Distinct Familiarity-Based Response Patterns for Faces and Buildings in Perirhinal and Parahippocampal Cortex

Abstract

An unresolved question in our understanding of the medial temporal lobes is how functional differences between structures pertaining to stimulus category relate to the distinction between item-based and contextually based recognition–memory processes. Specifically, it remains unclear whether perirhinal cortex (PrC) supports item-based familiarity signals for all stimulus categories or whether parahippocampal cortex (PhC) may also play a role for stimulus categories that are known to engage this structure in other task contexts. Here, we used multivoxel pattern analyses of fMRI data to compare patterns of activity in humans that are associated with the perceived familiarity of faces, buildings, and chairs. During scanning, participants judged the familiarity of previously studied and novel items from all three categories. Instances in which recognition was based on recollection were removed from all analyses. In right PrC, we found patterns of activity that distinguished familiar from novel faces. By contrast, in right PhC, we observed such patterns for buildings. Familiarity signals for chairs were present in both structures but shared little overlap with the patterns observed for faces and buildings on a more fine-grained scale. In the hippocampus, we found no evidence for familiarity signals for any object category. Our findings show that both PrC and PhC contribute to the assessment of item familiarity. They suggest that PhC does not only represent episodic context but can also represent item information for some object categories in recognition-memory decisions. In turn, our findings also indicate that the involvement of PrC in representing item familiarity is not ubiquitous.

Introduction

The functional organization of the medial temporal lobes (MTLs) remains a topic of intense debate in neuroscience. Much pertinent research in human and nonhuman species has focused on recognition memory, i.e., the ability to discriminate between previously encountered and novel stimuli, and the question of whether different MTL structures make distinct functional contributions. An influential view is that the hippocampus (Hip) plays a unique role in the encoding and subsequent recovery of episodic contextual information about a specific previous stimulus encounter, whereas perirhinal cortex (PrC) supports recognition based on the familiarity of the item itself (Aggleton and Brown, 1999; Eichenbaum et al., 2007; Mayes et al., 2007). However, there are also findings that point to category-specific contributions of different MTL structures to recognition memory (Davachi, 2006; Murray et al., 2007; Graham et al., 2010). An important, currently unresolved question is how category-specific effects relate to the distinction between item-based and contextually based recognition processes. This issue is of particular relevance for characterizing the functional contributions of PrC and parahippocampal cortex (PhC; Ranganath and Ritchey, 2012).

In the visual modality, the strongest category-specific neural responses have been observed for faces and scenes. Differential fMRI responses to faces are typically most pronounced in aspects of the lateral occipital and posterior fusiform gyrus (Kanwisher et al., 1997; for review, see Gobbini and Haxby, 2007) but have also been reported more rostrally in PrC (Tsao et al., 2008; Nasr and Tootell, 2012; O'Neil et al., 2013). In contrast, differential responses to scenes have been found predominantly in posterior aspects of PhC (Epstein and Kanwisher, 1998; Epstein, 2008). Scene-specific responses in PhC have informed the proposal that this structure represents episodic context in recognition memory and recall (Davachi, 2006; Eichenbaum et al., 2007). This role of PhC has been considered as distinct from both the role of PrC in the representation of objects and from the role of the Hip in item-context binding (Diana et al., 2007; Eichenbaum et al., 2007).

Interestingly, fMRI research has also revealed that preferential PhC responses extend beyond scenes to certain types of objects, including buildings (Aguirre et al., 1998; Cate et al., 2011). Although the exact object characteristics that “drive” PhC responses are only beginning to be understood, initial findings suggest that PhC is preferentially tuned to objects that are large (Konkle and Oliva, 2012), that evoke a sense of three-dimensional space (Mullally and Maguire, 2011), and/or that have navigational relevance (Janzen and van Turennout, 2004; Troiani et al., 2013). That PhC responds not only to scenes but also to certain types of objects is of direct relevance to its proposed role in recognition memory. Specifically, the summarized findings raise the question of whether the role of PhC in recognition memory is indeed limited to representing episodic contextual information or whether it also represents item-based familiarity for certain types of objects. Phenomenologically, buildings, like any other objects, can be recognized as “old,” even when episodic contextual detail about a specific previous stimulus encounter is absent. Corresponding item-based familiarity signals in PhC would be of broader theoretical significance toward understanding MTL organization, because they could suggest that familiarity is not invariably supported by PrC.

Here, we used high-resolution fMRI in combination with multivoxel pattern analyses (MVPA) to examine distributed patterns of activity in MTL structures that carry information about the perceived familiarity of three categories of objects, namely faces, buildings, and chairs.

Materials and Methods

Participants.

Nineteen right-handed participants took part in the study (21–30 years of age; mean age, 25.2 years; 12 females). They were screened for the absence of a history of neurological disorders. Data from one participant were excluded from all analyses as a result of excessive head movement (>5 mm in one plane) during scanning. Participants received financial compensation for their participation and provided informed consent according to procedures approved by the University of Western Ontario Health Sciences Research Ethics Board.

Stimuli and procedure.

Grayscale images depicting exemplars from three different object categories (faces, chairs, and buildings) were used as stimuli. Images of chairs and buildings were obtained from the internet using Google Image Search. Images of faces were obtained from a database we acquired for another study (O'Neil et al., 2009). All visual background was removed from the target object depicted in each image and replaced by an artificial, homogenous background without any space-defining features (Fig. 1). For example, sidewalks, lawns, and any visible horizon were removed from each image depicting a building. Presented in this manner, all items fulfill the definition of an object as being a discrete entity bounded by a single contour that does not have background elements or a horizon (a definition adopted in previous research that aimed to characterize the response properties of PhC regions; Troiani et al., 2013). Image size was constrained by a 375 × 250 pixel bounding box, and each object image was scaled to fit this box, with at least one dimension corresponding to its limits. Final item selection was optimized based on behavioral pilot data so as to match recognition accuracy and maximize the proportion of familiarity-based recognition responses across the three object categories. For each category, three sets of 40 items were selected. Two of these sets served as items presented before scanning during a study session and as corresponding targets in the subsequent recognition–memory test during scanning; items from the third set served as novel lures in the recognition task. Assignment of the three item sets as targets or lures was counterbalanced across participants.

Figure 1.

Experimental task design. During encoding, participants rated the relative attractiveness, comfort, or value for faces, chairs, and buildings, respectively. In the subsequent scanned recognition test stage, participants provided a rating of perceived item familiarity. They were asked to avoid voluntary attempts to recollect contextual details pertaining to a specific previous item encounter but were offered an opportunity to indicate recollection when it occurred involuntarily.

All participants completed an initial study session outside of the scanner that was preceded by a brief practice phase. The study session was separated into six blocked sequences of 40 trials, with blocks corresponding to the two sets of items from each of the three categories. Blocks were presented in an ABCCBA order, with each stimulus category corresponding to one of the letters. Stimuli were presented for 3000 ms each with a 2000 ms interstimulus interval, and participants were asked to rate the relative attractiveness, comfort, or value for faces, chairs, and buildings, respectively, using a five-point scale. For a schematic representation of the experimental design, refer to Figure 1.

After a delay of ∼1 h that allowed for scanning preparation, participants completed a recognition–memory test consisting of the 80 previously studied targets and 40 lures from each category, for a total of 360 trials. Items were distributed over eight functional runs, each comprising 45 trials with 10 previously studied and five novel items from each stimulus class. Each stimulus was presented for 2500 ms (corresponding to the length of acquisition of one MRI volume), with a jittered fixation baseline separating trials. Baseline fixation ranged from 2.5 to 15 s, and the order of trials and jitter were optimized within each run, using the OptSeq2 algorithm (http://surfer.nmr.mgh.harvard.edu/optseq/). Each of the three counterbalanced experiments used a unique jitter sequence and trial order. Participants viewed the stimulus displays through a mirror at a distance that yielded an approximate object size of 18° × 13° visual angle. Participants were asked to make recognition judgments with one of five different response buttons using an MRI-compatible keypad.

For their recognition judgments, participants were instructed to focus on their experienced item familiarity and provide a rating of perceived strength on a scale from one (least familiar) to four (most familiar). They were asked to respond quickly and avoid intentional attempts to recollect contextual details pertaining to a specific previous item encounter; however, they were offered an opportunity to indicate recognition based on spontaneous, involuntary recollection (R) with a separate response button when it occurred. Pertinent previous fMRI research suggests that participants can indeed control the extent to which they attempt to retrieve contextual details in recognition decisions strategically (Dobbins and Han, 2006; Quamme et al., 2010). The specific instructions were guided by those provided by Montaldi et al. (2010; for additional rationale, see Migo et al., 2012). Participants were informed that, despite their efforts to focus on familiarity, spontaneous recollection could be expected to occur on some trials. Recollection of contextual details was defined as any situation that involved conscious awareness of information about the past item encounter that was not included in the stimulus itself, such as internal thoughts and associations that were formed during the initial item encounter, or related uncontrolled external events (e.g., a knock on the door during the presentation of that item at study). Participants were familiarized with these response options in a practice phase that required them to justify any R response with a description of the contextual detail retrieved. They were also informed that two-thirds of the items to be presented had been encountered in the study phase.

fMRI acquisition protocol.

All MRI data were acquired on a 3 T Siemens TIM Trio scanner with a high-resolution fMRI protocol optimized for MTL examination. Functional MRI volumes were collected using a T2*-weighted single-shot gradient echo-planar acquisition sequence [repetition time (TR), 2500 ms; echo time (TE), 26 ms; slice thickness, 2 mm; in-plane resolution, 2 × 2 mm; field of view (FOV), 220 × 220 mm; matrix size, 110 × 110; flip angle, 90°]. Each functional volume included 37 contiguous slices collected in an interleaved manner. For each experimental run, 176 volumes were collected. To optimize MR signal in the anterior temporal lobes, a transverse orientation was chosen with the intent to include the entire temporal lobes and as much visual cortex as possible. This slice selection resulted in full coverage of the ventral aspects of occipital and full coverage of the entire temporal lobes in all participants, with exclusion of the most dorsal aspects of frontal, parietal, and occipital cortices. A saturation band was applied during functional runs to minimize artifacts related to eye movements and the sinus cavity. T1-weighted anatomical images were obtained using an MPRAGE sequence (192 slices; TR, 2300 ms; FOV, 240 × 256 mm; matrix size, 240 × 256; flip angle, 9 mm; TE, 4.25 ms; 1 mm isotropic voxels).

fMRI data preprocessing.

fMRI data were preprocessed in native space using BrainVoyager QX version 2.3 (Brain Innovation). Functional images were slice-scan time corrected, three-dimensional motion corrected with reference to the functional volume taken just before the anatomical scan, and high-pass filtered using a Fourier basis set of two cycles per run (including linear trend). Images were then coregistered with the anatomical set, aligned with the anterior commissure–posterior commissure plane, and smoothed using a three-dimensional Gaussian kernel with a full-width at half-maximum of 3 mm. Functional data were convolved using a standard double gamma hemodynamic response function (Friston, 1998). Participant-specific GLMs of these data allowed for extraction of z-scored trial-specific β estimates in all voxels of interest. β estimates derived from a modeled HRF were chosen as target measure for the MVPA (i.e., as classifier input) because they are particularly well suited to account for overlap in the hemodynamic response in fast-event-related designs (Misaki et al., 2010). Changes in mean intensity were modeled by including them as predictors of no interest in the participant-specific GLMs.

Anatomical ROI definition.

The main structures of interest in our investigation were PrC, PhC, and the Hip. To conduct our MVPA, we obtained anatomically defined ROIs in native MRI space with manual tracing separately in each participant, using an established MR-based protocol that specifies anatomical landmarks for demarcation of these MTL structures and their boundaries (Pruessner et al., 2000, 2002). Specifically, we obtained ROIs for PrC, PhC, anterior Hip (aHip), posterior Hip (pHip), and the entire Hip in each hemisphere. Entorhinal cortex was identified at the time of tracing to aid in the identification of anatomical landmarks but was not considered for the functional analyses. The distinction between aHip and pHip was introduced based on recent findings pointing to distinct functional specialization and followed the protocol described by Poppenk and Moscovitch (2011), with the uncal apex serving as the critical boundary. Figure 2 depicts a graphical representation of these ROIs in a representative participant.

Figure 2.

Coronal structural MRI slices from a representative participant with anatomically defined ROIs used for MVPA of fMRI data. L, Left; R, right.

MVPA of fMRI data.

MVPA (for reviews and rationale, see Norman et al., 2006; Kriegeskorte and Bandettini, 2007; Tong and Pratte, 2012) was used to address two different types of questions. First, we used it to determine whether distributed patterns of activity in any of the MTL structures examined could reliably distinguish between the stimulus categories (i.e., faces, chairs, and buildings). Second, and more importantly, it allowed us to examine whether distributed patterns of activity could be identified that reflected a memory signal, i.e., differences between familiar and novel stimuli, for each stimulus category. In these analyses, we specifically focused on perceived familiarity given that our main interest was in understanding the role of MTL structures in explicit recognition judgments. To the extent that the response distribution for items perceived as familiar or novel (i.e., familiarity levels 3, 4 vs familiarity levels 1, 2, respectively) varied within participants and to the extent that such unequal item distributions can introduce unwanted classification biases, we introduced a sampling procedure that matched item numbers of perceived familiar and novel trials within each stimulus category. This sampling procedure operated in a pseudorandom manner and underwent 10 iterations, with the provision that all trials be included in the classification analysis at least one time. It reduced the number of trials in the condition with the larger number (familiar or novel) so that it corresponded with the number in the other condition of interest. For example, if one participant had 50 “familiar” and 42 “novel” responses for faces, the number of familiar trials included in the classification analysis for faces was reduced to 42. We decided to use 10 different sampling iterations in each familiar/novel classification based on initial pilot analyses in two participants that showed that classifier performance did not differ statistically when more sampling iterations (up to 100) were included.

Pattern classification analyses were conducted using the Princeton MVPA toolbox (http://www.pni.princeton.edu/mvpa) and custom MATLAB code (MathWorks). As a first step, we performed feature selection to reduce noise in the functional data. For each ROI, the voxels that appeared most informative for classification based on initial univariate GLMs were selected in each participant separately (ANOVA; for discussion, see Norman et al., 2006). Specifically, a one-factorial ANOVA, with number of conditions being equal to the number of conditions pertinent for the classification at hand (two or three), was conducted on the β weights of all voxels in each ROI and in each cross-validation. For all familiar/novel classifications, this analysis was also performed separately for each of the 10 iterations of item sampling. The resulting F values were then rank ordered across all voxels, and the voxels corresponding to the top 10% of that ranking were selected for the MVPA-based classification. Note that, with this type of feature selection, the number of voxels considered for classifier training still scales with the overall size of the ROI. To illustrate the outcome of this feature selection, the number of voxels (2 × 2 × 2 mm in size), averaged across participants, that were included in the subsequent MVPA for each ROI were as follows: right PrC, 55.1; left PrC, 54.9; right PhC, 41.2; left PhC, 41.2; right aHip, 26.2; left aHip, 24.3; right pHip, 30.5; and left pHip, 30.0.

A linear support vector machine (SVM; libSVM, http://www.csie.ntu.edu.tw/∼cjlin/libsvm) was used for classification of the β values for the various experimental conditions, with a linear kernel function and a constant cost parameter of C = 1. The SVM was trained on all trials minus one from each of the conditions that were included in the analysis at hand; those trials not included in the training dataset subsequently served as test trials for assessment of classification performance. This procedure was repeated multiple times such that all trials served as the test stimuli for classification, providing a k-fold cross-validation (Duda et al., 2001; Chadwick et al., 2010), where k reflects the number of trials that were included in the relevant analysis. For each trial in the test set, the classifier provided probability estimates that reflected the likelihood that the activity pattern across voxels associated with that trial corresponded to each of the different conditions included in the classification. These estimates were expressed in a binary manner such that classification was either correct (i.e., when the “true” experimental condition was assigned the highest probability) or incorrect. Classification accuracy was then calculated based on the percentage of test trials that were classified correctly in this binary schema. This calculation was performed and averaged across all 10 sampling iterations in the familiar/novel classifications to provide an unbiased performance estimate.

Finally, we also conducted additional cross-classification analyses to probe whether any decision boundary yielding above-chance classification between familiar/novel trials in a stimulus category might also allow for successful classification between familiar and novel trials from other stimulus categories. The MVPA cross-classification procedure used for this purpose used the same linear SVM, cross-validation technique, and sampling procedure described above. However, the sampling procedure was performed to equate the number of trials across four, rather than two, conditions (i.e., novel and familiar trials from two stimulus categories).

To obtain inferential statistics, we primarily examined whether classification performance was above chance (0.5 or 0.33 depending on whether two or three conditions were included, respectively), focusing on each ROI independently. For this purpose, we used a single-sample t test to test against a population mean of chance level, applying Bonferroni's correction based on the number of independent comparisons made (i.e., the number of ROIs or stimulus categories examined).

Results

Behavioral performance

Raw percentages of the different recognition-response types for each stimulus category are presented in Table 1. Visual inspection of these data demonstrates that recognition decisions for previously studied items were based more frequently on familiarity (F3 and F4 collapsed; mean percentages, 40.2, 41.0, and 43.5 for faces, chairs, and buildings, respectively) than on recollection (mean percentages, 11.2, 12.9, and 12.5 for faces, chairs, and buildings), as intended. Overall recognition performance, i.e., the ability to distinguish between previously studied items and novel lures, regardless of subjective recognition experience, was quantified using the discriminability index d′, which considers hits and false alarms in the context of signal-detection theory, considering F3, F4, and R as old responses. Familiarity estimates were calculated using d′ based on F1, F2, F3, and F4 responses and corrected assuming independence between familiarity and recollection (Yonelinas, 1999). A one-way ANOVA conducted on these measures revealed that performance was closely matched across stimulus categories; there were no significant differences between categories for overall discrimination (F_(2,51) = 0.81, p = 0.45), estimates of familiarity (F_(2,51) = 0.45, p = 0.64), or estimates of recollection (R hits − R false alarms; F_(2,51) = 0.01, p = 0.99). Although performance levels were low, because of the high similarity between all items within each stimulus category, familiarity discrimination was above chance for each category (all t₍₁₇₎ values >8.4, p values <0.001). We note that the limited number of R responses observed in the present study (collapsed across hits and false alarms; mean number of trials, 9.56, 11.67, and 11.06 for faces, chairs, and buildings, respectively) did not allow for any investigation of fMRI responses associated with recollection.

Table 1.

Recognition-response distribution and discrimination estimates for each stimulus category

Stimulus category	Responses to studied items (%)					Responses to novel items (%)					Discrimination d′
	1	2	3	4	R	1	2	3	4	R	Recognition	Familiarity
Faces	21.9 ± 3.1	26.7 ± 2.1	20.8 ± 1.4	19.4 ± 2.1	11.2 ± 2.0	42.9 ± 5.1	30.9 ± 3.1	16.3 ± 1.8	8.0 ± 1.9	2.0 ± 0.9	1.24 ± 0.06	0.64 ± 0.08
Chairs	21.4 ± 2.5	24.6 ± 1.5	20.7 ± 2.1	20.3 ± 1.8	12.9 ± 3.6	46.5 ± 3.4	26.8 ± 2.2	15.4 ± 1.4	7.8 ± 1.6	3.5 ± 1.2	1.26 ± 0.06	0.66 ± 0.06
Buildings	24.3 ± 3.5	19.7 ± 1.4	18.4 ± 1.7	25.1 ± 2.5	12.5 ± 2.6	48.0 ± 5.6	25.4 ± 2.6	14.2 ± 2.5	10.2 ± 1.6	3.1 ± 1.1	1.34 ± 0.06	0.73 ± 0.07

For behavioral signal-detection and fMRI analyses, novel responses correspond to 1 and 2 collapsed, with familiar responses corresponding to 3 and 4 collapsed. Numbers reflect mean ± SEM.

With respect to reaction times, a 2 × 3 ANOVA (response type × stimulus category) revealed neither a main effect of category (F_(2,102) = 2.00, p = 0.14) nor a significant interaction (F_(2,102) = 2.13, p = 0.12), providing additional evidence that all three stimulus categories were matched behaviorally. This analysis did reveal a significant main effect of response type (F_(2,102) = 44.39, p < 0.001), with R responses (mean ± SD, 1336.2 ± 233.9 ms) being faster than judgments of familiarity (F1–F4; mean ± SD, 1648.8 ± 261.4 ms).

fMRI results: between-category classification

With our MVPA, we first sought to determine whether distributed patterns of activity across voxels in any of the anatomically defined ROIs in the MTL could discriminate between the three stimulus categories. For this purpose, we considered all novel trials regardless of the recognition responses provided by the participants, with an equal number of trials for each stimulus category. The one-sample t tests conducted with a Bonferroni correction for eight comparisons (i.e., ROIs) revealed that classification performance was above chance in right PrC (t₍₁₇₎ = 3.96, p < 0.01 corrected), left PrC (t₍₁₇₎ = 4.56, p < 0.01 corrected), right PhC (t₍₁₇₎ = 12.89, p < 0.001 corrected), left PhC (t₍₁₇₎ = 9.13, p < 0.001 corrected), and right pHip (t₍₁₇₎ = 3.46, p < 0.05 corrected), with all other regions (left aHip, right aHip, and left pHip) showing classifier accuracy not significantly greater than chance (all p values >0.37). Although these data point to category-specific representations in some MTL regions, they do not allow for any inferences as to whether all stimulus categories can be discriminated from one another. Figure 3 presents the results of additional analyses that focused on pairwise classifications. Activation patterns in right PrC (t₍₁₇₎ = 3.82, p < 0.01 corrected), left PrC (t₍₁₇₎ = 3.19, p < 0.05 corrected), right PhC (t₍₁₇₎ = 12.04, p < 0.001 corrected), and left PhC (t₍₁₇₎ = 7.25, p < 0.001 corrected) were sensitive to differences between faces and buildings (p values for remaining ROIs were >0.11). These regions were also sensitive to differences between faces and chairs (right PrC, t₍₁₇₎ = 3.46, p < 0.05 corrected; left PrC, t₍₁₇₎ = 5.67, p < 0.001 corrected; right PhC, t₍₁₇₎ = 7.31, p < 0.001 corrected; left PhC, t₍₁₇₎ = 7.14, p < 0.001 corrected). Another region that showed above-chance classification for these categories was the right pHip (t₍₁₇₎ = 3.66, p < 0.05 corrected; all other p values >0.41). Last, for discriminations between buildings and chairs, classifiers for activation patterns in right PrC (t₍₁₇₎ = 3.19, p < 0.05 corrected), right PhC (t₍₁₇₎ = 5.41, p < 0.001 corrected), and left PhC (t₍₁₇₎ = 6.06, p < 0.001 corrected) produced above-chance performance (all other p values >0.30). Together, these results revealed evidence for category-specific representations reflected in sensitivity to differences between all pairs of stimulus categories in PrC as well as in PhC, with the most clear-cut pattern emerging in the right hemisphere. Evidence for category-specific representations in the Hip was limited.

Figure 3.

Pairwise MVPA classification of different stimulus categories in left and right PrC, PhC, aHip, and pHip. Classification was based on examination of all trials in which novel stimuli were presented. Dashed lines indicate chance level for classification. Numbers within bars represent the number of participants for whom classification performance was numerically above chance level. All error bars indicate the SEM calculated across participants. *p < 0.05, **p < 0.01, ***p < 0.001.

fMRI results: within-category classification of familiar and novel trials

Although the previous analyses revealed category-specific effects in both PrC and PhC, their selective focus on novel items does not speak to mnemonic signals as such. In other words, the results of these analyses do not imply that these structures carry information about the perceived familiarity of items in any of the relevant categories. Therefore, we next examined whether subjectively perceived familiarity would also be reflected in patterns of activity in PrC and PhC that are category specific. For these analyses, trials were considered regardless of objective item status and were collapsed across familiarity ratings F1 and F2 (novelty or low familiarity) and contrasted with familiarity ratings F3 and F4 (high familiarity). This approach ensured that SVM training could be based on a sufficiently large number of trials (≥24) for the familiarity-based classification in each category and individual participant. The average number of trials included for the classification of faces, chairs, and buildings at each familiarity level (i.e., low vs high) was 39.8, 39.7, and 38.3, respectively. The results of these analyses are presented in Figure 4. We found evidence for activation patterns that reflected subjectively perceived familiarity in both PrC and PhC but not in the Hip in either hemisphere. Specifically, in right PrC, patterns of activity could be classified according to subjective familiarity with above-chance accuracy for faces (t₍₁₇₎ = 4.77, p < 0.001 Bonferroni corrected for three comparisons, i.e., number of stimulus categories) and chairs (t₍₁₇₎ = 3.47, p < 0.01 corrected) but not for buildings (t₍₁₇₎ = 0.17, p = 0.87). Further, classifier accuracy for both faces and chairs was significantly greater than for buildings in right PrC (t₍₃₄₎ = 3.83, p < 0.01; t₍₃₄₎ = 2.68, p < 0.05, respectively; corrected). In contrast, in right PhC, patterns of activity could be classified according to subjective familiarity with above-chance accuracy for buildings (t₍₁₇₎ = 3.91, p < 0.01 corrected) and chairs (t₍₁₇₎ = 5.27, p < 0.001 corrected) but not faces (t₍₁₇₎ = 1.20, p = 0.25). Classifier accuracy for both buildings and chairs was significantly greater than for faces in right PhC (t₍₃₄₎ = 2.42, p < 0.05; t₍₃₄₎ = 2.48, p < 0.05, respectively; corrected). In the left hemisphere, classifier performance in PrC did not exceed chance level for any stimulus category (faces, t₍₁₇₎ = 1.15, p = 0.26; chairs, t₍₁₇₎ = 1.95, p = 0.06; buildings, t₍₁₇₎ = 0.44, p = 0.67). Classification accuracy was above chance in left PhC for chairs (t₍₁₇₎ = 3.53, p < 0.01) but not faces (t₍₁₇₎ = 0.05, p = 0.96) or buildings (t₍₁₇₎ = 0.72, p = 0.48).

Figure 4.

MVPA classification of perceived familiar versus novel trials for the three different stimulus categories in left and right PrC, PhC, aHip, and pHip. Dashed lines indicate chance level. Numbers within bars represent the number of participants for whom classification performance was numerically above chance level. All error bars indicate the SEM calculated across participants. *p < 0.05, **p < 0.01, ***p < 0.001.

Critically, in the Hip, we found no evidence for patterns of activation that allowed for classification based on subjective familiarity for any stimulus category. This held regardless of whether we assessed the pHip or aHip and regardless of whether we did this in the right or the left hemisphere (all p values >0.15). One issue to consider in these analyses is that the four hippocampal ROIs were generally smaller than those in PrC and PhC. Moreover, because they focused only on the anterior or posterior section separately, they would not capture diagnostic patterns that might be distributed across the entire Hip. Accordingly, we also determined whether results would differ if we examined the whole Hip in a single analysis (for each hemisphere). Note that, in these analyses, the right hippocampal ROIs were on average, and in the majority of the participants, numerically larger in volume than the ROIs for right PrC and right PhC. Still, we found no evidence for patterns of responses in the right Hip that allowed for classification based on subjective familiarity for faces (mean ± SEM, 0.50 ± 0.01), chairs (mean ± SEM, 0.49 ± 0.01), or buildings (mean ± SEM, 0.50 ± 0.02). Similarly, we found no such evidence when the Hip was considered as a whole in the left hemisphere for faces (mean ± SEM, 0.49 ± 0.01), chairs (mean ± SEM, 0.50 ± 0.01), or buildings (mean ± SEM, 0.49 ± 0.01).

Although our primary interest was in perceived familiarity, we also explored whether patterns of responses in any ROI would allow for successful classification based on objective item status (i.e., previously studied vs new items regardless of recognition response). No significant effects were revealed for any stimulus category in any region examined (PrC, PhC, aHip, pHip, entire Hip in left or right hemisphere; all p values >0.05). These divergent results for classification based on perceived (i.e., subjective) familiarity versus objective item status mirror previous fMRI findings obtained with MVPA for the whole brain in the context of recognition memory for faces (Rissman et al., 2010).

fMRI Results: between-category cross-classification and voxel overlap analysis

Given that our MVPA revealed signals related to subjectively perceived familiarity for two stimulus categories in both PrC (faces and chairs) as well as PhC (buildings and chairs), an important question that arises is whether the patterns of activity that could be classified successfully are indeed category specific. For example, the analyses conducted so far would not rule out that above-chance classifier performance in right PrC for faces and chairs is based on a common familiarity signal. Likewise, they do not rule out that the familiarity signal for buildings and chairs in PhC is shared. To address category specificity of the observed familiarity signals more directly, we first explored the extent of overlap between voxels that contribute to the classification of familiar and novel responses in one stimulus category and those that contribute to classification in the other. Toward this end, we examined overlap at the level of voxels that were selected in the initial feature-selection procedure to be the most informative for use in the SVM. We found that 52% of the voxels in right PrC that entered the classification analysis for faces also entered the classification for chairs (and vice versa). The corresponding averaged value for overlap in right PhC in the context of classification of buildings and chairs was 53%. These results suggest that the distributed memory representations for the categories supported by each of these MTL structures show partial overlap. In the next step, we examined this issue more formally with inferential statistics by conducting a cross-classification analysis for faces and chairs in right PrC and for buildings and chairs in right PhC. For example, if familiarity-based classification in PrC is based on strongly overlapping distributed representations for faces and chairs, these analyses should reveal that the pattern of activity that distinguishes between familiar and novel faces can also be used to discriminate between familiar and novel chairs and vice versa. The classification accuracies we obtained for these cross-classification analyses in right PrC and right PhC are presented in Figure 5. Again, these analyses were conducted using the item-sampling procedure described previously that matched item numbers across both stimulus class and recognition responses (with 33.7 trials included on average). Critically, cross-classification was at chance in right PrC when the linear decision boundary of the SVM trained on faces was used to classify familiar and novel chairs (mean, 0.50; p > 0.45 uncorrected) as well as when the SVM trained on chairs was used to classify familiar and novel faces (mean, 0.49; p > 0.58). Cross-classification performance for buildings and chairs in right PhC was equally unsuccessful (mean, 0.52; p > 0.18; and mean, 0.51; p > 0.32). These data suggest that the distributed patterns of voxels that contribute to classification of familiar versus novel faces and chairs in PrC and those that contribute to classification of familiar versus novel buildings and chairs in PhC show some specificity within each of these structures.

Figure 5.

MVPA cross-classification for the two stimulus categories for which reliable familiarity-based classification was found in right PrC and PhC. Within-category classification performance is shown for reference in the left column of each data pair as displayed in Figure 4. Cross-classification data are displayed in green in the right column. A, Classification and cross-classification of familiarity signals for faces and chairs in PrC. B, Classification and cross-classification of familiarity signals for buildings and chairs in PhC. Dashed lines indicate chance performance level. Numbers within bars represent the number of participants for whom classification performance was numerically above chance level. All error bars indicate the SEM calculated across participants. Cross-classification performance did not exceed chance level in any condition.

In the interest of completeness, we also conducted the remaining cross-classification analyses that can be brought to bear on our data in right PrC and right PhC (i.e., cross-classification involving buildings in PrC and faces in PhC). None of these analyses revealed any significant effects (all p values >0.11, uncorrected).

Discussion

In the current study, we examined distributed patterns of activity in the MTL that are associated with the perceived familiarity of faces, buildings, and chairs. Our primary analyses of interest focused on patterns of activity that distinguished between subjectively experienced familiar or novel items in each object category. In right PrC, MVPA revealed such familiarity signals for faces but not buildings. In contrast, in right PhC, we observed familiarity signals for buildings but not faces. Familiarity signals for chairs were present in both structures but shared limited overlap with the patterns we observed for faces and buildings. Contrasting with these findings for PrC and PhC, we found no evidence for familiarity signals in the Hip for any of the object categories examined.

A number of studies have previously documented category-specific responses in the MTL (Litman et al., 2009; Preston et al., 2010; Staresina et al., 2011), some of which were also based on MVPA of fMRI data (Diana et al., 2008; Liang et al., 2013). As in the literature on category-specific effects in the ventral visual stream more broadly, the most widely used comparison in these studies has been between faces and scenes. In research concerned with item-based familiarity, such comparisons can pose challenges for interpretation. When a scene (such as a bedroom) is endorsed as being familiar, the response may be based on individual objects (e.g., furniture), the spatial relationships between these objects, or the geometric properties of the background (e.g., shape and size of the room). Unless the relationships between target scenes and their distractors are systematically manipulated (e.g., Cleary et al., 2012), the source of information in the stimulus display that guides the memory judgment is unknown and may even vary from trial to trial (for discussion, see Preston et al., 2010; Martin et al., 2012).

In the present study, we addressed the potential role of PhC in item familiarity by examining familiarity signals for items from three different object categories, all of which were presented without any scene context. Results from our initial examination of MVPA classification (when only novel stimuli were considered) revealed that patterns of activity in both PrC and PhC were sensitive to differences between all three object categories and that the most consistent differences were present in the right hemisphere. Critically, patterns of activity that gave rise to reliable classification of memory signals, more specifically item-based familiarity, revealed noticeable functional differences between both structures. In right PrC, familiarity signals could be classified reliably only for faces and chairs. In contrast, in right PhC, familiarity signals could be classified reliably only for buildings and chairs. This pattern of findings suggests that PhC, like PrC, carries information about the familiarity of objects. However, both structures differ in their response profile across the three different object categories examined here. To the extent that these results were obtained with analyses that excluded any recollection-based behavioral responses, this difference in neural response profile cannot be captured with reference to the distinction between items and episodic context.

The preferential responses in PrC and PhC we observed for faces and buildings in familiarity-based judgments are consistent with findings from previous research in other tasks. For example, recent evidence from fMRI studies in human and nonhuman primates points to the presence of a cortical patch in rostral aspects of the collateral sulcus that shows preferential responses to faces even during passive viewing (Tsao et al., 2008; Rajimehr et al., 2009; Ku et al., 2011; Nasr and Tootell, 2012). Other studies have revealed preferential responses to buildings in PhC (Aguirre et al., 1998; Maguire et al., 2001; Cate et al., 2011; Nasr and Tootell, 2012) and have shown adaptation effects in perceptual judgments (Pourtois et al., 2009).

That familiarity signals for chairs were found to be present in PrC as well as in PhC suggests that reference to stimulus category alone may ultimately be insufficient to account for the difference in response profiles exhibited by both structures. What other factors might determine whether reliable familiarity signals are observed in PrC or PhC then? The current findings would be consistent with the idea that these structures are differentially sensitive to specific stimulus dimensions that are often correlated in various object categories (Op de Beeck et al., 2008). Based on the characterization of PhC responses in other studies, we speculate that critical dimensions may include the size, mobility, and sense of space that objects evoke (Cate et al., 2011; Mullally and Maguire, 2011; Konkle and Oliva, 2012; Troiani et al., 2013). From a functional perspective, these dimensions are essential determinants of the navigational relevance of objects (Janzen and van Turennout, 2004; Troiani et al., 2013). In such a framework, PhC may carry familiarity signals for objects that tend to be large, immobile, and evoke a sense of three-dimensional space. Conversely, PrC may carry familiarity signals for objects that are smaller in real-world size and are not tied to a specific geographical location. Familiarity for chairs would be represented in both PrC and PhC because chairs are characterized by some stimulus attributes that drive PrC and others that drive PhC. Namely, like faces, they are not tied to a specific location. Like buildings, they can evoke a sense of surrounding three-dimensional space (Mullally and Maguire, 2011). An account that makes reference to object dimensions, rather than categories, could also explain why the voxels within each structure that carry information about the familiarity for one object category, although not allowing for cross classification, show some overlap with those that are diagnostic for the familiarity of another. Given that fMRI studies in recognition memory have most commonly focused on verbal stimuli, with no systematic manipulation of specific dimensions of the referent objects, we acknowledge, however, that extant evidence that speaks to this proposal is currently limited.

Another explanation as to why certain types of objects preferentially engage PhC could invoke the degree to which different objects elicit contextual associations. In particular, it has been suggested that PhC is involved in representing contextual associations for those objects (e.g., pillow) that are typically encountered in the same context (e.g., bed or bedroom; Bar and Aminoff, 2003). These types of contextual associations can be seen as semantic in nature, in that they specify a typical context that may or may not hold for any specific episodic encounter. It seems unlikely that such semantic contextual associations would be of diagnostic value in familiarity-based recognition memory decisions that require discrimination between studied and non-studied exemplars within restricted object categories, such as the buildings used in the current study. Although some buildings may be more likely to elicit such contextual associations than others, these associations, to the extent that they become conscious at the time of encoding and retrieval, would be captured with recollective rather than familiarity-based responses. Together with other recent evidence showing that the contextual effects in PhC reported by Bar and Aminoff (2003) may in fact reflect scene imagery (Epstein and Ward, 2010) or the sense of surrounding space that some objects evoke (Mullally and Maguire, 2011), these considerations argue against the idea that the differences in item-based familiarity signals we observed for PhC and PrC reflect differential influences of semantic contextual associations.

In the current study, the primary focus of analysis centered on discrete MTL structures that were defined anatomically. Another approach in the literature has been to examine functional gradients that cross the boundary of both structures in the parahippocampal region (Litman et al., 2009; Staresina et al., 2011; Liang et al., 2013). We focused on differences between discrete structures because our primary objective was to evaluate theoretical claims about functional specialization that have been proposed to be present at this neuroanatomical scale (Eichenbaum et al., 2007; Graham et al., 2010). These models of functional MTL organization have typically been informed by differences in cytoarchitectonic composition and anatomical connectivity of PrC and PhC (Lavenex and Amaral, 2000; Manns and Eichenbaum, 2006). In the future, it will be important to develop and test models of MTL organization that directly map distributed patterns of mnemonic signals to more fine-grained neuroanatomical subdivisions in neocortical MTL structures (e.g., Ding and Van Hoesen, 2010) and to corresponding differences in connectivity.

Concerning Hip function, we observed that distributed patterns of activation in this structure did not allow for any reliable classification of familiarity signals. These results held across both hemisphere and across anterior and posterior aspects of the Hip, regardless of stimulus category. Because the patterns probed were specific to trials that were not accompanied by recollection of episodic contextual information, this finding is consistent with the influential idea that the Hip does not support item-based familiarity (Aggleton and Brown, 1999). However, we recognize that, considered in isolation, this hippocampal finding reflects a null effect. As such, it is amenable to alternate interpretations and could also be accommodated by the proposal that the Hip only carries memory representations of high strength, which are associated with high levels of behavioral accuracy (Squire et al., 2007). Nevertheless, we note that recent data from intracranial EEG recordings strongly argue against a single-process account of MTL organization that attributes functional differences between PrC and Hip to any differential sensitivity of both structures to a unidimensional strength signal (Horner et al., 2012; Staresina et al., 2012).

In conclusion, our findings indicate that both PrC and PhC contribute to the assessment of item familiarity. They show that the role of PhC is not limited to representing information about context and that PrC is not involved in representing familiarity for all object categories. Considered within the larger literature reviewed, these findings suggest that a comprehensive model of MTL organization for PrC, PhC, and Hip will ultimately require consideration of representational distinctions that include, but go beyond, the division between item and context information.