Medial Temporal Lobe Coding of Item and Spatial Information during Relational Binding in Working Memory

Abstract

Several models have proposed that different medial temporal lobe (MTL) regions represent different kinds of information in the service of long-term memory. For instance, it has been proposed that perirhinal cortex (PRC), parahippocampal cortex (PHC), and hippocampus differentially support long-term memory for item information, spatial context, and item–context relations present during an event, respectively. Recent evidence has indicated that, in addition to long-term memory, MTL subregions may similarly contribute to processes that support the retention of complex spatial arrangements of objects across short delays. Here, we used functional magnetic resonance imaging and multivoxel pattern similarity analysis to investigate the extent to which human MTL regions independently code for object and spatial information, as well as the conjunction of this information, during working memory encoding and active maintenance. Voxel activity patterns in PRC, temporopolar cortex, and amygdala carried information about individual objects, whereas activity patterns in the PHC and posterior hippocampus carried information about the configuration of spatial locations that was to be remembered. Additionally, the integrity of multivoxel patterns in the right anterior hippocampus across encoding and delay periods was predictive of accurate short-term memory for object–location relationships. These results are consistent with parallel processing of item and spatial context information by PRC and PHC, respectively, and the binding of item and context by the hippocampus.

Introduction

Medial temporal lobe (MTL) subregions, including the hippocampus, parahippocampal cortex (PHC), and perirhinal cortex (PRC), are known to contribute to long-term memory. Several models propose that the PRC supports memory for item information in an event, the PHC supports memory for an event's spatial or contextual information, and the hippocampus binds item and spatial/contextual information (Eacott and Gaffan, 2005; Davachi, 2006; Diana et al., 2007; Eichenbaum et al., 2007; Konkel and Cohen, 2009; Montaldi and Mayes, 2010; Ranganath, 2010; Ranganath and Ritchey, 2012; Nadel and Peterson, 2013; Pertzov et al., 2013). This framework has received support from neuropsychological and functional magnetic resonance imaging (fMRI) studies of long-term memory for object and spatial or contextual stimuli (for review, see Ranganath, 2010; Staresina et al., 2011; Hannula et al., 2013; Liang et al., 2013; Staresina et al., 2013).

More recent evidence has suggested that PRC, PHC, and hippocampus may be involved in cognitive domains outside of long-term memory (Curtis et al., 2000; Ranganath and D'Esposito, 2001; Stern et al., 2001; Cabeza et al., 2002; Gazzaley et al., 2004; Hannula et al., 2006; Lee et al., 2006a; Piekema et al., 2006; Barense et al., 2007; Bussey and Saksida, 2007; Murray et al., 2007; Finke et al., 2008; Rissman et al., 2008; Graham et al., 2010). For instance, activation of these regions has reliably been observed during working memory (WM) tasks requiring retention of information over very short delays (Davachi and Goldman-Rakic, 2001; Ranganath and D'Esposito, 2001; Stern et al., 2001; Cabeza et al., 2002; Hasselmo and Stern, 2006; Hannula and Ranganath, 2008; Han et al., 2013; Newmark et al., 2013). The exact roles of MTL regions in WM are controversial, however, and it is unclear whether these regions play distinct roles in the encoding and maintenance of item, context, and bound item-in-context information in WM, as is seen in long-term memory (Jeneson and Squire, 2012).

In the current study, we identified regions involved in the encoding and maintenance of object and location information in WM by applying multivoxel pattern similarity analysis (Kriegeskorte et al., 2008, 2009) to an fMRI dataset previously analyzed with univariate methods (Hannula and Ranganath, 2008). Participants were scanned while performing a WM task that required active maintenance of the locations of sets of household objects over a short delay. We tested whether activity patterns across voxels carried information about the object or spatial information to be maintained in each trial. We predicted that similarity between voxel patterns in PRC elicited by each trial would track the number of objects that trials had in common. Pattern similarity in PHC voxels was predicted to track the number of occupied locations that trials had in common. Finally, we predicted that accurate decisions on the WM task, which required binding of object and location information, would be associated with increased similarity of hippocampal voxel activity patterns between the encoding and delay phases.

Materials and Methods

Participants

Eighteen right-handed human participants (10 female) underwent fMRI scanning at the University of California, Davis (UC Davis), Imaging Research Center. All procedures were approved by the UC Davis Institutional Review Board.

Materials

Stimuli consisted of a 126 rendered scenes generated with Punch! Home Design Software. Six scenes were used to illustrate the task in the prescanning instructions phase of the experiment. The remaining scenes were divided randomly into eight lists of 15 scenes, one of which was randomly selected to use as practice stimuli, and the final seven of which were presented during scanning for each subject, for a total of 105 experimental scenes.

Every scene contained four household objects, each displayed in one of four cells on a three-by-three grid (Fig. 1A). All scenes shared identical three-dimensional first-person viewpoint angle, grid size, color, position, background color, and lighting; scenes differed only according to their objects and occupied cells. Objects in a scene were chosen pseudorandomly from a set of nine total objects, such that every object was used equally often across scenes, but the four-object combination was always unique. Object locations were also chosen pseudorandomly, with each object appearing no more than seven times in each spatial location. Therefore, any given pair of scenes could be described as having zero, one, two, or three objects in common, independent of the location of the objects on the grid, and zero, one, two, three, or four spatial locations in common, independent of the objects contained in each location (Fig. 1B). Because each scene contained a unique four-object combination, no two scenes had four objects in common. Additionally, because the experiment was originally designed to minimize the repetition of object–location associations across trials, no two trials had more than two object–location associations in common.

Figure 1.

A, Example trial. At study, four household objects were presented from a first-person perspective in four locations on a three-by-three grid. After a delay, the grid was presented with a 90° viewpoint rotation. At test, object–location bindings were identical on one-third of trials (match condition, pictured). On remaining trails, either the positions of two objects were switched (mismatch-swap condition) or one object was moved to a new location (mismatch-position condition). B, Trial pair feature overlap schematic. All pairs of trials had zero, one, two, or three objects in common, independent of location; and zero, one, two, three, or four locations in common, independent of object.

Procedure

Participants were scanned while performing a relational working memory task in which memory for object–location associations was critical for accuracy. In the current study, to examine brain activity related to object and location representations, analysis focused on the encoding and delay periods of each trial; due to variability in the object–location information presented at testing in different conditions (see below), activation during the test period was not considered here. Below, we provide an abridged description of the experimental procedure. For full procedural details, see Hannula and Ranganath (2008).

Before scanning, participants were familiarized with all nine objects, and underwent an instructions and practice phase. During fMRI scanning, each participant completed seven blocks of 15 trials, for a total of 105 experimental trials (Fig. 1A). During the encoding period of each trial, a scene was presented for 3 s, and participants were required to encode all four object–location associations. During the 11 s delay period, participants were required to perform a 90° mental rotation of the stimulus grid, maintaining the relative object–location associations. Following the delay period, a 90° viewpoint-rotated test scene was presented for 3 s. On a third of the trials, test stimulus object–location associations were identical to those presented during the encoding period, given the 90° rotation (the match condition). On another third of the trials, the same four objects occupied the same four locations, but two of the object–location associations had been switched (mismatch-swap condition). On the final third of trials, the same four objects were presented, but one object was shifted to a new location by one cell (mismatch-position condition). Each scanning block contained equal numbers of trials in each condition, and conditions were presented in a pseudorandom order within each block. During each test phase, participants were required to indicate via a button press whether they were viewing a match, mismatch-swap, or mismatch-position test stimulus. A fixation cross was then presented for an intertrial interval of 9, 11, or 13 s.

fMRI acquisition

MRI scanning was conducted at the UC Davis Imaging Research Center on a 3 T Siemens Trio scanner equipped with an eight-channel head coil. Gradient echoplanar imaging (EPI) time series data (TR, 2000 ms; TE, 25 ms; FOV, 220°; matrix size, 64 × 64) consisted of 34 axial slices with a 3.4 mm slice thickness, resulting in a voxel size of 3.4375 × 3.4375 × 3.4 mm. Coplanar high-resolution T1-weighted images were also acquired to facilitate cross-participant registration.

fMRI analysis

Preprocessing and task modeling

EPI time series underwent brain extraction, motion and slice timing correction, and bandpass temporal filtering (0.005–0.25 Hz) in FSL (http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/). To minimize the influence of preprocessing steps on trial-by-trial voxel patterns before pattern information analysis, spatial smoothing of functional volumes was omitted. For each participant, a set of linear regressions was conducted to isolate BOLD activation evoked by the study, delay, and test periods of each trial, according to methods developed by Mumford et al. (2012). Briefly, for each study, delay, and test period of every trial, a general linear model (GLM) was constructed (i.e., 315 single-trial GLMs in total) in which a single covariate modeled a mini-block, indicating the onset and duration of the period of interest, convolved with a canonical hemodynamic response function (HRF). Study and test period blocks were modeled for 3 s; to separate estimates of the study and delay period signal, delay period blocks were modeled as the middle 7 s in the 11 s delay window. Covariates of no interest then included a single vector modeling HRF-convolved onsets of every other trial period and motion vectors. Single-trial parameter estimate (β) images corresponding to the evoked response of each trial period of interest were computed using ordinary least-squares regression. To remove noisy trials, separately for study and delay periods, single-trial β images with atypically high (top 5%) mean absolute z-score (based on the distribution of β estimates for each gray matter voxel across all trials of a given task phase) were excluded from further analysis. The remaining study and delay phase single-trial β images were then entered into pattern analysis pipelines.

Multivoxel pattern similarity analysis procedures

General analysis scheme.

Representation of object information and representation of location information were examined independently by comparing voxel activity patterns across trials, separately for the encoding and delay phases. To relate sustained voxel activity patterns to WM accuracy, encoding period voxel patterns were then compared with delay period patterns within the same trial (Fig. 2). In all analyses, to obtain voxelwise localization of pattern similarity effects, a whole-brain searchlight pattern analysis approach was applied (Kriegeskorte et al., 2006). Pattern analysis was conducted in native functional space (3.4385 × 3.4375 × 3.4 mm voxels), and the resulting images were normalized and resliced to 2 mm³ voxels for group comparison.

Figure 2.

Pattern similarity searchlight analysis schematic diagram. Item and location representation analyses were conducted separately for WM encoding and delay phases: for each voxel in the brain, trial–pairwise correlation matrices were generated based on β values from a sphere centered on that voxel. Trial pairs were sorted once according to the level of object overlap and once according to the level of location overlap. Correlation coefficients were averaged within overlap category. Bound item–location representation analysis: for every trial, for each brain voxel, encoding and delay β values from a sphere centered on that voxel were vectorized and correlated. Trialwise correlations were sorted and averaged according to WM judgment accuracy at testing. cor, Correct trial; inc, incorrect trial.

Because a priori predictions focused on the MTL, voxelwise group statistics were first calculated only for MTL voxels using an anatomical mask constructed with the WFU Pickatlas (Maldjian et al., 2003), consisting of parahippocampal gyrus, hippocampus, uncus, and amygdala and dilated in three dimensions by 4 mm. Exploratory whole-brain group statistics were then calculated using a mask including all other gray matter voxels (MNI probabilistic tissue atlas, p(gray matter) ≥0.9), excluding voxels contained in the MTL mask. The reliability of voxelwise effects was determined based on permutation tests (separately for MTL and whole-brain results) using the Randomize function in FSL (10,000 permutations; Nichols and Holmes, 2002) with threshold-free cluster enhancement (TFCE; Smith and Nichols, 2009). The TFCE-based corrected voxelwise significance threshold was set at p_(FWE) <0.05 for all comparisons.

Object and location representations during encoding and delay.

For every voxel in the brain, a spherical region of interest (ROI) with a five-voxel diameter (∼8.5 mm radius) was constructed around that voxel. This searchlight size was determined to be sufficiently large to provide reliable pattern similarity estimates (21 voxels per searchlight, on average) but still small enough to be sensitive to voxel patterns contained within MTL subregions. Center voxels near the edge of the brain with spheres containing fewer than nine voxels were excluded from analysis. Separately for encoding and delay phase single-trial β images, parameter estimates across all brain voxels within a sphere were extracted for every trial, resulting in a 105-column matrix of activation pattern vectors for each sphere. Based on these activation pattern vectors, a trial-by-trial pairwise correlation matrix was calculated using Pearson's r. Trial pairs were then sorted according to the number of objects in common (zero, one, two, or three), independent of location, and the number of locations in common (zero, one, two, three, or four), independent of object content. Thus, every trial pair was categorized twice: once based on its level of object overlap, and once on its level of location overlap. Correlation coefficients were then averaged for each category of trial pairs. Average correlation coefficients were z-transformed and assigned to the center voxel of each sphere, resulting in whole-brain pattern information images corresponding to each level of object and location overlap.

To identify voxels where activity patterns contained information about the object or spatial location dimensions of stimuli, zero overlap pattern information images were subtracted from the images corresponding to the maximum level of object or location overlap. In the object analysis, the maximum level of overlap was three objects in common; in the location analysis, the maximum level of overlap was four locations in common. The resulting pattern information difference images were normalized to the MNI-152 template in a two-step process via rigid-body transformation to brain-extracted individual high-resolution structural images using FLIRT (Jenkinson et al., 2002) and entered into one-sample t tests in FSL.

The above searchlight analysis was designed to reveal brain regions that showed an increase in pattern similarity between the minimum (zero objects; zero locations) and maximum (three objects; four locations) levels of information overlap, independent of pattern similarity estimates at intermediate levels of information overlap. We expected that pattern similarity would increase monotonically and incrementally across each overlap level; however, it was also possible that these results were driven entirely by differences in pattern similarity at the endpoints. To better understand the response profiles of these brain regions, we extracted single-participant similarity estimates (mean z-scores) from each level of object and location overlap using spherical ROIs (with a 5 mm radius for anatomical specificity) centered on the peak voxels of clusters resulting from the searchlight analysis and performed an exploratory function-fitting analysis. Two mixed-effects regression models were fit to object and location similarity extractions, controlling for ROI, using maximum likelihood estimation and type III Wald χ² tests. The first model tested for the linear effect of location overlap; the second model tested for a “hockey-stick” effect with a threshold increase at the maximum overlap level. We then compared the fit of these two models to each set of extractions using the Akaike Information Criterion (AIC). Lower AIC score indicated better model fit, regardless of sign (positive or negative), and two models with a difference in AIC score of ≤7 were taken to be indistinguishable (Burnham and Anderson, 2002). To determine the relative contribution of linear and threshold effects to each information function, a third model was then fit to the data combining both linear and hockey-stick terms, and the change in model fit with the additional term was subjected to a χ² test. Additionally, to determine the selectivity of object and location coding in regions identified in the searchlight analyses, object similarity estimates were extracted from ROIs centered on location similarity searchlight peaks, and location similarity estimates were extracted from ROIs centered on object similarity searchlight peaks. These estimates were entered into full-factorial ANOVAs (type III sums of squares) with within-subjects factors overlap level (minimum vs maximum) and ROI.

To characterize the temporal dynamics of pattern similarity throughout the evolution of a WM trial, we also plotted “pattern similarity time courses” that provided a closer look at the time course of object and location information effects throughout the trial. Motion effects were removed from preprocessed time series data via general linear modeling, and activity patterns were extracted from each residual time point based on PRC and PHC voxels identified in the study phase searchlight analysis. On a time point-by-time point basis, we then calculated object and location pattern similarity indices (r_{maximum overlap} − r_{zero overlap}), where voxel patterns from a TR of 1 from each trial were correlated with voxel patterns at TR = 1 from all other trials, then again for a TR of 2, and so on, for all time points across a trial (as well as for several time points before trial onset and after trial offset). Time points from noisy trials (identified based on single-trial β estimates, as described above) were excluded. Resulting pattern similarity time courses were averaged across subjects and compared with trial phase timing, accounting for an expected hemodynamic response lag.

Encoding delay similarity as a function of WM test accuracy.

As in the object representation and location representation analyses, five-voxel-diameter spheres were constructed around every voxel in the brain, and voxelwise activation patterns within a sphere for the encoding and delay phase of each trial were vectorized. In this analysis, however, to evaluate the consistency of voxel patterns across encoding and delay phases, encoding pattern vectors were correlated with delay pattern vectors on a trial-by-trial basis, resulting in one correlation coefficient for each trial (excluding noise trials, as described above). Trials were then sorted according to whether the participant responded correctly or incorrectly at test, and encoding delay correlations were averaged within response type. Average correlation coefficients were z-transformed and assigned to the center voxel of each sphere, resulting in whole-brain pattern information images corresponding to encoding-delay similarity for correct and incorrect trials. To identify voxels where activity patterns contained information about bound object–location representations, incorrect pattern information images were subtracted from correct pattern information images. The resulting pattern information difference images were normalized as described above and entered into one-sample t tests in FSL.

Results

Object and location representation during WM encoding and delay

We first analyzed activity patterns during each WM trial to identify brain regions involved in the encoding and maintenance of the object or location information present in each memory set. We predicted that voxel patterns in PRC would carry information about the objects present in a scene, whereas PHC would carry information about the spatial arrangement of objects in a scene.

To identify areas of the brain involved in encoding of object information, we contrasted voxel pattern similarity between trial pairs with three objects in common against pattern similarity between trial pairs with zero objects in common. This contrast revealed that voxels in left PRC, amygdala, and ventral temporopolar cortex, as well as a small cluster of voxels in left posterior fusiform cortex, carried information about objects encoded in WM (Fig. 3). The corresponding analysis did not yield suprathreshold object-sensitive voxels during the delay period; however, exploratory analysis at a more liberal statistical threshold (p < 0.001, uncorrected) yielded voxel patterns carrying information about the objects being maintained across the WM delay period in right PRC (peak at 26, 2, −40 MNI coordinates) and ventral temporopolar cortex (peak at 24, 12, −34 MNI coordinates). Whole-brain analysis did not reveal any additional voxels coding for object information outside of the MTL.

Figure 3.

Distinct regions code for object and location information during WM encoding. Study phase searchlight results for greater pattern similarity between trial pairs with three objects compared with zero objects in common (red) in PRC, temporopolar cortex (TPC), and amygdala, and between-trial pairs with four locations compared with zero locations in common (blue) in PHC and posterior hippocampus. All p_(FWE) values <0.05.

To identify areas of the brain coding for spatial location information, we contrasted pattern similarity between trial pairs with four locations in common against pattern similarity between trial pairs with zero locations in common. This analysis revealed regions in left parahippocampal cortex and right posterior hippocampus that carried information about the locations being encoded in WM (Fig. 3). This contrast additionally yielded a small cluster of voxels in left posterior fusiform cortex that partially overlapped with object-sensitive voxels in that region. Exploratory whole-brain analysis yielded additional significant voxels coding for location information during the encoding period in left lingual gyrus (peak at −8, −80, 6 MNI coordinates) and lateral occipital complex (peak at −36, −80, 24 MNI coordinates). Analysis focused on the delay period yielded suprathreshold location-sensitive voxels in the left parahippocampal cortex and posterior hippocampus, but in more anterior regions compared with study phase location effects (Fig. 4). Whole-brain analysis yielded no additional significant location-sensitive voxels during the delay period.

Figure 4.

PHC and posterior hippocampus code for location information during active maintenance. Delay phase searchlight results for greater pattern similarity between trial pairs with four locations compared with zero locations in common (blue) in PHC and posterior hippocampus. All p_(FWE) values <0.05.

To more fully characterize the representation of object and spatial information in the areas identified by the searchlight, we extracted similarity estimates (z-scores) for each level of information overlap using sphere ROIs (5 mm radius) centered on the peak voxel for each of the above-mentioned brain regions. Visual examination suggested that object and location regions exhibited strikingly different response profiles (Fig. 5). We performed an exploratory function-fitting analysis to determine whether the response profiles of these regions were best described as incremental and approximately linear or as threshold-like functions driven by differences between endpoints. In PRC, amygdala, and temporopolar cortex, object similarity extractions were best fit by the linear function (χ²_{(1, N = 18)} = 20.89, p < 0.001, AIC = −1153.5), as opposed to the threshold function (χ²_{(1, N = 18)} = 9.17, p < 0.005, AIC = −1142.6). Moreover, the addition of a threshold term to the linear model did not significantly improve the model fit (χ²_{(3, N = 18)} = 0.62, p = 0.89, ΔAIC = −5.4). In PHC and posterior hippocampus, location similarity extractions were best fit by the threshold function (χ²_{(1, N = 18)} = 82.79, p < 0.001, AIC = −735.47), as opposed to the linear function (χ²_{(1, N = 18)} = 41.74, p < 0.001, AIC = −704.48), and the addition of a linear term to the threshold model did not significantly improve the overall model fit (χ²_{(2, N = 18)} = 0.97, p = 0.62, ΔAIC = −3.0). For both types of information, there were no significant interactions between function fit and ROI.

Figure 5.

Distinct information functions in regions that code for object and location information. Study phase pattern similarity in PRC, temporopolar cortex (TPC), and amygdala (Amyg) increases monotonically with increasing levels of object overlap, as best fit with a linear function. Study phase pattern similarity in PHC and posterior hippocampus (postHF) exhibits a threshold increase with increasing levels of location overlap, as best fit with an exponential function. Shaded areas represent the SEM across subjects at each overlap level for each ROI.

We conducted an exploratory analysis testing for subthreshold location sensitivity in the areas identified in the object searchlight and subthreshold object sensitivity in regions identified in the spatial searchlight. However, in location-sensitive regions (posterior hippocampus and PHC) there were no detectable differences in pattern similarity between trial pairs with zero and three objects in common, and in object-sensitive regions (PRC, amygdala, and temporopolar cortex) there were no detectable differences in pattern similarity between trial pairs with zero and four locations in common (F values <1.21, p values >0.05). A similar pattern of results was seen in object- and location-sensitive regions detected during the delay period.

To examine the dynamics of object and location information effects within a trial, on a time point-by-time point basis, we calculated across-trial pattern information indices using PRC and PHC voxels detected in the study phase searchlight analysis. Resulting subject-averaged pattern similarity time courses (Fig. 6) illustrated that object patterns in PRC and location patterns in PHC were instantiated early in the study phase and persisted across the delay phase, particularly for PHC location information, which is consistent with results from the searchlight analysis.

Figure 6.

Information coding in PRC and PHC is sustained across encoding and delay. Pattern similarity time courses are shown for object coding (r_{three objects in common} − r_{zero objects in common}) in PRC (red) and location coding (r_{four locations in common} − r_{zero locations in common}) in PHC (blue) voxels identified in the searchlight analysis. Positive pattern similarity change values indicate the presence of object or location coding. Trial onset begins at time = 0, and x-axis trial phase ranges are estimated based on the expected HRF response time.

Encoding delay pattern similarity as a function of WM decision accuracy

Another objective of the current study was to identify brain regions involved in the maintenance of encoded object–location relations across the memory delay period. We expected that accurate WM performance relied on maintenance of the object–location relations processed during the encoding period throughout the active retention delay (Hannula and Ranganath, 2008). Accordingly, we examined the fidelity of voxel patterns between the encoding and delay periods within the same trial, separately for accurate and inaccurate trials. More specifically, we calculated the similarity between the voxel pattern elicited during the study phase and the voxel pattern elicited during the delay phase, separately for each trial. Using a searchlight analysis, we identified regions where the mean pattern similarity values were higher for trials that were subsequently associated with correct WM decisions than for trials that were subsequently associated with incorrect decisions. We predicted that the hippocampus would be identified in this contrast, in light of theories suggesting a role for the hippocampus in object–location binding (Cohen et al., 1997; Davachi, 2006; Ranganath, 2010). Consistent with this prediction, we found that encoding delay pattern similarity in the right anterior hippocampus was higher for correct trials compared with incorrect trials (Fig. 7). No other MTL regions were identified in this contrast, and exploratory whole-brain analysis yielded no significant voxels outside of the MTL. No significant voxels were identified by the inverse contrast (incorrect > correct).

Figure 7.

Hippocampal pattern similarity between encoding and delay is predictive of accurate maintenance of object–location relations. Searchlight results for greater pattern similarity between encoding and delay phases of correct WM trials compared with incorrect trials. All p_(FWE) values <0.05.

Discussion

In the current study, we investigated whether MTL regions carry information about the spatial and object information that is currently relevant in a WM task. Results revealed that PRC, amygdala, and temporopolar cortex carried information about the objects encoded in each trial, regardless of the spatial arrangement of those objects. Information about spatial configuration was detected in PHC and posterior hippocampus, regardless of object content. Finally, the successful maintenance of object–spatial relations was associated with the integrity of pattern information across encoding and delay phases in the anterior hippocampus. These findings suggest that PRC, PHC, and the hippocampus are differentially involved in processing object, spatial, and object–location binding information during WM encoding and delay, which is consistent with models that are typically used to explain results in long-term memory (Davachi, 2006; Ranganath, 2010).

Several previous fMRI studies have contrasted MTL responses to object and scene (or space) stimuli using either univariate (Litman et al., 2009; Preston et al., 2010; Staresina et al., 2011, 2013; Hannula et al., 2013; Liang et al., 2013; Mundy et al., 2013) or multivariate (Diana et al., 2008; Hannula et al., 2013; Liang et al., 2013; LaRocque et al., 2013) analysis methods. However, in these studies, low-level visual features such as spatial frequency (Zeidman et al., 2012), retinal size (Sato and Nakamura, 2003; Arcaro et al., 2009), or textural information (Cant and Goodale, 2011; Cant and Xu, 2012) could have influenced differential PRC and PHC responses to object and spatial information. A critical, novel aspect of the current study is that all of the stimuli were complex scenes, and the same stimuli were entered into orthogonal object and spatial information analyses. The present results, therefore, suggest that functional distinctions between PRC and PHC transcend low-level stimulus characteristics and provide new evidence that these areas simultaneously extract information about objects and spatial configurations from complex visual input.

The distribution of object and spatial coding effects in this study corresponds well with what might be expected on the basis of resting-state functional connectivity studies of the MTL (Kahn et al., 2008; Libby et al., 2012), and with a recent framework proposing that PRC and PHC are components of two distinct corticohippocampal systems (Ranganath and Ritchey, 2012). According to this framework, PRC preferentially affiliates with an anterior temporal object-processing network that includes temporopolar cortex and amygdala, whereas PHC is preferentially connected with a posterior medial context-processing network including posterior hippocampus, lingual gyrus, and lateral occipital complex. In the current study, regions carrying information about objects were restricted to the anterior temporal system, and regions carrying information about spatial configuration were restricted to the posterior medial system. Previous work has shown that the intrinsic network organization of these regions predicts similarities in voxel pattern information during long-term memory encoding (Ritchey et al., 2014). The current study demonstrates that this distinction also applies to WM processing.

Analysis of pattern information functions revealed that, for object-coding regions, pattern similarity increased linearly with increasing overlap in object content. These results are consistent with the idea that each object within the scene was represented individually in PRC, amygdala, and temporopolar cortex. Although these regions likely perform different functions, single-unit recording studies in monkeys (Nakamura et al., 1992, 1994; Sato and Nakamura, 2003) and humans (Quiroga et al., 2005) have reported object-selective neural responses in these regions and evidence for viewpoint invariance, which would be particularly useful for the mental rotation demands of the current task. Single-unit recording studies have also reported evidence consistent with roles for amygdala and temporopolar cortex, along with PRC, in recognition and/or maintenance of individual objects across delays (Wilson and Rolls, 1993; Nakamura and Kubota, 1995).

In areas of PHC and posterior hippocampus that were sensitive to spatial location encoding, pattern information functions were consistent with coding of coherent spatial configurations. Pattern similarity in these areas was selectively increased for trial pairs in which the entire set of spatial locations was repeated. These results align with univariate fMRI studies showing that PHC signal tracks even slight changes in spatial configurations of objects over both short (Pihlajamäki et al., 2005) and longer (Düzel et al., 2003) delays, and with neurophysiological studies in rodents showing that place field (O'Keefe and Dostrovsky, 1971) coding in dorsal hippocampus (possibly homologous to human posterior hippocampus) undergoes dramatic remapping between different spatial contexts (Bostock et al., 1991; Lever et al., 2002; Wills et al., 2005). Furthermore, the data provide novel support for frameworks suggesting that MTL regions (including the hippocampus) are involved in high-level perceptual, as well as mnemonic, processing (Bussey and Saksida, 2007; Epstein, 2008; Graham et al., 2010; Aly et al., 2013).

Using a univariate fMRI approach, Hannula and Ranganath (2008) showed that the overall activation magnitude in the hippocampus during encoding predicted accurate WM decisions, but this effect was not detected during the delay phase. In contrast, in the multivariate analyses reported here, the fidelity of pattern information across encoding and delay in the anterior hippocampus distinguished between correct and incorrect trials. Although this analysis does not unequivocally reveal the kind of information represented by the hippocampus, we note that accurate performance on the WM task required the maintenance of detailed information about the unique set of object–location bindings on each trial. Accordingly, the evidence is consistent with hippocampal involvement in the encoding and maintenance of bound item–spatial relations (Hasselmo and Stern, 2006; Kumaran, 2008; Ranganath et al., 2014). The current result aligns with recent neuropsychological (Hannula et al., 2006; Olson et al., 2006; Finke et al., 2008; Yee et al., 2014) and neuroimaging (Piekema et al., 2006; Hannula and Ranganath, 2008; Cashdollar et al., 2009; Lee and Rudebeck, 2010; Poch et al., 2011) evidence for hippocampal involvement in item–spatial relational binding during WM, and supports theories of a role for the hippocampus in item–context binding across cognitive domains (Ranganath, 2010).

Previous studies have shown that the content (Harrison and Tong, 2009; Serences et al., 2009; Riggall and Postle, 2012) and precision (Emrich et al., 2013) of actively maintained information can be decoded from voxel activity patterns in sensory cortex. Although we found that voxel patterns in visual cortical areas carried spatial information during WM encoding, during the delay period, information coding was pronounced only in the MTL; sensory (i.e., visual cortical) regions were not identified in the reported contrasts. One potential explanation for this effect is that our task differed from typical WM tasks, in that it required maintenance of a complex, relational representation, which may particularly require MTL regions (Barense et al., 2007; Pertzov et al., 2013; Yonelinas, 2013; Yee et al., 2014).

Because this experiment was originally designed to examine univariate activation magnitude differences between trials, we could not address certain relevant questions about MTL coding for item and spatial information. Most notably, because no two trials contained four objects in common, it is unknown whether, if trial pairs contained a full set of overlapping objects, the increase in object coding would be graded in proportion with the current findings, or whether the increase would be disproportionate, similar to the threshold-like spatial configuration coding observed in posterior regions for four locations in common. Additionally, because no two trials contained more than two of the same objects in the same locations, we were unable to investigate neural coding for item–spatial binding separately for study and delay phases, as we did with item and spatial information. However, based on differences in the study delay pattern similarity between correct and incorrect trials, we predict that sensitivity to item–spatial relations would be identified in anterior hippocampus during both study and delay. These limitations could be addressed in future studies designed specifically for multivoxel pattern similarity analysis.

The involvement of PRC, PHC, and hippocampus in WM has long been the subject of debate (Alvarez-Royo et al., 1992; Cave and Squire, 1992; Alvarez et al., 1994; Baddeley et al., 2011; Jeneson and Squire, 2012). The current study not only demonstrates that these regions are involved in WM for relational information, but also provides evidence that each region codes for different features of stimuli during WM encoding and delay. These results bear a striking resemblance to findings from long-term memory studies (Davachi et al., 2003; Pihlajamäki et al., 2005; Diana et al., 2007; Staresina and Davachi, 2008; Litman et al., 2009; Preston et al., 2010; Howard et al., 2011; Staresina et al., 2011; Hannula et al., 2013; Liang et al., 2013) and converge with studies of MTL involvement in perception (Lee et al., 2006a,b; Barense et al., 2007; Bussey and Saksida, 2007; Murray et al., 2007; Graham et al., 2010; Aly et al., 2013). Together, a growing literature suggests that the best characterization of MTL subregion functional differences over either short or long delays may be information based, with PRC and PHC as hubs of distinct cortical systems for processing item and context information (Ranganath and Ritchey, 2012; Ritchey et al., 2014), and the hippocampus binding item and context information (Cohen et al., 1997; Davachi, 2006; Ranganath, 2010).