Hippocampal structure predicts cortical indices of reactivation of related items

Abstract

One of the key components of relational memory is the ability to bind together the constituent elements of a memory experience, and this ability is thought to be supported by the hippocampus. Previously we had shown that these relational bindings can be used to reactivate the cortical processors of an absent item in the presence of a relationally bound associate (Walker et al., 2014). Specifically, we recorded the event-related optical signal (EROS) when presenting the scene of a face-scene pair during a preview period immediately preceding a test display, and demonstrated reactivation of a face-processing cortical area (the superior temporal sulcus, STS) for scenes that had been previously paired with faces, relative to scenes that had not. Here we combined the EROS measures during the same preview paradigm with anatomical estimates of hippocampal integrity (structural MRI measures of hippocampal volume and diffusion tensor imaging measures of mean fractional anisotropy and diffusivity) to provide evidence that the hippocampus is mediating this reactivation phenomenon. The study was run in a sample of older adults aged 55–87, taking advantage of the high amount of hippocampal variability present in aging. We replicated the functional reactivation of STS during the preview period, specific to scenes previously paired with faces. Crucially, we also found that this phenomenon is correlated with structural hippocampus integrity. Both STS reactivation and hippocampal structure predicted subsequent recognition performance. These data support the theory that relational memory is sustained by an interaction between hippocampal and cortical sensory processing regions, and that these functions may be at the basis of episodic memory changes in normal aging.

Introduction

It has long been known that, when a person recollects an event, s/he does not simply remember a recording of it but, rather, a reconstruction of multiple elements of that memory stored in various parts of the cortex (Norman and O’Reilly, 2003). During this reconstruction the same cortical processors that were active at the initial encounter are reactivated (Marr, 1971, Norman and O’Reilly, 2003, Johnson and Rugg, 2007, Rugg et al., 2008, Hofstetter et al., 2012). For example it has been shown that the areas of cortex that were used to process a face or a location are again active when a person is asked to remember those items (O’Craven and Kanwisher, 2000). This phenomenon is not specific to faces or locations but has been demonstrated across a wide range of stimulus types such as colors (Simmons et al., 2007), tools (Chao et al., 2002), and words (Johnson et al., 2009, Hofstetter et al., 2012), to name a few. Recently we and others were able to show that not only individual items can be reactivated in the cortex, but that items can reactivate other relationally-bound items from the same event (Hofstetter et al., 2012, Staresina et al., 2012, Zeithamova et al., 2012, Oudiette et al., 2013, Staresina et al., 2013, Walker et al., 2014). It has been hypothesized that the hippocampus is a critical structure in the process of storing and retrieving the multiple pieces of information constituting a relational memory (Cohen and Eichenbaum, 1993, Eichenbaum, 2000, Eichenbaum and Cohen, 2001, Norman and O’Reilly, 2003). Although the importance of the hippocampus in relational memory is supported by a large amount of data (Hannula et al., 2006, Konkel et al., 2008, Watson et al., 2013), its critical role in reactivating relationally bound items has yet to be demonstrated. In this paper we demonstrate this link by showing that variability in hippocampal volume and connectivity in normally aging older adults is highly correlated with the extent of reactivation of cortical representations and with a person’s ability to reactivate related information.

For the purposes of this paper we define reactivation as the activation, during retrieval, of the same cortical processor(s) used during the initial presentation of that item. Furthermore, we are interested in reactivation of relationally-bound information. Some studies investigating reactivation used some type of semantic cue that was known to participants prior to the experiment (e.g., the name of the object) in order to elicit reactivation. In such cases, however, reactivation could simply be the result of a semantic association established over a long period of time, and not specifically linked to a particular episode. In the case of relational memory, instead, we are interested in reactivation of an associated item after the presentation of another item arbitrarily paired with it during a single study episode. Evidence for relational memory in this case would therefore come from demonstrating the reactivation of cortical processors related to the processing of one item elicited by the presentation of the episodically-paired second item, even in the physical absence of the first item. In this case, evidence for relational memory reactivation would come from finding that a particular item elicits activation of a cortical region not normally involved in its processing, but involved instead in the processing of a stimulus type that was paired with it in a single previous episode.

There are strong theoretical bases for the involvement of the hippocampus in relational memory. It is generally accepted that the hippocampus is important in the formation and retrieval of declarative memories (Cohen and Squire, 1980). The hippocampus is believed to relationally bind together and store arbitrary associations (Cohen and Eichenbaum, 1993, Eichenbaum, 2000, Eichenbaum and Cohen, 2001). A considerable body of empirical evidence demonstrates the critical role of the hippocampus in creating and storing flexible associations after just one exposure (Hannula et al., 2006, Hannula et al., 2007, Konkel et al., 2008, Hannula and Ranganath, 2009, Warren et al., 2010, Zeithamova and Preston, 2010, Duff et al., 2013). Furthermore, it is thought that the hippocampus can then use these relational bindings to reactivate an item in the presence of a relationally bound associate (Cohen and Eichenbaum, 1993, Eichenbaum, 2000, Eichenbaum and Cohen, 2001, Norman and O’Reilly, 2003).

We (Walker et al., 2014) were able to show evidence for reactivation in a study using a face-scene preview paradigm. In this paradigm, unique face and scene exemplars (novel and never repeated in the course of the study) are presented together at encoding. At test, for each trial, a scene is presented ahead of the test display (scene preview). We found that scene previews that were previously studied with a face showed reactivation of the same face processing regions found to be active when encoding those faces. Crucially, such reactivation was not found for novel scenes not previously paired with faces. We termed this type of reactivation “relational reactivation.” Using a similar paradigm, Hannula and Ranganath (2009) had participants study pairs of faces and scenes and then tested the participants using a three-forced-choice recognition task to identify which face went with a scene, with a scene preview immediately prior to the test display. They found that hippocampal activity was related to later performance during the scene preview but not during the actual test display. Taken together these two studies lead to the prediction that hippocampal activity is associated with reactivating the face that was originally paired to the scene.

Others have reported similar evidence of hippocampal mediation of the relational reactivation process (Hofstetter et al., 2012, Staresina et al., 2012, Zeithamova et al., 2012, Staresina et al., 2013, Gordon et al., 2014). These studies show increased functional connectivity between the hippocampus and other parts of the medial temporal lobe (MTL) thought to be responsible for specific processing of stimuli during retrieval. Zeithamova et al. (2012) were able to show a correlation between activity in the anterior MTL and an overall pattern of reactivation in the ventral visual stream when a participant was imagining a related item, indicating the possibility of an association between hippocampal activity and the overall pattern of reactivation. Similarly, Gordon et al. (2014) also found a correlation between hippocampal activity and reactivation of patterns of activity associated with people and places.

The current study extends previous research by examining whether structural hippocampal integrity is associated with the degree of reactivation of paired memory representations in the cortex. There is a strong link between hippocampal volume (controlling for intracranial volume) and overall relational memory performance (Maguire et al., 2000, Erickson et al., 2009, Chaddock et al., 2010). Furthermore, measures of water diffusion in the hippocampus such as fractional anisotropy (FA) and mean diffusivity (MD), both thought to index white matter integrity, have also been linked to overall and associative memory ability. Specifically, individuals with high mean FA and low MD in the hippocampus perform better across a range of memory tasks (Charlton et al., 2006, Carlesimo et al., 2010), although not everyone has found a link between mean FA and memory performance (Carlesimo et al., 2010). Here we examined whether hippocampal structure, as measured through hippocampal volume, mean FA, and MD, was associated with the ability to relationally-reactivate representations in the cortex. In order to maximize hippocampal variability we chose to use older adults in our study. As people age, their hippocampi starts to atrophy, and there is also evidence of white matter degradation (Raz et al., 2005, Walhovd et al., 2005, Charlton et al., 2006). These changes create greater variability in both hippocampal size as well as white matter measures among adults, especially older adults. Older adults are also known to show decline in episodic memory, albeit with a large variability across individuals (Morcom et al., 2003, Buckner, 2004, Duverne et al., 2009, Fabiani, 2012). It is this greater variability that we harnessed to test whether hippocampal structure is associated with the ability to relationally reactivate items in the cortex.

To observe reactivation we employed a modified version of the paradigm used by Walker et al. (2014). In this paradigm participants study pairs of faces and scenes and then are tested on those pairs using a yes/no recognition task. As in the paradigm employed by Hannula and colleagues (2006, 2009), the critical aspect is that prior to every test display there is a scene preview. However, in our paradigm, instead of having only old scenes, some of the scenes are novel (i.e., never studied with a face before). By contrasting old scenes that were previously paired with a face and novel scenes that were never paired with a face, we can examine the extent to which participants are reactivating the face representation areas during the scene preview. In order to examine both the temporal and spatial dynamics of that reactivation we used the event related optical signal (EROS, Gratton and Fabiani, 2010). This technique uses a combination of temporal and spatial resolution to determine not only “where” activity is taking place but also “when” the activity is taking place, allowing investigators to examine the order of activation of various areas of the brain, instead of just establishing that those areas were active during a particular trial type.

In this experiment participants studied pairs of faces and scenes, first viewing either a face or a scene individually followed by the pair together (see Figure 1). At test, participants were given an old/new recognition test for each of the face-scene pairs, each preceded by a scene preview. We found activity during scene previews in the posterior superior temporal sulcus (STS). The STS is an area known to be part of the network involved in processing faces, being shown to be active during both general face processing (Puce et al., 1998, Puce et al., 2003, Grill-Spector et al., 2004, Fairhall and Ishai, 2007) as well as in social judgments about a face (Puce et al., 1998, Hoffman and Haxby, 2000). It is also easily accessible by our imaging technique (which has limited penetration inside the head). The activity was greater for “old” scene previews compared to new or “novel” scene previews (which had no face associated with them), and was elicited in the same region that was activated by faces presented alone during the study phase (localizer) even though no faces were present. Critically, this reactivation was associated with hippocampal volume, FA, and MD such that those older adults with smaller hippocampi and less intact white matter tracts within the hippocampus were impaired at reactivating the faces. Further, all three measures were related to episodic memory performance.

Figure 1.

Schematic representation of the experimental paradigm, illustrating the sequence of stimuli during study (a), where the top row represents a face-first trial and the second row represent a scene-first trial, and test (b). Note that during test scenes were always presented first, generating a scene preview period.

Materials and Methods

Participants

Eighteen right-handed older adults participated in this study for a payment of $15 an hour. Three participants were excluded from the analysis due to withdrawal from the experiment prior to completion, leaving a total of 15 participants (8 women; mean age = 68.30 SD= 8.90; age range: 55–88 years old). Demographic information, including scores on the he Wechsler Memory Scale – Third Edition that was given one week prior to optical data collection, can be found in Table 1. All participants indicated that they had normal or corrected to normal vision and were not taking medications that would affect the central nervous system. Informed consent was obtained from each participant and all procedures were approved by the University of Illinois Institutional Review Board.

Table 1.

Mean (with Standard Deviation in Parentheses) Demographic Characteristics

Measure	M	SD
Age (years)	68.30	8.90
Education (years)	15.45	3.66
Wechsler Memory Scale	117.33	13.02

Stimuli

The stimuli consisted of 444 full-color face images (294 female faces) selected from a previously normed faces database (Althoff and Cohen, 1999) and 592 scenes from Brand X© photography. The faces were all sized to 384 × 384 pixels and the scenes were all sized to be 1024 × 768 pixels, filling the entirety of the screen.

Procedures

After signing an informed consent form, each participant was fit with an EROS recording helmet (see below) and were given a practice block of 12 study trials followed by 12 test trials so that s/he could get used to the timing of the trials. During the practice block, participants were given instructions to create a story linking the face and the scene together and to use that story to retrieve that same association during test. Following the practice block the participant completed six study/test blocks; they were allowed to take breaks in between blocks as necessary.

Study Block

Study blocks consisted of 72 study trials, divided into two sets of 36 trials. Each set of study trials started with a 1-s fixation cross. As can be seen in Figure 1A, the study trials started with a face and scene being shown individually for 750 ms each. Half the study trials had the face shown first (“face-first” trials) and half the trials had the scene shown first (“scene-first” trials). After the face and scene were shown individually, the face was superimposed on the center of the scene for 1500 ms. Participants were instructed to study each of the pairings as they were to be tested on those pairings later on. The blocks were divided by short breaks to minimize movement artifacts.

Test Block

Following each study block was a corresponding test block of 72 test trials, testing the pairs of items studied in the immediately preceding study block. These test trials were divided up into three sets of 24 test trials, with each set beginning with a 1-s fixation cross. As can be seen in Figure 1B, a test trial consisted of a scene being presented for 1000 ms (the scene preview), followed by a fixation cross for 1000 ms, followed by the face superimposed on the center of the scene for 2500 ms. Participants were instructed to respond using a button box as to whether or not a specific face-scene pair was studied during the previous study block (old-new judgment). There were three types of test trials: match, re-pair, and novel. Match test trials were test trials in which the face and the scene being tested had been presented together during the study phase. The re-pair test trials were comprised of faces and scenes that had been presented in the study phase, but had not been paired together. The novel test trials were comprised of a novel scene with a previously studied face. The correct response for match trials was “old” as those pairs were previously studied together, whereas the correct response for re-pair and novel trials was “new” as the pairs in those trials were not studied together. Participants were asked to respond only once the face had appeared, and were explicitly told not to respond during the scene preview or fixation. Every test trial ended in a fixation screen presented for 1500 ms.

Counterbalancing for the study and test blocks consisted of four lists of 432 face-scene pairings created by randomly pairing 432 of the faces and 432 of the scenes and then randomly assigning each pair to each study and test type condition. Each pair was then randomly assigned to one of six blocks of 72 items each with the stipulation that each block contained 12 of each study-test type combination ([face first, scene first]x[match, re-pair, novel]). Of the remaining scenes, 144 were then randomly assigned to replace the scenes in the novel test trials. Four different lists were created to ensure that every scene was tested in one of the three categories. No scene and face were paired together more than once across these lists. The remaining 12 faces and 16 scenes were used to create a practice block of 12 study trials followed by 12 test trials. These faces and scenes were not included in the four lists and the practice trials were the same for all participants.

Optical Recording

Optical data were recorded using six synchronized ISS model 96208 frequency domain oxymeters (Imagent®; ISS, Inc., Champaign, IL). The light sources were laser diodes emitting light at the wavelength of 830 nm (max amplitude: 10 mW, mean amplitude after multiplexing: 1 mW) modulated at 110 MHz. Optic fibers were used to channel each light to the surface of the scalp. The detectors were fiber optic bundles (diameter = 3 mm) connected to photomultiplier tubes (PMTs). The PMTs were fed with a current modulated at 110.0625 kHz, generating a heterodyning frequency of 6.25 kHz. The output current from the PMTs was digitized at 50 kHz, affording 8 points per heterodyning cycle. A time-multiplexing approach was used to record from sixteen sources for each detector. In this approach, each source was switched on for 1.6 ms, and off for 24 ms. This allowed to record for a total of 10 heterodyning cycles (80 points) for each multiplexing time unit. However, to avoid cross-talk, the first two cycles were discarded, and the remaining 64 points were subjected to a fast Fourier transform for computation of DC (average) intensity, AC (amplitude), and relative phase delay (in degrees and later converted to picoseconds). Only phase delay data are reported here.

Source and detector fibers were mounted on a modified motorcycle helmet. Our montage consisted of 24 detectors and 64 sources (see Figure 2), covering most of the cortical surface (darker grey areas in figure 3). Source-detector distances ranged between 15 and 94 mm. To avoid cross talk, the sources were arranged such that during any given time division of the multiplexing cycle only one source was within 6 cm of any given detector. This allowed us to record from 384 channels (pairings of source and detector) at 39.0625 Hz.

Figure 2.

Locations of the optical sources (red) and detectors (orange) used for recording the event-related optical signal (EROS) in a representative subject.

Figure 3.

Spatial maps based on group level Z-statistics of the EROS data projected on sagittal brain surfaces (left hemisphere view). Dark gray shading represents the brain area sampled by the recording montage. The light green rectangle indicates the STS ROI. (a) Activity during study trials for face-first trials versus scene-first trials in the left STS at 409 ms and 435 ms. (b) Activity during the scene preview for previously studied scenes versus completely novel scenes in the left STS at 742 ms and 767 ms.

The locations of the sources and detectors were digitized with a Polhemus “3Space” ® (Colchester, VT) 3D digitizer and co-registered with a volumetric T1-weighted MR image for each subject (Whalen, Maclin, Fabiani, & Gratton, 2008). The co-registered data were then Talairach-transformed to permit registration across subjects. The phase data were corrected offline for phase wrapping, pulse artifacts were removed (Gratton and Corballis, 1995), and the data were low-pass filtered to 5 Hz (Maclin et al., 2003). Channels with standard deviations of the phase greater than 150 ps were excluded from further analysis (for further details of these analytic steps, see (Gratton and Fabiani, 2007).

Optical Statistical Analyses

The phase data were divided into epochs around stimulus events of interest with 204.8 ms pre-stimulus baseline and 768 ms post-stimulus recording for the study phase. The time locking event of interest was the onset of the first stimulus of a study trial (either the face in trials where the face was presented first or the scene in trials when the scene was presented first). For the test phase the phase data were also divided into epochs around the stimulus events of interest with a 204.8 ms baseline, but the post-stimulus recording consisted of 2022 ms so that it could include the scene preview and the fixation cross, both shown before the face is presented in the test trial.

In-house software “OPT-3D” (Gratton, 2000) was used to reconstruct the optical path for each channel spatially, combine channels whose mean diffusion paths intersected for a given brain volume (voxel) and to compute group-level statistics. The resel size of the cortical projections were determined by the independence of the error terms at various voxel distances computed using the methods described by Worsley et al. (1999). An 8-mm Gaussian filter (based on a 2 cm kernel) was used to spatially filter the data. The group-level statistics were then converted to Z-scores and compared to critical Z-scores based on the number of resels within an ROI and the subsequent correction for multiple comparisons. These Z scores are then orthogonally projected onto images of the sagittal surfaces of the brain in Talairach space (Talairach and Tournoux, 1988).

Due to its high spatial and temporal resolution (which may inflate the number of comparisons), statistical analysis of EROS during both the study and test phase was limited to ROIs selected a priori. Whole-brain analyses, as are often done in fMRI, which has only high spatial resolution, are not practical with EROS data as the number of data points (one for every resel at every time point) would make the correction for multiple comparisons too severe. Thus we focused on the STS and the DLPFC, the areas that were shown to be involved in reactivation in younger adults (Walker et al., 2014) and that have been shown to be important in the processing of faces (Puce et al., 1998, Puce et al., 2003, Grill-Spector et al., 2004, Fairhall and Ishai, 2007) and in the top-down control of memory retrieval (Miller and Cohen, 2001), respectively. These areas are also easily accessible with optical imaging, whereas other potential areas of interest such as the fusiform gyrus and the ventrolateral prefrontal cortex could not be accessed with the instrumentation used in the current study (For the spatial extent of areas covered see Figures 3–6). As in Walker et al. (2014), the boundaries for the STS ROI were defined in Talairach space (Talairach and Tournoux, 1988) as y = −65 to −43, and z = −8 to 20 and y = −72 to −45, and z = −3 to 24 for the left and right STS, respectively. These boundaries were based on the peak activation in the STS for faces in previous fMRI work (Bonda et al., 1996, Kanwisher et al., 1997, Puce et al., 1998, Haxby et al., 1999, Hoffman and Haxby, 2000, Ishai et al., 2000, Puce et al., 2003, Grill-Spector et al., 2004, Ishai et al., 2005, Fairhall and Ishai, 2007). The boundaries for the DLPFC ROI were defined as y = 10 to 50, and z = 15 to 35 for both the left and right DLPFC. These boundaries were based on the spatial extent of Brodmann’s areas 9 and 46.

Figure 6.

(a) Mean right hippocampal FA and (b) mean hippocampal MD are significantly correlated with the average relational-reactivation EROS activity in STS.

Additionally, in order to control for multiple comparisons we also limited our analyses to temporal intervals of interest (IOIs) at both study and test. Previous work using intracranial event-related potentials and EROS has shown activity in the STS to the presentation of faces at around 170ms, between 200 and 650 ms and around 700 ms (Allison et al., 1999, Walker et al., 2014), therefore for our analyses at study we used an IOI of 150 to 750ms. As in Walker et al. (2014) we limited our scene preview (i.e., reactivation) analyses to the time range of 500–1500 ms. This is based on work that has shown that eyes disproportionately start to fixate on the matching face in the time range of 500–1500 ms in a similar paradigm if more than one face is present (Hannula et al., 2007). However, for exploratory reasons we also report any activity that was found to be significant or marginally significant with a correction for multiple comparisons (see Table 3). Correction for multiple comparisons across voxels was applied based on the number of independent resolution elements (resels) within each ROI using random field theory (Friston et al., 1995, Gratton, 2000, Maclin et al., 2003).

Table 3.

Locations and Statistical Analyses for Areas Found to be Active During Study (Faces > Scenes) and Scene Preview (Old Scenes >New Scenes)

Study
Location	Time Period	Location (y,z)	Zobs	Zcrit
Left STS	409–435 ms	(−46, 1)	2.651	2.64
Scene Preview
Location	Time Period	Location	Zobs	Zcrit
Left STS	742 ms	(−58, 7)	2.323	2.64
	767 ms	(−58, 9)	2.692	2.55
	972 ms	(−44, −3)	2.415	2.51
	1561 ms	(−44, −3)	2.572	2.60
	1843 ms	(−44, 7)	3.711	2.84
Left DLPFC	255 ms	(32, 33)	−2.777	−2.68
	563–588 ms	(42, 29)	−3.011	−2.77
	742 ms	(12, 32)	−2.702	−2.85
	1612 ms	(48, 24)	−2.593	−2.90
Right DLPFC	255 ms	(17, 33)	2.833	2.83
	614 ms	(9, 32)	2.871	2.86

Structural Analyses

Images were collected on a Siemens Magnetom Trio 3T whole body MRI scanner. A standard 12-channel birdcage head coil was used and head motion was restricted with foam padding. High-resolution 3D MPRAGE (TI = 900ms; flip angle = 9°; .9 mm isotropic voxels) structural images were acquired parallel to the anterior commissure-posterior commissure (AC-PC) axis. Structural scans were acquired 3–11 months (average = 7.2 months) prior to optical data collection for the previous study of blood flow(Zimmerman et al., 2014).

Automatic segmentation of the hippocampus was performed using Freesurfer (v 5.3; details about the subcortical segmentation process have been described in Fischl et al. (2002) and Fischl et al. (2004). Intracranial volume (ICV), also calculated by Freesurfer, was used to correct for overall head size (see Buckner (2004). By regressing each ROI volume onto ICV, a slope (b) was obtained for the relationship between ROI and IVC. The resulting slope was used to normalize each volume for head size (normalized volume = raw brain volume – b(IVC- mean IVC);(Raz et al., 2005, Erickson et al., 2009, Head et al., 2009).

Diffusion-Weighted Analyses

Diffusion-weighted images were acquired during the same session as the structural scan. Due to technical difficulties data from one of the participants was lost. The diffusion-weighted images were acquired with a TR = 4,400 ms, TE = 98 ms, and 1.72 mm² in-plane resolution. Thirty-two 3 mm slices were obtained parallel to the anterior-posterior commissure plane with no inter-slice gap. The protocol consisted of four T2-weighted images (b-value = 0 s/mm²) followed by two repetitions of 30-direction diffusion-weighted echo planar imaging scans (b-value = 1,000 s/mm²).

Preprocessing and analyses were performed using tools from the FDT (Functional MRI of the Brain Diffusion Toolbox;(Behrens et al., 2003a, Behrens et al., 2003b) from FSL (Functional MRI of the Brain Software Library, v 5.0, (Smith et al., 2004, Jenkinson et al., 2012). Eddy-current distortions were corrected using affine registration of all volumes to a target volume with no diffusion weighting. The hippocampal ROI was used from the Free-Surfer segmentation of the structural MRI. Average FA and MD values were calculated by averaging across all of the voxels in the hippocampal ROI mask from the FA and MD maps, respectively, created using FDT.

Statistical Analysis

In order to control for the possible effects from demographic variables, all correlations reported are partial correlations, controlling for age, sex, and years of education.

Results

Behavioral Results at Test Display

Table 2 provides accuracy and reaction times for each condition. Overall participants made correct old-new judgments on 64% (M = .64, SD = .11) of the trials with an average response time of 1298.08 ms (SD = 121.59). There was an overall effect of test trial type on both accuracy [F (2, 42) = 4.52, p = .016], and response time [F (2, 42) = 47.69, p < .001]. Bonferroni-corrected pair-wise t tests showed that participants were marginally more accurate on novel trials (M = .75; SD =.16) than match/”old” trials (M = .64; SD = .19; t (14) = 2.19, p = .023), and were significantly more accurate on novel trials than re-pair trials (M = .55; SD = .19; t (14) = 2.69, p = .009). There was no difference between match or re-pair trials [t (14) = .0974, p = .103]. In terms of reaction time, participants were faster to respond during novel trials (M =1078 ms; SD = 249) than either match (M = 1404 ms; SD = 184; t (14) = −3.47, p = .002) or repair trials (M = 1411ms; SD = 236; t (14) = −4.0774, p = <.001). There was no difference in response time between match trials and re-pair trials, t (14) = −0.10, p = .46. The decrease in reaction time and the increase in accuracy for novel compared to either the match or re-pair trials could be due to the difference in item types at the test display. Both the match and re-pair trials required relational memory to detect whether or not the pair was old whereas novel trials only needed the participant to detect that the scene was novel.

Table 2.

Proportion of Correct Recognition Responses and Response Times for Each Trial Type

	Response Accuracy		Response Time (ms)
Test Trial Type	M	SD	M	SD
Match	.64	.19	1404	184
Re-Pair	.55	.19	1411	236
Novel	.75	.16	1078	249

Analysis of EROS Results

Activity at Study

The purpose of this analysis was to determine, before relational memory is established, that certain brain regions (and in particular the STS) only show activation after the presentation of faces and not of scenes (localizer task). This was achieved by examining the EROS data collected during the study phase, and contrasting the activity elicited by faces in the face-first trials with the activity elicited by scenes in the scene-first trials. Figure 3a shows the statistical maps of EROS data for the contrast faces (first) > scenes (first) at study. As expected this contrast revealed significant preferential activation for faces over scenes in the left STS. This preferential activation took place from 409–435 ms (peak Z = 2.651, Z_crit = 2.64; Talairach coordinates: y = −46, z = 1). Subsequent analyses revealed that these differential effects were due to activation (compared to a pre-stimulus baseline) in the face-first condition, and not to a significant negative deviation from baseline for the scene-first condition. At no point during study did the STS show activation levels for scenes that were significantly greater than baseline values. The statistical analyses are summarized in Table 3.

Activity at Test During Scene Preview

This analysis was run to determine whether participants are using the scene preview to reactivate the associated face in preparation for the upcoming test display. Specifically, this was accomplished by contrasting the EROS activity elicited by the preview of scenes that were previously paired with faces (i.e., those presented during the preview for match and re-pair trials) with that elicited by scenes that were not previously paired with faces (novel scenes). The focus was on face-specific areas (i.e., STS), which should show activation in the first condition, but not the second. Figure 3b shows the peak activations of this contrast in each significant time interval (see also Table 3 for the results of the statistical analyses). The data confirmed our predictions: there was significant activation in the left STS at around 767 ms after scene preview onset (peak Z = 2.692, Z_crit = 2.55; Talairach coordinates: y = −58, z = 9), and marginally significant activation in the left STS at around 742 ms (peak Z = 2.323, Z_crit = 2.64; Talairach coordinates: y = −58, z = 7) and again at 972 ms (peak Z = 2.415, Z_crit = 2.51; Talairach coordinates: y = −44, z = 3).

A conjunction analysis (Price and Friston, 1997) was performed to determine the amount of overlap between the left STS activity for faces during study and for old scene previews at test. As we predicted, we did find overlap between the study (409–435 ms) and scene preview (767 ms) activations at a combined threshold of p < .04. As can be seen in Figure 4, the overlap between the two activations takes place toward the center of the left STS region of interest (ROI).

Figure 4.

Spatial map of the conjunction analysis projected onto the left sagittal surface at a threshold of p < .04 for the conjunction (p < .2 for each condition separately). Blue represents the activity for faces > scenes from 409–435 ms during study, green represents the activity for old scenes > new scenes from 742 – 767 ms during the scene preview, and red represents the overlap between those two activations.

Additionally, EROS activity in DLPFC (BA 9, 46) was also examined as this region has been shown to be active in this paradigm before (Walker et al., 2014) and is well known to be involved in top-down or controlled processing (Miller and Cohen, 2001). Significantly greater activation in the left DLPFC for new scenes greater than old scenes was found early on during the scene preview around 255 ms (peak Z = −2.777, Z_crit = −2.68; Talairach coordinates: y = 32, z = 33) and from 563–588 ms (peak Z = −3.011, Z_crit = −2.77; Talairach coordinates: y = 42, z = 29). A marginally significant effect in the opposite direction was observed around 742 ms (peak Z = −2.702, Z_crit = −2.85; Talairach coordinates: y = 12, z = 32). However, in the right DLPFC an opposite pattern (with greater activity for old scenes than new scenes) was observed around 255 ms (peak Z = 2.833, Z_crit = 2.8; Talairach coordinates: y = 17, z = 33) and 614 ms (peak Z = 2.871, Z_crit = 2.86; Talairach coordinates: y = 9, z = 32). A summary of the results of the statistical analyses of the EROS data is presented in Table 3.

Functional Connectivity (Cross-Correlational) Analyses

Forward- and backward-lagged cross-correlation analyses were performed to elucidate the possible connections between the activity observed in the prefrontal cortex and that observed in the STS. For these analyses, the STS voxel showing the peak value in the old-new preview scene contrast at a latency 767 ms was selected as the seed point. The time course of the activity at this seed voxel was correlated with that in other voxels, using different lags (negative for backward cross-correlations, and positive for forward cross-correlations). The backward correlation analysis (in which activity in other regions predicted the seed voxel activity) revealed a significant correlation between activity in the seed voxel (left STS) and prior activity in the left DLPFC at a lag of −179 ms (peak Z = 4.328, Z_crit = 3.32); marginally significant correlations were observed at lags of −281 ms (peak Z = 2.829, Z_crit = 3.23) and −153 ms (peak Z = 3.160, Z_crit = 3.32). The forward correlation analysis (in which activity in the seed voxel predicted activity in other voxels) showed a significant correlation between activity in the seed voxel (left STS) and subsequent activity in the left DLPFC at a lag of 358 ms (peak Z = 3.528, Z_crit = 3.29) and the right DLPFC at a lag of 179 ms (peak Z = 3.432, Z_crit = 3.39), with a marginally significant correlation in the right DLPFC at a lag of 332 ms (peak Z = 2.464, Z_crit = 2.64). Thus, activity in the left STS was both predicted by and predictive of activity in DLPFC. Additionally activity in the left STS was correlated with activity in the right STS at lag of −25 ms (peak Z = 3.846, Z_crit = 3.34), lag 0 (peak Z = 4.640, Z_crit = 3.40), and lag 25 ms f (peak Z = 3.572, Z_crit = 3.30, indicating that the time courses of the difference between the activity elicited by old and new scenes during the preview period in STS was similar in the two hemispheres.

Structural Analyses

Structural images were also acquired to assess the extent to which the ability to reactivate the related item is associated with hippocampal volume. After normalization of the hippocampal volumes for overall ICV, we found that the average hippocampal volume for participants in our study was 7949.12 mm³ (SD = 1296.88 mm³). If the hippocampus was involved in the process of relational reactivation, one would predict that those individuals with relatively intact hippocampi would be able to reactivate the related item; in contrast, those individuals whose hippocampi have atrophied to a greater extent (i.e. those with smaller hippocampal volumes) would show decreased or non-existent reactivation of the cortex. As predicted, we did find that hippocampal volume showed a positive correlation with overall reactivation in the left STS (r = .71, p = .007).

Additionally, as the behavioral task relies heavily on the ability to reactivate the related face to the presented screen to indicate whether or not the pair is new or old, we tested whether hippocampal volume was associated with overall performance on the task. Indeed, hippocampal volume was also significantly correlated with task performance (r = .72, p = .006), showing that those with smaller hippocampi did not perform as well on the task as those with larger hippocampi (see Figure 5).

Figure 5.

Bilateral hippocampal volume normalized by head size is significantly correlated with (a) ability to reactivate face representations represented by the average difference between the EROS activity elicited by old (previously-paired) and new (not-previously paired) in STS at 767 ms after the onset of the preview period, in z scores and (b) behavioral accuracy on the subsequent recognition task.

To ensure that the correlations we are reporting are hippocampal specific we also looked at the volumes of the thalamus and the caudate nucleus as control areas. We found no significant correlation between normalized thalamus volume and reactivation nor between normalized thalamus volume and subsequent behavioral task performance (both p > .14). Similarly we did not find any correlation between normalized caudate volume and reactivation nor did we find any correlation between caudate volume and subsequent behavioral performance (both p > .10).

Diffusion-Weighted Analyses

Previous studies have linked higher mean FA and lower MD in the hippocampus to better behavioral performance in memory tasks. To assess how FA and MD in the hippocampus are related to the ability to reactivate representations in the cortex, the correlations between these measures and the overall reactivation level in the left STS (measured by the amplitude of EROS difference between old and new scene at 767 ms from the onset of the preview period in each subject) were computed. The correlation between bilateral mean FA in the hippocampus and left STS re-activation did not reach significance (r = .36, p = .10). Analyses based on computing the correlations separately for mean FA values from each hemisphere showed a significant correlation for the right (r = .47, p =. 045) but not the hippocampus (r = .35, p = .11). It is important to note, however, that the correlations in the left and right hippocampi are not significantly different from each other (Z =.34, p = .367), and that the mean FA in both the left and right hippocampi are significantly correlated with subsequent behavior, r = .64, p = .009 and r = .56, p = .02, respectively (see Figure 6).

Similar analyses were conducted for MD. As can be seen in Figure 5, a robust correlation was found between bilateral hippocampal MD and the EROS reactivation measure (r = −.531, p= .031) indicating that individuals with higher MD in the hippocampus are also better able to reactivate representations in the cortex. The mean MD in the hippocampus also correlated with subsequent performance, r = −.743, p = .002.

Discussion

The goal of this experiment was to show a link between the integrity of the hippocampus and the observed phenomenon of relational reactivation. The data indicated a highly significant correlation between hippocampal volume and the extent to which a participant could reactivate an area of the cortex related to face processing (the left STS) in the absence of a face, but in the presence of a related scene, which would otherwise not reactivate this area. Furthermore, the data also showed that mean FA in the right hippocampus and MD within the hippocampus are correlated with the extent to which a participant could reactivate the left STS, such that those with higher mean FA and lower MD within the hippocampus showed greater reactivation. All three of these measures were related to overall ability to perform the subsequent relational memory task. These results are consistent with the hypothesis that relational bindings can be used to reactivate stimuli-specific processing areas in the cortex even in the absence of that type of item, and that this process is mediated by the hippocampus.

In order to establish that the hippocampus was related to relational reactivation, it was necessary first to establish that relational reactivation took place. Even before this claim could be made, it was necessary to establish that activity elicited by a previously-paired scene in a face processing region (such as the STS) could be interpreted as “relational reactivation” of the associated face representation. To this end, it needs to be shown that, normally, scenes do not elicit activity in this region. This was accomplished by showing that, when each of the two items (face and scene) are presented alone during the study phase, STS responds to faces but not to scenes. Specifically we found that part of the left STS showed activity greater for faces than scenes from around 409 ms to around 435 ms. This left STS activity is consistent with past literature demonstrating that the STS is part of the face processing network (Puce et al., 1998, Allison et al., 1999, Puce et al., 2003, Grill-Spector et al., 2004, Fairhall and Ishai, 2007, Gratton et al., 2013). In addition, the EROS results were consistent with those observed in a previous study in younger adults (Walker et al., 2014), with only a slight delay (409–435 ms vs. 383–409 ms), which could be expected on the basis of the older age of the subjects in the current study. Unlike our previous findings we did not see any activity in the right STS for processing faces. This contrasts with data from the face processing literature, as most studies report that face processing either elicited bilateral STS activation or greater activity in the right STS than the left (Kanwisher et al., 1997, Puce et al., 1998, Hoffman and Haxby, 2000, Ishai et al., 2000, Puce et al., 2003, Grill-Spector et al., 2004, Fairhall and Ishai, 2007). It is important to note, however, that the subjects on most of these studies were younger adults, and that they were asked to either passively view or answer simple questions about the faces. It is possible that the lack of significant right STS activity could be due to a difference in processing due to aging, the instructions in the present task to “try and create a story combining the faces and the scenes,” or some interaction between the two.

These findings, however, indicated that the left STS is the most logical place to study for evidence of relational reactivation in the sample and paradigm used in the current study. Examining activity in this region after the presentation of old (i.e., previously-paired with faces) and new (i.e., not previously-paired with faces) scenes during the preview period, we observed evidence of relational reactivation (i.e., a difference between the brain response to old and new scenes) at approximately 767 ms (see Figures 2 and 3), as well as other latencies. As with the activity observed during study, the area and the timing of the activation are slightly delayed with respect to what we previously found with the younger adults (716–742 ms, (Walker et al., 2014). Again, this latency difference is consistent with age-related slowing in processing. We did not, however, find an earlier activation in the older adults as we did with the younger adults. This could be due to the difference between the populations, and to the greater heterogeneity of the older sample with respect to the younger sample. The activity observed in older adults for the contrast between old scenes and new scenes was in the right direction around the same time period as that found with younger adults but was never even marginally significant. It could be that older adults either do not have or have a smaller initial reactivation of the cortex; alternatively, the current experiment may have lower power than the previous one for this type of comparison, due to greater variability across subjects linked to age-related processes, such as hippocampal atrophy.

It should not be assumed that the activity observed in STS during the scene preview constitutes the entirety of the reactivated face representation. Since there is a distributed network of processors that are active when viewing a face, it should be assumed that the reactivation must be distributed across the entire (or at least an extended portion) of this network – most of which is not accessible to EROS measurement because of its depth. The activity observed in the STS is only an indicator that relational reactivation is taking place. Importantly, however, there is evidence based on multi-voxel pattern analysis of fMRI data suggesting that the STS is one of the areas holding differential representations for faces (Gratton et al., 2013). Future experiments could look at how the entire system of processors is reactivated in concert with each other.

A potential alternative interpretation for the STS activity during scene preview is that it may reflect a response elicited by the repetition of scenes. In fact, as we contrasted scenes that were previously paired with faces with novel scenes, it is theoretically possible that any observed activity during the scene preview could be due to greater STS activity for repeated scenes as compared to novel scenes. However, the STS did not show increases in activity to the initial presentations of the scenes during either the initial study or during the presentation of novel scenes during test. It therefore appears unlikely that an area that is inactive during the initial presentation of a stimulus suddenly becomes activated upon the second presentation of that same stimulus. Whereas we cannot rule out this interpretation, it is far more likely that activity in the STS during the scene preview is due to face related activity, an interpretation corroborated by research using other methods (Kanwisher et al., 1997, Puce et al., 1998, Hoffman and Haxby, 2000, Puce et al., 2003, Grill-Spector et al., 2004, Fairhall and Ishai, 2007).

With relational reactivation established we turned on the question of whether or not hippocampal anatomical integrity is associated with the ability to reactivate the cortex. Indeed, a strong correlation was observed between bilateral hippocampal volume and STS reactivation. Additionally, hippocampal volume was also found to be correlated with subsequent behavioral performance. These results are consistent with the predictions made by the relational memory theory (Cohen and Eichenbaum, 1993, Eichenbaum, 2000, Eichenbaum and Cohen, 2001), with previous studies showing positive correlations between hippocampal volume and relational memory performance (Maguire et al., 2000, Erickson et al., 2009, Chaddock et al., 2010), and substantial work demonstrating the critical nature of the hippocampus in the creation and use of relational memories (Hannula et al., 2006, Hannula et al., 2007, Konkel et al., 2008, Hannula and Ranganath, 2009, Warren et al., 2010, Zeithamova and Preston, 2010, Zeithamova et al., 2012, Duff et al., 2013).

These correlations were present not only when volumetric measures of the hippocampus were considered, but also when other measure of structural integrity, more related to fiber tracts, were considered. Specifically, significant correlations emerged between individual variations in the reactivation process and both right hippocampal mean FA and bilateral hippocampal MD. This is consistent with previous work demonstrating that these diffusion measures, especially MD, are correlated with overall memory ability (Charlton et al., 2006, Carlesimo et al., 2010). It is not entirely clear why we only find the mean FA of the right hippocampus to be correlated with ability to reactivate in the cortex, especially since it is on the contralateral side from where reactivation is most evident. One possible explanation is that there is an association between bilateral hippocampal mean FA and reactivation, but the statistical power was insufficient to fully capture the association. This is supported by the observation that correlations between mean FA and memory performance tend to be more elusive than those obtained using MD as the predictor (Carlesimo et al., 2010). Regardless, our findings not only demonstrate the role of the hippocampus in memory, but they take the relational memory theory outside of its traditional domain within the MTL and emphasize the importance of hippocampal-cortical interactions in this phenomenon. These results support the idea that one of the hippocampus’s roles is to use relational bindings to reactivate and bring online memories stored in the cortex and that possible degradation to this system (e.g. atrophy) results in a disruption in the ability to reactivate and perform a relational memory task.

Although we are here tentatively attributing the individual differences in the reactivation process to hippocampus atrophy, another explanation is possible: Namely, some individuals may have had smaller (or less well myelinated) hippocampi to begin with. Previous longitudinal research has shown that in fact hippocampus size does reduce with age, and that white matter within the hippocampus tends to lose some of its myelin (Raz et al., 2005, Walhovd et al., 2005, Charlton et al., 2006). However, since we did not have a longitudinal component in our study (i.e., subjects were only evaluated at one point in time) we cannot distinguish between these two hypotheses. Future research could examine how the rate of change in hippocampal size (and integrity) could be associated with changes in relational memory ability and relational reactivation over time.

It is also important to note that the correlations between STS reactivation and the hippocampal measures seem to be driven primarily by four participants with higher activation in the STS. While it is concerning that the effects are driven by just a few participants, those participants constitute more than 25% of our sample. Furthermore, they are not outliers from the data set, and are 4 of the 5 subjects with behavior performance exceeding .6. It could be postulated that the hippocampus must have a sufficient amount of integrity in order to trigger reactivation in the STS, and generate a good performance level in the task. Further studies could attempt to model this pattern of activity and examine whether other cortical sensory processing regions show similar patterns of activity.

Another caveat is that we cannot speak to the specificity of the reactivation activity for each particular face-scene pair, as we examined average activity across trials rather than in individual trials. There is a possibility that the observed activity indexes a general relational reactivation of face-related regions rather than the reactivation of specific face representations. This reactivation of a face processing region is correlated with subsequent accuracy at test. As such, whether general or specific, this relational reactivation does subserve the retrieval of the specific face representation at some point. Future studies using a multi-voxel pattern analysis approach could be used to determine whether hippocampal size and integrity are related to the specificity of reactivation activity.

The data also indicated greater activity in the right DLPFC for scenes that were previously paired with a face versus those that were not. These activations were in line with previous research demonstrating DLPFC activity in this type of task (Hannula and Ranganath, 2009, Walker et al., 2014) as well as other studies investigating brain activity associated with the encoding and recall of relational items (Dobbins et al., 2002, Cabeza et al., 2003, Lepage et al., 2003, Duzel et al., 2004, Murray and Ranganath, 2007). Conversely, greater activation for novel scenes as compared to scenes previously paired with a face was observed in the left DLPFC at several time points. This result runs counter to previously published imaging data with these types of tasks (Dobbins et al., 2002, Cabeza et al., 2003, Lepage et al., 2003, Duzel et al., 2004, Murray and Ranganath, 2007, Hannula and Ranganath, 2009, Walker et al., 2014) and the current theories that would expect that older adults generally over-recruit prefrontal areas when retrieving items from memory (Cabeza et al., 2000, Rajah et al., 2010). This difference could be due to older participants paying specific attention to the patterns of novel versus old scenes. Previous research has shown that concentrating on switching between different types of pairings such as novel-old to old-old is associated with greater DLPFC activity (Dolan and Fletcher, 1997). As novel scenes only constitute 1/3 of the scene previews, there is a far greater likelihood that the novel scene previews were immediately preceded by old scene previews, constituting a switch, than novel scene previews immediately preceding old scene previews. It is possible that the older adults were paying attention to these changes, resulting in greater DLPFC activity for novel scenes than old scenes.

The functional connectivity analyses indicated that DLPFC activity is related to the observed relational reactivation in STS. The cross-correlation analysis used the voxel from the left STS showing the largest reactivation effect as seed. Reactivation activity in this seed voxel could be predicted by activity in the left DLPFC activity (as indicated by the backward cross-correlation analysis), and predicted activity in both left and right DLPFC (as indicated by the forward cross-correlation analysis). These findings are consistent with the idea that the DLPFC is important in these types of relational memory tasks for top-down control and the ability to maintain information over a delay (Fuster and Alexander, 1971, D’Esposito and Postle, 1999, Rowe et al., 2000, Rowe and Passingham, 2001).

Conclusions

The current study sought to investigate the link between the hippocampus and the ability to use relational bindings to reactivate associated items. Its main goals were to establish that relational reactivation took place and that this reactivation was associated with hippocampal integrity. In both cases, the results confirmed the predictions: reactivation of the same cortex used for initially processing faces at study was found to follow the presentation of scenes (in the absence of any face) that were paired with a face but not of scenes that were never paired with a face. This reactivation took place at around 767 ms following the presentation of the scene and was systematically preceded and followed by DLPFC activity. Critically, the data indicated that this reactivation was highly correlated with hippocampal volume such that the larger the hippocampal volume, the more reactivation was observed in the cortex. This reactivation was also correlated with mean FA within the right hippocampus and MD within the hippocampus bilaterally, such that the higher mean FA in the right hippocampus and the lower mean MD across both hippocampi, the greater the reactivation observed in the cortex. This study provides direct evidence of an association between the hippocampus and the ability to relationally reactivate items in the cortex. These findings support the idea that the hippocampus is critical for the use of relational bindings, a main tenet of the relational memory theory (Cohen and Eichenbaum, 1993, Eichenbaum, 2000, Eichenbaum and Cohen, 2001). Furthermore, these results demonstrate the interaction between the hippocampus and cortical processors to reactivate relationally-bound information stored throughout the cortex.

Highlights.

• Hippocampus plays a critical role in relational memory

• Hippocampal volume correlated with magnitude of reactivation in sensory cortex

• MR diffusion measures within hippocampus correlated with magnitude of reactivation

• Older adults show relational reactivation like that found in younger adults

• Magnitude of reactivation is related to subsequent memory