A way forward for design and analysis of neuroimaging studies of memory consolidation

Abstract

Several novel ideas and suggestions were made in response to our discussion paper (Tallman et al., this issue). Careful consideration of the content and context of memory while accounting for the neuroanatomy and functional specialization of the hippocampus may reveal more consistent patterns in fMRI studies of memory consolidation. Below we address these ideas as well as issues that arise when interpreting the fMRI signal in memory consolidation studies. In addition, we describe new analyses suggested by the commentators that clarify our findings with respect to current theories.

Introduction

The functional role of the hippocampus and its interactions with the cortex in memory consolidation are highly debated. In a discussion paper (Tallman et al., this issue), we examined changes in brain activity and changes in the functional connectivity of the hippocampus and ventromedial prefrontal cortex across 1 hour to 1 month. Specifically, participants (N = 21) studied photos of indoor or outdoor scenes 1 hour, 1 day, 1 week, or 1 month before a memory retrieval test in the MRI scanner where they viewed scenes, made old/new recognition memory judgments, and gave confidence judgments on a 1 to 6 scale (1 = definitely new, 2 = probably new, 3 = maybe new, 4 = maybe old, 5 = probably old, or 6 = definitely old). We observed changes in hippocampal-cortical functional connectivity and cortical brain activity as memories aged. Additionally, we examined the extent that concomitant changes in memory strength across the time periods affected the memory consolidation effects. Overall, the findings were very similar regardless of whether the effects of memory strength were minimized.

The interpretation of studies examining memory consolidation, particularly fMRI studies, remains challenging due to stimuli with different degrees of episodic richness, different experimental designs, and different neuroimaging analysis approaches. Theories of memory consolidation, such as systems consolidation theory (SCT) or multiple trace theory (MTT), make differential predictions for whether or not the hippocampus is necessary for memory retrieval at different time points. This presents issues for interpreting changes in brain function since brain activity and functional connectivity are typically taken as a proxy for the relevance for regions in memory retrieval.

We received 10 commentaries (published in this issue) that highlight current relevant considerations for the interpretation of changes in brain activity and functional connectivity when examining memory consolidation (Berdugo-Vega & Gräff; Feld & Gerchen; Gellerson & Simmons; Gilmore, Audrain, & Martin; Gobbo & Tse; Kirwan; Manns; Runyan & Brooks; Santangelo; Yang). These commentaries provided insight about interpretation of our brain activity and functional connectivity patterns in light of additional memory consolidation theories, as well as interactions between the anatomical divisions of the hippocampus and memory content and context. Additionally, the commentaries highlighted the importance of the careful selection of statistical models and interpreting statistical significance in changes in functional brain measures, particularly null effects. In this response, we address these considerations in the following sections: theoretical frameworks of memory consolidation, meta-memory judgments and memory content and context, hippocampal subregions and the hippocampal anterior-posterior axis, interpretation of ‘active’ clusters in memory consolidation studies, and examining changes in hippocampal activity across time periods.

Theoretical frameworks of memory consolidation

In our study, we observed brain activity in the neocortex that increased or decreased as a function of the age of the memory. We did not observe any significant changes in hippocampal brain activity as a function memory age, but we did observe decreasing hippocampal-cortical functional connectivity with increasing memory age. Overall, the patterns of cortical brain activity and hippocampal-cortical functional connectivity we observed support systems consolidation theory (SCT). We discussed our hippocampal and cortical results with respect to both SCT and multiple trace theory (MTT) models, and the difficulties that arise when interpreting a null effect as support for MTT. Our study was not designed to test other memory consolidation theories that account for memory content and context (i.e., transformation theory and contextual binding theory).

Berdugo-Vega and Gräff (this issue) and Runyan and Brooks (this issue) suggested other theories that may better account for our findings. Specifically, indexing theory posits that the hippocampus indexes neocortical neurons that represent information necessary for both encoding and later retrieval (Teyler & Rudy, 2007). Indexing theory predicts that hippocampal activity may remain constant while hippocampal functional connectivity decreases with the neocortex over time. Therefore, our observed pattern of no significant changes in hippocampal activity and decreasing hippocampal-neocortical appears to support indexing theory. We are cautious about this idea because our study was only powered to detect brain activity changes with memory age. It was not powered to definitively test that brain activity remained constant across the time periods (Feld & Gerchen, this issue). Therefore, future studies are needed to adequately test for null effects of memory age. Unified theory (Runyan et al., 2019) is another theory that may align well with our results. That theory states that, unlike systems consolidation theory, that neocortical and hippocampal changes in memory traces during the cellular consolidation phase co-occur and these changes can happen rapidly. The relatively fast completion of cellular consolidation is the rate-limiting step in the relatively slow process of systems consolidation. Our observation that neocortical activity exhibited the most dramatic changes early in the consolidation process (i.e., between the 1-hour and the 1-day conditions) may reflect the rapid changes in neocortical activity described by this theory. Yet, it is unclear if the processes occurring from 1 hour to 1 day result from cellular consolidation, the beginnings of systems consolidation, or a combination of both processes. Moreover, it is unclear how the neuroimaging signal would be affected by changes associated with cellular versus systems consolidation. In order to ensure that changes in systems consolidation are evaluated, it would be prudent to exclude time periods shorter than 48 hours (Sutherland & Lehmann, 2011). Since SCT and MTT are based on patient memory abilities examined over relatively long time frames (years), studies that examine memory consolidation across years to decades may be better positioned to detect cortical changes that better match SCT or MTT predictions (e.g., Smith & Squire, 2009).

Meta-memory judgments and memory content and context

In our study, analysis of brain activity changes as a function of memory age was carried out irrespective of memory/confidence judgments (i.e., for all targets) as well for all targets when the influence of confidence ratings and response times (i.e., memory quality or memory strength) were minimized. Regardless which method was used to analyze the data, hippocampal brain activity did not significantly change as a function of memory age.

Several commentaries proposed that memory context may influence the degree to which brain activity changes as a function of memory age. For example, the hippocampus (Gellerson & Simmons, this issue) and ventromedial prefrontal cortex (Santangelo, this issue) may exhibit little to no changes in brain activity so long as contextual information is retained across the time periods examined. Yang (this issue) and Gilmore, Audrain, and Martin (this issue) further specify that the number of details (or subjective reexperiencing) may affect hippocampal activity patterns. Similarly, Kirwan (this issue) suggests that findings showing that hippocampal activity reflects memory specificity could also influence findings in memory-consolidation studies because memory specificity is lost as memories age.

Although our study did not directly measure memory details, we did examine the contribution of changes in memory strength (i.e., confidence ratings and response times) on our findings. We carried out two analyses that differed in whether the substantial changes in memory strength across the time periods affected brain activity analysis across those time periods (analysis of all targets versus amplitude-modulated analysis). Our findings were almost identical regardless whether or not the effect of memory strength changes across the time periods were minimized. To reduce the effect of memory strength on our findings even further, we have now carried out an analysis limited to the high confidence hits. This approach also minimizes the effect of memory strength on brain activity changes across the time periods; however, this new analysis did not reveal a significant change in hippocampal activity as a function of memory age. Instead, this analysis revealed clusters in prefrontal and parietal cortex that partially overlapped with findings when all trials were used. Therefore, regardless of whether changes in memory strength were allowed to affect brain activity data we did not observe significant changes in the hippocampus. However, other information, such as the number of details, the vividness of the memory, and how specific the memory is, are relevant characteristics that should be examined in future studies to clarify their role in the consolidation of different types of memory. For example, our use of a simultaneous memory/confidence judgment introduced a meta-memory judgment into the brain activity signal (Gobbo, Mitchell-Heggs, & Tse, this issue; Manns, this issue) which may differentially affect brain activity for the different time periods. Studies designed to test these hypotheses, such as measuring memory details or reexperiencing/metamemory through cued or free recall methods, will be better suited to address the different accounts of hippocampal functioning across consolidation theories.

Hippocampal subregions and the hippocampal anterior-posterior axis

It is important to consider the specialization of subregions when interpreting the functional role of the hippocampus in fMRI studies of memory consolidation (Gilmore, Audrain & Martin, this issue). In particular, the hippocampal subregions may be specialized for different memory functions (e.g., CA1: memory consolidation/late retrieval; CA3/DG: encoding, Mueller et al., 2011) and perhaps the specialization of the hippocampus along its long-axis (anterior vs. posterior; see below). Therefore, observed changes in the functional role of the hippocampus for recent and remote memory retrieval may be driven by different contributions of one or more subregions or across the long-axis. High-resolution anatomical studies of subregions can shed light on necessary subregions for recent and remote retrieval in patient populations. For fMRI, unlike high-resolution anatomical studies, analysis of hippocampal subregions requires a high-resolution acquisition, which we did not use for our study.

Transformation theory and its extension, neural-psychological representation correspondence (NPRC), suggest the posterior hippocampus represents memory details while the anterior hippocampus represents memory gist (Gilboa & Moscovitch, 2021; Sekeres et al., 2017). Because details are forgotten more quickly than gist, posterior hippocampal activity is predicted to decline with memory age while anterior hippocampal activity is predicted to remain relatively unchanged. Our study was not well-suited to address the interaction between memory detail versus gist and their neural representations. We examined hippocampal brain activity separately for the anterior or posterior hippocampus and found that regardless which subregion was examined, the results matched those of the whole hippocampus. That is, we did not observe significant changes in hippocampal activity across the time periods.

In contrast to our brain activity findings for the hippocampus, our whole hippocampus functional connectivity findings did reveal significant changes in hippocampal-parahippocampal cortex connectivity that decreased with increasing memory age. For this response paper, we conducted additional functional connectivity analyses using the anterior and posterior portions of the hippocampus as separate seed regions of interest, separated at the level of the disappearance of the uncus. Unlike our reported findings when all trials were used, no significant clusters were obtained when using either the anterior or posterior hippocampus as seed regions. When the effects of memory quality were minimized for all trials, we found that anterior hippocampal functional connectivity with the bilateral posterior cingulate and left cerebellum decreased with memory age. No brain regions were identified when posterior hippocampal functional connectivity was examined. The exploratory analysis we previously reported at a less stringent probability level revealed that the whole hippocampus was connected to a larger network of cortical and subcortical regions, all of which exhibited reductions in functional connectivity with memory age. A subset of brain regions that highly overlapped with this larger network was also obtained when anterior hippocampal functional connectivity was examined (see, Table 1 and Figure 1). Similarly, an analysis of posterior hippocampal functional connectivity identified many of the same brain regions (see, Table 2, compare with Table 2 from Tallman et al., this issue), though there was little overlap with the whole hippocampal functional connectivity network (see, Figure 1). These additional analyses revealed that our published findings appear to be more representative of anterior versus posterior hippocampal functional connectivity. In addition, regardless of which hippocampal seed regions were used and which cortical or subregions were identified, our finding that hippocampal functional connectivity decreases with memory age remained consistent.

Table 1.

Brain regions where retrieval-related anterior hippocampal functional connectivity was associated with the age of the memory.

Brain Region	Vol. (mm3)	MNI Coordinate			B.A.	M.P.
		X	Y	Z
Connectivity with the anterior hippocampus seed (p < 0.001) (no regions identified)
AM Connectivity with the anterior hippocampus seed (p < 0.001)
B. Posterior Cingulate Ctx.	297	−1	−39	17	23,31	↓
L. Cerebellum	351	−20	−41	−19	-	↓
Connectivity with the anterior hippocampus seed (p < 0.01) (no regions identified)
AM Connectivity with the anterior hippocampus seed (p < 0.01)
R. Parahippocampal Ctx./Inf. Temporal G./R. Fusiform G.	1755	42	−19	−21	20	↓
L. Insula/Caudate/Cingulate Ctx./Thalamus	1377	−21	−29	27	23,31	↓
R. Insula/Caudate/Hippocampus/Posterior Cingulate Ctx./Mid./Sup. Temporal G./L. Thalamus	6858	22	−36	10	13	↓
L. Insula/Caudate/Hippocampus/Parahippocampal Ctx./Sup. Temporal G.	2268	−39	−39	9	13	↓
L. Cerebellum	1728	−21	−39	−40	−	↓
L. Cerebellum	1296	−21	−41	−15	−	↓
B. Post. Central G./Paracentral Lob./Precuneus	2754	→2	−57	62	5,7	↓
R. Fusiform G./Mid. Occipital G./Inf. Temporal G.	1944	43	−65	−6	37	↓

Note. Functional connectivity significantly changed across the four time periods according to a power function (voxel-wise threshold of p < 0.001 or p < 0.01 [exploratory analysis], cluster-wise threshold of p < 0.05). AM (amplitude-modulated analysis) indicates that connectivity changed across time periods when the effect of concomitant changes in behavior were minimized. Functional connectivity decreased in a relatively monotonic pattern (M.P., ↓) across time periods. B.A., Brodmann area; B., Bilateral; Ctx., Cortex; Inf., Inferior; L., Left; Lob., Lobule; Mid., Middle; Post., Posterior; R., Right; Sup., Superior; Vol., Volume.

Figure 1.

Overlap of retrieval-related functional connectivity for the whole hippocampus and hippocampal subregions. Clusters indicate regions where functional connectivity changed as function of memory age when the effects of behavior were minimized for all trials. A. Extensive voxel overlap (green) of anterior hippocampus (yellow) and whole hippocampus (blue, previously reported Tallman et al., this issue). B. Same as A, except posterior hippocampus was used as the seed. There was minimal voxel overlap of posterior hippocampus and whole hippocampus connectivity. Posterior hippocampal and whole hippocampal functional connectivity networks had minimal voxel overlap, yet were generally identified the same brain regions (see Table 2 and Table 2 in Tallman et al., this issue). Voxel-wise p < 0.01, cluster-wise p < 0.05. L, Left.

Table 2.

Brain regions where retrieval-related posterior hippocampal functional connectivity was associated with the age of the memory.

Brain Region	Vol. (mm3)	MNI Coordinate			B.A.	M.P.
		X	Y	Z
Connectivity with the posterior hippocampus seed (p < 0.001) (No regions identified)
AM Connectivity with the posterior hippocampus seed (p < 0.001) (No regions identified)
Connectivity with the posterior hippocampus seed (p < 0.01) (No regions identified)
AM Connectivity with the posterior hippocampus seed (p < 0.01)
B. Posterior Cingulate Ctx./Precuneus	5373	−3	−43	36	7,23,30,31	↓
L. Sup. Temporal G.	1809	−49	−57	13	39	↓
L. Sup./Med. Frontal	1539	−17	60	0	10,37,39	↓
G./Mid. Temporal G. L. Cerebellum	1323	−21	−38	−30	-	↓

Note. Functional connectivity significantly changed across the four time periods according to a power function (voxel-wise threshold of p < 0.001 or p < 0.01 [exploratory analysis], cluster-wise threshold of p < 0.05). AM (amplitude-modulated analysis) indicates that connectivity changed across time periods when the effect of concomitant changes in behavior were minimized. Functional connectivity decreased in a relatively monotonic pattern (M.P., ↓) across time periods. B.A., Brodmann area; B., Bilateral; Ctx., Cortex; L., Left; Med., Medial; Mid., Middle; Sup., Superior; Vol., Volume.

Theories of memory consolidation are based on the premise of changing connections in the memory trace and these changes can be examined using functional connectivity. Memory details may also be important to consider when examining hippocampal-neocortical functional connectivity. For instance, it is posited that changes in hippocampal-medial prefrontal cortex interactions are important for detailed memories, but for memories that are part of an existing schema (Gilboa & Moscovitch, 2021). Due to the orthogonal nature of task activation and functional connectivity analyses, careful examination of the relationship between task activation and task functional connectivity within the same studies may inform current theories of memory consolidation.

Interpretation of ‘active’ clusters in memory consolidation studies

The literature examining brain activity in memory consolidation studies is mixed. Most of the interest has centered around the hippocampus given the differential theoretical predictions for its role as a function of the age of the memory. Some studies have found evidence that hippocampal activity decreases with memory age, increases with memory age, or does not significantly change as a function of memory age (for review, see Tallman et al., this issue). Part of the difficulty with interpretation of the role of the hippocampus in memory retrieval as a function of memory age pertains to how data are analyzed and interpreted. For example, one approach seeks to determine whether hippocampal clusters can be detected at each time period examined. A cluster detected in the remote but not the recent condition is taken as support for MTT and the opposite pattern is taken as support for SCT. Detection of clusters at both time periods is also taken as support for MTT, and this issue is addressed below. This ‘cluster detection’ approach seems reasonable given that MTT and SCT theories were based on patient lesion data and predictions pertain to whether the hippocampus is necessary for both recent and remote memory retrieval (MTT) or whether it is necessary only for recent memory retrieval (SCT).

The difficulty with this approach is that the detection of a hippocampal cluster does not indicate whether the hippocampus is necessary for memory retrieval. Beyond the well-understood idea that fMRI data are correlational, not causational, there is another problem with this approach. Specifically, as is the case with most fMRI studies of memory, detection of a hippocampal cluster depends on the level of hippocampal activity associated with the memory retrieval conditions, the level of hippocampal activity associated with the baseline condition, and the error estimates for these conditions. Because there is no true baseline signal in fMRI, isolating changes that reflect a true ‘memory retrieval signal’ is challenging. Even for a typical ‘rest’ condition or passive baseline task, hippocampal activity has been shown to be higher than task-related activity (Buckner et al., 2008; Stark & Squire, 2001). Baseline-related hippocampal activity can reduce, eliminate, or even reverse the direction of memory retrieval-related activity. Depending on these values, one could observe a cluster in the hippocampus reflecting activity that is significantly higher or lower than baseline (see, Figure 2., gray and blue examples, respectively), or one could detect no hippocampal clusters (see, Figure 2., orange example). Importantly, these findings about hippocampal clusters could all be true even when hippocampal activity significantly changes across conditions (see slopes of gray, blue, and orange lines in Figure 2.). Moreover, because this method depends on detecting a significant difference between the baseline and memory conditions, large error estimates for only one of the time periods condition could influence the results. This is likely to occur for when analysis is limited to correct trials or high-confidence trials, as forgetting reduces the number of trials in the remote versus the recent conditions. For example, this larger error estimate for the remote condition can lead to a failure to detect a significant cluster for that time period (see, Figure 2., green circle for remote condition). Thus, detection of a hippocampal cluster for a recent or a remote condition does not reveal much about the role of this region in memory consolidation.

Figure 2.

Hypothetical examples of brain activity in a region of interest for recent and remote conditions relative to baseline (dashed line). Circles with error bars that do not cross baseline result in significant clusters, but circles with error bars that do cross baseline do not result in significant clusters. Grey and blue examples result in significant clusters (relative to baseline) for the recent and remote conditions and exhibit significant reductions in brain activity. The orange example does not result in significant clusters for the recent or remote conditions, but it does exhibit a significant reduction in brain activity. The green example results in a significant cluster for the recent, but not the remote condition, but brain activity does not change across these conditions.

More modern approaches compare brain activity across recent and remote conditions. This method directly tests whether activity changes with memory age (see gray, blue, and orange lines in Figure 2.). One example of this lesson comes from a study of changes in brain activity in an anatomical hippocampal region of interest for memories that were 3-days old or 3-months old (Harand et al., 2012). They observed a significant decrease in hippocampal activity from 3-days to 3-months, a result that supports SCT. Yet, they interpreted their findings in support of MTT because hippocampal clusters were detected for both conditions in the whole-brain analysis (consistent with the blue result in Figure 2.). Thus, even within the same study, the cluster detection approach and the activity comparison approach are at odds. Given the difficulties with interpretation of the cluster detection approach, we believe a better way forward for the field is to use the activity comparison approach. In addition, examination of more than two time periods will also improve interpretation of findings because brain activity that follows a consistent pattern (either increasing or decreasing across time periods) is less likely to be affected by idiosyncratic differences across the time periods.

Examining changes in hippocampal activity across time periods

In line with the idea of comparing activity across more than 2 conditions (described above), we used a power function to model brain activity across four conditions. While a power function is the best fitting curve for forgetting rates (Ebbinghaus, 1885; Murre et al., 2015; Wixted & Ebbesen, 1997) and it has been used previously in other memory consolidation studies (Takashima et al., 2006), Feld and Gerchen (this issue) suggest that different non-linear functions may be more appropriate. Specifically, memories tested after a night of sleep experience less forgetting than memories tested before sleep. This memory ‘boost’ for a 1-day condition causes a deviation from a true power function. Although it is unclear how much ‘boost’ to add to the 1-day condition, it is still better to examine more than one reasonable function when analyzing changes in brain activity across time periods (e.g., Smith & Squire, 2009).

It is worth noting that we also analyzed our data using repeated measures ANOVA across the four time periods in an effort to capture brain activity patterns that changed with memory age at different rates. Although this analysis did not reveal a hippocampal cluster, it did reveal cortical clusters where activity appeared to change much like a power function (there was a 78% overlap of voxels between the ANOVA and power function analyses). Because our time periods were an interval scale and were not equally spaced, linear models appear to be more appropriate. Nevertheless, formal model comparisons will be beneficial to include in future studies.

When testing for differences in brain activity across conditions, it is also important to consider how to interpret null findings. For hippocampal activity in our study, we chose to examine activity in several different ways to specify how activity was or was not changing across conditions. Regardless how it was analyzed, we could not find evidence that brain activity significantly changed across time periods. This ‘null’ effect is sometimes taken as support for MTT, because MTT posits that the hippocampus is necessary for episodic memory retrieval regardless of the time period. Similarly, detection of hippocampal clusters at recent and remote time periods is taken as support for MTT. Feld and Gerchen (this issue) make important points about how to directly test for null effects if this is a predicted pattern of results. If these types of methods are not followed, it can lead to incorrect interpretation of data in underpowered studies. We agree with these ideas and believe the field would benefit from using proper methods to test for predictions of no effect of memory age on brain activity.

Finally, neurophysiologists may find it surprising that changes in brain activity can even be detected using fMRI. Manns (this issue) suggests that neurophysiological changes associated with memory consolidation in the hippocampus should be relatively difficult to detect with methods as temporally and spatially imprecise as fMRI and where the signal is only indirectly related to neural activity. The concomitant strengthening of some synapses and weakening of others that are thought to evolve during memory consolidation could be expected to result in a net zero level of brain activity. Although the blood oxygen level dependent (BOLD) signal is an indirect measure of neural activity, it is thought to reflect local field potentials (Logothetis et al., 2001). Given that approximately 65% of extant studies report significant changes in hippocampal activity across time periods, it appears that memory consolidation-related changes in brain activity are detectable with BOLD fMRI. Studies that incorporate single-cell recording, local field potential recordings, and BOLD fMRI could inform these ideas. For example, preoperative fMRI studies of epilepsy patients being considered for surgery and single-cell recordings in the same patients would provide all these data in humans.

Conclusion

Our study provided evidence for decreasing hippocampal-cortical functional connectivity over one hour to one month in support of systems consolidation theory. Although we did not observe brain activity changes in the hippocampus over this time frame, our response addresses the cautions of interpreting ‘null effects’ in fMRI signal as support for MTT. Additional study designs that measure memory for the context in which the information was learned, memory details, and methods that reduce the effect of meta-memory judgments at the time of memory retrieval may reveal more consistent fMRI patterns with memory consolidation theories. Investigating the differential contributions of the long-axis of the hippocampus or the hippocampal subregions may reveal functional specialization for specific processes in memory consolidation.