Declarative Memory

Abstract

Neuroimaging of declarative memory is not an endeavor divorced from psychology but, instead, is another path through which a more complete understanding of memory has emerged. Specifically, neuroimaging allows us to determine if differences between memory states emerge from quantitatively or qualitatively distinct underlying encoding operations. Further, it has allowed for greater specification of the putative control operations adopted when we make decisions about our memories. We describe some examples of insights provided by neuroimaging into the many and varied processes that support encoding and retrieval of declarative memory.

Colloquially, we often refer to memory as a singular object or thing that can be, for example, “bad,” “good,” or “lost.” However, psychologists have long realized that memory is a complex, temporally extended process that transforms one’s current thoughts or perceptions into a durable record that can later be recovered and used to inform decision making. Understanding memory’s dynamics and the interdependence among its stages remains a fundamental challenge. While creative experimental designs and models allow researchers to infer the nature of varied processes contributing to memory, there are limits to the success of this inference when examining only memory outcome. Partially addressing this impediment, functional brain imaging has allowed researchers to more precisely catalogue the nature and flow of information processing from the beginnings of memory formation through to the eventual successful recovery and implementation stages. As we briefly review below, the dynamic picture of memory’s unfolding suggests that multiple processes, beginning at the initial formation of a memory and ending at the act of retrieval, contribute to memory experience. The primary contribution of functional imaging to understanding memory lies in its ability to reveal multiple processes, as evidenced by differential response patterns in distinct brain regions during memory encoding and retrieval.

MULTIPLE ROUTES TO MEMORY: ENCODING

Encoding refers to cognitive and neural processes by which an event is transformed into a neural representation that can later be used to adaptively guide behavior. It is important to note that, from a cognitive perspective, encoding is not a singular process but rather a stage during which observers engage in a variety of operations such as perceiving, attending to, and working with internal and external events (Craik, Hasselmo, & Davachi, 2007). From a neural perspective, successful encoding engenders the formation of an engram or mnemonic trace that results from the brain mechanisms supporting the above-mentioned cognitive operations (Schacter, Norman, & Koutstaal, 1998). Importantly, this engram can vary in its future potential; it may be sufficient to support some types of retrieval goals but inadequate for others. For example, it may be sufficient to support remembering that one has an appointment but not the location of the appointment.

One critical debate in psychology has been whether different expressions of memory reflect qualitatively different underlying mechanisms or, instead, quantitative differences in underlying memory strength. Using functional imaging, researchers have begun to systematically observe neural activity during the initial encoding stages of memory formation and to link this activity with success or failure during different types of later memory tests. Such designs are referred to as subsequent memory designs, because the relationship between the initial encoding activity and subsequent memory outcomes is of key interest (Paller & Wagner, 2002). One fundamental question addressed by these designs is whether multiple, functionally distinct encoding processes underlie memory formation. For example, is recognizing a familiar person among a crowd of strangers supported by the same encoding process that allows one to recollect the origins of that person’s perceived familiarity? Although much elegant behavioral work has addressed this topic (see Yonelinas, 2002, for review), the debate has been difficult to resolve using only behavioral test performance.

Using the subsequent memory paradigm, one can reason that if a single encoding process, such as trace strength, is responsible for variations in later memory outcome, then brain regions should exhibit (a) little or no activation during the processing of items later forgotten, (b) more activation during the encoding of items later recognized albeit with no episodic details and, finally, (c) even more activation during event encoding leading to later item recognition accompanied by the recovery of episodic details. If, instead, distinct encoding processes support later item familiarity and episodic recollection, distinct encoding patterns should emerge in different brain regions, some correlating with later item memory and others with later contextual recollection.

In order to test this, Davachi, Mitchell, and Wagner (2003) scanned subjects while they were shown a list of study adjectives, being cued before the presentation of each study item to perform one of two encoding tasks with it (Fig. 1A). Memory for the individual events was later probed using a two-step memory test that allowed identification of trials for which subjects later either (a) forgot the study item, (b) recognized the item itself but could not remember the context (in this case, the encoding task) in which they had encountered it (an index of item recognition), or (c) recognized the item and correctly remembered the context (an index of item recognition plus episodic recollection). Importantly, the results from this study and numerous other subsequent studies (Davachi, 2006) show that activation in the hippocampus during encoding correlates with subjects’ later ability to recollect episodic details surrounding prior events—or later relational memory—but does not differentiate between individual test items later recognized and those forgotten. Critically, this same pattern is not seen in all medial temporal lobe brain regions. Specifically, activation in an adjacent region, the perirhinal cortex, during the very same events correlates with later item recognition irrespective of the recovery of episodic details (Fig. 1B).

Fig. 1.

Encoding paradigm (A) and activation in medial temporal lobe areas as related to subsequent memory (B). In the encoding trial, subjects were scanned while being shown a list of study adjectives (e.g., “happy,” “snowy”); before the presentation of each study item, they were instructed to perform one of two encoding tasks (i.e., imagine, read) with it. Encoding activation (as % signal change) in the hippocampus did not predict item recognition when probed later, but it did predict subjects’ ability to recollect the task in which they processed the items (i.e., item + source); conversely, encoding activation in the perirhinal cortex predicted later successful item recognition but not recollection of episodic details (Davachi, Mitchell, & Wagner, 2003).

These results provided compelling evidence in favor of (at least) two distinct encoding processes that contribute to recognition memory outcome. Importantly, similar distinctions between the hippocampus and the perirhinal cortex have also been seen during the act of remembering (Eichenbaum, Yonelinas, & Ranganath, 2007). Thus, there is a “mirror” pattern emerging, such that the hippocampus is important both for relational encoding and for retrieval that includes the recovery of relational information. On the other hand, the perirhinal cortex appears important for item encoding and for item recognition independent of contextual recovery.

The above example describes qualitative distinctions between different forms of recognition memory measured in response to a memory cue that is an exact copy of what was encountered during encoding. However, much of what we refer to as memory is free recall, the process of bringing to mind information without such a cue. Previous experimental and theoretical work has asked whether free recall is supported by similar operations as recognition memory or by distinct ones. Specifically, it had been proposed that free recall may benefit both from processes that enhance the distinctiveness of individual items and from those that help forge links between study items on a list. This is in contrast to item recognition that is proposed to benefit only from processes that make individual items distinctive irrespective of their links to other study items. However, previous behavioral work focused on how free recall differs from item recognition but did not address how such recall may differ from recollection, or relational recognition.

To address this question using brain data, we asked whether there are quantitative or qualitative differences between encoding processes that lead to later relational recognition as compared to item recognition and those that allow for later free recall (Staresina & Davachi, 2006). We found evidence for both quantitative and qualitative differences (Fig. 2). Specifically, a quantitative pattern emerged in the hippocampus and ventrolateral prefrontal cortex (PFC), two regions previously shown to be important for relational encoding and controlled semantic retrieval. These findings suggest that these operations contribute in a quantitative manner to both relational recognition and to free recall but that they are less critical for item recognition. A different pattern emerged in other brain regions. Activation in the dorsolateral PFC and posterior parietal cortex was selectively enhanced for trials later free recalled but not those later recognized. This qualitative effect suggests that some cognitive operation(s) distinct from semantic retrieval and relational binding contribute selectively to successful later free recall but not to different levels of recognition memory. Importantly, interpretation of the latter result was greatly informed by extant psychological theories positing that free recall benefits from higher-order inter-item organizational strategies (Tulving, 1966). While some caution is warranted in embracing the interpretation that the noted activations in the dorsolateral PFC and posterior parietal cortex are driven by inter-item processing, further work can verify its validity by directly manipulating inter-item processing and surveying activation in these brain regions.

Fig. 2.

Quantitative and qualitative subsequent memory effects. In the subsequent memory paradigm, subjects encountered a list of concrete nouns on colored backgrounds during encoding and were asked to imagine the referent of each noun in the presented color and to determine if this combination is likely found in the real world (i.e. is plausible or not; A). Subjects were later asked to free recall as many words as possible, and then their recognition memory for the words and the associated colors was assessed (B). The hippocampus and ventrolateral prefronal cortex exhibited a quantitative effect across memory conditions (item recognition, associative recognition, and free recall; C), suggesting that the process supported by those regions is important for both associative binding that supports later recognition memory as well as for later free recall. By contrast, the dorsolateral prefrontal cortex and posterior parietal cortex exhibited a qualitative difference between associative recognition and free recall (D), suggesting that the computations carried out in these brain regions are differentially important for free recall and do not contribute to the ability to later recognize associated episodic details (Staresina & Davachi, 2006).

Taken together, these two brief examples demonstrate the important interplay between psychological theory and neuroimaging data. The neuroimaging findings demonstrate that different forms of recognition-memory outcome—from basic item familiarity to detailed recollections—can be predicted based on the success of qualitatively distinct encoding processes. Likewise, psychological theory has greatly contributed to the interpretation of neuroimaging findings, as described in the second example.

MULTIPLE ROUTES TO MEMORY: RETRIEVAL

Occasionally, a memory will just pop into one’s head wholly uninvited. Although we have all had such involuntary recollections, they are arguably not the norm, and memory often reflects a goal-directed act. In these situations, remembering relies upon control processes that guide search, evaluate recovered content, and potentially reformulate search strategies (Schacter et al., 1998). Enumerating and characterizing these putative control processes based on retrieval accuracy measures alone has been challenging, because different combinations of distinct processes can lead to the same accuracy outcome. For example, one might fail to identify an item as originating from a previous event because of inefficient search or an evaluation failure (Tulving, 1983).

Functional imaging research has played a major role in our understanding of retrieval control processes in at least two ways. First, along with early neuropsychological research, functional imaging has served to highlight the prominent role of the PFC during memory retrieval and, in so doing, has firmly established memory retrieval as a form of higher-order decision making. Second, imaging has provided an important source of convergent evidence for theorizing about the numbers and types of control processes that guide retrieval.

Here we briefly focus on two interrelated control processes, termed semantic elaboration and retrieval description, that constitute part of a general capacity often referred to as cue elaboration (Norman & Bobrow, 1979; Schacter et al., 1998). Critically, these processes are thought to render memory retrieval intelligent and contextually appropriate. Semantic elaboration allows the observer to transform the initial memory probe or query into a representation that more closely highlights the characteristics of the actual event stored in memory. This modified representation then becomes the retrieval description that both guides memory search and is used to evaluate whether recovered information is sufficient for a particular retrieval situation. Importantly, these two processes may interact repeatedly throughout the course of a retrieval attempt. For example, when asked what one had for breakfast, one doesn’t consider all food items but instead actively considers typical breakfast foods. This initial semantic elaboration might then be further constrained through remembering partial information about the event. For example, if one remembers being in a hurry during the to-be-recovered event, then the list of potential food items could be greatly restricted to items that are quickly prepared (e.g., cereal). The benefit of this interactive process is that, if successful, it results in activation of many of the event features that were likely the focus of consideration during the initial experience. Such processing has been referred to as “transfer appropriate” based on the near-universal finding that successful remembering is highly dependent upon the extent to which features considered at retrieval match those that were most relevant during encoding (Morris, Bransford, & Franks, 1977).

One approach to investigating cue elaboration with functional imaging is to manipulate the nature of memory queries across probe items with a similar exposure history. The key idea is to hold constant the potential memory evidence of the stimuli and investigate how different memory goals recruit different brain regions, particularly those in the PFC. Initial work using this paradigm demonstrated enhanced recruitment of left dorsolateral and ventrolateral PFC regions when queries demanded recovery of contextual information, compared to when subjects were instructed to select items based on their relative familiarity or novelty (Dobbins, Foley, Schacter, & Wagner, 2002), and this pattern has remained highly stable across changes in materials and relative task difficulty across different investigations.

Because remembering context is often less successful than simply detecting novel or familiar items, it was tempting to ascribe this large PFC recruitment to increased mental effort or task difficulty and/or to the actual recovery of episodic content (retrieval success). However, several aspects of the designs have largely ruled out these interpretations. For example, the same pattern of left lateralized ventrolateral PFC recruitment occurs regardless of whether performance in the context-memory task is more or less successful than in the comparison item-memory task. Furthermore, activity levels during context-memory judgments are often extremely similar for both correct and incorrect judgments, again demonstrating that the activation region does not reflect the recovery of episodic information (Dobbins et al., 2002). Thus, left lateral PFC activity does not simply reflect trial difficulty or eventual outcome but instead the manner in which subjects approach memory problems that require recovery of specific contextual details.

Although ruling out constructs such as effort and task difficulty is important, work using contextual-memory tasks is beginning to more specifically implicate sub-regions of the PFC in semantic-elaboration and retrieval-description processes. For example, the left anterior ventrolateral PFC appears specifically tied to semantic elaboration of retrieval probes. As an illustration, Figure 3 shows two proximal left ventrolateral regions (indicated by arrows in the central brain image), only one of which appears to be specifically tied to semantic elaboration. Both regions were inactive when subjects had to detect novel items (green lines), suggesting no role for the left ventrolateral PFC for retrieval based upon simple feelings of item familiarity or novelty. However, the regions showed a qualitative difference across context-memory judgments depending on what the subjects were trying to remember. When trying to remember experiences tied to specific semantic, but not physical, features of the probes, the anterior ventrolateral region was active (Fig. 3, left graph). In contrast, the more posterior region was generally active whenever retrieval was contextually specific, irrespective of what contextual details needed to be recovered. This region showed elevated activity for trials in which subjects tried to remember experiences linked to either semantic or physical features of the probes (Fig. 3, right graph). Finally, a right ventrolateral region (not shown) was preferentially active for trials in which the physical features of the items were likely most critical. Such dissociations highlight the spatial selectivity of functional imaging compared to neuropsychological investigations, and they demonstrate that subjects selectively elaborate different features of the memory probes depending on their retrieval goals. Overall, then, ventrolateral PFC regions appear to play a key role in different forms of probe elaboration, and this control process is recruited whether or not episodic retrieval is ultimately successful.

Fig. 3.

Functional specialization of the left ventrolateral prefrontal cortex (PFC) during memory retrieval attempts. Subjects answered one of three different memory questions for triplets of items. Critically, two of the three items had been studied and one was novel. For the two studied items, one was previously bigger in size compared to its size in the test triplet, whereas one was previously smaller in size compared to its size in the test triplet. Additionally, during prior study, the two old items were rated using different semantic encoding tasks. Whereas one item was rated as pleasant or unpleasant, the other was rated as either living or nonliving. The previous size and type of prior rating task were fully crossed for the two studied retrieval probes of each test triplet. (The object triplets seen during each retrieval trial were different, although identical objects are shown in the figure for simplification.) During retrieval, when the question above the triplet read “Bigger Before?”subjects were to select the item that was perceptually bigger when previously studied. Because it is only the prior perceptual features of the items that are relevant, and not the prior rating task performed on the items, these trials are labeled Perceptual Context trials. In contrast, when the question read “Pleas. task before?” subjects were to select the item that was rated as pleasant or unpleasant when previously studied. Now the prior perceptual sizes of the probes were irrelevant and instead subjects were to recover prior conceptual context information (Conceptual Context trials). Finally, when the question read “New Item?” they were to select the item that had not been viewed before, and it was assumed that neither prior perceptual nor conceptual context information would be retrieved (Novelty trial). The line plots show the reconstructed blood oxygenation level dependent (BOLD) responses observed in two areas of the left ventrolateral PFC during the three different types of retrieval questions. In the anterior site (left graph), activation during Conceptual Context trials was greater than during Perceptual Context and Novelty trials. These latter two trials failed to deviate from baseline and overall this suggests that this region was uniquely involved in recovering memories related to conceptual features of the items. In contrast, the more posterior ventrolateral PFC region (right graph) showed activation for both Conceptual and Perceptual Context trials, suggesting a more general role in context-memory retrieval. Thus the pattern of activation observed in the left ventrolateral PFC critically depended upon what the subjects were attempting to recover about their prior experiences; the left anterior ventrolateral PFC appeared to be uniquely tied to elaborating the conceptual features of the probes (Dobbins & Wagner, 2005).

In order to elaborate upon the most relevant characteristics of memory probes, subjects must have at least a rough initial understanding of what it is they are attempting to recover and its behavioral significance. It is this initial template that guides elaboration and responding that is termed a retrieval description. For example, in the matched-probe paradigm we discussed, subjects must appreciate that they earlier participated in two rating tasks linked to different features of the probes and that only recollections concordant with the indicated task should govern item selection. However, as we noted, retrieval descriptions and semantic elaboration are often interactive, and this makes it difficult to isolate regions that may be unique to each. To overcome this, we built in a delay between the appearance of retrieval questions and the probes to which they were directed, in order to isolate the neural response to memory-orienting questions (e.g., “Which item did you encounter in the preceding pleasantness task?” following a task in which participants judged whether items encountered in a list were pleasant or unpleasant) from the neural response linked to the probes themselves. The prediction was that, to the extent that context-memory retrieval requires a more complex retrieval description than item-familiarity or novelty-based judgments do, the context-memory questions would evoke greater activity earlier in the trial, perhaps even before the actual memory probes were present.

Indeed, when the responses to memory questions were isolated from those to the upcoming memory probes, preparatory activity was greater for contextually specific memory questions than for item-based ones (Dobbins & Han, 2006). More specifically, regions along the left precentral gyrus (Fig. 4, area indicated by arrow in brain image) showed greater activity when subjects were preparing to assess the upcoming probes for context-memory information than they did when subjects were instead preparing to assess the probes for perceived novelty. Importantly, this activation difference was clear even before the probe items were presented. Given that this region is also implicated in verbal working memory, the findings suggest that context memory requires a more detailed or demanding description of candidate prior experiences and their response relevance and that this information is brought into working memory early in the stream of context-retrieval processing. In contrast, when the upcoming task is not specifically relevant to one’s prior experiences, there is considerably less preparatory activity during the cue or question phases of the trial. Importantly, this early working-memory demand implicates regions distinct from the left ventrolateral areas that instead appear to be involved in elaboration of probes. Further supporting this functional dissociation between retrieval description and probe elaboration, left ventrolateral regions such as those shown in Figure 3 did not show differential activity for context- and item-memory trials until the probes were actually present, demonstrating that this region contributes to processing features of the probe items.

Fig. 4.

Early recruitment of posterior prefrontal cortex (PFC) regions during contextual memory questions. During retrieval, subjects were shown object triplets containing two items that were previously studied and one novel item. Critically, for the studied items, each had been encoded using a different rating task. Whereas one of the studied items had been rated in terms of its pleasantness, the other had been rated with respect to the realism of its rendering. During retrieval, subjects were asked to select items associated with one of the prior rating tasks (e.g, “Pleasant Task?”) or to pick the novel items (“New Item?”). The key manipulation was the temporal separation of the memory question and the object triplet to which the memory question applied (5.5, 7.5, or 9.5 seconds). Because the questions preceded the triplets, activations in response to the questions could be isolated from activations that instead required the probe triplets to also be present. As shown in the line graph, the left posterior PFC demonstrated increased activity in response to context-memory questions (e.g., “Pleasant Task?”) compared to familiarity-based memory question (e.g., “New Item?”), even before memory probes were presented. The findings suggest that the posterior PFC is important for planning context retrieval and that contextually specific retrieval questions place a greater demand on these PFC regions even before subjects process the identity of the actual probe items to be evaluated (Dobbins & Han, 2006).

CONCLUSION

Ancient characterizations of the mind likened memory formation to impressions stamped in wax that could be read off at a later date when necessary. More than a generation of systematic behavioral memory investigation has yielded characterizations vastly more complex than this ancient analogy; and, even at this nascent stage, functional-imaging research has expanded the appreciated complexity even more. Synthesizing behavioral principles and functional-imaging findings has only just begun, but one significant contribution of imaging research is already clear—namely, that it serves to continually challenge the tendency to seek simple answers to complex questions.

Recommended Reading

Davachi, L. (2006). (See References). A recent review of the literature on neuroimaging during episodic encoding, focusing on the evidence in favor of item and relational distinctions in the medial temporal lobe as well as domain-specific encoding in medial temporal lobe cortex.

Eichenbaum, H., Yonelinas, A.P., & Ranganath, C. (2007). (See References). A recent cross-species review highlighting both encoding and retrieval patterns in the medial temporal lobe as they relate to item and relational distinctions, focusing specifically on the psychological constructs of recollection and familiarity.

Simons, J.S., & Spiers, H.J. (2003). Prefrontal and medial temporal lobe interactions in long-term memory. Nature Reviews Neuroscience, 4, 637–648. A nice discussion of how the prefrontal cortex and the medial temporal lobe might interact in the service of successful remembering.

Mitchell, K.J., & Johnson, M.K. (2000). Source monitoring: Attributing mental experiences. In E. Tulving & F.I.M. Craik (Eds.), The Oxford Handbook of Memory (pp. 179–195). New York: Oxford University Press. Presentation of a framework designed to account for how episodic detail or evidence is used in making source-memory decisions.

Moscovitch, M. (1992). Memory and working with memory: A component process model based on modules and central systems. Journal of Cognitive Neuroscience, 4, 257–267. Provides a framework for thinking about how prefrontally mediated decision processes are recruited during the attempted recovery of episodic memories.