Using pictures and words to understand recognition memory deterioration in amnestic mild cognitive impairment and Alzheimer’s disease: A review

Abstract

Difficulty recognizing previously encountered stimuli is one of the earliest signs of incipient Alzheimer’s disease (AD). Work over the last 10 years has focused on how patients with AD and those in the prodromal stage of amnestic mild cognitive impairment (aMCI) make recognition decisions for visual and verbal stimuli. Interestingly, both groups of patients demonstrate markedly better memory for pictures over words, to a degree that is significantly greater in magnitude than their healthy older counterparts. Understanding this phenomenon not only helps to conceptualize how memory breaks down in AD, but also potentially provides the basis for future interventions. The current review will critically examine recent recognition memory work using pictures and words in the context of the dual-process theory of recognition and current hypotheses of cognitive breakdown in the course of very early AD.

Introduction

At its most fundamental level, everyday memory relies on our ability to identify and recall objects, people, and locations that we have previously encountered. In a number of situations, we are asked to discriminate between these previously encountered items and similar ones. Decades of research have investigated how these discrimination judgments (i.e., recognition decisions) are made in healthy and diseased populations. Alzheimer’s disease (AD) is the most common memory disorder, currently affecting nearly 27 million individuals worldwide, with the projected number of cases expected to quadruple by 2050 [1]. Memory problems are among the most frequent reasons cited for admission to residential facilities [2] and delaying admission by only one month would result in saving the US healthcare system an estimated $4 billion annually [3]. While disease-modifying and curative therapies are being aggressively pursued, behavioral interventions to help manage or ameliorate episodic memory deficits are paramount in the interim. With this in mind, recent neuropsychology and cognitive neuroscience research has turned to identifying areas of memory and cognition that remain relatively intact in AD and its precursor, amnestic mild cognitive impairment (aMCI).

To this end, one recent area of memory investigation in patients with aMCI and AD is the picture superiority effect [4]. The picture superiority effect refers to markedly better memory for pictures than for words, and our laboratory has been specifically interested in this phenomenon in the course of early AD. In fact, one recent study showed that the magnitude of the picture superiority effect is greater in patients than in healthy older adults [5••]. Understanding this phenomenon not only helps to conceptualize how memory breaks down in AD, but also potentially provides the basis for future interventions. The current paper will review recent work using pictures and words to understand how recognition memory breaks down during the course of aMCI and AD, as well as provide some clinical and diagnostic implications from this work.

Recognition Memory in amnestic mild cognitive impairment and Alzheimer’s disease

To realize the potential of the picture superiority effect in patients, we must first review how recognition decisions are made. Though theories of a single recognition process continue to be advanced, most recognition literature suggests a two-process approach [see 6 for review]. This dual-process theory posits that recognition can occur on the basis of the independent processes of familiarity and recollection. The former is described as an acontextual, vague sense that an item, person, or location has been previously encountered, while the latter is described as the retrieval of specific, context-bound details of a previous item, person, or location. These two constructs are often commonly experienced in daily life. For example, the unexpected sight of a particular man on a crowded city street may elicit an immediate feeling of knowing him without being able to produce any specific details about who he is or how he is known. After some deliberation, details may come to mind regarding the man’s identity—the salesman at the Apple store you visited one week earlier. In this example, that vague sense of familiarity is verified by detailed recollection of the man as the salesman that sold you an iPhone for your spouse’s birthday present last week.

Over the last decade, memory researchers have focused on understanding how familiarity and recollection are involved in recognition decisions in patients with AD, with more recent focus on patients with aMCI. Alzheimer-related brain pathology typically begins in the medial temporal lobes (MTLs), even before clinical symptoms arise [7]. Braak and Braak (1991) [8] demonstrated that neurofibrillary tangle pathology typically develops first in perirhinal regions (Brodmann area 35), followed by entorhinal cortex, and then hippocampus proper. By the time a clinical diagnosis is made, there is dense neurofibrillary tangle involvement in the hippocampus and entorhinal, perirhinal, and parahippocampal cortices [9,10]. These MTL regions overlap heavily with those proposed to be involved in recollection. Although other regions, such as the frontal lobes [11] and parietal lobes [12], contribute to the process, the hippocampus has been extensively linked to recollection [13], and lesion studies have provided countless examples of patients with hippocampal lesions demonstrating impaired recollection [see 13]. Given the significant involvement of the hippocampus to recollection, it is not surprising that recollection is severely impaired in both aMCI and mild AD [14-16].

Although there is little debate as to whether recollection is impaired early in the course of AD, the consensus on familiarity is far less agreed upon. Familiarity appears to be a complex cognitive process based on the reactivation of perceptual (form-based) or conceptual (meaning-based) representations, which rely on numerous brain regions [17]. Like recollection, familiarity has been linked to MTL structures, particularly perirhinal and parahippocampal cortices [13,18,19]. However, lesion and neuroimaging studies have provided evidence that familiarity is far more diffuse, relying on other cortical and subcortical regions. For example, extended hippocampal system lesions appear to leave familiarity intact [20-22], while lesions to the lateral prefrontal cortex do not [23]. A 2007 meta-analysis showed that recollection activated MTL regions to a far greater extent than familiarity [17]. Further, fMRI work has shown that as perceived strength of familiarity increased, activity in a number of brain regions increased linearly, while hippocampal activity was not modulated by changes in familiarity strength [24].

Given the rather diffuse and complex nature of familiarity, it is not surprising that results of investigations of familiarity in patients with aMCI and mild AD have been disparate, generally with findings showing intact familiarity for pictures but impaired familiarity for words [5••,14-16,25-30). Although the question remains as to whether the process of familiarity is impaired, studies using the remember/know paradigm suggest that patients with aMCI and AD subjectively experience familiarity in a similar manner as their healthy counterparts [27,29]. However, it has been proposed that in the face of impaired recollection, patients with AD become over-dependent on familiarity [31,32] and potentially misinterpret the strength or accuracy of the familiarity signal [33]. To understand how AD patients might face recognition decisions, imagine for a second that you were unable to experience recollection. In the example provided above about seeing a familiar man on a crowded city street, ultimately the flood of recollected information resolved the subjective sense of familiarity. Recollected details allowed you to place the man in context, and subsequently verify your feeling of familiarity. In contrast, patients with aMCI and AD likely experience familiarity in a similar manner, but do not have the benefit of recollection to support or inhibit their familiarity-based recognition [32,34]. This leads to poor discrimination, and this “unmonitored” familiarity has been linked with elevated false recognition and a liberal response bias in patients with AD [35-37]. Increased false recognition has been hypothesized to have a number of clinical consequences; perhaps misidentifying the above-mentioned man as a friend, or causing even more dangerous errors such as falsely remembering to have turned off the stove [33].

After initial recognition occurs based on familiarity and/or recollection, a final executive-based cognitive process occurs in the recognition decision process. Research has highlighted the need for post-retrieval monitoring and verification of the contents of memory [38]. It is likely that this post-retrieval process is not integrated specifically into models of recognition memory (i.e., single or dual process models) because it occurs with other types of decisions, such as semantic judgments [39]. In recognition memory studies, this post-retrieval processing is associated with right dorsolateral prefrontal regions and likely pertains to the accuracy and completeness of information retrieved from memory. Studies using event-related potentials (ERPs) and functional magnetic resonance imaging (fMRI) show that activity in right prefrontal regions increases when the contents of memory are evaluated for details and features, such as contextual information [40,41] or when the retrieved information is not sufficient for the task being performed [42]. In other words, this late frontal activity is associated with the ongoing evaluation and monitoring of the product of the retrieval attempt and perhaps initiates subsequent attempts.

In addition to the absence of recollection, it has been proposed that executive-based post-retrieval monitoring and verification is impaired in patients with AD [43,44]. One hypothesis is that diminished executive abilities do not allow patients with AD to properly monitor, verify, or inhibit responding based on familiarity alone [44]. Studies in patients using fMRI have shown diminished activity in prefrontal regions during the cognitive control of memory [45,46]. Further, a behavioral study found that patients with AD demonstrated diminished retrieval monitoring compared to healthy older adults [47]. However, behavioral work has also shown that, in some situations where recognition judgments need to be attributed to previous experience [48] or post-retrieval meta-memory is required [49,50], patients with AD appear to be able to utilize some types of memorial post-retrieval processing.

The literature on post-retrieval processing in patients with aMCI has been sparse. Using standard neuropsychological tests of executive functioning, work has shown that performance on tasks of executive-based inhibition remains relatively intact in aMCI, while tasks that require planning and sequencing do not [51]. A subsequent study highlighted that the interaction between memory and executive functioning remained preserved in aMCI and suggested that memory-related executive functioning impairment was a potential marker of conversion from aMCI to clinical AD [52]. More recently, it has been reported that elevated false memory rates increased the diagnostic sensitivity and specificity for AD over aMCI and other dementias [53,54]. There have not been many studies directly investigating recognition memory post-retrieval monitoring and verification in patients with aMCI. One study from our laboratory found that the ERP late right prefrontal old/new positivity associated with post-retrieval monitoring and verification of the contents of memory was similar in patients with aMCI to healthy older adults for pictures, but not for words [15]. Interestingly, although patients with clinical AD in Gallo et al. (2007) [47] showed significantly diminished retrieval monitoring compared to their healthy peers, patients showed enhanced retrieval monitoring for pictures compared to words. Gallo et al. (2007) [47] speculated that the distinctive perceptual information provided by pictures might impel patients to engage in retrieval monitoring, which appears to remain intact in aMCI. Though the answer is far from clear, it appears as though pictures are a special class of stimuli that allow patients with aMCI, and potentially AD, to engage memory and cognitive processes that remain relatively intact.

The picture superiority effect in amnestic mild cognitive impairment and Alzheimer’s disease

Over 50 years of research has focused on the picture superiority effect. To account for better memory for pictures than words, three basic theories have been advanced. First, the dual-coding account proposes that pictures are at an advantage over words because pictures evoke both a verbal code and an image code, while words only evoke a verbal code [55]. This dual encoding of pictures might allow them to be more easily remembered, as two stored representations potentially lead to a higher probability of retrieval success. A second explanation is the distinctiveness account, which suggests that pictures provide more highly distinctive visual features at encoding than words, making them more memorable [56]. The third alternative is the semantic processing account, which proposes that the picture superiority effect is a result of pictures allowing for deeper and more elaborate conceptual processing than words [57,58]. The main difference among all three theories is the relative contribution of perceptual and conceptual information to the picture superiority effect. Though the debate still exists as to the theory behind the picture superiority effect, cognitive psychology studies agree that pictures enhance recollection compared to words in healthy young and older adults [59-62]. However, this assertion likely does not explain the robust picture superiority effect in aMCI and AD. These patients have equally severe recollection impairment for pictures and words, leading to the hypothesis that enhanced familiarity for pictures must account for the picture superiority effect. Recently, Embree, Budson, and Ally (2012) [5••] confirmed this hypothesis by using receiver operating characteristic (ROC) curves to show that in patients with aMCI, estimates of familiarity were similar to healthy older adults for pictures but not for words.

Turning back to the main theories of the picture superiority effect, the distinctiveness account suggests that pictures provide more distinctive visual-perceptual representations at encoding, making them more memorable. As in healthy memory [63], perhaps the distinctive visual information works to enhance familiarity of pictures over words in patients with aMCI and mild AD. Processing fluency, or the ease with which information is processed, is enhanced when a stimulus is re-processed in a subsequent encounter, regardless of whether the individual was aware of the original exposure. Fluency plays an essential role in familiarity-based recognition [64,65] and likely contributes to the phenomenological experience of familiarity [66-68]. Indeed, previous work in patients with aMCI and mild AD suggest that perceptual fluency remains intact and can contribute to increased recognition performance in these patients [25,69-71]. In this type of work, perceptual fluency refers to ease at which patients process only the physical characteristic of visual stimuli. Given that familiarity judgments have been strongly associated with fMRI activation of middle occipital gyri on recognition memory tasks [72], we have hypothesized that intact earlier visual processing areas within the ventral-visual-perirhinal pathway allow patients to utilize perceptual fluency or perceptually-based familiarity to enhance memory for pictures over words [5••]. This reliance on the posterior regions of the ventral-visual-perirhinal stream is likely responsible for the enhanced occipital activation seen in studies of patients with aMCI and those at genetic risk for AD compared to controls [73-76].

An alternative hypothesis is that pictures allow for deeper and more elaborate conceptual processing than words [57,63]. Although early studies of patients with AD using word generation tasks suggested that conceptual fluency was impaired [77], more recent work has shown that patients with aMCI and mild AD can successfully rely on conceptual fluency and extract conceptual meaning from pictures to enhance memory over words [28,78,79]. Further, the neural correlates thought to underlie conceptual processing of pictures remain intact in patients with aMCI [15,48]. It is likely that deterioration of the semantic network very early in the AD process [80] contributes to impaired conceptual processing, familiarity, and subsequent recognition of words. Indeed, aberrant semantic network activation has been proposed to contribute heavily to the pattern of memory loss associated with aMCI [81•,82]. Perhaps semantic network involvement signifies initial functional memory problems in the verbal domain that can be used as a marker of disease onset and progress. We propose that if a patient has forgotten conceptual information about a word [83], or degraded semantic networks prevent him or her from elaborately processing the meaning of a word [84], pictures can serve as this cue. Indeed, it has been hypothesized that pictures enhance semantic gist in patients and allow them to gain access more easily to the full meaning of words [85]. For example, if a patient is presented with the word “shoe” at study, he or she is left to generate an internal prototype of “shoe”; spread within the semantic network may result with only the concept of “foot.” In contrast, when a patient is shown a picture of a “shoe,” he or she may more easily be able to make the conceptual associations of “Converse,” “Chuck Taylors,” “hi-tops,” and “I used to wear Chuck Taylor hi-tops when I played basketball in high school.” This more elaborate conceptual processing of the picture might allow patients to utilize more effectively familiarity at test.

We have hypothesized that greater conceptual benefit from pictures, along with enhanced perceptual fluency, allows patients to better monitor their sense of familiarity [5••], which in turn leads to enhanced accuracy [4,30] and decreased false recognition [86] compared to words. In contrast, when studying words, patients are generally left to conceptual processing with very limited perceptual information to help generate and monitor familiarity at test [28], which is compounded by the fact that patients with mild AD have difficulty using mental imagery to enhance verbal encoding [84]. The difference in how patients with aMCI utilize familiarity and post-retrieval monitoring for pictures versus words is worthy of continued investigation. However, perhaps more importantly, what are the clinical considerations and implications of the picture superiority effect in patients with aMCI and mild AD?

Conclusions and clinical considerations

As outlined in the review of experimental work above, it appears as though disruption to the semantic network within the domain of memory might be the first signal of deteriorating recognition memory performance [82], which explains why verbal tests are most sensitive in picking up early deficits dissociating aMCI from healthy aging memory [87] and the robust picture superiority effect in patients with aMCI [5••]. Due to these aberrant semantic networks, patients with aMCI are unable to extract the gist from verbal information [81•], but do so without problem for pictures [78]. In addition to a standard verbal memory measure in a clinical evaluation, perhaps including a task examining one’s abilities to extract gist information from words and pictures could help to diagnose patients with aMCI at a much earlier point, and possibly dissociate patients with aMCI owing to Alzheimer’s pathology from other etiologies contributing to mild memory problems (e.g., depression, medication). Identifying these patients very early in the disease course is critical to implementing interventions and cognitive rehabilitation. In fact, work has shown that patients with aMCI can improve cognition and functional status when interventions are applied early in the disease course [88,89].

As the disease progresses, more cognitive domains are affected. Typically, patients with aMCI have some type of executive dysfunction related to cognitive flexibility and planning, but monitoring and inhibition processes tend to remain intact [15,51]. In contrast, patients with AD have impaired post-retrieval processing, likely owing to frontal lobe pathology, that is thought to get worse as the disease state becomes more severe [90]. It has been proposed that the interplay between memory and executive function may serve as a potential marker of conversion from aMCI to clinical AD [52]. Clinically, false recognition has been used as a putative marker of impaired executive post-retrieval monitoring of memory. Patients with AD demonstrate elevated false recognition to both semantically related and, more importantly, non-related test items [35, 54]. Clinicians are encouraged to examine the type of errors made on neuropsychological tests of memory, which may be indicative of disease state or functional status.

Experimentally, pictures have been shown to reduce false recognition significantly in patients with AD (Beth et al., 2009). However, to our knowledge this has never been examined as a way to reduce false memory in a real world or clinical trial setting. With respect to improving overall memory performance, there has been relatively little work published on cognitive rehabilitation in patients with clinical AD, and in general those results have been mixed [91]. Most work has been focused on learning novel face-name pairs, but recent work has examined re-learning of previously held information (e.g., object names). Using techniques such as errorless learning and cognitive stimulation therapy, a handful of studies have shown improvement in memory and cognitive functioning in patients with AD [92, and see 93 for earlier review]. In the future, intervention and cognitive rehabilitation with AD patients may benefit from the use of pictures. For example, reminder systems can use pictures rather than word lists for reminders and medication management. Additionally, recent work has shown that patients with AD demonstrate relatively intact discrimination when forced to rely on global characteristics of visual objects rather than specific perceptual details [71]. Perhaps working with patients to focus on the gist or conceptual information rather than specific details can help with new learning. Moreover, helping them to discard ineffective strategies, such as focusing on item-specific details that require recollection, can be just as effective for improving memory.

As a final note, when conceptualizing episodic memory in clinical evaluations, the influence of other cognitive domains on memory should not be neglected. This is complicated by the fact that standard neuropsychological measures tend to not be “process pure.” For example, measures of visual memory often require a planned visuomotor response (typically drawing), which relies heavily on intact visuospatial skills and executive functioning [94]. The patient needs to approach the copy of these complex figures with an organized and systematic plan to perform well on the subsequent recall and recognition portions of the test. Given that healthy older adults and patients with aMCI likely have decrements in the executive ability to plan ahead and sequence [51,95], we are potentially over-estimating their visual memory impairment. In contrast, verbal memory tests, such at the Hopkins Verbal Learning Testing or the California Verbal Learning Test, which allow for semantic clustering and categorical cues, can provide an artificial boost in memory performance for those with relatively intact executive skills. In this situation, we may be under-estimating verbal memory impairment in patients who are purely amnestic. In addition to these practical clinical considerations, future clinical test development is encouraged to incorporate novel experimental methods and analyses. Some more recent experimental tests of memory and executive functioning are thought to be highly sensitive and specific to many disorders of aging, which could potentially provide improved acumen and assessment in the clinical domain [see 96•]. These sensitive and specific tests are more likely to readily reveal subtle deficits, leading to earlier diagnosis and providing a potential window for nonpharmacological and disease modifying therapies.