Memory Awareness Disruptions in Amnestic Mild Cognitive Impairment: Comparison of Multiple Awareness Types for Verbal and Visuospatial Material

Abstract

BACKGROUND

Successful memory is normally accompanied by explicit awareness of retrieval and confidence in the accuracy of the retrieval product. Prior findings suggest that these features of metamemory can be dissociated from retrieval accuracy in Amnestic Mild Cognitive Impairment (aMCI). However, the literature on this question contains variable and conflicting results, likely because of differences in experimental conditions. We sought to systematically evaluate memory awareness disruptions in aMCI using multiple measures and stimulus formats within the same individuals.

METHODS

Memory awareness was tested with global predictions and postdictions, judgments of learning, confidence level ratings, and modified feeling-of-knowing ratings in tasks of visuospatial and verbal memory. These tests were administered to 14 individuals with aMCI and 15 healthy, age-matched controls. Memory awareness accuracy was calculated as the correspondence between subjective judgments and memory performance.

RESULTS

Individuals with aMCI demonstrated impaired global and trial-level retrospective task awareness for visuospatial and verbal stimuli. Additionally, modified feeling-of-knowing awareness was impaired selectively for verbal stimuli. Statistical effect sizes for global awareness impairments were comparable to impairments in several objective neuropsychological memory assessments.

DISCUSSION

Memory awareness (metamemory) disruptions in aMCI were most evident for a subset of subjective judgment types and task input modalities. These findings advance understanding of the nature of memory impairments in aMCI and support the utility of incorporating memory awareness testing to better characterize memory integrity in older adults.

Background

There is a pressing need to improve detection of neural and cognitive changes that presage Alzheimer’s disease (Petersen, 2004). Memory impairment occurs in early stages of the disease and is known as amnestic mild cognitive impairment (aMCI) (Petersen, Smith, Waring, Ivnik, Tangalos, & Kokmen, 1999). Although aMCI is chiefly characterized by poor objective memory performance (i.e., long-term recognition or recall of previously studied material), another aspect of aMCI is impaired awareness of memory performance.

Disrupted memory awareness is characterized by a discrepancy between memory task performance and awareness of the performance level based on introspective confidence judgments. This disruption is similar to the concept of “anosognosia” (Babinksi, 1914). Although anosognosia typically involves reduced awareness of motor and perception deficits (e.g., hemiplegia, hemianopia), such awareness deficits may also occur in dementias that affect frontal, temporal, and parietal brain networks responsible for awareness, memory, and language (Orfei et al., 2007; Pannu & Kaszniak, 2005; Agnew & Morris, 1998; Perrotin et al., 2015; Kaszniak & Edmonds, 2010; Gefen et al., 2012; De Carolis et al., 2015). For instance, individuals with dementia presumed due to Alzheimer’s disease have been shown to exhibit decreased functional activity in medial prefrontal and anterior temporal regions that correlate with disrupted self-appraisal of general memory performance (Zamboni et al., 2013).

Memory awareness disruption is clinically problematic because a decoupling of memory and awareness could prevent individuals from effectively communicating concerns about memory difficulties to caregivers, thereby delaying clinical intervention and increasing caregiver burden (Spalletta, Grirardi, Caltagirone, & Orfei, 2012; Seltzer, Vasteling, Yoder, & Thompson; Cosentino et al., 2015; Rymer et al., 2002). It is possible that tests for memory awareness deficits may provide sensitive and early behavioral indicators of neurodegenerative decline (Spaletta et al., 2014; Souchay, 2007; Wilson et al, 2015; Mograbi, Brown, & Morris, 2009), thereby facilitating early detection of Alzheimer’s disease (e.g., Marri et al., 2001).

Memory awareness disruption has been identified in individuals with dementia due to Alzheimer’s disease, but findings are inconsistent across studies potentially due to disparities in assessment techniques and heterogeneity in degree of impairment (e.g., Cosentino, Metcalfe, Butterfield, & Stern, 2007; Cosentino, 2014; Shaked et al., 2014; Souchay, Isingrini, & Gil, 2002; Schmitter-Edgecombe & Seelye, 2011; Rosen et al., 2010; Banks & Weintraub, 2008; Cosentino & Stern, 2005; Piras, Piras, Orfei, Caltagirone, & Spalletta, 2016; Roberts, Clare, and Woods, 2009). According to a recent meta-analysis by Piras et al. (2016), only ten studies to date have assessed subjective awareness of general cognitive concerns specifically in MCI, and of these, only three studies have quantified awareness deficits that are specific to memory in MCI. Akhtar, Moulin, and Bowie (2006) used judgments of learning made during the encoding phase of a verbal cued recall paradigm. Although individuals with MCI provided significantly lower judgment of learning ratings, there were no differences in the coupling of these ratings to memory accuracy versus healthy controls, leading to the conclusion that memory awareness is intact in MCI. In contrast, Perrotin, Belleville, & Isingrini (2007) used feeling-of-knowing judgments made during the retrieval phase of a verbal paired associates task. The authors reported significantly disrupted verbal memory awareness (i.e., performance overconfidence) in individuals with MCI compared to healthy controls. In the third study, Anderson and Schmitter-Edgecombe (2010) had participants provide accuracy judgments for both recalled and unrecalled items during the retrieval phase of a verbal paired-associates paradigm. Although those with MCI displayed significantly worse memory awareness for unrecalled items (i.e., feeling-of-knowing judgments), they were equivalent to healthy controls on awareness for recalled items (confidence level ratings). Thus, previous studies have used widely disparate assessment methods for memory awareness disruptions, and have arrived at different conclusions.

While the possible utility of tracking awareness in MCI is important and should be explored further (e.g., Roberts, Clare & Woods, 2009), there is not yet enough evidence to evaluate generalizability or whether particular assessment methods or materials would more precisely characterize the nature of impaired memory awareness in aMCI. For instance, no previous studies have systematically and comprehensively evaluated multiple memory awareness assessment methods within-subjects in order to identify specific types of awareness disruption in aMCI. Additionally and importantly, no studies to date have tested for differences in memory awareness between visual and verbal domains in aMCI.

To address these needs, we developed a battery of memory awareness measures by drawing on an extensive literature (e.g., Nelson & Narens, 1994; 1994; Flavell, 1979; Koriat, 2007). We focused on five distinct memory awareness measures that involve task-specific subjective judgments regarding the success of memory processing during the acquisition, retention, and retrieval phases of recognition memory testing. We included global self-ratings of aggregate memory performance made prospectively before testing (global predictions) as well as retrospectively after testing completion (global postdictions). We also included three distinct types of trial-level performance self-ratings, as previous findings in healthy individuals indicate distinctions between global and item-specific memory awareness assessments (e.g., Koriat, 2007; Connor, Dunlosky, & Hertzog, 1997). For item-level assessments, we gathered prospective judgments made for each item during learning (judgments of learning) as well as judgments made in hindsight for each item during subsequent testing (confidence level ratings) We also used a novel variation of “feeling-of-knowing” procedure during which explicit feedback was given during memory testing for unretrieved items, and individuals then rated the likelihood that memory will be correct for those items in a subsequent recognition memory test (e.g., Hart, 1965; Koriat, 2000). We refer to these as “modified feeling-of-knowing judgments from here on out. For each assessment type, the concordance between estimated and actual performance scores were obtained in order to compute measures of global-level and item-level awareness (Connor, Dunlosky, & Hertzog, 1997). To do so, we used the Goodman-Kruskal gamma (γ ) correlation, a measure of rank correlation that estimates the associative strength of dyads when both variables are measured on the ordinal level (Goodman & Kruskal, 1954; Nelson, 1982). We assessed memory performance using recognition memory tests for both verbal and visuospatial stimuli, providing a total of 18 measures for each participant (nine separate test formats for each stimulus type), and we compared awareness scores for individuals with aMCI diagnoses to those of age-matched healthy controls.

Methods

Participants and Clinical Characterization

Participants were 14 individuals with aMCI diagnoses and 15 age-matched cognitively healthy controls. Demographic variables, including age and neuropsychological performance scores for each group are summarized in Table 1.

Table 1.

Demographics and neuropsychological results for individuals with aMCI and health controls

Measures	Controls N=14 M (SD)	aMCIs N=15 M (SD)	Cohen’s d
Females/Males	8/6	9/6
Age	74.70 (6.32)	77.81 (6.50)
MMSE	27.33 (0.72)	25.53 (1.85)	1.28**
Logical Memory 1	14.67 (3.60)	8.46 (3.77)	1.68**
Digit Span Forward	7.00 (1.30)	6.30 (1.31
Digit Span Backward	5.60 (1.12)	4.38 (1.19)	1.05*
Category Fluency A	20.53 (5.04)	16.07 (3.33)	1.04*
Category Fluency V	15.33 (3.71)	10.85 (3.13)	1.30**
Trails A	30.60 (13.31)	50.15 (33.34)	0.77*
Trails B	72.80 (32.23)	207.58 (254.40)	0.74†
Digit Symbol	46.07 (7.46)	37.00 (12.26)	0.89*
Logical Memory II	13.36 (4.40)	6.38 (4.33)	1.59**
Boston Naming (30-item)	28.46 (1.60)	27.15 (3.60)
Boston Naming (60-item)	56.69 (4.27)	52.67 (8.34)
RAVLT Trials 1-5	53.30 (12.07)	32.67 (10.31)	1.83**
RAVLT Int	5.61 (2.43)	3.83 (3.47)	0.59**
RAVLT Delayed Recall	11.53 (3.15)	4.17 (2.98)	2.40**
RAVLT Recognition	28.70 (1.60)	27.15 (3.60)	1.42*
CERAD Constructions	8.92 (1.80)	8.50 (1.38)

Notes: Category Fluency A= Category Fluency for Animals, Category Fluency B= Category Fluency for Vegetables, CERAD= Consortium to Establish a Registry for Alzheimer’s disease, MMSE= Mini-Mental State, RAVLT= Rey Auditory Verbal Learning Test, Trails=Trail-Making Test. Effect sizes computed as Cohen’s d (d) are listed only for significant differences between groups. Cohen’s d values: 0.01=very small, 0.2=small, 0.50=medium, 0.80=large, 1.20=very large.

^**Between-groups differences: p<0.01,

^*p<0.05,

^†p=0.05

All participants were enrolled in the Clinical Core registry of the Northwestern Alzheimer’s Disease Center. Individuals with aMCI and normal controls carefully selected and invited to participate at the time of their scheduled annual research visit. This study was approved by the Institutional Review Board of Northwestern University. Participants provided informed written consent prior to study participation and were remunerated for their participation.

Individuals with aMCI were diagnosed by consensus of a group of behavioral neurologists and neuropsychologists in a specialized dementia center, based on recent research diagnostic criteria (see Albert et al., 2011; Petersen, 2011). A diagnosis of aMCI was made if the participant or study partner reported cognitive symptoms and if one or more scores on memory tests were below 1.5 standard deviations below the mean for their age and scores on tests in other domains were relatively normal. Daily living activities were preserved. Controls performed within normal ranges on all neuropsychological measures (Table 1). All participants in the recruitment source were assessed using the Mini Mental Status Exam (MMSE) (Folstein, Folstein, & McHugh, 1975) as well as the neuropsychological test battery of the Uniform Data Set (UDS 2.0) of the Alzheimer Disease Centers program (Weintraub et al., 2009), supplemented with additional tests including the Rey Auditory Verbal Learning Test (Rey, 1941).

Verbal Stimuli

Verbal-format test stimuli were 252 high-frequency English words including one to three syllables and between four and nine letters in length (mean number of syllables =1.84, SD=0.09; mean length=5.87 letters, SD=0.15). Words were approximately matched on frequency using the English Lexicon Database (mean Log KF frequency=8.65, SD=0.24) (Balota et al., 2007). Words were also chosen to be emotionally neutral based upon the Affective Norms for English Words (ANEW) database (mean valence= 5.24, SD=1.67; mean arousal= 4.41, SD=2.08) (Bradley & Lang, 1999). Word test fragments were created for each word by preserving the first three letters and replacing subsequent letters with dashes (e.g., Cleary and Greene, 2000; Lupker, Harbluk, & Patrick, 1991) Verbal stimuli were printed in 32 point black font and centered on 21.60 × 27.9 cm white pages. Six verbal study-test blocks (for judgments of learning and confidence level ratings) each used a set of 24 words, and three verbal study-test blocks (for modified feeling-of-knowing judgments) each used a set of 36 words. All blocks were matched on word frequency, length, and number of syllables.

Visuospatial Stimuli

Visuospatial-format test stimuli were 252 geometric color kaleidoscope images (Voss & Paller, 2009). Images were 13 × 13 cm and were centered on 21.60 × 27.9 cm white pages. Six visuospatial study-test blocks (judgments of learning and confidence level ratings) each used a set of 24 images, and three study-test blocks (modified feeling-of-knowing judgments) each used a set of 36 images.

Test Design

Examples of test stimuli and an overview of the general study design are displayed in Figure 1A. The experiment was divided into three test types differing in combination of study-phase and test-phase memory awareness assessments (judgments of learning, confidence level ratings, modified feeling-of-knowing judgments). For each stimulus type (verbal and visuospatial), there were three judgment of learning study-test blocks, three confidence level rating study-test blocks, and three modified feeling-of-knowing study-test blocks, each with unique stimuli. Our design used multiple discrete study-test blocks in order to increase the stability of memory awareness deficit scores for each test format. Scores were averaged across the three distinct study-test blocks separately for each test format. All tests were on paper, and experimenters recorded participant ratings. After providing written consent, participants were comfortably seated opposite the experimenter at a distance of approximately 60 cm. Participants were read instructions detailing the procedure, after which they viewed an example stimulus. The experimenter then confirmed that they understood the instructions prior to proceeding. Stimulus order for each block was assigned using a random number generator. All study-test blocks included global predictions and global postdictions. Study-test blocks of all formats were intermixed and administered in a partially counterbalanced order across participants using a Latin square. Participants took breaks between study-test blocks as needed. All procedures were completed during an approximately 4-hour experimental session.

Figure 1.

(A) Example stimuli and the basic framework for visuospatial and verbal memory awareness tests. (B) An illustration of the different memory awareness test types. Each test type incorporated a different item-level judgment, but all tests included a global prediction and postdiction of memory performance.

Judgments of Learning

For judgment-of-learning study-test blocks (Fig 1B), participants studied 12 items presented individually for three seconds each. For each study item, they were asked to provide a self-paced prospective 1-10 rating based on how likely they believed they were to remember that item on a later memory test immediately after the three seconds of viewing. Participants were told to provide a “10” if they were extremely confident they would remember the item later, “5” if they were somewhat confident, and a rating of “1” if they were extremely unconfident. Importantly, individuals were verbally reminded of the 1-10 rating scale for each of the 12 study trials, and they were encouraged to use the entire scale of ratings. After a one-minute break at the conclusion of the study list, participants then viewed 24 items during the test phase, presented individually. 12 of the test stimuli had been seen previously during the study phase and 12 of them were new. For verbal tests, individuals viewed a word fragment and were asked to indicate “yes” or “no” based on whether they could complete the fragment with a word that they had previously studied. For visuospatial tests, individuals viewed an image and were asked to indicate “yes” or “no” if they had studied that exact image earlier. Three aMCIs and three controls were excluded for Verbal judgment of learning analyses due to invariability in ratings (e.g., providing a rating of 9 on each trial despite instructions to use the entire scale). One control and one aMCI were excluded from each group for visuospatial judgment analyses for the same reason.

Confidence Level Ratings

For confidence level rating study-test blocks (Fig 1B), participants studied 12 items for three seconds each, separated by a one-minute break after which they viewed 24 test items, presented individually. 12 of the test stimuli had been seen previously during the study phase and 12 of them were new. For each test item, participants were asked to provide a self-paced retrospective 1-10 rating based on how confident they were that they had just provided a correct answer. As with judgments of learning, individuals were encouraged to use the entire scale of ratings on each test. One control participant was excluded from verbal confidence analyses due to invariability in ratings.

Modified Feelings-of-Knowing

For modified feeling-of-knowing study-test blocks (Fig 1B) the study phase was identical to that for confidence level ratings. During the test phase, participants were informed when they incorrectly answered “no” to a studied word fragment or image and were then allowed to view the correct answer. After feedback on each incorrect “no” response, individuals were asked to provide a self-paced 1-10 rating (using the whole scale), based on how likely they felt they were to remember the item on a later memory test. Following the retrieval phase, participants then completed an additional yes/no recognition test comprising 12 studied items interleaved with 12 unstudied items. Unlike judgments of learning and confidence level ratings that occurred for each test trial, the occurrence of modified feeling-of-knowing trials varied widely across individuals (range: 1-27 trials). We therefore excluded any participant with fewer than four total instances of modified feeling-of-knowing judgments across study-test blocks. Analyses were thus excluded for 3 aMCIs and 4 controls for both verbal and visuospatial tests.

Global Task Judgments (predictions and postdictions)

For each study-test block participants provided a global prospective prediction after the study phase and prior to beginning the test phase (Fig 1B). Participants were told that there were 24 possible correct answers on the test to follow, after which they rated how many answers they believed that they would correctly identify (out of 24). Similarly, after completing the test phase, participants rated how many correct answers (out of 24) they believed that they had just provided in retrospect. It should be noted that for global retrospective judgments, participants were guided to reflect on the test answers themselves rather than anchoring their postdictions solely on their predictions. Global γ correlations were computed for all aMCIs and all 15 controls for both verbal and visuospatial tests.

Data Analysis

Recognition memory performance was measured using d′, which is the standardized difference between the means of hit and false-alarm distributions (Macmillan & Creelman, 2005). Higher d′ values indicate better discrimination of old from new stimuli and zero indicates no ability to discriminate. Scores were collapsed across the type of awareness assessment (judgments of learning, confidence level ratings, modified feeling-of-knowing judgments) separately for each stimulus modality.

The relationship between performance self-ratings and objective memory accuracy was computed using nonparametric Goodman-Kruskall gamma (γ) correlations. To assess the correspondence between trial-level ratings and memory accuracy (i.e., resolution), the γ correlation was computed for each study-test block by correlating self-ratings across trials (judgments of learning, confidence level ratings, modified feeling-of-knowing judgments ) made at either study or test with the corresponding response accuracy across trials (at test for judgments of learning and confidence level ratings, and at the final post-test for modified feeling-of-knowing judgments). The γ correlation values were then collapsed for each awareness assessment type for each subject and then compared between groups for each stimulus type.

For global ratings we computed γ for each participant by correlating actual global memory performance with estimated global performance collapsed across the three awareness assessments (judgments of learning, confidence level ratings, modified feeling-of-knowing judgments). This was done separately for predictions and postdictions and for both visuospatial and verbal tests. We then compared mean γ estimates between groups for each global awareness judgment type and stimulus type.

Finally, we analyzed the degree to which global calibration may have changed over the course of the study. We first computed global bias scores as the signed discrepancy between raw memory performance and raw predictive and postdictive global estimates for each of the nine study-test blocks for each person. We then sorted each of the study-test blocks based on the order in which they were administered during the experiment. Finally, we correlated bias scores for individuals with the order of the nine study-test block presentations, thus allowing us to observe the relationship between global awareness calibration and time (i.e., to assess for any change in bias over time) for each group and stimulus modality.

Results

Memory performance

Overall memory performance was calculated as the number of answers correct out of 24 possible. Mean memory performance and standard deviations are reported in Table 2. As expected, recognition performance (d’) for participants with aMCI was significantly worse than for control subjects, as indicated by a main effect of group in a mixed repeated-measures ANOVA including group (aMCI; control) and stimulus type (verbal; visuospatial) as factors [Figure 2A; F(1,26)=23.41, p< 0.001, η_p² =0.47]. There was also a main effect of stimulus type [F(1,26)=11.53, p= 0.002, η_p² = 0.31], indicating that both groups performed worse for visuospatial than verbal stimuli. The interaction of group with stimulus type was not significant [F(1,26)=0.77, p=0.50] (Table 2, Figure 2A).

Table 2.

Mean memory performance, mean recognition performance (d′) and mean gamma correlations for item-level and global memory awareness measures, for individuals with aMCI and controls for verbal and visuospatial tests. Significant between-groups differences and corresponding standardized effect sizes (Cohen’s d) are listed in the right column.

Verbal Tests	Controls	aMCIs	Cohen’s (d)
Memory Performance	19.44 (2.18)	15.30 (2.51)	1.76 **
Recognition Performance (d′)	2.41 (1.16)	1.15 (0.88)	1.23 **
Judgments of Learning (γ)	0.18 (0.33)	0.09 (0.29)	0.28
Confidence Level Ratings (γ)	0.47 (0.25)	0.31 (0.26)	0.61 †
Feeling-of-Knowing Judgments (γ)	0.65 (0.26)	0.23 (0.49)	1.07 *
Global Predictions (γ)	0.18 (0.45)	−0.14 (0.86)	0.46
Global Postdictions (γ)	0.35 (0.32)	−0.05 (0.42)	1.10 **
Visuospatial Tests
Memory Performance	17.63 (1.99)	14.75 (1.99)	1.44 **
Recognition Performance (d′)	1.61 (0.75)	0.73 (0.50)	1.40 **
Judgments of Learning (γ)	0.03 (0.33)	0.06 (0.23)	0.10
Confidence Level Ratings (γ)	0.36 (0.22)	0.18 (0.27)	0.73 *
Feeling-of-Knowing Judgments (γ)	0.12 (0.44)	0.15 (0.51)	0.06
Global Predictions (γ)	0.16 (0.29)	0.15 (0.42)	0.16
Global Postdictions (γ)	0.30 (0.19)	−0.06 (0.33)	1.38 **

Notes: Effect sizes computed as Cohen’s d (d) are listed only for significant differences between groups. Cohen’s d values: 0.01=very small, 0.2=small, 0.50=medium, 0.80=large, 1.20=very large.

^**Between-groups differences p<0.005,

^*p<0.05,

^†p=0.06.

Standard deviations are listed in parentheses.

Figure 2.

(A) Overall recognition memory performance for individuals with aMCI and controls. (B) Item-level judgments of learning comparing groups and modalities. (C) Item-level confidence ratings comparing groups and modalities. (D) Item-level modified feeling-of-knowing judgments comparing groups and modalities. (E) Global memory awareness predictions comparing groups and modalities. (F) Global memory awareness postdictions comparing groups and modalities. Between-groups memory awareness differences ** p<0.005, * p<0.05, † p=0.06.

Item-by-item Confidence ratings

Mean item-by-item confidence ratings are provided in Table 3. We observed a general yet non-significant pattern of lower mean ratings for participants with aMCI (all ps > 0.07).

Table 3.

Mean confidence ratings for each of the three item-level memory awareness tests and mean global awareness estimates collapsed across all three tests types for individuals with aMCI and healthy controls

	Controls	aMCIs
Verbal Tests
Judgments of Learning	7.46 (1.58)	6.13 (2.01)
Confidence Level Ratings	7.11 (1.39)	5.88 (1.96)
Feeling-of-Knowing Judgments	6.06 (2.26)	5.40 (1.46)
Global Predictions	16.08 (4.47)	11.22 (4.01)**
Global Postdictions	15.00 (4.49)	11.19 (4.34)*
Visuospatial tests
Judgments of Learning	6.78 (1.68)	5.79 (2.07)
Confidence Level Ratings	6.79 (1.53)	5.58 (2.01)
Feeling-of-Knowing Judgments	5.97 (2.11)	5.41 (1.47)
Global Predictions	13.78 (3.95)	10.19 (4.08)*
Global Postdictions	13.85 (4.11)	10.39 (4.53)*

Note: Item-level ratings were made using a 1-10 scale. Global ratings were made using a 0-24 scale. We observed no significant group differences in mean item-level ratings (all ps <0.07).

^**Between-groups alpha p=0.005,

^*p<0.05,

Standard deviations are listed in parentheses.

Prospective memory awareness at encoding: Judgments of Learning

Mean verbal judgment of learning γ correlations did not differ from zero for those with aMCI [t(10)=1.07, p=0.31], they were only marginally different from zero for controls [t(11)=1.94, p=0.08], and they did not differ between groups [t(21)= 0.70, p=0.49, d=0.28]. Mean visuospatial judgment of learning γ correlations also did not differ from zero for those with aMCI [t(13)=1.01, p=0.33], or controls [t(12)=0.33, p=0.74], and they did not differ between groups [t(25)=0.29, p=0.77, d=0.10] (Figure 2B). These results indicate that participants with aMCI and controls were both relatively poor at making this type of judgment for both verbal and visuospatial modalities, with no evidence of impairment in aMCI (Table 2, Figure 2B).

Retrospective memory awareness at test: Confidence level ratings

Mean verbal confidence level rating γ correlations were significantly greater than zero for individuals with aMCI [t(13)=3.82, p=0.002], and controls [t(13)=6.81, p <0.001]. Values were marginally lower for those with aMCI compared to controls for verbal confidence level ratings [t(26)=2.01, p=0.06, d=0.61]. Mean visuospatial confidence level rating γ correlations were significantly greater than zero for those with aMCI [t(13)=2.94, p=0.01], and controls [t(14)=6.44, p <0.001], and they were significantly lower for those with aMCI compared to controls [t(27)=2.07, p=0.04, d=0.73] (Table 2, Figure 2C). The aMCI group thus exhibited similar deficits for this type of judgment across modalities.

Memory awareness after retrieval failure: Modified feeling-of-knowing judgments

Whereas mean verbal modified feeling-of-knowing γ correlations did not differ from zero for individuals with aMCI [t(10)=1.56, p=0.15], they were robustly greater than zero for controls [t(13)=8.18, p <0.001], and they were significantly lower for those with aMCI compared to controls [t(20)=2.46, p=0.02, d=1.07]. Conversely, mean visuospatial γ correlations were no different from zero for either those with aMCI [t(10)=0.96, p =0.36], or controls [t(10)=0.86, p =0.40], and they did not differ between groups [t(20)=0.16, p=0.88, d=0.06] (Table 2, Figure 2D). In contrast to the verbal and visuospatial deficits found for confidence level ratings, individuals with aMCI exhibited modified feeling-of-knowing deficits that were specific to verbal stimuli.

Global Confidence Ratings

Mean global confidence ratings collapsed across all three item-level test types are provided in Table 3. Mean verbal global prediction ratings were significantly higher for controls than for aMCIs [t(27)=3.08, p=0.005, d=1.14]. Mean verbal global postdiction ratings were also significantly higher for controls than for those with aMCI [t(27)=2.32, p=0.28, d=0.86]. Similarly, Mean visuospatial global predictions were significantly higher for controls than for those with aMCI [t(27)=2.40, p=0.023, d=0.89] as were mean visuospatial global postdictions [t(27)=2.16, p=0.04, d=0.79].

Global Memory awareness

For verbal global predictions, mean γ correlations were no different than zero for individuals with aMCI for predictions [t(13)=0.63, p=0.54] or postdictions [t(13)=0.44, p =0.66]. Mean γ estimates were also no different than zero for controls on predictions [t(14)=1.54, p=0.15], however they were robustly greater than zero for postdictions [t(14)=4.24, p =0.0008]. Group comparisons indicated that mean verbal global γ correlations were significantly lower for those with aMCI compared to controls for postdictions [t(27)= 2.91, p=0.007, d=1.10], but not predictions t[(27)=1.22, p=0.23, d=0.46] (Table 2, Figure 2 E-F).

For visuospatial stimuli global judgments, mean γ correlations did not differ from zero for participants with aMCI for predictions [t(13)=1.33, p =0.20] or postdictions [t(13)=0.68, p=0.51]. In contrast, mean global visuospatial γ correlations for controls were marginally greater than zero for predictions [t(14)=2.07, p=0.56], and they were robustly greater than zero on postdictions [t(14)=6.11, p <0.001]. As with the verbal tests above, group comparisons indicated that global estimates were significantly poorer for those with aMCI compared to controls for postdictions [t(27)= 3.58, p=0.001, d=1.38], but they did not differ between groups for predictions [t(27)=0.07, p=0.94, d=0.16] (Table 2, Figure 2 E-F). Taken together, these results suggest that individuals with aMCI exhibited significant deficits for global awareness calibration for both verbal and visuospatial stimuli that were specific to postdictions.

Change in global memory awareness over time

Next we computed bias scores for global predictions and postdictions for each individual for both test modalities (verbal and visual). Positive bias scores indicated global overconfidence, negative bias scores indicate global underconfidence, and a score of zero indicates perfect awareness calibration relative to actual performance. We assessed changes in global memory awareness over time by correlating bias scores across individuals with the order of the nine study-test block presentations.

For verbal tests, Pearson coefficients for healthy controls suggested a numerical pattern of increasing underconfidence (i.e., worse calibration) over the course of the experiment for both predictions (r = −0.26) and postdictions (r = −0.45). Coefficients for those with aMCI showed a similar numerical pattern of increased verbal underconfidence (i.e., worse calibration) over time for predictions (r = −0.40), and an opposite pattern of decreased underconfidence (i.e., better calibration) for postdictions (r = 0.53). Importantly, none of these correlations were statistically significant (all p values > 0.15), indicating that global verbal memory awareness for both individuals with aMCI and controls remained statistically unchanged over time.

For visuospatial tests, Pearson coefficients for healthy controls indicated a significant pattern of decreasing underconfidence (i.e., better calibration) over the course of the experiment for predictions (r = 0.73, p=0.02), and a similar trend for postdictions (r = 0.58 p=0.10). Interestingly, coefficients for individuals with aMCI showed a significant pattern of decreasing underconfidence (i.e., better calibration) for both predictions (r = 0.74, p=0.02) and postdictions (r = 0.81, p=0.008). This last pattern demonstrates that, unlike for verbal tests, those with aMCI were able to reduce their global calibration biases over the course of the experiment for visuospatial tests in a manner similar to controls.

Standardized effect sizes for memory performance versus memory awareness

We computed standardized effect sizes for between-group differences using Cohen’s d for each of the memory awareness assessment methods and for neuropsychological tests, thereby allowing for determination of the assessment type that most strongly differentiated performance of those with aMCI from controls. As indicated in Table 1 for neuropsychological measures and Table 2 for memory awareness measures, Cohen’s d effect sizes ranged from “small” to “very large” across variables in standard deviation units based on Cohen’s benchmarks (Cohen, 1988, see Table 2 for effect-size interpretation). The largest effect sizes for memory awareness were observed for global postdictions for both visuospatial and verbal stimuli. The effect size estimate for visuospatial postdictions in particular was approximately equivalent to that of objective visuospatial memory performance. Although effect sizes for other awareness measures ranged from small to large (Cohen, 1988), no awareness measure yielded statistically higher effect sizes than objective memory performance for both stimulus domains.

Discussion

We describe the assessment of memory awareness disruptions in aMCI more comprehensively than in previous studies, including assessment of prospective and retrospective global and item-by-item memory awareness measures for both verbal and visuospatial stimuli. The most robust evidence for memory awareness deficits was identified as a pattern of global miscalibration specific to retrospective awareness for both types of stimuli. This is consistent with previous findings demonstrating impaired absolute calibration scores for source judgments in dementia due to Alzheimer’s disease (Dodson et al., 2011) and worse global memory awareness in dementia due to Alzheimer’s disease despite unimpaired confidence level ratings (Gallo, Cramer, Wong, & Bennett, 2012). In the present study we show that global ratings are also sensitive to memory awareness deficits in aMCI and that they provide higher sensitivity than other standard awareness assessment types (judgments of learning, confidence level ratings, modified feeling-of-knowing judgments). Furthermore, we provide evidence that global memory awareness dysfunction discriminates individuals with aMCI from controls at effect-size levels comparable to objective memory performance (including that on several standard neuropsychological tests).

Unsurprisingly, we found lower global performance estimates in participants with aMCI (Table 3), and this is consistent with previous findings linking poor subjective beliefs of memory ability to objective memory performance in aMCI compared to healthy older adults (Cook & Marsiske, 2006) and individuals with dementia due to Alzheimer’s disease (Berry, Williams, Thomas, & Blair, 2015). It is important to point out that even though overall memory performance and global performance ratings were significantly lower in aMCI, the correspondence between performance and ratings was significantly disrupted in aMCI, specifically for postdictions. This suggests that while older adults with aMCI may have some awareness that that they are performing poorly on a memory task, they are unaware of the degree to which they are performing poorly. Instead, the global deficits we provide evidence for in aMCI are expressions of a disrupted ability to link actual memory task performance with subjective memory confidence when that confidence is provided retrospectively after task completion.

Deficits in calibrating global ratings to objective performance largely persisted in individuals with aMCI despite the fact that testing over multiple study-test blocks could have provided opportunities to adjust ratings over time based on performance. However, patterns in the present results provide some optimistic evidence that individuals with aMCI may be able to adjust their global calibration (i.e., mitigate their underconfidence) over the course of the experiment for both prospective and retrospective global judgments. This suggests that, despite expressing substantially disrupted memory awareness, those with aMCI may still possess the ability to flexibly adjust their awareness-accuracy correspondence, particularly for visuospatial memory, over time when given practice (e.g., Akhtar, Moulin, & Bowie, 2006). The lack of significant adjustment for global verbal predictions in aMCIs in the present study is interesting considering a report of diminished verbal memory performance in aMCI positively related to subjective memory complaints by Gifford et al. (2015). Specifically, Gifford and colleagues found that global awareness in aMCI, as measured using one subjective memory question, was positively related to impaired free recall and delayed recall in the RAVLT task. While the findings of Gifford suggest that some general subjective awareness of verbal memory disruption may be preserved, our findings suggest that even if a signal of general memory awareness exists, individuals with aMCI may not be able to flexibly adjust (i.e., improve) task-based global awareness using this signal, and this inability may be specific to verbal memory tasks. This represents an important distinction between general questionnaire-based awareness inventories, such as those typically used to measure anosognosia, and task-based (i.e., online) global judgments of memory performance. It is important to note that Gifford et al. used free and delayed recall of whole words, whereas the present study involved cueing of whole words using fragments. Nonetheless, as Gifford et al. (2015) point out, serial list-learning techniques appear to be highly sensitive to episodic memory changes in aMCI, and our results are consistent with verbal episodic memory awareness being particularly affected. Furthermore, our results suggest an important distinction between global and trial-level awareness judgments. Indeed, Silva and colleagues (2017) previously reported relatively accurate online (i.e., trial-level) prospective and retrospective assessments of verbal memory, despite inaccurate and “dysfunctional” global performance judgments in individuals with Alzheimer’s disease. Extending these results, our findings suggest that global memory performance judgments made in hindsight (i.e., postdictions) may be highly sensitive to awareness disruption for both verbal and visuospatial memory

Another possible influence on our patterns of global calibration change over time might be the underconfidence-with-practice (UWP) effect. The UWP effect is the finding that, in healthy young adults, calibration bias tends to shift from overconfidence to underconfidence with each subsequent repetition of a testing procedure (Koriat, Sheffer, & Ma’ayan, 2002). Although limited in scope, literature suggests that older adults may not exhibit the same UWP effect typically observed with younger adults (e.g., Rast & Zimprich, 2009). Our findings for verbal testing suggest that underconfidence increases with repeated testing for healthy older adults, whereas individuals with aMCI only experience a similar change for verbal predictions (and an opposite pattern for verbal postdictions). Individuals with aMCI and Controls both exhibited a pattern of decreased underconfidence for visual predictions and postdictions, which is opposite of what the UWP would predict. It should be noted that a fundamental difference exists between our global judgments and those typically observed with the UWP effect. whereas the UWP effect is usually computed by averaging JOL judgments during a learning phase and comparing them to average performance (e.g., Finn and Metcalfe, 2015), ours are singular judgments made directly before beginning a task or directly after completion.

Although global miscalibration for postdictions was the most robust indicator of disrupted memory awareness in aMCI, our findings for other assessment types also contribute to understanding how specific forms of trial-level awareness may decouple from memory in aMCI. We found that modified feeling-of-knowing-based awareness was significantly impaired in aMCI for verbal but not visuospatial stimuli, which is consistent with previous findings in individuals with Alzheimer’s disease (Cosentino et al., 2015; 2016) and in aMCI (Perrotin, Belleville, & Isingrini, 2007). Perrotin et al. (2007) found feeling-of-knowing-based awareness deficits for verbal memory in aMCI, and our findings of selective verbal modified feeling-of-knowing deficits are consistent with this account, however it should be noted that Perrotin et al. did not directly compare verbal and visuospatial stimuli. Although some findings involving individuals with Alzheimer’s disease have indicated disrupted memory awareness for prospective judgments of learning (Cosentino, Metcalfe, Butterfield, & Stern, 2007), we did not identify this pattern of disruption in aMCI. It is important to note that our gamma correlations for judgments of learning were no better than chance for both individuals with aMCI and controls, and thus this particular testing format may have limited sensitivity to learning-based deficits of awareness. Even so, this finding is consistent with Akhtar, Moulin, & Bowie (2006), who reported no significant disruption in prospective memory monitoring in MCI using a task similar to ours. Finally, to our knowledge, we are reporting the first evidence for reduced correspondence between confidence level ratings and objective performance for both stimulus formats, indicating that this type of memory awareness disruption is also present in aMCI. This is in contrast to Anderson and Schmitter-Edgecombe (2010) who reported no such disruption. One explanation for this discrepancy is that whereas we assessed confidence level ratings and modified feeling-of-knowing judgments in discrete tests, Anderson and Schmitter-Edgecombe combined both judgments into one test format, thus the two judgment types may have influenced one another.

In accordance with Anderson and Schmitter-Edgecombe (2010), we report memory awareness disruptions for modified feelings-of-knowing in verbal memory. This specificity to verbal memory testing is notable given theorizing regarding cognitive mechanisms for the mnemonic subtype of anosognosia (Ansell & Bucks, 2006; Silva, Pinho, Macedo, Souchay, & Moulin, 2017) and tip-of-the tongue states in aging (Schwartz & Brown, 2014). In one model proposed by Ansell and Bucks (2007), mnemonic anosognosia is thought to arise from disruption of a process that normally compares newly learned information to existing information stored in long-term semantic memory. When comparison is disrupted, individuals cannot determine whether the confidence experienced during stimulus processing is due to retrieval of existing semantic information, retrieval success of the new episodic information, or a form of heuristic processing signal indicating what verbal information should be retrievable (Schwartz & Brown, 2014). Because the visuospatial stimuli used in our experiment did not have pre-existing representations on which such erroneous assignments of prospective confidence could be based, feeling-of-knowing disruption would have been expected primarily for verbal materials, which was confirmed in our participants with aMCI. Our finding is also consistent with Perrotin and colleagues (2007) who demonstrated impaired episodic feeling-of-knowing accuracy for verbal memory testing in individuals with aMCI.

It should be noted that the modified feeling-of-knowing task used in the present study is different from the typical recall-judge-recognize (RJR) format used by many researchers. Instead of having participants study paired associates and recall a word when cued with another word at test, we chose to use a recognition judgment cued by word fragments at test. The first reason for doing this was to remain consistent with the procedure in our visuospatial tests. Second, previous research has shown that even when individuals are able to identify a studied word from a fragment at test, they can still use subjective confidence to reliably discriminate between studied and nonstudied items (Cleary and Greene, 2000). An additional theoretical view of feelings-of-knowing is that judgments are made based upon fluency or familiarity rather than on direct access to a memory trace (e.g., Koriat, 2000). Another view is that judgments are based on the outcome of performance (i.e., the content of information retrieved or retrieval failure). In our modified feeling-of-knowing task, we had individuals base their initial recognition judgments on a feeling of prior occurrence, and by giving them explicit feedback upon retrieval failure, we elicited a shift toward relying on direct information access. Indeed, some researchers have proposed that performance feedback should only serve to improve the accuracy of memory awareness (e.g., Finn and Tauber, 2005). Our results suggest that individuals with aMCI may not be able to capitalize on this switch from heuristically- driven to access-driven strategies, and this may explain the significant impairment we observe for our aMCI group versus controls in verbal testing using our modified procedure.

It is important to note that for several of our tests, data were excluded from analyses for individuals who exhibited no variability in their trial-by-trial memory confidence ratings. For example, one participant provided a rating of “8” on all trials for visual judgment-of-learning testing. While it is common to exclude such cases from analyses for statistical reasons, these patterns may be theoretically meaningful for individuals with aMCI. For instance, it is possible that, rather than misunderstanding the ratings procedure, this individual had an inherent belief that their likelihood of subsequent memory retrieval was fixed across items. This pattern could actually reflect that this person anchored their judgments on task-irrelevant information or that they exhibited an inability to adjust self-awareness due to a decoupling of performance and awareness during the task. Mograbi, Brown, and Morris (2009) describe a phenomenon known as “petrification of the self”, whereby the reciprocal relationship between brain areas responsible for memory and those giving rise to “the self” is disrupted. This disruption prevents the integration of new information into self-knowledge, resulting in rigidity of a mechanism that depends critically on flexibility and continuous updating. According to this perspective, individuals with dementia due to Alzheimer’s disease experience loss of memory updating that result in self-reflections of performance being inaccurate and unmodified over time. Indeed, Agnew and Morris (1998) have proposed a mnemonic form of anosognosia, whereby impairment in memory function could harm updating of information about task performance, leading to an “auto-evaluation that does not change in time”.

Despite general acknowledgement that fronto-parietal networks are important for awareness of a variety of cognitive functions (e.g., Banks & Wientraub, 2008) little is known regarding neural mechanisms for disrupted memory awareness in aMCI. Some studies have identified abnormal fMRI activation and reduced volume of temporal lobe regions in association with poor awareness of deficits in MCI and early dementia due to Alzheimer’s disease (Zamboni et al., 2013). Some research has also correlated frontal lobe dysfunction with loss of insight and anosognosia in dementia due to Alzheimer’s disease (e.g., Mograbi, Brown, and Morris, 2009). One influential cognitive model describes the relationship between memory and the self through frontal lobe activity (i.e., self-knowledge) exerting top-down control upon posterior (i.e., temporal and parietal) cortical regions, which then reciprocally feed back to frontal cortex thereby resulting in memory monitoring and updating of self-knowledge (e.g., Fernandez-Duque, Baird, & Posner, 2000). It is likely that greater specificity in identification of functional network disruptions that produce memory awareness disruption in aMCI (e.g., Vannini et al., 2017) will require adoption into neuroimaging research of measures with greater sensitivity to memory awareness, such as those provided here.

As mentioned in the outset, the term “anosognosia” originally referred to disrupted awareness of motor or perceptual deficits (Babinski, 1914). Subsequently, “disruption of awareness” or “lack of insight” has been used to describe impoverished cognizance of various neurological and neuropsychological symptoms, decreased awareness of general mental illness in a broad psychiatric sense, and even self-protective denial from a psychodynamic perspective. It is important to note that literature on awareness in dementia is not only broad, but it is largely mixed and inconsistent in findings. It is conceptually critical to define our meaning of awareness and to compare it to other meanings in order to frame the present study accurately (e.g.,. Markova et al., 2005). We are therefore framing our results specifically within the domain of episodic memory awareness in the amnestic subtype of MCI. The primary empirical assessments we used draw from findings in experimental cognitive psychology and metacognition, and they measure visual and verbal recognition memory performance. We further distill this awareness of episodic recognition to reflect item-specific and global task-based recognition memory awareness. It is possible that deficits we observed in recognition memory awareness may translate to tasks involving recall memory, semantic memory, and perhaps even implicit memory, and this is an important subject for future testing. Empirical work is also needed to determine whether these memory-specific deficits in aMCI translate to memory for activities of daily living as well as additional domains of awareness in motor functioning, emotion, personality, and non-memory related cognition.

Limitations and future directions

One limitation of the current study is that sample sizes were rather small in both of our participant groups, although they were comparable with several previously published studies on memory awareness in aMCI (see Piras et al., 2016 Table 2). Additionally, the repeated-measures design of our study allowed for robust detection of effects on memory and awareness at levels comparable to brief test-retest procedures performed in routine clinical visits with older adults. However, this required an intensive testing protocol (~4 hours), and variance in participant attention and fatigue could have impacted either memory performance or memory awareness. We again note that stimulus order for each of the study-test block was randomized, and study-test blocks for all test formats were intermixed and balanced across participants in order to avoid these potential confounds. In contrast to the comprehensive study design we presently report (akin to an omnibus test), individual tests incorporating memory and awareness measures are quite brief to administer. Researchers should also seek to shed light on intra-individual variance in the ability to adjust memory awareness over time, as presumably some individuals demonstrated better flexibility than others despite all having a clinical diagnosis of aMCI. Relatedly, future research should aim to replicate and extend our findings with larger samples to determine if statistically non-significant correlations, such as those shown for change in global awareness bias over time related for verbal testing, are in fact meaningful. Finally, it is possible that gamma correlations are not the ideal technique for assessing correspondence between global task estimates and overall performance due to potential inflation of coefficient estimates. Future work should confirm the validity of this novel technique and compare it to alternative techniques for measuring global memory awareness.

Another excellent direction for future research is to extend understanding of the relationship between subjective cognitive decline in healthy older adults and disrupted awareness in aMCI using longitudinal measures. Previous work has suggested that subjective cognitive decline in otherwise healthy older adults, particularly for memory-related functioning, may be especially diagnostic of impending decline leading to dementia (e.g., Rabin et al., 2015). Interestingly, patients with MCI have been shown to demonstrate subjective cognitive complaints more often than healthy older adults, despite that these complaints do not always accompany observable memory impairments (Studer Donati, Popp, & Van Gunten, 2014). An important question concerns whether older individuals who self-report cognitive complaints early on would eventually exhibit lack of awareness accompanying aMCI diagnosis later. Some have proposed that subjective cognitive complaints alone may not be diagnostically helpful, as not all individuals with MCI report them (Mitchell, 2008). This is potentially due to temporal differences in disease progression. For example, If an otherwise healthy older adult starts to communicate awareness of a memory problem through subjective complaints, it could be assumed that some awareness of memory remains intact. It is also possible that individuals who transition into aMCI begin to demonstrate a gradual disconnect between memory and awareness even while memory continues to decline, consequently becoming less likely to express their concerns to caregivers. This lack of communication could ultimately prevent timely clinical intervention, thus underscoring the need to better understand the relationship between memory, memory awareness, and subjective as well as objective changes in each over time.

Conclusion

In a recent study, Rajan and colleagues (2015) concluded that cognitive impairment can manifest in excess of 10 years prior to a clinical diagnosis of Alzheimers disease. Furthermore, a recent longitudinal analysis by Wilson et al. (2015) suggests that awareness of memory performance may begin to decouple from memory itself as early as 2.6 years prior to the onset of dementia. Indeed, memory complaints have been positively linked to prodromal neuropathological brain changes including amyloid-β deposition (Snitz et al., 2015) and to increased risk for further cognitive decline (Cooke & Marsiske, 2006; Grill, Vinters, & Monsell, 2015; Ryu, Lee, Kim, & Lee, 2016). Incorporating memory awareness assessments such as those in the present study into routine clinical diagnostic tests could provide a very simple and cost-effective means of tracking memory awareness over time and thus potentially enhancing early detection of aMCI. For example, global performance predictions and postdictions took less than 30 seconds to obtain per study-test block in our study, yet they provided discrimination between performance of individuals with aMCI and controls at levels comparable to objective memory testing (including several common neuropsychological measures). While item-level awareness measures were less robust than those for global awareness, this study provides important information distinguishing the utility of confidence level ratings and modified feelings-of-knowing based on whether tested memory is verbal or visual in nature. Although researchers will need to further evaluate the diagnostic utility of specific memory awareness measures, we have provided evidence that these measures could be easily incorporated into existing objective tests administered during formal visits with older adults, therefore yielding tests with heightened sensitivity to aMCI. Our findings also provide important replication of previous effects in a burgeoning yet critical area of research, and we provide evidence for unique effects that we hope will help motivate selection of such test formats for inclusion into longitudinal studies and clinical applications.