Metacognition in the behavioral variant of frontotemporal dementia and Alzheimer’s disease

Abstract

Objective

Impaired self-awareness is characteristic of nearly all dementias, including Alzheimer’s disease (AD), but the deficit is most severe in the behavioral variant of frontotemporal dementia (bvFTD). The prominence of frontal pathology in bvFTD suggests that failure of online monitoring, the process by which individuals monitor their own cognitive processing in real time, is an important contributor. Metacognitive research offers several approaches to measure self-assessment, some more and others less sensitive to online monitoring. The goal of this study was to assess metacognition in bvFTD using several approaches, and compare the results with those in AD.

Methods

We examined metacognition in 12 patients with bvFTD, 14 with AD and 35 healthy controls using Feeling of Knowing (FOK), Ease of Learning (EOL), Judgment of Learning (JOL), and Retrospective Confidence Rating (CR) tasks, as well as response to feedback about performance.

Results

BvFTD and AD were both impaired at FOK compared with controls, although AD showed some sparing. Both groups were similarly impaired at CR and neither group was impaired at JOL after accounting for memory performance. Most strikingly, bvFTD patients failed to appropriately adjust their predictions about future memory performance even after receiving explicit feedback that they had performed worse than they expected.

Conclusions

Both bvFTD and AD show deficits in online monitoring, although the deficit appears more severe in bvFTD. The insensitivity of bvFTD patients to overt feedback may point to unique mechanisms, possibly frontally mediated, that add to their severe lack of self-awareness.

Patients with neurodegenerative dementias are frequently unaware that they have a disease, or fail to appreciate the degree to which their disorder is impacting their function (Agnew & Morris, 1998; Banks & Weintraub, 2008; Mendez & Shapira, 2011; Rosen, 2011; Williamson et al., 2009). This deficit, often referred to as anosognosia (Agnew & Morris, 1998; Cosentino & Stern, 2005; Rosen, 2011), carries its own financial burden and risks for morbidity and causes significant frustration for patients and their families (Adler, 2007; Rymer et al., 2002; Seltzer, Vasterling, Yoder, & Thompson, 1997; Wild, Rodden, Grodd, & Ruch, 2003). Although anosognosia occurs in many forms of dementia (Leritz, Loftis, Crucian, Friedman, & Bowers, 2004; O'Keeffe et al., 2007; Wagner & Cushman, 1994), it is particularly prominent in the behavioral variant of frontotemporal dementia (bvFTD) (Banks & Weintraub, 2008; Williamson, et al., 2009). This association is so strong that lack of awareness of deficits has traditionally been considered mandatory for a diagnosis of bvFTD (Neary et al., 1998).

The mechanisms underlying anosognosia are poorly understood. Although memory deficits seem to contribute in some patients (Duke, Seltzer, Seltzer, & Vasterling, 2002; Migliorelli et al., 1995) this cannot fully explain the problem because anosognosia occurs even when memory deficits are not prominent, or are taken into account (Dodson et al., 2011; Mendez & Shapira, 2011; O'Keeffe, et al., 2007). Neuropsychological studies have linked anosognosia to executive dysfunction (Lopez, Becker, Somsak, Dew, & DeKosky, 1994; Mangone et al., 1991; Michon, Deweer, Pillon, Agid, & Dubois, 1994) and imaging studies have implicated the frontal lobes, particularly on the right (Reed, Jagust, & Coulter, 1993; Salmon et al., 2006; Starkstein et al., 1996). None of these findings provide direct insight into the specific psychological causes of anosognosia or the precise mechanisms underlying its severity in bvFTD. Several theories have been proposed to account for anosognosia in neurological diseases (Agnew & Morris, 1998; McGlynn & Schacter, 1989; Rosen, 2011; Stuss & Alexander, 2000). All share the assumption that specific brain systems, at least partially based in the frontal lobes, subserve online monitoring functions that allow individuals to become conscious of poor performance in real time. Failure of these online monitoring systems is one hypothesized mechanism leading to anosognosia. Based on these theories it follows that, to better understand the mechanisms leading to anosognosia and why it might be more severe in some patients than others, researchers must examine performance on tasks sensitive to online monitoring.

One area of psychological investigation that provides methodology relevant to anosognosia is the study of metacognition. This term refers to the processes by which we understand and alter our own thinking (Flavell, 1979; Nelson & Narens, 1994). A variety of techniques have been used to assess metacognition, usually for memory abilities (metamemory). Some of these tasks are thought to tap into online monitoring more than others. Paradigms often used include: 1) ease of learning tasks where participants predict their performance based on a description of the task, 2) judgment of learning tasks where participants estimate their likelihood of recalling items they have already studied, 3) feeling of knowing tasks, where participants estimate their likelihood of recognizing material that they have previously studied and tried to recall, and 4) retrospective confidence ratings, where participants estimate their confidence that their recent attempt to recognize stimuli they had studied was correct (Nelson & Narens, 1994). Furthermore, metacognitive judgments can be performed at a global level (e.g. estimating how many of 20 items you will be able to recall), or at an item-by-item level (e.g. estimating the likelihood of recalling each item in a list, regardless of overall performance (Connor, Dunlosky, & Hertzog, 1997)). It has been demonstrated that these metacognitive judgments are not all equivalent. Judgment of learning is more predictive of memory performance than ease of learning (Leonesio & Nelson, 1990). Furthermore, while judgment of learning and feeling of knowing are both correlated with performance, they are not correlated with each other, suggesting that they tap different processes (Leonesio & Nelson, 1990). Global judgments, and to some extent even item-by-item judgment of learning, are thought to reflect factors inherent to the testing situation to some degree. Such factors have been referred to as intrinsic factors in the stimuli (e.g. the memorizeability of the words), and extrinsic factors in the testing (e.g. time allotted to study (Connor, et al., 1997; Koriat, 2008; Souchay, 2007)). An example of how such factors could influence global ratings would be if a person uses the perceived difficulty of a task and their general feeling about how the average person would perform to predict their own performance, rather than basing their estimate on their own abilities. In such a case, assuming their estimate of average performance is accurate, the discrepancy between their estimated and actual performance would solely reflect the degree to which their performance deviated from average, and would not necessarily reflect online monitoring at all.

Of all the metamemory tasks, feeling of knowing, which requires subjects to evaluate the strength of a particular item in memory, has been linked most closely with online monitoring (Souchay, Isingrini, Clarys, Taconnat, & Eustache, 2004). Feeling of knowing is also closely associated with the frontal lobes, being correlated with executive function (Souchay, et al., 2004) and being impaired in patients with frontal lobe lesions (Janowsky, Shimamura, & Squire, 1989; Modirrousta & Fellows, 2008; Pannu & Kaszniak, 2005; Pannu, Kaszniak, & Rapcsak, 2005; Pinon, Allain, Kefi, Dubas, & Le Gall, 2005; Schnyer et al., 2004; Schwartz & Bacon, 2008; Vilkki, Servo, & Surma-aho, 1998). In contrast, judgment of learning is not impaired in patients with frontal lobe lesions when performance on the task is taken into account (Modirrousta & Fellows, 2008).

Based on the prominence of frontal pathology and theories of anosognosia, we would expect bvFTD to be associated with a deficit in online monitoring, which could be demonstrated with a feeling of knowing paradigm. A few studies have examined metacognition in bvFTD, but none using feeling-of-knowing. These prior studies have demonstrated impaired judgment of learning and retrospective confidence ratings (Banks & Weintraub, 2008; Souchay, Isingrini, Pillon, & Gil, 2003; Williamson, et al., 2009). One study found poor correlation between performance on standard neuropsychological tasks and estimation of performance after completing the tasks (Eslinger et al., 2005). All used global judgments. One study looking at item-by-item error tracking in bvFTD suggested a deficit in online monitoring, although this task did not have patients assess their performance (O'Keeffe, et al., 2007).

An additional issue that has not been examined in bvFTD is the response to overt feedback. While not necessarily related to online monitoring, feedback, when available, is clearly an important contributor to metacognitive monitoring and control (Hacker, Bol, & Keener, 2008). A large body of evidence implicates frontal lobe circuitry in monitoring feedback (Ridderinkhof, van den Wildenberg, Segalowitz, & Carter, 2004), and frontal injury impairs adjustment of behavior in response to feedback (Wheeler & Fellows, 2008). In bvFTD, behavioral problems and anosognosia occur (at least anecdotally) despite many attempts by family and others to alter patients’ behavior. This suggests that impaired response to feedback may contribute to their deficits.

In contrast to bvFTD, metacognition has been assessed more extensively in Alzheimer’s disease (AD), where the level of anosognosia is less severe than in bvFTD. Metacognitive assessments in AD have indicated intact sensitivity to intrinsic features of cognitive stimuli, such as memorizeability (Moulin, Perfect, & Jones, 2000), and mixed results for retrospective confidence ratings (Dodson, et al., 2011; Moulin, James, Perfect, & Jones, 2003). Global and item-by-item judgment of learning are impaired in AD (Ansell & Bucks, 2006; Cosentino, Metcalfe, Butterfield, & Stern, 2007; Duke, et al., 2002; Souchay, et al., 2003). Feeling of knowing is also impaired (Souchay, Isingrini, & Gil, 2002) although other studies have suggested some preservation of online monitoring because AD patients adjust their estimates of task performance after repeated attempts at memorization (Ansell & Bucks, 2006; Duke, et al., 2002). If online monitoring is impaired in AD, but less than in bvFTD, this would be consistent with prior studies indicating that anosognosia is less severe, and frontal lobe pathology is less extensive in AD than bvFTD (Banks & Weintraub, 2008; Rabinovici, Seeley, et al., 2007; Salmon et al., 2008; Williamson, et al., 2009)

The goal of the current study was examine metacognition in bvFTD and AD using a variety of approaches including both global judgments and item-by-item feeling of knowing. Although we did not include direct measures of anosognosia, further characterization of metacognitive deficits in bvFTD and AD would provide more information about potential mechanisms for the severe anosognosia in bvFTD. Based on the prior studies of metacognition in AD and the profound frontal involvement in bvFTD, we expected feeling of knowing to be impaired in both disorders, with the worst impairment in bvFTD. We used a metamemory task because this paradigm has been most successful in demonstrating feeling of knowing impairments in AD (Cosentino, et al., 2007; Souchay, 2007). Lastly, we incorporated a condition to assess the effect of feedback on metacognitive judgments, with the hypothesis that bvFTD would be impaired at adjusting to this feedback.

Methods

Participants

Clinical characterization

61 participants were studied including 12 patients with bvFTD, 14 patients with AD, and 35 healthy controls (Table 1). BvFTD and AD patients were consecutively recruited from a large study of bvFTD and related disorders, the goal of which is to develop new diagnostic approaches for FTD, including novel behavioral and imaging methods (PI BLM). Patients are enrolled from a variety of sources, including the UCSF Memory and Aging Center neurobehavioral clinic, and direct referrals to the research project from colleagues. Patients were diagnosed using published criteria (McKhann et al., 1984; Neary, et al., 1998) after a comprehensive evaluation including neurological history and examination, nursing assessment, laboratory evaluation, and a previously described neuropsychological assessment of memory, executive function, language, and mood (Kramer et al., 2003). The neuropsychological battery includes the MMSE (Folstein, Folstein, & McHugh, 1975), tests of working memory (digit span backwards), visuospatial function (copy of a complex figure), visuospatial memory (memory of a figure after 10 minutes), confrontational naming (15 items from the Boston Naming Test (Kaplan, Goodglass, & Wintraub, 1983), a brief syntax comprehension task with five questions requiring participants to point to pictures corresponding to specific sentences (e.g. point to the picture of the woman being kissed by the man), five calculations, set-shifting (modified version of the Trails B task (Reitan, 1958)), and tests of verbal fluency (words beginning with the letter ‘D’ or ‘H’ and animals), and non-verbal fluency (design fluency (Delis, Kaplan, & Kramer, 2001)). The Geriatric Depression Scale (GDS, (Yesavage et al., 1983)) is used to assess mood. All the clinical data for each patient are reviewed at a consensus conference including a neuropsychologist, nurse and physician, during which a clinical diagnosis is established.

Table 1.

Word Pairs, Associate Frequency, Written Frequency¹ and Imageability²

Probe Word	Written Frequency	Imageability	Target Word	Written Frequency	Imageability	Associative Frequency
Dress	67	595	Table	198	582	0
Face	371	581	Turn	233	384	.01
Wedding	32	594	Breakfast	53	586	.01
Gun	118	613	Belt	29	494	.02
Hospital	110	602	Ward	25	*	.03
Cigar	10	619	Pipe	20	598	.02
Cancer	25	567	Ill	39	*	.02
Soda	3	544	Bottle	76	619	.01
Bomb	36	606	Noise	37	*	.02
Earthquake	9	*	Focus	40	*	.01
Farmer	23	*	Ground	186	513	.01
Heart	173	617	Building	160	578	0
Ball	110	622	Salt	46	570	0
Theatre	29	*	Ceiling	31	557	0
Sofa	6	597	Ankle	8	613	0
Park	94	573	Cake	13	624	0
Car	274	638	Room	383	545	0
Baby	62	608	Cook	47	504	0
Rose	86	623	Gate	37	545	0
Keys	34	*	Coffee	78	618	0

^1,2See Methods for sources

^*No imageability data available for these words

Controls were recruited through community educational offerings at UCSF and advertisements in local newspapers for participants interested in research on normal aging. They were required to have no cognitive complaints, and to bring to the evaluation a knowledgeable informant who could confirm the absence of cognitive or behavioral difficulties. They underwent the same neuropsychological testing as patients. Final confirmation of their status as control participants was established at the same type of consensus conference used to diagnose patients. Because normal variation in cognitive performance encompasses a wide range of scores, low scores on one or two cognitive tasks were not considered sufficient to exclude participants from the control group, unless they were associated with subjective cognitive complaints or concerns from the informant, but scores greater than 1 standard deviation below age-matched norms on more than two cognitive tasks would exclude a participant from the control group.

Subjects were not paid for participation, although parking and other costs were reimbursed.

Testing of Metacognitive Ability

Overview

Metacognitive abilities were tested using a 20-item paired associates learning paradigm, similar to prior studies of metacognition (Connor, et al., 1997; Souchay, et al., 2002), and tasks requiring subjects to predict and evaluate the success of their learning several times during the session. The order of the tasks is depicted in Figure 1. As described below, the paradigm allowed the assessment of several types of metacognitive judgments including: ease of learning and judgment of learning (using global judgments), feeling of knowing, and retrospective confidence ratings (using item-by-item judgments). Also, by asking people to predict their recall and recognition abilities separately we could assess whether patients could still appreciate that recognition should be easier than recall and adjust their estimates accordingly (an assessment of sensitivity to the extrinsic factors described above). Lastly, to assess response to feedback, we added a global performance prediction trial at the end of the paradigm after participants are explicitly told how they have performed.

Figure 1.

Order of tasks in testing session with metacognitive measures in black boxes

Stimuli

The list of memory probes was generated specifically for this study using randomly chosen words from the Edinburgh Word Association Thesaurus (http://www.eat.rl.ac.uk), which contains a corpus of words for which paired associate frequencies have been identified using a sample of normal participants. The frequency of the associate for each target varied between 0 and 0.03, and the mean frequency was 0.008. The list of word pairs used, along with their written frequency (from(Kucera & Francis, 1967)), imageability (from the MRC Psycholinguistic Database; http://www.psych.rl.ac.uk/, (Wilson, 1988)) and associative frequency for the second of each word pair (from http://www.eat.rl.ac.uk) are provided in Table 1. The word pairs were presented visually on a computer screen in white letters on a black background.

Procedures

Ease of learning, memory encoding and judgment of learning (global judgments)

Participants were informed that they would be required to learn a list of 20 word pairs (e.g. PURSE-MONEY) and that after learning the list they would subsequently be shown the first word in each pair and asked to remember the second word in that pair. After seeing two examples, they were asked to say how many pairs they would likely be able to recall (Recall Prediction 1, ease of learning). They were then told that they would be shown the first word in each pair again, this time accompanied by eight choices, including the word that had been the second word in the pair, and they would be asked to choose the correct word from among these eight choices. They were asked to predict how many of the words they would recognize from among the choices (Recognition Prediction 1).

The 20 word pairs were then presented consecutively to the participant on a computer screen for 3 seconds each, and simultaneously read aloud by the experimenter. After studying the 20 word pairs, participants were asked again to say how many pairs they thought they would recall (Recall Prediction 2, judgment of learning), followed by a second recognition prediction.

Free recall

After a 10-minute delay, during which they performed non-verbal reaction time tasks on the computer, participants were shown the first word in each pair, one at a time, and asked to recall the associate for each. If they did not remember the word associate, they were encouraged to guess.

Feeling of knowing (item-by-item) and recognition memory

Next, participants were shown the first word in each pair, one a time, and reminded that they would be shown a list of eight choices with the correct associate among them. For each probe, they were asked to estimate their likelihood of correctly recognizing its pair from the choice of eight using the following scale: 1 – I will definitely recognize it, 2 – I am pretty confident I’ll recognize it, 3 – I am not too sure I’ll recognize it, 4 – I’ll just be guessing. After providing feeling of knowing ratings for all the word pairs, participants were shown the first word in each pair, one at a time, along with eight choices, and asked to choose the one which had been paired with the first word.

Retrospective confidence ratings (item-by-item)

After the recognition task, participants were again shown the first word in each pair, one a time, and asked to rate their confidence that they had chosen the correct associate from among the eight choices. They used the same rating scale as that used for feeling of knowing.

Response to feedback

Immediately after completing the recall, feeling of knowing, recognition and retrospective confidence rating tasks, participants were told how many of the twenty pairs they had recalled correctly. Immediately following this feedback they were asked to make another recall prediction (Recall Prediction 3). Specifically, the experimenter said the following: “You correctly recalled X of the twenty word pairs. If I gave you a similar list of 20 word pairs to remember, how many do you think you would remember?” This was then repeated for recognition. This judgment was delayed until after the recognition task and retrospective confidence ratings so that the feedback would not affect these ratings, but this third prediction came immediately after the feedback.

Analysis

Demographics and cognitive performance

Demographics and scores on cognitive testing, including recall and recognition for the paired associates task, were compared across diagnostic groups using ANOVA and chi-square tests as appropriate.

Metacognitive skills

The global judgments of recall performance and feeling of knowing ratings yielded several variables for analysis:

We used the ease of learning judgments to assess sensitivity to intrinsic and extrinsic factors in the task. To examine this, we subtracted Recall Prediction 1 from Recognition Prediction 1. Because recognition is easier than recall, this difference should be positive in all subjects, and should reflect mostly extrinsic factors (e.g. the parameters of the task). We compared the difference between recognition and recall predictions across groups using ANOVA.

The change between Predictions 1 and 2 allowed us to examine how the experience of studying the list affected metamemory judgments. This would, to some extent, represent sensitivity to intrinsic factors of the stimuli (i.e. memorability). We subtracted Recall Prediction 2 (after studying the list) from Recall Prediction 1 (before studying the list), and compared the difference across groups using ANOVA.

Judgment of learning was assessed based on Recall Prediction 2. The accuracy of judgment of learning was assessed by subtracting the actual number of words recalled from Recall Prediction 2 and comparing this difference across groups using ANOVA. We performed this analysis both before and after covarying for recall performance.

Feeling of knowing ratings were used by participants to indicate how confident they were that they would recognize the paired associate, ranging between no confidence and very high confidence. Thus, the responses were treated as an ordinal set of responses, where each rating level represents a stronger or less strong feeling that recognition will be successful. In theory, these ratings should be determined by the strength of the memory trace, which can be represented by the accuracy of recognition (i.e. items that are correctly recognized would be associated with a stronger memory trace and a higher likelihood of choosing a more confident rating). To quantify this relationship, we used a mixed-effects ordinal logistic model (Skrondal & Rabe-Hesketh, 2004), which used the ordered four-level ratings as the outcome variable and recognition accuracy as the predictor variable. The analysis yields an odds ratio that reflects the likelihood that a subject would choose a more confident rating if the memory trace for that word pair was stronger, as indicated by correct recognition. We included group diagnosis (controls, AD, bvFTD) and the interactions between recognition accuracy and diagnosis to see if the relationship between accuracy and feeling of knowing differs across groups. We included random intercepts to accommodate correlation of ratings within participants. We assessed the statistical significance of the diagnosis-accuracy interactions using a likelihood ratio test and the results of the model fit. The model was fitted using routines in the GLLAMM software package (Rabe-Hesketh, Pickles, & Skrondal, 2001). Retrospective confidence ratings were analyzed using the same approach.

In order to assess whether differences in feeling of knowing accuracy across diagnostic groups might be partially or completely explained by other patient characteristics, each of the measures in the neuropsychological battery was introduced into the ordinal logistic regression model described above and differences in the odds ratios were compared with and without each of these variables. Any substantial changes in the adjusted and unadjusted odds ratios were noted; substantial changes indicate confounding.

We were also interested in characterizing the pattern of ratings in each group, regardless of accuracy, to see whether bvFTD patients took a different approach to the ratings than other groups. For this analysis, the most important metric is the frequency with which each of the rating levels was used in each diagnostic group, rather than their relationship to accuracy. Thus, we used a similar mixed-effects logistic model (Skrondal & Rabe-Hesketh, 2004), this time treating the ratings as a four level categorical, rather than ordinal outcome variable. The model included diagnosis as the predictor variable, and participant-specific random intercepts to accommodate correlation of ratings within each participant. We assessed whether the distribution of ratings for bvFTD and AD differed significantly from controls using a Wald test and the results of the model fit; statistically significant coefficients for the diagnostic variables would indicate significant differences in rating distributions between groups. We fit the model using routines in the GLLAMM software package (Rabe-Hesketh, et al., 2001).

Response to feedback or Recall Prediction 3 occurred immediately after participants were told explicitly how they had performed. We reasoned that this third prediction could be dependent partly on participants’ original opinions about their performance, as well as the direct feedback they received. In particular, we hypothesized that the more their actual performance differed from their prediction of recall (i.e. the worse their judgment of learning accuracy), the more they would be forced to revise their prediction. Thus, we used the judgment of learning accuracy as a predictor variable, along with diagnosis and the interaction between diagnosis and judgment of learning accuracy, in a linear regression model with the Recall Prediction 3 as the outcome variable.

In cases where ANOVA indicated a significant effect across groups, we used Tukey’s test for post-hoc pairwise comparisons.

Hypotheses

Ease of learning/sensitivity to intrinsic and extrinsic factors

Based on prior studies (Moulin, et al., 2000), we expected that AD patients would show intact sensitivity to extrinsic and intrinsic factors, and thus would increase their ease of learning judgment for recognition relative to recall, and would alter their estimate of future performance once they had studied the words to the same degree as controls (recall prediction 1 minus recall prediction 2). No prior studies have addressed this issue in bvFTD, and we had no specific hypotheses about their performance in this area.

Judgment of learning

Based on prior studies (Ansell & Bucks, 2006; Duke, et al., 2002; Souchay, et al., 2003), we expected both AD and bvFTD patients to overestimate the number of word pairs they would recall, but we hypothesized that this impairment would not be significant if actual performance was taken into account.

Feeling of knowing

Based on their severe frontal pathology, we hypothesized that bvFTD would show inaccurate feeling of knowing, indicated by a low odds ratio for the ordinal logistic regression, and a significant difference in this odds ratio compared with controls. Based on prior findings (Ansell & Bucks, 2006; Duke, et al., 2002; Souchay, et al., 2003), we expected AD to be impaired versus controls as well, but with some sparing relative to bvFTD. We had no specific hypotheses about patterns of ratings.

Retrospective confidence ratings

One prior study indicated that retrospective confidence ratings are intact in AD (Moulin, et al., 2003), although another study found deficits (Dodson, et al., 2011). No prior studies have examined this task in bvFTD.

Response to feedback

Based on their clinical characteristics (see introduction) we expected bvFTD to be impaired in adjusting to feedback. Based on recent studies indicating that AD patients react emotionally when they are doing poorly on cognitive testing (Mograbi, Brown, & Morris, 2009), we expected that AD patients might be more sensitive to feedback.

Results

Group Demographics and Cognitive Performance

Both the AD and bvFTD groups were impaired relative to the healthy controls on several cognitive tests, including the paired associates memory task, but there were no significant differences between bvFTD and AD (Table 2).

Table 2.

Comparisons of Demographics and Neuropsychological Test Results¹ Across Three Groups

	Controls	bvFTD	AD
N	35	12	14
Agea	67.8 (6.8)	59 (7.1)b	62.4 (9.2)
Males/Females	16/19	6/6	9/5
Education (yrs)	16.9 (3.5)	15.8 (2.2)	16.3 (2.9)
MMSE (max=30)a	29.6 (0.7)	27.1 (2.7)b	25.8 (2.6)c
Visual Memory (Figure recall 10 minutes)a	12.3 (3.1)	7.8 (4.4)b	5.1 (3.5)c
Modified Trails Timea	26.8 (13.8)	78.8 (40.4)b	62.5 (35.7)c
Design Fluency, # correcta	11.9 (3.7)	6.1 (4.4)b	8.4 (3)c
Phonemic Fluencya	15.9 (5)	9.4 (6.3)b	12.4 (4.2)c
Category Fluencya	24 (4.2)	12.3 (5.3)b	11.9 (5)c
Backwards Digit Spana	5.4 (1.3)	3.9 (1.6)b	4 (1.2)c
Abbreviated BNT (max = 15)a	14.6 (0.8)	12.3 (2.9)b	13.1 (2.1)c
Syntax Comprehension (max=5)a	4.9 (0.3)	4.3 (0.8)	4.6 (0.6)
Modified Rey-Osterrieth Copy (max = 17)	15.4 (1)	14.8 (1.1)	14.5 (2.5)
Calculations (max=5)a	4.9 (0.3)	4.2 (1.3)b	3.8 (1.1)c
GDS (max=30)a	1.9 (2)	7.8 (9.8)b	9.6 (7.1)c
Paired Associates Performance (verbal memory)
Recall (number, max = 20)a	8 (3.2)	3.5 (4.7)b	1.9 (3.7)c
Recognition (number, max = 20)a	13.3 (3.3)	8.8 (5.2)b	5 (5.5)c

^1.Data are mean and (standard deviation)

^aSignificant across groups by ANOVA;

^bSignificant bvFTD vs. Controls;

^cSignificant AD vs. Controls

Metacognition

The main metacognitive findings are summarized in Figures 2–4 and Table 3.

Figure 2.

Recall predictions and performance across groups

Figure 4.

Recall prediction 3 (post-feedback judgment, Y-axis) vs. judgment of learning accuracy (X-axis) in Controls(a), bvFTD(b), and AD (c).

Table 3.

Summary of Metacognitive Findings Across Groups

Measure	Controls	bvFTD	AD	Group comparisons
Judgment of learning accuracya	2.29	6.75	4.43	bvFTD = Controls = ADa
Feeling of knowing accuracyb	OR = 0.13	OR = 0.87	OR = 0.39	bvFTD < Controls >ADb
Retrospective confidence accuracyb	OR = 0.04	OR = 0.45	OR = 0.42	bvFTD < Controls > ADc
Response of prediction to feedbackc	B = -0.28	B = 0.42	B = 0.22	bvFTD ≠ Controls = AD

^aExpressed as difference between Recall Prediction 2 (after list learning) and number of associates recalled.

^bExpressed as log odds ratio for relationship between recognition accuracy and feeling of knowing/retrospective confidence ratings

^cExpressed as the slope for the relationship between judgment of learning accuracy and Recall Prediction 3 (post feedback).

^aNo impairment in bvFTD or AD compared with controls after controlling for memory performance

^bImpaired in bvFTD, intact in AD, though not as good as controls

^cImpaired in bvFTD compared with controls

Ease of learning/sensitivity to intrinsic and extrinsic factors (hypothesis 1)

Recognition predictions were higher than recall predictions in all groups, with no significant differences between them ([F(2,58) = 1.44, p = 0.244)]. The difference in the change in estimate between bvFTD patients and controls was 1.7 (CI_.95 = -0.77, 4.17) and this difference for AD vs. controls was 1.56 (CI_.95 = -0.78, 3.89). After encoding the 20 paired associates, all participant groups lowered their predictions (Figure 2). There was no significant difference across groups in the change between the first and second recall prediction [F(2,58) = 0.76, p = 0.47]. The difference in the change in estimate between bvFTD patients and controls was 1.55 (CI_.95 = -1.18, 4.28) and this difference for AD vs. controls was -0.2 (CI_.95 = -0.18, 2.5). Thus, both AD and bvFTD patients appropriately lowered their estimates based on task conditions and experience with the specific stimuli.

Judgment of learning (hypothesis 2)

Recall Prediction 2 was higher than actual recall in all groups (Figure 2). BvFTD patients rated themselves similar to controls, suggesting that much of the difference in judgment of learning accuracy was due to impaired memory performance. When controlling for actual recall performance, the judgment of learning accuracy did not differ significantly in either patient group compared with controls. The difference in the change in estimate between bvFTD patients and controls was 0.38 (CI_.95 = -3.31, 4.07) and this difference for AD vs. controls was -2.33 (CI_.95 = -6.08, 1.4).

Feeling of knowing (hypothesis 3)

Feeling of knowing accuracy: The mixed-effects ordinal logistic regression examining the relationship between recognition accuracy and feeling of knowing revealed significant differences across groups; the likelihood ratio test of recognition accuracy - diagnosis interaction was 19.01 with 2 degrees of freedom (p = 0.00007). The diagnostic group-specific odds ratios assessing the association of recognition accuracy with feeling of knowing were 0.13 (CI_.95 = 0.09, 0.18) for controls, 0.87 (CI_.95 = 0.26, 1.47) for bvFTD and 0.39 (CI_.95 = 0.17, 0.61) for AD. That is, the association of recognition accuracy with feeling of knowing was statistically significant only for controls and AD subjects. In both groups, subjects with higher recognition accuracy scores had greater odds of having lower (more confident) feeling of knowing than subjects with lower recognition accuracy scores. In bvFTD, the confidence interval included the null value of 1, indicating that there was no significant relationship between FOK ratings and accuracy. The coefficients for the interaction between diagnosis and recognition accuracy were 1.90 (CI_.95 = 1.13, 2.27) for bvFTD vs. controls and 1.10 (CI_.95 = 0.46, 1.74) for AD vs. controls, indicating that both groups were significantly impaired at feeling of knowing relative to controls. There was a suggestion of a difference between bvFTD and AD in feeling of knowing accuracy, but the difference was not statistically significant (coefficient of 0.80, CI_.95 = -0.10, 1.69, p = 0.08).

Adjustment for each of the potential confounding variables in Table 1 (one at a time) did not produce substantial changes in the estimated odds ratios. For controls, the adjusted odds ratios ranged from 0.10 to 0.19, while the range was 0.72 to 0.87 for bvFTD and 0.35 to 0.44 for AD. All adjusted confidence intervals for controls and AD subjects indicated statistically significant associations, while all confidence intervals for bvFTD subjects included the null value of 1.

Patterns of ratings

The results from the mixed-effects multinomial logistic model fit showed that the distribution of ratings for bvFTD and AD differed significantly from controls (Wald test=27.33 on 6 degrees of freedom, p = 0.0001). The estimated coefficients indicated that bvFTD subjects used “moderately confident” and “not very confident” less often and used “just guessing” more often than controls, while AD subjects used “moderately confident”, “not very confident” and “just guessing” more often than controls. We plotted averages of subject-specific category ratings in each diagnostic group (Figure 3a) to describe differences in the rating distribution over the groups. Consistent with the Wald test results, the plot suggests that controls distributed their responses fairly evenly across all four feeling of knowing ratings, but used the “just guessing” rating less than the others. In contrast, AD patients’ responses were skewed toward lower ratings. BvFTD patients’ responses were quite aberrant. Most of their responses fell into the highly confident or just guessing categories.

Figure 3.

Frequency distribution of feeling of knowing (a) and retrospective confidence ratings (b) across groups

Retrospective confidence ratings

The pattern of responses for retrospective confidence ratings across groups was similar to that seen in feeling of knowing, although bvFTD patients showed a less aberrant pattern of ratings (Figure 3b). The mixed-effects ordinal logistic regression examining the effect of recognition accuracy on retrospective confidence ratings revealed significant differences across groups; the likelihood ratio test of recognition accuracy – diagnosis interaction was 73.62 with 2 degrees of freedom, p < 0.00001. The diagnostic group-specific odds ratios assessing the association of recognition accuracy with feeling of knowing was 0.04 (CI_.95 = 0.02, 0.05) for controls, 0.45 (CI_.95 = 0.17, 0.74) for bvFTD and 0.42 (CI_.95 = 0.16, 0.68) for AD. That is, the association of recognition accuracy with retrospective confidence ratings was statistically significant for all diagnostic groups and indicated that subjects with higher recognition accuracy scores had greater odds of having lower (more confident) retrospective confidence ratings than subjects with lower recognition accuracy scores. The coefficient for the interaction between bvFTD (vs. controls) and accuracy was 2.50 (CI_.95 = 1.69, 3.11), and this coefficient for AD was 2.40 (CI_.95 = 1.69, 3.11) indicating that the relationship between retrospective confidence and accuracy was significantly different in bvFTD and AD compared with controls. As in the analyses of feeling of knowing, adjustment for the potential confounding variables in Table 1 produced only small changes in the estimated odds rations for retrospective confidence ratings indicating that these variables did not confound the associations between recognition accuracy and retrospective confidence ratings.

Response to feedback

Recall Prediction 3, which came after direct feedback about recall performance, was lower than all prior predictions in all three groups (Figure 3). However, bvFTD patients did not lower their prediction commensurate with their score (Recall prediction 3 was 5 points larger than actual recall in the bvFTD group, vs. 0 in controls and 2 in AD). Linear regression confirmed a significant interaction between diagnosis and judgment of learning accuracy in predicting Recall Prediction 3 [F(5,55) = 3.78, p = 0.005]. The diagnosis - judgment of learning interaction coefficient for bvFTD vs. controls was 0.7 (CI_.95 = 0.25, 1.15), and for AD vs. controls it was 0.5 (CI_.95 = -0.15, 1.15), indicating that the influence of judgment of learning accuracy on recall prediction 3 was significantly different compared with controls only in bvFTD. Looking at the relationship in each group (Table 2 and Figure 4), the slope was -0.28 (CI_.95 = -0.53, - 0.02) in the controls, indicating that the larger the discrepancy between expected and actual recall, the lower the prediction for Recall Prediction 3 (Figure 4a). The estimated slopes were 0.42 (CI_.95 = -0.1, 0.94) in bvFTD (Figure 4b), and 0.22 (CI_.95 = -0.41, .86) in AD (Figure 4c).

Discussion

This study demonstrated significant impairments in feeling of knowing accuracy in bvFTD and AD, consistent with a hypothesized deficit in online monitoring. The deficit appeared more severe in bvFTD, in that there was some sparing of feeling of knowing accuracy in AD but no significant relationship between feeling of knowing ratings and actual recognition accuracy in bvFTD. The direct comparison of feeling of knowing accuracy between bvFTD and AD was nearly significant (p = 0.08). Even more strikingly, in predicting their performance patients with bvFTD failed to appropriately adjust their predictions even after receiving overt feedback. Although we did not include any measures of anosognosia in this study, such as self-ratings of everyday functions (Banks & Weintraub, 2008; Eslinger, et al., 2005; Williamson, et al., 2009), the finding provides support for the idea that failures in online monitoring and response to feedback could be important contributors to the severe level of anosognosia in bvFTD. Future studies could address this issue by directly linking feeling of knowing and feedback response to anosognosia.

The impaired feeling of knowing in bvFTD is consistent with prior findings indicating that this task is compromised in frontal lobe disease (Modirrousta & Fellows, 2008; Pinon, et al., 2005; Schnyer, et al., 2004; Shimamura & Squire, 1986; Vilkki, Surma-aho, & Servo, 1999). In contrast to our findings, the pattern of feeling of knowing ratings in most of these prior studies was similar in lesioned patients compared with controls. In one study, however, patients with Korsakoff’s syndrome showed decreased use of more cautious ratings, similar to the pattern in bvFTD (Shimamura & Squire, 1986). Some authors have suggested that different types of metacognitive judgment are sensitive to dysfunction in different frontal regions (Pannu, et al., 2005; Schnyer, et al., 2004). Thus, the location or severity of frontal lobe disease in bvFTD may account for the unique pattern of feeling of knowing ratings in bvFTD. This will have to be investigated in future studies using neuroimaging. The psychological interpretation of the aberrant rating pattern in bvFTD is not straightforward. It may reflect a lack of motivation to make a real effort on metacognitive monitoring tasks. Recent discussions of brain mechanisms for monitoring performance have highlighted the influence of motivation on these systems (Taylor, Stern, & Gehring, 2007). Alternatively, more profound impairment in metacognitive monitoring capacity may result in this pattern of ratings. Future studies examining metacognition in bvFTD will have to include experimental approaches that address these competing alternatives.

BvFTD patients did not appropriately adjust their performance predictions even after direct feedback. This is consistent with prior studies indicating that patients with frontal lobe injury, particularly in the ventromedial prefrontal cortex (vmPFC, an area that is severely affected in bvFTD (Liu et al., 2004; Rabinovici, Seeley, et al., 2007)), are insensitive to negative feedback (Bechara, Damasio, Damasio, & Anderson, 1994; Wheeler & Fellows, 2008). Although a large body of neurophysiological investigation has linked the medial frontal regions to feedback and error-related processing (for reviews, see (Ridderinkhof, Ullsperger, Crone, & Nieuwenhuis, 2004; Simons, 2009; Taylor, et al., 2007)), the role of feedback on metacognitive monitoring in brain disease has not been studied. We recently demonstrated that metacognitive deficits in patients with neurodegenerative disease correlate with atrophy in the vmPFC. We hypothesized that vmPFC may be responsible for generating error signals when performance does not match expectations, and that absence of these error signals due to vmPFC damage may contribute to metacognitive deficits (Rosen et al., 2010). The lack of appropriate response to feedback in bvFTD supports this possibility.

As predicted from prior studies (Cosentino, et al., 2007; Souchay, et al., 2002), AD patients were also significantly impaired at feeling of knowing relative to controls, but showed some preservation of this ability. The value for their odds ratio was significant, indicating that their memory trace, as represented by their recognition accuracy, influenced their feeling of knowing ratings. This is consistent with prior studies indicating some preservation in online monitoring in AD (Ansell & Bucks, 2006; Duke, et al., 2002). Although the difference between bvFTD and AD on feeling of knowing judgments was not significant, it was nearly so (p = 0.08), and bvFTD patients showed no significant relationship between recognition accuracy and feeling of knowing. Given that feeling of knowing has been strongly linked to the frontal lobes (Janowsky, et al., 1989; Modirrousta & Fellows, 2008; Pannu & Kaszniak, 2005; Pannu, et al., 2005; Pinon, et al., 2005; Schnyer, et al., 2004; Schwartz & Bacon, 2008; Vilkki, et al., 1998), the finding that feeling of knowing abilities in AD are intermediate between those in bvFTD and controls is consistent with the fact that frontal lobe disease is present in both AD and bvFTD, but is more severe and diffuse in bvFTD (Rabinovici, Seeley, et al., 2007). Anosognosia is also more severe in bvFTD than AD, and so the current findings support the idea that differences in online monitoring may contribute to the differences in anosognosia between the groups.

BvFTD and AD patients showed relatively normal metamemory in some areas, for instance in downgrading their performance estimates appropriately as they gained more experience with the items they would need to recall, and appropriately estimating that recognition performance would be better than recall. This suggests that bvFTD and AD patients are still sensitive to extrinsic (e.g. type of memory task) and intrinsic (e.g. memorizeability of the stimuli) factors and underscores the fact that metacognitive tasks dependent on these aspects of metamemory may not tap strongly into the self-monitoring deficits in these patients. Accordingly, global judgment of learning accuracy, which has been linked to these extrinsic and intrinsic factors (Connor, et al., 1997), was not significantly impaired in either patient group once performance was taken into account.

It is also notable that retrospective confidence ratings were similarly impaired in bvFTD and AD. This is consistent with prior findings from our group and others indicating that bvFTD and AD patients are impaired in retrospectively evaluating their performance (Banks & Weintraub, 2008; Dodson, et al., 2011; Eslinger, et al., 2005; Williamson, et al., 2009), as well as studies indicating that retrospective confidence ratings, in contrast to feeling of knowing judgments, are closely associated with memory abilities and thus may be dependent on different brain structures than feeling of knowing (Modirrousta & Fellows, 2008; Shimamura & Squire, 1988).

Some caveats are worth noting. First, the sample sizes for the patient groups were small because of the relatively low frequency of bvFTD and the fact that, in the age range typical of bvFTD, AD is also relatively uncommon. This likely limited our ability to demonstrate a significant difference in feeling of knowing between bvFTD and AD. Second, the diagnosis of bvFTD was not confirmed pathologically in this group (all are still living). Impairments in self-monitoring likely represent the specific neuroanatomical involvement in bvFTD rather than the neuropathology per se, as prior studies have indicated that patients with pathological or imaging evidence of AD pathology still show the behavioral signs of bvFTD if brain regions normally affected in bvFTD are involved (Rabinovici, Furst, et al., 2007). Third, we specifically assessed metamemory, as opposed to other types of metacognitive judgments. Prior studies have indicated that bvFTD is associated with poor metacognitive knowledge for other types of tasks as well (Eslinger, et al., 2005; Williamson, et al., 2009), but there is also data suggesting that metacognitive abilities are not uniformly affected in bvFTD (Banks & Weintraub, 2008). Further study of metacognition across cognitive and social domains of function is warranted.

Furthermore, it should be noted that the details of our experimental protocol could influence the outcome through multiple mechanisms. For instance, the linguistic features of our word lists (written frequency, associative frequency, imageability), and other aspects of our paradigm, including having to make judgments about learnability and knowing that the task would involve metamemory, could be associated with retroactive and proactive interference with memory, distraction due to task-switching, and many aspects of the task could influence motivation. The potential for the combined effects of these various factors is difficult to quantify or predict a-priori, and the effects may interact with diagnosis, given that functions such as motivation, concentration and related functions are at least partially frontally mediated. To the extent that these effects might influence memory performance, it is reassuring that memory performance was not different in the two patient groups. Similarly, the lack of difference in performance on standard executive function tasks across patient groups suggests that the patient groups were not dis-similar in terms of typical executive functions including concentration (e.g. fluency) and set-shifting (e.g. Trails task). Of course, both patient groups were impaired in these functions compared with controls, but accounting for these variables did not have a significant impact on the metacognitive outcome measures. Thus, while it is important to acknowledge that these factors could have contributed to our findings in ways that we cannot account for completely, we would conclude based on our analysis that the findings here provide insight into metacognitive monitoring despite the potential confounds associated with these tasks and patient populations. With regard to motivation, which is a frontal lobe function particularly affected in bvFTD (Liu, et al., 2004) that was not directly measured here, as noted above in our discussion of the patterns of feeling-of-knowing ratings, motivation is clearly a factor that may explain the differential metacognitive findings across groups and this will need to be explicitly addressed in future studies.

Lastly, although we outline hypothetical links between deficits in metacognition, in particular feeling of knowing, response to feedback and frontal lobe pathology, these relationships will have to be investigated in future studies that use neuroimaging and appropriate measures of anosognosia.