Neural correlates of the Dunning-Kruger effect

Abstract

The Dunning-Kruger effect (DKE) is a metacognitive phenomenon of illusory superiority in which individuals who perform poorly on a task believe they performed better than others, yet individuals who performed very well believe they under-performed compared to others. This phenomenon has yet to be directly explored in episodic memory, nor explored for physiological correlates or reaction times. We designed a novel method to elicit the DKE via a test of item recognition while electroencephalography (EEG) was recorded. Throughout the task, participants were asked to estimate the percentile in which they performed compared to others. Results revealed participants in the bottom 25th percentile over-estimated their percentile, while participants in the top 75th percentile under-estimated their percentile, exhibiting the classic DKE. Reaction time measures revealed a condition-by-group interaction whereby over-estimators responded faster than under-estimators when estimating being in the top percentile and responded slower when estimating being in the bottom percentile. Between-group EEG differences were evident between over-estimators and under-estimators during Dunning-Kruger responses, which revealed FN400-like effects of familiarity supporting differences for over-estimators, whereas “old-new” memory event-related potential effects revealed a late parietal component associated with recollection-based processing for under-estimators that was not evident for over-estimators. Findings suggest over- and under-estimators use differing cognitive processes when assessing their performance, such that under-estimators may rely on recollection during memory while over-estimators may draw upon excess familiarity when over-estimating their performance. Episodic memory thus appears to play a contributory role in metacognitive judgements of illusory superiority.

Graphical Abstract

1 |. INTRODUCTION

The fool doth think he is wise, but the wise man knows himself to be a fool (Shakespeare, 1601).

The Dunning-Kruger effect (DKE) describes the phenomenon in which poor performers on a task tend to over-estimate their performance while high performers on a task tend to under-estimate their performance. Over-estimation has been a topic of interest throughout recorded history, as early as the time of Socrates, who was noted by Plato for identifying that “…it is likely that neither of us knows anything worthwhile, but he thinks he knows something when he does not, whereas when I do not know, neither do I think I know; so I am likely to be wiser than he to this small extent, that I do not think I know what I do not know,” and later with Charles Darwin noting more simply that “Ignorance more frequently begets confidence than does knowledge” (Darwin, 2009/1871). Also implied by these timeless observations is that these metacognitive illusions are bi-directional, such that the most competent individuals who perform highest tend to also under-estimate their abilities (for Review see Zell et al., 2019).

Misperceptions of inaccurate belief in one’s abilities are a common cognitive phenomenon that can happen to anyone (including the authors and readers) and can lead to serious problems that are often preventable. For example, the misbelief that the Titanic was unsinkable led to loss of over 1,500 lives (Bartlett, 2012; Lord, 1955, 1986), and in modern times, the COVID-19 pandemic has been noted for widespread over-estimation of abilities to manage the pandemic concurrent with early under-estimation of its worldwide impact by many world health organizations, governments, and media alike (with a few notable exceptions). Conversely, when the most competent people hold back their contributions from teams or society because they think others are better suited, the effects can also lead to serious problems that can negatively impact society. For example, we could lose the most competent people for leadership positions and instead embrace those who merely think they are the best (incorrectly). Therefore, it is important to understand how and why these inaccurate judgements of one’s abilities relative to others occur so that they can be prevented.

In 1999, the study of perceived abilities compared to others was further characterized by social psychologists David Dunning and Justin Kruger. In their landmark paper, Dunning and Kruger conducted several studies showing that bottom performers on a test of humour judgements, logical reasoning and grammar abilities over-estimated their performance percentile and that, conversely, top performers under-estimated their performance percentile. The DKE is thus a psychological phenomenon in which a mismatch occurs between one’s perceived ability compared to their peers and the reality of one’s actual performance percentile on a given task. Low performers (individuals who do not earn high scores on a test using an objective scale) tend to over-estimate their performance percentile on a task while high performers (individuals who earn high scores measured on an objective scale) tend to under-estimate their performance percentile on the same task, with the direction of this perceptual mismatch extending in both directions (Sieber, 1979). Empirically, this paradigm has been used successfully in many different tasks to elicit the DKE on such tasks as microeconomics college examinations (Ryvkin, Krajč, & Ortmann, 2012), logical reasoning (Schlösser, Dunning, Johnson, & Kruger, 2013), cognitive reflection (Pennycook, Ross, Koehler, & Fugelsang, 2017), size judgements (Sanchez, 2016), finance (Atir, Rosenzweig, & Dunning, 2015) and computer programming (Critcher & Dunning, 2009). More broadly, the effect has been referred to in contexts of driving (Svenson, 1981), aviation (Pavel, Robertson, & Harrison, 2012) and professors rating their own teaching skills (Cross, 1977). However, throughout the extant work, an account of the cognitive processes leading to these illusory experiences has yet to be fully explored.

1.1 |. The Dunning-Kruger Effect

Most paradigms used to research the DKE follow a similar format: participants are given a task such as a series of logical reasoning problems or math problems, and after they finish the task in its entirety, they are asked to estimate their overall percentile estimate (e.g. compared to others) and objective score on the task. That is, their metacognitive judgement is measured as a single data point assessed at the conclusion of the study and represents their aggregated assessment of performance across many trials. That approach has not, however, permitted the ability to measure multiple instances of the cognitive phenomenon per person and has precluded being able to collect simple statistical measures of central tendency, such as reaction times (RTs). Accordingly, extant approaches have also precluded the collection of neuroscientific measures that rely upon measures of multiple trials per person (electroencephalography [EEG], fMRI, etc.), but which may illuminate contributory cognitive processes.

In social psychology, there is the related phenomenon known as the “better-than-average-effect” (BTAE), whereby people are found to rate themselves above the 50th percentile in comparison with their peers (for Review see Alicke & Govorun, 2005; Brown, 1986, 2010; Chambers & Windschitl, 2004; Hartwig & Dunlosky, 2014; Moore & Healy, 2008; Sedikides, Gaertner, & Cai, 2015; Zell et al., 2019). The BTAE reflects a general self-evaluation bias (Kwan, John, Kenny, Bond, & Robins, 2004; Moore & Healy, 2008), though as noted by Zell et al. the DKE emphasizes the bi-directional illusions of over-estimation and under-estimation.

This pattern of findings has persisted indirectly in research on memory as well (Hirst et al., 2015; Kvavilashvili, Mirani, Schlagman, Foley, & Kornbrot, 2009), providing indications that the DKE may manifest in memory. People can often exhibit illusory over-confidence in their memories (Chua, et al, 2012; Koriat & Ma’ayan, 2005; Nelson & Narens, 1990; DeSoto & Roediger, 2014; Wells, et al., 2002). For example, so-called “flashbulb memories” are some of the most salient memories yet are found to be no more accurate than less salient memories (Brown & Kulik, 1977; Neisser & Harsch, 1992). Other experiments studying the “false fame” phenomenon demonstrate that familiarity with names can lead to falsely recognizing them as famous later if the context of how the names were learned was not recollected (Dywan & Jacoby, 1990; Jacoby, Kelley, Brown, & Jasechko, 1989; Jacoby Woloshyn, & Kelley, 1989, 2004). Related research on estimates of memory comes from studies of the criminal justice system, where eye-witness testimony of memory has been found to be notoriously unreliable, and laboratory studies find that participants over-estimate their memory accuracy (Heaton-Armstrong, Shepherd, Gudjonsson, & Wolchover, 2006; Loftus, 1975; Loftus & Zanni, 1975; Nadel & Sinnott-Armstrong, 2012; Pena, Klemfuss, Loftus, & Mindthoff, 2017; Schacter & Loftus, 2013). Together, these converging results demonstrate a link that may inherently exist between the two domains of memory and metacognition, yet which remains largely unexplored with direct studies.

1.2 |. Theoretical Accounts of the DKE

One account of the effect that Dunning and Kruger proposed was that the reason for low performers’ incorrect percentile estimation is due to meta-ignorance or twofold ignorance (Kruger & Dunning, 1999), in which poor performers are unaware that they are ignorant of the details needed to correctly complete the task and that double ignorance bolsters feelings of false superiority (Schacter, 2012). More simply, poor performers do not have the knowledge to complete the task correctly, and because they do not know their answers are incorrect, they believe they are performing well (i.e. “ignorance is bliss”) (Schlösser et al., 2013). While this is a very useful behavioural description, it does little to advance an understanding of the cognitive processes involved in the pervasive illusion.

Dunning and Kruger also used what they coined “reach-around-knowledge” to explain low performers’ inflated belief in their abilities, which refers to a person’s unique knowledge gained from previously participating in a similar task to the task presented and generalizing their past experiences to the current experience (Dunning, 2011) (for alternative views of the DKE, see Gignac & Zajenkowski, 2020; Karjc & Ortmann, 2008; Kreuger & Mueller, 2002; Mahmood, 2016; Sullivan, Ragogna, & Dithurbide, 2018). Although the concept of “reach-around-knowledge” is not operationally defined and lacks a substantive construct grounded in cognitive psychology, it nevertheless provides a useful platform from which to expand into the theoretical constructs of memory. “Reach-around-knowledge” refers to changes in current behaviour based upon prior experience, which is fundamentally a defining feature of memory (Rudy, 2013), and as such, it recognizes a key role that memory processes may play in contributing to this metacognitive illusion.

1.3 |. Familiarity and Recollection

There is a rich empirical history of memory processes being both theoretically and operationally defined and studied. Two of these cognitive processes of episodic memory that may contribute to the DKE are familiarity and recollection (Eichenbaum et al., 2008; Yonelinas, 1999, 2002; Yonelinas, Aly, Wang, & Koen, 2010). These processes align closely with the general concepts of the “reach-around-knowledge” account, and importantly, we can draw upon this platform of memory processes in approaching the DKE in a systematic manner, which will be discussed in depth in the sections below.

The cognitive processes of familiarity and recollection have featured prominently in theoretical models of episodic memory for several decades (Diana, Yonelinas, & Ranganath, 2008; Eichenbaum, Yonelinas, & Ranganath, 2007; Rugg & Curran, 2007; Squire, Wixted, & Clark, 2007; Yonelinas, 1999, 2002; though see Wixted, 2007 and Wixted & Mickes, 2010 for alternative views), and it is possible that understanding of familiarity and recollection processes in memory may help explain a proportion of variance in the DKE. Recollection is typically operationalized as the declarative retrieval of episodic information of both the item and context bound together into a cohesive retrieval of the episodic event (for review, see Diana et al., 2008) and in empirical studies is usually associated with the retrieval of contextual information bound together with the item of the event, such as source memory (Addante, Ranganath, & Yonelinas, 2012; for reviews, see Eichenbaum et al., 2007; Ranganath, 2010; Yonelinas et al., 2010). The item in the event, however, may be retrieved without recollection, via reliance upon familiarity, which is typically conceptualized as retrieval of an item from a prior episode but without the associated contextual information in which it occurred. Familiarity occurs, for instance, when a person can remember that someone seems recognizable from the past but cannot retrieve specific information of who the person is or from where they know them. Recollection, on the other hand, would be remembering precisely who someone is and how you know them from a prior episode of one’s past experience.

Each of these two memory phenomena has been found to be dissociable cognitive processes (Yonelinas, 2002), with dissociable neural substrates (Ranganath et al.,), dissociable among amnesia patients (Addante, Ranganath, Olichney, & Yonelinas, 2012; Bowles et al., 2007), and with distinct patterns of electrophysiology at the scalp (Addante, Ranganath, & Yonelinas, 2012; Mecklinger & Bader, 2020; Rugg & Curran, 2007; Rugg et al., 1998). Physiologically, familiarity has been associated with event-related potential (ERP) differences in old and new memory trials during a negative-going peak at the mid-frontal scalp sites at approximately 400–600 ms post-stimulus, called the mid-frontal old-new effect, or FN400 (for frontal-N400 effect). On the other hand, recollection has been associated with differences between memory conditions occurring at a peak in the ERP at the parietal region of the scalp from approximately 600 to 900 ms called the late parietal component, or LPC (Addante, Ranganath, Olichney, et al., 2012; Addante, Ranganath, & Yonelinas, 2012; Leynes et al., 2005; for reviews, see Friedman, 2013; Mecklinger & Bader, 2020; Rugg & Curran, 2007).

1.4 |. A proposed memory-based framework of the DKE

Many of the accounts of the DKE have focused primarily upon interpretations based upon metacognition and competency (Adams & Adams, 1960; Ehrlinger & Dunning, 2003; Kruger & Dunning, 1999; Oskamp, 1965; Pennycook et al., 2017; Ryvkin et al., 2012; Sanchez & Dunning, 2017). However, it is likely that the DKE could also be contributed to by memory experiences in one’s past influencing the real-time processing of the current information—either via explicit or implicit means. Based upon the converging literature from memory and metacognition, a viable alternative theory we postulate to explain the DKE is that illusory superiority may also be driven, at least in part, by increased familiarity from prior experience of one’s past with the tested materials (e.g. Chua et al., 2012).

People may use a decision heuristic inducing a sense of performing well despite a lack of specific retrieval of the relevant details that would be involved in marking competency with the material. In this view, the experience of lacking a distinct recollection for but being generally familiar with material will lead people to assume that they are competent and successful at the task. This scenario would be associated with increased FN400 amplitudes in ERPs for inaccurate over-estimators. In that case, it would be a relatively “dangerous” combination to have insufficient recollection but excessive familiarity with a given topic, stimuli or information (Chua et al., 2012) because it could lead to inaccurate over-estimates of one’s abilities and competencies. Accordingly, under-estimators may be marked by having had sufficient recollection of the studied material (e.g. competency), such that these instances are associated with an LPC, while also leading people to recollect the extent of noncriterial information that their cognizance acknowledges could still be relatively wrong (Parks, 2007; Parks & Yonelinas, 2007; Yonelinas & Jacoby, 1996), hence lowering their estimated scores relative to other people. In this case, the excess of recollection signal would outweigh the noise of the familiarity signal.

1.5 |. Current Study

The current paradigm has been designed to study the metacognitive decision-making process as it occurs in real time during DKE percentile estimates that are provided by participants throughout an item recognition memory test. The investigation is focused upon examining the DKE as it applies to judging one’s performance compared to others (e.g. percentile ranking) since that is most directly comparable to the existing studies of DKE research, and not to estimation of one’s own raw performance, per se. Based upon the memory framework for making metacognitive decisions noted above, broadly, we hypothesized that low performers who tend to over-estimate their percentile ranking may do so because of familiarity for previous experiences in similar situations with markedly less reliance on recollection; and that high performers who tend to under-estimate their percentile ranking may use more recollection to accurately outperform their peers. Accordingly, we hypothesized that a larger FN400 will be evident in the group-level ERPs for low performers compared to high performers and that there would be a larger LPC evident in the ERPs of high performers compared to low performers.

2 |. METHODS

2.1 |. Participants

The experiment was conducted as approved by the California State University—San Bernardino Institutional Review Board protocol for research on human subjects. Participants were recruited through a combination of methods including advertisements placed around CSUSB or through the school-wide research pool SONA. Participants recruited through advertisements were paid $10 an hour for sessions that lasted approximately 2 hr. The study consisted of 61 right-handed participants (48 female) who were students at California State University, San Bernardino, and reported being free from neurological and memory problems. Four participants’ data were not used due to noncompliance issues (pressed only 1 button throughout the task or ignored experimenter’s instructions), and one participant did not have usable data due to a experimenter error that resulted in the loss of that data. Two participants did not have usable EEG data due to excess motion artefacts/noise that resulted in a majority of trials being excluded from EEG but were included in behavioural analyses. This presented a behavioural data set of N = 56, and EEG data set of N = 54. 56.5% of the participants were self-reported to be Hispanic, 22.6% Caucasian, 11.3% Asian and 9.7% identified as more than one or another ethnicity. The average age was 23.52 years old (SD = 4.82). None of the participants reported any visual, medical or physical issues that would interfere with the experiment. Most participants spoke English as their first language (N = 47), and the 15 whom indicated speaking a different language first had been speaking English for an average of 16.73 years (SD = 4.74).

2.2 |. Procedure

Participants arrived at the laboratory and completed informed consent and demographic information forms via voluntary self-report. The paradigm used was a modified item recognition confidence test, which is built from similar paradigms successfully used in our laboratory’s prior research (Addante, 2015; Addante, Ranganath, Olichney, et al., 2012; Addante, Ranganath, & Yonelinas, 2012; Addante, Watrous, Yonelinas, Ekstrom, & Ranganath, 2011; Roberts, Clarke, Addante, & Ranganath, 2018) and described in further detail below (Figure 1). This paradigm consisted of an encoding phase containing four study sessions, in which participants studied 54 words in each session, and a retrieval phase, containing six test sessions in which the participant’s memory was tested for 54 words in each session. They viewed a total of 324 words, 216 of which were presented in the encoding phase and 116 of which were unstudied items (new items).

FIGURE 1. Memory Retrieval and Dunning-Kruger Testing Paradigm.

Left: Participants indicated their confidence for item memory and source memory. For every 10^th stimuli presented, the participants viewed the Dunning-Kruger Estimate: asking participants to estimate the percentile in which they believe they are performing up to that point on the task in relation to other students.

During encoding, participants were given instructions to make a simple decision about the word presented on the screen. Subjects were asked to either judge whether the item was manmade or whether the item was alive and conditions were counterbalanced. The stimuli were presented on a black computer screen in white letters. To begin a trial, a screen with a small white cross at the centre was presented for one of three randomly chosen inter-stimuli interval times: 1 s, 2.5 or 3 s. Then, the stimulus word appeared in the middle of the screen with “YES” presented to the bottom left of the word and “NO” presented to the bottom right of the word. The participants indicated their answer by pressing buttons corresponding to “YES” and “NO” with their index and middle fingers, respectively, and this response was self-paced by the participant. After the participants responded, they viewed a blank black screen at a random duration of 1, 2.5 or 3 s. After the blank screen, the small white cross appeared at the centre of the screen to begin the next trial. This cycle continued until all 54 words in all four lists were presented. Between each list, participants were read the instructions for the next task to prevent carryover effects of the preceding encoding task and to ensure they correctly switched between the animacy and the manmade decision task.

After the encoding phase was complete, the EEG cap was sized and ocular electrodes were attached. EEG was recorded using the actiCHamp EEG Recording System with a 32-channel electrode cap conforming to the standard International 10–20 System of electrode locations. Each subject was tested individually inside a sound-attenuating chamber. Stimulus presentation and behavioural response monitoring were controlled using Presentation software on a Windows PC. EEG was acquired at a rate of 1,024 Hz. Subjects were instructed to minimize jaw and muscle tension, eye movements and blinking. EOG was monitored in the horizontal and vertical directions, and these data were used to eliminate trials contaminated by blinks, eye movements or other related artefacts. Five ocular electrodes were applied to the face to record electrooculograms (EOG): two above and below the left eye in line with the pupil to record electrical activity from vertical eye movements, two on each temple to record electrical activity from horizontal eye movements and one in the centre of the forehead just above the eyebrows as a reference electrode. The EEG cap was placed on the participant’s head and prepared for electrical recording. Gel was applied to each cap site, and impedances were lowered below 15 KOhms via gentle abrasion to allow the electrodes to obtain a clear electrical signal.

After the EEG cap was in place, the participant began the retrieval phase. The participants were read instructions asking them to judge whether the stimulus word presented was old (studied during the encoding phase) or new (not studied in the encoding phase; Figure 1). As in the encoding phase, all stimuli words were presented in white font on a black screen. At the beginning of each trial, participants were presented with a word in the middle of the screen, the numbers “1,” “2,” “3,” “4” and “5” evenly spaced beneath the word, the word “New” on the left by the number “1,” and the word “Old” on the right under the number “5.” Participants pressed any number between “1” and “5” to indicate whether they confidently believed the word was old (“5”), believe the word was old but was not confident (“4”), did not know if the word was old or new (“3”), believe the word was new but was not confident (“2”) or confidently believed the word was new (“1”) (Addante, Ranganath, Olichney, et al., 2012; Addante, Ranganath, & Yonelinas, 2012; Addante et al., 2011). This prompt was subject-paced. Participants were told to choose the response that gave us the most accurate reflection of their memory and to respond as quickly and accurately as possible. Immediately after the item recognition judgement, participants were asked to answer a source memory confidence test indicating if the word came from the animacy decision task or the manmade decision task during encoding on a scale of 1 to 5, which was also subject-paced (Addante, Ranganath, Olichney, et al., 2012; Addante, Ranganath, & Yonelinas, 2012; Addante et al. 2015, 2011; Roberts et al., 2018). After responding, participants viewed a blank black screen at a random duration of 1, 2.5 or 3 s. Participants were instructed to blink only during this blank screen and avoid blinking during the screens with a small cross or stimuli.

2.3 |. Dunning-Kruger In-test Questions

After the source memory test for each 10th word presented during the memory test, the Dunning-Kruger estimate was presented. Participants received instructions asking them to estimate the percentile in which they believed they were performing up to that point in the test compared to other students who would participate in the study (subjects were instructed to focus on generic memory performance and to use their item memory as the primary context). During the test phase, the word “Percentile?” was presented as a prompt for their estimate with the numbers “<60%,” “60’s,” “70’s,” “80’s” and “90%+” evenly spaced beneath it. The Dunning-Kruger estimate was subject-paced.

2.4 |. Dunning-Kruger post-test questions

At the conclusion of the memory retrieval test, participants answered four post-test questions. First, they were asked to “Estimate your score on the whole test.” Participants were prompted to respond on a 5-point scale with “1” meaning below 60%, “2” meaning between 60% and 69%, “3” meaning between 70% and 79%, “4” meaning between 80% and 89% and “5” meaning above 90%. The second question they were asked was the following: “In what percentile did you perform on the whole test?” The participants were prompted to respond on a 5-point scale with “1” meaning below the 60th percentile, “2” meaning between the 60th and 69th percentiles, “3” meaning between the 70th and 79th percentiles, “4” meaning between 80th and 89th percentiles and “5” meaning in the 90th percentile or above. The first questions measured perceived objective score on the entire memory test, while the second question measured perceived relative score in relation to other students taking the memory test. These post-test prompts allowed us to test for the DKE at a between-subjects level to be sure the effect can be elicited using an episodic memory task. During analyses, participants were grouped into quartiles based on their percentile score on the test, allowing us to average each group’s responses and test them against the other group’s average responses to determine significant differences. They were also grouped by errors in percentile estimates; groups of over-estimators, correct estimators and under-estimators (also referred to as estimator groups later) were also made to investigate potential differences in cognitive strategies (see below).

The two additional post-test questions were as follows: (a) “Rate your memory in everyday life” and (b) “How difficult was this entire test?” For the first question, participants responded on a 5-point scale with “1” meaning very poor, “2” meaning poor, “3” meaning moderate, “4” meaning good and “5” meaning very good. For the second prompt, participants responded on a 5-point scale with “1” meaning very hard, “2” meaning hard, “3” meaning moderate, “4” meaning easy and “5” meaning very easy.

2.5 |. Dunning-Kruger Groupings

To maintain consistency towards replicating the original report by Kruger and Dunning (1999), subjects were grouped for analyses in the same fashion as the original paper by separating participants into four quartiles depending on their test accuracy and investigating group differences among those quartiles. The way that subjects were selected for their respective group membership was based upon their performance on the item recognition test, divided into four quartiles. Subjects’ accuracy scores on the memory task (measured as the probability of a hit minus the probability of a false alarm, pHit-pFA) were ranked from smallest to largest and split into quartiles of performance (<25%, >25% to 50%, >50% to 75%, and >75%), and participants who fell into these quartiles comprised the low, 2nd, 3rd and highest quartile.

Participants were then regrouped into what we call “estimator groups” or participants that over-estimated, correctly estimated and under-estimated their percentile ranking. To make these estimator groups, first, the percentile rankings described above were given scores of 1 through 5 that directly corresponded to the scale used by subjects to estimate their performance percentile group. For example, a participant who scored in the 21st percentile was assigned a value of 1 while a participant who scored in the 82nd percentile was assigned a value of 4. This allowed the subtracting of each actual percentile score from the participant’s estimated percentile score (estimated percentile—actual percentile) on the post-test measure, thus obtaining a value of how accurately participants estimated their percentile ranking (we used the post-test relative Dunning-Kruger estimate, so as to be consistent with the original approach used in Kruger and Dunning (1999), although we also conducted a paired t test between the average of the in-test Dunning-Kruger responses (M = 3.14, SD = 0.81) for each person to the post-test relative Dunning-Kruger response (M = 3.16, SD = 0.78) and found that the two scores did not differ, t(55) = 1.30, p = .20). Positive values indicated over-estimations, values of 0 indicated correct estimations, and negative values indicated under-estimations. As an example, a participant who estimated their score to be in the range of the 80–89th percentile (which would correspond to a response of 4 on the response scale) and yet actually performed in the 74th percentile (a corresponding response of 3) would be categorized as an over-estimator. Thus, these new groups became our over-estimators (N = 38), correct estimators (N = 8) and under-estimators (N = 10). From these group memberships, the ERP analyses were conducted in accordance with standard practices in the field of including only subjects who met a sufficient number of trials in each condition of the ERP comparison for signal-to-noise ratio (SNR; see Methods section below for full details) (over-estimators (N = 36), correct estimators (N = 8) and under-estimators (N = 10)).

2.6 |. Electrophysiological Analyses

Physiological measurements of brain activity were recorded using EEG equipment from Brain Vision, LLC. All EEG data were processed using the ERPLAB toolbox using MATLAB (Delorme & Makeig, 2004; Lopez-Calderon & Luck, 2014). The EEG data were first re-referenced to the average of the mastoid electrodes, passed through a high-pass filter at 0.1 Hz and then down-sampled to 256 Hz. The EEG data were epoched from 200 ms prior to the onset of the stimulus to 1,200 ms after the stimulus was presented, and then categorized based on performance and response accuracy.

Independent component analysis (ICA) was performed using InfoMax techniques in EEGLAB (Bell & Sejnowski, 1995) to accomplish artefact correction, and then, resulting data were individually inspected for artefacts, rejecting trials for eye blinks and other aberrant electrode activity. During ERP averaging, trials exceeding ERP amplitudes of ±250 mV were excluded. Using the ERPLAB toolbox (Lopez-Calderon & Luck, 2014), automatic artefact detection for epoched data was also used to identify trials exceeding specified voltages, in a series of sequential steps as noted below. The Simple Voltage Threshold function identified and removed any voltage below −100 ms. The Step-Like Artifact function identified and removed changes of voltage exceeding a specified voltage (100 uV in this case) within a specified window (200 ms), which are characteristic of blinks and saccades. The Moving Window Peak-to-Peak function is commonly used to identify blinks by finding the difference in amplitude between the most negative and most positive points in the defined window (200 ms) and compared the difference to a specified criterion (100 uV). The Blocking and Flatline function identified periods in which the voltage does not change amplitude within the time window. An automatic blink analysis, Blink Rejection (alpha version), used a normalized cross-covariance threshold of 0.7 and a blink width of 400 ms to identify and remove blinks (Luck, 2014).

In order to maintain sufficient SNR, all comparisons relied upon including only those subjects whom met a criterion of having a minimum number of 12 artefact-free ERP trials per condition being contrasted (Addante, Ranganath, & Yonelinas, 2012; Gruber & Otten, 2010; Kim et al., 2009; Otten et al., 2006; c.f. Luck, 2016). ERPs of individual subjects were combined to create a grand average, and mean amplitudes were extracted for statistical analyses. Topographic maps of scalp activity were created to assess the spatial distribution of effects. For ERP figures, a 30 Hz low-pass filter was applied to ERPs so as to parallel the similar “smoothing” function that ensues from taking the mean voltage between two latencies during standard statistical analyses (i.e. Addante, 2015). ERP results are reported for representative electrode sites but were also found to be reliable at surrounding 3-site clusters of electrodes unless otherwise noted.

2.7 |. Behavioural Results

Recognition memory response distributions for recognition of old and new items are displayed in Table 1. Item recognition accuracy was calculated as the proportion of hits (M = 0.81, SD = 0.11) minus the proportion of false alarms (M = 0.24, SD = 0.14) (i.e. pHit-pFA) (Addante et al., 2011; Addante, Ranganath, Olichney, et al., 2012; Addante, Ranganath, & Yonelinas, 2012). Participants performed item recognition at relatively high levels (Max = 0.87, Min = 0.18, M = 0.57, SD = 0.15) which was greater than chance probability, t(55) = 3.59, p < .001. In addition, participants’ accuracy for high confidence item recognition trials (“5’s”) was significantly greater than low confidence item recognition trials (“4’s”), t(55) = 9.04, p < .001. Source memory response distributions for recognition of old and new items are displayed in Table 2. Source memory accuracy values were collapsed to include high and low source confidence responses which were then divided by the sum of items receiving a correct and incorrect source response to calculate the proportion (Addante, Ranganath, Olichney, et al., 2012; Addante, Ranganath, & Yonelinas, 2012; Roberts et al., 2018). Mean accuracy for source memory was 0.30 (SD = 0.19) and was reliably greater than chance, t(55) = 11.78, p < .001. The results of item memory confidence and source memory confidence scores and ERPs replicated the previous findings of Addante, Ranganath, and Yonelinas (2012), as reported in further detail by Muller (2019).

TABLE 1.

Distribution of responses for each item response as a proportion of all memory responses

Item Recognition Confidence	1	2	3	4	5
All Old Items	0.09	0.07	0.04	0.21	0.60
All New Items	0.43	0.23	0.10	0.15	0.08
Animacy Task	0.13	0.06	0.04	0.16	0.60
Manmade Task	0.08	0.04	0.03	0.14	0.71

TABLE 2.

Distribution of responses for each source response as a proportion of all memory responses

Source recognition confidence	1	2	3	4	5
All old items	0.14	0.14	0.22	0.17	0.33
All new items	0.05	0.08	0.70	0.09	0.08
Animacy task	0.24	0.17	0.27	0.16	0.16
Manmade task	0.11	0.11	0.17	0.2	0.41

2.7.1 |. Dunning-Kruger Performance Judgements

The distribution of responses for each Dunning-Kruger response category for the post-test and in-test Dunning-Kruger responses is shown in Table 3. When plotted against actual performance, results from subjects’ reported performance estimates revealed that the canonical DKE was evident, thereby replicating the DKE and extending it to our novel episodic memory paradigm (Figure 2). To quantify and analyse this effect, the participants were first split into quartiles based on memory accuracy (the procedure for grouping of subjects into groups based upon estimated performance versus actual performance was described in detail earlier in the Methods). The average memory test accuracy, organized by quartile and each quartile’s respective average post-test Dunning-Kruger response, is listed in Table 4. A 3 × 5 ANOVA analysing DK responses using factors of group (over-, under- and correct estimators) and response (DK judgements of 1, 2, 3, 4 and 5) revealed no differences in how subjects distributed their metacognitive judgements across the scale (F(8,265) = 0.58, p = .79), indicating no evident differences in how groups of subjects were responding to the task overall.

TABLE 3.

Distribution of responses for each Dunning-Kruger response, as a proportion of all Dunning-Kruger responses

DKE type	<60%	60%–69%	70%–79%	80%–89%	>90%
In-test DK responses	0.05	0.20	0.39	0.29	0.07
Post-test DK responses	0.02	0.11	0.54	0.30	0.04

FIGURE 2. Actual and estimated performance percentiles.

Participants were separated by their actual percentile ranking. The low group consists of those in the first quartile (less than or equal to 25%), the second group consists of those in the second quartile (>25% and <50%), the third group consists of those in the third quartile (>50% and <75%), and the high group consists of those in fourth quartile (>75%). Participants who performed in the first quartile showed the most over-estimation while participants who performed in the fourth quartile showed under-estimation of their actual percentile.

TABLE 4.

Average recognition memory test accuracy and average posttest and in-test Dunning-Kruger relative response by quartile

Quartile	Accuracy	Average post-test DK relative response	Average in-test DK relative response
Top (N = 14)	0.74 (0.06)	3.50 (0.65)	3.26 (0.73)
3rd (N = 14)	0.62 (0.02)	3.29 (0.99)	3.33 (1.01)
2nd (N = 14)	0.55 (0.04)	2.79 (0.80)	2.79 (0.81)
Bottom (N = 14)	0.38 (0.08)	3.43 (0.51)	3.17 (0.62)

Standard deviations are in parentheses.

The bottom quartile (N = 14, M = 2.43, SD = 0.51, t(26) = 17.69, p < .001), 2nd quartile (N = 14, M = 1.79, SD = 0.80, t(26) = 8.33, p < .001) and 3rd quartile (N = 14, M = 1.43, SD = 1.28, t(26) = 4.16, p < .001) significantly over-estimated their percentile ranking while the top quartile significantly under-estimated their percentile ranking (N = 14, M = −0.79, SD = 0.89, t(26) = −3.29, p = .003). Furthermore, the magnitude of the errors made by each group decreased systematically as their percentile group increased: the bottom quartile over-estimated their percentile by 62.56%, the 2nd quartile over-estimated by 37.95%, the 3rd quartile over-estimated by 14.56%, and the top quartile under-estimated by 8.30%. Together, these basic findings provide evidence that the DKE was successfully elicited by our memory paradigm, one that to our knowledge has not been shown before, thereby extending the DKE phenomenon directly to episodic memory.

Figure 3 displays the raw performance of each subject in item recognition as a function of Dunning-Kruger groupings for their estimates of the percentile group in which they thought they were performing (over-estimator, under-estimator or correct estimator). Performance of the over-estimators on the item recognition memory task (N = 38, M = 0.501 SD = 0.11, SE = 0.02) was significantly less than the correct estimators (N = 8, M = 0.65, SE = 0.02) (t(44) = 3.71, p < .001) and under-estimators (N = 10, M = 0.75, SE = 0.02) (t(46) = 6.66, p < .001), though was reliably greater than chance-level performance (t(37) = 27.341, p < .001). The worst-performing subjects (over-estimators) performed otherwise normatively, above the impaired levels of hippocampal amnesia patients (M = 0.30) that we have reported in previous work using the same paradigm (Addante, Ranganath, Olichney, et al., 2012) and were otherwise consistent with normative performance of healthy adults in published prior studies using the same paradigm (Addante, Ranganath, & Yonelinas, 2012; Addante et al., 2011; Roberts et al., 2018).

FIGURE 3. Recognition Memory Performance per Estimation Group.

Left: Accuracy on Item Recognition Memory Test for Over-estimators, Under-estimators, and Correct-Estimators. Raw scores of each subject are shown plotted as grouped by estimates of performance percentile relative to the group. Memory accuracy was measured as probability of a hit minus the probability of a false alarm, plotted on the y-axis; black bars represent mean values per group. Right: Receiver operating characteristic curve of item recognition memory performance for each of the three groups. Y-axis plots the proportion of hits, x-axis plots the proportion of false-alarms for each level of confidence.

2.7.2 |. Parameter estimates of decision processes during memory retrieval

A possible account of the DKE results is that subjects may have differentially engaged with the task or that results may reflect different decision-making strategies. To assess this possibility, we conducted analyses to quantify whether subjects were using any discernably different decision processes reflecting differential engagement in the memory task, using the ROC Toolbox (Koen, Barrett, Harlow, & Yonelinas, 2017) to calculate parameter estimates of their decision criterion (C), response bias (B), recollection (R_o) and familiarity (F) process contributions to performing the memory task (Koen, Barrett, et al., 2017; Parks & Yonelinas, 2009; Yonelinas, 2002, 2004; Yonelinas et al., 2010). A one-way ANOVA was performed to identify potential differences among groups (under-estimators, correct estimators and over-estimators) on each of the parameters. There were no reliable differences observed among groups for recollection estimates (F(2,52) = 0.75, p = .48), decision criterion (F(2,52) = 1, p = .38) or response bias (F(2,52) = 0.32, p = .73) (Table 5).

TABLE 5.

Memory performance parameter estimates

DK group	Recollection estimate (Ro)	Familiarity estimate (F)	Criterion (C)	Bias (B)
Over-estimators (N = 36)	0.37 (0.23)	0.73 (1.08)	−0.08 (0.42)	1.14 (1.01)
Correct estimators (N = 8)	0.42 (0.26)	1.30 (0.28)	0.06 (0.30)	1.33 (0.77)
Under-estimators (N = 10)	0.48 (0.32)	1.70 (0.45)	−0.20 (0.39)	0.96 (1.01)

Note:: Parameter estimates derived from the dual-process signal detection model. SDs are in parentheses.

Analyses of groups did reveal a significant effect for the parameter estimates of familiarity (F(50,2) = 14.35, p < .001) (one subject of the over-estimating group was removed as an outlier for exceeding three standard deviations from the mean) consistent with the groups’ defined memory differences noted earlier (Figure 3). This effect was further explored with follow-up between-group t tests, which revealed that each group was reliably different from each other in their estimates of familiarity used during item recognition: Under-estimators (N = 8, M = 1.70, SD = 0.45, SE = 0.14) were greater than correct estimators (N = 10, M = 1.30, SD = 0.28, SE = 0.10) (t(16) = 2.22, p = .041); Correct estimators were greater than over-estimators (N = 35, M = 0.89, SD = 0.46, SE = 0.08) (t(41) = 2.389, p = .022), and under-estimators were greater than over-estimators (t(43) = 4.96, p < 0.001; Figure 3), corresponding to the underlying performance differences on the memory test between groups. Thus, outside of their core performance on the task, it appears that subjects were meaningfully engaged in the task in similar ways unattributable to factors of strategies, task engagement or decision-making differences of the groups.

2.7.3 |. Response Speed for Dunning-Kruger Judgements

RTs were unable to be measured in previous studies of the DKE because extant studies were limited by including only a single measure of self-estimate of performance at the end of a task, which is not adequate for RT analyses. The current study, however, collected thirty DKE judgements per subject (n = 30), which permitted analysis of response times during these phenomena, in order to gain insight into how the different groups might have performed the task differently (Table 6). A one-way ANOVA first compared general RTs collapsed across all DKE metacognitive responses (“1” through “5”) between the three groups of over-estimators (N = 38), under-estimators (N = 10) and correct estimators (N = 8), revealing no significant differences in overall response times among groups (F(2, 52) = 0.41, p = .67).

TABLE 6.

Response distribution proportions of Dunning-Kruger responses and mean reaction times, SDs and sample size for in-test Dunning-Kruger judgements organized by estimator group

Group	Dunning–Kruger judgements	1	2	3	4	5
Over-estimators (n = 36)	Response distribution	0.05	0.19	0.39	0.28	0.09
	Reaction time	2,204	2064	1948	2044	1656
	SD	628	641	644	860	544
	N per response	10	23	33	27	13
Correct estimators (n = 8)	Response distribution	0.09	0.28	0.33	0.25	0.05
	Reaction time	1,447	2,323	2,018	1,920	2,275
	SD	263	987	890	733	360
	N per response	2	6	7	5	2
Under-estimators (n = 10)	Response distribution	0.01	0.21	0.35	0.38	0.05
	Reaction time	1,918	2,074	2,166	1,996	2,579
	SD	--	1,249	543	770	478
	N per response	1	5	9	9	3
Combined correct and under-estimators (n = 18)	Response distribution	0.04	0.24	0.34	0.32	0.05
	Reaction time	1,604	2,209	2,101	1,969	2,457
	SD	330	1,062	693	729	635
	N per response	3	11	16	14	5

Note.: Means and SD are in ms.

However, because our hypotheses were specifically interested in how people made illusory metacognitive judgements of being either among the best or the worst performers compared to others, we also specifically analysed the RTs for when subjects reported performing either the best (“5”) or the worst (“1”), as a function of estimator group to explore potential differences in cognitive strategies used to make these illusory self-estimates. For this analysis, the under-estimator group (N = 10) is inherently defined as having a limited number of trials of responding that they believed they were the best percentile, and so, this naturally reduced the sample of available subjects with sufficient trials for these sensitive behavioural analyses (N = 3). Although the current paradigm has been previously established as being sensitive to small samples of the same sizes (Addante, 2015; Addante, Ranganath, Olichney, et al., 2012), we nevertheless sought to increase the sample size of those who did not exhibit the errors of illusory superiority. Therefore, we collapsed the under-estimator group (N = 3) together with the available subjects of correct estimators whom had responses in these otherwise rare categories (N = 5) to create a larger group for analyses (N = 8); the over-estimators with available trials in these conditions were N = 23. We used a two-factor between-subjects ANOVA to determine whether any mean differences existed among the RTs for Dunning-Kruger groups (over-estimators versus correct and under-estimators), while they judged themselves to be in the highest (response of “5”) and lowest (response of “1”) performance groups, respectively. The ANOVA revealed a significant condition-by-group interaction between Dunning-Kruger groups and responses, F(1,27) = 8.35, p = .008, which we explored with planned t tests.

Within group, the RTs for over-estimators when rating themselves as performing in the worst (<60th) percentile (DK response of “1”; M = 2,204 ms, SD = 628 ms, N = 10) were significantly slower than when over-estimators rated themselves in as being in the best 90th percentile (DK response of “5”; M = 1,656 ms, SD = 544 ms, N = 13), t(21) = 2.24, p = .04. Alternatively, the combined group of correct + under-estimators exhibited RTs with the opposite pattern, showing a slower response time when rating themselves in the 90th percentile or above (M = 2,457 ms, SD = 634 ms, N = 5) that was marginally faster when rating themselves as performing less than the 60th percentile (M = 1,604 ms, SD = 329 ms, N = 3; t(6) = −2.12, p = .08) (Figure 4).

FIGURE 4. Reaction Times.

Left: Mean reaction times of high and low percentile estimation by Dunning-Kruger groups. Participants’ belief of performing in the 59th percentile or below corresponds to response of ‘1’ on the task and performing in the 90^th percentile or above corresponds to response of ‘5’. The reaction times are separated by over-estimators and the combined group of correct- & under-estimators collapsed due to relatively small sample sizes individually for these response bins. Mean reaction times are reported in milliseconds. Each black dot represents the raw score of an individual subject for each respective condition. Error bars represent standard error of the mean. Right: reaction times for the memory encoding task per each group. Black bar represents the mean for each group. Note that one participant’s data for encoding RT was excluded from analysis because of having exceeded three standard deviations from the mean (see text): that participant’s data is also excluded from the figure, all other data depicted did not meet the outlier criteria.

Between groups, over-estimators were significantly faster (M = 1,656 ms, SD = 544 ms, N = 13) than the collapsed group of accurate and under-estimators (M = 2,457 ms, SD = 635 ms, N = 5), when each were responding that they thought they were doing the best (i.e. in the 90th percentile or above), t(16) = −2.68, p = .02). The converse pattern was evident when rating themselves in the 59th percentile and lower (Figure 4, Table 6), as the over-estimators responded relatively more slowly (M = 2,204 ms, SD = 628 ms, N = 10) than the combined group of correct and under-estimators (M = 1,604 ms, SD = 330 ms, N = 3), though this general pattern was found not to be significant for the current sample sizes, t(11) = 1.56, p = .15. When the same analyses were performed for RTs of the other DK ratings (2’s and 4’s) using a 2×2 two-factor repeated-measures ANOVA, there were no significant effects of either condition, group or interaction thereof in the lesser grades of responses (all F’s < 1), indicating the interaction effect of RTs found earlier was constrained to the highest and lowest performance estimates.

Overall, the significant findings in RTs were an exploratory analysis using small samples that, like all scientific findings, will benefit from corroboration by independent laboratories. However, these results persisted despite the small sample sizes of the groups, and the patterns suggest that future studies using larger groups may find similar patterns. The pattern of responding revealed evidence that people who erred to over-estimate their abilities were also responding faster when they believed they were doing the best and slower to say they were the worst, whereas more accurate under-estimators were slower to say they were the best and relatively quicker to say they were the worst.

2.7.4 |. Response Times at Encoding

One possibility to account for the DKE is that group-level differences could be due to how people encoded the information into memory (Addante et al., 2015; Craik & Lockhart, 1972); indeed, early accounts of the DKE have posited that results can be due to competency of subjects that can be corrected by improving information acquisition (Kruger & Dunning, 1999). Similarly, subjects’ over-confidence at retrieval could have come from excess fluency at encoding providing feelings that the information was “easily learned,” leading them to rely upon intuitive perceptions of fluency and feelings of familiarity at retrieval that they incorrectly misattributed as better performance (Leynes & Addante, 2016; Leynes & Zish, 2012; Whittlesea & Leboe, 2000, 2003).

To inform these possibilities, we analysed mean encoding times as a general measure of information processing while participants encoded each item, using two-tailed between-group t tests, and excluded one outlier in the group of over-estimators due to exceeding three standard deviations from the mean (Figure 4). Over-estimators (N = 37, M = 1,289 ms, SD = 319, SE = 52) responded faster than under-estimators (N = 10, M = 1,651 ms, SD = 655, SE = 207) during encoding (t(45) = −2.48, p = .016); Correct estimators (N = 8, M = 1,159 ms, SD = 331, SE = 117) also performed marginally faster than the under-estimators (t(16) = 1.92, p = .071). There were no reliable differences evident between RTs of over-estimators and correct estimators (t(43) = 1.04, p = .303) (Figure 4). These findings appear to indicate that under-estimators may have performed better due to having spent (slightly) more time exposed to information tested later.

2.8 |. Electrophysiological Results

The EEG data were analysed in several systematic steps to probe possible differences between metacognitive judgements and cognitive strategies, and as noted in the Methods section, ERP analyses included only subjects who maintained a minimum number of valid ERP trials for both of the ERP conditions being compared, which resulted in somewhat smaller sample sizes from the original N = 61 and is noted in each reported result’s degrees of freedom. First, we assessed the data for general ERP differences that could be identified between the tasks of memory and metacognition judgements. To do this, we assessed the ERPs for decisions in all of the Dunning-Kruger judgements collapsed together compared to the ERPs for all item memory judgements collapsed together (Figure 5). This revealed that ERP activity for the metacognitive DKE decisions was significantly greater than that for memory judgements, starting from approximately 300 ms and continuing through 1,000 ms at almost every electrode site. These effects were maximal at the central parietal site of Pz through 800 ms (300–500 ms: t(53) = 10.69, p < .001; 400–600 ms: t(54) = 15.19, p < .001; 600–900 ms: t(53) = 9.79, p < .001) and similarly reliable at several surrounding sites, upon which time the effects became evident as maximal at mid-frontal site Fz from 900 to 1,200 ms (t(53) = 6.46, p < .001) with similar effects at surrounding sites, consistent with prior ERP findings of parietal and anterior P300a/b effects for novelty processing and oddball paradigms (Curran, 2004; Kishiyama, Yonelinas, & Lazzara, 2004; Knight, 1996; Knight & Scabini, 1998; Woodruff, Hayama, & Rugg, 2006). This basic finding established a foundation that ERPs during the metacognitive judgements of the DKE were reliably distinct from memory-related activity, which we continued further investigation.

FIGURE 5. Comparison of ERPs for memory judgments vs Dunning-Kruger judgments.

a) Topographic difference maps of ERPs for all item memory judgments compared to all Dunning-Kruger judgments (DK judgments minus memory judgments). Each topographic map is range normalized according to their maximum and minimum values per latency. Warmer colors represent more positive-going voltage differences, with scales for each noted beneath each map. b) ERPs for memory and DK tasks at central parietal size Pz; x-axis is time in milliseconds, y-axis is μV. c) ERPs for memory and metacognition tasks at mid-frontal site Fz.

Are there differences in how DKE groups were making their memory judgements? We next investigated physiological differences in memory (“old-new” effects of hits - correct rejections) as a function of the different kinds of DKE groups (i.e. over-estimators and under-estimators). When looking early in time (400–600 ms) during the latencies characteristic of familiarity-based processing (FN400; Addante, Ranganath, Olichney, et al., 2012; Addante, Ranganath, & Yonelinas, 2012; Rugg & Curran, 2007), a 2 × 2 repeated-measures ANOVA with factors of condition (ERP amplitudes for hits, correct rejections) and group (over-estimators [N = 36], under-estimators [N = 10]) at mid-frontal site Fz revealed no significant effects of either factor or interaction. However, at the centro-parietal site Pz there was a significant main effect of Condition (F(1,46) = 5.63, p = .022) and a reliable condition × group interaction (F(1,44) = 5.0, p = .030). This was explored with a follow-up between-group t test, which found under-estimators had a significantly higher amplitude (M = 1.39, SD = 1.59) of old-new effects than over-estimators (M = 0.25, SD = 1.38) that occurred maximally at centro-parietal site Pz but was diffuse across several sites in the posterior scalp, t(44) = 2.24; p = .03. As this difference was evident in the parietal region instead of the expected left frontal region characteristic of the FN400, it may indicate an early activation of recollection activity but does not preclude other possible interpretations of its functional significance related to familiarity or implicit processing (Addante, 2015; Voss et al., 2010, 2012; Voss & Paller, 2007, 2017).

Later in time, from 600 to 900 ms at left parietal site P3, a 2 × 2 repeated-measures ANOVA with factors of condition (ERP amplitudes for hits, correct rejections) and group (over-estimators, under-estimators) revealed a main effect of Condition (F(1,46) = 7.36, p = .009) and a condition × group interaction (F(1,44) = 9.91, p = .007). This finding was qualified by a between-group t test, which revealed that the under-estimator group (M = 1.96, SD = 1.35) had significantly larger LPC effects (hit—correct rejection amplitudes) than the over-estimator group (M = 0.30, SD = 1.72), t(44)= 2.81; p = .01 (Figure 6). This finding was similar across adjacent electrodes and suggests that the under-estimator group, which consists of the highest performing individuals, relied on using more recollection than the over-estimator group did in making their memory judgements. Accordingly, as the illusory over-estimators constituted the lowest performing individuals, it is possible that one reason why they performed lower was because of a relative reduction in their reliance upon recollection.

FIGURE 6. Difference waves of recognition memory ERP effects for Dunning-Kruger Groups.

Top left: Topographic maps show group-difference waves for memory effects (hits minus correct rejections) for Dunning-Kruger groups of Over- and Under-Estimators at left parietal electrode P3, map is range normalized to maximum and minimum microvolts. Top right: ERPs of difference waves in memory effects (hits minus correction rejections) for each group (over- and under-estimators) at P3 from 600–900 ms; y-axis of zero represents no differences between memory conditions’ ERPs, and shaded areas depict standard error of the mean for each group. Warmer colors represent more positive-going voltage differences. Bottom left: individual raw amplitudes of the difference wave from 600–900ms, categorized by group. Bottom right: individual amplitudes of hits and correct rejections at P3 from 600–900ms for each subject, categorized by group.

How do metacognitive judgements differ among over- and under- estimators? To investigate this core question, we analysed group-level differences in ERPs between the over- and under-estimators by DKE judgement (all responses collapsed together). There were significant differences in ERP amplitude between the under-estimators and over-estimators at mid-frontal electrode Fz from 400 to 600 ms (M_{Over-Estimators} = 4.16, SD = 5.09; M_{Under-Estimators} = 0.55, SD = 4.40; t(44) = −2.04, p = .048) and adjacent frontal sites, such that ERPs for over-estimators were far more positive than those for the under-estimators (Figure 7). One suggestion from these results is that the frontal effect at 400–600 ms may be characteristic of the FN400 ERP effect related to familiarity-based processing, in that over-estimators may be relying on the less specific memory process of familiarity or intuitions of increased fluency to guide making their metacognitive judgements, instead of relying upon the more distinct recollection-related processes (Yonelinas et al., 2010) that evidently appears to be supporting the people who were under-estimating their performance relative to the group.

FIGURE 7. ERPs of Dunning-Kruger Estimates by Dunning-Kruger Groups.

Topographic maps show ERPs of collapsed Dunning-Kruger responses (Dunning-Kruger judgments 1, 2, 3, 4, and 5 combined) for Over-Estimators compared to Under-Estimators from 400–600 ms. Topographic map is range normalized to maximum and minimum values, warmer colors represent more positive-going voltage differences. Right: ERPs for Dunning-Kruger judgments of over-estimators and under-estimators at mid-frontal site of Fz.

2.8.1 |. Brain-Behaviour Relationships Between Memory and Metacognition

The preceding results prompted the question of whether there is a systematic relationship evident between memory and metacognition effects at the individual subject level. Across subjects in groups of both over-estimators and under-estimators, the magnitude of the LPC effect that occurred between 600 and 900 ms at P3 (hits-CR) was found to be significantly correlated with the proportion of hits made by subjects (r = .318, p = .031, N = 46) and also negatively correlated with response speeds for times when high confidence hits went on to receive correct source memory responses (r = −.305, p = .039, N = 46; Figure 8). For the under-estimator group, magnitude of the LPC effect was found to be correlated in the negative direction with the average Dunning-Kruger response given in-test by each subject (r = −.798, p = .006, N = 10) but did not exhibit any relationship for the over-estimators (r = −0.014, p > .10, N = 36) (Figure 8), indicating that for the higher-performing under-estimators, LPC effects would reliably predict what the ensuing average Dunning-Kruger estimate of that subject would be for their estimated task performance, such that the larger LPC magnitudes predicted relatively lower performance estimates.

FIGURE 8. Relationships of Behavioral and Brain Measures for Memory and Metacognition.

X-axis represents the magnitude of the LPC effect for both under-estimator and over-estimator groups combined (LPC measured as ERPs for hits minus correct rejections at left parietal site P3 from 600 – 900ms during item recognition memory test (top left, top right, bottom left panels). Bottom right panel x-axis represents the amplitude of mid-frontal ERPs for metacognitive judgments from 400–600ms during the in-test Dunning-Kruger performance estimate task, separated by group. Y-axis represents the proportion of successful item memory hits (judgments of 4 or 5 for ‘old’ status items during memory retrieval task) (top left); reaction times in milliseconds to recollection-related trials in which subjects got both an high item confidence hit and source memory judgment correct (top right); average in-test performance estimate given by subjects during the metacognitive DK judgments.

Overall, these findings converge to reveal that the larger LPC magnitudes were related to higher proportions of hits on the memory test and with faster response times for recollection-related items of high confidence hits with correct source memory (Addante, Ranganath, & Yonelinas, 2012; Addante et al., 2011; Roberts et al., 2018). Hence, the LPC was related to recollection, and the more recollection signal (LPC) a subject had predicted, the more likely they were to under-estimate their memory performance via their average DK responses in the metacognitive judgements (Figure 8). Recollection thus apparently led people to exhibit more humble metacognitive self-awareness. What is special about the under-estimators during their decisions to avoid the pitfalls of illusory superiority? One line of evidence we found was that in the under-estimator group, the magnitude of the ERPs for metacognitive judgements from 400 to 600 ms at mid-frontal site of Fz exhibited a significant positive correlation with the average in-test Dunning-Kruger response given by subjects (r = .774, p = .009, N = 10), though this relationship was not evident for the over-estimators (r = .025, p = .886, N = 36) who exhibited the larger FN400-like effects (Figures 7 and 8). This suggested that the relative lack of familiarity-based processing in the under-estimators appears to be governing them towards reporting a lesser estimation of their performance in the task.

3 |. DISCUSSION

The current study assessed multiple measures of DKE estimates interspersed throughout an ongoing episodic memory test while EEG was recorded. The results from behavioural measures first revealed that the memory paradigm was successful at eliciting the DKE. Participants were separated into performance quartiles, and their actual percentile ranking in the group was plotted alongside their estimated percentile ranking (Figure 2). The lowest performing participants in the bottom quartile were found to have substantially overestimated how highly they ranked in their groups, while the highest performing participants moderately under-estimated their actual ranking. This basic finding was important to identify as a starting point in a novel paradigm for studying the DKE in episodic memory, and its establishment permitted us to continue to explore the data in more specified ways for both behavioural and electrophysiological domains.

3.1 |. Behavioural Findings

The current study’s paradigm permitted meaningful collection of RTs for multiple Dunning-Kruger judgements that could be analysed at a group level, which prior studies of the DKE have not been able to investigate due to their onetime measures of metacognitive performance estimates at the completion of a study. We found over-estimators were discernably faster than under-estimators in judging themselves to be in the top percentile, but they were slower to judge themselves as being in the bottom percentile; accordingly, under-estimators were marginally slower to report being in the best performance and quicker to claim they were doing poorly. One possible account of these results comes from using Kruger and Dunning’s (1999) model of double ignorance by low performers. That account posits that low performers (a) do not know the answer and (b) do not know that their answer is incorrect. Accordingly, the better-performing under-estimators would be self-aware enough to take pause in responses claiming they are doing well because they know the ways in which they might have also failed (due to their competence), and likewise also would be guided by a more humble competence (knowing also what they may not know as well as what they could know) to be quicker in believing they were performing poorly. The correlation analysis of RTs with physiological signals (Figure 8) suggests that these judgements may be based upon recollection processing which have greater accuracy at the expense of longer processing times. Some general messages that emerge from this evidence of the classic speed/accuracy trade-off include “don’t rush” and “speed kills.”

3.2 |. Neurophysiological findings

We began exploring the neurophysiology of the DKE by examining brain activity for general differences in processing among the memory and metacognition tasks. ERPs between memory trials and trials of estimates for performance percentile were found to differ reliably beginning from approximately 300 ms into the epoch and continuing throughout the epoch to 1,200 ms at almost every electrode site but being maximal first at posterior parietal sites and then later at mid-frontal regions. This pattern of ERPs is consistent with established properties of P300 ERP effects (P3a and P3b effects) that are known to have the same distributions of topography and latency of across early/late and posterior/anterior regions, respectively, and which have been well-established as being associated with novelty processing or oddball tasks (Dien, Spencer, & Donchin, 2003; Otten & Donchin, 2000; Simons, Graham, Miles, & Chen, 2001). This is consistent with the current paradigm in that the DKE judgements were uncommon trials that appeared among the common memory trials in the test, and would have been salient stimuli for eliciting an orienting effect of attention as a novelty item (Kishiyama, Yonelinas, & Lazzara, 2004; Knight, 1996; Knight & Scabini, 1998).

We next explored whether the differential metacognitive judgements were associated with differential ERP patterns. When brain activity of all Dunning-Kruger responses was investigated together, over-estimators were found to have a higher ERP amplitude than under-estimators at frontal electrode sites during 400–600 ms, consistent with known properties of the FN400 effect of familiarity-based processing (Addante, Ranganath, & Yonelinas, 2012; for review see Rugg & Curran, 2007). These early ERP effects suggest that the errors of illusory superiority may be caused by an over-reliance on a generic sense of familiarity similar to what has been found in research on the false fame effect (Jacoby, Kelley, et al., 1989; Jacoby, Woloshyn, et al., 1989; Jacoby, Woloshyn, & Kelley, 2004), as opposed to the more specific recollecting of the clear details from their past encounters which would instead provide the contextual cues to guide proper placement of one’s perceptual judgements. Under-estimators (those who performed best), on the other hand, exhibited a larger LPC than over-estimators did from 600 to 900 ms during memory judgements, indicating that these under-estimators may be making their decisions by reliance upon the clearer details of recollected information, as opposed to the fuzzy sense of familiarity that can come with less accuracy (Yonelinas et al., 2002; Yonelinas et al., 2010).

Overall, the analyses of ERPs during the DKE judgement provided novel insight, revealing a mid-frontal FN400-like effect, suggesting that there is different processing for over- and under- estimators when making their percentile judgements. Over-estimators had a larger ERPs than under-estimators at mid-frontal sites from 400 to 600 ms, which is the characteristic position and latency of the FN400 that has been a putative neural correlate of familiarity in many prior studies (Addante, Ranganath, Olichney, et al., 2012; for Reviews see Curran, 2000; Friedman, 2013; Rugg & Curran, 2007; Mecklinger & Bader, 2020). Thus, each group was apparently arriving at fundamentally different metacognitive conclusions because they were relying upon, or being influenced by, different processes of memory, such as familiarity and/or fluency. This was mirrored in the behavioural data of RTs, which revealed a crossover interaction pattern of responding.

3.3 |. A Memory-based Framework for Metacognitive Judgements

In the introduction, we postulated that a memory-based framework could account for the illusory errors seen in the DKE, whereby these errors (and successes) can be guided at least in part based upon differences in the cognitive processes of familiarity and recollection. In such a model, familiarity would be seen as providing the foundational cognitive processing associated with a heuristic used by people unsure about the details of their past performance on the task and thus guiding them to erroneously over-estimate how well they think they did (over-guessing based upon it feeling familiar but lacking details; Voss & Paller, 2010; Whittlesea, 1993, 2002; Whittlesea, Jacoby, & Girard, 1990; Whittlesea & Leboe, 2000, 2003; Whittlesea & Williams, 2000). On the other hand, recollection would be seen as the cognitive process constraining people’s abilities to correctly retrieve their memories of the past experiences with richness, detail and contextual information bound together with the item of the episode (Diana et al., 2008; Eichenbaum et al., 2007). Thus, having the cognitive process of recollection available would guide people to make self-assessments of performance that are more conservatively constrained by the details of the facts of that prior experience, thereby avoiding the risk of incorrectly assuming an over-performance based merely on it seeming familiar acontextually.

Taken together with the behavioural findings in RTs, it appears that over-estimators were “quick to brag,” whereas the high performers were slow to judge themselves as being best and their caution was associated with better scores. Moreover, ERP data suggested that recollecting the past with clear context and details may be an important part to helping keep us humble, whereas relying upon mere feelings of familiarity may be what is leading us to over-estimating ourselves. Thus, what may guide better evaluative decisions is the slower process of recollecting the combination of items bound in context (Addante, Ranganath, & Yonelinas, 2012; Diana et al., 2008; Eichenbaum et al., 2007).

A related interpretation comes from research on familiarity, which has identified a contribution of guessing, or fluency, to familiarity judgements that are included in its decision heuristic (Voss, Lucas, & Paller, 2010; Whittlesea, 1993, 2002; Whittlesea et al., 1990; Whittlesea & Leboe, 2000, 2003; Whittlesea & Williams, 2000). This fluency may evidently be leading people to jump to the wrong conclusions about themselves relative to others, similar to what has been found in the false fame effect (Jacoby, Kelley, et al., 1989; Jacoby Woloshyn & Kelley, 2004). This interpretation is consistent with prior accounts of differences being due to people’s task competency (Adams & Adams, 1960; Ehrlinger & Dunning, 2003; Kruger & Dunning, 1999; Oskamp, 1965; Pennycook et al., 2017; Ryvkin et al., 2012; Sanchez & Dunning, 2017; Schlösser et al., 2013) if differences are understood as being due to how people encoded the initial mnemonic information. Those who did not encode information well would not be likely to recollect that information later (i.e. poor attention, motivation or distraction; Addante et al., 2015; Craik, Eftekhari, & Binns, 2018; Craik, Luo, & Sakuta, 2010; Craik, Naveh-Benjamin, Ishaik, & Anderson, 2000; Fernandes, Moscovitch, Ziegler, & Grady, 2005; Galli, Gebert, & Otten, 2013; Middlebrooks, Kerr, & Castel, 2017; Weeks & Hasher, 2017) nor would they later be able to accurately calibrate how well they were actually performing while using heuristics of familiarity and fluency (Mecklinger & Bader, 2020; Whittlesea, 1993, 2002; Whittlesea et al., 1990; Whittlesea & Leboe, 2000, 2003; Whittlesea & Williams, 2000).

Accordingly, analysis of RTs during encoding revealed that over-estimators responded faster than under-estimators did during encoding, which is consistent with a large body of prior findings on fluency (Alter & Oppenheimer, 2009; Bader & Mecklinger, 2017; Bruett & Leynes, 2015; Castel, McCabe, & Roediger, 2007; Cermak, Verfaellie, Sweeney, & Jacoby, 1992; Doss, Bluestone, & Gallo, 2016; Hertzog, Dunlosky, Robinson, & Kidder, 2003; Kurilla & Westerman, 2008; Leynes & Addante, 2016; Leynes & Zish, 2012; Li, Gao, Wang, & Guo, 2015; Nie, Xiao, Liu, Zhu, & Zhang, 2019; Serra & Dunlosky, 2005; Thapar & Westerman, 2009; Volz, Schooler, & von Cramon, 2010; Westerman, 2008; Whittlesea & Leboe, 2000, 2003). Thus, subjects could have responded more quickly to items by virtue of the items seeming more fluent. While these findings from encoding RTs appear to indicate that illusory over-estimators may have relied upon enhanced fluency to believe the information was more easily learned (hence believing they are performing better), it is also challenged by the finding of their having the same response time as the correct estimators did. Alternatively, it appears that under-estimators may have performed better due to having spent (slightly) more time learning information better, again supporting their later task competence. Future work will benefit from both empirical manipulations of fluency and physiological measures during encoding to better resolve such possibilities (e.g. Bader & Mecklinger, 2017; Bruett & Leynes, 2015; Leynes & Addante, 2016; Leynes & Zish, 2012).

3.4 |. Alternative Interpretations

Studies of decision-making have provided ERP evidence that P300 effect timing and slope are each associated with evidence accumulation in decision-making tasks (Boldt, Schiffer, Waszak, & Yeung, 2019; O’Connell, Dockree, & Kelly, 2012; Twomey, Murphy, Kelly, & O’Connell, 2015). One possibility for the current results of group differences in ERPs during the performance estimates is that they may reflect differential decision-making and evidence accrual among subjects (for a similar model, see Urai & Pfeffer, 2014). By this account, over-estimators may have relied upon insufficient evidence accrual to make their inaccurate decisions (consistent with the features of a familiarity-based signal detection process; Yonelinas et al., 2010, 2002), whereas the under-estimators may have been slower to believe they were doing best because of evidence accrual occurring more slowly for a slower-growing integration signal (Summerfield & Tickle, 2015; Twomey et al., 2015) (which is consistent with a threshold model of recollection; Parks & Yonelinas, 2009; Yonelinas et al., 2010, 2002; Yonelinas & Parks, 2007). The correlation results we found were consistent with this, in that larger P3 signal magnitudes for Dunning-Kruger decisions predicted higher performance estimates in the under-estimators, as they presumably had more accrued more evidence to support those better judgements (Boldt, 2019; O’Connell et al., 2012; Twomey et al., 2015).

However, there are also a few lines of evidence weighing against this, which suggest that the results may not reflect core differences among groups in decision-making processes, attention to the task or use of different strategies during the task. First, there were no differences across groups in their Dunning-Kruger response distributions, nor in their overall RTs to the Dunning-Kruger decision task, which would be predicted by such accounts. Second, there were no differences across groups in quantification of their use of any decision criterion shifts (C), nor sensitivity to response bias (B). Hence, while it is always possible there could be some other decision-making factor or differences that are driving the observed effects, none of the four direct measures of such indications revealed any evidence for it.

3.5 |. Limitations and Considerations for Future Research

While the current experiment provides several novel contributions to the understanding of the DKE, it also leaves room for future explorations to build upon. For instance, the current study followed standard convention in calculating Dunning-Kruger comparisons and categorized people based upon their estimates of their percentile relative to others, and future work would benefit from exploring potential differences when categorizing groups instead based upon the accuracy of people’s estimates of their own individual performance scores (not estimates of one’s percentile group). More specifically, in the current study, there were no differences evident for which percentile group people thought they were performing in only differences in how they actually performed on the memory task. This meant that some people’s estimates were correct, and most were incorrect, but everyone thought they were performing as “above average” compared to everyone else. Groupings for the current results were thus driven in part by memory task performance and in part by one’s ability to intuit or “know” how others perform, similar to the “false consensus effect” (Bauman & Gehar, 2002; Marks & Miller, 1987; Ross, Greene, & House, 1977; see also McIntosh, Fowler, Lyu, and Della Sala, 2019, for related findings on task performance).

Knowledge about others thus plays an important role in this comparative effect, and in the current study, we intentionally gave no guidance to participants about others. While we did collect post-test ratings of participants’ own self-estimates, those single trials did not differ from the average in-test ratings about group percentile and also cannot be used for analyses requiring many trials for obtaining effective signal-to-noise ratio in ERPs or reaction time measures. Given that the DKE has always traditionally been measured as people’s estimate of themselves compared to others (i.e. percentile estimates of group placement), the current investigation sought to preserve consistency with that extant literature; future work could certainly benefit from extending investigation into accuracy of one’s own performance score.

The current work also maintains inherent limitations of all initial explorations: findings remain to be assessed for generalizability, tested for its boundary conditions and independently investigated for replicability across other sample sizes and experimental variables. In particular, some of our behavioural findings required relying upon relatively small sample sizes (i.e. RTs of high and low estimates), and though the current paradigm has been previously found to be effective in prior studies using even smaller sample sizes of clinical patients (Addante, 2015; Addante, Ranganath, Olichney, et al., 2012), the current findings should serve as preliminary discoveries to motivate future work exploring larger group sizes. As noted earlier, such exploratory analyses, like all scientific findings, will benefit from corroboration by independent laboratories. It should be noted, though, that most people in the study exhibited over-estimating errors (the goal we sought to study), so the majority of our relatively large sample (N = 61) were defined as not being in the smaller group of under-estimators. Nevertheless, in exploring these effects we maintained rigorous controls of inclusion criteria for trials to gain effective signal-to-noise ratio (see Methods), and it is attested to by the reliable effects observed in the current work that small sample sizes inherently created a stronger challenge to achieve (which we overcame).

Additionally, while the electrophysiological results are compelling in suggesting memory effects contributing to the Dunning-Kruger phenomena, we should be cautious to avoid an over-reliance upon reverse inference (Paller, Lucas, & Voss, 2012; Poldrack, 2011) as other cognitive processes can also contribute to ERP effects, too, such as implicit fluency and conceptual priming (Voss & Paller, 2010a, 2010b; Voss, Lucas, & Paller, 2012; Leynes & Zish; Leynes & Addante, 2016; though see comments in Addante, Ranganath, & Yonelinas, 2012; Addante, Ranganath, Olichney, et al., 2012; Bader & Mecklinger, 2017; Bridger et al., 2012; Mecklinger et al., 2012). While future work would benefit from explorations in those directions, the current work is grounded in an extensive literature of established ERP findings (Addante, Ranganath, Olichney, et al., 2012; Addante, Ranganath, & Yonelinas, 2012; Rugg et al., 1998 for Review see Rugg & Curran, 2007) and we observed systematic relationships among behavioural and physiological correlates of the cognitive processes (Figure 8; Addante et al., 2011; Stiers, Falbo, Goulas, van Gog, & de Bruin, 2016; Macleod & Donaldson, 2017).

A final limitation to the current work is that the authors, too, may be inherently subject to the pervasiveness of the DKE’s biases and be over-estimating its value, misinterpreting results or unaware of counterfactual evidence. We hope that the current research can serve in providing value for motivating future research investigating these findings in more depth, extend them and test them against competing hypotheses.

4 |. SUMMARY AND CONCLUSIONS

In conclusion, the current study adds to the literature by a series of small steps: first, it represents the first physiological measures of the DKE, as well as reaction time measures of the phenomenon. Second, the study represents an integrative new paradigm that was developed to permit measuring multiple recurring trials of Dunning-Kruger metacognitive judgements, which others can now use to extend our understanding further. Third, this paradigmatic innovation made possible the ability to capture the DKE in a complex episodic memory task which extends the body of work on the DKE to episodic memory tasks of item and source memory confidence measures. Together, these innovations revealed convergent insight into why people differ in this phenomenon. We hope that it offers future researchers a pathway forward to continued exploring of this phenomenon.

This pernicious psychological phenomenon of over-estimating our abilities relative to others has been observed throughout both history and cultures by philosophers such as Socrates and Confucius (Socrates from Apology by Plato, 21d; Confucius, trans. 1938/500; Confucius, 1938), is cautioned against by ancient texts including Judeo, Christian, Polynesian and Islamic traditions (Proverbs 12:15; 1 Corinthians, 3:18; Qur’an 31:18), noted by laureates (Shakespeare, 1998) and scientists (Charles Darwin, 2009/1871) alike, and persists today throughout the modern age—including throughout university professors, provosts, deans and peer review (Cross, 1977; Huang, 2013; Bradley, 1981) while extending to leaders occupying both the highest and lowest offices. The basic premise of the DKE is thus seemingly a fundamental force that shapes our socio-psychological universe in similar ways that gravity shapes the backdrop of our physical universe: persisting through time and affecting everyone at some level. It takes work, with self-awareness, to avoid the pitfalls of illusory superiority and surely benefits from practice and informed feedback.

We show here that one way to do that is to avoid relying on intuition, fluency and familiarity to make quick judgements; instead, results encourage relying on recollection of details and slower responses to reduce errors of illusory superiority when comparing to others. More experimentation is needed, but the present work identifies some of the cognitive processes involved in the errors that can lead to the leadership and safety hazards of over- and under-belief in one’s abilities compared to others. We hope that this research can serve to inspire new explorations endeavouring to discover the neural correlates of our psychological processes, towards a better understanding of ourselves and the truth of human behaviour.

Abbreviations:

DKE

Dunning-Kruger effect

EEG

Electroencephalogram

ERP

Event-related potential