Establishing norms for error-related brain activity during the arrow Flanker task among young adults

Abstract

Psychological assessments typically rely on self-report and behavioral measures. Augmenting these with neurophysiological measures of the construct in question may increase the accuracy and predictive power of these assessments. Moreover, thinking about neurophysiological measures from an assessment perspective may facilitate under-utilized research approaches (e.g., brain-based recruitment of participants). However, the lack of normative data for most neurophysiological measures has prevented the comparison of individual responses to the general population, precluding these approaches. The current work examines the distributions of two event-related potentials (ERPs) commonly used in individual differences research: the error-related negativity (ERN) and error positivity (Pe). Across three lab sites, 800 unselected participants between the ages of 18 and 30 performed the arrow version of a Flanker task while EEG was recorded. Percentile scores and distributions for ERPs on error trials, correct trials, and the difference (ΔERN, ΔPe; error minus correct) at Fz, Cz and Pz are reported. The 25^th, 50^th, and 75^th percentile values for the ΔERN at Cz were −2.37 μV, −5.41 μV, and −8.65 μV, respectively. The same values for ΔPe at Cz were 7.51 μV, 11.18 μV, and 15.55 μV. Females displayed significantly larger ΔPe magnitudes and smaller ΔERN magnitudes than males. Additionally, normative data for behavioral performance (accuracy, post-error slowing, and reaction time) on the Flanker task is reported. Results provide a means by which ERN and Pe amplitudes of young adults elicited by the arrow Flanker task can be benchmarked, facilitating the classification of neural responses as ‘large,’ ‘medium,’ or ‘small’. The ability to classify responses in this manner is a necessary step towards expanded use of these measures in assessment and research settings. These norms may not apply to ERPs elicited by other tasks, and future work should establish similar norms using other tasks.

1. Introduction

Over the past several years, a number of initiatives such as the National Institute of Mental Health’s (NIMH) Research Domain Criteria (RDoC; Kozak & Cuthbert, 2016), the Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017), and the Addictions Neuroclinical Assessment (ANA; Kwako, Momenan, Litten, Koob, & Goldman, 2016) as well as a number of independent research laboratories (e.g. Cornet, 2015; Patrick, Brislin, & Sturmey, 2017) have called for the increased incorporation of neurophysiological measures into psychological assessment. The impetus behind this idea is that biological measures have the potential to increase the objectivity and precision of more traditional (e.g., questionnaire-based) assessments. Although several neurobiological measures have already shown promise in this regard (Iacono, Carlson, Malone, & McGue, 2002; Nelson et al., 2013; Patrick et al., 2006; Teitelbaum, Teitelbaum, Nye, Fryman, & Maurer, 1998), little progress has been made in using these measures to distinguish between participants in research or applied settings.

Barriers to using neurophysiological measures as assessment tools could include the cost or tolerability of procedures involved in obtaining the measure or mixed findings regarding the direction of effects as they relate to a particular attribute or trait. However, even for measures that are affordable and well-tolerated (e.g., electroencephalography, EEG) or have shown consistent findings, one major barrier to classifying an individual’s neurophysiological score has been the lack of normative data available for such measures. That is, when the goal is to interpret individual-level responses rather than compare group means, norms are necessary (e.g., intelligence quotient scores are only meaningful because they are referenced against the population). Without well-established empirical benchmarks, it is difficult to determine whether an individual’s response is ‘large,’ ‘small,’ or within the normal range. Furthermore, the lack of well-established norms for most neurobiological measures has impeded research progress. In the self-report domain (e.g., for trait or state anxiety), it is common for researchers to report their sample means as well as published norms, to assure readers that values obtained in their sample reflect those of the target population. However, this practice is not common in the neuroscientific literature, because norms are not available. That is, establishing norms for a measure of interest would allow for direct comparisons of sample distributions and the general population. Moreover, the establishment of norms might help with task optimization by facilitating understanding of the range and distribution of responses captured by a particular task. Further, while researchers often assume that their samples accurately represent a population of interest, establishing norms would allow for direct comparisons of sample distributions to a population distribution. Additionally, norm-referenced neurobiological responses could be used to recruit participants for research studies, allowing researchers to consider neurobiological function as an independent variable or inclusion/exclusion criterion. Although NIMH has advocated for this approach (Insel et al., 2010), which is prevalent in other areas of medicine (e.g., selecting participants with high blood pressure to participate in a study of cardiovascular disease), it is rarely used.

In comparison to other neurophysiological measures such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET), EEG is a noninvasive, portable, and relatively economical technique. Therefore, EEG measures are well-suited for widespread use and assessment in applied settings (e.g., a clinician’s office) as well as research recruitment settings that may require large numbers of potential participants to be screened. The error-related negativity (ERN) and error positivity (Pe) are event-related potentials (ERPs) involved in error processing and are excellent candidates for integration into behavioral and psychological assessment. The ERN is a frontocentrally maximal, negative-going ERP component, peaking 0–100 ms following commission of an error, and the Pe is a centro-parietally maximal, positive-going ERP evident 200–400 ms following response (Falkenstein et al., 1990; Gehring et al., 1993). Both the ERN and the Pe are larger for error versus correct responses; these same responses on correct trials are referred to as the correct-related negativity (CRN) and correct positivity (Pc), respectively.

The ERN has been proposed to signal the need for behavioral adjustment and increased executive control prior to conscious awareness of error commission (Dehaene, 2018; Holroyd and Coles, 2002; Lo, 2018; Nieuwenhuis et al., 2001; Van Veen and Carter, 2006). From a neuroanatomical perspective, individual differences in the ERN might reflect differential activation in the anterior cingulate cortex (ACC). For example, single-cell recordings in mice (Averbeck, 2017) and fMRI work in humans (Botvinick et al., 2004) have both linked the magnitude of the ERN to activity in the ACC. On the neurotransmitter level, the ERN is affected by dopamine-related genes (Biehl et al., 2011) and pharmacological manipulations of dopamine (De Bruijn et al., 2004). Meanwhile, there is substantial dopaminergic input to the ACC (Beckmann et al., 2009; Ko et al., 2009; Lumme et al., 2007); therefore, dopamine levels and the ACC may work together to drive individual differences in the ERN. The Pe, which is thought to measure the motivational significance and conscious processing of errors (Dehaene, 2018; Nieuwenhuis et al., 2001; Overbeek et al., 2005; Shalgi et al., 2009), might reflect similar but separate neurobiological mechanisms. Though fewer studies have assessed the neural origins of the Pe there is evidence to suggest that it too may originate from the ACC, although from more rostral areas (Herrmann et al., 2004; van Boxtel et al., 2005). The Pe might also be modulated by dopamine levels, although evidence suggests that the dopamine-related genes that regulate the Pe and ERN are distinct (Biehl et al., 2011).

Both components display good test-retest reliability (Larson et al., 2010; Weinberg and Hajcak, 2011) and internal consistency (Olvet and Hajcak, 2009a) across a variety of brief and easy-to-administer tasks (Meyer et al., 2013). Therefore, the ERN and Pe are well-suited for use in individual differences research and as potential neurophysiological assessment tools. Further, these measures have also been shown to increase the predictive power of assessment. For example, smaller ERNs have been found to indicate increased likelihood of relapse among cocaine users, controlling for the extent of drug use and method of administration (Marhe et al., 2013). In addition, the ERN has been shown to predict the onset of childhood anxiety disorders, above and beyond other risk factors such as current anxiety symptoms and parental psychopathology (Meyer et al., 2015), and may differentiate between anxiety and depression (Bress et al., 2015; Weinberg et al., 2012a). In patients with psychosis, smaller ERNs predict greater inexpressivity (affective flattening, alogia) 4 years later, controlling for current symptomology (Foti et al., 2016). Moreover, the ERN may hold promise as an endophenotype. For example, individuals at risk for substance abuse disorders exhibit smaller ERNs (Euser et al., 2013) and ERNs and CRNs in obsessive-compulsive disorder appear to be independent of symptom severity, suggesting that they may relate to trait rather than state variability (Riesel et al., 2015). Though less work has examined the Pe as a predictor of individual differences, the Pe appears to be related to variability in reward sensitivity (Boksem et al., 2006), and may predict likelihood of re-arrest among offenders with greater precision than traditional risk factors (e.g., age at release from prison; alcohol and drug use; Steele et al., 2015). In sum, the ERN and Pe may be well-suited to clinical, academic, and forensic assessments.

Beyond applied settings, the ability to compare individual or small sample data to a normative distribution may also be useful in research settings. Normative distributions allow researchers to examine whether a given sample is representative of the population (e.g., whether sample outliers are truly outliers with respect to the population at large). Further, prospective research designs have traditionally used self-report measures of symptomology and risk factors such as familial history to identify individuals who are at risk for clinical disorders or phenomena such as psychosis (e.g. Bernard et al., 2017; Brewer et al., 2005; Pantelis et al., 2003; Velakoulis et al., 2006; Yung et al., 2005), and treatment research has typically sought out participants who meet criteria for a particular disorder. While this research has been fruitful, it has been limited by assumptions regarding the validity of the current psychiatric diagnostic system as well as by a focus on the more downstream indicators of psychopathology (e.g., symptoms) rather than the underlying systems that may give rise to these outcomes. Since the advent of RDoC, researchers have been encouraged to consider psychopathology as stemming from variation or extreme scores in one or more underlying psychological/biological systems that span the continuum of functioning (from normal to abnormal). From this perspective, it should be possible to recruit participants according to their scores on neurophysiological variables that correspond to variation along dimensions of relevance to psychopathology and to examine their corresponding associations with other units of analysis (e.g., behavior, self-report), functional outcomes or other domains. This approach should increase understanding of both normative and abnormal functioning and should help close gaps between clinical profiles and underlying neurobiology. Indeed, such a recruitment approach has been explicitly advocated by the National Institute of Mental Health (Insel et al., 2010). Importantly, psychophysiological measures that are used for these purposes should be trait-like/stable. The magnitude of the ERN is reliable over time (Olvet and Hajcak, 2009a; Segalowitz et al., 2010; Weinberg and Hajcak, 2011), is associated with stable individual differences (Pailing and Segalowitz, 2003) and is heritable (Anokhin et al., 2008), suggesting its suitability in this regard (Weinberg et al., 2012b). However, to-date, the lack of normative data has prevented contextualization of individual response as ‘large’ or ‘small’, hindering such a recruitment approach.

As such, the current study sought to establish norms for the CRN, ERN, Pc, and Pe using an, unselected sample of 800 participants. This sample is comparable in size to studies that aim to establish norms for assessments in non-clinical young adult samples (e.g. Crawford et al., 2003; Crawford and Henry, 2003; Gilmore and Cuskelly, 2009; Henry et al., 2002; Pelham Jr. et al., 2005; Sinclair et al., 2012) including the WAIS-IV (Wechsler, 2008) and the STAI (Spielberger et al., 1983). The current sample was collected across three lab sites. Multi-site approaches to large-scale ERP analysis are becoming more common, and have been employed to meet the challenge of recruiting certain psychiatric samples (Hesselbrock et al., 2001; Seidman et al., 2006; Walker et al., 2007). Pooling data across multiple sites not only increases sample size, but also increases sample diversity, which is important particularly in the context of norm development. Furthermore, some forms of diversity across sites may be unrelated to samples. For example, there may be variability in data recording methods or experimental procedures, such as task length.

Here, we used a version of the Eriksen Flanker task (Eriksen and Eriksen, 1974) because it is the most widely used task for eliciting the ERN and Pe (e.g., Hall, Bernat, & Patrick, 2007; Herrmann, Römmler, Ehlis, Heidrich, & Fallgatter, 2004; Holroyd & Coles, 2002; Olvet & Hajcak, 2008) and because it appears to be more reliable than other tasks used to elicit these measures (Riesel et al., 2013). However, while the Flanker is commonly used to elicit the ERN and Pe, the number of trials varies widely (e.g., 399 trials, Suchan et al., 2018; 300 trials, Suzuki et al., 2017; 176 trials, Taylor et al., 2018). The current study included data elicited using tasks of three different lengths. This reflects in part the data available across the three labs for analyses (i.e., slightly different versions of the same task), but should have the added benefit of increasing the generalizability of results beyond a single task length.

To examine how characteristics of the ERPs might vary across their distributions, participants were grouped based on their percentile score for each response. Average amplitudes, grand average waveforms, scalp distributions, and LORETA source estimates are presented for each group. In addition to the ERPs, we report norms for accuracy and reaction time measures derived from the same task. The effects of task length, lab site, age, and gender on each ERP and behavioral measure were examined, and normative distributions stratified by these variables are reported as necessary. Additionally, the relationship between behavioral measures and ERP measures was examined, to establish whether variation in behavior might account for substantial variation in ERP magnitude. By identifying normative and extreme values for the CRN, ERN, ΔERN, Pc, Pe, ΔPe and related behavioral responses (accuracy, reaction time and post-error slowing), we provide reference points for future work with the aim of facilitating the incorporation of neurophysiological measures into psychological assessment.

2. Methods

2.1. Participants

Participants were 907 unselected individuals, recruited across three lab sites (Lab 1, Lab 2, Lab 3). We excluded participants from the final analyses if they committed fewer than 6 errors (n = 74) ), as previous work indicates a minimum of 6 errors is necessary for a reliable estimate of the ERN in a Flanker task (Meyer et al., 2013; Olvet and Hajcak, 2009b; Pontifex et al., 2010). We also excluded participants that had < 50% accuracy¹ (n = 11) or had excessive EEG artifacts (i.e., > 50% of trials; n = 22). The final sample consisted of 800 participants (overall 59.25% female; Lab 1, 58.33% female; Lab 2, 60.75% female; Lab 3, 56.86% female) with ages ranging from 18–30 years (overall M = 20.22, SD = 2.56; M_{Lab 1} = 19.28, SD_{Lab 1} = 1.53; M_{Lab 2} = 21.52, SD_{Lab 2} = 3.24; M_{Lab 3} = 19.97, SD_{Lab 3} = 2.27). Full demographic characteristics of the final sample are presented in Table 1.

Table 1.

Participant demographics.

Demographic Information	N (%) (Total N = 800)
Gender (%)
Male	326 (40.75%)
Female	474 (59.25%)
Race/Ethnicity (%)
White/Caucasian	461 (57.625%)
Black/African-American	17 (2.125%)
Hispanic/Latino	67 (8.375%)
Native American	2 (0.25%)
Asian	203 (25.375%)
Other/Multiracial	45 (5.625%)
Unknown/Did not answer	5 (0.625%)

2.2. Flanker Task

Participants completed an arrow version of the Eriksen Flanker task (Eriksen and Eriksen, 1974) used widely to elicit the ERN (Hall et al., 2007; Herrmann et al., 2004; Holroyd and Coles, 2002; Olvet and Hajcak, 2008). On each trial, participants viewed five white arrows presented for 200 ms against a black background; they were asked to respond as quickly and as accurately as possible to indicate the direction of the middle arrow by pressing the left or right mouse button. Participants had up to 2000 ms from the onset of the stimulus to respond. Half of the trials were congruent (< < < < < or > > > > >), whereas the other half were incongruent (e.g., < < > < < or > > < > >). Trial order was random for each participant and the intertrial interval varied from 1000 – 2000 ms, during which time participants viewed a white fixation cross centered on a black background. Participants received short breaks throughout the task. Across lab sites, some participants performed a longer version of the task (i.e., 300 or 330 trials) while other participants performed a shorter version of the task (150 trials). Lab 1 exclusively used a length of 330 trials (n = 168), Lab 2 used a mix of 150 trials (n = 255) and 330 trials (n = 173), and Lab 3 exclusively used 300 trials (n = 204). Task length was included in subsequent analyses, as detailed below.

2.3. EEG Recording and Data Reduction

All three labs used the ActiChamp amplifier system and ActiCaps (Brain Products, Gilching Germany) to record continuous EEG. Thirty-two electrode sites were used based on the 10/20 system. The electrooculogram (EOG) was recorded from four facial electrodes: two electrodes placed approximately 1 cm above and below the eye, forming a bipolar channel to measure vertical eye movement and blinks, and two electrodes placed approximately 1 cm beyond the outer edges of each eye, forming a bipolar channel to measure horizontal eye movements. One lab did not record horizontal EOG and instead used channel FT9 referenced against FT10 for subsequent horizontal ocular correction. EEG data was digitized at a 24-bit resolution with a sampling rate of 500 Hz or 1000 Hz; data that was originally sampled at 1000 Hz was later downsampled to 500 Hz in order to keep the sampling rate consistent throughout the dataset. For some participants, a 60-Hz low-pass filter was used at acquisition. For the remaining participants, data was not filtered during acquisition.

All data were processed together at Texas A&M University using BrainVision Analyzer 2 (Brain Products GmbH, Gilching, Germany). Data were segmented for each trial beginning 400ms before participant response and lasting until 800ms following participant response. The signal from each electrode was referenced offline to the average of the left and right mastoids (TP9/10) and band-pass filtered with high-pass and low-pass filters of 0.1 and 30 Hz, respectively. Eye blink and ocular corrections used the method developed by Miller, Gratton and Yee (1988). Artifact analysis was used to identify a voltage step of more than 50.0 μV between sample points, a voltage difference of 300.0 μV within a trial, and a maximum voltage difference of less than 0.50 μV within 100-ms intervals. Trials were also inspected visually for remaining artifacts, and data from individual channels containing artifacts were rejected on a trial-to-trial basis. Following artifact rejection, the number of correct trials (M_Fz = 228.30, Min_Fz = 80, Max_Fz = 324; M_Cz = 229.40, Min_Cz = 90, Max_Cz = 324; M_Pz = 228.50, Min_Pz = 77, Max_Pz = 324) and error trials (M_Fz = 28.09, Min_Fz = 6, Max_Fz = 142; M_Cz = 28.30, Min_Cz = 6, Max_Cz = 142; M_Pz = 28.23, Min_Pz = 6, Max_Pz = 142) at each electrode of interest was examined. Correct and error trials were averaged separately and baseline correction was performed using a 200 ms window from −400 to −200 ms before behavioral response. We chose this period for baseline correction, rather than a baseline closer to response, as baselines more distal from responses allow for a more reliable ERN measure with fewer error trials (Sandre et al., in prep) and to avoid subtracting out portions of the ERN that might occur just prior to response.

The ERN and CRN were scored as the average activity on error and correct trials, respectively, from 0 to 100 ms after response; ΔERN was calculated as the difference between the ERN and CRN (ERN minus CRN). The Pe and Pc were scored as the average activity on error and correct trials, respectively from 200 to 400 ms after response; ΔPe was calculated as the difference between the Pe and Pc (Pe minus Pc). Mean area amplitudes rather than peak-related measures were used as mean area amplitudes are less affected by signal-to-noise ratios, trial-level changes in component latency, task length and overlapping ERP components, and are more consistent across electrode sites (Clayson et al., 2013; Luck, 2018, 2005). Although we computed ERPs at three electrode sites (Fz, Cz and Pz), we focus demographic analyses and presentation of normative data on electrode Cz, because this is where ΔERN and ΔPe were maximal (Boksem et al., 2006; Foti et al., 2012; Herrmann et al., 2004; Ladouceur et al., 2006; Schroder et al., 2013; Segalowitz et al., 2010). We chose to focus on the sites of maximal amplitude for error minus correct difference measures (Cz), rather than the raw ERN or Pe, as the difference measures should be less affected by, among other things, proximity to reference, cerebrospinal fluid density and skull thickness (Chauveau et al., 2004; Luck, 2005; Pfefferbaum, 1990) and are commonly employed in the broader error-processing literature. However, as a comparison point and for researchers accustomed to using somewhat different scoring locations for these components, we also present means and percentile cutpoints for the components at electrodes Fz, Cz and Pz (Tables 2 and 3).

Table 2.

Amplitudes (μV) corresponding to percentile cut-points and means (standard deviation) for ΔERN, ERN, and CRN at Fz, Cz and Pz.

	%ile	Fz	Cz	Pz
ΔERN	90th	−10.73	−11.76	−7.76
	75th	−7.99	−8.65	−5.14
	50th	−4.67	−5.41	−2.51
	25th	−1.93	−2.37	0.24
	10th	0.51	0.68	2.86
	M (SD)	−5.03 (4.43)	−5.62 (4.98)	−2.51 (4.25)
ERN	90th	−8.87	−5.28	0.59
	75th	−4.95	−0.90	3.77
	50th	−1.29	3.39	7.05
	25th	1.99	7.19	10.59
	10th	4.83	11.03	14.39
	M (SD)	−1.58 (5.53)	3.27 (6.56)	7.19 (5.37)
CRN	90th	−2.03	2.58	3.99
	75th	0.41	5.53	6.45
	50th	3.33	8.83	9.46
	25th	6.18	12.40	12.86
	10th	9.57	15.88	16.06
	M (SD)	3.52 (4.43)	9.02 (5.29)	9.69 (4.66)

Note: ΔERN = ERN difference (error minus correct); %ile = percentile.

Table 3.

Amplitudes (μV) corresponding to percentile cut-points and means (standard deviation) for ΔPe, Pe, and Pc at Fz, Cz and Pz.

	%ile	Fz	Cz	Pz
ΔPe	90th	13.76	20.28	19.38
	75th	9.79	15.55	15.23
	50th	6.03	11.18	11.01
	25th	2.48	7.51	7.35
	10th	−0.83	4.15	4.57
	M (SD)	6.16 (6.02)	11.72 (6.31)	11.53 (5.86)
Pe	90th	17.69	24.29	22.49
	75th	11.20	18.85	17.20
	50th	6.53	13.70	12.71
	25th	2.12	9.41	8.84
	10th	−1.38	5.41	5.07
	M (SD)	7.23 (7.65)	14.30 (7.51)	13.27 (6.58)
Pc	90th	6.61	8.28	6.79
	75th	2.94	5.23	3.84
	50th	0.45	2.36	1.55
	25th	−2.27	−0.13	−0.67
	10th	−4.14	−2.88	−2.91
	M (SD)	0.76 (4.48)	2.52 (4.49)	1.65 (3.85)

Note: ΔPe = Pe difference (error minus correct); %ile = percentile.

2.4. Behavior

Accuracy was calculated as the percentage of correct responses. Reaction time (RT) was calculated as time between stimulus onset and participant response. RTs for correct responses (correct RT) and errors (error RT) were computed separately and ΔRT was calculated as the difference between correct RT and error RT (correct RT – error RT). Post-error slowing (PES) was calculated as the mean difference between response time on trials following an error and response time on trials immediately preceding errors. Subjects with ΔRT (N = 10) or PES (N = 14) greater than 3 standard deviations from the mean were removed from analyses involving ΔRT and PES, respectively. No outliers were identified for accuracy.

2.5. Data Analyses

Participants with ERN, CRN, ΔERN, Pc, Pe and ΔPe values greater than 3 standard deviations from the mean were identified at each electrode and eliminated from subsequent analyses involving that component and electrode. Distributions of each ERP were then examined for normality. Traditional tests of normality, such as the Shapiro-Wilk test, are generally considered overpowered for large samples such as the present sample, causing relatively inconsequential deviations from normality to be considered significant (Ghasemi and Zahediasl, 2012). As such, density plots and Q-Q plots for each ERP measure were visually examined to determine normality. Visual inspections of the plots determined that distributions of the ERPs across the sample and when examined separately within males and females were relatively normal². Further analyses examining ERP measures focused on amplitudes at Cz, where ΔERN and ΔPe were maximal.

The overall distributions of RT and accuracy were also examined for normality, with correct and error RT examined individually. Correct RT and error RT distributions were positively skewed, with most participants’ RTs being on the lower end of the distribution. Accuracy was negatively skewed, with most participants performing quite well. Log transformations of RT measures resulted in adequately normal distributions³. However, a log transformation of accuracy did not normalize the distribution; therefore, nonparametric tests were used for analyses. The distribution of ΔRT (correct RT – error RT) was examined and appeared normal, as did the distribution of PES.

The internal consistency of ERPs (ERN, CRN, ΔERN, Pe, Pc, and ΔPe) and behavioral measures (accuracy, correct RT, error RT, PES, and ΔRT) was assessed by calculating separate averages for even and odd trials. Pearson correlation coefficients were computed between these averages, and these coefficients were corrected using the Spearman-Brown formula.

To examine the effect of task length on ERPs, we examined only the first 150 trials for each participant and compared distributions to those obtained using all available trials for each participant. These analyses included slightly fewer participants (N = 724) than were included in the overall dataset, because some participants failed to make 6 errors within first 150 trials. Within this truncated dataset, the effect of lab site, which was partially confounded with task length, was also examined using one-way ANOVA tests.

The effect of gender on ERP measures was assessed using independent t-tests. The influence of gender on correct and error RT was examined using independent t-tests in which RT was log transformed. Independent t-tests were also used to examine the relationships between gender and ΔRT and gender and PES. The relationship between gender and accuracy was examined using a Wilcoxon sum-rank test due to the significant skew in accuracy. In the event that gender had a significant effect on ERP magnitude, the effect was examined again controlling for all behavioral measures. Because of the skew in many behavioral measures, a rank ANCOVA was used (Olejnik and Algina, 2008) where each behavioral measure rank-transformed, then included in the ANCOVA as a covariate.

Because the age of the sample was heavily skewed toward the younger end of the 18–30 cutoff range (M = 20.22, Med. = 19), the relationships between age and ERPs as well as age and all behavioral measures were assessed using Kendall (nonparametric) correlations. If age did significantly predict ERP magnitude, the effect was examined again while controlling for all behavioral measures. To control for the effects of behavior when assessing the influence of age on ERP measures, nonparametric partial correlations were used where each behavioral variable and age were IVs and ERP magnitude was the DV, then the partial correlation between age and ERP was assessed.

The relationships between behavioral measures and ERP measures were also examined. Due to the skew in accuracy, relationships between accuracy and ERP magnitudes were examined using Kendall (nonparametric) correlations. Relationships between raw RT measures (error RT and correct RT) and raw ERP measures (ERN, CRN, Pe and Pc) were examined using log-transformed RT measures and parametric correlations. Parametric correlations were also used to examine the relationships between difference measures (ΔERN, ΔPe, ΔRT and PES). In tables throughout the manuscript the 25^th, 50^th, and 75^th percentiles are used to present the quartiles of each distribution; in addition, we present the 10^th and 90^th percentiles to illustrate values corresponding to extreme scores. For each percentile group, source estimates for ΔERN and ΔPe were derived from LORETA source estimates were derived using BrainVision Analyzer (Brain Products, 2014) using 50 ms windows around peak error activity (20–70 ms for the ERN, 200–250 ms for the Pe). Of interest was whether the source estimates for the ERN and Pe were qualitatively similar at these different percentile groups, indicating that they represent different degrees of activation within the same neural network. All statistical analyses were performed using R version 3.5.1 (R Core Team, 2016).

3. Results

Figure 1 depicts grand average waveforms for error and correct trials and the spatial distribution of voltage differences for error minus correct trials, in the time-windows of the ERN and Pe, shown across all participants. Tables 2 and 3 present percentile cutpoints for ERN, CRN, ΔERN, Pe, Pc, and ΔPe at Fz, Cz and Pz. Table 4 presents percentile group cutpoints, means and standard deviations for accuracy, correct RT, error RT, ΔRT, and PES.

Figure 1.

Grand average waveforms at Cz for all participants and scalp distributions of the voltage differences for error minus correct trials during the time-windows in which the ERN/CRN (0–100ms after response, left) and the Pe/Pc were scored (200–400 ms after response, right).

Table 4.

Percentile cut-points and means (standard deviation) for behavior.

%ile	Accuracy (%)	ΔRT (ms)	Error RT (ms)	Correct RT (ms)	PES (ms)
90th	95.21	118.93	271.30	334.67	77.67
75th	92.57	94.96	283.65	355.17	54.49
50th	89.09	73.77	301.36	378.38	32.21
25th	85.91	56.13	319.87	407.75	16.31
10th	82.67	41.86	350.54	442.13	0.80
M (SD)	88.82 (5.56)	76.90 (32.06)	307.67 (39.85)	384.94 (44.27)	36.40 (31.63)

Note: ΔRT = reaction time difference (correct minus error); %ile = percentile; PES = post-error slowing

3.1. ERP distributions by percentile group

Figures 2 and 3 depict distributions of each ERP measure by percentile group at Cz, where ΔERN and ΔPe were maximal. To examine how variation in each raw ERP measure (ERN, CRN, Pe and Pc) was related to variation in difference measures (ΔERN and ΔPe), subjects were grouped based on the magnitude of their ΔERN and ΔPe. Raw ERP measures are presented within these groups in Table 5, which reports mean and standard deviation amplitudes for the ERN, CRN, Pe and Pc, shown separately for percentile groups defined by ΔERN and ΔPe amplitudes⁴. Compared to the CRN and Pc, the ERN and Pe varied more consistently with ΔERN and ΔPe across percentile groups, suggesting that variation in ΔERN and ΔPe may be driven primarily by error rather than correct trials⁵. LORETA source estimates were also derived for ΔERN and ΔPe. Across percentile groups, the ΔERN consistently yielded source estimates from the medial prefrontal gyrus, anterior cingulate, and cingulate gyrus, while ΔPe received additional contributions from the posterior cingulate, cuneus and precuneus. Figures 4 and 5 present grand average waveforms at electrode Cz, as well as scalp topographies for error minus correct trials and LORETA source estimates, shown separately for percentile groups defined by ΔERN (Figure 4) and ΔPe (Figure 5) magnitudes.

Figure 2.

Distribution of mean ΔERN, ERN and CRN amplitudes by ΔERN percentile group. The y-axis displays participant count and the x-axis displays ERP amplitude; percentile groups are denoted in the legend. The mean for each distribution is denoted by a vertical dashed line.

Figure 3.

Distribution of mean ΔPe, Pe and Pc amplitudes by ΔPe percentile group. The y-axis displays participant count and the x-axis displays ERP amplitude; percentile groups are denoted in the legend. The mean for each distribution is denoted by a vertical dashed line.

Table 5.

Mean ERP amplitudes (SD) by percentile group.

%ile group	ΔERN (μV)	ERN (μV)	CRN (μV)	ΔPe (μV)	Pe (μV)	Pc (μV)
90th–100th	−14.73 (2.59)	−4.36 (5.67)	10.37 (5.50)	23.93 (3.08)	25.40 (5.12)	1.47 (4.84)
75th–90th	−10.09 (0.88)	−0.16 (5.52)	9.93 (5.44)	17.64 (1.35)	20.27 (5.28)	2.63 (5.20)
50th–75th	−7.11 (0.95)	2.58 (5.54)	9.70 (5.53)	13.20 (1.28)	15.48 (4.41)	2.75 (4.04)
25th–50th	−3.94 (0.85)	4.91 (4.60)	8.85 (4.62)	9.38 (1.07)	12.23 (4.19)	2.84 (4.14)
10th–25th	−1.04 (0.89)	6.81 (5.51)	7.85 (5.49)	5.93 (0.96)	8.58 (4.88)	2.65 (4.56)
0–10th	2.89 (2.18)	9.91 (6.45)	7.02 (6.06)	1.67 (2.03)	4.82 (6.22)	3.15 (5.56)

Note: Participants were assigned to percentile groups based on ΔERN and ΔPe at Cz, respectively. ΔERN = ERN difference (error minus correct); ΔPe = Pe difference (error minus correct); %ile = percentile.

Figure 4.

Grand average waveforms at Cz, scalp distributions of the voltage differences for error minus correct trials from 0–100 ms after response, and source localization estimates for ΔERN, shown separately for each percentile group. Groups are ordered from largest to smallest ΔERN from the top left to the bottom right.

Figure 5.

Grand average waveforms at Cz, scalp distributions of the voltage differences for error minus correct trials from 200–400 ms after response, and source localization estimates for ΔPe, shown separately for each ΔPe percentile group. Groups are ordered from largest to smallest ΔPe from the top left to the bottom right.

3.2. Reliability

Internal consistency was moderate-to-high for both ΔERN (r = .76) and ΔPe (r = .82; ps < .0001). Likewise, ERN (r = .87), CRN (r = .98), Pe (r = .88) and Pc (r = .97) all displayed high internal consistency (ps < .0001). In general, ERPs on correct trials showed the highest internal consistency, most likely due to the larger number of correct compared to error trials.

Accuracy was highly internally consistent (r = .90), as was correct RT (r = .98; ps < .001). Error RT (r = .87) and ΔRT (r = .81) displayed moderate-to-high internal consistency (ps < .001). As was the case for ERPs, behavioral measures with more trials (accuracy and correct RT) likely displayed greater internal consistency due to the larger number of correct compared to error trials. PES displayed poor internal consistency (r = .33, p < .001).

3.3. Task length and lab site

Table 6 displays means, standard deviations and percentile cutoff scores for each ERP measure at Cz, shown separately using the full dataset versus only the first 150 trials. Overall, there appeared to be little difference between ERP distributions derived using the full dataset and those obtained using only the first 150 trials. This was also the case for ERPs measured from Fz and Pz; distributions from all trials at these electrodes compared to the distributions from only the first 150 trials are presented in supplementary materials.

Table 6.

Amplitudes (μV) corresponding to percentile cut-points and means (standard deviation) for ΔERN, ERN, CRN, ΔPe, Pe, and Pc at Cz from the full dataset compared to the first 150 trials only.

	ΔERN		ERN		CRN		ΔPe		Pe		Pc
%ile	All Trials	First 150	All Trials	First 150	All Trials	First 150	All Trials	First 150	All Trials	First 150	All Trials	First 150
90th	−11.76	−12.42	−5.28	−5.57	2.58	2.36	20.28	20.86	24.29	24.88	8.28	7.67
75th	−8.65	−9.24	−0.90	−1.24	5.53	5.37	15.55	16.03	18.85	18.69	5.23	4.85
50th	−5.41	−5.62	3.39	3.13	8.83	8.60	11.18	11.90	13.70	13.68	2.36	1.76
25th	−2.37	−2.24	7.19	7.12	12.40	12.07	7.51	7.71	9.41	9.46	−0.13	−0.67
10th	0.68	1.50	11.03	11.67	15.88	15.85	4.15	4.11	5.41	4.66	−2.88	−3.48
M (SD)	−5.62 (4.98)	−5.73 (5.48)	3.27 (6.56)	3.04 (6.72)	9.02 (5.29)	8.82 (5.21)	11.73 (6.31)	12.15 (6.65)	14.30 (7.51)	3.04 (6.72)	2.52 (4.49)	8.82 (5.21)

Note: ΔERN = ERN difference (error minus correct); ΔPe = Pe difference (error minus correct); %ile = percentile.

ERN, CRN, Pe, Pc, and ΔPe magnitudes did not significantly differ across lab site (ps > .09). ΔERN did significantly differ across lab site, F(2,714) = 5.67, p < .01. Tukey’s HSD post-hoc tests revealed that on average, Lab 1 (M = −4.39 μV) yielded smaller ΔERN magnitudes than the other two (M_{Lab 2} = −5.98 μV; M_{Lab 3} = −6.27 μV; ps < .01); the size of this effect was small (η²_p = .016). When behavioral measures, age and gender were controlled for, the effect of lab site was marginally significant (F = 3.05, p = .048) and the effect size was negligible (η²_p = .0017).

3.4. Gender

Table 7 presents percentile cutpoints, means, and standard deviations for each ERP measure at Cz, shown separately for males and females (this information for ERP measures at Fz and Pz can be found in supplementary materials). ERN magnitude did not significantly differ by gender (p = .74). However, CRN magnitude did differ by gender (t(735.54) = 4.30, p < .001, d = 0.31). The CRN was more negative in females (M = 8.37 μV, SD = 5.43) than in males (M = 9.97 μV, SD = 4.95). The effect of gender on CRN remained significant after controlling for all behavioral measures (F(1, 789) = 6.69, p = .01). Additionally, there was a significant effect of gender on ΔERN (t(672.67) = 5.30, p < .001, d = 0.39), such that males exhibited larger ΔERNs (M = −6.75 μV; SD = 4.98) compared to females (M = −4.86 μV, SD = 4.84). The effect of gender on ΔERN remained significant after controlling for all behavioral measures (F(1, 760) = 18.45, p < .001). Gender did not influence Pe magnitude (p = .32). However, Pc magnitude differed by gender (t(743.54) = 5.31, p < .001, d = 0.37), such that males displayed more positive Pc responses (M = 3.50 μV, SD = 4.20) than females (M = 1.85 μV, SD = 4.67). The effect of gender on Pc remained significant after controlling for all behavioral measures (F(1, 764) = 18.71, p < .001). Finally, there was a small but significant influence of gender on ΔPe (t(729.09) = 2.29, p = .02, d = 0.16). Females had larger ΔPes (M = 12.15 μV, SD = 6.51) than males (M = 11.13 μV, SD = 5.96). However, the effect of gender on ΔPe was no longer significant after controlling for all behavioral measures (F(1, 759) = 3.13, p = .08).

Table 7.

Amplitudes (μV) corresponding to percentile cut-points and means (standard deviation) for ΔERN, ERN, CRN, ΔPe, Pe, Pc at Cz by gender.

	ΔERN		ERN		CRN		ΔPe		Pe		Pc
%ile	Males	Females	Males	Females	Males	Females	Males	Females	Males	Females	Males	Females
90th	−12.96	−11.02	−5.92	−4.85	3.78	1.83	19.13	20.58	24.25	24.30	9.26	7.33
75th	−9.51	−7.96	−0.70	−0.98	6.64	4.85	14.88	16.05	19.18	18.75	5.92	4.80
50th	−6.74	−4.78	3.39	3.40	9.61	8.02	10.28	11.55	13.64	13.74	2.92	1.77
25th	−3.48	−1.68	7.35	7.16	13.39	11.72	7.08	7.62	10.21	9.04	0.81	−1.13
10th	−0.63	1.25	10.66	11.28	16.34	15.40	3.92	4.22	6.57	4.57	−1.22	−3.92
M (SD)	−6.75 (4.98)	−4.86 (4.84)	3.18 (6.50)	3.34 (6.50)	9.97 (4.95)	8.37 (5.43)	10.41 (5.30)	12.15 (6.51)	14.65 (7.20)	14.06 (7.72)	3.50 (4.12)	1.85 (4.61)

Note: ΔERN = ERN difference (error minus correct); ΔPe = Pe difference (error minus correct); %ile = percentile

Table 8 presents percentile cutpoints, means and standard deviations for behavioral measures, shown separately for males and females. Females were more accurate than males, W = 707174, p = .04, however the effect size was negligible (M_Male = 88.41%, SD_Male = 5.19; M_Female = 89.10%, SD_Male = 5.33; d = 0.13). There was also a significant effect of gender on correct RT (t(734.62) = 4.69, p < .0001), with males (M = 376.13 ms, SD = 40.61) responding faster than females (M = 391.00 ms, SD = 45.68; d = 0.34). There was also a trend for males to respond faster than females on error trials, although the effect size was negligible (p = .05; d = 0.14). ΔRT also differed for males and females (t(722) = 2.96, p < .01), such that males showed less of a difference between correct and error RT (M = 72.89 ms, SD = 32.06) than females (M = 79.68 ms, SD = 33.10; d = 0.21), most likely due to gender differences in correct RT. Finally, PES differed by gender (t(750) = 3.04, p < .01), such that females slowed more following errors (M = 39.18 ms, SD = 33.28) than males (M = 32.42 ms, SD = 28.67; d = 0.21).

Table 8.

Percentile cut-points and means (standard deviation) for behavior by gender.

	Accuracy (%)		ΔRT (ms)		Correct RT (ms)		Error RT (ms)		PES (ms)
%ile	Male	Female	Male	Female	Male	Female	Male	Female	Male	Female
90th	94.31	95.42	109.25	123.55	331.42	330.01	269.65	271.68	67.64	84.30
75th	92.00	93.03	87.32	98.92	350.12	350.80	281.57	285.27	47.42	58.91
50th	88.73	89.13	70.36	75.80	371.17	377.78	299.00	302.77	29.94	34.51
25th	85.67	86.06	53.69	58.85	396.23	408.66	317.76	321.94	15.27	16.99
10th	82.05	83.14	39.30	43.00	424.49	438.43	343.19	353.29	−1.60	1.91
M (SD)	88.41 (5.19)	89.10 (5.33)	72.89 (30.09)	79.68 (33.10)	376.13 (40.61)	391.00 (45.68)	304.41 (36.21)	309.91 (42.06)	32.42 (28.67)	39.18 (33.28)

Note: ΔRT = RT difference (correct minus error); %ile = percentile; PES = post-error slowing.

3.5. Age

Age was not significantly related to CRN (τ = −.03, p = .23) or ERN (τ = −.02, p = .49). Age was significantly related to ΔERN such that older participants tended to display larger ΔERN amplitudes, however the relationship was weak (τ = −.07, p < .01). The effect of age on ΔERN remained significant after controlling for all behavioral measures (τ = −.07, p < .01). Age was also significantly related to Pe (τ = −.08, p < .01), Pc (τ = .06, p = .02), and ΔPe (τ = −.13, p < .001). These effects remained significant after controlling for all behavioral measures (τ_Pe = −.07, p < .01; τ_Pc = .06, p = .01; τ_ΔPe = −.13, p < .001). As participant age increased, the size of the Pe and ΔPe decreased while the size of the Pc increased. However, the magnitude of these relationships was small.

Age did not significantly influence accuracy (τ = 0.00, p = .97), correct RT (τ = −.01, p = .57), error RT (τ = .04, p = .11), or PES (τ = −.02, p = .40). Age did significantly influence ΔRT (τ = −.06, p = .02), such that older participants showed a smaller difference between error and correct RTs, however the relationship was very weak.

ERPs and behavior ⁶

Accuracy was weakly but significantly related to CRN magnitude (τ = −.08, p < .01) and ERN magnitude (τ = −.08, p < .01), such that participants with better performance (greater accuracy) showed larger ERNs and CRNs. Accuracy was not significantly related to ΔERN (τ = −.02, p = .41), but was related to Pc (τ = −.08, p < .001), Pe (τ = .11, p < .001) and ΔPe (τ = .28, p < .001). As accuracy increased, Pc decreased while Pe and ΔPe increased.

Correct RT was significantly related to both CRN (r = −.35, p < .001) and Pc (r = −.24, p < .001). Participants with faster RTs on correct trials had smaller Pc amplitudes and larger CRNs. Error RT was not related to ERN (r = 0.00, p = .97), but was significantly related to Pe (r = −.12, p < .001). Participants who responded more quickly on error trials had smaller Pes; however, this relationship was weak.

ΔRT was significantly correlated with ΔERN (r = .18, p < .001) and ΔPe (r = .09, p = .01). Participants with larger ΔRTs (correct minus error) had smaller ΔERNs and larger ΔPes. PES was significantly correlated with ΔERN (r = .12, p < .001) and CRN (r = −.18, p < .001). Participants who slowed more after error trials had smaller ΔERNs and larger CRNs. However, PES was not significantly correlated with ERN (r = −.04, p = .21). PES was not related to ΔPe (r = .04, p = .23) or Pe (r = −.04, p = .25) but was related to Pc (r = .11, p < .01) such that participants who slowed more after errors had larger Pc amplitudes.

4. Discussion

The current study reports norms for the ERN, Pe and behavioral responses in a large sample of college-aged individuals, elicited using the Eriksen Flanker task. The CRN, ΔERN, and Pc significantly differed for males and females, as did many behavioral measures on the Flanker. As such, gender-stratified distributions for ERPs and Flanker performance are presented. Additionally, significant, but weak, relationships between age and ERPs were present, while age was not related to most behavioral measures. Examination of raw ERPs across percentile groups indicated that variation in ΔERN and ΔPe were related more to variation in error-related than correct-related ERPs. Furthermore, source analyses of ΔERN and ΔPe yielded consistent estimates across percentile groups, suggesting that variation across individuals was associated with differing magnitudes of activation in similar neural networks. We present these data in order to facilitate the comparison of individual responses and smaller samples with the overall distribution of the population, which has utility for both research and applied settings.

4.1. ERPs

Many prior studies have linked the ERN and Pe to individual differences in behavior, self-reported traits or symptoms (Boksem et al., 2006; Hajcak and Foti, 2008; Hirsh and Inzlicht, 2010; Pailing and Segalowitz, 2003; Weinberg et al., 2012b). Moreover, several studies have found that the ERN and/or Pe predict outcomes such as the likelihood of re-arrest (Steele et al., 2015) or the onset of psychiatric disorders (Meyer et al., 2015), even when controlling for more traditional predictors (e.g. Foti et al., 2016; Marhe et al., 2013). Large-scale normative data provided here should help in the interpretation of inter-individual variability in these promising biomarkers.

An examination of prior work suggests that mean values for these ERP components tend to vary considerably among smaller samples (Bartholow et al., 2005; Horowitz-Kraus, 2011; Jackson et al., 2015; Olvet et al., 2010; Steele et al., 2016; Weinberg et al., 2010); therefore, larger samples such as that employed in the current study are needed to approximate true population values. There are a number of variables that may be responsible for differences observed across prior work, some of which were analyzed here (e.g., effects of gender and age). In addition, the influence of methodological choices such as different recording systems, the time windows chosen for ERP scoring and ocular correction procedures on ERP amplitudes is largely unknown (Luck, 2005), and may contribute to differences in the ERN and Pe observed across samples.

Indeed, there is a great deal of diversity in ERP scoring methods. We chose to score error-related ERPs using mean area amplitudes as this method is relatively robust to suboptimal signal-to-noise ratios (Clayson et al., 2013) and overlapping ERP components (Luck, 2005), as well as because mean area measures are less susceptible to variability in component latency and task length (Luck, 2018). We recognize, however, that many researchers choose to score error-related ERPs using peak measures. We chose a baseline of −400 to −200 ms prior to response because baselines that are distal from the response yield more reliable quantification of the ERN with fewer trials (Sandre et al., in prep) and avoid subtracting out portions of the ERN that might occur just prior to response; however, baseline periods used to measure the ERN vary widely (Gehring et al., 2012). Because these methodological choices may affect ERP distributions, it is recommended that researchers who wish to apply the norms presented here exercise caution if other methods are used.

Difference scores such as ΔERN should control for some variation in data recording or processing choices observed across studies because effects should be similar across all conditions (i.e., error-related and correct-related ERPs). In line with this notion, raw condition amplitudes such as the ERN and CRN tend to vary more substantially between studies than condition differences (Bartholow et al., 2005; Horowitz-Kraus, 2011; Jackson et al., 2015; Olvet et al., 2010; Weinberg et al., 2010). Moreover, ΔERN magnitudes seem more robust to variability in study design, as suggested by superior cross-task concordance of ΔERN compared to raw condition measures (Meyer et al., 2013). In light of this, ΔERN might be better suited to normative comparisons than raw ERN or CRN values.

Nonetheless, for ERP data collected and processed using methods similar to those employed here, normative data for the ERN and CRN reported here might prove very useful. Prior work suggests that variation in the ERN and CRN might reflect important individual differences (Bates et al., 2002; Endrass et al., 2010, 2008; Meyer et al., 2017). In the current work, raw ERP scores (ERN, CRN, Pe, and Pc) also displayed higher internal consistency than difference measures (ΔERN and ΔPe), with correct-related responses exhibiting the highest internal consistency. Reliability for ΔERN in particular is not ideal (r = .76). Clayson and Miller (2017) suggest a reliability of at least .80 for ERP measures in well-established research areas and .70 for measures in budding research areas; as such, the reliability of the ΔERN (but not the ERN and CRN) is a concern for individual differences work. Furthermore, previous work demonstrates that the use of difference measures can lead to suppressor effects that may obscure the true relationship between error- and correct-related responses and external measures such as anxiety (Meyer et al., 2017). Instead, the use of unstandardized residuals or raw scores with an interaction term is suggested. While unstandardized residuals are not amenable to normative comparisons, the distributions of ERN, CRN, Pe and Pc reported here can help future researchers that opt for interaction terms over difference measures, provided they use methods similar to those in the current work.

Another potential issue with subtraction measures is lack of clarity regarding whether the difference score is attributable primarily to a condition of interest (e.g., error trials) or to a “control” condition (e.g., correct trials). Here, this is less of a concern, as variation in ΔERN and ΔPe across percentile groups appeared to be driven primarily by error rather than correct trials (Table 5). This pattern would seem to be in line with interpretations of ΔERN and ΔPe as primarily reflecting error sensitivity/processing rather than processing of correct responses (e.g. Moser et al., 2013; Shalgi et al., 2009; Weinberg et al., 2012). Furthermore, results observed here suggest that not only are changes in ΔERN driven primarily by individual differences in ERN (not CRN), but that this is true for individuals at the low, middle and high ends of the ΔERN distribution. In line with prior work (Fischer et al., 2016), we observed an association between gender and ΔERN, such that males had larger ΔERNs than females. While the current study is not optimized to determine the reasons for this gender difference, one possibility concerns dopaminergic release. Computational models have suggested that the ERN is driven by changes in the rate of dopamine firing in response to an outcome that is worse than expected (Holroyd and Coles, 2002; Nieuwenhuis et al., 2002), such that higher levels of dopamine are associated with larger ERNs. This theory has been supported by work examining the effects of dopamine-related genes (Biehl et al., 2011) and pharmacological manipulations (De Bruijn et al., 2004) on the ERN. Meanwhile, PET work indicates that males exhibit more robust dopamine release in the striatum than females, possibly due to estrogen differences between males and females (Munro et al., 2006). Additionally, although the relationship between ACC morphology and ERN magnitude is not well understood, gender differences in the morphology of the ACC have also been documented (Huster et al., 2007; Mann et al., 2011; Yücel et al., 2001). In line with previous work (Herrmann et al., 2004), LORETA estimates in the current sample identified the ACC/cingulate cortex as a likely source of the ERN, and morphological gender differences in this area might contribute to gender differences in the ERN.

In any case, the gender difference in ΔERN amplitude does seem to be consistent in previous literature; Another study using a smaller, older sample (198 healthy individuals between the ages of 18 and 52; Larson, South, & Clayson, 2011) also concluded that males had larger ΔERN amplitudes than females. In contrast to the current work, Larson and collegues (2011) also found that males displayed larger ΔPe magnitudes than females; this difference might be due to the different tasks employed or the difference in age ranges. Here, correct-related responses (CRN and Pc) also differed by gender in directions consistent with effects observed for ΔERN and ΔPe; on the other hand, gender did not affect error-related responses (ERN and Pe). This pattern of results, along with increased RTs, better accuracy, and increased post-error slowing in females (outlined below), might indicate a more general effect of gender on response monitoring rather than error-specific gender differences. Future attempts to benchmark an individual’s CRN, Pc, ΔERN or ΔPe against our overall distributions would be advised to reference the separate percentile values provided for each gender.

ERN and CRN amplitudes were not associated with age. While the effects of age on ΔERN, Pc, Pe and ΔPe were statististically significant, the effect sizes of the relationships were very small (τ = −.07 for ΔERN, τ = .06 for Pc, τ = −.08 for Pe, τ = −.13 for ΔPe). This may be in part because of the relatively restricted age range of participants in the current study (18–30 years). Nonetheless, prior work by Fischer and colleagues (2016) employed a slightly larger age range (18–40 years) and failed to find a significant association between age and ΔERN. Work that has employed adolescent samples may help inform the current results. That is, in a sample of 550 adolescents (M_AGE = 14.39), Weinberg and colleagues (2016) found a much smaller ΔERN (M = −2.66 μV; SD = 3.88) than observed here. In addition, Santesso and Segalowitz (2008) found that ΔERN was smaller for individuals aged 15–16 compared to those aged 18–20.

Therefore, one possibility is that ΔERN magnitude may increase with age, which would be in line with the direction of the association observed here, and which would fit with evidence suggesting that the brain regions involved in generation of the ERN (i.e., the anterior cingulate cortex) are still developing throughout adolescence and early adulthood (Kelly et al., 2009; Rubia et al., 2007). Similarly, dopaminergic systems that might underlie ERN responses continue developing through adolesence, and the trajectory of the development predicts an increase in ERN magnitude as age increases (Segalowitz and Dywan, 2009). Taken together, results suggest that the norms provided here should be regarded as specific to the age range employed (18–30 years).

ΔERN magnitude differed significantly by lab site in the current sample, though the effect size was small. Moreover, controlling for gender, age and behavior differences across labs reduced the effect of site sufficiently that it only barely reached significance. In considering the reason for lab differences in ΔERN magnitude, recording differences seem unlikely, because the effect was only evident for ΔERN, not ERN or CRN (nor any Pe-related measures). However, the labs were located in somewhat different geopolitical regions (Canada, Indiana, Texas), and some work suggests that differences in cultural and political views may affect ΔERN magnitude (Amodio et al., 2007; Kitayama and Park, 2014; Weissflog et al., 2013). Regardless, integrating data from different lab sites makes the results more likely to generalize to other settings. That is, because we collapsed across three lab sites, our results can be taken as a reasonably good approximation of the distribution of ΔERN in the average unselected college-aged population.

Participants who made fewer errors had larger ERNs. The relationship between accuracy and the ERN has been examined previously, with results evincing a moderately strong to absent/nonsignificant association (Carp and Compton, 2009; Herrmann et al., 2004; Vocat et al., 2008). Here, the association between accuracy and the ERN was relatively weak (τ = −.08), though previous work that used a similarly-sized sample reported a stronger relationship in the same direction (Fischer et al., 2017). It is difficult to explain why different magnitudes of association between accuracy and the ERN have been observed. Fischer and colleagues (2017) varied the physical distance between arrows in the Flanker task across trials, which should have increased response conflict when arrows were closer together, modulating the ERN (Danielmeier et al., 2009). Moreover, Fischer and colleagues (2017) used a shorter ITI (either 250 ms or 700 ms) compared to the range used in the current work (1000 ms - 2000 ms), which has also been shown to affect both accuracy and the size of the ERN, even when controlling for number of errors (Compton et al., 2017). Both of these differences in task design could have affected the relationship between the ERN and accuracy. Other possible moderators include sample differences or methodological choices such as ERP scoring methods. As such, future work might wish to assess potential moderators of relationships between ERPs and behavior. Interestingly, Fischer and colleagues found that the association between accuracy and the ERN did not depend on the number of errors per participant used to quantify the ERN. Therefore, it would seem that larger ERNs among participants who are more accurate might be attributable to individual differences in the way errors are processed, rather than, for example, a decline in ERN amplitude as more and more errors are made.

Compared to accuracy, we generally observed stronger relationships between RT and ERPs, with the strongest relationships evident between correct RTs and correct-related ERPs. Participants who responded faster on correct trials had larger CRNs and smaller Pc amplitudes. Of note, however, the relatively large effect of gender on ERN, CRN and ΔERN remained significant even after controlling for all behavioral measures. Because research has not typically controlled for behavior when examining error- or correct-related ERPs (i.e., behavior and ERPs are allowed to covary), we chose not to report ERPs stratified by accuracy or RT. Nonetheless, we believe that a better understanding of associations between behavior and error-related ERPs may be fruitful, particularly in regards to characterizing individual differences. For example, how do individuals with the same sized ERN but different levels of accuracy differ from or resemble each other? What about individuals with different sized ERNs but similar levels of accuracy? Put differently, controlling for associations between behavior and the ERN could yield information about individual differences that is not otherwise apparent.

4.2. Behavior

The present study provides the first normative examination of the distribution of behavior in the Erisken Flanker task. Females tended to respond more accurately than males, although the effect size was small (d = 0.13). Overall, females responded more slowly than males, though the effect of gender on RT was larger for correct (d = 0.34) versus error trials (d = 0.14), with the latter difference presenting at a trend level of significance (p = .05). Previous work corroborates the direction of this difference, with females responding more slowly than males on the Flanker task in the sample of 198 participants collected by Larson and colleagues (2011) as well as the sample of 863 collected by Fischer and colleagues (2016). Furthermore, the study by Fischer and colleagues found a larger gender difference for correct RT than error RT, in line with the current work.

In the present sample, females also displayed a smaller difference in RT between correct and error trials (ΔRT) than males, as well more pronounced PES than males. Prior work provides conflicting data regarding gender differences in PES; while Fischer and colleagues (2016) also found greater PES in females than males, Larson and colleagues (2011) found no gender differences. This may be a result of the smaller sample size collected by Larson and colleagues (2011) compared to those collected by Fischer and colleagues and in the current work, both of which were highly powered to detect the relatively small effect. In sum, where an individual falls within the distribution of RT scores, and likely PES, probably differs by gender, and gender-specific norms should be employed (see Table 8).

Age did not have a significant effect on accuracy, correct RT, error RT, or PES. The effect of age on ΔRT was statistically significant, but the magnitude of this effect was very small (τ = −.06). These results are in line with previous work, which found no effect of age on RT measures in a similar sample (Fischer et al., 2016). Nonetheless, conclusions are limited, given the restricted age range of our sample.

As was the case with ERPs, behavioral measures that were derived from a greater number of trials (accuracy and correct RT) displayed the highest internal consistency, followed by error-related measures (error RT), with difference measures (ΔRT and PES) exhibiting worse internal consistency. While reliability for ΔRT (r = .70) was not ideal, reliability for PES (r = .33) was exceptionally poor. To our knowledge, the current work is the first large-scale examination of the internal reliability of PES. The results here suggest that PES is not reliable enough to track meaningfully with individual differences (e.g., as assessed via self-report or other measures) and should not be used as a prescreening measure for data recruitment or for clinical applications.

4.3. Future Directions and Challenges

The current work is intended to serve as an initial step toward facilitating the inclusion of ΔERN and ΔPe in psychological assessment or novel research applications (e.g., brain-based participant recruitment). However, depending on the trait or attribute of interest, additional challenges may emerge. For example, when assessing psychiatric risk or clinical status, it is likely that distributions of the ERN corresponding to ‘normal’ and ‘abnormal’ will overlap substantially. Therefore, while a number of studies have reported a larger mean ΔERN in individuals with anxiety disorders compared to controls (Hajcak et al., 2003; Ladouceur et al., 2006; Moser et al., 2013; Olvet and Hajcak, 2008), classification at the individual level may not be possible using ΔERN alone. Along these lines, Patrick and colleagues (2013) have recently suggested that the combination of multiple indicators of a single psychological trait (e.g., questionnaire data, peripheral psychophysiology, neurophysiology, behavioral response), may yield a more accurate assessment than any one of these measures alone. As an example of this type of approach, a recent study examining 444 participants found that a combination of reward-related electrocortical activity and self-report data predicted first-onset depression better than either variable when used in isolation (Nelson et al., 2018). Although substantial work remains in translating these approaches to the clinic (e.g., how best to combine these various measurements into a single indicator at the individual level), it is possible that greater differentiation between ‘normal’ and ‘abnormal’ may be achieved if more than one indicator is considered simultaneously.

It is also possible that measuring the same ERP elicited by different tasks might be more valuable than using a single task. The distribution reported here was derived only from performance on the Flanker task; this task is the most commonly used task to elicit the ERN and ERNs elicited by the Flanker correlate well with those elicited by other tasks (Riesel et al., 2013), making it a good starting point for establishing normative data for the ERN. However, while correlations between ERNs elicited by the Flanker and other tasks are reasonably good, they are far from perfect – correlations ranged from .37 to .49. Therefore, it is possible that other tasks might yield different error-related ERP distributions than the Flanker task, and that comparing an individual’s ERN (or Pe) response to normative distributions on multiple tasks might provide better predictive or diagnostic power than using only one task.

In accordance with general guidelines for the norming of psychological assessments, the norms reported here should be considered specific to the procedures and task used – the arrow version of the Flanker task – and should not be assumed to apply to other tasks such as the Go/No-Go task. Future work might wish to examine large-sample distributions of the ERN and Pe elicited by other tasks to compare with these norms. Additionally, given the relationship between task performance (accuracy) and ERP magnitude reported in previous work (Fischer et al., 2017), researchers who use the norms reported here to interpret data from arrow version of the Flanker task are cautioned to consider the potential effects of participant performance on ERP magnitude. Indeed, we acknowledge the need for future work to assess the effects of task parameters (such as timing and stimuli differences) and performance factors on the distributions of the ERN and Pe, and to take these factors into consideration when applying norms.

Other barriers to the integration of neurophysiological measures in psychological assessment must be addressed as well. For example, most laboratory tasks, including the Eriksen Flanker task, were designed to elicit robust condition differences across individuals, rather than to distinguish between individuals (Hajcak et al., 2017; Hedge et al., 2017). This is in contrast to more traditional assessment measures, such as self-report questionnaires, which are designed to differentiate between individuals. Therefore, in order to maximize utility for psychological assessment, it may be necessary to redesign or select tasks with this end goal in mind. Tasks that better measure individual differences might yield behavioral and ERP distributions with greater variance than those reported in the current study, or possibly even multimodal distributions, allowing for clearer differentiation among subgroups of the distributions. Additionally, different versions of a task might be created such that particular versions differentiate optimally between individuals at either high or low ends of a trait spectrum.

A significant limitation of the current study is that the our sample lacked ethnic diversity. Several ethnic groups were underrepresented in relation to their representation in the general population. Additionally, socioeconomic data was not collected for the majority of the sample, and thus was not reported. However, because the sample consisted largely of college students, it is likely that the sample was skewed towards higher incomes and greater education with respect to the general population. Therefore, normative data in the current work is likely most applicable to a primarily white and relatively educated sample, as are most studies using samples composed of college students. Future work may wish to examine the distribution of the ERN and CRN in more diverse samples to assess the influence of ethnicity and socioeconomic variables, and to provide norms for these samples.

Along these lines, the current work examined only individuals aged 18–30. Crucially, however, previous work indicates that the size of the ERN might increase with age throughout childhood and adolescence (Santesso and Segalowitz, 2008; Segalowitz and Dywan, 2009). As such, the norms reported here are likely not applicable to individuals outside of the age range of the current sample; norms for other age groups are needed. Further, the age distribution of our sample was skewed toward the lower end of the 18–30 age range; future work should establish whether larger samples in the 25–30 years range corroborate the results reported here. It is important to note that not only do average ERN magnitudes differ with age among children and adolescents, but that there is greater variability in the amplitude and timing of the ERN within and across children compared to adults (Segalowitz and Dywan, 2009). This might be in part to the differing rates at which the neural processes underlying the ERN develop across children. The variation presents an important challenge for future work wishing to establish norms for developing brains.

5. Conclusions

Neurophysiological measures, particularly when used in combination with other trait indicators (e.g., self-report and behavioral measures, other biological measures), have the potential to improve precision in psychological assessment and research. As reliable measures that are relatively affordable and easy to elicit, ERN and Pe hold substantial promise in this regard. Here, we present data that represents a necessary step towards these ends by establishing normative data for a large sample of healthy young adults. While further obstacles must be addressed before ERP measures can be used towards these aims, the current work provides a base upon which future work can build.