Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jun 2.
Published in final edited form as: Behav Ther. 2021 Sep 2;53(2):196–207. doi: 10.1016/j.beth.2021.07.004

Measuring Emotion Dysregulation in Children with Attention-Deficit/Hyperactivity Disorder: Revisiting the Factor Structure of the Emotion Regulation Checklist (ERC)

Melanie R Silverman a, Randi Bennett b, Leah Feuerstahler a, Jill Stadterman a, Anthony Steven Dick c, Paulo Graziano c, Amy Krain Roy a,b
PMCID: PMC9162246  NIHMSID: NIHMS1800853  PMID: 35227398

Abstract

Emotion dysregulation (ED) is prevalent among youth with Attention-Deficit/Hyperactivity Disorder (ADHD) and significantly impacts functioning. Nuanced measurement of ED is central to understanding its role in this disorder and informing treatment approaches. The present study examined the factor structure of the Emotion Regulation Checklist (ERC) among children with ADHD with and without Oppositional Defiant Disorder (ODD). Exploratory factor analysis (EFA) conducted in a sample of 328 youth (mean age=6.08) with ADHD indicated a four-factor solution, comprised of the following factors: Negative Emotion Lability, Positive Emotion Lability, Socially Appropriate Affect, and Socially Incongruent Affect. The Negative and Positive Emotion Lability subscales assess the reactivity of negatively and positively valenced emotions, respectively. The Socially Appropriate and Socially Incongruent Affect subscales provide an assessment of social-emotional functioning. All subscales discriminated between children with ADHD only and ADHD with co-morbid ODD, such that children with ODD had greater emotional lability and social-emotional difficulties. This revised factor structure of the ERC facilitates a uniquely brief, yet multifaceted and specific, assessment of emotional difficulties in children with ADHD that can inform treatment planning and operationalize emotional reactivity and social-emotional functioning in future research efforts.

Keywords: emotion regulation, ADHD, children, factor analysis, assessment

Emotion dysregulation (ED) underlies a variety of developmental psychopathologies and is prevalent among youth with externalizing disorders including Attention-Deficit/Hyperactivity Disorder (ADHD; Eisenberg et al., 1996; Röll et al., 2012; Suveg & Zeman, 2004). While ED is not part of the diagnostic nosology of ADHD, there is clear evidence that dysregulated attention and poor inhibitory control characteristic of this disorder extend to emotions, with 25 to 45% of children with ADHD exhibiting ED (Shaw et al., 2014). Children with ADHD and co-occurring ED exhibit greater impairment in psychosocial and academic functioning than those with ADHD alone (Shaw et al., 2014; Wehmeier et al., 2010). Given the prevalence of ED in this population and its impact on functioning, it is critical to establish nuanced measures of emotion regulation difficulties to facilitate development of targeted treatment approaches. One frequently used measure of ED, the Emotion Regulation Checklist (ERC; Shields & Cicchetti, 1997) could be leveraged to more precisely assess difficulties with emotion regulation in this clinical population. For this reason, the present study sought to explore the factor structure of the ERC in children with ADHD.

The term “emotion regulation” is commonly used to refer to multiple aspects of emotional functioning. Theoretical frameworks include two phases of the emotion regulation process: emotion generation and emotion regulation. Emotion generation refers to the emergence of emotion(s) in response to an internal or external stimulus, and regulation refers to the modulation of that emotion(s) across time (Bunford et al., 2015; Fernandez et al., 2016; Sheppes et al., 2015). Difficulties in either of these phases can emerge with emotions of any valence; both positive and negative emotions can be dysregulated and adversely impact functioning. In a meta-analysis of youth with ADHD, difficulties with emotional reactivity were most prominent, followed by difficulties with emotion regulation (Graziano & Garcia, 2016). Neuroimaging studies of children with ADHD have demonstrated that such difficulties with emotion regulation, as evidenced by emotional lability and severe temper outbursts are associated with disruptions in neural circuits (Hulvershorn et al., 2014; Roy et al., 2018). For example, in a sample of children with ADHD, those with high emotional lability were found to have altered intrinsic functional connectivity (iFC) of the amygdala relative to children with lower emotional lability (Hulvershorn et al., 2014). Prior work by Sjöwall et al. (2013) has also found that youth with ADHD have greater parent-reported difficulty returning to baseline following both positive and negative emotional episodes than typically-developing (TD) children, even when controlling for conduct problems and internalizing symptoms. Further, both dysregulation of anger and exuberance were found to predict whether or not a child was diagnosed with ADHD, over and above executive functioning abilities.

Oppositional defiant disorder (ODD) commonly co-occurs in youth with ADHD (Angold, 1999; Steinberg & Drabick, 2015) and may contribute to emotional lability, particularly in the context of negative emotions (Dunsmore et al., 2016). Children with ODD also exhibit difficulties with emotional functioning in social situations (e.g. unemotionality, lack of empathy, etc.; Bunford et al., 2015), that are not typically observed in children with ADHD without this comorbidity. Given the described variety of ED weaknesses in children with ADHD, both with and without ODD, it is critical to have measures that quantify these specific areas of emotional functioning. Such assessment tools would allow for improved understanding of the ways in which different aspects of ED play a role in ADHD and for specialized development of targeted treatment plans.

There are many levels of analysis at which to assess emotion regulation. The National Institute of Mental Health’s Research Domain Criteria (RDoC) indicates that constructs should be explored across eight units of analysis: genes, molecules, cells, circuits, physiology, behavior, self-report, and paradigms (Insel et al., 2010). While studying ED at each of these levels will help advance our empirical understanding (Fernandez et al., 2016), self- and informant-report approaches remain standard in clinical practice. Such measures, though not without limitations, provide unique insights into children’s emotion regulation abilities across settings, impact on daily functioning, and caregiver perception (Zeman et al., 2007). There are a wide range of questionnaire measures that assess emotion regulation and dysregulation in pediatric samples (for review see Adrian et al., 2011). While many measures exclusively elicit information about negative emotions, the Emotion Regulation Checklist (ERC) is an informant-report measure that assesses the regulation of both positive and negative emotions in children across two subscales: the Lability/Negativity subscale (15 items), which reportedly assesses “lack of flexibility, mood lability, and dysregulated negative affect”, and the Emotion Regulation subscale (8 items), which reportedly assesses “situationally appropriate affective displays, empathy, and emotional self-awareness” (Shields & Cicchetti, 1997, p. 910). The ERC was initially validated in a sample of maltreated and non-maltreated children ages six- to twelve-years-old attending a week-long summer camp with counselors who had spent approximately 35 hours with the children providing the ratings. The factor structure of the measure was evaluated via principal-components factor analysis with varimax rotation (Shields & Cicchetti, 1997). Since publication, the measure has been widely used, both in the United States and internationally (Meybodi et al., 2018; Molina et al., 2014; Reis et al., 2016). However, there have not been further evaluations of the psychometric properties of the measure in clinical samples.

Given the aforementioned significance of difficulties with varied aspects of emotion regulation among children with ADHD and the paucity of psychometric evaluations of the ERC in clinical samples, the present study sought to explore the underlying dimensions of the ERC in this population. The current sample of children with ADHD exhibited a wide range of ED, including individuals with significant irritability, characterized by clinically significant temper outbursts and/or ODD, and those recruited specifically for the absence of such comorbidities. Thus, the current clinical sample provides a rich context in which to explore the dimensions of the ERC. The current investigation had four primary aims: (1) to examine the model fit of the current ERC factor structure in a sample of children with ADHD, (2) to use exploratory factor analysis to examine alternative underlying dimensions of the ERC in this population, and (3) to explore the clinical utility of the newly-identified ERC subscale scores to show differences between children with ADHD and ODD, and children with ADHD without ODD.

Methods

Data were collected across two sites. At Fordham University (Site One), data from two studies exploring temper outbursts in children were analyzed retrospectively. Both studies were approved by Institutional Review Boards. Parents/legal guardians provided signed informed consent and children provided oral or written (age seven and older) assent. At Florida International University (Site Two), data came from an ongoing longitudinal study examining the heterogeneity of emotion regulation among young children with ADHD. The study was approved by the Institutional Review Board. Parents/legal guardians provided signed informed consent and children who were seven-years-old provided oral assent.

Participants

The total sample included 328 parents of children ages four- to nine-years-old (259 males; mean age=6.08±1.63) who completed the ERC (n=154 enrolled at Site One and n=174 enrolled at Site Two). All children included in the current analyses had a confirmed diagnosis of ADHD, as determined by methods described below.

For Site One, children ages five- to nine-years-old were recruited across two groups: those with ADHD and clinically significant temper outbursts defined as three or more outbursts per week lasting at least 10 minutes, and those with ADHD without significant outbursts. Children with a history of Autism Spectrum Disorder, psychosis, Post-Traumatic Stress Disorder, or an IQ score of less than 75 were excluded. Additionally, one of the two studies included an MRI component; thus, children in that study were excluded for neurological disorders (e.g. epilepsy), psychoactive medication (except for stimulants, which were discontinued 72 hours prior to the study visit; n=11), and other MRI contraindications including metal of any kind in the body, previous surgeries, previous head injury, injury involving metal, and medical devices containing metal, such as braces.

For Site Two, the parents and children ages four- to seven-years old were invited to participate in an assessment to determine study eligibility if a parent (1) endorsed clinically significant levels of ADHD symptoms (six or more symptoms of either Inattention or Hyperactivity/Impulsivity according to the DSM-5 [5th ed.; DSM-5; American Psychiatric Association, 2013]) or reported a previous diagnosis of ADHD, (2) indicated that the child was currently displaying clinically significant academic, behavioral, or social impairments as measured by a score of three or higher on a seven-point impairment rating scale (Fabiano et al., 2006), and (3) reported that the child was not taking any psychotropic medication. Participants were also required to be enrolled in school during the previous year, have a measured IQ of 70 or higher, have no confirmed history of an Autism Spectrum Disorder, and be able to attend an 8-week summer treatment program (STP-PreK; Graziano et al., 2014) prior to the start of the next school year.

Procedures

Across both studies, children and their caregivers were recruited from local schools and mental health agencies and from the community via brochures, radio and newspaper ads, and open houses/parent workshops. Legal guardians contacted the clinic and were directed to the study staff for screening questions to determine eligibility.

At Site One, those who were determined to be eligible based on a telephone screen were scheduled for an in-person visit. Diagnostic information was obtained by doctoral students and post-doctoral fellows through parent interviews using the Schedule for Affective Disorders and Schizophrenia- Child Version (KSADS-PL; Kaufman et al., 1997). This included the KSADS-PL screener for Autism Specturm Disorders (ASD). Final diagnoses were determined using case conference consensus with a licensed clinical psychologist, and children suspected to have a possible diagnosis of ASD were excluded from the study. Parents also completed additional interview and questionnaire measures regarding their children’s behavioral, academic, and emotional functioning. Children completed questionnaire, cognitive assessment, and behavioral measures. Those relevant to the current analyses are described below. At the conclusion of the study visit, families were compensated for their time.

At Site Two, those who were determined to be eligible based on a telephone screen were scheduled for an intake during which ADHD diagnosis and comorbid disruptive behavior disorders were assessed through a combination of parent structured interview (Computerized-Diagnostic Interview Schedule for Children [C-DISC]; Shaffer et al., 2000) and parent and teacher ratings of symptoms and impairment (Disruptive Behavior Disorders Rating Scale, Impairment Rating Scale; Fabiano et al., 2006; Pelham et al., 1992), as is recommended practice (Pelham et al., 2005). Dual Ph.D. level clinician review was used to determine diagnosis and eligibility. In addition to completing the C-DISC, parents completed various questionnaires regarding their children’s behavioral, academic, and emotional functioning, including those relevant to this investigation, described below. To screen for ASD, parents also completed the Autism Screening Rating Scale Short Form (ASRS; Goldstein & Naglieri, 2009) and children with elevated T-scores of 70 or higher were excluded from the study. Children also completed a series of social-emotional/self-regulation and cognitive assessment tasks in the laboratory. Families of children with ADHD received a summer camp intervention at either no cost via a federal grant or at a subsidized cost via a local grant, and all families received compensation.

Measures

Parents from both sites completed the Emotion Regulation Checklist (ERC), a 24-item questionnaire that utilizes a 4-point Likert scale (i.e., 1=never to 4=almost always) and yields two subscales according to the original psychometric evaluation (Shields & Cicchetti, 1997): the Lability/Negativity subscale (15 items), and the Emotion Regulation subscale (8 items). Internal consistency of the Emotion Regulation and Lability/Negativity subscales in the present sample were 0.62 and 0.82, respectively.

A subset of parents at both sites (n=249) completed the Behavioral Assessment System for Children Parent Rating Scales (BASC PRS) - Child or Preschool Forms as a dimensional measure of child behavioral, cognitive, emotional, and learning difficulties (Reynolds & Kamphaus, 2004, 2015). At Site One, parents completed the BASC Second Edition and at Site Two, parents completed the BASC Third Edition. On this measure, respondents rate item frequency on a Likert-type scale (“never”, “sometimes”, “often”, “always”). Parents of five-year-old children completed the BASC PRS Preschool (Site One: n=14; Site Two: n=100) and parents of children six-years-old and older completed the BASC PRS Child (Site One: n=112; Site Two: n=23). For the current study, Hyperactivity and Attention Problems T-scores were used to assess for differences in symptom severity in these domains between children with ADHD only and ADHD with co-morbid ODD. Given the retrospective nature of the study, item-level data was only available for Site Two and within this sample, internal consistencies of these subscales were good (BASC PRS Preschool Hyperactivity = 0.85, BASC PRS Preschool Attention Problems = 0.86, BASC PRS Child Hyperactivity = 0.78, BASC PRS Child Attention Problems = 0.71).

Data Analysis

Model fit of the original two-factor structure was examined with confirmatory factor analysis using maximum likelihood estimation with robust standard errors and a mean and variance adjusted test statistic. Items 4, 5, 9, 11, 16, and 18 were reverse coded and assigned to load onto the Emotion Regulation and Lability/Negativity factors per the original manuscript (Shields & Cicchetti, 1997). This model was assessed with the following fit indices: log-likelihood chi-square test, standardized root mean square of the residual (SRMSR), root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker Lewis index of factoring reliability (TLI). SRMSR and RMSEA values of 0.10 or less and CFI and TLI values of 0.90 or greater are generally indicative of good fit. SRSMR and RMSEA of 0.06 or less and CFI and TLI values of 0.95 or greater are generally indicative of excellent fit (Finch & West, 1997).

The factor structure of the ERC was subsequently re-examined using exploratory factor analysis of inter-item polychoric correlations. A smoothed polychoric correlation matrix was calculated using pairwise complete data. At entry, items were reverse-coded such that higher scores represented greater emotion dysregulation. Eigenvalues were calculated and the number of factors was determined via Cattell’s scree plot and Horn’s parallel analysis, which has been found to perform well with polychoric correlation matrices (Timmerman & Lorenzo-Seva, 2011). Factors were extracted using principal axis factoring with oblimin rotation, allowing factors to correlate. Factor loadings of individual items of 0.4 and higher were considered meaningful and items with loadings of less than 0.4 were removed. Cronbach’s alphas were calculated for each factor in the final solution.

Subscale scores were constructed by taking the simple sum of responses to each of the identified dimensions. Relationships between subscale scores and age were evaluated via bivariate correlations. Bonferroni correction for multiple comparisons was applied and the threshold for significance was 0.0125 (0.05/4 comparisons). Differences in subscale scores between children with ADHD and co-morbid ODD and with ADHD without ODD were explored via one-way ANCOVAs. The Bonferroni correction for multiple comparisons was applied and the threshold for significance was 0.0125 (0.05/4 comparisons). These analyses were repeated excluding the 11 children from Site One who were taking stimulant medication. While most procedures for measurement invariance between groups make use of confirmatory, rather than exploratory, factor analysis, it is inadvisable to use the same dataset for both exploratory and confirmatory analyses. Given this, differences in the functioning of the measure between the two diagnostic groups were assessed by testing for significant differences in the inter-item correlation matrices between diagnostic groups with Jennrich’s test for the equality of correlation matrices (Jennrich, 1970) and calculating the root mean square difference (RMSD) of unique inter-item correlations across the two groups. If measurement invariance holds, then the population correlation matrices for each group will be equivalent to each other. All analyses were conducted with the statistical software R (R Core Team, 2013), using the lavaan package or SPSS (IBM SPSS Statistics, Version 25.0).

Results

Demographic and Clinical Characteristics

Demographic and clinical characteristics of the sample are presented in full in Table 1. Of the 328 children (mean age=6.08 ± 1.63), 78.96% were male, 69.51% were Caucasian, and 57.01% were Hispanic. The entire sample met criteria for ADHD and 61.3% of the children met criteria for co-morbid ODD. Participants from Site One were older (7.48±1.16 years) than participants from Site Two (4.85±0.75 years), t(326)=24.56, p<0.001).

Table 1.

Demographic and Clinical Characteristics

Characteristic Mean (SD) or Count (%)
Whole Sample (N =328) Site One Sample (N =154) Site Two Sample (N =174)
Mean age in years (SD) 6.08 (1.63) 7.48 (1.16) 4.85 (0.75)
Males (%) 259 (78.96) 120 (77.92) 139 (79.88)
Site
 Site One (%) 154 (46.95) 154 (100) 0 (0)
 Site Two (%) 174 (53.04) 0 (0) 174 (100)
Race
 White (%) 228 (69.51) 76 (49.35) 152 (87.35)
 Black (%) 57 (17.37) 42 (27.27) 15 (8.62)
 Asian (%) 10 (3.04) 8 (5.19) 2 (1.15)
 Mixed (%) 22 (6.70) 17 (11.04) 5 (2.87)
 Other (%) 11 (3.35) 11 (7.14) 0 (0)
Ethnicity
 Hispanic (%) 187 (57.01) 43 (27.92) 144 (82.76)
 Missing (%) 3 (0.91) 2 (1.29) 1 (0.57)
Diagnoses
 ADHD without ODD (%) 127 (38.71) 85 (55.19) 42 (24.14)
 ADHD & ODD (%) 201 (61.28) 69 (44.81) 132 (75.86)
Stimulant Medication 11 (3.35) 11 (7.14) 0 (0)
Open in a new tab

Confirmatory Factor Analysis of Prior Model

The fit of the previously identified two-factor model was assessed. The log-likelihood chi-square test rejected the two-factor model, X2(229) = 509.82, p < 0.001. It is well-known that the log-likelihood chi-square test is overly powerful and almost always rejects model fit (MacCallum, 1990). However, additional fit indices also indicated that the original two-factor model had poor fit (SRMSR=0.108; RMSEA index=0.062; TLI=0.531; CFI=0.575).

Exploratory Factor Analysis

Cattell’s scree plot and Horn’s parallel analysis indicated a four-component solution. Based on these results, a four-component structure was retained. The eigenvalues of the polychoric correlation matrix for the four components were 6.29, 2.72, 2.05, and 1.81. Most items had salient loadings of 0.4 or greater; four items were removed for low loadings. See Table 2 for a complete list of factor loadings.

Table 2.

Exploratory Factor Analysis: Factor Loadings

ERC Item Negative Emotion Lability Positive Emotion Lability Socially Appropriate Affect Socially Incongruent Affect
1.* Cheerful child .51 −.18 .33 .17
2. Mood swings .65 .12 .11 .17
4.* Transitions well .40 .11 .35 −.29
5.* Recover quickly .47 −.01 .33 −.13
6. Easily frustrated .81 −.02 −.15 .08
8. Angry outbursts .93 .00 .00 .00
14. Responds angrily .63 .28 .00 −.03
13. Exuberant outbursts .17 .74 .05 −.03
17. Overly exuberant −.17 .72 −.01 .04
20. Impulsive .22 .61 −.05 −.11
22. Displays exuberance −.05 .78 −.04 .08
3.* Positive to adults −.02 −.01 .61 .10
7.* Positive to peers −.01 −.01 .69 .12
21.* Empathic .02 .00 .58 .11
23.* Appropriately negative −.29 −.12 .44 −.12
10. Pleasure others’ distress .00 .22 .19 .45
16. Sad or listless .23 .00 .01 .56
18. Flat affect .02 −.02 .10 .65
19. Negative to peers .11 .14 .19 .56
24. Negative in play .06 .39 .10 .40
Items Removed
9. Delay gratification .15 .22 .21 −.30
11.* Modulate excitement −.05 .22 .33 −.35
12. Whiny or clingy .23 .26 −.03 .20
15.* Can say when feeling negative emotion −.04 −.08 .36 .11
Open in a new tab

Note. Loadings ≥ 0.40 are bolded.

*

indicates item is reverse scored.

As shown in Table 2, after oblimin rotation, the first factor was comprised of seven items primarily concerning the reactivity of negative emotions and the scale was thus named Negative Emotion Lability. The second factor was comprised of four items assessing exuberance and impulsivity. This factor was named Positive Emotion Lability. The third factor contained four items regarding adaptive displays of emotions in social contexts and was named Socially Appropriate Affect. The fourth factor contained five items generally describing atypical displays of emotions in social situations and was named Socially Incongruent Affect.

Subscales for each of the four factors were derived by summing responses to the included items. For Negative Emotion Lability, Positive Emotion Lability, and Socially Incongruent Affect, higher scores represent greater dysregulation. For Socially Appropriate Affect, items were recoded such that higher scores represent greater regulation. Cronbach’s alpha for Negative Emotion Lability and Positive Emotion Lability were good and 0.83 and 0.77, respectively. For Socially Appropriate Affect and Socially Incongruent Affect, Cronbach’s alpha values were acceptable and 0.62 and 0.65, respectively (Boyle et al., 2015). Correlation coefficients between the four subscales ranged from 0.03 to 0.37 (see Table 3). The subscales were not correlated with age after Bonferroni correction for multiple comparisons (p’s > 0. 125).

Table 3.

Subscale Score Correlation Coefficients

Subscale NEL PEL SAA SIA
NEL - .37* −.19* .39*
PEL - .03 .33*
SAA - −.21*
SIA -
Open in a new tab

NEL = Negative Emotion Lability; PEL = Positive Emotion Lability;

SAA = Socially Appropriate Affect; SIA = Socially Incongruent Affect

*

indicates a significant correlation at the 0.01 level

Subscale Score Differences between Children with and without ODD

Preliminary analyses indicated that the inter-item correlation matrices were different between children with co-morbid ODD and children without ODD, but that the size of the effect was relatively small, X2(380, N=328)=486.04, p<0.001, RMSD=0.12. As such, the measure appears to be functioning similarly between groups, but there may be some lack of invariance worthy of further investigation and the below results are interpreted with caution. There were no significant differences in sex (p > .05) or BASC PRS Hyperactivity and Attention Problems T scores (p’s > 0.05) between children with co-morbid ODD and children without ODD. For Site One, full diagnostic data was available and there were no differences in number of co-morbid diagnoses beyond ADHD and/or ODD between the two diagnostic groups (p’s > 0.05). Diagnostic groups did differ in site, X2(1, N=328)=33.20, p<0.001, age, t(326)=5.46, p<0.001, race, X2(4, N=328)=25.03, p<0.001, and ethnicity, X2(1, N=325)=7.01, p=0.008, and thus these variables were controlled for in the subsequent analyses. Six children were excluded from the following analyses for missing ethnicity data. One additional child was excluded from the Negative Emotion Lability analysis and two additional children were excluded from the Positive Emotion Lability analysis for missing composite ERC item(s).

As shown in Figure 1, significant differences in all four subscale scores were observed between children with and without comorbid ODD as determined by one-way ANCOVAs. Negative Emotion Lability scores were higher in children with co-morbid ODD (M=18.53, SD = 4.14) than those of children without ODD (M=15.28, SD=4.15); F(1,318)=69.35, p<0.001. Similarly, for Positive Emotion Lability, subscale scores of children with co-morbid ODD (M=10.36, SD=2.80) were higher than those of children without ODD (M=9.06, SD=2.82); F(1,317)=15.20, p<0.001. For Socially Appropriate Affect, scores of children without co-morbid ODD (M=11.90, SD=2.48) were higher than those of children with ODD (M=11.33, SD=2.20); F(1,319)=7.33, p=0.007. Socially Incongruent Affect scores of children with co-morbid ODD (M=7.29, SD=2.05) were higher than those of children without ODD (M=6.71, SD=2.02); F(1,319)=4.04, p=0.001. These analyses were repeated excluding the 11 children who were taking stimulant medication and all of the above differences remained significant (p’s < 0.0125).

Figure 1.

Figure 1.

Open in a new tab

Diagnostic Group Subscale Score Differences

Note. NEL = Negative Emotion Lability; PEL = Positive Emotion Lability; SAA = Socially Appropriate Affect; SIA = Socially Incongruent Affect. Error bars represent ± 1 S.E.

*p < 0.01

**p ≤ 0.001

Discussion

Assessing specific areas of weakness in emotion regulation is central to improving our understanding of ED in ADHD and developing targeted treatment plans in the clinic. The present study sought to explore the factor structure of the ERC in two community-based samples of children with ADHD. Confirmatory factor analysis demonstrated that the original two-factor model had poor fit. Rather, four factors were identified via exploratory factor analysis: Negative Emotion Lability, Positive Emotion Lability, Socially Appropriate Affect, and Socially Incongruent Affect. Two factors appear to assess positive and negative emotional reactivity. The Negative Emotion Lability subscale assesses the frequency of negative emotional reactions, or the rapid escalation of negative emotions such as anger or frustration. The Positive Emotion Lability subscale is comparable insofar as it assesses the frequency of children’s heightened emotional displays, but it primarily targets positive emotions such as exuberance or excitement. The other two scales assess the display of emotions specifically in the social context. Socially Appropriate Affect assesses the frequency of children’s adaptive emotional responses in social interactions with adults and peers, while the frequency of atypical emotional displays in social situations is assessed by the Socially Incongruent Affect subscale. These new factors, and resulting subscales, expand the utility of the ERC beyond assessment of general emotional reactivity and regulation, to more nuanced evaluation of emotional responding, particularly in children with ADHD.

Emotion regulation, as described above, includes two component processes: the generation of an emotion and the subsequent modulation (e.g., up- or down-regulation) of that emotion across time. Considered within this framework, the Positive Emotion Lability and Negative Emotion Lability subscales predominately assess emotion generation, or the emergence of negative and positive emotional states. For example, they include items such as “Is easily frustrated” and “Displays exuberance that others find intrusive or disruptive.” These factors do not overtly reflect downstream regulation of these emotions. In fact, the items that are suggestive of regulation such as “Is able to delay gratification” and “Can modulate excitement in emotionally arousing situations” did not load on any factors. As a result, this new ERC factor structure does not overtly assess the regulation, or children’s ability to effectively down-regulate high arousal emotions after they emerge. At the same time, it must be acknowledged that the utility of questionnaire measures to distinguish between emotional reactivity and emotion regulation is inherently limited. For instance, a child with low scores on the Negative or Positive Emotion Lability subscales, may experience intrusively strong emotions infrequently, or may have strong regulation skills that allow them to efficiently modulate their emotions such that parents do not observe, and therefore do not report, heightened emotional reactivity. This is a challenge of using parent-report measures as they rely upon observation of the child’s behavior. However, self-report measures of emotion regulation for younger children are limited. For example, one frequently used self-report measure in children, the Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CA; Gullon & Taffe, 2012), was standardized in a sample of youth ages 10 to 18 years. In the research setting, physiological measures, such as electroencephalography (EEG), galvanic skin response (GSR) and heart rate variability (e.g., respiratory sinus arrythmia), assessed during emotion-eliciting tasks provide objective assessment of emotion processing at a timescale that allows for differentiation between emotion generation and regulation abilities. However, such measures have limited utility in the clinic; thus, refined parent-report assessment tools, such as the ERC, are needed.

Children with ADHD and ODD scored significantly higher on both Negative and Positive Emotion Lability scales than children with ADHD alone, indicating that the measure captures the increased difficulty with emotional reactivity among children with this comorbid presentation. It was expected that children with ADHD and ODD would exhibit higher scores on Negative Emotion Lability as ERC items such as “Is prone to angry outbursts / tantrums easily” overlap with symptoms of ODD. However, the significantly higher scores on Positive Emotion Lability suggest that increased reactivity in children with ADHD and ODD is not specific to negatively valenced emotions. Prior longitudinal studies indicate that exuberance and difficulty regulating positive emotions in toddlerhood predict disruptive behaviors as well as social competence in early childhood (Degnan et al., 2011; Rydell et al., 2003). Thus, having a have a brief measure of exuberance and positive emotional reactivity in children for the continued assessment of this domain has significant utility.

The Socially Appropriate Affect and Socially Incongruent Affect subscales provide a brief assessment of strengths and weaknesses in social-emotional functioning, that was not available in the original ERC factor structure. Although these scales had relatively lower, albeit still valid, internal consistencies than the other subscales, their potential clinical and empirical utility is significant given that difficulties in this domain are often present among children with ADHD (Nijmeijer et al., 2008; Nixon, 2001; Solanto et al., 2009). Children with ADHD and co-occurring ODD had lower scores on the Socially Appropriate Affect subscale and higher scores on the Socially Incongruent Affect subscale than children with ADHD alone, which is consistent with prior literature indicating that children with ODD exhibit atypical or cold emotional responses in social situations (Bunford et al., 2015). Further, as children with ADHD commonly demonstrate symptoms of autism spectrum disorders (Grzadizinski et al., 2011; Reiersen et al., 2007), this structure of the ERC allows clinicians to efficiently assess relevant social difficulties that may be important treatment targets. Of note, several items included in the Socially Incongruent Affect subscale could be interpreted to reflect irritability or dysphoria (i.e., “Responds negatively to neutral or friendly overtures by peers”; “Seems sad or listless”), and considering children’s scores from this perspective may provide useful clinical information indicating that further assessment of irritability or depressive symptoms is needed. Thus, the ERC may have utility in identifying how the quality of emotional responses to others may underlie broader social and/or emotional difficulties present in children with ADHD.

There are several limitations to the current study and areas for growth in future work. First, the study sample was predominately male. While this is consistent with the higher prevalence of externalizing disorders among boys (Copeland et al., 2011; Merikangas et al., 2010), the generalizability of the findings to female patients may be limited. Second, while the present clinical sample had valuable variation in emotion regulation abilities, the absence of a typically-developing group may limit the representation of the full range of emotion regulation abilities as potentially measured by the ERC and the current subscales may not capture the variation present in typically-developing samples. This may also contribute to the absence of correlations between subscale scores and age in the present sample. Third, there were clear advantages of utilizing two study samples such as increasing the sample size, expanding the age range, and enhancing sociocultural diversity and thus, improving the external validity of the findings. However, there was limited consistency in the measures used across sites aside from the ERC and the BASC PRS, which prevented additional psychometric evaluation of the new subscales. Future studies are needed to test the construct validity of these subscales using an established clinical measure completed by an independent reporter (e.g. teachers) or behavioral paradigms, as well as to examine their test-retest reliability. Fourth, the development of additional items to specifically assess regulation and increase the reliability of the social affect scales could also significantly strengthen the measure. Fifth, confirming this new factor structure in an independent sample with similar clinical characteristics would be informative. Further, multi-group confirmatory factor analysis with robust sample sizes would allow for a formal assessment of measurement invariance between ADHD without ODD and ADHD with ODD groups. Finally, confirming this factor structure in other populations with developmental psychopathology and neurodevelopmental disorders would expand the clinical and research utility of the measure beyond ADHD. Given the common co-occurrence of ADHD and ASD, future work examining this factor structure in this comorbid population or among children with ASD alone may provide valuable insights into emotion dysregulation in these populations. In addition, the Socially Appropriate Affect and Socially Incongruent Affect subscales could be compared between such clinical groups and in relation to extant measures of social emotional functioning, such as the Social Responsiveness Scale (SRS; Costantino & Gruber, 2012). Last, while the current retrospective study sample did not facilitate it, exploring differences in subscale scores between children with ADHD combined or predominately hyperactive/impulsive presentation and those with ADHD predominately inattentive presentation would be an interesting line of future research in light of theories suggesting these groups differ in emotion regulation and reactivity (Martel, 2009).

In sum, the present study identified four dimensions of the ERC in a sample of children with ADHD. The Positive and Negative Emotion Lability subscales provide an important assessment tool particularly relevant to children with ADHD who often exhibit functionally impairing difficulties regulating both positive and negative emotions. The Socially Appropriate Affect and Socially Incongruent Affect subscales extend beyond typical emotion regulation measures and tap into social domains that are also known to be impaired in children with ADHD, as well as in children with other commonly co-occurring disorders such as ODD and autism spectrum disorders. Leveraging the diverse ERC items in this manner could aid clinicians in specifying areas of difficulty in emotion functioning and identifying treatment targets. For instance, while results from the ERC would need to be considered in the larger context of the patient, if a parent strongly endorses items on the Negative and/or Positive Emotion Lability subscale, it may be indicated to work with the child to develop strategies to regulate strong emotions (e.g. identifying common triggers and generating practical alternatives to outbursts such as diaphragmatic breathing or reappraisal) and engage parents in the implementation of a reinforcement schedule for adaptive coping with strong emotions. In contrast, a child who is low on the Socially Appropriate Affect subscale may benefit more from a social skills group or role plays in individual sessions, while elevated scores on the Socially Incongruent Affect scale may suggest a need for further assessment of mood functioning and/or callous unemotional behavior. This new factor structure has the potential to enhance the utility of the ERC as a brief, yet multifaceted, measure of emotional functioning that can be used clinically to identify treatment targets, and empirically, to develop a more nuanced understanding of emotion regulation difficulties in clinical child populations.

Acknowledgments

This work was supported by the National Institute of Mental Health [grant numbers R01MH091140-01 to A.K.R.; R01MH112588 to P.G. and A.S.D.; and R56MH108616 to P.G. and A.S.D] and by the National Institute of Diabetes and Digestive and Kidney Diseases [grant number R01DK119814 to P.G. and A.S.D].

Footnotes

We have no conflicts of interest to disclose.

References

  1. Adrian M, Zeman J, & Veits G (2011). Methodological implications of the affect revolution: A 35-year review of emotion regulation assessment in children. Journal of Experimental Child Psychology, 110(2), 171–197. 10.1016/j.jecp.2011.03.009 [DOI] [PubMed] [Google Scholar]
  2. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: Author. [Google Scholar]
  3. Angold A, Costello EJ, & Erkanli A (1999). Comorbidity. Journal of Child Psychology and Psychiatry, 40(1), 57–87. 10.1111/1469-7610.00424 [DOI] [PubMed] [Google Scholar]
  4. Boyle GJ, Saklofske DH, & Matthews G (2015). Criteria for selection and evaluation of scales and measures. In Boyle GJ, Saklofske DH, & Matthews G (Eds.), Measures of personality and social psychological constructs (pp. 3–15). Academic Press. [Google Scholar]
  5. Bunford N, Evans SW, & Wymbs F (2015). ADHD and emotion dysregulation among children and adolescents. Clinical Child and Family Psychology Review, 18(3), 185–217. 10.1007/s10567-015-0187-5 [DOI] [PubMed] [Google Scholar]
  6. Copeland W, Shanahan L, Costello EJ, & Angold A (2011). Cumulative prevalence of psychiatric disorders by young adulthood: a prospective cohort analysis from the Great Smoky Mountains Study. Journal of the American Academy of Child & Adolescent Psychiatry, 50(3), 252–261. 10.1016/j.jaac.2010.12.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Constantino JN, & Gruber CP (2012). Social responsiveness scale: SRS-2. Torrance, CA: Western Psychological Services. [Google Scholar]
  8. Degnan KA, Hane AA, Henderson HA, Moas OL, Reeb-Sutherland BC, & Fox NA (2011). Longitudinal stability of temperamental exuberance and social–emotional outcomes in early childhood. Developmental Psychology, 47(3), 765–780. 10.1037/a0021316 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Dunsmore JC, Booker JA, Ollendick TH, & Greene RW (2016). Emotion socialization in the context of risk and psychopathology: Maternal emotion coaching predicts better treatment outcomes for emotionally labile children with oppositional defiant disorder. Social Development, 25(1), 8–26. 10.1111/sode.12109 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Eisenberg N, Fabes RA, Guthrie IK, Murphy BC, Maszk P, Holmgren R, & Suh K (1996). The relations of regulation and emotionality to problem behavior in elementary school children. Development and Psychopathology, 8(1), 141–162. 10.1017/s095457940000701x [DOI] [Google Scholar]
  11. Fabiano GA, Pelham WE, Waschbusch DA, Gnagy EM, Lahey BB, Chronis AM, Onyango AN, Kipp H, Lopez-Williams A, & Burrows-MacLean L (2006). A practical measure of impairment: Psychometric properties of the impairment rating scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology, 35(3), 369–385. 10.1207/s15374424jccp3503_3 [DOI] [PubMed] [Google Scholar]
  12. Fernandez KC, Jazaieri H, & Gross JJ (2016). Emotion regulation: A transdiagnostic perspective on a new RDoC domain. Cognitive Therapy and Research, 40(3), 426–440. 10.1007/s10608-016-9772-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Finch JF, & West SG (1997). The investigation of personality structure: Statistical models. Journal of Research in Personality, 31(4), 439–485. 10.1006/jrpe.1997.2194 [DOI] [Google Scholar]
  14. Goldstein S, & Naglieri JA (2009). Autism spectrum rating scales (ASRS). Multi-Health System. [Google Scholar]
  15. Graziano PA, & Garcia A (2016). Attention-deficit hyperactivity disorder and children’s emotion dysregulation: A meta-analysis. Clinical Psychology Review, 46, 106–123. 10.1016/j.cpr.2016.04.011 [DOI] [PubMed] [Google Scholar]
  16. Graziano PA, Slavec J, Hart K, Garcia A, & Pelham WE (2014). Improving school readiness in preschoolers with behavior problems: Results from a summer treatment program. Journal of Psychopathology and Behavioral Assessment, 36(4), 555–569. 10.1007/s10862-014-9418-1 [DOI] [Google Scholar]
  17. Grzadzinski R, Di Martino A, Brady E, Mairena MA, O’Neale M, Petkova E, Lord C & Castellanos FX (2011). Examining autistic traits in children with ADHD: does the autism spectrum extend to ADHD?. Journal of Autism and Developmental Disorders, 41(9), 1178–1191. 10.1007/s10803-010-1135-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gullone E, & Taffe J (2012). The Emotion Regulation Questionnaire for Children and Adolescents (ERQ–CA): A psychometric evaluation. Psychological Assessment, 24(2), 409. 10.1037/a0025777 [DOI] [PubMed] [Google Scholar]
  19. Hulvershorn LA, Mennes M, Castellanos FX, Di Martino A, Milham MP, Hummer TA, & Roy AK (2014). Abnormal amygdala functional connectivity associated with emotional lability in children with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 53(3), 351–361. 10.1016/j.jaac.2013.11.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp. [Google Scholar]
  21. Insel T, Cuthbert B, Garvey M, Heinssen R, Pine D, Quinn K, Sanislow C, & Wang P (2010). Research Domain Criteria (RDoC): Toward a toward a new classification framework for research on mental disorders. American Journal of Psychiatry, 167(7), 748–751. 10.1176/appi.ajp.2010.09091379 [DOI] [PubMed] [Google Scholar]
  22. Jennrich RI (1970). An asymptotic χ2 test for the equality of two correlation matrices. Journal of the American Statistical Association, 65(330), 904–912. 10.1080/01621459.1970.10481133 [DOI] [Google Scholar]
  23. Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, Williamson D, & Ryan N (1997). Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): Initial reliability and validity data. Journal of the American Academy of Child and Adolescent Psychiatry, 36(7), 980–988. 10.1097/00004583-199707000-00021 [DOI] [PubMed] [Google Scholar]
  24. MacCallum RC (1990). The need for alternative measures of fit in covariance structure modeling. Multivariate Behavioral Research, 25(2), 157–162. 10.1207/s15327906mbr2502_2 [DOI] [PubMed] [Google Scholar]
  25. Martel MM (2009). Research Review: A new perspective on attention-deficit/hyperactivity disorder: emotion dysregulation and trait models. Journal of Child Psychology and Psychiatry, 50(9), 1042–1051. 10.1111/j.1469-7610.2009.02105.x [DOI] [PubMed] [Google Scholar]
  26. Merikangas KR, He JP, Brody D, Fisher PW, Bourdon K, & Koretz DS (2010). Prevalence and treatment of mental disorders among US children in the 2001–2004 NHANES. Pediatrics, 125(1), 75–81. 10.1542/peds.2008-2598 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Meybodi FA, Mohammadkhani P, & Pourshahbaz A (2018). Psychometric properties of the Persian version of the Emotion Regulation Checklist. World Family Medicine Journal/Middle East Journal of Family Medicine, 16(2), 187–192. 10.5742/mewfm.2018.93260 [DOI] [Google Scholar]
  28. Molina P, Sala MN, Zappulla C, Bonfigliuoli C, Cavioni V, Zanetti MA, Baiocco R, Laghi F, Pallini S, De Stasio S, Raccanello D, & Cicchetti D (2014). The Emotion Regulation Checklist - Italian translation. Validation of parent and teacher versions. European Journal of Developmental Psychology, 11(5), 624–634. 10.1080/17405629.2014.898581 [DOI] [Google Scholar]
  29. Nijmeijer JS, Minderaa RB, Buitelaar JK, Mulligan A, Hartman CA, & Hoekstra PJ (2008). Attention-deficit/hyperactivity disorder and social dysfunctioning. Clinical Psychology Review, 28(4), 692–708. 10.1016/j.cpr.2007.10.003 [DOI] [PubMed] [Google Scholar]
  30. Nixon E (2001). The social competence of children with attention deficit hyperactivity disorder: A review of the literature. Child Psychology and Psychiatry Review, 6(4), 172–180. 10.1111/1475-3588.00342 [DOI] [Google Scholar]
  31. Pelham WE, Fabiano GA, & Massetti GM (2005). Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34(3), 449–476. 10.1207/s15374424jccp3403_5 [DOI] [PubMed] [Google Scholar]
  32. Pelham WE, Gnagy EM, Greenslade KE, & Milich R (1992). Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 31(2), 210–218. 10.1097/00004583-199203000-00006 [DOI] [PubMed] [Google Scholar]
  33. R Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/. [Google Scholar]
  34. Reiersen AM, Constantino JN, Volk HE, & Todd RD (2007). Autistic traits in a population-based ADHD twin sample. Journal of Child Psychology and Psychiatry, 48(5), 464–472. 10.1111/j.1469-7610.2006.01720.x [DOI] [PubMed] [Google Scholar]
  35. Reis AH, De Oliveira SES, Bandeira DR, Andrade NC, Abreu N, & Sperb TM (2016). Emotion Regulation Checklist (ERC): Preliminary studies of cross-cultural adaptation and validation for use in Brazil. Temas em Psicologia, 24(1), 77–96. 10.9788/TP2016.1-07 [DOI] [Google Scholar]
  36. Reynolds CR, & Kamphaus RW (2004). Behavior assessment system for children (2nd ed.). Bloomington, MN: NCS Pearson, Inc. [Google Scholar]
  37. Reynolds CR, & Kamphaus RW (2015). Behavior assessment system for children (3rd ed.). Bloomington, MN: NCS Pearson, Inc. [Google Scholar]
  38. Röll J, Koglin U, & Petermann F (2012). Emotion regulation and childhood aggression: Longitudinal associations. Child Psychiatry and Human Development, 43(6), 909–923. 10.1007/s10578-012-0303-4 [DOI] [PubMed] [Google Scholar]
  39. Roy AK, Bennett R, Posner J, Hulvershorn L, Castellanos FX, & Klein RG (2018). Altered intrinsic functional connectivity of the cingulate cortex in children with severe temper outbursts. Development and Psychopathology, 30(2), 571–579. 10.1017/S0954579417001080 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Rydell AM, Berlin L, & Bohlin G (2003). Emotionality, emotion regulation, and adaptation among 5-to 8-year-old children. Emotion, 3(1), 30–47. 10.1037/1528-3542.3.1.30 [DOI] [PubMed] [Google Scholar]
  41. Shaffer D, Fisher P, Lucas CP, Dulcan MK, & Schwab-Stone ME (2000). NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC- IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 39(1), 28–38. 10.1097/00004583-200001000-00014 [DOI] [PubMed] [Google Scholar]
  42. Shaw P, Stringaris A, Nigg J, & Leibenluft E (2014). Emotion dysregulation in attention deficit hyperactivity disorder. American Journal of Psychiatry, 171(3), 276–293. 10.1176/appi.ajp.2013.13070966 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Sheppes G, Suri G, & Gross JJ (2015). Emotion regulation and psychopathology. Annual Review of Clinical Psychology, 11(1), 379–405. 10.1146/annurev-clinpsy-032814-112739 [DOI] [PubMed] [Google Scholar]
  44. Shields A, & Cicchetti D (1997). Emotion regulation among school-age children: the development and validation of a new criterion Q-sort scale. Developmental Psychology, 33(6), 906–916. 10.1037/0012-1649.33.6.906 [DOI] [PubMed] [Google Scholar]
  45. Sjöwall D, Roth L, Lindqvist S, & Thorell LB (2013). Multiple deficits in ADHD: Executive dysfunction, delay aversion, reaction time variability, and emotional deficits. Journal of Child Psychology and Psychiatry, 54(6), 619–627. 10.1111/jcpp.12006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Solanto MV, Pope-Boyd SA, Tryon WW, & Stepak B (2009). Social functioning in predominantly inattentive and combined subtypes of children with ADHD. Journal of Attention Disorders, 13(1), 27–35. 10.1177/1087054708320403 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Suveg C, & Zeman J (2004). Emotion regulation in children with anxiety disorders. Journal of Clinical Child and Adolescent Psychology, 33(4), 750–759. 10.1207/s15374424jccp3304 [DOI] [PubMed] [Google Scholar]
  48. Timmerman ME, & Lorenzo-Seva U (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209–220. 10.1037/a0023353 [DOI] [PubMed] [Google Scholar]
  49. Wehmeier PM, Schacht A, & Barkley RA (2010). Social and emotional impairment in children and adolescents with ADHD and the impact on quality of life. Journal of Adolescent Health, 46(3), 209–217. 10.1016/j.jadohealth.2009.09.009 [DOI] [PubMed] [Google Scholar]
  50. Zeman J, Klimes-Dougan B, Cassano M, & Adrian M (2007). Measurement issues in emotion research with children and adolescents. Clinical Psychology: Science and Practice, 14(4), 377–401. 10.1111/j.1468-2850.2007.00098.x [DOI] [Google Scholar]

RESOURCES