The Early Childhood Irritability-Related Impairment Interview (E-CRI): A Novel Method for Assessing Young Children’s Developmentally Impairing Irritability

Abstract

Irritability is a substrate of more than one dozen clinical syndromes. Thus, identifying when it is atypical and interfering with functioning is crucial to the prevention of mental disorder in the earliest phase of the clinical sequence. Advances in developmentally based measurement of irritability have enabled differentiation of normative irritable mood and tantrums from indicators of concern, beginning in infancy. However, developmentally sensitive assessments of irritability-related impairment are lacking. We introduce the Early Childhood Irritability-Related Impairment Interview (E-CRI), which assesses impairment associated with irritable mood and tantrums across contexts. Reliability and validity are established across two independent samples varied by developmental period: the Emotional Growth preschool sample (EmoGrow; N=151, M=4.82years) and the When to Worry infant/toddler sample (W2W; N=330, M=14months). We generated a well-fitting two-factor E-CRI model, with tantrum- and irritable mood-related impairment factors. The E-CRI exhibited good interrater, test–retest, and longitudinal reliability. Construct and clinical validity were also demonstrated. In both samples, E-CRI factors showed association to internalizing and externalizing problems, and to caregiver-reported concern in W2W. Tantrum-related impairment demonstrated stronger and more consistent explanatory value across outcomes, while mood-related impairment added explanatory utility for internalizing problems. The E-CRI also showed incremental utility beyond variance explained by the Family Life Impairment Scale (FLIS) survey indicator of developmental impairment. The E-CRI holds promise as an indicator of impairment to inform identification of typical versus atypical patterns reflecting early emerging irritability-related syndromes in the initial phase of the clinical sequence.

Irritability is of high transdiagnostic relevance, serving as a substrate of more than a dozen Diagnostic and Statistical Manual of Mental Disorders (DSM) with substantial neurodevelopmental underpinnings representing alterations in negative valence systems (e.g., frustrative nonreward; Avenevoli, Blader, & Leibenluft, 2015; Kessel et al., 2016; Wakschlag et al., 2018). Irritability reflects relative dispositional tendencies toward angry affect, including the likelihood of reactivity to blocked goal attainment with frustrated outbursts (i.e., temper tantrums) and/or angry mood states (Wakschlag et al., 2018). This anger proneness is a normally distributed trait in the population, which is relatively stable yet also exhibits meaningful developmental change over time (Hawes et al., 2016; Wakschlag et al., 2015; Wiggins, Mitchell, Stringaris, & Leibenluft, 2014). Temper tantrums are the most salient expression of irritability in young children, as exhibited by excessive frustration reactions with concomitant decrements in emotion regulation capacity, but there is also emergent evidence that irritable mood may be dissociable early on (Roy et al., 2013). Distinguishing normative variation from clinically salient irritability is particularly challenging in early childhood (here defined as infancy–preschool age), as temper tantrums are normative expressions of frustration (Wakschlag et al., 2012). Irritability is also a core feature of early childhood disruptive behavior, which is one of the most common reasons for mental health referral, and is a contributor to the substantial increase in pharmacologic intervention with young children (Comer, Chow, Chan, Cooper, & Wilson, 2013; Wakschlag, Tolan, & Leventhal, 2010).

We have developed a suite of survey and observational tools—namely, the Multi-Dimensional Assessment Profile of Disruptive Behavior (MAP-DB) and the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS), to distinguish among the atypical behavioral features of irritability in young children in terms of frequency, context, and dysregulation (Wakschlag et al., 2008, 2012). Using these tools, dimensional and contextual patterning of irritability have demonstrated clinical and predictive utility (Frost, Jelinek, Bernard, Lind, & Dozier, 2018; Wakschlag et al., 2015; Wiggins et al., 2018) and similar patterns have been detected using alternative methods and samples (Belden, Thompson, & Luby, 2008; Carlson, Danzig, Dougherty, Bufferd, & Klein, 2016; Dougherty et al., 2013). When developmentally specified features of irritability, such as context and dysregulation, are incorporated in early childhood assessment, this sharpens typical–atypical distinctions and also predicts internalizing and externalizing syndromes at older ages, as well as functional disruptions in prefrontal regions, such as the lateral prefrontal and anterior cingulate cortices and striatum, and decrements in error monitoring (Althoff et al., 2017; Deveney et al., 2013; Grabell et al., 2017; Kessel et al., 2016; Li, Grabell, Wakschlag, Huppert, & Perlman, 2017; Roy et al., 2017).

The critical missing piece in this developmentally based toolkit for determining clinically salient patterns of irritability is corollary determination of impairment based on interference with developmentally expectable capacities (e.g., increased frustration tolerance as children develop). Functional impairment reflects the extent to which patterns of symptoms and behavior restrict an individual’s ability to fulfill expected roles and participate in life activities (Angold & Costello, 2000; World Health Organization, 2001). In the case of young children, functional impairment can also manifest in disruptions in the family’s functioning (Mian, Soto, Briggs-Gowan, & Carter, 2018). This is an area that has received scant attention in studies of the clinical salience of young children’s irritability (Biedzio & Wakschlag, 2019; Bufferd, Dyson, Hernandez, & Wakschlag, 2016). Given the high rates of normative variability in early childhood, impairment is particularly important to examine as it may pinpoint those young children at the outset of a clinical pathway and reduce the rate of false positives (Healey, Miller, Castelli, Marks, & Halperin, 2008). Further, dimensional approaches, while critical to capturing clinical phenomenology within a neurodevelopmental framework, identify dynamic probabilistic risk rather than static clinical conditions. This makes characterization of impairment particularly crucial, as there are young children for whom irritability is maladaptive versus those who have compensatory or coping strategies that result in adaptive functioning despite this dispositional tendency. For example, we have shown that impaired versus nonimpaired irritable children differ in effectiveness of employing prefrontal resources to manage frustration (Grabell et al., 2017).

Using general indicators of impairment, several studies have shown that heightened irritability is associated with functional impairment and growth of impairment over time (Dougherty et al., 2015; Wiggins et al., 2018). Tantrum severity is also a major driver of impairment (Carlson et al., 2016). However, impairment in these studies has not been assessed in an irritability-specific manner, and as a result, may emanate from co-occurring clinical features and/or more general problems in developmental functioning that often co-occur (e.g., language; Roberts et al., 2018). The Preschool Age Psychiatric Assessment (PAPA; Egger & Angold, 2004), the only comprehensive diagnostic interview validated for preschoolers, assesses developmental impairment, but training and administration are time-intensive and proprietary, and impairment is not irritability specific.

Thus, there is a need for a relatively brief, irritability-specific, developmentally based, non-proprietary impairment interview that can be used with a variety of clinical assessment tools (i.e., the E-CRI is meant as a companion to any parent report or observed measures of clinically salient irritability in young children). We believe that assessment of impairment within context is particularly important based on prior work on the clinical salience of contextual variation in irritability and related behaviors (Dirks, De Los Reyes, Briggs Gowan, Cella, & Wakschlag, 2012; Petitclerc et al., 2015). We also build on theorized differential mechanisms related to impairments in neural structure and function that govern frustrative nonreward and related behavioral dysregulation versus chronic irritable mood state (Grabell et al., 2017; Leibenluft, 2017). Explication of whether these phenotypic differences are related to variations in impairment would be informative toward advancing this more finely parsed approach to irritability. We have also found clinically that it is difficult for some parents to make subtle distinctions in terms of which features of irritable mood and tantrums are “responsible” for interference with functioning, particularly with brief survey methods. Developmentally based, semistructured standardized methods operationalize clinical judgment in a manner that enables use by clinical and research interviewers, including (a) developmentally grounded training and anchors that establish what is normative and what is atypical at varying ages, and (b) ordinal ratings that specify a continuum of concern rather than relying on clinician inference. This structure is designed to “get the best of both worlds” (i.e., to integrate clinical knowledge while at the same time obtaining equivalent data across children; established methodologically via interrater reliability; Wakschlag et al., 2005; Westen & Weinberger, 2004), and (c) using well-defined anchor points based on integrative judgments that reflect an overall assessment of the atypicality of the child’s behavior. This goes beyond the presence or absence of discrete behaviors to weigh the quality of behavior and its salience based on age appropriateness and context (Wakschlag et al., 2005).

We hypothesized that the development of an irritability-specific interview method that allows for probing of such nuanced distinctions would have added value for characterization of patterns of irritability and the extent to which they interfere with developmental adaptation, relative to survey impairment measures alone. Further, we postulate that such differentiation is not only scientifically salient but, ultimately, will be highly informative for tailored treatment planning. The “healthier, earlier” imperative to enhance early identification of probabilistic risk at the earliest phase of the clinical sequence (Finlay-Jones et al., 2019; Wakschlag et al., 2019), along with increasing evidence that clinically salient patterns of irritability are identifiable by 12 months of age (Biedzio & Wakschlag, 2019), point toward the need for in-depth assessments of impairment in a clinically meaningful manner from the age 1 year.

To this end, we developed the Early Childhood Irritability-Related Impairment Interview (E-CRI). The E-CRI is a semistructured interview designed to assess early childhood impairment emanating from young children’s tantrums and irritable moods across varied contexts. Our goal was to develop an interview that could be flexibly deployed in clinical, research, and practice settings for individuals at varied levels of training (i.e., would not require a licensed clinician). Irritability-related, developmentally based impairment refers to the extent to which irritability (a) interferes with the child’s and family’s ability to engage in normal and expectable activities of daily living and achieve developmental tasks (e.g., self-control, cooperative play) and/or (b) detracts from harmonious relationships (e.g., walking on eggshells, aversive interactions).

Method

Overview

The reliability and preliminary validity of the E-CRI was tested in two independent early childhood irritability studies: (a) the preschool-age Emotional Growth study (EmoGrow; N = 151) and (b) the infant/toddler When to Worry study (W2W; N = 330; Fishburn et al., 2019; Quinones-Camacho, Fishburn, Camacho, Wakschlag, & Perlman, 2019; Note: The full W2W cohort comprises 350 infant/toddlers; 20 children were excluded from the analytic sample due to missing E-CRI data). Both samples were demographically diverse (see Table 1). At enrollment, the mean age of participants was 4.8 years in EmoGrow (range 3.8–5.9) and 14 months in W2W (range 12–18 months). Each sample provided information about E-CRI’s psychometric structure. The EmoGrow study enabled testing of E-CRI’s test–retest reliability (n = 30) and longitudinal stability (n = 142). The W2W study enabled testing of the E-CRI’s incremental utility relative to a survey measure of impairment.

Table 1.

Emotional Growth (EmoGrow) Study and When to Worry (W2W) Study Sample Descriptives

	EmoGrow (n = 151)		W2W (n = 330)
Variable	n	%	n	%
Child sex
Female	71	47.0	153	46.4
Male	80	53.0	177	53.6
Racea
Non-Hispanic White	97	64.7	137	41.5
Hispanic	7	4.7	85	25.8
Non-Hispanic African American	34	22.7	76	23.0
Other	12	8.0	32	9.7
Income
Less than $10,000	0	0.0	34	10.4
$10,000-$24,999	30	19.9	35	10.7
$25,000-$49,999	23	15.2	55	16.8
$50,000-$99,999	52	34.4	71	21.7
$100,000 or more	46	30.5	132	40.4
	Mean	SD	Mean	SD
Age (years)a	4.82	0.60	1.17	0.14

^aIndicates sample differences at p < .001.

Assessments

The Early Childhood Irritability-Related Impairment Interview

The E-CRI was developed based on decades of clinical and research experience with young irritable children by the authorship team. It is a semistructured parent interview designed to capture variation in the extent to which young children’s irritable behavior and mood interferes with developmental and social functioning. To ensure that interference is more than a transient perturbation (e.g., due to fussiness related to teething), the E-CRI allows for waxing and waning of behavior within a 3-month recall period. This time frame was based on prior evidence that it is one that parents can easily and accurately recall (Norquist, Girman, Fehnel, DeMuro-Mercon, & Santanello, 2012). This is in contrast to most developmental survey measures (such as the MAP-DB and FLIS, described below), which use a 1-month reporting period. The expanded recall period of the E-CRI interview mirrors that of other early childhood interviews (Egger & Angold, 2004), reflecting enhancement of recall with interviewer probing. The interview begins with open-ended probes to assess the general presence and interference of the child’s irritability, followed by more in-depth probing of specific instances of impairment related to irritable moods and tantrums in different contexts. Each context has multiple questions to elicit greater specificity. Because little is known about how the various expressions of irritability link to functional impairment and how that varies across contexts, the E-CRI probes about tantrum- and irritable mood-related impairment across six different settings and relationship contexts (i.e., home, out and about, with siblings, with peers, with nonparental adults, and in school/day care). It was designed to assess irritability-related impairment from infancy–preschool age with attentiveness to varied capacities across these age periods. Interview structure and ratings are parallel across these periods. Variation in typical settings and expectations for younger versus older children were incorporated to ensure developmental face validity (e.g., toddlers: “participate in household routines like changing diapers”; preschoolers: “expected activities like cleanup”).

E-CRI Training and Administration.

The E-CRI takes approximately 30 minutes to administer. Interviews were administered and coded by research assistants (RAs) with a bachelors’ degree or higher level of education. RAs were trained to reliability by author anderson, a licensed clinical psychologist with expertise in early childhood clinical assessment. Training included a developmental and clinical conceptual overview of irritability and impairment, emphasizing developmentally normative versus atypical expressions of tantrums and moods across the early childhood period, and varying developmental capacity at each age period. The E-CRI defines developmentally based impairment as the extent to which the child’s tantrums and irritable mood interfere with the child’s and family’s ability to engage in mutual relationships, participate in normal and expectable activities of daily living, and achieve developmental tasks. Scores are rated from 0 to 3 with 0–1 considered to be no or mild (normatively expectable) impairment and 2–3 of clinical concern indicating substantial or near complete impairment (see Supplemental Table 1 for rating descriptors and interview introduction).

Once general training was completed, RAs rated taped interviews to achieve reliability (80% exact agreement for at least five interviews). Three RAs administered and coded the interviews in W2W, and one RA administered and coded the EmoGrow interviews. Interviews were generally coded in real time or shortly afterward if review was necessary. Ongoing reliability discussions prevented coder drift, with double coding by EA, who served as the criterion.

For each setting, the first questions were open-ended and inquired about the extent to which the child’s tantrums and irritable mood interfered with functioning. Based on our prior findings of sociocultural and individual variation in parents’ use of terms to describe irritable behavior in young children, irritability descriptors were tailored to match the terms provided by parents at the outset of the interview (e.g., “tantrum” and “grumpy mood” to “having a fit” and “showing attitude,” respectively). The semistructured nature of the interview enabled parents to recount descriptions using their own words. If spontaneously reported, interviewers then probed (e.g., “Has there ever been a time, frequency, duration, intensity, and most intense behavior?”). Interviewers flexibly employed probes about specific impacts, family accommodations, and/or contexts. Probes were used to draw out examples and to provide information needed to make a judgment about impairment (see Figure 1).

FIGURE 1.

Illustration of the Decision Tree for scoring parental responses in the Early Childhood Irritability-Related Impairment Interview (E-CRI). General Decision Tree: 1. Starting Probe: Please tell us about ways in which… This question looks to the general picture and how the behaviors/moods can affect the situation and people in it. The probe allows the parent to respond without thinking about answering a specific question; they are allowed to start the conversation however they see fit. It gives us clues as to what to focus on in the directed questions. 2. Behavior/Mood Specific Probes: Start with general questions about whether outbursts occur in the particular context, then delve into the specifics of what happens, how often, how long, and how intense, as well as who is it affecting and what is the transition out of the outburst like. Extra detail is necessary when a situation seems vague or the parent is not giving enough detail for you to be able to confidently score. Some questions could include asking how the child can calm down from outbursts/moods, how these affect the others, whether the child has varying levels of outbursts/moods, whether the parent is able to see the outbursts/moods coming, or whether these seem to occur out of nowhere or for no reason. 3. Summarizing Questions: All questions get asked regardless of how the parent responds to the starting probe or the specific questions about behavior/mood, as this provides a fuller picture and offers different situations of which parents may not have thought about. If the parent answers yes to any of the general questions, ask about frequency and for more context (when/how it affects, which person, etc.). These questions might lead you back to the general outburst/mood sections for more situations/detail. SCORING: Scoring factors include frequency, intensity, length of time, pervasiveness, and disruptiveness. All these factors are taken into consideration when scoring. The context and pervasiveness are very important for determining severity of outbursts/moods. Frequency, intensity, and length of time are key points to determining the disruptiveness of the outbursts/moods. Also, keep the normative expressions of anger/frustration in mind when scoring. A child who has a bad mood once a week when tired is different from a child who has a bad mood three to four times per week when told to go to bed.

Impairment ratings consisted of integrative judgments in which raters used all the information available to assess how severely irritability interfered with child and/or family functioning. Quantitative (frequency) and qualitative (intensity and duration) dimensions, pervasiveness, and whether mood or behavior led to avoidance or modification of typical activities or routines were considered when arriving at an impairment severity rating. Parental perception of level of severity was integrated into the ratings but was not the sole basis on which ratings were made. This enabled raters to account for parental framing that might be under- or overreporting impairment based on concrete behavioral examples that belie parental judgment about interference. For example, a parent who reports little impairment when asked directly about interference but in response to probing describes interference so substantial that the family has accommodated this by avoiding or reducing opportunity (e.g., no tantrums in the grocery store because dysregulation precludes taking the child with them). Thus, children with different patterns may receive the same rating since it reflects the extent of interference, not the specific behavior.

For example, a moderate to high impairment score (rating of 2 or 3) would be given to both (a) a child who is reported to have severe but infrequent tantrums because the family actively avoids contexts that set the child off, even if the parent downplays the impact of these tantrums; as well as to (b) a child with mild to moderate irritability that is regularly causing family distress in a parent who has a low tolerance for misbehavior. Such profile heterogeneity also mirrors the varied factors weighted in clinical decision making (i.e., children who present differently are identified as having the same “problem”). Figure 1 highlights the impairment decision tree.

Validating measures

Irritability

The MAP-DB temper loss subscale served as our indicator of irritability severity. Parents rated their child’s irritability using the temper loss subscale of the MAP-DB, which assesses behavior over the past month (Wakschlag et al., 2012). The EmoGrow study utilized the MAP-DB preschool version and the W2W study utilized the MAP-DB infant/toddler version (Biedzio & Wakschlag, 2019). These versions are highly overlapping with minor modifications to the original preschool version to more fully cover expressions of irritability in younger children (e.g., addition of crying items). Importantly, the temper loss subscale captures variations in quality, intensity, and context of irritability. To enhance normal–abnormal differentiation, items are rated on an objective frequency 6-point Likert scale. The MAP-DB has been extensively validated at preschool age (an age at which normal–abnormal differentiation is especially challenging; Wakschlag et al., 2018), and the infant/toddler, school-age, and adolescent versions have been validated more recently to enable developmentally meaningful dimensional measurement across ages (Biedzio & Wakschlag, 2019; Kaat et al., 2018). A set of shared items, consistent across age periods, is supplemented with developmentally specific items to enable cross-age and longitudinal modeling. Summary scores had high internal consistency across samples (α = .96).

Outcomes

Each sample utilized a well-validated measure of internalizing/externalizing problems, with measurement variation reflecting differences in the age of the samples. The Child Behavior Checklist (CBCL 1.5–5) was used in the EmoGrow preschool sample as an indicator of internalizing and externalizing problems (Achenbach & Rescorla, 2000). In the W2W, the Brief Infant Toddler Social Emotional Assessment (BITSEA), was used to generate internalizing/dysregulation and externalizing scales (α = .71 and .54, respectively; Briggs-Gowan, Carter, Irwin, Wachtel, & Cicchetti, 2004). As an additional outcome of clinical salience in W2W, we created a caregiver concern index based on the mean of three survey items about caregiver worry regarding the child’s behavior and mood (α = .75; Briggs-Gowan et al., 2013).

Impairment

In addition to the E-CRI, the W2W study included the Family Life Impairment Scale (FLIS), a well-validated survey measure of developmental impairment (Mian et al., 2018), enabling a test of E-CRI interview method incremental utility over and above a survey measure of impairment. The FLIS assesses the extent to which children’s emotions and behavior interfere with developmentally expected participation and functioning over a 1-month period (Mian et al., 2018). We created a FLIS summed score to reflect multiple child and family contexts of impairment (α = .73).

Validation samples

The E-CRI validation study derives from two independent samples (see Table 1). The EmoGrow sample comprises 151 preschoolers recruited from the community via flyers and ads on public transportation. The W2W sample comprises 330 infants/toddlers, recruited via screening, social media, ads, and flyers. Samples were broadly comparable in terms of child sex, but differed by race/ethnicity with W2W including more Hispanic participants and EmoGrow including more non-Hispanic White participants, χ²(3) = 35.6, p < .0001, reflecting regional differences.

Results

Modeling overview

Evaluation of Measurement Structure

We first fit a series of structural equation models to reduce the E-CRI’s impairment ratings to interpretable summary scores. The multitrait multimethod (MTMM) framework (Kenny & Kashy, 1992), which is used when every variable reflects both a trait and a method or domain of assessment, was extended for the present purpose of parsing trait (i.e., impairment related to mood or tantrums) and domain (i.e., context). A feature of MTMM is that trait factors are free to correlate with one another, whereas method factors are assumed to be independent conditional on traits. We are using an MTMM approach to account for the fact that individual items vary both by trait and context; ignoring these structural interrelationships could bias measurement. This does not suggest that variation in behavior across contexts based on same reporter is equivalent to variation in behavior across contexts in approaches when reporters are different (as in the more traditional MTMM; Kenny & Kashy, 1992). Here the MTMM approach accounts for systematic variation by trait and context.

We first fit MTMM models separately, then tested for factorial invariance using a multiple group version of this model. All structural equation models were fit using Mplus (Muthén & Muthén, 2015) and all other analyses were conducted using SAS (2003). Factor scores were generated from the final retained versions of each MTMM model and used in all subsequent analyses (see Supplemental Figure 1 for path model).

Tests of Reliability and Validity

We evaluated 2- to 4-week test–retest reliability (n = 30) and 1-year longitudinal stability (n = 142) in the EmoGrow sample, and interrater reliability in both samples via kappa statistics for item-level ratings and correlations for full scales. We then tested the validity of the E-CRI in terms of bivariate associations with related constructs, clinical validity in univariate multiple regressions examining association to internalizing/externalizing outcomes and caregiver concern, and incremental utility over and above the FLIS survey in W2W. All regression analyses controlled for child age and gender. We tested the contribution of each E-CRI domain (i.e., tantrum and mood) separately and jointly.

All items showed the highest endorsement rates in the “no” (0) and “minimal” (1) impairment categories, with more severe impairment categories used much more sparingly. Items were more consistent within context than within domain. EmoGrow preschoolers were rated as significantly more impaired than W2W infants across all domains and contexts, which may reflect the greater demands placed at older ages (and hence, greater opportunity for functional impairment). Contexts showed a consistent ordering across samples in terms of frequency of endorsement: Impairment at home was most frequently endorsed, whereas impairment with peers was the least frequently endorsed. Impairment ratings for the school/child care context were missing for a large portion of the sample, thus this context was omitted in subsequent analyses (see Supplemental Table 2 for rating distribution).

Measurement Structure Evaluation

MTMM Models

We first fit MTMM models to W2W and EmoGrow samples separately (see Table 2). MTMM models were selected because of their ability to handle the method variance in this scale. Unidimensional models with a single combined E-CRI trait showed universally poor fit (W2W: CFI = .894, RMSEA = .101; EmoGrow: CFI = .851, RMSEA = .168). In contrast, MTMM models with separate mood- and tantrum-related impairment “trait” factors and context-specific “method” factors showed excellent fit to the data (CFIs > .99, RMSEAs = .022, .015). Mood and tantrum latent factors were moderately correlated in both W2W (r = .580) and EmoGrow (r = .513) samples. Multiple group models showed no loss of fit when factor loadings were constrained across samples (${\text{χ}}^{2_8}=5.57$, p = .695; see Table 2). This allowed us to make comparisons between scores across samples (Meredith, 1993). Given the measurement invariance and excellent fit of the two-factor model, separate mood- and tantrum-related E-CRI impairment factors were used in all subsequent models.

Table 2.

Fit Statistics for MTMM Models by Sample^a

Model	CFI	TLI	RMSEA	Mood and tantrum-related impairment factor correlation
EmoGrow—single trait	.851	.791	.168
W2W—single trait	.894	.841	.101
EmoGrow—mood and tantrum-related impairment factors	.999	.998	.015	.513
W2W—mood and tantrum-related impairment factors	.995	.992	.022	.580
Combined—free	1.000	1.000	.005	.573 (W2W) .464 (EmoGrow)
Combined—invariant	1.000	1.002	.000	.573 (W2W) .464 (EmoGrow)

Note. MTMM = multitrait multimethod; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; WTW = When to Worry. Constraining factor loadings to invariance yields a nonsignificant loss of fit (${\text{χ}}^{2_8}=5.57$, p = .695).

^aModel fit statistics from structural equation models of the Early Childhood Irritability-Related Impairment Interview (E-CRI). The two combined data models are fit as multiple group models, with EmoGrow data comprising one group and W2W data comprising the other.

Reliability and Validity

Interrater Reliability (Both Samples)

A subset of interviews were double coded for interrater reliability by the clinical co-author (12% for W2W and 21% for EmoGrow). Interrater reliability was high across domains and contexts for both samples, both for total scores and individual items. Tantrum- and mood-related impairment scores correlations ranged from .94 to .97 in both samples. Individual rating reliability ranged from K = .63 to 1.00 across samples.

Test–Retest and Longitudinal Reliability (EmoGrow)

Reliability was high for both tantrum- and mood-related impairment. Over a 2- to 4-week period (mean = 17.93 days, range: 14–28 days), test–retest intraclass correlation coefficients (ICCs) were .69 and .73 for tantrum- and mood-related impairment scores, respectively (ps < .001). One year stability was moderate, ICCs of .59 and .49 (ps < .001) for tantrum- and mood-related impairment, respectively.

Construct Validity (Both Samples)

In both samples, E-CRI impairment domain scores were significantly associated with like constructs in the expected direction (see Table 3a–b). Associations were generally stronger to E-CRI tantrum- versus mood-related impairment across both samples. Age and impairment were not strongly related (mean: r = .02).

Table 3.

Construct Validity of the E-CRI: Bivariate Associations

a. EmoGrow Preschool Sample
	E-CRI tantrum-related impairment	E-CRI mood-related impairment	MAP-DB temper loss	CBCL externalizing	CBCL internalizing
E-CRI tantrum-related impairment	1.000
E-CRI mood-related impairment	.612	1.000
MAP-DB temper loss total	.634	.491	1.000
CBCL externalizing	.518	.329	.601	1.000
CBCL internalizing	.365	.325	.532	.588	1.000
b. W2W Infant/Toddler Sample
	E-CRI tantrum-related impairment	E-CRI mood-related impairment	MAP-DB temper loss total	FLIS impairment score	Caregiver-reported concern index	BITSEA externalizing	BITSEA internalizing
E-CRI tantrum-related impairment	1.000
E-CRI mood-related impairment	.614	1.000
MAP-DB temper loss total	.497	.351	1.000
FLIS impairment score	.480	.350	.574	1.000
Caregiver-reported concern index	.457	.322	.616	.461	1.000
BITSEA externalizing	.419	.307	.558	.537	.392	1.000

Note. E-CRI = Early Childhood Irritability-Related Impairment Interview; MAP-DB = Multidimensional Assessment Profile of Disruptive Behavior; CBCL = Child Behavior Checklist. All ps < .001.

Note. Ns vary based on missing data (n = 291–330); E-CRI = Early Childhood Irritability-Related Impairment Interview; MAP-DB = Multidimensional Assessment Profile of Disruptive Behavior; FLIS = Family Life Impairment Scale; BITSEA = Brief Infant Toddler Social Emotional Assessment. All ps < .001.

Clinical Validity (Both Samples)

In univariate multiple regressions, the clinical validity of the E-CRI was demonstrated in relation to externalizing and internalizing outcomes, with patterns broadly comparable across samples (see Table 4a–b and Table 5a–b). When E-CRI domains were considered singly, both domains were significantly associated with externalizing and internalizing problems. However, when E-CRI domains were jointly considered, tantrum-related impairment demonstrated a stronger and more consistent association to both internalizing and externalizing outcomes. In contrast, E-CRI mood-related impairment only explained independent variance for internalizing problems. Tantrum-related impairment was also strongly associated with caregiver-reported concern in joint models (see Table 5c).

Table 4.

Clinical Validity of the E-CRI: EmoGrow Preschool Sample

a. CBCL Externalizing Problems
	Baseline				E-CRI mood factor only				E-CRI tantrum factor only				E-CRI tantrum and mood factors
Parameter	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β
Intercept	50.636	6.112	<.001		48.833	5.790	<.001		45.359	5.301	<.001		45.389	5.317	<.001
Age	−1.173	1.270	.357	−.076	−.887	1.202	.462	−.058	−.100	1.101	.927	−.007	−.114	1.105	.918	−.007
Gender	1.090	1.531	.478	.118	1.884	1.458	.198	.203	1.284	1.316	.331	.139	1.355	1.333	.311	.146
E-CRI tantrums									1.391	.190	<.001	.519	1.336	.241	<.001	.499
E-CRI mood					1.035	.239	<.001	.339					.102	.275	.711	.033
R2		.008				.120				.273				.273
b. CBCL Internalizing Problems
	Baseline				E-CRI mood factor only				E-CRI tantrum factor only				E-CRI tantrum and mood factors
Parameter	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β
Intercept	42.214	6.474	<.001		40.331	6.143	<.001		38.129	6.072	<.001		38.282	6.035	<.001
Age	1.011	1.345	.453	.062	1.310	1.275	.306	.081	1.842	1.261	.146	.113	1.770	1.254	.160	.109
Gender	−0.309	1.622	.849	−.032	.520	1.547	.737	.053	−.158	1.507	.916	−.016	0.206	1.513	.892	.021
E-CRI tantrum-related impairment									1.077	.218	<.001	.380	.795	.274	.004	.281
E-CRI mood-related impairment					1.081	.253	<.001	.335					.526	.312	.095	.163
R2		.004				.114				.146				.162

Note. CBCL = Child Behavior Checklist; E-CRI = Early Childhood Irritability-Related Impairment Interview.

Note. CBCL = Child Behavior Checklist; E-CRI = Early Childhood Irritability-Related Impairment Interview.

Table 5.

Clinical Validity of the E-CRI: W2W Infant/Toddler Sample

a. BITSEA Externalizing Problems
	Baseline				E-CRI mood factor only				E-CRI tantrum factor only				E-CRI tantrum and mood factors
Parameter	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β
Interceptl	−.837	.796	.294		−.418	.768	.586		.148	.742	.842		.148	.741	.842
Age	.160	.056	.004	.165	.133	.054	.014	.136	.092	.052	.079	.094	.092	.052	.077	.095
Gender	.501	.195	.011	.294	.444	.188	.018	.261	.491	.179	.006	.289	.477	.179	.008	.280
E-CRI tantrum-related impairment									.366	.049	<.001	.399	.324	.060	<.001	.353
E-CRI mood-related impairment					.243	.047	<.001	.284					.065	.056	.244	.076
R2		0.052				.132				.207				.210
b. BITSEA Internalizing/Dysregulation Problems
	Baseline				E-CRI mood factor only				E-CRI tantrum factor only				E-CRI tantrum and mood factors
Parameter	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β
Intercept	1.091	1.435	.448		2.042	1.348	.131		2.752	1.354	.043		2.754	1.333	.040
Age	.181	.101	.074	.105	.117	.095	.215	.068	.065	.095	.496	.038	.066	.093	.478	.039
Gender	−.023	.352	.948	−.008	−.146	.330	.659	−.049	−.035	.327	.916	−.012	−.103	.323	.749	−.034
E-CRI tantrum-related impairment									.612	.089	<.001	.380	.405	.109	<.001	.251
E-CRI mood-related impairment					.546	.083	<.001	.362					.324	.101	.001	.215
R2		.011				.140				.151				.180
c. Caregiver-Reported Concern Index
	Baseline				E-CRI mood factor only				E-CRI tantrum factor only				E-CRI tantrum and mood factors
Parameter	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β
Intercept	−.341	.208	.103		−.226	.199	.257		−.072	.190	.705		−.072	.190	.707
Age	.041	.015	.005	.164	.034	.014	.017	.134	.022	.013	.093	.089	.022	.013	.092	.090
Gender	−.006	.051	.907	−.014	−.023	.049	.636	−.053	−.010	.046	.828	−.023	−.014	.046	.765	−.031
E-CRI tantrum-related impairment									.104	.012	<.001	.442	.094	.016	<.001	.396
E-CRI mood-related impairment					.068	.012	<.001	.310					.017	.014	.240	.077
R2		.027				.122				.216				.220

Note.BITSEA = Brief Infant Toddler Social Emotional Assessment; E-CRI = Early Childhood Irritability-Related Impairment Interview.

Note.BITSEA = Brief Infant Toddler Social Emotional Assessment; E-CRI = Early Childhood Irritability-Related Impairment Interview.

Note. E-CRI = Early Childhood Irritability-Related Impairment Interview.

Incremental Validity (W2W)

When E-CRI incremental utility was considered over and above the FLIS survey, the E-CRI consistently explained unique variance in models where each of its domains were separately considered (see Table 6a–c). The FLIS also contributed unique variance in all models. When E-CRI domains were jointly considered, incremental utility varied by outcome: E-CRI tantrum-related impairment added incremental utility to BITSEA externalizing problems and caregiver-reported concern (see Table 6a and 6c), whereas E-CRI mood-related impairment added value for BITSEA internalizing/dysregulation problems (see Table 6b).

Table 6.

Incremental Utility of the E-CRI: W2W Infant/Toddler Sample

a. BITSEA Internalizing/Dysregulation Problems
	Baseline				E-CRI mood factor only				E-CRI tantrum factor only				E-CRI tantrum and mood factors
Parameter	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β
Intercept	.424	1.236	.732		1.077	1.214	.376		1.297	1.250	.300		1.345	1.235	.277
Age	.074	.087	.396	.043	.052	.085	.544	.030	.037	.087	.667	.022	.040	.086	.645	.023
Gender	−.122	.303	.687	−.041	−.181	.296	.542	−.060	−.112	.299	.708	−.037	−.166	.296	.575	−.055
FLIS	.236	.023	<.001	.515	.203	.024	<.001	.443	.199	.026	<.001	.433	.193	.026	<.001	.420
E-CRI tantrums									.283	.092	.002	.176	.124	.106	.246	.077
E-CRI mood					.322	.079	<.001	.213					.266	.093	.004	.176
R2		.272				.311				.295			.315
b. BITSEA Externalizing Problems
	Baseline				E-CRI mood factor only				E-CRI tantrum factor only				E-CRI tantrum and mood factors
Parameter	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β
Intercept	−1.250	.679	.066		−1.038	.679	.128		−.695	.683	.310		−.689	.684	.314
Age	.102	.048	.033	.105	.095	.048	.046	.098	.079	.047	.096	0.81	.079	.047	.095	.082
Gender	.431	.166	.010	.253	.412	.165	.013	.242	.437	.163	.008	.257	.431	.164	.009	.253
FLIS	.135	.013	<.001	.519	.124	.013	<.001	.477	.111	.014	<.001	.427	.111	.014	<.001	.425
E-CRI tantrums									.180	.050	<.001	.197	.161	.059	.007	.176
E-CRI mood					.105	.044	.018	.122					.032	.051	.539	.037
R2		.317				.330				.347				.347
c. Caregiver-Reported Concern Index
	Baseline				E-CRI mood factor only				E-CRI tantrum factor only				E-CRI tantrum and mood factors
Parameter	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β	Est.	SE	P	β
Intercept	−.438	.187	.020		−.359	.185	.053		−.229	.183	.212		−.227	.184	.217
Age	.028	.013	.031	.113	.026	.013	.047	.103	.020	.013	.119	.079	.020	.013	.118	.079
Gender	−.021	.046	.653	−.047	−.028	.045	.529	−.065	−.019	.044	.669	−.043	−.021	.044	.633	−.048
FLIS	.030	.004	<.001	.449	.026	.004	<.001	.388	.021	.004	<.001	.311	.021	.004	<.001	.308
E-CRI tantrum-related impairment									.070	.013	<.001	.294	.063	.016	<.001	.268
E-CRI mood-related impairment					.039	.012	.001	.178					.010	.014	.448	.047
R2		.225				.253				.291				.292

Note.BITSEA = Brief Infant Toddler Social Emotional Assessment; E-CRI = Early Childhood Irritability-Related Impairment Interview; FLIS = Family Life Impairment Scale.

Note.BITSEA = Brief Infant Toddler Social Emotional Assessment; E-CRI = Early Childhood Irritability-Related Impairment Interview; FLIS = Family Life Impairment Scale.

Note.BITSEA = Brief Infant Toddler Social Emotional Assessment; E-CRI = Early Childhood Irritability-Related Impairment Interview; FLIS = Family Life Impairment Scale.

Discussion

In this paper, we introduced the E-CRI, the first parent-interview method specifically designed as a tool to characterize irritability-related impairment across varied contexts. The semistructured, domain based, and contextualized nature of the interview enabled probing that drew out parental reflections on the extent to which their young children’s irritable mood and tantrums interfered with daily routines and social functioning, while also allowing rater judgments that weighted salience of information. Increasing emphasis on identification at the earliest phase of the clinical sequence underscores the importance of developmentally sensitive measures of impairment. The specific focus of the E-CRI on irritability-related impairment is also important for differentiating general developmental impairment (which may emanate from co-occurring problems, such as language; Roberts et al., 2018) from specific interference of irritability with children’s adaptive behavior. The E-CRI may also provide a template for assessing impairment specific to other aspects of psychopathology (e.g., impulsivity). The E-CRI joins a developmentally specified early childhood irritability assessment toolkit we developed, including the MAP-DB survey and the DB-DOS observation (Biedzio & Wakschlag, 2019). To date, these methods have primarily been employed in research. With the E-CRI companion impairment measure, we aim to advance the clinical use of this toolkit beginning in infancy (Wakschlag et al., 2019).

A strength of this study is demonstration of the E-CRI’s reliability and validity across two independent samples. The two samples, taken together, spanned the infant/toddler and preschool periods, providing first-stage reliability and validity of the E-CRI across these developmentally varying age bands. It is likely that the added value of the E-CRI emanates from the opportunity to follow parents’ leads in descriptions of their children’s behavior, probe specifics, and weight the clinical salience of information to inform an integrated judgment of the extent to which the child’s irritability is interfering with age-expectable relationships and activities (illustrated in Figure 1). Parental ratings of impairment include biases and varying thresholds for “interference,” as well as accommodation to children’s tantrum behavior and irritable mood, often without conscious awareness. Strength in measurement is gained by grounding these reports in context and the flexibility to probe occurrences in a manner that increases parents’ reflections on how their own behavior and others’ may be shaped by the goal of preventing irritable outbursts. E-CRI ratings weight parental judgments about impairment based on the extent to which these are consonant with detailed examples elicited by probes. An important avenue for future research exploration will be to examine how parental characteristics (e.g., parental psychopathology, parenting styles) influence parental judgments.

Of note, we structured the E-CRI to include assessment across varied contexts to provide a richer profile of irritability. The unique contribution of this contextual information could not be tested here. Future studies in larger samples might test the utility of context-specific ratings versus a single global rating to optimize measurement efficiency.

In models where both E-CRI impairment domains were jointly considered, tantrum-related impairment demonstrated more substantial and consistent clinical and incremental utility relative to irritable mood-related impairment. The greater explanatory value of tantrum-related impairment may reflect the fact that tantrums are highly salient, as their intensity, frequency, and dysregulation are aversive and disruptive. Because these need to be “managed” (e.g., decisions about whether to disrupt grocery shopping when a tantrum is intense) more than a child’s mood, it is not surprising that in-depth knowledge that explicates setting-based accommodations to manage such behavior would add incremental knowledge. In contrast, sullen, grumpy moods may dampen the affective atmosphere but are less likely to dominate and disrupt those around them, particularly at younger ages. It is also not surprising that internalizing problems were the outcomes in which mood-related impairment consistently explained unique variance, as mood is a defining feature of these syndromes, particularly for depressive symptoms. The general consistency of these patterns across age periods and samples is evidence of their robustness in these community samples. Of note, while validation of the E-CRI across the early childhood period is a strength, the inability to test developmental differences across these age periods within the same sample is a limitation. For example, the EmoGrow sample was more impaired but it is not possible to discern whether this is a sample or developmental difference based on our design.

These findings are a first step toward validation of the E-CRI. Validation in clinical populations and in relation to traditional clinical syndromes (e.g., traditional DSM disorders and syndromes) will be a critical step toward establishing the E-CRI’s clinical utility. This should include replicating the relative utility of tantrum- versus mood-related impairment and determining their distinct contributions to prediction of specific syndromes and their co-occurrence.

Incorporations of brain/behavior patterns of atypicality in exploration of impairment patterns will also advance greater neurodevelopmental understanding of psychopathology (Wakschlag et al., 2018). Finally, given the decades-long “waiting period” from discovery to integration into clinical use, rapid translation of the E-CRI for clinical use should be a priority.

Neurodevelopmental frameworks, such as the Research Domain Criteria (RDoC), encourage early identification and prevention at the earliest phase of the clinical sequence (Mittal & Wakschlag, 2017). Until recently, application of this framework was limited by a dearth of methods that captured clinical phenomenology and related impairment with developmental sensitivity. The E-CRI adds to a growing armamentarium of methods designed to advance and prevent early emergence and chronicity of mental disorder. Its specificity to irritability is of special importance given that irritability is the most robust transdiagnostic indicator of mental health risk in young children (Wakschlag et al., 2019).