Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Mar 1.
Published in final edited form as: Behav Ther. 2019 Nov 26;51(2):310–319. doi: 10.1016/j.beth.2019.09.006

The Internal Structure of the Aberrant Behavior Checklist Irritability Subscale: Implications for Studies of Irritability in Treatment Seeking Youth with Autism Spectrum Disorders

Joel Stoddard 1,2, Jodi Zik 2, Carla Mazefsky 3, Briar DeChant 1,2, Robin Gabriels 1,2
PMCID: PMC7080324  NIHMSID: NIHMS1544636  PMID: 32138940

Abstract

Given the prominence of the Aberrant Behavior Checklist (ABC), Irritability Subscale (ABC-I), in treatment outcome studies, we conducted a critical examination of its internal consistency and relationship to other measures of irritability in 758 psychiatrically hospitalized youth with Autism Spectrum Disorder. In exploratory and confirmation samples, we conducted factor and bifactor analyses to describe the internal structure of the ABC-I. Our results suggest that the ABC-I roughly represents a unidimensional construct of irritability, as indicated by a general factor in bifactor analysis. In addition to irritability, subordinate factors are presented that represent tantrums, verbal outbursts, self-harm, and negative affect. Notably, self-harm items explain a large proportion of variance independent of irritability. Therefore, their contribution in analyses of treatment effects should be considered. Further study or revision of the ABC-I may improve convergent validity with transdiagnostic formulations of irritability as well as prevent confound from self-harm in treatment studies for irritability in ASD.

Keywords: Aberrant Behavior Checklist, Irritability, Autism Spectrum Disorder

Autism Spectrum Disorder (ASD), a disorder of social communication impairments and restricted repetitive patterns of behavior (American Psychiatric Association, 2013), is strongly associated with severe mood problems such as irritability (Mandy, Roughan, & Skuse, 2014; Simonoff et al., 2012). Here, we evaluated the internal structure and measurement properties of a widely used caregiver-reported measure of irritability in ASD, the Aberrant Behavior Checklist Irritability Subscale (ABC-I; Aman, Singh, Stewart, & Field, 1985; Kaat, Lecavalier, & Aman, 2014). By doing so, we aimed to evaluate its utility for measuring irritability in treatment seeking youth with ASD.

Irritability was identified as a prominent behavioral correlate of ASD and was operationalized in the ABC well before the contemporary psychiatric interest in irritability (Aman et al., 1985). The ABC contains an irritability subscale (ABC-I) comprised of items reflecting temper outbursts, negative mood, aggression, and self-harm behavior (Supplemental Table 1). Of the ABC dimensions, the ABC-I subscale explains the most variation in parent report of problem behaviors (please see Kaat et al., 2014 for a large analysis of N=1893 youth with ASD and selective review of five prior studies). The ubiquity of the irritability subscale on studies of ASD is evidenced by its frequent use as an outcome measure for treatment effects for medication, behavioral, and complementary interventions (e.g., Fung et al., 2016; Gabriels et al., 2015; Hollander et al., 2004; Shea et al., 2004). According to ClinicalTrials.gov, 54 registered treatment studies have used the ABC-I subscale as an outcome measure and 28 studies have used the ABC-I subscale as a selection measure (as of July 11,2019).

The formulation of irritability in ASD, and its prominent operationalization in the ABC, deserves examination for two major reasons. First, as noted above, it often represents irritability in treatment studies of ASD as well as other conditions including neurodevelopmental and disruptive disorders. Second, conceptualization of pathologic irritability has evolved transdiagnostically in pediatric mental health since the ABC was developed. Current transdiagnostic formulations empirically define irritability as an increased sensitivity to provocation toward negative emotions, especially anger, and temper outbursts (Deveney, Stoddard, Evans & Chavez, 2019, Brotman, Kircanski, Stringaris, Pine, & Leibenluft, 2017; Holtzman, O’Connor, Barata, & Stewart, 2015; Leibenluft & Stoddard, 2013). Therefore, it is important to examine how the ABC-I subscale represents irritability and how it relates conceptually to other constructions of irritability (Avenevoli, Blader, & Leibenluft, 2015; Toohey & DiGiuseppe, 2017).

A first step towards understanding how the ABC-I subscale represents irritability in ASD is to formally examine its internal structure. Theoretically, caregiver responses to each item on the ABC-I subscale are indicators of a unitary latent construct representing irritability in youth with ASD, an argument based on factor analyses of the ABC and expert opinion (see the study and review by Kaat et al. 2014). Given discrepant results and marginal fits of prior factor analytic models (Kaat et al., 2014), it is reasonable to examine the latent structure of the ABC for all of its accepted components, including the ABC-I subscale, which is the focus of this report. Given the prominence of the ABC-I subscale in interventional studies, an analysis of its structure can inform ongoing investigations, secondary analyses, and interpretation of prior work.

Some evidence suggests that the ABC-I subscale has a multidimensional structure. Specifically, preliminary work suggests that there might be a self-harm component separable from irritability. Brinkley and colleagues (2007) conducted a preliminary factor analysis of the ABC with 275 children with ASD, suggesting that self-harm items of the ABC-I subscale represent a distinct construct from irritability. Similarly, in a community sample of 601 youth with intellectual disability, Brown et al. discovered that self-harm items were empirically distinct from others in the ABC-I subscale (Brown, Aman, & Havercamp, 2002). In an exploratory factor analysis of N=1,130 children with ASD, Kaat and colleagues found that separating self-harm items from the ABC-I subscale was a reasonable alternative model (Kaat et al. 2014). Item wise, re-examination of aripiprazole treatment effects on the ABC-I subscale suggested that aripiprazole may reduce parent reported temper outbursts but not self-harm (Aman et al., 2010). If this is the case, then temper outbursts and self-harm may be distinct features underlying pathophysiology or treatment prescription. Such findings indicate that self-harm items may represent a separable latent construct from the overall construct of irritability (Youngstrom, 2008). In any case, these reports suggest that the ABC-I subscale may have a multidimensional structure.

In addition to considering the internal structure of the ABC-I subscale, its convergent validity with external measures of irritability is also important. Two other measures of irritability have been used in individuals with ASD: the Affective Reactivity Index (Mikita et al., 2015), which is also often used to measure irritability in non-ASD clinical studies, and the Emotion Dysregulation Inventory (EDI), short form (Mazefsky et al., 2018a; Mazefsky, Yu, White, Siegel, & Pilkonis, 2018b). Other than in youth with ASD, irritability has often been measured by an irritability subscale of caregiver report on the Child Behavioral Checklist (CBCL; Achenbach, Rescorla, & Maruish, 2004) of the Achenbach Behavioral Assessment System (Stringaris, Zavos, Leibenluft, Maughan, & Eley, 2012). For this study, the EDI and the CBCL were also included as available data sets.

In this study, we conducted an empirical analysis of the ABC-I subscale in 758, well-phenotyped, treatment-seeking youth with ASD who were admitted to inpatient psychiatric hospital care. These treatment-seeking youth represent individuals who may be candidates for treatment trials for which the ABC-I subscale is commonly used. As summarized above, prior work suggests a need to examine the internal structure the ABC-I subscale and test its associations with other measures of irritability. From prior work, we expected that the ABC-I subscale would represent a single irritability construct (Kaat et al. 2014). However, we also expect that subordinate constructs may be present that are important to consider. If our hypotheses are supported, then an estimate of the reliability of the ABC-I subscale, its suitability for measuring a unitary construct, and its association with external measures will provide evidence for its validity as a measure of irritability for treatment seeking youth with ASD.

Methods

Participants.

All participants were members of the Autism Inpatient Collection (AIC) cohort of the Autism and Developmental Disorders Inpatient Research Collaborative. Data extracted from this sample were from the March 13, 2018 data release (n=937). Accrual details have been published previously (Siegel et al., 2015). Briefly, participants were recruited from individuals admitted for treatment at six psychiatric specialty inpatient care units for children and adolescents with neurodevelopmental disorders (n=2312). Children between the ages of 4–20 years old with a score of ≥12 on the Social Communication Questionnaire (SCQ; Rutter, Bailey, & Lord, 2003) or high suspicion of ASD from the inpatient clinical treatment team were eligible for enrollment. ASD diagnoses were confirmed with the administration of the Autism Diagnostic Observation Schedule-2 (ADOS-2; Lord et al., 2012) by evaluators trained to research reliability standards set by the test authors. Exclusion criteria included not having a parent available who was proficient in English or the child with ASD having prisoner status. Of those eligible (n=1993), 1229 enrolled for further assessment. The ADOS-2 supported a clinical diagnosis of ASD in n=937, who were included in the AIC. Caregivers for n=758 of those in the AIC completed the Aberrant Behavior Checklist (ABC) at admission and were included in this study. Participants were 4-20 years old, M (SD) = 13.0 (3.4), and 19.9% female. Note, the sample was largely comprised of early adolescents with a median age of 13 years, 50% of the sample was between 10.5-15.6 years of age, and 95% of the sample was aged between 6.1 and 18.8 years. Their adaptive behavior was measured with the Vineland II (VABS; Sparrow, Cicchetti, & Balla, 2005) Adaptive Behavior Scale. Overall the sample had impaired adaptive behavior scores, with an Adaptive Behavior Composite M (SD) = 57.1 (17.1), and VABS Communication scale M (SD) = 59.4 (18.6).

Measures

The ABC is an empirically constructed measure of the problem behaviors on which caregivers rate 58 items on a 4-point Likert scale of severity, ranging from “not at all a problem” to “the problem is severe in degree.” Responses on the first edition, community version of the ABC (Aman & Singh, 2017) were coded as integers ranging from 0-3. The ABC has a 15-item irritability subscale, the ABC-I. The items include prompts such as “Injures self on purpose,” “Temper tantrums/outbursts,” “Cries and screams inappropriately,” and “Irritable and whiny.”

A three-item sum from the Child Behavior Checklist (CBCL; Achenbach et al. 2004) has been used as an index of irritability outside of the ASD population. The CBCL irritability score is based on the sum of three items (“stubborn, sullen, or irritable,” “sudden changes in mood or feelings,” and “temper tantrums or hot temper”) (Stringaris, Zavos, Leibenluft, Maughan, & Eley, 2012).

The Emotion Dysregulation Inventory (EDI), Reactivity Subscale, is a caregiver report measure of the irritability construct recently validated in youth with ASD based on exploratory and confirmatory factor analysis and item response theory analysis (Mazefsky et al., 2018b). Responses are converted to EDI theta scores from the normalization sample of N=1755 youth with ASD.

Theory

A measure intended to represent just one construct should not show evidence of overly representing many competing constructs. We use two indicators of how well a measure estimates a unitary construct (Revelle & Zinbarg, 2009). The first indicator, worst split half reliability, is from a model free approach. In this method, the items of a measure are randomly divided in two parts. The within reporter reliability between the two parts are calculated. The worst split half reliability represents the lower bound of reliability of a measure. If items on a scale consistently indicate a single underlying construct at a reasonable level, within-reporter, worst split half reliability on a measure should be high, e.g. 0.8 which allows for 20% of the total variance of responding to be error. Scores less than this indicate that there may be poor internal reliability. For example, the measure may be imprecise, respondents may be unreliable, or the measure may represent more than one latent construct.

The second indicator of sufficiently measuring a unitary construct are parameters from a model-based approach. By proposing a specific structure to the latent constructs that a measure represents, the model-based approach gives more information than the model-free approach. This structure may be discovered and tested through factor analyses. The assumption of the ABC-I subscale is that it indicates a single construct operationalized as significant loadings of each item onto a factor common to all items, which is often called a general factor. A hierarchical factor analysis approach tests for additional complexity by modeling factors that are subordinate to the general factor and are typically called grouping factors. Perhaps the most famous example is that variance in most intelligence tests is best explained by a general factor for intelligence and subordinate factors, representing different aspects of a specific intelligence test. A major question for the model-based approach is how many grouping factors should be allowed. This may be done through a traditional exploratory and confirmatory factor analytic approach that is described below.

Once a model has been selected, the first assessment is to test how well the general factor is represented by the items of a measure. The statistic, Explained Common Variance (ECV), is defined as the ratio of variance explained by a general factor to the variance explained by the general plus any subordinate grouping factors. For a single scale, like the ABC-I subscale, an ECV > 0.5 suggests that a scale acceptably measures a unitary construct of interest as represented by the general factor (an example provided by Reise, 2012). This allows for about half of the total explained response variance to be attributed to subordinate factors.

From all models, we report hierarchical omega (ωh), which is the reliability for the unitary construct of interest. Hierarchical omega is the proportion of all response variance attributable to the general factor partialled out from all other sources of variance. Thus, it accounts for substructure that may be present in a scale that is presumed to measure a single latent variable (Zinbarg, Yovel, Revelle, & McDonald, 2006). Of note, it is increasingly becoming recognized that Cronbach’s alpha is inappropriate for scales such as the ABC-I subscale, which, as demonstrated below, have complex latent structures that fail tau-equivalence assumptions. Hierarchical omega may be confused with ECV. It is important to keep in mind that hierarchical omega is a measure of reliability and can be interpreted in that way (e.g. like Cronbach’s alpha or worst split half reliability). On the other hand, ECV is a standardized indicator of the relative strength of the general factor to grouping factors with an emerging set of standards for interpretation. For a full discussion of these models and statistics, please see Rodriguez, Reise & Haviland, 2016.

Calculations

To examine the internal structure of the ABC-I subscale, we used an established procedure of hierarchical exploratory factor analysis followed by bifactor confirmatory factor analysis. This method has been implemented by Revelle (2018) in R package psych. Data were randomly split into two equal halves. In one half, an exploratory hierarchical factor analysis demonstrates the structure of the measure. Best fit models were estimated with hierarchical factor analysis with oblimin rotation with the number of group factors, determined by model comparison measures, and one general factor estimated through the Schmid-Leiman transformation (Zinbarg et al. 2006). From this first half, we determined the best fit number of factors by model comparison, report preliminary values of ωh and ECV, and display the graphical result of individual item loadings. In the second half of the sample, we confirmed the discovered structure by confirmatory bifactor analysis with a general factor and orthogonal grouping factors. The number of grouping factors was determined by the first stage of the analysis. Construction of the bifactor model followed the procedure implemented in the psych function OmegaSEM, where individual item assignment to orthogonal grouping factors is first determined by their greatest factor loadings from a hierarchical factor analysis. R packages psych with bifactor analysis in lavaan. From this analysis we report final values of ωh and ECV and display the structure of the model. Item assignment and loadings are displayed in Supplemental Table 1. All analyses used maximum likelihood fit.

In sum, we make inferences on the number of grouping factors from EFA and exploratory sample and on internal consistency from the CFA and confirmation sample. This strategy reduces the chance of overfitting the number of grouping factors in the confirmation sample. In addition, hierarchical omega and ECV are more commonly interpreted from a bifactor model as used in the CFA (Rodriguez, Reise & Haviland, 2016). EFA models are compared by χ2, Bayesian information criteria (BIC), and root mean standard error of approximation (RMSEA). The model that minimizes both indices and significantly differed from reduced models in a chi-squared test was selected. For the CFA, BIC, RMSEA, the 90% confidence interval (CI) of the RMSEA, comparative fit index (CFI), and standardized root mean square residual (SRMR) are reported. Convention suggests that an acceptable fit is indicated by the following limits: RMSEA <.08, lower bound of 90%CI RMSEA <.05, CFI >.90, and SRMR<.08.

Post hoc analyses examined the effects of age, verbal ability, and the relationship between the ABC-I subscale and other measures of irritability (i.e., the EDI Reactivity and CBCL Irritability subscales). Please note, it has recently been reported that the examination and calculation of reliability measures with polychoric correlations lead to biased overestimates of reliability metrics (Revelle & Condon, 2018). Therefore, all analyses were done with Pearson’s correlations. However, since using Pearson’s correlations for a 4-point scale may be debatable, bifactor analyses were repeated with polychoric correlations to ensure the same solution (Supplemental Information).

All analyses were done using R version 3.5.3 (CRAN) using the psych package version 1.8.12 (Revelle, 2018) and the lavaan package version 0.6-3 (Rosseel 2012). All analytic code and output are posted at www.github.com/joelStod/IrrInASD.

Results

Exploratory analyses

An exploration of the internal structure of the 15-item ABC-I subscale in a random subsample (n=379) is presented in Table 1. In a model free examination of the reliability of the ABC-I subscale, the worst split half reliability is .66, a modest value suggesting a complex latent structure or imprecision in the ABC-I subscale. A model-based exploration of the internal latent structure suggests that a general factor plus four grouping factors best explains n=379 parent reports of their children’s symptoms on the ABC-I (Figure 1a). The ECV was .43; this may be interpreted as the underlying unitary construct represented by the ABC-I subscale that explains 43% of the variance explained by all the factors (general and grouping). The reliability of measuring this unitary construct with the ABC-I subscale is ωh =.64.

Table 1.

Exploring the internal structure of ABC-I

Model RMSEA BIC β ωh ECV χ2(df) Δχ2(Δdf) p
ABC-I 0.66
 General .234 1385.32 1919.70 (90)
 General + 2 .115 −3.76 .38 .30 447.49 (76) 1472.21 (14) <.001
 General + 3 .078 −168.97 .65 .47 205.09 (63) 242.40 (13) <.001
 General + 4 .058 −189.04 .64 .43 113.78 (51) 91.31 (12) <.001
 General + 5 .065 −136.09 .63 .42 101.41 (40) 12.37 (11) 0.337
Open in a new tab

Internal consistency of the ABC irritability scale (ABC-I) by worst split-half reliability (β) and hierarchical exploratory factor analysis. For exploratory factor analysis, the model indicates how many grouping factors are included with a general factor. A general factor with four subordinate factors provided the best fit by fit indices (RMSEA and BIC) and significance testing. RMSEA=root mean square error of approximation, BIC=Bayesian information criterion, β=smallest split half reliability, ωh= hierarchical omega (general factor saturation), ECV=explained common variance.

Figure 1.

Figure 1.

Open in a new tab

The internal structure of the ABC-I in the exploration (n=379) and confirmation (n=379) analyses. The ABC-I is best described by a general factor (g) with four grouping factors representing tantrums (F1), self-harm (F2), verbal outbursts (F3), and negative affect (F4). Arrows represent loadings > .2.

Confirmatory analyses

In the confirmation sample (n=379), worst split half reliability of the ABC-I subscale was 0.72. The confirmatory bifactor analysis with the best fit model from the exploratory factor analysis, the general plus four-group factor model, fit well and is displayed in Figure 1b, with fit statistics χ2(75)=171.91, p<.001, RMSEA=.058 [90% CI=.047-.070], BIC=13517.42, CFI=.97, SRMR=.038. Similar, to the exploratory analysis (Table 1), a purely unidimensional model with only a general factor was a poor fit RMSEA=.214 [90% CI=.205-.224]. The unidimensional model was a significantly worse fit than the bifactor model (BIC for the unidimensional model was greater than for the bifactor model at 14915.26, Δχ2(Δdf)=1486(15), p<.001). Taken together, these results provide good support for a multidimensional, bifactor structure to the ABC-I subscale.

To evaluate whether the ABC-I subscale is a sufficient measure of its intended underlying construct in this group of patients, we evaluated the confirmatory bifactor model, ECV=.54. This is a modest value, suggesting variance is equally distributed between the general and grouping factors. It may be interpreted as “unidimensional enough” (see Reise, 2012) with a fair reliability of ωh=.77.

Some properties of the structure should be highlighted. Grouping factors suggested items cluster into facets of ABC-I subscale, which we term tantrums, negative affect, self-harm, and verbal outbursts. All items loaded onto the general factor (λ≥.3) except item 25 probing depressed mood (λ=.125 se=.058, z=2.167, p=.03). Items with weaker loadings onto the general irritability factor (Figure 1b) include three items strongly loading onto a self-harm group factor (items 2, 50, and 52) and item 3 probing aggressive behavior. The self-harm group factor was notable in that describing most of the explained variance, 25%, after the general factor, with the other three grouping factors together representing only 21% of explained variance.

Post hoc analyses

Given the broad age range, a confirmatory analysis was done on (n=377) youth within the narrower interquartile age range of 10.6-15.7 years. The confirmatory bifactor analysis with a general factor plus four group factors was a significant and good fit (χ2(75)=154.23, p<.001, RMSEA= .053 [90% CI = .041-.065], BIC=13681.97, CFI=.973, SRMR=.038) with similar loadings (Supplemental Information).

Associations between the sum of the items of the ABC-I subscale, the general and grouping factor scores, age, CBCL irritability subscale, EDI Reactivity, and Vineland adaptive behavior and communication subscales are displayed in Table 2. Measures of irritability were all significantly correlated (sum of the ABC-I items, general factor scores, EDI Reactivity, and CBCL irritability subscales; r’s≥.64, familywise error corrected p’s <.001). Of the grouping factor scores for the ABC-I subscale, only the tantrum group was significantly related to external measures of irritability (EDI Reactivity r=.43, corrected p<.001; CBCL Irritability r=.35, corrected p<.001). The self-harm group was significantly negatively associated with Vineland adaptive behavior composite (r=−.25, corrected p=<.001) and communication (r=−.25, corrected p<.001).

Table 2.

Pearson’s correlations between measures

ABC-I (sum) Tantrums Self-harm Verbal Outbursts Negative Affect Age EDI Reactivity CBCL Irritability Vineland Ad. Bhx. Vineland Comm.
ABC-I (g) 0.94 0.18 0.06 0.20 0.16 −0.27 0.62 0.71 −0.13 −0.10
Tantrums 0.20 −0.08 −0.17 −0.23 −0.02 0.43 0.35 0.04 0.07
Self-harm 0.36 0.00 −0.05 0.02 0.08 −0.05 −0.25 −0.25
Verbal Outbursts 0.25 −0.24 −0.01 0.06 −0.01 −0.11 −0.12
Negative Affect 0.17 0.02 0.01 0.13 0.16 0.18
Age −0.22 −0.12 −0.20 −0.37 −0.26
EDI Reactivity 0.61 0.52 −0.01 0.04
CBCL Irritability 0.64 0.02 0.04
Vineland Ad. Bhx. −0.19 0.93
Open in a new tab

Pearson’s correlations between the ABC-I, its model-based scores, and measures of irritability and communication in the confirmatory sample (n=379). Bold correlations reflect p values <.05 corrected for familywise error by the Holm-Bonferroni method. g= “irritability,” the general factor scores of the ABC-I. Tantrums/Self-harm/Verbal Outbursts/Negative Affect = grouping factor scores. EDI = Emotional Dysregulation Inventory, available in n=339. CBCL = Child Behavioral Checklist, available in n=(197). Vineland was available in n=309 with subscales Ad. Bhx. = adaptive behavior and Comm. = communication.

Discussion

Irritability is an important dimension of problem behavior in ASD. Its operationalization via the ABC-I subscale has had substantial influence on representing irritability in treatment trials. Here, we examine how the ABC-I subscale represents irritability by examining its internal structure. We show that the commonly used ABC-I subscale sufficiently represents a unidimensional construct (irritability), but also that it is multidimensional in nature with at least four subordinate components: tantrums, self-harm, verbal outbursts, and negative affect. Of these, the self-harm items comprise a factor that explains the next largest parent report variance after the common irritability factor. When using the scale to measure irritability or conducting secondary analyses, it is important to be aware of its complex structure. Some considerations for accommodating the ABC-I subscale structure are given below.

Of the components of the ABC-I subscale, self-harm is notable in that it accounts for 25% of explained variance independent of the general irritability factor. The amount of variance explained by the self-harm grouping factor reduces the ECV of general factor representing irritability. This observation is consistent with prior work suggesting that self-harm items may represent a distinct latent variable in youth with intellectual disabilities (Brown et al., 2002) and in youth with ASD (Brinkley et al., 2007). Prior work shows that ABC-I subscale scores may be lower in youth with self-harm (Brinkley et al., 2007). Contrary to this, in the current study, there are significant but modest positive loadings of self-harm items on the general irritability factor in both our exploration and test samples (Figure 1). The current study is consistent with prior work showing a positive loading of these items onto the ABC-I factor (Kaat et al. 2014). Thus, their inclusion the ABC-I is fair. However, those that use the ABC-I subscale may need to be cautious in the implications of the inclusion of these items.

Most importantly, users of the ABC-I subscale should carefully consider the competing, substantial independent explanation of parent-report variance by its self-harm component. Treatment associations with the ABC-I subscale may be differentially affected by it or other components, as demonstrated by Aman and colleagues (2010) for the effect of aripiprazole on temper outburst but not self-harm. The variance independently explained by the self-harm component may confound generalization and understanding of the specificity of treatment effects as well as treatment targets. Another consideration is that the self-harm component explains a unique feature of youth with ASD worthy of investigation, perhaps allowing for a more precise consideration of treatment effects. For example, in this study, self-harm, but not other components of the ABC-I subscale, was negatively associated with adaptive behavior and communication (Table 2). Methods to examine the effects of self-harm items on findings include post hoc item-wise analysis (e.g., Aman et al. 2010) and explicitly modeling the latent structure of the ABC-I in analyses.

Correlations with other measures of irritability showed that the ABC-I subscale and its general factor were similarly associated with the EDI Reactivity and CBCL Irritability subscales, r’s=.62 and .71, respectively. The level of association is modest by psychometric convention, but comparable to the associations between different irritability measures (Deveney et al. 2019). For example, consider the association between the EDI reactivity and CBCL irritability subscale is r=.52.

With regards to item-wise loadings, there are three notable features of the results. First, depressed mood, item 25, did not substantially load onto the general irritability factor in either the exploratory or confirmatory analyses. This is consistent with prior work showing that depressed mood weakly loads with the irritability subscale (Brinkley et al., 2007; Brown et al., 2002; Kaat et al., 2014). This finding is also consistent with recent psychometric analysis of the EDI in a large ASD sample, where factor analysis and item response theory analysis supported separation of items on the Reactivity scale (resembling the transdiagnostic construct of irritability) from Dysphoria items (Mazefsky et al., 2018b). Second, items probing irritable mood, item 14, and mood changes, item 36, cross loaded in the exploratory factor analysis on grouping factors negative affect and tantrums. Their loadings onto the common irritability factor were strong, so both items indicate the irritability construct. However, the cross loading indicates that their assignment to negative affect or tantrums may be ambiguous. This is particularly true of mood changes, item 36, which showed a slightly stronger loading with tantrums in the exploratory analysis but a slightly stronger loading with negative affect, where it was assigned, in the confirmatory analysis. Please note, cross loadings are not seen in the bifactor analysis because the model assigns items to only one grouping factor. Third, stamps, item 47, and demands, item 29, load well on the irritability factor in both models, but more so in confirmatory analysis. In the confirmatory relative to the exploratory analysis, they more strongly load on the irritability factor and do not significantly load on the tantrums grouping factor. Such differences in loadings may be attributable to differences in sampling or model construction.

Several aspects of our design should be considered when interpreting these findings. First, this was a sample of psychiatric inpatients with ASD, and we could not explore differences in the structure of irritability between those with and without ASD due to a limited sample of youth who did not have ASD. This will be necessary to understand whether the current findings on the ABC-I subscale applies to other youth. Second, our sample includes a broad age range, but not a sufficient number in narrow age ranges that may be of interest (e.g. early childhood or late adolescence) to best examine effects of age on the structure of the ABC-I. Third, the exploratory and the confirmatory analyses result from different models on different samples. Some differences are to be expected from sampling and model differences, indicating important sources of variation, such as the ambiguous assignment of mood changes to tantrum or negative affect grouping factors. In addition, unaccounted for sources of measurement error may have influenced the findings. Future research may investigate the most appropriate model of internal structure and construction of the ABC-I as well as its measurement invariance to specific patient characteristics. However, the two models are largely similar. This similarity in different samples as well as the model free worst split half reliability results suggest that the main findings on reliability and putative internal structure will be reproducible. Finally, our findings are most generalizable to caregiver report of individuals with ASD who present for inpatient treatment. The internal structure of the ABC-I may vary for youth with ASD in other states, e.g. not treatment seeking, and by reporter.

Conclusion.

This study represents an important first step towards explaining the structure of the ABC-I subscale and comparing it to other formulations of irritability, with implications for research and clinical care. Importantly, it suggests the potential to optimize the measurement of irritability through the ABC-I subscale by carefully examining the self-harm and mood items.

Supplementary Material

1

  • The Aberrant Behavioral Checklist, Irritability Subscale, (ABC-I) is commonly used in clinical trials.

  • Irritability is adequately measured by the ABC-I with fair reliability.

  • It has a complex internal structure with four separable components.

  • The self-harm component explains a large portion response variance independently of irritability.

Acknowledgements:

Participants were recruited and data obtained in partnership with the Autism and Developmental Disorders Inpatient Research Collaborative (ADDIRC) through use of Autism Inpatient Collection (AIC) data. The ADDIRC is made up of the co-investigators: Matthew Siegel, MD (PI) (Maine Medical Center Research Institute; Tufts University), Craig Erickson, MD (Cincinnati Children’s Hospital; University of Cincinnati), Robin L. Gabriels, PsyD (Children’s Hospital Colorado; University of Colorado, School of Medicine), Desmond Kaplan, MD (Sheppard Pratt Health System), Carla Mazefsky, PhD (UPMC Western Psychiatric Hospital; University of Pittsburgh), Eric M. Morrow, MD, PhD (Bradley Hospital; Brown University), Giulia Righi, PhD (Bradley Hospital; Brown University), Susan L Santangelo, ScD (Maine Medical Center Research Institute; Tufts University), and Logan Wink, MD (Cincinnati Children’s Hospital; University of Cincinnati). Collaborating investigators and staff: Jill Benevides, BS, Carol Beresford, MD, Carrie Best, MPH, Katie Bowen, LCSW, Briar Dechant, BS, Tom Flis, BCBA, LCPC, Holly Gastgeb, PhD, Angela Geer, BS, Louis Hagopian, PhD, Benjamin Handen, PhD, BCBA-D, Adam Klever, BS, Martin Lubetsky, MD, Kristen MacKenzie, BS, Zenoa Meservy, MD, John McGonigle, PhD, Kelly McGuire, MD, Faith McNeil, BS, Joshua Montrenes, BS, Tamara Palka, MD, Ernest Pedapati, MD, Kahsi A. Pedersen, PhD, Christine Peura, BA, Joseph Pierri, MD, Christie Rogers, MS, CCC-SLP, Brad Rossman, MA, Jennifer Ruberg, LISW, Elise Sannar, MD, Cathleen Small, PhD, Nicole Stuckey, MSN, RN, Barbara Tylenda, PhD, Brittany Troen, MA, R-DMT, Mary Verdi, MA, Jessica Vezzoli, BS, and Deanna Williams, BA.

We would like to extend a special thank you to AIC families for making this study possible.

This study was supported with funding from NIMH K23 MH113731 (JS), NICHD R01 HD079512 (CM), and the Ritvo-Slifka Award for Innovation in Autism Research (CM). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflicts of Interest Statement

The authors declare no conflicts of interest.

References

  1. Aman MG, Kasper W, Manos G, Mathew S, Marcus R, Owen R, & Mankoski R (2010). Line-item analysis of the Aberrant Behavior Checklist: results from two studies of aripiprazole in the treatment of irritability associated with autistic disorder. Journal of Child Adolescent Psychopharmacology, 20(5), 415–422. doi: 10.1089/cap.2009.0120 [DOI] [PubMed] [Google Scholar]
  2. Aman MG, & Singh NN (2017). Aberrant Behavior Checklist: Community/residential Manual: Slosson Educational Publications, Incorporated. [Google Scholar]
  3. Aman MG, Singh NN, Stewart AW, & Field CJ (1985). Psychometric characteristics of the aberrant behavior checklist. American Journal of Mental Deficiency, 89(5), 492–502. [PubMed] [Google Scholar]
  4. American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders (5th ed.). Arlington, VA: American Psychiatric Publishing. [Google Scholar]
  5. Avenevoli S, Blader JC, & Leibenluft E (2015). Irritability in Youth: An Update. Journal of the American Acadademy Child Adolescent Psychiatry, 54(11), 881–883. doi: 10.1016/j.jaac.2015.08.012 [DOI] [PubMed] [Google Scholar]
  6. Brinkley J, Nations L, Abramson RK, Hall A, Wright HH, Gabriels R, … Cuccaro ML (2007). Factor analysis of the aberrant behavior checklist in individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 37(10), 1949–1959. doi: 10.1007/s10803-006-0327-3 [DOI] [PubMed] [Google Scholar]
  7. Brotman MA, Kircanski K, Stringaris A, Pine DS, & Leibenluft E (2017). Irritability in youths: A translational model. American Journal of Psychiatry, 174(6), 520–532. doi: 10.1176/appi.ajp.2016.16070839 [DOI] [PubMed] [Google Scholar]
  8. Brown EC, Aman MG, & Havercamp SM (2002). Factor analysis and norms for parent ratings on the Aberrant Behavior Checklist-Community for young people in special education. Research in Developmental Disabilities, 23(1), 45–60. doi: 10.1016/S0891-4222(01)00091-9 [DOI] [PubMed] [Google Scholar]
  9. Fung LK, Mahajan R, Nozzolillo A, Bernal P, Krasner A, Jo B, … Hardan AY. (2016). Pharmacologic treatment of severe irritability and problem behaviors in autism: A systematic review and meta-analysis. Pediatrics, 137 Suppl 2, S124–135. doi: 10.1542/peds.2015-2851K [DOI] [PubMed] [Google Scholar]
  10. Gabriels RL, Pan Z, Dechant B, Agnew JA, Brim N, & Mesibov G (2015). Randomized controlled trial of therapeutic horseback riding in children and adolescents with autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 54(7), 541–549. doi: 10.1016/j.jaac.2015.04.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Hollander E, Phillips A, King BH, Guthrie D, Aman MG, Law P, … Robinson R. (2004). Impact of recent findings on study design of future autism clinical trials. CNS Spectrums, 9(1), 49–56. 10.1017/S109285290000835X [DOI] [PubMed] [Google Scholar]
  12. Holtzman S, O’Connor BP, Barata PC, & Stewart DE (2015). The Brief Irritability Test (BITe): A measure of irritability for use among men and women. Assessment, 22(1), 101–115. doi: 10.1177/1073191114533814 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Kaat AJ, Lecavalier L, & Aman MG (2014). Validity of the Aberrant Behavior Checklist in children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44(5), 1103–1116. doi: 10.1007/s10803-013-1970-0 [DOI] [PubMed] [Google Scholar]
  14. Leibenluft E, & Stoddard J (2013). The developmental psychopathology of irritability. Development and Psychopathology, 25(4 Pt 2), 1473–1487. doi. 10.1017/S0954579413000722 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Lord C, Rutter M, DiLavore P, Risi S, Gotham K, & Bishop S (2012). Autism Diagnostic Observation Schedule Second Edition (ADOS-2) Manual (Part 1): Modules 1–4. Torrance, CA: Western Psychological Services. [Google Scholar]
  16. Mandy W, Roughan L, & Skuse D (2014). Three dimensions of oppositionality in autism spectrum disorder. Journal of Abnormal Child Psychology, 42(2), 291–300. doi: 10.1007/s10802-013-9778-0 [DOI] [PubMed] [Google Scholar]
  17. Mazefsky CA, Day TN, Siegel M, White SW, Yu L, & Pilkonis PA (2018a). Development of the Emotion Dysregulation Inventory: A PROMIS® ing method for creating sensitive and unbiased questionnaires for autism spectrum disorder. Journal of Autism and Developmental Disorders, 48, 3736–3746. doi: 10.1007/s10803-016-2907-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Mazefsky CA, Yu L, White SW, Siegel M, & Pilkonis PA (2018b). The Emotion Dysregulation Inventory: Psychometric properties and item response theory calibration in an autism spectrum disorder sample. Autism Research, 11(6), 9280–941. doi: 10.1002/aur.1947 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Mikita N, Hollocks MJ, Papadopoulos AS, Aslani A, Harrison S, Leibenluft E, … Stringaris A (2015). Irritability in boys with autism spectrum disorders: an investigation of physiological reactivity. Journal of Child Psychology and Psychiatry, 56(10), 1118–1126. doi: 10.1111/jcpp.12382 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Reise SP (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. doi: 10.1080/00273171.2012.715555 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Revelle WR (2018). psych: Procedures for Personality and Psychological Research [Computer Software], Northwestern University, Evanston, Illinois, USA. [Google Scholar]
  22. Revelle WR, & Condon DM (2018). Reliability In Irwing P, Booth T, & Hughes DJ (Eds.), The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development. London: John Wily & Sons. [Google Scholar]
  23. Revelle WR, Zinbarg RE (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika.74(1):145–154. doi: 10.1007/s11336-008-9102-z [DOI] [Google Scholar]
  24. Rodriguez A, Reise SP, & Haviland MG (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2),137–150. doi: 10.1037/met0000045 [DOI] [PubMed] [Google Scholar]
  25. Rosseel Y (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. [Google Scholar]
  26. Rutter M, Bailey A, & Lord C (2003). SCQ. The Social Communication Questionnaire. Torrance, CA: Western Psychological Services. [Google Scholar]
  27. Shea S, Turgay A, Carroll A, Schulz M, Orlik H, Smith I, & Dunbar F (2004). Risperidone in the treatment of disruptive behavioral symptoms in children with autistic and other pervasive developmental disorders. Pediatrics, 114(5), e634–641. doi: 10.1542/peds.2003-0264-F [DOI] [PubMed] [Google Scholar]
  28. Siegel M, Smith KA, Mazefsky C, Gabriels RL, Erickson C, Kaplan D, … Santangelo SL (2015). The autism inpatient collection: Methods and preliminary sample description. Molecular Autism, 6(61), e1–10. doi: 10.1186/s13229-015-0054-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Simonoff E, Jones CR, Pickles A, Happe F, Baird G, & Charman T (2012). Severe mood problems in adolescents with autism spectrum disorder. Journal of Child Psychology and Psychiatry, 53(11), 1157–1166. doi: 10.1111/j.1469-7610.2012.02600.x [DOI] [PubMed] [Google Scholar]
  30. Sparrow SS, Cicchetti DV, & Balla DA (2005). Vineland Adaptive Behavior Scales:(Vineland II), Survey Interview Form/caregiver Rating Form. Livonia, MN: Pearson Assessments. [Google Scholar]
  31. Stringaris A, Zavos H, Leibenluft E, Maughan B, & Eley TC (2012). Adolescent irritability: Phenotypic associations and genetic links with depressed mood. American Journal of Psychiatry, 169(1), 47–54. doi: 10.1176/appi.ajp.2011.10101549 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Toohey MJ, & DiGiuseppe R (2017). Defining and measuring irritability: Construct clarification and differentiation. Clinical Psychology Review, 53, 93–108. doi: 10.1016/j.cpr.2017.01.009 [DOI] [PubMed] [Google Scholar]
  33. Youngstrom EA (2008). Evidence-based strategies for the assessment of developmental psychopathology: Measuring prediction, prescription, and process In Craighead WE, Miklowitz DJ, & Craighead LW (Eds.), Psychopathology: History, Diagnostis, and Empirical Formulations. Hoboken, NJ: John Wiley & Sons. [Google Scholar]
  34. Zinbarg RE, Yovel I, Revelle W, & McDonald RP (2006). Estimating generalizability to a latent variable common to all of a scale’s indicators: A comparison of estimators for ωh. Applied Psychological Measurement, 30(2), 121–144. doi: 10.1177/0146621605278814 [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1544636-supplement-1.docx (27.3KB, docx)

RESOURCES