Abstract
The presence of Callous-Unemotional (CU) traits delineates a subgroup of youth with severe antisocial behavior. However, debate surrounds the best method to assess CU traits. This study examined the factor structure of the parent-reported Inventory of Callous Unemotional Traits (ICU) among high risk 9-year olds (N = 540) and its predictive validity over one year. Confirmatory Factor Analysis showed support for a three-factor bifactor model and revised two-factor model using a shortened ICU. Within a three-factor bifactor framework the general callous-unemotional (CU) traits factor and specific uncaring factor scores were related to higher externalizing and lower internalizing behavior problems at ages 9.5 and 10.5. Findings were replicated using teacher-reported outcomes. However, results suggest the need for item refinement and highlight the utility of a two-factor solution using a shortened ICU. In particular, the meaning of the unemotional items is discussed in relation to the conceptualization of CU traits.
Keywords: psychometric, psychopathy, antisocial behavior, psychopathology
In the last 20 years, research has examined callous unemotional (CU) traits among antisocial youth as a theoretical downward extension of the affective features of adult psychopathy (Frick, O’Brien, Wootton, & McBurnett, 1994). Measures of CU traits assess behaviors, such as deficits in empathic concern, shallow affect, and lack of guilt. In recent years, several reviews have summarized the literature showing that the presence of CU traits is related to more severe antisocial behavior in childhood and adolescence and that these traits identify a homogenous subgroup of children with specific risk factors who may require tailored interventions (Frick & White, 2008; Frick, Ray, Thornton, & Kahn, 2014). In recognition of the growing body of research that has demonstrated the utility of assessing CU traits during middle childhood and adolescence, a specifier for the diagnosis of Conduct Disorder based on conceptualizations of CU traits was added to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition and termed ‘with limited prosocial emotions’, (DSM-5; American Psychiatric Association, 2013). Measurement of CU traits thus remains a significant research focus with important clinical implications. However, gaps remain in our knowledge of the underlying construct of CU behavior1 and how best to measure it, especially among school-aged children. In response to the shortcomings of previous measures, the Inventory of Callous-Unemotional Traits (ICU; Frick, 2004) was developed to comprehensively assess CU behavior, but remains at the center of debate, particularly in relation to its psychometric properties and the extent to which these properties inform conceptualizations of CU behavior (see Lahey, 2014). In the current study, we examined the factor structure and construct validity of the parent-reported ICU. In particular, we examined the widely-employed three-factor bifactor (3FBF) and newly proposed two-factor (2F) solutions, and tested the predictive validity of ICU scores in a large, high-risk sample of 9.5-year-olds followed longitudinally over a year.
Factor structure of the Inventory of Callous-Unemotional Traits (ICU)
The ICU comprises 12 positively- and 12 negatively-worded items, including four items included in the ‘limited prosocial emotions’ DSM-5 specifier that index CU behavior (‘I care about how well I do at school’, ‘I feel guilty when I do something wrong’, ‘I do not show emotions’, and ‘I am concerned about the feelings of others’). In terms of factor structure, the best fit for the 24-item ICU has typically been obtained by models specifying a 3FBF solution (Essau, Sasagawa, & Frick, 2006). Bifactor models represent an appealing way to model multidimensionality, specifying a general factor that captures shared variance across all items, while simultaneously modeling specific variance of separate dimensions within subsets of items. Bifactor models are common in the intelligence literature (e.g., Carroll, 1993), where conceptualizations of the structure of mental ability comprise both general and specific skills. Bifactor models have also been applied to antisocial behavior, including among studies of adult psychopathy (e.g., Patrick, Hicks, Nichol, & Krueger, 2007) and to model different forms of aggression among antisocial youth (e.g., Tackett, Daoud, De Bolle, & Burt, 2013). Generally, these studies have demonstrated an overarching general factor (e.g., psychopathy or aggression) but with underlying specific factors (e.g., interpersonal, affective, or lifestyle traits).
In the ICU 3FBF model, items load onto three specific factors (callousness, uncaring, and unemotional), while simultaneously loading onto a general CU behavior factor. A 3FBF solution has been replicated in studies of the self-reported ICU among forensic (N =248, ages 12–20, Kimonis et al., 2008) and community samples that represent different ages periods and countries (Table 1; e.g., N=347, ages 12–18, Fanti, Frick, & Georgiou, 2009; N=455, ages 14–20, Roose, Bijttebier, Decoene, Claes, & Frick, 2010; N=540, ages 10–14, Ciucci, Baroncelli, Franchi, Golmaryami, & Frick, 2014). Nevertheless, because much research on the ICU has focused on healthy community samples recruited from schools or on forensic/clinical samples, there is a need for studies that assess dimensional samples that include a full range of antisocial behavior and CU behavior.
Table 1.
N | Country | Sample type | Age range | % female | df | χ2 | GFI | AGFI | CFI | RMSEA | SRMR | AIC | CAIC | χ2/df | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Essau et al. (2006) | 1443 | Germany | Com. | 13–18 | 46 | ||||||||||
| |||||||||||||||
1 factor | 252 | 2475.366 | .731 | .680 | .120 | 2571.366 | 2831.682 | ||||||||
3 factor | 249 | 2213.166 | .783 | .738 | .109 | 2316.166 | 2596.808 | ||||||||
3 bifactor | 228 | 1824.942 | .819 | .762 | .103 | 1968.942 | 2365.143 | ||||||||
3 bifactor + MI | 203 | 1034.464 | .894 | .844 | .078 | 1228.464 | 1762.235 | ||||||||
| |||||||||||||||
Kimonis et al. (2008) | 248 | USA | Foren. | 12–20 | 24 | ||||||||||
| |||||||||||||||
1 factor | 252 | 890.76 | .50 | .10 | 3.53 | ||||||||||
Original 4 factor hierarchical | 249 | 800.13 | .57 | .10 | 3.21 | ||||||||||
3 bifactor | 231 | 502.30 | .79 | .07 | 2.17 | ||||||||||
3 factor hierarchical (deleted 2 and 10) | 206 | 471.25 | .79 | .07 | 2.29 | ||||||||||
3 bifactor (deleted 2 and 10) | 187 | 343.52 | .87 | .06 | 1.84 | ||||||||||
| |||||||||||||||
Fanti et al. (2009) | 347 | Cyprus | Com. | 12–18 | 49 | ||||||||||
| |||||||||||||||
1 factor | 252 | 931.88 | .572 | .089 | .090 | 20,833.12 | |||||||||
3 factor | 249 | 722.77 | .702 | .075 | .082 | 20,630.01 | |||||||||
3 bifactor | 228 | 508.35 | .824 | .058 | .061 | 20,455.59 | |||||||||
3 bifactor + MI | 212 | 372.12 | .919 | .047 | .050 | 20,401.36 | |||||||||
| |||||||||||||||
Roose et al. (2010) | 455 | Belgium | Com. | 14–20 | 44 | ||||||||||
| |||||||||||||||
1 factor youth report | 2133.43 | .72 | .66 | .77 | .13 | 2229.49 | 2475.21 | 8.47 | |||||||
1 factor parent report | 1070.74 | .63 | .56 | .76 | .15 | 1166.74 | 1360.51 | 4.25 | |||||||
1 factor teacher report | 1297.01 | .52 | .43 | .78 | .19 | 1393.01 | 1574.81 | 5.15 | |||||||
1 factor combined report | 901.61 | .61 | .54 | .84 | .15 | 997.61 | 1179.41 | 3.58 | |||||||
3 factor youth report | 998.95 | .85 | .81 | .89 | .08 | 1100.95 | 1362.09 | 4.01 | |||||||
3 factor parent report | 722.49 | .72 | .66 | .85 | .11 | 824.49 | 1030.37 | 2.90 | |||||||
3 factor teacher report | 816.34 | .64 | .56 | .86 | .14 | 4000.29 | 4091.19 | 3.28 | |||||||
3 factor combined report | 574.55 | .71 | .65 | .92 | .10 | 676.55 | 869.71 | 2.31 | |||||||
3 bifactor youth report | 674.53 | .89 | .86 | .92 | .07 | 818.53 | 1187.20 | 2.96 | |||||||
3 bifactor parent report | 375.12 | .83 | .78 | .93 | .07 | 519.12 | 809.78 | 1.65 | |||||||
3 bifactor teacher report | 534.03 | .73 | .64 | .90 | .11 | 678.03 | 950.73 | 2.34 | |||||||
3 bifactor combined report | 348.31 | .80 | .74 | .96 | .07 | 492.31 | 765.01 | 1.53 | |||||||
Feilhauer et al. (2012) | 383 | Holland | Mixed | 8–20 | 0 | ||||||||||
| |||||||||||||||
1 factor | 252 | 1054.56 | .777 | .70 | .105 | .093 | 1411.140 | 1648.646 | 4.18 | ||||||
3 factor | 249 | 846.639 | .829 | .78 | .086 | .087 | 1048.295 | 1300.644 | 3.40 | ||||||
3 factor hierarchical | 249 | 846.639 | .829 | .78 | .086 | .087 | 1048.295 | 1300.644 | 3.40 | ||||||
3 bifactor | 228 | 706.725 | .861 | .82 | .077 | .077 | 882.459 | 1238.718 | 3.09 | ||||||
Houghton et al. (2013) | 268 | Australia | Com. | 7–13 | 43 | ||||||||||
| |||||||||||||||
3 factor | 167 | 380.09 | .85 | .07 | |||||||||||
2 factor (unemotional + callous versus uncaring) | 169 | 417.43 | .82 | .07 | |||||||||||
2 factor (callous versus uncaring) | 103 | 280.32 | .86 | .08 | |||||||||||
2 factor + MI (callous versus uncaring) | 95 | 221.63 | .90 | .07 | |||||||||||
Ezpeleta et al. 2013 | 620 | Spain | Strat. | 3–4 | 50 | ||||||||||
| |||||||||||||||
1 factor | 1055 | 6706.30 | .77 | .093 | |||||||||||
3 factor | 1041 | 3670.30 | .89 | .064 | |||||||||||
3 bifactor | 1004 | 2864.20 | .92 | .055 | |||||||||||
Ciucci et al. 2014 | 540 | Italy | Com. | 10–14 | 53 | ||||||||||
| |||||||||||||||
1 factor | 209 | 836.44 | .86 | .83 | .67 | .07 | .07 | 924.44 | 4.00 | ||||||
3 factor | 206 | 616.93 | .89 | .87 | .78 | .06 | .06 | 710.93 | 2.99 | ||||||
3 bifactor | 187 | 426.52 | .93 | .91 | .87 | .05 | .05 | 558.52 | 2.28 | ||||||
3 factor hierarchical | 198 | 442.06 | .93 | .91 | .87 | .05 | .05 | 552.06 | 2.23 | ||||||
Hawes et al. 2014 | 250 | USA | Clinic | 6–12 | 0 | ||||||||||
| |||||||||||||||
1 factor | 252 | 797.66 | .78 | .09 | |||||||||||
3 factor | 249 | 629.352 | .84 | .08 | |||||||||||
3 bifactor | 228 | 553.36 | .87 | .08 | |||||||||||
Revised 2 factor (callous versus uncaring) | 53 | 100.21 | .97 | .06 | |||||||||||
Revised 2 bifactor (similar fit but selected 2 factor as it provided, ‘a more parsimonious model’ (p.7) | 43 | 86.57 | .97 | .06 |
Note. Com =community (i.e., normative, healthy, or school-based sample); Foren =forensic; Strat =stratified. Degrees of freedom vary across previous studies depending on item set being used (e.g., 22 items instead of 24 items).
Limitations of 3FBF ICU models
Despite advances in research examining models, there are also a number of limitations associated with the 3FBF for the ICU, including poor-to-acceptable model fit indices (see Table 1), marginally acceptable internal consistency of the unemotional subscale, the need to remove items, and error terms being specified to correlate according to modification indices. Further, the callousness subfactor is largely comprised of negatively-worded items, whereas the uncaring factor is comprised of positively-worded items, suggesting that the 3FBF structure may be driven by method variance related to response styles. Finally, an increasing number of studies have not replicated a 3FBF (see Table 1). Across a range of samples using the youth-reported ICU, studies have reported solutions for models with five (N = 383; ages 8–18, Feilhauer, Cima, & Arntz, 2012), two (N = 268, ages 7–13, Houghton, Hunter, & Crow, 2013), and three factors (N = 620, ages 3–4, Ezpelata, Osa, Granero, Penelo, & Domenech, 2013), as well as a recently proposed two-factor solution using a 12-item version of the ICU (N = 250, ages 6–12, Hawes et al., 2014).
However, despite these alternative model solutions and the limitations outlined above, results of studies supporting a 3FBF model have been the justification for which subsequent studies have simply used a summed subscale or total ICU scores within analyses. Notably, studies are also using the parent-reported ICU to assess CU behavior in young children (e.g., Somech & Elizur, 2012), despite the fact that only two previous studies have examined the factor structure of the parent-reported ICU, and both reported inadequate fit for a 3FBF model (Hawes et al., 2014; Roose et al., 2010; Table 1). It is troublesome that the field is moving forward on less than solid psychometric grounds in the context of the new DSM-5 specifier, which highlights the need for reliable measurement of CU behavior for diagnosis and classification. In particular, questions surround the use of the ICU for diagnosis of limited prosocial emotions and, specifically, which version of the ICU (e.g., 24-item or 12-item; e.g., Hawes et al., 2014) best assesses the CU behavior construct, particularly using parent report.
Construct validity of the ICU – externalizing and internalizing behavior outcomes
Indeed, aside from factor structure, an important aspect of psychometric work relating to the ICU surrounds its construct validity. First, previous studies have supported the utility of total summed ICU scores, which typically exhibit high internal consistencies. Further, total self-reported ICU scores correlate positively with externalizing outcomes (e.g., Fanti et al., 2009; Kimonis et al., 2008). Total scores derived from the parent-reported ICU are also related to more behavior problems (e.g., at-risk adolescents, ages 13–17, N = 70, Berg et al., 2013; also see Roose et al., 2010) and criminogenic risk (detained adolescents, ages 12–18, N = 94; White, Cruise, & Frick, 2013). Second, callous and uncaring subscale scores are strongly related to externalizing and aggressive outcomes, in line with findings reported for total ICU scores. However, even though the construct is ‘callous-unemotional’, unemotional subscale scores typically fail to explain unique variance in outcomes and demonstrate inconsistent correlation with measures of externalizing behavior (e.g., Berg et al., 2013). Recently Hawes and colleagues proposed a revised 2F model for the ICU, comprised of only callous and uncaring subfactors (unemotional items were dropped, with the exception of item 6, ‘does not show emotions’), based on an examination of item-total correlations and item-response theory. Their 2F solution demonstrated good model fit, high internal consistency, acceptable test-retest reliability, convergent validity, and replication in an independent sample. This initial revision of the ICU highlights the need for further examination of the validity of ‘unemotional’ items within the context of youth CU behavior.
CU behavior has also been examined in relation to internalizing problem behaviors. Broadly, results mirror those from the adult psychopathy literature, with the theory being that CU behavior is related to fearlessness, low anxiety, and low internalizing (Lykken, 1995). For example, one study found that CU behavior was related to fewer internalizing symptoms over time (N = 1862, ages 5–8 at baseline, Pardini, Stepp, Hipwell, Stouthamer-Loeber, & Loeber, 2012). However, ICU scores have also been shown to predict higher levels of internalizing problems (e.g., Berg et al., 2013; Essau et al., 2006). One explanation for these discrepant findings may derive from the fact that externalizing behavior problems are often strongly associated with internalizing symptoms. Thus, the direction of association between CU behavior and anxiety may be positive, until concurrent externalizing behavior is accounted for, when it becomes negative (i.e., cooperative suppression; see Frick et al., 2014; Lilienfeld, 2003). However, this issue is yet to be addressed in longitudinal analyses using the parent-reported ICU and controlling for overlap between internalizing and externalizing.
Implications of a 3FBF model
Finally, studies have rarely considered the meaning of the 3FBF solution. In particular, the meaning of specific factors may be different when partialling out variance in a general CU behavior factor. Equally, it is unclear what a general factor means in terms of capturing correlation between items once unique variance relating to specific subfactors has been accounted for. Somewhat surprisingly, no studies to date have examined unique associations between specific factors and externalizing or internalizing outcomes, accounting for variance explained by a CU general factor. Rather, studies have solely examined associations between summed subscale scores based on the 3FBF model, even though the 3FBF model implies a need for general and specific factor scores (see Lahey, 2014). Indeed, when summed total ICU scores are used, potentially unique variance captured by specific factors is lost. However, harnessing a 3FBF may produce more precise associations with relevant behavioral outcomes. It is thus important to examine whether a 3FBF predicts criterion variables differently than summed scores.
Gaps in the literature
Taken together, a number of gaps thus emerge in the ICU literature. First, although the parent-reported ICU is already widely in use, there remains a need for its validation, particularly among high-risk samples of youth who exhibit a range of scores on measures of antisocial behavior. In addition, the majority of previous studies that have examined the ICU have assessed samples with wide age ranges (Table 1), making it difficult to draw conclusions about its factor structure during specific developmental periods. Further, no studies to date have examined the longitudinal and predictive validity of the parent-reported ICU across informants. From a prevention perspective this is an important question especially if we can clarify whether ICU scores identify children at risk of developing more entrenched behavior problems before they reach clinical levels, particularly in youth already at high risk for later psychopathology. Second, studies examining construct validity have typically been cross-sectional in design, making it difficult to draw conclusions about the longitudinal predictive value of the ICU. The few longitudinal studies have typically focused on treatment or forensic samples, among which ICU scores predicted risk of future poor outcomes (e.g., White, Frick, Lawing, & Bauer, 2013). Third, while the results from studies that have examined externalizing problems have generally been consistent, studies are needed that examine associations between the ICU and internalizing problems and take into account any overlap between internalizing and externalizing symptoms. Finally, no previous studies have considered the predictive validity of the 3FBF scales versus summed ICU scores.
Aims of current study
The current study seeks to clarify the factor structure of the ICU and its construct validity in a number of ways. First, we examined the parent-reported version of the ICU in a large sample (N = 540; 50% female) of 9-year-olds at risk for conduct problems. We used reports from both primary and alternative caregivers. For primary caregiver reports on the ICU, we compared four models reported in previous studies (one factor, three oblique correlated factors, 3FBF, and revised 2F model) using Confirmatory Factor Analysis (CFA) and replicated findings with alternative caregiver-reported ICU data (e.g., co-parent, father, grandmother). Within a 3FBF framework, we examined associations between ICU scores and primary caregiver versus teacher reported outcomes age 9.5 and predictions to age 10.5. We focused on narrow symptom outcomes for broad-band externalizing (aggressive vs. rule-breaking) and internalizing (anxious-depressed vs. withdrawn-depressed). We computed models for internalizing behaviors while controlling for the overlap between internalizing and externalizing symptoms. We also compared findings using summed scores based on the 3FBF versus 2F solutions specifically with externalizing behavior outcomes. The major goal of the current study was to comprehensively examine the factor structure of the ICU and its predictive validity over time in a way that informed psychometrics of the measure, the use of summed versus factor scores, proposed revisions to the measure, and our understanding of the CU behavior construct.
Methods
Participants
Participants included 731 mother–child dyads recruited between 2002 and 2003 from Women, Infants, and Children (WIC) Nutritional Supplement Program programs in the metropolitan areas of Pittsburgh, PA, and Eugene, OR, and in and outside of Charlottesville, VA (Dishion et al. 2008). Participants were originally recruited to be part of a randomized controlled trial of the Family Check-Up, a preventative intervention for use in high-risk environments to address normative challenges facing parents from toddlerhood onwards (see Dishion et al., 2008). Families were invited to participate if they had a son or daughter between age 2 years 0 months and 2 years 11 months. Recruitment risk criteria were defined as 1 SD above normative means or established clinical cut points on screening measures in at least two of the following three domains: (a) child behavior problems (e.g., conduct problems - Eyberg Child Behavior Inventory; Robinson, Eyberg, & Ross, 1980), (b) primary caregiver problems (e.g., maternal depression, daily parenting stress, or self-report of substance), and (3) sociodemographic risk (low education or low family income). Thus, children in the study selected as ‘high risk’ based on established risk factors for later conduct problems. Specifically, aside from socioeconomic or family risk, families qualified for the original study if children scored in the clinical range on the Intensity or Problem Scales of the Eyberg Behavior Inventory, which comprised 44% of the sample at recruitment, making the sample community-based but enriched/over-sampled for those with early conduct problems (see Dishon et al., 2008; Robinson et al., 1980). However, because the sample was a community (versus clinical) sample and not all children met inclusion criteria based on this definition of clinically-meaningful frequencies of conduct problems, there was variability in the frequency of child conduct problems.
Of the 1,666 families who had children of the appropriate age and who were contacted across study sites, 879 met the eligibility requirements (52% in Pittsburgh, 57% in Eugene, and 49% in Charlottesville), and 731 (83.2%) consented to participate. The children in the sample had a mean age of 29.9 months (SD = 3.2) at the age 2 assessment (approximately 2.5 years old). Across sites, primary caregivers self-identified as belonging to the following ethnic groups: 28% African American, 50% European American, 13% biracial, and 9% other groups. During screening, more than 66% of enrolled families had an annual income < $20,000, and the average number of family members per household was 4.5 (SD = 1.63). Forty-one percent of the sample had a high school or general education diploma. Following the baseline assessment, half the sample was randomly assigned to receive the Family Check-Up intervention (see Dishion et al., 2008); thus intervention status is used as a covariate in all analyses. Of 731 families who initially participated, we had ICU data for 540 (74%) at age 9.5. Of the 540 children with ICU data at age 9.5, we had alternative caregiver reports on the ICU for 401 (74%) children, primary caregiver-reported data for 404 children (75%) at age 10, teacher-reported data for 358 children at age 9.5 (66%), and teacher-reported data for 318 children at age 10.5 (59%). Selective attrition analyses conducted via Chi-squared tests or ANOVAs indicated that there were no differences between children for whom we did and did not have ICU data for according to intervention status (p = .40), race (p = .19), family income (p = .19), and initial, baseline levels of child problem behavior as reported by either primary (p > .70) or alternative (p > .90) caregivers. However, parent education was lower among those families for whom we did not obtain age 9.5 ICU data from - those lost to follow-up were less likely to have at least a high school education (p > .001).
Measures
Recruitment began when children were age 2 and annual assessments (with the exception of age 6) were conducted at family homes using a variety of questionnaires, interviews, assessor impressions, and videotaped observations. From age 7.5 onwards, we also collected data from teachers. The current study uses questionnaire data collected from homes (primary caregiver and alternative caregiver reports) and schools (teacher reports) at ages 9.5 and 10.5 years old. The majority of primary caregivers at age 9.5 were biological mothers (90%). Alternative caregivers were most typically a biological father (45%), the romantic partner of the child’s mother (11%), a grandparent (10%), or an aunt/uncle (4%).
Demographics questionnaire – covariates
Primary caregivers completed a demographics questionnaire at age 2 (Dishion et al., 2008). Consistent with past studies in this sample, child gender was coded as female=0 (n = 271; 50.2%); male=1 (n = 269; 49.8%). Child’s race was coded as ‘Caucasian/other’=0 (n = 304; 56.3%); ‘Black African-American/biracial’=1 (n = 236; 43.7%). Ethnicity was coded as ‘non-Hispanic’=0 (n = 474; 87.8%); ‘Hispanic’=1 (n = 64; 11.9%). Consistent with past studies in this sample, parent education was coded as ‘less than high school=0 (n = 113; 20.9%) and ‘high school and beyond’ =1 (n = 428; 79.1%. Gross annual family income was coded as ≤ $14,999=0 (n = 268; 49.6%); ≥ $15,000=1 (n = 272; 50.4%). Finally, as data were collected from multiple sites and sites differed with respect to the urbanicity and ethnic/racial composition of participants, location was included as a covariate to account for these potential differences. Note that the pattern of findings is unchanged if we include quasi-continuous parent education and family income variables. Further, the cut-points reported represent meaningful differences between groups within our relatively high-risk sample.
CU Traits (age 9.5)
We assessed CU traits at age 9.5 via primary and alternative caregiver reports on the 24-item ICU (Frick, 2004). Each item is rated on a 4-point scale (0 =not true; 1 =somewhat true; 2 =very true; 3 =definitely true). Self-reported ICU data were not collected.
Externalizing and Internalizing problem behavior (ages 9.5 and 10.5)
Primary caregivers completed the Child Behavior Checklist (CBCL; Achenbach, 1991a) and teachers completed the Teacher Report Form of the Child Behavior Checklist (TRF; Achenbach, 1991b). Both questionnaires consist of an externalizing (33 items for the CBCL and 34 for the TRF) and internalizing (31 items for CBCL and 35 for TRF) problem behavior scale. The externalizing scale consists of two subscales, measuring aggressive (e.g., defiant and talks back, disrupts class discipline) and rule-breaking (e.g., steals, fights) behaviors. The internalizing scale comprises withdrawn-depressed (e.g., likes to be alone, withdrawn), somatic complaints (e.g., over-tired, headaches) and anxious-depressed (e.g., fears mistakes, needs to be perfect) subscales. To examine more precise associations within broad-band externalizing and internalizing scales, models were examined focusing on aggressive versus rule-breaking subscale anxious-depressed versus withdrawn-depressed subscales.
Analytic strategy
Aim 1 – To examine the factor structure of the ICU
First, we computed inter-item polychoric correlations for the ICU using primary and alternative caregiver reports2. We then used CFA in Mplus version 7.2 (Muthén & Muthén, 2014) to compare model fit for one-factor, correlated three-, 3FBF, and revised 2F solutions for the primary caregiver-reported ICU. We also tested the 3FBF and revised 2F model using alternative-caregiver reports enabling corroboration of model fit within our sample. Models were estimated with mean and variance adjusted weighted least squares estimation (WLSMV) for use with ordinal items (Flora & Curran, 2004). We considered model fit to be adequate if the Root Mean Square Error of Approximation (RMSEA) and Comparative Fit Index (CFI) values met guidelines (i.e., RMSEA<.06 and CFI >.95; Hu & Bentler, 1999). Because we used WLSMV estimation, we carried out corrected chi-square differences test with DIFFTEST in Mplus (Muthén & Muthén, 2014). We examined descriptive statistics and zero-order correlations between summed ICU total and subscale scores for the 3FBF and revised 2F solutions. Internal consistencies of summed ICU scores were assessed using Cronbach’s alpha.
Aim 2 – To test cross-sectional and longitudinal construct validity of the ICU
We computed cross-sectional and longitudinal zero-order correlations between age 9.5 summed total and subscale primary caregiver-reported ICU scores and both primary caregiver- and teacher-reported externalizing and internalizing scores at ages 9.5 and 10.5. Next, we computed a series of path models to examine the prediction of primary caregiver- and teacher-reported externalizing and internalizing problem behaviors by 3FBF general and specific factor scores. We computed separate models to examine prediction of aggressive versus rule-breaking behavior and anxious-depressed versus anxious-withdrawn behavior. We examined cross-sectional associations at age 9.5 and longitudinal associations with age 10.5 scores (controlling for autoregressive effects). We were thus able to examine the pattern of findings for a general CU factor when variance in specific factors was accounted for and vice versa. However, for purposes of enabling comparison with studies that have computed summed total and subscale scores and for practical translation, we also examined associations with aggressive versus rule-breaking behavior within regression models using summed scores. As before, to be transparent with the data, we computed separate models for cross-sectional and longitudinal associations and examined both within and across informant associations. Finally, we compared findings for regression models examining associations between ICU summed scores and aggressive versus rule-breaking behavior when summed scores were based on the 3FBF solution versus the revised 2F solution. In all models examining cross-sectional and longitudinal associations with externalizing and internalizing subscales, we controlled for intervention status, project location, child gender, race, and ethnicity, parent education, and family income.
Results
Factor structure of the ICU
We computed polychoric correlations among items of the primary caregiver-reported ICU (Table 2). There were modest-moderate correlations among items3. Consistent with previous studies, we dropped item 10, ‘does not let feelings control him/her’ (e.g., Ciucci et al., 2014) as higher endorsement of this item was not related to endorsement of other ICU items. It thus appeared that raters were interpreting item 10 as indexing a desirable behavior. In addition, items 15 (‘always tries his/her best’) and 23 (‘works hard on everything’), which have similar item content, were highly related (r =.75). We dropped item 23 as it caused difficulties in the model estimation stage, which appeared to be related to an issue of multicollinearity. We found similar associations among items using alternative-caregiver reports (not shown for brevity, but available on request). We thus computed all models using 22 of the original 24 ICU items.
Table 2.
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1. Expresses his/her feelings open | |||||||||||||||||||||||
2. Does not know right from wrong. | .10 | ||||||||||||||||||||||
3. Is concerned about school work | .24 | .19 | |||||||||||||||||||||
4. Does not care who s/he hurts | .10 | .52 | .16 | ||||||||||||||||||||
5. Feels bad or guilty | .36 | .30 | .37 | .39 | |||||||||||||||||||
6. Does not show emotions | .39 | .51 | .05 | .30 | .27 | ||||||||||||||||||
7. Does not care about being on time | .09 | .33 | .15 | .37 | .07 | .26 | |||||||||||||||||
8. Concerned about feelings of others | .32 | .43 | .33 | .56 | .61 | .29 | .16 | ||||||||||||||||
9. Does not care if s/he is in trouble | .19 | .43 | .34 | .57 | .39 | .36 | .31 | .32 | |||||||||||||||
10. Does not let feelings control | −.21 | −.07 | −.06 | −.06 | −.19 | −.05 | −.003 | −.23 | −.001 | ||||||||||||||
11. Does not care about doing things well | .13 | .39 | .20 | .44 | .25 | .31 | .52 | .22 | .51 | .05 | |||||||||||||
12. Seems very cold and uncaring. | .22 | .57 | .18 | .53 | .39 | .57 | .33 | .39 | .55 | −.005 | .47 | ||||||||||||
13. Easily admits to being wrong. | .42 | .30 | .30 | .40 | .45 | .16 | .24 | .41 | .31 | −.20 | .26 | .39 | |||||||||||
14. It is easy to tell how s/he is feeling | .53 | .23 | .22 | .24 | .42 | .41 | .20 | .41 | .28 | −.14 | .21 | .37 | .45 | ||||||||||
15. Always tries his/her best. | .26 | .35 | .31 | .42 | .34 | .23 | .44 | .37 | .40 | −.21 | .55 | .35 | .38 | .42 | |||||||||
16. Apologizes (“says s/he is sorry”) | .35 | .36 | .31 | .49 | .59 | .35 | .26 | .52 | .43 | −.19 | .41 | .39 | .54 | .51 | .58 | ||||||||
17. Tries not to hurt others feelings | .30 | .51 | .26 | .50 | .48 | .31 | .26 | .58 | .46 | −.26 | .39 | .52 | .48 | .42 | .55 | .70 | |||||||
18. Shows no remorse | .25 | .50 | .23 | .58 | .45 | .44 | .24 | .42 | .57 | −.08 | .41 | .58 | .35 | .28 | .35 | .49 | .38 | ||||||
19. Is expressive and emotional | .33 | .05 | .15 | .02 | .30 | .35 | .15 | .26 | .07 | −.02 | .17 | .14 | .21 | .45 | .23 | .26 | .24 | .04 | |||||
20. Does not put time into things | .08 | .40 | .17 | .38 | .15 | .24 | .43 | .18 | .39 | −.02 | .68 | .30 | .31 | .12 | .54 | .29 | .29 | .37 | .001 | ||||
21. Feelings of others are unimportant | .13 | .51 | .13 | .51 | .27 | .33 | .25 | .38 | .52 | −.08 | .46 | .52 | .24 | .21 | .29 | .37 | .37 | .60 | .01 | .31 | |||
22. Hides his/her feelings | .34 | .24 | .14 | .23 | .10 | .47 | .19 | .16 | .19 | −.03 | .29 | .50 | .18 | .31 | .12 | .14 | .12 | .25 | .14 | .18 | .25 | ||
23. Works hard on everything | .23 | .21 | .27 | .29 | .33 | .11 | .37 | .31 | .35 | −.15 | .48 | .17 | .39 | .30 | .75 | .51 | .45 | .27 | .26 | .54 | .23 | .05 | |
24. Does things to make others feel good | .37 | .34 | .31 | .40 | .44 | .35 | .23 | .53 | .39 | −.14 | .37 | .42 | .43 | .46 | .49 | .63 | .65 | .39 | .30 | .30 | .30 | .19 | .53 |
Note. Item 10 inversely related to other ICU items and excluded from subsequent analyses. High overlap between items 23 and 15 caused problems in model estimation stage; item 23 excluded from subsequent analyses.
We examined one-factor, three-correlated factor, 3FBF, 3FBF with correlated residuals, and revised 2F models for the primary caregiver-reported ICU (Table 3 and Figure 1). The one-factor model showed poor fit to the data although the moderate loadings of all 22 items onto a general factor were notable and hinted at shared variance among items. The three-correlated factor model fit the data better than the one-factor model (Δχ2(3) = 254.76, p < .001); however, the fit was still poor. The 3FBF model fit the data better than a three-correlated factor model (Δχ2(19) = 224.24, p < .001), but the fit was only just acceptable and a number of items on the uncaring subscale loaded in the opposite direction and several were non-significant (Table 3). Because of this poor fit, we examined modification indices. We allowed error terms of five pairs of items to correlate, although we only specified correlations when there was overlap in item content (Figure 1). However, only one of five item pairs was the same as specified in a previous study (see Fanti et al., 2009 who specified 16 pairs of items to correlate, including one overlapping pair with us, items 15 and 20). It is noteworthy, however, that other studies incorporating modification indices have not consistently specified how many items (e.g., Essau et al., 2006) or which items were specified to correlate (e.g., Ciucci et al., 2014). Based on items having negative items loadings on their specific factors, we specified four items (8, 3, 5, & 13) to only have loadings on the general factor. This 3FBF model with modification indices showed good fit to the data (Table 3). We also examined the fit of this 3FBF model specifying the same correlated residuals for the alternative caregiver-reported ICU. Loadings were similar and the fit was good, demonstrating corroboration of our 3FBF model across informants (Table 3). We also compared model fit for males versus females. We conducted multi-group analyses comparing model fit when factor loadings and intercepts were fixed versus freed using the DIFFTEST procedure. We found that the fixed model showed significantly better fit, suggesting that loadings were similar across males and females. Finally, we examined a revised 2F model with 12 ICU items (Hawes et al., 2014). This newly-proposed two-factor solution showed good model fit for both primary caregiver (see Table 3) and alternative caregiver reports (χ2(53) = 126.98, p < .001; CFI = .98; RMSEA = .05; details available on request). However, because the 2F model is based on a different item set than the 3FBF, we could not compare model fit directly.
Table 3.
One factor | Three correlated factor | Three-factor bifactor | Three-factor bifactor with MI | Three-factor bifactor with MI (alternative caregiver report) | Two-factor | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||
Callous | Uncar | Unemo | G | Callous | Uncar | Unemo | G | Callous | Uncar | Unemo | G | Callous | Uncar | Unemo | Callous | Uncar | ||
1. Expresses feelings openly | .47*** | .63*** | .42*** | .52*** | .43*** | .51*** | .51*** | .65*** | ||||||||||
6. Does not show emotions. | .55*** | .71*** | .51*** | .39*** | .52*** | .38*** | .42*** | .23*** | .55*** | |||||||||
14. Easy to tell how s/he is feeling | .60*** | .81*** | .56*** | .54*** | .58*** | .52*** | .50*** | .36*** | ||||||||||
19. Expressive and emotional | .35*** | .47*** | .30*** | .43*** | .31*** | .42*** | .24*** | .58*** | ||||||||||
22. Hides feelings from others | .35*** | .45*** | .30*** | .35*** | .29*** | .37*** | .28*** | .33*** | ||||||||||
2. Does not know right from wrong | .60*** | .65*** | .54*** | .37*** | .53*** | .42*** | .37*** | .49*** | ||||||||||
4. Does not care who is hurts | .68*** | .73*** | .61*** | .38*** | .60*** | .44*** | .77*** | .36*** | .78*** | |||||||||
7. Does not care about being on time | .44*** | .48*** | .36*** | .40*** | .30*** | .46*** | .16*** | .68*** | ||||||||||
8. Concerned about feelings of others | .68*** | .73*** | .73*** | −.07ns | .73*** | - | .73*** | - | .73*** | |||||||||
9. Does not care if in trouble | .64*** | .69*** | .56*** | .43*** | .55*** | .47*** | .52*** | .49*** | .74*** | |||||||||
11. Does not care about doing well | .65*** | .70*** | .51*** | .63*** | .45*** | .51*** | .38*** | .56*** | .61*** | |||||||||
12. Seems very cold and uncaring | .69*** | .73*** | .63*** | .37*** | .61*** | .43*** | .71*** | .37*** | .77*** | |||||||||
18. Shows no remorse | .66*** | .71*** | .59*** | .43*** | .58*** | .48*** | .55*** | .49*** | .78*** | |||||||||
20. Does not put the time into things | .56*** | .60*** | .42*** | .55*** | .33*** | .44*** | .31*** | .43*** | ||||||||||
21. Feelings of others unimportant | .56*** | .61*** | .46*** | .48*** | .45*** | .55*** | .35*** | .48*** | .67*** | |||||||||
3. Concerned about school work | .40*** | .41*** | .43*** | −.17* | .43*** | - | .40*** | - | ||||||||||
5. Feels bad or guilty | .65*** | .68*** | .70*** | −.16* | .70*** | - | .68*** | - | .70*** | |||||||||
13. Easily admits to being wrong | .60*** | .62*** | .63*** | −.02ns | .64*** | - | .69*** | - | ||||||||||
15. Always tries his/her best | .68*** | .71*** | .71*** | .07ns | .61*** | .29*** | .59*** | .38*** | ||||||||||
16. Apologizes to persons | .80*** | .82*** | .80*** | .19*** | .79*** | .29*** | .78*** | .27*** | .83*** | |||||||||
17. Tries not to hurt others feelings | .78*** | .81*** | .77*** | .42*** | .77*** | .35*** | .79*** | .27*** | .82*** | |||||||||
24. Does things to make others feel good | .71*** | .74*** | .71*** | .24*** | .71*** | .26*** | .68*** | .26*** | .74*** | |||||||||
| ||||||||||||||||||
Χ2 | 1510.95*** | 1096.73*** | 891.29*** | 603.32*** | 428.75*** | 173.42*** | ||||||||||||
df | df =209 | df =206 | df =187 | df =186 | df = 186 | df =53 | ||||||||||||
CFI; RMSEA | .83; 10 | .89; .08 | .91; .08 | .95; .06 | .95; 05 | .98; .05 |
Note.
p <.05;
p<.01;
p<.001.
Items paraphrased for brevity. All parent-reported ICU except if stated. MI =modification indices. In the three-factor bifactor model with MI, error terms of the following items were allowed to correlate: 15 with 07; 15 with 11; 22 with 12; 20 with 11; 20 and 15. In addition, items 8, 3, 5, and 13 were specified to only have general CU factor variance (no specific factor variance). Note that a 2BF (i.e., general CU factor and two specific factors of callous and uncaring) ran into estimation problems as the uncaring items loaded strongly (all loadings > .70) onto the general factor. Thus, the two-correlated factor appears to offer a more parsimonious method to express the variance in these two factors.
Total and subscale summed scores
Based on our 3FBF solution, we created a 22-item 3FBF total score, a 9-item callous subscale score, a 4-item uncaring score, and a 5-item unemotional subscale score. Based on the 2F model, we created a 12-item 2F total score, a 7-item callous subscale score, and a 5-item uncaring subscale score. There were moderate inter-correlations among unemotional, callous, and uncaring subscales within 3FBF summed scores (range, r = .31–.52, p < .001) and between callous and uncaring 2F scores (r = .49, p < .001) (Table 4). There was high internal consistency for total ICU scores (3FBF, α = .87; 2F, α = .84), and the callous (3FBF, α = .78; 2F, α = .76) and uncaring subscales (3FBF, α = .81; 2F, α = .84), which were similar across the two solutions, and acceptable internal consistency for the 3FBF unemotional subscale (α = .65). The total 2F score featured 12 of the 22 items included in the total 3FBF score, and not surprisingly, they were highly related (r = .94, p < .001). The 2F callous subscale score featured six of the 9 items included in the 3FBF callous subscale and an additional item that appeared in the 3FBF unemotional scale; these scales were strongly associated (r = .93, p < .001). Similarly, the 2F uncaring scale featured three of the four 3FBF uncaring items and two different items (one from 3FBF general factor with no specific variance, and one from 3FBF callous scale), and they were also strongly related (r= .90, p < .001). Based on bivariate associations, it thus appears that the refined ICU produces scale scores that are similar to those derived from the full version of the measure. Nevertheless, the utility of dropping the unemotional items (and having no unemotional subscale) required further investigation via an examination correlates. We focused our results on models computed within a 3FBF framework so we could examine associations with externalizing and internalizing outcomes for a general factor controlling for variance explained by specific factors and vice versa. However, we also computed total and subscale summed scores and examined associations with externalizing outcomes, which enabled comparability of our findings with previous studies and allowed us to compare the pattern of findings for total and subscale scores derived from the 3FBF versus 2F solutions.
Table 4.
Three-factor bifactor solution (3FBF) | Revised two-factor solution (2F) (Hawes et al., 2014) | ||||||
---|---|---|---|---|---|---|---|
| |||||||
Total | Callous | Uncaring | Unemotional | Total | Callous | Uncaring | |
Total ICU score (22 items) | |||||||
Callous subscale score (9 items) | .77*** | ||||||
Uncaring subscale score (4 items) | .84*** | .52*** | |||||
Unemotional subscale score (5 items) | .69*** | .31*** | .45*** | ||||
Total ICU score (12 items) | .94*** | .78*** | .84*** | .51*** | |||
Callous subscale score (9 items) | .75*** | .93*** | .49*** | .37*** | .82*** | ||
Uncaring subscale score (5-items) | .86*** | .48*** | .90*** | .49*** | .90*** | .49*** | |
| |||||||
M | 18.33 | 3.33 | 4.51 | 4.45 | 8.13 | 2.20 | 5.92 |
SD | 8.83 | 3.51 | 2.84 | 2.49 | 5.36 | 2.70 | 3.50 |
α | .87 | .78 | .81 | .65 | .84 | .76 | .84 |
Note.
p <.05;
p<.01;
p<.001
Cross-sectional and longitudinal construct validity of the ICU
Cross-sectional and longitudinal bivariate correlations – summed scores
There were strong associations within primary caregiver and teacher reports of externalizing problem behavior from ages 9.5–10.5, and moderate associations between primary caregiver and teacher reports of scores (see Table 5), suggesting convergence across time and informants. There were modest-to-strong zero-order associations between primary caregiver-reported ICU summed scores for the 3FBF and 2F solutions and primary caregiver-reported externalizing symptoms, including the aggressive and rule-breaking subscales (range, r =.16–.60, p < .01). The magnitude of associations was greater for total, callous, and uncaring scores than for unemotional scores. There were modest-moderate zero-order associations between 3FBF and 2F primary caregiver-reported summed ICU total scores and callous and uncaring subscale scores and teacher-reported externalizing scores (range, r =.10–.25, p < .10). Associations with 3FBF unemotional scores were smaller in magnitude and less likely to be significant (range, r = .09, ns, −.13, p < .13). For internalizing symptoms, zero-order associations within primary caregiver reports were high over time (range, r = .41–.71, p < .001). However, associations between primary caregiver and teacher reports of internalizing problem behavior were lower in magnitude (range, r =. 05, ns, −.29, p < .001; Table 6). Finally, we found moderate positive zero-order associations between primary caregiver-reported ICU and internalizing scores (range, r = .05, ns, −.44, p < .001). Associations between primary caregiver-reported ICU scores and teacher-reported internalizing were smaller and less likely to be significant, although still tended to be positive in directionality.
Table 5.
Primary caregiver-reported | Teacher-reported | |||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
age 9.5 | age 10.5 | age 9.5 | age 10.5 | |||||
| ||||||||
Agg | RuleB | Agg | RuleB | Agg | RuleB | Agg | RuleB | |
Three-factor bifactor | ||||||||
Total ICU (n =22) (PC) | .56*** | .53*** | .47*** | .44*** | .18** | .17** | .21*** | .23*** |
Callous subscale (PC) | .57*** | .55*** | .45*** | .43*** | .14** | .10† | .15** | .14* |
Uncaring subscale (PC) | .50*** | .47*** | .44*** | .40*** | .19*** | .18** | .19** | .20*** |
Unemotional subscale (PC) | .18*** | .19*** | .18*** | .16** | .09† | .11* | .09 | .13** |
| ||||||||
Revised two-factor | ||||||||
Total ICU (n =12) (PC) | .61*** | .57*** | .50*** | .47*** | .20*** | .18*** | .23*** | .25*** |
Callous subscale (PC) | .57*** | .54*** | .45*** | .43*** | .17** | .13* | .18** | .18** |
Uncaring subscale (PC) | .49*** | .45*** | .42*** | .39*** | .16** | .18** | .20*** | .23*** |
Note.
p < .10;
p <.05;
p<.01;
p<.001.
Ext =Externalizing; Agg =Aggression; RuleB =Rule-breaking; P = primary caregiver-reported; T =teacher-reported.
Table 6.
Primary caregiver-reported | Teacher-reported | |||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
age 9.5 | age 10.5 | age 9.5 | age 10.5 | |||||
| ||||||||
Anx | With | Anx | With | Anx | With | Anx | With | |
Three-factor bifactor | ||||||||
Total ICU (n =22) (PC) | .20*** | .42*** | .19*** | .37*** | .05 | .13* | .05 | .15** |
Callous subscale (PC) | .29*** | .45*** | .23*** | .36 | .02 | .10† | .08 | .16** |
Uncaring subscale (PC) | .14** | .30*** | .17** | .30*** | .08 | .13* | .05 | .09 |
Unemotional subscale (PC) | .05 | .31*** | .05 | .25*** | .02 | .13* | −.04 | .13* |
| ||||||||
Revised two-factor | ||||||||
Total ICU (n =12) (PC) | .20*** | .40*** | .21*** | .37*** | .07 | .13* | .08 | .14* |
Callous subscale (PC) | .26*** | .44*** | .22*** | .35*** | .04 | .09 | .08 | .15** |
Uncaring subscale (PC) | .11* | .27*** | .16** | .29*** | .07 | .13* | .05 | .08 |
Note.
p < .10;
p <.05;
p<.01;
p<.001.
Anx=Anxious-depressed; With=Withdrawn-depressed; PC = primary caregiver-reported; T=teacher-reported.
Cross-sectional and longitudinal construct validity – 3FBF latent model (Tables 7 & 8)
Table 7.
Within informant | Across informant | |||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
Primary caregiver reports age 9.5 | Primary caregiver reports age 10.5 | Teacher reports age 9.5 | Teacher reports age 10.5 | |||||
| ||||||||
Aggressive | Rule-breaking | Aggressive | Rule-breaking | Aggressive | Rule-breaking | Aggressive | Rule-breaking | |
Primary-caregiver-reported 3FBF model of ICU age 9.5 | ||||||||
‘General’ CU factor score | .51*** | .46*** | .09† | .16** | .11† | .11† | .16** | .22*** |
Callous specific factor | .41*** | .43*** | .06 | .08 | .10 | .03 | .01 | .03 |
Uncaring specific factor | .15*** | .15*** | .09 | .05 | .14 | .12 | −.10 | −.09 |
Unemotional specific factor | −.21*** | −.13* | −.03 | −.07 | .01 | .02 | −.13† | −.10 |
Note.
p < .10;
p <.05;
p<.01;
p<.001.
All models controlled for intervention group, gender, race, ethnicity, and project site. Models predicting age 10.5 outcomes controlled for age 9.5 primary caregiver-reported aggressive or rule-breaking behavior as relevant. Controlling for concurrent internalizing disorder did not change the pattern of findings – results are not shown for brevity.
Table 8.
Within informant | Across informant | |||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
Primary caregiver reports age 9.5 | Primary caregiver reports age 10.5 | Teacher reports age 9.5 | Teacher reports age 10.5 | |||||
| ||||||||
Withdrawn- depressed |
Anxious- depressed |
Withdrawn- depressed |
Anxious-depressed | Withdrawn- depressed |
Anxious- depressed |
Withdrawn- depressed |
Anxious-depressed | |
Primary-caregiver-reported 3FBF model of ICU age 9.5 | ||||||||
‘General’ CU factor score | .02 (.28***) | −.27*** (.08) | −.09† (.05) | −.12** (.05) | .07† (.14*) | .01 (.06) | −.06 (.06) | −.15* (.03) |
Callous specific factor | .15** (.40***) | −.05 (.30***) | −.06 (.08) | −.10* (.06) | .001 (.03) | −.03 (.001) | .19** (.10†) | .08 (.02) |
Uncaring specific factor | .02 (.12†) | .001 (.15†) | .04 (.18*) | −.01 (.13†) | −.05 (.12†) | −.02 (.07) | .01 (−.10) | .13 (−.10) |
Unemotional specific factor | .38*** (.30***) | .16*** (.07) | .11 (.09) | .03 (.02) | .12 (.30***) | .003 (.03) | .13 (−.10) | −.06 (.06) |
Note.
p < .10;
p <.05;
p<.01;
p<.001.
All models controlled for intervention group, gender, race, ethnicity, and project site. Models predicting age 10.5 outcomes controlled for age 9.5 primary caregiver-reported aggressive or rule-breaking behavior as relevant. Models predicting internalizing symptoms include controlling for externalizing disorder. However, estimates when concurrent externalizing behavior is not controlled for are shown in italics and parentheses. These results highlight cooperative suppression effects of externalizing behavior in relation to associations between ICU scores and internalizing symptoms.
In cross-sectional models at age 9.5, higher ICU general factor scores were associated with higher levels of both primary caregiver- and teacher-reported aggressive and rule-breaking behaviors (Table 7). Higher callous and uncaring specific scores were also related to higher primary caregiver-reported aggressive and rule-breaking behaviors. However, the unemotional specific factor predicted fewer primary caregiver-reported aggressive and rule-breaking behaviors, and was unrelated to teacher-reported outcomes in cross-sectional models. In longitudinal autoregressive models that controlled for earlier externalizing behavior problems, the ICU general factor significantly predicted increases in both teacher and primary caregiver-reported rule-breaking. General ICU factors scores also predicted increases in teacher-reported aggressive behavior at age 10.5 (prediction of primary caregiver-reported aggressive behavior was a trend). It is noteworthy that the effect of the ICU general factor was greater in magnitude for the prediction of rule-breaking (primary caregiver-reported, β = .16, p < .01; teacher-reported, β = .22, p < .001) compared with the prediction of aggression (primary caregiver-reported, β = .09, p < .10; teacher-reported, β = .16, p < .01). None of the specific factors accounted for unique variance in primary caregiver-reported aggressive or rule-breaking from ages 9.5–10.5.
When we examined associations with internalizing outcomes (anxious-depressed vs. withdrawn-depressed), we compared the pattern of effects when we did and did not control for concurrent externalizing behavior. In cross-sectional models, we found that a higher ICU general factor score was related to lower primary caregiver-reported anxious-depressed scores at age 9.5 (Table 8), but only after accounting for concurrent externalizing behavior. Accounting for variance explained by the general CU behavior factor however, we found that unemotional and callous specific factors were related to higher withdrawn-depressed scores, and unemotional scores were also related to higher anxious-depressed scores. There were no significant associations between the ICU general or specific factors scores and teacher-reported outcomes in cross-sectional models at age 9.5 (Table 8). In longitudinal autoregressive models (Table 8), we found that ICU general factor scores were related to decreases in primary caregiver- and teacher-reported anxious-depressed scores and primary caregiver-reported withdrawn-depressed scores from ages 9.5 to10.5, but again, only after controlling for concurrent externalizing behavior at age 10.5. Scores on the callous specific factor were also related to decreases in primary caregiver-reported anxious-depressed behavior. In line with cross-sectional models, we found that the unemotional specific factor was related to increases in primary caregiver- and teacher-reported withdrawn-depressed scores. Likewise, the callous specific factor predicted increases in teacher-reported withdrawn-depressed scores. We also report estimates when we did not control for concurrent externalizing (in italics and parentheses; Table 8), the results of which reinforce the importance of considering cooperative suppression effects between externalizing and internalizing behavior in relation to associations with CU behavior. Specifically, the direction of effects between the general ICU factor and primary caregiver-reported anxious-depressed score reversed when taking account of comorbid externalizing behavior symptoms (see Discussion).
Cross-sectional and longitudinal construct validity - summed scores
We re-examined associations with externalizing and internalizing outcomes using summed total and subscale scores within regression analyses. The pattern of findings broadly mirrored that obtained when associations were examined within a 3FBF framework. For brevity, we thus only present results from examining associations with externalizing outcomes (Table 9; results of models examining associations with internalizing outcomes available on request from authors). We also examined associations using summed scores based on the 2F model to test whether this more parsimonious set of items performed similarly to the 22-item set. Both the 3FBF and 2F produced summed total scores and callous and uncaring subscale scores that were cross-sectionally related to higher primary caregiver- and teacher-reported aggressive and rule-breaking behavior. In longitudinal autoregressive models, summed total scores of the 3FBF and 2F models predicted increases in primary caregiver and teacher reports of rule-breaking at age 10.5. In longitudinal autoregressive models, the uncaring subscale of both model solutions was related to increases in rule-breaking behavior across informant. As with the 3BF analyses however, we found that unemotional summed scores were cross-sectionally related to lower primary caregiver reported aggressive and rule-breaking behavior, accounting for overlap with other subscales4.
Table 9.
Within informant | |||||
---|---|---|---|---|---|
| |||||
Primary caregiver reports age 9.5 | Primary caregiver reports age 10.5 | ||||
| |||||
Aggressive | Rule-breaking | Aggressive | Rule-breaking | ||
3FBF | Model 1 – total ICU score | ||||
Total ICU (n =22) (PC) | .55*** | .52*** | .03 | .08* | |
| |||||
Model 2 – subscale scores | |||||
Callous subscale (PC) | .43*** | .44*** | .02 | .03 | |
Uncaring subscale (PC) | .32*** | .26*** | .03 | .08† | |
Unemotional subscale (PC) | −.11*** | −.07† | −.001 | −.02 | |
| |||||
2F | Model 3 – total revised ICU score | ||||
Total ICU (n =12) (PC) | .59*** | .56*** | .03 | .10* | |
| |||||
Model 4– total revised subscale scores | |||||
Callous subscale (PC) | .43*** | .43*** | .003 | .03 | |
Uncaring subscale (PC) | .27*** | .23*** | .03 | .08† |
Across informant | |||||
---|---|---|---|---|---|
| |||||
Teacher reports age 9.5 | Teacher reports age 10.5 | ||||
| |||||
Aggressive | Rule-breaking | Aggressive | Rule-breaking | ||
3FBF | Model 5 – total ICU score | ||||
Total ICU (n =22) (PC) | .15** | .13** | .08 | .16** | |
| |||||
Model 6 – subscale scores | |||||
Callous subscale (PC) | .10† | .04 | −.01 | .06 | |
Uncaring subscale (PC) | .09 | .11† | .11† | .10 | |
Unemotional subscale (PC) | −.01 | .02 | −.03 | −.01 | |
| |||||
2F | Model 7 – total revised ICU score | ||||
Total ICU (n =12) (PC) | .17*** | .15** | .09† | .17** | |
| |||||
Model 8– total revised subscale scores | |||||
Callous subscale (PC) | .16** | .08 | .01 | .07 | |
Uncaring subscale (PC) | .06 | .09† | .12* | .14* |
Note.
p < .10;
p <.05;
p<.01;
p<.001.
Ext =Externalizing; Agg =Aggression; RuleB =Rule-breaking; PC = primary caregiver-reported; T =teacher-reported. All models controlled for intervention group, gender, race, ethnicity, and project site. Models predicting age 10.5 outcomes controlled for age 9.5 primary caregiver-reported aggressive or rule-breaking behavior as relevant. Controlling for concurrent internalizing disorder did not change the pattern of findings – results are not shown for brevity.
Discussion
In the current study, we addressed a number of questions surrounding the parent-reported ICU. We found acceptable model fit for a 3FBF model with correlated residuals for both primary and alternative caregiver reports, thus providing some corroboration of this structure among a sample of high-risk children aged 9.5. We also found good model fit for a revised 2F model using a reduced 12-item pool, providing support for the proposal by Hawes and colleagues (2014) to focus only on callous and uncaring, and to trim item content. Total and subscale scores from the 3FBF and 2F models showed acceptable-to-high internal consistencies. An examination of the cross-sectional and longitudinal construct validity of scores within a 3FBF framework suggested that the ICU provides predictive validity in relation to covert forms of antisocial behavior indexed via a measure of rule-breaking behavior, although effect sizes of this prediction were modest in magnitude within autoregressive models. Our results speak both to the assessment of CU behavior using this measure and to the construct itself.
Aim 1 – To examine the factor structure of the ICU
We found that a 3FBF model, with modification indices guiding correlation of five pairs of items, showed the best model fit for primary and alternative caregiver reports on the ICU. It is unclear how similar our use of modification indices is to those employed in previous studies, which have not always reported the correlations specified among item residuals (e.g., Ciucci et al., 2014; Essau et al., 2006). The need to employ modification indices is a limitation associated with the ICU that we had wanted to avoid. However, the necessity for this approach in both the current study and previous studies suggests overlapping item content within and between factors that may be compounded further by similar semantic item structure (e.g., 8 items begin with ‘does not’). Further, it is noteworthy that the factor loadings of the uncaring specific factor seemed to be the most affected within the bifactor model, suggesting that ‘uncaring’ may be most closely aligned with a general ‘CU’ factor (Pardini, Hawes, Burke, & Loeber, 2014). Thus, although we replicated the most commonly-reported factor structure (i.e., 3FBF), the psychometric properties of the ICU continue to appear far from robust. Importantly, from a modeling perspective, the 2F solution of Hawes and colleagues (2014) had good model fit and appears to offers a more parsimonious assessment of a central callous and uncaring construct than the 3FBF model. This 2F model could also prove to be more stable across samples as modification indices and other specific changes were not needed. The 2F solution is supported by another recent study that assessed children aged 7–12, where CFA showed that a 2F model comprising callous and uncaring dimensions fit the data best (Houghton et al., 2013).
Aim 2 – To test cross-sectional and longitudinal construct validity of the ICU
Externalizing problem behavior
Within a 3FBF framework, the ICU general factor was cross-sectionally related to higher aggressive and rule-breaking behavior. The general factor also predicted increases in rule-breaking behavior from ages 9.5–10.5 across informants and settings. This finding highlights that by leveraging shared variance among ICU items, and controlling for unique variance of specific factors, we tapped a construct with incremental validity in relation to the development of behavior problems, particularly covert antisocial behavior. Nevertheless, the practical utility of a ‘general bifactor’ model may be limited given the inability to model a meta-factor with individual client data. Interestingly however, summed total and uncaring subscale scores implied by both the 3FBF and 2F models also predicted increases in rule-breaking from ages 9.5–10.5, after accounting for earlier rule-breaking. It was surprising that despite the widespread use of the ICU and the predominance of the 3FBF, no previous studies had examined associations of ICU scores with relevant outcome variables within a 3FBF (see Lahey, 2014). The fact that we found a similar pattern of findings when analyses were carried out using summed scores or within the 3FBF framework is striking. Ultimately, assessment is a practical enterprise, and our results provide justification for future studies to use a summed total score using either 22 ICU items or the revised 12-item ICU to create a total score with predictive utility. Further, the results suggest that uncaring items may be particularly pertinent for identifying those youth at risk for displaying increases in rule-breaking behavior among high-risk children. At the same time, effect sizes were modest in magnitude, especially for longitudinal autoregressive models, which may be due to the high stability in externalizing behavior. Nevertheless, the incremental validity of the ICU scores suggests that knowing about CU behavior as indexed via the ICU may be helpful for targeting or tailoring specific intervention or treatment components to different groups of youth (e.g., Dadds et al., 2014).
Whereas total and uncaring scores were reliably related to more externalizing problems across informants, we found that unemotional subfactor scores were related to lower scores when we controlled for variance in general CU behavior within a 3FBF framework. This effect was particularly notable for cross-sectional parent-reported aggressive and rule-breaking symptoms. As few studies have examined the longitudinal predictive validity of ICU within a 3FBF framework, we interpret our findings with caution. One possibility is that once variance in general CU behavior and specific uncaring and callousness factors is partialled, the variance remaining in unemotionality actually relates to emotional resiliency or a lack of externalizing problem behavior, and thus represents a marker of positive mental health. We return to an evaluation of the unemotional part of the CU behavior construct later in this discussion.
Internalizing problem behavior
Within a 3FBF, we obtained expected negative associations between the general CU behavior factor and anxious-depressed scores. Notably, the general factor predicted lower anxious-depressed scores at age 10.5, controlling for autoregressive effects, and across primary caregiver- and teacher-reported outcomes. Thus, our results support the notion that the shared general variance within ICU items is related to lower anxiety, in line with the defining characteristics of affective aspects of psychopathy among adult samples (Frick et al., 2014; Lykken, 1995). Findings were replicated using summed scores for the total ICU and uncaring scores implied by both the 3FBF and 2F model for primary caregiver-reported anxious-depressed scores. However, this pattern of effects only emerged when we controlled for comorbid externalizing symptoms; otherwise ICU general scores were related to higher anxious-depressed scores. This reversal in the direction of associations is notable, and represents possible cooperative suppression (see Frick et al., 2014). By partialing the variance in this way, we appeared to be accounting for comorbidity between externalizing and internalizing, which may be underpinned by some higher-order dimension, such as or behavioral dysregulation or negative emotionality (for further discussion of this issue, see Lilienfeld, 2003; Hyde, Byrd, Votruba-Drzal, & Hariri, & Manuck, 2014). Only when this overlapping negative emotionality was accounted for were we able to obtain expected negative associations between CU behavior and anxiety (also see Loney, Frick, Clements, Ellis, & Kerlin, 2003). At the same time, our results highlight that while partialing variance is a useful approach to demonstrate construct validity within the context of statistical modeling, the reality may be more complex with many children presenting with problems across multiple domains of functioning.
In contrast to the negative association between ICU total scores and anxious-depressed symptoms, we found callousness to be related to higher withdrawn-depressed scores even after controlling for concurrent externalizing problems. It may be that this finding emerged as a statistical artifact with a 3FBF framework. However, a similar pattern of findings was found using summed scores in regression models. Further, Hawes and colleagues (2014) also reported a positive association between ICU callous scores and measures of internalizing behavior problems. Thus, when variance in other subscales or a general CU behavior factor is partialled, it appears that callousness may relate to parents or teachers endorsing children as being socially-withdrawn, isolated, or low in mood. This finding highlights the need for careful consideration of the wording of callous items. In particular, it is noteworthy that the uncaring subscale is comprised of positively-worded items, whereas the callous subscale is negatively-worded. Future studies are needed to examine whether the same pattern of associations is achieved when callousness is assessed with positive-worded items and uncaring with negatively-worded items. Moreover, it may indicate that the suppression effects discussed above do not generalize to all types of internalizing, only those focused on symptoms of anxiety (vs. depression).
The meaning of ‘unemotional’
We found that when callous, uncaring, or general factor scores were controlled for, unemotional behavior was related to more withdrawn-depressed symptoms. In conjunction with the finding that unemotional scores were related to lower aggressive and rule-breaking scores, our results provide thus support for the conclusions of Hawes and colleagues (2014) that consideration needs to be given to conceptualizations of ‘unemotional’ among children and adolescents. In particular, the ICU unemotional items may not be doing a good job of capturing ‘unemotionality’ as it relates to the nomological network of CU behavior. Interestingly, in a previous paper assessing this sample in the preschool years (Hyde et al., 2013a), we developed a ‘home-grown’ measure of CU-like behavior and found that traditional ‘unemotional’ items did not load with items indexing callousness and uncaring.
Alternatively, it may be that ‘unemotionality’ as indexed by the ICU is interpreted by informants as withdrawn, anhedonic, or shy behavior, which differs somewhat from conceptualizations of the unemotional component of CU behavior indexing reduced anxiety or fearlessness (cf., low fear in psychopathy; Lykken, 1995). In support of this notion, Ezpeleta and colleagues (2013) reported a positive association between teacher-reported unemotionality and anxiety among preschoolers. A review of individual cases suggested that teachers did appear to be rating ‘unemotional’ children as shy, socially phobic, and unable to express feelings (Ezpeleta et al., 2013, p. 102). At the same time, it is possible that once variance in callous and uncaring are partialled, unemotionality predicts covert aggression not captured in either our aggressive and rule-breaking subscales (e.g., lying that remains undetected; relational aggression).
Another explanation is that the general CU behavior factor acts as a suppressor variable, obscuring the relationship of the unemotional subfactor with criterion-related variables when considered in the 3FBF framework. Indeed, in zero-order correlations, unemotional scores were associated with higher externalizing, albeit with effect sizes that were smaller in magnitude than for those for callous or uncaring. However, this conclusion is difficult to reconcile with the positive associations we found between unemotional and internalizing problem behaviors. Further, while a lack of emotional responsivity has been documented at a neurobiological level for antisocial youth with CU behavior (i.e., reduced amygdala responsivity to others’ fear and reduced responsivity to punishment; see Hyde, Shaw, & Hariri, 2013b), it is less clear that children with high CU behavior are, in fact, less emotionally expressive, as implied by the ICU unemotional items (e.g., ‘hides feelings from others’). Indeed, previous studies have shown that adolescents with high CU behavior display significant negative emotionality, including anger, anxiety, and depression (e.g., Kimonis, Frick, Cauffman, Goldweber, & Skeem, 2012).
As such, there is a need for refinement of items to capture more accurately what is meant by ‘unemotional’. For example, some youth may be callous/uncaring but display fearlessness and low anxiety, whereas other may be callous/uncaring and show high levels of anxious or fearful behavior as a result of early environmental risk (Kimonis et al., 2012; Waller et al., 2013). Indeed, classic descriptions of psychopathy typically focus on low fear rather than a lack of emotionality in general (Lykken, 1995). Thus, revised items are needed to capture distinction in CU behavior in the context of negative emotional responses (including anger or anxiety) versus low levels of fear. Incorporating assessment of temperamental dimensions, such as negative emotionality or prosociality may help to differentiate between antisocial subgroups of youth with and without CU behavior (Lahey, 2014). At the same time, one item of the original unemotional scale (‘does not show emotions’) was retained in the shortened ICU, albeit as part of the callous factor (Hawes et al., 2014). Thus, it appears that continued consideration of the emotional displays of children (or lack thereof) is warranted to provide the affective context for any callous or uncaring behavior (see Rowe, 2014). However, our results relating to unemotionality highlight that the CU behavior construct, at least that assessed by the ICU, is relatively narrow and indexes only some of the personality traits linked to adult psychopathy. Thus the CU behavior construct should not be synonymously equated with psychopathy.
Implications for assessment of the CU construct
There are a number of implications for assessment of the CU construct. First, the results provide justification for practitioners to assess CU behavior using a sum of 22 ICU items, as implied by our 3FBF, or using the revised 12-item set proposed by Hawes and colleagues (2014) which both added unique variance in relation to the prediction of future behavior problems across informants and settings. Use of these ICU summed scores may help in the diagnosis of the DSM-5 ‘limited prosocial emotions’ specifier. Second, the 3FBF implies separable components of the CU behavior construct. However, we also found unexpected correlations with criterion-related variables that undermine conceptualizations of the subfactors, particularly the unemotional subfactor. The ‘unemotional’ items do not appear to operate as intended in the nomological basis of CU behavior and thus may not be useful clinically or conceptually. Taken together, these points suggest that in using the ICU for assessment, the most meaningful and reliable predictive validity is derived via use of a latent general or summed total score. However, our results also highlight the need for alternative self-, parent-, or teacher-reported measures of CU behavior with stronger psychometric properties. Further, the need for alternative methodologies is implied, especially in relation to how items are interpreted (i.e., callousness as withdrawal or anhedonia). One new assessment approach, the ‘Clinical Assessment of Prosocial Emotions’ (CAPE; Frick, 2013) is currently under development, and is designed to assess CU traits via semi-structured and self-reported interviews. Finally, our analyses were conducted within a dimensional framework, and as such, we cannot speak to the application of the ICU within person-centered analyses or for categorical “diagnosis”. However, evidence from neuroimaging studies has demonstrated divergent patterns of amygdala reactivity among youth with Conduct Disorder based on their level of CU behavior (e.g., amygdala hyporeactivity to threat for high CU behavior versus amygdala hyperreactivity to threat for low CU behavior; Viding et al., 2012), which represents a strong, objective test of the discriminant validity of ICU scores. In this example, a cut-off score was created using a median split on ICU total scores (Viding et al., 2012, p. 1110). However, future studies are needed to establish the validity of cutoff scores using the ICU or other measures of CU behavior, including the CAPE.
Strengths and limitations
There were a number of strengths to the study, including a fairly large sample size, having children all assessed at the same age, use of a prospective longitudinal design, use of multiple informants, and corroboration of findings across informants. However, there were several limitations. First, we focused on low-income children with risk factors across multiple domains, including sociodemographic risk and early child problem behavior. Thus, it is unclear whether our results would generalize to children from higher-income families with fewer risk factors. At the same time, our findings represent a useful complement to previous studies that have examined the ICU factor structure and that have focused on adolescents (e.g., Fanti et al., 2009), community or school samples (e.g., Ezpeleta et al., 2013), and incarcerated youth (Kimonis et al., 2008). Results from the current study likely bridge the distribution across these studies in having some children low and others very high on externalizing. Second, while we collected data from teachers for externalizing and internalizing outcomes, we only had primary and alternative caregiver reports for the ICU. Nevertheless, it remains unclear who the best informant is for assessing youth CU behavior and our results suggest a need to investigate alternative ways to find about these emerging aspects of personality, especially in relation to concerns regarding the validity of self-reported measures of psychopathy (e.g., Lilienfeld, 1994). Finally, there was some attrition in our sample by 10.5 years old. We used FIML to accommodate missing data for models testing prediction by ICU scores for the 540 families for whom we had ICU data at age 9.5, although it is unclear whether the prediction of outcomes by primary caregiver-reported ICU scores would have differed among those lost to follow-up.
Conclusions
We found support for use of a general CU behavior score based on 22-items of the ICU, which was related to future rule-breaking behavior and lower anxious-depressed behavior problems over a one year period across informants and settings. Thus, our study supports the use of the ICU among high-risk children to identify those at risk for developing covert forms of antisocial behavior. However, the effect size of predictions was not large, and future work is indicated for better assessment and identification of youth with particularly accelerating forms of externalizing behavior problems. In particular, our study presents a number of questions moving forward for this relatively young field. While there appears to be utility in using a total score that captures shared variance across this set of items, more psychometric work is needed to examine the item wording and specific correlates of subfactors, particularly the unemotional subfactor. Indeed, total summed ICU scores derived from a refined 12-item version of the measure showed comparable longitudinal predictive validity to the original item set. Further, the model fit for the 2F solution for these 12 items was very good, and this model appears to offer a parsimonious method of assessing callous and uncaring behaviors with more robust psychometric support.
Acknowledgments
This research was supported by Grant 5R01 DA16110 from the National Institutes of Health, awarded to Dishion, Shaw, Wilson, & Gardner. Aidan Wright’s efforts were supported by Grant F32 MH097325 from the National Institute of Mental Health. We thank families and staff of the Early Steps Multisite Study and Dustin Pardini and Samuel Hawes for helpful comments on an earlier draft of the manuscript. We also thank three anonymous reviewers and the editor for valuable feedback during the review process.
Footnotes
Note that we label CU ‘traits’ as behaviors through the rest of the manuscript because, particularly at the age we focus on, we are primarily assessing observable behaviors and it remains an open empirical question as to how ‘trait-like’ the construct is (see Waller, Gardner, Hyde, 2013).
For ease of interpretation, we reverse coded all ‘positively’ worded items from the ICU so that all items indexed higher CU traits score
Note, throughout all the correlational analyses, we considered estimates of .15–.30 to indicate ‘modest’ correlation; estimates of .30–.50 to indicate ‘moderate’ correlation; and estimates greater than .50 to indicate ‘strong’ correlation.
Note that we also examined whether gender moderated associations between ICU scores and outcomes, both cross-sectionally and longitudinally. Only 1 of 12 models tested featured a significant interaction (p < .05) thus for brevity, we do not report results.
Conflict of interest statement: No conflicts declared.
Contributor Information
Rebecca Waller, Department of Psychology, University of Michigan.
Aidan G.C. Wright, Department of Psychology, University of Pittsburgh
Daniel S. Shaw, Department of Psychology, University of Pittsburgh
Frances Gardner, Center for Evidence-Based Social Intervention, Department of Social Policy and Intervention, University of Oxford.
Thomas J. Dishion, Department of Psychology, University of Arizona
Melvin N. Wilson, Department of Psychology, University of Arizona
Luke W. Hyde, Department of Psychology, Center for Human Growth and Development, Survey Research Center of the Institute for Social Research, University of Michigan
References
- Achenbach TM. Manual for the Child Behavior Checklist/4–18 and 1991 Profiles. Burlington, VT: University of Vermont, Department of Psychiatry; 1991a. [Google Scholar]
- Achenbach TM. Manual for the Teacher Report Form and 1991 Profiles. Burlington, VT: University of Vermont, Department of Psychiatry; 1991b. [Google Scholar]
- American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5. Washington, DC: Author; 2013. [Google Scholar]
- Berg JM, Lilienfeld SO, Reddy SD, Latzman RD, Roose A, Craighead LW, Pace TW, Raison CL. The Inventory of Callous and Unemotional Traits: A construct-validation analysis in an at-risk sample. Assessment. 2013;20(5):532–544. doi: 10.1177/1073191112474338. [DOI] [PubMed] [Google Scholar]
- Carroll JB. Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press; 1993. [Google Scholar]
- Ciucci E, Baroncelli A, Franchi M, Golmaryami FN, Frick PJ. The association between callous-unemotional traits and behavioral and academic adjustment in children: further validation of the inventory of callous-unemotional traits. Journal of Psychopathology and Behavioral Assessment. 2014;36(2):189–200. [Google Scholar]
- Dadds M, Allen J, McGregor K, Woolgar M, Viding E, Scott S. Callous-unemotional traits in children and mechanisms of impaired eye contact during expressions of love: A treatment target? Journal of Child Psychology and Psychiatry. 2014;55:771–780. doi: 10.1111/jcpp.12155. [DOI] [PubMed] [Google Scholar]
- Dishion T, Shaw D, Connell A, Gardner F, Weaver C, Wilson M. The Family Check-Up with high-risk indigent families: Preventing problem behavior by increasing parents’ positive behavior support in early childhood. Child Development. 2008;79:1395–1414. doi: 10.1111/j.1467-8624.2008.01195.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Essau CA, Sasagawa S, Frick PJ. Callous-unemotional traits in a community sample of adolescents. Assessment. 2006;13:454–469. doi: 10.1177/1073191106287354. [DOI] [PubMed] [Google Scholar]
- Ezpeleta L, Osa NDL, Granero R, Penelo E, Domènech JM. Inventory of callous-unemotional traits in a community sample of preschoolers. Journal of Clinical Child & Adolescent Psychology. 2013;42:91–105. doi: 10.1080/15374416.2012.734221. [DOI] [PubMed] [Google Scholar]
- Fanti K, Frick PJ, Georgiou S. Linking callous-unemotional traits to instrumental and non-instrumental forms of aggression. Journal of Psychopathology and Behavioral Assessment. 2009;31:285–298. [Google Scholar]
- Feilhauer J, Cima M, Arntz A. Assessing callous unemotional traits across different groups of youths: Further cross cultural validation of the Inventory of Callous-Unemotional Traits. International Journal of Law and Psychiatry. 2012;35:251–262. doi: 10.1016/j.ijlp.2012.04.002. [DOI] [PubMed] [Google Scholar]
- Flora D, Curran P. An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods. 2004;9:466–491. doi: 10.1037/1082-989X.9.4.466. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Frick PJ, O’Brien BS, Wootton JM, McBurnett K. Psychopathy and conduct problems in children. Journal of Abnormal Psychology. 1994;103:700–707. doi: 10.1037//0021-843x.103.4.700. [DOI] [PubMed] [Google Scholar]
- Frick PJ. Inventory of Callous-Unemotional Traits. University of New Orleans; 2004. Unpublished rating scale. [Google Scholar]
- Frick PJ, White SF. Research review: The importance of callous-unemotional traits for developmental models of aggressive and antisocial behavior. Journal of Child Psychology & Psychiatry. 2008;49:359–375. doi: 10.1111/j.1469-7610.2007.01862.x. [DOI] [PubMed] [Google Scholar]
- Frick PJ. The Clinical Assessment of Prosocial Emotions: Version 1.1 (CAPE 1.1) University of New Orleans; 2013. [Accessed, July 2014]. Unpublished manual. from http://labs.uno.edu/developmental-psychopathology/documents/CAPE%20Manual[1].pdf. [Google Scholar]
- Frick PJ, Ray JV, Thornton LC, Kahn RE. Can callous-unemotional traits enhance the understanding, diagnosis, and treatment of serious conduct problems in children and adolescents? A comprehensive review. Psychological Bulletin. 2014;140:1– 57. doi: 10.1037/a0033076. [DOI] [PubMed] [Google Scholar]
- Hawes SW, Byrd AL, Henderson CE, Gazda RL, Burke JD, Loeber R, Pardini DA. Refining the parent-reported inventory of callous–unemotional traits in boys with conduct problems. Psychological Assessment. 2014;26:256–266. doi: 10.1037/a0034718. [DOI] [PubMed] [Google Scholar]
- Houghton S, Hunter S, Crow J. Assessing callous unemotional traits in children aged 7-to 12-years: a confirmatory factor analysis of the inventory of callous unemotional traits. Journal of Psychopathology and Behavioral Assessment. 2013;35:215–222. [Google Scholar]
- Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:424–453. [Google Scholar]
- Hyde LW, Shaw DS, Gardner F, Cheong J, Dishion TJ, Wilson M. Dimensions of callousness in early childhood: Links to problem behavior and family intervention effectiveness. Development and Psychopathology. 2013a;25:347–363. doi: 10.1017/S0954579412001101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hyde LW, Shaw DS, Hariri AR. Understanding youth antisocial behavior using neuroscience through a developmental psychopathology lens: Review, integration, and directions for research. Developmental Review. 2013b;33:168–223. doi: 10.1016/j.dr.2013.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hyde LW, Byrd A, Votruba-Drzal E, Hariri A, Manuck S. Amygdala reactivity and negative emotionality: Divergent correlates of antisocial personality and psychopathy traits in a community sample. Journal of Abnormal Psychology. 2014;123:214–224. doi: 10.1037/a0035467. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kimonis ER, Frick PJ, Skeem JL, Marsee MA, Cruise K, Munoz LC, Aucoin KJ, Morris AS. Assessing callous-unemotional traits in adolescent offenders: Validation of the Inventory of Callous- Unemotional Traits. International Journal of Law and Psychiatry. 2008;31:241–252. doi: 10.1016/j.ijlp.2008.04.002. [DOI] [PubMed] [Google Scholar]
- Kimonis ER, Frick PJ, Cauffman E, Goldweber A, Skeem J. Primary and secondary variants of juvenile psychopathy differ in emotional processing. Development and Psychopathology. 2012;24:1091. doi: 10.1017/S0954579412000557. [DOI] [PubMed] [Google Scholar]
- Lahey BB. What we need to know about callous-unemotional traits: Comment on Frick, Ray, Thornton, and Kahn (2014) Psychological Bulletin. 2014;140:58–63. doi: 10.1037/a0033387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lilienfeld SO. Comorbidity between and within childhood externalizing and internalizing disorders: reflections and directions. Journal of Abnormal Child Psychology. 2003;31:285–291. doi: 10.1023/a:1023229529866. [DOI] [PubMed] [Google Scholar]
- Lilienfeld S. Conceptual problems in the assessment of psychopathy. Clinical Psychology Review. 1994;14:17–38. [Google Scholar]
- Loney B, Frick P, Clements C, Ellis M, Kerlin K. Callous-unemotional traits, impulsivity, and emotional processing in adolescents with antisocial behavior problems. Journal of Clinical Child and Adolescent Psychology. 2003;32:66–80. doi: 10.1207/S15374424JCCP3201_07. [DOI] [PubMed] [Google Scholar]
- Lykken DT. The antisocial personalities. Hillsdale, NJ: Lawrence Erlbaum; 1995. [Google Scholar]
- Muthén LK, Muthén BO. Mplus. Los Angeles, CA: Muthén & Muthén; 2014. [Google Scholar]
- Pardini D, Stepp S, Hipwell A, Stouthamer-Loeber M, Loeber R. The clinical utility of the proposed DSM-5 callous-unemotional subtype of conduct disorder in young girls. Journal of the American Academy of Child & Adolescent Psychiatry. 2012;51:62–73. doi: 10.1016/j.jaac.2011.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pardini D, Hawes S, Burke J, Loeber R. Structural Coherence and External Relationships of the Revised ICU. Presentation at the Annual Conference of the American Psychology-Law Society; New Orleans. 2014.2014. [Google Scholar]
- Patrick C, Hicks B, Nichol P, Krueger R. A bifactor approach to modeling the structure of the psychopathy checklist-revised. Journal of Personality Disorders. 2007;21:118–141. doi: 10.1521/pedi.2007.21.2.118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Robinson E, Eyberg S, Ross A. The standardization of an inventory of child conduct problem behaviors. Journal of Clinical Child Psychology. 1980;9:22–28. [Google Scholar]
- Roose A, Bijttebier P, Decoene S, Claes L, Frick PJ. Assessing the affective features of psychopathy in adolescence: A further validation of the Inventory of Callous and Unemotional Traits. Assessment. 2010;17:44–57. doi: 10.1177/1073191109344153. [DOI] [PubMed] [Google Scholar]
- Rowe R. Commentary: Integrating callous and unemotional traits into the definition of antisocial behavior–a commentary on Frick et al. (2014) Journal of Child Psychology and Psychiatry. 2014;55:549–552. doi: 10.1111/jcpp.12253. [DOI] [PubMed] [Google Scholar]
- Somech LY, Elizur Y. Promoting self-regulation and cooperation in pre-kindergarten children with conduct problems: A randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry. 2012;51:412–422. doi: 10.1016/j.jaac.2012.01.019. [DOI] [PubMed] [Google Scholar]
- Tackett JL, Daoud SL, De Bolle M, Burt SA. Is relational aggression part of the externalizing spectrum? A bifactor model of youth antisocial behavior. Aggressive Behavior. 2013;39:149–159. doi: 10.1002/ab.21466. [DOI] [PubMed] [Google Scholar]
- Viding E, Sebastian C, Dadds M, Lockwood P, Cecil C, De Brito S, McCrory E. Amygdala response to preattentive masked fear in children with conduct problems: the role of callous-unemotional traits. American Journal of Psychiatry. 2012;169:1109–1116. doi: 10.1176/appi.ajp.2012.12020191. [DOI] [PubMed] [Google Scholar]
- Waller R, Gardner F, Hyde L. What are the associations between parenting, callous-unemotional traits, and antisocial behavior in youth? A systematic review of evidence. Clinical Psychology Review. 2013;33(4):593–608. doi: 10.1016/j.cpr.2013.03.001. [DOI] [PubMed] [Google Scholar]
- White SF, Cruise KR, Frick PJ. Differential correlates to self-report and parent-report of callous-unemotional traits in a sample of juvenile sexual offenders. Behavioral Sciences & the Law. 2009;27:910–928. doi: 10.1002/bsl.911. [DOI] [PubMed] [Google Scholar]
- White SF, Frick PJ, Lawing K, Bauer D. Callous–Unemotional Traits and Response to Functional Family Therapy in Adolescent Offenders. Behavioral Science and the Law. 2013;31:271–285. doi: 10.1002/bsl.2041. [DOI] [PubMed] [Google Scholar]