Assessment of Psychopathic Traits in an Incarcerated Adolescent Sample: A Methodological Comparison

Abstract

Analyses of convergent validity and group assignment using self-report, caregiver-report and interview-based measures of adolescent psychopathy were conducted in a sample of 160 incarcerated adolescents. Results reveal significant convergent validity between caregiver-report measures of adolescent psychopathy, significant convergent validity between self-report measures of adolescent psychopathy and an interviewer rating scale, but not between the caregiver-report measures and their corresponding self-report measures nor between the caregiver-report measures and the interviewer rating scale. Analyses of group assignment were also poorer than expected among all the measures with none evidencing significant agreement with the expert-rated device (Hare Psychopathy Checklist-Youth Version; PCL-YV), the most common forensic instrument used in clinical practice. Part of the poor agreement may be related to the poor psychometric performance of the callous-unemotional subscale of most of these measures and the low response rates from caregivers (N=35). These findings suggest that the measures do not provide an interchangeable assessment of callous-unemotional traits and suggest that further refinement of the measurement of callous-unemotional traits in youth may be warranted.

Psychopathy is a severe personality disturbance comprised of a constellation of maladaptive behaviors including glibness, pathological lying, lack of remorse and callousness, comorbid with a history of impulsive, antisocial, parasitic, and conning behaviors. This modern conception of psychopathy, including the current scoring criteria, is most directly influenced by the work of Cleckley (1941, 1976). Currently, the most widely used measure to assess psychopathy in adults is the Psychopathy Checklist-Revised (PCL-R; Hare 1991, 2003). The PCL-R combines a semi-structured clinical interview with comprehensive file review to assess 20 items derived from Cleckley’s (1941, 1976) work. Tests of reliability and predictive validity have won the measure its popularity and widespread use in forensic settings (Bolt et al. 2004; Hare 2003).

Given the widespread forensic utility of the PCL-R in adults, researchers have begun to test the applicability of the psychopathy construct in children and adolescents. Within the DSM-IV-TR nomenclature, Conduct Disorder (CD; American Psychiatric Association 2000) most closely resembles the youth psychopathy construct, however, some argue that the diagnostic criteria for CD over-relies on disruptive behavior and fails to capture the interpersonal and affective features of these behavioral problems; those that are thought to be core features of psychopathy and at the root of violent and antisocial behavior (Frick et al. 1994; Hare 1991, 1998, 2003). Moreover, previous studies have shown psychopathic traits in youth are predictive of violent offending in adulthood (Corrado et al. 2004; Gretton et al. 2004; Vincent et al. 2008) which suggests the potential utility of the construct above and beyond the assessment of conduct problems in understanding aggressive and antisocial youth (Farrington 2005; Frick 2007; Lynam 1997).

Early psychometric studies of adolescent psychopathy employed a modified version of the PCL-R which omitted or altered items not applicable to juveniles and found significant associations between the adolescent modified PCL-R scores and conduct problems in adolescents (Brandt et al. 1997; Forth et al. 1990; Myers et al. 1995; Rogers et al. 1997). Based on this work, Forth and colleagues developed a youth version of the Psychopathy Checklist (PCL-YV; Forth et al. 2003). Like the PCL-R, the PCL-YV is a semi-structured interview augmented by file-based review rating scale, but is developed for use with male and female offenders aged 12 to 18. The current literature suggests that the PCL-YV successfully captures adolescent features of the adult psychopathy construct (Forth and Burke 1998; Kosson et al. 2002). A review of 19 studies found that like in adults, scores on the PCL-YV vary as a function of setting with incarcerated adolescents scoring higher than adolescents on probation and adolescents on probation scoring higher than adolescents in the community (Forth et al. 2003).

Early factor analysis of the PCL-YV revealed a two-factor model similar to that found in adults (Harpur et al. 1989; Neumann et al. 2006), isolating psychopathic personality features from antisocial behaviors (Frick et al. 1994). Frick (1995) described this distinction as callous-unemotional (CU) behavior versus an impulsive/antisocial (I/A) behavioral pattern. The initial finding of the two-factor model in adolescents sparked a flurry of research into the callous-unemotional behavior exhibited by these youth (Barry et al. 2000; Christian et al. 1997; Dadds et al. 2005; Jones and Viding 2007). Some of these findings include youth with callous-unemotional traits exhibiting worse conduct problems, but not experiencing poorer quality of parenting than youth who exhibit conduct problems alone (Edens et al. 2008; Wootton et al. 1997). Also, the conduct problems of youth exhibiting callous-unemotional traits appear less influenced by environmental factors, such as low socio-economic status, school and neighborhood influences (Viding et al. 2005; Viding and Jones 2008), and have a more stable pattern of severe conduct problems (Frick et al. 2003, 2005).

Researchers argued, however, that the two-factor model did not adequately capture the construct of adolescent psychopathy and that three or four-factor models would be more descriptive and appropriate (Cooke and Michie 2001). Subsequent studies have found that three and four factor models were, indeed, better fits for adolescent data than the two factor model (Neumann et al. 2006; Salekin et al. 2006). For example, studies comparing the PCL-YVand a self-report measure of psychopathy, the Youth Psychopathic traits Inventory (YPI; Andershed et al. 2002), have reported somewhat conflicting results. Dolan and Rennie (2006) found that YPI scores correlated with the behavioral factor of the PCL-YV while Skeem and Cauffman (2003) found that the YPI correlated more strongly with the interpersonal factor of the PCL-YV. Similarly mixed results have been found in the predictive and incremental validity of the PCL-YV and the Child Psychopathy Scale (CPS-S; Lynam 1997) and the Antisocial Process Screening Device - Youth Version (APSD-Y; Frick and Hare 2001; Falkenbach et al. 2003; Murrie and Cornell 2002). Other investigators have found that PCL-YV scores were more strongly correlated with a violent offense history than a caregiver version of the APSD scores (Murrie et al. 2004; Salekin 2008). In contrast, an examination of a modified form of the CPS-S, APSD-P and the PCL-YV found that the self-report measures more strongly related to institutional infractions and the days required to progress in treatment than the PCL-YV (Spain et al. 2004).

With the increased consideration of psychopathic features in antisocial youth, debate over the validity of the use of such a construct with youth has ensued (Hart et al. 2002; Petrila and Skeem 2003). Some contend that the relative malleability of the youth personality and the negative consequences of false positive psychopathy classifications in juvenile courts warrant careful consideration of the construct and the measures used to assess it (Seagrave and Grisso 2002; Sharp and Kine 2008). For example, Edens et al. (2001) argue that behaviors such as the need for stimulation, impulsivity, lack of realistic long-term goals, and irresponsibility are features of adolescence, itself, and measurement of them using psychopathy measures could be inappropriately inflate the presence of these behaviors in youth with conduct problems. Counter-arguments support the use of such measures because in doing so, high-risk youth have the potential to be identified for more intensive treatment interventions (Forth et al. 2003), thus staving off negative outcomes, such as substance abuse and continued criminal behavior, in adulthood. Regardless of the argument made for or against the use of this construct in youth, close inspection of the performance of these youth psychopathy measures is necessary to avoid false-positive identification and the potential negative outcomes associated with such an incorrect identification (Frick 2002).

Current Study

Because of conflicting results in the discriminate validity of these measures, the convergent validity of these assessments remains in question. Also in question is the convergent validity of these assessments with criminal convictions. Previous studies have shown that PCL was correlated with the number of institutional charges for aggressive and violent behavior and previous violent charges and convictions (Forth et al. 1990), but little is known about the relationship between the rest of these instruments and criminal convictions. The current study aimed to address these issues by administering the most commonly used assessments of youth psychopathy to a sample of maximum-security adolescent inmates. The four self-report measures administered were the APSD, the ICU (Inventory of Callous-Unemotional Traits – Youth version; Frick 2003), the CPS and YPI. In addition, caregiver versions of the APSD, ICU and CPS were also administered as was the PCL-YV. It was hypothesized that given the construction of the self-report and the caregiver-report (same measure and scoring with one adapted to a caregiver) that these measures would have good convergent validity. Because anti-social attitudes and behavior in adolescents are often better assessed by self-report than parent or caregiver-report (Jolliffe et al. 2003; Kamphaus and Frick 2002), it was also hypothesized that the self-report measures would have stronger convergent validity with the PCL-YV than would the caregiver-report measures. In addition, the widespread use of the PCL-YV relies on its ability to accurately delineate group membership (i.e., those high on psychopathy from those low on psychopathy). A second aim of this study was to assess the agreement of group assignment among the aforementioned self, caregiver and interview-based measures of youth psychopathy and criminal convictions. It was further hypothesized that the self-report measures would have good agreement with the PCL-YV in group assignment.

Method

Participants

Participants were 190 adolescents (22.5% female) incarcerated at a southwestern US juvenile maximum-security detention facility and were part of a larger functional neuroimaging study investigating the neurocognition of callous conduct disordered youth (NIMH R01 MH071896). Participants were incarcerated for crimes that included murder, attempted murder, manslaughter, armed robbery, assault, domestic assault, rape, arson, weapons possession, burglary, theft, fraud, drug possession/distribution, probation/parole violations and criminal mischief. The mean age of the sample participants was 17 (SD=1.00, range 14–19) years of age at the time of assessment. Thirty adolescent participants withdrew from the study. Nine of these participants were transferred to another juvenile facility, 17 of these participants were excluded for fMRI safety reasons and four stated that they no longer wished to participate leaving 160 adolescents for the final analyses. Attempts to obtain caregiver-report materials were made from all caregivers of the 160 adolescents, but despite repeated attempts to obtain this information, only 35 parents or guardians submitted completed caregiver-report information. Sample adolescent participants were 73% Hispanic,¹ 11% Caucasian, 7% American Indian or Alaska Native, 6% Black or African American and 3% more than one race or ethnicity.

Inclusion Criteria

Participants included in the current study met the inclusion criteria for the parent study that were as follows: (a) currently incarcerated, (b) between 12 and 19 years old, (c) right-handed, (d) no history of head injury resulting in a loss of consciousness greater than 30 min, (e) no history of psychosis or first degree relative with psychosis, (f) a fourth grade English reading level and (g) estimated IQ greater than 80.

Procedures and Ethical Considerations

Initial contact with potential study participants was made through announcements by research staff at the facility. Meetings were scheduled with the guardians of interested youth where parental/guardian informed consent and minor assent were obtained. Participants were informed of their right to discontinue participation at any point during the course of the study. Participants were also informed that their participation was in no way associated with their status at the facility, their probation status and that there were no direct benefits to them. Participant remuneration was paid at the rate of the hourly wage at the facility. All procedures were approved by the Human Research Review Committee at the research institution and correctional facility where the study was conducted.

The self-report assessments and PCL-YV were conducted in two separate sessions with individual participants to minimize participant fatigue and as a design procedure to minimize the influence of method variance resulting from instruments being administered at the same time. Participants who consented to participate in the study were interviewed in private offices at the facility by bachelor and master level research staff trained in the administration and scoring of the PCL-YV. Demographic and collateral information, and criminal convictions were obtained from institutional files.

Assessment Materials

Antisocial Process Screening Device - Youth Version (APSD-Y)

The APSD (Frick and Hare 2001) is a 20—item self-report scale developed to assess behavior in youth ages nine to 18 that is similar to the adult construct of psychopathy and assessed by the Psychopathic Checklist – Revised (PCL-R; Hare 1991). Each item is scored either 0 (not at all), 1 (sometimes true), or 2 (definitely true). Factor analyses reveal three dimensions: Narcissism, Impulsivity and Callous-Unemotional (Frick et al. 2000). The total ASPD-Y has shown adequate internal consistency reliability of 0.78–0.81, however, the internal consistency reliability for the subscales is more moderate, ranging from 0.50 to 0.68 (Munoz and Frick 2007). In the current study, the Cronbach alpha for the Total scale was 0.75, 0.71 for the Narcissism subscale, 0.57 for the Impulsivity subscale, but only 0.32 for the Callous-Unemotional. For scales that yield poor reliability (usually less than 0.70), especially if the scale consists of fewer than 10 items, it is recommended that mean inter-item correlation and the range of the values be inspected and reported (Cortina 1993; Pallant 2007). Values below 0.3 on these indices indicate that the items are not correlating well with the total scale and that the item may be measuring something different from the scale as a whole. The inter-item correlations for the Impulsivity subscales were below what is generally considered acceptable (mean 0.21; range 0.02 to 0.38), and the mean inter-item correlation for the Callous-Unemotional subscale were largely within unacceptable ranges (mean 0.08, range −0.29 to 0.47).

Antisocial Process Screening Device - Parent Version (APSD-P)

The APSD-P (Frick and Hare 2001) is a 20-item scale directly translated from the APSD-Y to be completed by a parent or other guardian for youth ages nine to 18. In the current study, the Cronbach alpha for the Total Scale was 0.89, 0.80 for the Narcissism dimension, 0.65 for the Impulsivity dimension, and 0.63 for the Callous-Unemotional traits dimension. The inter-item correlation values somewhat low for the Impulsivity and Callous-Unemotional dimensions (mean 0.28, range 0.06 to 0.44; 0.23, range −0.02 to 0.55, respectively).

Child Psychopathy Scale - Self-Report Version (CPS-S)

The CPS-S (Lynam 1997) is a 50-item self-report measure adapted for youth ages 12 to 18 to directly assess the 13 of the 20 items measured by the PCL-R (glibness, untruthfulness, boredom susceptibility, manipulation, lack of guilt, poverty of affect, callousness, parasitic lifestyle, behavioral dyscontrol, lack of planning, impulsiveness, unreliability, and failure to accept responsibility). Each item is scored 0 (No) or 1 (Yes) and reflect the respondent’s personal style of doing things as well as manner in which he or she gets along with other people. The CPS-S correlates significantly with other measures of delinquency (Lynam 1997) and has been shown to have adequate internal consistency: 0.87 for the Total Scale, 0.68 for the Callous-Uemotional subscale and 0.71 for the Antisocial Behavior subscale (Spain et al. 2004). In the current study, The Cronbach alpha for the Total Scale was 0.84, 0.81 for the Antisocial Behavior subscale and 0.67 for the Callous-Unemotional subscale. Inter-item correlations for the Callous-Unemotional subscale were largely unacceptable (mean 0.07; range −0.82 to 0.75).

Child Psychopathy Scale - Caregiver version (CPS-C)

The CPS-C (Lynam 1997) is a 52-item measure completed by a parent or guardian for youth ages 12 to 18. All but two items are direct translations of the CPS-S. Two additional items were added to the caregiver version: “Is s/he able to see how other people feel?” and “Does s/he feel things very strongly? Are his/her feelings intense?” In the current study, the Cronbach alpha for the Total scale was 0.84, 0.82 for the Antisocial Behavior scale and 0.67 for the Callous-Unemotional scale. The inter-item correlations for the Callous-Unemotional scale were largely unacceptable (mean 0.07; range −0.82 to 0.75).

Inventory of Callous-Unemotional Traits - Youth Version (ICU-Y)

The ICU (Frick 2003) is a 24-item self-report measure developed from the Callous-Unemotional scale of the Antisocial Process Screening Device (APSD; Frick and Hare 2001) for use in youth ages 13 to 18. The ICU was developed to overcome the limitations of this scale of the APSD which has demonstrated only moderate internal consistency reliability largely due to the small number of items and three point rating system (Munoz and Frick 2007). The six items that encompass the Callous-Unemotional scale of the APSD were expanded to the 24-items of the ICU and put on a four-point Likert type scale from 0 (not at all true) to 3 (definitely true). Factor analyses reveal three factors: Callousness, Uncaring and Unemotional and adequate internal consistency reliability (0.73) (Kimonis et al. 2008a, b). In the current study, the Cronbach alpha for the Total scale was 0.82, 0.81 for the Callousness scale, 0.80 for the Uncaring scale and 0.60 for the Unemotional scale. The mean inter-item correlations for the Unemotional scale were largely unacceptable (mean 0.23; range 0.04 to 0.63).

Inventory of Callous-Unemotional Traits - Parent Version (ICU-P)

The parent version of ICU (Frick 2003) is a 24-item scale that is a direct translation of the items of the ICU-Y to be completed by a parent or guardian for youth ages 13 to 18. In the current study, the Crohbach alpha for the Total scale was 0.89, 0.83 for Callousness, 0.84 for Uncaring and 0.72 for Unemotional.

Youth Psychopathic traits Inventory (YPI)

The YPI (Andershed et al. 2002) is a 50-item self-report measure designed to measure core features of psychopathy in youth ages 12 to 20. The YPI consists of ten subscales with factor analyses showing these subscales form three factors: Grandiose-Maniuplative (including dishonest charm, grandiosity, lying and manipulation), Callous-Unemotional (including remorselessness, unemotionality and callousness) and Impulsive-Irresponsible (including thrill-seeking, impulsivity and irresponsibility) (Andershed et al. 2002). The YPI has shown good convergent validity with other measures of antisocial and callous-unemotional traits (Andershed et al. 2007; Dolan and Rennie 2006; Skeem and Cauffman 2003). The test-retest reliability has also indicated good stability at 0.73 (Skeem and Cauffman 2003). In the current study, the Cronbach alpha for the Total scale was 0.92, 0.90 for the Grandiose-Manipulative subscale, 0.81 for the Callous-Unemotional subscale, and 0.85 for the Impulsive-Irresponsible subscale.

Psychopathy Checklist - Youth Version (PCL-YV)

The PCL-YV (Forth et al. 2003) is a 20-item interviewer-rating scale similar to the Psychopathy Checklist – Revised, Second Edition (PCL-R, Hare 2003) developed to assess the behavior of youth ages 12 to 18. Factor analyses reveal four facets: Interpersonal, Affective, Lifestyle, and Antisocial. Inter-rater reliability in the present study was calculated for 20% of the interviews and reveal alpha of 0.89 for Total Score, 0.70 for Interpersonal, 0.78 for Affective, 0.82 for Lifestyle, and 0.84 for Antisocial facets. In the current study, Cronbach alpha for the Total Score was 0.82, 0.64 for the Interpersonal facet, 0.64 for Affective facet, 0.57 for Lifestyle facet, and 0.59 for Antisocial facet. The inter-item correlations for the Interpersonal and Affective facets were generally within acceptable ranges (mean 0.31, range 0.22 to 0.44; mean 0.32, range 0.12 to 0.58, respectively). The mean inter-item correlations for the Lifestyle and Antisocial facets were, however, low (mean 0.22, range 0.03 to 0.34; mean 0.29, range 0.07 to 0.61, respectively).

Criminal Convictions

Criminal convictions were acquired from the participants’ criminal records. Total convictions were calculated for each participant as well as coded as violent or nonviolent.

Results

Statistical Package for the Social Sciences 16.0 (SPSS) was used to perform the statistical analyses of the data for this study. To increase the ecological validity of the study, all convergent validity and group classification analyses were conducted on the published subscales of the measures. The published subscales are also those most likely be used in clinical and forensic settings. The means and standard deviations for the youth psychopathy assessment measures and criminal convictions are presented in Table 1.

Table 1.

Intercorrelations, means and standard deviation of total scale youth psychopathy assessment measures

Measure	1	2	3	4	5	6	7	8	9	10	11	M	SD
1. APSD-P	–	0.45 n=37	0.68* n=31	0.41 n=34	0.74* n=40	0.28 n=35	0.25 n=34	0.11 n=44	−0.09	−0.14	−0.03	17.64	7.03
2. APSD-Y		–	0.38 n=27	0.78* n=89	0.32 n=33	0.62* n=131	0.62* n=115	0.36* n=145	−0.03	−0.06	0.00	15.17	4.57
3. CPS-C			–	0.29 n=26	0.65* n=29	0.21 n=27	0.22 n=25	0.06 n=32	0.11	−0.10	0.21	24.44	7.56
4. CPS-S				–	0.47 n=31	0.65* n=111	0.67* n=100	0.44* n=122	−0.06	0.10	−0.07	20.69	7.38
5. ICU-P					–	0.16 n=32	0.35 n=31	−0.04 n=40	−0.15	−0.29	0.00	29.85	10.49
6. ICU-Y						–	0.47* n=114	0.27* n=140	0.04	−0.02	0.06	26.68	8.40
7. YPI							–	0.35* n=121	−0.03	0.05	−0.05	119.64	21.00
8. PCL-YV								–	0.15	0.10	0.14	22.74	5.48
9. Total Convictions									–	0.32	0.95*	7.67	6.74
10. Violent Convictions										–	0.00	1.56	2.19
11. Nonviolent Convict.											–	5.95	6.34

– = data not applicable. APSD-P Antisocial Process Screening Device (Parent Version) (APSD-P; Frick and Hare 2001); APSD-Y Antisocial Process Screening Device (Youth Version) (APSD-Y; Frick and Hare 2001); CPS-C Child Psychopathy Scale (Caregiver Version) (CPS-C; Lynam 1997); CPS-S Child Psychopathy Scale (Self-report Version) (CPS-S; Lynam 1997); ICU-P Inventory of Callous-Unemotional Traits (Parent Version) (ICU-P; Frick 2003); ICU-Y Inventory of Callous-Unemotional Traits (Youth Version) (ICU-Y; Frick 2003); PCL-YV Psychopathy Checklist - Youth Version (PCL-YV; Forth et al. 2003); YPI Youth Psychopathic Traits Inventory (YPI; Andershed et al. 2002). The Bonferroni procedure was used to hold the familywise (FW) Type 1 error rate at p_FW=0.05; the Type 1 error rate per individual test was set at p_T=p_FW/28=0.0018. p_FW is based on the familywise Type 1 error rate; p_T is based on the Type 1 error rate per measurement procedure test.

^*p_FW<0.05; p_T<0.0018. Correlation coefficients in bold denote relationship significant at p<0.05

Analyses of Convergent Validity

The relationships between the measures of youth psychopathy were investigated using Pearson product–moment correlation coefficient and are presented in Table 1. The Bonferroni procedure was used to hold the familywise (FW) Type 1 error rate at p_FW=0.05. Preliminary analyses were also performed to ensure no violation of the assumptions of normality, linearity and homoscedasticity. The four self-report total scale scores (APSD-Y, CPS-S, ICU-Y, and YPI) were significantly correlated with the total scale score of the PCL-YV (r=0.36, r=0.44, r=0.27, r=0.35, p_T<0.0018, respectively). In addition, all self-report total scale scores were significantly intercorrelated (see Table 1). In contrast, none of the parent-report total scale scores (APSD-P, CPS-C, and ICU-P) were significantly correlated with the PCL-YV (r=0.11, r=0.06, r=−0.04, p_T>0.0018), nor did they correlate with their corresponding self-report scales (APSD-Y, CPS-S or ICU-Y). The parent-report total scale scores did, however, significantly correlate with each other (see Table 1). Surprisingly, no scale was correlated with the total number of criminal convictions, number of violent criminal conviction or number of nonviolent criminal convictions (see Table 1).

The self- and caregiver-report measure subscales and PCL-YV facet inter-correlations are presented in Table 2. There are a few patterns among these inter-correlations that stand in relief. For example, the callous-unemotional sub-scale of the YPI significantly correlated with the Affective facet of the PCL-YV (r=0.32, p_T<0.0018), but it was the only YPI subscale to correlate with any of the PCL-YV facets. In addition, the unemotional and callous subscales of the ICU-Y and the narcissism subscale of the APSD-Y did not significantly correlate with any of the PCL-YV facets nor did any of the subscales of the APSD-P, ICU-P and CPS-C (Table 3). Subscale inter-correlations between the self-report measures (APSD-Y, CPS-S and ICU-Y) and their corresponding caregiver-report versions can be found in Table 4. Like the measure total scale inter-correlations, there were no significant relations between the self-report measure subscales and the parent-report subscales.

Table 2.

Intercorrelations of Youth Psychopathy Self-Report and PCL-YV Assessment Measures Subscales/Facets

Measure	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18
APSD-Y
1. Narcissism	–	0.48*	0.19	0.35*	0.53*	0.45*	0.21	0.13	0.60*	0.38*	0.34*	0.22	0.14	0.18	0.11	−0.06	−0.05	−0.04
2. Impulsivity		–	0.12	0.61*	0.50*	0.44*	0.16	0.15	0.34*	0.48*	0.40*	0.16	0.06	0.26	0.28*	0.00	−0.12	0.05
3. CU			–	0.43*	0.52*	0.41*	0.65*	0.31*	0.05	0.21	0.32*	0.08	0.45*	0.28*	0.39*	0.03	0.06	0.02
CPS-S
4. Behavioral				–	0.51*	0.38*	0.43*	0.12	0.22	0.54*	0.39*	0.03	0.29	0.32*	0.44*	−0.02	−0.09	0.00
5. CU					–	0.60*	0.53*	0.27	0.53*	0.60*	0.45*	0.21	0.40*	0.21	0.25	−0.03	0.09	−0.07
ICU-Y
6. Callous						–	0.36*	0.12	0.42*	0.42*	0.57*	−0.04	0.23	0.24	0.12	0.04	0.09	0.01
7. Uncaring							–	0.27	0.10	0.12	0.33*	−0.03	0.40*	0.15	0.34*	0.08	−0.03	0.10
8. Unemotional								–	0.04	0.13	0.14	0.10	0.10	0.02	0.12	0.06	0.09	0.03
YPI
9. Grandiosity									–	0.51*	0.55*	0.27	0.07	0.16	0.13	−0.10	0.04	−0.12
10. Impulsivity										–	0.60*	0.12	0.18	0.28	0.27	0.09	−0.14	0.13
11. CU											–	0.17	0.32*	0.22	0.25	0.00	0.06	−0.03
PCL-YV
12. Interpersonal												–	0.37*	0.31*	0.26	−0.06	0.03	−0.06
13. Affective													–	0.45*	0.52*	0.16	0.13	0.13
14. Lifestyle														–	0.48*	0.20	0.09	0.19
15. Anti-social															–	0.15	0.05	0.15
16. Total Convictions																–	–	–
17. Violent Convictions																	–	–
18. Nonviolent Convictions																	–

– = data not applicable. CU = Callous-Unemotional. The Bonferroni procedure was used to hold the familywise (FW) Type 1 error rate to the same as that for the Total Scale, p_FW=0.05; the Type 1 error rate per individual test was set at p_T=p_FW/28=0.0018. p_FW is based on the familywise Type 1 error rate; p_T is based on the Type 1 error rate per measurement procedure test.

^*p_FW<0.05; p_T< 0.0018. Correlation coefficients in bold denote relationship significant at p<0.05

Table 3.

Intercorrelations of youth psychopathy caregiver-report, YPI and PCL-YV assessment measures subscales/facets

Measure	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18
APSD-P
1. Narcissism	–	0.76*	0.56*	0.50	0.70*	0.40	0.56*	0.51*	0.31	0.10	0.32	0.06	0.18	0.03	0.32	−0.12	−0.11	−0.09
2. Impulsivity		–	0.53*	0.67*	0.46	0.34	0.50*	0.47	0.21	0.04	0.07	−0.14	−0.06	0.14	0.37	0.06	−0.15	0.18
3. CU			–	0.41	0.33	0.53*	0.54*	0.32	0.19	0.26	0.25	0.15	−0.09	0.16	0.19	−0.05	−0.06	−0.02
CPS-C
4. Behavioral				–	0.59*	0.52*	0.61*	0.34	0.24	0.21	0.16	−0.06	−0.17	0.26	0.41	0.03	−0.24	0.20
5. CU					–	0.32	0.43	0.24	0.11	0.06	0.27	0.19	0.19	−0.13	0.36	0.09	0.01	0.12
ICU-P
6. Callous						–	0.43	0.12	0.27	0.32	0.29	−0.16	−0.04	0.08	0.19	−0.10	−0.31	0.05
7. Uncaring							–	0.45	0.34	0.24	0.08	−0.08	−0.18	0.15	0.18	−0.09	−0.25	0.05
8. Unemotional								–	0.09	−0.05	−0.03	−0.03	−0.15	−0.07	0.00	−0.12	−0.07	−0.07
YPI
9. Grandiosity									–	0.55*	0.51*	0.27	0.07	0.16	0.13	−0.10	0.04	−0.12
10. Impulsivity										–	0.58*	0.12	0.18	0.28	0.27	0.09	−0.14	0.13
11. CU											–	0.17	0.32*	0.22	0.25	0.00	0.06	−0.03
PCL-YV
12. Interpersonal												–	0.37*	0.31*	0.26	−0.06	0.03	−0.06
13. Affective													–	0.45*	0.52*	0.16	0.13	0.13
14. Lifestyle														–	0.48*	0.20	0.09	0.19
15. Anti-social															–	0.15	0.05	0.15
16. Total Convictions																–	–	–
17. Violent Convictions																	–	–
18. Nonviolent Convictions																	–

– = data not applicable. CU = Callous-Unemotional. The Bonferroni procedure was used to hold the familywise (FW) Type 1 error rate to the same as that for the Total Scale, p_FW=0.05; the Type 1 error rate per individual test was set at p_T=p_FW/28=0.0018. p_FW is based on the familywise Type 1 error rate; p_T is based on the Type 1 error rate per measurement procedure test.

^*p_FW<0.05; p_T< 0.0018. Correlation coefficients in bold denote relationship significant at p<0.05

Table 4.

Intercorrelations between caregiver-report and corresponding self-report assessment measures subscales

Measure	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
APSD-P
1. Narcissism	–	0.76*	0.56*	0.50	0.70*	0.40	0.56*	0.51*	0.47	0.25	0.23	0.35	0.40	0.33	0.31	0.27
2. Impulsivity		–	0.53*	0.67*	0.46	0.34	0.50*	0.47	0.46	0.25	0.17	0.46	0.23	0.24	0.15	0.15
3. CU			–	0.41	0.33	0.53*	0.54*	0.32	0.33	0.25	0.23	0.25	0.20	0.09	0.11	0.22
CPS-C
4. Behavioral				–	0.59*	0.52*	0.61*	0.34	0.34	0.20	0.12	0.35	0.18	0.12	0.08	0.09
5. CU					–	0.32	0.43	0.24	0.27	0.07	0.20	0.13	0.25	0.14	0.13	0.19
ICU-P
6. Callous						–	0.43	0.12	0.26	0.05	0.10	0.30	0.40	0.14	0.21	−0.05
7. Uncaring							–	0.45	0.30	0.30	0.02	0.49	0.24	0.06	0.09	0.17
8. Unemotional								–	0.16	0.18	0.02	0.17	0.14	0.11	0.18	0.05
APSD-Y
9. Narcissism									–	0.48*	0.19	0.35*	0.53*	0.45*	0.21	0.13
10. Impulsivity										–	0.12	0.61*	0.50*	0.44*	0.16	0.15
11. CU											–	0.43*	0.52*	0.41*	0.65*	0.31*
CPS-S
12. Behavioral												–	0.51*	0.38*	0.43*	0.12
13. CU													–	0.60*	0.53*	0.26
ICU-Y
14. Callous														–	0.36*	0.11
15. Uncaring															–	0.27
16. Unemotional																–

– = data not applicable. CU = Callous-Unemotional. The Bonferroni procedure was used to hold the familywise (FW) Type 1 error rate to the same as that for the Total Scale, p_FW=0.05; the Type 1 error rate per individual test was set at p_T=p_FW/28=0.0018. p_FW is based on the familywise Type 1 error rate; p_T is based on the Type 1 error rate per measurement procedure test.

^*p_FW<0.05; p_T< 0.0018. Correlation coefficients in bold denote relationship significant at p<0.05

Group Classification

In addition to analyses of convergent validity, analyses were conducted to evaluate the extent to which assessment procedures agreed in the classification of offender youth. Classification agreement was evaluated using the Kappa (κ) Measure of Agreement and the procedure established in Hare (1983, 1985). A Kappa coefficient may be interpreted as the consistency of two different diagnostic tests in determining group assignment (Pallant 2007). A Kappa coefficient of 1.00 represents complete agreement and 0 represents chance agreement. According to Peat (2001) a value of 0.50 for Kappa represents moderate agreement, above 0.70 represents good agreement and above 0.80 represents very good agreement. A Kappa of 0.40 or below represents poor agreement (Landis and Koch 1977). Because of the large number of coefficients calculated among the instruments (28), the family-wise Type 1 error rate for each set of combinations was held at 0.05 by testing individual Kappas for significance at the 0.0018 level.

Three-Groups Analyses

In the first set of analyses, each distribution was divided into three groups based on the distribution of scores around the mean. For example, the low group consisted of scores less than 0.5 SD below the mean, the medium group consisted of scores 0.5 SD below the mean to 0.5 SD above the mean and the high group consisted of scores greater than 0.5 SD above the mean for each distribution. Although, there were Kappa coefficients among some of the self-report measures (APSD-Y & CPS-S, κ=0.46; APSD-Y & ICU-Y, κ=0.21; APSD-Y & YPI, κ=0.38; CPS-S & ICU-Y, κ=0.32; CPS-S & YPI, κ=0.48) and among some of the caregiver-report measures (APSD-P & ICU-P, κ=0.36; CPS-C & ICU-P, κ=0.37) were statistically significant, none reached moderate agreement. In addition, none of the self-report measures or the caregiver-report measures reached statistically significant agreement with the PCL-YV or statistically significant agreement with total criminal convictions, violent convictions or nonviolent convictions.

Two-Groups Analyses

Inspection of the data indicated that a portion of the poor agreement between the measures in the three-groups analyses was related the adolescents in the medium-psychopathy group. Because of this and because low and high groups are often used in research investigations, a second set of Kappa coefficients were calculated using the low and high scores from each distribution. These results can be found in Table 5. Comparing only these extreme scores improved agreement for the same distributions that were statistically significant in the three group analyses with agreement ranging from moderated to perfect. As in the three-group analyses, no distribution reached statistically significant agreement with the PCL-YV or statistically significant agreement with total criminal convictions, violent convictions or nonviolent convictions, however.

Table 5.

Kappa coefficients of agreement and intraclass correlation coefficients between high- and low-psychopathy groups defined by each measurement procedure

Measure		1	2	3	4	5	6	7	8	9	10	11
1. APSD-P	κ	–	0.52	0.57	0.41	1.00*	0.10	−0.04	0.22	−0.21	0.07	−0.06
	ICC		0.54 n=17	0.59 n=12	0.42 n=17	1.00* n=18	0.66 n=11	−0.09 n=17	0.23 n=18	−0.25 n=17	0.07 n=11	0.22 n=13
2. APSD-Y	κ		–	1.00*	0.83*	0.67	0.42*	0.77*	0.28	−0.03	−0.17	−0.16
	ICC			1.00* n=10	0.83* n=58	0.62* n=17	0.75* n=50	0.77* n=54	0.29 n=60	−0.03 n=60	−0.17 n=38	−0.03 n=41
3. CPS-C	κ			–	0.46	1.00*	0.70	0.33	−0.09	0.00	−0.40	−0.24
	ICC				0.48 n=11	1.00* n=12	0.73 n=7	0.36 n=8	−0.10 n=12	0.00 n=8	−0.42 n=7	0.19 n=7
4. CPS-S	κ				–	0.76*	0.80*	0.81*	0.32	−0.02	−0.05	−0.12
	ICC					0.76* n=17	0.81* n=52	0.81* n=53	0.32 n=58	−0.02 n=55	−0.05 n=41	0.09 n=43
5. ICU-P	κ					–	0.06	0.39	0.11	0.08	−0.20	0.25
	ICC						0.40 n=13	0.41 n=16	0.12 n=18	0.10 n=14	−0.22 n=10	−0.18 n=8
6. ICU-Y	κ						–	0.57*	0.32	−0.10	−0.10	−0.02
	ICC							0.57* n=46	0.19 n=56	−0.10 n=64	−0.11 n=34	0.00 n=49
7. YPI	κ							–	0.32	0.08	−0.08	0.13
	ICC								0.32 n=56	0.08 n=53	−0.08 n=33	0.03 n-40
8. PCL-YV	κ								–	0.08	0.06	0.17
	ICC									0.08 n=70	0.06 n=47	−0.13 n=53
9. Total Convictions										–	–	–
10. Violent Convictions											–	–
11. Nonviolent Convictions												–

– = data not applicable. κ = Kappa Coefficient of Agreement. ICC = Intraclass Correlation Coefficient. The Bonferroni procedure was used to hold the familywise (FW) Type 1 error rate at p_FW=0.05; the Type 1 error rate per individual test was set at p_T=p_FW/28=0.0018. p_FW is based on the familywise Type 1 error rate; p_T is based on the Type 1 error rate per measurement procedure test.

^*p_FW<0.05; p_T<0.0018

Median-Split Analyses

Because some investigators divide a sample into two groups at the median, a third set of Kappa coefficients were calculated. Please refer to Table 6. This methodology resulted in the poorest agreement and reduced the number of distributions that reached statistical significance and resulted in only two that reached moderate agreement. No distribution reached statistically significant agreement with total criminal convictions, violent convictions or nonviolent convictions.

Table 6.

Kappa coefficients of agreement and intraclass correlation coefficients between median-split low- and high- psychopathy groups defined by each measurement procedure

Measure		1	2	3	4	5	6	7	8	9	10	11
1. APSD-P	κ	–	0.25	0.35	0.24	0.65*	0.10	0.02	0.14	−0.07	0.07	−0.01
	ICC		0.25 n=37	0.36 n=31	0.24 n=34	0.66* n=40	0.10 n=35	0.02 n=34	0.14 n=44	−0.07 n=39	0.09 n=39	−0.02 n=39
2. APSD-Y	κ		–	0.32	0.36*	0.37	0.42*	0.56*	0.16	−0.05	0.00	−0.04
	ICC			0.33 n=27	0.37* n=117	0.38 n=33	0.42* n=131	0.57* n=115	0.16 n=145	−0.05 n=36	0.00 n=137	−0.05 n=136
3. CPS-C	κ			–	0.23	0.38	−0.04	0.08	0.19	−0.06	−0.14	0.02
	ICC				0.24 n=26	0.39 n=29	−0.04 n=27	0.08 n=25	0.19 n=32	−0.06 n=28	−0.14 n=28	0.02 n=28
4. CPS-S	κ				–	0.29	0.46*	0.34*	0.16	0.04	0.03	−0.04
	ICC					0.30 n=31	0.46* n=111	0.35* n=100	0.15 n=122	0.04 n=115	0.03 n=116	−0.05 n=115
5. ICU-P	κ					–	0.06	0.24	−0.06	0.00	−0.11	0.06
	ICC						0.06 n=32	0.25 n=31	−0.06 n=40	0.00 n=35	−0.13 n=35	0.08 n=35
6. ICU-Y	κ						–	0.33	0.11	−0.09	−0.12	−0.07
	ICC							0.34* n=114	0.11 n=140	−0.09 n=131	−0.13 n=132	−0.07 n=131
7. YPI	κ							–	0.26	−0.03	0.04	−0.03
	ICC								0.26 n=121	−0.03 n=116	0.04 n=116	−0.03 n=116
8. PCL-YV	κ								–	0.08	0.21	0.06
	ICC									0.08 n=150	0.22 n=151	0.06 n=150
9. Total Convictions										–	–	–
10. Violent Convictions											–	–
11. Nonviolent Convictions												–

– = data not applicable. κ = Kappa Coefficient of Agreement. ICC = Intraclass Correlation Coefficient. The Bonferroni procedure was used to hold the familywise (FW) Type 1 error rate at p_FW=0.05; the Type 1 error rate per individual test was set at p_T=p_FW/28=0.0018. p_FW is based on the familywise Type 1 error rate; p_T is based on the Type 1 error rate per measurement procedure test.

^*p_FW<0.05; p_T<0.0018

Intraclass Correlations

Because of the problems with loss of power and precision associated with categorizing continuous data, intraclass correlations were calculated for all of the distributions using two-way random effects of absolute agreement. Doing so yielded statistically significant agreement among some of the distributions that did not show statistically significant agreement using the Kappa Measure of Agreement in the low, medium and high psychopathy group assignment. There were no improvements in agreement for examinations of low and high psychopathy group assignment or for group assignment using a median-split to form low and high groups. It should be noted that just as with the Kappa Measure of Agreement, no distribution showed statistically significant agreement with the PCL-YV using intraclass correlation coefficient calculations (Please see Tables 5 and 6) nor did any distribution reach statistically significant agreement with total convictions, violent convictions or nonviolent convictions..

Discussion

The current study sought to examine the convergent validity and extent to which psychopathy self-report, care-giver report and interview-based assessment procedures agreed in their classification of offender youth as well as criminal convictions. The findings of the study generally did not support the hypothesis that these measures of adolescent psychopathy would have good convergent validity. Although the self-report measures (APSD-Y, CPS-S, ICU-Y, and YPI) had good convergent validity with the interview-based PCL-YV, the caregiver-report versions of these measures were not significantly correlated with their corresponding self-report versions nor were they significantly correlated with the PCL-YV. More surprising is that none of the scales were significantly correlated with the number of total criminal convictions, violent convictions or nonviolent convictions. It must be noted, however, that the lack of convergent validity between the caregiver-report versions and the other measure versions may be the result of the low number of caregiver-report versions included in the study. Although, the intercorrelations between the caregiver-report measures were significant, such a low response rate suggests that there may be bias in this group of respondents and that they may not be representative of all of the caregivers in the sample.

An additional surprising finding from that study was that group classification analyses generally revealed low agreement across youth psychopathy assessment methods and no agreement with criminal convictions. While nearly all same method measures procedures reached statistically significant group agreement with each other, none reached statistically significant group agreement with the PCL-YV or criminal convictions using the Kappa Measure of Agreement. While there was some improvement in group classification using intraclass correlation coefficients (ICC), none of the self-report or caregiver-report measures reached statistically significant agreement with the PCL-YV or with criminal convictions.

The lack of convergent validity between adolescent psychopathy measures has been shown to be moderated by several factors, however. For example, the age of the youth, the problem, and parent characteristics (De Los Reyes and Kazdin 2005) affect the agreement among raters (e.g., self-report & caregiver-report). A meta-analysis by Achenbach et al. (1987) found that agreement for informants’ ratings of psychopathology was greater for younger children than adolescents leading the authors to conclude that younger children’s behavior may be more observable by informants or the behavior may be more cross-situationally consistent. The existing literature also suggests that while caregiver-report is reliable for some forms of psychopathology (Verhulst and van der Ende 1991), antisocial attitudes and behavior are more reliably assessed using self-report or clinician-report methods, especially in adolescents (Jolliffe et al. 2003). Adolescents at this developmental stage tend to have less adult supervision and, as was the case with our sample, adolescents with severe conduct problems typically come from families where there are histories of out-of-home placements and where the parents have not had enough recent contact with the adolescent to provide current ratings of their child’s functioning or characteristics (Jolliffe et al. 2003; Loney et al. 2003). Lastly, the caregivers of adolescents in our sample were transient and obtaining behavioral reports from them was exceedingly difficult despite repeated attempts and monetary remuneration. De Los Reyes and Kazdin (2005) describe such transient families as those experiencing a high level of family or parental stress resulting from financial and other family dynamic strains and have reported these processes contribute to increased discrepancy between caregiver-reports and those of other informants.

Also, because of the correlation among same method assessments in the current study, common method variance is a concern. This concern is tempered by several considerations, however. The first is that self-report instruments were significantly correlated with the PCL-YV, an instrument of different methodology. Secondly, the PCL-YV was conducted at a different setting which is a procedural strength guarding against common method variance (Podsakoff et al. 2003; Reio 2010). Lastly, common method variance is of greatest concern when the same method is used to measure different traits. This is not the case in this study and instruments measuring the same construct should be correlated.

Although, there may be several reasons for the lack of convergent validity and group classification agreement between the various methods assessing adolescent psychopathy, the lack of convergent validity and group classification agreement between the various measures and criminal convictions remains distressing, however. One possible explanation is that there may be restricted range in criminal conviction data because this is an incarcerated sample. The restricted range in criminal conviction data may obfuscate the relationship with the assessment methods. Regardless of the reasons for the lack of convergent validity or group agreement, this finding is important when considering the forensic utility of these instruments for future placement or adjudication of these youth.

Study Limitations

Several limitations of this study should be considered in evaluating the generalizability of these findings. The first limitation is the low number of caregiver-report measures that were obtained from parents or caregivers in this study. Although a greater range of these measures would certainly have improved agreement, our study highlights a potential difficulty with using caregiver-report measures of psychopathy (or other caregiver-report measures of disruptive behavior) with incarcerated youth simply from a pragmatic perspective. As discussed above, measuring psychopathy by caregiver-report in an incarcerated sample is an unrealistic method given the level of distress in many of these families.

A second factor affecting the generalizability of the findings is poor psychometric performance of the callous-unemotional subscales. The internal consistency of the callous-unemotional traits subscales of the self-report and caregiver-report measures, except the YPI, were below commonly accepted levels (e.g., below 0.70; Pallant 2007). In addition, inspections of corrected item-total correlations were low (e.g., less than 0.30; Pallant 2007) suggesting that many of the items of these subscales are measuring something different from the scale as a whole, warranting the consideration of further refinement of the measurement of callous-unemotional traits. It is possible that these subscales tapped other psychopathology exhibited by these youth and future studies should investigate other psychopathologies that have similar clinical presentations (flat affect, lack of caring, etc.) in attempts to refine these measures. A related limitation of this study and possible explanation for the other measures poor agreement with the PCL-YV was the low Crohnbach alpha values observed among the PCL-YV’s four facets. These low alphas values were not exclusive to this study, however, (Edens et al. 2008; Spain et al. 2004; Vitacco et al. 2006) and are likely related to the abbreviated number of items on these scales (cf. Forth et al. 2003).

Implications of Findings and Conclusions

Being able to validly assess youth psychopathy, and differentiate it from conduct disorder, is important not only for understanding the construct of psychopathy, but also for valid forensic assessment and appropriate treatment recommendations. The consequences associated with correctly identifying youth with psychopathic traits are significant. Incorrectly identifying a youth as having psychopathic traits where none actually exist is potentially stigmatizing while not correctly identifying a youth who does possess such traits misses a potential treatment opportunity. Evidence is mounting that suggests that early, individualized interventions are effective in ameliorating the antisocial behavior of these youth (Caldwell and Van Rybroek 2005; Frick 2001). The greatest tragedy is in not correctly identifying youth who would be candidates for such services.

The findings of poor agreement among the measures of this study also argue against the interchangeable use of these assessment modalities in clinical and forensic settings, especially against the interchangeable use of the caregiver-report when assessing youth psychopathology. Our findings suggest that, at the very least, not enough information exists as to the validity and utility of such caregiver-report measures in clinical and forensic settings.

Lastly, this study was unique in that it is one of the few studies of youth psychopathy that relies on data from a large Hispanic sample. While the ethnic composition of the study offers a unique contribution to the literature, it calls for the replication of the findings in other racial and ethnic groups. Just as in other forms of psychopathology (e.g., depression), it is possible that different groups simply present different symptomology. It further emphasizes the need for sound assessment and early, valid detection of behavior and attitudes that are associated with poor outcomes in diverse groups so that such youth receive the services necessary to remediate their problematic behavior.