Validity Study of the CBCL 6–18 for the Assessment of Emotional Problems in Youth With ASD

Abstract

Youth with autism spectrum disorder (ASD) often present with emotional problems such as anxiety and depression (American Psychiatric Association, 2013). A recent study of the Child Behavior Checklist 6-18 (CBCL; Achenbach & Rescorla, 2001) indicated good sensitivity but relatively low specificity for identifying emotional problems in youth with ASD. The current study examined the extent to which variance in the CBCL's Anxious/Depressed, Withdrawn/Depressed, Internalizing Domain, and Total Problems scales was accounted for by symptoms of emotional problems relative to ASD symptoms. Correlation and multiple regression analyses indicated that these scales measured anxiety and depression but a small statistically significant proportion of variance in Total Problems scores was also accounted for by ASD symptoms. Results contribute to the emerging evidence base for the inclusion of the CBCL in assessment protocols for assessing emotional and behavioral problems in youth with ASD.

Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder characterized by impairments in social communication and interaction and the presence of restrictive and repetitive patterns of behaviors, interests, and activities (American Psychiatric Association, 2013). Youth with ASD can also present with emotional disorders such as anxiety and depression (American Psychiatric Association, 2013; also see Gadow, Guttmann-Steinmetz, Rieffe, & DeVincent, 2012; Ghaziuddin, Ghaziuddin, & Greden, 2002; van Steensel, Bogels, & Perrin, 2011; White, Oswald, Ollendick, & Scahill, 2009). Recent reports indicate a period of increased risk in school-age youth with ASD (see Vickerstaff, Heriot, Wong, Lopes, & Dossetor, 2007), particularly for depression (e.g., Hofvander et al., 2009). In addition, behavior disorders have been reported to co-occur with emotional disorders (e.g., Mattila et al., 2010; Quek, Sofronoff, Sheffield, White, & Kelly, 2012). Emotional problems can persist over time (see Gadow, DeVincent, Pomeroy, & Azizian, 2004; Mash & Dozois, 2003; Simonoff et al., 2008) and result in additional functional impairment for the affected individual over and above what is typically associated with the core clinical features of ASD (Matson & Nebel-Schwalm, 2007; Mattila et al., 2010). These findings indicate the need for routine monitoring and assessment to ensure early detection and disorder-specific intervention (Magyar & Pandolfi, 2012; Mayes, Calhoun, Murray, Ahuja, & Smith, 2011).

However, several factors make it difficult to assess for emotional disorders in many youth with ASD. These include atypical symptom presentation because of idiosyncratic communication and behavioral topographies, limited ability to provide accurate self-reports due to ASD and associated developmental problems such as communication and intellectual impairments, and apparent symptom overlap between ASD characteristics and symptoms of anxiety and depression (e.g., see Stewart, Barnard, Pearson, Hasan, & O'Brien, 2006). These assessment challenges can increase the risk for either a delay or failure to accurately identify co-occurring conditions and problems, potentially forestalling appropriate treatment.

Psychometrically sound parent report measures can greatly assist evaluators when assessing youth with ASD for co-occurring emotional problems. The Child Behavior Checklist 6–18 (CBCL; Achenbach & Rescorla, 2001) is one of the most widely researched parent measures of emotional and behavioral problems in youth ages 6–18 years. Numerous studies support its reliability and validity across many different clinical samples (see Berubé & Achenbach, 2010). A recent study supported its factor structure, the reliability of its major scales and subscales, and good sensitivity for identifying emotional and behavioral problems in youth with ASD (Pandolfi, Magyar, & Dill, 2012), but results indicated generally low specificity. The authors speculated that the low specificity may be related to the possibility that the CBCL was also measuring symptoms of ASD including those that may overlap with other kinds of emotional and behavioral problems. In the present psychometric study of a sample of youth with ASD, correlation and simultaneous multiple regression examined the extent to which variance in CBCL scores was accounted for by symptoms of emotional problems relative to ASD symptoms.

The four CBCL scales of interest used in these analyses included the Anxious//Depressed, Withdrawn/Depressed, Internalizing Domain, and Total Problems scales. All of these scales assess for emotional problems, including Total Problems, which assesses for a combination of emotional and behavioral problems, a clinical presentation often observed in youth with ASD (American Psychiatric Association, 2013). Their relationship with the following measures was examined: (a) the Schedule for Affective Disorders and Schizophrenia-Childhood Version (K-SADS; Kaufman, Birmaher, Brent, Rao, & Ryan, 1996), an interview measure of emotional and behavioral problems in youth, and (b) the current behavior algorithm of the Autism Diagnostic Interview-Revised (ADI-R; Rutter, LeCouteur, & Lord, 2003).

We hypothesized that the variance in CBCL scores would be primarily accounted for by the measures of emotional problems (K-SADS). Although some have attempted to identify subsets of CBCL items predictive of an ASD diagnosis (see Ooi, Rescorla, Ang, Woo, & Fung, 2011; So et al., 2013), the CBCL was developed to be a measure of emotional and behavioral problems and not ASD-specific problems. If the hypothesis was supported, results from this study might appeal to readers working with youth with ASD in clinical and school settings. Our findings can help inform the most appropriate clinical uses and interpretation of the CBCL when evaluating emotional problems in ASD within the context of a multimethod multi-informant assessment protocol.

METHOD

Data Analyzed

The archival data analyzed were collected from parents of youth with ASD (N = 76), ages 6 to 18 years (M = 12 years, 0 months, SD 3 years, 3 months). Participants were recruited from the local community= to enroll in a large federally funded study of genotype and phenotype in ASD. All research procedures were approved by the University of Rochester's Institutional Review Board. Each youth participant met research criteria for ASD that included diagnostic algorithms from the ADI-R and the Autism Diagnostic Observation Schedule (ADOS; Lord, Rutter, DiLavore, & Risi, 2002) as well as expert clinical consensus. Participants met ADI-R criteria for ASD if their Reciprocal Social Interaction score was at or above cutoff and if their Communication score was within two points of the cutoff. ADOS criteria for ASD were met if the participant's score was at or above the ASD cutoff for the module administered.

Psychiatric assessments were completed for all youth participants. The psychiatric assessment included a parent-completed medical history form; direct clinical observation of the youth; a semistructured interview of the youth participant (if possible, based on communication level); the K-SADS screening interview with the parent as reporter; and a parent-completed CBCL, 91% of which were completed by the mother (65% of CBCL protocols were completed by the mother in the normative sample; see Achenbach & Rescorla, 2001). The Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev; DSM-IV-TR) diagnostic criteria (American Psychiatric Association, 2000) were used and diagnostic decisions were informed by all data sources. The evaluation team included a clinical psychologist, social workers, school psychologists, and bachelor's level research associates. All evaluators had experience assessing children and youth with ASD and were trained to reliably administer the ADI-R, ADOS, and K-SADS according to the research protocol. Table 1 presents participant characteristics.

TABLE 1.

Sample Characteristics (N = 76)

	Frequency
	No.	%
Age classification
Total 6- to 11-year-olds	40	52.63
Gender male:	37	92.50
Total 12- to 18-year-olds	36	47.37
Gender male:	29	80.56
Overall gender: Male	66	86.84
Diagnostic status
Any DSMa diagnosis	59	77.63
Any anxiety disorder	35	46.05
Any depressive disorder	15	19.74
Race
Asian	2	2.63
Black	1	1.32
White	73	96.05
Ethnicity: Non-Hispanic	76	100.00
Socioeconomic statusb
Major business/Professional	28	37.33
Medium business/Minor professional	31	41.33
Skilled craftsman/Clerical/Sales	11	14.67
Machine operator/Semiskilled	3	4.00
Unskilled laborer/Menial service	2	2.67
Full scale IQ
≥ 70	67	88.16
Vineland classificationc
Low	51	67.11
Moderately low	17	22.37
Adequate	8	10.53

^aDiagnostic and Statistical Manual of Mental Disorders.

^bHollingshead (1975)scale, n = 75.

^cPercentages do not sum to 100% due to rounding error.

The gender distribution for the entire sample was similar to that reported for the general ASD population (see American Psychiatric Association, 2000, 2013), with 86.84% male. Males comprised approximately 62% of the CBCL's normative sample. There were an approximately equal number of participants falling within each of the CBCL's two normative age groups, 6–11 (52.63%) and 12–18 years (47.37%). The sample was predominantly middle to upper middle class, White, and non-Hispanic. Most were diagnosed with at least one emotional and behavioral disorder (77.63%) and nearly half were diagnosed with two or more disorders (47.37%). Consistent with other reports (e.g., Gadow et al. 2012), a rather high percentage of participants was diagnosed with anxiety disorders (46.05%) and depression (19.74%). Many participants obtained CBCL scores in the borderline and clinically significant ranges on the Anxious/Depressed (43.42%), Withdrawn/Depressed (44.74%), Internalizing Domain (69.74%), and Total Problems (73.68%) scales. This sample also presented with various behavior disorders including attention-deficit/hyperactivity disorder (38.16%) and oppositional defiant disorder (7.89%), a finding consistent with other reports in the literature regarding high rates of emotional and behavioral comorbidity.

Table 1 shows that most participants had FSIQs ≥70. The specific intelligence test used with each participant was selected based on participant age and general language and communication ability. Measures included the Wechsler Intelligence Scale for Children-Fourth Edition (Wechsler, 2003), Wechsler Adult Intelligence Scale-Third Edition (Wechsler, 1997), Stanford-Binet Intelligence Scales-Fifth Edition (Roid, 2003), and the Leiter International Performance Scale-Revised (Roid & Miller, 1997). Table 1 also indicates that the majority of participants presented with significant adaptive deficits as evidenced by scores significantly below age expectations on the Vineland Adaptive Behavior Scales (Sparrow, Balla, & Cicchetti, 1984): Adaptive Behavior Composite (M = 65.17, SD = 15.22), Communication Domain (M = 77.04, SD = 18.72), Daily Living Skills Domain (M = 68.96, SD = 20.65), and Socialization Domain (M = 63.89, SD = 17.84).

Measures

CBCL 6–18

The norm-referenced CBCL 6–18 is a dimensional (as opposed to categorical) measure of emotional and behavioral problems. It contains several scales that are arranged hierarchically, a structure that emerged through factor analysis (see Achenbach & Rescorla, 2001). Each level of the hierarchy reflects the breadth of coverage of a variety of emotional and behavioral problems. At the top of this hierarchy are the broadly focused Internalizing Domain and the Externalizing Domain. The Internalizing Domain is a broad measure of emotional problems: it is an aggregate of anxiety and depression symptoms that subsumes three more narrowly focused syndrome scales: Anxious/Depressed, Withdrawn/Depressed, and Somatic Complaints. The Externalizing Domain is an aggregate measure of behavioral problems and includes the Rule Breaking Behavior and Aggressive Behavior syndrome scales. A Total Problems score is also available, which quantifies the overall extent of both emotional and behavioral problems based on responses to all CBCL items including those on the three remaining syndrome scales: Social Problems, Thought Problems, and Attention Problems.

The parent or primary caregiver completes the CBCL. Each item is scored on a 3-point Likert scale (0 = Not True, 1 = Somewhat or Sometimes True, or 2 = Very True or Often True) to describe=the child's behavior during the preceding 6 months. All CBCL scales have a t-score mean of 50 and standard deviation of 10 and different norms are provided for each gender across the two normative age groups 6–11 and 12–18 years.

Pandolfi et al. (2012) provided evidence supporting many psychometric properties of the CBCL in a well-characterized sample of youth with ASD. Results supported the reliability and unidimensionality of the syndrome scales and the two-factor Internalizing-Externalizing model. ROC analyses indicated that the Internalizing Domain, Total Problems, Anxious/Depressed, and Withdrawn/Depressed scales all had good sensitivity for detecting anxiety and/or depression (see Pandolfi et al., 2012).

K-SADS

The K-SADS is a semistructured diagnostic interview based on the DSM-III-R (American Psychiatric Association, 1987) and DSM-IV (American Psychiatric Association, 1994), both categorical classification systems. It assesses for psychopathology in children and adolescents across 20 diagnostic areas (Ambrosini, 2000; Kaufman et al., 1996). The measure contains a screening interview and supplemental diagnostic interviews. Reliability and validity data are generally favorable (see Ambrosini, 2000; Kaufman et al., 1997) and the K-SADS has been used in published studies of youth with ASD (see Gjevik, Eldevik, Fjaeran-Granum, & Sponheim, 2011; Leyfer et al., 2006).

The K-SADS data analyzed in this study were collected from screening items belonging to the anxiety disorders and depression scales. These scales included Panic Disorder, Separation Anxiety, Avoidant Disorder/Social Anxiety, Agoraphobia/Specific Phobia, Overanxious Disorder/Generalized Anxiety Disorder, Obsessive-Compulsive Disorder, and Depression. Items assess for symptoms of these disorders, and ratings reflect a Likert scale (0 = No Information Available, 1 = Not Present, 2 = Subthreshold, and 3 = Threshold). We created two K-SADS scores from these screening interviews: a K-SADS Anxiety score and a K-SADS Depression score with each reflecting the raw score sum of the items that comprised the anxiety disorders and depression screening interviews, respectively (see Kaufman et al., 1996). These two scores were used for the data analyses because each quantified a broad range of symptoms comparable to the CBCL's syndrome scale scores.

ADI-R

The ADI-R is a semistructured diagnostic interview used to discriminate children and youth with ASD from those with other disorders. The technical manual provides data to support its psychometric properties (see Rutter et al., 2003). The ADI-R can yield two algorithms, and the diagnostic algorithm assesses for the presence of ASD symptoms throughout a person's life with particular focus on the time span between 4 and 5 years of age. The current behavior algorithm, used in this study, covers a time frame comparable to the CBCL and K-SADS.

All ADI-R items reflect a Likert scale and are generally scored 0 (Absence of Abnormal Behavior) to 3 (Significantly Abnormal or Qualitatively Different Behavior), although some items are only scored 0–2. To compute the algorithm scores, item scores of 3 are replaced with a score of 2 so that some items do not disproportionately overinfluence the results. Item raw scores from the current behavior algorithm were summed, and this sum of item scores was used in the data analyses to capture the full range of ASD symptoms for each participant.

Data Analysis

Correlation and simultaneous multiple regression analyses included the K-SADS and ADI-R predictors and the four CBCL scales that assess for symptoms of anxiety and/or depression: Anxious/Depressed, Withdrawn/ Depressed, the Internalizing Domain, and Total Problems. These scales served as the criterion variables in the regression analyses. Pearson correlations quantified the relationship between predictor and criterion variables and reflected validity coefficients. Simultaneous multiple regression was chosen given the exploratory nature of the study. It was possible that the CBCL scales measured both emotional problems and ASD symptoms and each predictor variable was evaluated in relation to how much variance it accounted for in the CBCL scale scores over and above the other predictors.

Raw score data were analyzed because participants’ ages crossed over the CBCL's two normative age ranges. For all analyses, the K-SADS and ADIR scores were centered: each participant's score was subtracted from the sample mean. The use of centered scores reduces nonessential collinearity among the predictors in multiple regression and increases both statistical power and interpretability of results (Cohen, Cohen, West, & Aiken, 2003). All analyses were performed using SPSS 18.0.

We computed the same statistics in all regression analyses: R², the squared semipartial correlation (sr²) between each predictor and the criterion, and their associated significance tests. An F test evaluated whether the predictors collectively accounted for a statistically significant proportion of variance in the CBCL criterion (a test of R²). T tests evaluated the statistical significance of each individual predictor, which was tantamount to testing for the significance of sr²: the proportion of variance that each predictor uniquely accounted for in the criterion. Assumptions of linear regression were evaluated using qualitative and quantitative methods.

We conducted a power analysis to determine how many participants were needed to detect at least medium-size effects (R² = .20, sr² = .10), values commonly tested in behavioral research when the true population values are unknown (see Cohen et al., 2003). Using the formulas and power charts in Cohen et al. (2003), a total of 48 participants were needed to have an 80% chance of rejecting the null hypothesis that R² = 0 (α = .05). A total of 65 participants were required to have the same chance of rejecting the null hypothesis that sr² = 0 ( participants α = .05). Because our N = 76, we had a sufficient number of to detect at least medium-size effects while keeping the Type I and Type II error rates at conventional levels.

RESULTS

Reliability of Measures

Scale reliability for the CBCL, K-SADS, and ADI-R data was estimated by Guttman's Lambda-2 (λ-2; Guttman, 1945), which is preferred to coefficient alpha (see Green & Yang, 2009; Sijtsma, 2009). All λ-2 indices were above .70, a commonly desired threshold (Netemeyer, Bearden, & Sharma, 2003): Anxious/Depressed = .87, Withdrawn/Depressed = .79, Internalizing Domain = .89, and Total Problems = .94 (n = 75). Five items from Total Problems were not included in the reliability analysis due to zero variance: Drinks alcohol without parents’ approval; Sets fires; Smokes, chews, or sniffs tobacco; Truancy, skips school; and Uses drugs for nonmedical purposes. Reliability was also acceptable for K-SADS Anxiety (.81), K-SADS Depression (.79), and the ADI-R (.72). Two items from the K-SADS Depression scale also were eliminated from the reliability analysis due to zero variance: Suicidal Acts-Seriousness and Suicidal Acts-Medical Lethality.

Regression Analyses

Assumptions

Regression assumptions were assessed using several plots and statistical tests. These analyses did not indicate violations of the assumptions or significant outliers that could have biased the results. Pearson correlations among the predictors themselves ranged from .128 between KSADS Anxiety and the ADI-R to .457 (p < .001) between K-SADS Anxiety and K-SADS Depression, which was the only significant correlation. Therefore, multicollinearity was not a problem.

Anxious/Depressed scale

The Pearson correlations between Anxious/Depressed and the K-SADS Anxiety (r = .684, p < .001) and K-SADS Depression scores (r = .664, p < .001) were both statistically significant and moderately large. The correlation between Anxious/Depressed and the ADI-R (r = .098, p > .05) was not statistically significant. Thus, the Anxious/Depressed scores correlated significantly with the interview-based anxiety and depression scores but not with the ASD scores.

Table 2 shows the results of the regression analysis. R² was statistically significant (R²= .628, p < .001), and the K-SADS Anxiety (sr² = .187) and Depression (sr² = .158) scores each uniquely accounted for a significant proportion of variance in Anxious/Depressed scores (both p's < .001). The ADI-R uniquely accounted for less than 1% of the variance, which was not significant.

TABLE 2.

Regression Results for CBCL Anxious/Depressed Scale

Variable	β a	SEβ b	95% CI c	sr2 d
Constant	7.171*	0.380	6.414, 7.929
K-SADSe Anxiety	0.615*	0.102	0.411, 0.818	.187
K-SADS Depression	1.390*	0.252	0.888, 1.892	.158
ADI-Rf	–0.039	0.044	–0.126, 0.048	.004
R2	.628
R2adj	.613
F	40.554*

^aUnstandardized regression coefficient.

^bStandard error of the regression coefficient.

^cNinety-five percent confidence interval for the regression coefficient.

^dSquared semipartial correlation.

^eSchedule for Affective Disorder and Schizophrenia-Childhood Version.

^fAutism Diagnostic Interview-Revised.

^*p < .001.

Withdrawn/Depressed scale

The correlation between Withdrawn/ Depressed and K-SADS Depression (r = .546, p < .001) and K-SADS Anxiety (r = .390, p < .001) were statistically significant. Withdrawn/Depressed did not correlate significantly with the ADI-R (r .164, p > .05). Thus, higher K-SADS Depression and Anxiety scores were= associated with higher Withdrawn/Depressed scores.

Table 3 shows the multiple regression results. The K-SADS Depression, K-SADS Anxiety, and ADI-R collectively accounted for 32.7% of the variance in the Withdrawn/Depressed scale, which was statistically significant (p < .001). The K-SADS Depression score (sr² = .168, p < .001) uniquely accounted for a statistically significant proportion of variance; however, the K-SADS Anxiety (2.1%) and ADI-R scores (< 1%) uniquely accounted for a nonsignificant percentage of variance in Withdrawn/Depressed.

TABLE 3.

Results for CBCL Withdrawn/Depressed Scale

Variable	β a	SEβ b	95% CI c	sr2 d
Constant	4.632*	0.331	3.971, 5.292
K-SADSe Anxiety	0.132	0.089	–0.045, 0.310	.021
K-SADS Depression	0.933*	0.220	0.495, 1.370	.168
ADI-Rf	0.027	0.038	–0.049, 0.103	.005
R2	.327
R2adj	.299
F	11.682*

^aUnstandardized regression coefficient.

^bStandard error of the regression coefficient.

^cNinety-five percent confidence interval for the regression coefficient.

^dSquared semipartial correlation.

^eSchedule for Affective Disorder and Schizophrenia-Childhood Version.

^fAutism Diagnostic Interview-Revised.

^*p < .001.

Internalizing Domain

The Internalizing Domain had moderately large and statistically significant correlations with K-SADS Anxiety (r = .650, p < .001) and K-SADS Depression (r = .655, p < .001). Its correlation with the ADI-R (r = .137) was small and not statistically significant (p > .05).

Table 4 shows the results for the regression. The overall R² = .584 was statistically significant (p < .001). The sr² values indicated that= only the K-SADS scores uniquely accounted for a statistically significant proportion of variance in the Internalizing Domain. K-SADS Depression uniquely accounted for 16.2% of the variance and K-SADS Anxiety accounted for 15.3%. The ADI-R uniquely accounted for less than 1% of the variance, which was not significant.

TABLE 4.

Results of CBCL Internalizing Domain

Variable	β a	SEβ b	95% CI c	sr2 d
Constant	14.500*	0.692	13.121, 15.879
K-SADSe Anxiety	0.955*	0.186	0.585, 1.326	.153
K-SADS Depression	2.427*	0.458	1.514, 3.341	.162
ADI-Rf	–0.018	0.080	–0.177, 0.141	<.001
R2	.584
R2adj	.567
F	33.710*

^aUnstandardized regression coefficient.

^bStandard error of the regression coefficient.

^cNinety-five percent confidence interval for the regression coefficient.

^dSquared semipartial correlation.

^eSchedule for Affective Disorder and Schizophrenia-Childhood Version.

^fAutism Diagnostic Interview-Revised.

^*p < .001.

Total Problems

Total Problems had statistically significant correlations with K-SADS Anxiety (r = .539, p < .001), K-SADS Depression (r = .669, p < .001), and the ADI-R (r = .294, p < .05). Table 5 shows the regression results. R² was significant (R² = .544, p < .001) and both K-SADS Anxiety (sr² = .052, p < .01) and Depression significant predictors. The ADI-R (sr² = .221, p < .001) were statistically uniquely accounted for 2.8% of the variance in Total Problems, which was also statistically significant (p < .05).

TABLE 5.

Results of CBCL Total Problems Scale

Variable	β a	SEβ b	95% CI c	sr2 d
Constant	52.654***	1.981	48.704, 56.604
K-SADSe Anxiety	1.525**	0.532	0.464, 2.586	.052
K-SADS Depression	7.758***	1.313	5.141, 10.376	.221
ADI-Rf	0.477*	0.228	0.022, 0.931	.028
R2	.544
R2adj	.525
F	28.681***

^aUnstandardized regression coefficient.

^bStandard error of the regression coefficient.

^cNinety-five percent confidence interval for the regression coefficient.

^dSquared semipartial correlation.

^eSchedule for Affective Disorder and Schizophrenia-Childhood Version.

^fAutism Diagnostic Interview-Revised.

^*p < .05.

^**p < .01.

^***p < .001.

DISCUSSION

General Findings

This study extends the emerging evidence base describing the CBCL's measurement properties in youth with ASD. Results indicated that the CBCL's Anxious/Depressed, Withdrawn/Depressed, and Internalizing Domain were measures of emotional problems in the present sample. Total Problems measured emotional problems but a small statistically significant percentage of variance was accounted for by ADI-R scores. These findings provide additional validity evidence to support the inclusion of the CBCL into screening and diagnostic assessment protocols for emotional problems in youth with ASD, an important practice given the DSM-5 requirement that an ASD diagnosis specify whether an individual presents with or without emotional and behavioral conditions (American Psychiatric Association, 2013).

Part of our rationale for conducting this study was to explore why a previous diagnostic accuracy analysis indicated rather low specificity for the CBCL when assessing for emotional problems in youth with ASD (see Pandolfi et al., 2012). Results of the present study indicated that ASD symptoms were not a major factor. It may be possible that the different conceptual frameworks related to emotional and behavioral problems that underlie the CBCL and DSM account for low specificity. The dimensional CBCL was developed to assess continuously distributed emotional and behavioral syndromes and not categorical disorders (see Achenbach & Rescorla, 2001), which characterizes DSM classification. Low specificity could result from comparing the results of a dimensional measure (CBCL) with diagnostic outcomes based on a categorical classification system (DSM). The CBCL scales reflect relatively small samples of behavior and they may not be able to better discriminate among the different categories of anxiety and depressive disorders in the DSM. When using the CBCL, it may be reasonable to expect higher sensitivity relative to specificity for identifying more narrowly defined categorical DSM-based disorders.

Limitations and Areas for Future Research

A few methodological limitations should be addressed in future research. First, replication studies should include participants from more diverse racial-ethnic, socioeconomic, psychiatric, and developmental backgrounds. Second, future studies may target CBCL protocols completed by fathers to better determine the effect that specific respondents may have on CBCL data. Future studies could also look to replicate our methodology within each gender and CBCL normative age group separately. Analyses may also be replicated separately on groups of youth with ASD with and without intellectual disability. Such analyses can help establish clinical practice guidelines for the assessment of emotional problems within specific subgroups of the larger ASD population. Finally, similar studies with larger samples should examine the CBCL's Externalizing Domain, Rule Breaking Behavior, Aggressive Behavior, Social Problems, Thought Problems, Attention Problems and Somatic Complaints as this will extend validity evaluation to all CBCL scales. Further investigation of Total Problems is warranted because it was meant to measure behavioral as well as emotional problems.

The CBCL should be evaluated for its association with other parent report measures of emotional problems. This may include rating scales like the Behavioral Assessment System for Children-2nd Edition (Reynolds & Kamphaus, 2004) and other interview measures, such as the Anxiety Disorders Interview Schedule (Silverman & Albano, 1996). The CBCL's associations with other measures of ASD, such as the Childhood Autism Rating Scale-2 (Schopler, Van Bourgondien, Wellman, & Love, 2010) could also be explored. Studying associations with other measures of emotional problems and ASD is important because different measures may tap slightly different aspects of these constructs. Quantifying the CBCL's relationships with a wide range of measures may allow researchers and clinicians alike to better understand exactly what components of anxiety and depression the CBCL is measuring in youth with ASD. This knowledge would help evaluators select a set of complementary measures that allow for a comprehensive assessment individually tailored to the specific pattern of presenting problems. This in turn can more fully inform the breadth and intensity of treatment.

Implications for Practice

Psychometric evidence suggests that the CBCL scales evaluated here can be interpreted for youth with ASD similar to the way they are for the general population. Total Problems reflects the broadest CBCL scale and elevations suggest the presence of an emotional and/or behavioral problem. Because Total Problems measures both emotional and behavioral problems, practitioners should interpret this global scale in light of the results obtained across all CBCL syndrome scales (see Achenbach & Rescorla, 2001) and within the context of ASD symptom profiles. If an emotional problem is suspected, clinicians can obtain more specific information by analyzing scores on the Internalizing Domain and its syndrome scales.

An elevated score on the Internalizing Domain suggests the presence of one or more emotional problems. Because this scale is a composite of both anxiety- and depression-related problems, analysis of the more narrowly defined Anxious/Depressed and Withdrawn/Depressed syndrome scales can help clarify the specific nature of the internalizing problem. As an aggregate measure, it is possible for the Internalizing Domain to be elevated in the absence of clinically elevated syndrome scales. The dimensional nature of the CBCL allows for the detection of subthreshold anxiety or depression problems that may jointly contribute to overall emotional distress. We recommend that children with ASD who have elevated scores on the Anxious/Depressed, Withdrawn/Depressed, and/or Internalizing Domain scales be referred for a comprehensive diagnostic evaluation.

We also recommend that the CBCL be used with other methods of assessment to assist with differential diagnosis. For example, the CBCL should be used with direct clinical observation, parent and child interview, and teacher report, which is consistent with best practice in ASD assessment (Ozonoff, Goodlin-Jones, & Solomon, 2005). Functional behavioral assessment might be particularly helpful in cases where emotional problems are accompanied by externalizing behaviors such as aggression (see Matson & Laud, 2007). The use of multiple assessment methods can assist in the differential diagnosis process and help identify appropriate treatment.

Given that emotional problems such as anxiety and depression are reportedly prevalent in school-age youth with ASD, this study is an important step in the validation of the CBCL for use in evaluating emotional problems within this population of youth. Within the larger context, we believe this study helps move the field toward more precise and valid measurement of psychopathology in youth with ASD, which has implications for the quality of life and long-term outcomes of those affected individuals. Additional work in this area should focus on continued examination of the CBCL as well as other measures to inform evidence-based assessment practice in ASD and associated disorders.