Predicting Anxiety Diagnoses and Severity with the CBCL-A: Improvement Relative to Other CBCL Scales?

Abstract

The Child Behavior Checklist (CBCL) is a widely used parent-report of child and adolescent behavior. We examined the ability of the CBCL-A scale, a previously published subset of CBCL items, to predict the presence of generalized anxiety disorder (GAD), separation anxiety disorder (SAD), and social phobia (SoP), as well as anxiety severity, among 488 youth randomized in the Child Anxiety Multimodal Study (CAMS). We predicted that the CBCL-A’s unique inclusion of items related to somatic symptoms would better identify anxiety disorder and severity than other CBCL scales, given that somatic complaints are often key features of anxiety among youth. Results support the use of the anxiety-based CBCL subscales as first-line screeners for generally elevated symptoms of anxiety, rather than tools to identify specific anxiety disorders. Although somatic symptoms are often reported and included in diagnostic criteria for certain anxiety disorders (e.g., SAD, GAD), the unique combination of somatic and non-somatic symptoms for the CBCL-A subscale did not increase its ability to consistently predict the presence of specific anxiety disorders.

Introduction

Anxiety disorders are common among youth, with epidemiological studies indicating prevalence rates between 10% and 20% (Chavira, Stein, Bailey, & Stein, 2004; Kessler, et al., 2005). Left untreated, problematic anxiety often leads to psychosocial impairment (Essau, Conradt, & Petermann, 2000), including poor outcomes in academic performance, social development, and mental health in adulthood (Albano, Chorpita, & Barlow, 2003; Pine, Cohen, Gurley, Brook, & Ma, 1998; Puleo, Conner, Benjamin, & Kendall, 2011; Velting, Setxer, & Albano, 2004). Anxiety is often overlooked by teachers and primary care doctors (Kendall, Panichelli-Mindel, Sugarman, & Callahan, 1997; Wren, Scholle, Heo, & Comer, 2003), making sensitive identification of anxious youth of paramount importance.

Parent reports may be especially important because parents are often the initiators of mental health services for youth (Choudhury et al., 2003). Furthermore, parent reports are critical when assessing younger children, as these youth are sometimes unable to express their internal distress, or make the connection between somatic symptoms and anxiety (Langley, Bergman, & Piacentini, 2002). Particularly in settings where time and cost-effective measures are needed, questionnaire-based parent-reports play an important role because they are easy to administer and require fewer resources than interviews and observational methods (Yates & Taub, 2003).

The Child Behavior Checklist (CBCL; Achenbach, 1991; Achenbach, Dumenci, & Rescorla, 2003) is a widely used, standardized measure that assesses parent-reported behaviors, problems, and competencies for children and adolescents 6–18 years of age. The CBCL consists of two subscales targeting anxiety problems. The Internalizing syndrome scale (CBCL-INT) and Anxious/Depressed (CBCL-A/D) scale were developed empirically to identify reliable clusters of items that reflect internalizing symptoms that often covary (Achenbach, 1995). The CBCL also includes a Somatic Complaints scale (CBCL-SOM), a scale that does not directly measure anxiety but assesses for physiological symptoms that may relate to anxiety. The Internalizing scale and Anxious/Depressed syndrome scale reliably distinguish between treatment-seeking anxious youth and community youth (Aschenbrand, Angelosante, & Kendall, 2005; Seligman, Ollendick, Langley, & Baldacci, 2004), including international samples (Brasil & Bordin, 2010). However, these broad symptom scales nonetheless reflect a mixture of internalizing-type problems and are unable to reliably distinguish between anxious and affective problems (e.g., depression; Wadsworth, Hudziak, Heath, & Achenbach, 2001).

Although the broad syndrome scales reliably distinguish anxious and non-anxious youth, they have only modest correspondence to the commonly used DSM-IV nosology (Kendall et al., 2007; Nakamura, Ebesutani, Bernstein, & Chorpita, 2009). This discrepancy can be problematic given that the DSM system provides an important connection to the research literature, facilitates communication between health care professionals, and can be a requirement for financial reimbursement (Achenbach & Dumenci, 2001; Doucette, 2002). Achenbach and colleagues (2003) created a series of DSM-oriented scales, including the 6-item CBCL Anxiety Problems subscale (CBCL-AP), based on expert agreement of item similarity to DSM-IV criteria. The psychometric properties of the CBCL-AP scale are comparable to those of the syndrome scales and it has been shown to reliably identify anxious youth (Achenbach et al., 2003; Nakamura et al., 2009). The CBCL-AP scale has shown greater discriminatory ability than the broader Anxious/Depressed syndrome scale for anxiety in general, as well as for distinguishing between specific anxiety diagnoses (i.e. generalized anxiety disorder [GAD], separation anxiety disorder [SAD], specific phobia [SPPH]; Ebesutani et al., 2010).

The addition of the CBCL-AP scale provided a much-needed connection to common diagnostic procedures, but research has provided mixed results about the incremental clinical utility of the CBCL-AP. The CBCL-AP scale evidenced only “fair” ability to predict anxiety disorder diagnosis by clinician assessment (AUC = .65) or by child-reported impairment (AUC = .70), and revision of the item content of the CBCL-AP scale has been recommended (Ferdinand, 2008). Importantly, this scale does not include somatic symptoms, which can weigh heavily in DSM-based diagnoses for certain anxiety disorders (e.g., GAD).

To enhance correspondence between CBCL subscales and DSM anxiety disorder criteria, Kendall and colleagues (2007) developed an anxiety scale (i.e., the CBCL-A) that includes the addition of 10 somatic symptoms already included in the measure to the CBCL-AP scale. An initial evaluation of the CBCL-A revealed adequate ability to discriminate between youth with and without anxiety disorders (t(215) = 1.31, p < .05), and evidenced improved prediction of anxiety status relative to the CBCL Anxious/Depressed syndrome and Internalizing subscales. Research also supports the sensitivity of the CBCL-A scale to capture anxiety symptom reduction at posttreatment (Kendall et al., 2007; Kley, Heinricks, Bender, & Tuschen-Caffier, 2011). The discriminatory abilities of the CBCL-A between anxious and nonanxious youth were comparable to the CBCL-AP subscale in the initial study (Kendall et al., 2007), but were somewhat lower when discriminating between ICD-10 diagnostic classifications among both in-and out-patient groups (Pauschardt, Remschmidt, & Mattejat, 2010), suggesting the need for replication with larger samples of anxious youth.

The inclusion of items about somatic symptoms (e.g., “nausea, feels sick,” “aches or pains”) was designed to make the CBCL-A more adept at identifying anxiety disorders, given that bodily reactions are important features within DSM-based characterization of many anxiety disorders and corresponding semi-structured interviews. However, anxiety disorders differ in the report and weight of somatic symptoms in diagnosis. Indeed, an evaluation of the report of physical symptoms within the current sample, using the Physical Symptoms Checklist (Emslie et al., 2006), found that diagnoses of GAD and SAD were significantly related to the number and severity of child-reported somatic symptoms, while SoP was not (Crawley et al., in press). Somatic symptoms are defined in the diagnostic criteria for GAD and SAD, although they are not included in the criteria for SoP (APA, 2000). In contrast to somatic symptoms of SoP, which may occur more in the context of peers, somatic symptoms may play a greater role in the parent-identification of GAD and SAD because the display of these disorders likely occur more frequently in the presence of parents. As such, it would be important to consider how youth with these disorders differ in parent-reported CBCL scores.

The present study examined the ability of the CBCL-A scale to differentially predict the presence of specific anxiety disorders (GAD, SoP, SAD) and their severity within a large sample of children and adolescents. Additionally, we extended previous research by examining the relative predictive ability of the CBCL-A as compared to the other CBCL scales (CBCL-INT, CBCL-A/D, CBCL-AP, CBCL-SOM). Given that the CBCL-A fared better than the broad internalizing scales (Kendall et al., 2007), it was hypothesized that the inclusion of somatic-related items in the CBCL-A scale would increase the sensitivity of this scale to detect the presence of specific anxiety diagnoses relative to the CBCL-AP, particularly for those disorders in which somatic symptoms play a key role (e.g., GAD, SAD). Additionally, it was predicted that the CBCL-A scale would be a superior predictor of anxiety severity as compared to other CBCL scales.

Method

Participants

The present study examined data from the 488 youth who participated in the Child/Adolescent Anxiety Multimodal Study (CAMS), which was a randomized clinical trial (RCT) that examined the relative efficacy of CBT, pharmacotherapy, and combined treatment for anxiety disorders among youth aged 7–17 years (Walkup et al., 2008). Participants were recruited at six different sites across the United States. A total of 3066 subjects were screened by telephone. Of those, 542 met the study’s eligibility criteria and were invited for an assessment. Participants were excluded if they had an unstable medical condition, endorsed major school refusal (e.g. missing more than 25% of school days in the most recent term), did not speak English, had an IQ below 80, were pregnant, endorsed suicidal or homicidal ideation, were taking a non-stimulant psychoactive medication, or had a previous unsuccessful trial with cognitive behavioral therapy or a psychotropic medication (for full inclusion/exclusion, see Compton et al., 2010). Four hundred and eighty-eight youth met diagnostic criteria for a principal diagnosis of GAD, SoP, or SAD, and were randomized into one of four treatment conditions as part of a larger RCT (for a description of the methods see Compton et al., 2010; for the clinical characteristics of the sample see Kendall et al., 2010). Youth with comorbid diagnoses were included as long as the disorder was co-principal or secondary to one of the aforementioned anxiety diagnoses. Participants with a diagnosis of major depression, bipolar disorder, pervasive-developmental disorder, schizophrenia, schizoaffective disorder, or untreated attention-deficit/hyperactivity disorder (ADHD) were excluded.

The present study examined the 488 youths assessed at randomization (M_age = 10.70, SD = 2.80). The sample was 78.9% Caucasian (n = 385), 9.0% African American (n = 44), 2.5% Asian American (n = 12), 1.2% American Indian (n = 6), and 8.4% other (n = 41). Gender was evenly distributed, as approximately 50.4% (n = 246) of the sample was male. Youth commonly met criteria for more than one diagnosis, both by their own and their parents’ report (see Table 3). Comorbidity with disorders other than SAD, SoP, and GAD was also common, as 270 participants (55.33%) met criteria for one or more other disorders by the composite report. The most common of these comorbidities included ADHD, oppositional-defiant disorder, and obsessive-compulsive disorder.

Table 3.

Prevalence of ADIS-IV diagnostic groupings

	Parent Report n (%)	Youth Report n (%)	Composite Report n (%)
All Three	186 (38.1)	73 (15.0)	163 (33.4)
GAD+SoP	121 (24.8)	110 (22.5)	150 (30.7)
GAD+SAD	32 (6.6)	38 (7.8)	48 (9.8)
SoP+SAD	32 (6.6)	23 (4.7)	38 (7.8)
GAD only	33 (6.8)	51 (10.5)	33 (6.8)
SoP only	62 (12.7)	29 (5.9)	40 (8.2)
SAD only	14 (2.9)	49 (10.0)	16 (3.3)
No diagnosis	8 (1.6)	115 (23.6)	--

Note: Diagnoses indicate clinician severity rating (CSR) ≥ 4. ADIS = Anxiety Disorder Interview Schedule; GAD = generalized anxiety disorder; SoP = social phobia; SAD = separation anxiety disorder.

Measures

Child behavior checklist (CBCL; Achenbach, 1991; Achenbach & Rescorla, 2001)

The CBCL (Achenbach & Rescorla, 2001) is a 118-item questionnaire completed by parents that assesses a child’s emotional/behavioral problems, and social and academic competencies. Items are rated on a 3-point scale ranging from 0 to 2, with higher scores indicating greater problems. The CBCL has been standardized with normative reference points indicating a child’s functioning relative to the population (Achenbach & Edelbrock, 1983; Achenbach & Rescorla, 2001). The CBCL consists of broad Internalizing and Externalizing scales, as well as eight syndrome subscales (see Table 1 for itemized listing of subscales examined herein).

Table 1.

Child Behavior Checklist (CBCL) scale items

CBCL Item	CBCL- A	CBCL- AP	CBCL- A/D	CBCL- INT	CBCL- Som
5. Enjoys little				X
9. Cannot get his/her mind off certain thoughts	X
11. Clings to adults or too dependent	X	X
14. Cries a lot			X	X
29. Fears certain animals, situations, or places other than school	X	X	X	X
30. Fears going to school.	X	X	X	X
31. Fears he/she might think or do something bad	X		X	X
32. Feels he/she has to be perfect.	X		X	X
33. Feels unloved			X	X
35. Feels worthless			X	X
42. Rather be alone				X
45. Nervous, high-strung, or tense.	X	X	X	X
46. Nervous moments or twitching.	X
47. Nightmares				X	X
49. Constipated				X	X
50. Too fearful or nervous.	X	X	X	X
51. Feels dizzy				X	X
52. Feels too guilty			X	X
54. Overtired				X	X
56a. Aches or pains	X			X	X
56b. Headaches	X			X	X
56c. Nausea, feels sick	X			X	X
56d. Eye problems				X	X
56e. Skin problems				X	X
56f. Stomachaches or cramps.	X			X	X
56g. Vomiting				X	X
65. Won’t talk				X
69. Secretive				X
71. Self-conscious or easily embarrassed	X		X	X
75. Shy or timid.	X			X
91. Talks about killing self			X	X
102. Lacks energy				X
103. Sad				X
111. Withdrawn				X
112. Worries	X	X	X	X

NOTE: CBCL-A = Child Behavior Checklist-anxiety scale; CBCL-INT = Child Behavior Checklist-internalizing dimension; CBCL-A/D = Child Behavior Checklist-Anxious/Depressed syndrome; CBCL-AP = Child Behavior Checklist-Anxiety Subscale; CBCL-SOM = Child Behavior Checklist- Somatic Subscale.

Anxious/depressed subscale (CBCL-A/D)

The CBCL-A/D consists of 13 items relating to anxious and depressive symptomatology. The CBCL-A/D has demonstrated good retest reliability (r = .86) and inter-rater reliability (r = .77; Achenbach, 1991). The scale has been shown to discriminate between youth with and without a depression diagnosis (Biederman, Faraone, Mick & Moore, 1996), as well as youth with and without an anxiety disorder diagnosis (Aschenbrand et al., 2005; Biederman et al., 1993; Edelbrock & Costello, 1998). Internal consistency for CBCL-A/D scale in present sample was .83 (Cronbach’s alpha).

Somatic Complaints subscale (CBCL-SOM)

The CBCL-SOM is an 11-item subscale that reflects a range of physiological symptoms (e.g. constipation, dizziness, headaches). The CBCL-SOM has good retest reliability (r = .95) and adequate interrater reliability (r = .52; Achenbach, 1991). Internal consistency for CBCL-SOM scale in present sample was .88 (Cronbach’s alpha).

Internalizing subscale (CBCL-INT)

The CBCL-INT is comprised of 33 items from the Anxious/Depressed, Somatic Complaints, and Withdrawn syndrome subscales. The CBCL-INT has demonstrated high retest reliability (r = .89) and adequate inter-rater reliability (r = .66; Achenbach, 1991). The scale has the ability to discriminate between children with anxiety disorders and those with or without externalizing disorders (Aschenbrand et al., 2005; Seligman et al., 2004). The CBCL-INT did not perform well when discriminating between youths with anxiety and affective disorders (Seligman et al., 2004). Internal consistency for CBCL-INT scale in present sample was .93 (Cronbach’s alpha).

Anxiety Problems Subscale (CBCL-AP; Achenbach, Dumenci, & Rescorla, 2003)

The CBCL-AP is a 6-item scale that represents symptoms of GAD, SAD, and specific phobia. The CBCL-AP is a DSM-oriented scale that was constructed based on experts’ selection of items that reflect DSM-IV diagnostic criteria. The CBCL-AP has demonstrated good retest reliability (Achenbach, Dumenci, & Rescorla, 2003), and some convergent and divergent validity (Nakamura et al., 2009). Concurrent validity on the CBCL-AP has ranged from fair to good (Ebesutani et al., 2010; Ferdinand, 2008). Internal consistency for CBCL-AP scale in present sample was .67 (Cronbach’s alpha).

Anxiety Subscale (CBCL-A; Kendall et al., 2007)

The 16-item CBCL-A includes the same 6 items on the CBCL-AP and 10 additional items regarding somatic symptoms. The CBCL-A has demonstrated high internal consistency, some construct validity, sensitivity to treatment, and has discriminated anxious youth from nonanxious youth (Kendall et al., 2007). Internal consistency for CBCL-A/D scale in present sample was .75 (Cronbach’s alpha).

Anxiety Disorder Interview Schedule for Children and Parents (ADIS-C/P; Silverman & Albano, 1996)

The ADIS-C/P is a semistructured diagnostic interview that assesses child psychopathology, particularly anxiety disorders. Composite diagnoses were derived from separate interviews with parents and children. Clinical severity ratings (CSRs) were assigned by clinicians using a 9 point scale (i.e. 0–8), with a “0” indicating no impairment, and an “8” signifying severe impairment. Impairment was defined as impacting several domains of the child’s life and/or causing significant distress. A CSR ≥ 4 designated a DSM-IV diagnosis. The ADIS C/P has demonstrated excellent inter-rater reliability (r = .98 for the parent interview and .93 for the child interview) and high retest reliability (k > .70 for both interviews; Silverman & Nelles, 1988; Silverman, Saavadra, & Pina, 2001). The ADIS C/P also has shown strong concurrent validity (Wood, Piacentini, Bergman, McCracken, & Barrios, 2002), and sensitivity to treatment-produced change (e.g., Kendall et al., 2008). Agreement between diagnosticians was high for anxiety severity (CSR; r = .85).

Procedure

Independent evaluators (IEs) administered the ADIS C/P and were monitored for reliability (diagnostic status, κ = .82–.88). IEs consisted of MA-level psychologists, social workers, doctorate-level psychologists, child psychiatrists, and a nurse practitioner. The 488 youths and their parents were assessed by IEs and interviewed separately whenever possible. IEs assigned principal anxiety diagnoses based on the symptoms that were most interfering and distressing. Parent and child diagnoses were obtained from CSRs ascertained directly from interviews with each respective participant. Composite diagnoses were formed based on both the parent- and child-reports. Given the high rates of comorbidity in the present study (as is typical for anxiety disorders in youth; Kendall et al, 2010), our sample was divided into orthogonal diagnostic groups to reflect their comorbid anxiety presentation. Diagnostic groups included GAD only, SoP only, SAD only, GAD + SoP, GAD + SAD, SoP + SAD, and All Three (concurrent diagnoses of GAD, SoP, and SAD). For comorbid groups, overall CSRs were analyzed using weighted averages of CSRs of the anxiety diagnoses. In addition to the diagnostic interview, the mother of the anxious youth completed the CBCL (Achenbach & Rescorla, 2001) as part of a larger battery of self-report measures. For the present study, totals for the CBCL-INT, CBCL-A/D, CBCL-AP, CBCL-SOM, and CBCL-A scales were calculated. Raw scores on each of the CBCL anxiety scales were used to predict the presence and severity of each diagnostic group as measured by the ADIS C/P. Use of raw scores facilitated comparison across subscales, given that the CBCL-A, a relatively new scale, does not have normative data available for T-score comparisons. Missing data were minimal and not problematic: due to missing/incomplete data, only 7 cases (1.4%) were removed in the analysis of the CBCL-INT, CBCL-A/D, and CBCL-AP subscales, and only 9 cases (1.8%) were removed in the analysis of the CBCL-A and CBCL-SOM subscales.

Data Analysis

To determine the relation between the CBCL-A scale and DSM-IV anxiety disorders, ROC analyses were conducted. These analyses investigated how well CBCL-A scores, as a dimensional predictor, predicted the presence or absence of each diagnostic grouping set as a dichotomous outcome. Area-under-the-curve (AUC) values reflect the relative strength of prediction of our dichotomous variables (presence/absence of diagnostic groups) from our continuous variables (CBCL subscale scores). AUC values falling between .50–.70 indicate poor prediction, .70 −.80, fair; .80 −.90, good; and .90 −1.00, excellent prediction (Ferdinand, 2008). The relation between CBCL-A scores and both parent-, youth-, and composite-report of anxiety diagnoses were entered into separate ROC analyses. Binormal comparative ROC analyses investigated the relative sensitivity of each CBCL scale (CBCL-A, INT, A/D, AP, SOM) in identifying the presence of each diagnostic group for pairings in which subscales were significant independent predictors of presence of diagnostic group. Such an approach is robust to the potential effects of high multicollinearity between CBCL subscales. Chi-square estimates were examined, with the value farthest from 0.5 (i.e., chance estimate) representing a more sensitive predictor of each given disorder.

Hierarchical regression analyses assessed the relative ability of CBCL subscales (CBCL-A, INT, A/D, AP, SOM) to predict severity of anxiety diagnostic group.¹ Comparisons between subscales in predicting anxiety disorder CSR were examined for pairings in which subscales were significant independent predictors of anxiety disorder CSR. Interaction terms between the CBCL-A and each of the other subscales (INT, A/D, AP, SOM) were entered into the second step of a hierarchical regression to assess for differences in predictive ability. Separate models were run for each diagnostic group by each report type (i.e., composite, parent, youth).

Results

Means and standard deviations of CBCL scale raw scores across diagnostic groups and report types, as well as correlations between subscales, are found in Table 2. Regarding normality, all values of skew and kurtosis were within acceptable limits. All VIF and tolerance statistics fell within acceptable limits (all VIF < 10, all tolerance > 1). Prevalence of anxiety disorder diagnostic groups are provided in Table 3.

Table 2.

Means, Standard Deviations, and Correlation between CBCL Scales

CBCL scales	N	M	SD	Avg. T-Score			r
					A	INT	A/D	AP	SOM
CBCL-A	479	13.52	5.38	-	-
CBCL-INT	481	18.79	8.49	67.75	.86	-
CBCL-A/D	481	9.76	4.44	68.09	.85	.85	-
CBCL-AP	481	6.15	2.70	68.92	.84	.68	.80	-
CBCL-SOM	479	4.67	3.45	63.92	.69	.72	.45	.42	-

Note. CBCL = Child Behavior Checklist; CBCL-A = Child Behavior Checklist-anxiety scale; CBCL-INT = Child Behavior Checklist-internalizing dimension; CBCL-A/D = Child Behavior Checklist-Anxious/Depressed syndrome; CBCL-AP = Child Behavior Checklist-Anxiety Subscale; CBCL-SOM = Child Behavior Checklist- Somatic Subscale; GAD = generalized anxiety disorder; SoP = social phobia; SAD = separation anxiety disorder. All listed correlations statistically significant at p < .01.

Receiver Operating Characteristics (ROC) Analyses: Presence of Disorder

Results of the CBCL-A ROC analyses predicting parent diagnosis revealed AUC values of 0.68 for All Three (p < .001), 0.48 for GAD+SoP (p = .60), 0.52 for GAD+SAD (p = .74), 0.40 for SoP+SAD (p = .06), 0.39 for GAD only (p < .05), 0.30 for SoP only (p < .001), and 0.50 for SAD only (p = 0.99). Results of the CBCL-A ROC analyses predicting youth diagnosis revealed AUC values of 0.63 for All Three (p < .001), 0.54 for GAD+SoP (p = .17), 0.58 for GAD+SAD (p = .13), 0.51 for SoP+SAD (p = .51), 0.39 for GAD only (p = .01), 0.37 for SoP only (p < .01), and 0.44 for SAD only (p = 0.32). Results of ROC analyses of CBCL-A predicting composite diagnosis revealed AUC values of 0.65 for All Three (p < .001), 0.47 for GAD+SoP (p = .35), 0.53 for GAD+SAD (p = .51), 0.43 for SoP+SAD (p = .13), 0.40 for GAD only (p = .054), 0.27 for SoP only (p < .001), and 0.47 for SAD only (p = 0.70). Although CBCL-A scores yielded significant sensitivity for the identification of All Three (all report types), GAD only (parent, youth report), and SoP only (all report types), AUC interpretive guidelines indicate that all individual AUC values for these diagnostic groups fell in the poor range. Furthermore, some of these AUC values are less than 0.5 (e.g., AUC = 0.27 for SoP only, composite report), indicating that these scale scores are actually significant negative predictors of our diagnostic groups.

Likewise, AUC values for the other scales (INT, A/D, AP, SOM) predominantly all in the poor range, and many were not statistically significant from chance (0.5). In addition to the CBCL-A subscale, the following subscales were significant predictors of disorder presence: by parent report, A/D significantly predicted presence of All Three (AUC = 0.65, p< .001), SoP+SAD (AUC = 0.34, p<.01), and SoP only (AUC = 0.32, p<.001); INT significantly predicted presence of All Three (AUC = 0.62, p<.001), GAD only (AUC = 0.36, p<.05), and SoP only (AUC = 0.42, p<.05); AP significantly predicted presence of All Three (AUC = 0.71, p<.001), GAD+SoP (AUC = 0.43, p<.05), GAD+SAD (AUC = 0.61, p<.05), GAD only (AUC = 0.36, p=.01), and SoP only (AUC = 0.25, p<.001); and SOM significantly predicted presence of All Three (AUC = 0.61, p<.001), and SoP only (AUC = 0.36, p<.001). By youth report, A/D significantly predicted presence of All Three (AUC = 0.60, p< .05), GAD+SoP (AUC = 0.59, p<.01), and SoP only (AUC = 0.39, p<.05); INT significantly predicted presence of All Three (AUC = 0.60, p<.05), GAD+SoP (AUC = 0.60, p<.01), GAD only (AUC = 0.43, p<.01), and SoP only (AUC = 0.42, p<.05); AP significantly predicted presence of All Three (AUC = 0.61, p<.01), GAD+SAD (AUC = 0.62, p<.05), GAD only (AUC = 0.42, p<.05), and SoP only (AUC = 0.39, p<.05); SOM significantly predicted presence of All Three (AUC = 0.62, p<.01), SAD+GAD (AUC = 0.61, p<.05), and SoP only (AUC=0.35, p<.001). By composite report, A/D significantly predicted presence of All Three (AUC = 0.62, p< .001), SoP+SAD (AUC = 0.35, p<.01), and SoP only (AUC = 0.28, p<.001); INT significantly predicted presence of All Three (AUC = 0.62, p<.001), SAD only (AUC = 0.36, p<.05), and SoP only (AUC = 0.36, p<.01); AP significantly predicted presence of All Three (AUC = 0.68, p<.001), GAD+SoP (AUC = 0.40, p=.001), GAD+SAD (AUC = 0.61, p<.01), GAD only (AUC = 0.36, p<.05), and SoP only (AUC = 0.25, p<.001); and SOM significantly predicted presence of All Three (AUC = 0.60, p<.001), and SoP only (AUC = 0.31, p<.001).

Comparative ROC analyses were conducted to assess the relative prediction of anxiety diagnosis by each CBCL subscale pairing with the CBCL-A in which individual ROC analyses indicated significant differences from chance for each subscale (see Tables 4 and 5). These analyses indicated that the CBCL-A was identified as a better predictor of the All Three diagnostic group compared to A/D (by all report types), to INT (by parent and composite report, no differences by youth report), and SOM (by parent and composite report, no differences by youth report). No differences were found between the CBCL-A and AP subscales in identification of presence of All Three diagnostic group by any report type. With regard to presence of the GAD only group, no differences were found between the CBCL-A and INT or AP subscales by the parent or youth report. Regarding presence of SoP only, CBCL-A subgroup emerged as a stronger negative predictor than INT (by all report types) and SOM (by parent report). The AP subscale emerged as a stronger negative predictor of SoP only, as compared to the CBCL-A by the parent repot of diagnosis. No differences in the predictive abilities of CBCL-A and A/D for SoP only were identified by any report type. Table 5 summarizes the ROC results.

Table 4.

Binormal comparative ROC analyses

Parent Report		AUC	S. E.	χ2
All Three	CBCL-A	0.68***	0.02	--
	vs. A/D	0.65***	0.02	7.75**
	vs. INT	0.62***	0.03	22.14***
	vs. AP	0.71***	0.02	2.05
	vs. SOM	0.61***	0.03	12.61***
GAD only	CBCL-A	0.38*	0.04	--
	vs. INT	0.36*	0.04	0.34
	vs. AP	0.36**	0.04	0.64
SoP only	CBCL-A	0.30***	0.03	--
	vs. A/D	0.32***	0.03	0.59
	vs. INT	0.42*	0.04	36.03***
	vs. AP	0.25***	0.03	7.28**
	vs. SOM	0.36***	0.04	3.98*
Youth Report
All three	CBCL-A	0.63***	0.03	--
	vs. A/D	0.60*	0.03	4.09*
	vs. INT	0.06*	0.03	2.50
	vs. AP	0.61**	0.03	1.05
	vs. SOM	0.62**	0.03	0.19
GAD only	CBCL-A	0.38**	0.04	--
	vs. INT	0.38**	0.04	0.01
	vs. AP	0.39*	0.04	0.12
SoP only	CBCL-A	0.37**	0.04	--
	vs. A/D	0.39*	0.04	0.83
	vs. INT	0.42*	0.04	4.54*
	vs. AP	0.39*	0.04	1.27
	vs. SOM	0.35***	0.04	0.25
Composite Report
All Three	CBCL-A	0.66***	0.03	--
	vs. A/D	0.62***	0.03	4.60*
	vs. INT	0.62***	0.03	6.61**
	vs. AP	0.68***	0.03	2.66
	vs. SOM	0.60***	0.03	5.34*
SoP	CBCL-A	0.27***	0.04	--
	vs. A/D	0.28***	0.04	0.16
	vs. INT	0.36**	0.04	13.69***
	vs. AP	0.25***	0.04	1.13
	vs. SOM	0.31***	0.04	2.17

Note: AUC = Area Under the Curve; GAD = generalized anxiety disorder, SAD = separation anxiety disorder; SoP = social phobia; CBCL-A = Child Behavior Checklist-anxiety scale; CBCL-INT = Child Behavior Checklist-internalizing dimension; CBCL-A/D = Child Behavior Checklist-Anxious/Depressed syndrome; CBCL-AP = Child Behavior Checklist-Anxiety Problems Subscale; CBCL-SOM = Child Behavior Checklist- Somatic Subscale. Degrees of freedom = 1 for all models.

^*p <.05,

^**p <.01,

^***p<.001.

Table 5.

Binormal ROC Comparisons Summary

Diagnosis		Subscale Comparison (CBCL-A vs.)
	A/D	INT	AP	SOM
All Three	CBCL-Aδ	CBCL-Aα,γ	ns	CBCL-Aα,γ
GAD + SoP	--	--	--	--
GAD + SAD	--	--	--	--
SAD + SoP	--	--	--	--
GAD only	--	ns	ns	--
SAD only	--	--	--	--
SoP only	ns	CBCL-A* δ	AP* α	CBCL-A* α

Note: This table indicates which CBCL subscale was indicated to be a better predictor of diagnosis as ascertained by comparative ROC analyses. Significance by report type indicated by superscript: α = by parent report; β = by youth report, γ = by composite report, δ = by all report types. All Three = concurrent comorbid diagnoses of GAD, SAD, and SoP; GAD = generalized anxiety disorder, SAD = separation anxiety disorder; SoP = social phobia; CBCL-A = Child Behavior Checklist-anxiety scale; CBCL-INT = Child Behavior Checklist-internalizing dimension; CBCL-A/D = Child Behavior Checklist-Anxious/Depressed syndrome; CBCL-AP = Child Behavior Checklist-Anxiety Subscale; ns = nonsignificant comparison.

^*indicates that subscale was stronger negative predictor, rather than positive.

Hierarchical Regression Analyses: Clinician Severity Ratings

All CBCL subscales were initially examined independently in their prediction of disorder CSR; results are presented in Table 6. The CBCL-A subscale was then compared against other subscales when initial regression analyses indicated that both subscales significantly predicted CSR. When the CBCL-A was compared against all other subscales in ability to predict CSR of diagnostic group, only two significant differences emerged. Specifically, the CBCL-A arose as a stronger predictor of GAD only by the parent report than SOM, t(475) = −2.16, p < .05. Additionally, INT emerged as a stronger predictor of SoP only by youth report than the CBCL-A, t(475) = −2.07, p<.05. Otherwise, no significant differences in strength of predictive ability for anxiety disorder CSR were identified for most of the CBCL subscale pairings (all interaction ps > .05).

Table 6.

Hierarchical Regression Analyses: Individual prediction to diagnostic group

Diagnostic Group		Parent	β	Composite
			Child
All Three	CBCL-A	.26***	.11	.17*
	A/D	.24***	.19	.14
	INT	.25***	.10	.16*
	AP	.32***	.21	.16*
	SOM	.08	.06	.11
GAD+SoP	CBCL-A	.10	.08	.22**
	A/D	.13	.06	.27***
	INT	.17	.10	.28***
	AP	.17	.07	.25**
	SOM	.02	.05	.06
GAD +SAD	CBCL-A	.14	−.06	.22
	A/D	.19	−.06	.17
	INT	.25	−.09	.26
	AP	.25	−.09	.21
	SOM	.23	.03	.37**
SoP +SAD	CBCL-A	.13	.53**	.20
	A/D	.15	.49*	.12
	INT	.10	.47*	.19
	AP	.15	.37	.06
	SOM	.02	.30	.07
GAD only	CBCL-A	.34***	.14**	.25***
	A/D	.34***	.15***	.20***
	INT	.27***	.15***	.23***
	AP	.33***	.05	.20***
	SOM	.19***	.14**	.20***
SAD only	CBCL-A	.28***	.21***	.23***
	A/D	.18***	.11*	.13**
	INT	.10*	.10*	.07
	AP	.41***	.27***	.36***
	SOM	.18***	.21***	.19***
SoP only	CBCL-A	.16***	.13**	.11*
	A/D	.17***	.13**	.10
	INT	.27***	.20***	.21***
	AP	.10*	.02	.03
	SOM	.03	.06	−.21

Note: Subscales examined in represent separate models. All Three = concurrent comorbid diagnoses of GAD, SAD, and SoP; GAD = generalized anxiety disorder, SAD = separation anxiety disorder; SoP = social phobia; CBCL-A = Child Behavior Checklist-anxiety scale; CBCL-INT = Child Behavior Checklist-internalizing dimension; CBCL-A/D = Child Behavior Checklist-Anxious/Depressed syndrome; CBCL-AP = Child Behavior Checklist-Anxiety Subscale; CBCL-SOM = Child Behavior Checklist- Somatic Subscale.

^*p≤.05

^**p<.01

^***p≤.001.

Discussion

The CBCL is among the most commonly used scales in clinical, school, and community settings. Its comprehensive assessment of multiple problem areas makes it more appealing and convenient than multiple problem-specific questionnaires. The CBCL subscales provide a variety of ways to assess psychopathology including both continuous, empirically driven syndromes and categorical, DSM-based disorders. The present study extends prior work by exploring the relative benefits of using the CBCL-A, given its unique combination of somatic and non-somatic symptoms of anxiety, in comparison to other CBCL scales in identifying the presence and severity of specific anxiety disorder presentations

The results of the comparative ROC analyses indicate that the CBCL-A emerged as a stronger predictor of youth with comorbid diagnoses of the combined GAD, SAD, and SoP group (All Three diagnostic group) over the A/D, INT, and Somatic subscales, although its performance was not statistically different from the AP scale. This finding was largely consistent across the parent and composite diagnostic reporters. No predictive advantage was identified across other subscale comparisons, and in fact the CBCL-A and AP subscales were identified as the strongest negative predictors of presence of SoP only (see Table 5). Furthermore, subscales’ low-level AUC values were often not significantly different from chance (0.5), and those that were, remained predominantly in the poor range (AUC < .70). Such a pattern likely indicates that the CBCL (and its many anxiety-oriented subscales) may not be a sensitive screener for specific anxiety disorders, as much as it is useful for identifying the presence of generally high symptoms of anxiety. This interpretation is bolstered by the consistent ability of all subscales in the prediction of the All Three conglomerate diagnostic grouping. Similarly, findings from the regression analyses, indicated that although many of the individual subscales significantly predicted disorder CSR, there was little predictive difference when subscales were compared to one another. Such findings make sense when one considers that the anxiety-specific CBCL subscales were designed to identify symptoms of elevated anxiety, generally, rather than specific anxiety diagnoses.

It is interesting that the inclusion of somatic symptoms did not result in increased sensitivity to predict specific anxiety disorders, given that they were thought to make the CBCL-A more adept at identifying anxiety disorders for which somatic symptoms are often a presenting concern. Considering the separate disorders, it was expected that the CBCL-A would not be a better predictor of diagnostic categories that included SoP given that symptoms often manifest in the context of peers and there are no somatic diagnostic criteria. This expectation was supported by findings that the CBCL-A emerged as a stronger predictor of the SoP only subgroup than the INT subscale. However, most of the CBCL subscales were significant negative predictors of this subgroup, possibly indicating that the CBCL subscales are not particularly sensitive in the identification of this diagnostic group. For GAD, the presence of somatic symptoms is neither necessary nor sufficient to meet diagnostic criteria; a child may be diagnosed with GAD without any somatic symptoms (e.g. meeting secondary criteria of irritability and sleep problems). Furthermore, research on GAD in adults suggests that worry is associated with a reduced range of physiological variability, including parasympathetic deficiency (Borkovec, Ray & Stöber, 1998). With regard to SAD, although somatic symptoms are specific criteria in the DSM definition of this disorder, it is possible that parents doubt the validity of somatic complaints by youth (i.e., view them as instrumental to avoid separation). Nonetheless, while there is evidence that anxiety disorders broadly are associated with somatic symptoms (Ginsburg, Riddle, & Davies, 2006; Hofflich, Hughes, & Kendall, 2006), individual differences and heterogeneity of presentation within individual disorders may account for the lack of differences between CBCL scales, as well as the difficulty for some subscales to accurately identify the diagnostic groups used in this study.

The current study did reveal some distinctions between CBCL subscales. The ROC results suggest the superior predictive ability of DSM-oriented scales (CBCL-A and CBCL-AP) in identifying SAD, by either parent or composite report, compared to the broadband (CBCL-INT) and symptom (CBCL-A/D) scales, although this finding was not consistent for the other anxiety disorders (i.e., GAD, SoP). The A/D scale outperformed the CBCL-A scale in predicting the presence of GAD by the composite report. This difference may be explained by the substantial rates of comorbidity between GAD and MDD (adjusted lifetime OR = 5.7; Grant, Hasin, Stinson, et al., 2005), overlap in phenotypic presentation (e.g., negative affectivity), and overlap in DSM-IV diagnostic criteria (e.g., fatigue, sleep disturbance, difficulty concentrating; APA, 2000) between GAD and mood disorders, including major depressive disorder (Mennin, Heimberg, Fresco, & Ritter, 2008). Although youth with MDD were not included, those with subthreshold mood symptoms remained (Kendall et al., 2010). Furthermore, the INT subscale outperformed the CBCL-A in identifying SoP by both the parent and composite report of diagnosis. The INT subscale also pulls from parent-report of somatic symptoms, but includes symptoms of withdrawal likely to represent youth with SoP, who often engage less with others and focus inward. However, given that AUC values for these scales in identifying SoP diagnoses fell in the poor range, and no significant differences were found in the ability of these scales to predict diagnostic severity, these differences may not be clinically useful.

Our findings have clinical implications. The ROC analyses found that the All Three diagnostic group, by each reporter, was most frequently and sensitively identified by each subscale score. These results support the use of the CBCL as a general screener, rather than a tool for the identification of specific anxiety diagnoses. Perhaps anxiety is best viewed as a single unitary construct, with the separate disorders being less meaningful than is implied by the DSM categorical system (e.g., Weems, 2008). In practice, clinicians might use the CBCL to identify generally elevated anxiety, and use disorder-specific measures (e.g., SPAIC for SoP; Beidel, Turner, & Morris, 1995) or diagnostic interviews (e.g., ADIS-IV; Silverman & Albano, 1996) subsequently, as needed. However, findings also indicate some weaknesses in the CBCL scales’ ability to identify particular diagnostic groups. In particular, for diagnostic groups comprised of youth with SoP (i.e., SoP only, SAD+SoP), most subscales emerged as significant negative predictors (i.e., higher subscale scores predicting absence of disorder), with AUC values less than 0.5 that were significantly different from chance. It may be that the parent-reported answers to CBCL scale questions do not accurately tap into the clinical presentation of this disorder.

This study is not without limitations. First, comorbidity across the three diagnoses was high. Our diagnostic grouping procedure aimed to account for the frequent concurrent presentation of disorders, although some of our groupings were less common, resulting in numbers per group (e.g., SAD only by parent report n = 14). As such, some of our groups may be underpowered to represent their clinical presentation for the present analyses. Second, normative data were not available for the CBCL-A. As such, analyses concern mean raw scores rather than normed T-scores. Although the present data provide meaningful diagnostic information, cautions should be used when comparing the CBCL-A scores against those of other CBCL subscales. Additionally, concordance between the parent-reported principal anxiety disorder and parent-completed CBCL may reflect shared method variance. However, we have attempted to account for this possibility by comparing parent report of diagnosis with youth and composite report in order to lend some convergent validity.

The present results indicate directions for future research. For example, the establishment of normative T-scores for the CBCL-A scale would permit normative comparisons with other CBCL scales. Additionally, CBCL subscales may be compared in their ability to identify other anxiety disorders, for which somatic symptoms are important features, particularly panic disorder. Such an analysis was not feasible within the present sample given the rates of these other diagnoses in our sample. The CBCL-A/D scale, in particular, includes items representing both anxious and depressive symptoms and future research could examine the ability of these scales to discriminate samples of youth with non-anxiety affective disorders (e.g., depression, dysthymia).