An Analysis of the Child Behavior Checklist Anxiety Problems Scale’s Predictive Capabilities

Abstract

The Child Behavior Checklist (CBCL) is widely used to assess behavioral and emotional problems in youth. The CBCL Diagnostic and Statistical Manual (DSM)-Oriented Anxiety Problems subscale (CBCL-AP) was developed for the identification of DSM-IV anxiety disorders. Using data from 298 youth aged 6- to 18, the CBCL-AP scale was examined to determine its ability to differentially predict, via Receiver Operating Characteristics (ROC) analysis, the presence of (a) generalized anxiety disorder (GAD), (b) separation anxiety disorder (SAD), (c) specific phobia (SPPH), or (d) the presence of any of these disorders. Independent Evaluators (IEs) administered the Anxiety Disorders Interview Schedule for Children (ADIS-C/P) to determine the presence of an anxiety disorder. The ability of the CBCL-AP to predict to anxiety disorders was compared to the ability of the CBCL Anxious/Depressed (CBCL-A/D) scale and the seven empirically derived CBCL syndrome subscales and five DSM-Oriented subscales to predict anxiety disorder diagnoses. Results revealed that CBCL-AP scores significantly predicted all diagnoses. CBCL-A/D scores significantly predicted SAD (AUC = 0.67), GAD (AUC = 0.69), and the presence of any of the three disorders (AUC = 0.72), but not the presence of SPPH (AUC = 0.52). Although the CBCL-AP scale may not be a substitute for extensive diagnostics, it has demonstrated utility as an instrument for assessing anxiety and can serve to identify anxious youth in need of mental health services.

Anxiety disorders are common psychological problems affecting children and adolescents (Merikangas, Nakamura, & Kessler, 2009) with data indicating a 31.9% prevalence rate for any anxiety disorder among 13-18-year olds (Merikangas et al., 2010). Data from the National Comorbidity Survey–Adolescent Supplement (NCS-A) showed prevalence rates of 2.2% for generalized anxiety disorder (GAD), 2.4% for agoraphobia, 7.6% for separation anxiety disorder (SAD), 9.1% for social anxiety disorder, and 19.3% for specific phobia (SPPH) (Merikangas et al., 2010). Other data suggest an anxiety disorder prevalence rate of 20% (Chavira, Stein, Bailey, & Stein, 2004). In addition to interfering symptoms, anxiety is associated with multiple functional impairments (Swan & Kendall, 2017). Youth with anxiety disorders are at increased risk of alcohol abuse in adolescence (Schuckit & Hesselbrock, 1994), and those who do not receive treatment are at greater risk of atypical psychosocial development, substance abuse later in life, and further mental health problems as adults (Connolly et al., 2007; Essau, Conradt, & Petermann, 2000; Kim-Cohen et al., 2003; Pine, Cohen, Gurley, Brook, & Ma, 1998; Puleo, Conner, Benjamin, & Kendall, 2011; Wolk, Kendall & Beidas, 2015).

The identification of anxiety in youth plays a central role in optimal mental health services. The assessment of youth anxiety is best when multiple sources (parents, the child, independent evaluators, teachers) are considered (Kazdin & Weisz, 1998; Kendall et al., 2000). Accurate assessment is especially important in light of the fact that primary care physicians and teachers may not notice anxiety issues in youth (Kendall, Panichelli-Mindel, Sugarman, & Callahan, 1997; Wren, Scholle, Heo, & Comer, 2003). Because parents typically initiate treatment on behalf of their child (Choudhury et al., 2003), and because youth may not recognize (or lack the ability to articulate) the interference or impairment anxiety may be causing in their family relationships, school, or other social interactions, parent report is considered an indispensable component for assessing youth anxiety (Langley, Bergman, & Piacentini, 2002). Given the importance of assessment, semi-structured diagnostic interviews with both parents and children are a part of the process, but diagnostic interviews require both trained assessors and a substantial amount of time. As such, easily administered, cost-effective, and efficient parental questionnaires fill an important need (Yates & Taub, 2003).

The Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001) is a parent-report questionnaire used to assess behavioral and emotional problems, as well as competencies, in children and adolescents aged 6-18. The CBCL first established empirically derived subscales, including the Internalizing syndrome scale (CBCL-INT) and the Anxious/Depressed (CBCL-A/D) scale, both of which indicate internalizing problems and focus on anxiety symptoms (Achenbach, 1995); both have been shown to differentiate between youth with and without anxiety disorders (Aschenbrand, Angelosante, & Kendall, 2005; Seligman, Ollendick, Langley, & Baldacci, 2004). Such empirically derived dimensional subscales, however, were not intended to reflect the DSM diagnostic categories (Kendall et al., 2007; Nakamura, Ebesutani, Bernstein, & Chorpita, 2009). Given the role that the DSM plays in mental health services, research, and communication about treatment, Achenbach and colleagues (2003) developed DSM-Oriented Scales to establish a greater connection between CBCL scores and DSM classifications, and the Anxiety Problems subscale (CBCL-AP) has shown mixed evidence regarding its ability to differentiate between anxious and non-anxious youth (Achenbach et al., 2003; Nakamura et al., 2009).

Using Area Under the Curve (AUC) values, Ferdinand (2008) found that the CBCL-AP demonstrated “poor” ability to predict anxiety disorder diagnosis by clinician severity rating (AUC = 0.65) and a “fair” ability to predict anxiety disorder diagnosis by parent-reported impairment (AUC = 0.70). Lacalle, Ezpeleta, and Domenech (2012) also found the CBCL-AP a “poor” predictor of anxiety disorders (AUC = 0.67) diagnosed using the Diagnostic Interview for Children and Adolescents (DICA-IV; Reich, 2000). Ebesutani et al. (2010) reported that when compared to the CBCL-A/D, the CBCL-AP had significantly greater AUC values, demonstrating both a superior ability to discriminate youth with any of the anxiety disorders specifically targeted by the CBCL-AP scale (GAD, SAD, and/or SPPH) from those without any of the three disorders, and a superior ability to discriminate those with GAD, SAD, and/or SPHH from those with an affective disorder but no anxiety disorder. Using parent reports of outpatients and inpatients referred for mental health services, Pauschardt, Remschmidt, and Mattejat (2010) found that the CBCL-AP demonstrated an ability to predict GAD, SAD, and/or SPHH diagnosis (AUC = 0.70 for both a sample of outpatients and a sample of inpatients). However, using the CBCL-AP, fully 23-24% of youth with anxiety disorders were not properly recognized, and therefore were considered false negatives, whereas 41-42% of those without an anxiety disorder were false positives as they had scores surpassing the scale’s cut point. Additionally, Pauschardt et al. (2010) found that the CBCL-AP demonstrated a stronger ability to predict GAD, SAD, and/or SPPH diagnosis than the CBCL-A/D for both a sample of inpatients (AP AUC = 0.70; A/D AUC = 0.52) and a sample of outpatients (AP AUC = 0.70; A/D AUC = 0.61). None of these studies, however, make a direct comparison between the ability of the CBCL-AP and the CBCL-A/D to predict specific anxiety disorder diagnoses.

How well does the DSM-Oriented Anxiety Problems scale of the CBCL predict anxiety disorders in youth? Can the CBCL-AP predict specific anxiety disorders in youth significantly better than the CBCL-A/D or any other CBCL subscales? The present study investigated the extent to which the parent reported CBCL-AP scale can differentially predict the presence of specific anxiety disorder diagnoses (GAD, SAD, or SPPH) within a sample of children and adolescents presenting for treatment at an outpatient clinic. Although many studies of anxiety disorders in youth include social anxiety disorder, this disorder was excluded from our analyses because the DSM-IV version of the CBCL-AP does not include items consistent with social anxiety disorder diagnostic criteria. The ability of the CBCL-AP to predict anxiety disorder diagnosis in youth was also compared to the CBCL-A/D, and also to the predictive ability of the non-anxiety-related CBCL empirically based syndrome subscales (i.e., Aggressive Behavior, Attention Problems, Rule-Breaking Behavior, Somatic Complaints, Social Problems, Thought Problems, Withdrawn/Depressed) as well as CBCL DSM-Oriented non-anxiety subscales (i.e., Affective Problems, Somatic Problems, ADHD Problems, Oppositional Defiant Problems, and Conduct Problems). It was hypothesized that the CBCL-AP scale would significantly predict an anxiety disorder diagnosis (i.e. the presence of at least one of the following: SAD, GAD, or SPPH) as determined by an Independent Evaluator’s (IE’s) Clinical Severity Rating (CSR) from a diagnostic interview. Additionally, parent Global Interference Ratings (GIR) from the diagnostic interview were used as diagnostic criteria to examine the ability of the CBCL-AP to predict anxiety disorder diagnoses. It was also hypothesized that AUC values for the SAD/GAD/SPPH group (i.e. a diagnosis of any of the three) would be greater than AUC values for any specific anxiety disorder (SAD, GAD, or SPPH) individually. Further, it was predicted that the agreement between the CBCL-AP and DSM diagnoses will be higher when ADIS parent impairment ratings (GIR) are used as the criterion, relative to the use of diagnostician CSRs as the criterion. Finally, it was predicted that the CBCL-AP subscale would be able to predict disorder diagnosis in youth significantly better than the CBCL-A/D subscale and the other CBCL subscales.

Method

Participants

Pretreatment data from 298 children and adolescents (6- to 18-year-olds) seeking treatment in an anxiety disorders clinic affiliated with a mid-Atlantic University were examined. The sample was 81.5% Caucasian (n = 243), 5.0% African American (n = 15), 3.7% Asian American, (n = 11), 3.7% Hispanic (n = 11), and 5.7% identified as “other race/ethnicity” (n = 17). Gender was evenly distributed: 51.3% (n = 153) of the sample was male. Comorbidity with disorders other than SAD, GAD, and SPPH was not uncommon, with 194 youth (65.1%) meeting diagnostic criteria for at least one other disorder by parent report.

Measures

Child Behavior Checklist (CBCL; Achenbach, 1991; Achenbach & Rescorla, 2001).

The CBCL (Achenbach & Rescorla, 2001) is a 118-item parent questionnaire that assesses a youth’s behavioral and emotional problems as well as social and academic competencies. Items are rated as “Not True” (0), “Somewhat or Sometimes True” (1), or “Very True or Often True” (2). Items can be summed to obtain broad-band Internalizing and Externalizing scale scores, eight syndrome subscales scores, and six DSM-oriented scale scores. Substantial normative data are available for children ranging from ages 6 to 18, and the syndrome and DSM-oriented scales have demonstrated good reliability and validity (Achenbach & Rescorla, 2001; Achenbach, Dumenci, & Rescorla, 2003). The DSM-Oriented scales were developed by a group of psychiatrists and psychologists with an expertise in child psychopathology. The experts were asked to rate the consistency of each CBCL item with particular DSM-IV categories as either 0 (not consistent with the DSM category), 1 (somewhat consistent with the DSM category), or 2 (very consistent with the DSM category).

Anxiety Problems Subscale (CBCL-AP; Achenbach, Dumenci, & Rescorla, 2003).

The CBCL-AP is a 6-item, DSM-oriented scale to assess GAD, SAD, and specific phobia, and includes the following items: “clings to adults or too dependent,” “fears certain animals, situations, or places other than school,” “fears going to school,” “nervous, high-strung, or tense,” “too fearful or nervous,” and “worries.” The CBCL-AP has shown convergent and divergent validity (Nakamura et al., 2009), retest reliability (ICC = 0.95, p < 0.001; Achenbach, Dumenci, & Rescorla, 2003), and has demonstrated fair to good concurrent validity (Ebesutani et al., 2010; Ferdinand, 2008).

Anxious/Depressed Subscale (CBCL-A/D; Achenbach, Dumenci, & Rescorla, 2003).

The CBCL-A/D is a 13-item empirically derived syndrome subscale related to anxiety and depression symptom and has shown good inter-rater reliability (r = 0.77) and retest reliability (r = 0.86; Achenbach, 1991).

Anxiety Disorder Interview Schedule for Children and Parents (ADIS-C/P; Silverman & Albano, 1996).

The ADIS-C/P is a semi structured diagnostic interview that primarily assesses child anxiety disorders as well as other forms of child psychopathology. Diagnoses were derived from interviews with parents, and clinical severity ratings (CSRs) were assigned by doctoral students in clinical psychology using a 0 to 8 point scale, with higher ratings indicating more severe impairment (i.e. a rating of “0” indicating no impairment, a rating of “4” indicating clinically significant impairment, and a rating of “8” indicating an extreme level of impairment). If a sufficient number of parent reported symptoms are present for a given disorder, the parent is asked to rate the extent to which symptoms interfere with their child’s life. CSRs of 4 or greater indicate a DSM-IV clinical level diagnosis. As a comparison with CSRs, Global Interference Ratings (GIRs) were also obtained from interviews with parents, with the GIR assigned by the parents themselves. Similar to CSR, GIR ratings fall within a 0 to 8 point scale, with higher ratings indicating more severe interference (i.e. a rating of “0” indicating no interference in the child’s life, and a rating of “8” indicating an extreme level of interference). The ADIS-C/P has demonstrated internal consistency (k =0.80 to 0.84), retest reliability (k= 0.65 to 0.88 over a period of 7-14 days; Silverman & Nelles 1988; Silverman, Saavadra, & Pina, 2001), and concurrent validity with the Multidimensional Anxiety Scale for Children (March, 1998; Wood, Piacentini, Bergman, McCracken, & Barrios, 2002).

Procedure

IEs were doctoral students in clinical psychology and postdoctoral fellows trained to reliability and who administered the ADIS-C/P to all participants and their parents. IEs assigned anxiety disorder diagnoses based on CSRs, and a GIR was obtained from parents and was included in the analysis (>= 4). The sample was divided into the following diagnostic groups: SAD (i.e. presence of SAD diagnosis), GAD (i.e. presence of GAD diagnosis), SPPH (i.e. presence of SPPH diagnosis), and SAD/GAD/SPPH (i.e. diagnoses of any of the three anxiety disorders). Parents completed the CBCL as part of their visit to the clinic. Raw scores for the CBCL-AP were summed and used to establish CBCL-AP T-scores, which were then examined for relation to ADIS-P GIRs of 4 or greater. CBCL-AP T-scores were then examined for relation to ADIS-P CSRs of 4 or greater as assigned by diagnosticians.

Data Analysis

Receiver Operating Characteristics (ROC) analyses were conducted to determine the relationship between the CBCL-AP scale and anxiety disorders (SAD, GAD, SPPH, or SAD/GAD/SPPH). ROC analyses investigated the extent to which a dimensional score (i.e. CBCL-AP scores) predicted the dichotomous presence or absence of each diagnostic group. In four ROC analyses, SAD, GAD, SPPH, and SAD/GAD/SPPH (based on clinician assigned CSR ratings of 4 or greater) were entered as dependent variables with the CBCL-AP score entered as a predictor. The same analyses were then run using parent assigned GIRs of 4 or greater in place of CSR ratings. The resulting area-under-the-curve (AUC) values indicate the strength of the prediction. AUC values in the 0.50 to 0.70 range indicate poor prediction, whereas 0.70 to 0.80 indicates fair prediction, 0.80 to 0.90 is good prediction, and 0.90 to 1.00 is excellent prediction (Ferdinand, 2008). Using z-test comparisons, for each of the four diagnostic groups for the total sample (n = 298), AUC values for diagnostician CSR ratings were compared to the AUC values for parent GIR ratings to determine whether or not AUC values were significantly different between groups. A similar analysis was run for an “Under 12 Years Old” (<12) group (n = 186) and a “12 Years Old and Above” (>=12) group (n = 112). For each diagnostic group and for both CSRs and GIRs, z-tests were conducted to compare <12 group AUC values to >=12 group AUC values. Additionally, within the diagnostician CSR diagnostic groups, SAD/GAD/SPPH AUC values were compared to the AUC values for individual disorder diagnoses (i.e. for the diagnostician CSR group, z-tests were conducted comparing the SAD/GAD/SPPH AUC value to the SAD AUC value, the GAD AUC value, and the SPPH AUC value).

Similar ROC analyses were conducted to determine the relationship between the CBCL-A/D scale and anxiety disorders (SAD, GAD, SPPH, or SAD/GAD/SPPH). In four ROC analyses, SAD, GAD, SPPH, and SAD/GAD/SPPH (based on clinician assigned CSR ratings of 4 or greater) were entered as dependent variables with the CBCL-A/D score entered as a predictor. Then, once again, the same analysis was run using parent assigned GIRs of 4 or greater in place of CSR ratings. The resulting eight AUC values (using the CBCL-A/D subscale) were then compared using z-tests to the corresponding AUC values derived from using the CBCL-AP values to see if there were any significant differences.

Results

Table 1 presents prevalence of CSR- and GIR-based ADIS-C/P/DSM-IV diagnoses. CBCL-AP scale T-scores were calculated (M = 60.96, SD = 10.77) and used for ROC analyses. Results of the CBCL-AP ROC analyses (see Table 2) predicting diagnosis based on diagnostician CSR revealed AUC values of 0.73 for SAD (p < 0.001), 0.64 for GAD (p < 0.001), 0.57 for SPPH (p < 0.05), and 0.74 for SAD/GAD/SPPH (p < 0.001). CBCL-AP scores demonstrated fair prediction for identifying SAD and SAD/GAD/SPPH diagnosis based on diagnostician CSR ratings. However, the CBCL-AP demonstrated poor prediction for GAD and SPPH individually.

Table 1.

DSM-IV diagnoses based on ADIS-P CSR and GIR.

DSM-IV diagnoses	ADIS-P-CBCL comparison sample CSR ≥ 4 (n, %)	ADIS-P-CBCL comparison sample GIR ≥ 4 (n, %)
SAD	77 (25.8)	68 (22.8)
GAD	198 (66.4)	157 (52.7)
SPPH	106 (35.6)	86 (28.9)
SAD/GAD/SPPH	252 (84.6)	202 (67.8)

Note: SAD = separation anxiety disorder; GAD = generalized anxiety disorder; SPPH = specific phobia

Table 2.

ROC analyses with CBCL-AP and CBCL-A/D scales as predictors and ADIS-P clinician CSR and parent GIR as dependent variables.

DSM-IV diagnosis	CBCL-AP			CBCL-A/D			AUC Difference
Based on clinician CSR	AUC	S.E.	p	AUC	S.E.	p	CBCL-AP vs A/D
SAD	0.73	0.03	0.00	0.67	0.04	0.00	z = 2.27, p = 0.02
GAD	0.64	0.04	0.00	0.69	0.03	0.00	z = 2.03, p = 0.04
SPPH	0.57	0.03	0.04	0.52	0.04	0.68	z = 2.25, p = 0.02
SAD/GAD/SPPH	0.74	0.04	0.00	0.72	0.04	0.00	z = .78, p = 0.44
Based on parent GIR	AUC	S.E.	p	AUC	S.E.	p
SAD	0.72	0.03	0.00	0.66	0.03	0.00	z = 2.02, p = 0.04
GAD	0.66	0.04	0.00	0.71	0.03	0.00	z = .2.21, p = 0.03
SPPH	0.58	0.03	0.03	0.52	0.04	0.66	z = 2.31, p = 0.02
SAD/GAD/SPPH	0.77	0.04	0.00	0.75	0.04	0.00	z = 0.99, p = 0.32

Note: SAD = separation anxiety disorder; GAD = generalized anxiety disorder; SPPH = specific phobia

Results of the CBCL-AP ROC analyses predicting diagnosis based on Parent GIR revealed AUC values of 0.72 for SAD (p < 0.05), 0.66 for GAD (p < 0.05), 0.58 for SPPH (p < 0.05), and 0.77 for SAD/GAD/SPPH (p < 0.05). Consistent with ROC analyses predicting diagnostician CSR ratings, CBCL-AP scores were fair predictors of SAD and SAD/GAD/SPPH diagnoses, but poor predictors of GAD and SPPH diagnoses individually based on parent GIR.

We hypothesized that the agreement between the CBCL-AP and DSM diagnoses would be higher when GIRs are used as the criterion as compared to using CSRs. However, for the total sample (n = 298), there were no significant differences between the diagnostician CSR AUC values and the parent GIR AUC values (all p’s > 0.05) for all four CBCL-AP diagnostic groups (SAD, GAD, SPPH, SAD/GAD/SPPH). This same analysis was run for both the <12 group (n = 186) and the >=12 group (n = 112) to examine age and the differences between CSR AUC and GIR AUC values and we found no significant differences for either group. Further, for all four diagnostic groups and examining both CSRs and GIRs, we found no significant differences between the <12 AUC values and the >=12 AUC values.

When comparing the diagnostician CSR diagnostic groups to one another, there was a significant difference between the SAD AUC value and the SPPH AUC value (z = 3.37, p = 0.00), and between the SAD/GAD/SPPH AUC value and the SPPH AUC value (z = 3.13, p = 0.00). All other comparisons were not significant (all p’s> 0.05)

CBCL-A/D scores were significantly sensitive to the extent that they could identify a SAD, GAD, and SAD/GAD/SPPH diagnosis based on both diagnostician CSR and parent GIR (see Table 2). CBCL-A/D scores, however, were poor predictors of SAD and GAD based on clinician CSR, and fair predictors of SAD/GAD/SPPH and GAD based on parent GIR. When examining prediction of anxiety disorder diagnosis based on CSR ratings, the CBCL-AP subscale was a significantly better predictor of SAD (z = 2.27, p = 0.02) and SPPH (z = 2.25, p = 0.02), while the CBCL-A/D subscale was a significantly better predictor of GAD than the CBCL-AP (z = 2.03, p = 0.04). No significant differences were observed between the CBCL-AP and CBCL-A/D in predicting SAD/GAD/SPPH diagnoses (p = n.s.).

When examining prediction of anxiety disorder diagnosis based on parent GIR ratings, the CBCL-AP subscale was a significantly better predictor than the CBCL-A/D of SAD (z = 2.02, p = 0.04) and SPPH (z = 2.31, p = 0.02), while the CBCL-A/D subscale was a significantly better predictor of GAD than the CBCL-AP (z = 2.21, p = 0.03). No differences were observed between the CBCL-AP and CBCL-A/D in predicting SAD/GAD/SPPH diagnoses (p = n.s.).

Results demonstrated that, in addition to the CBCL-AP (AUC = 0.73, p = 0.00) and CBCL-A/D (AUC = 0.67, p = 0.00) subscales, the following subscales were significant predictors of SAD diagnosis based on CSRs: Aggressive Behavior (AUC = 0.61, p = 0.01), Somatic Complaints (AUC = 0.68, p = 0.00), Social Problems (AUC = 0.60, p = 0.01), Thought Problems (AUC = 0.64, p = 0.00), Affective Problems (AUC = 0.63, p = 0.00), Somatic Problems (AUC = 0.70, p = 0.00), and Oppositional Defiant Problems (AUC = 0.60, p = 0.01). Although these subscales were predictors of SAD, the CBCL-AP was the strongest of all subscales in predicting SAD diagnosis, significantly better than all subscales except for the Somatic Complaints and Somatic Problems subscales. The CBCL-A/D subscale was a better predictor of SAD than all other subscales except for the CBCL-AP, Somatic Complaints, and the Somatic Problems subscales. The CBCL-A/D subscale was not a significantly better predictor compared to any of the other subscales that significantly predicted SAD diagnosis.

For GAD diagnosis based on CSRs, results demonstrated that in addition to the CBCL-AP (AUC = 0.64, p = 0.04) and CBCL-A/D (AUC = 0.69, p = 0.03) subscales, the following subscales also were significant predictors of GAD diagnosis: Aggressive Behavior (AUC = 0.60, p = 0.01), Social Problems (AUC = 0.58, p = 0.04), Thought Problems (AUC = 0.62, p = 0.00), Affective Problems (AUC = 0.60, p = 0.01), and Oppositional Defiant Problems (AUC = 0.58, p = 0.03). The CBCL-A/D subscale was the strongest predictor of GAD diagnosis, significantly better than all subscales except for Thought Problems. The CBCL-AP subscale was the second strongest predictor of GAD diagnosis, though it was not a significantly better predictor than any of the other subscales that significantly predicted GAD diagnosis.

The present results indicated that, in addition to the CBCL-AP (AUC = 0.74, p = 0.00) and CBCL-A/D (AUC = 0.72, p = 0.00) subscales, the following subscales were significant predictors of SAD/GAD/SPPH diagnosis based on CSRs: Aggressive Behavior (AUC = 0.65, p = 0.00), Somatic Complaints (AUC = 0.61, p = 0.02), Social Problems (AUC = 0.61, p = 0.02), Thought Problems (AUC = 0.66, p = 0.00), Affective Problems (AUC = 0.60, p = 0.04), and Oppositional Defiant Problems (AUC = 0.62, p = 0.02). Based on AUC values, the CBCL-AP was the strongest predictor of SAD/GAD/SPPH diagnosis and was a significantly better predictor than all subscales except for the CBCL-A/D and Thought Problems subscales. Based on AUC values, the CBCL-A/D subscale was the second strongest predictor of SAD/GAD/SPPH diagnosis and was significantly better predictor than all subscales except for the CBCL-AP and Thought Problems subscales.

Discussion

The ease of administration, minimal time burden, and availability of the CBCL make it a useful measure for the assessment of anxiety in youth. The present study found that the CBCL-AP was a significant predictor of the presence or absence of the DSM-IV anxiety disorders (SAD, GAD, SPPH) that it is intended to target. Given this finding, when the time or resources for a diagnostic interview are lacking, the CBCL-AP can serve as a useful instrument for screening youth for anxiety and identifying youth in need of mental health services for anxiety. However, given that the CBCL-AP was not a good predictor of specific anxiety disorder diagnoses other than SAD, it would not be the only data considered when making an anxiety disorder diagnosis.

Using both CSR and GIR of 4 or greater from the ADIS-C/P as dependent variables, CBCL-AP scores were significantly sensitive to the extent that they could identify all four diagnostic groups (SAD, GAD, SPPH, and SAD/GAD/SPPH). AUC values were highest for the SAD/GAD/SPPH diagnostic group, with AUC values for the SAD group second highest of the four groups examined. AUC values were likely highest for the SAD/GAD/SPPH group because this group is the broadest/most inclusive. In other words, an AUC value should be higher for the SAD/GAD/SPPH group than for any single diagnostic category because there are additional chances for the CBCL-AP to predict a positive diagnosis. For the single disorder diagnostic categories, SAD may have had the largest AUC values because several of the CBCL-AP subscales items (e.g., “clings to adults or too dependent” and “fears going to school”) map closely onto the clinical presentation of youth with separation anxiety disorder. Further, symptoms of SAD are much more observable than the more internalized symptoms typically associated with GAD, perhaps making SAD a more salient concern for parents. Contrary to study hypotheses, across all four diagnostic groups, there was no significant difference between the AUC values when using CSR as compared to GIR for disorder diagnosis criteria.

Although we did find a significant difference between the ability of the CBCL-AP and the CBCL-A/D to predict SAD, GAD, and SPPH diagnosis, it is important to keep in mind that the CBCL-A/D subscale includes five of the six items that comprise the CBCL-AP. The only item the CBCL-AP has that the CBCL-A/D does not is “clings to adults or too dependent,” the absence of which may explain why the magnitude of the difference in AUC values when using the CBCL-AP versus the CBCL-A/D was the largest for SAD as compared to the three other diagnostic groups.

The results of our CBCL-AP ROC analyses were generally in line with other studies that have examined the ability of the CBCL-AP to predict anxiety disorder diagnosis in youth. In a study using the Multiaxial Classification of Child and Adolescent Psychiatric Disorders (MC/ICD-10; WHO, 2008) for diagnostic criteria, Pauschardt et al. (2010) demonstrated SAD/GAD/SPHH AUC values of 0.70 (p = 0.001) for both a sample of youth psychiatric outpatients and inpatients. In a study including both the Youth Self-Report (Achenbach, 1987) and the CBCL, Ferdinand (2008) examined the ability of the CBCL-AP scale to predict anxiety disorders (SAD, GAD, SPPH, or any of the three) using both parent rated impairment (GIR) and CSR as predictors. With the exception of SPPH, all the AUC values in our analysis were greater than the AUC values found by Ferdinand though not by much. However, unlike Ferdinand’s study, the AUC values in our analysis for SAD and SAD/GAD/SPHH were sufficiently elevated to be considered “fair” predictors for anxiety disorder diagnosis. As mentioned earlier, Ferdinand’s AUC value of 0.70 (p = 0.001) for SPPH was higher than our SPPH AUC value of 0.57 (p< 0.05). It was somewhat surprising to see our relatively low AUC values for the SPPH diagnostic group. As one of the CBCL-AP items is “fears certain animals, situations, or places other than school” we might expect a higher level of correspondence to a DSM-IV specific phobia diagnosis, especially considering a relative lack of ambiguity regarding the underlying concept that particular CBCL-AP items appears intended to capture. However, only one of the items on the CBCL-AP specifically targets SPPH. A youth with SPPH may not present as worried or tense, may not fear going to school, and may not cling to parents unless around a feared stimulus, leading to lower CBCL-AP scores despite having clinically significant anxiety.

Interestingly, in a study comparing the predictive abilities of the CBCL DSM-Oriented subscales to the CBCL Syndrome Scales, Ebesutani et al. (2010) found AUC values well above 0.80 for all four diagnostic categories (i.e. SAD, GAD, SPPH, and SAD/GAD/SPPH), with the CBCL-A/D demonstrating an AUC value of 0.80 (p 0.001). In the Ebesutani et al. (2010) study, the Children’s Interview for Psychiatric Syndromes, Parent Version (P-Chlps; Weller et al., 1999) was used to determine diagnostic criteria as opposed to the ADIS-C/P for diagnostic criteria used in the current study, preventing the possibility of a direct comparison.

There are study limitations. First, method variance could have influenced AUC values, as parents verbally reported on their child in an interview and responded to the CBCL questionnaire. Second, the sample was largely Caucasian, fairly well-educated and treatment seeking, and conclusions may not generalize beyond such participants. Finally, although the CBCL-AP scale, at six items, is not extensive and does not capture all components of anxiety, it appears reasonable as a brief measure of broader anxiety symptomology.

Despite these limitations, the present findings are valuable by indicating the sensitivity and specificity of the CBCL-AP for identifying anxiety disorder diagnoses. Taken together, these findings indicate that the CBCL-AP represents a quick, relatively low-burden screening instrument for anxiety disorder diagnoses that has a fair ability to predict clinically interfering anxiety. Given the relatively little time required to complete the CBCL-AP, it could serve as an efficient screening measure for primary care offices or schools where short, accurate assessments are needed.