The Revised Child Anxiety and Depression Scales (RCADS): Psychometric Evaluation in Children Evaluated for ADHD

Abstract

Co-occurring internalizing symptoms are common and important to assess in children with attention-deficit/hyperactivity disorder (ADHD). One frequently used child self-report measure of internalizing symptoms is the Revised Child Anxiety and Depression Scales (RCADS), yet the psychometric properties of the RCADS remain unexamined in children referred for ADHD specifically. The present study evaluated the RCADS in 117 children (ages 8–12; 66% male) evaluated for suspected ADHD at an ADHD specialty clinic (83% met criteria for ADHD). In addition to the RCADS, children completed measures of social anxiety and depression. Parents completed the RCADS-Parent Version (RCADS-P) in addition to other measures of internalizing and externalizing symptoms. Children and parents both completed a measure of aggression. Factor structure, reliability, and convergent/discriminant validity of the RCADS were examined. Results supported the six-factor structure of the child-report RCADS (Separation Anxiety, Social Phobia, Generalized Anxiety Disorder, Panic Disorder, and Obsessive Compulsive Disorder, Major Depressive Disorder). The RCADS demonstrated adequate reliability as well as convergent and discriminant validity with other child ratings. The total anxiety score on the RCADS also demonstrated convergent and discriminant validity with parent measures, though the depression score on the RCADS did not. Findings provide preliminary psychometric support for the RCADS in children referred for ADHD.

Children with attention-deficit hyperactivity disorder (ADHD) frequently experience comorbid internalizing symptoms. Although rates of internalizing comorbidities vary across studies, due in part to differences in diagnostic assessment procedures (e.g., structured vs. semi-structured interviews; Cohen, O’Connor, Lewis, Velez, & Malachowski, 1987; Hodges, 1993), it is clear that children with ADHD are more likely than their typically developing peers to have an anxiety and/or depression diagnosis (Angold, Costello, & Erkanli, 1999; Pliszka, 2015). It is likely that far more have subthreshold internalizing symptoms that will reach clinically diagnostic levels later in development (Shankman et al., 2009). Not only are comorbid anxiety and depressive symptoms common among children with ADHD, but they are also associated with more functional impairment and clinical complexity. The co-occurrence of ADHD and internalizing disorders is associated with more sleep difficulties, greater academic problems, higher rates of suicidality, and worse family and peer functioning compared to children and adolescents with ADHD alone (Becker, Luebbe, & Langberg, 2012; Daviss, 2008; Hvolby, 2015; Schatz & Rostain, 2006). Furthermore, internalizing problems in children with ADHD are associated with higher treatment costs (Guevara, Lozano, Wickizer, Mell, & Gephart, 2001) and poorer medication response (Al Ghriwati et al., 2017).

The high rates of psychiatric comorbidity among children with ADHD and their impact on functional outcomes and treatment makes the “need to assess accurately comorbid conditions a constant clinical reality” (Hunsley & Mash, 2007, p. 40). Assessing for internalizing symptoms among children with ADHD is recommended by the American Academy of Pediatrics (2011) and the American Academy of Child and Adolescent Psychiatry (2007). However, few measures assessing internalizing symptoms have been validated in children with suspected or confirmed ADHD. Reliable and valid measures of anxiety and depression that can be used in research as well as in assessment and intervention are critical.

A multi-informant approach is optimal when assessing for anxiety and depression in children (Klein, Dougherty, & Olino, 2005; Silverman & Ollendick, 2005). Although parents can often provide timelines and descriptions of children’s observable internalizing symptoms, they may be less aware of children’s symptoms that are not as easily observed (e.g., feelings of worthlessness, excessive worry). For this reason, children themselves are also recognized as valid informants of their internalizing symptoms (Becker, Jensen-Doss, Kendall, Birmaher, & Ginsburg, 2016; DiBartolo & Grills, 2006). Gathering child-report information may be especially important among children with suspected ADHD, given the overlap between symptoms of ADHD and internalizing disorders including restlessness, irritability, and difficulty concentrating, which can make accurate diagnosis difficult. That is, overlap in symptom presentation can make assessment more challenging when ADHD and an internalizing disorder are both suspected, or may inflate rates of comorbidity (Diler et al., 2007; Jarrett & Ollendick, 2008; Tannock, 2000). Thus, gathering internalizing information from multiple informants, including children themselves, may be crucial when assessing ADHD.

There are a number of validated child self-report measures for assessing anxiety and depression in children. Many of these, however, assess only anxiety or depression and often must be purchased, which may limit their use in community-based practice settings or large-scale research studies. One free (though copyrighted) measure that includes both anxiety and depression in a single scale is the Revised Child Anxiety and Depression Scales (RCADS; Chorpita, Yim, Moffitt, Umemoto, & Francis, 2000). The RCADS, which consists of 47 items rated on a four-point scale, is designed to align with Diagnostic and Statistical Manual of Mental Disorders (DSM) nosology.¹ The RCADS yields six scores: five anxiety scales (Separation Anxiety, Social Phobia, Generalized Anxiety Disorder, Panic Disorder, and Obsessive Compulsive Disorder) and one depression scale (Major Depressive Disorder). Furthermore, total anxiety and total internalizing (anxiety and depression) scales are generated. An added benefit of the RCADS is that it has been translated into multiple languages, such as Dutch, Greek, Korean, Spanish, Swedish, and Turkish, which allows for it to be used across a diverse range of populations. These and other translations, as well as scoring tools, can be readily found online (http://www.childfirst.ucla.edu/Resources.html).

To date, the self-report RCADS has been validated in a range of populations and settings, including school (Bouvard, Denis, & Roulin, 2015; de Ross, Gullone, & Chorpita, 2002) and clinic (Chorpita, Moffitt, & Gray, 2005; Gormez et al., 2017) samples spanning in age from 8 to 19 years old. These studies have supported the RCADS six-factor structure, as well as its convergent and discriminant validity (Bouvard et al., 2015; Chorpita et al., 2005; Chorpita et al., 2000; de Ross et al., 2002; Esbjørn, Sømhovd, Turnstedt, & Reinholdt-Dunne, 2012; Gormez et al., 2017; Mathyssek et al., 2013; Sterling et al., 2015). These studies provide important psychometric support for the RCADS in children and adolescents. Furthermore, among several measures included in a review of child self-report measures for clinical use, it was concluded that the RCADS may be the most sensitive to treatment change given its level of specificity in anxiety dimensions (Wolpert, Cheng, & Deighton, 2015).

Although the RCADS has been identified as a valuable tool for evaluating internalizing symptoms in a number of samples and populations, the child-report RCADS has never been examined in children referred for ADHD specifically. This is important since assessment tools should have psychometric evidence in the clinical samples in which they are being used, such that “supporting psychometric evidence must be available for each purpose for which an instrument or assessment strategy is used” (Hunsley & Mash, 2007, p. 33). Surprisingly few studies have examined the psychometric properties of child self-report internalizing measures outside of community samples or clinical samples of children with internalizing problems (Silverman & Ollendick, 2005). We are aware of only one study that has examined the psychometric properties of internalizing measures in children with ADHD. In the Multimodal Treatment of ADHD (MTA) sample of 579 children (ages 7–10 years) with ADHD, March et al. (1999) found support for the reliability and validity of self-ratings on the Multidimensional Anxiety Scale for Children (MASC). The RCADS, which includes both anxiety and depressive symptoms, remains unexamined in a sample of children clinically-referred for ADHD.

It is especially important to examine the psychometric properties and validity of internalizing measures in children referred for ADHD given ongoing concerns about possible biases in self-report ratings in children with ADHD. That is, there is evidence that children with ADHD may provide positively biased self-reports of their social, behavioral, and academic functioning (Emeh, Mikami, & Teachman, 2018; Hoza et al., 2004; Owens, Goldfine, Evangelista, Hoza, & Kaiser, 2007), though recent studies suggest this phenomenon may not be as prevalent or pronounced as previously thought (Bourchtein, Langberg, Owens, Evans, & Perera, 2017; Jiang & Johnston, 2017). Of note, although internalizing symptoms may attenuate positive bias in competencies in youth with ADHD (Swanson, Owens, & Hinshaw, 2012), it is unknown whether any bias exists in self-ratings of internalizing symptoms (as opposed to self-ratings of competence). If child self-report of internalizing symptoms demonstrates cross-informant convergent validity (e.g., child-rated anxiety correlates with parent-rated anxiety), then this would provide some evidence for the validity of self-report ratings of internalizing in children with ADHD. Further, a study by Tannock (2000) demonstrated that children with ADHD who endorsed anxiety showed lower levels of self-confidence and greater impairment in daily activities than children with ADHD whose parents reported anxiety in their child but the child did not self-report, suggesting children may be more valid informants than parents in this domain. Given the unique challenges of assessing for internalizing symptoms among a sample of children with ADHD, as well as possible links to positive biases in competency ratings, it is important to evaluate the reliability and factor structure of internalizing measures in children evaluated for ADHD.

Accordingly, the goal of the present study was to conduct a preliminary investigation of the psychometric properties of the RCADS in a sample of children who were referred to a specialty clinic for possible ADHD, thus increasing generalizability to children who are evaluated for suspected ADHD (i.e., although all children in the present study were referred for possible ADHD, not all children received an ADHD diagnosis). We examined the factor structure and reliability of the RCADS, as well as the convergent and discriminant validity of the RCADS with both child- and parent-report measures. Based on previous research (Bouvard et al., 2015; Chorpita et al., 2005; de Ross et al., 2002; Gormez et al., 2017), we hypothesized that the six-factor structure of the RCADS would best fit the data. We also hypothesized that the RCADS would have good reliability, convergent validity, and discriminant validity with child and parent measures of anxiety, depression, and externalizing behaviors.

Methods

Participants

Participants were 117 children ages 8–12 years recruited for an ongoing study through the standard clinical intake flow at an outpatient clinic specializing in pediatric ADHD. All participants were being evaluated in the clinic for possible ADHD, and 83% (n = 97) met full criteria for ADHD. All children had an IQ ≥ 70 (Range = 71–129) based on the Kaufman Brief Intelligence Scale, Second Edition (KBIT-2; Kaufman & Kaufman, 2004). Sample characteristics, including ADHD and comorbid diagnoses based on the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS; Kaufman et al., 1997) interview conducted with the child’s caregiver, are provided in Table 1. Among children with ADHD, 8 (8.2%) met criteria for an internalizing disorder diagnosis; among children without ADHD, 1 (5.0%) met criteria for an internalizing diagnosis. The majority of children (n = 103; 88.0%) were not on psychotropic medications at the time of their evaluation.

Table 1.

Sample Characteristics (N = 117)

	M ± SD
Age	9.38 ± 1.36
Estimated IQa	102.21 ± 14.28
	N (%)
Sex
Male	77 (65.8%)
Female	40 (34.2%)
Race/Ethnicity
White	90 (76.9%)
Black	21 (17.9%)
Hispanic	3 (2.6%)
Asian	2 (1.7%)
Native American	1 (0.9%)
ADHD Diagnosisb	97 (82.9%)
Combined Type	45 (46.4%)
Inattentive Type	51 (52.6%)
Hyperactive-Impulsive Type	1 (1.0%)
Comorbid Internalizing Diagnosesb
Depression/Dysthymia	2 (1.7%)
Generalized Anxiety Disorder	3 (2.6%)
Separation Anxiety Disorder	1 (0.9%)
Social Phobia	4 (3.4%)
Panic Disorder	1 (0.9%)
Obsessive-Compulsive Disorder	0 (0%)
Specific Phobia	0 (0%)
Post-traumatic Stress Disorder	1 (0.9%)
Any Anxiety Disorder	9 (7.7%)
Any Internalizing Disorder	9 (7.7%)
Comorbid Externalizing Diagnosesb
Oppositional Defiant Disorder	37 (31.6%)
Conduct Disorder	1 (0.9%)

Note. ADHD = attention-deficit/hyperactivity disorder.

^aEstimated intelligence quotient (IQ) determined using the Kaufman Brief Intelligence Scale, Second Edition (KBIT-2).

^bDiagnoses established using the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS) administered to parents.

Procedures

This study was reviewed and approved by the Institutional Review Board. Families were recruited through the standard clinical intake flow at an outpatient clinic specializing in the diagnosis and treatment of ADHD. Parents provided informed consent and children provided assent.

Measures

Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS).

The K-SADS (Kaufman et al., 1997) is a semi-structured diagnostic interview with good reliability and validity. The disruptive behavior disorder (including ADHD), mood disorder, and anxiety disorder modules were administered to a caregiver of all participants in the present study. If any K-SADS screening item is endorsed as positive, a full module is administered which covers the DSM symptoms of the disorder in question. Included as part of the interview are questions regarding psychosocial functioning, impairment and age of onset, and rule out questions related to medical and other causes, including other mental health disorders. The K-SADS was administered by individuals with Master’s or doctoral degrees in clinical psychology. All interviewers were trained by experienced interviewers, which included a didactic training focused on DSM nosology and differential diagnosis, scoring a previously recorded interview, observing interviews, and being observed before interviewing independently. In addition, one interview per interviewer was randomly selected to be scored by another interviewer. We achieved 100% reliability between interviewers on this reliability check.

Kaufman Brief Intelligence Scale, Second Edition (KBIT-2).

The KBIT-2 (Kaufman & Kaufman, 2004) is a brief culturally-sensitive standardized assessment measure comprised of Verbal and Nonverbal scales, which together yield a full scale estimate of IQ. Both scales have good test-retest reliability, and internal consistency reliabilities in the ages of interest all exceed .90 (Kaufman & Kaufman, 2004).

Revised Child Anxiety and Depression Scales (RCADS) and RCADS-Parent Version (RCADS-P).

As described above, the RCADS (Chorpita et al., 2000) is a 47-item child self-report measure that assesses anxiety and depression disorder symptoms on a four-point scale (0 = never, 3 = always). In addition to a depression scale (10 items; e.g., “I feel sad or empty”), the RCADS has five anxiety scales: separation anxiety (7 items; “I worry about being away from my parents”), generalized anxiety (6 items; e.g., “I worry about things”), panic disorder (9 items; e.g., “All of a sudden I feel really scared for no reason at all”), social phobia (9 items; e.g., “I worry what other people think of me”), and obsessive-compulsive (6 items; e.g., “I have to do some things just the right way to stop bad things from happening”). The RCADS has good psychometric properties and has demonstrated excellent reliability and validity in clinical and nonclinical samples (Chorpita et al., 2005; Chorpita et al., 2000; Esbjørn et al., 2012; Gormez et al., 2017; Mathyssek et al., 2013). The reliability and convergent/discriminant validity of the RCADS in this sample is examined in the Results section below.

The RCADS-P (Ebesutani, Bernstein, Nakamura, Chorpita, Weisz, et al., 2010; Ebesutani et al., 2011) is a parent-report parallel to the child self-report RCADS. The RCADS-P has the same 47 items, response scale, and factor structure as the RCADS. The RCADS-P has also demonstrated excellent reliability and validity in clinical and nonclinical samples (Ebesutani, Bernstein, Nakamura, Chorpita, Weisz, et al., 2010; Ebesutani et al., 2011). In addition, the RCADS-P has been validated in ADHD-referred children specifically (Becker et al., 2017). Mean scale scores were used in the present study (separation anxiety α = .81, generalized anxiety α = .82, social phobia α = .89, panic disorder α = .76, obsessive-compulsive α = .68, depression α = .71, total anxiety α = .91, total internalizing α = .92).

Convergent Validity.

Two child self-report measures and one parent-report measure were used to examine convergent validity.

Children’s Depression Inventory (CDI).

The CDI (Kovacs, 1992) is a 27-item self-report measure of children’s depressive symptoms. Each item has three response options (scored 0 to 2), with some items reverse-scored. There is substantial support for the reliability and validity of the CDI (Huang & Dong, 2014; Kovacs, 1992; Sun & Wang, 2015). Previous studies have found a strong correlation (rs = 0.57 to 0.80) between the CDI and the RCADS depression subscale (Chorpita et al., 2005; Chorpita et al., 2000; de Ross et al., 2002; Sandin, Chorot, Valiente, & Chorpita, 2010; Weems, Zakem, Costa, Cannon, & Watts, 2005). In the present study, a total mean scale score was used (α = .88).

Social Anxiety Scale for Children – Revised (SASC-R).

The SASC-R (La Greca & Stone, 1993) is a 22-item child self-report measure of social anxiety (4 of the 22 items are filler items that are not scored). Each item is rated on a five-point scale (1 = not at all, 5 = all the time). There is substantial evidence supporting the reliability and validity of the SASC-R (Ginsburg, La Greca, & Silverman, 1998; La Greca & Stone, 1993; Reijntjes, Dekovic, & Telch, 2007). Although the SASC-R is specific to social anxiety, studies have shown that scores on the SASC-R are strongly correlated with other, broader measures of anxiety in children (Inderbitzen & Hope, 1995; Kearney, 2007; La Greca, 1999), including the RCADS anxiety subscales and total internalizing score (Sandin et al., 2010). As recommended, a total sum score was used (α = .68).

Child Behavior Checklist (CBCL).

The CBCL is a caregiver-report measure of children’s emotional and behavioral problems (Achenbach & Rescorla, 2001). Items are rated on a three-point scale (0 = not true, 1 = somewhat or sometimes true, 2 = very true or often true). In the present study, the DSM-oriented anxiety (6 items) and depression (i.e., Affective Problems; 13 items) scales were used. The DSM-oriented scales have demonstrated good internal consistency as well as convergent and discriminant validity with other parent- and self-report symptom scales and with DSM diagnoses as determined by clinical interviews (Achenbach & Rescorla, 2001; Ebesutani, Bernstein, Nakamura, Chorpita, Higa-McMillan, et al., 2010; Nakamura, Ebesutani, Bernstein, & Chorpita, 2009).

Discriminant Validity.

One child self-report measure and two parent-report measures were used to examine discriminant validity.

Aggression.

Both children and parents completed the Dodge and Coie (1987) aggression scale, which consists of six items of proactive and reactive aggression rated on a five-point scale (1 = never, 5 = almost always). This measure has construct and criterion validity for use as both a parent-report and child self-report measure (Dodge, Lochman, Harnish, Bates, & Pettit, 1997; Fite et al., 2011; Waschbusch, Willoughby, & Pelham, 1998). In the present study, total mean scale scores were calculated (child-rated aggression α = .68, parent-rated aggression α = .82).

Vanderbilt ADHD Diagnostic Parent Rating Scale (VADPRS).

The VADPRS (Wolraich et al., 2003) was used to assess children’s ADHD and oppositional defiant disorder (ODD) symptoms. The VADPRS includes all 18 DSM ADHD symptoms and all 8 DSM ODD symptoms. Each item is rated on a four-point scale (0 = never, 3 = very often). In the present study, mean scale scores were calculated and internal consistencies were excellent (ADHD α = .89, ODD α = .92).

Analytic Approach

Missing data.

The CBCL was available for 112 participants in the current study. There were no differences in demographic characteristics, RCADS, or RCADS-P scores between children with and without CBCL data (all ps > .10).

Factor structure.

Confirmatory factor analyses (CFAs) were calculated using Mplus v.7.0 (Muthén & Muthén, 1998–2017). Four competing models were calculated in line with Ebesutani et al. (2011): the original six-factor model, a five-factor model in which GAD and depression items were combined to form a general factor (see Lahey et al., 2008, for rationale for this approach), a two-factor depression and anxiety model (with all five anxiety disorder subscales on a single factor), and a one-factor model thought to reflect general negative affect. We considered the Likert-scale items as ordered categorical data and therefore used the weighted least squares with mean and variance adjustment (WLSMV) estimator. Missing data were handled via pairwise correlation with available data. Model fit was assessed with multiple indices, with the following indicating acceptable fit: comparative fit index (CFI) >0.90, and root mean square error of approximation (RMSEA) <0.08 (Hu & Bentler, 1999; Yu, 2002). Simulations testing performance of these indices of fit have assumed continuous data, rather than ordered categorical data. For models with ordered categorical indicators and weighted least squares estimation, the weighted root square residual (WRMR) has also been proposed, with values <1.00 considered adequate and values <0.90 representing excellent fit (Yu, 2002). To compare models, chi-square difference tests were conducted using the DIFFTEST option in Mplus (see Asparouhov & Muthén, 2006 for technical details regarding DIFFTEST).

Reliability.

As in previous RCADS validation studies (Chorpita et al., 2005; Chorpita et al., 2000; Esbjørn et al., 2012; Gormez et al., 2017; Mathyssek et al., 2013), we assessed the reliability of the RCADS scores using Cronbach alpha coefficients, omega reliability (which, unlike alpha does not assume tau-equivalence and may provide a more realistic estimate of reliability in a given sample, especially for models that are multidimensional; see Dunn, Baguley, & Brunsden, 2014), item-total correlations, and alpha-if-item-deleted values. In behavioral research, coefficients ≥0.70 are generally considered acceptable (Nunnally, 1978; Schmitt, 1996).

Convergent and discriminant validity.

Convergent and discriminant validity of the RCADS were examined using both child- and parent-report measures. Correlations were conducted to evaluate whether RCADS scores were positively associated in expected ways with internalizing symptoms as well as less positively associated with externalizing behaviors. A correlation of 0.10 is considered a small effect, a correlation of 0.30 is considered a medium effect, and a correlation of 0.50 is considered a large effect (Cohen, Cohen, West, & Aiken, 2003). In addition, Steiger’s z-tests for dependent correlations were conducted to examine whether RCADS scores were significantly more strongly associated with the convergent validity variables than the discriminant validity variables. Although correlations were conducted for all of the RCADS subscales, the depression, total anxiety, and total internalizing scores on the RCADS were the focus of the z-test analyses.

Comparing children with and without elevated RCADS scores.

We examined whether children with clinically at-risk or elevated RCADS scores (i.e., T-scores ≥65) differed from children with non-elevated RCADS scores in self- and parent-reported functioning on the internalizing and externalizing measures. Finally, we examined whether children with elevated RCADS scores were more likely than children with non-elevated RCADS scores to have been diagnosed with an internalizing disorder on the K-SADS parent interview.

Results

Factor Structure

The original six-factor structure of the RCADS was tested against three other models representing various competing conceptualizations of the relations among internalizing symptoms. The six-factor model fit the data adequately and, compared to all other models, the six-factor model best fit the data as demonstrated by the significant chi-square difference tests². Fit statistics for competing models are shown in Table 2. In the six-factor model, all items loaded significantly (ps < .05) on their respective factors: Separation Anxiety (λs = .48 to .92), Social Anxiety (λs = .30 to .83), Generalized Anxiety (λs = .55 to .87), Panic (λs = .32 to .83), Obsessive-Compulsive (λs = .45 to .71), and Depression (λs = .36 to .73). See Table 3 for each item’s factor loading. As summarized in Table 4, factor correlations ranged from .61 to .86³.

Table 2.

Confirmatory Factor Analyses of the Revised Child Anxiety and Depression Scale (RCADS) in Children Referred for ADHD (N = 117)

							Comparing to 6-factor model
Model	χ2	df	p	RMSEA (90%CI)	CFI	WRMR	Δχ2 (df)	p
6-Factora	1193.19	1019	<.001	.04 (.03, .05)	.94	.98	--	--
5-Factor (combine GAD and DEP)	1209.20	1024	<.001	.04 (.03, .05)	.93	1.00	18.93(5)	.002
2-Factor (ANX and DEP)	1255.87	1033	<.001	.04 (.03,.05)	.91	1.06	78.08(14)	<.001
1-Factor (Negative Affect)	1278.29	1034	<.001	.05 (.04,.05)	.90	1.08	91.76(15)	<.001

Note. For chi-square difference testing, the DIFFTEST option in Mplus was used because models were estimated using weighted least squares with mean and variance adjustments. ANX = anxiety. DEP = depression. GAD = generalized anxiety disorder.

^a= best fitting model

Table 3.

Cronbach Alpha, Cronbach Alpha if Item Deleted, Item-Total Correlations, and Factor Loadings of RCADS in Children Evaluated for ADHD

Scale	Alpha/Omega	Item	Alpha if Item Deleted	Item-Total Correlation	Factor Loading
Separation Anxiety (7 items)	0.79/0.87	5	0.77	0.48	0.48
		9	0.76	0.52	0.57
		17	0.77	0.46	0.53
		18	0.78	0.42	0.85
		33	0.76	0.55	0.68
		45	0.74	0.64	0.92
		46	0.75	0.60	0.78
Generalized Anxiety (6 items)	0.79/0.87	1	0.80	0.35	0.56
		13	0.77	0.48	0.55
		22	0.73	0.67	0.81
		27	0.72	0.68	0.87
		35	0.75	0.55	0.82
		37	0.76	0.52	0.67
Panic Disorder (9 items)	0.78/0.87	3	0.79	0.31	0.32
		14	0.78	0.37	0.70
		24	0.75	0.54	0.62
		26	0.76	0.52	0.69
		28	0.74	0.65	0.83
		34	0.76	0.51	0.66
		36	0.76	0.50	0.61
		39	0.77	0.41	0.56
		41	0.76	0.47	0.77
Social Phobia (9 items)	0.76/0.83	4	0.76	0.30	0.30
		7	0.75	0.38	0.40
		8	0.74	0.41	0.50
		12	0.73	0.48	0.72
		20	0.75	0.37	0.66
		30	0.72	0.59	0.63
		32	0.72	0.54	0.73
		38	0.75	0.35	0.55
		43	0.71	0.58	0.83
Obsessive-Compulsive (6 items)	0.65/0.75	10	0.61	0.36	0.61
		16	0.64	0.29	0.45
		23	0.55	0.52	0.71
		31	0.60	0.38	0.60
		42	0.62	0.34	0.48
		44	0.61	0.38	0.62
Depression (10 items)	0.72/0.81	2	0.71	0.32	0.48
		6	0.72	0.25	0.36
		11	0.70	0.43	0.51
		15	0.70	0.40	0.60
		19	0.68	0.55	0.57
		21	0.69	0.45	0.54
		25	0.70	0.44	0.68
		29	0.70	0.43	0.73
		40	0.73	0.19	0.41
		47	0.70	0.43	0.54
Total Anxiety (37 items)	0.92	--	--	--	--
Total Internalizing (47 items)	0.92/0.97	--	--	--	--

Table 4.

Intercorrelations and Descriptive Statistics of the RCADS in Children Evaluated for ADHD

RCADS Subscale	1	2	3	4	5	6	Total Anxiety	RCADS Total
1. Separation Anxiety	--						0.80	0.78
2. GAD	0.57 (0.76)	--					0.79	0.80
3. Panic Disorder	0.61 (0.86)	0.48 (0.69)	--				0.80	0.79
4. Social Phobia	0.51 (0.71)	0.56 (0.76)	0.52 (0.72)	--			0.81	0.78
5. OCD	0.49 (0.72)	0.62 (0.78)	0.58 (0.79)	0.59 (0.80)	--		0.80	0.78
6. Depression	0.41 (0.61)	0.55 (0.74)	0.47 (0.68)	0.40 (0.67)	0.43 (0.61)	--	0.56	0.70
Mean Scale Scores
Mean	0.73	0.86	0.53	0.95	0.88	0.78	0.78	0.78
SD	0.64	0.62	0.48	0.53	0.58	0.47	0.45	0.42
Range	0–2.71	0–2.67	0–2.11	0–2.56	0–2.67	0–2.30	0–2.22	0–2.04
T-Scores
Mean	50.41	44.22	48.86	45.51	46.60	49.66	46.36	46.88
SD	11.81	11.04	10.27	10.06	10.42	11.85	11.14	11.17
Range	33–102	29–77	36–83	25–77	29–81	30–97	27–84	27–81
% with T-score ≥ 65	8.5%	6.0%	8.6%	5.1%	7.7%	8.5%	7.7%	8.5%
% with T-score ≥ 70	6.0%	5.1%	6.0%	2.6%	2.6%	3.4%	4.3%	4.3%

Note. N = 117. All correlations were significant at p < .001. Correlations outside parentheses and descriptive statistics are based on raw scores. Correlations based on factor scores are in parentheses. T-scores are based on normative data in the RCADS User’s Guide (Chorpita, Ebesutani, & Spence, 2015). ADHD = attention-deficit/hyperactivity disorder. GAD = generalized anxiety disorder. OCD = obsessive-compulsive disorder. RCADS = Revised Child Anxiety and Depression Scales.

Reliability

Cronbach’s alpha coefficients and omega reliabilities, in addition to alpha-if-item-deleted and item-total correlation values, are reported in Table 3. All of the alpha coefficients were >0.70 with the exception of the obsessive-compulsive scale (α = 0.65). Of note, the internal consistency for the obsessive-compulsive scale would not be improved by removing any single item from this scale. All omega coefficients were ≥0.75. The total anxiety and total internalizing scales demonstrated excellent reliability (>0.90).

Convergent and Discriminant Validity

Convergent and discriminant validity correlations with child self-report and parent-report measures are reported in Table 5.

Table 5.

Convergent and Discriminant Validity of the Revised Child Anxiety and Depression Scales (RCADS)

	Convergent Validity
	Child-Report Measures		Parent-Report Measures
RCADS Scale	Anxiety (SASC-R)	Depression (CDI)	Anxiety (RCADS-P)	Depression (RCADS-P)	Anxiety (CBCL)	Depression (CBCL)
Separation Anxiety	.43***	.23***	.28**	.03	.30**	.06
GAD	.42***	.40***	.04	-.02	.16†	.05
Panic Disorder	.39***	.40***	.20*	.02	.20*	.05
Social Phobia	.57***	.26**	.20*	.03	.23*	.13
OCD	.45***	.33***	.08	−.02	.20*	.01
Depression	.34***	.71***	.17†	.12	.21*	.17†
Total Anxiety	.57***	.40***	.21*	.02	.27**	.13
RCADS Total	.57***	.50***	.22*	.04	.28**	.15

	Discriminant Validity
	Child-Report Measures	Parent-Report Measures
	Aggression (PRAS)	Aggression (PRAS)	ADHD (VADPRS)	ODD (VADPRS)
Separation Anxiety	.25**	−.03	.01	−.05
GAD	.25**	−.03	.01	−.03
Panic Disorder	.38**	.06	.08	.13
Social Phobia	.25**	.06	.08	.03
OCD	.28**	.05	.05	−.01
Depression	.34***	.02	−.03	.07
Total Anxiety	.35***	.03	.06	.02
RCADS Total	.38***	.03	.05	.03

Note. ADHD = attention-deficit/hyperactivity disorder. ANX = anxiety. CBCL = Child Behavior Checklist. CDI = Children’s Depression Inventory. GAD = generalized anxiety disorder. OCD = obsessive-compulsive disorder. ODD = oppositional defiant disorder. PRAS = Proactive Reactive Aggression Scale. RCADS = Revised Child Anxiety and Depression Scales. RCADS-P = Revised Child Anxiety and Depression Scales–Parent Version. SASC-R = Social Anxiety Scale for Children-Revised. VADPRS = Vanderbilt ADHD Diagnostic Parent Rating Scale.

^†p < .10.

^*p < .05.

^**p < .01.

^***p < .001.

Convergent/discriminant validity with child self-report measures.

RCADS ratings were moderately-to-strongly correlated with child-rated internalizing symptoms on both the SASC-R and CDI (rs = 0.23 to 0.71), with all but two of the 16 correlations ≥0.30. The RCADS social phobia subscale was strongly correlated with the SASC-R social anxiety total score (r = 0.57), and the RCADS depression subscale was strongly correlated with the CDI depression total score (r = 0.71). RCADS ratings were significantly moderately correlated with child-rated aggression, though correlations were generally smaller in magnitude (rs = 0.25 to 0.38), with 4 of the 8 correlations <0.30.

Steiger’s z-tests supported the specificity of the RCADS anxiety and depression scales in relation to the SASC-R and CDI. Specifically, RCADS total anxiety was more strongly associated with SASC-R anxiety (r = 0.57) than with CDI depression (r = 0.40; z = 1.96, p = .05) or with aggression (r = 0.35, z = 2.32, p = .02). Likewise, RCADS depression was more strongly associated with CDI depression (r = 0.71) than with SASC-R anxiety (r = 0.34; z = 4.53, p < 0.001) or aggression (r = 0.34, z = 4.74, p < .001).

Convergent/discriminant validity with parent-report measures.

As summarized in Table 5, RCADS ratings were negligibly-to-moderately correlated with parent-rated internalizing symptoms on both the RCADS-P and CBCL (rs = -0.02 to 0.30). Evidence for convergent validity of the RCADS was found in relation to parent-rated anxiety (12 of 16 correlations were statistically significant) but not in relation to parent-rated depression (0 of 16 correlations were statistically significant). Likewise, whereas RCADS total anxiety was significantly correlated with both RCADS-P and CBCL anxiety (rs = 0.21 and 0.27, respectively; ps = .02 and .004, respectively), RCADS depression was not significantly correlated with either RCADS-P or CBCL depression (rs = 0.12 and 0.17, respectively; ps = .18 and .069, respectively). Parent-child correlations for the specific RCADS and RCADS-P anxiety subscales were moderate-to-strong for separation anxiety (r = 0.41, p < .001), modest for panic disorder (r = 0.19, p = .046) and social phobia (r = 0.17, p = .06), and absent for generalized anxiety (r = 0.07, p = .49) and obsessive-compulsive (r = -0.03, p = .77).

The RCADS demonstrated discriminant validity with parent-reported aggression, ADHD symptoms, and ODD symptoms, with all correlations nonsignificant and negligible (rs = -0.05 to 0.13, with only one correlation >0.10; all ps > .05). Steiger’s z-tests were used to examine whether the magnitude of correlations of child-reported internalizing symptoms with parent-reported internalizing was stronger than the magnitude of child-reported internalizing symptoms with parent-reported ADHD/externalizing. These analyses generally indicated that the RCADS total internalizing score was significantly more strongly correlated with parent-rated anxiety than with parent-rated externalizing behaviors. Specifically, RCADS total internalizing was significantly more strongly correlated with CBCL anxiety than with aggression (z = 2.13, p = .03) or ODD symptoms (z = 2.47, p = .01), and RCADS total internalizing was also marginally more strongly correlated with CBCL anxiety than with ADHD symptoms (z = 1.88, p = .06). In addition, RCADS total internalizing was marginally more strongly correlated with RCADS-P anxiety than with aggression (z = 1.79, p = .07) or ODD symptoms (z = 1.70, p = .09).

Comparing Children with and without Elevated RCADS Total Internalizing Scores

As summarized in Table 4, 10 children (8.5%) had at-risk or clinically elevated T-scores (i.e., ≥65) on the RCADS total internalizing scale. Levene’s test for homogeneity of variance was significant for RCADS-P depression and ODD, and so the Welch’s t-statistic is reported below for these variables. Independent samples t-tests indicated that children with elevated RCADS internalizing scores had significantly higher SASC-R, CDI, and RCADS-P anxiety scores compared to children without elevated RCADS internalizing scores (ts = 3.40, 4.11, and 2.68, respectively; all ps < .01) as well as marginally higher CBCL anxiety scores (t = 1.81, p = .07). Children with and without elevated RCADS internalizing scores did not differ in RCADS-P depression or CBCL depression scores (ts = 1.64 and -0.08, respectively; both ps > .10).

Children with elevated RCADS internalizing scores did have higher child self-reported aggression compared to children without elevated RCADS scores (t = 2.19, p = .03). Children with or without elevated RCADS internalizing scores did not differ in parent-reported aggression, ADHD symptoms, or ODD symptoms (ts = -0.17, 0.68, and 1.03, respectively; all ps > .05).

Children with elevated RCADS internalizing scores were not more likely than their peers to have received an internalizing disorder diagnosis on the K-SADS parent interview (χ²(1) = 2.33, p = .13). Only two of the 10 children with elevated RCADS internalizing scores received an internalizing disorder diagnosis on the K-SADS parent interview. Conversely, seven children received an internalizing disorder diagnosis on the K-SADS parent interview but did not have an elevated RCADS score.

Discussion

This is the first study to examine the psychometric properties of the RCADS in children referred for ADHD. We found preliminary support for the six-factor RCADS structure, its reliability, and its convergent and discriminant validity. These findings indicate that it is important for future research to more systematically examine the RCADS as a potentially useful assessment tool in studies examining internalizing symptoms in children with ADHD, as well as in clinical practice where it is important to assess and monitor internalizing symptoms.

Previous studies have found support for the six-factor structure, reliability, and convergent/discriminant of the child-report RCADS in school based samples (Chorpita et al., 2000; de Ross et al., 2002) and general clinical samples (Chorpita et al., 2005; Gormez et al., 2017). Our study extends these findings to children referred for suspected ADHD. Psychometric support for the RCADS in children with ADHD specifically is important since children with ADHD frequently experience co-occurring internalizing symptoms, or in some children internalizing symptoms may be mistaken for ADHD symptoms (e.g., inattention) (AACAP; 2007; AAP, 2011). As such, it is not only important to assess for internalizing symptoms in youth with ADHD (or suspected ADHD), but also to capture the child’s own perspective as part of a multi-informant approach to assessing internalizing symptoms in youth (Connolly, Bernstein, & Work Group on Quality Issues, 2007; Hunsley & Mash, 2007).

Promising support for both convergent and discriminant validity was found when the RCADS was examined in relation to other child self-report measures. Ratings on the RCADS were moderately-to-strongly correlated with other internalizing measures. Moreover, the RCADS total anxiety score was more strongly correlated with SASC-R scores than with CDI or aggression scores, and the RCADS depression score was likewise more strongly correlated with CDI scores than with SASC-R or aggression scores. This demonstrates specificity in the convergent and discriminant validity of the RCADS, at least with other child self-report measures. In addition, in our sample the correlation between the RCADS depression subscale and the CDI was 0.71, which is almost identical to the correlation of 0.70 reported in two separate studies examining associations between the RCADS and CDI (Chorpita et al., 2005; Chorpita et al., 2000). In terms of anxiety, we were only able to examine the RCADS in relation to a measure of social anxiety specifically, and both the RCADS social phobia subscale and total anxiety subscale had a correlation of 0.57 in relation to the SASC-R social anxiety measure. The only other study that included both the RCADS and SASC-R used a shortened, Spanish version of the RCADS in a school-based sample of adolescents in Spain and found a correlation of 0.71 between the RCADS social phobia subscale and SASC-R (Sandin et al., 2010). Although the correlation magnitude was somewhat lower in our sample, multiple methodological and sampling differences between studies make it unclear why this may have been the case. For instance, the Spain study used a shortened, translated RCADS with 30 rather than 47 items, was conducted in a large (N = 544) school/community-based sample rather than our smaller (N = 117) clinical sample, and included youth ages 10–17 years whereas the current study included youth ages 8–12 years. Nevertheless, the RCADS social anxiety subscale as well as total anxiety and internalizing scores were strongly associated with the SASC-R in our sample.

Less clear support for convergent and discriminant validity was found when examining the RCADS in relation to parent-report measures, which is not entirely surprising since low-to-moderate parent-child agreement, particularly for internalizing symptoms, is well-documented (Achenbach, McConaughy, & Howell, 1987; De Los Reyes et al., 2015). First, there was little evidence of convergent validity when examining agreement between subscales on the RCADS and RCADS-P, with separation anxiety being the only subscale that demonstrated moderate agreement. In a school-based sample, Ebesutani et al. (2011) did find significant correlations between the RCADS subscales and the corresponding RCADS-P subscales, though they also found the strongest agreement for separation anxiety (r = 0.39) compared to the other subscales (rs = 0.14 to 0.21). This is consistent with other recent research indicating parent-child agreement to be higher for separation anxiety and school refusal as compared to other facets of anxiety, which makes sense since parents may be most likely to directly observe separation anxiety/school refusal behaviors (Becker et al., 2016).

In considering the various parent measures used to examine convergent and discriminant validity, RCADS anxiety generally demonstrated convergent and discriminant validity in relation to parent-report measures of internalizing symptoms and externalizing behaviors, though RCADS depression did not. That is, RCADS depression subscale scores had small, nonsignificant associations with parent-reported depression on both the RCADS-P and CBCL. Other studies using both rating scale and interview methods have also reported low agreement between parent and child depressive symptoms (Angold et al., 1987; Mesman & Koot, 2000), though it does not appear in the broader literature that there is lower agreement for depression than for anxiety (Cole, Hoffman, Tram, & Maxwell, 2000; Herjanic & Reich, 1982). However, rates of depression are lower than anxiety in school-aged children (Zahn-Waxler, Klimes-Dougan, & Slattery, 2000), as well as in children with ADHD (Angold et al., 1999), which may have contributed to the lower parent-child agreement for depression specifically that was found in our study. Considered together, although the RCADS did not demonstrate convergent/discriminant validity with parent-reported depression, support for both convergent and discriminant validity was found for child-reported depression and both child- and parent-rated anxiety, providing initial support for the convergent and discriminant validity of the RCADS in children with ADHD.

Interestingly, few children met criteria for clinically elevated (i.e., T-score ≥70) scores on the RCADS (2.6% to 6.0% depending on subscale), and sample mean T-scores were in the normative range (see Table 4). Additional studies that also include non-ADHD-referred children (as well as other clinical samples) will be needed to determine whether children with ADHD have similar or different patterns of responding when assessing internalizing symptoms. However, it is important to note that youth frequently display subthreshold symptoms of psychopathology (Lewinsohn, Shankman, Gau, & Klein, 2004) which frequently develop into full syndrome disorders in adulthood (Shankman et al., 2009). Furthermore, among the children with elevated RCADS scores in our study, few received an internalizing diagnosis per a diagnostic interview with the child’s parent, underscoring the importance of gathering self-report of internalizing symptoms in children referred for possible or suspected ADHD.

In considering the multi-informant assessment of internalizing symptoms in children referred for possible ADHD, one benefit of the RCADS is that it has a parallel parent version (RCADS-P) that can readily be used for cross-informant comparisons (Ebesutani, Bernstein, Nakamura, Chorpita, Weisz, et al., 2010; Ebesutani et al., 2011). The RCADS-P was recently validated in children evaluated for possible ADHD (Becker et al., 2017). Specifically, in a sample of 372 children, the RCADS-P demonstrated adequate internal consistency as well as convergent and discriminant validity with other parent ratings (less clear evidence was found for convergent and discriminant validity in a subsample of 162 children with teacher ratings). In addition, the RCADS-P demonstrated good-to-excellent diagnostic efficiency and sensitivity/specificity relative to an internalizing disorder diagnosis on the K-SADS parent interview (Becker et al., 2017). Unfortunately, a diagnostic interview was not conducted with children themselves in the current study, which would be an important contribution in future research. Nevertheless, the current findings coupled with findings from the RCADS-P study indicates that these parallel internalizing measures may be useful in the multi-informant assessment of internalizing symptoms in youth with suspected ADHD. Examination of the RCADS in clinical practice, including both assessment and treatment settings, will be needed to inform whether and when to use the RCADS in clinical care.

This study had several strengths, including use of a sample of children referred to an ADHD specialty clinic and multiple informants. Yet, several limitations are important to note. First, this study was not initially designed to examine the RCADS and, as such, our examination of convergent validity for child-rated anxiety was limited to a measure of social anxiety as opposed to a general or multi-dimensional measure of anxiety. In addition, the cross-sectional design of our study precluded our ability to examine other important aspects of psychometric validation such as test-retest reliability or predictive validity. Our sample was comprised of school-aged children referred for ADHD, with few participants having elevated internalizing symptoms. The reason for this is unclear, and perhaps represented a dampening bias in the reporting of internalizing symptoms among children with ADHD stemming from overly positive reporting (Swanson et al., 2012), a possibility warranting empirical scrutiny. Alternatively, parents of less severe clinical presentations (of ADHD and/or internalizing) may have been more likely than parents of more clinically severe children to enroll in the study, though we unfortunately do not have data to test whether this was the case. In any event, additional research will be needed to evaluate the psychometric properties of the RCADS in samples with more severe internalizing symptoms, including adolescents with ADHD when rates of internalizing symptoms (especially depression) may increase. It will also be important for future studies to use larger, more adequately-powered samples (see Footnote 3).

In addition, our study did not include a comparison group of typically developing children without ADHD, and a key step in future research would be to examine whether the RCADS is invariant between youth with and without ADHD. Additionally, although the K-SADS is a reliable and often used tool to establish diagnoses in research studies, no teacher report was obtained and the information obtained was related to the current presentation; given the nonpathognomonic nature of ADHD symptoms it will be important to replicate our findings with a sample of children who undergo a more thorough clinical assessment that includes teacher ratings and a more thorough history of past functioning. Finally, although it has been argued that children are in the unique position to report on behaviors across different situations and that their perspective should be captured in any assessment (La Greca, 1990), and that children are often better reporters of internal states that are not readily observable for parents (Moretti, Fine, Haley, & Marriage, 1985; Rey, Schrader, & Morris-Yates, 1992), others have argued that children with ADHD have a positive bias and may not be the best reporters of their own competence (Owens et al., 2007). Although recent evidence calls into question how ubiquitous this phenomenon is in youth with ADHD specifically (Bourchtein et al., 2017; Jiang & Johnston, 2017), future work will need to examine whether these findings are replicated when others’ ratings of internalizing are utilized. It would also be informative for future research to examine associations between internalizing symptoms and positive bias (Swanson et al., 2012), including whether child or parent report is more clearly associated with objective functional outcomes or treatment response in children with ADHD. Given these considerations, findings from the present study provide initial psychometric evidence for the RCADS in children referred for ADHD.