Making high‐quality measures available in diverse contexts—The psychometric properties of the Revised Child Anxiety and Depression Scale in a Norwegian sample

Abstract

Objectives

Recent initiatives have recommended the Revised Child Anxiety and Depression Scale (RCADS) for use in research and as patient‐reported outcome in health care globally. We aimed to investigate, for the first time, whether the psychometric properties of the anxiety and depression youth self‐report measures, RCADS‐47 and RCADS‐25, generalize to a Norwegian setting.

Methods

We examined gender and age differences in symptomatology among 592 children (mean age 10.7 years), and conducted a psychometric investigation of the internal reliability, structural validity, measurement invariance and convergent validity of the RCADS‐47 and RCADS‐25 youth versions.

Results

Girls reported higher levels of anxious and depressive symptoms than boys, but no age differences were observed. Reliability coefficients for the RCADS‐47 and RCADS‐25 scales indicated good internal consistency. Structural validity for RCADS‐47 and RCADS‐25 was supported by confirmatory factor analyses results. For both measures, strong gender‐based measurement invariance was present. Convergent validity of the RCADS‐47 and RCADS‐25 with other well‐established self‐report measures for anxiety (Multidimensional Anxiety Scale for Children) and depression (The Short Mood and Feelings Questionnaire) was supported.

Conclusion

The RCADS‐47 and RCADS‐25 youth versions are valid and reliable instruments for measuring symptoms of anxiety and depression in a Norwegian setting. The results add to the evidence supporting RCADS's cross‐cultural validity.

1. INTRODUCTION

Anxiety and depression are common in children and adolescents (Polanczyk et al., 2015), frequently co‐exist (Melton et al., 2016) and are associated with impairment in social, emotional, academic, and family functioning that can extend into adulthood if left untreated (Swan & Kendall, 2016). Early and targeted intervention is important to prevent further developments of problems (Giesen et al., 2007), but health systems have yet to find an adequate response to the burden of mental disorders. Despite substantial increases in the provision of treatment the last decades, the prevalence of mood disorders has not been reduced (Jorm et al., 2017), and the gap between the need for treatment and its provision is still large all over the world (World Health Organization, 2013).

There is a need for a more coordinated response to this problem from health, social and research sectors. One barrier is the large variation in measurement instruments used for mental health outcomes. Even when measuring the same constructs, results from different instruments are difficult to compare (Wahl et al., 2014). The competition between the many instruments cause data gaps, fragmentation, and inconsistencies, all of which hamper the potential to inform quality improvement efforts. An international standard set of health outcome measures may therefore be a necessary step to ensure that research leads to tangible improvements. Such standards have the potential to enhance and accelerate the understanding of mental health disorders and their treatments (Obbarius et al., 2017). Recent initiatives therefore seek to attain global standardized mental health measurement (Krause et al., 2021). Some of the largest funders of mental health research worldwide, such as the National Institute of Mental Health (NIMH) and Wellcome Trust, endorse these initiatives, making funding obtainment for new research related to depression and anxiety in children and adolescents conditional on using two specific consensus‐based measures (NIMH, 2020).

One of the two consensus‐based measures recommended to use with children and adolescents worldwide is the Revised Child Anxiety and Depression Scale (RCADS; Chorpita et al., 2000). RCADS is available as a full 47‐item version and as a 25‐item short version. The full RCADS‐47 provides an accurate indication of primary problems with its six subscales based on the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM‐IV; American Psychiatric Association, 1994). The items can further be combined into aggregated anxiety, depression, or total internalizing scales. The RCADS 25‐item short version (Ebesutani et al., 2012) includes 25 items from the full version, and can also be combined into anxiety, depression, and total internalizing scales. Although the full RCADS‐47 version was initially chosen to be included in the set of consensus‐based instruments, the shorter RCADS‐25 version ended up as the final recommendation due to its brevity (Krause et al., 2021). In terms of feasibility, both versions of RCADS have several advantages over other measures. They are freely available at no monetary cost, in contrast to many other instruments. The RCADS‐25 is short. Although the RCADS‐47 is longer, it is the briefest measure aligned with the DSM‐IV (Southam‐Gerow et al., 2008). Furthermore, both RCADS‐47 and RCADS‐25 assess anxious and depressive symptoms concurrently. This is useful given the high comorbidity of the two disorders and the increasing number of transdiagnostic interventions available (Ehrenreich & Chu, 2014; Hunsley & Mash, 2007). Previous studies in both community and clinical samples in several countries have reported good psychometric properties for RCADS‐47 (e.g., Chorpita et al., 2005; Esbjørn et al., 2012; Mathyssek et al., 2013; Piqueras et al., 2017). Its discriminant validity is superior to some of the most common measures of anxiety and depression, such as the Revised Children's Manifest Anxiety Scale (RCMAS; Reynolds & Richmond, 1978) and Children's Depression Inventory (CDI; Kovacs, 1980) (see Chorpita et al., 2005). Although less widely validated than RCADS‐47, good psychometric properties have thus far also been reported for RCADS‐25 (Ebesutani et al., 2012).

To build confidence in the comparison of research findings, measures should be validated prior to usage in new cultural contexts. It is especially important that translated versions that have undergone cross‐cultural adaptation demonstrate their psychometric properties, as subtle changes could alter the instrument's psychometric properties (Chang et al., 1999). The RCADS‐47 and RCADS‐25 have demonstrated promising psychometric abilities in various countries in Europe, Africa, Asia, and the Americas (Piqueras et al., 2017). Western European data from the measures exist and a recent systematic literature review (Eidet et al., 2022) found that, in Scandinavia, there are existing Danish and Swedish peer‐reviewed studies which have reported favorable psychometric properties of RCADS‐47 (e.g. Esbjørn et al., 2012). To the best of our knowledge, however, no psychometric properties of the RCADS‐25 have been reported in Scandinavia yet. And thus far, there is no documentation of the psychometric properties of the Norwegian RCADS full or short version. In the present study, we therefore aimed to investigate whether the psychometric properties of the RCADS‐47 and RCADS‐25 child self‐report instruments could be generalized to a Norwegian setting. We examined the internal reliability and the structural validity of the RCADS‐47 full version and the RCADS‐25 short version. Further, we explored measurement invariance across gender. Finally, the convergent validity of RCADS‐47 and RCADS‐25 with other well‐established measures of anxiety and depression was investigated.

2. METHOD

2.1. Participants and procedure

The current study was a part of a larger trial of an indicated transdiagnostic preventive intervention, the ECHO study (clinicaltrials.gov NCT04263558; see Neumer et al., 2021). Children and parents at participating schools received oral and written information about the study, inviting children who experienced symptoms of anxiety or depression to participate in the study. Children with valid parental consent completed electronic surveys at school. The surveys were mostly forced choice to minimize the amount of missing data. Altogether 43 public schools in urban and rural areas, from 24 municipalities and boroughs across three regions in Norway (south‐east, central, and north), took part in the present study. The participating children (n = 592) had a mean age of 10.7 years and attended fourth through sixth grade.

To investigate psychometric properties, the RCADS youth self‐report was included as a pre‐intervention measure for a subsample of the participants in the larger trial (all participants from two out of five data collection waves; fall and spring term 2020/2021) alongside the anxiety measure Multidimensional Anxiety Scale for Children (MASC; March et al., 1997) youth report and The Short Mood and Feelings Questionnaire (SMFQ; Angold et al., 1995) youth report. The current article thus presents data from this subsample.

2.2. Measures

2.2.1. The Revised Child Anxiety and Depression Scale (RCADS‐47 youth version)

The full RCADS‐47 (Chorpita et al., 2000) youth version is a 47‐item self‐report anxiety and depression rating scale for children and youth aged 8–18 years. Though a parallel caregiver version exists, the present study examined the youth self‐report only. Respondents rate how often each item applies to them on a 4‐point Likert scale. The RCADS‐47 is based on the Spence Children's Anxiety Scale (SCAS; Spence, 1997), and includes six subscales which corresponds with the DSM‐IV classifications for anxiety and depressive disorders: separation anxiety disorder, generalized anxiety disorder, panic disorder, social phobia, obsessive–compulsive disorder (OCD), and major depressive disorder. Scores can also be aggregated into Total Anxiety (sum of the five anxiety subscales) and Total Internalizing (sum of all six subscales). Although obsessive–compulsive disorder has been re‐categorized and excluded from the anxiety disorders category in the DSM‐V (APA, 2013), the manual still recognizes the high comorbidity between OCD and anxiety disorders. As both often share treatment approaches (such as cognitive behavioral therapy with exposure), it is useful to continue to assess OCD in line with anxiety disorders (Ebesutani et al., 2017). Chorpita et al. (2005) found good internal consistency for the six RCADS subscales, with α ranging from 0.78 (SAD) to 0.88 (GAD). Furthermore, they found good convergent and discriminant validity, and RCADS demonstrated greater correspondence to specific diagnostic syndromes than traditional measures of anxiety and depression. The RCADS is freely available from the developers in 19 languages, including Norwegian (www.childfirst.ucla.edu/resources).

The abbreviated RCADS‐25 youth self‐report is a 25‐item scale developed by shortening the five separate anxiety scales of the RCADS‐47, keeping three items from each original anxiety subscale, resulting in a broad 15‐item anxiety scale (Ebesutani et al., 2012). The 10 items measuring depression were retained as in the original depression subscale. The items in RCADS‐25 are aggregated to anxiety, depression, or total internalizing scales. Previous studies have corroborated the construct validity (Young et al., 2020) of the RCADS‐25 as well as high internal reliability, with α of 0.86 (anxiety scale) and 0.80 (depression scale) (Ebesutani et al., 2012). In the present sample, the RCADS‐47 was distributed and results concerning the shortened RCADS‐25 have been tested using the 25 items from the original RCADS‐47.

2.2.2. MASC

MASC (March et al., 1997) is a 39 item self‐report measure for children and youth 8–19 years. Symptoms of anxiety are rated on a 4‐point Likert‐scale. MASC contains 4 subscales: Physical Symptoms, Social Anxiety, Separation Anxiety/Panic, and Harm Avoidance. The measure has demonstrated robust psychometric properties internationally (e.g., Rynn et al., 2006). The Norwegian version of MASC has been documented for children in fourth through sixth grade (Martinsen et al., 2017; Villabø et al., 2012). Internal consistency for MASC subscales in the present study were consistent with previous Norwegian findings (Martinsen et al., 2017), with the reliability coefficient omega (ω; McDonald, 1999) for the subscales ranging from 0.66 (Harm Avoidance) to 0.89 (Social Anxiety and Physical Symptoms). The original 4‐factor structure was supported by acceptable confirmatory factor analyses (CFA)‐fit in the present sample (n = 592); comparative fit index (CFI) = 0.92, Tucker‐Lewis index (TLI) = 0.92, Root Mean Square Error of Approximation (RMSEA) = 0.053.

2.2.3. The SMFQ

SMFQ (Angold et al., 1995) is a 13‐item self‐report measure rated on a 3‐point Likert‐scale to assess core symptoms of depression in children and youth 6–18 years. According to previous international (Katon et al., 2008; Thabrew et al., 2018) and Norwegian (Lundervold et al., 2013; Martinsen et al., 2019) studies, SMFQ has high internal consistency. The reliability found in the present study was good (ω = 0.90). SMFQ is unidimensional (Messer et al., 1995), which in the present study was corroborated by acceptable CFA‐fit for a one‐factor model (n = 591); CFI = 0.98, TLI = 0.98, RMSEA = 0.066.

2.2.4. Demographics variables

Parents provided demographic information (age, gender, grade) upon completing parental consent forms.

2.3. Data analyses

Bias Corrected and accelerated (BCa; Efron, 1987) bootstrapped t‐tests (1000 replications) for independent samples (see Kelley, 2005), along with the unbiased standardized effect size (Hedge's g; Hedges, 1981; see also Cumming, 2012), were used to assess gender differences in symptoms of anxiety or depression. Pearson correlations were used to investigate the association between age and symptoms.

Reliability, for both the full sample and by gender, was assessed with the internal consistency coefficient McDonald's omega (ω; McDonald, 1999) and corresponding BCa bootstrapped 95% confidence interval (CI) (10,000 replications). Omega can be interpreted the same way as Cronbach's Alpha, but it does not depend on items being tau‐equivalent and provides a more accurate approximation of a scale's internal structure than Cronbach's Alpha (Dunn et al., 2014; Revelle & Zinbarg, 2009).

CFAs were carried out to investigate construct validity. As the required sample size depends on values in the covariance matrix unknown before the study, there is no consensus on estimation methods for CFA sample‐sizes (Fayers & Machin, 2007). With the large RCADS‐47 model, it is reasonable to assume that a few hundred participants would be required and that 589 participants were sufficient to ensure power and low bias in parameters interest (see Myers et al., 2011; Wolf et al., 2013). Given the ordinal metric nature of the items, the robust weighted least square (WLSMV) estimator, using a diagonal weight matrix, and theta parameterization was chosen for all analyses. Latent variables were scaled by fixing the first loading of each factor to 1. There was no missing data. Error variances were kept uncorrelated in all analyses. Factor loadings above 0.5 were considered acceptable (Hulland, 1999). For RCADS‐47, the CFA was based on the theorized 6‐factor structure equivalent to the RCADS subscales (Chorpita et al., 2000). For RCADS‐25, the CFA was based on an a priori 2‐factor structure of the two subscales (anxiety and depression). To evaluate the models' goodness‐of‐fit, we considered the CFI, TLI, and the RMSEA with its corresponding 90% CI. Higher CFI and TLI values indicate a better fit, and we considered CFI and TLI values above 0.90, and 0.95 as indicative of acceptable and good fit, respectively (Hu & Bentler, 1999). With RMSEA, lower values indicate a better fit, and RMSEA below 0.08 and 0.06 were considered as indicative of acceptable and good model fit, respectively. Limitations of the χ ²‐test includes test sensitivity that increases with sample size. This problem is compounded when accompanied by many degrees of freedom, which means that model size also affects the χ ²‐test (Moshagen, 2012). Due to these issues, we largely relied on alternative fit indices.

Measurement invariance tests across gender were performed following a 4‐step procedure (Bowen & Masa, 2015). First, CFAs for each gender were run separately using the same approaches as in the CFAs previously described. Then we ran multigroup models with both genders to investigate configural, metric (weak) and scalar (strong) invariance. In the configural model, factor loadings and thresholds across groups were free. In the metric model, factor loadings and the first threshold were held equal across groups. In the scalar model, factor loadings and thresholds were held equal across groups. Like the χ ²‐tests, χ ²‐difference testing is sensitive to sample size and model size. Alternative fit indices are much less sensitive to sample size and more sensitive to lack of invariance than χ ²‐based tests of measurement invariance (Meade et al., 2008). In terms of the model comparison, we therefore reviewed CFI change (ΔCFI) and RMSEA change (ΔRMSEA). The ΔCFI and ΔRMSEA cut‐offs used were 0.002 and 0.007, respectively, as suggested by Meade et al. (2008).

Pearson correlations (r) were used to evaluate convergent validity of RCADS anxiety and depression scales with the MASC and the SMFQ. In addition, partial correlation analyses were employed to evaluate the correlations while controlling for potential effects of gender. Correlation coefficients were interpreted in line with the European Federation of Psychologists' Associations' (EFPA, 2013) guidelines on congruent validity coefficients (Inadequate <0.55; Adequate = ≥0.55 to <0.65; Good ≥0.65 to <0.74; Excellent ≥0.75, in absolute value).

Preliminary analyses were performed with IBM SPSS Statistics version 22. Omega estimation and CFAs were performed using Mplus version 7.4.

3. RESULTS

A total of 4280 children attended the schools participating in the larger study of an indicated preventive intervention. Parents of 603 children (14%) consented to their child's participation. Before the study, 11 children (2%) withdrew. Of the remaining 593 children, 592 (98.8%) responded to the survey. Of the responders, 592 completed the MASC, 591 completed both the MASC and SMFQ, and 589 children completed all three measures (RCADS, MASC and SMFQ). Within each instrument, children had completed all the items and there was no missing data.

The age ranged from 9.1 to 12.0 years, with a mean age of 10.7 years (SD = 0.52), with 29 (5%), 296 (50%) and 267 (45%) children attending fourth, fifth and sixth grade, respectively. There were 238 boys (40%) and 354 girls (60%).

3.1. Gender differences in symptoms

Mean scores for all measures are shown in Table 1. Girls reported higher levels of both anxious and depressive symptoms. Bootstrapped independent samples t‐tests showed that the gender differences in means were significant for all subscales, although the MASC subscale Harm Avoidance was approaching a non‐significant level. The effect sizes were small to moderate (Cohen, 1988), with Hedge's g ranging from 0.27 to 0.55. The effect size for RCADS‐47 anxiety was larger than for the abbreviated RCADS‐25 scale: RCADS‐47 anxiety (g = 0.44), RCADS‐25 anxiety (g = 0.36). Both RCADS‐47 and RCADS‐25 anxiety scales showed larger effect sizes than the RCADS‐47 and RCADS‐25 depression scales (which were identical, containing the same items) (g = 0.28).

TABLE 1.

RCADS‐47, RCADS‐25 MASC and SMFQ means and SD for the full sample, and by gender, along with t‐test significance value and standardized effect size (Hedges g)

Measure/Subscale	Full sample	Boys	Girls
	Mean (SD)	Mean (SD)	Mean (SD)	p	Hedges g
RCADS‐47 (n = 589)		n = 235	n = 354
Social phobia	12.4 (6.5)	10.3 (6.2)	13.7 (6.4)	<0.001	−0.55
Panic disorder	8.3 (6.3)	6.9 (5.8)	9.1 (6.5)	<0.001	−0.35
Separation anxiety	6.1 (4.7)	5.1 (4.5)	6.8 (4.6)	<0.001	−0.36
Generalized anxiety	7.7 (4.4)	6.9 (4.4)	8.2 (4.4)	0.002	−0.28
Obsessive‐compulsive	6.2 (3.9)	5.6 (3.5)	6.6 (4.0)	<0.001	−0.27
Depression	10.1 (5.8)	9.1 (5.6)	10.7 (5.8)	<0.001	−0.28
Total anxiety	40.6 (22.5)	34.9 (21.2)	44.5 (22.5)	<0.001	−0.44
Total internalizing	50.7 (27.4)	44.0 (25.8)	55.2 (27.5)	<0.001	−0.42
RCADS‐25 (n = 589)		n = 235	n = 354
Anxiety	15.0 (9.1)	13.1 (8.5)	16.3 (9.3)	<0.001	−0.36
Depression	10.1 (5.8)	9.1 (5.6)	10.7 (5.8)	<0.001	−0.28
Total internalizing	25.1 (14.2)	22.2 (13.3)	27.0 (14.4)	<0.001	−0.35
MASC‐C (n = 592)		n = 238	n = 354
Physical symptoms	15.9 (7.4)	14.0 (7.2)	17.2 (7.3)	<0.001	−0.44
Harm avoidance	18.1 (3.9)	17.7 (4.2)	18.4 (3.7)	0.048	−0.18
Social anxiety	15.5 (6.5)	13.6 (6.7)	16.7 (6.0)	<0.001	−0.50
Separation/Panic scale	12.3 (5.6)	10.7 (5.7)	13.5 (5.3)	<0.001	−0.52
MASC total	61.9 (18.7)	56.0 (19.0)	65.9 (17.4)	<0.001	−0.55
SMFQ‐C (n = 591)		n = 237	n = 354
	9.4 (6.1)	8.1 (5.6)	10.2 (6.3)	<0.001	−0.34

Note: All tests two‐tailed.

Abbreviations: MASC, Multidimensional Anxiety Scale for Children; RCADS, Revised Child Anxiety and Depression Scale; SD, standard deviation; SMFQ, Short Mood and Feelings Questionnaire.

3.2. Age differences in symptoms

Children's age did not appear related to symptoms of anxiety and depression. Pearson correlations revealed no significant associations between age and the total internalizing scores for RCADS‐47 (r = −0.05, p = 0.266) or RCADS‐25 (r = −0.03, p = 0.520). Nor were there significant associations between age and RCADS‐47 total anxiety (r = −0.06, p = 0.169) or RCADS‐25 anxiety (r = −0.04, p = 0.292), or between age and the RCADS‐47 or RCADS‐25 depression subscale (r = 0.004, p = 0.931). Similarly, Pearson correlations between age and MASC (r = −0.06, p = 0.166), and age and SMFQ (r = −0.06, p = 0.153) were very weak and non‐significant.

3.3. RCADS‐47

3.3.1. RCADS‐47 reliability

The internal consistency of the RCADS‐47 subscales (Table 2) was good, with omega ranging from 0.81 (Obsessive‐Compulsive) to 0.92 (Panic Disorder and Social Phobia). Omega was also estimated for each gender separately. Good internal consistency was indicated for both genders on all subscales. There were some gender discrepancies in the omega point estimates, particularly for the subscales Separation Anxiety and Obsessive‐Compulsive disorder, but there was considerable overlap between the confidence intervals for boys and girls also for these two subscales.

TABLE 2.

RCADS‐47 omega coefficients and their confidence intervals

Subscale	Girls (n = 354)	Boys (n = 235)	Full sample (= 589)
	ω (95% CI)	ω (95% CI)	ω (95% CI)
Panic disorder	0.92 (0.90, 0.93)	0.91 (0.88, 0.93)	0.92 (0.90, 0.93)
Major depression	0.86 (0.84, 0.88)	0.85 (0.81, 0.88)	0.86 (0.84, 0.88)
Separation anxiety	0.81 (0.76, 0.84)	0.85 (0.82, 0.88)	0.83 (0.80, 0.85)
Generalized anxiety	0.88 (0.86, 0.90)	0.89 (0.87, 0.91)	0.89 (0.87, 0.90)
Obsessive‐compulsive	0.82 (0.79, 0.85)	0.77 (0.71, 0.81)	0.81 (0.78, 0.83)
Social phobia	0.90 (0.88, 0.92)	0.91 (0.88, 0.92)	0.91 (0.90, 0.92)

Abbreviations: CI, Confidence Interval; ω, omega.

3.3.2. RCADS‐47 confirmatory factor analysis

The RCADS six‐factor model (n = 589) showed a good fit with the data; χ ² = 2643.895 (1019), p < 0.001; CFI = 0.95; TLI = 0.95 and RMSEA = 0.052 (90% CI = 0.050, 0.054). The CFI and TLI indicated a good model fit (>0.95), and so did the RMSEA (<0.06). All factor loadings (Table 3) were acceptable, ranging from 0.52 to 0.92, and significant (p < 0.001). The hypothesized 6‐factor structure was therefore supported.

TABLE 3.

RCADS‐47 truncated items and standardized (STDYX) parameter estimates with standard errors

Parameter	Standardized estimate (standard error)
Social phobia BY
4. Worry to do poorly	0.78 (0.02)
7. Scared to take a test	0.66 (0.03)
8. Worry someone is angry with me	0.79 (0.02)
12. Worry to do badly at school	0.82 (0.02)
20. Worry to look foolish	0.85 (0.02)
30. Worry about making mistakes	0.84 (0.02)
32. Worry what people think of me	0.88 (0.01)
38. Afraid to talk in front of class	0.60 (0.03)
43. Afraid to make a fool of myself	0.83 (0.02)
Separation anxiety BY
5. Afraid of being home alone	0.69 (0.03)
9. Worry about being away from parent	0.68 (0.03)
17. Scared to sleep alone	0.71 (0.03)
18. Nervous going to school	0.80 (0.03)
33. Afraid in crowded places	0.70 (0.03)
45. Worry when going to bed	0.88 (0.02)
46. Scared to stay away overnight	0.58 (0.04)
Generalized anxiety BY
1. Worry about things	0.75 (0.03)
13. Worry something awful happens to family	0.71 (0.03)
22. Worry bad things will happen to me	0.89 (0.01)
27. Worry something bad happens	0.93 (0.01)
35. Worry about what will happen	0.89 (0.02)
37. Think about death	0.68 (0.03)
Panic disorder BY
3. Funny feeling in stomach when problems	0.70 (0.03)
14. Feel as if can't breathe	0.81 (0.02)
24. Heart beats fast when problems	0.78 (0.02)
26. I suddenly tremble or shake	0.83 (0.02)
28. Feel shaky when problems	0.79 (0.02)
34. Scared for no reason	0.87 (0.02)
36. Dizzy for no reason	0.77 (0.02)
39. Heart suddenly beats quickly	0.83 (0.02)
41. Worry about getting a scared feeling	0.85 (0.02)
Obsessive‐compulsive BY
10. Bad or silly thoughts	0.79 (0.02)
16. Keep checking things are done right	0.56 (0.03)
23. Can't get bad thoughts out of mind	0.78 (0.02)
31. Think of special thoughts to stop bad things	0.68 (0.03)
42. Have to do some things over and over again	0.58 (0.03)
44. Have to do things just the right way	0.78 (0.02)
Major depression BY
2. Feel sad or empty	0.82 (0.02)
6. Nothing is much fun anymore	0.66 (0.03)
11. Trouble sleeping	0.62 (0.03)
15. Problems with appetite	0.62 (0.03)
19. No energy	0.55 (0.04)
21. Tired a lot	0.52 (0.04)
25. Cannot think clearly	0.78 (0.02)
29. Feel worthless	0.81 (0.02)
40. Don't want to move	0.82 (0.02)
47. Feel restless	0.56 (0.03)
Generalized anxiety WITH
Separation anxiety	0.81 (0.02)
Panic disorder WITH
Separation anxiety	0.76 (0.02)
Generalized anxiety	0.81 (0.02)
Social anxiety WITH
Separation anxiety	0.70 (0.03)
Generalized anxiety	0.82 (0.02)
Panic disorder	0.76 (0.02)
Obsessive‐compulsive WITH
Separation anxiety	0.80 (0.02)
Generalized anxiety	0.88 (0.01)
Panic disorder	0.89 (0.01)
Social anxiety	0.81 (0.02)
Major depression WITH
Separation anxiety	0.75 (0.03)
Generalized anxiety	0.79 (0.02)
Panic disorder	0.90 (0.01)
Social anxiety	0.77 (0.02)
Obsessive‐compulsive	0.86 (0.02)

Note: All parameter estimates were significant at the level of p ˂ 0.001.

Abbreviation: RCADS, Revised Child Anxiety and Depression Scale.

3.3.3. RCADS‐47 measurement invariance across gender

Models fitted for boys and girls separately showed good fit with the data (Table 4). No modifications were made. Model fit for the configural model with both groups was also good, indicating configural invariance and that the six‐dimensional structure of the RCADS applied to both genders. From the configural to the metric model, the ΔCFI was −0.001, while the RMSEA did not deteriorate, indicating metric (weak) invariance. This finding can be interpreted as items having similar weights and that the same latent factors were measured in each group. From the metric to the scalar model, neither CFI nor RMSEA deteriorated. Scalar (strong) invariance was thus supported. This finding implies that the differences in the proportion of responses in each category was caused by factor mean differences. RCADS‐47 scores can therefore be interpreted the same way across gender.

TABLE 4.

Model fit statistics from RCADS‐47 invariance testing (n = 589)

Model	χ 2 (p)	df	RMSEA (90% CI)	ΔRMSEA	TLI	CFI	ΔCFI
Single group solutions
Boys (n = 235)	1423.06 (p < 0.001)	1019	0.041 (0.035, 0.046)	‐	0.964	0.966	‐
Girls (n = 354)	2087.00 (p < 0.001)	1019	0.054 (0.051, 0.058)	‐	0.954	0.950	‐
Measurement invariance
Configural	3428.86 (p < 0.001)	2083	0.048 (0.045, 0.051)	‐	0.954	0.957	‐
Metric	3478.54 (p < 0.001)	2079	0.048 (0.045, 0.051)	0.000	0.955	0.956	−0.001
Scalar	3533.85 (p < 0.001)	2167	0.046 (0.044, 0.049)	−0.002	0.958	0.957	0.001

Abbreviations: CFI, Comparative Fit Index; CI, confidence interval; df, degrees of freedom; RMSEA, root mean square error of approximation; TLI, Tucker‐Lewis Index.

3.3.4. RCADS‐47 convergent validity

The Pearson correlation between RCADS‐47 total anxiety and MASC total was 0.81 (p < 0.001), and the partial correlation controlled for gender was 0.80 (p < 0.001). The convergent validity, then, was in the “excellent” category (≥0.75) according to the EFPA guidelines on congruent validity coefficients. The correlation between RCADS‐47 major depression and the SMFQ total scores was 0.75 (p < 0.001), and the partial correlation was also 0.75 (p < 0.001). There was thus evidence of excellent convergent validity also for the RCADS‐47 depression scale. As expected, there were also associations between anxiety symptoms and depressive symptoms. The Pearson correlation between RCADS‐47 total anxiety and SMFQ was 0.71 (p < 0.001), and the partial correlation controlled for gender was 0.70 (p < 0.001). The correlation between RCADS‐47 depression and MASC was 0.68 (p < 0.001), and the partial correlation controlled for gender was 0.67 (p < 0.001).

3.4. RCADS‐25

3.4.1. RCADS‐25 reliability

Omega coefficients for the RCADS‐25 subscales anxiety (ω = 0.91, 95% CI: 0.90–0.92) and depression (ω = 0.86, 95% CI: 0.84–0.88) indicated good internal consistency. Omega estimated for each gender separately indicated good internal consistency both for boys (anxiety ω = 0.90, 95% CI = 0.88–0.92; depression ω = 0.85, 95% CI = 0.81–0.88) and girls (anxiety ω = 0.91, 95% CI = 0.89–0.92; depression ω = 0.86, 95% CI = 0.84–0.89).

3.4.2. RCADS‐25 CFA

The CFA for the a priori hypothesized RCADS‐25 two‐factor model demonstrated an acceptable fit with the data: (n = 589); χ ² = 1134.1 (274), p < 0.001; CFI = 0.94; TLI = 0.93; RMSEA = 0.073 (90% CI = 0.069, 0.077). CFI and TLI indicated an acceptable (>0.90) fit, and so did the RMSEA (<0.08), in line with findings from previous studies (Young et al., 2020). Standardized factor loadings (Table 5) ranged from 0.53 to 0.84, and all factor loadings were significant (p ˂ 0.001).

TABLE 5.

RCADS‐25 truncated items and Standardized (STDYX) parameter estimates with standard errors

Parameter	Standardized estimate (standard error)
Anxiety BY
2. Worry to do poorly	0.66 (0.03)
3. Afraid of being home alone	0.56 (0.03)
5. Worry something awful happens to family	0.66 (0.03)
6. Afraid in crowded places	0.61 (0.03)
7. Worry what people think of me	0.78 (0.02)
9. Scared to sleep alone	0.60 (0.03)
11. Dizzy for no reason	0.75 (0.02)
12. Have to do some things over and over again	0.57 (0.03)
14. I suddenly tremble or shake	0.80 (0.02)
17. Think of special thoughts to stop bad things	0.65 (0.03)
18. Think about death	0.64 (0.03)
20. Worry about getting a scared feeling	0.81 (0.02)
22. Afraid to make a fool of myself	0.76 (0.02)
23. Have to do things just the right way	0.76 (0.02)
25. Worry something bad happens	0.84 (0.02)
Major depression BY
1. Feel sad or empty	0.80 (0.02)
4. Nothing is much fun anymore	0.68 (0.03)
8. Trouble sleeping	0.60 (0.03)
10. Problems with appetite	0.65 (0.03)
13. No energy	0.56 (0.03)
15. Cannot think clearly	0.77 (0.02)
16. Feel worthless	0.82 (0.02)
19. Don't want to move	0.82 (0.02)
21. Tired a lot	0.53 (0.03)
24. Feel restless	0.55 (0.03)
Anxiety WITH depression	0.90 (0.01)

Note: All parameter estimates were significant at the level of p ˂ 0.001.

Abbreviation: RCADS, Revised Child Anxiety and Depression Scale.

3.4.3. RCADS‐25 measurement invariance across gender

At the first step in the RCADS‐25 gender invariance testing, separate models for boys and girls demonstrated acceptable fit with the data (Table 6). Configural invariance was supported by acceptable fit indices for the configural model. Metric (weak) invariance was indicated by ΔCFI = −0.001 and no deterioration in RMSEA from the configural to the metric model. From the metric to the scalar model neither CFI nor RMSEA deteriorated, indicating scalar (strong) invariance. RCADS‐25 scores can thus be interpreted the same way across gender.

TABLE 6.

Model fit statistics from RCADS‐25 invariance testing (n = 589)

Model	χ 2 (p)	df	RMSEA (90% CI)	ΔRMSEA	TLI	CFI	ΔCFI
Single group solutions
Boys (n = 235)	530.79 (p < 0.001)	274	0.063 (0.055, 0.071)	‐	0.941	0.946	‐
Girls (n = 354)	804.67 (p < 0.001)	274	0.074 (0.068, 0.080)	‐	0.936	0.942	‐
Measurement invariance
Configural	1301.12 (p < 0.001)	548	0.068 (0.064, 0.073)	‐	0.939	0.944	‐
Metric	1336.05 (p < 0.001)	571	0.067 (0.063, 0.072)	−0.001	0.940	0.943	−0.001
Scalar	1344.70 (p < 0.001)	619	0.063 (0.058, 0.068)	−0.004	0.948	0.946	0.003

Abbreviations: CFI, Comparative Fit Index; CI, confidence interval; df, degrees of freedom; RMSEA, root mean square error of approximation; TLI, Tucker‐Lewis Index.

3.4.4. RCADS‐25 convergent validity

The Pearson correlation between the RCADS‐25 anxiety subscale and MASC total score was 0.78 (p < 0.001), and the partial correlation controlled for gender was 0.77 (p < 0.001). The correlation between the RCADS depression subscale and the SMFQ total score was 0.76 (p < 0.001) and the partial correlation controlled for gender was 0.75 (p < 0.001). The two RCADS‐25 subscales thus demonstrated convergent validity with MASC and SMFQ, falling within the “excellent” category (r ≥ 0.75) according to the EFPA guidelines on congruent validity coefficients.

4. DISCUSSION

The aim of this study was to conduct a psychometric investigation of the RCADS‐47 and RCADS‐25 youth versions. For both scales, we examined gender and age differences, reliability, structural validity, gender‐based measurement invariance and convergent validity.

Findings indicated that the Norwegian versions of RCADS‐47 and RCADS‐25 have strong psychometric properties. All RCADS‐47 and RCADS‐25 subscales showed good reliability in terms of internal consistency, in line with previous studies from several other countries (see Piqueras et al., 2017).

Structural validity for RCADS‐47 and in RCADS‐25 was supported by CFAs indicating that the expected dimensionality of a 6‐factor structure for RCADS‐47 and the 2‐factor structure of RCADS‐25 generalized to a Norwegian setting. Strong measurement invariance across gender was demonstrated for both measures. This means that the factor structure applies equally well for both genders, and that differences between the groups can be interpreted unambiguously.

Importantly, as far as convergent validity for the RCADS (both full and short version) depression and anxiety subscales can be judged from correlations with the well‐documented MASC and the SMFQ, convergent validity was strongly supported. This was also the finding when correlations were controlled for gender. Correlations between the RCADS‐47 and RCADS‐25 anxiety scales and the anxiety measure MASC were excellent. The RCADS‐47 and RCADS‐25 depression scale also had excellent correlation with the depression measure SMFQ. There were therefore good indications that both the RCADS‐47 full version and the RCDAS‐25 short version are valid measures of anxiety and depression.

We observed gender differences in symptoms of anxiety and depression, with girls reporting higher levels of anxious and depressive symptoms than boys. The effects sizes for gender differences were larger for the anxiety scales than depression. There is abundant evidence that gender differences, with girls having increased risk for anxiety and depression, increase during adolescence (e.g., Larsson et al., 2016; Leikanger et al., 2012), although the evidence for gender differences in internalizing disorders among prepubescent children is inconclusive in the literature (see Costello et al., 2005; Thapar et al., 2012; Zahn‐Waxler et al., 2008). We did not observe significant age differences in symptoms, but this may be due to the limited (and prepubescent) age range in the current study. The means observed in the present study were higher than the means for population‐based Icelandic (Olason et al., 2004) and American (Chorpita et al., 2000) samples among children the same age. This was expected and has a natural explanation in the recruitment strategy for the main study of an indicated intervention where the aim was to recruit children with more anxious or depressive symptoms than their peers. As such, the participating children's symptomatology should not be seen as representative of the general population.

Strengths of the study included negligible dropout, high response rate, a decent sample size in which both genders were represented, and sufficient variation in scores. An important study limitation to consider is that the sample is recruited as part of a larger study of an indicated preventive intervention for children with symptoms of anxiety or depression. While the response rate for the children with parental consent was high (99.8%), the participants represented 14% of the total children attending the participating schools. Children who signed up for the intervention will, on average, most likely have experienced more symptoms of anxiety and depression than the general population (e.g., Kösters et al., 2015). It is also possible that the parents who consented to their child's participation differed from the population on demographic or clinical characteristics. Another important limitation to consider was the narrow age‐range of the sample (9–12 years), which meant that we could not explore the psychometric properties for the entire age‐range for which the RCADS is intended (8–18 years). Although the present study utilized the advantage of having access to data from the full RCADS‐47 scale to perform analyses comparing responses in both full and abbreviated RCADS versions from the same sample, another possible study limitation to consider is that the RCADS‐25 was not distributed in an independent sample. As the 25 items of RCADS‐25 are included in the original RCADS‐47, the psychometric properties of RCADS‐25 were examined using only these items drawn from the full scale. It is possible that participants would respond differently to items when they are included in a somewhat longer questionnaire than if the 25 items were administered alone. Finally, the present study did not include clinical assessment such as a clinical interview. This would have been valuable to examine RCADS‐scores in relation to clinical judgment.

There is an international drive to standardize mental health outcomes, as this may harmonize data and accelerate the understanding of mental health disorders and their treatments. Emphasis has been placed on finding consensus‐based instruments that are brief, accessible, and valid. Regarding children and adolescents, consensus have been reached on the RCADS being a preferred instrument. However, it is important to establish the validity of instruments and translated versions in new settings. This was the first study to examine the psychometric properties of RCADS‐25 in Scandinavia, and the first study to investigate the psychometric properties of RCADS‐47 in Norway.

In conclusion, findings from the present study indicated that RCADS‐47 and RCADS‐25 are valid and reliable instruments for measuring symptoms of anxiety and depression in Norwegian settings. In combination with other cross‐cultural research on these instruments, the results contribute to the evidence of a robustness of the construct validity and a stable performance of the RCADS‐47 and RCADS‐25 across cultures and settings (Piqueras et al., 2017; Young et al., 2020). Both the full and the short version of the RCADS can examine comorbid and differentiating symptoms of internalizing disorders, and they balance evidence‐based assessment with feasibility demands for use in the real world. Preference for the full or short version of RCADS may depend on the specific study or clinical focus. The RCADS‐47, with its six subscales, offers more information than the RCADS‐25, which may be valuable in some contexts. Though RCADS‐25 provides a less detailed assessment, it is, one of the briefest existing measures to assess symptoms of anxiety and depression concurrently. This may be especially important when considering the participant burden of survey batteries, and its brevity was an important reason why the shorter version of RCADS version was included in a proposed international standard of health and research outcome measures.

Suggestions for future research would be to address both norm‐scores and clinical validation of the RCADS‐47 and RCADS‐25 in Scandinavia. As RCADS is intended for the ages of 8–18 years, investigating psychometric properties across a wider age‐span in a population‐based sample would also be useful.

CONFLICT OF INTEREST

The authors have no competing interests.

ETHICS STATEMENT

The authors assert that all procedures complied with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The study was approved by the Regional Committees for Medical and Health Research Ethics (REK)—South East Norway (reference number 2019/1198) and the Norwegian Centre for Research Data (reference number 152745), and a Data Protection Integrity Act (DPIA) was carried out prior to the study commencement.

Lisøy, C. , Neumer, S.‐P. , Waaktaar, T. , Ingul, J. M. , Holen, S. , & Martinsen, K. (2022). Making high‐quality measures available in diverse contexts—The psychometric properties of the Revised Child Anxiety and Depression Scale in a Norwegian sample. International Journal of Methods in Psychiatric Research, 31(4), e1935. 10.1002/mpr.1935

DATA AVAILABILITY STATEMENT

Research data are not shared.