Skip to main content
PMC home page
Springer logoLink to Springer
. 2010 Feb 24;38(5):683–694. doi: 10.1007/s10802-010-9394-1

The Relation of Severity and Comorbidity to Treatment Outcome with Cognitive Behavioral Therapy for Childhood Anxiety Disorders

Juliette Margo Liber 1,3,✉, Brigit M van Widenfelt 1, Adelinde J M van der Leeden 1, Arnold W Goedhart 1, Elisabeth M W J Utens 2, Philip D A Treffers 1
PMCID: PMC2880239  PMID: 20180011

Abstract

The present study investigated the impact of comorbidity over and above the impact of symptom severity on treatment outcome of Cognitive Behavioral Therapy for children with anxiety disorders. Children (aged 8–12, n = 124) diagnosed with an anxiety disorder were treated with a short-term CBT protocol. Severity was assessed with a composite measure of parent-reported behavior problems. Two approaches to comorbidity were examined; “total comorbidity” which differentiated anxiety disordered children with (n = 69) or without (n = 55) a co-occurring disorder and “non-anxiety comorbidity’ which differentiated anxious children with (n = 22) or without a non-anxiety comorbid disorder (n = 102). Treatment outcome was assessed in terms of Recovery, represented by post-treatment diagnostic status, and Reliable Change, a score reflecting changes in pre- to post-treatment symptom levels. Severity contributed to the prediction of (no) Recovery and (more) Reliable Change in parent-reported internalizing and externalizing symptoms and self-reported depressive symptoms. Total and non-anxiety comorbidity added to the prediction of diagnostic recovery. Non-anxiety comorbidity added to the prediction of Reliable Change in parent reported measures by acting as a suppressor variable. Non-anxiety comorbidity operated as a strong predictor that explained all of the variance associated with severity for self-reported depressive symptoms. The results support the need for further research on mechanisms by which treatment gains in children with higher symptom severity and non-anxiety comorbidity can be achieved.

Keywords: Childhood anxiety disorders, Cognitive behavior therapy, Comorbidity

Severity and Comorbidity as Predictors of CBT Outcome for Childhood Anxiety Disorders

A recent review supports the efficacy of Cognitive Behavioral Therapy (CBT) for Childhood Anxiety Disorders (CAD) and accords CBT the status of an empirically supported treatment (Cartwright-Hatton et al. 2004). However, a significant number of children (20 to 60 percent) that participate in research trials for CAD, do not show an adequate treatment response (Compton et al. 2002). Predictors of treatment response are of clinical and theoretical value (Rapee 2000) for identifying the mechanisms that facilitate or hinder treatment recovery. Two possible child predictors for treatment outcome for CAD are initial symptom severity and comorbidity.

In previous studies symptom severity has been associated with less favorable treatment outcomes in school-refusing youth, in which baseline school attendance represented severity (Layne et al. 2003). In another study the long-term outcome of child CBT for anxiety disorders was also predicted by severity as represented by the median split of the pretreatment scores on child-reported anxiety (Manassis et al. 2004). In a study of exposure-based CBT for phobic and anxiety disorders in youth mixed findings were reported: child-reported anxiety and depression severity predicted less favorable treatment outcome, but parent reported severity measures did not show any predictive value (Berman et al. 2000). In contrast, in a study of CBT for children with anxiety disorders parent reported child withdrawal symptoms predicted poor outcome while no associations were found with child-reported (e.g. anxiety and depression) severity measures (Southam-Gerow et al. 2001). Together, these results suggest that initial symptom severity is a good predictor of treatment outcome of CBT for CAD. However, we found no studies on the predictive value of measures of overall severity operationalized as total symptom count or total problem scores.

To date, most studies investigating the treatment outcome of CBT for CAD have not found an impact of diagnostic comorbidity on treatment recovery (Flannery-Schroeder et al. 2004; Kendall et al. 2001; Rapee 2003). One study found an association between comorbid depression and poor treatment response (Berman et al. 2000). Rapee (2003) found no differences in treatment response at post-treatment, and negligible differences at 6 months follow-up between three groups of anxious children and adolescents; an anxiety disorder and no comorbid disorder group, a more than one anxiety disorder group and an anxiety disorder and comorbid disorder other than anxiety (mood disorder or externalizing disorder) group. At 6 months follow-up there was a slight increase of father-reported internalizing symptoms and mother-reported externalizing symptoms in the children with other comorbidity, whereas children with one or more anxiety disorders and no non-anxiety comorbid disorders still showed a decline in internalizing symptoms.

While studies suggest that CBT is equally effective for anxiety disordered children with or without comorbid disorders, it is premature to assume that a standard brief CBT program used in research trials will work as well for complex cases such as those that present themselves in non-research settings (Southam-Gerow et al. 2008). In addition, comorbidity has repeatedly been suggested to negatively impact treatment processes (e.g., Kennard et al. 2005). For clinical practice it is important to assess whether comorbidity is a clinically significant co-occurrence (Starcevic 2005) with possible unique effects on treatment outcome. To our knowledge research examining the unique impact of comorbidity on treatment outcome for childhood anxiety disorders above and beyond the impact of severity is scarce.

In the present study we use a measure of overall severity (i.e., the Total Problem score of the CBCL that has been validated as a measure indicating caseness, see Kasius et al. (1997)). We further study the impact of both continuous and categorical outcome measures to examine the unique impact of comorbidity. Comorbidity and overall severity, though closely related, are for the purposes of this study viewed as distinct concepts, with comorbidity referring to the co-occurrence of two or more psychiatric disorders, and overall severity referring to the number of symptoms regardless of diagnostic status.

Two patterns of co-occurrence of psychiatric diagnoses are examined: total comorbidity, defined as any pattern of co-occurrence, and non-anxiety comorbidity, defined as the co-occurrence of one or more anxiety and one or more non-anxiety disorders. Total comorbidity reflects the broad definition of comorbidity as the co-occurrence of two or more psychiatric diagnoses (Starcevic 2005). Non-anxiety comorbidity originates from the proposal of Angold et al. (1999) to make a distinction between two types of comorbidity that tend to have different implications: homotypic comorbidity (i.e., comorbidity between disorders within a diagnostic grouping, such as anxiety disorders) and heterotypic comorbidity (i.e., comorbidity between disorders from different diagnostic groupings). As the high level of comorbidity of different anxiety disorders may reflect the possible artificiality of the subdivision of anxiety disorders into many different syndromes (Caron and Rutter 1991), we decided to focus on non-anxiety comorbidity, i.e. the heterotypic comorbidity of one or more anxiety disorders and one or more non-anxiety disorders. Total comorbidity in contrast reflects the number of disorders regardless the nature of the comorbid disorder(s).

It is important when studying predictors of treatment outcome to effectively evaluate the nature and degree of change that has occurred as a result of therapy. Estimation of pre- to post-treatment change is often inferred by calculating differences between pre and post-treatment scores on primary outcome measures, e.g., by using a repeated measures design including time. A serious drawback of using pre-post difference scores is that measurement-error (error-based regression to the mean) is not taken into account (Hageman and Arrindell 1999). Hageman and Arrindell therefore proposed procedures to establish Reliable Change and clinically significant change. Furthermore, recovery usually reflects the clinical post-treatment symptom-status given the pretreatment symptom status; e.g., a disorder that was present before treatment, is present (i.e., no recovery) or absent (recovery) at post treatment. If continuous outcome measures are used to estimate recovery, then both pre-post change and a cut-off score (dividing ‘normal’ from ‘abnormal’ scores) for post-treatment should be used (e.g. recovery is defined as: a significant pre-post change and a post-treatment score in the normal range). In recent years, it has become more common for treatment outcome researchers to report on clinically significant and meaningful change (i.e., Shortt et al. 2001; Silverman et al. 1999). Even these studies did not take measurement-error into account when using continuous measures in computing treatment recovery.

The purpose of the present study was to investigate the unique impact of pretreatment comorbidity on treatment outcome for childhood anxiety disorders above and beyond the impact of severity. The study focuses on the impact of total and non-anxious comorbidity and overall severity on outcome by addressing the following questions: (1) Does total and/or non-anxious comorbidity predict recovery above and beyond overall severity? (2) Does total and/or non-anxious comorbidity predict Reliable Change in self-reported anxiety and parent-reported internalizing symptoms above and beyond overall severity? (3) Does total and/or non-anxious comorbidity predict Reliable Change in non-anxiety symptoms (self-reported depressive symptoms and parent-reported externalizing symptoms) above and beyond overall severity?

Method

Participants

The present study is part of a larger treatment outcome study (see Liber et al. 2008a). Eligible for participation were children aged 8–12 years referred between September 2002 and December 2005 to the anxiety and depression outpatient clinic at the Child and Adolescent Psychiatry Department, Leiden University Medical Center and Erasmus Medical Center, Sophia Children’s Hospital in Rotterdam, in the Netherlands. Children diagnosed with a primary disorder of Separation Anxiety Disorder (SAD), Generalized Anxiety Disorder (GAD), Social Phobia (SOP) or Specific Phobia (SP) were eligible for participation. Children were excluded if they had an IQ below 85 or Pervasive Developmental Disorder. Children with primary Obsessive Compulsive Disorder and Posttraumatic Stress Disorder were also excluded. The reason for exclusion was that in previous research trials OCD and PTSD were not treated with the same CBT treatment protocol as other anxiety disorders, but rather CBT protocols specific for OCD or PTSD. Further, given that Panic Disorder is uncommon in childhood, it was also not included in the present study. As part of the routine procedure of both settings, all children and their parents were interviewed with the Anxiety Disorders Interview Schedule (ADIS-C/P; Silverman and Albano 1996).

A total of 142 children principally diagnosed with an Anxiety Disorder and their parents were asked to participate in the present study. Of these 142 children and their families, 133 subjects gave informed consent to participate. Children who received medication for ADHD (n = 5) were not excluded from the study. Dosage of medication was kept constant during the study as a constant dosage of medication for ADHD was considered unlikely to confound treatment effects. Effects of the medication for ADHD on anxiety were expected to be stabilized by giving children a constant dosage. We did not exclude these children in order to facilitate generalization to normal care. Given that there were only five children on medication the power was not sufficient to investigate if medication interacted with treatment.

Participants were randomly assigned in sequences of 6 to either group or individual treatment. Sixty-two children (36 boys and 26 girls) participated in the group treatment, and 71 children (38 boys and 33 girls) were individually treated. The primary diagnoses of the children were SAD (n = 53), GAD (n = 39), SOP (n = 22) or SP (n = 19). The social economic status (SES) was low for 19 children, medium for 61 children and high for 53 children (Central Bureau of Statistics Netherlands 2001). Sixteen children had a non-Dutch ethnic background: Turkish (n = 2), Moroccan (n = 3), Surinamese (n = 5) and other (n = 6).

Nine children dropped out of treatment, which resulted in a sample of 124 treatment completers. One hundred and twenty-three mothers and 108 fathers of children who completed the treatment also participated (for a description of the parents, see Liber et al. 2008b). Children who completed treatment or dropped out of treatment did not differ in the number of pretreatment disorders (t(131) = 0.57, p = ns).

Two types of comorbidity were defined, in line with one of the approaches to comorbidity described in the introduction. As defined by Starcevic (2005), a dichotomy of anxious children with no comorbid disorders (n = 55) and anxious children with one or more comorbid disorders (n = 69; total comorbidity) was created. According to the approach by Caron and Rutter (1991) a dichotomy of children with one or more anxiety disorders (n = 102) and a group of children with one or more anxiety disorder(s) and a non-anxiety comorbid disorder (n = 22) was created. There were no significant differences between these groups with regard to SES, age, gender or primary diagnosis. All results are based on the sample of children who completed the treatment unless otherwise specified. Diagnostic information and demographic data for the children who completed treatment are presented in Table 1.

Table 1.

Demographic Data on Diagnostic Groups

Variable   Total Comorbidity Non-anxiety comorbidity
Comorbid Groups Anx (n = 55) ComTot (n = 69) Anx-s (n = 102) Anx-Other (n = 22)
Gender
 Male 26 42 56 12
 Female 29 27 46 10
Age
 Mean (years) 10.06 10.07 10.02 10.30
 SD 1.24 1.32 1.28 1.34
SES
 Low 8 11 15 4
 Middle 21 36 46 11
 High 26 22 41 7
Primary Diagnosis
 SAD 21 29 39 11
 GAD 17 19 30 6
 SP 8 11 16 2
 SOP 9 10 17 3
Comorbid Diagnosis
 SAD – 8 5 3
 GAD – 24 17 7
 SP – 28 23 5
 SOP – 9 4 5
 Agoraphobia – 2 2 0
 ADHD – 12 – 12
 ODD – 6 – 6
 Depression – 2 – 2
 Dysthymia – 6 – 6
Open in a new tab

Anx single anxiety disorder, ComTot more than one disorder, Anx-s one or more anxiety disorders, Anx-Other one or more anxiety disorders and a co-occurring non-anxiety disorder. All group differences were nonsignificant at an alpha level of 0.05 (χ² and t-tests). SAD Separation Anxiety Disorder, GAD Generalized Anxiety Disorder, SP Specific Phobia, SOP Social Phobia, ADHD Attention Deficit Hyperactivity Disorder, ODD Oppositional Defiant Disorder.

Procedure

Children and parents were interviewed with the ADIS-C/P (Silverman and Albano 1996) and further assessed with the routine assessment procedure to confirm clinical diagnosis of the child. The routine procedure at both sites included at least one psychiatric consult, intelligence testing, and assessment of school and family functioning. Symptoms of anxiety and depression were assessed with the Multi-dimensional Anxiety Scale for Children (MASC; March 1997) and the Children’s Depression Inventory (CDI; Kovacs 1992). Clinical case conferences were held to exclude possible diagnoses not covered in the ADIS-C/P (i.e., Pervasive Developmental Disorder).

After the initial routine assessment verbal and written consent were obtained from the parents as well as children above age 11. Children were then randomly assigned to individual or group CBT (see Liber et al. 2008a). Prior to starting treatment parents and children were asked to complete additional questionnaires (see Measures). All children participating received a manual-based 10 session weekly cognitive behavioral treatment and their parents received 4 sessions of cognitive behavioral parent training (FRIENDS; Barrett and Turner 2000). One week post-treatment children and parents were again interviewed with the ADIS-C/P and post-treatment measures were administered to both children and parents. The present study is part of a larger study on a stepped care model investigating the effect of an additional treatment protocol for nonresponders to the initial CBT program used in the present study. Follow-up data could be affected by the additional treatment and are therefore not included in the present study.

Measures

Diagnostic Interview

The ADIS-C/P is a semi-structured interview schedule and was administered to both parents and children pre- and post-treatment to obtain clinical information and derive DSM-IV diagnoses (American Psychiatric Association 2000). The ADIS-C/P is a reliable instrument organized according to DSM-IV criteria and yields kappa coefficients for SAD, SOP, SP and GAD ranging from 0.62 to 0.92 for both the child and the parent interview (Silverman et al. 2001). The ADIS-C/P has strong evidence for providing reliable and valid diagnoses and sensitivity to clinical change in treatment outcome research (Silverman and Ollendick 2005). A Dutch translation of the ADIS-C/P was made in close consultation with the original authors and used in the present study (Siebelink and Treffers 2001). Guidelines for adequate translation and cross-cultural adaptation were followed as described in Van Widenfelt et al. 2005.

Experienced clinicians or master level students administered the ADIS-C/P pretreatment. Clinicians of both institutions met several times to ensure that the procedures and decision-making were alike. Master level students were trained by observing live and videotaped interviews and completed an exam to prove adequate administration of the interview. The training protocol and examination criteria can be obtained from the first author upon request. Supervision was provided and the reports were reviewed and discussed to ensure that administration, scoring and reporting would not drift.

Self-report Measures

Information on self-reported child anxiety, anxiety related and depressive symptoms were obtained by administering the Dutch versions of the MASC and CDI. The MASC is a 39 item general measure of anxiety translated into Dutch by Utens and Ferdinand (2000). The internal reliability of the total score (Cronbach’s alpha of 0.87 for boys and 0.88 for girls) and the test-retest reliability (3 months) are excellent (intra class correlation coefficient of 0.87 for the total score for children) (March et al. 1997, 1999). Reliability analyses of the Dutch version revealed a Cronbach’s alpha of 0.93 (N = 299, age 8–12) and a test-retest correlation of 0.81 for a school based population (n = 196, age 8–12).

The CDI is a 27-item scale suited for monitoring changes in a child’s mood (Kovacs 1992). A Dutch version of the CDI (Koot and van Widenfelt 2000) was used as a continuous measure of depressed mood in the present study. The CDI has good internal consistency (alphas ranging from 0.71 (outpatient group) to 0.89) and acceptable test-retest reliability (correlation of 0.75). The Dutch translation showed good psychometric properties, the Cronbach’s alpha was 0.82 for elementary school children (N = 649, age 8–12).

Parent-report Measures

The Child Behavior Checklist (CBCL) is a well-known and researched 113 item scale that assesses child behavior problems by parents (Achenbach and Rescorla 2001). Verhulst et al. (1996) translated the CBCL into Dutch and collected norm data on 1,422 clinically referred children. This resulted in a standardized questionnaire with sound psychometric properties for the scales ‘total behavior problems’, ‘externalizing problems’ and ‘internalizing problems’. Cronbachs alpha’s ranged for the above-mentioned scales for the clinical norm group from 0.72 to 0.87. For the analyses in the present study the internalizing problem scale (CBCL-Int) and the externalizing problem scale (CBCL-Ext) of the 2002 translation was used (Verhulst 2002). Norms for the Dutch 2002 version are not available yet. Cronbachs alpha for the CBCL-Int, CBCL-Ext and CBCL total, for fathers and mothers, ranged for the present study from 0.88 to 0.96. For analytic purposes a composite measure (“severity score”) was calculated consisting of the mean Z-scores of mother and father reported problem behavior (CBCL total score).

Treatment

Children were treated with the Dutch translation of the FRIENDS program (Barrett and Turner 2000; Utens et al. 2001). Results from previous research indicate that FRIENDS is an effective treatment for childhood anxiety disorders (Shortt et al. 2001). The FRIENDS program is based on the Coping Cat workbook (Kendall et al. 1990). FRIENDS is a manualized treatment based on a theoretical framework with three main target areas for change: physical symptoms, cognitive processes and coping skills. Children are taught coping techniques such as relaxation and breathing exercises to learn to cope with physical symptoms of anxiety. Children are also taught to challenge negative cognitions, irrational beliefs and negative self-talk by changing them into helpful cognitions, realistic beliefs and positive self-talk. Increased awareness of avoidant behavior is stimulated, as well as the development of problem solving and coping skills. In the second half of the treatment, gradual exposure is introduced. Attempts to cope constructively with anxiety are positively rewarded.

Children were assigned to either individual or group CBT by sequential randomization. No significant differences in outcome (free of any anxiety disorders at post treatment) between individual and group treatment were found (χ² (1, 124) = 0.55, p = 0.46) (Liber et al. 2008a).

Treatment Integrity

Adherence was checked as part of the treatment integrity measures. All therapy sessions were videotaped; a random selection of 30% of the tapes of individual sessions and all tapes of the group sessions were checked for adherence. (For the training procedure of the coders; see Liber et al. in press). Each session contained 6–12 therapeutic activities, coders watched entire therapy sessions and then rated how well the therapist met the aims of each activity on a four point Likert-scale ranging from 1 (extremely well) to 4 (not at all). Scores 1 and 2 were recoded into ‘met’ (1) and scores 3 and 4 were recoded into ‘not met’ (0) in order to calculate the percentage of adherence to the treatment protocol. It should be noted that adherence to the treatment protocol is an implementation check, and not an evaluation of the competence of the therapists. Percentages of ‘met’ were calculated per available session, and subsequently averaged over the available sessions per child. Results indicated that therapists adhered satisfactorily to the treatment protocol (94% of the child sessions and 85% of the parent sessions were provided as intended). Adherence rates for child sessions were higher than adherence rates for parent sessions (t(117) = 6.83, p < 0.01). The total and non-anxiety comorbidity groups did not differ with regard to adherence to the treatment protocol (total comorbidity child sessions; t(117) = 0.56, p = ns, total comorbidity parent sessions; t(114) = 1.08, p = ns, non-anxiety comorbidity child sessions; t(117) = 1.11, p = ns; non-anxiety comorbidity parent sessions; t(114) = 1.60, p = ns). Twenty-three therapists conducted the therapy sessions; six were doctoral students and 17 were licensed psychologists. Therapists at each institute met regularly to discuss the treatment and were supervised by two experienced licensed cognitive behavioral therapists. Every three to four months the therapists of the two institutions met to prevent therapist drift between institutions.

Data Analytic Strategy

Statistical Analyses

Regression analyses with hierarchical/ blockwise entry were performed to investigate whether comorbidity has an impact on treatment outcome above and beyond symptom severity. Regression methods are regarded as robust methods for use with clinical trials (Vickers 2005). Prediction models with recovery as outcome measure are conducted using logistic regression, prediction models with change as outcome measure are performed using multiple regression. As there is empirical evidence that severity does (Doss and Weisz 2006; Manassis et al. 2004), and comorbidity does not predict child anxiety CBT outcome (Kendall et al. 2001; Rapee 2003), severity is entered in the first block, and total and non-anxiety comorbidity are entered in the second block. Thereby investigating whether comorbidity has an impact on treatment outcome above and beyond symptom severity. As total and non-anxiety comorbidity are substantially related (Spearman’s r = 0.42, p < 0.001), regression analyses were run twice, with total comorbidity in the first model and non-anxiety comorbidity in a separate second model.

Reliable Change and Treatment Recovery

In the present study Reliable Change and Treatment Recovery are included as dependent variables. Change was determined by Reliable Change scores (RC scores). Reliable Change is calculated using a method that corrects for error-based regression to the mean (RC score). RC scores were computed for the CBCL-Int and Ext, the MASC and CDI. RC scores are used because they represent the most precise estimation of the true pre-post differences and are a more conservative approach than using the observed difference score (for a detailed description see Liber et al. 2008b). Negative RC scores reflect a reliable reduction in symptoms; improved RC score is defined by an RC score below−1.65. In the present study, treatment recovery is a dichotomy of ‘recovered’ versus ‘not/ partially recovered’. Treatment Recovery on the continuous anxiety measures (i.e., the MASC and the CBCL-Int) was defined as an improved RC score and a post score indicating ‘normal functioning’ as determined by the CSINDIV score. The CSINDIV score was computed using cutoff type c (see Hageman and Arrindell 1999). A CSINDIV score <−1.65 is used to conclude that the individual client has passed the cutoff for ‘normal’ functioning. Treatment Recovery on the CBCL-Int for mothers and father were combined: if the CBCL-Int RC score and CSINDIV score of one of the parents were both <−1.65, the outcome was considered successful (coded as 1) unless the CBCL-Int RC score of the other parent indicated deterioration, i.e. a RC score >1.65 (‘deteriorated’) (coded as 0). The CS of the CBCL Int scores for mothers and fathers showed a correlation of 0.67 (p < 0.001).

Treatment Recovery is also determined by post treatment diagnostic status (presence (1) or absence (0) of any anxiety disorder as assessed with the ADIS C/P). Treatment recovery based on diagnostic status indicates that children were free of any anxiety disorder at post-treatment according to the overall ADIS-C/P severity rating; child, parent and clinicians severity ratings were below threshold (below 4).

Treatment Recovery was determined for anxiety measures only, given the focus of the treatment. Reliable Change was calculated for anxiety measures as well as measures on externalizing and depressive symptoms to explore potential differential effects of the predictors. Treatment Recovery and Reliable Change are both presented as they function as complementary indices of treatment outcome; Treatment Recovery has a stronger focus on clinically relevant outcome.

Results

Pre-treatment Comparisons

To explore whether comorbidity and severity were related, biserial correlation coefficients were calculated. Biserial correlation coefficients revealed small to moderate correlation coefficients between total comorbidity and severity, and between non-anxiety comorbidity and severity (r b = 0.20, r b = 0.37 respectively).

Comorbidity and Severity as Predictors for Recovery on Anxiety Measures

Logistic regression analyses showed that recovery on the MASC, CBCL-Int and ADIS-C/P was significantly predicted by severity (see Table 2). The Odds Ratios (OR) indicate that recovery was less likely for children with higher levels of pretreatment overall severity. In the second block of the logistic regressions comorbidity status was entered: total comorbidity (step 2A) or non-anxiety comorbidity (step 2B). Total and non-anxiety comorbidity added significantly to the prediction of recovery on the ADIS-C/P. Children with non-anxiety and total comorbidity were less likely to have recovered. Adding non-anxiety comorbidity significantly increased the prediction of recovery on the MASC, with a growth of 6% of Nagelkerke’s R 2; a measure of pseudo-variance that ranges from 0 to 1. However, the OR of severity was no longer significant and the OR of non-anxiety comorbidity only approached significance (p < 0.10), indicating that severity and non-anxiety comorbidity are overlapping predictors of recovery on the MASC.

Table 2.

Logistic Regression Predicting Recovery on MASC, CBCL-Int and ADIS-C/P

Variables B OR (Exp(B)) 95% CI of OR χ 2 Step Nagk R 2 Model
Analysis for MASC
 Step 1 6.12* 0.07
  Severity* −0.53 0.59 0.38–0.91
 Step 2A 0.27 0.07
  Severity* −0.51 0.60 0.39–0.94
  Total Comorbidity −0.21 0.81 0.38–1.77
 Step 2B 4.12* 0.11
  Severity −0.38 0.69 0.43–1.09
  NAC+ −1.27 0.28 0.07–1.07
Analysis for CBCL-Int#
 Step 1 9.55** 0.14
  Severity** −1.04 0.35 0.17–0.75
 Step 2A 2.99+ 0.18
  Severity** −1.18 0.31 0.14–0.68
  Total Comorbidity 1.03 2.79 0.83–9.36
Analysis for ADIS-C/P
 Step 1 22.06*** 0.22
  Severity*** −1.07 0.34 0.21–0.57
 Step 2A 6.92* 0.29
  Severity*** −1.00 0.37 0.22–0.61
  Total Comorbidity* −1.08 0.34 0.15–0.76
 Step 2B 5.37* 0.27
  Severity*** −0.88 0.41 0.25–0.70
  NAC* −1.63 0.20 0.04–0.95
Open in a new tab

MASC Multidimensional Anxiety Scale for Children; CBCL-Int Child Behavior Checklist, Internalizing Scale; ADIS-C/P Anxiety Disorders Interview Schedule, Child & Parent;   NAC non-anxiety comorbidity; MASC, CBCL and ADIS-C/P recovery = 1, no recovery = 0; Non-anxiety comorbidity and total comorbidity; present = 1, absent = 0; B = Logistic Coefficient; OR = Odds Ratio; Nagk R 2 = Nagelkerke R 2; Zero to five outliers were removed from a model;

# Step 2B not realized because of a low expected number of recovered cases with non-anxiety comorbidity (as only 16% of all cases was recovered on CBCL-Int, the expected number recovered cases with non-anxiety comorbidity was 3.2). + p < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001.

Comorbidity and Severity as Predictors for Reliable Change on Anxiety Measures

Multiple regression analyses with severity (in the first step) and comorbidity status (second steps) as predictor variables did not show any significant association between the predictors and Reliable Change on the MASC. As can be seen in Table 3, severity was a significant predictor of Reliable Change in CBCL-Internalizing symptoms. Entering non-anxiety comorbidity (in Step 2B) resulted in a better prediction with a higher absolute beta value of severity. The higher absolute value indicates a suppressor situation (see e.g. Paulhus et al. 2004). In addition to testing the significance of R 2Δ, we used the Sobel test to evaluate the significance of the difference between the beta values of severity (see MacKinnon et al. 2000). Identical with the result of testing R 2Δ, we found that the negative effect of severity was significantly more negative after non-anxiety comorbidity was included in the equation: z = 2.06, p < 0.05. As the correlation of non-anxiety comorbidity with Reliable Change in CBCL-Internalizing is near zero (r = 0.06), this suppressor situation has been called classical suppression. Classical suppression implies mutual suppression (Paulhus et al. 2004), i.e. the stronger prediction found by combining both predictors indicates that non-anxiety comorbidity removes irrelevant variance from severity and vice versa. In a configuration with a classical suppressor variable both predictors suppress some of each others irrelevant variance. The removal of this ‘contamination’ may result in more ‘pure’ measures. As indicated by the negative beta value of severity and the positive beta value of non-anxiety comorbidity, children with more parent reported overall severity are more likely to show greater pre to post-treatment changes, but to a lesser extent if they suffer from a comorbid disorder other than anxiety.

Table 3.

Multiple Regression Predicting Reliable Change on CDI, CBCL-Int and CBCL-Ext

Variables B SE(B) β R 2Δ R 2 Model
Analysis for CBCL-Int#
 Step 1 0.15*** 0.15***
  Severity*** −0.56 0.13 −0.39
 Step 2B 0.04* 0.19***
  Severity*** −0.70 0.14 −0.48
  NAC* 0.76 0.33 0.22
Analysis for CDI#
 Step 1 0.07** 0.07**
  Severity** −0.20 0.07 −0.27
 Step 2B 0.06** 0.13***
  Severity −0.12 0.08 −0.15
  NAC** −0.53 0.19 −0.27
Analysis for CBCL-Ext#
 Step 1 0.16*** 0.16***
  Severity*** −0.56 0.12 −0.40
 Step 2B 0.06** 0.22***
  Severity*** −0.70 0.12 −0.50
  NAC** 0.97 0.31 0.28
Open in a new tab

NAC non-anxiety comorbidity; CDI Children’s Depression Inventory; CBCL Child Behavior Checklist, Int Internalizing Scale and Ext Externalizing Scale; Non-anxiety comorbidity and total comorbidity: present = 1, absent = 0. Three to five outliers were removed from a model; + p < 0.10; *: p < 0.05; **: p < 0.01; ***: p < 0.001; # Step 2A not computed because no significant association was found with total comorbidity.

Severity and Comorbidity as Predictors for Reliable Change in Depressive and Externalizing Symptoms

Multiple regression analyses were used to predict reliable change in depressive and externalizing symptoms. Again, severity was entered in the first step and total comorbidity and non-anxiety comorbidity respectively in the second step. Severity significantly predicted Reliable Change in self-reported depressive symptoms (CDI) and in parent reported externalizing problems (CBCL-Ext, see Table 3). The negative beta-values for the severity scores in the prediction of reliable change on the CDI and CBCL-Ext scales indicate more improvement (e.g., negative RC values) for children with higher levels of severity.

The addition of ‘non-anxiety comorbidity’ significantly increased the prediction of reliable change on the CDI and the CBCL-Ext. Interestingly, severity was no longer a significant predictor of the CDI after entering ‘non-anxiety comorbidity’ (step 2B), thereby indicating that ‘non-anxiety comorbidity’ explained all of the variance in CDI improvement associated with severity.

The higher beta of severity in predicting improvement in CBCL-Ext at step 2B, indicates that ‘non-anxiety comorbidity’ could be a suppressor variable. This was affirmed by the significant value of R2Δ and by the result of the Sobel test: z = 2.55, p < 0.05, indicating that the negative effect of severity was significantly more negative after non-anxiety comorbidity was included in the equation. Non-anxiety comorbidity is again a classical suppressor as near-zero correlation (r = 0.10) was found between this predictor and the dependent variable. So again, a stronger predictive value of both predictors was found, indicating the removal of irrelevant variance from overall severity by non-anxiety comorbidity and vice versa. Change in pre- to post treatment symptoms is more likely for children with a higher level of overall severity, but to a lesser extent if they suffer from a comorbid disorder other than anxiety. Treatment format did not improve the models1.

Discussion

In the present study, children with a higher level of pre-treatment symptom severity were less likely to have recovered at post-treatment. Further, children with any comorbid disorder (total comorbidity) at pre-treatment were less likely to be free of a DSM-IV anxiety disorder at post-treatment. Pre-treatment non-anxiety comorbidity added to the prediction of recovery when recovery was assessed with child reported anxiety symptoms. Thus both increased symptom severity and having a nonanxiety comorbid disorder at pretreatment predicts recovery when outcome is assessed using self-reports of anxiety. Total comorbidity did not add to the prediction of recovery using child-reported anxiety nor parent-reported internalizing symptoms over and above the contribution of symptom severity.

Children with a higher level of pretreatment severity need greater decreases in symptoms to reach a subthreshold level of symptoms, therefore we also examined pre to posttreatment Reliable Change. Severity and comorbidity did not contribute to the prediction of Reliable Change when based on self-reported anxiety symptoms. Though children with a higher level of overall severity and non-anxiety comorbidity are less likely to show clinically significant recovery in terms of clinician rated DSM-IV diagnoses, they are equally likely to gain from the treatment when children are asked to report on anxiety symptoms. Conversely, severity predicted greater Reliable Change in parent reported internalizing and externalizing symptoms, and child reported depressive symptoms. Having a non-anxiety comorbidity added to this prediction when assessing parent-reported internalizing, externalizing and child-reported depressive symptoms, but in different ways. Non-anxiety comorbidity was the dominant predictor for Reliable Change in self-reported depressive symptoms, incorporating the predictive value of severity. In the predictions of Reliable Change of parent-reported internalizing and externalizing child symptoms, non-anxiety comorbidity could be classified as a classical suppressor variable. Severity is strongly associated with more improvement when irrelevant variance in common with non-anxiety comorbidity is removed, therefore researchers should consider removing the irrelevant variance of parent-reported pre-treatment overall severity when considering relations between non-anxious comorbidity and parent reported change in internalizing and externalizing problems.

The results indicate the importance of assessing recovery both in terms of treatment recovery and Reliable Change scores, as well as the investigation of treatment outcome from a multi-informant perspective. Similar to Doss and Weisz (2006), we found higher initial severity to be predictive of both greater treatment gains (reliable change) and higher levels of remaining symptoms. In line with previous studies we found only a modest predictive value of total comorbidity on treatment outcome (i.e., Flannery-Schroeder et al. 2004; Kendall et al. 2001; Rapee 2003). However, the presence of a non-anxiety comorbid disorder at pretreatment added significantly to predictions of treatment recovery based on a clinical interview and child-reported anxiety symptoms and for Reliable Change on depressive, internalizing and externalizing symptoms.

The current findings may help identify those children that need additional treatment after a basic CBT program. Children with a higher level of overall severity might benefit from a stepped care approach (Bower and Gilbody 2005) in which additional treatment is necessary to reach the subthreshold level of symptoms. A generic program might be sufficient, as children with higher overall severity tend to show greater changes with such programs. If they also suffer from a non-anxiety comorbid disorder then they might benefit from a combined, modular or prescriptive treatment (Chorpita et al. 2004) in which non-anxiety problems are also targeted. Future studies on stepped care, modular or prescriptive treatment could evaluate strategies targeting anxiety as well as strategies targeting comorbid problems for these children.

There might be a positive spin-off of this generic anxiety-focused CBT treatment leading to a decrease in co-occurring depressive symptoms and externalizing symptoms. A greater change in depressive symptoms was associated with the presence of having another disorder at pretreatment. Similarly, another treatment outcome study also reported greater decreases in depressive scores in depressed adolescents with pretreatment comorbid anxiety disorders. These adolescents had more room for improvement as they reported significantly higher depression scores at intake (Rohde et al. 2001). We did find an effect of non-anxiety comorbid disorders on externalizing symptoms; non-anxiety comorbidity did have a negative impact on change in externalizing symptoms in anxious children when suppressor effects were taken into account. A recent study evaluated the effectiveness of an intervention for anxiety disorders and comorbid aggression (Levy et al. 2007). Children were treated with either a CBT program for anxiety only or a CBT intervention targeting both anxiety and comorbid aggression. Both interventions led to significant reductions in externalizing and internalizing parent reported problem behavior, and the combined treatment program did not show a higher effectiveness. Unfortunately, the study did not assess severity separately. Similar to Levy et al. we found no associations between the presence of non-anxiety comorbidity and reductions of internalizing and externalizing parent-reported problems with anxiety-focused CBT. However, we also investigated the combined prediction of comorbidity and overall severity and found significant suppressor effects, indicating that ‘pure’ non-anxiety comorbidity has a negative impact on reduction of internalizing and externalizing problems. In addition, there is sufficient evidence from meta-analyses that certain treatments are effective for certain types of problems, e.g. parent training for externalizing problems or exposure for phobia (Chambless and Ollendick 2001). Investigation of the combined impact of comorbidity and overall severity on the potency of treatments for various primary diagnoses may inform us how to enhance treatments for primary disorders as well as comorbid conditions.

Important assets of the present study are the multi-informant perspective on treatment outcome, assessment of changes not only in anxiety but also in anxiety-related and comorbid symptoms, use of a clinically meaningful method to define treatment recovery and computation of reliable pre to post-treatment changes.

For the present study, children with comorbid conditions other than anxiety were treated as one group, whereas the investigation of children with comorbid externalizing disorders versus comorbid affective disorders could also prove fruitful. Comorbidity of anxiety with an affective disorder or comorbidity with an externalizing disorder may be more common in community samples and therefore of clinical value to investigate. As only 17% of the children in the current study showed a comorbid disorder other than anxiety, the power did not permit a further splitting of this group. Moreover, the sample of children with a comorbid disorder other than anxiety (n = 22) included 4 children with both a mood disorder and an externalizing disorder. Though the sample size in the present study is quite large, it was designed to investigate main effects and not designed for interaction effects. An adequate test of an interaction requires an even larger sample size than for detecting a simple main effect (Shoham-Salomon and Hannah 1991). These considerations point at the need for an adequately powered trial testing the mechanisms through which comorbidity might interact with treatment.

A further limitation of the present study is that we only assessed at post-treatment the presence or absence of anxiety disorders, and not the absence or presence of affective or externalizing disorders. Assessing comorbid diagnoses could give further insight into the impact of comorbidity on the treatment process. Kendall et al. (2001) found that children with the continued presence of comorbidity were less likely to recover from their primary pretreatment diagnosis. Additionally, five children were on a consistent dose of medication for ADHD.It is not known whether the negative impact of comorbid conditions on anxiety and externalizing symptoms or the positive impact on depressive symptoms would have been stronger if children had not been on medication for ADHD. However, it was deemed unethical to withhold medication from children with ADHD for research purposes.

It can be argued that severity should not be investigated using a sole additive construct with symptoms of one sort counting the same as that of another, i.e., itching and suicidal ideation. However, differentiating between symptoms and prioritizing some above others would be difficult to calculate and not be representative of the severity construct. Future studies could incorporate an impairment measure such as the DSM global functioning scale in addition to parent reported overall severity.

In sum, severity predicted less favorable child, parent and clinician reported outcomes but greater pre- to post-treatment changes in parent-reported internalizing symptoms. Non-anxiety comorbidity added significantly to these predictions in a number of ways: as a separate predictor (in predicting recovery on ADIS), as predictor overlapping with severity (in predicting recovery on the MASC), as suppressor variable (in predicting Reliable Change on internalizing and externalizing CBCL-scales) and as dominant predictor (in predicting Reliable Change on the CDI). Future studies but also clinicians should take into account the potential impact of symptom severity on outcome. As severity and comorbid disorders only explain part of the variance in outcome a further understanding of valid predictors is essential to understand who does not benefit sufficiently from CBT for CAD. Replication of the current findings with severity and non-anxiety comorbidity as separate constructs are necessary to further our understanding on how symptoms and comorbid patterns of problem behavior change with anxiety focused CBT.

Acknowledgements

This study was financed in part by the Netherlands Foundation for Mental Health, Utrecht. We would like to thank all children and their parents for their participation in this research project. We also thank the clinicians and therapists of the Anxiety and Depression Unit of Curium-LUMC and the Sophia Children’s Hospital/ Erasmus MC for their support and participation.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Footnotes

1

Children were randomly assigned to individual CBT or group CBT. Though previous analyses showed no differences in outcome between these formats (Liber et al. 2008a), interaction effects for comorbidity x treatment format were not yet examined. Therefore all analyses were rerun including interaction effects of our initial predictors with treatment format. None of the models improved by including treatment format.

References

  1. Achenbach TM, Rescorla LA. Manual for the ASEBA school-aged forms & profiles. Burlington: University of Vermont, Research Centre for Children, Youth and Families; 2001. [Google Scholar]
  2. American Psychiatric Association . Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington: Author; 2000. [Google Scholar]
  3. Angold A, Costello JE, Erkanli A. Comorbidity. Journal of Child Psychology and Psychiatry, and Allied Disciplines. 1999;40:57–87. doi: 10.1111/1469-7610.00424. [DOI] [PubMed] [Google Scholar]
  4. Barrett PM, Turner C. Friends for children: Group leader’s manual. Bowen Hills: Australian Academic Press; 2000. [Google Scholar]
  5. Berman SL, Weems CF, Silverman WK, Kurtines WM. Predictors of outcome in exposure-based cognitive and behavioral treatments for phobic and anxiety disorders in children. Behavior Therapy. 2000;31:713–731. doi: 10.1016/S0005-7894(00)80040-4. [DOI] [Google Scholar]
  6. Bower P, Gilbody S. Stepped care in psychological therapies: access, effectiveness and efficiency - Narrative literature review. British Journal of Psychiatry. 2005;186:11–17. doi: 10.1192/bjp.186.1.11. [DOI] [PubMed] [Google Scholar]
  7. Caron C, Rutter M. Comorbidity in child psychopathology: concepts, issues and research strategies. Journal of Child Psychology and Psychiatry. 1991;32:1063–1080. doi: 10.1111/j.1469-7610.1991.tb00350.x. [DOI] [PubMed] [Google Scholar]
  8. Cartwright-Hatton S, Roberts C, Chitsabesan P, Fothergill C, Harrington R. Systematic review of the efficacy of cognitive behaviour therapies for childhood and adolescent anxiety disorders. British Journal of Psychiatry. 2004;43:421–436. doi: 10.1348/0144665042388928. [DOI] [PubMed] [Google Scholar]
  9. Central Bureau of Statistics Netherlands . Standaard Beroepenclassificatie 1999 (Standardized Classification of Occupations) Voorburg/Heerlen: Statistics Netherlands; 2001. [Google Scholar]
  10. Chambless DL, Ollendick ThH. Empirically supported psychological interventions: controversies and evidence. Annual Review of Psychology. 2001;52:685–716. doi: 10.1146/annurev.psych.52.1.685. [DOI] [PubMed] [Google Scholar]
  11. Chorpita BF, Taylor AA, Francis SE, Moffitt C, Austin AA. Efficacy of modular cognitive behavior therapy for childhood anxiety disorders. Behavior Therapy. 2004;35:263–287. doi: 10.1016/S0005-7894(04)80039-X. [DOI] [Google Scholar]
  12. Compton SN, Burns BJ, Egger HL, Robertson E. Review of the evidence base for treatment of childhood psychopathology: Internalizing disorders. Journal of Consulting and Clinical Psychology. 2002;70:1240–1266. doi: 10.1037/0022-006X.70.6.1240. [DOI] [PubMed] [Google Scholar]
  13. Doss AJ, Weisz JR. Syndrome co-occurrence and treatment outcomes in youth mental health clinics. Journal of Consulting and Clinical Psychology. 2006;74:416–425. doi: 10.1037/0022-006X.74.3.416. [DOI] [PubMed] [Google Scholar]
  14. Flannery-Schroeder E, Suveg C, Safford S, Kendall PC, Webb A. Comorbid externalizing disorders and child anxiety treatment outcomes. Behaviour Change. 2004;21:14–25. doi: 10.1375/bech.21.1.14.35972. [DOI] [Google Scholar]
  15. Hageman WJ, Arrindell WA. Clinically significant and practical! Enhancing precision does make a difference. Reply to McGlinchey and Jacobson, Hsu, and Speer. Behavior Research and Therapy. 1999;37:1219–1233. doi: 10.1016/S0005-7967(99)00036-4. [DOI] [PubMed] [Google Scholar]
  16. Kasius MC, Ferdinand RF, Van den Berg H, Verhulst FC. Associations between different diagnostic approaches for child and adolescent psychopathology. Journal of Child Psychology and Psychiatry. 1997;38:625–632. doi: 10.1111/j.1469-7610.1997.tb01689.x. [DOI] [PubMed] [Google Scholar]
  17. Kendall, P. C., Kane, M., Howard, B., & Siqueland, L. (1990). Cognitive-behavioral therapy for anxious children: Treatment manual. Philadelphia: Department of Psychology: Temple University.
  18. Kendall PC, Brady EU, Verduin TL. Comorbidity in childhood anxiety disorders and treatment outcome. Journal of the American Academy of Child & Adolescent Psychiatry. 2001;40:787–794. doi: 10.1097/00004583-200107000-00013. [DOI] [PubMed] [Google Scholar]
  19. Kennard BD, Ginsburg GS, Feeny NC, Sweeney M, Zagurski R. Implementation challenges to TADS cognitive-behavioral therapy. Cognitive and Behavioral Practice. 2005;12:230–239. doi: 10.1016/S1077-7229(05)80028-8. [DOI] [Google Scholar]
  20. Koot HM, Van Widenfelt BM. Dutch translation of Kovac’s children’s depression inventory. Oegstgeest: Curium-LUMC; 2000. [Google Scholar]
  21. Kovacs M. Children’s Depression Inventory (CDI) manual. Toronto: Multi-Health System; 1992. [Google Scholar]
  22. Layne AE, Bernstein GA, Egan EA, Kushner MG. Predictors of treatment response in anxious-depressed adolescents with school refusal. Journal of the American Academy of Child and Adolescent Psychiatry. 2003;42:319–326. doi: 10.1097/00004583-200303000-00012. [DOI] [PubMed] [Google Scholar]
  23. Levy K, Hunt C, Heriot S. Treating comorbid anxiety and aggression in children. Journal of the American Academy of Child & Adolescent Psychiatry. 2007;46:1111–1118. doi: 10.1097/chi.0b013e318074eb32. [DOI] [PubMed] [Google Scholar]
  24. Liber JM, Van Widenfelt BM, Utens EMWJ, Ferdinand RF, Van der Leeden AJM, Van Gastel W, Treffers PhDA. No Differences between group versus individual treatment of childhood anxiety disorders in a randomized clinical trial. Journal of Child Psychology and Psychiatry. 2008a;49:886–893. doi: 10.1111/j.1469-7610.2008.01877.x. [DOI] [PubMed] [Google Scholar]
  25. Liber JM, Van Widenfelt BM, Goedhart AW, Utens EMWJ, Van der Leeden AJM, Markus MT, Treffers PhDA. Parenting and parental anxiety and depression as predictors of treatment outcome for childhood anxiety disorders: Has the role of fathers been underestimated? Journal of Clinical Child and Adolescent Psychology. 2008b;37:747–758. doi: 10.1080/15374410802359692. [DOI] [PubMed] [Google Scholar]
  26. Liber JM, McLeod BD, Van Widenfelt BM, Goedhart AW, Van der Leeden AJM, Utens EMWJ, Treffers PhDA. Examining the relation between the therapeutic alliance, treatment adherence, and outcome of cognitive behavioral treatment for children with anxiety disorders. Behavior Therapy. 2009 doi: 10.1016/j.beth.2009.02.003. [DOI] [PubMed] [Google Scholar]
  27. MacKinnon DP, Krull JL, Lockwood CM. Equivalence of the mediation, confounding, and suppression effect. Prevention Science. 2000;1:173–181. doi: 10.1023/A:1026595011371. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Manassis K, Avery D, Butalia S, Mendlowitz SL. Cognitive-behavioral therapy with childhood anxiety disorders: functioning in adolescence. Depression and Anxiety. 2004;19:209–216. doi: 10.1002/da.10133. [DOI] [PubMed] [Google Scholar]
  29. March JS. Multidimensional Anxiety Scale for children: Technical Manual. Toronto: Multi-Health Systems Inc.; 1997. [Google Scholar]
  30. March JS, Parker JDA, Sullivan K, Stallings P, Conners K. The Multidimensional Anxiety Scale for Children (MASC): factor structure, reliability, and validity. Journal of the American Academy of Child & Adolescent Psychiatry. 1997;36:554–565. doi: 10.1097/00004583-199704000-00019. [DOI] [PubMed] [Google Scholar]
  31. March JS, Sullivan K, Parker JDA. Test-retest reliability of the MULTIDIMENSIONAL ANXIETY SCALE for Children. Journal of Anxiety Disorders. 1999;13:349–358. doi: 10.1016/S0887-6185(99)00009-2. [DOI] [PubMed] [Google Scholar]
  32. Paulhus DL, Robins RW, Trzesniewski KH, Tracy JL. Two replicable suppressor situations in personality research. Multivariate Behavioral Research. 2004;39:303–328. doi: 10.1207/s15327906mbr3902_7. [DOI] [PubMed] [Google Scholar]
  33. Rapee RM. Group treatment of children with anxiety disorders: outcome and predictors of treatment response. Australian Journal of Psychology. 2000;52:125–129. doi: 10.1080/00049530008255379. [DOI] [Google Scholar]
  34. Rapee RM. The influence of comorbidity on treatment outcome for children and adolescents with anxiety disorders. Behavior Research and Therapy. 2003;41:105–112. doi: 10.1016/S0005-7967(02)00049-9. [DOI] [PubMed] [Google Scholar]
  35. Rohde P, Clarke GN, Lewinsohn PM, Seeley JR, Kaufman NK. Impact of comorbidity on a cognitive-behavioral group treatment for adolescent depression. Journal of the American Academy of Child & Adolescent Psychiatry. 2001;40:795–802. doi: 10.1097/00004583-200107000-00014. [DOI] [PubMed] [Google Scholar]
  36. Shoham-Salomon V, Hannah MT. Client-treatment interaction in the study of differential change processes. Journal of Consulting and Clinical Psychology. 1991;59:217–225. doi: 10.1037/0022-006X.59.2.217. [DOI] [PubMed] [Google Scholar]
  37. Shortt AL, Barrett PM, Fox TL. Evaluating the FRIENDS program: a cognitive-behavioral group treatment for anxious children and their parents. Journal of Clinical Child Psychology. 2001;30:525–535. doi: 10.1207/S15374424JCCP3004_09. [DOI] [PubMed] [Google Scholar]
  38. Siebelink BM, Treffers PhDA. ADIS-C anxiety disorders interview schedule for DSM-IV (Dutch Version) Lisse: Swets & Zeitlinger BV; 2001. [Google Scholar]
  39. Silverman WK, Albano AM. The clinician manual for the anxiety disorders interview schedule for DSM-IV. San Antonio: The Psychological Corporation; 1996. [Google Scholar]
  40. Silverman WK, Ollendick TH. Evidence-based assessment of anxiety and its disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology. 2005;34:380–411. doi: 10.1207/s15374424jccp3403_2. [DOI] [PubMed] [Google Scholar]
  41. Silverman WK, Kurtines WM, Ginsburg GS, Weems CF, Rabian B, Serafini LT. Contingency management, self-control, and education support in the treatment of childhood phobic disorders: a randomized clinical trial. Journal of Consulting and Clinical Psychology. 1999;67:675–687. doi: 10.1037/0022-006X.67.5.675. [DOI] [PubMed] [Google Scholar]
  42. Silverman WK, Saavedra LM, Pina AA. Test-retest reliability of anxiety symptoms and diagnoses with the Anxiety Disorders Interview Schedule for DSM-IV: child and parent versions. Journal of the American Academy of Child & Adolescent Psychiatry. 2001;40:937–944. doi: 10.1097/00004583-200108000-00016. [DOI] [PubMed] [Google Scholar]
  43. Southam-Gerow M, Kendall PC, Weersing RV. Examining outcome variability: correlates of treatment response in a child and adolescent anxiety clinic. Journal of Clinical Child Psychology. 2001;30:422–436. doi: 10.1207/S15374424JCCP3003_13. [DOI] [PubMed] [Google Scholar]
  44. Southam-Gerow MA, Chorpita BF, Miller LM, Gleacher AA. Are children with anxiety disorders privately referred to a university clinic like those referred from the public mental health system? Administration and Policy in Mental Health. 2008;35:168–180. doi: 10.1007/s10488-007-0154-7. [DOI] [PubMed] [Google Scholar]
  45. Starcevic V. Psychiatric comorbidity: concepts, controversies and alternatives. Australasian Psychiatry. 2005;13:375–378. doi: 10.1080/j.1440-1665.2005.02216.x. [DOI] [PubMed] [Google Scholar]
  46. Utens EMWJ, Ferdinand RF. Multidimensional Anxiety Scale for Children (Dutch translation) Rotterdam: Department of Child and Adolescent Psychiatry Erasmus MC- Sophia Childrens Hospital Rotterdam; 2000. [Google Scholar]
  47. Utens EMWJ, De Nijs P, Ferdinand RF. FRIENDS for children- manual for group leaders (Dutch translation) Rotterdam: Department of Child and Adolescent Psychiatry Erasmus MC- Sophia Childrens Hospital Rotterdam; 2001. [Google Scholar]
  48. Van Widenfelt BM, Treffers PhD, De Beurs E, Siebelink BM, Koudijs E. Translation and cross-cultural adaptation of assessment instruments used in psychological research with children and families. Clinical Child and Family Psychology Review. 2005;8:135–147. doi: 10.1007/s10567-005-4752-1. [DOI] [PubMed] [Google Scholar]
  49. Verhulst FC. Child behavior checklist for children aged 6–18(Dutch translation of the Child Behavior Checklist 6–18) Rotterdam: Academisch Ziekenhuis Rotterdam/ Erasmus Universiteit Rotterdam; 2002. [Google Scholar]
  50. Verhulst FC, Van der Ende J, Koot HM. Manual for the child behavior checklist (in Dutch) Rotterdam: Department of Child and Adolescent Psychiatry, Erasmus Medical Centre/Sophia; 1996. [Google Scholar]
  51. Vickers AJ. Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches. Psychosomatic Medicine. 2005;67:652–655. doi: 10.1097/01.psy.0000172624.52957.a8. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Abnormal Child Psychology are provided here courtesy of Springer

RESOURCES