Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Feb 18.
Published in final edited form as: J Clin Child Adolesc Psychol. 2011;40(4):607–617. doi: 10.1080/15374416.2011.581618

Longitudinal Relations Between Depressive Symptoms and Externalizing Behavior in Adolescence: Moderating Effects of Maltreatment Experience and Gender

Matthew Brensilver 1, Sonya Negriff 2, Ferol E Mennen 2, Penelope K Trickett 2
PMCID: PMC5316287  NIHMSID: NIHMS850067  PMID: 21722032

Abstract

Among the explanations for the high rates of co-occurrence between depressive symptoms and externalizing behavior is the possibility of direct causal associations between the two symptom groups. However, the mechanisms by which co-occurrence arises may not be the same across etiologically significant variables. A gender-balanced sample of 303 adolescents (ages 9–12 at the first assessment) with carefully assessed histories of maltreatment experience and 151 demographically matched nonmaltreated adolescents were assessed over the period of 1 year. Multiple-group cross-lagged panel analyses assessed the equivalence of longitudinal relations between depressive symptoms and externalizing behavior for gender/maltreatment status groups. Consistent with previous findings, the results suggest that girls, particularly maltreated girls, who exhibit early externalizing behavior are at high risk for the development of subsequent depressive symptoms.

Understanding the etiology, course, and correlates of adolescent psychopathology is important for enhancing treatment options. Ideally, interventions should target the mechanisms by which psychopathology is maintained in developmentally sensitive ways. Many interventions for adolescents represent downward extensions of intervention models developed for adults. As such, it is particularly relevant to characterize distinctive properties of adolescent psychopathology so that interventions can be optimally targeted.

One important characteristic of psychopathology that is evident during youth is comorbidity or covariation between different clusters of symptoms. Most investigations of the co-occurrence of symptoms have employed categorical models of psychopathology, despite evidence to suggest dimensional distributions of psychopathological symptoms (Watson, 2005). In both categorical and dimensional conceptualizations of psychopathology, however, patterns of comorbidity emerge. This phenomenon has been documented not only in clinical samples, where co-occurring diagnoses are more likely to be present, but also in community samples. A meta-analysis of general population studies conducted by Angold, Costello, and Erkanli (1999) found that the existence of depression or conduct disorder in youth confers substantial risk for the other disorder with an odds ratio of 6.6. In a clinical sample of adolescent boys, Lahey, Loeber, Burke, Rathouz, and McBurnett (2002) found an average correlation of .35 between symptoms of the two disorders across six waves of data. Greene et al. (2002) suggested that between one third and one half of a clinical sample with oppositional defiant disorder met criteria for major depression.

Almost two decades ago, Clarkin and Kendall (1992) called comorbidity the “premiere mental health challenge.” Subsequent research has begun to address this issue, but Jensen (2003) highlighted the dearth of extant research. Understanding the sources of comorbidity has important implications for understanding etiology, course, prevention, and treatment of psychopathology (Biederman, Faraone, Mick, & Lelon, 1995). Different mechanisms explaining comorbidity carry distinct implications for intervention and research programs. Several competing explanations for the phenomena have been offered, although it is possible that multiple explanatory factors operate.

One prominent explanation for the phenomenon has been termed pathogenic comorbidity. This refers to the hypothesis that early symptomatology compromises functioning, which in turn leads to the emergence of a new constellation of symptoms (Klein & Riso, 1993). A causal explanation would require that (a) one set of symptoms precedes the other, (b) there is an association between the symptom clusters, and (c) this association is nonspurious. The pathogenic hypothesis suggests that depressive symptoms, or externalizing behavior, act directly as a causal factor for the other disorder. Of course, the relationship may be mediated by a complex circuit of causal mechanisms, but the most stringent version of the hypothesis implies that the second disorder or set of symptoms would not emerge in the absence of the first. There is some theoretical precedent to suggest such a scenario, particularly for the causal effect of externalizing behavior on depression. Externalizing behavior is associated with various functional impairments including academic underachievement (Hinshaw, 1992; Masten et al., 2005), lower social competence (Renouf, Kovacs, & Mukerji, 1997), and peer rejection (Dodge, 1983). These developmental impairments are, in turn, predictors of depression.

This research was synthesized in Capaldi’s (1991, 1992) “failure model,” which suggested that conduct problems were prospectively related to depressive symptoms. In this model, poor peer and parental relationships and academic failures mediate the relation between conduct problems and depression. Little and Garber (2005) assessed mediational models linking externalizing behavior and depression. They hypothesized that externalizing behavior, defined by Achenbach’s (1991) broadband cluster of symptoms, generates dependent social stressors. Dependent social stress refers to stressful life events that are catalyzed by the individual’s own behavior (i.e., interpersonal conflicts). In a gender-balanced sample, they found that the prospective relationship between externalizing behavior and depression was partially mediated by dependent stressors. Independent stressors (i.e., those entirely unrelated to the behavior of the individual) did not mediate the relationship. Masten et al. (2005) assessed a sample at 7, 10, and 20 years. They tested a model, consistent with the failure model, where externalizing problems in childhood undermined academic success during adolescence, and in turn predicted internalizing symptoms (operationalized as depression, anxiety, and somatization) in early adulthood. Data provided evidence for this “developmental cascade,” particularly among girls. There was no evidence for a similar causal effect of internalizing behavior; that is, internalizing behavior was either unrelated or inversely related to subsequent externalizing behavior. In a contribution to the literature on girls, Measelle, Stice, and Hogansen (2006) assessed developmental trajectories of several symptom domains including depression and antisocial behavior in a sample of 493 adolescent girls. They found that initial antisocial symptoms were associated with the escalation of depression over a period of 5 years. Wiesner (2003) assessed depression and delinquency using a four-wave latent-variable cross-lagged design in a mixed-gender sample of 15- and 16-year-old adolescents. Delinquency, operationalized as violence, theft, and property destruction, was predictive of subsequent depression in one of three crossed paths for boys and two of three paths for girls. Lahey et al. (2002) assessed the prognostic power of conduct disorder over the course of seven annual assessments of psychopathology. Analyses revealed that initial conduct problems predicted depression at the six subsequent assessments, when controlling for baseline depression. The reverse was not the case. However, these prospective relations were not specific to depression; that is, conduct disorder also presaged anxiety and attention-deficit symptoms.

Although more evidence suggests externalizing behavior as a predictor of subsequent depression, the data are not consistent. Wolff and Ollendick (2006) acknowledged that depression may exert a direct effect on conduct problems, although the theoretical basis is less clear. It is conceivable that the hopelessness associated with depression might diminish the deterrent effect of punishments associated with delinquency. Further, the acting-out behavior could serve a self-regulatory mechanism, whereby the excitement of externalizing behavior counteracts the vegetative aspects of depression. Several studies have marshaled evidence of this relationship. In a sample of middle adolescent boys, Beyers and Loeber (2003) assessed concurrent and prospective relations of delinquency variety and depression. A valuable feature of these analyses is the inclusion of risk factors in the models that are common to both depression and delinquency. They found that even after controlling for the effects of poor parent–adolescent communication, family socioeconomic status, peer delinquency, low academic achievement and aggression, concurrent and longitudinal effects of depression on delinquency were found. Wiesner (2003) found bidirectional influences among girls, though not boys. Similarly, Measelle et al. (2006) identified the same bidirectional effects in girls. In a sample of boys with attention deficit/hyperactivity disorder, depression was related to subsequent conduct disorder, but the reverse was not the case (Drabick, Gadow, & Spafkin, 2006).

MODERATORS OF THE PROSPECTIVE RELATIONS BETWEEN DEPRESSION AND EXTERNALIZING BEHAVIOR

Gender

Characterizing psychopathology among adolescents necessitates an examination of relevant moderating variables. Relations between depression and externalizing behavior may vary across etiologically significant variables, such as gender or child maltreatment. Studies of the developmental similarities and differences between genders are especially relevant, as relatively few investigations have examined prospective changes of externalizing behavior among girls. Among extant studies, key differences in the timing and nature of symptoms have been observed for boys and girls (Hay, 2007; Nolen-Hoeksema, 2001). Suggestions from the literature indicate that externalizing behavior among girls may portend more negative depression outcomes (Loeber & Keenan, 1994; Rutter, Caspi, & Moffitt, 2003). Other studies have found distinctive patterns of relations between internalizing and externalizing dimensions for boys and girls (Bukstein, Glancy, & Kaminer, 1992; Clark et al., 1997), although inconsistency in the nature of differences have emerged (Marmorstein & Iacono, 2001). In a cross-sectional study, Rowe, Maughan, and Eley (2006) found similar mediational relationships between externalizing behavior and depression for boys and girls, but the authors lamented the dearth of extant data and encouraged further exploration of the moderating role of gender. Given the paucity of data regarding the prospective changes of depression and externalizing behavior in girls, the question of the moderating effect of gender warrants further research.

Maltreatment Experience

The inclusion of maltreatment in studies of comorbidity is motivated by two lines of evidence. First, as maltreatment is a risk factor for both depressive symptoms and externalizing behavior, its exclusion from pathogenic comorbidity studies could confound results. Second, maltreatment may function as interactive, rather than simply additive risk factor for negative outcomes. High rates of co-occurring symptoms have been documented in maltreated samples (Cicchetti & Toth, 2005), and existing evidence suggests that maltreatment represents a risk factor for both depression (Weiss, Longhurst, & Mazure, 1999) and externalizing behavior (Smith & Thornberry, 1995; McLeer, Callaghan, Henry, & Wallen, 1994). Several studies demonstrate the salience of maltreatment for studies of the comorbidity of depression and externalizing behavior. Keiley, Lofthouse, Bates, Dodge, and Pettit (2003), using a methodology borrowed from multitrait–multimethod approaches, created orthogonal internalizing, externalizing, and covarying factors in a longitudinal study of nonclinical adolescents. Harsh punishment was related not only to externalizing behavior but also to covariation between externalizing and internalizing behavior. Several studies have found that child maltreatment or harsh punishment distinguishes individuals with only internalizing or externalizing symptoms from those with comorbid conditions (Ge, Best, Conger, & Simons, 1996; Meller & Borchardt, 1996; Simic & Fombonne, 2001; Whitbeck, Hoyt, & Bao, 2000).

In addition to evidence that maltreatment confers risk for both depressive and externalizing problems, some literature suggests that maltreatment amplifies the effects of risk factors in a nonadditive manner. Cicchetti’s (Cicchetti & Lynch, 1993; Cicchetti & Toth, 1995, 2005) developmental psychopathology framework highlights the interactive nature maltreatment experience, whereby complex transactional patterns serve to potentiate or attenuate the effects of other risk factors. Accumulating evidence from behavioral genetic studies suggest that maltreatment not only functions merely as an isolated risk factor but also interacts in nontrivial ways with biological vulnerabilities to produce externalizing behavior (Caspi et al., 2002; Foley et al., 2004) and depression (Kaufman et al., 2004). This series of publications illustrates the interaction between maltreatment and underlying vulnerabilities. Thus, it is plausible that maltreatment may deplete the resources required to successfully recover from psychopathological state, thereby further compromising functioning. That is, externalizing behavior may portend more negative outcomes for maltreated adolescents, who lack some of the internal or external resources that facilitate a return to healthy functioning. These lines of evidence encourage an examination of the equivalence of longitudinal relations of psychopathology between maltreated and nonmaltreated youth.

THE PRESENT STUDY

The present study utilized a cross-lagged panel analysis design with an ethnically diverse, gender-balanced, urban sample of young adolescents in order to determine the relations between depressive symptoms and externalizing behavior across time. This represents a test of the tenability of pathogenic theory of comorbidity in this age group. The current study extends previous findings by investigating two important variables that may moderate the relationships between depression and externalizing behavior: gender and maltreatment experience. Due to the number of studies that have examined externalizing behavior with exclusively male samples, the potential moderating effect of gender is important to examine.

A multigroup structural equation modeling framework was used to determine whether the longitudinal relations between depressive symptoms and externalizing behavior are similar across boys and girls and across maltreated and comparison children. It was hypothesized that in maltreated girls, there is a downstream effect of early externalizing behavior on subsequent depressive symptoms. Further, it was hypothesized that this effect would be zero among the nonmaltreated boys. Available evidence does not provide a clear basis for hypotheses regarding this parameter among maltreated boys and nonmaltreated girls. As substantially less evidence suggests a downstream effect of depressive symptoms on externalizing behavior, we do not anticipate finding evidence for this in any of the four groups. Figure 1 displays the model described.

FIGURE 1.

FIGURE 1

Open in a new tab

Cross-lagged panel analysis of depressive symptoms and externalizing behavior.

METHODS

Participants

Data for this study were obtained from Time 1 (T1) and Time 2 (T2) of a longitudinal study on the effects of maltreatment on adolescent development. There were 454 adolescents and their caretakers enrolled in the study. T2 took place approximately 12 months following the initial assessment, and 86% of families (N=392) returned for the second wave of data collection. Of the 62 attriters, 19 chose to leave the study, 12 moved out of the country, and 31 were unable to be scheduled due to difficulties with contact or attending the assessment. The sample characteristics at T1 and T2 are displayed in Table 1.

TABLE 1.

Sample Characteristics For Each Group at Each Measurement Point

Demographic Variable Group and Measurement Point
Maltreated
 Comparison
Time 1 Time 2 Time 1 Time 2
N 303 250 151 142
Age (standard
 deviation)		 10.84
(1.15)		 12.02
(1.21)		 11.11
(1.15)		 12.28
(1.26)
Gender (%)
 Male 50 48 60 60
 Female 50 52 40 40
Ethnicity (%)
 African American 40 40 32 32
 Latino 35 36 47 45
 Caucasian 12 11 10 11
 Biracial 13 13 11 12
Annual household income (%)
 $0-$14,999 37 35 13b 11
 $15,000–29,999 32 34 27 28
 $30,000–44,999 12 13 24 26
 $45,000–59,999 9 10 13 14
 $60,000–74,999 5 5 10 11
 Over $75,000 3 2 11 11
Living Arrangement (%)
 With Parent 52 63a 93b 94
 Foster Care or Kinship Care 48 37 7 6
Open in a new tab
a

Maltreatment group at T1 significantly different from maltreatment group at T2 (p<.05).

b

Maltreatment group different than comparison group (p<.05).

Procedures

Following approval from the University of Southern California Institutional Review Board and the Los Angeles County Juvenile Court, we collaborated with Los Angeles County Department of Child and Family Services (DCFS) to acquire a sample of maltreated adolescents. Beginning in 2002 and completed in 2004, the recruitment gathered 303 young adolescents according to the following inclusion criteria: (a) a DCFS case being opened in the preceding month due to the credible report of maltreatment or the extreme threat of maltreatment; (b) age between 9 and 12 years; (c) child identified as Latino, African American, or Caucasian; and (d) child currently residing within 10 specified zip codes within Los Angeles County. The zip codes were chosen to be accessible to the research site, to contain significant numbers of children of the three ethnicities, and to have substantial numbers of maltreated children. A letter went to the caretaker of each child describing the study and enclosing a postcard indicating their willingness or unwillingness to participate. In all, 77% of families agreed to participate. The decision to use official records ensures representation of more severe forms of maltreatment that are often missed in less targeted methods of sampling (Cichetti & Toth, 2005). However, the unitary labels attached to cases by DCFS are often insufficient to document the complexity of the participants’ maltreatment experience. Accordingly, the research team abstracted the case records to obtain data on the type of maltreatment, severity, and chronicity (see Trickett, Mennen, Kim, & Sang, 2009, for more details of this process). Seventy-two percent of the sample experienced general or severe neglect, 49% physical abuse, 20% sexual abuse, 48% emotional abuse, and 52% caretaker incapacity. Fifty-seven percent had more than one type of maltreatment experience. The average number of referrals to a child protection agency was 4.9 (SD = 3.3) with a range of 1 to 17.

A comparison sample of 151 adolescents was collected from the same zip codes, using a list of family names obtained from a marketing firm. Fifty percent of the invited families agreed to participation. Comparison participants had no previous reports or documented incidents of child maltreatment. The demographic characteristics for age, neighborhood, and ethnicity were similar to the maltreatment group. The comparison group had a significantly smaller percentage of families (13%) living in poverty (defined as less than $15,000 annual household income) than maltreatment families (37%). As DCFS must often remove children from dangerous environments, the proportion of maltreated youth living with a biological parent (54%) was notably lower than the living arrangements of comparison youth (94% with biological parent).

Assessments were conducted at an urban research university. After assent and consent were obtained from the adolescent and their caretaker, the adolescent was administered an array of questionnaires and tasks during a 4-hr protocol. The measures used in the following analyses represent a subset of the questionnaires administered during the protocol, which also included hormonal, cognitive, and behavioral measures. Both the child and caretaker were paid for their participation according to the National Institutes of Health Normal Volunteer Program.

Measures

Depressive symptoms

The Children’s Depression Inventory (CDI) is a 27-item self-report measure representing a downward extension of the Beck Depression Inventory (Kovacs, 1981). The items are rated on a 3-point scale assessing the frequency of an emotional state or behavior. Myers and Winters (2002) reviewed the evidence to support the CDI as a measure of depression in children and adolescents. Its subscales include Negative Self-Esteem, Anhedonia, Ineffectiveness, Negative Mood, and Interpersonal Problems. One potential criticism of the CDI is its broad definition of the construct of depression. Specifically, the Interpersonal Problems subscale taps aspects of irritability and aggression with items such as “I get into fights all the time.” Although legitimate arguments can be made that irritability in youth represents one manifestation of depression, Myers and Winters suggested that the CDI’s discriminant validity suffers as a result of the broad definition of the construct. So as to avoid artifactual overlap with the measure of externalizing behavior, the interpersonal problems subscale was excluded in the current study.

Aggression and delinquency

The Youth Self-Report (YSR; Achenbach, 1991) is an extensively researched tool for adolescent assessment that has been translated into more than 60 languages (Achenbach & Rescorla, 2001). It represents the self-report version of a multi-informant strategy that includes assessment of teacher and parental views of the child. Items are rated on a 3-point scale assessing the frequency of occurrence, ranging 0 (not true), 1 (somewhat or sometimes true) and 2 (very or often true). In the current study, only the 17-item Aggression subscale and 16-item Delinquency subscale were utilized. The Aggression subscale includes items such as “I have a hot temper,” and the Delinquency subscale includes items such as “I disobey at school.” Although the Attention subscale of the YSR is typically considered as representative of the externalizing construct, previous work suggests attention problems represents a “mixed syndrome” (Lambert et al., 2003). In contrast, the Aggression and Delinquency subscales have previously been found to load unequivocally on an externalizing factor (Lambert et al., 2003; O’Keefe, Mennen, & Lane, 2006).

Covariates

Covariates included in the analyses included child’s age at T1, ethnicity (Black, White, Latino, and biracial), annual caretaker household income (1 = under $4,999; 12 = over $120,000) and a composite variable from the Brief Symptom Inventory (Derogatis & Melisaratos, 1983) representing caretaker depression and anxiety. Ethnicity and caretaker income were included as covariates as these variables are related with psychological distress (Costello, Keeler, & Angold, 2001) and were not precisely balanced across group comparison tests. Similarly, age shows important relationships with both depression and externalizing behavior and is thus relevant for inclusion as a covariate. Fluid and crystallized intelligence was measured by the Woodcock–Johnson Psycho-Educational Battery (Woodcock & Johnson, 1989). Childhood temperament and prenatal exposure to alcohol and nicotine were reported retrospectively by the caretaker.

Data Analysis

Cross-lagged panel analysis

Cross-lagged panel analyses afford the opportunity to examine hypothesized causal processes between two different constructs. Not only is the autoregressive effect of a construct on itself examined, but the crossed paths hypothesize that the construct exerts an impact on the second construct, even after adjusting for the autoregressive effect. The full model was fit to the total sample prior to conducting multiple group analyses. Depressive symptoms and externalizing behavior were allowed to correlate at T1, and the disturbance terms for the endogenous T2 variables were allowed to correlate. Covariates included ethnicity, age, parental income, and parental depression/anxiety. T2 variables were regressed on each of the covariates.

To conduct the multiple group analysis, first factorial invariance was tested. Ensuring that the measures function equivalently across groups is necessary if results are to be unambiguously interpreted (Horn & McArdle, 1992). Factorial invariance analyses with the four groups (comparison/girl, comparison/boy, maltreated/girl, maltreated/boy) were conducted by imposing cross-group equality constraints on the factor loadings. Next, significant differences between groups for each cross-lagged path was assessed by first freely estimating the parameter and then by imposing equality constraints. Significant decrements in model fit following parameter constraints were assessed with the chi-square difference test and indicated that a constrained parameter was significantly different between groups. Fit indices such as the chi-square goodness of fit statistic, the root mean square error of approximation (RMSEA), and comparative fit index (CFI) were used to evaluate the fit of the model to the data. The RMSEA accounts for sample size in estimating the error of approximation per degree of freedom with values less than .05 are indicative of good fit and values less than .08 indicate acceptable fit (Browne & Cudeck, 1993). The CFI (Bentler, 1990) is an incremental fit index that assesses the improvement of the specified model over a model assuming zero covariances among variables. The traditional standard for adequate fitting models is a CFI of .90 or higher. However, Hu and Bentler (1999) have subsequently suggested that the conventional cutoff may be too low and found that a CFI of .95 is a better guideline.

Treatment of missing data and outliers

Item-level missingness was extremely low, with most variables exhibiting less than 1% missing. For these instances, imputation using Schafer’s (1999) NORM software program was used to complete the data set. The substantive analyses were conducted with the structural equation modeling software AMOS 18 (Arbuckle, 2009) and implemented full information maximum likelihood procedures to incorporate data from participants lacking data for an entire measure. The attrition rate of 14% is relatively low, given the distressed nature of the sample. Examination of patterns of missingness revealed that maltreatment experience and elevated caregiver depression and anxiety, as measured by the Brief Symptom Inventory, were related to attrition. Models included these variables so as to minimize bias in parameter estimation (Schafer & Graham, 2002). The data were also inspected for influential observations that could artificially distort the model for the vast majority of the data. McClelland (2000) argued that there are principled, statistically informed procedures for determining outlying values which emerge from a different subpopulation. Multivariate outliers, which included seven subjects, were excluded before hypothesis testing was conducted.

RESULTS

Descriptive Statistics

Descriptive statistics are displayed in Table 2, and bivariate correlations are displayed in Table 3. Means for depressive symptoms are similar among the four groups, although maltreated boys at study entry feature higher levels of externalizing behavior than the other groups (effect size d~.3).

TABLE 2.

Descriptive Statistics for Children’s Depression Inventory (CDI) and Youth Self-Report (YSR)

Time 1
 Time 2
Maltreated
Boys		 Maltreated
Girls		 Comparison
Boys		 Comparison
Girls		 Maltreated
Boys		 Maltreated
Girls		 Comparison
Boys		 Comparison
Girls
CDIa
N 148 149 89 61 118 126 82 57
M 8.56 8.81 6.91 8.66 6.94 7.91 6.40 6.58
SD 6.94 6.80 5.31 6.08 5.09 6.55 4.94 5.04
YSRb
N 148 149 89 61 118 126 83 57
M 13.64 11.21 11.20 11.59 12.54 11.56 11.56 12.25
SD 9.63 7.21 7.59 6.83 8.34 7.60 6.56 8.82
Open in a new tab
a

CDI excluding the Interpersonal Problems subscale.

b

YSR including the Aggression and Delinquency subscales.

TABLE 3.

Correlations Between Time 1 (T1) and Time 2 (T2) Variables for Boys and Girls

T1 Externalizing T1 Depression T2 Externalizing T2 Depression
T1 Externalizing .87 .41* .43* .25*
T1 Depression .50* .86 .23* .56*
T2 Externalizing .44* .33* .86 .33*
T2 Depression .15* .42* .46* .83
Open in a new tab

Note: Correlations for girls displayed below diagonal; boys displayed above diagonal. Reliabilities (Cronbach’s α) for entire sample displayed along diagonal are bolded.

*

p<.05.

Cross-Lagged Panel Model

First, the full model was fit to the total sample and this fit the data well, χ2(58)=117.60, p=.000 (CFI=.969, RMSEA=.048). Next, a four-group (comparison/boys, comparison/girls, maltreated/boys, maltreated/girls) multiple-group model was estimated with the measurement weights restricted (factorial invariance) but the autoregressive and crossed paths allowed to freely vary for each group. This model was significantly different from the unrestricted model, so partial measurement invariance was tested. Partial measurement invariance was achieved by freeing the equality constraints across groups on the measurement loadings for T1 negative self-image, T1 YSR delinquency, and T2 YSR delinquency. The partial measurement invariance model did not differ significantly from the unrestricted model, Δχ2(Δdf=15)=16.32, p > .05. This model was compared with one in which all the structural parameters were restricted to be equal across groups. The model fit showed that the restricted model was significantly different from the measurement weights restricted model, Δχ2(Δdf=12)=36.79, p < .01, and indicated that one or more of the structural parameters were significantly different across groups. For the crossed path from T1 externalizing to T2 depressive symptoms, comparison (β=−.02, ns) and maltreated boys (β=.06, ns) were significantly different from maltreated girls (β=.75, p=.00), Δχ2(Δdf=1)=15.92, p < .01; Δχ2 (Δdf=1)=14.59, p < .01. In addition, comparison girls (β=.43, p=.08) appeared to evidence a weaker effect than maltreated girls, Δχ2(Δdf=1)=3.63, p=.06. Figure 2 displays these parameter estimates. These results indicate that maltreated girls showed the strongest relationship between early externalizing behavior and later depression. For the rest of the groups this association was not significant. For maltreated girls, baseline externalizing problems explained 22% of the variance in T2 depressive problems. The crossed path from T1 depressive symptoms to T2 externalizing was not significantly different from zero in any of the four groups.

FIGURE 2.

FIGURE 2

Open in a new tab

Multigroup cross-lagged panel analysis with maltreatment status=gender as moderating variable. Note: The models report on standardized coefficients from multiple-group models. When found invariant across group, the estimate was standardized across the full sample. ** p < .01.

To assess the robustness of the moderated effect of earlier externalizing behavior on subsequent depression, several additional analyses were conducted. First, a two-group model featuring boys and girls was fit with the maltreatment variables as predictors of psychopathology at T2. This test was conducted to determine if the crossed effect among girls would be diluted by the inclusion of maltreatment as a risk factor. Results indicated that the prospective effect of externalizing behavior on depression persisted for girls. Second, a single group model was fit with girls that featured additional covariates. Within the four-group model, the inclusion of covariates beyond those identified above exceeded the reasonable limit on the number of parameters being estimated, given the sample size. In this model—in addition to ethnicity, age, parental income, parental psychopathology—the adolescent’s fluid intelligence, crystallized intelligence, childhood temperament, and prenatal exposure to alcohol and nicotine were added as covariates. After accounting for the effects of these variables on the T2 outcome variables, the prospective effect of externalizing behavior on depression persisted. These analyses increase confidence in the robustness of the observed crossed effect for girls.

For the autoregressive paths, there were significant group differences only in the path from T1 depressive symptoms to T2 depressive symptoms. Further testing showed that maltreated and comparison girls were not significantly different and did not indicate a significant autoregressive effect (β=−.03, ns). This is not unexpected, as the measurement period coincided with significant pubertal changes, when the depressive profile of girls changes substantially. Comparison male individuals (β=.72, p < .01) were significantly higher than all other groups. This may have been a function of this group’s lower symptom levels. Maltreated boys (β=.59, p < .01) were also significantly different from maltreated girls, Δχ2(Δdf=1)=4.14, p=.04.

DISCUSSION

This study utilized a cross-lagged panel analysis design to determine the relations between depression and externalizing behavior across two time points. The results are consistent with evidence that early externalizing symptomatology contributes to the escalation of depressive symptoms. This finding is consistent with Capaldi’s (1991, 1992) failure model. Of importance is the fact that this effect was confined to girls and appeared strongest specifically among maltreated girls. This provides some evidence for pathogenic comorbidity: Among girls, early externalizing behavior may exert a direct effect on later depression even after controlling for demographic factors, caretaker psychopathology, cognitive factors, prenatal risk factors, and childhood temperament. The findings of a moderated effect is consistent with prior evidence suggesting that externalizing behavior carries greater risk for later internalizing distress among girls, particularly for depression (Loeber & Kennan, 1994; Moffitt, Caspi, Rutter, & Silva, 2001). Future studies should seek to replicate this finding. If replicated, the results illustrate the important prognostic value of externalizing behavior among girls and, even more specifically, for maltreated girls. This highlights a strategic target for intervention.

Future studies might profitably investigate the mediating mechanisms linking these variables. Previous work has suggested that academic competencies may be compromised by externalizing behavior, which in turn makes individuals more susceptible to depression (Masten et al., 2005). Other mediational hypotheses such as the failure of externalizing youth to adequately develop nurturing peer relationships might be investigated. The association between depressive symptoms and externalizing behavior may be explained by from a transactional framework (Cicchetti & Toth, 2005), whereby exposure to maltreatment is associated with the selection of environments that are more likely to potentiate psychological problems. Antisocial peer affiliations may partially explain the observed link between externalizing behavior and subsequent depression. Why this effect is found only among girls, and particularly among maltreated girls, remains an important question, but it is possible that girls pay an especially high social cost for externalizing symptoms and this contributes to depressive symptoms. Efforts to link this gender specific effect may have useful implications for thinking about psychopathology more generally.

It is important to note that although this provides preliminary evidence for pathogenic comorbidity, it is possible that the observed phenomenon reflects risk factors predisposing for both depression and externalizing behavior. That is, the shared etiological theory of comorbidity predicts consistent relationships across time between forms of psychopathology. It is relevant however, that depression and externalizing behavior did not show reciprocal relationships. Instead, the relationship was unidirectional and only significant among girls. That we only witnessed the predictive power of externalizing behavior makes the shared etiological theory less plausible as the sole explanation for the current findings.

Examining the stability of depressive symptoms and externalizing behavior across groups showed that externalizing behavior evidenced moderate and similar stability in all groups. However, the stability of depressive problems did vary based on group. Comparison boys evidence the most stability, followed by maltreated boys. There was no significant association between depressive symptoms at T1 and T2 for either maltreated or comparison girls. Perhaps for girls, because the study covered the period of early puberty when there is a known increase in depressive symptoms around Tanner stage 3 (Angold, Costello, & Worthman, 1998), we should expect less stability across 1 year.

The results of this study must be considered in light of several limitations. The relatively short period of adolescence featured precludes the possibility of making broad developmental interpretations. The developmental window characterized by the present research should be extended in future work. The use of a nonclinical sample may limit the generalizability of the findings. Although clinical samples are known to over-represent comorbid diagnoses and therefore may not provide the best avenue for understanding these issues, it would be important to determine if clinical populations evidence the same relationships. The lack of a clinician-administered diagnostic interview might be considered another limitation of the study. Although the difficulties of relying exclusively on diagnostic interviews have been enumerated (Achenbach & Edelbrock, 1984), future investigations are encouraged to combine information from self-reports and parent-report measures with diagnostic interviews. It is commonly noted that even well-fitting structural models do not permit definitive conclusions. Tomarken and Waller (2003) demonstrate that even in relatively simple models, there often exist many models with identical implied covariance matrices and, thus, identical fit statistics. In addition, alternative nonequivalent models exist that might fit the data as well or better than the specified model. The finding that the crossed effect observed for girls is carried primarily by maltreated girls must be interpreted cautiously. The apparent moderating effect of maltreatment among girls could reflect power considerations; that is, the comparison girls group was substantially smaller than the group of maltreated girls.

A final limitation of the current study is the lack of a genetically informative design. Twin studies, for example, have made important contributions by modeling the covariation between disorders as a function of additive genetic, shared environment, and nonshared environmental factors (Neale & Kendler, 1995). In this framework, the covariation itself becomes the dependent variable explained by different factors. Results from such studies suggest powerful genetic influences, although environmental factors remain a significant determinant of symptom covariation even when accounting for the influence of genetic factors (Burt, Krueger, McGue, & Iacono, 2003; O’Connor, McGuire, Reiss, Hetherington, & Plomin, 1998).

Implications for Research, Policy, and Practice

Explanations of comorbidity have tended to presume similarity across etiologically significant group variables. However, this current research suggests that the mechanisms by which comorbidity arises varies across gender. Future research should attend carefully to gender and maltreatment experience when examining the mechanisms by which depression and externalizing behavior co-occur. Among girls, and perhaps maltreated girls specifically, externalizing behavior may be an important prognostic indicator of psychopathological processes and portend negative outcomes. As such, externalizing behavior may serve as a strategic target for intervention. Clinical interventions targeting externalizing behavior may have salutary effects on depressive trajectories. Studies assessing interventions for girls with externalizing behavior might additionally assess depression as an outcome.

REFERENCES

  1. Achenbach TM. Integrative guide for the 1991 CBCL/4-18, YSR, and TRF profiles. University of Vermont, Department of Psychiatry; Burlington: 1991. [Google Scholar]
  2. Achenbach TM, Edelbrock CS. Psychopathology of childhood. Annual Review of Psychology. 1984;35:227–256. doi: 10.1146/annurev.ps.35.020184.001303. [DOI] [PubMed] [Google Scholar]
  3. Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms and profiles. University of Vermont, Center for Children, Youth & Family; Burlington: 2001. [Google Scholar]
  4. Angold A, Costello EJ, Erkanli A. Comorbidity. Journal of Child Psychology and Psychiatry. 1999;40:57–87. [PubMed] [Google Scholar]
  5. Angold A, Costello EJ, Worthman CM. Puberty and depression: The roles of age, pubertal status, and pubertal timing. Psychological Medicine. 1998;28:51–61. doi: 10.1017/s003329179700593x. [DOI] [PubMed] [Google Scholar]
  6. Arbuckle JL. Amos 18 user’s guide. Amos Development; Crawfordsville, FL: 2009. [Google Scholar]
  7. Bentler PM. Comparative fit indices in structural models. Psychological Bulletin. 1990;107:238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
  8. Beyers JM, Loeber R. Untangling developmental relations between depressed mood and delinquency in male adolescents. Journal of Abnormal Child Psychology. 2003;31:247–266. doi: 10.1023/a:1023225428957. [DOI] [PubMed] [Google Scholar]
  9. Biederman J, Faraone S, Mick E, Lelon E. Psychiatric comorbidity among referred juveniles with major depression: Fact or artifact? Journal of the American Academy of Child and Adolescent Psychiatry. 1995;34:579–590. doi: 10.1097/00004583-199505000-00010. [DOI] [PubMed] [Google Scholar]
  10. Browne MW, Cudeck R. Alternative ways of assessing fit. In: Bollen KA, Long JS, editors. Testing structural equation models. Sage; Newbury Park, CA: 1993. pp. 136–162. [Google Scholar]
  11. Bukstein OG, Glancy LG, Kaminer Y. Patterns of affective comorbidity in a clinical population of dually diagnosed adolescent substance abusers. Journal of the American Academy of Child and Adolescent Psychiatry. 1992;31:1041–1045. doi: 10.1097/00004583-199211000-00007. [DOI] [PubMed] [Google Scholar]
  12. Burt AS, Krueger RF, McGue M, Iacono W. Parent–child conflict and the comorbidity among childhood externalizing disorders. Archives of General Psychiatry. 2003;60:505–513. doi: 10.1001/archpsyc.60.5.505. [DOI] [PubMed] [Google Scholar]
  13. Capaldi DM. The co-occurrence of conduct problems and depressive symptoms in early adolescent boys: I. Familial factors and general adjustment at grade 6. Development and Psychopathology. 1991;3:277–300. doi: 10.1017/s0954579499001959. [DOI] [PubMed] [Google Scholar]
  14. Capaldi DM. Co-occurrence of conduct problems and depressive symptoms in early adolescent boys: IT. A 2-year follow-up at Grade 8. Development and Psychopathology. 1992;4:125–144. doi: 10.1017/s0954579499001959. [DOI] [PubMed] [Google Scholar]
  15. Caspi A, McClay J, Moffitt TE, Mill J, Martin J, Craig IW, et al. Role of genotype in the cycle of violence in maltreated children. Science. 2002;297:851–854. doi: 10.1126/science.1072290. [DOI] [PubMed] [Google Scholar]
  16. Cicchetti D, Lynch M. Toward an ecological/transactional model of community violence and child maltreatment: Consequences for children’s development. Psychiatry. 1993;56:96–118. doi: 10.1080/00332747.1993.11024624. [DOI] [PubMed] [Google Scholar]
  17. Cicchetti D, Toth SL. A developmental psychopathology perspective on child abuse and neglect. Journal of the American Academy of Child and Adolescent Psychiatry. 1995;34:541–565. doi: 10.1097/00004583-199505000-00008. [DOI] [PubMed] [Google Scholar]
  18. Cicchetti D, Toth SL. Child maltreatment. Annual Review of Clinical Psychology. 2005;1:409–438. doi: 10.1146/annurev.clinpsy.1.102803.144029. [DOI] [PubMed] [Google Scholar]
  19. Clark DB, Pollock N, Bukstein OG, Mezzich AC, Bromberger JT, Donovan JE. Gender and comorbid psychopathology in adolescents with alcohol dependence. Journal of the American Academy of Child and Adolescent Psychiatry. 1997;36:1195–1203. doi: 10.1097/00004583-199709000-00011. [DOI] [PubMed] [Google Scholar]
  20. Clarkin JE, Kendall PC. Comorbidity and treatment planning: Summary and future directions. Journal of Consulting and Clinical Psychology. 1992;60:904–908. [PubMed] [Google Scholar]
  21. Costello EJ, Keeler GP, Angold A. Poverty, race/ethnicity, and psychiatric disorder: A study of rural children. American Journal of Public Health. 2001;91:1494–1498. doi: 10.2105/ajph.91.9.1494. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Derogatis LR, Melisaratos N. The Brief Symptom Inventory: An introductory report. Psychological Medicine. 1983;12:595–605. [PubMed] [Google Scholar]
  23. Dodge KA. Behavioral antecedents of peer social status. Child Development. 1983;54:1386–1399. [Google Scholar]
  24. Drabick DA, Gadow KD, Sprafkin J. Co-occurrence of conduct disorder and depression in a clinic-based sample of boys with ADHD. Journal of Child Psychology and Psychiatry. 2006;47:766–774. doi: 10.1111/j.1469-7610.2006.01625.x. [DOI] [PubMed] [Google Scholar]
  25. Foley DL, Eaves LJ, Wormley B, Silberg JL, Maes HH, Kuhn J, et al. Childhood adversity, monoamine oxidase a genotype, and risk for conduct disorder. Archives of General Psychiatry. 2004;61:1–7. doi: 10.1001/archpsyc.61.7.738. [DOI] [PubMed] [Google Scholar]
  26. Ge X, Best KM, Conger RD, Simons RL. Parenting behaviors and the occurrence and co-occurrence of adolescent depressive symptoms and conduct problems. Developmental Psychology. 1996;32:717–731. [Google Scholar]
  27. Greene RW, Biederman J, Zerwas S, Monuteaux MC, Goring JC, Faraone SV. Psychiatric comorbidity, family dysfunction, and social impairment in referred youth with oppositional defiant disorder. American Journal of Psychiatry. 2002;159:1214–1224. doi: 10.1176/appi.ajp.159.7.1214. [DOI] [PubMed] [Google Scholar]
  28. Hay DF. The gradual emergence of sex differences in aggression: Alternative hypotheses. Psychological Medicine. 2007;37:1527–1537. doi: 10.1017/S0033291707000165. [DOI] [PubMed] [Google Scholar]
  29. Hinshaw SP. Externalizing behavior problems and academic underachievement in childhood and adolescence: Causal relationships and underlying mechanisms. Psychological Bulletin. 1992;111:127–55. doi: 10.1037/0033-2909.111.1.127. [DOI] [PubMed] [Google Scholar]
  30. Horn JL, McArdle JJ. A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research. 1992;18:117–144. doi: 10.1080/03610739208253916. [DOI] [PubMed] [Google Scholar]
  31. Hu L, Bentler PM. Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:1–55. [Google Scholar]
  32. Jensen PS. Comorbidity and child psychopathology: Recommendations for the next decade. Journal of Abnormal Child Psychology. 2003;31:293–300. doi: 10.1023/a:1023281513936. [DOI] [PubMed] [Google Scholar]
  33. Kaufman J, Yang B-Z, Douglas-Palumberi H, Houshyar S, Lipschitz D, Krystal J, et al. Social supports and serotonin transporter gene moderate depression in maltreated children. PNAS. 2004;101:17316–17321. doi: 10.1073/pnas.0404376101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Keiley MK, Lofthouse N, Bates JE, Dodge KA, Pettit GS. Differential risks of covarying and pure components in mother and teacher reports of externalizing and internalizing behavior across ages 5 to 14. Journal of Abnormal Child Psychology. 2003;31:267–283. doi: 10.1023/a:1023277413027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Klein DN, Riso LP. Psychiatric disorders: Problems of boundaries and comorbidity. In: Costello CG, editor. Basic issues in psychopathology. Guilford; New York: 1993. pp. 19–66. [Google Scholar]
  36. Kovacs M. Rating scales to assess depression in school-age children. Acta Paedopsychiatrica. 1981;46:305–315. [PubMed] [Google Scholar]
  37. Lahey BB, Loeber R, Burke J, Rathouz PJ, McBurnett K. Waxing and waning in concert: Dynamic comorbidity of conduct disorder with other disruptive and emotional problems over 7 years among clinic-referred boys. Journal of Abnormal Psychology. 2002;4:556–567. doi: 10.1037//0021-843x.111.4.556. [DOI] [PubMed] [Google Scholar]
  38. Lambert MC, Schmitt N, Samms-Vaughan ME, An JS, Fairclough M, Nutter CA. Is it prudent to administer all items for each child behavior checklist cross-informant syndrome? Evaluating the psychometric properties of the youth self-report dimensions with confirmatory factor analysis and item response theory. Psychological Assessment. 2003;15:550–568. doi: 10.1037/1040-3590.15.4.550. [DOI] [PubMed] [Google Scholar]
  39. Little SA, Garber J. The role of social stressors and interpersonal orientation in explaining the longitudinal relation between externalizing and depressive symptoms. Journal of Abnormal Psychology. 2005;3:432–433. doi: 10.1037/0021-843X.114.3.432. [DOI] [PubMed] [Google Scholar]
  40. Loeber R, Keenan K. Interaction between conduct disorder and its comorbid conditions: Effects of age and gender. Clinical Psychology Review. 1994;14:497–523. [Google Scholar]
  41. Marmorstein NR, Iacono WG. An investigation of female adolescent twins with both major depression and conduct disorder. Journal of the American Academy of Child & Adolescent Psychiatry. 2001;40:299–306. doi: 10.1097/00004583-200103000-00009. [DOI] [PubMed] [Google Scholar]
  42. Masten AS, Roisman GI, Long JD, Burt KB, Obradovic J, Riley JR, et al. Developmental cascades: Linking academic achievement and externalizing and internalizing symptoms over 20 years. Developmental Psychology. 2005;41:733–746. doi: 10.1037/0012-1649.41.5.733. [DOI] [PubMed] [Google Scholar]
  43. McClelland GH. Nasty data: Unruly, ill–mannered observations can ruin your analysis. In: Reis HT, Judd CM, editors. Handbook of research methods in social and personality psychology. Cambridge University Press; Cambridge, England: 2000. pp. 393–411. [Google Scholar]
  44. McLeer SV, Callaghan M, Henry D, Wallen J. Psychiatric disorders in sexually abused children. Journal of the American Academy of Child and Adolescent Psychiatry. 1994;33:313–319. doi: 10.1097/00004583-199403000-00003. [DOI] [PubMed] [Google Scholar]
  45. Measelle JR, Stice E, Hogansen JM. Developmental trajectories of co-occurring depressive, eating, antisocial, and substance abuse problems in female adolescents. Journal of Abnormal Psychology. 2006;115:524–538. doi: 10.1037/0021-843X.115.3.524. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Meller WH, Borschardt CM. Comorbidity of major depression and conduct disorder. Journal of Affective Disorders. 1996;39:123–126. doi: 10.1016/0165-0327(96)00031-6. [DOI] [PubMed] [Google Scholar]
  47. Meyers K, Winters NC. Ten-year review of rating scales. II: Scales for internalizing disorders. Journal of the American Academy of Child and Adolescent Psychiatry. 2002;41:634–659. doi: 10.1097/00004583-200206000-00004. [DOI] [PubMed] [Google Scholar]
  48. Moffit TE, Caspi A, Rutter M, Silva PA. Sex differences in antisocial behavior. Cambridge University Press; Cambridge, UK: 2001. [Google Scholar]
  49. Neale MC, Kendler KS. Models of comorbidity for multi-factorial disorders. American Journal of Human Genetics. 1995;57:935–953. [PMC free article] [PubMed] [Google Scholar]
  50. Nolen-Hoeksema S. Gender differences in depression. Current Directions in Psychological Science. 2001;10:173–176. [Google Scholar]
  51. O’Connor TG, McGuire S, Reiss D, Hetherington EM, Plomin R. Co-occurrence of depressive symptoms and antisocial behavior in adolescence: A common genetic liability. Journal of Abnormal Psychology. 1998;107:27–37. doi: 10.1037//0021-843x.107.1.27. [DOI] [PubMed] [Google Scholar]
  52. O’Keefe M, Mennen FE, Lane CJ. An examination of the factor structure for the youth self report in a multiethnic population. Research on Social Work Practice. 2006;16:315–325. [Google Scholar]
  53. Renouf AG, Kovacs M, Mukerji P. Relationship of depressive conduct and comorbid disorders in childhood. Journal of the American Academy of Child and Adolescent Psychiatry. 1997;36:998–1004. doi: 10.1097/00004583-199707000-00023. [DOI] [PubMed] [Google Scholar]
  54. Rowe R, Maughan B, Eley TC. Links between antisocial behavior and depressed mood: The role of life events and attributional style. Journal of Abnormal Child Psychology. 2006;34:293–302. doi: 10.1007/s10802-006-9032-0. [DOI] [PubMed] [Google Scholar]
  55. Rutter M, Caspi A, Moffitt T. Using sex differences in psychopathology to study causal mechanisms: Unifying issues and research strategies. Journal of Child Psychology & Psychiatry & Allied Disciplines. 2003;44:1092–1115. doi: 10.1111/1469-7610.00194. [DOI] [PubMed] [Google Scholar]
  56. Schafer J. NORM: Multiple imputation of incomplete multivariate data under a normal model, Version 2.03. State College; Department of Statistics, The Pennsylvania State University: 1999. [Google Scholar]
  57. Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychological Methods. 2002;7:147–177. [PubMed] [Google Scholar]
  58. Simic M, Fombonne E. Depressive conduct disorder: Symptom patterns and correlates in referred children and adolescents. Journal of Affective Disorders. 2001;62:175–185. doi: 10.1016/s0165-0327(99)00201-3. [DOI] [PubMed] [Google Scholar]
  59. Smith CA, Thornberry T. The relationship between child maltreatment and adolescent involvement in delinquency. Criminology. 1995;33:451–481. [Google Scholar]
  60. Tomarken AJ, Waller NG. Potential problems with “well fitting” models. Journal of Abnormal Psychology. 2003;112:578–595. doi: 10.1037/0021-843X.112.4.578. [DOI] [PubMed] [Google Scholar]
  61. Trickett PK, Mennen FE, Kim K, Sang J. Emotional abuse in a sample of multiply maltreated, urban young adolescents: Issues of definition and identification. Child Abuse and Neglect. 2009;33:27–35. doi: 10.1016/j.chiabu.2008.12.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Watson D. Rethinking the mood and anxiety disorders: A quantitative hierarchical model for DSM–V. Journal of Abnormal Psychology. 2005;114:522–536. doi: 10.1037/0021-843X.114.4.522. [DOI] [PubMed] [Google Scholar]
  63. Weiss EL, Longhurst JG, Mazure CM. Childhood sexual abuse as a risk factor for depression in women: Psychosocial and neurobiological correlates. American Journal of Psychiatry. 1999;156:816–828. doi: 10.1176/ajp.156.6.816. [DOI] [PubMed] [Google Scholar]
  64. Whitbeck LB, Hoyt DR, Bao W. Depressive symptoms and co-occurring depressive symptoms, substance abuse, and conduct problems among runaway and homeless adolescents. Child Development. 2000;71:721–732. doi: 10.1111/1467-8624.00181. [DOI] [PubMed] [Google Scholar]
  65. Wiesner M. A longitudinal latent variable analysis of reciprocal relations between depressive symptoms and delinquency during adolescence. Journal of Abnormal Psychology. 2003;112:633–645. doi: 10.1037/0021-843X.112.4.633. [DOI] [PubMed] [Google Scholar]
  66. Wolff JC, Ollendick TH. The comorbidity of conduct problems and depression in childhood and adolescence. Clinical Child and Family Psychology Review. 2006;9:201–220. doi: 10.1007/s10567-006-0011-3. [DOI] [PubMed] [Google Scholar]
  67. Woodcock RW, Johnson MB. Woodcock–Johnson Psycho-Educational Battery—Revised. DLM Teaching Resources; Allen, TX: 1989. [Google Scholar]

RESOURCES