Abstract
Objective
Using a general population sample, we aimed to examine the adult outcomes of children who present with severe problems with self-regulation defined as being concurrently rated highly on attention problems, aggressive behavior, and anxious-depression on the Child Behavior Checklist (the CBCL-Dysregulation Profile).
Method
2,076 children from 13 birth cohorts aged 4 to 16 were drawn from Dutch birth registries in 1983. Child Behavior Checklists (CBCLs) were completed by parents at baseline when children from the different cohorts were ages 4-16 and sampled every 2 years for the next 14-years. At year 14 both CBCL and DSM interview data were collected. Logistic regression was used to compare and contrast outcomes for children with and without dysregulation, as measured by the latent-class-defined CBCL-DP. Sex and age were covaried and concurrent DSM diagnoses were included in regression models.
Results
Presence of childhood CBCL-DP at Wave 1 was associated with increased rates of adult anxiety disorders, mood disorders, disruptive behavior disorders, and drug abuse 14 years later. After controlling for co-occurring disorders in adulthood, the associations with anxiety and disruptive behavior disorders with the CBCL-DP remained while the others were not significant.
Conclusions
A child reported to be in the CBCL-DP class is at increased risk for problems with regulating affect, behavior, and cognition in adulthood.
Keywords: CBCL, bipolar disorder, substance use, suicide, dysregulation
INTRODUCTION
Cross-sectional, longitudinal, genetic, neuroimaging, and treatment studies all have been used to improve the ability to understand the pathophysiology and treatment of children with ADHD, mood and anxiety disorders. Similar efforts focused on the study of child bipolar disorder, although reported, have not yet led to a consensus about how to define the syndrome. Adequate characterization of these children is a priority 1. These children have been labeled as “explosive” 2, “highly co-morbid” 3, and, most controversially, the broad phenotype of pediatric bipolar disorder or severe mood dysregulation4. Despite the rigorous debate about how best to diagnose these children, there is no debate about the severity of their problems5–11. The combination of severe anxious/depressed, impulsive-aggressive behavior, and cognitive problems (especially severe inattention) puts these children at high risk for negative outcomes such as adult personality disorders, substance abuse, and a broad range of adult psychopathology 8,12,13. Our group is just one of many who are invested in advancing how best to describe these children so that subsequent treatment research may lead to better outcomes. The objective of the study presented here was to examine the longitudinal outcomes of children with what we have described as suffering from ‘dysregulation’ 14 in a non-clinical, general population sample and to examine whether children with broadly-defined dysregulation would grow up to have a single disorder (such as bipolar disorder) or multiple disorders associated with impairment in self-regulation. Self-regulation as a concept is thought to be divided into three components: affective, behavioral, and cognitive. We theorize that children with impaired self-regulation (i.e. “dysregulation”) will be impaired in all three of these domains and, consequently, will grow up with a heterogeneous set of outcomes all related to this core process. Impairment in the ability to regulate mood (depression, bipolar disorder) or worry/cognitions (anxiety), to regulate behavior (disruptive behaviors, substance use), or to regulate cognitions (ADHD) would be possible outcomes. A further thought behind using the term “dysregulation” is our experience that clinicians use this term frequently to describe their patients without any empirical description of the term. Consequently, we sought to describe what it looks like so that it could be studied.
Dysregulation across affect, behavior, and cognition (ABC’s) – three aspects of self-regulatory behavior 15 can be measured using the Child Behavior Checklist (CBCL). Children who cannot perform these ABC’s of self-regulation are relatively common 16,17 and have been found to be at markedly increased risk for suicidal thoughts and behavior 5,10. Their characteristic profile on the Child Behavior Checklist (CBCL) shows elevated scores on the anxious-depressed (A/D), aggressive behavior (AB), and attention problems (AP) scales. This combination of problems has been shown to be heritable 17 and to be stable throughout childhood with that stability due to genetic factors 18. It has been shown to have specific molecular genetic linkage peaks in high risk samples 19, to be associated with specific problems in the serotonergic system 11 and abnormal thyroid function 6, to be observed by multiple informants 20, and to be highly associated with parent and self-reported suicidal thoughts and behavior 5. In short, there have been a number of studies that point to the potential validity of the CBCL-DP in the study of highly dysregulated children. However, more work needs to be done to determine the validity of this profile. We have termed this profile the CBCL-Dysregulation Profile (CBCL-DP) although it has also been called the CBCL-Juvenile Bipolar Disorder profile 17, the CBCL-Pediatric Bipolar Disorder profile 12, and the CBCL-mania proxy 21 because of reported associations with this pattern in children with bipolar disorder 9,22 and in children of bipolar parents 23. While associated with a bipolar disorder diagnosis when chronic irritability is allowed as the hallmark of a bipolar disorder diagnosis 22,the profile appears less predictive of DSM defined bipolar disorder in studies using discrete episodes as diagnostic 24 or prominent elation and/or grandiosity 10. With this in mind, it seemed likely that the adult outcomes of the CBCL-DP would likely have outcomes similar to Severe Mood Dysregulation (SMD) 8,12,13 or chronic, non-episodic irritability 25 than bipolar disorder 26.
To date there have been a variety of longitudinal studies that address the adult outcomes of single disorders such as ODD27, CD 28, ADHD 29, and affective disorders 30-32, some of which controlled for co-occurring disorders 33. These findings have helped the field move towards consensus on how to assess and treat these disorders. However, there is relatively less known about the outcomes of children who struggle with both severe internalizing and externalizing symptoms, such as children with co-occurring ADHD, ODD, and MDD, or children who meet criteria for the CBCL-DP. There have been two studies that have looked at the outcomes of the CBCL-DP. Meyer and colleagues provided the first evidence for the predictive validity of the CBCL-DP. They retrospectively examined CBCL data from 101 adult children of mothers with mood disorders and demonstrated that children with the CBCL-DP subsequently manifested elevated rates of mood disorders, substance abuse, and personality disorders in adulthood – all areas of impaired self-regulation 8. Biederman and colleagues recently performed a similar longitudinal follow-up in a high risk family study of 204 children with ADHD. They demonstrated that children with ADHD who also demonstrated the CBCL-DP had markedly elevated rates of bipolar disorder, major depressive disorder, disruptive behavior disorders, and overall worse outcomes 7.4 years later 12.
Although both the Meyer and Biederman studies are important steps towards understanding the long-term course, the generalizability of the findings are limited to the sample populations used. In both the children were selected from populations recruited for high rates of either mood disorders or attention disorders. In addition, the two groups used different approaches to defining the CBCL-DP phenotype. Biederman and colleagues used a sum of the T-scores for individuals greater than 210 (i.e. on average T-scores greater than 70 on each component scale). Meyer and colleagues used T-scores greater than 60 at any assessment time in childhood on each component scale.
Here we aim to build on prior work in two ways. First, we study the longitudinal course of DP in a representative general population sample followed prospectively for 14 years. Second, we will use our prior Latent Class Analyses (LCA) definition of the phenotype to assure we are following children who meet rigorous quantitative criteria for DP. The advantage to LCA is that it applies statistical criteria to which children score highly on all three scales rather than forcing researchers to establish arbitrarily a “best” cutpoint. We have previously used LCA to demonstrate the relative heritability of the CBCL-DP profile and its relation to suicidality in a cross-sectional model 5 and to demonstrate the cross-informant reliability of the profile 20.
We apply the LCA method to data on children who were first studied at ages 4–18 with the CBCL and subsequently assessed at 2-year increments for 14 years. We report on the adult outcomes using a standardized DSM interview 14-years later. Based on our model of the DP as measuring impaired self-regulation and building upon the previous 2 outcomes studies of these children, we hypothesized that the DP would: a) predict DSM diagnoses of substance abuse, bipolar disorder, and mood disorders and b) predict suicidal behavior in adulthood.
METHODS
Subjects and Procedure
Sample
This sample is based on data derived from a seven-wave longitudinal study of behavioral and emotional problems in children that started in 1983, the Zuid Holland study 34. While details of the study can be found elsewhere, the original sample of 2,600 children from 13 birth cohorts aged 4 to 16 was drawn from municipal registers that list all residents in the Dutch province of Zuid-Holland. A random sample of 100 children of each gender and age of Dutch nationality was drawn. Of the 2,447 parents contacted, 2,076 (i.e., 84.8%) provided usable data. Data from the 371 individuals who were sent forms, but who did not return forms, was not available. The sample in 1983 consisted of 49% boys. This sample was approached every two years until 1991 (Waves 1–5) and again in 1997 (Wave 6) and in 2007 (Wave 7). Data from Waves 1 – 6 were used as the primary outcome measure for this project as this allowed for the youngest child in the sample to become an adult, creating a total follow-up period of 14 years. Lifetime outcome data were also examined at Wave 7. The number of participants at each wave is shown in Table 1 along with the measures collected across the time points. Analyses were performed on those individuals who had DSM (n=1580) and suicide data at Wave 6 (4 missing, final n=1576). The mean age at Wave 1 was 9.9 (SD = 3.7) and the mean age at Wave 6 was 24.5 (SD = 3.8). Assessments at each wave were approved by the Committee for Medical Ethics, University Hospital/Erasmus University, Rotterdam. After explaining the procedures, each participant provided informed consent. De-identified data analysis was additionally approved by the Institutional Review Board of the University of Vermont.
Table 1.
Longitudinal data available in Zuid-Holland Longitudinal Study.
| Wave | Cohort Year | Ages | Years follow-up | CBCL | ASR | DSM Diagnosis |
|---|---|---|---|---|---|---|
| 1 | 1983 | 3–17 | 0 | 2076 | - | - |
| 2 | 1985 | 6–17 | 2 | 1412 | - | - |
| 3 | 1987 | 8–19 | 4 | 1374 | - | - |
| 4 | 1989 | 9–18 | 6 | 1116 | - | - |
| 5 | 1991 | 12–19 | 8 | 954 | - | - |
| 6 | 1997 | 18–32 | 14 | - | 1615 | 1580 |
| 7 | 2007 | 27–41 | 24 | 1365 | 1339 |
Note: ASR = Adult Self Report (from which suicidality data are taken); CBCL = Child Behavior Checklist.
Measures
At Waves 1 through 6 parents filled in the appropriate version of the ASEBA rating scales 35 including the Child Behavior Checklist (CBCL). After reaching age 18, the adult form of the CBCL, the Adult Behavior Checklist (ABCL) was used instead. Participants themselves filled out the Adult Self Report after they turned 18. Good reliability and validity estimates of the Dutch version of the ASEBA checklists have been confirmed 36. At Wave 6 data were collected on DSM diagnoses in the last 12 months as measured by standardized diagnostic interview using the computerized version of the Composite International Diagnostic Interview (CIDI) 37 and sections on paper of the Diagnostic Interview Schedule (DIS) for ADHD and ODD which not included in the CIDI. The CIDI has shown good test-retest and excellent inter-rater reliability for DSM diagnoses and is the most commonly used instrument for DSM diagnosis in epidemiological studies 38. In addition, lifetime diagnostic data on the CIDI and DIS were examined at Wave 7.
Attrition
The effects of attrition in the Zuid-Holland Longitudinal Study have been described in detail by the earlier reports 39. In general, differences were slight but some were statistically significant between participants who completed the study and those who dropped out before the CIDI interview was performed. Subjects who dropped out were significantly older (0.44 years, p = 0.023) when enrolled, were more often male (10% more males, p <0.001), and had a significantly lower SES (0.23 point difference on 6 point scale, significant at p = 0.004) but with low effect sizes (Cohen’s d = 0.12 for age, 0.03 for SES and Phi = 0.093 for sex) Total problems scores did not differ at study entry between completers and drop outs (0.86 point difference).
Data Analyses
LCA Determination of Profiles as Predictors
To determine which individuals fit into the DP profile, latent class analysis (LCA) was performed at Wave 1. A form of person-centered data analysis for categorical data, LCA uses the observed item endorsement profiles of respondents and attempts to derive a small number of mutually exclusive respondent classes, with each class having its own set of symptom endorsement probabilities 40. The parameter estimates that result from LCA are (1) probabilities of class membership assignment for individuals and (2) symptom endorsement probabilities for each class. Latent class models were fitted by means of an EM algorithm 41, using the program Latent Gold 4.04 42. Trichotomous (0, 1, 2) responses from each item of the AP, AB, and AD scales were entered as dichotomous variables (0 = 0 and 1 or 2 = 1), similar to previous procedures 5 and on the recommendation of the CBCL manuals 35. Models estimating 1-class through 9-class solutions were compared. To identify the best model, a combination of bootstrapping techniques and change in the Bayesian Information Criterion (BIC) between different solutions was employed. The BIC is a goodness-of-fit index that considers the rule of parsimony. The BIC and bootstrapping have both been demonstrated to yield solid fits in Monte Carlo simulation experiments 43. Sex and age were entered as covariates. Once an adequate model fit was obtained, the variables were dropped to see if model fits worsened. Once an adequate model fit was obtained at Wave 1, this model fit was used to estimate latent classes in subsequent waves using the “Known Class” function in Latent Gold. Individuals were therefore placed into classes at each wave based on whichever class had the highest probability of class membership in that wave. Individuals falling into the class with the highest elevations on AP, AB, and A/D were chosen as being in the DP class at each time point.
Wave 6 Outcome variables
Because this was a representative longitudinal sample with expectedly low rates of DSM diagnoses, our outcome variables were aggregated at the level of diagnostic categories. Follow-up diagnostic categories included the following: any disruptive disorder, any mood disorder, any anxiety disorder, any drug abuse, and any suicidal thoughts or behavior (which included any positive responses on the Adult Self Report of “I deliberately try to hurt or kill myself” or “I think about killing myself.”). Similar categories were generated for lifetime diagnoses at Wave 7.
Comparison of Predictors to Outcomes
Logistic regression was used in SPSS 15.0 with sex and class membership at Wave 1 as predictor (with Class 1, the lowest symptom class, as a reference) and each Wave 6 variable as outcomes. Next, a second set of logistic regressions was performed. Individuals who were in the DP class at ANY wave in Waves 1 – 5 were included as a “DP ever” group and compared to a reference group of individuals who were NEVER in a DP class in any Wave 1–5 (a “DP never” group). In addition, co-occurring diagnoses at Wave 6 were included in the model, which allowed for the unique contribution of dysregulation to eventual outcome even with sex and other co-occurring diagnoses considered. A final set of logistic regressions analyzed the relations between class membership in the DP class in Wave 1 and lifetime DSM diagnoses at Wave 7. This analysis additionally compared the lifetime DSM Wave 7 outcomes of children who had borderline clinical or clinical levels of Total Problems on the CBCL at Wave 1, but were not in the DP class.
RESULTS
Latent Class Analyses
A 7-class solution with sex and age as covariates fit the Wave 1 data best. Dropping age or sex worsened the model fit. Model fit indices are provided in Table S1, available online. The average T-scores for the resultant latent classes at Wave 1 are shown in Figure 1. The DP latent class consisted of 3.8% of the general population sample at Wave 1, and averaged 3.5% across all waves, consistent with previous work in general population and twin samples in the U.S. and the Netherlands 5,20. 7.4% of the girls and 7.8% of the boys fell into the DP latent class at sometime between Waves 1–5. The class membership from wave to wave was relatively consistent with an average of 46% (stdev = 9.56%) of the boys and 39.5% of the girls (stdev = 10.08) having persistent DP (i.e. being in the DP class in Waves 1–5). Overall Kappas for class membership ranged from wave to wave between 0.29 and 0.42 and were always significant at the p<0.001 level, but because these numbers reflect those with CBCL data, they are over-represented with those individuals who were sampled first when they were very young. Of those individuals who had CBCL data from one wave to the next, on average 43% of individuals who were in the DP class at one wave were in that class in the following wave. About 2% of individuals in the other classes, on average, moved into the DP class from one wave to the next.
Figure 1.

Mean T-scores for the 7 latent classes defined at Wave 1. Note: A/D = anxious-depressed; AGG = aggressive behavior; AP = attention problems; CBCL = Child Behavior Checklist; DP = dysregulation profile; RBB = rule breaking behavior; SOM = somatic complaints; THTP = thought problems; W/D = withdrawn
Longitudinal Logistic Regression Analyses
The number and proportion of individuals in each class at Wave 1 who were diagnosed at Wave 6 with any anxiety, mood, disruptive behavior, or drug abuse disorder or who had suicidality are presented in Table 2. Results of the logistic regression analysis to determine the significance of those findings are presented in Table 3. Table 3 demonstrates first, predictably, that female sex is associated with anxiety disorders, mood disorders, and suicidality at follow-up as evidenced by odds ratios significantly greater than 1. Male sex is associated with higher rates of drug abuse and disruptive behavior disorders as evidenced by odds ratios significantly less than 1. Being in one of the latent classes (2, 3, 6, and the DP class) at Wave 1 increase the likelihood of a diagnosis of any anxiety disorder at Wave 6. Only being in the DP latent class at Wave 1, however, is significantly associated with increased diagnosis of mood disorders, disruptive behavior disorders, and drug use at Wave 6. These significant associations also held for lifetime diagnoses at Wave 7. The incidence of DSM-IV bipolar I or II disorder at Wave 6 was very low -- only 6 individuals, or 0.4%, with DSM diagnoses at Wave 6. Of these 6 individuals, 1 of them (16.67%) was ever in the DP class. 3 of these individuals were lost to follow-up at Wave 7 and none met the criteria for bipolar disorder at Wave 7. Being over the borderline clinical cutpoint for Total Problems at Wave 1 but not in the DP class predicted only lifetime disruptive behavior disorders [p = 0.007, OR = 3.563 (1.40, 9.10)].
Table 2.
Number (proportion) of DSM-based diagnoses and suicidality at Wave 6 by Wave 1 Latent Class. Only participants with diagnostic and suicide data at Wave 6 included.
| Class | Wave 1 Number | Wave 6: Any Anxiety | Wave 6: Any Mood | Wave 6: Any DBD | Wave 6: Any Drug | Wave 6: Suicidali ty | Wave 6: Any Major Depressi on | Wave 6: Any Bipolar Disorder |
|---|---|---|---|---|---|---|---|---|
| Male |
||||||||
| Class 1. No symptoms | 112 (0.15) | 3 (0.03) | 1 (0.01) | 6 (0.05) | 1 (0.01) | 1 (0.01) | 1 (0.01) | 0 (0.00) |
| Class 2. Mild attention problems and social problems | 241 (0.33) | 14 (0.06) | 5 (0.02) | 15 (0.06) | 4 (0.02) | 6 (0.03) | 4 (0.02) | 1 (0.00) |
| Class 3. Moderate aggressive behavior with attention problems and rule-breaking | 162 (0.22) | 10 (0.06) | 2 (0.01) | 12 (0.07) | 3 (0.02) | 4 (0.02) | 2 (0.01) | 0 (0.00) |
| Class 4. Mild externalizing | 105 (0.14) | 6 (0.06) | 2 (0.02) | 7 (0.07) | 1 (0.01) | 1 (0.01) | 2 (0.02) | 0 (0.00) |
| Class 5. Mild internalizing | 41 (0.06) | 1 (0.02) | 1 (0.02) | 0 (0.00) | 1 (0.02) | 0 (0.00) | 1 (0.02) | 0 (0.00) |
| Class 6. Moderate internalizing with attention problems | 32 (0.04) | 1 (0.03) | 2 (0.06) | 3 (0.09) | 1 (0.03) | 1 (0.03) | 2 (0.06) | 0 (0.00) |
| Class 7. DP Class | 36 (0.05) | 2 (0.06) | 2 (0.06) | 6 (0.17) | 4 (0.11) | 1 (0.03) | 2 (0.06) | 0 (0.00) |
| Total Males | 729 (1.0) | 37 (0.05) | 15 (0.02) | 49 (0.07) | 15 (0.02) | 14 (0.02) | 14 (0.02) | 1 (0.00) |
| Female | ||||||||
| Class 1. No symptoms | 270 (0.32) | 35 (0.13) | 16 (0.06) | 3 (0.01) | 1 (<0.01) | 7 (0.02) | 16 (0.06) | 0 (0.00) |
| Class 2. Mild attention problems and social problems | 126 (0.15) | 24 (0.19) | 13 (0.10) | 0 (0.00) | 0 (0.00) | 3 (0.02) | 12 (0.10) | 1 (0.01) |
| Class 3. Moderate aggressive behavior with attention problems and rule-breaking | 70 (0.08) | 22 (0.31) | 5 (0.07) | 3 (0.04) | 0 (0.00) | 5 (0.07) | 4 (0.06) | 1 (0.01) |
| Class 4. Mild externalizing | 113 (0.13) | 17 (0.15) | 12 (0.11) | 2 (0.02) | 1 (0.01) | 5 (0.04) | 10 (0.09) | 2 (0.02) |
| Class 5. Mild internalizing | 160 (0.19) | 21 (0.13) | 14 (0.09) | 1 (0.01) | 0 (0.00) | 1 (0.01) | 13 (0.08) | 1 (0.01) |
| Class 6. Moderate internalizing with attention problems | 87 (0.10) | 23 (0.26) | 8 (0.09) | 0 (0.00) | 0 (0.00) | 2 (0.02) | 7 (0.08) | 0 (0.00) |
| Class 7. DP Class | 21 (0.02) | 6 (0.28) | 3 (0.14) | 1 (0.05) | 1 (0.05) | 2 (0.10) | 3 (0.14) | 0 (0.00) |
| Total Females | 847 (1.0) | 148 (0.17) | 71 (0.08) | 10 (0.01) | 3 (<0.01) | 25 (0.03) | 65 (0.08) | 5 (0.01) |
Note: Proportions within each class for each sex are in parentheses. DBD = Disruptive Behavior Disorder, DP = Dysregulation Profile.
Table 3.
Results of logistic regression using all classes from Wave 1 predicting diagnostic information at Wave 6.
| Any Anxiety Disorder | Any Mood Disorder | Any DBD | Any Drug Abuse | Any Suicidality | Any Major Depression | Any Bipolar Disorder | |
|---|---|---|---|---|---|---|---|
| Variables | OR | OR | OR | OR | OR | OR | ORa |
| Sex b reference class = male | 4.99 (3.34 – 7.45) | 4.68 (2.58 – 8.48) | 0.21 (0.10 – 0.42) | 0.20 (0.05 – 0.74) | 2.00 (1.01 – 3.98) | 4.43 (2.40 – 8.27) | 5.17 (0.63 – 43.32) |
| Class 1. No symptoms | Ref | Ref | Ref | Ref | Ref | Ref | 0 (0) |
| Class 2. Mild attention problems and social problems | 1.77 (1.08 – 2.90) | 1.80 (0.90 – 3.63) | 1.07 (0.45 – 2.53) | 1.26 (0.22 – 7.12) | 1.50 (0.56 – 4.05) | 1.57 (0.76 – 3.22) | 1.65 (0.30 – 9.05) |
| Class 3. Moderate aggressive behavior with attention problems and rule– breaking | 2.72 (1.59 – 4.63) | 1.16 (0.46 – 2.89) | 1.66 (0.70 – 3.97) | 1.44 (0.23 – 8.93) | 2.42 (0.89 – 6.59) | 0.97 (0.37 – 2.55) | 1.16 (0.14 – 10.0) |
| Class 4. Mild externalizing | 1.36 (0.78 – 2.39) | 1.86 (0.89 – 3.89) | 1.33 (0.51 – 3.45) | 1.31 (0.18 – 9.46) | 1.77 (0.63 – 4.98) | 1.57 (0.73 – 3.37) | 3.13 (0.57 – 17.2) |
| Class 5. Mild internalizing | 1.02 (0.58 – 1.78) | 1.60 (0.78 – 3.28) | 0.25 (0.03 – 1.97) | 1.15 (0.10 – 12.83) | 0.22 (0.03 – 1.81) | 1.48 (0.71 – 3.10) | 1.37 (0.16 – 11.82) |
| Class 6. Moderate internalizing with attention problems | 2.30 (1.30 – 4.06) | 1.95 (0.86 – 4.40) | 1.13 (0.30 – 4.28) | 1.70 (0.15 – 19.06) | 1.20 (0.31 – 4.60) | 1.73 (0.75 – 4.02) | 0 (0) |
| Class 7. DP Class | 2.45 (1.04 – 5.77) | 3.30 (1.14 – 9.62) | 3.71 (1.29 – 10.67) | 11.62 (2.13 – 63.28) | 3.30 (0.83 – 13.06) | 4.44 (2.40 – 8.23) | 0 (0) |
Note: Class 1 is the reference group against which the odds ratios are calculated. Odds ratios (OR) are from logistic regression with 95% confidence intervals. OR significant at p < 0.05 are in bold. Beta = unstandardized beta from logistic regression; DBD = Disruptive Behavior Disorder, DP = Dysregulation Profile; ref = reference class.
Unable to fit with logistic regression because of the small cell numbers, these are based on simple contingency tables
Sex is coded such that larger numbers indicate higher ratios in females.
Next, the results from the more restricted logistic regressions are presented in Table 4. The number of individuals classified as being in the DP class was increased by allowing for ever having been in the DP latent class (Ever in DP latent class in Table 4). Wave 6 other diagnoses were allowed into the logistic regression as predictors to examine whether there are any unique contributions to having been in the DP class versus the DP class being better characterized as a “highly comorbid” group. Table 4 demonstrates that allowing all concurrent diagnoses to compete with ever having been in the DP class does reduce some of the associations. However, significant associations remain. As expected, the co-occurrence of symptoms that have been observed previously are present including anxiety with mood disorders, disruptive behavior disorders (DBDs) with drug abuse, and DBDs with anxiety. Sex is also associated independently with all disorders, although interestingly not with suicidality. Ever having been in the DP class, however, remains significantly associated with anxiety disorders and DBDs at the p < 0.05 level. While numerically similar to the order of magnitude seen with anxiety and DBDs, the odds ratios for association of the DP class with mood disorders, drug abuse, and suicidality are not significant (p = 0.08, 0.06, and 0.08, respectively).
Table 4.
Results of logistic regression for ever being in the Dysregulation Profile class across Waves 1–5 predicting diagnostic information at Wave 6 controlling for concurrent diagnoses at Wave 6.
| Any Anxiety Disorder | Any Mood Disorder | Any DBD | Any Drug Abuse | Any Suicidality | |
|---|---|---|---|---|---|
| Variables | OR | OR | OR | OR | OR |
| Sex a reference class = male | 3.99 (2.67 – 5.97) | 4.51 (2.36 – 8.63) | 0.13 (0.06 – 0.27) | 0.13 (0.03 – 0.52) | 0.92 (0.42 – 2) |
| Any Anxiety Disorder | n/a | 4.72 (2.91 – 7.67) | 2.64 (1.18 – 5.92) | 1.02 (0.23 – 4.57) | 2.79 (1.29 – 6.05) |
| Any Mood Disorder | 4.78 (2.95 – 7.75) | n/a | 2.42 (0.85 – 6.87) | 10.30 (2.72 – 39.07) | 13.49 (6.2 – 29.34) |
| Any DBD | 2.64 (1.20 – 5.78) | 2.77 (1.02 – 7.5) | n/a | 4.54 (1.43 – 14.47) | 2.68 (0.82 – 8.69) |
| Any Drug Abuse | 0.94 (0.21 – 4.09) | 10.06 (2.76 – 36.74) | 4.67 (1.44 – 15.20) | n/a | 0.87 (0.13 – 5.62) |
| Ever in DP Class | 1.99 (1.15 – 3.45) | 1.9 (0.92 – 3.92) | 2.16 (1.03 – 4.54) | 3.00 (0.95 – 9.44) | 2.33 (0.91 – 5.93) |
Note: Odds ratios (OR) are from logistic regression with 95% confidence intervals. For Ever in DP Class and all diagnostic information, never being in the class and not having the diagnosis are the reference categories. Odds ratios significant at p < 0.05 are in bold. Some cells are marked as “n/a” because a variable cannot be in the model as an outcome and a predictor. Bipolar and Major Depression could not be included in this analysis because they are nested within the Any Mood Disorder category. Beta = unstandardized beta from logistic regression; DBD = Disruptive Behavior Disorder, DP = Dysregulation Profile.
Sex is coded such that larger numbers indicate higher ratios in females.
DISCUSSION
Childhood dysregulation as measured by the CBCL-DP is significantly associated with a wide variety of psychopathology in adulthood, including anxiety disorders, mood disorders, disruptive behavior disorders, and drug abuse. It is remarkable that any specific pattern of behavior reported by parents in childhood would be associated with such a wide variety of adult psychopathologic conditions. We considered it likely that the addition of current adult diagnoses into the analysis would cause these associations to disappear. What we found, however, was that the addition of current adult diagnoses into the model, while weakening the association with having been in the DP class and adult outcomes, does not negate them. These findings suggest that childhood impairment in affective, behavioral, and cognitive domains can manifest as multiple types of adult psychiatric disorders. This is consistent with the findings from Leibenluft, Stringaris, and others demonstrating that the end result of chronic, non-episodic irritability appears to be depressive disorders and disruptive behavior disorders 13,25,27,44,45. The finding of the significant co-occurrence of internalizing and externalizing disorders in adulthood for children who were in the DP class as children is also consistent with findings of mood lability in childhood being a predictor of co-occurring internalizing/externalizing disorders 46. In fact, if one generates an outcome that is any internalizing along with any externalizing disorder by including those individuals who scored positive for either a mood or an anxiety disorder and any of the disruptive behavior disorders, the odds ratio for Wave 1 prediction of this outcome was 19.04 (1.89 – 191.67) for the DP class. Thus, it appears that the CBCL-DP predicts later disorders related to mood, cognitive, and behavioral dysregulation, and their co-occurrence -- much the same as the concept of mood lability, “non-episodic irritability” and the SMD construct 13.
The CBCL-DP is postulated to reflect global deficits in self-regulation, particularly those processes related to effortful control as defined by Eisenberg and others 47. These difficulties represent a temperamentally based risk factor that has been found to be associated with broad array of psychopathology. Clark has hypothesized that much of psychopathology and patterns of comorbidity can be conceptualized as combinations of three main temperament dimensions that reflect tendencies of negative affectivity, positive affectivity, along with self-regulatory abilities 48. While tendencies toward positive and negative affects are generally inversely related, we have previously identified, using person-based analyses, groups of children who manifest relatively high levels of both approach and avoidance in the context of reduced regulatory abilities 49. This group, defined in temperament terms, bears remarkable similarity with the CBCL-DP group, which has obviously been defined from a psychopathology scale. Understanding these parallels can provide some insight as to how this particular group manifests such a seemingly wide range of DSM defined disorders over time.
In addition to the positive associations between the DP and subsequent psychiatric disorders, there are other findings worth highlighting. The DP class is clearly the latent class most associated with later drug use. Furthermore, much of this association is mediated through the relations of mood and disruptive behavior disorders. We examined this further by including alcohol abuse into a model. The DP class was not association with alcohol use disorders. Instead, Class 2, the class with mild attention problems and social problems, was most associated with alcohol abuse or dependence. This is consistent with disorders of drug use being more related to core difficulty in self-regulation skills separable from alcohol use 50 and alcohol-related disorders being more related to social problems and internalizing symptoms as has been observed in the Collaborative Study on the Genetics of Alcoholism (COGA) study 51. Given the documented links between childhood externalizing disorders and later substance use 52, 53, it is possible that some of this association is due to children with combined internalizing-externalizing problems such as those children with the DP. These data indicate that childhood dysregulation is a pathway to substance use disorders.
Given earlier findings from our group and others on the association between CBCL-DP and childhood suicidal thoughts and behavior 5,10, it is a bit surprising that the association with suicidal thoughts or behavior did not reach significance. The relation between current suicidality and current anxiety and mood disorders is strong enough to overshadow the prediction of current suicidality from DP in the past. Alternatively, it might be that associations between the DP and suicidality are not as strong in adulthood as they are in childhood.
Similar to the findings of Meyer et al, we demonstrated adult outcomes for the DP consistent with problems in self-regulatory behavior. While Meyer and colleagues and Biederman et al 12 demonstrated a significant association with the DP and bipolar disorder in adulthood, we did not demonstrate that here. The reason for this discrepancy is possibly because both Meyer and Biederman had a higher prevalence of bipolar disorder in adulthood in their samples that were selected from at-risk populations rather than a general population sample as was used here. Only 1 out of the 6 individuals who went on to have bipolar disorder in adulthood had demonstrated the DP in the 14 prior years. It is possible that the method of diagnosing bipolar disorder varied across these studies. The interview used here (the CIDI) measures bipolar disorder using DSM/ICD-9 criteria. The Biederman et al12 and Meyer et al 8 studies used a different instrument, the SCID, as the assessment tool which may explain some of the variance. Another possibility is that the use of LCA to determine the DP class might have been the determining factor as the other two studies used T-scores. We re-analyzed these data using a T-score cutpoint approach and found that the results were, essentially, unchanged. An alternative interpretation is that, at least in a general population sample, children in the DP Class are very unlikely to go on to have bipolar disorder in adulthood. Given that 57 individuals fell into the DP Class, with no individuals eventually having bipolar disorder, one would expect that the maximum percentage who might go on to have bipolar disorder in this population would be 5.3%, or 3 out of 57, based on the “rule of three” used when there are zero numerators in an equation 54. Thus, while this might be a risk profile for bipolar disorder in a high-risk sample, it appears to be a risk for other problems in a general population sample.
From this we can derive that all studies thus far of the DP either in clinically-relevant populations or, now, in a general population sample point to heterotypic outcomes of these children in adulthood. Could it be that the profile is simply measuring comorbidity or severity? There is clearly more work to be done. The Dysregulation Profile is consistently confounded with severity (i.e. almost all children with this profile are profoundly affected) and comorbidity. So how will one ever separate out the role of having this profile from having either generally severe psychopathology or multiple diagnoses? Severity of symptoms is certainly important, as is comorbidity 55. The only arguments in the literature currently to suggest that the CBCL-DP might be more than the sum of its component parts is work by McGough et al 19 who demonstrated different genetic markers associated with the CBCL-DP than with bipolar disorder, depression, or attention problems. Similarly, our group has demonstrated a different genetic architecture to the CBCL-DP than to AP, AGG, or A/D alone 17,18. Work presented here, however, demonstrates that having a high total problems score when not in the DP class, leads to a different pattern of outcomes in adulthood. Children with elevated scores overall but without the pattern of elevated scores on attention, aggression, and anxious-depression end up at risk for disruptive behavior disorders in adulthood, but not drug abuse or mood disorders. However, more work needs to be done to examine the relations between comorbidity, severity, and the DP. We are currently conducting studies of these children using severity-matched controls and are examining alternative models of co-occurring disorders but this work is not yet complete.
It’s worthwhile to point out the limitations of the present study. Overall, the incidence of certain outcomes was low, necessitating the collapsing of several diagnoses into combined categories. Also, attrition could have played a role, given that children who had worse outcomes may be more difficult to find for follow-up. It is possible that the attrition of significantly more males, individuals with lower SES, and individuals who were slightly older may have effected the follow-up diagnostic determination. However, the effect sizes of these differences were at a level that is unlikely to have confounded the results. Further arguing against attrition as having a major role is the fact that individuals who followed up had the same total problems score at enrollment as those who did not follow-up. Nevertheless, it could have contributed to less severe outcomes in the individuals who did report at 14-years. However, even with systematic attrition, modeling within longitudinal datasets has suggested that the effects on later regressions are minimal 56.
There remains the possibility that, while seemingly representative of the Dutch population at baseline, the sample at follow-up may not be representative due to attrition or to the ascertainment strategy. For example, the 4:1 female:male ratio in mood disorders and the 1:7 female:male ratio in disruptive behavior disorders is higher than would be predicted. However, because the regression models controlled for sex, the associations with the DP and later outcomes appear sound. With regard to the analyses of suicidal behavior, the suicide questions from the ASR unfortunately may be endorsed by individuals engaging in non-suicidal self-injurious behavior, so it is not particularly specific to actual suicidal thoughts or behavior. However, work that we have done on over 4700 individuals with concurrently collected ASR and Beck Depression Inventory (which has questions more specific to suicidal thoughts and/or behavior), demonstrates a high correlation (r = 0.6) between the two suicidal behavior items on the ASR and suicidal ideation items on the BDI (unpublished data).
An important limitation and concern of this work is the concern that these children have problems in multiple domains – that is, children with elevations on all three of these scales often have elevations in the other scales of the CBCL. We have attempted to isolate children with specific elevations on these three scales by performing the latent class analysis with the items from those scales. Inclusion of Total Problems into any of the logistic regression models drowns out any specificity to any outcome (i.e. no latent classes predict outcomes when total problems are in the models because it accounts for so much of the variance and because it is so highly correlated, by definition, with problems in any domain). It remains difficult to study the most severely affected children with reported problems in nearly every domain, but we have attempted to look at the patterns of outcome when the problems are not specific to the DP Class but rather to generally high total problems It’s also worthwhile to point out that children with many “clearly defined” psychiatric conditions using other constructs, if severely affected, will have problems in many domains of psychopathology.
Regardless of these limitations, the finding of a risk profile which has been associated with poor concurrent symptoms in childhood that further predicts later severe adult outcomes argues for continued study of childhood dysregulation using the CBCL.
Supplementary Material
Acknowledgments
This work was partially supported by NIMH Grant K08MH082116 (R. Althoff, PI). The NIMH had no role in study design; collection, analysis or interpretation of data; in writing of the report; or in the decision to submit the paper for publication.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Althoff RR. Dysregulated children reconsidered. J Am Acad Child Adolesc Psychiatry. 2010;49(4):302–305. [PubMed] [Google Scholar]
- 2.Greene RW. The Explosive Child: A New Approach for Understanding and Parenting Easily Frustrated, Chronically Inflexible Children. 2. New York: Harper Collins; 2001. [Google Scholar]
- 3.Carlson GA. Who are the children with severe mood dysregulation, a.k.a. "rages"? Am J Psychiatry. 2007;164(8):1140–1142. doi: 10.1176/appi.ajp.2007.07050830. [DOI] [PubMed] [Google Scholar]
- 4.Leibenluft E, Charney DS, Towbin KE, Bhangoo RK, Pine DS. Defining clinical phenotypes of juvenile mania. Am J Psychiatry. 2003;160(3):430–437. doi: 10.1176/appi.ajp.160.3.430. [DOI] [PubMed] [Google Scholar]
- 5.Althoff RR, Rettew DC, Faraone SV, Boomsma DI, Hudziak JJ. Latent class analysis shows strong heritability of the child behavior checklist-juvenile bipolar phenotype. Biol Psychiatry. 2006;60(9):903–911. doi: 10.1016/j.biopsych.2006.02.025. [DOI] [PubMed] [Google Scholar]
- 6.Holtmann M, Duketis E, Goth K, Poustka L, Boelte S. Severe affective and behavioral dysregulation in youth is associated with increased serum TSH. J Affect Disord. 2009;129(1–2):184–188. doi: 10.1016/j.jad.2009.06.009. [DOI] [PubMed] [Google Scholar]
- 7.Holtmann M, Goth K, Wockel L, Poustka F, Bolte S. CBCL-pediatric bipolar disorder phenotype: severe ADHD or bipolar disorder? J Neural Transm. 2008;115(2):155–161. doi: 10.1007/s00702-007-0823-4. [DOI] [PubMed] [Google Scholar]
- 8.Meyer SE, Carlson GA, Youngstrom E, et al. Long-term outcomes of youth who manifested the CBCL-Pediatric Bipolar Disorder phenotype during childhood and/or adolescence. J Affect Disord. 2009;113(3):227–235. doi: 10.1016/j.jad.2008.05.024. [DOI] [PubMed] [Google Scholar]
- 9.Mick E, Biederman J, Pandina G, Faraone SV. A preliminary meta-analysis of the child behavior checklist in pediatric bipolar disorder. Biol Psychiatry. 2003;53(11):1021–1027. doi: 10.1016/s0006-3223(03)00234-8. [DOI] [PubMed] [Google Scholar]
- 10.Volk HE, Todd RD. Does the Child Behavior Checklist juvenile bipolar disorder phenotype identify bipolar disorder? Biol Psychiatry. 2007;62(2):115–120. doi: 10.1016/j.biopsych.2006.05.036. [DOI] [PubMed] [Google Scholar]
- 11.Zepf FD, Wockel L, Poustka F, Holtmann M. Diminished 5-HT functioning in CBCL pediatric bipolar disorder-profiled ADHD patients versus normal ADHD: susceptibility to rapid tryptophan depletion influences reaction time performance. Hum Psychopharmacol. 2008;23(4):291–299. doi: 10.1002/hup.934. [DOI] [PubMed] [Google Scholar]
- 12.Biederman J, Petty CR, Monuteaux MC, et al. The child behavior checklist-pediatric bipolar disorder profile predicts a subsequent diagnosis of bipolar disorder and associated impairments in ADHD youth growing up: a longitudinal analysis. J Clin Psychiatry. 2009;70(5):732–740. doi: 10.4088/JCP.08m04821. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Brotman MA, Schmajuk M, Rich BA, et al. Prevalence, clinical correlates, and longitudinal course of severe mood dysregulation in children. Biol Psychiatry. 2006;60(9):991–997. doi: 10.1016/j.biopsych.2006.08.042. [DOI] [PubMed] [Google Scholar]
- 14.Ayer L, Althoff R, Ivanova M, et al. Child Behavior Checklist Juvenile Bipolar Disorder (CBCL-JBD) and CBCL Posttraumatic Stress Problems (CBCL-PTSP) scales are measures of a single dysregulatory syndrome. J Child Psychol Psychiatry. 2009;50(10):1291–1300. doi: 10.1111/j.1469-7610.2009.02089.x. [DOI] [PubMed] [Google Scholar]
- 15.Fitzsimons GM, Bargh JA. Automatic Self-Regulation. In: Baumeister RF, Vohs KD, editors. Handbook of Self-Regulation. New York: Guilford Press; 2004. pp. 151–170. [Google Scholar]
- 16.Holtmann M, Bolte S, Goth K, et al. Prevalence of the Child Behavior Checklist-pediatric bipolar disorder phenotype in a German general population sample. Bipolar Disord. 2007;9(8):895–900. doi: 10.1111/j.1399-5618.2007.00463.x. [DOI] [PubMed] [Google Scholar]
- 17.Hudziak JJ, Althoff RR, Derks EM, Faraone SV, Boomsma DI. Prevalence and genetic architecture of Child Behavior Checklist-juvenile bipolar disorder. Biol Psychiatry. 2005;58(7):562–568. doi: 10.1016/j.biopsych.2005.03.024. [DOI] [PubMed] [Google Scholar]
- 18.Boomsma DI, Rebollo I, Derks EM, et al. Longitudinal stability of the CBCL-juvenile bipolar disorder phenotype: A study in Dutch twins. Biol Psychiatry. 2006;60(9):912–920. doi: 10.1016/j.biopsych.2006.02.028. [DOI] [PubMed] [Google Scholar]
- 19.McGough JJ, Loo SK, McCracken JT, et al. CBCL Pediatric Bipolar Disorder Profile and ADHD: Comorbidity and Quantitative Trait Loci Analysis. J Am Acad Child Adolesc Psychiatry. 2008;47(10):1151–1157. doi: 10.1097/CHI.0b013e3181825a68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Althoff RR, Rettew DC, Ayer LA, Hudziak JJ. Latent Class Cross-informant Agreement of the Child Behavior Checklist Dysregulation Profile. Psychiatry Res. 2010 doi: 10.1016/j.psychres.2010.05.002. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Galanter CA, Carlson GA, Jensen PS, et al. Response to methylphenidate in children with attention deficit hyperactivity disorder and manic symptoms in the multimodal treatment study of children with attention deficit hyperactivity disorder titration trial. J Child Adolesc Psychopharmacol. 2003;13(2):123–136. doi: 10.1089/104454603322163844. [DOI] [PubMed] [Google Scholar]
- 22.Faraone SV, Althoff RR, Hudziak JJ, Monuteaux M, Biederman J. The CBCL predicts DSM bipolar disorder in children: a receiver operating characteristic curve analysis. Bipolar Disord. 2005;7(6):518–524. doi: 10.1111/j.1399-5618.2005.00271.x. [DOI] [PubMed] [Google Scholar]
- 23.Wals M, Hillegers MH, Reichart CG, Ormel J, Nolen WA, Verhulst FC. Prevalence of psychopathology in children of a bipolar parent. J Am Acad Child Adolesc Psychiatry. 2001;40(9):1094–1102. doi: 10.1097/00004583-200109000-00019. [DOI] [PubMed] [Google Scholar]
- 24.Diler RS, Birmaher B, Axelson D, et al. The Child Behavior Checklist (CBCL) and the CBCL-bipolar phenotype are not useful in diagnosing pediatric bipolar disorder. J Child Adolesc Psychopharmacol. 2009;19(1):23–30. doi: 10.1089/cap.2008.067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Stringaris A, Cohen P, Pine DS, Leibenluft E. Adult outcomes of youth irritability: a 20-year prospective community-based study. Am J Psychiatry. 2009;166(9):1048–1054. doi: 10.1176/appi.ajp.2009.08121849. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Birmaher B, Axelson D, Goldstein B, et al. Four-year longitudinal course of children and adolescents with bipolar spectrum disorders: the Course and Outcome of Bipolar Youth (COBY) study. Am J Psychiatry. 2009;166(7):795–804. doi: 10.1176/appi.ajp.2009.08101569. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Stringaris A, Goodman R. Longitudinal outcome of youth oppositionality: irritable, headstrong, and hurtful behaviors have distinctive predictions. J Am Acad Child Adolesc Psychiatry. 2009;48(4):404–412. doi: 10.1097/CHI.0b013e3181984f30. [DOI] [PubMed] [Google Scholar]
- 28.Robins LN. Deviant children grown up. Eur Child Adolesc Psychiatry. 1996;5 (Suppl 1):44–46. doi: 10.1007/BF00538543. [DOI] [PubMed] [Google Scholar]
- 29.Mannuzza S, Klein RG. Long-term prognosis in attention-deficit/hyperactivity disorder. Child Adolesc Psychiatr Clin N Am. 2000;9(3):711–726. [PubMed] [Google Scholar]
- 30.Fichter MM, Kohlboeck G, Quadflieg N, Wyschkon A, Esser G. From childhood to adult age: 18-year longitudinal results and prediction of the course of mental disorders in the community. Soc Psychiatry Psychiatr Epidemiol. 2009;44(9):792–803. doi: 10.1007/s00127-009-0501-y. [DOI] [PubMed] [Google Scholar]
- 31.Geller B, Zimerman B, Williams M, Bolhofner K, Craney JL. Adult psychosocial outcome of prepubertal major depressive disorder. J Am Acad Child Adolesc Psychiatry. 2001;40(6):673–677. doi: 10.1097/00004583-200106000-00012. [DOI] [PubMed] [Google Scholar]
- 32.Todd RD, Botteron KN. Etiology and genetics of early-onset mood disorders. Child Adolesc Psychiatr Clin N Am. 2002;11(3):499–518. doi: 10.1016/s1056-4993(02)00013-5. [DOI] [PubMed] [Google Scholar]
- 33.Copeland WE, Shanahan L, Costello EJ, Angold A. Childhood and adolescent psychiatric disorders as predictors of young adult disorders. Arch Gen Psychiatry. 2009;66(7):764–772. doi: 10.1001/archgenpsychiatry.2009.85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Verhulst FC, Akkerhuis GW, Althaus M. Mental health in Dutch children: (I). A cross-cultural comparison. Acta Psychiatr Scand Suppl. 1985;323:1–108. doi: 10.1111/j.1600-0447.1985.tb10512.x. [DOI] [PubMed] [Google Scholar]
- 35.Achenbach TM, Rescorla LA. Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, & Families; 2001. [Google Scholar]
- 36.Verhulst FC, van der Ende J, Koot HM. Handleiding voor de CBCL/4-18. Rotterdam: Sophia Kinderziekenhuis/Academisch Ziekenhuis Rotterdam/Erasmus Universiteit Rotterdam, Afdeling Kinder-en jeugdpsychiatrie; 1996. [Google Scholar]
- 37.Essau CA, Wittchen H-U. An overview of the Composite International Diagnostic Interview (CIDI) Int J Meth Psychiatric Res. 1993;3:79–85. [Google Scholar]
- 38.Andrews G, Peters L. The psychometric properties of the Composite International Diagnostic Interview. Soc Psychiatry Psychiatr Epidemiol. 1998;33(2):80–88. doi: 10.1007/s001270050026. [DOI] [PubMed] [Google Scholar]
- 39.Hofstra MB, Van Der Ende JAN, Verhulst FC. Adolescents' self-reported problems as predictors of psychopathology in adulthood: 10-year follow-up study. Br J Psychiatry. 2001;179(3):203–209. doi: 10.1192/bjp.179.3.203. [DOI] [PubMed] [Google Scholar]
- 40.Goodman L. Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika. 1974;61:215–231. [Google Scholar]
- 41.Dempster A, Laird N, Rubin D. Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B. 1977;39:1–38. [Google Scholar]
- 42.Vermunt JK, Magidson J. Latent GOLD User’s Guide. Belmont, MA: Statistical Innovations Inc; 2000. [Google Scholar]
- 43.Nylund KL, Asparouhov T, Muthen BO. Deciding on the number of clases in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling. 2007;14(4):535–569. [Google Scholar]
- 44.Leibenluft E, Cohen P, Gorrindo T, Brook JS, Pine DS. Chronic versus episodic irritability in youth: a community-based, longitudinal study of clinical and diagnostic associations. J Child Adolesc Psychopharmacol. 2006;16(4):456–466. doi: 10.1089/cap.2006.16.456. [DOI] [PubMed] [Google Scholar]
- 45.Stringaris A, Goodman R. Three dimensions of oppositionality in youth. J Child Psychol Psychiatry. 2009;50(3):216–223. doi: 10.1111/j.1469-7610.2008.01989.x. [DOI] [PubMed] [Google Scholar]
- 46.Stringaris A, Goodman R. Mood lability and psychopathology in youth. Psychol Med. 2009;39(8):1237–1245. doi: 10.1017/S0033291708004662. [DOI] [PubMed] [Google Scholar]
- 47.Eisenberg N, Spinrad TL. Emotion-related regulation: sharpening the definition. Child Dev. 2004;75(2):334–339. doi: 10.1111/j.1467-8624.2004.00674.x. [DOI] [PubMed] [Google Scholar]
- 48.Clark LA. Temperament as a unifying basis for personality and psychopathology. J Abnorm Psychol. 2005;114(4):505–521. doi: 10.1037/0021-843X.114.4.505. [DOI] [PubMed] [Google Scholar]
- 49.Rettew DC, Althoff RR, Dumenci L, Ayer L, Hudziak JJ. Latent profiles of temperament and their relations to psychopathology and wellness. J Am Acad Child Adolesc Psychiatry. 2008;47(3):273–281. doi: 10.1097/CHI.0b013e318160b403. [DOI] [PubMed] [Google Scholar]
- 50.Fernandez-Serrano MJ, Perez-Garcia M, Schmidt Rio-Valle J, Verdejo-Garcia A. Neuropsychological consequences of alcohol and drug abuse on different components of executive functions. J Psychopharmacol. 2009 doi: 10.1177/0269881109349841. [DOI] [PubMed] [Google Scholar]
- 51.Schuckit MA, Tipp JE, Bucholz KK, et al. The life-time rates of three major mood disorders and four major anxiety disorders in alcoholics and controls. Addiction. 1997;92(10):1289–1304. [PubMed] [Google Scholar]
- 52.Wilens TE. The nature of the relationship between attention-deficit/hyperactivity disorder and substance use. J Clin Psychiatry. 2007;68 (Suppl 11):4–8. [PubMed] [Google Scholar]
- 53.Wilens TE, Biederman J, Adamson J, et al. Association of bipolar and substance use disorders in parents of adolescents with bipolar disorder. Biol Psychiatry. 2007;62(2):129–134. doi: 10.1016/j.biopsych.2006.11.022. [DOI] [PubMed] [Google Scholar]
- 54.Hanley JA, Lippman-Hand A. If nothing goes wrong, is everything all right? Interpreting zero numerators. JAMA. 1983;249(13):1743–1745. [PubMed] [Google Scholar]
- 55.Caron C, Rutter M. Comorbidity in child psychopathology: Concepts, issues and research strategies. J Child Psychol Psychiatry. 1991;32(7):1063–1080. doi: 10.1111/j.1469-7610.1991.tb00350.x. [DOI] [PubMed] [Google Scholar]
- 56.Wolke D, Waylen A, Samara M, et al. Selective drop-out in longitudinal studies and non-biased prediction of behaviour disorders. Br J Psychiatry. 2009;195(3):249–256. doi: 10.1192/bjp.bp.108.053751. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
