Abstract
Background
Oppositional Defiant Disorder (ODD) is a disruptive behavioral disorder; however, increasing evidence emphasizes irritable mood as a primary symptom of ODD.
Objectives
This study investigated whether heterogeneous groups (classes) of individuals can be differentiated based on ODD sub-dimensions (irritability and defiance) or on overall ODD symptoms longitudinally. We also examine associations between ODD trajectory class and comorbid substance use (heavy episodic drinking, cannabis use), mental health (depression and anxiety) and behavioral symptoms (ADHD, aggression and substance use) in both adolescence and young adulthood (controlling for adolescent levels of each of these concerns).
Method
Data were from a randomly recruited community sample of 662 Canadian youth (T1 ages 12–18) followed biennially for 10 years (T6 ages 22–29).
Results
Growth mixture models revealed trajectories classes of ODD based on severity of symptoms. A three-class solution provided the best fit with Low (n = 119; 18%), Moderate (n = 473; 71.5%), and High (n = 70; 10.6%) ODD classes. Class trajectory differences were similarity based on symptoms severity (rather than type) for symptom sub-dimensions (irritability defiance). Adolescent and young adult substance use, mental health symptoms, and behavioral problems were significantly higher for the High ODD trajectory class compared to both other classes. Youth in the Moderate ODD trajectory class also showed higher comorbid symptoms in adolescence and young adulthood, compared to the Low ODD trajectory class.
Conclusion
Early identification of children and adolescents with high or moderate ODD symptoms and interventions that simultaneously address defiance and irritability are supported by the findings.
Keywords: oppositional defiant disorder, irritability, defiance, growth mixture model, transitions
Résumé
Contexte
Le trouble oppositionnel avec provocation (TOP) est un trouble du comportement perturbateur; toutefois, des données probantes croissantes soulignent que l’humeur irritable est un symptôme primaire du TOP.
Objectifs
La présente étude a investigué si les groupes (classes) hétérogènes de personnes qui peuvent être différentiées au mieux selon les sous-dimensions (irritabilité et défi) ou selon les symptômes généraux du TOP longitudinalement. Nous examinons également les associations entre la classe de trajectoire du TOP et l’utilisation de substances comorbide (lourde consommation d’alcool épisodique, utilisation de cannabis), la santé mentale (dépression et anxiété) et symptômes comportementaux (TDAH, agression et utilisation de substances) tant chez les adolescents que chez les jeunes adultes (contrôler les niveaux adolescents de chacun de ces problèmes).
Méthode
Les données provenaient d’un échantillon communautaire recruté au hasard de 662 jeunes Canadiens (âges T1 2–18) suivis tous les deux ans pendant 10 ans (T6 âges 22–29).
Résultats
Des modèles de mélange de croissance ont révélé des classes de trajectoire du TOP basées sur la gravité des symptômes. Une solution en trois classes a fourni le meilleur ajustement avec des classes de TOP faible (n = 119; 18 %), modérée (n = 473; 71,5 %), et élevée (n = 70; 10,6 %). Les différences de classes de trajectoire étaient également basées sur la gravité des symptômes (plutôt que sur le type) des sous-dimensions des symptômes (irritabilité, défi). L’utilisation de substances chez les adolescents et les jeunes adultes, les symptômes de santé mentale et les problèmes de comportement étaient significativement plus élevés pour la classe de la trajectoire élevée du TOP comparé aux deux autres classes. Les jeunes de la classe de trajectoire modérée du TOP présentaient aussi des symptômes comorbides plus élevés à l’adolescence et au jeune âge adulte, comparé à la classe de trajectoire faible du TOP.
Conclusion
L’identification précoce des enfants et des adolescents présentant des symptômes élevés ou modérés du TOP et les interventions qui prennent en charge simultanément le défi et l’irritabilité sont soutenues par les résultats.
Mots clés: trouble oppositionnel avec provocation, irritabilité, défi, modèle de mélange de croissance, transitions
Lifetime prevalence of Oppositional Defiant Disorder (ODD) is estimated to be 10% (males = 11%; females = 9%)(1).Of those with lifetime ODD, 92% meet criteria for at least one other lifetime DSM-5 disorder, including mood (45.8%), anxiety (62.3%), impulse-control (68.2%), and substance use (47.2%) disorders (1). ODD is a disruptive behavioral disorder; however, increasing evidence points to irritable mood as a primary symptom in children and youth experiencing ODD (2). Research also suggest that ODD symptoms have effects beyond adolescence and are also associated with disruptions of healthy development of relationships and of educational and occupational success by young adulthood (3–6).
Considerable cross-sectional research using factor analyses with community (7–13) −and clinic-referred samples of children and adolescents (14, 15) demonstrate that ODD comprises two important sub-dimensions that have been labeled: irritability and defiance dimensions. Irritable symptoms of ODD (i.e., touchy and easily annoyed, often resentful or angry, often loses temper) typically correlate concurrently and prospectively with mood disorders; whereas defiant symptoms (i.e., argues, blames others, defies authorities) are related more strongly, but not exclusively, to externalizing problems (see reviews 2, 16). A third dimension, hurtful, spiteful, or vindictive, has received somewhat less attention, in part, given assessments of these symptoms are often limited to one item. Research has also shown that by young adulthood irritability symptoms of ODD predict subsequent internalizing problems (anxiety and depression); whereas, defiance symptoms are associated with both internalizing and externalizing symptoms and with substance use by young adulthood (2). ODD, as a severe multi-dimensional concern, can disrupt the course of development for some children and youth, particularly if undiagnosed or untreated (17).
Defiant and irritable symptoms typically co-occur in individuals experiencing ODD and longitudinal research is lacking. It remains unclear whether children and youth can be classified into clinically distinct groups based on the sub-dimensions (e.g., high in defiance or low in irritability) over time (17). The correlation between these two constructs is typically high in samples of children and adolescents (e.g., r = 0.81 to 0.91) (7) and young adults (e.g., r = 0.70 to 0.89) (18). Longitudinal research is needed to distinguish between groups of adolescents and youth with varying levels of defiance and irritability symptoms in order to better understand the thresholds, patterns, and the developmental course of these concerns across the transition from adolescence to young adulthood (17–19).
Past cross sectional research suggest that ODD symptoms in affected individuals do not reflect distinct sub-dimensions. Rather most individuals with ODD report symptoms of both defiance and irritability. In their review of the evidence for the ODD dimensions, Evans et al.(17) conclude that current findings “do not support cleaving ODD into distinct subtypes or disorders (of either defiance or irritability); rather, ODD items include heterogeneous variability that is accounted for by a general ODD factor, as well as dimensions of irritability and defiant behavior” (p. 38). The present study uses longitudinal data from cohorts of Canadian youth who were ages 12 to 18 in 2003. We include six waves of data collected longitudinally over 10 years. We use growth mixture modeling (GMM) to classify the longitudinal trajectories of ODD symptoms considered together. We also classified youth based on each of the two sub-dimensions, to investigate whether distinct subtype trajectories (of either defiance or irritability) were evident across this transition.
Previous cross-sectional research with children and youth using latent class analysis (LCA) has identified three (or four) class solutions that include a low or no symptoms class, a defiant or irritable symptoms class, and a class experiencing both irritable and defiant symptom (14, 20–26). However, these classes confound symptom numbers (severity) and sub-dimension types (22). For example, in research with male youth from the Zurich Juvenile Detention Centre, Abei et al.(20) identified classes that combine severity and sub-dimensions; including, a no-ODD subtype, a severe ODD subtype, and two moderate ODD subtypes comprised of either defiant or irritable symptoms. Other cross-sectional research identified classes that differ in severity of symptoms (low, moderate, and high), rather than in distinctive sub-dimensions (24, 25). Longitudinal research is needed to assess patterns of ODD symptoms and sub-dimensions across time in order to disentangle the trajectories of ODD and the sub-dimensions as well as their relations with comorbid problems.
Previous longitudinal research that included five of the six waves of data used in this study (18) examined the trajectories of irritability and defiance variables in growth curve analyses. Defiance symptoms declined over time, whereas irritability remained relatively stable from ages 12 to 25. In addition, the associations of irritability with internalizing were stronger than associations of irritability with conduct problems and defiance was associated with both internalizing and conduct problems in mid-adolescence. However, defiance was more highly related to internalizing than to conduct problems by early adulthood (ages 18 to 25) (18). Our previous research also showed that higher overall ODD symptoms in adolescence predicted lower educational attainment (for males only), lower occupational prestige, higher debt (for females only), and more trouble paying for necessities and medical attention, and greater perceived workplace stress (18). Increases in ODD symptoms across the transition from adolescence to young adulthood also predicted worsening educational attainment and annual income for males, and higher debt and greater perceived personal conflict in the workplace for females, as well as greater job instability for both males and females (4)
The current study investigates whether there are heterogeneous groups (classes) of youth in a community sample who can be classified based on their ODD symptoms over time. We also examined trajectory classes for the sub-dimensions, but we did not hypothesize that there would be distinctive trajectory classes for irritability and defiance symptoms given the previous review by Evans et al.(17).. We use GMM to classify youth’s reports of ODD symptoms across a decade spanning adolescence to young adulthood (using six waves of data). In addition, we examine the associations of these class trajectories at their onset in adolescence and ten years in young adulthood (controlling for adolescent levels of these comorbid concerns with substance use (cannabis and alcohol), mental health symptoms (depression and anxiety), and behavior problems (Attention-Deficit/Hyperactivity Disorder [ADHD] and aggression).
Method
Participants and Procedure
The Victoria Healthy Youth Survey (V-HYS) (27) is a 10-year prospective longitudinal study of Canadian youth followed biennially for six assessments (T1; N = 662; 48% male Mage = 15.5, SD = 1.9) to 2013 (T6; N = 478; 45% male; Mage = 26.8, SD = 2.0). Males were slightly more likely to be lost to follow-up compared to females (i.e., males comprised 48% of the sample at T1 and 45% at T6; χ2 (1, 662) = 8.77, p = .003). Participants from higher socioeconomic status (SES) families (T1: M = 6.79, SD = 1.66; F (1, 636) = 19.39, p < .001) were more likely to be retained in the study than participants from lower SES families (M = 6.05, SD = 1.94).
In 2003, participants were recruited from a random sample of 9,500 telephone listings; 1,036 households with an eligible youth (ages 12 to 18 years) were identified. Of these, 662 agreed to participate in the study. The sample was representative of the Greater Victoria Areas population surveyed (27). Youth and the parent or guardian for youth under age 18 gave written consent for participation at each wave, and youth received a gift certificate at each interview. A trained interviewer administered the V-HYS individually in the youth’s home or another private place. To enhance privacy, the portion of the V-HYS questionnaire dealing with private topics (i.e., sexual experiences) was self-administered and placed in a sealed envelope not accessible to the interviewer. Retention rates were high at all assessments: 87% (T2), 81% (T3), 69% (T4), 70% (T5), and 72% (T6). The university’s research ethics board approved the research protocol.
Measures
Dimensions of ODD were measured using six items from the Brief Child and Family Phone Interview (BCFPI) (28) which assesses DSM-5 criteria for child and adolescent psychiatric disorders, including ODD symptoms (irritability and defiance symptoms). Items were rated on a three-point Likert scale (0 = never, 1 = sometimes, or 2 = often) in response to the question, “Do you notice that you are […item].” Items for the irritability subscale (three items) are “easily annoyed by others?” “angry and resentful?” and “cranky or irritable)?” Items for the defiance subscale (three items) are “argue a lot with others?” “defiant or talk back to people?” and “blame others for your own mistakes?” Scores ranged from 0 to 12 for ODD total symptoms (six items). Reliability estimates (alphas) for these ordinal scales were obtained using polychoric correlations. Alphas ranged from 0.79 to 0.84 for symptoms of ODD (total score), 0.74 to 0.81 for irritability, and 0.69 to 0.79 for defiance, with the exception of Wave 4 (alpha = 0.61) for defiance. The ODD subscale has been shown to be invariant across sex and time in previous studies using the V-HYS data.17 Within-time correlations between the irritability and defiance sub-dimensions were moderate at each time point (r’s = 0.47 to 0.58).
Demographic variables
Participants self-reported their gender and age at T1. As an estimate of socioeconomic status (SES), participants reported parent occupations were coded from 1 to 9 using the Hollingshead Occupational Status Scale (29). The highest level of occupational prestige for either parent was used as a measure of SES.
Substance use
In order to assess heavy episodic drinking (HED), participants were asked how often they had five or more drinks on one occasion in the past year: 0 = never, 1 = a few times a year, 2 = a few times a month, 3 = once a week, and 4 = more than once a week. The definition of a standard drink was provided (30). To assess the frequency of cannabis use, participants were asked: “how often marijuana (pot, hash) was used in the past 12 months.” Responses were given on a five-point scale: 0 = never, 1 = a few times a year, 2 = a few times a month, 3 = once a week, and 4 = more than once a week.
Depressive and anxiety symptoms
The BCFPI (28) assesses DSM-IV criteria for child and adolescent psychiatric disorders, including internalizing symptoms (i.e., depressive symptoms and anxiety). The BCFPI uses six items for each disorder and has demonstrated strong psychometric properties with the present sample.27 Items for each domain are rated on a three-point Likert scale (0 = never, 1 = sometimes, or 2 = often). Sum totals are used for this study (ranges = 0 to 12). Polychoric alphas were good for both depression and anxiety at T1 and T6, respectively: 0.87 and 0.92 for depression and 0.81 and 0.88 for anxiety.
ADHD and aggression
The BCFPI (28) also assessing symptoms of inattention and hyperactivity (i.e., ADHD). Total scores (six items; range 0 to 12) are used and polychoric alphas were 0.74 and 0.82 for T1 and T6, respectively. Aggression was assessed using nine items from Buss and Perry (31). Participants were asked to rate nine items (e.g., “once in a while I can’t control the urge to strike another person”) on a five-point Likert scale (1 = extremely uncharacteristic of me to 5 = extremely uncharacteristic of me, range 1 to 36). This was not assessed at T1 or T2, so T3 scores were used as baseline scores, when youth were in the 16- to 22-years-old cohort. Cronbach’s alphas were 0.85 at T3 and 0.85 at T6.
Statistical Analyses
All analyses were fit using Mplus version 8.7 (32) using full-information maximum likelihood to address missing data and with the robust maximum likelihood estimator that adjusts for potential non-normality in the data by estimating robust standard errors using a Huber White Sandwich estimator. GMM was used to assess whether a given population is composed of multiple subpopulations across time (33). The observed variables were the ODD total symptom scores (a composite of both irritability and defiance symptoms) for each of six assessments. We also fit growth mixtures for irritability and defiance symptoms separately; however, given their similarities in classifying youth, we use the combined ODD trajectories to classify youth. Cross tabulation of the three defiant and irritable classes showed only one youth was high in in defiance and low in irritability only four were high in irritability and low in defiance. See Table 1.
Table 1.
Cross-tabulation of modal class assignments for irritability and defiance growth mixture classes
| Defiance Low Class N |
Defiance Medium Class N |
Defiance High Class N |
Row Totals N |
|
|---|---|---|---|---|
| Irritability Low Class | 128 (61%) | 80 | 1 | 209 |
| Irritability Medium Class | 75 | 222 (63%) | 53 | 350 |
| Irritability High Class | 4 | 44 | 55 (53%) | 103 |
| Column Totals | 207 | 346 | 109 | 662 |
Note: Percentages indicate class overlap for low/low, medium/medium and high/high irritability and defiance classes using row totals for irritability.
The ODD scores for the six time points were used to identify ODD growth trajectories (classes). First, we determined the number of distinct classes to retain through a class enumeration process. We used standard fit indices and the theoretical meaningfulness of the classes to select the number of latent classes to retain. Fit indices included the Bayesian Information Criterion (BIC) and the Aikake Information Criteria (AIC), where lower values indicate a better-fitting model, the significance on the Lo–Mendell–Rubin Adjusted Likelihood Ratio Test (LMR-LRT), Bootstrap Likelihood Ratio Test (BLRT) as well as the usefulness of the latent classes given the number of individuals in each class.34 Entropy, the quality of class separation is reported but was not used to determine the number of classes to retain because it is not a fit index. We examined the functional form of the data and determined that a quadratic function fit the data best. For example, for ODD symptoms using a likelihood ratio test the linear functional form fit the data significantly worse than the quadratic ((Δ −2LL = 53.85, ΔDF = 6, p < .001).
We used the standard three-step approach (34) to fit the mixture models and examine adolescent (T1; ages 12–18) predictors (i.e., substance use, mental health, and behavioral problems) of the classes using multinomial logistic regression. Then, linear regression was used to examine differences in the same young adult (T6; ages 22–29) outcomes (i.e., substance use, mental health, and behavior problems). All models controlled for demographic variables including age, sex, and SES). Young adult models (T6) also accounted for T1 levels of the outcome variables. We examined these predictors and outcomes in three separate models grouped by concern (substance use, mental health, and behavioral problems).
Results
Descriptive Statistics
Correlations between irritability and defiance were moderate at each time point (T1 r = 0.47; T2 r = 0.51; T3 r = 0.57; T4 r = 0.49; T5 r = 0.54; T6 r = 0.58), Correlations between ODD symptoms and the substance use, mental health, and behavioral problem variables were also in the expected directions (see Supplemental Table S1, available from the authors).
Growth Mixture Models of Classes of ODD and the Sub dimensions
Using separate GMMs for ODD, irritability, and defiance, we tested an increasing number of class solutions iteratively to determine the best fitting model see (Table 2 and Figure 1). Model testing ceased when the additional class provided a poorer fit and was not different qualitatively from the more parsimonious model. A three-class solution provided the best fit for the ODD variable reflecting differences in the severity of ODD symptoms across each assessment Low (n = 119, 18.0%), Moderate (n = 473; 71.5%), and High (n = 70, 10.6%) classes (Table 1 and Figure 1). Similarly, three-class solutions provided the best fit reflecting severity of symptoms for irritability symptoms: Low (n = 209, 31.6%), Moderate (n = 350; 52.9%), and High (n = 103, 15.6%) classes and for defiance symptoms: Low (n = 207; 31.3%), Moderate (n = 346; 52.3%), and High (n = 109, 16.5%). Given the moderately high positive correlations between irritability and defiance at T1 and T6, the three-class solutions for each, and the similar trajectories for irritability and defiance, all subsequent analyses focus on the three overall ODD classes only.
Table 2.
Model fit indices for latent class models examined for ODD, Irritability, and Defiance Symptoms.
| AIC | BIC | LMR | BLRT | entropy | Class 1 | Class 2 | Class 3 | Class 4 | |
|---|---|---|---|---|---|---|---|---|---|
| ODD total symptoms model | |||||||||
| one-class | 13025.91 | 13093.34 | |||||||
| two-class | 12986.81 | 13085.7 | p < 0.01 | p < 0.001 | 0.671 | 107 (16%) | 555 (84%) | ||
| three-class | 12963.93 | 13094.29 | p = 0.04 | p < 0.001 | 0.574 | 119 (18%) | 473 (71%) | 70 (11%) | |
| four-class | model does not converge | ||||||||
|
| |||||||||
| Irritability symptoms model | |||||||||
| one-class | 10926.52 | 10966.97 | |||||||
| two-class | 10228.99 | 10287.42 | p < 0.001 | p < 0.001 | 0.712 | 409 (62%) | 253 (38%) | ||
| three-class | 10014.25 | 10090.67 | p < 0.001 | p < 0.001 | 0.716 | 209 (32%) | 350 (53%) | 103 (15%) | |
| four-class | 9973.992 | 10068.39 | p = 0.11 | p < 0.001 | 0.669 | 40 (6%) | 151 (23%) | 154 (23%) | 317 (48%) |
|
| |||||||||
| Defiance symptoms model | |||||||||
| one-class | 10380.381 | 10420.838 | |||||||
| two-class | 9773.94 | 9832.38 | p < 0.001 | p < 0.001 | 0.708 | 227 (34%) | 435 (66%) | ||
| three-class | 9645.671 | 9722.09 | p = 0.008 | p < 0.001 | 0.642 | 207 (31%) | 346 (52%) | 109 (17%) | |
| four-class | 9607.22 | 9701 | p = 0.35 | p < 0.001 | 0.67 | 346 (52%) | 32 (5%) | 182 (28%) | 102 (15%) |
Note:
p < 0.05;
p < 0.01;
p < 0.001;
BIC= Bayesian information criterion AIC= Aikake Information Criteria, LMR Lo–Mendell–Rubin Adjusted Likelihood Ratio Test, BLRT = Bootstrap Likelihood Ratio Test
Figure 1.
a) Growth mixture model classes showing Low, Moderate, and High Trajectories across the transition for adolescence to young adulthood (Time 1 to 6) for a) ODD symptoms; b) Irritability symptoms; and, c) Defiance symptoms.
Mean differences and standard errors for irritability and defiance at T1 and T6 for each of the classes are presented in Table 3. Significantly, from both a clinically meaningful and statistical perspective at T1, symptoms of defiance were approximately two times greater in the High compared to the Moderate class, and symptoms of irritability were approximately two times greater in the High versus Moderate class. Similarly, at T6 symptoms of defiance were 1.5 times greater in the High compared to the Moderate class, and symptoms of irritability were 1.6 times greater in the High versus Moderate class.
Table 3.
Means and standard errors of irritability and defiance symptoms by ODD latent classes
| 1. Low ODD class (n = 123) | 2. Moderate ODD class (n = 464) | 3. High ODD class (n = 74) | Class comparisons (Wald test) | ||||
|---|---|---|---|---|---|---|---|
|
|
|
|
|||||
| Mean | SE | Mean | SE | Mean | SE | ||
| Irritability T1 | 0.95 | 0.09 | 2.25 | 0.07 | 4.20 | 0.15 | 1 < 2***; 2 < 3***; 1 < 3*** |
| Defiance T1 | 1.03 | 0.11 | 2.01 | 0.07 | 3.90 | 0.15 | 1 < 2***; 2 < 3***; 1 < 3*** |
| Irritability T6 | 0.72 | 0.07 | 2.24 | 0.07 | 3.63 | 0.23 | 1 < 2***; 2 < 3***; 1 < 3*** |
| Defiance T6 | 0.66 | 0.06 | 1.25 | 0.06 | 2.95 | 0.27 | 1 < 2***; 2 < 3***; 1 < 3*** |
p < 0.001
SE = Stanadard Error
Mental Health, Substance Use, and Behavioral Problems as Predictors of Class Severity
As shown in Table 4, adolescent levels of HED, cannabis use, depression and anxiety, and behavioral problems were all significantly higher for youth in the High ODD trajectory class compared to those in both the Low and Moderate ODD classes. In addition, youth in the moderate ODD class had significantly higher symptoms of depression, ADHD and aggression compared to youth in the Low ODD class (but not for HED, cannabis and anxiety).
Table 4.
Multinomial regression analysis showing ODD symptoms trajectories class differences in adolescents (T1; ages 12–18) predicted by age, sex, SES, substance use, mental health, and behavioral problem indicators. The Low ODD class (n=199) is the reference group
| 2. Moderate ODD class (n = 473) | 3. High ODD class (n = 70) | Pairwise comparisona | |||
|---|---|---|---|---|---|
|
|
|
|
|||
| Est (SE) | OR | Est (SE) | OR | ||
| Substance use | |||||
| Sex (male = 0) | 0.15 (0.38) | 1.16 | −0.66 (0.37) | 0.52 | 2 > 3*** |
| Age | −0.12 (0.17) | 0.89 | −0.25 (0.15) | 0.78 | |
| SES | −0.05 (0.12) | 0.95 | −0.23 (0.11) | 0.79 | 1 > 3*; 2 > 3** |
| HED | 0.03 (0.28) | 1.03 | 0.01 (0.27) | 1.01 | |
| Cannabis | 0.28 (0.29) | 1.33 | 0.81 (0.25) | 2.24 | 1 < 3***; 2 < 3*** |
| Mental health | |||||
| Sex (male = 0) | −0.04 (0.29) | 0.96 | −0.41 (0.37) | 0.66 | |
| Age | −0.17 (0.08) | 0.85 | −0.11 (0.10) | 0.90 | 1 > 2*; |
| SES | −0.05 (0.08) | 0.96 | −0.16 (0.10) | 0.85 | |
| Depression | 0.22 (.09) | 1.24 | 0.47 (0.10) | 1.59 | 1 < 2**; 1 < 3***; 2 < 3*** |
| Anxiety | 0.08 (0.06) | 1.09 | 0.38 (0.08) | 1.46 | 1 < 3***; 2 < 3*** |
| Conduct problems | |||||
| Sex (male = 0) | 1.28 (0.46) | 3.61 | −0.12 (0.58) | 0.89 | 1 < 2**; 2 > 3*** |
| Age | 0.16 (0.10) | 0.86 | −0.18 (0.13) | 0.84 | |
| SES | −0.04 (0.10) | 0.96 | −0.19 (0.13) | 0.82 | |
| ADHD | 0.47 (0.12) | 1.60 | 0.71 (0.14) | 2.04 | 1 < 2***; 1 < 3***; 2 < 3** |
| Aggression (T3) | 0.24 (0.08) | 1.27 | 0.36 (0.09) | 1.43 | 1 < 2***; 1 < 3***; 2 < 3*** |
SES = socioeconomic status; HED = Heavy episodic drinking; ADHD = Attention-deficit/hyperactivity disorder.
p < 0.05;
p < 0.01;
p < 0.001
1=Low, 2=Moderate, 3=High ODD class.
Using assessments collected in young adulthood (T6) and controlling for adolescent levels of each variable, significant differences between the High ODD trajectory class and the other two classes were evident for each of the mental health, substance use, and behavioral problem variables assessed (see Table 5). The Moderate ODD class also had significantly greater increases in symptoms of depression, anxiety, ADHD and aggression than youth in the Low ODD class, but not for the substance use variables.
Table 5.
Linear regression was used to examine differences in young adult substance use, mental health, and conduct problem outcome means and comparisons by ODD growth mixture classes. All models control for the outcome measures at T1, sex, age, and socioeconomic status.
| 1. Low ODD class | 2. Moderate ODD class | 3. High ODD class | Group comparisons Wald test | ||||
|---|---|---|---|---|---|---|---|
|
|
|
|
|||||
| M | SE | M | SE | M | SE | ||
|
| |||||||
| (n = 99) | (n = 428) | (n = 133) | |||||
| HED T6 | 1.04 | 0.32 | 1.40 | 0.13 | 1.88 | 0.13 | 1 < 3**; 2 < 3** |
| Cannabis T6 | 1.40 | 0.21 | 1.54 | 0.08 | 4.63 | 0.07 | 1 < 3***; 2 < 3*** |
| (n = 121) | (n = 432) | (n = 108) | |||||
|
| |||||||
| Depression T6 | 0.70 | 0.11 | 2.65 | 0.19 | 6.59 | 0.34 | 1 < 2***; 1 < 3***; 2 < 3*** |
| Anxiety T6 | 3.10 | 0.22 | 5.34 | 0.22 | 8.92 | 0.30 | 1 < 2***; 1 < 3***; 2 < 3*** |
| (n = 116) | (n = 332) | (n = 88) | |||||
|
| |||||||
| ADHD T6 | 2.04 | 0.16 | 4.12 | 0.19 | 5.26 | 0.36 | 1 < 2***; 1 < 3***; 2 < 3** |
| Aggression T6 | 3.41 | 0.43 | 5.54 | 0.42 | 17.29 | 1.10 | 1 < 2***; 1 < 3***; 2 < 3*** |
Note: HED = Heavy episodic drinking; ADHD = Attention-deficit/hyperactivity disorder.
p < 0.05;
p < 0.01;
p < 0.001.
Discussion
This study examines ODD symptom across the transition from adolescence to young adulthood in a large longitudinal Canadian sample. Given ODD symptoms are often thought to dissipate after adolescence, our findings of continuity in symptom classes based on seriousness of symptoms across the ten years show the need to advocate for and undertake both early and ongoing treatment of ODD. The study also raises attention to the need for treating symptoms of irritability, as well as defiance, in youth with ODD to prevent further negative mental health and behavioral consequences.
The goal of this study was to examine of the heterogeneity in the trajectories of ODD symptoms in a community representative sample of youth across ten years, across the transition from adolescence (ages 12 to 18) to young adulthood (ages 22 to 29). We found trajectory differences that reflected the severity of ODD symptoms (for defiance and irritability and for the combined symptoms) across this salient life transition. Subgroup classes for youth with combined ODD symptoms of defiance and irritability showed that Low (18.0%), Moderate (71.5%), and High (10.6%) levels of overall ODD symptoms best characterized the ODD class trajectories over time. In assessments of comorbid symptoms, the High ODD class had higher levels for all predictors of substance use (cannabis use), mental health (depression and anxiety), and behavioral problems (ADHD and aggression) in adolescence and young adulthood (also including HED) compared to the Moderate and Low ODD classes. The Moderate ODD class reported more depressive symptoms and behavioral problems (ADHD and aggression) than the Low ODD class in adolescence and more symptoms of both mental health (depression and anxiety) and behavioral problems (ADHD and aggression) in young adulthood (but not more substance use concerns).
Our findings also showed trajectory class differences based on severity of symptoms for irritability and defiance. Consistent with a review of this research by Evans et al. (2), the findings did not distinguish separate groups of youth by sub-dimensions. Specifically, cross tabulations of the defiance and irritability subclasses classes did not reveal youth in high irritability/low defiance or low irritability/high defiance classes. Cross-sectional studies, with predominately children and adolescents have identified three- or four-class solutions based on the severity of symptoms as well as irritable and defiant subtypes (21–22,35–36).However, the frequency of youth experiencing defiance without irritability was low and the current study is among few examining the longitudinal trajectories in symptoms which may be a more accurate representation of symptom experiences over time. It is possible that within the large percentage of youth classified in the Moderate class (71.5%), there may be additional classes that are not captured here due to low numbers of youth. In adolescence, the Moderate ODD class did not differ from the Low class on HED, cannabis use, and anxiety; however, on average, they did have more symptoms of depression, ADHD, and aggression. Consistent with Copeland et al. (19), some level of ODD symptoms and substance use may be normative and intermittent at ages 9 to 18. However, here, as young adults, youth in the Moderate ODD class showed higher levels of depression, anxiety, ADHD, and aggression compared to the Low ODD class suggesting further refinements are needed to distinguish youth at-risk for young adult problems within the Moderate ODD class. Continuation of ODD symptoms into the transition to young adulthood and the cumulative impact of moderate levels of both irritability and defiance may also cascade into comorbid mental health and behavioral problems. Early identification and treatment of even moderate levels of ODD symptoms may prevent these negative outcomes.
In some previous research, irritable (angry) mood appears to occur without the behavioral manifestations of ODD (6); however, defiance rarely occurred without irritable mood (19, 37). Increasing research suggests that irritability may be a trans-diagnostic symptom that characterizes many mental health and behavioral disorders (19, 37, 38). In addition, there is some evidence that irritability may be distinguishable from other concerns that have irritability as one component (e.g., ODD, depression, bipolar disorder) (2, 37, 38). Irritability may also be a multidimensional construct reflecting mood or emotional regulation problems (angry, grouchy, grumpy) rather than behavioral problems (behavioral expressions of intense anger, temper outbursts, screaming) as well as chronic and episodic phases. Following DSM-5, symptoms of ODD-irritability in the current study (i.e., easily annoyed, cranky, angry, and resentful) tap mood disruptions that may not be similar to irritability that characterizes other disorders (tense, dysthymic, moody, cranky). In contrast, ODD-defiance items (argues a lot, defiant, talks back and blame others for mistakes) are observable behavioral symptoms, but are not the same as severe manifestations of intense anger by breaking things or temper outbursts. Further research into the quality and functional impairments related to irritability experienced by individuals with ODD may reveal additional differences from other diagnoses.
Similar to findings by Copeland et al. (19) the parallel trajectories of irritability and defiant symptoms of ODD found in the current research suggest these mood and behavioral dimensions may be distinguishable variables but they co-occur in youth with ODD. Variable centered research that demonstrates the associations of irritability and defiance with specific comorbid concerns may suggest that one subtype may occur without the other. The current study adds to the growing number of studies (2) that do support subgroups of ODD youth who manifest as defiance without irritability or of irritability without defiance.
A majority of youth in the current study (71.5%) were classified in the Moderate ODD class across adolescence and young adulthood; however, the persistence of these negative feelings may be particularly difficult for adolescents to regulate and may be dismissed as normative changes accompanying adolescence and go untreated. By young adulthood, even the moderate levels of persistent ODD symptoms may disrupt functioning as they are not tolerated in work, community, and school settings. Thresholds of symptoms related to pathology may be difficult to establish with the brief screening tool used here; however, consistent with Copeland et al. (19), the current research makes clear that persistently moderate levels detected in even a short screening measure may be implicated in the development of young adult mental health problems and behavioral concerns.
Implications
Our findings show that youth’s experiences of moderate and high levels of symptoms of ODD are typically characterized by both irritability and defiance. ODD levels persisted from adolescence to young adulthood. Moreover, experiences of ODD were associated, in adolescences and young adulthood, with risks for comorbid depression, anxiety, behavioral problems, and cannabis use.
A combination of genetic, temperament, and parenting or other contextual factors lie at the basis of high levels of ODD symptoms and may sustain these over time. Research on the etiology and development of ODD suggests that adolescents with ODD showed difficulties in self-regulation and anger as children and that their parents may have had difficulties in attachments and child management (39). By adolescence and young adulthood, high levels of ODD symptoms are increasingly non-normative and annoying to parents, teachers, employers, and peers. This may lead to conflicts in relationships that, in turn, exacerbate perceptions that others are to blame for their problems in youth with ODD. The persistence of symptoms into young adulthood suggest early identification and treatments are needed.
Evidence-based treatments for ODD commonly focus on behavioral and anger management (often with a parenting behavior management component) or alternatively, on problem solving or cognitive interventions that target social information processing errors and disruptive interactions with peers (41, 42). Given the severity and persistence of both irritability and defiance symptoms found here, support for parents and treatments of children’s irritability, emotional regulation, and attachment disruptions may also be needed (39, 43). Pharmacotherapeutic treatment options of both aggression and chronic irritability in youth with ODD has begun to be assessed but conclusions are preliminary (44). Our findings do not support treatment options that based on the unsubstantiated view there are distinguishable subgroups of youth with ODD who experience irritability with or without defiance manifestations (40).
Limitations
Our data are generalizable to Canadian youth comprising predominately Caucasian samples. All data were reported by the youth themselves, which may contribute to shared reporter variance. Outcome data were collected in face-to-face structured interviewers and coded by trained research assistants (i.e., educational attainment, occupational prestige, income, work characteristics). Consistency across multiple assessments spaced two years apart (18) increase our confidence in the findings. Slightly more males and youth from lower income families were lost to follow-up. While the effects of family SES are included in our models to adjust for potential bias due to missing data for low-income participants, our models may underestimate these youth. Our analyses also focused on the symptom levels in this community-based sample rather than clinical diagnoses. However, our findings concur with previous research on depressive symptoms (45), suggesting that subclinical symptoms of mental health problems in adolescence can have long term negative outcomes.
Conclusion
While there is a tendency to view ODD symptoms of irritability and defiance as evidence of bad temper or moodiness, consistently high levels of these concerns can compromise the achievement of developmental task appropriate to later adolescence and young adulthood. Early identification and interventions is needed to address both mood regulation and anger management. More research into the etiology of comorbid irritable mood and behavioral manifestations of this serious disorder is needed. Costs to families trying to manage these concerns can be high. Defiance, anger, and touchiness may be responded to by repeated school suspensions that further impede development. Risks for substance use and deviant behavior and depression in young adulthood also suggest early intervention is important.
Acknowledgments
The Victoria Healthy Youth Survey study and this research were supported by grants from the Canadian Institutes of Health Research (#88476; #79917; #192583; #130500; SHI-155410). We appreciate the contributions of all youth who participated in this research for over a decade. The research study received approval from the University of Victoria’s Human Research Ethics Board (PROTOCOL NUMBER: 09-292)
Footnotes
Conflict of Interests
The authors have no conflicts of interest related to this research.
References
- 1.Nock MK, Kazdin AE, Hiripi E, Kessler RC. Lifetime prevalence, correlates, and persistence of oppositional defiant disorder: Results from the National Comorbidity Survey Replication. Journal of Child Psychology and Psychiatry. 2007 Jul;48(7):703–13. doi: 10.1111/j.1469-7610.2007.01733.x. [DOI] [PubMed] [Google Scholar]
- 2.Evans SC, Burke JD, Roberts MC, Fite PJ, Lochman JE, Francisco R, Reed GM. Irritability in child and adolescent psychopathology: An integrative review for ICD-11. Clinical Psychology Review. 2017 Apr 1;53:29–45. doi: 10.1016/j.cpr.2017.01.004. [DOI] [PubMed] [Google Scholar]
- 3.Burke JD, Rowe R, Boylan K. Functional outcomes of child and adolescent oppositional defiant disorder symptoms in young adult men. Journal of Child Psychology and Psychiatry. 2014 Mar;55(3):264–72. doi: 10.1111/jcpp.12150. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Leadbeater BJ, Ames ME. The longitudinal effects of oppositional defiant disorder symptoms on academic and occupational functioning in the transition to young adulthood. J Abnorm Child Psychol. 2017 May;45(4):749–763. doi: 10.1007/s10802-016-0190-4. [DOI] [PubMed] [Google Scholar]
- 5.Loeber R, Burke JD. Developmental pathways in juvenile externalizing and internalizing problems. Journal of Research on Adolescence. 2011 Mar;21(1):34–46. doi: 10.1111/j.1532-7795.2010.00713.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Stringaris A, Cohen P, Pine DS, Leibenluft E. Adult outcomes of youth irritability: A 20-year prospective community-based study. Am J Psychiatry. 2009 Sep;166(9):1048–54. doi: 10.1176/appi.ajp.2009.08121849. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Burke JD, Boylan K, Rowe R, Duku E, Stepp SD, Hipwell AE, Waldman ID. Identifying the irritability dimension of ODD: Application of a modified bifactor model across five large community samples of children. J Abnorm Psychol. 2014 Nov;123(4):841–51. doi: 10.1037/a0037898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ezpeleta L, Granero R, De La Osa N, Penelo E, Domènech JM. Dimensions of oppositional defiant disorder in 3-year-old preschoolers. Journal of Child Psychology and Psychiatry. 2012 Nov;53(11):1128–38. doi: 10.1111/j.1469-7610.2012.02545.x. [DOI] [PubMed] [Google Scholar]
- 9.Krieger FV, Leibenluft E, Stringaris A, Polanczyk GV. Irritability in children and adolescents: Past concepts, current debates, and future opportunities. Braz J Psychiatry. 2013;35(Suppl 1(0 1)):S32–9. doi: 10.1590/1516-4446-2013-S107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Rowe R, Costello EJ, Angold A, Copeland WE, Maughan B. Developmental pathways in oppositional defiant disorder and conduct disorder. Journal of Abnormal Psychology. 2010 Nov;119(4):726. doi: 10.1037/a0020798. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Stringaris A, Goodman R. Longitudinal outcome of youth oppositionality: Irritable, headstrong, and hurtful behaviors have distinctive predictions. Journal of the American Academy of Child & Adolescent Psychiatry. 2009 Apr 1;48(4):404–12. doi: 10.1097/CHI.0b013e3181984f30. [DOI] [PubMed] [Google Scholar]
- 12.Stringaris A, Maughan B, Copeland WS, Costello EJ, Angold A. Irritable mood as a symptom of depression in youth: Prevalence, developmental, and clinical correlates in the Great Smoky Mountains Study. J Am Acad Child Adolesc Psychiatry. 2013 Aug;52(8):831–40. doi: 10.1016/j.jaac.2013.05.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Whelan YM, Stringaris A, Maughan B, Barker ED. Developmental continuity of oppositional defiant disorder subdimensions at ages 8, 10, and 13 years and their distinct psychiatric outcomes at age 16 years. J Am Acad Child Adolesc Psychiatry. 2013;52(9):961–969. doi: 10.1016/j.jaac.2013.06.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Burke JD. An affective dimension within oppositional defiant disorder symptoms among boys: Personality and psychopathology outcomes into early adulthood. J Child Psychol Psychiatry. 2012;53(11):1176–1183. doi: 10.1111/j.1469-7610.2012.02598.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Drabick DA, Gadow KD. Deconstructing oppositional defiant disorder: Clinic-based evidence for an anger/irritability phenotype. J Am Acad Child Adolesc Psychiatry. 2012 Apr;51(4):384–93. doi: 10.1016/j.jaac.2012.01.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Waldman ID, Rowe R, Boylan K, Burke JD. External validation of a bifactor model of oppositional defiant disorder. Molecular psychiatry. 2021 Feb;26(2):682–93. doi: 10.1038/s41380-018-0294-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Evans SC, Roberts MC, Keeley JW, Rebello TJ, Peña F, Lochman JE, et al. Diagnostic classification of irritability and oppositionality in youth: A global field study comparing ICD-11 with ICD-10 and DSM-5. Journal of Child Psychology and Psychiatry. 2021 Mar;62(3):303–12. doi: 10.1111/jcpp.13244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Leadbeater BJ, Homel J. Irritable and defiant sub-dimensions of ODD: Their stability and prediction of internalizing symptoms and conduct problems from adolescence to young adulthood. J Abnorm Child Psych. 2015 Apr;43(3):407–21. doi: 10.1007/s10802-014-9908-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Copeland WE, Wolke D, Shanahan L, Costello EJ. Adult functional outcomes of common childhood psychiatric problems: A prospective, longitudinal study. JAMA Psychiatry. 2015 Sep;72(9):892–9. doi: 10.1001/jamapsychiatry.2015.0730. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Aebi M, Barra S, Bessler C, Steinhausen HC, Walitza S, Plattner B. Oppositional defiant disorder dimensions and subtypes among detained male adolescent offenders. Journal of Child Psychology and psychiatry. 2016 Jun;57(6):729–36. doi: 10.1111/jcpp.12473. [DOI] [PubMed] [Google Scholar]
- 21.Althoff RR, Kuny-Slock AV, Verhulst FC, Hudziak JJ, van der Ende J. Classes of oppositional-defiant behavior: Concurrent and predictive validity. Journal of Child Psychology and Psychiatry. 2014 Oct;55(10):1162–71. doi: 10.1111/jcpp.12233. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Herzhoff K, Tackett JL. Subfactors of oppositional defiant disorder: Converging evidence from structural and latent class analyses. Journal of Child Psychology and Psychiatry. 2016 Jan;57(1):18–29. doi: 10.1111/jcpp.12423. [DOI] [PubMed] [Google Scholar]
- 23.Kuny AV, Althoff RR, Copeland W, Bartels M, Van Beijsterveldt CE, Baer J, Hudziak JJ. Separating the domains of oppositional behavior: comparing latent models of the conners’ oppositional subscale. J Am Acad Child Adolesc Psychiatry. 2013 Feb;52(2):172–183.e8. doi: 10.1016/j.jaac.2012.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Boylan K, Rowe R, Duku E, Waldman I, Stepp S, Hipwell A, Burke J. Longitudinal profiles of girls’ irritable, defiant and antagonistic oppositional symptoms: Evidence for group based differences in symptom severity. J Abnorm Child Psychol. 2017 Aug;45(6):1133–1145. doi: 10.1007/s10802-016-0231-z. [DOI] [PubMed] [Google Scholar]
- 25.Roetman PJ, Siebelink BM, Vermeiren RRJM, Colins OF. Classes of oppositional defiant disorder behavior in clinic-referred children and adolescents: Concurrent features and outcomes. Can J Psychiatry. 2021 Jul;66(7):657–666. doi: 10.1177/0706743720974840. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Aebi M, Plattner B, Metzke CW, Bessler C, Steinhausen HC. Parent- and self-reported dimensions of oppositionality in youth: Construct validity, concurrent validity, and the prediction of criminal outcomes in adulthood. J Child Psychol Psychiatry. 2013 Sep;54(9):941–9. doi: 10.1111/jcpp.12039. [DOI] [PubMed] [Google Scholar]
- 27.Leadbeater B, Thompson K, Gruppuso V. Co-occurring trajectories of symptoms of anxiety, depression, and oppositional defiance from adolescence to young adulthood. Journal of Clinical Child & Adolescent Psychology. 2012 Nov 1;41(6):719–30. doi: 10.1080/15374416.2012.694608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Cunningham CE, Boyle MH, Hong S, Pettingill P, Bohaychuk D. The Brief Child and Family Phone Interview (BCFPI): 1. Rationale, development, and description of a computerized children’s mental health intake and outcome assessment tool. J Child Psychol Psychiatry. 2009 Apr;50(4):416–23. doi: 10.1111/j.1469-7610.2008.01970.x. [DOI] [PubMed] [Google Scholar]
- 29.Bornstein MH, Hahn CS, Suwalsky JTD, Haynes OM. Socioeconomic status, parenting, and child development. Mahwah: Lawrence Erlbaum Associates Publishers; 2003. Socioeconomic status, parenting, and child development: The Hollingshead four-factor index of social status and the socio-economic index of occupations; pp. 29–82. [Google Scholar]
- 30.Evans-Polce RJ, Vasilenko SA, Lanza ST. Changes in gender and racial/ethnic disparities in rates of cigarette use, regular heavy episodic drinking, and marijuana use: Ages 14 to 32. Addict Behav. 2015 Feb;41:218–22. doi: 10.1016/j.addbeh.2014.10.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Buss AH, Perry M. The aggression questionnaire. Journal of Personality and Social Psychology. 1992;63(3):452–459. doi: 10.1037/0022-3514.63.3.452. [DOI] [PubMed] [Google Scholar]
- 32.Muthén LK, Muthén BO. Mplus User’s Guide. Eighth Edition. Los Angeles, CA: Muthén & Muthén; 1998–2017. [Google Scholar]
- 33.Jung T, Wickrama KA. An introduction to latent class growth analysis and growth mixture modeling. Social and Personality Psychology Compass. 2008 Jan;2(1):302–17. [Google Scholar]
- 34.Asparouhov T, Muthén B. Auxiliary variables in mixture modeling: Three-step approaches using M plus. Structural equation modeling: A multidisciplinary Journal. 2014 Jul 3;21(3):329–41. [Google Scholar]
- 35.Gomez R, Stavropoulos V, Van Doorn G. Factor structure of teacher ratings of the ODD symptoms in children. Psychiatr Q. 2018 Mar;89(1):219–234. doi: 10.1007/s11126-017-9527-6. [DOI] [PubMed] [Google Scholar]
- 36.Wesselhoeft R, Stringaris A, Sibbersen C, Kristensen RV, Bojesen AB, Talati A. Dimensions and subtypes of oppositionality and their relation to comorbidity and psychosocial characteristics. European Child & Adolescent Psychiatry. 2019 Mar;28(3):351–65. doi: 10.1007/s00787-018-1199-8. [DOI] [PubMed] [Google Scholar]
- 37.Hawes MT, Carlson GA, Finsaas MC, Olino TM, Seely JR, Klein DN. Dimensions of irritability in adolescents: longitudinal associations with psychopathology in adulthood. Psychological Medicine. 2019:1–9. doi: 10.1017/S0033291719002903. [DOI] [PubMed] [Google Scholar]
- 38.Stringaris A, Goodman R, Ferdinando S, Razdan V, Muhrer E, Leibenluft E, Brotman MA. The Affective Reactivity Index: A concise irritability scale for clinical and research settings. Journal of Child Psychology and Psychiatry. 2012 Nov;53(11):1109–17. doi: 10.1111/j.1469-7610.2012.02561.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Craig SG, Hernandez CS, Moretti MM, Pepler DJ. The mediational effect of affect dysregulation on the association between attachment to parents and oppositional defiant disorder symptoms in adolescents. Child Psychiatry and Human Development. 2021;52(5):818–828. doi: 10.1007/s10578-020-01059-5. [DOI] [PubMed] [Google Scholar]
- 40.Zachary C, Jones DJ. The role of irritability in the treatment of behavior disorders: A review of theory, research, and a proposed framework. Clin Child Fam Psychol Rev. 2019 Jun;22(2):197–207. doi: 10.1007/s10567-018-00272-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Burke JD, Loeber R. Evidence-based interventions for oppositional defiant disorder in children and adolescents. Handbook of evidence-based interventions for children and adolescents. 2017:181–91. [Google Scholar]
- 42.Martel MM. The clinician’s guide to oppositional defiant disorder: Symptoms, assessment, and treatment. Elsevier Academic Press; 2019. [Google Scholar]
- 43.Derella OJ, Johnston OG, Loeber R, Burke JD. CBT-enhanced emotion regulation as a mechanism of improvement for childhood irritability. Journal of Clinical Child & Adolescent Psychology. 2019 Mar 29;48(sup1):S146–54. doi: 10.1080/15374416.2016.1270832. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Tourian L, LeBoeuf A, Breton JJ, Cohen D, Gignac M, Labelle R, Guile JM, Renaud J. Treatment options for the cardinal symptoms of Disruptive Mood Dysregulation Disorder. J Can Acad Child Adolesc Psychiatry. 2015 Winter;24(1):41–54. [PMC free article] [PubMed] [Google Scholar]
- 45.Peters AT, Shankman SA, Deckersbach T, West AE. Predictors of first-episode unipolar major depression in individuals with and without sub-threshold depressive symptoms: A prospective, population-based study. Psychiatry Res. 2015 Dec 15;230(2):150–6. doi: 10.1016/j.psychres.2015.08.030. [DOI] [PMC free article] [PubMed] [Google Scholar]

