Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 May 1.
Published in final edited form as: J Abnorm Child Psychol. 2017 May;45(4):743–748. doi: 10.1007/s10802-016-0196-y

ODD Symptom Network During Preschool

Tess E Smith 1, Christine A Lee 2, Michelle M Martel 3, Marni E Axelrad 4
PMCID: PMC5311058  NIHMSID: NIHMS810641  PMID: 27523818

Abstract

Several different conceptualizations of Oppositional Defiant Disorder (ODD) symptoms have been proposed, including one undivided set of symptoms (DSM-IV-TR; APA, 2000); two domains of symptoms subdivided into affective and behavioral; and three domains of symptoms subdivided as angry/irritable, argumentative/defiant, and spiteful. The current study utilizes a novel approach to examining the division of ODD symptoms through use of network analysis. Participants were 109 preschoolers (64 male) between the ages of three and six (M=4.34 years, SD=1.08) and their parents and teachers/caregivers, who provided ratings of ODD symptoms. Results are consistent with 1-, 2-, and 3- cluster solutions of ODD, but perhaps provide most support for the 3-cluster solution. In addition, results support the idea that negative affect, particularly anger, forms the core of the ODD symptom network during preschool. These results suggest the importance of targeting anger in preschool interventions for ODD.

Keywords: ODD, network, clusters, anger

Oppositional Defiant Disorder (ODD) is a Disruptive, Impulse-Control, and Conduct Disorder that is characterized by a pattern of angry, hostile, and/or defiant behaviors and interactions (American Psychiatric Association [APA], 2013). ODD can be diagnosed as young as the preschool period (age 3; Keenan & Wakschlag, 2002) and is thought to be caused in part by coercive parent-child interactions (Lahey, Moffitt, & Caspi, 2003). The disorder also exhibits moderate heritability, and is substantially stable over time, particularly through childhood (Lavigne et al., 2001; Pihlakoski et al., 2006). ODD is associated with a number of negative and costly outcomes, such as poor family relations, academic problems, and high comorbidity with other disruptive behavior problems including conduct problems, aggression, and hyperactivity-impulsivity (Campbell, Spieker, Burchinal, Poe, & National Institute of Child Health and Human Development Early Child Care Research Network, 2006; Posner et al., 2007; Spira & Fischel, 2005).

Several conceptualizations of ODD have been proposed. Although DSM-IV-TR proposed just one set of symptoms for ODD (APA, 2000), DSM-5 divides the symptoms of ODD into three domains: angry/irritable (three symptoms: loses temper, touchy, angry), argumentative/defiant (four symptoms: argues, defies, annoys others, blames others), and spiteful (one symptom; APA, 2013). Further, recent bifactor modeling by Burke and colleagues (2014) suggests an alternative conceptualization of ODD that instead divides it into two domains: affective, or irritable (three symptoms: loses temper, touchy, angry) versus behavioral (five symptoms: argues, defies, annoys others, blames others, spiteful). Therefore, one-, two-, or three-cluster solutions may be reasonable.

In support of subdividing these symptoms into domains, theoretical and empirical work by Stringaris and Goodman (2009) suggests that ODD symptom domains exhibit differential patterns of associations with comorbid disorders. For example, angry/irritable ODD symptoms during childhood predict the development of internalizing and emotional disorders, such as depression, whereas argumentative/defiant ODD symptoms predict later attention-deficit/hyperactivity disorder (ADHD), conduct disorder (CD), delinquency, and callous-unemotional traits (Burke, Hipwell, & Loeber, 2010; Stringaris & Goodman, 2009; Whelan, Stringaris, Maughan, & Barker, 2013). Prior work suggests the vindictive symptom domain exhibits no specific longitudinal associations with particular disorders or outcomes. However, the vindictive symptom domain is theorized to be a better indicator of the later development of CD than the argumentative/defiant domain (Stringaris & Goodman, 2009; Whelan et al., 2013). Finally, theory suggests that negative affect, and irritability in particular, may be at the core of ODD symptoms, explaining the disorder's comorbidity with many other disorders, including mood disorders, other disruptive behavior problems, and ADHD (Burke, Hipwell, & Loeber, 2010; Martel, 2009; Stringaris & Goodman, 2009).

The current study utilizes a novel approach to examining the division of ODD symptoms through use of network analysis. This type of analysis provides unique insights into how symptoms of a disorder relate to each other and which symptoms cluster together (Borsboom & Cramer, 2013). It also provides information about which symptoms are core, or most central, to the overall network of symptoms. The current study is the first to empirically examine how ODD symptoms cluster. Specifically, we evaluate whether ODD symptoms appear to best cluster into one overall, two (affective versus behavior) or three (angry/irritable, argumentative/defiant, and spiteful) groups, as well as whether affective and irritability symptoms form the core of the ODD symptom cluster.

Methods

Participants

Overview

Participants were 109 preschoolers between the ages of 3 and 6 (M=4.34 years, SD=1.08) and their primary caregivers (hereafter termed parents for simplicity; 67% mothers with the remaining 33% fathers+mothers together, fathers only, foster parents, or grandmothers with guardianship). As shown in Table 1, 59% of the sample was male, and 36% of the sample represented an ethnic or racial minority (28% African American), coded dichotomously (0=non-Hispanic Caucasian; 1= any ethnic or racial minority). Family income ranged from below $20,000 to above $100,000 annually. Parental highest educational level ranged from grade school to doctorate, and family employment ranged from unemployed to full-time weekly.

Table 1.

Demographic and Descriptive Information on Sample

Control (c) ODD-only (o) ADHD-only (a) ODD+ADHD(oa)
n=30 n=18 n=18 n=43
M (SD) M (SD) M (SD) M (SD)
Age* 4.28 (1.07) 5.07 (1.19) 5.03 (.95) 4.89 (1.08)
Boys n(%) 14 (46.7) 10 (55.6) 13 (72.2) 27 (62.8)
Ethnic Minority n(%)* 7 (23.3) 2 (11.2) 10 (55.6) 17 (39.5)
Family Income (mode)* 1, 3 5 0 0
ODD symptoms (P)** 2.971,2 (3.08) 10.01,3 (6.02) 5.833,4 (3.28) 11.602,4 (7.24) c<a<o<oa
ADHD symptoms (P)** 8.601,2,3 (6.86) 19.731,4 (12.98) 26.722,5 (9.09) 35.263,4,5 (13.5) c<o<a<oa
    Inattention** 3.771,2,3 (3.87) 8.931,4 (6.77) 11.392,5 (5.88) 16.03,4,5 (7.29) c<o<a<oa
    Hyper-Impulsive** 4.831,2,3 (3.76) 10.81,4,5 (6.62) 15.332,4,6 (5.43) 19.263,5,6 (7.04) c<o<a<oa
ODD symptoms (T)** 2.641 (3.63) 3.802 (3.68) 5.173 (3.97) 11.841,2,3 (6.68) c, o, a<oa
ADHD symptoms (T)** 10.081,2 (9.11) 7.783,4 (7.97) 37.601,3 (10.16) 36.322,4 (8.51) c, o<a, oa
    Inattention** 4.151,2 (4.26) 3.893,4 (3.95) 23.201,3,5 (2.95) 17.952,4,5 (5.75) c, o<a, oa
    Hyper-Impulsive** 5.641,2 (5.47) 3.893,4 (4.17) 14.671,3 (7.99) 18.372,4 (5.41) c, o<a, oa
Open in a new tab

Note.

*

p<.05

**

p<.01.

Subgroup differences based on chi-square or ANOVA with follow-up LSD post hoc tests indicated with like superscripts. Family income modes: 0=annual income less than $20,000, 1=between $20,000 and $40,000, 2=between $40,000 and $60,000, 3=between $60,000 and $80,000, 4=between $80,000 and $100,000, and 5=over $100,000 annually. (P)=Parent report. (T)=Teacher report. Gift Delay Peek and Touch=high scores indicate better control.

Based on multistage and comprehensive diagnostic screening procedures (detailed below), preschoolers were recruited into two groups: those with Disruptive Behavior Disorders (DBD; n=79), subdivided into those with ADHD-only (n=18), Oppositional-Defiant Disorder (ODD)-only (n=18), and ADHD+ODD (n=43); and children without DBD (n=30). For more details about the demographics of each group see Table 1. The non-DBD group included preschoolers with minimal and subthreshold symptoms to provide a more continuous measure of symptoms, consistent with research suggesting that DBD may be better captured by continuous dimensions than categorical diagnosis and to be sensitive to the young age of the sample (Haslam et al., 2006; Marcus & Barry, 2011). No siblings were included.

Recruitment and Identification

Participants were recruited from an urban, Southern United States community primarily through direct mailings to families with children between the ages of three and six and internet postings, as well as through advertisements in newspapers and flyers posted at doctors’ offices, community centers, daycares, and on campus bulletin boards. Two sets of advertisements were utilized; one set of advertisements targeted children between ages 3 and 6 with disruptive behavior problems and/or attention problems and a second set of advertisements targeted children between ages 3 and 6 without these types of problems. After recruitment, all families passed through a multi-gated screening process. An initial telephone screening was conducted to rule out children prescribed psychotropic medication or children with neurological impairments, intellectual disability, psychosis, autism spectrum disorders, any seizure history, head injury with loss of consciousness, or other major medical conditions. Only 10 families were screened out at this phase.

Families were mailed teacher/caregiver questionnaires one week prior to the laboratory visit and instructed to provide the questionnaires to children's teacher, daycare provider, or babysitters who then mailed the completed questionnaires back to the university. When available (i.e., available on 50% of participating families), teacher/caregiver report on DBD symptoms was obtained via report on the Disruptive Behavior Rating Scale (DBRS; Barkley & Murphy, 2006). Response rate did not differ based on child DBD diagnostic group, χ2(3) = .59, p = .90, ethnic/racial minority status, χ2(1) = 1.73, p = .19, or family income, t(97) = 1.82, p =.07.

Parents and preschoolers attended a campus laboratory visit where they completed written and verbal informed consent procedures prior to data collection and consistent with the university Institutional Review Board, the National Institute of Mental Health, American Psychiatric Association guidelines, and in compliance with national and local legislation. At this visit, parents completed the Kiddie Disruptive Behavior Disorders Schedule (K-DBDS, Leblanc et al., 2008), a semi-structured diagnostic interview modeled after the Schedule for Affective Disorders and Schizophrenia for School-Age Children administered by a trained graduate student. The K-DBDS demonstrates high test-retest reliability and high inter-rater reliability in the preschool population (Leblanc et al., 2008). In the current study, fidelity to interview procedure was determined by calculation of reliability of blind interviewer ratings of DBD symptoms on a randomly-chosen 10% of families. Inter-rater clinician agreement was adequate for symptoms (ICC=.97). All families were compensated $50 for their participation.

Ultimately, clinical diagnoses and groupings were determined by the Principal Investigator, a licensed clinical psychologist, after a review of parent ratings on the KDBDS and (when available) teacher/caregiver ratings on the DBRS, consistent with current best practice guidelines for current diagnosis (Pelham et al., 2005).

Measures

Symptom Counts

Parent and teacher/caregiver reports on symptoms were available via the Disruptive Behavior Rating Scale (DBRS; Barkley & Murphy, 2006), which assesses symptoms using a 0 to 3 scale for a more continuous dimension. Symptom domain scores were calculated as sums of scores within each diagnostic subdomain (ODD symptoms, inattentive ADHD symptoms, hyperactive-impulsive ADHD symptoms). The DBRS has high internal consistency ranging from .78 to .96 in the preschool age range (Pelletier, Collett, Gimple, & Cowley, 2006). All scales for parent and teacher/caregiver report on the DBRS had high internal reliability (all alphas > .92) in the current sample. Primary analyses were conducted using parent report on the DBRS to maximize power.

Data Analytic Plan

A series of networks were computed using the R package qgraph (Epskamp, Cramer, Waldorp, Schmittmann, & Borsboom, 2012). We used the portion of the script and followed procedures provided in the supplemental materials from Borsboom and Cramer (2013). Networks were not specified to be directional, nor did we set a predetermined number of paths or strength of correlations. The ODD symptom networks were computed and then visualized using different colors to represent 2- and 3- symptom clusters, based on Burke (2014) and DSM (APA, 2013) recommendations respectively. Networks can be visually inspected to examine clustering and identify core symptoms. Statistical indices, called measures of centrality, were calculated to quantify aspects of the network, particularly node centrality, using the tnet package in R (Freeman, 1979; Opsahl, Agneessens, & Skvoretz, 2010). We used two indices of centrality: Closeness and Degree. Closeness represents the inverse of the sum of distance to all other nodes; higher numbers indicating that a node is more central to the network relative to the other items. Degree represents the sum of the weights of the relations with which a node is involved and captures the strength of the relations that a node has with all other nodes (Opsahl & Panzarasa, 2009). Again, higher numbers indicate higher centrality.

Results

The ODD symptom network can be viewed in Figure 1. First, this network was visualized using DSM-5 recommendations of three ODD symptom clusters: angry/irritable (three symptoms: loses temper, touchy, angry), argumentative/defiant (four symptoms: argues, defies, annoys others, blames others), and spiteful (1 symptom). As can be seen in Figure 1, symptoms seemed relatively evenly disbursed throughout the network, and centrality indices ranges were relatively small (closeness=.1409-.1442; degree=4.144-4.350). Visually, angry was the clear central, or core, symptom, in the middle and linking all other symptoms. This was also evident through measures of centrality which were high for angry (closeness=.1434; degree=2.276), as well as for annoys (closeness=.1442; degree=4.350) and argues (closeness=.1431; degree=4.281). The angry/irritable symptoms appeared to form the midline of the cluster being linked and directly between all other symptoms. Argumentative/defiant symptoms formed the outside of the cluster, scattered around the angry/irritable symptoms. Spiteful was linked to angry, loses temper, and argues, and looked more removed than many of the other symptoms in the cluster. This impression was confirmed with indices of central, as spiteful showed the lowest indices of central with closeness of .1411 and degree of 4.134, along with blames (closeness=.1409; degree=4.144).

Figure 1.

Figure 1

Open in a new tab

3-Cluster DSM ODD Symptom Domains: Angry/Irritable, Argumentative/Defiant, and Spiteful

Next, the ODD symptom network was visualized based on Burke's (2014) bifactor findings of two ODD symptom clusters: affective, or irritable (three symptoms: loses temper, touchy, angry) versus behavioral (five symptoms: argues, defies, annoys others, blames others, spiteful). As shown in Figure 2, this network is identical to the prior network, just differing in the colors in which symptoms are visualized. Similar to the Figure 1, Figure 2 suggests that affective, or irritability, symptoms form the midline of the cluster with the behavioral symptoms falling along the periphery. Yet, based on centrality indices, angry, annoys, and argues form the center of the cluster rather than angry, touchy, and temper as Burke's model would suggest. Therefore, overall, network analysis appears to provide slightly more support for the DSM-5 three-cluster model than the Burke two-cluster model.

Figure 2.

Figure 2

Open in a new tab

2-Cluster ODD Symptom Domains: Affective and Behavioral

Discussion

Network analysis of ODD symptoms suggests that ODD symptoms form a relatively homogenous network of which angry is a central symptom. Other affective or irritable symptoms also appear to be relatively central to the network with behavioral symptoms falling along the periphery, particularly spiteful. Results could be viewed as consistent with DSM-IV-TR (2000) one-cluster, DSM-5 (2013) three-cluster, or Burke's (2014) two-cluster model of ODD symptoms, but perhaps provide most support for the DSM-5 (2013) three-cluster solution. In addition, results support the idea that negative affect, particularly anger, form the core of the ODD symptom network during preschool.

Given the relative homogeneity of the symptom cluster, the DSM-IV-TR (2000) solution might also be appropriate. Yet, the centrality of the affective symptoms to the behavioral symptoms provides some support for Burke's (2014) bifactor model. Finally, spiteful seemed somewhat removed from most of the other ODD symptoms, providing some support for the DSM-5 three-cluster solution. Therefore, these findings may be most in line with recent hierarchical models that suggest that one-, two-, and three-cluster models may differentially fit the data depending on the level of analysis (Markon, 2010). Yet, the DSM-5 three-cluster solution seems best supported based on the relative centrality of affective symptoms and relative remoteness of the spiteful symptom in the preschool ODD symptom network. Validation of these different clustering solutions with external validation criteria will be an important next step since prediction of the development of comorbid problems is particularly important.

Regardless of the cluster solution used, results clearly suggest that negative affect is the core of the ODD symptom cluster (Burke, Hipwell, & Loeber, 2010; Martel, 2009; Stringaris & Goodman, 2009), particularly angry. Therefore, anger may be driving many of these other affective and behavioral symptoms. This is in line with recent work suggesting the importance of irritability in ODD, externalizing problems, and psychopathology in general (Burke et al., 2014; Burke, Hipwell, & Loeber, 2010; Leibenluft, Blair, Charney & Pine, 2003; Stringaris, 2011). This is also consistent with temperament theory suggesting negative emotionality is related to deficits in self-regulation and behavioral problems, including antisocial behavior and criminal justice involvement (DeLisi & Vaughn, 2014). As a result, preschool interventions for ODD symptoms might effectively focus on trying to teach parents how to coach their children in emotion regulation strategies in addition to focusing on parental behavioral management of behavioral problems (Eyberg, Nelson, & Boggs, 2008), although validation of such approaches in preschool are needed.

Additionally, results might shed some light on the controversial addition of Disruptive Mood Dysregulation Disorder (DMDD) to DSM-5, which is defined by two symptoms: irritable-angry mood and temper outbursts (APA, 2013). Those who meet criteria for both DMDD and ODD are only given the diagnosis of DMDD. However, current results suggest that angry forms the core of ODD, with other affective symptoms, including loses temper, being directly linked to all other symptoms. Therefore, results suggest that the two symptoms of DMDD are the core symptoms of ODD, which may drive all other symptoms. Exploratory analyses, examining the subsample of participants with only ODD (n=15), without comorbid ADHD, further support this idea, as loses temper became the central symptom of ODD (see online supplementary material). This is in line with recent longitudinal work suggesting DMDD symptoms are often components of other disorders, particularly ODD (Mayes et al., 2015). Of course, DMDD cannot be diagnosed until age 6, so it will be interesting to statistically examine symptom networks of both DMDD and ODD in older children to better understand what, if anything, differentiate the disorders.

Of course, the current study is limited in its focus on one measure, one age range, and lack of external validation measures. These are important directions for future work and replication initiatives. Additionally, it is possible that the variability in reporters and ADHD comorbidity within the small sample might have influenced the outcome. Exploratory analyses on the ODD only group, however, suggest results remain largely consistent in individuals without comorbid ADHD (see supplementary online material). Results of the current study support a three-cluster conceptualization of ODD and suggest the importance of angry as being a driving symptom, perhaps serving as a useful intervention target.

Supplementary Material

10802_2016_196_MOESM1_ESM

Table 2.

Network Indices of Centrality for ODD Symptom Network

Closeness Degree
Temper .1418 4.182
Touchy .1429 4.262
Angry .1434 4.276
Blames .1409 4.144
Annoys .1442 4.350
Argues .1431 4.281
Defies .1430 4.259
Spiteful .1411 4.134
Open in a new tab

Note. Closeness represents the distance between each singe node and all other nodes; degree represents the strength of the relationship each node has with all other nodes. Higher numbers are more central to the network. Bolded numbers represent those most central; italicized numbers represent those least central.

Acknowledgments

We are indebted to the families who made this study possible. This research was supported by National Institute of Health and Human Development Grant 5R03 HD062599-02 to M. Martel. The authors declare no conflicts of interest. This study seeks to compare and contrast different strategies for ODD symptom clustering and identify core symptoms of ODD during preschool. Another study, also in press at JACP, examines the structure of the externalizing spectrum, including ADHD and ODD symptoms, from preschool into adolescence. A final study, in press at CPS, examines developmental change in the ADHD symptom network between preschool and young adulthood seeking to identify common core symptoms of ADHD across age ranges. The main overlap is between ODD symptom network examination in the first two papers; however the papers have entirely different goals and mostly diverging samples (with the exception of the preschool sample which overlaps between studies). Analytic strategy is also the same in both papers.

Footnotes

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Contributor Information

Tess E. Smith, Department of Psychology, University of Kentucky, 111G Kastle Hall, Lexington, KY 40506

Christine A. Lee, Department of Psychology, University of Kentucky, 111 Kastle Hall, Lexington, KY 40506

Michelle M. Martel, Department of Psychology, University of Kentucky, 207C Kastle Hall, Lexington, KY 40506

Marni E. Axelrad, Department of Pediatrics, Psychology Section, Baylor College of Medicine, 6701 Fannin, Suite 1630, Houston, TX 77030

References

  1. American Psychiatric Association . Diagnostic and statistical manual of mental disorders: DSM-IV-TR. American Psychiatric Association; Washington, DC: 2000. [Google Scholar]
  2. American Psychiatric Association . Diagnostic and statistical manual of mental disorders—Fifth ed. American Psychiatric Association; Washington, D.C.: 2013. [Google Scholar]
  3. Barkley RA, Murphy KR. Attention-Deficit Hyperactivity Disorder: A clinical workbook. 3rd Ed. The Guilford Press; New York: 2006. [Google Scholar]
  4. Borsboom D, Cramer AOJ. Network analysis: An integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology. 2013;9:91–121. doi: 10.1146/annurev-clinpsy-050212-185608. [DOI] [PubMed] [Google Scholar]
  5. Burke JD, Boylan K, Rowe R, Duku E, Stepp SD, Hipwell AE, Waldman ID. Identifying the irritability dimension of ODD: Application of a modified bifactor model across five large community samples of children. Journal of Abnormal Psychology. 2014;123:841. doi: 10.1037/a0037898. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Burke JD, Hipwell AE, Loeber R. Dimensions of oppositional defiant disorder as predictors of depression and conduct disorder in preadolescent girls. Journal of the American Academy of Child & Adolescent Psychiatry. 2010;49:484–492. doi: 10.1097/00004583-201005000-00009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Campbell SB, Spieker S, Burchinal M, Poe MD, The NICHD Early Child Care Research Network Trajectories of aggression from toddlerhood to age 9 predict academic and social functioning through age 12. Journal of Child Psychology and Psychiatry. 2006;47:791–800. doi: 10.1111/j.1469-7610.2006.01636.x. [DOI] [PubMed] [Google Scholar]
  8. DeLisi M, Vaughn MG. Foundation for a temperament-based theory of antisocial behavior and criminal justice system involvement. Journal of Criminal Justice. 2014;42(1):1025. [Google Scholar]
  9. Epskamp S, Cramer AOJ, Waldorp LJ, Schmittman VD, Borsboom D. Qgraph: Network visualizations of relationships in psychometric data. Journal of Statistical Software. 2012;48:1–18. [Google Scholar]
  10. Eyberg SM, Nelson MM, Boggs SR. Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child & Adolescent Psychology. 2008;37:215–237. doi: 10.1080/15374410701820117. [DOI] [PubMed] [Google Scholar]
  11. Freeman LC. Centrality in social networks conceptual clarification. Social Networks. 1979;1:215–239. [Google Scholar]
  12. Haslam N, Williams B, Prior M, Haslam R, Graetz B, Sawyer M. The latent structure of attention-deficit/hyperactivity disorder: A taxonomic analysis. Australian and New Zealand Journal of Psychiatry. 2006;40:639–647. doi: 10.1080/j.1440-1614.2006.01863.x. [DOI] [PubMed] [Google Scholar]
  13. Keenan K, Wakschlag LS. Can a valid diagnosis of disruptive behavior disorder be made in preschool children? The American Journal of Psychiatry. 2002;159:351–358. doi: 10.1176/appi.ajp.159.3.351. [DOI] [PubMed] [Google Scholar]
  14. Lahey BB, Moffitt TE, Caspi A, editors. Causes of conduct disorder and juvenile delinquency. Guilford Press; 2003. [Google Scholar]
  15. Lavigne JV, Cicchetti C, Gibbons RD, Binns HJ, Larsen L, DeVitto C. Oppositional Defiant Disorder with onset in preschool years: Longitudinal stability and pathways to other disorders. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40:1393–1400. doi: 10.1097/00004583-200112000-00009. [DOI] [PubMed] [Google Scholar]
  16. Leblanc N, Boivin M, Dionne G, Brendgen M, Vitaro F, Tremblay RE, Pérusse D. The development of hyperactive–impulsive behaviors during the preschool years: The predictive validity of parental assessments. Journal of Abnormal Child Psychology. 2008;36:977–987. doi: 10.1007/s10802-008-9227-7. [DOI] [PubMed] [Google Scholar]
  17. Leibenluft E, Blair RJR, Charney DS, Pine DS. Irritability in pediatric mania and other childhood psychopathology. Annals of the New York Academy of Sciences. 2003;1008:201–218. doi: 10.1196/annals.1301.022. [DOI] [PubMed] [Google Scholar]
  18. Marcus DK, Barry TD. Does attention-deficit/hyperactivity disorder have a dimensional latent structure? A taxometric analysis. Journal of Abnormal Psychology. 2011;120:427–422. doi: 10.1037/a0021405. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Markon KE. Modeling psychopathology structure: A symptom-level analysis of Axis I and II disorders. Psychological medicine. 2010;40:273–288. doi: 10.1017/S0033291709990183. [DOI] [PubMed] [Google Scholar]
  20. Martel MM. Research Review: A new perspective on attention- deficit/hyperactivity disorder: emotion dysregulation and trait models. Journal of Child Psychology and Psychiatry. 2009;50:1042–1051. doi: 10.1111/j.1469-7610.2009.02105.x. [DOI] [PubMed] [Google Scholar]
  21. Mayes SD, Mathiowetz C, Kokotovich C, Waxmonsky J, Baweja R, Calhoun SL, Bixler EO. Stability of disruptive mood dysregulation disorder symptoms (irritable-angry mood and temper outbursts) throughout childhood and adolescence in a general population sample. Journal of Abnormal Child Psychology. 2015;43:1543–1549. doi: 10.1007/s10802-015-0033-8. [DOI] [PubMed] [Google Scholar]
  22. Opsahl T, Agneessens F, Skvoretz J. Node centrality in weighted networks: Generalizing degrees and shortest paths. Social Networks. 2010;32:245–251. [Google Scholar]
  23. Opsahl T, Panzarasa P. Clustering in weighted networks. Social Networks. 2009;31:155163. [Google Scholar]
  24. Pelham WE, Jr, Fabiano GA, Massetti GM. Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology. 2005;34:449–476. doi: 10.1207/s15374424jccp3403_5. [DOI] [PubMed] [Google Scholar]
  25. Pelham WE, Foster M, Robb JA. The economic impact of Attention-Deficit/Hyperactivity Disorder in children and adolescents. Ambulatory Pediatrics. 2007;7:121–131. doi: 10.1016/j.ambp.2006.08.002. [DOI] [PubMed] [Google Scholar]
  26. Pelletier J, Collett B, Gimple G, Cowley S. Assessment of disruptive behaviors in and School Situations Questionnaire. Journal of Psychoeducational Assessment. 2006;24:318. [Google Scholar]
  27. Pihlakoski L, Sourander A, Aromaa M, Rautava P, Helenius H, Sillanpaa M. The continuity of psychopathology from early childhood to preadolescence: A prospective cohort study of 3012-year-old children. European Child and Adolescent Psychiatry. 2006;15:409–417. doi: 10.1007/s00787-006-0548-1. [DOI] [PubMed] [Google Scholar]
  28. Posner K, Melvin GA, Murray DW, Gugga SS, Fisher P, Skrobala A, Kollins S. Clinical presentation of attention-deficit/hyperactivity disorder in preschool children: The preschoolers with attention-deficit/hyperactivity treatment study (PATS). Journal of Child and Adolescent Psychopharmacology. 2007;17:547–562. doi: 10.1089/cap.2007.0075. [DOI] [PubMed] [Google Scholar]
  29. Spira EG, Fischel JE. The impact of preschool inattention, hyperactivity, and impulsivity on social and academic development: A review. Journal of Child Psychology and Psychiatry. 2005;46:755–773. doi: 10.1111/j.1469-7610.2005.01466.x. [DOI] [PubMed] [Google Scholar]
  30. Stringaris A. Irritability in children and adolescents: A challenge for DSM-5. European Child & Adolescent Psychiatry. 2011;20:61–66. doi: 10.1007/s00787-010-0150-4. [DOI] [PubMed] [Google Scholar]
  31. Stringaris A, Goodman R. Longitudinal outcome of youth oppositionality: irritable, headstrong, and hurtful behaviors have distinctive predictions. Journal of the American Academy of Child & Adolescent Psychiatry. 2009;48:404–412. doi: 10.1097/CHI.0b013e3181984f30. [DOI] [PubMed] [Google Scholar]
  32. Whelan YM, Stringaris A, Maughan B, Barker ED. Developmental continuity of oppositional defiant disorder subdimensions at ages 8, 10, and 13 years and their distinct psychiatric outcomes at age 16 years. Journal of the American Academy of Child & Adolescent Psychiatry. 2013;52:961–969. doi: 10.1016/j.jaac.2013.06.013. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

10802_2016_196_MOESM1_ESM
NIHMS810641-supplement-10802_2016_196_MOESM1_ESM.docx (90KB, docx)

RESOURCES