Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Dec 29.
Published in final edited form as: JAMA Psychiatry. 2014 Sep;71(9):1015–1024. doi: 10.1001/jamapsychiatry.2014.763

Subtyping attention-deficit/hyperactivity disorder using temperament dimensions: toward biologically based nosologic criteria

Sarah L Karalunas 1,*, Damien Fair 1,2,3, Erica D Musser 4, Kamari Aykes 2, Swathi P Iyer 5, Joel T Nigg 1,2
PMCID: PMC4278404  NIHMSID: NIHMS646390  PMID: 25006969

Abstract

Importance

Psychiatric nosology is limited by behavioral and biological heterogeneity within existing disorder categories. The imprecise nature of current nosological distinctions limits both mechanistic understanding and clinical prediction. Here, we demonstrate an approach consistent with the NIMH Research Domain Criteria (RDoC) initiative to identifying superior, neurobiologically-valid subgroups with better predictive capacity than existing psychiatric categories for childhood Attention-Deficit Hyperactivity Disorder (ADHD).

Objective

Refine subtyping of childhood ADHD by using biologically-based behavioral dimensions (i.e. temperament), novel classification algorithms, and multiple external validators. In doing so, we demonstrate how refined nosology is capable of improving on current predictive capacity of long-term outcomes relative to current DSM-based nosology.

Design, Setting, Participants

437 clinically well-characterized, community-recruited children with and without ADHD participated in an on-going longitudinal study. Baseline data were used to classify children into subgroups based on temperament dimensions and to examine external validators including physiological and MRI measures. One-year longitudinal follow-up data are reported for a subgroup of the ADHD sample to address stability and clinical prediction.

Main Outcome Measures

Parent/guardian ratings of children on a measure of temperament were used as input features in novel community detection analyses to identify subgroups within the sample. Groups were validated using three widely-accepted external validators: peripheral physiology (cardiac measures of respiratory sinus arrhythmia and pre-ejection period), central nervous system functioning (via resting-state functional connectivity MRI), and clinical outcomes (at one-year longitudinal follow-up).

Results

The community detection algorithm suggested three novel types of ADHD, labeled as “Mild” (normative emotion regulation); “Surgent” (extreme levels of positive approach-motivation); and “Irritable” (extreme levels of negative emotionality, anger, and poor soothability). Types were independent of existing clinical demarcations, including DSM-5 presentations or symptom severity. These types showed stability over time and were distinguished by unique patterns of cardiac physiological response, resting-state functional brain connectivity, and clinical outcome one year later.

Conclusions and Relevance

Results suggest that a biologically-informed temperament-based typology, developed with a discovery-based community detection algorithm, provided a superior description of heterogeneity in the ADHD population than any current clinical nosology. This demonstration sets the stage for more aggressive attempts at a tractable, biologically-based nosology.

Introduction

Psychiatric nosology remains exclusively syndromic, depending on clusters of signs and symptoms rather than biologically-based measures. This dependence introduces fundamentally confounding factors limiting a better understanding of mental illness [14]. For these reasons, the National Institute of Mental Health, under the Research Domain Criteria (RDoC) initiative [2], has emphasized broad neurobiologically-based dimensions (e.g. positive/negative valence, cognitive control) to refine existing nosology. The long-term goal for this initiative, and the field in general, is to improve neurobiological validity, clinical prediction, and treatment matching [2]. While it is clear that eventually the nosology will need to be revised, a tractable approach to advance the current classification system is not agreed upon. For example, no study of alternative nosology has demonstrated both biological validity and clinical prediction within the same sample.

We address this problem for the case of Attention Deficit Hyperactivity Disorder (ADHD). Like other psychiatric categories, ADHD is suspected of identifying children with diverse etiologies [5, 6] but DSM-IV subtypes (and thus DSM-5 presentations) fail to elucidate clear differences in pathophysiology or clinical course, in part due to their own instability over time [5, 79]. To better parse heterogeneity some experts have advised looking beyond existing symptom lists toward phenotypic measures that can be represented dimensionally and have well-theorized relationships to neurobiological systems [2, 1012], which is the core of the RDoC approach. At the same time, phenotype measures that retain clinical applicability are desirable. ADHD is a good starting point for achieving RDoC aims because children with the disorder can be characterized in terms of several features that are best represented as dimensions and have well-theorized relationships to biological systems.

How are novel types to be empirically identified and validated? Clustering algorithms can be expected to yield varying results depending on the algorithm used and features selected as input [13] and therefore are, in themselves, exploratory [14]; the main question concerns whether clusters are clinically useful. Our key considerations were (a) input features, (b) choice of clustering method, and (c) multiple external validators.

Phenotypic features

For the features, we used a well-accepted, low-cost, parent-rated measure of child temperament. Temperament traits can be described as a set of biologically-based behavioral and emotional tendencies. While not the only approach available, these traits extend beyond single disorder clinical symptoms to expand coverage of positive and negative emotion systems, as well as attentional capacities [1517]. These in turn are directly related to the RDoC domains of negative valence systems, positive valence systems/approach motivation, and cognitive control, respectively. They also are related to, although not identical with, personality traits in adults, and are relatively easily assessed via parent-report questionnaires, which can be easily translated into clinical applications. Temperament traits also have distinct neurobiological correlates with a substantial literature in humans and animals [1820]. In humans, temperament domains are differentially related to peripheral nervous system indicators [2125], as well to specific neural circuitry [e.g. amygdala, hippocampus, insula, and striatum, anterior cingulate cortex, and prefrontal cortex, 26, 27]. The amygdala, in particular, plays a key role in emotion and motivation, and so is one important brain correlate of temperament domains [17, 28, 29].

Clustering methods

In the current report we follow up on our earlier work [10] that utilizes a clustering method known as community detection, which stems from the mathematical discipline of graph theory [30]. Graph theory concerns the study of networks, where networks are sets of nodes or vertices joined in pairs by lines or edges. Community structure in networks refers to the existence of densely connected groups of nodes, with only sparse connections between the groups. In our case, individuals with ADHD who share similar temperament traits may cluster to form specific subtypes of the disorder. Community detection is a widely used optimization clustering method (another well-known optimization clustering method is mixture modeling). As a clustering approach it has the attractive features of being data-driven (in that the number of communities does not have to be pre-specified in the model) and provides a quantitative measure of group robustness to chance variations in the data structure.

External Validation

For external validation, we consider (a) peripheral physiology, (b) central nervous system physiology (MRI), and (c) clinical outcome.

Methods

Participants and Diagnostic Procedures

Data are reported for 437 children: NControl=190 (46.3% male); NADHD=247 (71.6% male) between the ages of 7–11. Participants were drawn from the Oregon ADHD Program, a child cohort for which the community-based recruitment and enrollment procedures and multi-informant assessment procedures for ADHD diagnosis are published elsewhere [3133] and described in the Supplement. Sample characteristics for the ADHD and control groups are presented in eTable 1 in the Supplement for background. Ethics approval was obtained from the Institutional Review Board at Oregon Health & Science University. A parent/legal guardian provided written informed consent, and children provided written assent.

Temperament Ratings

A parent/guardian completed the widely used Temperament in Middle Childhood Questionnaire [TMCQ, 34]. Items form 16 subscales and three higher order factors that are hypothesized to be influenced by underlying neurobehavioral systems. List of scales and reliabilities in the current sample (which were all satisfactory) are provided in the Supplement.

Community Detection Analyses

Community detection [30, 35] is an optimization clustering method. As used here, a matrix of child-by-child correlations was created based on the 16 TMCQ subscales, standardized to the sample mean and standard deviation. Community detection was applied to the child matrix using Rubinov and Sporns’ [35] weight-conserving modularity algorithm [36]. This algorithm starts with each node (i.e. child) being the sole member of a community and iteratively produces partitions whose conjunction produces the largest increase in the quality index (Q) until Q reaches a maximum. In our case, Q is a weighted combination of the modularity for positive relationships (Q+) and negative relationships (Q) introduced by Rubinov and Sporns [35]. We provide the formal definition in the Supplement. Conceptually, Q represents the overall segregation between identified communities, with higher values indicating stronger separation of communities. Importantly, this and other modularity algorithms are not deterministic, and thus can yield slightly different solutions on different runs. To generate a single, stable depiction of structure, final group assignments for each child were based on the modal group assignment across 10 runs of the modularity algorithm. A secondary validation method [Variation of Information Algorithm (VOI), 37] was also used to evaluate internal validity (group robustness to chance data variation).

External Validation

Physiological Recording

Physiological response data were available for 178 children with ADHD and 128 control children. Temperament traits are conceptually related to positive and negative emotion domains, and so recording was completed during an emotion induction/suppression task with positive and negative emotion conditions, as well as during an emotionally-neutral baseline condition, exactly as described in prior publications [31]. Respiratory sinus arrhythmia (RSA), a measure of heart rate variability strongly influenced by parasympathetic regulation and hypothesized to be related to emotion regulation [38], was derived from the electrocardiogram (ECG) signal using MindWare® Heart Rate Variability software V.2.6 [39] and was quantified as the high frequency component (>0.15 Hz) of the R-R peak time series (in 60 second epochs). Standard methods for artifact detection were followed as published in our prior work [40] and are also provided in the Supplement.

Cardiac pre-ejection period (PEP) was derived from ECG and impedance cardiography in 60 second epochs using MindWare V.2.6 [41]. It was calculated as the time interval in milliseconds from the onset of the Q-wave to the B-point of the dZ/dt wave using published methods [42]. PEP reactivity is a generally agreed upon measure of sympathetic influence on the heart, while the interpretation of tonic PEP is more complex.

Neural Functioning

Resting-state functional connectivity data (rs fcMRI; 420–600 seconds) are reported for 39 ADHD (NMild=18, NSurgent=11, NIrritable=10) and 15 typically-developing children. Participants were scanned using a 3.0 Tesla Siemens Magnetom Tim Trio scanner with a twelve-channel head-coil using methods we have published [4547].

Functional images were processed to reduce artifacts and connectivity was processed following published methods [48] and are reproduced in the Supplement. Motion, a known confound in functional MRI studies, was corrected and quantified several ways [46, 49] (for details see the Supplement).

As noted earlier, one major candidate for neural involvement in temperament is the amygdala, so it was chosen as our single seed region for this demonstration analysis. Left and right amygdala regions of interest (ROIs) were obtained for subjects using FMRIB's Integrated Registration and Segmentation Tool (FIRST), distributed with FMRIB Software Library (FSL) [50, 51]. The ROIs were created in MNI atlas space and then converted to Talairach atlas space. All subsequent operations were performed on the Talairach atlas-transformed ROIs.

For all subjects, resting BOLD time series for each region of interest was correlated with all other voxels in the brain, generating voxelwise functional connectivity maps. We performed two-sample, two-tailed t-tests on all potential connections (Fischer transformed r-values) between community-defined subgroups groups (assuming unequal variance; p≤0.05), as well as between each community-defined subgroup and controls. One-sample tests were run for within- group connectivity maps. To account for multiple comparisons, thresholding based on Monte Carlo simulation was implemented [52]. To obtain multiple comparisons corrected, p<0.05 voxel clusters, a threshold of 53 contiguous voxels with a Z-value >2.25 was used.

Longitudinal Follow-up

One-year longitudinal follow-up data on clinical features and TMCQ ratings are reported for 101 children with ADHD (98% of those currently due for follow up in this ongoing study).

Results

Community Detection Analyses

Subgrouping was done in a multi-step fashion using community detection. At the first step, including all 437 children in the sample, community detection identified three profiles of children (Q=.55) with sizes of 192, 181, 64. Profile 1 included 82% of the control children and only 13% of the ADHD children. Profiles 2 and 3 together included 87% of the ADHD children and only 18% of non-ADHD controls. Separation of the ADHD and non-ADHD samples was not attributable to sample differences in age or gender (see Supplement). We concluded that our initial clustering analysis recovered a primarily “control” profile and 2 “ADHD” profiles. Thus, at the second step, the analysis was conducted on the ADHD and control samples separately with the aim of better characterizing variation within those two populations. Results from the controls revealed weakly divided, marginal subgroups and therefore were not subjected to further validation (see Supplement).

Community detection using the 247 children with ADHD identified as the modal solution three profiles (referred to herein as “types”) with n= 64 (25.9%), 85 (34.4%), and 98 (39.7%) based on modal type assignments. The quality quotient suggested strong separation of the groups (Q=.50). VOI analyses indicated that types were robust to modest random perturbations in the data (see eFigure 1 in Supplement). Types reproduced with similar quality scores after a random split replication of the ADHD population (Q=.46, Q=.50).

Clinically, Type 1 was more impulsive and inattentive (all p<.01) than controls, as well as less affiliative (all p<.05), but otherwise did not differ from controls on any temperament domain. Type 1 had milder impulsivity, inhibition, and attentional dyscontrol than the other two ADHD types (all p<.001). We labeled Type 1 “Mild ADHD.” Type 2 had more severe impulsivity than Type 1, and also less shyness and more assertiveness/dominance, high-intensity pleasure seeking, and activity level than either of the other two types or than controls (all p<.001); we labeled this type “Surgent ADHD.” Type 3, in addition to greater impulsivity and attentional dyscontrol than the Mild type, also had more negative emotionality than the other two types or the controls (all p<.001). This included higher levels of anger, discomfort, fear, and sadness, and lower levels of soothability (all p<.001), suggesting this is a group of children prone to anger, tantrums, and irritable behavior as described in recent literature [53]. Type 3 was thus labeled “Irritable ADHD.” Figure 1 provides a spring-embedding graph for depiction of degree of group separation in terms of the distance between subjects. Figure 2 depicts the profile for each group on the 16 temperament scales.

Figure 1. Spring-embedded visualization of temperament groups.

Figure 1

Graphical representation of the community detection results in the ADHD sample, which shows many strong correlations among individuals in the same temperament type and fewer between-type connections. Nodes represent individual’s in each temperament group (Blue: Mild, Red: Surgent, Green: Irritable) and connecting edges indicate correlations between individuals. (Note: Correlations are thresholded at .50 for purposes of visual representation; however, reported results are based on an unthresholded modularity algorithm as explained in Methods.)

Figure 2. Temperament type profiles.

Figure 2

TMCQ scores for each of the three temperament types identified in the ADHD sample. Scores are shown as z-scores relative to the control sample mean (such that 0 on the Y axis is the mean of the typically-developing sample). Standard errors are shown. Scores were reversed for some scales as follows: for Inhibition high scores indicate less inhibitory control; for Attentional Focus, high scores mean poorer focus; for Shyness, high scores mean less shy; for Soothability, high scores indicate less soothabtility. Panel A) shows scores on temperament domains related to Cognitive Control; B) shows scores for Surgency domains; and C) shows scores for Negative Emotion Domains. Two scales that did not differentiate types (Openness and Perceptual Sensitivity) are not show, but scores for these scales are reported in eTable 2 in the Supplement.

The DSM-5-based presentations of ADHD were split across the types suggesting the temperament types did not reduce to DSM presentations. Additional clinical and demographic description can be found in eTable 2 in the Supplement.

External Validity: I. Peripheral Physiology

Physiological measurements of peripheral nervous system response were recorded for 53 Mild, 61 Surgent, and 64 Irritable children with ADHD and 128 controls. Group differences were examined in 2(Valence: Positive/Negative)×2(Condition: Induction/Suppression)×4(Group) repeated-measures ANOVAs with gender covaried [54]. The ANOVA matrix was decomposed following procedures recommended by Keppel [55] with embedded correction for Type I error using a Fisherian decomposition strategy and an LSD test for simple comparisons.

Pre-Ejection Period

There were baseline differences in PEP between groups (F[1, 305]=2.73, p=.044, η2=0.03). Post-hoc tests indicated that the Surgent group had significantly shorter PEP values than typically-developing children or the other ADHD types. Similarly, tonic PEP scores in each of the four emotion task conditions were compared. Only the group main effect was significant (F[3, 303]=3.95, p=.009, η2=0.05). Post-hoc tests again confirmed that the Surgent type had shorter PEP values than the typically-developing controls, Irritable, and Mild ADHD types. None of the groups showed significant change in PEP from baseline to task (i.e. PEP reactivity scores did not differ from zero, all p>.05), and there were no significant main or interaction effects when PEP reactivity scores were examined for the emotion task (all p>.05). These results are summarized in Figure 2 and in eTable 3 in the Supplement.

RSA

RSA change from baseline to task was assessed, with positive scores (increases) interpreted as increased parasympathetic activity and negative scores (decreases) as reduced parasympathetic activity [56]. A main effect of group (F[3, 303]=3.97, p=.047, η2=0.03) was qualified by a 3-way interaction of Valence×Condition × Group (F[3, 303]=4.09, p=.018, η2=0.03). The follow up simple comparisons revealed that the Irritable type had a smaller increase in RSA than both the control and Mild groups during the two negative emotion conditions (all p<.05).

The Surgent Type differed from the other three groups during the negative suppression condition (p=.036, η2=0.04) because they showed no change in RSA in this condition from baseline. In the positive emotion conditions, the Surgent group also significantly differed from the other ADHD types and non-ADHD controls (p=.012, η2=0.05). Where other ADHD types and typically-developing children increased RSA, the Surgent group decreased RSA (p=.012, η2=0.05), consistent with withdrawal of parasympathetic activity (see Figure 2 and eTable 3).

Thus, data suggest that the Surgent and Irritable types have distinct peripheral physiological profiles as compared to each other and to typically-developing children. The Mild types, in contrast, showed a physiological profile similar to that of typically-developing children.

External Validation: II. Neural Connectivity

We examined functional connectivity of the amygdala. Findings related to each group were distributed throughout the cortex. We highlight a few of these differences here and they are reported fully in Table 1 and shown in Figure 4.

Table 1.

ROI table for between-group comparisons.

Surgent- Irritable Type Comparison
Region x y z Effect
Anterior Insula −36 21 5 Surgent < Irritable
−44 15 4 Surgent < Irritable
36 17 −4 Surgent < Irritable
36 18 6 Surgent < Irritable
Posterior Insula −36 −24 −11 Surgent > Irritable
Mid-/Post-Cingulate 7 −38 59 Surgent > Irritable
−8 −42 65 Surgent > Irritable
Temporal Lobe- Anterior/Superior −58 3 4 Surgent > Irritable
47 15 −9 Surgent > Irritable
Temporal Lobe- Inferior −40 −9 −35 Surgent > Irritable
Temporal Lobe- Medial −41 −10 −20 Surgent > Irritable
Temporal Lobe- Superior 61 −53 8 Irritable > Surgent
Occipital Lobe 8 −85 32 Irritable < Surgent
13 −84 20 Irritable < Surgent
Parahippocampus −33 −20 −24 Surgent > Irritable
Parietal Lobe −20 −36 56 Surgent > Irritable
Mild- Irritable Type Comparison
Region x y z Effect
Anterior Insula −29 20 8 Mild < Irritable
−37 14 7 Mild < Irritable
Temporal Lobe- Inferior −65 −20 −16 Mild > Irritable
Occipital Lobe −1 −90 8 Irritable < Mild
Occipital Lobe 13 −87 13 Irritable < Mild
Mild- Surgent Type Comparison
Region x y z Effect
Dorsal Medial Frontal 1 −22 56 Surgent < Mild
Mid-/Post-Cingulate 8 −40 56 Surgent > Mild
−10 −44 59 Mild < Surgent
Precuneus/Posterior Cingulate 15 −54 50 Mild < Surgent
Ventral Medial Prefrontal Cortex 2 34 6 Surgent < Mild
Control-Mild Type Comparison
Region x y z Effect
Occipital Lobe −3 −92 0 Control<Mild
−19 −81 −5 Control<Mild
−27 −93 −5 Control<Mild
−18 −99 −4 Control>Mild
21 −82 5 Control<Mild
26 −94 −3 Control<Mild
−24 −100 5 Control<Mild
Lateral Frontal 49 21 28 Control>Mild
Precuneus/Posterior Cingulate 4 −52 52 Control<Mild
−10 −43 51 Control<Mild
Inferior Parietal Lobe −55 −30 39 Control<Mild
−44 −36 42 Control<Mild
Anterior Insula −22 −7 18 Control<Mild
Cerebellum −22 −79 −28 Control>Mild
−31 −71 −27 Control<Mild
Caudate 19 −17 22 Control<Mild
Control-Surgent Comparison
Region x y z Effect
Medial Frontal −18 53 10 Control<Surgent
−30 54 1 Control<Surgent
Cerebellum 41 −81 −34 Control>Surgent
Control-Irritable Comparison
Region x y z Effect
Anterior Insula −43 14 4 Control> Irritable
−35 22 3 Control>Irritable
Precuneus −1 −69 24 Control> Irritable
Medial Frontal −33 44 27 Control> Irritable

Figure 4. rs fcMRI conjunction maps.

Figure 4

Amygdala connectivity maps for each temperament type were directly compared to the two other types and a matched control population. Results from each of these comparisons are provided in Table 1. The figure is a conjunction map for the comparisons. For each comparison (i.e. Mild vs Surgent, Mild vs Irritable, Mild vs Control – and similar for each of the two other subgroups), a voxel was coded as either 0 (not significantly different between groups) or 1 (significantly different between groups). Maps based on this coding were summed such that voxels that never differ between groups will have a value of 0 and voxels that differ in all comparisons for that group have a value of 3. In A) the Mild type differed from other types in areas in the posterior cingulate/precuneus (black arrow). B) As with the Mild type, the Surgent type also showed areas in the posterior cingulate and precuneus (black arrow) where it was distinct from at least 2 other groups. C) The Irritable type was quite distinct from Mild, Surgent, and control populations in the anterior insula (black arrows) – a region important for emotional regulation and task level control.

The Irritable group displayed weaker connectivity (in the form of reduced negative relationships) from the amygdala to the anterior insula relative to all other groups (including non-ADHD controls) and weaker connectivity to the posterior cingulate (a key hub in the default network) relative to typically-developing and the Surgent children. The Mild type showed the reduced connectivity profile of the posterior cingulate/precuneus versus the typically-developing and Surgent groups. The Surgent type evidenced increased connectivity from the amygdala to cingulate/precuneus compared to the Mild and Irritable groups, but was similar relative to control populations. All groups had differential patterns of connectivity to unique dorsolateral prefrontal cortex regions compared to at least one other group (see Table 1). Overall, each ADHD type had a distinct central nervous system physiology.

External validation: III. Longitudinal Stability and Clinical Outcome

Community detection was repeated using Time 2 data. Again, the modal solution suggested three ADHD profiles (Q=.51), which could be characterized as Mild (n=18), Surgent (n=42), and Irritable (n=41).

Multinomial logistic regression [57] indicated that temperament types were more stable than expected by chance (χ2[4]=71.84, p<.001). In both the Surgent and Irritable types, over 70% of children were assigned to the same type at follow-up. In contrast, the Mild type often changed profile: 43.6% remained Mild type at Time 2; 41% became Surgent type and 15.4% became Irritable type (see eVideo 1 in Supplement for depiction of stability of types over time).

With regard to clinical outcomes, children in the Irritable type at Time 1 were more likely than those in the other two types to develop a new comorbid disorder over the course of longitudinal follow-up (χ2[2]=10.11, p=.006), with more than double the rate of onsets as compared to either of the other types or non-ADHD controls (11.1%, 18.2%, 11.1%, and 39.5% in controls, Mild, Surgent, and Irritable types, respectively). This prediction was superior to other nosological designations at baseline that may be related to prognosis: neither DSM-5 presentations nor groups based on the presence/absence of ODD symptoms (i.e. 0 symptoms versus 1+ symptoms) differed in their rates of new comorbid disorder onsets at Time 2 (χ2[1]=0.68, p=.409 and χ2[1]=2.54, p=.111, respectively).

Furthermore, when temperament types, DSM-5 presentations, and oppositional defiant disorder symptom status were entered into a logistic regression model, the temperament types were the best predictor of comorbidity onsets (p=.019, Odds Ratio=2.46). Neither the presence of Time 1 ODD symptoms (p=.563) nor ADHD DSM-5 presentation (p=.113) uniquely predicted new onsets at Time 2. Future work can examine other clinical designations that were less common in the sample.

Discussion

Here, we demonstrate an approach to revising psychiatric nosology that: (1) goes beyond clinical symptoms, in this case choosing temperament measures that are closely related to RDoC domains; (2) utilizes a novel, discovery-based clustering algorithm; (3) demonstrates external validation concurrently using peripheral and central physiology; and (4) shows superior clinical prediction versus existing clinical categories related to ADHD or ADHD with comorbidity. Results suggest that revising the nosology in the case of ADHD is tractable and will be biologically meaningful. The current results add to prior work in schizophrenia [e.g. 58], suggesting that the use of biologically-influenced indicators, optimization-based clustering techniques, and comparison on external validators is an informative approach for understanding heterogeneity in psychiatric disorders.

We identify three distinct types of ADHD based on temperament profiles: a “Mild” type whose members are characterized only by deficits in core ADHD symptom domains; a “Surgent” type characterized by high levels of positive approach-motivated behaviors and activity level, shorter PEP,, parasympathetic withdrawal in response to positive emotions, and atypical amygdala connectivity to medial frontal areas; and an “Irritable” type characterized by high levels of negative emotionality, weak parasympathetic response to negative emotional stimuli, reduced amygdala-insula connectivity, and a doubling of risk for onset of new behavioral or emotional disorders.

Crucially, we considered multiple validators, choosing a subset from the classic validators suggested by Robins and Guze [59] and Cantwell [60], including biological correlates and clinical course. We found promising results suggesting that these types outperform existing nosology of ADHD. Here, the Surgent and Irritable types survived every test of validation, including distinct physiology and functional connectivity, stability over time, and better clinical prediction than DSM presentations or the most common comorbid grouping, ODD. Thus, they emerge as promising, novel types of ADHD that support the promise in this approach generally, and that warrant further validation.

Although the Mild type showed distinct patterns of functional connectivity compared to other types, they showed normative peripheral physiology and assignment to this type was not particularly stable over time. Thus, this type was not as well-validated as the others. From the perspective of Wakefield’s [61] definition of mental disorder as including a psychological or physiological dysfunction, these youth did not as clearly have a disorder even though they just as clearly met DSM-5 ADHD criteria.

Cardiac physiology measures suggest that the neurobiology of the Irritable type is related to weak parasympathetic response to negative emotionality. This pattern is consistent with high levels of negative valence emotions and low soothability captured in the temperament profile, as well as with high rates of comorbidity in this group. In contrast, the neurobiology of the Surgent type is related to parasympathetic withdrawal to positive valence emotionality, which is potentially consistent with high sensation-seeking and approach-motivated behaviors in this group. Although no differences were seen in PEP reactivity, the Surgent type had shortened tonic PEP scores. Tonic PEP may be influenced by many factors, including sympathetic nervous system activity, preload, afterload, peripheral resistance, and biomechanical processes. It will be interesting in future studies to examine whether the source of this effect can be described in terms of abnormal sympathetic arousal, due to its historic interest in externalizing disorders, or whether these tonic PEP differences reflect other factors.

The Irritable group, which had abnormal parasympathetic response to negative emotions, also had prominent atypical connectivity between the amygdala and the anterior insula. Although imaging results are based on a small subsample of participants and should be considered preliminary, the insula is a region that has been long known to be important for monitoring and adjusting sympathetic and parasympathetic tone. More recently, it has also been shown to be heavily involved in higher order control processing [62, 63]. Together these data may highlight the region’s importance for emotional regulation and impulse control.

Several of the neural regions (e.g. DLPFC, anterior insula) differentiating the ADHD temperament types are also implicated in executive control of attention [67]. Thus, our results, while preliminary due to the small sample size, are consistent with Petersen & Posner’s [67] recent elaboration of their attention model suggesting a close relationship between the anatomical correlates of executive attention and the broader constructs of behavioral and emotional self-regulation [6769]. Additional work examining the cognitive profiles of each temperament-based group, as well as to understand the relationships between attention and self-regulation models [e.g. 70] will be an important areas for future research.

Perhaps most encouraging here was that our new temperament types of ADHD predicted clinical outcome. In this case, we focused on onset of new comorbid psychiatric disorder at one year follow-up as an index of clinical deterioration. The new categories provided improved prediction versus existing clinical indicators.

With regard to implications for ADHD itself, these findings are broadly consistent with, but suggest some modifications to, prior theory of ADHD temperament variation that inspired this work [64, 65], while being mostly consistent with and building on previous work exploring temperament subgroups in ADHD in a different sample [66]. Although the subgroups found here were largely stable over time, future work will need to address issues related to family history, genetics, replicability in other samples, and treatment response [59, 60].

In addition, despite well-described limitations of parental-report measures [71], current results suggest that parent-reported temperament characteristics (at least as measured in this study) may be related to important individual differences in underlying physiology, suggesting some validity and clinical applicability for these measures. Importantly, temperament traits have been investigated in many other disorders [65] and further exploration of temperament types in other DSM-defined disorders may be informative for understanding shared and unique risk profiles and revising psychiatric nosology. Similarly, detailed comparison and integration of results obtained using alternative clustering algorithms or based on different input features will be important.

In terms of the proposed RDoC research framework, parent-report on the TMCQ broadly captures RDoC domains of negative valence systems, positive valence systems/approach motivation, and cognitive control. Additional studies examining sub-constructs within these domains (e.g. frustrative non-reward, reward valuation, or response selection and inhibition, respectively) may further refine the picture provided here. In addition, here we focus on relationships between parent-report measures, reflecting the “self-report” unit of analysis in the RDoC matrix, and peripheral and central nervous system indicators, reflecting the physiology and circuit units of analysis, respectively. Additional work examining relationships with other units of analysis, such cells or genes, will also be informative.

Overall, we conclude that progress toward a neurobiologically-based nosology, as envisioned in the NIMH RDoC initiative [2], is tractable. In the illustration here, temperament profiles in children with ADHD can be related to meaningful differences in both central and peripheral nervous system and longitudinal course, setting the stage for improving nosology.

Supplementary Material

Supplementary
Video
Download video file (21.4MB, mp4)

Figure 3. Cardiac physiological response.

Figure 3

A) Mean RSA change from baseline and B) raw PEP for each of the emotion task epochs: negative induction (NI), negative suppression (NS), positive induction (PI), and positive suppression (PS), shown by temperament type.

Acknowledgments

Many special thanks to Colleen Schmitt, M.S. and Hilary Galloway-Long, B.A. for help with data and project management. Funding was provided by R01 MH059105 (Nigg), R01 MH086654 (Nigg), R01 MH096773 (Fair), R00 MH091238 (Fair), and F32 MH098632 (Karalunas).

Footnotes

Author Contributions: Karalunas: community detection analysis, behavioral, and longitudinal analyses, writing and manuscript preparation; Fair: functional connectivity analyses, community detection analysis, writing and manuscript preparation; Musser: cardiac physiology analyses, writing and manuscript preparation; Aykes: functional connectivity analyses; Iyer: community detection analysis; Nigg: theoretical conceptualization, hypotheses, project data collection and oversight, writing and manuscript preparation.

Conflicts of Interest: Dr. Karalunas reported no biomedical financial interests or potential conflicts of interest. Dr. Fair reported no biomedical financial interests or potential conflicts of interest. Dr. Musser reported no biomedical financial interests or potential conflicts of interest. Ms. Aykes reported no biomedical financial interests or potential conflicts of interest. Ms. Iyer reported no biomedical financial interests or potential conflicts of interest. Dr. Nigg reported no biomedical financial interests or potential conflicts of interest.

References

  • 1.Doyle AE, et al. Are endophenotypes based on measures of executive functions useful for molecular genetic studies of ADHD? Journal of Child Psychology and Psychiatry. 2005;46(7):778–803. doi: 10.1111/j.1469-7610.2005.01476.x. [DOI] [PubMed] [Google Scholar]
  • 2.Insel T, et al. Research domain criteria (RDoC): Toward a new classification framework for research on mental disorders. The American Journal of Psychiatry. 2010;167(7):748–751. doi: 10.1176/appi.ajp.2010.09091379. [DOI] [PubMed] [Google Scholar]
  • 3.Faraone SV. Genetics of childhood disorders: XX. ADHD, part 4: Is ADHD genetically heterogeneous? Journal of the American Academy of Child & Adolescent Psychiatry. 2000;39(11):1455–1457. doi: 10.1097/00004583-200011000-00022. [DOI] [PubMed] [Google Scholar]
  • 4.Todd RD. Genetics of childhood disorders: XXI. ADHD, Part 5: A behavioral genetic perspective. Journal of the American Academy of Child & Adolescent Psychiatry. 2000;39(12):1571–1573. doi: 10.1097/00004583-200012000-00023. [DOI] [PubMed] [Google Scholar]
  • 5.Khan SA, Faraone SV. The genetics of ADHD: a literature review of 2005. Current Psychiatry Reports. 2006;8(5):393. doi: 10.1007/s11920-006-0042-y. [DOI] [PubMed] [Google Scholar]
  • 6.Nigg J. Attention-Deficit/Hyperactivity Disorder: Endophenotypes, Structure, and Etiological Pathways. Current Directions in Psychological Science. 2010;19(1):24. [Google Scholar]
  • 7.Chhabildas N, Pennington BF, Willcutt EG. A comparison of the neuropsychological profiles of the DSM-IV subtypes of ADHD. Journal of Abnormal Child Psychology. 2001;29(6):529–540. doi: 10.1023/a:1012281226028. [DOI] [PubMed] [Google Scholar]
  • 8.Lahey BB, Pelham WE, Loney J, Lee SS, Willcutt E. Instability of the DSM-IV subtypes of ADHD from preschool through elementary school. Archives of General Psychiatry. 2005;62(8):896–902. doi: 10.1001/archpsyc.62.8.896. [DOI] [PubMed] [Google Scholar]
  • 9.Willcutt EG, et al. Validity of DSM–IV Attention Deficit/Hyperactivity Disorder Symptom Dimensions and Subtypes. 2012 doi: 10.1037/a0027347. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Fair D, Bathula D, Nikolas M, Nigg JT. Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD. Proceedings of the National Academy of Sciences of the United States of America. 2012 doi: 10.1073/pnas.1115365109. 10.1073/pnas.1115365109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Georgiades S, Szatmari P, Boyle M. Importance of studying heterogeneity in autism. Neuropsychiatry. 2013;3(2):123–125. [Google Scholar]
  • 12.Sanislow CA, et al. Developing constructs for psychopathology research: Research domain criteria. The Journal of Abnormal Psychology. 2010;119(4):631–639. doi: 10.1037/a0020909. [DOI] [PubMed] [Google Scholar]
  • 13.Everitt BS, Landau S, Leese M, Stahl D. Cluster Analysis. 5th ed. West Sussex, United Kingdom: John Wiley & Sons, Ltd; 2011. [Google Scholar]
  • 14.Jain AK. Data clustering: 50 years beyond K-means. Pattern Recognition Letters. 2010;31(8):651–666. [Google Scholar]
  • 15.Bates JE, Goodnight JA, Fite JE. In: Temperament and Emotion, in Handbook of Emotion. Lewis M, Haviland-Jones JM, Barrett LF, editors. New York, NY: The Guilford Press; 2008. [Google Scholar]
  • 16.Rothbart MK. Becoming who we are: Temperament and personality in development. New York, NY: Guidford Press; 2011. [Google Scholar]
  • 17.Smillie LD. Extraversion and Reward Processing. Current Directions in Psychological Science. 2013;22(3):167–172. doi: 10.1177/0963721412474460. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Depue RA, Lenzenweger MF. A neurobehavioral dimensional model of personality disturbance. Major theories of personality disorder. 2005;2:391–453. [Google Scholar]
  • 19.Kennis M, Rademaker AR, Geuze E. Neural correlates of personality: An integrative review. Neuroscience & Biobehavioral Reviews. 2012 doi: 10.1016/j.neubiorev.2012.10.012. [DOI] [PubMed] [Google Scholar]
  • 20.Zuckerman M. Psychobiology of personality. 2nd ed. New York, NY: Cambridge University Press; 2005. [Google Scholar]
  • 21.Beauchaine TP, Katkin ES, Strassberg Z, Snarr J. Disinhibitory psychopathology in male adolescents: Discriminating conduct disorder from attention-deficit/hyperactivity disorder through concurrent assessment of multiple autonomic states. Journal of Abnormal Psychology. 2001;110(4):610–624. doi: 10.1037//0021-843x.110.4.610. [DOI] [PubMed] [Google Scholar]
  • 22.Healy B, Treadwell A, Reagan M. Measures of RSA suppression, attentional control, and negative affect predict self-ratings of executive functions. Journal of Psychophysiology. 2011;25(4):164–173. [Google Scholar]
  • 23.Miller JG, et al. Children's dynamic RSA change during anger and its relations with parenting, temperament, and control of aggression. Biological Psychology. 2012 doi: 10.1016/j.biopsycho.2012.12.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Stifter CA, Dollar JM, Cipriano EA. Temperament and emotion regulation: the role of autonomic nervous system reactivity. Developmental psychobiology. 2011;53(3):266–279. doi: 10.1002/dev.20519. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Zalewski M, Lengua LJ, Wilson AC, Trancik A, Bazinet A. Emotion regulation profiles, temperament, and adjustment problems in preadolescents. Child Development. 2011;82(3):951–966. doi: 10.1111/j.1467-8624.2011.01575.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Ochsner KN, Silvers JA, Buhle JT. Functional imaging studies of emotion regulation: a synthetic review and evolving model of the cognitive control of emotion. Annals of the New York Academy of Sciences. 2012;1251(1):E1–E24. doi: 10.1111/j.1749-6632.2012.06751.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Whittle S, Allen NB, Lubman DI, Yücel M. The neurobiological basis of temperament: Towards a better understanding of psychopathology. Neuroscience & Biobehavioral Reviews. 2006;30(4):511–525. doi: 10.1016/j.neubiorev.2005.09.003. [DOI] [PubMed] [Google Scholar]
  • 28.Miskovic V, Schmidt LA. New directions in the study of individual differences in temperament: A brain–body approach to understanding fearful and fearless children. Monographs of the Society for Research in Child Development. 2012;77(2):28–38. [Google Scholar]
  • 29.Rothbart MK. Temperament, development, and personality. Current Directions in Psychological Science. 2007;16(4):207–212. [Google Scholar]
  • 30.Newman MEJ. Modularity and community structure in networks. Proceedings of the National Academy of Sciences of the United States of America. 2006;103(23):8577–8582. doi: 10.1073/pnas.0601602103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Musser ED, Galloway-Long HS, Frick PJ, Nigg JT. Emotion Regulation and Heterogeneity in Attention-Deficit/Hyperactivity Disorder. Journal of the American Academy of Child & Adolescent Psychiatry. 2013;52(2):163–171. e2. doi: 10.1016/j.jaac.2012.11.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Karalunas SL, Huang-Pollock CL, Nigg JT. Decomposing attention-deficit/hyperactivity disorder (ADHD)-related effects in response speed and variability. Neuropsychology. 2012;26(6):684–94. doi: 10.1037/a0029936. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Nikolas MA, Nigg JT. Neuropsychological performance and attention-deficit hyperactivity disorder subtypes and symptom dimensions. Neuropsychology. 2013;27(1):107. doi: 10.1037/a0030685. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Simonds J, Rothbart MK. Athens, GA: 2004. The Temperament in Middle Childhood Questionnaire (TMCQ): A computerized self-report measure of temperament for ages 7–10, in Occasional Temperament Conference. [Google Scholar]
  • 35.Rubinov M, Sporns O. Weight-conserving characterization of complex functional brain networks. Neuroimage. 2011;56(4):2068–2079. doi: 10.1016/j.neuroimage.2011.03.069. [DOI] [PubMed] [Google Scholar]
  • 36.Clauset A, Newman ME, Moore C. Finding community structure in very large networks. Physical Review E. 2004;70(6):066111. doi: 10.1103/PhysRevE.70.066111. [DOI] [PubMed] [Google Scholar]
  • 37.Karrer B, Levina E, Newman ME. Robustness of community structure in networks. Physical Review E. 2008;77(4):046119. doi: 10.1103/PhysRevE.77.046119. [DOI] [PubMed] [Google Scholar]
  • 38.Obradović J, Boyce WT. DEVELOPMENTAL PSYCHOPHYSIOLOGY OF EMOTION PROCESSES. Monographs of the Society for Research in Child Development. 2012;77(2):120–128. [Google Scholar]
  • 39.Mind Ware Heart Rate Variability. Gahanna, OH: MindWare Technologies; 2008. [Google Scholar]
  • 40.Musser ED, et al. Emotion regulation via the autonomic nervous system in children with attention-deficit/hyperactivity disorder (ADHD) Journal of Abnormal Child Psychology: An official publication of the International Society for Research in Child and Adolescent Psychopathology. 2011;39(6):841–852. doi: 10.1007/s10802-011-9499-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Mind Ware Impedance Cardiography. Gahanna, OH: MindWare Technologies; 2008. [Google Scholar]
  • 42.Berntson GG, Lozano D, Chen Y-J, Cacioppo JT. Where to Q in PEP. Psychophysiology. 2004;41:333–337. doi: 10.1111/j.1469-8986.2004.00156.x. [DOI] [PubMed] [Google Scholar]
  • 43.Cacioppo JT, et al. Autonomic cardiac control. II. Noninvasive indices and basal response as revealed by autonomic blockades. Psychophysiology. 1994;31(6):586–598. doi: 10.1111/j.1469-8986.1994.tb02351.x. [DOI] [PubMed] [Google Scholar]
  • 44.Schächinger H, Weinbacher M, Kiss A, Ritz R, Langewitz W. Cardiovascular indices of peripheral and central sympathetic activation. Psychosomatic medicine. 2001;63(5):788–796. doi: 10.1097/00006842-200109000-00012. [DOI] [PubMed] [Google Scholar]
  • 45.Fair D, et al. Differentiating combined and inattentive subtypes of ADHD with healthy controls using resting-state functional connectivity MRI. Frontiers in Systems Neuroscience. 2013;1 [Google Scholar]
  • 46.Fair DA, et al. Distinct neural signatures detected for ADHD subtypes after controlling for micro-movements in resting state functional connectivity MRI data. Frontiers in Systems Neuroscience. 2012;6 doi: 10.3389/fnsys.2012.00080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Fair DA, et al. Atypical DefaultNetwork Connectivity in Youth with Attention-Deficit/Hyperactivity Disorder. Biological Psychiatry. 2010;68(12):1084–1091. doi: 10.1016/j.biopsych.2010.07.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Miezin FM, Maccotta L, Ollinger J, Petersen S, Buckner R. Characterizing the hemodynamic response: effects of presentation rate, sampling procedure, and the possibility of ordering brain activity based on relative timing. Neuroimage. 2000;11(6):735–759. doi: 10.1006/nimg.2000.0568. [DOI] [PubMed] [Google Scholar]
  • 49.Power JD, Barnes KA, Snyder AZ, Schlaggar BL, Petersen SE. Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion. Neuroimage. 2012;59(3):2142–2154. doi: 10.1016/j.neuroimage.2011.10.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Patenaude B, Smith S, Kennedy D, Jenkinson M. FIRST-FMRIB’s integrated registration and segmentation tool; Human Brain Mapping Conference; 2007. [Google Scholar]
  • 51.Patenaude B, Smith SM, Kennedy DN, Jenkinson M. A Bayesian model of shape and appearance for subcortical brain segmentation. Neuroimage. 2011;56(3):907–922. doi: 10.1016/j.neuroimage.2011.02.046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Forman SD, et al. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magnetic Resonance in Medicine. 1995;33(5):636–647. doi: 10.1002/mrm.1910330508. [DOI] [PubMed] [Google Scholar]
  • 53.Leibenluft E. Severe mood dysregulation, irritability, and the diagnostic boundaries of bipolar disorder in youths. The American Journal of Psychiatry. 2011;168(2):129. doi: 10.1176/appi.ajp.2010.10050766. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Beauchaine TP, Hong J, Marsh P. Sex differences in autonomic correlates of conduct problems and aggression. Journal of the American Academy of Child & Adolescent Psychiatry. 2008;47(7):788–796. doi: 10.1097/CHI.0b013e318172ef4b. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Keppel G. Review of Statistical Analysis in Psychology and Education. PsycCRITIQUES. (5th ed.) 1981;26(9):723–723. [Google Scholar]
  • 56.Butler EA, Wilhelm FH, Gross JJ. Respiratory sinus arrhythmia, emotion, and emotion regulation during social interaction. Psychophysiology. 2006;43(6):612–622. doi: 10.1111/j.1469-8986.2006.00467.x. [DOI] [PubMed] [Google Scholar]
  • 57.Peng C-YJ, Lee KL, Ingersoll GM. An introduction to logistic regression analysis and reporting. The Journal of Educational Research. 2002;96(1):3–14. [Google Scholar]
  • 58.Lenzenweger MF, McLachlan G, Rubin DB. Resolving the latent structure of schizophrenia endophenotypes using expectation-maximization-based finite mixture modeling. Journal of Abnormal Psychology. 2007;116(1):16. doi: 10.1037/0021-843X.116.1.16. [DOI] [PubMed] [Google Scholar]
  • 59.Robins E, Guze SB. Establishment of diagnostic validity in psychiatric illness: its application to schizophrenia. The American Journal of Psychiatry. 1970 doi: 10.1176/ajp.126.7.983. [DOI] [PubMed] [Google Scholar]
  • 60.Cantwell DP. Classification of Child and Adolescent Psychopathology. Journal of Child Psychology and Psychiatry. 1996;37(1):3–12. doi: 10.1111/j.1469-7610.1996.tb01377.x. [DOI] [PubMed] [Google Scholar]
  • 61.Wakefield JC, First MB, Phillips KA, Pincus HA. Clarifying the distinction between disorder and nondisorder. Advancing DSM: Dilemmas in psychiatric diagnosis. 2003:23–55. [Google Scholar]
  • 62.Dosenbach NU, Fair DA, Cohen AL, Schlaggar BL, Petersen SE. A dual-networks architecture of top-down control. Trends in Cognitive Sciences. 2008;12(3):99–105. doi: 10.1016/j.tics.2008.01.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Dosenbach NU, et al. Distinct brain networks for adaptive and stable task control in humans. Proceedings of the National Academy of Sciences. 2007;104(26):11073–11078. doi: 10.1073/pnas.0704320104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Nigg J, Goldsmith HH, Sachek J. Temperament and Attention Deficit Hyperactivity Disorder: The Development of a Multiple Pathway Model. Journal of Clinical Child and Adolescent Psychology. 2004;33(1):42–53. doi: 10.1207/S15374424JCCP3301_5. [DOI] [PubMed] [Google Scholar]
  • 65.Nigg JT. Temperament and developmental psychopathology. Journal of Child Psychology and Psychiatry. 2006;47(3–4):395–422. doi: 10.1111/j.1469-7610.2006.01612.x. [DOI] [PubMed] [Google Scholar]
  • 66.Martel MM, Goth-Owens T, Martinez-Torteya C, Nigg JT. A person-centered personality approach to heterogeneity in attention-deficit/hyperactivity disorder (ADHD) Journal of Abnormal Psychology. 2010;119(1):186–196. doi: 10.1037/a0017511. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Petersen SE, Posner MI. The attention system of the human brain: 20 years after. Annual Review of Neuroscience. 2012;35:73. doi: 10.1146/annurev-neuro-062111-150525. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Posner M, Rothbart MK. Educating the human brain. Washington, DC, US: American Psychological Association; 2007. [Google Scholar]
  • 69.Rothbart MK, Sheese BE, Rueda MR, Posner MI. Developing mechanisms of self-regulation in early life. Emotion review. 2011;3(2):207–213. doi: 10.1177/1754073910387943. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Nigg JT, Casey B. An integrative theory of attention-deficit/hyperactivity disorder based on the cognitive and affective neurosciences. Development and Psychopathology. 2005;17(03):785–806. doi: 10.1017/S0954579405050376. [DOI] [PubMed] [Google Scholar]
  • 71.Kagan J, Snidman N, McManis M, Woodward S, Hardway C. One measure, one meaning: Multiple measures, clearer meaning. Development and Psychopathology. 2002;14(03):463–475. doi: 10.1017/s0954579402003048. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary
Video
Download video file (21.4MB, mp4)

RESOURCES