Heterogeneity and subtyping in attention-deficit/hyperactivity disorder— Considerations for emerging research using person-centered computational approaches

Abstract

Few if any experts believe that existing psychiatric diagnostic categories included in the DSM/ICD are actually discrete disease entities. Attention-deficit/hyperactivity disorder (ADHD) is emblematic of the problems in the existing psychiatric classification system. ADHD symptoms reliably cluster into two correlated dimensions in factor analysis. However, children with ADHD vary considerably in their symptom profiles, symptom trajectories, clinical outcomes, and biological and psychological correlates. Thus, the field has sought alternative approaches that harness the dimensions of emotional, cognitive, and behavioral functioning that underlie ADHD and other existing psychiatric categories to create informative phenotypes that improve clinical prediction and clarify etiology. Within ADHD, cognitive (neuropsychological) and temperament/personality features have received considerable attention. In some cases, subphenotypes based on these features appear to improve on existing classifications and could eventually be translated into clinical practice. This review summarizes findings from sub-phenotyping efforts in ADHD that use cognitive, emotion-related, and other features to highlight major considerations for research applying person-oriented approaches to inform an improved psychiatric nosology. Considerations related to feature selection, validation of newly proposed divisions, defining populations of interest, and incorporating a developmental perspective are all discussed.

The history of nosology in psychiatry, medicine, and biology has featured a fundamental split between two schools of thought (see ^{1, 2}). One school of thought assumes that categories exist in nature and the job of the empirical nosologist is to discover them. The second school of thought eschews this assumption as either false or unknowable, and instead proposes that nosology should be functionally useful for some purpose beyond simple description. The related Diagnostic and Statistical Manual (DSM) and International Classification of Diseases (ICD) approaches are the most widely used psychiatric classifications, but few if any experts believe that the disorders included in DSM/ICD are actually discrete diseases, and there are welldocumented limitations on the functional utility of the DSM categories as well (3). Thus, the field has sought alternative approaches that harness the dimensions of emotional, cognitive, and behavioral functioning that underlie existing categories to create informative phenotypes that improve clinical prediction and clarify etiology. Dimensionally-based approaches to psychiatric nosology are not new (4–6). Two currently influential approaches are the Research Domain Criteria initiative (RDoC (7–9)) and the Hierarchical Taxonomy of Psychopathology (HiTOP (10, 11)). RDoC places greater emphasis on linkage to biological systems while HiTOP emphasizes functional clinical description, but both seek to use quantitative traits to develop a new, or at least improved, empirically-informed nosology.

Attention-deficit/hyperactivity disorder (ADHD) is emblematic of the problems in existing psychiatric classification systems. ADHD inattention and hyperactivity-impulsivity symptoms reliably cluster into two correlated dimensions in factor analysis (some studies do find a third impulsivity dimension). However, children with ADHD vary considerably in their symptom profiles, symptom trajectories, clinical outcomes, and biological and psychological correlates (12–14). Sub-profiling of ADHD has been built into its conceptualization since at least DSM-III (15) when the option for diagnosis with and without hyperactivity was added to the diagnostic criteria. Several major theories of ADHD have also highlighted the potential for etiological heterogeneity (16–19). However, at present, there is no consensus about which subtypes will be informative nor even about which features are most relevant for subtyping efforts. As a result, calls for refinement of phenotypes have continued (20).

In this review we describe conceptual issues related to sub-phenotyping in ADHD using symptom-based, cognitive (neuropsychological), and personality/temperament features. This is not to suggest that these specific features are more important than a variety of others but rather to provide useful illustrative examples of the integration of categorical (e.g., DSM/ICD) and dimensional (e.g., HiTOP (10), RDoC (21)) approaches. We highlight research that has applied statistical and mathematical clustering algorithms, rather than studies using arbitrarily defined cutoffs, because of the improved statistical accuracy and reproducibility that these methods afford. However, we also discuss issues related to clinical translation and limitations of these approaches. The overall goal is to highlight conceptual and methodological considerations that we see as critical to revising current nosology.

Which features are relevant?

Clustering analyses are inherently feature dependent and exploratory (22)— the profiles or groups identified will depend on the features included in the model and then require validation in terms of their incremental improvement over existing clinical indicators or subgroups for a particular purpose. Relevant groups may exist but be missed due to incorrect feature selection. The opposite is also true. Input features may produce subgroups that reliably differ on model features but that are not clinically or etiologically useful. These challenges are analogous to familiar issues of reliability and validity in psychometric measurement theory. Groups that are not reliably detected with similar input features are unlikely to be useful. However, the existence of reliable groups, on its own, does not guarantee that groups are valid.

Here, we suggest that “valid” means that groups are informative in relation to: 1) etiological signal, 2) biological correlates, and/or 3) clinical prediction, recognizing that different features set may serve each of these purposes. In regard to clinical prediction, DSM-oriented categorical outcomes may be useful (e.g., prospectively predicting onset of new disorders over time is a marker of worsening clinical course). However, the same limitations in using them as predictors also weaken their value as outcomes. Considering a wider set of dimensional outcomes (e.g., functional impairment, ratings of positive adaptation) and granular defined outcomes (e.g., school dropout, encounters with law enforcement) will be important. Treatment response could also serve as an important outcome measure. Any improvements in the field’s ability to predict who will respond to specific treatments would be a large step forward. (As an anonymous reviewer rightly pointed out, treatment response could also be an input feature and used as a nosological probe to identify relevant groups for additional clinical study.)

DSM-based features

Common approaches that stay within the feature sets proposed by DSM have been to isolate subgroups based on comorbidity patterns (e.g., ADHD with and without conduct problems) or on symptom severity. Both approaches have been somewhat helpful as an initial rendering, but each has significant limitations. With regard to comorbidity, the associated disorders are themselves subject to arbitrary cutoffs, potentially adding to heterogeneity and decreasing diagnostic reliability (23, 24). With regard to symptom severity, sub-profiling using DSM ADHD symptoms demonstrates the problem of reliable but ultimately minimally informative groups. Latent class analyses with ADHD symptoms reliably identify classes corresponding to DSM inattentive and combined presentations, and these presentations are often further subdivided into severity classes (e.g., mild inattentive, severe inattentive) (25, 26). However, groups add little incremental contribution to standard clinical practice, which already routinely accounts for symptom severity. In addition, individuals do not remain the same profile over time (26), ultimately limiting the predictive utility.

Non-DSM domains as features

Recent efforts have emphasized a broader set of phenotypic features outside of the scope of formal ADHD diagnostic criteria (e.g., sluggish cognitive tempo, ^{27f, 28}). Here, we expand on two specific attempts to broaden phenotypic features using: 1) cognitive functioning and 2) temperament/personality (related to emotional functioning). Both domains have well-established psychometric measurement properties that add to their appeal for subgrouping. Both are relatively well-understood in terms of potential biological correlates and thus lead directly to mechanistic hypotheses in the case that profiles are identified. Both also have consistent group-level findings implicating them in ADHD and are believed to be important sources of heterogeneity within the disorder (17, 29–32). Thus, this work directly relates to the RDoC and HiTOP frameworks, but also highlights how feature selection can be informed by the extensive prior work on DSM categorical diagnoses.

Cognitive profiles in ADHD

Using cognitive features (33–40) multiple distinct ADHD profiles are consistently identified. However, studies are inconsistent with regard to whether observed profiles reflect qualitative differences (i.e., profiles with specific strengths and weaknesses in different cognitive domains) or only quantitative differences (i.e., cognitive variation in ADHD follows a severity pattern similar to symptom-based subgroups) among individuals. The former would be more promising, while the latter is more likely to reflect a non-specific pattern of being “worse” across many domains.

Numerous computational methods are available for identifying profiles. In the case of cognitive profiling in ADHD, two methods have been applied most often: latent profile analysis (LPA; ^{41, 42}) and community detection (43, 44). LPA represents one example of a supervised classification approach that tests how well data fit a specified model. It has the advantage of allowing direct comparison of the relative fit of models with different numbers of groups. Cognitive studies employing LPA have tended to identify profiles fully (35, 36) or partially (38, 40) explained by severity regardless of specific cognitive input feature set.

Community detection is an unsupervised approach which makes few assumptions. It has the advantage of several internal validation metrics to evaluate likelihood of “groupness” within the data. Studies employing community detection with cognitive input features in children, adolescents, and adults have often found evidence for qualitatively distinct profiles in ADHD rather than severity profiles (33, 34, 37). Although method-dependence is neither unexpected nor inherently problematic (22), studies that directly address clustering algorithm as a source of between-study variance are needed. Supervised and unsupervised approaches may ultimately be most valuable when used together given their different strengths (45, 46).

Regarding specific profiles, studies have used a wide variety of tasks and input features, so there is no clear consensus on which cognitive profiles will be most relevant in ADHD. Arguably, the two most consistently identified groups include a profile characterized by executive function deficits (i.e., problems with working memory and inhibition, which often though not always cluster together) and a profile characterized by slow and/or variable reaction times (33, 34). At least some standardization around domains of interest and measures will be an important next step in clarifying the reproducibility of cognitive profiles across studies and samples. (We return to issues of data harmonization at the end of this review.)

Turning to the question of validity, it is somewhat surprising that the clinical validation of subgroups based on cognitive features is relatively weak given the prominence of cognitive theories of ADHD. In studies that have identified severity-based cognitive groups, differences on other clinical outcomes are sometimes found, however, those outcomes also generally follow a severity pattern (35, 36). That is, the more impaired cognitive groups are simply worse on everything (e.g., (35) found that the more impaired cognitive groups had lower IQ, worse academic performance, and more depression comorbidity than less impaired groups). In other studies, the cognitive subgroups fail to differ on either concurrent (33, 37, 40) or prospective outcome measures (38).

Findings suggest that a general distinction between ADHD with and without cognitive impairment (as has been made for many years) may be meaningfully related to severity of outcomes, but that greater granularity in cognitive profiles, even if present, does not incrementally contribute to understanding outcomes. Alternatively, the scope of outcomes considered to-date may be too limited. Most studies have focused on ADHD or comorbid disorder symptoms (but see (33) for an example of an extensive set of clinical, academic, and sociometric outcomes). It is also possible that standard cognitive performance metrics are too multi-factorially determined and use of computational parameters that distinguish different aspects of information processing would yield more informative profiles (47, 48). Finally, studies to-date may have been underpowered to detect between-profile effects given that even relatively large clinical samples (up to ~700 (35) but more often ~100 (33, 36–38, 40)) become underpowered quickly when multiple subgroups are present.

As noted, however, subgroups may have different relevance depending on study goals and groups may be important for elucidating relevant aspects of neurobiological heterogeneity. Rossi et al. (39) recently identified two severity-based cognitive subgroups within ADHD that differed in terms of DTI-measured fractional anisotropy. Findings are intriguing; however, it was unclear whether differences could be explained by symptom severity. This emphasis on incremental validity— demonstrating that groups contribute unique information that would not be detected using existing groupings or measures— is critical to moving subtyping literature into clinical translation.

Temperament/personality-related profiles in ADHD

A decades-long literature has proposed that emotional dysregulation should be included as a core element of ADHD (49). More recently, there has been interest in anger regulation as a core problem in ADHD that may also extend across diagnostic boundaries (49–51). Temperament traits in children (and the related personality traits in adults (52, 53)) provide one framework for integrating emotional response and regulation (54) into models of psychopathology generally (52, 55) and ADHD specifically (56). Computational efforts to identify subgroups based on temperament/personality features, while still developing, appear quite productive. Growing evidence suggests that profiles can be retrieved and yield incremental information over and above ADHD severity (57–61).

Although some differences between studies are seen, work in young children (59), school-age children (57, 58) and young adults (61, 62) converge on at least partially reproducible profiles. First, a minority consistently emerge that show normative emotional functioning—their problems are primarily in features that make up the core ADHD symptom domains. Second, a group consistently emerges characterized by high surgency/extraversion/sensation-seeking. Finally, a group emerges that is characterized by negative affect/neuroticism. Results hold across multiple clustering approaches (58).

Clinical validation is promising. The identification of a group with high negative affect converges with growing evidence from DSM-framed research of the importance of irritability and emotional lability as determinants of impairment in ADHD (51, 63–65). Evidence is reasonably strong that the emotionally-dysregulated profiles predict clinical outcomes better than ADHD symptom severity, baseline comorbidity, or impairment (57–59, 62). Our own work and that of others suggests that the high negative affect/irritability profile prospectively predicts worsening clinical outcomes (57, 58, 62), but in other cases high surgency/extraversion profiles have been the primary predictive feature (59). These differences may relate to the developmental period (studies identifying surgency have included younger samples than those finding negative affect/irritability as the predictive feature), but may also suggest that a broad distinction between ADHD with and without emotional dysregulation is more clinically meaningful than a valence distinction. Future studies characterizing and refining specific profile description and emphasizing biological validation are needed.

Integration across feature sets

Understanding how features within specific domains operate to differentiate groups is promising but does not capitalize fully on a person-oriented framework, which seeks to understand the individual as a functioning whole (66). Traits may interact and the meaning of differences in one trait may depend on interactions with other characteristics. A small number of studies have begun to examine these cross-domain interactions. Van Dijk et al (67) used cognitive features in a clustering analysis and then compared cognitive groups on personality dimensions. They found little evidence for personality differences between the cognitive groups, suggesting partially orthogonal dimensions. Other studies have included both cognitive and emotional tasks in grouping analyses (e.g. an executive function battery plus either a reward discounting/sensitivity (36, 37, 40) or an emotion recognition (33) task). Findings are inconsistent with regard to whether distinct groups with cognitive and emotional impairment are identified (33, 37) or whether impaired groups tend to be impaired across domains (36, 40).

Understanding cross-domain interactions is necessary for unifying cognitive and emotional accounts of ADHD. Different profiles may emerge when multiple domains are considered simultaneously as input features. However, the task of integration is complicated by problems such as method variance (related to differences in how cognitive and emotional control tend to be measured) that can swamp other effects. Additional work to identify the functionally significant relationships among cognitive and emotional features in ADHD is needed to help clarify which features are relevant for grouping analyses.

Feature reduction

Integration across domains should be useful, but as the number of features increases so does the number of idiosyncratic ways that these features can cluster within individuals (each person could theoretically have a unique profile). Clustering approaches assume that a select number of frequently emerging profiles will tend to occur (66). Clustering algorithms differ in their robustness to irrelevant and/or inter-correlated features and in the ideal ratio of features to sample size (41, 46). Given these limitations, selection of a small number of theoretically-driven input features may increase the likelihood of identifying clinically-meaningful, replicable groups that are easily interpreted and translated into clinical application. On the other hand, extensive a priori data reduction of input features rests on the assumption that it is possible to know the essential features of a group before the group is identified, which has proven untrue in other fields (1). It increases the possibility of missing novel, previously unrecognized patterns in the data that are etiologically or clinically informative.

In the case of both cognitive and temperament/personality features, the problem of feature reduction benefits from a well-established literature that has identified relevant traits and described the expected relationships between them. In contrast, sub-phenotyping work that emphasizes neuroimaging input features is in its relatively early days. Here it is less clear whether use of high-dimensional input features can yield informative groups or whether this will identify groups with small, non-meaningful differences. Perhaps due to this concern, the small number of neuroimaging sub-phenotyping studies in ADHD have generally applied a theory driven approach in which a small number of specific features are selected, such as fMRI measured activation in specific regions of interest (68, 69) or EEG-measured spectral power in pre-identified frequency bands (70). These promising early studies suggest that neurobiologically-based subgroups can be identified. For example, (69) found unique profiles using reward system connectivity maps as input features and validated groups based on impulsivity and reward task performance and (70) identified groups with different EEG frequency patterns who also differed in cognitive performance and comorbidity. However, reproducibility has yet to be demonstrated and mechanistic interpretation of groups (particularly for EEG profiles) and how they explain clinical functioning is not yet clear.

Moving forward, a key challenge is to balance exploratory analyses using a broad feature set with theory-driven approaches that emphasize a smaller number of features. Several recent advances facilitate this work, including development of canonical connectivity maps for fMRI and development of statistical approaches that offer step-wise consideration of individual and group-level neuroimaging data in the process of feature selection (71).

Who should be included in analyses?

Identifying the population of interest is also critical. One basic, yet still unanswered question in ADHD is whether and how sex should be considered in subgrouping analyses (and variation in gender identity, while potentially higher in ADHD than typically-developing groups (72), has not even been considered). At least one study has found that differences in cognitive liability may mediate increased ADHD risk in males (73). Yet several cognitive sub-profiling studies have not found significant differences in sex distribution between cognitive subgroups (33, 35, 37). Hartung & Lefler (74) offer recommendations for reporting information about sex (and gender identity) that provide a starting point for accumulating information.

A second question involves how existing diagnostic categories should be incorporated into studies of heterogeneity. A purely dimensional perspective emphasizes how cognitive and emotional functioning operate to convey risk across existing diagnostic boundaries (and across typical and atypical development). From this perspective, over-sampling for psychopathology without regard for specific diagnosis might present the optimal strategy (75). This approach can identify transdiagnostically-relevant features.

However, to the extent that different features are relevant for different populations, their inclusion in a single-grouping analysis will emphasize large between-population differences at the expense of within-population variation. A common problem is that a clustering analysis yields profiles that largely replicate the typically-developing/not typically-developing distinction. This is analogous to using an extreme groups design— it prevents discovery of new clusters within the extremes. Recruiting specific diagnostic subgroups and conducting sub-phenotyping separately in clinical and typically-developing groups can help emphasize within-group variation.

Several cognitive studies have demonstrated a step-wise approach to the issue of sample selection, and there is now consistent evidence that cognitive variability in ADHD is nested within similar heterogeneity in typical development (33, 34, 36, 37, 40, 76). Thus, many ADHD-related cognitive impairments reflect quantitative differences in the proportion of children with a particular profile, but not a qualitative departure from normal developmental patterns. In contrast, while there is undoubtedly large normative temperament variation (77, 78), in ADHD studies have either failed to find strong evidence for distinct subgroups in typically-developing samples (57) or support broad distinctions between ADHD and typically-developing individuals but without evidence of nested variation when all children are included in analyses as once (59, 61, 62). This may be related to exclusion of “messy” or “subthreshold” cases that artificially limits the range of ratings in the typically-developing samples.

The question of how to understand ADHD variation in relation to normative variation is directly relevant to clinical application. Qualitatively distinct deviations from development may be most mechanistically important, whereas quantitative shifts that are also present for many individuals in the typically-developing population may be more likely to be epiphenomenon (76). Alternatively, models that consider quantitative feature shifts as part of an additive model (e.g., cognitive impairment is only relevant if the child also has emotional impairment) or as features that matter only in specific environmental contexts may be informative.

How should studies address development?

The majority of work in ADHD has asked, “What features are relevant for understanding ADHD-related heterogeneity?” This question assumes a static model of psychopathology rather than recognizing dynamic developmental continuity and discontinuity (66, 79–81). A more refined question may be, “What features are relevant at what points in development?” One way to ask this question can be addressed is by examining the extent to which a) the same profiles are identified at multiple developmental phases and b) the same individuals remain in each group over time.

With at least moderate stability, groups may be useful for clinical prediction and, in fact, our own work suggests that assignment to an negative affect/irritable profile is predictive of increased risk for worsening clinical outcomes, even when this assignment is not fully stable over time (58). An approach that considers data at several distinct time points separately and then analyzes stability over time has the benefit of mirroring clinical practice. In many cases a clinician will have only a single time point of information on which to base their diagnostic decisions (although they may have retrospective reports, these are unfortunately often unreliable).

Another approach is to ask, “What trajectories are relevant?” and to integrate development into the classification algorithm (82–84). This has the benefit of directly corresponding to the dynamic developmental changes that are theorized to contribute to and maintain pathological states but relies on the assumption that measures carry similar meaning over broad developmental periods. Karalunas et al. (76) provide an example of a trajectory approach using cognitive features. Latent class growth models characterizing working memory development in ADHD from age 7–13 identified 3 trajectory classes: a cognitively normative group; a group with moderate, stable impairment; and a group with moderate impairment who recovered or caught up in working memory development by age 13. Improvement in working memory was related to improvement in inattention symptoms, whereas baseline levels of cognitive impairment were not, highlighting the importance of considering developmental change as part of predictive algorithms.

Summary and Translation to Clinical Practice

Currently, the field faces considerable controversy about whether the existing psychiatric syndromes should be scrapped and a new system built (75) or nosology should be revised in a gradual fashion (85), and in either case, how to do so. Here, we suggest that ADHD reflects a useful construct with some developmental continuity but that it requires refinement to explain the observed equifinality and multifinality. Our proposal is for a blended approach that integrates the vast literature on reliability and validity of DSM/ICD categories with emerging knowledge of neurobiological bases of dimensions of psychological functioning (7, 86, 87). We highlighted cognition and temperament/personality as two well-established domains that may be informative for future studies.

Several specific approaches may be useful as the field moves forward. In terms of sample selection, studies that recruit and include children with subthreshold symptoms or in diagnostic grey areas (e.g., lack of parent/teacher agreement) offer an opportunity to capitalize on ADHD’s trait-like aspects while still retaining the capability of mapping clinically-relevant cut points. Studies that test generalizability of features across disorders will also be needed. In addition, while ADHD-control comparisons may continue to be useful for identifying features that are relevant for many members of the ADHD group, it may ultimately prove fruitful to shift away from studies seeking to identify markers or mechanism for a putative discrete disease (“ADHD”) and towards those identifying markers and mechanisms for specific subgroups or clinical outcomes (88). While the presence of discrete taxa is possible, we emphasize that different observable features may cluster in ways that are informative without requiring the assumption of correspondence to a “true” or “natural kind.”

To facilitate mapping across diagnostic boundaries, as well as to address concerns related to low power for validation when sub-profiles are compared, large datasets such as the Adolescent Brain Cognitive Development cohort (89) or the Philadelphia Neurodevelopmental Cohort (90) (or similar large cohorts in other nations) may provide an additional resource for tests of heterogeneity. Large samples also offer opportunity for cross-validation (a method in which datasets are partitioned into separate testing and confirmation sets) to test of the reproducibility of results. In smaller datasets such cross-validation must rely on sub-optimal approaches, such as the leave-one-out procedure, which can inflate model performance (46). Large sample size almost invariably necessitates a methodological trade-off in terms of breadth and/or depth of measures. One useful strategy may be to be to use large but shallowly phenotyped datasets for discovery purposes and smaller, more deeply characterized cohorts for fuller understanding of functional meaning and clinical utility.

While capturing heterogeneity can benefit from integrating across different datasets, such approaches raise additional challenges. Lack of data harmonization on either input features or outcomes substantially limits conclusions in much of the literature we summarize here. Problems with lack of data harmonization apply broadly to all levels of data (e.g., beahvioral, imaging, genetic). For outcome measures, efforts are underway to create consensus lists for specific disorders that could facilitate translation of findings into clinical practice and support efforts at replication (91) and cross-validation, but there are not yet any consensus outcomes in ADHD (92).

Finally, while statistical clustering approaches are increasingly easily implemented in research contexts, how these should be translated into clinical practice remains unclear. One possibility is that feature identification using clustering algorithms can be combined with decision-making algorithms that define optimal clinical cutoff scores (e.g., ROC approaches (93)), which would facilitate clinical translation. Ultimately, one could envision an updating diagnostic algorithm in which relevant input features are assessed at repeated intervals (e.g., at annual well-child pediatrician visits or over the course of specialty psychological or psychiatric care) and used to update risk scores or diagnosis over the course of a child’s development.