A Hierarchical Taxonomy of Psychopathology (HiTOP) Primer for Mental Health Researchers

Abstract

Mental health research is at an important crossroads as the field seeks more reliable and valid phenotypes to study. Dimensional approaches to quantifying mental illness operate outside the confines of traditional categorical diagnoses, and they are gaining traction as a way to advance research on the causes and consequences of mental illness. The Hierarchical Taxonomy of Psychopathology (HiTOP) is a leading dimensional research paradigm that synthesizes decades of data on the major dimensions of psychological disorders. In this article, we demonstrate how to use the HiTOP model to formulate and test research questions through a series of tutorials. To boost accessibility, data and annotated code for each tutorial are included at OSF (https://osf.io/8myzw). After presenting the tutorials, we outline how investigators can use these ideas and tools to generate new insights in their own substantive research programs.

Mental health research is at a crossroads. Historically, the field has relied on categorical diagnoses as the basic units of analysis, but there is mounting concern that diagnostic categories, as currently defined, are limiting much needed insights into disorder etiology, treatment, and prevention (e.g., Gordon & Redish, 2016). This criticism has put the focus on dimensional perspectives that prioritize phenotypes that cut across traditional diagnostic boundaries (Kotov et al., 2017; Kozak & Cuthbert, 2016).

The Hierarchical Taxonomy of Psychopathology (HiTOP) is an empirically derived model of the major dimensions of mental illness. It represents an alternative research paradigm that, as we argue below, has multiple advantages relative to categorical rubrics. Several publications have described HiTOP’s conceptual and empirical foundations (Kotov et al., 2017; Krueger et al., 2018), but as yet, there are no resources that explain how mental health researchers can design and interpret results from studies framed by the HiTOP system.¹ In this article, we provide tutorials and empirical examples for prototypical research questions that can be formulated and tested in the HiTOP framework. We also consider obstacles and limitations to this approach as well as ongoing methodological developments. Our aim is to make it clearer how the HiTOP model can be applied in mental health research across a variety of substantive areas and to give an overview of the relevant statistical methods. We include the data and annotated code for each of our examples in the Supplemental Material available online and cite additional references for more detailed technical information on the statistical methods throughout.

HiTOP

To put the tutorials in context, we begin with a brief review of the motivation behind and makeup of the HiTOP model of psychopathology. We refer readers to prior consortium publications for the full account of HiTOP’s dimensional structure (Kotov et al., 2017, 2020; Krueger et al., 2018; Krueger, Hobbs, et al., in press; Watson et al., in press) and its implications for substantive research on the origins, development, consequences, and treatment of psychopathology (Conway et al., 2019; Latzman et al., 2020; Ruggero et al., 2019; Waszczuk et al., 2020).

The HiTOP consortium formulated a new nosological system in response to problems with traditional categorical nosologies such as the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association [APA], 2013) and the International Classification of Diseases (ICD; World Health Organization, 2018). Issues such as rampant comorbidity, excessive within-diagnoses heterogeneity, and unreliability put significant constraints on the utility of categorical diagnoses for assessment and applied research (Clark et al., 2017; Widiger & Samuel, 2005).

Figure 1 illustrates the current working model for HiTOP. The upper part of the model is oriented around spectrum constructs: internalizing, antagonistic externalizing, disinhibited externalizing, detachment, thought disorder, and somatoform. The first five spectra are particularly well validated and are akin to dimensions described in consensus models of both adaptive and maladaptive personality (Markon et al., 2005), whereas somatoform has more modest empirical backing (e.g., Michelini et al., 2019). The shared characteristics of the six HiTOP spectra are captured in a superspectrum, the general factor of psychopathology (dubbed the p-factor by Caspi et al., 2014), which is presumed to pervade all mental disorder symptoms (albeit some more than others; Lahey et al., 2012). Moving down the hierarchy, the six broad dimensions divide into more homogeneous dimensions that are labeled subfactors. For instance, internalizing is subdivided into fear and distress, among others. These subfactors are, in turn, composed of lower order syndromes, which are intended to be empirically derived symptom constellations. In practice, our current understanding of the lower levels of the hierarchy is largely bound to DSM constructs, but this area of the literature is evolving rapidly, and HiTOP’s empirically derived syndromes are not expected to directly correspond to the syndrome concepts in the DSM (see e.g., Waszczuk, Kotov, et al., 2017). At the base of the hierarchy, symptom components and maladaptive personality traits represent the basic units of psychological disorders. We use terms such as superordinate and higher order to refer to broader dimensions in the model and terms such as subordinate and lower order for narrower dimensions.

Fig. 1.

Hierarchical Taxonomy of Psychopathology (HiTOP) consortium model. Constructs higher in the figure are broader and more general, whereas constructs lower in the figure are narrower and more specific. As explained in the main text, DSM diagnoses are not part of the HiTOP model. This departure from DSM’s approach to classification is represented in the figure by the solid line separating HiTOP from DSM concepts. Yet HiTOP is based on the same signs and symptoms that constitute DSM diagnoses, and the color scheme symbolizes where the primary features of DSM disorders are likely to fall in the HiTOP structure. There are too many symptom components and maladaptive traits—the most basic units of mental disorder represented at the base of the model—to depict here; see Figure 3 in Kotov et al. (2017) for a list. OCD = obsessive-compulsive disorder; ADHD = attention-deficit/hyperactivity disorder.

HiTOP as a Research Framework

The HiTOP model has a descriptive function; it delineates the dimensional phenotypes that characterize mental health conditions. At the same time, the model has clear implications for how psychological theories are formulated and tested and for how research is designed and interpreted. HiTOP dimensions change not only the way one asks questions about the causes, correlates, and consequences of mental illness but also the very questions that one asks.

Dimensions

The HiTOP model reflects a paradigmatic break from traditional classification schemes, which assume that someone either has or does not have a condition, with no middle ground. Instead, the HiTOP model conceptualizes psychopathology in terms of continuously distributed dimensions, an approach that is in sync with a great deal of research evidence (Haslam et al., 2020). This has implications for the sort of assessment information researchers use to test hypotheses about mental disorders. Specifically, the focus is on dimensional representations of mental health problems as measured directly or abstracted via latent variable modeling from observable signs and self-reported symptoms of mental disorder.

Hierarchy

HiTOP assumes that the building blocks of psychopathology are related to one another hierarchically and parallel the configuration of other individual difference domains, such as personality, intelligence, and executive functioning (e.g., Deary, 2012; Markon et al., 2005). This conceptualization means that mental health conditions can be construed at various levels of breadth. Consider social anxiety disorder: Using Figure 1 (and for another depiction, see Kotov et al., 2017), researchers can operationally define social anxiety in terms of (a) fine-grain symptom components, such as situational avoidance; (b) a cluster of symptom components that reflect common manifestations of social anxiety; (c) fear responses it shares with panic and other phobic disorders; (d) emotional disturbances common to all internalizing problems; and at the broadest level, (e) general distress and impairment, reflected in the p-factor, which saturates almost all psychopathology. The different levels are neither incommensurate nor mutually exclusive; they are equally valid representations of social anxiety in the HiTOP system, albeit differing in specificity, just as, for example, in biological classification, animals can have various taxonomic ranks (e.g., for dogs: kingdom = animalia, phylum = chordata, class = mammalia).

This perspective enables researchers to pinpoint the “level of analysis” that conveys the most predictive information for a given research context. Stated differently, they can identify what part or parts of a mental health condition are associated with outcomes of interest. This versatility opens up entirely new research questions and hypotheses. For example, is the link between posttraumatic stress disorder and romantic relationship difficulties best explained by its associations with (a) fairly specific features, such as hyperarousal and hostility (symptom component level); (b) many or all of its component parts (syndrome level); or (c) emotional dysfunction common to all anxiety and depressive conditions (spectrum level)? Research framed by DSM categories cannot empirically compare these hypotheses because the categories collapse individual differences across these various levels of a dimensional hierarchy into a single level of analysis (i.e., the binary syndrome). Research informed by HiTOP can shed new light on the nature of connections between psychological symptoms and their hypothesized causes and consequences.

Measurement

What does HiTOP measurement look like, practically speaking? How do hierarchically arrayed dimensions make their way into one’s data set? These are natural questions because many of the terms in the HiTOP model are less familiar than the DSM-5 (APA, 2013) labels to many researchers. Keep in mind, however, that HiTOP constructs comprise the same clinical phenomena that researchers and clinicians work with routinely—they are simply reorganized according to structural empirical evidence.

HiTOP constructs can be (a) accessed directly with existing measurement instruments and also (b) captured in the shared variance among scores on traditional measures using latent variable modeling. We now discuss each of these options in turn.

Direct assessment.

Many HiTOP constructs can be assessed with surveys and interviews that were developed with the hierarchical structure of psychopathology in mind. One advantage of this option is that these symptom scales were created after verifying certain psychometric desiderata, such as reliability and discriminant validity, in normative samples. These features are not assured when working with symptom-level data derived from assessments oriented around categorical diagnoses.

Several such empirically derived measures already exist. For instance, the Inventory of Depressive and Anxiety Symptoms (IDAS) was designed to measure the core components of the internalizing spectrum (Watson et al., 2007). Framed around the influential tripartite theory of anxiety and depression (and its successors; Clark & Watson, 1991; Mineka et al., 1998; Watson, 2009), the IDAS features empirically derived subscales that tap into homogeneous components of emotional problems, such as panic, insomnia, and social anxiety. A revised IDAS, the IDAS-II, includes additional scales related to posttraumatic stress, obsessive compulsive, and bipolar disorders (Watson et al., 2012). The Externalizing Spectrum Inventory is the analogue for the externalizing domain (Krueger et al., 2007). It contains 23 narrow-bandwidth facet scales (e.g., alcohol use, theft, fraud) that cohere empirically into three broader dimensions (i.e., general disinhibition, callous aggression, and substance abuse). The Schedule for Nonadaptive and Adaptive Personality (Clark et al., 2014), Five Factor Model personality disorder scales (Widiger et al., 2012), and 220- and 100-item versions of the Personality Inventory for DSM-5 (Krueger et al., 2012; Maples et al., 2015) all include various lower order scales that capture maladaptive personality traits characterizing clinical and personality disorders as well as broader trait concepts that reflect the major dimensions of Big Three and Big Five models of personality (e.g., Markon et al., 2005). In addition, some broadband clinical inventories—for example, MMPI-2-RF (Ben-Porath & Tellegen, 2008), the Personality Assessment Inventory (Morey, 1991), SPECTRA: Indices of Psychopathology (Blais & Sinclair, 2018)—have scales that structurally adhere broadly to contemporary psychopathology frameworks (e.g., Lee et al., 2017).

Finally, the HiTOP consortium is now in the process of developing a measure of the higher and lower order dimensions in six HiTOP domains: internalizing, detachment, thought disorder, somatoform pathology, and both disinhibited and antagonistic forms of externalizing psychopathology (Simms et al., 2021). After an omnibus self-report questionnaire is finalized, there are plans also to create informant-report and interview measures.

Latent variable modeling.

Factor analysis and structural equation modeling are families of statistical approaches that naturally align with hierarchical dimensional models. Latent variables can be used to model the higher order dimensions in the HiTOP framework (e.g., subfactors, spectra, and superspectra) by capturing the shared variance among psychopathology constructs in a data set. For example, in a study that assesses mental disorder diagnoses using a structured clinical interview, an internalizing latent variable could be modeled according to the patterns of comorbidity (i.e., covariance) among major depression, generalized anxiety, social anxiety, and panic disorders. Likewise, in a study that assesses a variety of self-reported symptoms of psychopathology using the Strengths and Difficulties Questionnaire (Goodman, 2001) or Youth Self-Report (Achenbach, 2009), broader internalizing and externalizing spectra can be modeled as correlated latent variables according to the overlap among the corresponding scales. For interested readers who have not used these techniques before, there are a number of gentle introductions to factor analysis and structural equation modeling available that cover the assumptions, specification, and evaluation of latent variable models (e.g., Brown, 2015; Kline, 2015).

More complex latent variable models can also be used to operationalize multiple levels of the HiTOP framework simultaneously. For example, the literature on the nature and correlates of general psychopathology (or p-factor) suggests that this dimension embodies individuals’ propensity to all forms of psychopathology (e.g., Caspi & Moffitt, 2018). As mentioned earlier, it rests at the apex of the HiTOP model and has been hypothesized to capture the nonspecific etiological factors that increase risk for all symptom domains (Lahey et al., 2017).

Bifactor modeling is a popular approach for representing the general factor (e.g., Caspi et al., 2014; Lahey et al., 2012). In these latent variable models, all of the observed variables—for example, categorical diagnoses or continuous scales of different symptom domains—load on a single general factor that captures their shared variance (general psychopathology), and the remaining variance is partitioned into uncorrelated, narrower specific factors (e.g., internalizing, externalizing, and thought disorder). Multiple limitations of the bifactor model have been highlighted in recent empirical and simulation research (e.g., Greene et al., 2019; Watts et al., 2019), which had led to an increasing number of studies using a higher order model in which a second-order latent variable (general psychopathology) is modeled atop several first-order latent variables (internalizing, externalizing, and thought disorder) to capture the correlations among them (see Tutorial 2).

Although covering the relative strengths and weaknesses of these approaches is beyond the scope of this article, we direct interested readers to a recent study that comprehensively compared the most popular models used in research on the structure of psychopathology (Forbes et al., 2021; see also Bornovalova et al., 2020). Comparing the reliability and validity of the latent variable models commonly used in research using a HiTOP framework, Forbes et al. (2021) found that the general factors of psychopathology in the bifactor and higher order models were both robust and performed similarly well. The results of the study boiled down to three core recommendations to researchers using these models to understand the structure and correlates of HiTOP constructs: (a) Look beyond traditional model fit indices to choose which model fits “best” for the data and research question, (b) examine the reliability of latent variables directly, and (c) be cautious when isolating and interpreting the unique effects of individual latent variables (see further discussion in Tutorial 1 and Note 3).

Common research designs

There are several ways that the HiTOP system tends to be applied in substantive research (for a review, see Conway et al., 2019). First, investigators have used latent variable modeling techniques to establish the number and nature of latent dimensions relevant to a psychopathology domain (e.g., Forbes et al., 2020; Michelini et al., 2019). Often called structural modeling, this line of inquiry resolves the dimensional phenotypes that might serve as research or assessment targets. Thus, we can consider it as a precursor to tests of psychopathology’s association with putative causes and outcomes.

Second, researchers study how the heterogeneous symptom components of a diagnosis (or small cluster of diagnoses) differentially relate to a given outcome. This approach elucidates the fine-grain clinical problems that contribute to an outcome of interest. Consider how major depression can be decomposed into affective, vegetative, and cognitive symptom components. Researchers can simultaneously evaluate the predictive validity of these dimensions in relation to clinically important outcomes (e.g., emergency room visits).

Third, investigators compare the criterion validity of dimensions at varying levels of breadth. The psychopathology phenotypes in these studies span multiple levels of the HiTOP hierarchy. The objective is to determine whether one level of the dimensional hierarchy (e.g., spectrum) compared with another (e.g., symptom component) better predicts important outcomes.

Tutorials

In this section, we present three tutorials for mental health researchers. Tutorial 1 illustrates how to partition a broad psychopathology phenotype into relatively narrow individual differences and, in turn, how to compare their prediction of an outcome of interest. Tutorial 2 derives a hierarchy of dimensions of emotional disorders from a battery of symptom-level data and then examines phenotypes across multiple levels in relation to romantic relationship quality. Tutorial 3 covers how to integrate higher order dimensions of mental disorder into developmental psychopathology designs.

Data, codebooks, and analysis code for all three tutorials covered in this article are available at OSF (https://osf.io/8myzw). We supply annotated code written for both the R (R Core Team, 2021) and Mplus (Version 8; Muthén & Muthén, 2017) software platforms.

Tutorial 1: comparing specific symptom components’ effects

Earlier we alluded to the fact that diagnostic heterogeneity complicates mental health research framed around binary disorders. When diverse clinical problems can contribute to a diagnosis, two people with the same condition can exhibit very different symptom profiles (e.g., Fried & Nesse, 2015). This within-diagnoses variability is problematic for applied researchers because it is difficult to know which components of a disorder explain its correlation with a particular risk factor or outcome.

The HiTOP framework allows more control over the issue of heterogeneity because it includes specific dimensions. Researchers working from a HiTOP perspective can focus on homogeneous dimensions at lower levels of the hierarchy if they want to pinpoint the specific processes that connect psychopathology and outcomes of interest. This degree of resolution is impossible with almost all DSM and ICD diagnoses, which combine constellations of loosely related symptoms and traits into individual diagnostic constructs (Smith et al., 2009).

Background.

In this tutorial, we used the IDAS, mentioned earlier, to examine the fundamental components of the internalizing domain as they relate to distress tolerance. Distress tolerance can be defined as the ability to maintain goal-directed activity in the face of discomfort, such as pain, traumatic memories, or obsessive thoughts (Leyro et al., 2010). Like other stress-reactivity outcomes, distress tolerance is empirically related to an array of internalizing conditions, including major depression, generalized anxiety disorder, and social phobia (e.g., Macatee et al., 2015). We analyzed the data without a priori hypotheses regarding the relative effect sizes of specific symptom dimensions, although we generally expected that greater severity of internalizing symptoms would be correlated with poorer distress tolerance.

Measures.

In this example, we used a data set of 145 university students who completed the IDAS and a self-report measure of distress tolerance, described below. The IDAS assesses 10 specific symptom dimensions from the internalizing domain: suicidality, lassitude, insomnia, appetite loss, appetite gain, ill temper, social anxiety, panic, traumatic intrusions, and well-being (see the bottom tier of Figure 1). It also features two broad scales that tap into the general features of internalizing problems: general depression and dysphoria. For simplicity, we omitted general depression from our analysis because it is highly correlated (r = .95) with dysphoria in this sample and, unlike dysphoria, contains items that overlap with the 10 lower order scales. Prior research has confirmed that the IDAS has favorable psychometric properties in diverse samples (e.g., Nelson et al., 2018). The 10 specific scales are reasonably but not excessively intercorrelated, and in factor analyses, they define a single underlying dimension (Watson et al., 2007).

Distress tolerance was assessed using the Distress Tolerance Scale (Simons & Gaher, 2005). This 15-item self-report inventory provides an overall index of how effectively people feel that they manage upsetting emotional states.

Results.

For descriptive statistics, see Table S1 in the Supplemental Material. The sample means for all IDAS scales closely parallel those previously reported in university student samples (Stasik-O’Brien et al., 2019). Correlations among the 10 lower order scales range from .02 to .67 (median r = .33; after reversing the sign of correlations between well-being and other scales).

To examine the effects for each IDAS dimension in isolation, we performed a series of simple (i.e., single-predictor) regressions of distress tolerance on IDAS scales. Table 1 shows that the standardized effects ranged from −0.60 to 0.39, and all were statistically significant at the .05 α level. The effect for dysphoria was largest by a substantial margin (standardized effect = −0.60), whereas ill temper, insomnia, and appetite gain (range = −0.20 to −0.27) had the most modest associations with self-reported distress tolerance. We next performed a multiple regression that included all 10 lower order scales as predictors—but omitted dysphoria to avoid collinearity problems—to examine each effect holding the others constant. Most effect sizes dropped appreciably in this analysis, consistent with the idea that all subscales are indicators of the same broader construct. None exceeded |0.20|, and all were smaller than |0.15| except panic and well-being, an index of positive affectivity at one pole and anhedonia at the other.² This general attenuation of effects suggests that there is redundancy among these components of internalizing problems and that most of their external validity may be accounted for by the core disposition (internalizing) that they share.

Table 1.

Regression Results for Distress Tolerance Outcomes for Tutorial 1

	Distress Tolerance Scale
	Simple regression				Multiple regression
Predictor	b	SE	p	β	b	SE	p	β
Dysphoria	−7.38	0.83	.000	−0.60	—	—	—	—
Lassitude	−5.11	0.94	.000	−0.41	−1.06	1.12	.344	−0.08
Insomnia	−3.10	1.00	.002	−0.25	−0.85	0.96	.378	−0.07
Appetite loss	−4.53	0.96	.000	−0.37	−0.62	1.14	.586	−0.05
Appetite gain	−2.46	1.01	.016	−0.20	−1.46	0.93	.117	−0.12
Well-being	4.82	0.95	.000	0.39	2.21	1.05	.037	0.18
Ill temper	−3.31	1.00	.001	−0.27	−1.41	0.94	.135	−0.11
Traumatic intrusions	−5.01	0.94	.000	−0.41	−0.48	1.26	.700	−0.04
Social anxiety	−4.79	0.95	.000	−0.39	−1.04	1.10	.345	−0.08
Panic	−5.41	0.93	.000	−0.44	−2.15	1.35	.114	−0.17
Suicidality	−4.21	0.97	.000	−0.34	−1.18	1.03	.253	−0.09

Note: All predictors were standardized before analysis.

Interpretation.

We conclude that many specific aspects of the internalizing domain are meaningfully related to self-reported distress tolerance. The largest effect was observed for the dysphoria scale, which was empirically derived to reflect the higher order internalizing dimension (Watson et al., 2007). Effects for the lower order symptom components were smaller, and there was some variability among them. Traumatic intrusions, lassitude, and panic symptoms exhibited the largest correlations with distress tolerance, whereas ill temper, insomnia, and appetite gain had the smallest. There was some evidence from the multiple regression that well-being had an independent association with the outcome, whereas other dimensions had very small unique effects.

For published projects that follow this general approach, see Waszczuk, Zimmerman, et al. (2017), Watson et al. (2013), and Watson et al. (2007).

Tutorial 2: criterion validity of higher order dimensions

We have emphasized that the HiTOP model organizes clinical phenotypes hierarchically. In this rubric, no single level optimally represents psychopathology. Broader dimensions are bound to be ideal for some research contexts and narrower dimensions for others; this tension is akin to the bandwidth-fidelity dilemma in personality psychology (e.g., Cronbach & Gleser, 1957). We expect that without the benefit of strong theory or prior evidence, researchers will find it difficult to intuit the level or levels of analysis that will have the largest empirical association with some risk factor or outcome. Therefore, we argue that researchers ought to test empirically how external correlates (e.g., etiological markers) map onto psychopathology at various levels of the hierarchy.

Suppose an investigator theorizes that depression predicts loafing in the workplace. A conventional way to test this idea in observational research is to estimate the correlation between loafing and major depressive disorder symptoms. We know, however, that syndromes such as major depression reflect a mixture of both broad (e.g., internalizing, distress) and specific (e.g., anhedonia, hostility) individual difference factors. Any of these might be most proximally linked to loafing. The sizes of these influences are impossible to know (and compare) until the researcher examines them directly.

Furthermore, the literature on these types of research questions often develops in disorder-specific silos. So whereas one investigator may have found an association between loafing and major depression symptoms, another may have found an association between loafing and generalized anxiety disorder (or panic, social anxiety, etc.). Understanding the extent to which these associations are specific to one construct over another compared with the extent to which they can be parsimoniously accounted for by higher order dimensions can have important implications for developing appropriately targeted interventions (Forbes et al., 2019).

In this tutorial, we examined the emotional correlates of healthy romantic relationships. We first established that a hierarchical, dimensional model makes sense of the pattern of covariation among anxiety and depression symptom scales. We then tested associations between relationship functioning and dimensional phenotypes across three levels of a hierarchical model of anxiety and depression, differentiating the specificity and generality of each association. This approach allowed us to evaluate the extent to which each construct in the hierarchical model is associated with the outcome of interest (i.e., a total effect) and then to disentangle the portion of that relationship that is unique to the construct of interest (i.e., a direct effect) as opposed to being accounted for by higher-order factors that capture variance shared with other constructs in the model (i.e., an indirect effect). This method can thus transcend multiple disorder-specific literatures and substantially advance our understanding of the hierarchical structure of the risks and outcomes of psychopathology.

Background.

People in satisfying romantic relationships report fewer emotional problems. Research across developmental, social/personality, and clinical fields has confirmed that the quality of romantic relationships is inversely associated with dysphoria, panic, worry, social anxiety, and the primary features of several other emotional disorders (e.g., South, 2021). This nonspecificity could reflect an association of relationship quality with broad psychopathology dimensions—such as fear, distress, internalizing, and even the general factor of psychopathology—that are manifested in various diagnosable emotional problems (e.g., South & Krueger, 2008). On the other hand, it could reflect multiple independent associations of relationship quality with specific emotional disorder symptoms or syndromes (e.g., panic, social anxiety).

In this example, we examined relationship quality in association with HiTOP syndrome-level, subfactorlevel, and spectrum-level constructs. We determined the degree to which relationship quality’s observed associations with the various syndromes were explained (i.e., statistically mediated) by higher order dimensions. The data set for this tutorial comes from a larger survey of romantic experiences and mental health in an Australian community sample (see Wong & Forbes, 2020). We selected a subset of respondents (N = 725) who reported that they currently were in a romantic relationship.

Measures.

Emotional problems were assessed using popular surveys for depression, generalized anxiety disorder, social phobia, panic disorder, and obsessive compulsive disorder. These were the Patient Health Questionnaire (Kroenke et al., 2001), Brief Measure for Assessing Generalized Anxiety Disorder (Spitzer et al., 2006), Social Phobia Inventory (Connor et al., 2000), Panic Disorder Severity Scale Self-Report (Newman et al., 2006), and Obsessive-Compulsive Inventory–Revised (Foa et al., 2002), respectively. All measures provided a continuous index of symptom severity. Relationship quality was assessed using the six-item Modified Quality of Marriage Index (QMI; Nazarinia et al., 2009). Despite its name, the QMI is often used to assess the quality of nonmarital romantic relationships, as we did in this example. Respondents rated their agreement with a series of statements about relationship satisfaction over the past 2 weeks (e.g., “I really feel like part of a team with my partner”) on a 6-point Likert scale. We used the QMI total score (i.e., mean of all item responses) as our index of relationship quality in this analysis.

Results.

The first part of the analysis involved creating latent dimensions from the set of observed symptom measures using confirmatory factor analysis (CFA). In factor-analytic language, this step generates the measurement model. Following the structure represented in Figure 1, we hypothesized that a fear dimension would account for the correlations among social phobia, panic, and obsessive compulsive symptoms, whereas a distress dimension would account for the correlation between depression and generalized anxiety. For the scale intercorrelations, see Table S3 in the Supplemental Material. The fear and distress dimensions are called first-order factors because they are just one step removed from the observed variables. Figure 1 also led us to expect that the fear and distress factors would be substantially correlated, reflecting a shared superordinate internalizing factor, which here represents a second-order factor.³

This hypothesized model fit the data well. Indices of model fit suggested that the internalizing, distress, and fear factors adequately represented the correlations among symptom scales: χ²(4) = 9.23, p = .06; comparative fit index (CFI) = .99; root mean square error of approximation (RMSEA) = 0.042; standardized root mean square residual (SRMR) = .015 (for details on how to evaluate factor models, see Brown, 2015, Chapter 5). The standardized factor loadings on the first-order factors ranged from 0.73 to 0.88, and the loadings of fear and distress on the internalizing factor were both 0.93. It was necessary to constrain the two (unstandardized) second-order loadings to be equivalent for model identification purposes (for information on identification, see Brown, 2015, Chapter 3).

The second step in the analysis involved regressing psychopathology variables on relationship quality. To answer our question about potential etiological pathways across multiple levels of the dimensional hierarchy, we estimated total, direct, and indirect effects of relationship quality on psychopathology outcomes (i.e., symptoms of depression, generalized anxiety disorder, panic, social anxiety, and obsessive compulsive disorder; distress and fear; and internalizing) one at a time. Figure 2 diagrams the distinction between direct and indirect effects in the context of our factor model for this data set. The total effect represents the bivariate (or “zero-order”) association between relationship quality and a psychopathology dimension. This effect does not adjust for shared variance between the psychopathology variable (e.g., panic symptoms) and the other variables in the model. Next, the direct effect represents the extent to which relationship quality and a psychopathology dimension are related above and beyond any higher order constructs (e.g., fear and internalizing). Thus, it gives the proportion of the total effect that is not statistically mediated by higher order psychopathology dimensions. For example, the direct effect of relationship quality on depression occurs independently of the internalizing and distress dimensions; it reflects the association between relationship quality and the part of depression that is independent of other emotional problems.

Fig. 2.

Direct and indirect effects of relationship quality on emotional-disorder symptoms. This diagram illustrates the regression of the internalizing factor and depression symptoms on relationship quality. The indirect effect of relationship quality on depression via internalizing is represented by the path a > b > c. In contrast, the direct effect of relationship quality on depression is represented by path d. These path names mirror the labeling scheme we used in our data analysis code in the Supplemental Material available online. For clarity of presentation, here we omit the indirect pathway from relationship quality to depression via the distress factor. The total effect of relationship quality on depression is the sum of the direct effect (relationship quality > depression), the indirect effect via internalizing (relationship quality > internalizing > distress > depression), and the indirect effect via distress (relationship quality > distress > depression). dep = depression; gad = generalized anxiety; pan = panic; soc = social phobia; obs = obsessions and compulsions. Rectangles represent observed variables, and ellipses represent latent factors. Short, single-headed arrows pointing to distress, fear, and the factor indicators reflect error variances. Single-headed arrows from internalizing to distress and fear represent second-order factor loadings; single-headed arrows from distress and fear to factor indicators are first-order factor loadings.

In contrast, the indirect effect reflects the proportion of the association between relationship quality and psychopathology that is mediated by higher order constructs.⁴ Note that there is no indirect effect of relationship quality on internalizing in this example because internalizing is at the apex of the hierarchy in our model. For more conceptual and quantitative information on total, direct, and indirect effects in path analysis (the foundation of factor analysis and structural equation modeling), we recommend textbooks on applied multiple regression/correlation analysis (e.g., Pedhazur, 1997, Chapter 18).⁵

The first five rows of Table 2 present the total effects of relationship quality on the symptom measures. All coefficients were negative (standardized effect range = −0.24 to −0.37) and indicated that, as expected, healthy relationships were linked with fewer emotional problems. The total effects for fear, distress, and internalizing were −0.32, −0.36, and −0.37, respectively. Again, we can conceptualize these effects as the bivariate correlations between relationship quality and psychopathology constructs.

Table 2.

Comparing the Total and Direct Effects of Relationship Quality on Symptom Dimensions at Different Levels of the Psychopathology Hierarchy for Tutorial 2

Psychopathology outcome	Total effect	Direct effect	Percentage accounted for by higher order factor(s)a
Depression	−0.37 (0.04)	−0.13 (0.03)	65%
Generalized anxiety	−0.26 (0.04)	0.14 (0.04)	100%
Social anxiety	−0.24 (0.04)	0.01 (0.04)	100%
Panic	−0.28 (0.07)	−0.07 (0.07)	77%
Obsessions and compulsions	−0.24 (0.04)	0.02 (0.04)	100%
Distress	−0.36 (0.04)	−0.05 (0.04)	86%
Fear	−0.32 (0.05)	0.05 (0.04)	100%
Internalizing	−0.37 (0.04)	−0.37 (0.04)b	—

Note: All effects are fully standardized estimates from Mplus (Version 8.0; Muthén & Muthén, 2017). Values in parentheses are standard errors. Boldface type indicates statistically significant effects (p < .001). Total effects represent the expected change in psychopathology outcomes (in standard deviation units) per standard deviation increase in relationship quality. Direct effects represent the fully standardized effect of relationship quality on psychopathology outcomes above and beyond any higher order dimensions (path d in Fig. 3). The indirect effect can be computed by subtracting the direct effect from the total effect. See the main text for more detail.

^aSome direct effects were opposite in sign from the corresponding total effects. Consider generalized anxiety, for example. Its “zero-order” association with relationship quality was −0.26 (total effect), whereas the partial association (after partialing out variance generalized anxiety shared with other symptom dimensions; direct effect) was 0.14. This pattern implies that relationship quality is inversely related to generalized anxiety on a bivariate level but positively related to the part of generalized anxiety that does not overlap with other symptom dimensions (i.e., suppression; Watson et al., 2013). When direct effects had opposite signs to the relevant total effects, we indicated that the percentage of the total effect accounted for by higher order factors (final column in the table) was 100%. In other words, these positive direct effects did not augment the overall (total) negative effect of relationship quality on the psychopathology outcome.

^bThe total and direct effects for internalizing are equivalent because internalizing is the highest level of the psychopathology hierarchy here. There are no broader (i.e., superordinate) dimensions through which relationship quality could have (indirectly) influenced internalizing. Because there are no superordinate (i.e., higher order) factors relative to internalizing in the model, the final column in this table is left blank.

Table 2 also lists the direct effects of relationship quality on the observed symptom outcomes. As noted above, the dependent variable in these regressions is the part of symptom scales that is not shared with other outcomes. This unshared, or unique, variation often is called the residual variance in factor analysis and structural equation modeling more generally. Likewise, the regressions of fear and distress on relationship quality reflect associations that are adjusted for the overlap each first-order factor has with the second-order internalizing factor. That is, the outcomes in these first-order factor regressions represent pathology that fear does not share with distress (and vice versa).

All direct effects for syndrome- and subfactor-level constructs were small compared with the total effects. Three out of five syndrome-level direct effects were approximately 0. However, the negative direct effect on depression was statistically significant. This result indicates that relationship health was negatively associated with the part of depression that remained after adjusting for the overlap depression has with other emotional problems. Stated differently, there is some, albeit modest, connection between relationship quality and depression even after taking into account what is shared between depression and internalizing more broadly. There was also a statistically significant direct effect on generalized anxiety symptoms. This effect was positive, which suggests that relationship quality was positively associated with the part of generalized anxiety that remained after adjusting for the higher order dimensions. Note that this positive direct effect with generalized anxiety symptoms is a suppressor effect (i.e., the partialed effect reversed in sign compared with the zero-order effect; see Watson et al., 2013); this positive direct effect is a corollary of the negative direct effect found for depression given that there were only two indicators for the distress latent variable here.⁶

The final column in Table 2 presents the percentage of the total effect for each outcome that was accounted for by factors higher in the hierarchy. For symptom scales, this column indicates the total effect accounted for by distress, fear, and internalizing factors. For distress and fear factors, this column indicates the total effect accounted for by internalizing. This percentage, in turn, represents the magnitude of the indirect effect, and it is computed by subtracting the direct effect from the total effect and dividing the difference by the total effect. These values demonstrate that for all symptom scales—with the exception of depression—the indirect effect (of relationship quality via the fear, distress, and internalizing factors) explained the large majority of the bivariate association with relationship quality. This same pattern was evident for the first-order factors. Regarding depression, roughly one third of the total effect was not mediated by the factors (i.e., was unique to depression).

Interpretation.

According to our analysis, much of the association between relationship quality and emotional problems operated at a higher order level. Put another way, after adjusting for higher order dimensions, there were relatively small and inconsistent associations between relationship quality and the DSM syndromes that clinicians tend to diagnose in routine practice.

This pattern of effects highlights a key methodological issue. The modal study of the correlation between close relationships and mental health involves a single disorder entity (see Egan & Smith, in press), such as social anxiety. But here we see that any associations of relationship functioning with any specific manifestation of the internalizing dimension (e.g., social anxiety symptoms) might reflect a broader association between relationship functioning and higher order dimensions such as internalizing (e.g., South et al., 2011). Reframing studies around a hierarchical model of psychopathology can therefore give a more complete account of how potential causal factors relate to mental health problems. As a result, theory becomes more nuanced. Given the present results, for example, we might posit that satisfying relationships buffer against all internalizing problems and, at a finer level of resolution, against depression-specific problems, too. This is more complex (and accurate) than hypotheses guided by DSM alone (e.g., satisfying relationships mitigate risk for major depression). Moreover, this theoretical development feeds into improved research design. That is, study design is more efficient to the extent that on the basis of prior results, researchers can target the emotional phenotypes most relevant to relationship functioning.

Although broad dimensions, such as internalizing, seem to have strong criterion validity across most published investigations, research on hierarchical models is bound to discover important roles of lower order dimensions. As we alluded to, depressive symptoms in our analysis remained significantly correlated with relationship quality after adjusting for the pathology depression shares with other emotional problems. (Note, however, that depression’s higher order [i.e., indirect] effect was twice as large as the depression-specific [i.e., direct] effect.) This result supports the idea that multiple levels of breadth can be important to consider in mental health research, depending on the association of interest. For instance, future research might benefit from testing both higher order and depression-specific pathways between relationship quality and depression.

For published examples of this multitiered approach to criterion-validity testing, see Forbes et al. (2020), Hamlat et al. (2019), Rodriguez-Seijas et al. (2015), and Sellbom et al. (2020). In addition, in this article’s Supplemental Material, we recasted Tutorial 1 as a latent variable model and demonstrated one way to estimate the criterion-validity effects of lower order symptom dimensions over and above a common factor representing the broad internalizing dimension.

Tutorial 3: developmental trends in latent psychopathology dimensions in adolescence

Developmentalists have been studying internalizing and externalizing variation for decades, dating back to Achenbach’s (1966) early factor analyses of child mental disorder symptoms. This body of research thus provides a guide for dimensional research across the entire life span (Beauchaine & McNulty, 2013). However, much of the evidence supporting the HiTOP model comes from cross-sectional studies of adults. Less is known about how HiTOP dimensions change over time, particularly in developmental stages associated with rapid shifts in risk for psychopathology, such as adolescence. Longitudinal research is needed to test the extent to which dimensional phenomena are developmentally coherent to ensure that developmental changes in the structure of psychopathology are captured in the HiTOP model (Caspi et al., 2020; Forbes et al., 2016).

Recent work has used longitudinal structural equation modeling techniques to identify continuity and change in psychopathology phenotypes—especially internalizing, externalizing, and the p-factor—in children (Olino et al., 2018), adolescents (Laceulle et al., 2015; Snyder et al., 2017), and older adults (Eaton et al., 2011). Formally testing longitudinal measurement invariance is important because it determines whether observed changes in latent dimensions can be interpreted as genuine change in that dimension over time or is reflective of fluctuations in measurement properties of the observed variables (Horn & McArdle, 1992). Once such measurement invariance is established, longitudinal research can further characterize the factors that predict the natural course of psychopathology and its downstream consequences.

Background.

The transition to adolescence marks a period of social upheaval. Youths tend to distance themselves from parents and invest heavily in peer relationships as a source of social and emotional support (Steinberg & Morris, 2001). Fostering friendships during adolescence is therefore a key developmental milestone.

Anxiety and depression can derail this process. Cross-sectional data demonstrate that youths diagnosed with emotional disorders report less satisfying peer relationships and more difficulty establishing and keeping friendships (Rapee et al., 2019). Sociometric studies document that anxious and depressed children and adolescents tend to be at the periphery of friendship groups (Prinstein & Giletta, 2016). These interpersonal processes affect and are affected by social competence—self-perceptions of one’s ability to have meaningful social relationships (Harter, 1999).

These observations may reflect a dynamic process connecting emotional problems and self-perceptions of social functioning. That is, anxiety and depression may limit social prospects, and peer relationship difficulties may, in turn, exacerbate feelings of anxiety and depression (La Greca & Harrison, 2005). This type of positive feedback loop portends an intractable course of anxiety and depression in adolescence and beyond.

We put this developmental psychopathological hypothesis to the test by examining dynamics between internalizing problems and social competence over three occasions, each separated by about 18 months, in a community sample at the transition to adolescence. We defined internalizing as the commonality among adolescents’ self-reports on three frequently used assessment instruments for anxiety and depressive symptoms. We hypothesized that self-perceived social competence and emotional problems would reciprocally influence one another over time, consistent with the idea of a positive feedback loop. The data for this tutorial come from the Genes, Environment, and Mood Study (for details, see Hankin et al., 2015). Participants (N = 682; 381 girls) were, on average, 12 years old (SD = 2.38) at Wave 1. About two thirds were White, 11% were Black, 9% were Asian or Pacific Islander, 1% were American Indian or Alaska Native, and 11% identified as multiracial or another race. Thirteen percent identified as Latinx.

As a precursor to testing these prospective effects, we investigated longitudinal measurement invariance in our internalizing construct, which was represented by three observed indicators of anxiety and depressive symptoms.

Measures.

Anxiety and depression were assessed with popular instruments that are well validated with regard to inferences about children’s internalizing symptoms: self-report of Children’s Depression Inventory (CDI; Kovacs, 1985), self-report of Manifest Anxiety Scale for Children (MASC; March et al., 1997), and caregiver report of the Strengths and Difficulties Questionnaire (SDQ; Goodman, 2001). Our analysis involved total scale scores from the CDI and MASC and the SDQ Emotional Symptoms subscale score, which reflects a mixture of anxiety and depressive symptoms (e.g., “I worry a lot”; “I am often unhappy, depressed, or tearful”). Higher scores on all three measures reflect more severe symptoms.

The Self-Perception Profile for Adolescents (Harter, 2012) was administered to measure youths’ attitudes about themselves along a number of dimensions, ranging from athletic ability to physical appearance to school performance. In the current analysis, we focused on the Social Competence subscale, which asks respondents to rate the extent to which they match one end or another of six bipolar items tapping into the number and quality of their friendships with peers (e.g., “some kids find it hard to make friends” vs. “other kids find it’s pretty easy to make friends”). Higher social competence scores reflect better functioning.

Results.

For the descriptive statistics for our observed variables, see Table S5 in the Supplemental Material. All contemporaneous indices of internalizing problems were significantly correlated with one another (rs = .15–.43), consistent with our plan to treat them as expressions of the same underlying factor. Social competence was also modestly to moderately related to observed internalizing symptoms cross-sectionally (rs = −.15 to −.48) and at adjacent waves (rs = −.12 to −.33), which confirmed the link between these two constructs.

We used latent variable modeling to define an internalizing factor as the common variance among CDI, MASC, and SDQ scores at all three assessment waves. We applied CFA to test for measurement invariance, a precondition for the cross-lagged panel models (CLPMs) we estimated to test our primary hypotheses (Horn & McArdle, 1992), of this longitudinal model. In the first step in this process, we tested a longitudinal CFA in which all factor loadings were freely estimated, called the unconstrained model. This model was a good fit to the data, χ²(15) = 33.23, p < .01; CFI = .99; RMSEA = 0.046; SRMR = 0.048, which suggests that the assumption of configural invariance—the same three observed variables serve as indicators of the internalizing factors across waves—was satisfied. All loadings were statistically significant and substantial, except for SDQ, which was slightly below conventional thresholds at the first two waves (standardized factor loadings at Waves 1–3 = 0.28, 0.27, 0.38).

Next, in a test of loading invariance, we fit a new model that constrained loadings of corresponding indicators (e.g., Wave 1 CDI, Wave 2 CDI, Wave 3 CDI) to take on the same (unstandardized) values across measurement occasions. This restriction did not significantly degrade model fit, difference in χ²(4) = 2.28, p = .68, which demonstrated that the three observed indicators measured the latent construct in approximately the same way over time. That is, the association between the internalizing factor and indicators did not change appreciably across waves. After confirming metric invariance, we could move forward with tests of how internalizing and social competence shape one another over time.

There are many ways to study the temporal dynamics of two related constructs (Usami et al., 2019). The key is to choose the analysis plan that most closely maps onto the developmental question at hand. In our case, we aimed to study how individual differences in internalizing problems prospectively predicted changes in self-perceived social competence (and vice versa). This question was a close match for the CLPM, which is a popular choice for developmental researchers studying codevelopment among two related processes (Orth et al., 2021). The CLPM requires that two constructs are measured on at least two occasions, and it estimates the prospective effects of each construct on the other above and beyond the observed within-constructs continuity in each process (McArdle, 2009).

The CLPM fit the data well, χ²(41) = 68.21, p < .01; CFI = .99; RMSEA = 0.034; SRMR = 0.047. It included both autoregressive (e.g., Wave 2 social competence regressed on Wave 1 social competence) and cross-lagged (e.g., Wave 2 social competence regressed on Wave 1 internalizing) paths, which represent continuity and reciprocal influences, respectively (see Fig. 3). The complete list of parameter estimates is presented in Table S6 in the Supplemental Material.

Fig. 3.

Cross-lagged panel model of internalizing and social competence over 3 years. All effect sizes are fully standardized (for full results, see Table S6 in the Supplemental Material available). All factor loadings were statistically significant at the .001 α level. Rectangles and circles represent observed and latent variables, respectively. Dashed lines denote statistically nonsignificant paths. com = social competence; int = internalizing; CDI = Children’s Depression Inventory; MASC = Manifest Anxiety Scale for Children; SDQ = Strengths and Difficulties Questionnaire. Asterisks indicate significant paths (***p < .001).

To foster model parsimony, we evaluated several sets of equality constraints that constrained parameters to the same value across waves, similar to the restrictions imposed in the test of loading invariance (see Curran & Bollen, 2001). This process resulted in fixing autoregressive, cross-lagged, and residual covariance paths to equality over time but not the prospective effect of social competence on internalizing, which was freely estimated (for details, see the Supplemental Material).

This final model, represented in Figure 3, also fit the data well, χ²(48) = 76.52, p < .01; CFI = .99; RMSEA = 0.032, SRMR = 0.049. The autoregressive paths for internalizing were fairly large (standardized effects range = 0.69–0.74), whereas they were more moderate for social competence (range = 0.42–0.44). There were effects of internalizing variation on subsequent social competence (range = −0.23 to −0.25) above and beyond the temporal stability of both constructs. The reverse effects (i.e., social competence on internalizing), represented in Figure 3 by dashed lines, were small and not statistically significantly different from zero (range −0.01 to 0.13).

Interpretation.

There were two main findings here. First, through measurement invariance tests, we observed that there was continuity in how the latent internalizing construct was expressed across 3 years in adolescence. A three-indicator measurement model adequately reproduced not only the correlations among observed anxiety and depression variables across waves (configural invariance), but also the different indicators related to the latent construct equivalently over time (metric invariance). This result suggests that internalizing, as a latent construct, was developmentally coherent through early adolescence when operationalized via these self-report symptom inventories.

Furthermore, levels of internalizing problems in this age group showed substantial rank-order stability in individual differences over the 3-year span, as seen in the autoregression coefficients (standardized effects ≈ 0.70). Other research has reported similar estimates over intervals this long (and longer; Krueger et al., 1998; Sunderland et al., 2013; Vollebergh et al., 2001). Much more investigation is needed into normative patterns of stability and change of HiTOP dimensions and how they align with results drawn from research on the time course of categorical diagnoses.

Second, the CLPM revealed that there is overlap in the development of internalizing and social competence. The two were strongly related cross-sectionally in this sample, as evidenced, for example, by a modelimplied correlation of −.57 between the two at Wave 1. Moreover, internalizing prospectively predicted decrements in social competence even after adjusting for continuity in both constructs across waves. These results tie into a robust literature on the negative effects of anxiety and depression on self-perceptions of ability to engage in and have meaningful peer relationships in adolescence (Harter, 1999). The reverse was not true: Social competence did not predict changes in internalizing levels. This null effect could signal that social competence is not an important predictor of internalizing trajectories, but it might also be a function-marked temporal stability in the internalizing spectrum mentioned above given that there is comparatively little variation left to predict over time.

This analytic framework can be applied to understanding the developmental coherence and longitudinal stability of HiTOP dimensions—and more broadly to characterizing the risks, consequences, and reciprocal relationships among HiTOP dimensions and their correlates across the life span.

Common Questions and Challenges

Is HiTOP for self-report measures only?

HiTOP grew out of factor analysis of self-reported and interviewer-rated symptoms and diagnoses. These were the data available in large data sets that were well suited to the type of latent variable modeling traditionally used to resolve the correlations among latent dimensions. For example, investigators have repeatedly mined the National Epidemiologic Survey on Alcohol and Related Conditions, a multiwave study involving interviewer ratings of mental disorder symptoms in more than 40,000 U.S. adults, for information on the empirical structure of psychopathology.

This need not be the predominant strategy for research on the utility of HiTOP constructs. First, informant reports are known to have incremental validity and, for some outcomes, superior predictive utility relative to a proband’s own account of mental health problems. Indeed, Achenbach’s seminal work on dimensional models of child psychopathology relied in large part on collateral sources such as parents, caregivers, and teachers (reviewed in Achenbach, 2020). This trend continues in current HiTOP-informed research (see e.g., Boudreaux et al., 2019).

Second, we recommend greater integration of psychopathology dimensions with experimental and laboratory-based designs involving behavioral observation, task performance, psychophysiology, and neuroimaging (e.g., Latzman et al., 2020; Snyder et al., 2019). This type of research can determine whether HiTOP dimensions relate more robustly than categorical diagnoses to behavioral and biological vulnerabilities.

For instance, there is a robust line of clinical research on fear conditioning in anxious populations. Anxious people exhibit more intense defensive responses to safety cues during conditioning (Duits et al., 2015). But it remains unknown whether disrupted fear learning is a general characteristic of all internalizing problems, is correlated predominantly with fear-based disorders (e.g., phobias), or is tied to specific diagnoses (e.g., posttraumatic stress disorder; e.g., Craske et al., 2009). A hierarchical, dimensional approach to psychopathology assessment—as reflected in, for example, the IDAS (see Tutorial 2)—could resolve this issue and significantly advance etiological theories of anxiety-based disorders.

Third, the concordance of verbal-report and other measures of psychopathology can be examined by integrating multiple measurement modalities into the same structural model. In one recent study, Hill et al. (2019) modeled a latent dimension of “neural reactivity” using event-related potentials derived from different experimental conditions and examined the correlations between this factor and a latent internalizing factor. In this same vein, Yancey et al. (2016) modeled the shared variance of heart rate, startle potentiation, and corrugator muscle tension expressed during an affective picture-viewing task along with verbal reports of fear and fearlessness. They found that this latent factor was meaningfully correlated with symptoms of DSM-defined fear disorders and other physiological risk factors.

Does HiTOP require large samples?

The short answer is “not necessarily.” As mentioned, much of the work on the latent structure of psychopathology has relied on epidemiological studies, partly because they tend to assess broad swaths of psychopathology. Using HiTOP constructs to test hypotheses in other fields is a different undertaking, and sample size requirements can vary widely depending on the research context.

The sample size necessary to estimate factor-analytic and structural equation models depends on the number of observed variables, the number of latent variables, the proportion of missing data, and the magnitude of both factor loadings and paths between latent variables (Wolf et al., 2013). Simple models with clearly defined factors and strong loadings can be estimated with sample sizes between 50 and 200 (Boomsma, 1983; MacCallum et al., 1999). In the least favorable conditions, these models may require sample sizes up to 450 and beyond. Should latent variable models not converge or should the investigator simply prefer a simpler method, it is possible to test the same hypotheses without using latent variable models. This is accomplished by estimating a path model between scale scores from tests that assess HiTOP constructs directly. This is the approach taken in Tutorial 1, for example. For list of HiTOP-friendly measures, see the HiTOP Clinical Network (n.d.). Even in small samples, however, it is worth attempting to estimate latent variable models. This is because path models assume that measures are perfectly reliable—an assumption that is almost never true—which attenuates power and increases the chance of errors (Cole & Preacher, 2014). When measures are unreliable, a structural equation model can detect effects half as large as those that could be detected using a path model (Wolf et al., 2013). Furthermore, investigators might have hypotheses about an effect that is moderate or large; power analysis in structural equation models will often focus on power for a specific parameter, not the overall fit of the model.

Researchers planning to use significance tests should perform power analyses for all study designs, including those involving HiTOP. When only observed variables are involved, the familiar power analysis methods will suffice. The pwr package (Version 1.3–0; Champely, 2020) for the R software environment implements many of the recommendations in Cohen (2013). For instance, in Tutorial 1, we would determine the sample size needed to detect the change in variance explained when adding the 10 lower order IDAS dimensions to a multiple regression that also included the internalizing dimension (assuming this was our primary hypothesis).

For investigators who want to compute power for effects in a structural equation model, there is a growing number of user-friendly tutorials and dedicated software packages that provide guidance (Wolf et al., 2013). Within the R software environment, there are several recently developed (and evolving) packages dedicated to power computations (see e.g., Jorgensen et al., 2020). The Mplus software has similar capabilities (e.g., Muthén & Muthén, 2002), as does SAS (MacCallum et al., 1996).

Does HiTOP work only in certain populations?

There is no ideal population for HiTOP-informed research. Again, a fair amount of work thus far has been conducted in epidemiological data sets, in which large samples afford much more precision around parameter estimates. Yet basic architectural elements of the HiTOP model appear to be replicable across more specific populations, including patient groups (reviewed in Kotov et al., 2017). Much of the structure also seems to be invariant across gender, developmental stage, and region of the world (Krueger et al., 2018).

We also note that one of HiTOP’s central tenets is that psychopathology phenotypes are dimensional. Thus, every person theoretically can be characterized by their location on a spectrum representing, say, mood lability, fear, or disinhibited externalizing. Very fine gradients are possible such that meaningful individual differences can be detected in most, if not all, populations.

Is HiTOP compatible with the Research Domain Criteria?

Beginning in 2007, the U.S. National Institute of Mental Health developed the Research Domain Criteria (RDoC) initiative as an alternative research framework to DSM. RDoC is not a classification system per se but a research heuristic that identifies biological and behavioral processes relevant to mental disorders (Kozak & Cuthbert, 2016). The RDoC framework is operationalized in a matrix (National Institute of Mental Health, n.d.) that crosses eight units of analysis (genes, molecules, cells, circuits, physiology, behavior, paradigms, and self-report) with six constructs (negative valence, positive valence, cognitive, social, arousal and regulatory, and sensorimotor systems).

There are some salient conceptual differences between HiTOP and RDoC (for an extended comparison of RDoC with current diagnostic systems, which overlaps with a HiTOP-RDoC comparison in several ways, see Clark et al., 2017). HiTOP is oriented around the subjective signs and symptoms of mental disorder that drive patients to get treatment and that practitioners encounter regularly in the clinic, whereas RDoC expands on this conceptualization of symptoms to include deviations from usual functioning of particular biobehavioral systems (e.g., neural system activity and behavior). This level of analysis focuses on both biology and psychology but has less emphasis on the importance of self-reported symptoms of psychopathology compared with HiTOP.

To the extent that research in the HiTOP and RDoC frameworks converge on the same fine-grain clinical phenomena as reliable phenotypes for research, these research paradigms can together identify the biological building blocks of the clinical constructs represented in the HiTOP model. With these goals in mind, recent scholarship from the HiTOP consortium has focused on developing a detailed interface between HiTOP and RDoC (Michelini et al., 2020; Perkins, Joyner, et al., 2020; Perkins, Latzman, & Patrick, 2020).

How does HiTOP address development?

Much of the literature underpinning the current HiTOP model has been conducted in cross-sectional studies and adult samples that focus on an incomplete range of psychopathology domains (Kotov et al., 2017). This has resulted in two substantial limitations when applying the HiTOP framework in developmental psychopathology research. First, the current framework has a static structure that does not account for potential changes in the empirical structure of psychopathology across development. For example, predispositions toward general psychopathology are evident in early childhood temperament such that propensities toward internalizing domains compared with externalizing domains of psychopathology emerge later in childhood and narrower and more varied symptoms and syndromes crystalize in adolescence and early adulthood (for a review, see Forbes et al., 2019). This may well manifest as noninvariance in dimensional models of psychopathology throughout development (e.g., the varying structure of the Achenbach System of Empirically Based Assessment inventories by age group; Achenbach, 2020). Studies that have examined the structure of psychopathology longitudinally—and across multiple developmental periods—have tended to converge on dimensions represented in the HiTOP model (Caspi et al., 2020; for a review, see Forbes et al., 2016). However, the extent to which these dimensions demonstrate longitudinal measurement invariance throughout development requires more systematic and comprehensive efforts and is a core focus of the HiTOP Developmental Workgroup (Kotov et al., 2021).

Second, the current working model spans approximately two thirds of the diagnoses in the DSM-5 (APA, 2013) but does not yet include important domains of child-onset psychopathology, such as autism spectrum disorder and other neurodevelopmental disorders. This limits the scope of research that can currently be accommodated in a HiTOP framework. The HiTOP Revisions Workgroup is currently working on ensuring that the model remains up to date with current research, and the first formal revision of the model is anticipated to incorporate a broader variety of psychopathology—including autism-related symptoms (e.g., Michelini et al., 2019; Noordhof et al., 2015).

Conclusions

Categorical diagnoses constrain the inferences we can make in mental health research. Is HiTOP a viable alternative? We believe so, but we acknowledge that this is ultimately an empirical question that must be tested scientifically. This article presented tutorials intended to help investigators pilot this framework—and evaluate its utility—in various applied research contexts.

We recognize that research habits are hard to break. Funding bodies, professional organizations, and training programs for many years institutionalized categorical disorders as research targets, notwithstanding some recent trends in the opposite direction (e.g., RDoC). We contend, however, that the practical benefits of a hierarchical, dimensional approach are worth the effort. HiTOP promises to unify unnecessarily fragmented research literatures, improve precision in psychological theories, and generate new substantive insights into the causes and consequences of mental illness.

The HiTOP framework is relevant to all researchers who study mental illness, not just those researchers who specialize in diagnosis, assessment, or psychometrics. We hope this article will help make HiTOP more accessible and promote widespread empirical evaluation of HiTOP’s utility as a research tool that has the potential of improving clinical practices and, in turn, clinical outcomes for people seeking mental health care.