The Hierarchical Taxonomy of Psychopathology (HiTOP) Is Not an Improvement Over the DSM

Abstract

In their response to our article (both in this issue), DeYoung and colleagues did not sufficiently address three fundamental flaws with the Hierarchical Taxonomy of Psychopathology (HiTOP). First, HiTOP was created using a simple-structure factor-analytic approach, which does not adequately represent the dimensional space of the symptoms of psychopathology. Consequently, HiTOP is not the empirical structure of psychopathology. Second, factor analysis and dimensional ratings do not fix the problems inherent to descriptive (folk) classification; self-reported symptoms are still the basis on which clinical judgments about people are made. Finally, HiTOP is not ready to use in real-world clinical settings. There is currently no empirical evidence demonstrating that clinicians who use HiTOP have better clinical outcomes than those who use the Diagnostic and Statistical Manual of Mental Disorders (DSM). In sum, HiTOP is a factor-analytic variation of the DSM that does not get the field closer to a more valid and useful taxonomy.

We thank DeYoung and colleagues (2022) for their commentary and appreciate the opportunity to debate the validity and usefulness of the Hierarchical Taxonomy of Psychopathology (HiTOP). DeYoung and colleagues claim that HiTOP is fundamentally different from the Diagnostic and Statistical Manual of Mental Disorders (DSM) and that it does not “group” people. In this commentary, we explain why we disagree and discuss three fundamental flaws with HiTOP.

HiTOP Is Not the Empirical Structure of the Symptoms of Psychopathology

HiTOP is the result of a dimensional-interpretation/simple-structure factor-analytic procedure (Thurstone, 1947) in which items are rotated to create nonoverlapping dimensions. This simple structure does not represent the complexity of the empirical structure of the symptom data. In fact, it “has no substantive justification whatsoever … from an explanatory point of view, it is plainly ridiculous to suppose that latent variables are uncorrelated (i.e., if these latent variables are taken to be substantively meaningful factors that refer to objective properties outside of the model)” (Borsboom, 2017b, p. 46, see also Guttman, 1992; McGrane & Maul Gevirtz, 2020; van Bork et al., 2017). HiTOP is not a data-driven realization of the structure of the symptoms of psychopathology because it was created using an arbitrary and inadequate representation of the dimensional space (Maraun, 1997; Turkheimer, 2017; Wittchen & Beesdo-Baum, 2018) Consequently, HiTOP may hinder progress on understanding the etiology of psychopathology because the model is wrong.

If a simple structure is not an adequate representation of the symptom data, then what is? The answer is that nobody knows. HiTOP researchers have never tested the structure of the data (facet theory; Guttman, 1992). It could be a radex, cylinder, circumplex, simplex, or other structure. The choice to use a simple-structure approach is based on convenience and tradition rather than empirical considerations (Turkheimer, 2017). The advantage of using a simple structure is that the results are easier to interpret; the disadvantage is that the results are likely to be an inaccurate representation of nature. It is akin to pouring water into an ice tray, freezing it, and then claiming the ice cubes are the empirical structure of ice.

Factor Analysis and Dimensional Ratings Do Not Fix the Problems Inherent to Descriptive Classification

HiTOP is a descriptive system like the DSM. It uses a like-goes-with-like approach in which people who report similar symptoms are thought to have similar mental-health problems and people with different symptoms are thought to have different mental-health problems. Symptom-based descriptive approaches can be very useful (e.g., DSM), but over time, classification should evolve from a system based on observable characteristics to one based in theory (Hempel, 1965). HiTOP is not on this evolutionary path. It replaces categories with dimensions and comorbidities with covariances, but it is not sufficiently differentiated in content or its fundamental approach to classification. It is a factor-analytic articulation of the content of the DSM.

Nevertheless, DeYoung and colleagues contend that HiTOP is “very different” than the DSM:

HiTOP takes a variable-centered, rather than a person-centered, approach to classification. Symptoms are grouped into a hierarchy of dimensions according to their likelihood of manifesting in the same individual. This is very different from nosologies, including the [DSM], that classify people into discrete categories. In HiTOP, people are not classified but rather described by their position on each symptom dimension in the framework. … Haeffel et al.’s confusion about the fact that HiTOP classifies symptoms renders a number of their specific arguments invalid or irrelevant.

(p. 280 PROOF 2)

We do not dispute that HiTOP is “variable-centered,” but the same is true for the DSM. As the consortium has written about their own work, “the HiTOP model might look novel at first glance, but it contains the same clinical phenomena that researchers are used to, just reorganized as dimensions” (Conway et al., 2021, p. 156). But more importantly, the symptom groupings in HiTOP and the DSM do not exist in a vacuum. Despite what DeYoung and colleagues claim, HiTOP does, in fact, group individuals. For any diagnostic tool to be useful, it must effectively differentiate individual human beings, thereby grouping them. If HiTOP does not or cannot do this, then it is an ineffective clinical tool. If HiTOP can effectively distinguish varying clinical presentations, then it does, in fact, group individuals.

In addition, the HiTOP methodology relies on between-subjects (i.e., interindividual) factor analysis of covariance between input items. This covariance is determined by calculating the deviations in each item from its mean and the relative agreement in deviations between items. Although not a formal rank-order statistic, these calculations rely on the relative rank order of individuals within the sample. Thus, the derivation of clinical targets via hierarchical factor-analytic methods does, in essence, group individuals according to their sample-wise position within a set of symptom items.

The key point here is that researchers and clinicians use symptom profiles to make judgments about people’s mental health and potential treatments, and these judgments require assumptions: (a) People describing similar symptoms have similar problems with a shared etiology and treatment, and (b) people describing different symptoms have different problems with different etiologies and require different treatments. An additional assumption of HiTOP is that people who describe similar symptoms share a common etiology for which genetic variants can be discovered. Unfortunately, these assumptions are unfounded and inconsistent with the complexity of nature (e.g., Haeffel et al., 2022).

This point can be illustrated with a “variable-centered” thought experiment in which four patients arrive at a hospital with the following complaints (see Table 1).

Table 1.

“Variable-Centered” Thought Experiment

Patient 1	Patient 2	Patient 3	Patient 4
Aches	Aches	Diarrhea	Cough
Dry cough	Dry cough	Nausea	Fatigue
Chills	Chills		Sore throat
Fever	Fever
Fatigue	Fatigue

A HiTOP-like symptom questionnaire is administered and scored. Results show that Patient 1 and Patient 2 score high on the “respiratory disorders” subfactor and have elevated scores on COVID-19. Patient 3 scores high on the “gastrointestinal disorders” subfactor and has elevated scores on the norovirus syndrome. Patient 4 has elevated scores on the “rhinovirus” (common cold) syndrome. Unfortunately, this descriptive approach led to incorrect conclusions (and the wrong treatment) for 75% of patients. Although Patients 1 and 2 reported the same symptoms, they suffered from COVID-19 and influenza H1N1, respectively (an example of equifinality). Patient 3 had COVID-19 (not norovirus) despite reporting a completely different symptom profile than Patient 1 (an example of multifinality). Patient 4 had bronchitis, not a cold. If a single-stranded RNA virus such as COVID-19 can lead to such highly heterogeneous symptom expressions, then imagine the complexity in mental-health symptoms that arise from the interplay of thousands of genes and environmental factors over decades of development.

In sum, HiTOP is not scientifically progressive and does not improve on the DSM. It does not matter how the symptom profiles are created (factor analysis vs. expert consensus) or operationalized (dimensions vs. categories) or the specific terms used to describe how people are grouped (described vs. classified). These differences do not alter the fact that both HiTOP and the DSM are symptom-based taxonomies that share the same underlying assumptions (e.g., symptom covariation is meaningful) and inherent limitations.

The field does not need two descriptive taxonomies, and despite its flaws, the DSM is more scientifically progressive than HiTOP. The DSM is clinically useful, contains more information (e.g., course, severity, duration, persistence, prevalence), and has greater potential to change over time. For example, the fifth edition of the DSM (American Psychiatric Association, 2013) incorporated the dimensional approach for use with personality disorders (e.g., Zimmermann et al., 2019). In contrast, HiTOP does not have many of the features found in a useful taxonomy, and the simple-structure statistical approach does not lend itself to falsification (e.g., positive manifold guarantees a general factor; factor solutions do not require the existence of latent variables; there are infinite well-fitting models; it cannot correct for equifinality and multifinality because it misses these cases). In sum, HiTOP is “not reflective of the ‘true’ complexity of psychopathological processes” (Wittchen et al., 2009, p. 201; Wittchen & Beesdo-Baum, 2018) and may contribute to incomplete and inaccurate understandings of the nature of mental illness (e.g., Achenbach, 2020; Eronen, 2021; Fisher et al., 2018; Funkhouser et al., 2021; Haywood et al., 2021; Kerridge et al., 2013; Witte et al., 2017).

HiTOP Is Not Ready to Use in Clinical Practice

We were pleased to learn that the HiTOP consortium is beginning to test its system in clinical settings, and we share its desire to improve psychiatric taxonomy. But it is concerning that DeYoung and colleagues continue to recommend HiTOP for use in clinical settings without knowing the results of these studies (or replicating them). Evidence should precede recommendations. Yet the HiTOP consortium continues to promote HiTOP as “ready for implementation now” (Conway et al., 2021, p. 156). We recommend that the following five questions be addressed empirically before using HiTOP in clinical settings:

1. Can clinicians reliably and validly create an aggregate of measures to assess the entire HiTOP systems?

2. Can clinicians reliably and validly interpret a HiTOP profile?

3. Can clinicians make reliable and valid treatment choices given a HiTOP profile?

4. Which treatments are effective for which HiTOP profiles?

5. Do clinicians who use HiTOP have better therapeutic outcomes than clinicians who use the DSM (e.g., randomly assign therapists to use HiTOP or DSM)?

We understand that HiTOP researchers are actively working to answer some of these questions, and it remains an open question to whether HiTOP will have clinical utility. But to date, there is not one published study that addresses one or more of these clinical questions. There is no evidence that clinicians can piece together and reliably interpret a HiTOP profile (consisting of a large number of facets with various levels of symptom data) and then make reliable and valid judgments about case conceptualization and treatment. In fact, decades of research on the fallibility of human judgment would suggest otherwise (e.g., Garb, 2005; Kahneman, 2011; Meehl, 1954). Furthermore, there is no evidence that having this symptom information will improve treatment outcomes. At least one study suggests it will not; Lima and colleagues (2005) showed that providing clinicians with HiTOP-like symptom information from the Minnesota Multiphasic Personality Inventory does not improve treatment outcomes. It is premature for HiTOP consortium researchers to recommend that clinicians start using HiTOP in real-world settings without conducting a single clinic-based study that directly compares it with the DSM (e.g., randomly assign therapists to use HiTOP or the DSM). There is no evidence that HiTOP leads to better clinical outcomes than the DSM, and HiTOP outcomes could be worse.

A founding principle of psychological science (McFall, 1991) is that people deserve assessments and interventions supported by evidence (i.e., adherence to the principle of “do no harm”). A statistically significant factor-analytic model does not guarantee real-world usefulness. We hope that the HiTOP consortium will take the same “data-driven” mindset it used to create the system to determine HiTOP’s readiness for real-world use. We, as researchers, need to hold ourselves to the same standards that we expect of clinicians (i.e., not using untested methods) or risk further eroding the public’s trust in clinical psychology (Baker et al., 2008).

Conclusion

I do not believe that we have an adequate classification system now, and it seems unlikely to me we would ever arrive at one by merely using factor analysis or statistical clustering.

(Borsboom, (2017b, p. 50)

A reliable and valid classification system is fundamental to progress in clinical psychology. It provides a common language for professionals, organizes knowledge (e.g., for information retrieval), and allows for prediction (e.g., treatment). Decisions to change or replace a classification system should be based on the results of scientific competition (e.g., tests of incremental validity). Empirical evidence must supersede popularity, endorsements, membership numbers, academic prowess, publication rates, and citation counts (cf. Kotov et al., 2021). To paraphrase Richard Feynman, it does not make a difference how beautiful your model is; it does not make a difference how smart you are, or what your name is, or how many publications you have; if the data do not support the model, then it is wrong (Feynman et al., 1964). Unfortunately, the field of psychology has a history of pursuing new and exciting ideas at the expense of developing a cumulative character. This perpetual cycle was described by Meehl (1978):

There is a period of enthusiasm about a new theory, a period of attempted application to several fact domains, a period of disillusionment as the negative data come in, a growing bafflement about inconsistent and unreplicable empirical results, multiple resort to ad hoc excuses, and then finally people just sort of lose interest in the thing and pursue other endeavors.

(p. 807)

A factor-analytic version of the DSM is not on the path to a more valid and etiologically based classification system; it does not solve clinical psychology’s classification problems. There is little reason to jump on the HiTOP bandwagon given the lack of evidence for its structure, clinical usefulness, and falsifiability. If clinical psychology is going to change the basis for how mental illness is conceptualized, assessed, and treated, then the new system should be better than the old system. At some point, this means moving on from the like-goes-with-like symptom approach to classification and focusing on more progressive, dynamic, novel, and diverse classification strategies and theories (e.g., Barlow et al., 2021; Beck & Haigh, 2014; Berenbaum, 2013; Borsboom, 2017a; Del Giudice & Haltigan, 2021; Follette & Houts, 1996; Gone & Kirmayer, 2010; Luyten & Blatt, 2011; Mansell et al., 2009; Molenaar, 2004; Robinaugh et al., 2021; Smith et al., 2009; Thomas & Sharp, 2019; Wilshire et al., 2021; Zachar & Kendler, 2017).