Psychiatric classifications: validity and utility

Abstract

Despite historical assumptions to the contrary, there is little evidence that the majority of recognized mental disorders are separated by natural boundaries. Diagnostic categories defined by their clinical syndromes should be regarded as ‘valid’ only if they have been shown to be truly discrete entities. Most diagnostic concepts in psychiatry have not been demonstrated to be valid in this sense, though many possess ‘utility’ by virtue of the information they convey about presenting symptoms, outcome, treatment response and, in some instances, aetiology. While researchers in genetics, neurobiology and population epidemiology are increasingly more likely to adopt a continuum/dimensional view of the variation in symptomatology, clinicians prefer to hold on to the categorical approach embodied in current classifications such as ICD‐10 and DSM‐5. Both points of view have plausible justification in their respective contexts, but the way forward may be in their conceptual reconciliation.

In his Philosophical Remarks, L. Wittgenstein commented that “the classifications made by philosophers and psychologists are as if one were to classify clouds by their shape”1. The metaphor is apt: clouds have fuzzy boundaries, tend to merge imperceptibly, and drift, being carried by invisible air currents. Observation and measurement of their movement predict, within a margin of error, the weather, yet the inner physical and chemical structure of clouds is hidden to the naked eye.

Wittgenstein's aphoristic remark applies equally well to the classifications developed by psychiatrists: the conceptual outlines of syndromes and putative disease entities tend to change with successive revisions of their classification, relative to their utility for predicting course, outcome and likely response to available treatments, even if their inner biological and psychological structure is not fully understood. The latter, the quest for validity of our concepts, remains an open agenda.

In anticipation of this, the protagonist of modern psychiatric nosology E. Kraepelin stated in one of his last articles, Patterns of Mental Disorder, that “it is necessary to turn away from arranging illnesses in orderly well‐defined groups, and to set ourselves the undoubtedly higher and more satisfying goal of understanding their essential structure”2. This goal of validity is yet to be attained.

THE NATURE OF PSYCHIATRIC CLASSIFICATIONS

The term nosology refers to the theory about the nature of medical conditions and the principles and rules of their classification. In psychiatry, we are still facing the recurrent question about the nosological status of the brain and mind disorders that constitute the core of the discipline. Are we dealing with discrete entities, or with graded continuous phenomena to which we can apply cut‐off points to separate “pathology” from “normal variation” and to determine the need for treatment? What is the relationship between the clinical manifestations of a disorder and the underlying brain dysfunction, pathological processes or predisposing genetic aberrations?

Notwithstanding the advances in the neuroscience and genetics of psychiatric disorders, many of the present‐day answers to these questions are a replay of debates that took place in the earlier periods of scientific psychiatry. This suggests that there may be inherent shortcomings in the nosological classifications in clinical psychiatry adopted since the beginning of the 20th century and all the way to the present versions of DSM and ICD.

Medical classifications are created with the primary purpose of meeting pragmatic needs related to diagnosing and treating people experiencing illnesses. Their secondary purpose is to assist the generation of new knowledge relevant to those needs (though progress in medical research usually precedes, rather than follows, improvements in classification). Simply stating that medical classifications classify diseases (or that psychiatric classifications classify disorders) begs the question, as the status of concepts like “disease” and “disorder” remains obscure3.

As pointed out by Scadding4, the concept of “disease” has evolved with the advance of medical knowledge, and is at present no more than “a convenient device by which we can refer succinctly to the conclusion of a diagnostic process which starts from recognition of a pattern of symptoms and signs, and proceeds, by investigation of varied extent and complexity, to an attempt to unravel the chain of causation”. “Disease”, therefore, is an explanatory construct integrating information about deviance from the population “norm”; characteristic clinical manifestations; characteristic pathology; underlying causes; and reduced biological fitness.

For a cluster of such attributes to be referred to as “a disease”, these characteristics must be shown to form a “real‐world correlational structure”5, which must be stable and distinct from other similar structures. The typical progression of knowledge starts with the identification of the clinical manifestations (the syndrome) and the deviance from the “norm”; understanding of the pathology and aetiology usually comes much later. However, there is no fixed point or agreed threshold beyond which a syndrome can be said to be “a disease”. Today, Alzheimer's disease, with dementia as its syndrome, characteristic brain morphology, tentative pathophysiology, and at least partially understood causes, is one of the few conditions in psychiatric classifications that approximates the disease construct. The majority of the “disorders” in our current classifications are, at best, described as syndromes6.

The essential task in the construction of a nosology of discrete disease entities is to identify internally cohesive clinical groupings based on established inter‐correlations among symptoms and syndromes (the cross‐section) and patterns of course and outcome (the longitudinal aspect). Individual groupings should be separated from one another by demonstrable natural boundaries, or a “zone of rarity”7. The test of their validity is the degree to which they are found to be associated with explanatory variables of deeper structural significance – potential causal factors, pathogenetic mechanisms, treatment response, as well as stability vis‐à‐vis demographic and cultural variation. However, nosological entities in psychiatry, constructed according to such idealized desiderata, have met with difficulties.

The first problem is that, on the examples of schizophrenia and affective disorders, the requirement of a close correspondence between the cross‐section of the disorder and the patterns of its course and outcome was never fully met. Recent attempts to identify in the early, high‐risk or prodromal state, symptoms and signs that reliably predict transition to full‐blown psychosis have not been successful8. It has been furthermore demonstrated in follow‐up studies that a proportion of initially “typical” schizophrenias may recover, while a proportion of “typical” manic‐depressive illnesses may run a chronic and disabling course. These observations could not be easily reconciled with the assumptions of the original “dichotomy” model of the two disorders.

The argument that recovering schizophrenias are not “true” cases of the disorder, and ought to be re‐diagnosed if lasting recovery occurs, contradicts the findings of two important World Health Organization (WHO) studies: the International Pilot Study of Schizophrenia (IPSS)9 and the subsequent Study on Determinants of Outcome of Severe Mental Disorders10. In the IPSS, cases were diagnosed in a restrictive way by applying three sets of criteria: clinician's diagnosis according to ICD; computer diagnosis using the CATEGO algorithm; and empirical grouping of cases by cluster analysis, on the basis of maximum shared characteristics. Patients who met simultaneously the three sets of criteria were designated as a “core” or “concordant” group of schizophrenia, that was expected to be more homogeneous than the rest of the cases. However, the follow‐up data did not reveal any significant differences in course and outcome between the concordant cases and the non‐concordant ones.

Such findings do not stand alone: a number of recent follow‐up studies confirm the notion that severe deterioration is not the typical outcome of schizophrenia, even if a very long follow‐up period is involved. According to the WHO Report on Recovery from Schizophrenia11, which integrated findings from several long‐term follow‐up studies conducted under the aegis of WHO, “the most striking overall finding… is that the current global status of over half of these subjects – 56% of the incidence group and 60% of the prevalence group – is rated as “recovered”. Nearly half have experienced no psychotic episodes in the last 2 years of follow‐up… These percentages accord fairly well with ratings of both current symptoms and functioning”11. These findings suggest that the prognosis of schizophrenia is an open‐ended dynamic process whose direction can, within limits, be modified at any point. The presumed “characteristic” psychopathological phenomena, such as the Schneiderian first‐rank symptoms12, did not appear to have prognostic significance.

A second shortcoming of the classical nosological system is its failure to separate consistently the two entities of schizophrenia and affective disorders. This has been known for a long time, but the difficulty was thought to reside in the imprecise definition of the diagnostic criteria, rather than in the existence of a large group of conditions which simply defy the dichotomy and exhibit the features of a clinical “hybrid”. This group has attracted a variety of diagnostic labels, including “schizoaffective disorder”13 or “unsystematic schizophrenias”14, and was classified alternately with the schizophrenias or with the affective disorders, but never found a comfortable place in either category. The existence of such “hybrid” cases poses the problem of defining the borderline between the two disorders. One alternative solution is to treat the poor prognosis schizophrenia and the good prognosis affective disorders as two extremes on a single clinical (and presumably genetic) continuum that could include all kinds of intermediate forms.

A third problem for which the classical nosological theory has failed to find an acceptable solution is the classification of the sub‐threshold, practically non‐pathological forms of cognitive and affective deviations and the unusual personalities which are encountered among biological relatives of schizophrenia patients. The importance and relative frequency of these variants were clearly recognized by Bleuler15, who coined the term “latent schizophrenia”, and they were subsequently reported by a bewildering variety of diagnostic labels: “ambulatory schizophrenia”16, “pseudoneurotic schizophrenia”17, “borderline schizophrenia” or “schizotypal personality disorder”18 and, more recently, “attenuated psychotic syndrome”19. None of these terms has been universally accepted, nor have their diagnostic criteria been unequivocally defined. Epidemiological and genetic evidence has provided support for a link of those subclinical conditions to “core” schizophrenia, strengthening the concept of a schizophrenia “spectrum”20. The spectrum forms related to the affective disorders have so far received less attention than the non‐psychotic satellites of schizophrenia, but the recognition of a syndrome of “masked depression”21 and the notion of an affective or cyclothymic personality disorder suggest that similar problems also exist on the affective side of the classical diagnostic dichotomy of the major psychiatric disorders. At present, the borderline forms are of limited therapeutic interest, since most cases do not require treatment, and there is little evidence that, if provided, treatment is effective. Their theoretical and research importance, however, is considerable, especially from the point of view of the genetics of the major psychotic disorders.

Although the range of possible aetiological factors that may give rise to psychiatric disorders is practically unlimited, the range of psychopathological syndromes, reflecting the brain's responses to a variety of noxae, is limited. Since a variety of aetiological factors may produce the same syndrome (and conversely, an aetiological factor may give rise to a spectrum of syndromes), the relationship between aetiology and clinical syndrome is an indirect one. In contrast, the relationship between the syndrome and its underlying pathophysiology, or specific brain dysfunction, is likely to be much closer. This was recognized long ago in the case of psychiatric illness associated with somatic and brain disorders, where clinical variation is restricted to a limited number of “organic” brain syndromes, or “exogenous reaction types”22. This was recently reconfirmed by evidence that many focal neurological diseases, neurodegenerative disorders and autoimmune encephalopathies can present with symptom pictures closely mimicking the symptomatology of “endogenous” disorders, such as schizophrenia23. In the complex psychiatric disorders, where aetiology is multifactorial, future research into specific pathophysiological mechanisms could be considerably facilitated by a better delineation of the syndromal status of diagnostic categories, providing a rationale for reinstating the syndrome as the basic unit in future versions of psychiatric classifications.

None of the many attempts to re‐shape the nosology of the major psychiatric disorders has been entirely satisfactory. There can be no doubt that the classical nosological hypothesis was a major step forward, introducing order and parsimony in a field that had previously been chaotic or arbitrarily subdivided. The least that could be said is that the nosological hypothesis helped to bring into focus issues which critics could oppose or endorse, thus contributing to a diversity of viewpoints that was fruitful in a developing discipline such as psychiatry. However, a more fundamental re‐thinking of the nosological theory underlying the classification of psychiatric disorders will require the development of a conceptual framework that allows a better integration of clinical, neurobiological, genetic and behavioural data.

DSM‐5 AND ICD‐10

Classifying in science involves forming categories or taxa for ordering natural objects or entities and assigning names to these categories. Ideally, the categories should be jointly exhaustive to account for all possible entities, and mutually exclusive. In biology, there is agreement that classifications reflect fundamental properties of biological systems and constitute “natural” classifications. This is not so with psychiatric classifications. First, the objects being classified in psychiatry are not “natural” entities but explanatory constructs. Secondly, the taxonomic units of “disorders” in DSM‐5 and ICD‐10 do not form hierarchies and contain no supraordinate, higher‐level organizing concepts. Therefore, DSM‐5 and ICD‐10 are not systematic classifications in the sense in which that term is applied in biology.

Social anthropologists have claimed that an analogue to current psychiatric classifications could be found in the so‐called indigenous or “folk” classifications of animals or plants, which do not consist of mutually exclusive categories, have no hierarchies, but may contain many rules applicable ad hoc 24. They are pragmatic and adapted to the needs of everyday life. In that sense, DSM‐5 and ICD‐10 are not systematic classifications, but they are useful tools of communication and play an important role in research, clinical management and teaching.

Many clinicians are aware that diagnostic categories are constructs, justified only by whether or not they provide a useful framework for organizing clinical experience and making predictions about outcome and the effects of treatment decisions. However, the generic term “disorder” (first introduced as a name for the unit of classification in DSM‐I in 1952) has no correspondence with either the concept of disease or the concept of syndrome in medical classifications. The data on which the majority of the current diagnostic rubrics in psychiatry are based consist primarily of reported subjective experiences and patterns of behaviour. Some of those rubrics correspond to syndromes in the medical sense, but many appear to be isolated symptoms, habitual behaviours, or personality traits. Thus, the ambiguous status of the “disorder” creates conceptual confusion and hinders the advancement of knowledge.

The fragmentation of psychopathology into a large number of “disorders”, of which many are merely symptoms, facilitates the proliferation of comorbid diagnoses which blur the distinction between true comorbidity (co‐occurrence of aetiologically independent disorders) and the spurious comorbidity that may be a feature of multifaceted but essentially unitary syndromes. It is, therefore, not surprising that disorders, as defined in the current versions of DSM and ICD, have a strong tendency to co‐occur, which suggests that “fundamental assumptions of the dominant diagnostic schemata may be incorrect”6.

VALIDITY AND UTILITY

While the reliability of psychiatrists’ diagnoses can be substantially improved by the use of explicit diagnostic criteria, their validity remains uncertain. What is meant by validity of a diagnostic concept in psychiatry is rarely discussed and few studies have addressed this question directly. Because the validity of diagnostic concepts, and of their defining criteria, is a critical issue, it is important to clarify what is implied by the term validity in the context of psychiatric diagnosis.

The word “valid”, derived from the Latin validus, means strong, and is defined as “well founded and applicable; sound and to the point; against which no objection can fairly be brought”25. In formal logic, validity is the characteristic of an inference that must be true if all its premises are true. However, there is no single agreed meaning of validity in science, although it is generally accepted that the concept addresses “the nature of reality”26, and that its definition is an “epistemological and philosophical problem, not simply a question of measurement”27.

The attribution of validity to scientific concepts and theories is in fact an unending quest: what was regarded as valid knowledge in the past is quickly superseded by new evidence, and this in the nature of scientific endeavour. In a thoughtful review of the subject, Zachar28 proposed the term comparative validity, to summarize the progression of scientific knowledge, which “emphasises rationally justified criteria we use to say that current theories/models are improvements on past theories/models”. In a similar vein, Aragona29 examined the “epistemological history” of the successive DSM editions, from DSM‐I (1952) to DSM‐5 (2013), and concluded that all systems share the same view of validity as a “correspondence to external reality”, with the ultimate ideal of validation by neurobiological data.

In psychology, the American Psychological Association's distinction between content, criterion‐related and construct validity30 still holds, since it provides criteria for the validity of psychological tests. Borrowing terminology from psychometric theory, psychiatrists have mainly been concerned with concurrent and predictive validity, partly because of their relevance to the issue of the validity of diagnoses. The ability to predict outcome, both in the absence of treatment and in response to specific therapies, has always been a key concern to physicians. In a seminal paper, Goodwin and Guze31 asserted that “diagnosis is prognosis”, and that the follow‐up is to the psychiatrist “what the postmortem is to the physician”. The types of validity currently employed in the context of psychiatric diagnosis – construct, content, concurrent and predictive – are borrowed off the shelf of psychometric theory in psychology. Few diagnostic concepts in psychiatry meet these criteria at the level of stringency normally required of psychological tests.

Despite such ambiguities, a number of procedures have been proposed to enhance the validity of psychiatric diagnoses in the absence of a simple measure. Thus, Robins and Guze32 outlined a program with five components: clinical description; laboratory studies; delimitation from other disorders; follow‐up studies; and family studies. This schema was later elaborated by Kendler33, who distinguished between antecedent validators (familial aggregation, premorbid personality, precipitating factors); concurrent validators (e.g., psychological tests); and predictive validators (diagnostic consistency over time, rates of relapse/recovery, response to treatment). Andreasen34 has proposed additional validators, such as findings of molecular genetics, neurochemistry, neuroanatomy, neurophysiology and cognitive neuroscience, suggesting that “the validation of psychiatric diagnoses establishes them as real entities”.

Such procedural criteria implicitly assume that psychiatric disorders are distinct entities, ignoring the possibility that disorders might merge into one another with no clear boundary in between. However, there is increasing evidence of overlapping genetic predisposition to schizophrenia and bipolar disorder, as well as to seemingly unrelated disorders, such as autistic spectrum, intellectual disability and, possibly, epilepsy. It is equally likely that the same environmental factors may contribute to several different syndromes. Should such findings be systematically replicated, their repercussion on future psychiatric classifications would be considerable. It has been proposed that variations in psychiatric symptomatology might indeed be better represented by “an ordered matrix of symptom‐cluster dimensions”35 than by a set of discrete categories. However, it would be premature at this time to discard the current categorical entities.

In contrast to validity, a diagnostic rubric may be said to possess utility if it provides non‐trivial information about prognosis and likely treatment outcomes, and/or testable propositions about biological and social correlates7. The term utility was first used in this sense by Meehl36, who wrote that “the fundamental argument for the utility of formal diagnosis… amounts to the same kind of thing one would say in defending formal diagnosis in organic medicine. One holds that there is a sufficient amount of etiological and prognostic homogeneity among patients belonging to a given diagnostic group so that the assignment of a patient to this group has probability implications which it is clinically unsound to ignore”36.

Many, though not all, of the diagnostic concepts listed in contemporary classifications such as DSM‐5 and ICD‐10 are useful to clinicians, whether or not the category in question is valid, as they provide information about the likelihood of recovery, relapse, deterioration, and social handicap; they guide treatment decisions, describe symptom profiles, or guide research into the aetiology of the syndrome. However, there is a critical difference between validity and utility. Validity is by definition an invariate attribution to a diagnostic category: there may be uncertainty about its justification because of lack of relevant empirical information, but in principle, a category cannot be “partly” valid7. Utility, on the other hand, is an incremental, graded characteristic that is partly context specific. Schizophrenia may be an invaluable concept to practicing psychiatrists, but of questionable use to researchers exploring the genetic basis of psychosis. For example, the DSM‐5 definition of schizophrenia is useful for predicting outcome, because some degree of chronicity is inbuilt. But a broader definition, covering a heterogeneous “schizophrenia spectrum”, is more useful for defining a syndrome with high heritability for genetic research.

THE VIEW FROM PSYCHIATRIC GENETICS

Can psychiatric genetics inform the nosology of mental disorders? Not so long ago, tentative findings of overlapping associations between candidate genes (NRG1, DTNBP1, G72/G30, DISC1, DISC2) in DSM‐IV schizophrenia and mood disorders raised the expectation that “over the coming years, molecular genetics will catalyse a reappraisal of psychiatric nosology” by conceptualizing “a spectrum of clinical phenotypes with susceptibility conferred by overlapping sets of genes”37.

Such reappraisal has not happened. However, recent whole‐genome association studies (GWAS), involving large, consortium‐pooled samples from multiple research centres, have indeed identified shared genetic variation of common single nucleotide polymorphisms across schizophrenia, bipolar disorder, major depression, autism spectrum and attention‐deficit/hyperactivity disorder38. The main contributor to these findings was the variation in calcium‐channel activity genes (CACNA1C and CACNB2), which appeared to have pleiotropic effects on a range of psychopathology. These findings reinforced the hope that, similarly to medical disciplines such as oncology and cardiology, psychiatry could move “beyond descriptive syndromes… towards a nosology informed by disease cause”39.

Further support for a trans‐diagnostic commonality of genomic variants underlying susceptibility risks was provided by the largest to date GWAS of schizophrenia40, which revealed multiple common polymorphisms converging upon individual genes and definable molecular pathways in the brain, involving glutamatergic synaptic and calcium channel functions, as well as a highly significant contribution of the immune system. Importantly, there was evidence of overlap between rare copy number variations associated with schizophrenia and rare de novo mutations observed in intellectual disability and autism spectrum disorders. However, instead of an imminent reappraisal of psychiatric classification, these novel findings add to the tremendous complexity of the genotype‐phenotype problem in common mental disorders.

In a recent review, Kendler41 outlined “possible scenarios” of biological coherence in the genomic findings, ranging from low coherence (clinical syndromes do not have specific underlying pathophysiology) to high coherence (risk genes and polymorphisms map to a single biological pathway underpinning a single disease process). Since psychiatric disorders are significantly more heterogeneous than other complex disorders, greater heterogeneity means also greater complexity, and emergent traits in the “mind‐brain” system may be “more remote from individual gene effects than those seen in other tissues”. For these reasons, we may be ill‐advised to call, under the sway of important novel findings, for a premature overhaul of psychiatric nosology.

CONCLUSION: THE WAY FORWARD

The present diagnostic manuals, ICD and DSM, are classifications of current diagnostic concepts, and not of “natural kinds”, such as people or diseases. There is little evidence that most recognized mental disorders, including the psychoses, are separated by natural boundaries. There is a growing understanding, supported by recent advances in genetic and neurobiological research, that many of the present diagnostic categories are end‐point phenotypes for heterogeneous gene networks, pathophysiological pathways, and environmental modifiers. Probably we shall see in the future increased experimentation with research‐based classifications and diagnostic tools, focusing on improving and refining the clinical utility of both categorical and dimensional models of psychopathology, and seeking a consilience between the two, leading to concordance.

Paraphrasing Jaspers’ dictum42, validity is an “idea in Kant's sense of the word… an objective which one cannot reach since it is unending, but all the same it indicates the path for fruitful research and supplies a valid point of orientation for particular empirical investigations”. This means that our primary concern should be the progressive refinement of the utility of the diagnostic concepts and tools, towards the enhancement of their phenomenological accuracy, predictive value and capacity to guide person‐focused treatment and management decisions.