Seeking Verisimilitude in a Class: A Systematic Review of Evidence That the Criterial Clinical Symptoms of Schizophrenia Are Taxonic

Abstract

This review examines whether there is evidence that the criterion symptoms of Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) (DSM-IV) schizophrenia are taxonic—that schizophrenia is not part of a single distribution of normality. Two taxometric methods, coherent cut kinetics (CCK) and latent variable modeling (LVM), are demonstrated to be sensitive to latent classes and, therefore, were regarded as providing relevant statistical evidence. A systematic literature search identified 24 articles describing analyses of 28 participant cohorts in which CCK or LVM methods were used with one or more criterion symptoms of schizophrenia. Virtually all analyses yielded results that, on first impression, favored taxonic over dimensional interpretations of the latent structure of schizophrenia. However, threats to the internal and external validity of these studies—including biased or inadequate analyses, violation of statistical assumptions, inadequate indicator screening, and the introduction of systematic error through recruitment and sampling—critically undermine this body of work. Uncertainties about the potential effects of perceptual biases, unimodal assessment, and item parceling are also identified, as are limitations in seeking to validate classes with single or double dissociations of outcomes. We conclude that there is no reason to seriously doubt a single-distribution model of schizophrenia because there is no evidence that provides a serious test of this null hypothesis. A second fundamental question remains outstanding: is schizophrenia truly a group of schizophrenias, with taxonic divisions separating its types? We make design and analysis suggestions for future research addressing these questions.

Introduction

The Psychotic Disorders Workgroup is seriously considering incorporating a dimensional component into the Psychotic Disorders section of Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) (DSM-V). There are many reasons for thinking that inclusion of dimensions in DSM-V will be a good thing, improving both clinical practice and research, perhaps even bringing a degree of verisimilitude to one of the most widely used systems for classifying schizophrenia. Nevertheless, as the field considers this change, it is important to ask whether there is evidence that greater verisimilitude may be found in a class than in a dimension. The objective of this review is to address this question. To begin, the scope of the question is specified, and the sorts of statistical evidence that are considered pertinent are identified and illustrated.

The latent structure of schizophrenia may be examined using numerous phenotype and endophenotype measures, such as of brain structure or function, cognition, or a broad range of clinical signs and symptoms. That is not the intention here. Rather, the question we aim to address concerns the clinical symptoms comprising Criterion A of DSM-IV schizophrenia. Also, although continuum, its synonyms and its antonyms appear frequently in schizophrenia research literature, they are applied to diverse referents. For instance, some speak of a continuum of psychotic experience: clinical phenotypes giving rise to need for care fall on a continuum with subclinical phenotypes that do not give rise to need for care. In contrast, others use continuum to refer to the latent structure of a population: a specified population is continuous or dimensional if all its members comprise 1 group but is taxonic or discontinuous if the population comprises 2 or more commingled groups. In this article, we will be restricting ourselves to the latter referent of continuum, the latent structure of populations. Finally, some might suggest the best way to address this question is by considering the balance of probabilities—to weigh evidence on both sides of the argument. However, our approach to this question will be to consider whether research that attempts to demonstrate a latent class structure represents a challenge to dimensional models. Thus, the question we aim to address is whether there is evidence that population distributions of the criterion symptoms of schizophrenia are taxonic.

Statistical Methods and Potentially Relevant Evidence

Researchers who have endeavored to examine this question have used one or more of 4 statistical approaches. The earliest of these studies involved exploring data—often discriminant function scores—for evidence of bimodality.^1–3 Aside from specific methodological limitations of this research, exchanges on bimodality show that bimodality is neither sufficient to indicate taxonicity nor an inevitable consequence of it.^4–7 Cluster analysis has also been employed in this context but has likewise been discredited as a means of addressing questions about latent structure.^8–10 Consequently, these methods and evidence stemming from them will not be considered further.

The 2 remaining approaches possibly yield evidence that is pertinent to the question at hand. First, coherent cut kinetic (CCK) methods emerged in the 1970s. CCK methods (eg, maximum covariance analysis [MAXCOV], mean above minus below a cut [MAMBAC], maximum eigenvalue analysis, among others) were developed primarily to address questions about the latent structure of the liability for schizophrenia, although they have much wider applications.¹¹ Second, latent variable methods (eg, latent class analysis [LCA], admixture analysis, multivariate normal mixture modeling) also became feasible during the 1970s, but it was not until 1982 that this approach was first applied to schizophrenia.^12–14

Understanding of the utility of these statistical methods and the meaning of information derived from them lags substantially behind the capacity to employ these, as is the case with many other statistical approaches. Therefore, in order to inform our review, we briefly describe these approaches in Supplementary Online Material (pS1–S6), illustrating the methods using 2 simulated datasets. It is not our intention to provide a detailed exposition of these analysis methods. These can be found in other sources.^7,11,15–22 Rather, it is useful to consider briefly the circumstances within which each of these procedures can or should be applied and the sorts of solutions these generate.

The Supplementary Online Material shows that CCK and generalized latent variable modeling (LVM) are sensitive to latent class structures (pS6–S14). Therefore, these methods were regarded as providing relevant evidence in the context of this review. However, we also included evidence from studies that used classical LCA alone—a procedure that cannot distinguish latent continuous from latent class structures. We do so chiefly because in many instances LCA results are interpreted as evidence of distinct classes, subgroups, syndromes, or types. Notwithstanding the inclusion of these studies, in addressing the research question, the primary objective was to evaluate the quality of the available evidence and threats to its internal and external validity. In the absence of robust internal and external validity, conjectures about theoretical and clinical implications may be premature.

Method

We searched entries in the MEDLINE database (1950–December 2007) to identify the intersection of 3 sets of publications: (a) those articles using one or more of the full or truncated text-word (ie, search suffix “.mp”) search terms psychosis, psychotic, schizoaffect, schizophren, or schizotyp; (b) those using one or more of the text-word search terms class, kind, category, type, subtype, taxa, taxon, taxanomic, taxonomic, categorial, categorical, continu, or discontinu; and (c) those using one or more of the text-word search terms latent, underlying, structure, structural, or discrete. This search identified 833 articles, of which 757 reporting human research were considered potentially relevant. We then examined each of these articles, first by reading the title and subsequently, as necessary, the abstract and the article itself, to identify articles that fell within the review inclusion criteria. Also, any article not excluded after reading the abstract was searched for citations to other potentially relevant articles, which were screened for inclusion in the same manner. Inclusion criteria were as follows.

1. Studies were included if one or more Criterion A (–like) symptoms were among one or more indicators included in a statistical analysis of latent structure. We excluded studies in which indicators were not adequately specified (eg, Hallmayer et al²³) or in which schizophrenia- or psychosis-related diagnoses were used as the primary indicators (eg, Young et al,¹² Mojtabai,²⁴ Peralta and Cuesta,²⁵ and Roy et al²⁶). The latter cannot, by definition, address the research question.

2. The indicators were derived from the application of clinical questionnaires, rating scales, interviews, or the review of clinical records. Analyses of data from nonclinical rating scales (eg, Raine's²⁷ Schizotypal Personality Questionnaire) were not included.

3. The analyses were published in or since 1950 in a peer-reviewed journal.

4. The analysis method identifies latent structures with mutually exclusive classes. Thus, studies using grade of membership analysis were not included (eg, Manton et al,²⁸ Jablensky and Woodbury,²⁹ and Cassidy et al³⁰).

From each article included in the review, we recorded cohort variables (sampling population, recruitment strategy, response rate, and inclusion and exclusion criteria), sample variables (size and demographic characteristics), indicator measurement variables (details of assessment instruments and administration procedures), indicator preparation variables (score derivation and coding, use of item parcels, metric, and evidence relating to violation of independence), analysis variables (statistical method, models subjected to analysis, model selection criteria, fit indices, group postvalidation, and corroborating analyses), and the key results of the analyses.

Results

We identified 24 relevant articles describing 38 analyses of 28 participant cohorts that met the inclusion criteria (table 1). Face-value reading of these studies suggests that a variety of latent class models may underlie variance in Criterion A and other indicators.

Table 1.

Studies Meeting the Inclusion Criteria

Source	Sample and Limits	Indicators and Methods	Analysis	Model and Labels (Prevalence)
Young (1982)13	New psychiatric admissions. Without endogenous depression or mania.	First-rank symptoms, formal thought disorder, and blunt affect. Method not reported.	LCA (n = 88)	Two classes, with parameter restrictions: schizophrenia 46%) and no schizophrenia (54%).
Goldstein et al (1990)31	Consecutive admissions to psychiatric hospital. With DSM-III schizophrenia.	Dysphoria, early onset, flat affect, persecutory delusions, poor premorbid social adjustment, winter birth, and family history of schizophrenia or a schizophrenia-spectrum disorder. Interview, records, and informants.	Simultaneous LCA, males (n = 171) and females (n = 161).	Two classes, with parameter restrictions: neurodevelopmental (males 69%, females 26%) and affective (males 31%, females 74%) types.
Jørgensen and Jensen (1990)32	Consecutive admissions to psychiatric hospital. With delusional psychosis.	Delusional thought content, auditory hallucinations, primary delusions, and blunt affect. Interview.	LCA (n = 88)	Two classes: prominent schizophrenia features (47%) and few or no features of schizophrenia (53%).
Parker et al (1991)33	Consecutively diagnosed patients. With melancholic depression. Without schizophrenia.	Hallucinations, delusions, or both; motor agitation or retardation, poverty of speech, or poor insight; sustained pessimism, hopelessness, helplessness, or worthlessness; no diurnal variation in mood; and constipation. Self-report and interview.	LCA (n = 125)	Two classes: psychotic depression (36%) and depression without psychosis (64%) types.
Castle et al (1994)34	First contact psychiatric patients, in catchment over 20 years. With nonaffective psychosis.	Dysphoria, early onset, family history of schizophrenia, poor premorbid social adjustment, persecutory delusions, restricted affect, and winter birth. Records.	Simultaneous LCA, male (n = 227) and female (n = 220).	Three classes, with parameter restrictions: neurodevelopmental (males 57%, females 26%), paranoid (males 43%, females 47%), schizoaffective (males 0%, females 26%) types.
Goldstein et al (1994)35	Psychotherapy trial participants and consecutive neuropsychology referrals. With schizophrenia.	Chronic negative symptoms, early educational problems, history of neurological events, and sex. Interview and records.	LCA (n = 49)	Two classes: developmental deficit (32%) and complement (68%).
Nestadt et al (1994)36	Unaffected relatives in multiply affected families, identified via psychiatric admissions. With ≥2 relatives with schizophrenia. Without psychosis.	Features of (A) schizoid (7 indicators), (B) paranoid (7), and (C) schizotypal (9) personality disorder. Interview, informants, and records.	LCA of (A), (B), and (C), and all combined (max n = 602)	(A) One class (100%).(B) One class (100%).(C) Four classes: paranoid (2%), schizoid (10%), positive symptom (2%), and unaffected (85%) types.Combined: 5 classes: paranoid (3%), schizoid (5%), positive symptom (1%), loner (19%), and unaffected (72%) types.
Tyrka et al (1995)37	Children of mothers with schizophrenia (67%) and of control mothers (33%).	Flat affect, passivity, peculiarity, poor prognosis, social anxiety, and social withdrawal. Teacher ratings, interview, and test performance.	MAXCOV (n = 311)	Two classes: liable for schizophrenia-spectrum disorder (overall, 49%; high risk, 58%; low risk, 28%) and complement.
Bell (1997)38 study 1	Medical/psychiatric center patients.	Cognitive disorganization items.	Unspecified CCK (n = 381)	Two classes: cognitive disorganization taxon (36%) and complement.
Bell (1997)38 study 2	Clozapine trial participants.	Cognitive disorganization items.	Unspecified CCK (n = 421)	Two classes: cognitive disorganization taxon (24%) and complement.
Bell (1997)38 study 3	Homeless, geriatric, and others, from mixed sources, multinational.	Cognitive disorganization items.	Unspecified CCK (n = 426)	Two classes: cognitive disorganization taxon (43%) and complement.
Kendler et al (1995)39	Siblings concordant for (A) schizophrenia or (B) nonaffective psychosis. A was subset of B.	Affective symptoms, age of onset, catatonia, course features, delusions, hallucinations, flat affect, mania, outcome, sex, and thought disorder. Interview, records, or both.	LCA (A) (n = 580), (B) (n = NR)	(A) Five classes: schizoaffective (29%), negative symptom schizophrenia (21%), chronic delusions and poor outcome (16%), paranoid schizophrenia (19%), and remitting/relapsing catatonic schizophrenia (15%).(B) Five classes: schizoaffective; negative symptom schizophrenia; chronic delusions and poor outcome; moderate positive, negative, and affective symptoms; and remitting/relapsing catatonic schizophrenia. Base rates not reported.
Kendler et al (1998)40	All schizophrenia patients on population case register, plus random subset of affective patients on same case register.	Psychotic, negative, manic, and depressive symptoms; predominance of affective symptoms; and illness course features (duration, deterioration, global course, and outcome). Interview.	LCA (n = 344)	Six classes: classic schizophrenia (26%), major depression (21%), schizophreniform (18%), bipolar schizomania (18%), schizodepression (15%), and hebephrenia (3%).
Battaglia et al (1999)41	General population, same-sex twins.	Constricted affect, ideas of reference, magical thinking, no close friends, odd behavior, odd speech, social anxiety, suspiciousness, and unusual perceptual experience. Interview.	LCA (n = 118)	Four classes: negative schizotypal (9%), socially anxious (17%), positive schizotypal (8%), and unaffected (66%)
Fossati et al (2001)42	Consecutive medical psychology/psychotherapy admissions. Without main schizophrenia-spectrum disorders.	Constricted affect, ideas of reference, magical thinking, no close friends, odd behavior, odd speech, social anxiety, suspiciousness, and unusual perceptual experience. Interview and self-report.	LCA (n = 564)	Four classes: broad schizotypal (6%), positive schizotypal (7%), suspicious socially anxious (12%), and unaffected (75%) types.
Peralta et al (2002)43	Consecutive psychiatric admissions. With one or more Criterion A symptoms.	(A) Index episode, (B) lifetime, and (C) interepisode global ratings of alogia, anhedonia, attention, avolition, catatonia, delusions, depression, flat affect, hallucinations, inappropriate affect, mania, odd behavior, and thought disorder. Interview, observation, significant others, and records.	LCA (n = 110)	(A) Four classes: psychotic (37%), mixed positive-negative (33%), schizomania (21%), and schizodepressive (9%) types.(B) Four classes: mixed or undifferentiated (40%), psychosis (24%), schizobipolar (22%), and schizodepressive (15%) types.(C) Three classes: remitted (60%), chronic psychosis (23%), and defect psychosis (17%) types.
Peralta and Cuesta (2003)44 and Cuesta et al (2007)45	Consecutive psychiatric admissions. With one or more Criterion A symptom or one or more SANS global items rated marked or severe.	(A) Index episode and (B) lifetime ratings of avolition, catatonia, delusions, depression, disorientation, hallucinations, inappropriate affect, insight, mania, odd behavior, onset, poverty of speech, residual symptoms, syndrome polymorphism, and thought disorder. (C) Reality distortion, disorganization, and negative feature factor scores. Interview and observation.	(A, B) LCA (n = 660) (C) MAXCOV and MAMBAC (n = 660)	(A) Five classes: schizophrenia (42%), psychosis (19%), schizomania (17%), schizodepression (12%), and cycloid (10%) types.(B) Five classes: schizophrenia (38%), atypical schizophrenia (22%), psychosis (20%), schizobipolar (12%), and schizodepression (8%) types.(C) Ambiguous results.
Blanchard et al (2005)46	Treatment outcome study participants; multisite. With DSM-III-R schizophrenia, schizoaffective disorder, or schizophreniform disorder.	Anhedonia-asociality, blunt affect, avolition-apathy, and alogia. Interview and observer ratings.	MAXCOV and MAMBAC (n = 238)	Two classes: negative symptom taxon (28%) and complement (72%).
Fossati et al (2005)47 study 1	Consecutive clinical psychology/psychotherapy admissions. Without schizophrenia-spectrum disorders.	Cognitive-perceptual, interpersonal, and disorganization summary indices of schizotypal personality disorder. (A) Interview and (B) self-report.	Multivariate normal mixture analyses (n = 721)	(A) Two classes: schizotypal (4%) and complement (96%) distributions.(B) Two classes: schizotypal (39%) and complement (61%) distributions.
Fossati et al (2005)47 study 2	Consecutive anxiety disorder admissions. Without schizophrenia-spectrum disorders.	Cognitive-perceptual, interpersonal, and oddness summary indices of schizotypal personality disorder. Interview.	Multivariate normal mixture analyses (n = 537)	Two classes: schizotypal (9%) and complement (91%) distributions.
Fossati et al (2005)47 study 3	General population convenience sample.	Cognitive-perceptual, interpersonal, and oddness summary indices of schizotypal personality disorder. Interview.	Multivariate normal mixture analyses (n = 225)	Two classes: schizotypal (1%) and complement (99%) distributions.
Haro et al (2005)48	Treatment outcome study participants; multi-site, nternational. With acute manic or mixed episodes and need for change in medication.	Severe mania at admission, current psychosis (hallucinations, delusions, or both), cannabis use (current), cannabis misuse (lifetime), alcohol misuse (lifetime), other substance misuse (lifetime). Interview.	LCA (n = 3536)	Three classes: minimal symptom (59%), psychosis with mania (28%), and substance-misuse (13%) types.
Murray et al (2005)49	All in catchment area. With psychotic illness within previous 5 years.	Sixty-two reality distortion, disorganization, negative, mania, and depression symptoms. Records.	LCA (n = 387)	Four classes: unipolar affective (19%), disorganization with reality distortion (28%), bipolar affective (23%), and affective psychosis without significant negative symptoms (30%) types.
Boks et al (2007)50	All psychiatric referrals. With suspected psychosis.	Counts of lifetime positive (13 symptoms), negative (10), disorganized (10), depression (11), and mania (8) symptoms. Interview.	LCA (n = 1056)	Six classes: bipolar schizomania (33%), schizodepression (26%), hebephrenia (14%), classic schizophrenia (14%), major depression (6%), and unaffected (7%) types.
Schmitz et al (2007)51 primary cohort	Treatment seekers with first-episode psychosis. With schizophrenia-spectrum mental disorder.	Abstract thinking, blunt affect, emotional withdrawal, rapport, social withdrawal, spontaneous conversation, and stereotyped thinking, at 12-months follow-up, with covariates including sex, age, and baseline negative symptoms. Method not reported.	Latent class regression (n =116)	Three classes: near complete remission (51%), worsened (27%), and partially improved (22%) negative symptom types.
Schmitz et al (2007)51 replication cohort	Patients with first-episode psychosis. With schizophrenia-spectrum mental disorder.	Abstract thinking, blunt affect, emotional withdrawal, rapport, social withdrawal, spontaneous conversation, and stereotyped thinking, at 12-months follow-up, with covariates including sex, age, and baseline negative symptoms. Method not reported.	Latent class regression (n = 59)	Three classes: near complete remission (63%), worsened (17%), and partially improved (20%) negative symptom types.
Shevlin et al (2007)52,53 NCS	General population (stratified, area probability) householders. Noninstitutionalized.	Paranoia, thought broadcasting, mind reading, passivity experiences, self-referential delusions, and visual, auditory, olfactory, and somatic hallucinations. Lay interview.	(A) Hybrid IRT-LCA (n = 5858)(B) LCA (n = 5854)	(A) 4 ordered classes: psychosis like (1%), intermediate psychosis (8%), low psychosis (27%), and unaffected (64%) types.(B) 4 classes: psychosis (2%), hallucination (6%), intermediate (6%), and unaffected (86%) types.
Shevlin et al (2007)52 NEMESIS	General population (stratified, random) householders. Noninstitutionalized.	Paranoia, thought broadcasting, mind reading, passivity experiences, self-referential delusions, and visual, auditory, olfactory, and somatic hallucinations. Lay and psychiatric interview.	Hybrid IRT-LCA (n = 7075)	Four ordered classes: psychosis-like (0.1%), intermediate psychosis (0.5%), low psychosis (3%), and unaffected (97%) types.

Note: LCA, latent class analysis; DSM-III R, Diagnostic and Statistical Manual of Mental Disorders (Third Edition Revised); MAXCOV, maximum covariance analysis; PANSS, Positive and Negative Syndrome Scale; CCK, coherent cut kinetics; MAMBAC, mean above minus below a cut; NCS, National Comorbidity Survey. NEMESIS, Netherlands Mental Health Survey and Incidence Study; IRT-LCA, item response theory-latent class analysis.

Two-Class Interpretations

Fourteen analyses concluded with 2-class interpretations. Nine of these were based on LVM and 5 on CCK methods. Reports based on LVM suggest that there are qualitative distinctions between schizophrenia and nonschizophrenia syndromes in new psychiatric admissions,^13,32 between neurodevelopmental and affective syndromes, or between neurodevelopmental and complement groups, in those with schizophrenia,^31,35 and that depression with psychosis differs qualitatively from depression without psychosis among those with melancholic depression.³³ A latent class structure underlying clinical interview ratings of schizotypal personality disorder has been replicated in 3 cohorts—2 nonschizophrenic psychiatric samples and a general population sample.⁴⁷

Turning to interpretations based on CCK methods, one study suggests there is a qualitative distinction between those with and without negative symptoms among those diagnosed with schizophrenia, schizoaffective disorder, or schizophreniform disorder.⁴⁶ An abstract describing 3 analyses suggests cognitive disorganization has a latent class structure among psychiatric patients.³⁸ Finally, liability for schizophrenia-spectrum disorders in offspring of probands with schizophrenia has a latent class structure.³⁷ Among the articles that were reviewed, only one article that used CCK methods failed to reach a 2-class interpretation.⁴⁵ Because the data on which this report was based were also subject to LCA and yielded a 5-class interpretation, the CCK findings are presented below in the subsection on “Five-Class Interpretations.”

Three-Class Interpretations

In 5 cases, findings have been interpreted as evidence of 3 latent classes. Twelve-month follow-up ratings of negative symptoms in 2 cohorts yielded remitted, partially remitted, and worsened outcome-related classes.⁵¹ Neurodevelopmental, paranoid, and schizoaffective classes were identified in first contact patients with nonaffective psychosis.³⁴ And in a study of patients in acute manic or mixed episodes, current psychosis was 1 of 6 indicators used to identify minimal symptom, psychosis with mania, and substance-misuse classes.⁴⁸ Finally, interepisode global ratings of schizophrenia and affective symptoms in consecutive admissions with at least one Criterion A symptom of schizophrenia yielded remitted, chronic psychosis, and defect psychosis classes.⁴³

Four-Class Interpretations

In this same study,⁴³ separate analyses of index episode ratings and lifetime ratings both suggested 4-class solutions. For index episode data, these included psychotic, mixed positive-negative, schizomania, and schizodepression types. For the lifetime ratings, the types were labeled mixed or undifferentiated, psychosis, schizobipolar, and schizodepressive.

Analyses of data from 6 other cohorts also concluded with 4-class interpretations. In a catchment area study of individuals with a recent history of psychotic illness, analyses of symptoms of schizophrenia and the affective disorders suggested 4 types: unipolar affective, bipolar affective, disorganization with reality distortion, and affective psychosis.⁴⁹ In 2 epidemiological surveys of general population samples, participants were screened with a narrower set of items measuring hallucinations and delusions. An LCA of data from one of these, the National Comorbidity Survey (NCS), yielded 4 classes: broad psychosis, intermediate psychosis, hallucination, and unaffected.⁵³ Subsequent hybrid item-response-LCA analysis of the NCS data also identified 4 classes (psychosis-like, intermediate psychosis, low psychosis, and unaffected), but these appeared to comprise or emanate from an underlying continuum.⁵² That is, the classes appeared to have a quantitative order or hierarchy. The latter finding was replicated with similar data from the second survey, the Netherlands Mental Health Survey and Incidence Study.⁵²

The remaining 3 cohorts were used to examine the latent structure of the criterial features of Diagnostic and Statistical Manual of Mental Disorders (Third Edition Revised) (DSM-III-R) and DSM-IV schizotypal personality disorder.^36,41,42 Classes identified from the first of these studies, of nonpsychotic relatives in families multiply affected by schizophrenia, were labeled positive symptom, social isolation, paranoid, and unaffected.³⁶ Second, analysis of data from a general population sample of same-sex twins identified negative schizotypal, socially anxious/suspicious, positive schizotypal, and unaffected classes.⁴¹ Third, analysis of data from psychiatric patients without an Axis I schizophrenia-spectrum disorder yielded broad schizotypal, unaffected, positive schizotypal, and suspicious classes.

Equal numbers of classes and equivalence in indicators permit comparison of the indicator profiles obtained in these 3 studies of schizotypal personality disorder. Moderate correspondence was evident between profiles obtained from the relative³⁶ and twin⁴¹ studies, respectively: the positive symptom class resembled the positive schizotypy class (r = .85), the social isolation class resembled the negative schizotypal class (r = .66), and the paranoid class resembled the socially anxious/suspicious class (r = .61). In contrast, correspondence between the relative³⁶ and patient⁴² studies, respectively, was less consistent: paranoid and suspicious were very similar (r = .91); positive symptom and positive schizotypy were moderately alike (r = .51), but the remaining 2 classes, social isolation and broad schizotypal, did not have strong resemblance (r = .25). Similar inconsistency was evident from comparison of the twin⁴¹ and patient⁴² studies, respectively: positive schizotypal and positive schizotypal (r = .62), socially anxious/suspicious and suspicious (r = .42), and negative schizotypal and broad schizotypal (r = −.07).

Five-Class Interpretations

In the aforementioned study of nonpsychotic relatives in families multiply affected by schizophrenia,³⁶ a 5-class solution was also obtained when schizotypal personality disorder ratings were combined with ratings on criterial features of schizoid and paranoid personality disorders: paranoid, schizoid, positive symptom, loner, and unaffected classes. Two other studies have identified 5-class solutions. In a study of siblings concordant for DSM-III-R schizophrenia, indicators of a broad range of affective and schizophrenia symptoms and course features yielded schizoaffective, negative symptom schizophrenia, chronic delusional, paranoid, and remitting/relapsing catatonic schizophrenia classes.³⁹ Subsequent analyses in which the sample was extended to include those concordant for nonaffective psychosis yielded similar schizoaffective, negative symptom schizophrenia, chronic delusional, and remitting/relapsing catatonic classes, whereas the fifth class was characterized by moderate negative, positive, and affective symptoms.³⁹ Several analyses have been completed on data obtained from a large sample of consecutive psychiatric admissions with one or more Criterion A symptoms of schizophrenia.^44,45 An LCA of index episode ratings of schizophrenia and affective symptom ratings yielded schizophrenia, psychosis, schizomanic, schizodepression, and cycloid classes.⁴⁴ In contrast, lifetime ratings of the same features yielded schizophrenia, atypical schizophrenia, psychosis, schizobipolar, and schizodepression types.⁴⁴ Subsequent CCK analyses (MAXCOV and MAMBAC) of reality distortion, disorganization, and negative factor scores obtained from the same sample yielded quite ambiguous results,⁴⁵ perhaps due to the presence of parataxonic correlations among the indicators.⁵⁴ (The presence of parataxonic correlations, ie, a mixture of positive and negative correlations, among indicators suggests the indicator set will not yield meaningful CCK results. This issue is discussed in the Supplementary Online Material [pS9].)

Six-Class Interpretations

Two studies concluded with 6-class interpretations. First, LCA of schizophrenia and affective symptoms and course feature indicators obtained on schizophrenia and affective patients identified from a population case register yielded 6 classes, including classic schizophrenia, major depression, schizophreniform, bipolar schizomania, schizodepression, and hebephrenia classes.⁴⁰ Second, in a study of psychiatric referrals with suspected psychosis, counts of lifetime positive, negative, disorganized, depression, and mania symptoms also yielded 6 classes: bipolar schizomania, schizodepression, hebephrenia, classic schizophrenia, major depression, and unaffected types.⁵⁰

Dimensional Interpretations

Among the 27 reports, 2 analyses, both reported by Nestadt et al,³⁶ concluded with a dimensional interpretation. Nestadt et al³⁶ performed separate LCAs on ratings of the criterial features of schizoid and paranoid personality disorders obtained from unaffected relatives in families that had at least 2 members diagnosed with schizophrenia. For each personality disorder, solutions to 2- and 3-class analyses were interpreted as indicative of a latent dimensional or latent class structure. However, because 1-class models were not reported, it is equally reasonable to interpret these findings as consistent with a model with 2 or more latent classes. Thus, strictly speaking, the analysis does not afford a dimensional interpretation.

Limitations Undermining Evidence Quality

These studies have almost all identified latent class rather than latent dimensional structures. However, there are a variety of important limitations affecting the internal and external validity of these findings (table 2). In many cases, these limitations significantly undermine the contribution of the research to understanding of the latent structure of Criterion A symptoms of schizophrenia.

Table 2.

Threats to Internal and External Validity of Studies Meeting Inclusion Criteria

		Internal	Internal		Internal	External
Cohort	No Item Screening or Appraisal	Biased Analysis Method	Insufficient Models Compared	Conditional Independence Not Tested	Commingled Sample	Constraint on Population
Young13	✓	✓	✓	✓		✓
Goldstein et al31	✓	✓	✓	✓		✓
Jørgensen and Jensen32	✓	✓	✓	✓		✓
Parker et al33		✓	✓	✓		✓
Castle et al34	✓	✓	✓	✓		✓
Goldstein et al35		✓	✓	✓	✓	✓
Nestadt et al36
Schizoid items	✓	✓	✓	✓		✓
Paranoia items	✓	✓	✓	✓		✓
Schizotypal items	✓	✓	✓✓	✓		✓
All items	✓	✓	✓✓	✓		✓
Tyrka et al37					✓	✓
Bell38 study 1	?			?
Bell38 study 2	?			?
Bell38 study 3	?			?	✓
Kendler et al39
Schizophrenia subset	✓	✓	✓✓✓	✓		✓
Nonaffective subset	✓	✓	✓✓✓	✓		✓
Kendler et al40	✓	✓	✓	✓	✓	✓
Battaglia et al41	✓	✓		✓
Fossati et al42	✓	✓		✓		✓
Peralta et al43
Index episode	✓	✓		✓		✓
Lifetime	✓	✓		✓		✓
Interepisode	✓	✓		✓		✓
Peralta and Cuesta44 and Cuesta et al45
Index episode, LCA	✓	✓		✓		✓
Lifetime, LCA	✓	✓		✓		✓
Factor scores, CCK	✓			✓		✓
Blanchard et al46				✓	✓	✓
Fossati et al47 study 1
Interview data		✓				✓
Self-report data		✓				✓
Fossati et al47 study 2		✓	✓			✓
Fossati et al47 study 3		✓	✓
Haro et al48		✓	✓✓✓		✓	✓
Murray et al49	✓	✓	✓✓✓	✓	✓	✓
Boks et al50	✓	✓				✓
Schmitz et al51 primary		✓		✓		✓
Schmitz et al51 replication		✓		✓		✓
Shevlin et al52,53 NCS
Hybrid IRT-LCA	✓		✓
LCA	✓	✓		✓
Shevlin et al52 NEMESIS	✓		✓

Note: LCA, latent class analysis; CCK, coherent cut kinetics; NCS, National Comorbidity Survey; NEMESIS, Netherlands Mental Health Survey and Incidence Study; IRT-LCA, item response theory-latent class analysis

By far, the most substantial problem affecting this body of work is the application of statistical methods that are biased in favor of finding latent classes. Given correlated indicators and assuming conditional independence (a key assumption of LCA that is discussed in the Supplementary Online Material [pS2,S4]), LCA will result in the rejection of the 1-class (dimensional) model and identification of at least 2 classes, regardless of the true latent structure.¹⁷ An equivalent problem exists with factor analysis. Factor analysis of correlated indicators will result in the identification of at least one factor, even though the correlation may be entirely attributable to a latent class structure. Indeed, there are several cohorts included in table 5 for which both dimensional and class interpretations of the same or equivalent data have been published (eg, Jørgensen and Jensen,³² Peralta et al,⁴³ Peralta and Cuesta,⁴⁴ Murray et al,⁴⁹ Jørgensen and Jensen,⁵⁵ and Peralta and Cuesta⁵⁶). The primary consequence of this class-finding bias is that LCA results, if not obtained concurrently with hybrid LVM or CCK results, cannot be construed as evidence that the latent structure is not dimensional.

Second, the key result from modeling methods is not the absolute quality or fit of the final model. Instead, the key result is how the final model (eg, of K classes) compares with the null model (K = 1 class) and to potential competing models in light of substantive considerations. In many of the studies, one or more of the null or adjacent models (ie, K − 1 classes, K + 1 classes) was not explicitly examined. Consequently, in these cases, it is unclear whether the reported interpretations are the best solutions given the available data. (A related but noncritical problem with an even greater number of studies is the failure to report results of the full range of models that were evaluated.)

Third, in many instances, the analysis approach required the assumption of within-class independence among the class indicators. (As discussed in the Supplementary Online Material, this assumption is met where the correlations among indicators are zero within each class [pS2].) Failure of this assumption is a significant threat to the validity of findings from LCA and latent regression analysis, whereas CCK methods can accommodate small to moderate conditional dependence.⁷ In a large number of studies, adherence to the assumption of conditional independence went unchecked. Yet, in some studies, it seems highly likely that this assumption was violated—as evidenced by significant overlap in the content of items included in analyses (eg, Nestadt et al³⁶).

Fourth, problems with conditional dependence are sometimes addressed with preliminary analyses during the indicator selection stage (eg, Tyrka et al³⁷ and Haro et al⁴⁸), along with other aspects of indicator quality, such as item validity and multidimensionality of measures. However, indicator quality received attention, albeit sometimes very little, in only 11 of the 38 analyses. There were several related problems. The presence of both positive and negative correlations among indicators (eg, Cuesta et al⁴⁵) makes the indicator set inappropriate for analysis with CCK methods and, if analyzed, renders the result largely meaningless. Indicator parcels, created by combining data from multiple items (eg, Parker et al³³), are likely to be less sensitive to latent structure than the constituent items.⁵⁷ Lastly, use of item sets that are multidimensional can lead to overextraction of classes with LCA.⁵⁸

Fifth, CCK and modeling methods are not intelligent systems. Rather, CCK methods are tools for detecting taxonic anomalies in outcome variables, and modeling methods are tools for recreating variance on the basis of mixture parameters. The principal consequence of this fact is that these methods detect or attempt to recreate, respectively, whatever may be present in the observed data. Therefore, recruitment and sampling procedures that contribute systematic error to observations will, at a minimum, obscure the true latent structure and, when more extreme, may create artifactual latent class structures. One critical source of systematic error variance prominent among the reviewed studies was the use of commingled samples. In 7 instances, cohorts were constructed by combining participants from 2 or more sources or recruited using 2 or more methods, such as from different countries,⁴⁸ different clinics,⁴⁶ and different referral sources^35,49 or subpopulations.^37,38,40 Systematic differences among commingled groups hinder detection of the true latent structure in several ways. First, evidence of taxonicity may be obscured or go undetected if the samples that are combined differ both in the magnitude of scores on indicators and, assuming latent taxonicity, in the latent structures (eg, where, eg, the prevalence of a latent class is not equal across the combined groups). Second, given evidence of taxonicity, the estimated prevalence of the latent classes will be inaccurate if the latent structures of the combined groups differ. Third, latent classes may be overextracted if the mean difference among samples adds sufficient variance to the combined data.

Some have suggested that systematic error from subsample differences may be removed prior to analysis, eg, by standardizing scores within subsamples before combining data.⁵⁹ Similarly, others have justified combining data on the basis of the absence of significant group differences on demographic and clinical measures.³⁵ However, these solutions will only lead to improved resolution of the taxometric evidence if the latent structures of the combined samples are the same. Neither of these solutions will result in models that generalize beyond the study where there is reason to believe that the latent structure differs across subsamples. In such situations, the only appropriate solution for generating generalizable results is to conduct parallel analyses within homogeneous subsamples.

Sixth, a second critical source of systematic bias was the imposition of arbitrary constraints on the sample population. This occurred frequently (table 2). For example, including only or excluding those who meet DSM-III-R criteria for schizophrenia or a related disorder places an arbitrary suppositional constraint on latent structure outcomes. The consequences of such constraints are unclear but may include under- or overextraction of classes, depending on where the criteria fall with respect to the true latent structure, or misspecification of the prevalence rates of latent classes. In one study of schizotypal personality disorder, patients meeting criteria for schizophrenia, schizoaffective or schizophreniform disorder, or delusional disorder were excluded but not those meeting criteria for psychosis not otherwise specified.⁴⁷ In another, excluding patients with affective disorders might have created a point of rarity.¹³ Ironically, such criteria are antithetical to the well-worn objective that many borrow from Plato, “to carve nature at its joints.” Although clearly not intended, such constraints imply that the true latent structure does not extend beneath these excluded groups or that the exclusion criteria are natural latent boundaries.

Measurement Uncertainties

Just as sample recruitment and screening methods can introduce artifacts, so too assessment methods and indicator construction. First, the modal method of assessment of indicators involved interviewer ratings. In many instances, these were made in the course of clinical assessment or in the context of diagnostic interviews where it may be expected that the raters’ conceptualizations of psychopathology, including diagnostic boundaries, or the imperatives of the assessment device introduced perceptual or rating biases. Although useful in some contexts, such biases have been demonstrated to modify the latent structure of data.⁶⁰ Thus, notwithstanding the limitations undermining evidence validity, the principal uncertainty here is whether observed class boundaries appear to align with established classification boundaries not because these are where true latent boundaries of psychopathology lie but because of convention in assessment practices.

Expanding on this point, although the clinical interview is widely regarded as yielding better quality data than other modes of assessment, it has unique disadvantages in taxometric analysis. First, as Strauss and others note, the practice of forcing reported experiences into presence or absence ratings is inconsistent with evidence that psychotic experience exists on a continuum.^5,61 Second, information obtained with other methods captures variability attributable to a wider range of potential etiological processes,⁶² perhaps because it is not subjected to perceptual biases. Third, one cannot avoid perceptual biases affecting observer ratings, even when observers are aware of the potential for bias.^63,64 These limitations create a similar uncertainty about the origin of structures identified with taxometric methods.

Third, with few exceptions, studies utilized a single mode of assessment, such as clinical ratings, rather than multiple modes of assessment (eg, clinical ratings plus indicators from self-report, accuracy or speed of task performance, and ratings on circumscribed tasks). A potential disadvantage of this practice is that method variance—ie, the degree to which resemblance among the methods used to assess indicators contributes systematically to variance in those indicators—may contaminate modeling solutions.⁶⁵ Method variance is not negligible.⁶⁶ Consequently, faced with evidence of a single continuous structure detected with LVM (eg, Shevlin et al⁵²), uncertainty will exist as to whether the structure is clinically interesting or a method artifact.

Fourth, item parceling refers to the practice of combining data from multiple items into a single indicator. This can be undertaken as an explicit process but is also effectively achieved when information from multiple sources is combined in a single assessment measure. The hazard associated with item parceling is that important variability in single items is countervailed when these are combined. More importantly, research demonstrates that latent models obtained with individual items can differ from those obtained from parcels made of those items.⁵⁷ It is unclear whether or to what degree the practice of item parceling in the reviewed studies may have affected model selection.

Relevance Limitations

Finally, other characteristics limit the relevance of some studies to our objectives. These include that few of the indicators included in the analyses represent Criterion A (–like) symptoms (eg, Parker et al,³³ Goldstein et al,³⁵ and Haro et al⁴⁸), that the indicators are of a narrow set of Criterion A symptoms (eg, Jørgensen and Jensen,³² Bell,³⁸ Shevlin et al,⁵² and Shevlin et al⁵³) and that the study is not primarily about the schizophrenia spectrum (eg, Parker et al³³ and Haro et al⁴⁸).

Structural Validation Vs Class Differences

In general, considerable attention was given to the validation of the latent structures that were identified. However, the standard of evidence accepted as validating class outcomes was generally low. Often, the standard that has been used is demonstration of single or double dissociations of concurrent measures or outcomes across classes. Single dissociations occur when outcome 1 is associated with class A but not class B; double dissociations occur where, in addition to the single dissociation, a second outcome, outcome 2, is associated with class B but not class A. The presence of a single or double dissociation is often interpreted as evidence that there is more than one intervening causal process. Unfortunately, as Dunn and Kirsner⁶⁷ have demonstrated, single and double dissociations are not logically inconsistent with single process or dimensional explanations. Instead, the rejection of single-process accounts on the basis of such dissociations requires additional assumptions about plausibility of single-process models and the selective influence of classes on outcomes. Thus, although single and double dissociations may provide evidence of quantitative differences among resultant groups, these types of dissociation do not necessarily validate a latent structure.

A better type of dissociation for validating structure involves demonstrating that 2 outcome variables obtained across classes are not monotonically related. A monotonic relationship is one in which outcomes are never negatively related (ie, 0 ≤ r ≤ 1) or never positively related (ie, −1 ≤ r ≤ 0). An important limitation of monotonic relationships is that it is always conceivable for a single process to account for both outcomes; a monotonic relationship is not logically inconsistent with a single disease process explanation.⁶⁷ In contrast, a nonmonotonic relationship is present when at some point the association between the outcome variables changes from being positive to negative or vice versa. Dunn and Kirsner⁶⁷ reason that such nonmonotonic or reverse associations are logically inconsistent with a single-process model given just one assumption. Specifically, if one assumes that a disease process has a monotonic output function, the presence of a nonmonotonic association among outcomes suggests that 2 or more disease processes are involved. In the context of validating latent class structures, reverse associations provide more compelling grounds for rejecting a dimensional latent structure because the nonmonotonicity implies 2 or more processes are in operation. In addition, if present, reverse associations also validate quantitative differences among groups. (The Supplementary Online Material [pS14–S15] demonstrates evidence of reverse associations among classes identified by Kendler et al.⁴⁰)

In summary, robust validation requires both dissociation (ie, difference) as well as reversal of association across 3 or more classes (reverse association cannot be demonstrated with only 2 classes). Consequently, it seems reasonable to expect that not all outcome measures will be useful for validating class structure. For example, there is little to be gained from demonstrating reverse association between 2 outcomes that are already known to depend on disparate processes (eg, auditory verbal learning and eye tracking or eye tracking and employment). In contrast, the outcome variables that strongly validate a latent class structure will be those that, at the outset, are thought to depend on a single process but that turn out not to. In other words, a latent class solution is validated when it leads to a new revelation about outcome.

Discussion

Taken at face value, CCK and LVM studies of Criterion A features of schizophrenia provide reason to question the adequacy of dimensional models of schizophrenia. In some cases, the class boundaries appear to fall along lines corresponding to the divisions of psychopathology in DSM-IV. Other cases suggest isolated Criterion A features are also sensitive to class boundaries. Still others suggest boundaries consistent with theoretical distinctions among neurodevelopmental and affective psychoses or between deficit and nondeficit forms of schizophrenia.

However, the great majority of this research is limited by significant threats to the validity of the findings. The primary consequence of these limitations is that this evidence does not present a serious threat to the notion that Criterion A features are continuously distributed within the general population.

Indeed, this body of research does not provide, collectively, a reasonable test of the hypothesis that the population distributions of Criterion A symptoms of schizophrenia are dimensional. Nor is it the case that the findings reviewed here are consistent with a dimensional viewpoint. Therefore, although these studies provide for hypothesis generation, it would be premature to consider potential theoretical or clinical implications of this evidence, such as the meaning that should or should not be attached to the latent structures that are identified.⁶⁸

The generalizability of this conclusion is constrained by the inclusion and exclusion criteria applied to studies in the review. Although these criteria served the focus on Criterion A symptoms, we consequently overlook an important body of CCK research. This research is predominantly of features of schizotypy (as distinct from schizotypal personality disorder) in school children and undergraduates and has relied on nonclinical assessment instruments, such as the Chapmans’^69–72 scales (eg, Blanchard et al,⁷³ Horan et al,⁷⁴ Korfine and Lenzenweger,⁷⁵ Lenzenweger,⁷⁶ Lenzenweger and Korfine,⁷⁷ Meyer and Keller,⁷⁸ and Rawlings et al⁷⁹), the Thinking and Perceptual Style Questionnaire (eg, Linscott⁸⁰ and Linscott et al⁸¹), and others (eg, Fossati et al⁸²), or the assessment of non–Criterion A variables (eg, Tyrka et al³⁷ and Erlenmeyer-Kimling et al⁸³). The findings reported in this body of research largely favor a taxonic view of schizotypy and, by implication, the liability for schizophrenia,⁵⁹ although this interpretation is not universally held.⁸⁴ Thus, as pointed out by one reviewer, consideration of a broader range of studies may provide grounds for an alternative conclusion.

It is also important to acknowledge that much of the research included in the review was at the frontier of taxometric investigations of schizophrenia. Whereas CCK and classical LCA methods have been available since the 1970s, even today CCK methods are not readily implemented, and the developments that allow for testing of sophisticated mixture models are also very recent. For both types of analysis, the boundary conditions governing their use—statistical power, indicator psychometrics and item parceling, model complexity, controlling for sample biases, appropriateness of fit indices—are not well tested. Equally, multiple methodological issues specific to schizophrenia research also need to be addressed.

Primary Questions Worth Testing

Schizophrenia is broader and more conceptually rich than what is captured descriptively in Criterion A symptoms. There are 2 primary taxometric questions about its distribution. First, just as psychotic experience falls on a continuum,^61,85 does schizophrenia exist as part of a single distribution of normality? Second, is schizophrenia truly a group of schizophrenias,⁸⁶ with taxonic divisions separating its types?

Consider the first question. One ostensible challenge to attempts to address question is the raft of schizophrenia-like disorders or states or traits that are distinguished in clinical practice (eg, substance-induced psychosis, schizotypal personality disorder, prodromal symptoms) but which may obscure evidence of a latent boundary. However, this presents a difficulty only if one takes a narrow view of schizophrenia as defined by the diagnostic criteria that demarcate it from these related entities. Instead, this first question requires a much broader concept of schizophrenia. For example, evidence from Tyrka et al³⁷ and others⁸³ suggests that there is a taxonic liability for schizophrenia that has a prevalence that is much greater than the rate of the schizophrenia diagnosis. This is consistent with evidence from many but not all taxometric studies of schizotypy.^{74–77,79–82,87} Taking a view of schizophrenia that is not only constrained to the DSM-IV definition but that also accommodates both incipient processes and benign outcomes, these findings pose a potential challenge to a fully dimensional view of schizophrenia. Specifically, the findings suggest that there is a nonarbitrary boundary that lies well within the range of nonpathological functioning. If substantiated using indicators, designs, and methodologies that address the limitations described above, such evidence would seriously undermine the notion that a single distribution of normality fully embraces schizophrenia.

If there is an affirmative response to the first question—a single distribution of normality embraces schizophrenia entirely—there is no merit in attempting to investigate the second. This notwithstanding, the second question is not so straightforward to address for several reasons. First, the theoretical models proposing subgroup of the schizophrenias are unlike the conceptual pigeonhole model implied by schizophrenia-spectrum classifications specified in DSM-IV. Indeed, this fact arouses suspicion about the theoretical significance of some of the latent variable models that suggest 6 or 5—perhaps even 4—classes. Instead, theories typically distinguish 2 or 3 theoretical syndromes (eg, Kraepelin's dichotomy, deficit vs nondeficit, reality distortion, disorganization, and negative), 2 ends of a continuum (eg, neurodevelopmental vs affective psychosis), or specify quantitative or qualitative transitions (eg, stress or sensitization, onset of psychosis) or a qualitative outcome (eg, need for care). Second, it is not clear that the indicators that may distinguish one class from a second will necessarily be suited to distinguishing these from a third. Indeed, it may be that research that addresses the first primary question—about whether a single distribution of normality fully embraces schizophrenia—identifies nonredundant (mutually exclusive, stochastically independent, or synergistic) classes depending on the indicators that are used. There is tentative evidence suggesting this is the case for schizotypy.⁷⁴ Conversely, indicators that are sensitive to 3 or more classes, such that classes differ on the indicator in a quantitative fashion, may yield ambiguous evidence. Fourth, addressing the question should properly involve an iterative taxometric mapping process—repeating the sort of work required for the first primary question for each of the boundaries proposed to exist among the group of schizophrenias.

Design and Analysis Issues

Thus, whether the population distribution of the criterion symptoms of schizophrenia is taxonic remains to be adequately addressed. The limitations of the research reviewed here, and consideration of what is known about taxometric methods, give rise to several design and analysis issues or suggestions that should be considered when attempting to address this question.

Pit Dimensional Hypotheses Against Categorical Ones

Dimensional and categorical interpretations should be permissible given the analysis method. CCK methods allow this—at least, to the degree that one can reject a null hypothesis corresponding to a dimensional latent structure.^59,88 A more liberal view is that CCK methods allow one to positively confirm either structure.^68,84 Also, generalized LVM allows for either interpretation in this latter liberal sense. In contrast, classical LCA, latent profile analysis, and exploratory and confirmatory factor analysis do not.

Modeling Methods Model; coherent cut kinetics Test

One of the main limitations of mixture modeling is that modeling does not necessarily reveal a portrayal of the true latent structure.^{17,18,40,58,89,90} The reason for this is very simply that modeling involves composing a mixed distribution from simpler constituents that may or may not coincide with true latent components but which, in any case, provide a mixed distribution that closely resembles the observed or manifest distribution. Neither substantive reasoning nor fit statistics can discern the verisimilitude of the model. Also, it is the case that distinctly different models can provide equally good fit to a single dataset.⁸⁹ In such circumstances, substantive reasoning is the only recourse.

In contrast, CCKs do not involve modeling. Instead, these methods involve looking for telltale anomalies within data that are inconsistent with a dimensional latent structure. Whether these anomalies are present or absent is the primary judgment required. Of course, there are limitations associated with judging the shape of a line, and substantive reasoning also enters into interpretation of CCK results. But the anomalies are not the product of a modeling process. Hence, modeling methods model, whereas CCK methods are the only methods that provide a test for the notion that the latent structure of an indicator set is continuous.^7,91

In Analysis, More Is More; With Model Complexity, Less Is More

CCK and latent variable methods each have disadvantages. The disadvantages of CCK methods include that these depend on judgments of qualitative features of covariance curves or other graphical output, these require multiple consistency tests that also do not have conventional quantitative thresholds (cf P < .05), these cannot be used to distinguish simultaneously among three or more latent classes, and these require indicators that are monotonically related within the mixed sample. The disadvantages of latent variable methods include that these cannot distinguish among alternative models that have similar descriptive power, these only yield results for models that researchers select for analysis, and these also depend on fit indices that are not well validated.

Addressing these limitations is challenging. To some degree, the burden of these limitations depends on the level of methodological complexity the researcher chooses to adopt. For example, parsimony is frequently considered when judging the outcomes of LVM. However, it is equally important to consider the merits of pursuing complex vs simple questions about latent structure. There is some merit in attempting to model the latent structure of broad sets of indicators, eg, spanning affective and psychotic disorders. Such analyses may provide an overview of relationships and possible class divisions and lead to specific hypotheses that can be tested. However, a critical test for the models that result from such analyses is whether the classes so identified can withstand the test provided by CCK methods.⁸⁸ If 2 classes cannot ultimately be distinguished using CCK methods, uncertainty remains about whether the modeling solution is only ostensible.

Because each statistical approach has both advantages and disadvantages, each has a role in research into the latent structure of schizophrenia. It is neither desirable nor necessary to rely solely on LVM or CCK. Rather utilization of both should be preferred. As Meehl⁶⁸ noted, when circumstances are appropriate, both approaches should yield the same result, as was the case in the simulations described in the Supplementary Online Material (pS9–S14). Also, scrutiny of simple refutable divisions between pairs of classes with CCKs can lead to stronger inferences about latent structure than modeling of broad areas of psychopathology.^88,92

Indicator Selection Is of the Greatest Importance

From the inception of modeling and CCK methods up to the present day, indicator selection is of utmost importance.^{7,11,13,54,83} No modeling or CCK study should be conducted, let alone published, without systematically evaluating indicators to prevent inappropriate indicators from remaining in the final modeling or CCK analysis. In many instances, it may be appropriate for the analysis strategy to be applied iteratively, using the procedures to identify poor indicators that are removed for subsequent analyses. There may also be ways of testing the validity (separation) of individual indicators prior to analysis, such as by using between-group comparisons. However, if preanalysis validity estimates are based on classifications that are under scrutiny (eg, schizophrenia vs bipolar disorder), the validity estimate rests on the assumption that the classifications are valid. In such cases, it is quite satisfactory, and certainly more defensible, to obtain bootstrapped estimates of validity in the course of CCK analysis.¹¹

The possibility of relaxing the conditional independence assumption in modeling is a great advantage provided there is a sound theoretical reason for doing so. Given the potential for interpretative problems to arise from common method variance, the independence assumption should perhaps only be relaxed where indicators are not derived using the same methodology. In any case, it is advisable that indicators submitted to CCK methods adhere as much as possible to the assumption of conditional independence. Thus, the conditional independence assumption is required for a comprehensive analysis strategy.

Select Indicators Spanning Multiple Modes and Levels of Analysis

The psychometric limitations and perceptual biases that affect clinical rating scale data are insurmountable in a rating paradigm.⁶⁰ Clinical ratings scale data should not be abandoned, but there is much to gain from procedures that include such ratings as part of a battery of indicators spanning multiple modes and multiple levels of observation (eg, self-report, cognitive performance, neurological signs, structural and functional neuroanatomy, neurophysiological markers).⁹² Also, although psychiatric interview is the modal assessment strategy, the limits of the data provided by this method—namely, dichotomous or trichotomous ratings—strongly argue against their use.

Theoretical accounts of schizophrenia identify core causal mechanisms or processes that operate at a fundamental level yet produce symptoms and signs at higher neurophysiological, cognitive, psychological, and social levels (eg, Andreasen⁹³ and Meehl⁹⁴). By restricting focus to just one high level of analysis (eg, psychological functioning assessed using a psychiatric interview), the array of potential explanations that must be considered becomes diffuse and variable: the methodological limitations considered above, basic psychometric properties of measures, the influence of nonspecific risk factors, as well as the core causal agent. By restricting focus to one lower level of analysis (eg, eye tracking or brain torque), the relevance of findings to the clinical phenotypes must rest on assumptions or reasoning.^68,94 In contrast to these 2 situations, if analyses of multilevel multimodal indicators yield consistent evidence of a clear latent structure, the multilevel and multimodal nature of the indicators affords a much stronger basis for interpreting the meaning of that structure. This approach also exposes theoretical models of schizophrenia to much more rigorous evaluation than tests of single level indicators.

Validating Class Structure

Class structures identified with LVM or CCK methods need to be validated using independent measures. The objective of the validation process is to establish that a single underlying process cannot explain outcomes on independent measures taken across classes. It is interesting to find single or double dissociations, but these, unlike reverse associations, do not attest to the potential latent structure of the groups. In the absence of reverse associations, the best way of validating class structure is, as argued above, demonstrating convergence of evidence from both modeling and CCK procedures.

Assuming a class structure is identified, validation of that structure does not imply that the problem of the classification of psychosis or schizophrenia has been solved. First, the crux of the classification problem is the absence of an identifiable etiology.⁹⁵ These statistical methods do not redress this limitation. Secondly, once validated, a class does not necessarily correspond to a specific etiology.^94,96 Thirdly, class structures identified with CCK or latent variable methods are not self-interpreting; the meaning that is applied to these classes is independent of the statistical evidence.⁶⁸

Conclusions and Future Directions

Much of the research included in this review was at the frontier of taxometric investigations of criterion symptoms of schizophrenia. However, fundamental limitations in statistical methods, measurement, and design suggest the face-value interpretations of reported findings are largely undermined if not unfounded. There is a critical dearth of well-conducted and well-analyzed studies of clinical symptoms of schizophrenia, studies using CCK methods, studies using generalized LVM, and studies using both. Consequently, the available evidence provides no serious challenge to the single-distribution model of schizophrenia nor is the evidence consistent with this viewpoint. The exciting thing is, however, that a serious test of this hypothesis is well within means.

Beyond issues concerning DSM-V, the question of whether schizophrenia exists as part of a single distribution of normality appeals as the most important question for research in the immediate future. Investigations into this question will necessarily involve large representative community samples of young adults and the assessment of multiple indicators. Given evidence on the prevalence psychotic experience,⁸⁵ sample sizes in the order of 1000–5000 will likely be required. Ideally, indicators will span a broad range of clinical-behavioral phenotypes and cognitive, neurophysiological, and neuroanatomical endophenotypes. Following our recommendations above, indicator data would be screened for suitability and subjected to LVM and CCK methods. Assuming class solutions are found, validation will depend on prospective measures of outcome.

Supplementary Material

Supplementary material is available at http://schizophreniabulletin.journals.org.