Exploring boundaries of schizophrenia: a comparison of ICD-10 with other diagnostic systems in first-admitted patients

The introduction of operational criteria for clinical and research diagnosis in psychiatry was motivated by alarmingly different diagnostic habits among British and American psychiatrists (1). With respect to schizophrenia, at least 15 different diagnostic systems may be identified in the literature over the last three decades (2). The operational approach in psychiatry, considered by many as a progress in the right direction, is also being increasingly criticized for serious epistemological shortcomings and for a number of negative pragmatic consequences for psychiatry as a profession (3-7). "The [operational] diagnosis has almost become a thing in itself - a certainty of 'concrete dimensions' [...] 'allegedly' being more data based, has even assumed the aura of allowing psychiatry to keep pace with the rest of medicine as a 'technological triumph'" (5). Thus, although the contemporary DSM and ICD criteria for schizophrenia are a convention of unknown validity as compared to their potential rival definitions (see 8 for an extensive discussion), these criteria are being increasingly considered by the residents and the younger colleagues as the criteria which somehow must reflect the true nature or essence of this disorder. Needless to say, the official diagnostic criteria exert powerful censoring effects on the editorial and funding policies.

The purpose of this study was to revive the issue of boundaries of schizophrenia by comparing the ICD-10 criteria to other diagnostic systems. Special attempt was made to assess the concurrent validity of ICD-10 and ICD-9. The study is a part of an ongoing prospective followup investigation of first-admission cases (The Copenhagen Prodromal Study).

METHODS

The sample consisted of 155 patients with age <40 years, consecutively first-admitted to the Department of Psychiatry at Hvidovre Hospital (catchment area: 130,000 inhabitants of the city of Copenhagen) during the period from September 1, 1998 to September 1, 2000. Patients who suffered from clear-cut affective disorders, organic disorders or severe substance abuse as a primary diagnosis, or as a clinically dominating secondary diagnosis, were not included. Severely psychotic and aggressive, or involuntarily admitted patients were not included, due to ethical concerns or because they were considered unable to undergo the full examination. The sample, according to the clinicians in charge, included clearly psychotic patients (approximately one third) as well as patients regarded as non-psychotic, yet possibly within the schizophrenia spectrum.

All patients received a 3-5 hour interview, consisting of the Operational Criteria Checklist (OPCRIT [9]) expanded with several items used in the Copenhagen High Risk Study (10), the Danish version of the Bonn Scale for the Assessment of Basic Symptoms (BSABS [11]), the Positive and Negative Syndrome Scale (PANSS [12]), the Premorbid Adjustment Scale (13) and the Global Assessment of Functioning (GAF [14]). This was supplemented by a 5-6 page typed summary of premorbid functioning, social history and detailed descriptions of psychopathological features, illustrated by verbatim quotes. All interviews were performed by a senior clinician (PH), trained in Germany in the use of the BSABS by the authors of the scale, and with extensive research interviewing experience from a genetic linkage study. Both the interviewer and another senior clinician, with research interview experience from the Danish- American adoption studies (LJ), completed a checklist comprising all single symptoms and other criteria required by several diagnostic systems: the St. Louis criteria (15), the Research Diagnostic Criteria (RDC [16]), the Flexible System, narrow and wide (17), the Vienna Research Criteria (2,18), the DSM-IV, the ICD- 8/9, and the ICD-10. The ratings were made on the basis of all materials available on each patient (i.e., the interview summary, chart notes by the staff clinicians and second informant interviews). Whenever a disagreement in the rating occurred, a consensus assessment of the symptom in question was performed by three clinicians (PH, LJ, JP). Only the consensus ratings are used here for data analyses. Schizophrenia diagnosis was generated by computerized algorithms written for each diagnostic system. For each patient, a family history of mental illness was obtained (19). All patients were tested (by JN) with the Thought Disorder Index (TDI [20,21]), which is a Rorschach-transcript based, quantitative and qualitative assessment of different aspects of formal thought disorder, subsequently scored by a group of psychologists trained and regularly supervised by Philip Holzman from Harvard University (one of the creators of the TDI). The inter-rater correlation of the total TDI scores between two independent raters performed on a random sample of 10 transcripts was 0.82 (p< 0.01) (22). The TDI responses were factor analyzed, resulting in two factors: one measuring severe formal thought disorder, comprising absurd responses, idiosyncratic symbolism, autistic logic and confabulation ('state' formal thought disorder), and a second factor, referring to trait-like semantic distortions ('trait' formal thought disorder), such as inappropriate distance, vagueness, peculiar verbalizations and queer responses.

We applied analysis of variance with the continuous variables as response, and the binary ICD-9 and ICD-10 schizophrenia diagnostic status as explanatory variables, and compared the parameter estimates for ICD-9 and ICD-10 schizophrenia by means of a likelihood ratio test size evaluated in a chi square distribution. Normality of the residuals from these models was checked both with the Shapiro-Wilks test and with a diagram of fractils. When both the test (p<0.05) and the plot demonstrated that normality could not be assumed, a transformation of the response variable with the logarithm or the square root function was tried and rechecked for normality. If the transformation worked successfully, the results from this model is presented. If normality still could not be assumed, we dichotomized the response variable into low and high scores (into, as far as possible, equally sized groups, Table 1). In this case (and for all originally binary responses, i.e. sex, marital status, family history), a binary logistic regression model was tested with the dichotomized (binary) outcome variable and ICD-9 and ICD-10 diagnostic status as explanatory variables, with a comparison of the estimates (odds ratios) for ICD-9 and ICD-10 schizophrenia. The odds ratios for the reference groups - non- ICD-9-schizophrenia and non-ICD-10-schizophrenia - were set to 1.00. PROC GENMOD in SAS 8.2 was applied for the analyses.

Table 1.

Descriptive characteristics of ICD-10 and ICD-9 combinations

Variable	Groups	Non-schizophrenic	Schizophrenic	Schizophrenic	Schizophrenic
		according to both	according to	according to	according to both
		ICD-9 and ICD-10	ICD-10 but not	ICD-9 but not	ICD-9 and ICD-10
			ICD-9	ICD-10
Familiar	Predisposed to schizophrenia	4	0	11	9
predisposition	Not predisposed	59	0	43	26
‘Trait’ formal	High score: 6-43 points	28	0	34	16
thought disorder	Low score: 0-5 points	38	0	16	18
‘State’ formal	High score: 2-20 points	22	0	23	14
thought disorder	Low score: 0-1 points	44	0	27	20
Sex	Male	25	0	19	17
	Female	41	0	35	18
Marital status	Married/cohabitating	40	0	25	12
	Single	22	0	27	22
PANSS	High score: 13-29 points	11	0	22	34
positive symptoms	Low score: 7-12 points	51	0	32	1
PANSS	High score: 13-31 points	13	0	31	29
negative symptoms	Low score: 7-12 points	52	0	20	6

RESULTS

Polydiagnostic assessments

The results of the polydiagnostic assessment of all 155 patients appear in Table 2, which indicates the numbers of patients with schizophrenia by each diagnostic system, numerical overlaps as well as the kappas of agreement between the systems. The ICD-10 simple schizophrenia category is considered separately, because it deviates substantially from the main ICD definition of schizophrenia. The ICD-10 (without the simple category) appears as the most conservative (n=35) and the ICD-9 as the most liberal (n=89) system. All ICD-10 schizophrenia cases, including the simple category, are covered by the ICD-9 definition. The best diagnostic agreement is observed between ICD-10 and DSM-IV (kappa 0.823) and RDC and DSM-IV (kappa 0.769). There are non-trivial differences between even rather conservative systems: e.g., 10 (28%) of the ICD-10 schizophrenia cases are not so diagnosed by the St. Louis criteria, whereas 18 (34%) of the RDC schizophrenia cases are not so diagnosed by the ICD-10.

Table 2.

Numbers of individuals and kappa agreements (in brackets) between different diagnostic criteria for schizophrenia in a sample of 155 first admissions

	ICD-9	Flexible	Research	Flexible	Vienna	DSM-IV	St. Louis	ICD-10	ICD-10
		System,	Diagnostic	System,	Research		criteria		Simplex
		wide	Criteria	narrow	Criteria
ICD-9	89
Flexible System, wide	71	84
	(0.595)
Research Diagnostic	46	45	52
Criteria	(0.397)	(0.422)
Flexible System,	52	52	36	52
narrow	(0.545)	(0.598)	(0.537)
Vienna Research	40	41	24	34	47
Criteria	(0.317)	(0.388)	(0.244)	(0.541)
DSM-IV	37	36	38	31	20	39
	(0.351)	(0.368)	(0.769)	(0.553)	(0.262)
St. Louis criteria	36	34	29	29	21	26	37
	(0.353)	(0.345)	(0.517)	(0.517)	(0.318)	(0.582)
ICD-10	35	33	34	31	20	32	25	35
	(0.356)	(0.346)	(0.701)	(0.606)	(0.309)	(0.823)	(0.602)
ICD-10 Simplex	8	7	0	5	3	0	3	0	8
	(0.078)	(0.064)	(-0.098)	(0.085)	(0.023)	(-0.094)	(0.053)	(-0.092)

There are substantial differences in which patients are actually labeled as schizophrenic. Thus, similar numbers in the cells of Table 2 reflect, to a substantial extent, different individuals (see Figure 1 for illustration of distributions of individual patients in four selected systems). After excluding the ICD-10 simplex category, there are only 14 cases diagnosed as schizophrenic by all 8 systems and 108 patients diagnosed as schizophrenic by at least one of the systems.

Figure 1.

Numbers of individuals with schizophrenia diagnosis according to four selected systems

Comparisons of the ICD-9 and ICD-10

The inter-rater (PH vs. LJ) reliability for the ICD-10 diagnosis was higher than for the ICD-9 (kappa 0.855 vs. 0.505). The median inter-rater kappa for the single symptoms was 0.587 (incoherence) for the ICD-10 and 0.603 (autism) for the ICD-9. All the 35 patients with ICD-10 schizophrenia had different constellations of symptoms; among the 89 patients with ICD-9 schizophrenia, 61 different symptom combinations were observed. The ICD-10 diagnosis was made on the basis of the criterion 1 solely (Schneiderian symptoms, voices from the body and/or bizarre delusions) in 4 individuals (11%), on the sole basis of the criterion 2 (assortment of second rank and negative symptoms) in 6 individuals (17%), and by both criteria 1 and 2 in 25 individuals (72%). Statistical comparisons of the ICD-9 and ICD-10 diagnoses (without simple schizophrenia) on a number of concurrent characteristics appear in Table 3.

Table 3.

Statistical comparisons of ICD-9 and ICD-10

Binary logistic regression	Odds ratio in the ICD-9	Odds ratio in the ICD-10	p value for comparison
	schizophrenic group	schizophrenic group	of odds ratios
	(95% CI)	(95% CI)
Family predisposition to schizophrenia	3.77 (1.13-12.65)	1.35 (0.49-3.70)	0.273
High score for ‘trait’ formal thought disorder	2.88 (1.34-6.22)	0.42 (0.17-1.03)	0.008
High score for ‘state’ formal thought disorder	1.70 (0.80-3.63)	0.82 (0.34-1.98)	0.309
Male sex	0.89 (0.42-1.88)	1.74 (0.73-4.14)	0.343
Married/cohabitating	0.51 (0.24-1.08)	0.59 (0.24-1.43)	0.838
High score for PANSS positive symptoms	3.19 (1.37-7.44)	49.45 (6.29-388.5)	0.007
High score for PANSS negative symptoms	6.20 (2.71-14.19)	3.12 (1.09-8.85)	0.387
Analysis of variance	Parameter estimate in the ICD-9	Parameter estimate in the ICD-10	p value for comparison
	schizophrenic group	schizophrenic group	of parameter estimates
	(p value)	(p value)
Age at first psychiatric symptoms	-0.31 (0.777)	1.05 (0.424)	0.511
Global Assessment of Functioning (GAF) score	-8.28 (<0.0001)	-15.46 (<0.0001)	0.051

There are no significant socio-demographic differences, but the ICD-10 schizophrenics tend to be more frequently male, whereas the first psychiatric symptoms tend to appear earlier in life among ICD-9 schizophrenics. The ICD-9 schizophrenia status is associated with a nearly four-fold and statistically significant risk for having a positive family history of schizophrenia, whereas this is not the case for ICD-10. Only the ICD-9 schizophrenia diagnosis is associated with a significantly increased level of 'trait' formal thought disorder as measured by the TDI. Both ICD-9 and ICD-10 schizophrenia diagnoses are associated with decreased levels of functioning measured by GAF (more so for ICD-10) and increased PANSS symptom levels (ICD-10 dramatically so with positive symptoms, and ICD-9 mainly with negative symptoms).

A re-analysis of the comparisons between ICD-9 and 10 after expanding the ICD-10 with the simplex category did not change the overall picture, except for increasing the significance of the association between ICD-9 schizophrenia and family history (odds ratio for ICD-9 = 4.64; ICD-10 = 0.84), making the association between ICD-10 schizophrenia and living alone significant (odds ratio 0.41; 95% CI = 0.17-0.98), increasing the association between the PANSS negative symptoms and ICD-9 (odds ratio for ICD-9 = 6.75; ICD-10 = 1.96) and decreasing the association between ICD-10 and the PANSS positive symptoms (odds ratio for ICD-9 = 3.89; ICD-10 = 5.21).

Finally, we looked into potential differences among the ICD-10 schizophrenia cases when diagnosed by criterion 1 only, 1 and 2 simultaneously, and criterion 2 only. The sample sizes were too small for a reliable statistical analysis: yet, in the Kruskal-Wallis analysis of variance, the criterion 1 plus 2 patients had the lowest mean GAF score (30.68 vs. 35.00 for patients fulfilling only criterion 1 and 43.17 for patients fulfilling only criterion 2, p = 0.054), whereas the sole criterion 2 patients had the most frequent family history of schizophrenia (4 out of 6 vs. 1 out of 4 for patients fulfilling only criterion 1 and 4 out of 25 for patients fulfilling both criteria, p = 0.043).

DISCUSSION

In contrast to other polydiagnostic studies, which dealt with predefined psychotic patients only (23-25), this sample was psychopathologically more variegated, reflecting a broader range of first-onset psychiatric contacts and so perhaps being more generalizable to a standard diagnostic setting outside a closed ward. It is likely that the polydiagnostic studies restricted to predefined psychotic patients inflated the concordance between the investigated systems.

Each of the diagnostic systems examined in the present study has its own rationale and background (e.g., the Flexible System is derived from statistical analyses of a large body of psychopatological data in more than 1000 identically assessed patients from nine different countries) but no convincing evidence of a superior validity. The interdiagnostic kappas (Table 2) suggest that RDC and DSMIV, as well as DSM-IV and ICD-10, are close to each other, reflecting historical modelling sequences. There are, however, quite substantial differences among the systems with respect to how many and which patients are diagnosed as schizophrenic (Table 2 and Figure 1).

These findings re-emphasize the need for a continuing debate and research on the boundaries of schizophrenia. The obvious arbitrariness of the contemporary diagnosis poses a serious problem for etiological research (especially genetic linkage research) and early intervention studies (e.g., how many of the so-called 'pre-onset' patients in one diagnostic system are already 'post-onset' in another system?). The issues of validity and reliability of psychiatric diagnosis are frequently conflated in the literature, yet a demonstration of a high reliability of a given diagnostic system does not make that system valid. In the case of schizophrenia, we are confronted with a viciously circular difficulty: we do not possess robust extra-clinical markers to anchor the diagnosis, while at the same time unclear phenotypic limits impede the etiological research. In the presented comparisons of ICD-9 and ICD-10 (Table 3), only formal thought disorder and family history may be considered as concurrent/construct validity indicators, i.e. as extra-clinical measures, yet essentially pertinent to the disease concept as such (26). Formal thought disorder is intrinsically linked to the notion of schizophrenia, and the TDI is currently its most sophisticated measure (27). The view of schizophrenia as aggregating in families is as old as the disease concept itself (28). On both indices, the ICD-10 appears to be less valid than the ICD-9. This may be linked to what Kendler (26) calls a non-empirical aspect of validity; in this case the conceptualisation of the phenomenological essence of the disorder. Thus, the ICD-9 seems to be more oriented towards the fundamental features of schizophrenia, such as autism and self-dissolution (29,30). These aspects become diluted in the ICD-10, through its strong emphasis on the flamboyant psychotic phenomena, and further trivialized through the notion of the so-called negative symptoms.

We are currently starting a 4-year follow-up of this sample in order to assess predictive validity of the different diagnostic systems (the most frequent type of validity testing today) (25,31). Yet, temporal persistence of diagnosis (chronicity) is an epistemologically dubious validity indicator. Moreover, the original Bleulerian-Kraepelinian emphasis focused on the stability of the autistic trait phenomena rather than on the continuity of the state indicators (i.e. psychotic symptoms). Finally, in practical terms, chronicity at outcome is often more reflective of the illness duration recorded at the initial assessment than of specific symptomatological constellations (34).