Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 May 9.
Published in final edited form as: Compr Psychiatry. 2005 JAN-FEB;46(1):38–42. doi: 10.1016/j.comppsych.2004.07.016

Sensitivity of ICD-10 diagnosis of psychotic disorders in the Israeli National Hospitalization Registry compared with RDC diagnoses based on SADS-L

Mark Weiser a,b,*, Kyra Kanyas c, Dolores Malaspina d, Philip D Harvey e, Ittai Glick a, Deborah Goetz d, Osnat Karni c, Avi Yakir c, Neil Turetsky c, Shmuel Fennig f, Daniella Nahon g, Bernard Lerer c, Michael Davidson a,b
PMCID: PMC3089940  NIHMSID: NIHMS164412  PMID: 15714193

Abstract

Objective

The Israeli National Psychiatric Hospitalization Registry is a nationwide list of all psychiatric hospitalizations in the country and has been widely used as a source of data for psychiatric research. This study assessed the sensitivity of the diagnosis of psychotic disorders (International Statistical Classification of Diseases, 10th Revision [ICD-10] F20.0–F29.9) and schizophrenia (ICD-10 F20.0–F20.9) in the Registry.

Method

Registry discharge diagnoses of psychotic disorders (ICD-10 F20.0–F29.9) and schizophrenia (ICD-10 F20.0–F20.9) were compared with research diagnoses derived from best-estimate procedures based on Research Diagnostic Criteria (RDC) using structured clinical research interviews, hospital records, and family information.

Results

Out of 169 patients meeting RDC for psychotic disorder, 150 also had a diagnosis of psychotic disorders in the Registry, yielding a sensitivity of 0.89.

Re-running this analysis for the narrow definition of schizophrenia identified 94 patients who were diagnosed with schizophrenia using RDC; 82 of those patients also had a diagnosis of schizophrenia in the Registry, yielding a sensitivity of 0.87.

Conclusion

In 87% to 89% of cases with psychotic disorders or with schizophrenia, Registry diagnoses agreed with RDC diagnoses, a rate of agreement comparable with those of other, similar registries. Because a large number of analyses derived from this and similar national registries will be published in the coming years, this constitutes relevant information.

1. Introduction

Research using clinical databases can be a valuable source of information, and psychiatric hospitalization registries have been a particularly rich source of research data [1–5]. The Israeli Ministry of Health maintains a national psychiatric hospitalization registry that includes diagnostic information for all psychiatric hospital admissions in the country since 1950. This registry is a rich source of psychiatric data and has been used in many studies [6–19].

One obvious caveat that arises when using clinical registries for research purposes is the accuracy of the diagnoses that they contain. Because the diagnoses are made on the basis of signs, symptoms, and history and not as research diagnoses, concerns regarding the validity and reliability of the diagnoses are pertinent.

Because most patients hospitalized suffer from schizophrenia and psychotic disorders, and as most of the studies using the Registry focused on these disorders, the purpose of this study was to evaluate the sensitivity of the diagnoses of schizophrenia and psychotic disorders. This was done by comparing best-estimate research diagnoses (obtained using a structured diagnostic research interview and treated as the gold standard) with Registry diagnoses for 169 cases.

2. Methods

2.1. Subjects

The subjects were 169 patients who had been hospitalized in psychiatric wards throughout Israel, who as part of clinical research projects had undergone a structured research diagnostic assessment. The patients were drawn from 2 separate clinical samples, the first being a sample of Jewish Israeli patients that was cross-sectionally recruited from admissions to the Department of Psychiatry, Hadassah-Hebrew University Medical Center and from referrals from other centers. The second sample consisted of Arab Israeli patients who were systematically recruited from the rosters of regional mental health centers. Only those patients who had been hospitalized were included in the current study. The research diagnostic assessments were conducted by the research team of the Biological Psychiatry Laboratory, Hadassah-Hebrew University Medical Center as part of their ongoing studies on the genetics of major psychiatric disorders. The studies had been approved by the Helsinki Committee (Internal Review Board) of the Hadassah-Hebrew University Medical Center and all subjects gave written informed consent after the project had been explained to them [20]. Subjects were 112 men (mean age at interview = 34.5 ± 11.1 years, mean duration of illness = 10.0 years) and 57 women (mean age at interview = 37.5 ± 10.0 years, mean duration of illness = 10.7 years). The subjects were 89 Israeli Jews and 78 Israeli Arabs. Data regarding the ethnic group of 2 subjects were missing.

Subjects had been interviewed with the Schedule for Affective Disorders and Schizophrenia—Lifetime Version (SADS-L) [21] and were questioned about psychiatric symptoms in the family according to the Family History Research Diagnostic Criteria [22]. Medical records of hospitalizations and clinic care were obtained. The completed SADS-L interview form, Family History Research Diagnostic Criteria information, and medical records were reviewed by 2 psychiatrists according to a best-estimate procedure [23] and in cases where consensus was not achieved, the conflict was resolved by a senior psychiatric researcher (BL). Lifetime diagnoses were established according to the Research Diagnostic Criteria (RDC) [24] using a best-estimate consensus procedure [23].

Registry diagnoses were compared with research diagnoses twice: first the Registry's last discharge diagnosis was compared with the research diagnosis. The second comparison of Registry diagnoses with research diagnoses was done by comparing the research diagnosis with the Registry discharge diagnosis that was assigned to the patient in the hospitalization that was closest in date to the research assessment. Because of missing values of the dates of the research assessment, this comparison was done for 148 subjects.

After obtaining approval from the local Internal Review Board, the Israeli identification numbers (similar to American Social Security numbers) of the subjects who had undergone the SADS-L assessment and were assigned a RDC diagnosis were sent to the Registry, where their ICD-10 Registry diagnosis was obtained. Before the file containing the Registry diagnoses and the RDC diagnoses was transferred from the managers of the Registry to the investigators, the identification numbers were removed, thus keeping the identification of the subjects confidential. For those patients who had been hospitalized more than once, discharge diagnoses from all hospitalizations were obtained.

To compare the diagnoses of psychotic disorders from the Registry with the RDC research diagnoses, we took 169 patients who met the criteria for RDC diagnosis of psychotic disorders (schizophrenia, schizo-affective disorder or unspecified functional psychosis) and recoded their diagnoses into yes/no for having psychotic disorders. For these patients, the Registry diagnoses were also recoded as yes/ no for having psychotic disorders, using ICD-10 categories of F20.0 to F29.9. One patient from the Registry with a diagnosis of schizotypal personality disorder (ICD-10 F21) was recoded as “no.”

In addition, we compared the RDC diagnosis of schizophrenia with the Registry diagnoses of schizophrenia. The research diagnosis of schizophrenia included all patients who after a SADS-L interview met RDC for schizophrenia, and the Registry diagnosis of schizophrenia included patients with ICD-10 diagnoses F20.0 to F20.9.

2.2. Statistical analyses

We used the RDC diagnosis as the “gold standard” to which the Registry diagnoses were compared, and the measure of sensitivity is used as the index of agreement between the RDC and the Registry's diagnosis. The sensitivity of the diagnosis was calculated as the proportion of the patients who were diagnosed by RDC as suffering from psychotic disorders or from schizophrenia, who received the same diagnosis from the Registry.

To rule out the presence of bias related to the presence of different ethnic groups in the sample, we compared the sensitivity of the Registry diagnoses between Arab Israelis and Jewish Israelis using Pearson's χ2 test.

Nineteen patients were diagnosed as suffering from a psychotic disorder according to RDC, but had not received a diagnosis of psychotic disorder in the Registry. We compared these patients with those who were diagnosed with a psychotic disorder both using RDC, and in the Registry; differences in gender were compared using Pearson's χ2 test and age at first hospitalization using independent-sample t tests.

3. Results

Out of 169 patients diagnosed as suffering from psychotic disorders using RDC, 150 also had a last discharge diagnosis of psychotic disorders in the Registry, yielding a sensitivity of 0.89.

The data were reanalyzed for the narrow diagnosis of schizophrenia, defined as ICD-10 diagnoses F20.0 to F20.9. Ninety-four out of 169 patients were diagnosed as suffering from schizophrenia using RDC, and 82 out of 169 also had a Registry last discharge diagnosis of schizophrenia, yielding a sensitivity of 0.87.

Out of the 75 participants who were diagnosed as suffering from psychotic disorders but not with schizophrenia using RDC, 71 participants were diagnosed with schizo-affective disorder, and the 4 additional participants were diagnosed with unspecified functional psychosis.

The sensitivity of diagnoses was computed again for the hospitalization that was closest to the date that the research assessment was performed and yielded similar results: 0.89 for psychotic disorders and 0.90 for the narrow diagnosis of schizophrenia.

To rule out the presence of bias related to the presence of different ethnic groups in the sample, we compared the sensitivity of the Registry diagnoses between Arab Israelis (psychotic disorders, 0.93; schizophrenia, 0.88) and Jewish Israelis (psychotic disorders, 0.89; schizophrenia, 0.86). These differences were not significant (Pearson's χ2 = 1.45, P = .229).

Nineteen patients were diagnosed as suffering from a psychotic disorder according to RDC, but had not received a diagnosis of psychotic disorder in the Registry. Fourteen of these patients were diagnosed as suffering from affective disorders (ICD diagnosis F30), 4 from substance abuse disorders (ICD diagnosis F10), and 1 patient was diagnosed as suffering from a personality disorder (ICD diagnosis F60).

Compared with the entire cohort, these patients did not differ in their mean age at first hospitalization (25.07 ± 8.53 vs 24.67 ± 14.61, t = 0.175, P = .862) or gender distribution (31.6% female and 68.4% male vs 33.7% female and 66.3% male in the entire cohort, χ2 = 0.044, P = .833).

4. Discussion

The main finding of this study is that the diagnoses of psychotic disorders and schizophrenia which appear in the Israeli National Psychiatric Hospitalization Registry exhibit high sensitivity, reaching 0.89 for psychotic disorders and 0.90 for the narrow definition of schizophrenia. These results are congruent with research on the reliability of hospitalization Registry diagnoses from other registries. A UK-based study showed an 85% concordance between psychiatric case register and International Statistical Classification of Diseases, Ninth Revision, diagnoses [25]. A study based on the Swedish National Inpatient Registry yielded an 86% concordance with the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, criteria of the broadly defined schizophrenia syndrome and 76% for the “narrow” definition of the syndrome [26]. Isohanni et al studied the diagnostic reliability of the Finnish National Hospital Discharge Register and discovered a high percentage of true positives (87%–100%) [27]. A study very similar to ours [28] evaluated the agreement between discharge diagnosis and best-estimate RDC diagnosis. The sensitivity for diagnosis of schizophrenia was found to be 0.62 and the specificity 0.84. However, this study focused on patients in their first psychotic episode, from which diagnoses are known to change. Another study [29] comparing hospital diagnoses with research diagnoses reported sensitivity of 0.53 to 0.66 and specificity of 0.69 to 0.96 for schizophrenia, schizophreniform, and schizo-affective disorders. Thus, the accuracy of the diagnoses in the Israeli Hospitalization Registry seems to be very much in line with other clinical diagnoses.

4.1. Limitations and strengths

A study design that might be preferable to the design used in this study would take a random selection of patients diagnosed with psychotic disorders or schizophrenia in the Registry and diagnose them using RDC. However, there is no reason to think that the patients that had undergone the diagnostic assessments as part of other research protocols would be effected by any particular bias, and their mean ages and duration of illness are compatible with those of the chronic mental patients which comprise most patients in the Registry. Moreover, the results of this study point to the fact that the Israeli Hospitalization Registry is not affected by an ethnic group bias.

Another potential limitation is that it is possible that the 169 cases that made up the research sample were unusually “classic” because they were ascertained as part of ongoing studies of the genetics of major psychiatric disorders. Furthermore, the agreement might have been somewhat inflated because the best-estimate diagnosticians had access to hospital records as well as the SADS-L interviews and family history information.

The use of face-to-face diagnostic interviews, family history information, medical records, and a best-estimate diagnostic procedure is a strength of this study, compared to others that assessed Registry diagnoses relying on chart review alone.

Another strength is the use of the RDC best-estimate procedure as a gold standard. Previous studies have shown that the reliability and validity of the best-estimate diagnosis procedure are very high [30,31].

The study's design prevented us from using additional measures of concordance that are common to many validation studies, namely, the measure of specificity and the k statistic. The main reason for using sensitivity as our only concordance measure was that the participating subjects were selected with an a priori RDC diagnosis of psychotic disorders. Specificity, which measures the proportion of subjects not diagnosed by both raters as suffering from each disorder, was not an appropriate measure in this case. Similarly, k, which is influenced by the unequal proportions of base-rate diagnoses and the resulting high proportion of chance agreement [32], was not used in this study because of the biased low results it would have yielded.

4.2. Conclusion

The diagnosis of psychotic disorders and schizophrenia in the Israeli National Psychiatric Registry is comparable to those of other hospitalization registries.

Because a large number of analyses derived from this and similar national registries will be published in the coming years, this constitutes relevant information.

Acknowledgments

This study was supported in part by the Stanley Medical Research Foundation (MD and MW); NARSAD (MD and MW); the German Israeli Foundation for Scientific Research, Jerusalem, Israel and Bonn, Germany (GIF, No. I-579-069.02/98) and the Israel Ministry of Science (Korean-Israeli Research Cooperation) (BL); the German Federal Ministry of Education and Research (BMBF) within the framework of German Israeli Project Cooperation (DIP) (MD), Bonn, Germany; and NIMH R01 MH59114 (MD), Bethesda, MD.

References

  • [1].Andreasen NC, Endicott J, Spitzer RL, Winokur G. The family history method using diagnostic criteria. Reliability and validity. Arch Gen Psychiatry. 1977;34(10):1229–35. doi: 10.1001/archpsyc.1977.01770220111013. [DOI] [PubMed] [Google Scholar]
  • [2].Baron M, Endicott J, Lerer B, Loth JE, Alexander JR, Simon R, et al. A pedigree series for mapping disease genes in bipolar affective disorder: sampling, assessment, and analytic considerations. Psychiatr Genet. 1994;4(1):43–55. doi: 10.1097/00041444-199421000-00007. [DOI] [PubMed] [Google Scholar]
  • [3].Cantor-Graae E, Pedersen CB, McNeil TF, Mortensen PB. Migration as a risk factor for schizophrenia: a Danish population-based cohort study. Br J Psychiatry. 2003;182:117–22. doi: 10.1192/bjp.182.2.117. [DOI] [PubMed] [Google Scholar]
  • [4].Dalman C, Broms J, Cullberg J, Allebeck P. Young cases of schizophrenia identified in a national inpatient register—are the diagnoses valid? Soc Psychiatry Psychiatr Epidemiol. 2002;37(11):527–31. doi: 10.1007/s00127-002-0582-3. [DOI] [PubMed] [Google Scholar]
  • [5].Davidson M, Reichenberg A, Rabinowitz J, Weiser M, Kaplan Z, Mark M. Behavioral and intellectual markers for schizophrenia in apparently healthy male adolescents. Am J Psychiatry. 1999;156(9):1328–35. doi: 10.1176/ajp.156.9.1328. [DOI] [PubMed] [Google Scholar]
  • [6].Faraone SV, Blehar M, Pepple J, Moldin SO, Norton J, Nurnberger JI, et al. Diagnostic accuracy and confusability analyses: an application to the Diagnostic Interview for Genetic Studies. Psychol Med. 1996;26(2):401–10. doi: 10.1017/s0033291700034796. [DOI] [PubMed] [Google Scholar]
  • [7].Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol. 1990;43(6):543–9. doi: 10.1016/0895-4356(90)90158-l. [DOI] [PubMed] [Google Scholar]
  • [8].Fennig S, Craig TJ, Tanenberg-Karant M, Bromet EJ. Comparison of facility and research diagnoses in first-admission psychotic patients. Am J Psychiatry. 1994;151(10):1423–9. doi: 10.1176/ajp.151.10.1423. [DOI] [PubMed] [Google Scholar]
  • [9].Fennig S, Rabinowitz J. Involuntary first admission of patients with schizophrenia as a predictor of future admissions. Psychiatr Serv. 1999;50(8):1049–52. doi: 10.1176/ps.50.8.1049. [DOI] [PubMed] [Google Scholar]
  • [10].Ginath Y, Rabinowitz J, Popper M, Mark M, Ritsner M. Patterns of use and changes in diagnosis during first admission. National Case Register Study. Psychopathology. 1996;29(1):39–45. doi: 10.1159/000284970. [DOI] [PubMed] [Google Scholar]
  • [11].Kemppainen L, Jokelainen J, Isohanni M, Jarvelin MR, Rasanen P. Predictors of female criminality: findings from the Northern Finland 1966 birth cohort. J Am Acad Child Adolesc Psychiatry. 2002;41(7):854–9. doi: 10.1097/00004583-200207000-00019. [DOI] [PubMed] [Google Scholar]
  • [12].Lerer B, Segman RH, Hamdan A, Kanyas K, Karni O, Kohn Y, et al. Genome scan of Arab Israeli families maps a schizophrenia susceptibility gene to chromosome 6q23 and supports a locus at chromosome 10q24. Mol Psychiatry. 2003;8(5):488–98. doi: 10.1038/sj.mp.4001322. [DOI] [PubMed] [Google Scholar]
  • [13].Lerner Y, Popper M, Zilber N. Patterns and correlates of psychiatric hospitalization in a nationwide sample. I. Patterns of hospitalization with special reference to the “new chronic” patients. Soc Psychiatry Psychiatr Epidemiol. 1989;24(3):121–6. doi: 10.1007/BF01788020. [DOI] [PubMed] [Google Scholar]
  • [14].Malaspina D, Harlap S, Fennig S, Heiman D, Nahon D, Feldman D, et al. Advancing paternal age and the risk of schizophrenia. Arch Gen Psychiatry. 2001;58(4):361–7. doi: 10.1001/archpsyc.58.4.361. [DOI] [PubMed] [Google Scholar]
  • [15].Moilanen K, Veijola J, Laksy K, Makikyro T, Miettunen J, Kantojarvi L, et al. Reasons for the diagnostic discordance between clinicians and researchers in schizophrenia in the Northern Finland 1966 Birth Cohort. Soc Psychiatry Psychiatr Epidemiol. 2003;38(6):305–10. doi: 10.1007/s00127-003-0638-z. [DOI] [PubMed] [Google Scholar]
  • [16].Munk-Jorgensen P, Kastrup M, Mortensen PB. The Danish psychiatric register as a tool in epidemiology. Acta Psychiatr Scand Suppl. 1993;370:27–32. doi: 10.1111/j.1600-0447.1993.tb05358.x. [DOI] [PubMed] [Google Scholar]
  • [17].Pedersen CB, Mortensen PB. Evidence of a dose-response relationship between urbanicity during upbringing and schizophrenia risk. Arch Gen Psychiatry. 2001;58(11):1039–46. doi: 10.1001/archpsyc.58.11.1039. [DOI] [PubMed] [Google Scholar]
  • [18].Popper M. The epidemiology of involuntary psychiatric hospitalization in Israel. Med Law. 1990;9(6):1226–34. [PubMed] [Google Scholar]
  • [19].Rabinowitz J, Fennig S. Differences in age of first hospitalization for schizophrenia among immigrants and nonimmigrants in a national case registry. Schizophr Bull. 2002;28(3):491–9. doi: 10.1093/oxfordjournals.schbul.a006956. [DOI] [PubMed] [Google Scholar]
  • [20].Rabinowitz J, Lichtenberg P, Kaplan Z, Mark M, Nahon D, Davidson M. Rehospitalization rates of chronically ill schizophrenic patients discharged on a regimen of risperidone, olanzapine, or conventional antipsychotics. Am J Psychiatry. 2001;158(2):266–9. doi: 10.1176/appi.ajp.158.2.266. [DOI] [PubMed] [Google Scholar]
  • [21].Rabinowitz J, Mark M, Popper M, Slyuzberg M, Munitz H. Predicting revolving-door patients in a 9-year national sample. Soc Psychiatry Psychiatr Epidemiol. 1995;30(2):65–72. doi: 10.1007/BF00794944. [DOI] [PubMed] [Google Scholar]
  • [22].Rabinowitz J, Slyuzberg M, Ritsner M, Mark M, Popper M, Ginath Y. Changes in diagnosis in a 9-year national longitudinal sample. Compr Psychiatry. 1994;35(5):361–5. doi: 10.1016/0010-440x(94)90276-3. [DOI] [PubMed] [Google Scholar]
  • [23].Reichenberg A, Weiser M, Rabinowitz J, Caspi A, Schmeidler J, Mark M, et al. A population-based cohort study of premorbid intellectual, language, and behavioral functioning in patients with schizophrenia, schizoaffective disorder, and nonpsychotic bipolar disorder. Am J Psychiatry. 2002;159(12):2027–35. doi: 10.1176/appi.ajp.159.12.2027. [DOI] [PubMed] [Google Scholar]
  • [24].Roy MA, Lanctot G, Merette C, Cliche D, Fournier JP, Boutin P, et al. Clinical and methodological factors related to reliability of the best-estimate diagnostic procedure. Am J Psychiatry. 1997;154(12):1726–33. doi: 10.1176/ajp.154.12.1726. [DOI] [PubMed] [Google Scholar]
  • [25].Spitzer R, editor. The schedule for affective disorders and schizophrenia, lifetime version. New York State Psychiatric Institute; New York: 1977. [Google Scholar]
  • [26].Spitzer RL, Endicott J, Robins E. Research Diagnostic Criteria: rationale and reliability. Arch Gen Psychiatry. 1978;35(6):773–82. doi: 10.1001/archpsyc.1978.01770300115013. [DOI] [PubMed] [Google Scholar]
  • [27].Isohanni M, Makikyro T, Moring T, et al. A comparison of clinical and research DSM-III-R diagnoses of schizophrenia in a Finnish national birth cohort. Clinical and research diagnoses of schizophrenia. Soc Psychiatry Psychiatr Epidemiol. 1997;32(5):303–8. doi: 10.1007/BF00789044. [DOI] [PubMed] [Google Scholar]
  • [28].Taiminen T, Ranta K, Karlsson H, Lauerma H, Leinonen KM, Wallenius E, et al. Comparison of clinical and best-estimate research DSM-IV diagnoses in a Finnish sample of first-admission psychosis and severe affective disorder. Nord J Psychiatry. 2001;55(2):107–11. doi: 10.1080/08039480151108507. [DOI] [PubMed] [Google Scholar]
  • [29].Weiser M, Reichenberg A, Rabinowitz J, Kaplan Z, Mark M, Bodner E, et al. Association between nonpsychotic psychiatric diagnoses in adolescent males and subsequent onset of schizophrenia. Arch Gen Psychiatry. 2001;58(10):959–64. doi: 10.1001/archpsyc.58.10.959. [DOI] [PubMed] [Google Scholar]
  • [30].Weiser M, Reichenberg A, Rabinowitz J, Kaplan Z, Mark M, Nahon D, et al. Gender differences in premorbid cognitive performance in a national cohort of schizophrenic patients. Schizophr Res. 2000;45(3):185–90. doi: 10.1016/s0920-9964(99)00190-5. [DOI] [PubMed] [Google Scholar]
  • [31].Zammit S, Allebeck P, Andreasson S, Lundberg I, Lewis G. Self reported cannabis use as a risk factor for schizophrenia in Swedish conscripts of 1969: historical cohort study. BMJ. 2002;325(7374):1199. doi: 10.1136/bmj.325.7374.1199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [32].Zilber N, Popper M, Lerner Y. Patterns and correlates of psychiatric hospitalization in a nationwide sample. II. Correlates of length of hospitalization and length of stay out of hospital. Soc Psychiatry Psychiatr Epidemiol. 1990;25(3):144–8. doi: 10.1007/BF00782743. [DOI] [PubMed] [Google Scholar]

RESOURCES