Skip to main content
PMC home page
International Journal of Methods in Psychiatric Research logoLink to International Journal of Methods in Psychiatric Research
. 2006 Mar 24;12(4):197–207. doi: 10.1002/mpr.157

Inter‐rater reliability of the Brief Psychiatric Rating Scale and the Groningen Social Disabilities Schedule in a European multi‐site randomized controlled trial on the effectiveness of acute psychiatric day hospitals

Matthias Schützwohl 1,✉, Joanna Jarosz‐Nowak 2, Jane Briscoe 3, Krzysztof Szajowski 2, Thomas Kallert 1
PMCID: PMC6878593  PMID: 14657976

Abstract

The objectives of this study were to report the inter‐rater reliability of the Brief Psychiatric Rating Scale (BPRS 4.0) and the Groningen Social Disabilities Schedule (GSDS‐II) as assessed in a randomized controlled trial on the effectiveness of psychiatric day hospitals spanning five sites in countries of Central and Western Europe.

Following brief training sessions, videotaped BPRS‐interviews and written GSDS‐vignettes were rated by clinically experienced researchers from all participating sites. Inter‐rater reliability often proved to be poor for items assessing the severity of both psychopathology and social dysfunction, but findings suggest that both instruments allow for the assessment of the presence or absence of specific psychopathological symptoms or social disabilities. Inter‐rater reliability at subscale level proved to be good for both instruments.

Results indicate that, with a brief training session and proper use of the instruments, psychopathology and social disabilities can be reliably assessed within cross‐national research studies. The results are of particular interest given that the need to conduct cross‐national multi‐site studies including countries with different cultural backgrounds increases. Copyright © 2003 Whurr Publishers Ltd.

Keywords: inter‐rater reliability, multi‐site psychiatric studies, EDEN study, Brief Psychiatric Rating Scale, Groningen Social Disabilities Schedule

Full Text

The Full Text of this article is available as a PDF (99.1 KB).

References

  1. Alonso J, Ferrer M, Romera B, Vilagut G, Angermeyer M, Bernert S, Brugha TS, Taub N, McColgen Z, de Girolamo G, Polidori G, Mazzi F, de Graaf R, Vollebergh WAM, Buist‐Bowman MA, Demyttenaere K, Gasquet I, Haro JM, Palacín C, Autonell J, Katz SJ, Kessler RC, Kovess V, Lépine JP, Arbabzadeh‐Bouchez S, Ormel J, Bruffaerts R. The European study of the epidemiology of mental disorders (ESEMeD/ MHEDEA 2000). Project: rationale and methods. International Journal of Methods in Psychiatric Research 2001; 11: 55–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. American Pychiatric Association . DSM‐IV. Diagnostic and Statistical Manual of Mental Disorders. 4 edn. Washington DC: APA, 1994. [Google Scholar]
  3. Bartko JJ, Carpenter WT Jr. On the methods and theory of reliability. Journal of Nervous and Mental Disease 1976; 163: 307–17. [DOI] [PubMed] [Google Scholar]
  4. Becker T, Knapp M, Knudsen HC, Schene AH, Tansella M, Thornicroft G, Vázquez‐Barquero JL, and the Epsilon Study Group . Aims, outcome measures, study sites and patient sample. Epsilon Study 1. British Journal of Psychiatry 2000, 177 (suppl. 39): s1–s7. [DOI] [PubMed] [Google Scholar]
  5. Becker T, Magliano L, Priebe S, Salize HJ, Schützwohl M, Kallert T. Evidence‐based mental health services research. The contribution of some recent EU‐funded projects In: Kirch W. (ed.) Public Health in Europe. 10 Years EUPA. Berlin: Springer, 2004. [Google Scholar]
  6. Becker T, Vázquez‐Barquero JL. The European perspective of psychiatric reform. Acta Psychiatrica Scandinavica 2001, 104 (Suppl. 419): 8–14. [DOI] [PubMed] [Google Scholar]
  7. Borkowf CB. A new nonparametric method for variance estimation and confidence interval construction for Spearman's rank correlation. Computational Statistics and Data Analysis 2000; 34: 219–41. [Google Scholar]
  8. Cicchetti DV, Volkmar F, Sparrow SS, Cohen D, Fermanian J, Rourke BP. Assessing the reliability of clinical scales when the data have both nominal and ordinal features: proposed guidelines for neuropsychological assessment. Journal of Clinical and Experimental Neuropsychology 1992; 14: 673–86. [DOI] [PubMed] [Google Scholar]
  9. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 1960; 20: 37–46. [Google Scholar]
  10. Edson R, Lavori P, Tracy K, Adler LA, Rotrosen J and the Veterans Affairs Cooperative Study Study Group . Inter‐rater reliability issues in multicenter trials, Part II: Statistical procedures used in Department of Veterans Affairs Cooperative Study #394. Psychopharmacology Bulletin 1997; 33: 59–67. [PubMed] [Google Scholar]
  11. Hafkenscheid A. Psychometric evaluation of a standardized and expanded Brief Psychiatric Rating Scale. Acta Psychiatrica Scandinavica 1991; 84: 294–300. [DOI] [PubMed] [Google Scholar]
  12. Hansson L. Outcome assessment in psychiatric service evaluation. Social Psychiatry and Psychiatric Epidemiology 2001; 36: 244–8. [DOI] [PubMed] [Google Scholar]
  13. Hedlund JL, Vieweg BW. The Brief Psychiatric Rating Scale (BPRS): a comprehensive review. J Oper Psychiatry 1980: 11: 48–65. [Google Scholar]
  14. Jarosz‐Nowak J. Modified kappa as an alternative measure of agreement between 2 and more than 2 raters. Wrocław University of Technology, Institute of Mathematics, 2002.
  15. Kallert T, Ganev K, Raboch J, Kastergiou A, Solomon Z, Maj M, Dembinskas A, Kiejna A, Nawka P, Torres‐Gonzales F, Kjellin L, Priebe S. Aims and methods of the EUNOMIA‐study (European Evaluation of Coercion in Psychiatry and Harmonisation of Best Clinical Practise). Paper presented at the Tenth Annual EUPHA meeting, Dresden, Germany, 2002.
  16. Kallert T, Priebe S, Kiejna A, Nawka P, Raboch J. The European Day Hospital Evaluation (EDEN) Study: An example of EC‐funded mental health services research In Raboch J, Doubek P, Zrzavecká I. (eds). Psychiatrie v medicínĕ a medicína v psychiatrii. Praha: Galen, 2002. [Google Scholar]
  17. Kallert TW, Priebe S, Schützwohl M, Glöckner M, Briscoe J and the EDEN study group . The role of acute day hospital treatment for mental health care: research context and practical problems of carrying out the international multi‐centre EDEN‐study In: Kirch W. (ed.) Public Health in Europe. 10 Years EUPA. Berlin: Springer, 2004. [Google Scholar]
  18. Knudsen HC, Vázquez‐Barquero JL, Welcher B, Gaite L, Becker T, Chisholm D, Ruggeri M, Schene AH, Thornicroft G, and the Epsilon Study Group . Translation and cross‐cultural adaptation of outcome measurements for schizophrenia. Epsilon Study 2. British Journal of Psychiatry 2002; 177 (suppl. 39): s8–s14. [DOI] [PubMed] [Google Scholar]
  19. Leese MN, White IR, Schene AH, Koeter MWJ, Ruggeri M, Gaite L. Reliability in multi‐site psychiatric studies. International Journal of Methods in Psychiatric Research 2001; 10: 29–42. [Google Scholar]
  20. Lukoff D, Liberman RP, Nuechterlein KH. Symptom monitoring in the rehabilitation of schizophrenic patients. Schizophrenia Bulletin 1986; 12: 578–602. [DOI] [PubMed] [Google Scholar]
  21. Marshall M, Crowther R, Almarez‐Serrano A, Creed F, Sledge W, Kluiter H, Roberts C, Hill E, Wiersma D, Bond G R, Huxley P, Tyrer P. Systematic reviews of effectiveness of day care for people with severe mental disorders. (1) Acute day hospital versus admission; (2) Vocational rehabilitation; (3) Day hospital versus outpatient care. Health Technology Assessment 2001; (5) 21. [DOI] [PubMed] [Google Scholar]
  22. Overall JE, Gorham DR. Brief Psychiatric Rating Scale. Psychological Reports 1962; 10: 799–812. [Google Scholar]
  23. Roncone R, Ventura J, Impallomeni M, Falloon IRH, Morosini PL, Chiaravalle E, Casacchia M. Reliability of an Italian standardized and expanded Brief Psychiatric Rating Scale (BPRS 4.0) in raters with high vs. low clinical experience. Acta Psychiatrica Scandinavica 1999; 100: 229–36. [DOI] [PubMed] [Google Scholar]
  24. Shrout PE. Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research 1998; 7: 301–17. [DOI] [PubMed] [Google Scholar]
  25. Ventura J, Green MF, Shaner A, Liberman RP. Training and quality assurance with the Brief Psychiatric Rating Scale: ‘The drift busters’. International Journal of Methods in Psychiatric Research 1993; 3: 221–44. [Google Scholar]
  26. Ventura J, Nuechterlein KH, Subotnik KL, Gutkind D, Gilbert EA. Symptom dimensions in recent‐onset schizophrenia and mania: a principal components analysis of the 24‐item Brief Psychiatric Rating Scale. Psychiatry Research 2000; 97: 129–35. [DOI] [PubMed] [Google Scholar]
  27. Wiersma D, de Jong A, Kraaijkamp HJM, Ormel J. GSDS II. The Groningen Social Disabilities Schedule, Second Version. University of Groningen, Department of Social Psychiatry, 1990.
  28. Wiersma D, de Jong A, Ormel J. The Groningen Social Disabilities Schedule: Development, relationship with I.C.I.D.H., and psychometric properties. International Journal of Rehabilitation Research 1988; 11: 213–24. [PubMed] [Google Scholar]
  29. World Health Organization . Schedules for Clinical Assessment in Neuropsychiatry, Version 2.1. Geneva: World Health Organization, 1999. [Google Scholar]

Articles from International Journal of Methods in Psychiatric Research are provided here courtesy of Wiley

RESOURCES