Reliability and Validity of the Calgary Depression Scale for Schizophrenia (CDSS) in Youth at Clinical High Risk for Psychosis

Abstract

Assessment of depression in individuals who are considered to be at clinical high risk (CHR) for psychosis is important as high rates of depression have been reported in CHR individuals. The Calgary Depression Scale (CDSS) is the most widely used scale for assessing depression in schizophrenia. It has excellent psychometric properties, internal consistency, inter-rater reliability, sensitivity, specificity, and discriminant and convergent validity. The aim of this study was to test the reliability and validity of the CDSS in a sample of youth at CHR for psychosis. Participants were assessed for depression, presence of axis 1 mood disorders, and prodromal symptoms using the CDSS, the Structured Clinical Interview for DSM-IV Disorders (SCID-1), and the Scale of Prodromal Symptoms (SOPS). The CDSS total score as well as all individual items, except “guilty ideas of reference,” were significantly associated with the presence of a major depressive disorder. Significant correlations were observed between CDSS total score and the dysphoric mood item on the SOPS. There was some evidence of overlap between the CDSS and both attenuated positive symptoms and negative symptoms as assessed by the SOPS. It is concluded that CDSS is a reliable scale suitable for assessing depression in individuals considered to be at CHR for psychosis.

1. Introduction

Assessment of depression in schizophrenia is clinically important due to the high prevalence of depression and suicidality in schizophrenia and to its impact on functioning (Harvey, 2011; Sanchez-Gistau et al., 2013; Siris, 2000). Specifically designed for individuals with schizophrenia, the Calgary Depression Rating Scale (CDSS) (Addington et al., 1990) was originally developed to address the issue of the overlap between depression and negative symptoms and has become the recommended scale to assess the severity of depression in schizophrenia (Collaborative Working Group on Clinical Trial Evaluations, 1998). In the past 20 years there has been a wealth of studies examining the CDSS (www.ucalgary.ca/cdss) including its translation into more than 36 languages. It has excellent psychometric properties, internal consistency, inter-rater reliability, sensitivity, specificity, and discriminant and convergent validity (Addington et al., 1992; Addington et al., 1994; Bernard et al., 1998; Lancon et al., 1999; Lancon et al., 2000; Muller et al., 1999; Sarro et al 2004). It has little overlap with positive and negative symptoms and no real correlation with extrapyramidal symptoms (Addington et al., 1994; Muller, 2002). Finally, the CDSS has demonstrated superiority relative to other depression rating scales, such as the Hamilton Depression Rating Scale and the Montgomery-Asberg Scale in terms of differentiating between depression and negative and positive symptoms and brevity of administration (Addington et al., 1996; Collins et al., 1996).

More recently, in the schizophrenia field, research has focused on individuals who are considered to be at clinical high risk (CHR) for psychosis based on the presence of attenuated psychotic symptoms (Addington and Heinssen, 2011). Approximately, 36% of those at CHR will convert to psychosis within 36 months (Fusar-Poli et al., 2012). High rates of depressed mood have been reported both in retrospective studies of the pre-psychotic period (Häfner et al., 2005) and in prospective CHR studies where approximately 40–50% meet DSM-IV criteria for major depression (Addington et al., 2012a; Fusar-Poli et al., 2012; Rietdijk et al., 2013). Furthermore, in a recent study with CHR youth, high levels of depression predicted poor functioning above and beyond negative symptoms (Fulford et al., 2013). Thus, the purpose of this paper is to investigate the reliability and validity of the CDSS in a sample of youth at CHR for psychosis.

2. Method

2.1 Sample

The sample consists of 155 participants ( 81 male, 74 female) at CHR for psychosis based on meeting Criteria of Prodromal Syndromes (COPS) using the Structured Interview for Prodromal Syndromes (SIPS) and the Scale of Prodromal Symptoms (SOPS) (McGlashan et al., 2010). Details on ascertainment, inclusion and exclusion criteria have been described in detail elsewhere (Addington et al., 2012a & b). Exclusion criteria included any current or lifetime axis I psychotic disorder, a prior history of treatment with an antipsychotic medicine, IQ < 70 and current history of a clinically significant central nervous system disorder.

2.2 Measures

The Structured Clinical Interview for DSM-IV Disorders (SCID-1) (First et al., 1995) was used to determine the presence of any axis 1 mood disorders. The Structured Interview for Prodromal Syndromes (SIPS) and the Scale of Prodromal Symptoms (SOPS) (McGlashan et al., 2010) were used to determine the Criteria of Prodromal Syndromes (COPS) and the presence and severity of prodromal symptoms. The COPS has three possible prodromal syndromes -attenuated positive symptom syndrome (APSS), genetic risk and deterioration (GRD) and/or brief intermittent psychotic syndrome (BIPS). APSS requires the presence of at least one attenuated psychotic symptom (unusual thought content, suspiciousness, grandiose ideas, perceptual abnormalities, or disorganized communication). Attenuated psychotic symptoms need to have been present in the past month, and have begun or worsened in the past 12 months. The GRD requires having a combination of both functional decline (at least a 30% drop in GAF score over the last month as compared to 12 months ago) and genetic risk (having either schizotypal personality disorder or a rst-degree relative with a schizophrenia spectrum disorder). The BIPS requires the presence of any one or more threshold positive psychotic episode that is too brief to meet diagnostic criteria for psychosis. Symptoms were assessed with the Scale of Prodromal Symptoms (SOPS), which consists of 19 items in 4 symptom domains: positive, negative, general, and disorganized.

Depression was assessed using the Calgary Depression Scale for Schizophrenia (Addington et al., 1993).

2.3 Procedures

Participants were recruited as part of a longitudinal study of predictors of psychosis. Individuals who responded to our recruitment efforts were screened on the phone and if suitable were invited in for an assessment to determine inclusion criteria (Addington et al. 2012b). All interested individuals signed informed consent. Participants were assigned a clinical rater who conducted all the semi structured interviews. CDSS was administered by six raters. Data in this paper was collected at the baseline assessment. Reliability of the diagnostic interview was conducted on a yearly basis. The intraclass correlation coefficient (ICC)for interrater reliability on the SOPS was 0.96. The interrater reliability on the CDSS was conducted by six raters. The ICC correlation for interrater reliability based on absolute agreement on the CDSS was 0.98 with 95% confidence interval of 0.94-to 0.99.

2.4 Data Analysis

Descriptive statistics were calculated for age, years of education, CDSS total scores, and SOPS negative and positive symptoms total scores. Frequencies of measures were calculated for marital status, race, living arrangements and mood disorder diagnoses. The intraclass correlation coefficient (ICC) measuring absolute agreement was used to assess the inter-rater reliability on the CDSS. Spearman’s and Pearson’s correlation coefficients were used to determine the association between the CDSS and the presence of major depressive disorder and between the CDSS, the dysphoric mood item from the SOPS general psychopathology scale and the SOPS positive and negative subscales. A receiver operating characteristic (ROC) curve analysis was performed to demonstrate the ability of the CDSS to separate those with and without a major depressive disorder (MDD) by plotting sensitivity versus (1-secificity) (i.e. the false-positive rate) across a range of rating scores on the CDSS and using the area under the ROC curve as a summary measure of the overall performance of the CDSS.

3. Results

3.1 Demographic characteristics

The mean age of the participants was 17.79 (SD=3.73) years, ranging from 13 to 35 years. The majority of the participants was single (94.19%), Caucasian (73.55%) and lived at home (81.29%). The average number of years of education achieved by participants was 10.97 (SD=2.45) years.

3.2 Clinical Symptoms

37.25% of participants met criteria for a current mood disorder with 25.49% meeting criteria for a major depressive disorder (MDD). Details are presented in Table 1. The mean CDSS total score was 5.46 (SD=4.65, range 0–20). The mean SOPS positive and negative total scores were 11.18 (SD=4.15 , range 0–30) and 10.40 (SD=6.03, range 0–36), respectively. The mean CDSS score of participants with no depressive disorder was 3.02 (SD=2.58), for those with a MDD was 10.46 (SD=4.25) and for those with diagnoses of other mood disorders 7.89 (SD=4.43). Since one of the goals of this paper is to determine the validity of the CDSS in a CHR sample by examining if it is predictive of a major depression, for those analyses, we have not included the 18 participants with diagnoses of dysthymic disorder, depression NOS or MDD in partial remission.

Table 1.

Frequency of diagnosis of mood disorders

Mood Disorder Diagnosis	Frequency
No mood disorder diagnosis	96 (62.75%)
Major depressive disorder	39 (25.49%)
Dysthymic disorder	4 (2.61%)
Depressive disorder NOS	5 (3.27%)
Major depressive disorder in partial remission	9 (5.88%)

3.3 Correlations between CDSS and SOPS

The Pearson’s correlations between CDSS total score and positive symptoms, negative symptoms, and the dysphoric mood item scores from the SOPS are presented in Table 2. Significant correlations were observed between CDSS total score and the total SOPS positive symptoms and the total SOPS negative symptoms. CDSS total scores were also significantly associated with the dysphoric mood item and with 4 of the 7 individual negative symptoms.

Table 2.

Pearson’s correlations between Calgary Depression Scale and Scale of Prodromal Symptoms

SOPS Subscale Measures	r -value
SOPS positive symptoms total score	0.24**
SOPS negative symptoms total score	0.38**
N1 Social anhedonia	0.35**
N2 Avolition	0.36**
N3 Decreased expression of emotion	0.17*
N4 Decreased experience of emotion and self	0.33**
N5 Decreased ideational richness	0.09
N6 Occupational functioning	0.11
G Dysphoric mood (SOPS General subscale)	0.56**

^*p < 0.05;

^**p < 0.01

3.4 CDSS and major depressive disorder

The Spearman’s correlations between a diagnosis of a MDD and the CDSS total and CDSS individual item scores are presented in Table 3. The total CDSS scores as well as all individual items, except “guilty ideas of reference,” were significantly associated with MDD.

Table 3.

Spearman’s correlations between CDSS and Presence of a Major Depressive Disorder (MDD)

CDSS Scores	rho-value
Depression	0.55***
Hopelessness	0.59***
Self-Depreciation	0.54***
Guilty ideas of reference	0.17
Pathological guilt	0.49***
Morning depression	0.44***
Early awakening	0.21*
Suicide	0.47***
Observed depression	0.61***
CDSS total score	0.69***

^*p < 0.05;

^***p < 0.001

3.5 ROC analysis

The area under the ROC curve was 0.94 with 95% confidence interval of 0.90 to 0.98( p<0.0001), which is in the excellent range of accuracy. This means that the CDSS is discriminative in terms of its ability to separate the group being tested into those with and without MDD .The sensitivities and specificities corresponding to different CDSS scores are presented in Table 4.

Table 4.

Sensitivity and Specificity for Different Rating Scores on the CDSS

CDSS Score	Sensitivity (%)	Specificity (%)
0	100	16
1	100	28
2	100	36
3	100	50
4	95	62
5	88	69
6	75	79
7	63	88
8	69	92
9	60	93
10	55	94
11	46	100
12	33	100
13	24	100
14	19	100
15	18	100
16	12	100
17	8	100
18	5	100
19	3	100
20	0	100
21	0	100

4. Discussion

The purpose of this study was to determine the reliability and validity of the CDSS, a scale that was empirically derived for use in individuals with established psychotic illness, in assessing depression in youth at CHR for psychosis. Results of this study were that the CDSS has high inter-rater reliability. It also appears to be a valid measure of depression as shown by its high correlation with the presence of a major depressive episode and the SOPS dysphoric mood item. All CDSS items and the total score were predictive of the presence of a major depressive disorder. The one exception was the item “guilty ideas of reference”. However this item was the least endorsed of all items even in those with a MDD, with a mean of 0.25 in the MDD group. Data from the ROC also supported that the CDSS adequately discriminates between those with and without a MDD. An examination of Table 3 suggests that to optimize both sensitivity and specificity a score of 6 may be a valid cut off point for probable depression with sensitivity of 75% and a specificity of 79%. Interestingly the sensitivity and specificity of the scale in this population is similar to these parameters in schizophrenia and the cut point is the same (Addington et al., 1993). This suggests that the CDSS is consistently measuring the same core phenomena in both the CHR group and in the population with schizophrenia.

However, in contrast to studies with populations with schizophrenia, the CDSS was significantly associated with negative symptoms and with positive symptoms. It has been suggested that individuals with attenuated or subthreshold psychotic symptoms who present for help are those who are distressed by their symptoms (Yung et al., 2006). Thus it is possible that the more severe the attenuated positive symptoms then the more depression will be observed. The overlap with SOPS negative symptoms is somewhat harder to interpret. However, little is known about the association between negative symptoms assessed by the SOPS compared to those assessed by other scales such as the Positive and Negative Syndrome Scale (PANSS). In a recent publication (Piskulic et al., 2012 ) using the SOPS, high levels of negative symptoms were observed in a large CHR sample with 82% exhibiting moderate to severe negative symptoms at the initial assessment with more than 50% demonstrating stable moderately severe negative symptoms one year later. Since these rates are higher than typically observed with PANSS rated negative symptoms and that improvement was observed over time it is possible that the SOPS rated negative symptoms in this project were less stable and potentially overlapped with depression at the initial assessment.

There are some limitations to this study. We did not include any other measures of depression and it may have been useful to use other measures such as the Beck Depression Inventory Scale (Beck et al., 1961). We have addressed above the concerns with the SOPS as a measure of negative symptoms. However, in conclusion the results of this study do support the use of the CDSS as a measure of depression in those at clinical high risk of psychosis.