Abstract
Background
The Psychotic Depression Assessment Scale (PDAS) is a rating scale dedicated to the measurement of severity in psychotic depression (PD). The aim of this study was to establish the PDAS cut-off for remission of PD as well as PDAS score-ranges for mild, moderate, and severe PD. The secondary aim was to test how remission, as defined by the PDAS, would perform as outcome measure when applied to the data from a large randomized controlled trial (RCT) in PD.
Methods
The study was based on data from the Study of Pharmacotherapy in Psychotic Depression (STOP-PD). The cut-off for remission on the PDAS and the severity-ranges for mild, moderate, and severe PD were defined using the Clinical Global Impression – Severity scale (CGI-S) as reference by means of pair-wise receiver operating characteristic (ROC) analyses. Subsequently, it was tested whether remission on the PDAS could separate the effects of Olanzapine+Sertraline vs. Olanzapine+Placebo through an intention-totreat, mixed-effects logistic regression of the data from STOP-PD.
Results
According to the ROC analyses, the ideal cut-off for remission of PD was a PDAS total score <8, while the severity-ranges for mild, moderate and severe PD were 8–15, 16–23, and >23 respectively. When applying the PDAS total score <8 (remission) as outcome on the STOP-PD data, treatment with Olanzapine+Sertraline performed significantly better than Olanzapine+Placebo (p<0.001).
Limitations
The STOP-PD was not designed specifically to answer the research questions of the present study.
Conclusions
According to this study, a total score <8 on the PDAS corresponds to remission of PD.
Keywords: Affective Disorders, Psychotic; Antidepressive Agents; Antipsychotic Agents; Psychiatric Status Rating Scales; Remission Induction
1. Introduction
Psychotic depression (PD) is a severe and debilitating condition, which needs intensive treatment and monitoring (Leadholm, et al, 2014; Ostergaard, et al, 2012; Ostergaard, et al, 2013; Rothschild, 2009). However, until recently there was no rating scale dedicated specifically for the measurement of severity of PD (Ostergaard, et al, 2015c). Therefore, we developed the Psychotic Depression Assessment Scale (PDAS) (Ostergaard, et al, 2014b; Ostergaard, et al, 2014c) based on data from the Study of Pharmacotherapy in Psychotic Depression (STOP-PD) (Meyers, et al, 2009). The PDAS consists of 11 items, the six items from the melancholia subscale (HAM-D6) (Bech, et al, 1975; Ostergaard, et al, 2014a) of the 17-item Hamilton Depression Rating Scale (HAM-D17) (Hamilton, 1960), plus five items related to psychosis from the Brief Psychiatric Rating Scale (BPRS) (Overall and Gorham, 1962). The 11 items are: depressed mood, guilt feelings, work and activities, psychomotor retardation, psychic anxiety and somatic symptoms (general) from the HAM-D6 and hallucinatory behavior, unusual thought content (delusions), suspiciousness, emotional withdrawal and blunted affect from the BPRS. When using Clinical Global Impressions (Guy, 1976) of severity (CGI-S) and improvement (CGI-I) as reference, the PDAS demonstrated clinical validity, responsiveness (sensitivity to change in illness severity) and scalability (Ostergaard, et al, 2014b). Taken together, these findings are consistent with using the sum of the individual PDAS item scores (i.e., the total score) as a measure for the overall severity of depressive and psychotic symptoms in PD. The validity of the PDAS in the measurement of the overall severity of PD was recently further supported by a Danish multicenter study (Ostergaard, et al, 2015b) of PD patients diagnosed according to the 10th edition of the International Classification of Disease (ICD-10) (World Health Organization, 1993). Furthermore, in a secondary analysis evaluating the PDAS as an outcome measure based on data from STOP-PD, it was shown that the scale was able to detect statistically significant differences in response rates when comparing treatment with Olanzapine+Sertraline to Olanzapine+Placebo (the former being superior) in patients with PD (Ostergaard, et al, 2014c). Finally, apart from its application as a rating scale measuring severity, the PDAS has also shown promising results as a tool to detect cases of PD among patients with depressive disorders in general (Park, et al, 2014a; Park, et al, 2014b).
In both clinical practice and in research studies of mental disorders, remission is a highly important outcome measure (Bech, 2012). Remission can be defined either from a clinical/nosological point of view - as the absence of illness (i.e., no longer meeting the defining criteria), or from a psychometric perspective - as a total score below a certain cut-off on a rating scale (Ostergaard, et al, 2015a). Remission of PD according to the PDAS has not been investigated. Therefore, the primary aims of this study was to establish the cut-off for remission of PD on the PDAS as well as PDAS score-ranges for mild, moderate, and severe PD. The secondary aim was to test how remission, as defined by the PDAS, would perform as an outcome measure when applied to the data from a large randomized controlled trial (RCT) in PD. Finally, we wished to define the score-changes on the PDAS, which correspond to clinically significant improvement.
2. Methods
2.1 Data source
For this study, we used data from the Study of Pharmacotherapy in Psychotic Depression (STOP-PD), which is a 12-week, randomized controlled trial (RCT) comparing the remission rates among PD patients treated with either Olanzapine+Sertraline or Olanzapine+Placebo (Meyers, et al, 2009). A total of 259 patients who met the criteria for MDD with psychotic features according to the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition, Text Revision (American Psychiatric Association, 1994) and presented with a minimum total score of 21 on the GRID-HAMD (a modified version of the HAM-D17) (Williams, et al, 2008) participated in the study. The inclusion also required presence of a delusion, rated as ≥2 on at least one of the conviction items of the Delusional Assessment Scale (Meyers, et al, 2006) and a severity score of ≥3 on the delusion item of the Schedule of Affective Disorders and Schizophrenia (SADS) (Spitzer and Endicott, 1979). During the 12-week trial (Meyers, et al, 2009), the participants were rated weekly for the first six weeks and then every other week on the following instruments: the GRID-HAMD, the Brief Psychiatric Rating Scale (BPRS) (Overall and Gorham, 1962) and the Clinical Global Impression – Severity (CGI-S) and – Improvement (CGI-I) scales (Guy, 1976).
Since the PDAS consists of six items from the HAM-D17 and five items from the BPRS, PDAS scores can be extracted from the STOP-PD dataset. In order to obtain the same scaling (0–4) on the PDAS items, the following conversion was made: ratings on the somatic symptoms (general) item (scored 0–2) was multiplied by 2 and the ratings on the five BPRS items (scored 1–7) was converted using this formula: (rating - 1) ×2/3.
2.2 Establishing cut-offs for remission and severity ranges
The cut-off for remission and the severity-ranges for mild, moderate, and severe PD were defined based on data from STOP-PD (Meyers, et al, 2009) using CGI-S as reference as follows: remission = CGI-S scores 1 (normal, not at all ill) and 2 (borderline ill), mild PD: CGI-S = 3 (mildly ill), moderate PD: CGI-S = 4 (moderately ill) and 5 (markedly ill), and severe PD: CGI-S = 6 (severely ill) and 7 (extremely ill). The CGI-S reference for remission (<3) was chosen based on results from a study of 7,131 patients with non-psychotic depression (Leucht, et al, 2013) showing that a CGI-S score of 2 corresponded to a HAM-D17 total-score between 6 and 8, which is equivalent to the cut-off for remission (<8) most commonly used for the HAM-D17 (Bech, 2012). The three cut-offs on the PDAS separating the severity levels outlined above were calculated by means of pair-wise receiver operating characteristic (ROC) analyses. These ROC analyses identified the PDAS values that resulted in the best separation between CGI-S = 1–2 vs. CGI-S = 3, CGI-S = 3 vs. CGI-S = 4–5, CGI-S = 4–5 vs. CGI-S = 6–7.
2.3 Testing the “pharmacological sensitivity” of remission on the PDAS
Following the establishment of the cut-off for remission on the PDAS, we tested its “pharmacological sensitivity” by using is as outcome in an analysis of the RCT data from STOP-PD (Olanzapine+Sertraline vs. Olanzapine+Placebo). The analysis of PDAS remission rates was carried out by means of an intention-to-treat, mixed-effects logistic regression (Hedeker and Gibbons, 1994) with a random intercept that included treatment and time (after baseline) as fixed effects and a treatment×time interaction effect. The analysis was performed with time expressed as the square root of week (√week) in line with our previous study of these data (Ostergaard, et al, 2014c). Potential differences in remission rates between the two treatment regimens were assessed by testing for statistical significance of the treatment×time interaction at the .05 level.
2.4. Establishing thresholds for clinically significant improvement on the PDAS
The threshold for clinically significant improvement on the PDAS was estimated by calculating the mean change in PDAS total score (from baseline) corresponding to CGI-I scores of 3 (minimally improved), 2 (much improved), and 1 (very much improved) based on data from STOP-PD. For this analysis, all 1,540 CGI-I ratings with scores from 1–3 from the 12-week trial and the associated values for change in PDAS total score were pooled.
3. Results
3.1 Cut-offs for remission and severity ranges
According to the ROC analyses (Table 1), the ideal cut-off for remission of PD on the PDAS was <8, while the severity-ranges for mild, moderate and severe PD were 8–15, 16–23, and >23 respectively. As shown in Table 1, these PDAS thresholds place the STOP-PD participants in the correct category (using CGI-S as reference) with acceptable sensitivity and specificity.
Table 1.
Cut-offs for remission and severity-ranges
| CGI-S | N | Mean PDAS (SD) | Cut-off | Sensitivity | Specificity |
|---|---|---|---|---|---|
| 1–2 (remission) | 427 | 4.5 (3.3) | 7 | 0.78 | 0.83 |
| 3 (mild PD) | 369 | 10.9 (4.7) | 15 | 0.79 | 0.83 |
| 4–5 (moderate PD) | 858 | 20.6 (6.5) | 24 | 0.68 | 0.71 |
| 6–7 (severe PD) | 162 | 27.4 (5.6) |
This table shows the mean PDAS scores and associated standard deviations for different values of CGI-S. The cut-off values separating remission vs. mild PD, mild PD vs. moderate PD, and moderate PD vs. severe PD were established by means of receiver operating characteristics (ROC) analyses. The sensitivity and specificity describes how well the PDAS cut-offs perform in terms of placing the STOP-PD participants in the correct severity category, using CGI-S as reference.
3.2 “Pharmacological sensitivity” of remission on the PDAS
The result of the intention-to-treat, mixed-effects logistic regression is shown in Figure 1. The combination of Olanzapine+Sertraline resulted in significantly higher remission rates compared to Olanzapine+Placebo (p<0.001).
Fig. 1.
The figure shows the proportion of STOP-PD participants reaching remission (PDAS totalscore <8) during treatment with Olanzapin+Sertraline (solid line) and Olanzapine+Placebo (dashed line) respectively. Potential difference between remission rates was tested using an intention-to-treat, mixed-effects logistic regression with a random intercept that included treatment and time (√week) as fixed effects and a treatment×time interaction effect. The P-value reflects the significance of the treatment×time interaction effect in the logistic regression.
3.3 Threshold for clinically significant improvement on the PDAS
The mean reductions in the PDAS total score corresponding to CGI-I scores of 3 (minimally improved), 2 (much improved), and 1 (very much improved) were 7.2 (SD=4.8), 14.5 (SD=6.5), and 21.8 (SD=6.3) respectively.
4. Discussion
The aim of this study was to establish the PDAS cut-off for remission of PD as well as PDAS score-ranges for mild, moderate, and severe PD. The secondary aim was to test how remission, as defined by the PDAS, would perform as an outcome measure when applied to the data from a large randomized controlled trial (RCT) in PD. Finally, we wished to define the score-changes on the PDAS, which correspond to clinically significant improvement.
Our results indicated that total-scores <8 on the PDAS resulted in the best separation of individuals in remission compared to those with mild PD. The “psychopathological” interpretation of this cut-off is best provided by consulting the web-version of the PDAS: http://psychoticdepressionassessmentscale.com It then becomes clear that a patient with a PDAS total score <8 can, by no means, meet diagnostic criteria for PD, which further supports this cut-off for remission. Similarly, a consultation of the web-version of the PDAS indicates that the mean PDAS scores (see Table 1) of patients with mild (10.9 [SD=4.7]), moderate (20.6 [SD=6.5]) and severe PD (27.4 [SD=5.6]) are indeed representative of clinical reality.
When applying PDAS remission as outcome to the STOP-PD RCT data (Meyers, et al, 2009), the results showed a significant superiority of Olanzapine+Sertraline over Olanzapine+Placebo. This finding shows that the cut-off for PDAS remission defined in this study is not only clinically valid, but also pharmacologically valid – i.e. useful as an outcome in clinical trials of pharmacological interventions in PD.
The final aim of this study was to establish a threshold for clinically significant improvement of PD on the PDAS. This part of our analysis showed that the PDAS total scores corresponding to CGI-I scores of 3 (minimally improved), 2 (much improved), and 1 (very much improved) were 7.2 (SD=4.8), 14.5 (SD=6.5), and 21.8 (SD=6.3) respectively. The consistent interval between these numbers suggests that a reduction of 7 points on the PDAS represents a clinically meaningful improvement. In accordance with this interpretation, a 7 point reduction in the PDAS total score will, in most cases, lead to a “reclassification” of the severity from severe to moderate PD, moderate to mild PD, or from mild PD to remission (Table 1). These findings are also consistent with results from a similar study of non-psychotic depression by Leucht et al. (Leucht, et al, 2013), in which CGI-I scores between 2 and 3 corresponded to a reduction on the HAM-D17 of 10 points, and a CGI-I score of 1 to a reduction of 20 points.
The primary limitation of this study is that STOP-PD was not designed specifically to define severity thresholds on the PDAS. Furthermore, in STOP-PD, the CGI and BPRS ratings were performed by psychiatrists, whereas the HAM-D17 ratings were conducted by trained research assistants with documented reliability (Meyers, et al, 2009). Since the PDAS is defined by items from both the HAM-D17 and the BPRS, it would have been better for the purpose of this study had the two ratings been performed by the same raters. However, both of these limitations also entail that it is extremely unlikely that the raters in STOP-PD have been biased towards favoring certain results on the PDAS. Future studies employing the PDAS available at http://psychoticdepressionassessmentscale.com should aim at validating the findings of the present study. Such a study, in which data on both the CGI-S and the PDAS will be obtained from 200 patients with PD, is currently being planned in Belgium and The Netherlands (personal communication, Tom Vermeulen, Collaborative Antwerp Psychiatric Research Institute, Psychiatric Hospital Duffel, Duffel, Belgium, August 22, 2015).
Another limitation of our study is that the definition of remission is “cross-sectional”, i.e., based on a single measurement. However, the severity of PD may fluctuate and therefore full remission, according to the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (American Psychiatric Association, 2013), requires two months free of significant symptoms. It would seem reasonable to operationalize this as PDAS total-scores <8 at consecutive ratings throughout two months. However, this definition should be subjected to validation.
In conclusion, based on the results of this study, we propose that remission of PD can be defined by total scores on the PDAS <8. Similarly, mild PD, moderate PD and severe PD can be defined by total scores from 8–15, 16–23, and >23 respectively. Finally, reductions of 7 points in the PDAS total score appear to represent clinically significant improvement. We believe that these findings constitute a significant contribution to the field. Indeed, the authors of a recent meta-analysis comparing the effect of antidepressant or antipsychotic monotherapy versus combination treatment emphasized that in order to “facilitate better comparability across trials, ideally, the psychiatric field should agree on… standardized outcomes of response, remission, and relapse in patients with psychotic depression”(Farahani and Correll, 2012). We believe that the findings of this study represent an important step in this direction.
Highlights.
The ideal cut-off for remission of PD was a PDAS total score <8, while the severity-ranges for mild, moderate and severe PD were 8–15, 16–23, and >23 respectively.
When applying PDAS remission as outcome to the STOP-PD RCT data, the results showed a significant superiority of Olanzapine+Sertraline over Olanzapine+Placebo.
A reduction of 7 points on the PDAS represents a clinically meaningful improvement.
Acknowledgements
None
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
REFERENCES
- American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th Edition. Washington, DC: Text Revision; 1994. [Google Scholar]
- American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th Edition. Washington, DC: 2013. [Google Scholar]
- Bech P, Gram LF, Dein E, Jacobsen O, Vitger J, Bolwig TG. Quantitative rating of depressive states. Acta Psychiatr. Scand. 1975;51:161–170. doi: 10.1111/j.1600-0447.1975.tb00002.x. [DOI] [PubMed] [Google Scholar]
- Bech P. Clinical Psychometrics. Oxford, UK: Wiley-Blackwell; 2012. [Google Scholar]
- Farahani A, Correll CU. Are antipsychotics or antidepressants needed for psychotic depression? A systematic review and meta-analysis of trials comparing antidepressant or antipsychotic monotherapy with combination treatment. J. Clin. Psychiatry. 2012;73:486–496. doi: 10.4088/JCP.11r07324. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guy W Clinical Global Impressions Scale. ECDEU Assessment Manual for Psychopharmacology. Rockville, MD, USA: US Department of Health, Education and Welfare pub no (AMD) 76-338, NIMH; 1976. [Google Scholar]
- Hamilton M. A rating scale for depression. J. Neurol. Neurosurg. Psychiatry. 1960;23:56–62. doi: 10.1136/jnnp.23.1.56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leadholm AK, Rothschild AJ, Nielsen J, Bech P, Ostergaard SD. Risk factors for suicide among 34,671 patients with psychotic and non-psychotic severe depression. J. Affect. Disord. 2014;156:119–125. doi: 10.1016/j.jad.2013.12.003. [DOI] [PubMed] [Google Scholar]
- Leucht S, Fennema H, Engel R, Kaspers-Janssen M, Lepping P, Szegedi A. What does the HAMD mean? J. Affect. Disord. 2013;148:243–248. doi: 10.1016/j.jad.2012.12.001. [DOI] [PubMed] [Google Scholar]
- Meyers BS, English J, Gabriele M, Peasley-Miklus C, Heo M, Flint AJ, Mulsant BH, Rothschild AJ STOP-PD Study Group. A delusion assessment scale for psychotic major depression: Reliability, validity, and utility. Biol. Psychiatry. 2006;60:1336–1342. doi: 10.1016/j.biopsych.2006.05.033. [DOI] [PubMed] [Google Scholar]
- Meyers BS, Flint AJ, Rothschild AJ, Mulsant BH, Whyte EM, Peasley-Miklus C, Papademetriou E, Leon AC, Heo M STOP-PD Group. A double-blind randomized controlled trial of olanzapine plus sertraline vs olanzapine plus placebo for psychotic depression: the study of pharmacotherapy of psychotic depression (STOP-PD) Arch. Gen. Psychiatry. 2009;66:838–847. doi: 10.1001/archgenpsychiatry.2009.79. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ostergaard SD, Rothschild AJ, Uggerby P, Munk-Jorgensen P, Bech P, Mors O. Considerations on the ICD-11 Classification of Psychotic Depression. Psychother. Psychosom. 2012;81:135–144. doi: 10.1159/000334487. [DOI] [PubMed] [Google Scholar]
- Ostergaard SD, Leadholm AK, Rothschild AJ. Persistent delusional theme over 13 episodes of psychotic depression. Acta Neuropsychiatr. 2013;25 doi: 10.1017/neu.2013.33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ostergaard SD, Bech P, Trivedi MH, Wisniewski SR, Rush AJ, Fava M. Brief, unidimensional melancholia rating scales are highly sensitive to the effect of citalopram and may have biological validity: Implications for the Research Domain Criteria (RDoC) J. Affect. Disord. 2014a;163:18–24. doi: 10.1016/j.jad.2014.03.049. [DOI] [PubMed] [Google Scholar]
- Ostergaard SD, Meyers BS, Flint AJ, Mulsant BH, Whyte EM, Ulbricht CM, Bech P, Rothschild AJ on behalf of the STOP-PD Study Group. Measuring psychotic depression. Acta Psychiatr. Scand. 2014b;129:211–220. doi: 10.1111/acps.12165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ostergaard SD, Meyers BS, Flint AJ, Mulsant BH, Whyte EM, Ulbricht CM, Bech P, Rothschild AJ on behalf of the STOP-PD Study Group. Measuring treatment response in psychotic depression: The Psychotic Depression Assessment Scale (PDAS) takes both depressive and psychotic symptoms into account. J. Affect. Disord. 2014c doi: 10.1016/j.jad.2013.12.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ostergaard SD, Papakostas GI, Fava M. Depression: Response and Remission. In: Price LH, Stolerman I, editors. Encyclopedia of Psychopharmacology. Berlin Heidelberg: Springer-Verlag; 2015a. [Google Scholar]
- Ostergaard SD, Pedersen CH, Uggerby P, Munk-Jorgensen P, Rothschild AJ, Larsen JI, Gotzsche C, Sondergaard MG, Bille AG, Bolwig TG, Larsen JK, Bech P. Clinical and psychometric validation of the psychotic depression assessment scale. J. Affect. Disord. 2015b;173:261–268. doi: 10.1016/j.jad.2014.11.012. [DOI] [PubMed] [Google Scholar]
- Ostergaard SD, Rothschild AJ, Flint AJ, Mulsant BH, Whyte EM, Leadholm AK, Bech P, Meyers BS. Rating scales measuring the severity of psychotic depression. Acta Psychiatr. Scand. 2015c doi: 10.1111/acps.12449. [Epub ahead of print]. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Overall JE, Gorham DR. The Brief Psychiatric Rating Scale. Psychol. Rep. 1962;10:799–812. [Google Scholar]
- Park SC, Choi J, Kim JM, Jun TY, Lee MS, Kim JB, Yim HW, Park YC. Is the Psychotic Depression Assessment Scale a useful diagnostic tool? The CRESCEND study. J. Affect. Disord. 2014a;166:79–85. doi: 10.1016/j.jad.2014.05.004. [DOI] [PubMed] [Google Scholar]
- Park SC, Ostergaard SD, Choi J, Kim JM, Jun TY, Lee MS, Kim JB, Yim HW, Park YC. Is the BPRS-5 subscale of the psychotic depression assessment scale a reliable screening tool for psychotic depression?: Results from the CRESCEND Study. J. Affect. Disord. 2014b;174C:188–191. doi: 10.1016/j.jad.2014.11.014. [DOI] [PubMed] [Google Scholar]
- Rothschild AJ. Clinical Manual for Diagnosis and Treatment of Psychotic Depression. Washington, DC, USA: American Psychiatric Publishing, Inc.; 2009. [Google Scholar]
- Spitzer R, Endicott j. Schedule for Affective Disorders and Schizophrenia. 3rd ed. Biometrics Research Dept, New York State Psychiatric Institute; 1979. [Google Scholar]
- Williams JB, Kobak KA, Bech P, Engelhardt N, Evans K, Lipsitz J, Olin J, Pearson J, Kalali A. The GRID-HAMD: standardization of the Hamilton Depression Rating Scale. Int. Clin. Psychopharmacol. 2008;23:120–129. doi: 10.1097/YIC.0b013e3282f948f5. [DOI] [PubMed] [Google Scholar]
- World Health Organization. Diagnostic criteria for research. Geneva: WHO; 1993. The ICD-10 Classification of Mental and Behavioural Disorders. [Google Scholar]

