Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jan 27.
Published in final edited form as: J Affect Disord. 2012 Jan 25;138(1-2):123–127. doi: 10.1016/j.jad.2011.12.003

Validation of BDI-II response and remission cut-points for assessment of seasonal affective disorder patients

Gloria M Reeves 1, Kelly J Rohan 2, Patricia Langenberg 3, Soren Snitker 4, Teodor T Postolache 5
PMCID: PMC5270383  NIHMSID: NIHMS352433  PMID: 22277151

Abstract

Background

The purpose of this study is to define treatment response and remission cut-point scores for the Beck Depression Inventory-Second Edition (BDI-II) when used to monitor a seasonal affective disorder (SAD) episode.

Methods

Data from two published randomized clinical trials for SAD were utilized to complete a ROC analysis to define response and remission thresholds for the BDI-II. The Structured Interview Guide for the Hamilton Rating Scale for Depression-Seasonal Affective Disorder Version (SIGH-SAD) was used as a reference standard. Data from the two trials included BDI-II and SIGH-SAD scores for patients at baseline, 6 weeks (post-treatment), and 1 year (next winter).

Results

BDI-II score of ≤9 was the derived criterion for remission of SAD, and BDI-II score decrease of 50% from baseline was the criterion for treatment response.

Limitations

Study participants were primarily female (94%) and Caucasian (80%) so demographic diversity of the sample was limited.

Conclusion

This study validated BDI-II scores compared to the SIGH-SAD reference standard. The BDI-II has greater potential for widespread use by clinicians than the SIGH-SAD to monitor SAD patients because it is a brief self-report instrument that can be conveniently administered in the waiting room.

Keywords: BDI-II, SIGH-SAD, Seasonal Affective Disorder

Introduction

Standardized outcome tools help the practicing clinician distinguish response (partial symptom reduction) from remission (full return to healthy functioning). This distinction is important in depression treatment, since patients often fail to achieve remission after an adequate treatment trial (Trivedi et al. 2006). A prospective, naturalistic study of individuals with a first lifetime major depressive episode found that patients with incomplete remission of depression experienced more rapid depression recurrence and had more chronic major depressive episodes (>2 years) compared to patients with full remission (Judd et al. 2000). Patients with residual depression symptoms have also demonstrated worse social/relationship and occupational outcomes compared to patients who achieved remission (Kennedy and Paykel 2004).

A challenge, however, in using outcome measures to determine treatment response and remission is that scores may not be validated for specific depression subgroups. One subtype of major depressive disorder is seasonal affective disorder (SAD). Patients with SAD experience marked changes in mood and activity over the fall/winter months and spontaneous remission of symptoms in the spring/summer. These patients often present with significant atypical depressive symptoms, including low energy, increased appetite/weight gain, hypersomnia, and carbohydrate craving (Rosenthal et al. 1984). Individuals with SAD often require ongoing monitoring because of the highly recurrent nature of this disorder (Sakamoto et al. 1995; Schwartz et al. 1996), and the seasonal pattern of symptoms allows for a more predictable time frame to assess for recurrence of symptoms. Prevention of subsequent mood episodes and early intervention is a major public health concern in SAD management (Rohan et al. 2009). In one study of over 1000 SAD patients, individuals reported an average of 13 previous seasonal depression episodes (Modell et al. 2005).

A standard clinical research tool to assess SAD symptom severity and treatment response is the Structured Interview Guide for the Hamilton Rating Scale for Depression-Seasonal Affective Disorder Version (SIGH-SAD; Williams et al. 1992). This semi-structured interview tool is not useful for widespread use in community treatment because of lengthy administration time and required training.

The Beck Depression Inventory-Second Edition (BDI-II; Beck et al. 1996b) is a commonly used treatment outcome measure for non-seasonal depression. The BDI-II can be used in both primary and specialty care settings because it is a self-report measure and takes only 5-10 minutes to complete.

To our knowledge, the BDI-II has never been calibrated to the SIGH-SAD, the reference standard in SAD research, in order to establish appropriate cut-point scores for SAD treatment response and remission. The purpose of this study was to identify appropriate BDI-II threshold scores to define “response” and “remission” in SAD treatment. We did so by comparing BDI-II and SIGH-SAD scores of SAD patients treated in a randomized clinical trial. Patient assessments using both instruments were completed at baseline, after the 6-weeks treatment phase, and at a 1-year naturalistic follow-up the next winter.

Methods

Participants

Participants were treated in one of two published randomized clinical trials for SAD (Rohan et al. 2004; Rohan et al. 2007). The first study (Rohan et al. 2004) was a 6-week randomized clinical trial of light therapy, group cognitive-behavioral therapy, or combination treatment for SAD. The second study (Rohan et al. 2007) was a 6-week randomized clinical trial with the same interventions plus a wait-list control group. Follow-up data were obtained for participants in both studies during January or February of the next winter season (Rohan et al. 2009).

Recruitment and full inclusion/exclusion criteria are detailed in Rohan et al. 2004 and Rohan et al. 2007. Diagnosis of recurrent major depressive disorder with seasonal pattern was confirmed by both Structured Clinical Interview for DSM-IV Axis I Disorders – Clinician Version (SCID-CV; First et al. 1995) and SIGH-SAD interview using published guidelines on criteria for current SAD episode (Terman et al. 1990). Research was conducted with approval by the Uniformed Services University of the Health Sciences Institutional Review Board.

Structured Interview Guide for the Hamilton Rating Scale for Depression (SIGH-SAD)

The SIGH-SAD consists of the 21-item Structured Interview Guide for the Hamilton Rating Scale for Depression (HAM-D) plus an 8-item subscale to assess atypical depressive symptoms associated with SAD. Inter-rater reliability for total SIGH- SAD scores was high, rs = .93-.96 at pre-treatment, .98-.99 at post-treatment (Rohan et al. 2004; Rohan et al. 2007) and r =.99 at 1-year (Rohan et al. 2009).

Beck Depression Inventory-Second Edition (BDI-II)

The Beck Depression Inventory-Second Edition (BDI-II) is an updated version of the amended Beck Depression Inventory (BDI-IA; Beck et al. 1993) that includes atypical depressive symptoms (i.e., fatigue, hypersomnia, hyperphagia), consistent with the DSM-IV criteria for major depression. Participants rate 21 items on a scale of 0 to 3 based on how they have been feeling over the last two weeks, and the total score is the sum of these items. The BDI-II has a high level of internal consistency (Coefficient α = .91) (Beck et al. 1996a), and has been shown to yield reliable, internally consistent, and valid scores in a primary care setting (Arnau et al. 2001).

Statistical Methods

We conducted a calibration analysis comparing SIGH-SAD and BDI-II scores pooled across the two studies at treatment endpoint (after 6 weeks of treatment) and at 1-year follow-up. The SIGH-SAD score was considered the reference standard for “response” (at least 50% reduction in SIGH-SAD score) and “remission” (SIGH-SAD score of ≤8) criteria. Remission criteria are consistent with prior SAD clinical trials (Terman et al. 1998; Koorengevel et al. 2001; Lavoie et al. 2009).

The BDI-II cut-points for treatment response and remission were determined using the 6-week outcome data. First, BDI-II scores were regressed against SIGH-SAD scores in separate models for the baseline and 6 weeks time points. Figure 1 displays BDI-II scores as a function of SIGH-SAD scores at baseline and at 6 weeks. For remission, a logistic regression model was developed with SIGH-SAD remission status (yes/no) at 6 weeks as the dependent variable, and BDI-II score at 6 weeks as the independent variable, providing data (potential cutpoints) for a receiver operating characteristic (ROC) curve analysis. The c-statistic (area under the curve, perfect calibration = 1) was calculated as a measure of goodness of fit.

Figure 1. BDI-II scores as a function of SIGH-SAD scores at baseline and 6 weeks.

Figure 1

Open in a new tab

Legend: BDI, Beck Depression Inventory-II

SIGH-SAD – Structured Interview guide for the Hamilton Rating Scale for Depression – Seasonal Affective Disorder Version

For response, defined as 50% or more reduction in SIGH-SAD score from baseline, a similar procedure was used. In this case, the % reduction in BDI-II score at 6 weeks was first calculated for each observation. Next, the logistic model was developed with SIGH-SAD response status (yes or no) as the dependent variable and percent reduction in BDI-II score at 6 weeks as the predictor variable. The ROC curve was plotted, and the c-statistic recorded.

The BDI-II cut-points for remission and response derived from the 6 week data were then used to determine the proportion of remitters and responders at 1-year follow up compared to the SIGH-SAD reference standard.

Results

The current calibration analysis included data only from subjects who completed both the SIGH-SAD and the BDI-II at a given time point. Across both studies, of the 72 individuals who were randomized to CBT, light therapy, or combination treatment (24 CBT, 25 LT, 23 CBT+LT), 55 provided data at the 1-year followup (18 CBT, 20 LT, 17 CBT+LT). The BDI-II cut-point for remission was determined as the value that provided the percentage of participants in remission closest to the SIGH-SAD percentage of remitters, in addition to optimal values of sensitivity, specificity, and correct classification. A similar procedure was followed for determination of the BDI-II cut-point for response.

For SAD treatment response, the BDI-II derived cut-point was 50% improvement (i.e., 50% reduction in BDI-II score from baseline). This cut-point correctly identified 76.3% of treatment responders at 6 weeks and 70.9% of responders at one year follow-up per the reference SIGH-SAD measure. Figure 2 illustrates the correct/incorrect classification of response at 6 weeks by the BDI-II cut-point (50% improvement) compared to the reference SIGH-SAD response score.

Figure 2. BDI-II scores at 6 weeks (percentage of baseline) as a function of SIGH-SAD scores at 6 weeks (percentage of baseline).

Figure 2

Open in a new tab

Treatment response is defined as 50% reduction in BDI-II

Reference standard is 50% reduction in SIGH-SAD

Additional lines indicate 50% baseline BDI-II and 50% baseline SIGH-SAD scores

Correct classification (n=58); Incorrect classification (n=11)

Regarding the primary clinical goal of remission, 36.8% of participants (N= 76) achieved SAD remission by 6 weeks and 36.4% were in remission at one year follow up (N=55) by SIGH-SAD score. Table 1 indicates the sensitivity/specificity and positive predictive value/negative predictive value associated with four possible remission cut-points on BDI-II, and calibration with the SIGH-SAD score ≤ 8. Table 2 indicates the c-statistic (area under the ROC curve) as well as the sensitivity and specificity for the BDI-II remission and response cut-points compared to the reference SIGH-SAD.

Table 1.

Sensitivity and specificity associated with four cut-points for BDI-II, and calibration with the SIGH-SAD score ≤ 8.

6 weeks post treatment One year post treatment One year post treatment
BDI cut-point Sensitivity N=34 Specificity N=42 Sensitivity N=20 Specificity N=35 Positive Predictive Value Negative Predictive Value
<= 8 67.7 73.8 85.0 71.4 63.0 89.3
<= 9 76.5 71.4 90.0 60.0 56.3 91.3
<= 10 88.2 66.7 90.0 45.7 48.7 88.9
<= 11 94.1 66.7 95.0 42.9 48.7 93.8
Open in a new tab

Table 2.

Sensitivity and specificity of BDI II cut-point scores in designating SAD “response” and “remission” compared to SIGH SAD:

c-statistic Sensitivity Specificity
Remission 6 weeks 0.84 76.5 71.4
1 year 90.0 60.0
Response 6 weeks 0.87 80.8 66.7
1 year 87.5 47.8
Open in a new tab

c-statistic is area under the ROC curve

Discussion

Based on our calibration analysis, BDI-II score of ≤9 was the derived criterion for remission of SAD, and BDI-II score decrease of 50% or more was the criterion for treatment response. The BDI-II may be a practical and useful tool for monitoring treatment response and remission in SAD patients. Assessing for remission is important because there are several treatment options available to optimize response, including light therapy, antidepressant medication, and cognitive-behavioral therapy. Of note, SIGH-SAD scores indicate that less than half of patients in the included studies achieved full remission after six weeks of carefully administered treatment by SAD experts, suggesting that most patients will require a longer duration of treatment or optimization of their current regimen.

The area under the curve (c-statistic) was .84 for remission and .87 for response. When ROC analysis is used for validating screening tools, a c-statistic of .80 indicates a scale is a good screening instrument and .90 indicates an excellent screening tool (DeSouza et al. 2009). Treatment response in non-seasonal depression intervention trials is often defined as 50% improvement on standardized rating scales (Keller 2003; Israel 2006). Because treatment response is defined as percentage improvement, there is considerable variability in the actual scores that denote response.

This study has the limitation that participants in the two SAD studies were mainly female (94%) and Caucasian (80%). It is possible that our derived calibration of the BDI-II does not extend to other demographic samples. The BDI-II can be a useful tool to monitor seasonal depression treatment response and has the advantages of short administration time, self report, and widespread use by both primary care and mental health clinicians compared to the semi-structured interview SIGH-SAD scale. Further research is needed to test use of this scale to monitor SAD treatment response for different demographic groups and use in different types of treatment settings. Although individuals with SAD typically experience spontaneous remission in spring/summer months, the recurrent nature of this subtype of depression indicates the need for careful, ongoing monitoring.

Acknowledgments

Role of the Funding Sources: The Uniformed Services University of the Health Sciences (USUHS) grant and NIH grant (specific grant numbers are included in the Acknowledgement section) funded the two studies that were included in the data analysis of this study. Dr. Reeves and Dr. Postolache's both received salary support from NIH grants for time involved in manuscript preparation).

Footnotes

Conflict of interest: The authors have no conflict of interest to report.

Contributors: All authors contributed to development of the study design and data analysis plan through a series of joint meetings. Author G.R wrote the primary draft of the paper and author P.L completed the data analysis and wrote the statistical analysis section. Author S.S. developed the figures and reviewed/edited the draft. Authors T.P. and K.R. also reviewed and edited the draft.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Arnau RC, Meagher MW, Norris MP. Psychometric evaluation of the Beck Depression Inventory-II with primary care medical patients. Health Psychol. 2001;20(2):112–119. doi: 10.1037//0278-6133.20.2.112. [DOI] [PubMed] [Google Scholar]
  2. Beck AT, Steer RA. Manual for the Beck Depression Inventory. Psychological Corporation; San Antonio, Texas: 1993. [Google Scholar]
  3. Beck AT, Steer RA, Ball R. Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. J Pers Assess. 1996a;67(3):588–597. doi: 10.1207/s15327752jpa6703_13. [DOI] [PubMed] [Google Scholar]
  4. Beck AT, Steer RA, Brown GK. Manual for Beck Depression Inventory II (BDI-II) Psychology Corporation; San Antonio, Texas: 1996b. [Google Scholar]
  5. DeSouza J, Jones LA, Rickards H. Validation of self-report depression rating scales in Huntington's Disease. Movement Disorders. 2009;25(1):91–96. doi: 10.1002/mds.22837. [DOI] [PubMed] [Google Scholar]
  6. First MB, Spitzer RL, Gibbon M. Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-Clinician Version) New York State Psychiatric Institute Biometrics Research Department; New York: 1995. [Google Scholar]
  7. Israel JA. Remission in depression: definition and initial treatment approaches. J Psychopharmacol. 2006;20(3 suppl):5–10. doi: 10.1177/1359786806064306. [DOI] [PubMed] [Google Scholar]
  8. Judd LL, Paulus MJ, Schettler PJ, Akiskal HS, Endicott J, Leon AC, Maser JD, Mueller T, Solomon DA, Keller MB. Does incomplete recovery from first lifetime major depressive episode herald a chronic course of illness. Am J Psychiatry. 2000;157(9):1501–1504. doi: 10.1176/appi.ajp.157.9.1501. [DOI] [PubMed] [Google Scholar]
  9. Keller MB. Past, present, and future directions for defining optimal treatment outcome in depression. JAMA. 2003;289(23):3152–3160. doi: 10.1001/jama.289.23.3152. [DOI] [PubMed] [Google Scholar]
  10. Kennedy N, Paykel ES. Residual symptoms at remission from depression: impact on long-term outcome. J Affect Disord. 2004;80(2-3):135–44. doi: 10.1016/S0165-0327(03)00054-5. [DOI] [PubMed] [Google Scholar]
  11. Modell JG, Rosenthal NE, Harriett AE, Krishen A, Asgharian A, Foster VJ, Metz A, Rockett CB, Wightman DS. Seasonal affective disorder and its prevention by anticipatory treatment with buproprion. XL Biol Psychiatry. 2005;58(8):658–667. doi: 10.1016/j.biopsych.2005.07.021. [DOI] [PubMed] [Google Scholar]
  12. Rohan KJ, Lindsey KT, Roecklein KA, Lacy TJ. Cognitive-behavioral therapy, light therapy, and their combination in treating seasonal affective disorder. J Affect Disord. 2004;80(2-3):273–283. doi: 10.1016/S0165-0327(03)00098-3. [DOI] [PubMed] [Google Scholar]
  13. Rohan KJ, Roecklein KA, Lacy TJ, Vacek PM. Winter depression recurrence one year after cognitive-behavioral therapy, light therapy, or combination treatment. Behav Ther. 2009;40(3):225–238. doi: 10.1016/j.beth.2008.06.004. [DOI] [PubMed] [Google Scholar]
  14. Rohan KJ, Roecklein KA, Tierney LK, Johnson LG, Lippy RD, Lacy TJ, Barton FB. A randomized controlled trial of cognitive-behavioral therapy, light therapy, and their combination for seasonal affective disorder. J Consult Clin Psychol. 2007;75(3):489–500. doi: 10.1037/0022-006X.75.3.489. [DOI] [PubMed] [Google Scholar]
  15. Rosenthal NE, Sack DA, Gillin C, Lewy AJ, Goodwin FK, Davenport Y, Mueller PS, Newsome DA, Wehr TA. Seasonal affective disorder. A description of the syndrome and preliminary findings with light therapy. Arch Gen Psychiatry. 1984;41(1):72–80. doi: 10.1001/archpsyc.1984.01790120076010. [DOI] [PubMed] [Google Scholar]
  16. Rush AJ, Kraemer HC, Sackeim HA, Fava M, Trivedi MH, Frank E, Ninan PT, Thase ME, Gelenberg AJ, Kupfer DJ, Regier DA, Rosenbaum JF, Ray O, Schatzberg AF. ACNP Task Force, 2006. Report by the ACNP task force on response and remission in major depressive disorder. Neuropsychopharmacology. 31(9):1841–1853. doi: 10.1038/sj.npp.1301131. [DOI] [PubMed] [Google Scholar]
  17. Sakamoto K, Nakadaira S, Kamo K, Kamo T, Takahashi K. A longitudinal follow-up of seasonal affective disorder. Am J Psychiatry. 1995;152:862–868. doi: 10.1176/ajp.152.6.862. [DOI] [PubMed] [Google Scholar]
  18. Schwartz PJ, Brown C, Wehr TA, Rosenthal NE. Winter seasonal affective disorder: a follow-up study of the first 59 patients of the National Institute of Mental Health Seasonal Studies Program. Am J Psychiatry. 1996;53(8):1028–36. doi: 10.1176/ajp.153.8.1028. [DOI] [PubMed] [Google Scholar]
  19. Strik JJ, Honig A, Lousberg R, Denollet J. Sensitivity and specificity of observer and self-report questionnaires in major and minor depression following myocardial infarction. Psychosomatics. 2001;42(5):423–428. doi: 10.1176/appi.psy.42.5.423. [DOI] [PubMed] [Google Scholar]
  20. Terman M, Amira L, Terman JS, Ross DC. Predictors of response and nonresponse to light treatment for winter depression. Am J Psychiatry. 1996;153(11):1423–1429. doi: 10.1176/ajp.153.11.1423. [DOI] [PubMed] [Google Scholar]
  21. Terman M, Terman JS, Rafferty B. Experimental design and measures of success in the treatment of winter depression by bright light. Psychopharmacol Bull. 1990;26(4):505–510. [PubMed] [Google Scholar]
  22. Trivedi MH, Rush AJ, Wisniewski SR, Nierenberg AA, Warden D, Ritz L, Norquist G, Howland RH, Lebowitz B, McGrath PJ, Shores-Wilson K, Biggs MM, Balasubramani GK, Fava M. STAR*D Study Team, 2006. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: implications for clinical practice. Am J Psychiatry. 163(1):28–40. doi: 10.1176/appi.ajp.163.1.28. [DOI] [PubMed] [Google Scholar]
  23. Williams JB, Link MJ, Rosenthal NE. Structured Interview Guide for the Hamilton Depression Rating Scale – Seasonal Affective Disorder Version (SIGH-SAD) New York State Psychiatric Institute; New York: 1992. [Google Scholar]

RESOURCES