Validity of Center for Epidemiologic Studies Depression (CES-D) scale in a sample of Iraq and Afghanistan Veterans

Abstract

Objectives:

Optimal depression screening necessitates measurement tools that are valid across varied populations and in the presence of comorbidities.

Methods:

This study assessed the test properties of two versions of the Center for Epidemiologic Studies Depression scale against psychiatric diagnoses established by the Mini International Neuropsychiatric Interview among a clinical sample of US Veterans deployed during Operations Enduring Freedom, Iraqi Freedom, and New Dawn. Participants (N = 359) recruited from two Department of Veterans Affairs hospitals completed a clinical interview, structured diagnostic interview, and self-reported measures.

Results:

Based on diagnostic interview and the Diagnostic and Statistical Manual of Mental Disorders 4th Edition criteria, 29.5% of the sample met diagnostic criteria for major depressive disorder and 26.5% met diagnostic criteria for post-traumatic stress disorder. Both Center for Epidemiologic Studies Depression-20 and Center for Epidemiologic Studies Depression-10 scales performed well and almost identically against the Mini International Neuropsychiatric Interview-major depressive disorder in identifying Veterans with major depressive disorder (Center for Epidemiologic Studies Depression-20 area under the Receiver Operating Characteristic curve 91%; Center for Epidemiologic Studies Depression-10 area under the ROC curve 90%). Overall, higher cut points for the Center for Epidemiologic Studies Depression scales performed better in correctly identifying true positives and true negatives for major depressive disorder (Center for Epidemiologic Studies Depression-20 cut point 18+ sensitivity 92% specificity 72%; Center for Epidemiologic Studies Depression-10 cut point 10+ sensitivity 92% specificity 69%).

Conclusions:

The specificity of the Center for Epidemiologic Studies Depression scales was poor among Veterans with co-occurring post-traumatic stress disorder (13% and 16%). Veterans with post-traumatic stress disorder who have a positive depression screen should have a more thorough assessment of mental health symptoms and comorbidities, rather than immediate diagnosis of and treatment for depression.

Introduction

Major depressive disorder (MDD) is a common mental health problem affecting nearly one out of every six adults in the United States.¹ This condition is associated with significant disability, symptom burden, and health care costs.^1,2 Identifying depression in community and primary care settings allows for targeting interventions,³ but screening tools for depression may not be equally valid across clinical populations, or in the context of medical or other psychiatric comorbidities.

Veterans have much higher levels of depression relative to the general US population,⁴ including younger cohorts of Veterans involved in the Iraq and Afghanistan conflicts.⁵ Veterans are also at higher risk for other debilitating mental health conditions, including post-traumatic stress disorder (PTSD) and traumatic brain injury.^6–8 Adequately assessing health needs of returning service members is important both for providing clinical care and allocating resources.

The accurate identification of MDD may pose significant challenges because it co-occurs and overlaps with other mental health conditions. A number of widely used scales may accurately identify depressive symptoms, although they may not correlate entirely with a clinical diagnosis. The Center for Epidemiologic Studies Depression scale (CES-D) has wide applicability and utility in identifying depressive symptoms in people with traumatic injury and disability in VA health care settings^9,10 and has consistently demonstrated excellent screening properties for detecting depression outcomes.¹¹ In addition, full and shortened versions of the CES-D scale have been used to assess depression outcomes, with the shortened scale demonstrating comparable test properties to the full scale when used as a screening tool to identify MDD.^12–14

There is a high degree of comorbidity and symptom overlap between PTSD and MDD among Veterans.^15–17 Patients with depression alone and patients with PTSD and depression together exhibit very different disease severities, symptomatologies, and progression. Further, patients diagnosed with both MDD and PTSD experience delayed response to depression treatment.¹⁶ The presence of PTSD may thus alter the properties of depression screening tests.

The objective of this study is to evaluate the validity of the CES-D (the full 20-item and shortened 10-item scales) at various cutoff points for a population of Veterans using a gold standard established by the Mini International Neuropsychiatric Interview (MINI-MDD). In order to assess the validity of the full and shortened CES-D scales, we examined the sensitivity, specificity, and receiver operating characteristic (ROC) curves. In addition, we examined these test properties for Veterans with PTSD and without PTSD.

Methodology

This was a cross-sectional study of a cohort of returned Veterans from Operations Enduring Freedom (OEF), Iraqi Freedom (OIF), and New Dawn (OND) who were either receiving care or had registered for care at VA health care facilities. Recruitment details of the study population are described elsewhere.⁸

Study sites

Data were drawn from two (one northern and one southeastern) of the five VA facilities with regional inpatient Polytrauma Rehabilitation Centers (PRCs). Both facilities are large, tertiary care hospitals that provide a broad range of comprehensive medical and mental health care, and both maintain a comprehensive list (OEF/OIF/OND registry) of current or former service members who have applied for VA services in their region. Full details of the parent study protocols, informed consent procedures, and Institutional Review Board approvals are published elsewhere.⁸ Informed consent procedures were approved by the Institutional Review Boards and Research and Development Committees for each of the two VA facilities (one northern and one southeastern) that conducted participant enrollment and data collection. As secondary data analysis of existing, de-identified clinical information, the current study was exempt from human subject review by the Institutional Review Board of Oregon Health & Science University.

Eligibility and recruitment

Eligible participants were deployed during OEF/OIF/OND between October 2001 through September 2010, able to read and write in English, and able to complete study forms, scales, and provide informed consent. Eligible individuals with moderate to high traumatic brain injury and who met criteria for minimal to moderate cognitive impairment¹⁸ required clearance by an attending physician in order to participate.

Measures

Electronic health record data

Descriptive information was extracted from electronic health records, including demographic information, physical and psychiatric diagnoses, and health history.

MINI diagnostic interviews

Psychiatric diagnoses were obtained via the MINI.¹⁹ The MINI is a brief, validated, structured clinical interview designed to provide reliable Axis I psychiatric diagnoses in accordance with the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) for various psychiatric disorders.²⁰ The MINI battery of clinical interviews administered to the study population yields diagnoses for a number of psychiatric conditions, including MDD and PTSD. Each MINI measure was dichotomized as having met criteria for having the condition or not, using the established cut points.¹⁹

Depressive symptoms

Depressive symptomatology was assessed using the full, self-reported 20-item Center for Epidemiologic Studies-Depression Scale (CESD-20) as well as the shortened 10-item CES-D (CESD-10) (see Appendix 1).^12,21 Total scores range from 0 to 60 for the CESD-20 scale and from 0 to 30 for the CESD-10, with higher scores indicating greater depressive symptoms. Previously, while the norm for the cut point of the CESD-20 has been 16 points, some authors have argued for cut points at 20 or above.^22–24 We assessed various cut points: 16+ and 18+ for the CESD-20 and 8+ and 10+ for the CESD-10.^12,13,25

Statistical analyses

We assessed the validity of the CESD-20 and CESD-10 against MDD, as defined by the MINI diagnostic interview. We assessed the sensitivity and specificity of the CESD-10 and the CESD-20 against the MINI-MDD. More specific properties of the CES-D scales were examined with ROC curves. The area under the ROC curve is interpreted as the probability that the result of the diagnostic test of a randomly selected individual with MDD will be greater than the result of the same diagnostic test from a randomly selected healthy individual. This approach also can be used to delineate cut points for the scales. The greater the area under the ROC curve the better the diagnostic test performs. We used the cut-offs of 0.9–1 to indicate excellent accuracy versus the MINI, 0.8–0.9 to indicate good accuracy, and 0.7–0.8 to indicate fair accuracy.²⁶ Because PTSD is a highly significant and frequently co-occurring condition for this population, we examined whether the performance of the CES-D was similar among Veterans with and without PTSD using the MINI-PTSD diagnostic assessment. We used Stata version 13.1 to conduct all analyses.

Results

This study included 359 Veterans, 91% of whom were male. The mean age was 35 years. The sample demographic characteristics are summarized in Table 1. Most participants were married or living with a partner (59%) and the vast majority were non-Hispanic white (77%). Few medical comorbidities were assessed in this population. 56% were experiencing clinically significant pain (assessed by a rating of usual pain intensity over the last week of >3 on the Numeric Rating Scale, ranging from 0 to 10) and 16% reported postconcussional disorder (assessed using DSM-IV diagnostic criteria). The MINI psychiatric interviews identified sample population prevalence of 29.5% for MDD, 26.5% for PTSD, 28.7% for agoraphobia, 23.1% for hypomania or mania, 18.1% for panic disorder, and 13.9% for alcohol dependence. Approximately half of the study population scored above the lower cut points on the CES-D depressive symptoms scales. The mean score for depressive symptoms was 18.5 for the CESD-20; for the CESD-10 the mean was 10.2.

Table 1.

Descriptive results of a sample of n = 359 US Veterans of Operations Enduring Freedom (OEF) Iraqi Freedom (OIF) New Dawn (OND).

Characteristic	N (%)	Mean (SD)	Range
Men	327 (91)
Age		35.1 (9.7)	20–66
Marital status
Married	188 (52)
Co-habitating	24 (6.7)
Divorced/separated	60 (16.7)
Never married	87 (24.2)
Years education (n = 356)		14.5 (2.0)	12–22
Race
Non-Hispanic white	278 (77.4)
Non-Hispanic black	34 (9.5)
Hispanic	38 (10.6)
Asian	6 (1.7)
Other	3 (0.8)
Clinically Significant Paina	201 (56)
Post-concussional disorder	58 (16.2)
Met criteria:
MINI-MDD	106 (29.5)
MINI-PTSD	95 (26.5)
MINI-Dysthymia	21 (6.6)
MINI-Panic disorder	65 (18.1)
MINI-ETOH dependence	50 (13.9)
MINI-Psychotic conditions	18 (5)
MINI-Agoraphobia	103 (28.7)
MINI-Hypomania or mania	83 (23.1)
MINI-Social phobia	34 (9.5)
CESD-20		18.5 (12.9)	0–56
Positive for depressive symptoms (16+ points)	182 (50.7)
Positive for depressive symptoms (18+ points)	169 (47.1)
CESD-10		10.2 (7.2)	0–30
Positive for depressive symptoms (8+ points)	205 (57.1)
Positive for depressive symptoms (10+ points)	175 (48.7)

SD: standard deviation; MINI-MDD: Mini International Neuropsychiatric Interview Diagnosis for Major Depressive Disorder; MINI-PTSD: Mini International Neuropsychiatric Interview Diagnosis for Post-Traumatic Stress Disorder; CES-D: Center for Epidemiologic Studies Depression Scale.

^aClinically Significant Pain was defined as “usual” pain intensity over the last week rated as >3 on the Numeric Rating Scale (range 0–10).

Both the CESD-20 and CESD-10 performed well against the MINI-MDD in identifying MDD at the various cut points examined (Table 2; Figure 1). The test properties of the CESD-20 were comparable for the 16+ cut point (sensitivity 95%, specificity 68%, area under ROC curve 91%) and the 18+ cut point (sensitivity 92%, specificity 72%, area under ROC curve 91%). For the CESD-10, the test properties were also fairly similar for the 8+ cut point (sensitivity 97%, specificity 60%, area under the ROC curve 90%) compared to the 10+ cut point (sensitivity 92%, specificity 69%, area under ROC curve 90%). Figure 1 illustrates the excellent test properties of the full and shortened CES-D scales against the MINI-MDD, and the very similar performance of the two CES-D scales.

Table 2.

Descriptive information for condition overlap between MINI-MDD and MINI-PTSD (met criteria) among US Veterans of OEF/OIF/OND.

Condition overlap for MINI-MDD and MINI-PTSD (met criteria), n (percent of total)
	MINI-MDD Positive	MINI-MDD Negative
MINI-PTSD Positive	64 (18%)	31 (9%)
MINI-PTSD Negative	42 (12%)	222 (62%)
CESD-20 and CESD-10 scores based on MINI-MDD and MINI-PTSD categories, mean(sd)
	MINI-MDD Positive	MINI-MDD Negative
MINI-PTSD Positive	34.0 (8.8)/18.6 (4.6)	25.4 (8.7)/14.1 (5.3)
MINI-PTSD Negative	28.3 (10.2)/15.8 (5.7)	11.1 (8.3)/6.2 (4.8)

MINI-MDD: Mini International Neuropsychiatric Interview Diagnosis for Major Depressive Disorder; MINI-PTSD: Mini International Neuropsychiatric Interview Diagnosis for Post-Traumatic Stress Disorder; SD: standard deviation; CES-D: Center for Epidemiologic Studies Depression Scale.

Figure 1.

Receiver Operating Characteristic (ROC) curves for MINI-major depressive disorder and CES-D.

The stratified analyses by Veterans with and without PTSD showed differences in the performance of the CES-D against the MINI-MDD. We examined both CES-D scales at the higher cut points because of the comparable test properties and higher specificities demonstrated by the CESD-20 at the 18+ cut point and the CESD-10 at the 10+ cut point. Both scales were more accurate in identifying MDD for Veterans without PTSD (for CESD-20: sensitivity 81%, specificity 80%, area under ROC curve 90%; for CESD-10: sensitivity 81%, specificity 77%, area under ROC curve 89%) (Table 3; bottom of Figure 2). For Veterans with PTSD, however, the specificity was quite low for both the CESD-20 (sensitivity 98%, 13% specificity, area under ROC curve 75%) and CESD-10 (sensitivity 98%, specificity 16%, area under ROC curve 73%) (Top of Figure 2; Table 4).

Table 3.

Validity of Center for Epidemiologic Studies Depression Scale (CES-D) Versions compared to the mini international neuropsychiatric interview among US Veterans of OEF/OIF/OND.

	Major Depressive Disorder using Mini International Neuropsychiatric Interview
CESD-20 (cut point 16+)
Sensitivity	95.28%
Specificity	67.98%
Area under ROC curve	0.91
CESD-20 (cut point 18+)
Sensitivity	91.51%
Specificity	71.54%
Area under ROC curve	0.91
CESD-10 (cut point 8+)
Sensitivity	97.17%
Specificity	59.68%
Area under ROC curve	0.90
CESD-10 (cut point 10+)
Sensitivity	91.51%
Specificity	69.17%
Area under ROC curve	0.90

CESD: Center for Epidemiologic Studies Depression Scale; ROC: Receiver Operating Characteristic.

Figure 2.

Receiver Operating Characteristic (ROC) curves for MINI-MDD and CES-D for Veterans with (n = 95) and without (n=264) PTSD.

Table 4.

Differences in validity by diagnostic subgroups among US Veterans of OEF/OIF/OND.

	Major depression using Mini International Neuropsychiatric Interview
CESD-20 (cut point 18+)
Veterans with PTSD	N = 95
Sensitivity	98.4%
Specificity	12.9%
Area under ROC curve	0.75
Veterans without PTSD	N = 264
Sensitivity	81.0%
Specificity	79.7%
Area under ROC curve	0.90
CESD-10 (cut point 10+)
Veterans with PTSD	N = 95
Sensitivity	98.4%
Specificity	16.1%
Area under ROC curve	0.73
Veterans without PTSD	N = 264
Sensitivity	81.0%
Specificity	76.6%
Area under ROC curve	0.89

PTSD: Post-Traumatic Stress Disorder; CESD: Center for Epidemiologic Studies Depression Scale; ROC: Receiver Operating Characteristic.

Discussion

Using data based on face-to-face clinical interviews and a validated clinical assessment tool as the gold standard, we found that both the CESD-20 and CESD-10 demonstrated excellent test properties in classifying MDD. We found only small differences in test properties between the CESD-20 and the CESD-10 against the MINI-MDD in any of our analyses. There was no evidence that the longer CESD-20 scale provided any meaningful benefits. We found moderate support for using the higher diagnostic cut points on both scales. Our results confirm the utility of using the CES-D scale, and the shortened CESD-10 battery of questions in particular, to screen Veterans for MDD.

Importantly, we found that the test properties of the CES-D are different for Veterans with and without PTSD. The CES-D scales performed better at identifying MDD for Veterans without co-occurring PTSD. For those with co-occurring PTSD, the specificity was quite low (13% and 16%) at the cut points we used. These findings could argue for a higher MDD cut point on the CES-D scale for Veterans with PTSD, but doing so would diminish the sensitivity of the tests (as seen in the upper graphs in Figure 2). Instead, we recommend that any Veteran with PTSD who has a positive depression screen should have a thorough assessment of depressive symptoms and comorbidities prior to diagnosis or treatment.

PTSD shows heterogeneous symptoms, and other research has found substantial symptom overlap between PTSD and MDD. Several key symptoms, especially numbness and dysphoria, appear to explain this overlap.¹⁵ Our findings are generally consistent with research suggesting that these two conditions may influence each other in significant ways.^16,17 Our results do not resolve whether the two are part of a single general mood disorder, as has been proposed.²⁷ In all, 18% of our study sample had co-occurring MDD and PTSD, which was considerably higher than the 9% found in a primary care VA population¹⁵ but a large number had only one of the conditions. Additional research can identify areas of overlap and differentiation, as well as the consequences (such as differences in longitudinal course) of various manifestations of PTSD and MDD.

It is likely that the PHQ-9, which the VA has adopted for use during clinical encounters,²⁸ operates similarly to the CES-D in terms of test properties.²⁹ However, there has been limited research about the test properties of the PHQ-9 among broader Veteran populations, or about its properties among Veterans with co-occurring PTSD. Further examination of how the CES-D and PHQ-9 operate similarly or differently is warranted, given utility and applicability of each in population-based and clinical settings, respectively.²⁹

There are several advantages to using screening survey measures. First, self-reported survey tools can quickly ascertain information from respondents at-large. As a result, health care systems such as the VA could feasibly distribute the CES-D questionnaire widely to difficult-to-reach populations. Second, clinical interviews such as the MINI-MDD require face-to-face assessments of patients conducted by trained professionals. In contrast, the CES-D need not be administered face-to-face, and data collection can be attained through various modes—phone-assisted interviews, mailed paper questionnaires, or via Internet access.³⁰ Measures used in self-reported surveys can be standardized for comparisons to other populations. Our study encourages the use of the CESD-10 scale, with a cut point of 10, but with the caveat that the test has low specificity among Veterans with co-occurring PTSD. It is possible that screening for specific symptoms of PTSD and depression (such as numbness and dysphoria)¹⁵ could promote more targeted mental health interventions, since the group in this overlap seems to experience the most severe symptoms.

Some aspects of our study may limit its clinical relevance. While the CES-D indices may be useful in identifying Veterans with likely MDD, they do not assess either the severity of depressive symptoms, or comorbid mental health diagnoses such as anxiety disorders, PTSD, and substance abuse, which are important factors for clinical care. It will be important in future research to clarify the degree and consequences of overlap between comorbid mental conditions²⁷ as well as with co-occurring medical conditions. Initial screening with the CESD-10 would require follow-up with additional clinical history, or another assessment tool, especially in the setting of PTSD. However, if the goal is to maximize identification, assessment, and treatment of Veterans with MDD, the CESD-10 may be a particularly useful and viable screening tool for Veteran populations at large.

Conclusion

This study confirmed the predictive accuracy of the CESD-20 and CESD-10 in identifying Veterans with MDD. The CESD-20 offered no advantages. However, the specificity of the CES-D for identifying MDD was low among Veterans with co-occurring PTSD, which encourages more detailed mental health assessment in that group.

Appendix 1

The original CES-D contains 20 items. The list below shows the original CESD-20 item notations, the corresponding numbers for the CESD-10 in parentheses. Positive items are in italics. Scoring is a simple sum (after reversing scores for the positive items).

During the past week:

• A1(1) I was bothered by things that usually don’t bother me.

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1–2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3–4 Days)
  • 3 Most or All of the Time (5–7 Days)

• E5(2) I had trouble keeping my mind on what I was doing.

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1–2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3–4 Days)
  • 3 Most or All of the Time (5–7 Days)

• F6(3) I felt depressed.

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1–2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3–4 Days)
  • 3 Most or All of the Time (5–7 Days)

• G7(4) I felt that everything I did was an effort.

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1-2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3-4 Days)
  • 3 Most or All of the Time (5-7 Days)

• H8(5) I felt hopeful about the future.

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1-2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3-4 Days)
  • 3 Most or All of the Time (5-7 Days)

• J10(6) I felt fearful.

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1-2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3-4 Days)
  • 3 Most or All of the Time (5-7 Days)

• K11(7) My sleep was restless.

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1-2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3-4 Days)
  • 3 Most or All of the Time (5-7 Days)

• L12(8) I was happy.

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1-2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3-4 Days)
  • 3 Most or All of the Time (5-7 Days)

• N14(9) I felt lonely.

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1-2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3-4 Days)
  • 3 Most or All of the Time (5-7 Days)

• T20(10) I could not get “going.”

  • 0 Rarely or None of the Time (Less than 1 Day)
  • 1 Some or a Little of the Time (1–2 Days)
  • 2 Occasionally or a Moderate Amount of Time (3–4 Days)
  • 3 Most or All of the Time (5–7 Days)