Shades of Blue and Gray: A Comparison of the Center for Epidemiologic Studies Depression Scale and the Composite International Diagnostic Interview for Assessment of Depression Syndrome in Later Life

Abstract

Background and Objectives

Psychiatric research lacks the equivalent of a thermometer, that is, a tool that accurately measures mental disorder regardless of context. Instead, the psychometric properties of scales that purport to assess psychopathology must be continuously evaluated. To that end, this study evaluated the diagnostic agreement between the eight-item Center for Epidemiologic Studies Depression Scale (CESD-8) and the Composite International Diagnostic Interview—short form (CIDI-SF) in the Health and Retirement Study (HRS).

Research Design and Methods

Data come from 17,613 respondents aged >50 from the 2014 wave of the HRS. Kappa coefficients were used to assess the agreement between the 2 instruments on depression classification across a range of thresholds for identifying case status, including variation across subgroups defined by age, race/ethnicity, and gender.

Results

The point prevalence of depression syndrome estimated by the CESD was higher than that estimated by the CIDI-SF (CESD: 9.9%–19.5% depending on the cutoff applied to the CESD vs CIDI-SF: 7.7%). Assuming CIDI-SF as the gold standard, the CESD yielded a sensitivity of 56.2%–70.2% and specificity of 84.7%–94.0% across the range of cutoffs. The agreement on depression classification was weak (κ = 0.32–0.44).

Discussion and Implications

Depression cases identified by the CESD have poor agreement with those identified by the CIDI-SF. Conceptually, psychological distress as measured by the CESD is not interchangeable with depression syndrome as measured by the CIDI-SF. Population estimates of depression among older adults based on the CESD should be interpreted with caution.

In psychiatric epidemiology, case identification is important for generating valid and comparable estimates of mental disorders in a population. The meaning of these estimates, whether for service planning or for identifying risk or protective factors, is a function of the validity and reliability of the case identification tools (Blum, 1962; Eaton, Hall, Macdonald, & Mckibben, 2007; Weissman, Sholomskas, Pottenger, Prusoff, & Locke, 1977; Williams, Tarnopolsky, & Hand, 1980). Self-report questionnaires or structured interviews administered by lay persons are often used to assess common mental disorders, such as depression syndrome, in population-based research because they are less burdensome and costly than clinical psychiatric interviews (Blum, 1962; Eaton, Mojtabai, Stuart, Leoutsakos, & Kuramoto, 2012; Levine, 2013; Weissman et al., 1977).

The Center for Epidemiologic Studies Depression Scale (CESD) is among the most widely used tools to assess depression syndrome in population-based research (Eaton, Smith, Ybarra, Muntaner, & Tien, 2004; Steffick, 2000). The original CESD is a 20-item instrument assessing depressed affect, positive affect, somatic complaints, and interpersonal relations over a 1-week period (Eaton et al., 2004; Radloff, 1977; Steffick, 2000; Weissman et al., 1977). Despite its popularity in research, the CESD has major limitations. Although the scale was originally designed as a continuous measure of depressive symptomatology (Eaton et al., 2004; Radloff, 1977; Steffick, 2000), it is commonly used for depression case identification based on dichotomized threshold scores (Polsky et al., 2005; Steffick, 2000; Suglia, Demmer, Wahi, Keyes, & Koenen, 2016; Zich, Attkisson, & Greenfield, 1990), and the optimal cutoff score remains unclear (Dozeman et al., 2011; Henry, Grant, & Cropsey, 2018; Poulin, Hand, & Boudreau, 2005; Radloff, 1977; Schein & Koenig, 1997; Steffick, 2000; Vilagut, Forero, Barbaglia, & Alonso, 2016; Zich et al., 1990). Most importantly, the validity of the CESD as a tool for identifying cases of depression syndrome, and differences in the prevalence of depression across population subgroups, is in question.

Brief History and Overview of Recent Validation Studies of the CESD

The CESD was developed in the Community Mental Health Epidemiology Program (CMH) which consisted of a two sites: Washington County, Maryland (MD) and Kansas City, Missouri (MO; Comstock & Helsing, 1977). The article describing the psychometric properties of the CESD has been cited more than 40,000 times to date (Radloff, 1977), in which the author stated:

The scale was designed for use in studies of the relationships between depression and other variables across population subgroups. To compare results from one subgroup to another, the scale must be shown to measure the same thing in both groups. Therefore, it will be shown that properties of the scale (validity, reliability, factor structure) are similar for the various population subgroups to be studied. (Radloff, 1977, p. 386)

To that end, Radloff’s analysis examined the reliability and criterion validity of the CESD in terms of “age, sex, race, and education[al],” to provide a rationale for using the CESD to make meaningful comparisons across these subgroups (Radloff, 1977).

In this validation study, the CESD was administered at the two CMH sites with a total sample size of 3,845 (n = 295 of whom were black; all black participants came from the Kansas City site). In terms of reliability, Radloff concluded: “Test–retest correlations were moderate (.40 or above) in all but three groups (Blacks, age under 25, and “need help” [for mental distress])” (Radloff, 1977, p. 400). To assess validity, “true” cases of depression were identified from two psychiatric facilities, one in Washington County MD (n = 70) and another in New Haven, Connecticut (n = 35); the diagnosis of these “true” cases was determined by psychiatric assessments and then compared with the CESD, which they were also administered. The racial/ethnic composition of these clinical samples are not described, nor is there discussion of whether the validity of the CESD varied by race.

This brief overview indicates that the CMH study was not well-designed for examining the CESD as a case-identification tool in the general population because the sample was insufficient for drawing conclusions about the reliability or validity of this measure across racial groups. Moreover, the CMH results indicated the CESD did not have adequately test–retest reliability across age and race. Indeed, in their comprehensive review of case-identification tools for depression, Eaton and colleagues (2007) identified no interviewer-blinded, English-language validation studies with a sample size greater than 100 that compared the CESD to diagnosis made by a clinical psychiatric interview (Eaton et al., 2007). Several recent investigations have cautioned against the use of the CESD as a stand-alone case-identification tool for depression syndrome (Eaton et al., 2004; Moon et al., 2017; Vilagut et al., 2016), particularly for persons with medical comorbidity, adolescents, and nonwhite minorities (Kim, Chiriboga, & Jang, 2009; Lu, Lindsey, Irsheid, & Nebbitt, 2017; Perreira, Deeb-Sossa, Harris, & Bollen, 2005; Schein & Koenig, 1997; Yang & Jones, 2008).

Finally, little is known about how the psychometric properties of CESD compare to other depression case-identification instruments in the general adult population. Very often the CESD is the only measure of mental health included in large population-based health surveys (e.g., National Longitudinal Study of Adolescent to Adult Health, Coronary Artery Development in Young Adults, and Multi-Ethnic Study of Atherosclerosis), and thus such comparisons are not possible.

Present Study

The goal of this study is to compare the psychometric properties of the CESD as a function of age, gender, and race/ethnicity with a criterion measure of major depression in the Health and Retirement Study (HRS), a nationally representative survey of U.S. adults aged >50. The HRS is unique in that it includes both an 8-item version of CESD and the short-form version of the Composite International Diagnostic Interview (CIDI-SF) module for major depression. This CIDI-SF module is derived from Diagnostic and Statistical Manual of Mental Disorders (DSM) criteria and has been extensively validated against clinical psychiatric interviews (i.e., the Structured Clinical Interview for DSM [SCID], Schedules for Clinical Assessment in Neuropsychiatry [SCAN]) in both white and nonwhite populations (Alegria et al., 2009; Andrews, Peters, Guzman, & Bird, 1995; Haro et al., 2006; Jackson, Neighbors, Nesse, Trierweiler, & Torres, 2004; Kessler et al., 2004, 2003; Wittchen, 1994). As a result, the HRS provides an opportunity for comparing the agreement between the CESD-8 and the CIDI-SF, and to assess whether these two instruments perform similarly as depression case-identification tools.

Methods

Study Population

The HRS is a longitudinal, nationally representative survey of U.S. adults aged more than 50 years old which oversamples Hispanics, African Americans, and Florida residents. Since 1992, approximately 20,000 respondents are interviewed biennially to assess a range of psychosocial characteristics, health history and health care utilization, and economic circumstances and employment history. Additional details of the study design and sampling method are described elsewhere (HRS, 2008). The HRS is approved by the Institutional Review Board at the University of Michigan (Weir, 2017).

The analytic sample for the present study consisted of 17,613 nonproxy respondents who completed the 2014 HRS Core interview and had complete data on the CESD-8 and the CIDI-SF screening questions. Respondents with missing data on these two assessments were excluded (n = 1,135). Those excluded were older, more likely to be male and nonwhite, had lower socioeconomic status, were less likely to be married, had poorer memory, and were more likely to have history of psychiatric problems (all p < .05; Supplementary Table 1).

Measures

The Composite International Diagnostic Interview—short form

The Composite International Diagnostic Interview—short form (CIDI-SF) depression module is a fully-structured diagnostic interview based on the DSM criteria for major depressive episode (MDE; Kessler, Andrews, Mroczek, Ustun, & Wittchen, 1998; Steffick, 2000). It has been validated against semi-structured clinical psychiatric examinations, and has moderate concordance with diagnosis of MDE from these exams (Eaton et al., 2007; Eaton, Neufeld, Chen, & Cai, 2000; Jackson et al., 2004). The CIDI-SF depression module assesses eight symptom domains of MDE (i.e., sadness, anhedonia/lost interest, sleeping disturbances, appetite changes, fatigue, guilt/worthlessness, concentration problems, thinking about death) experienced over the past 12 months, as shown in Supplementary Table 2 (Kessler et al., 1998; Steffick, 2000). Psychomotor symptoms (e.g., agitation, retardation), while part of the DSM criteria for MDE, are not assessed in this module; however, these are also the least common MDE symptoms in general population samples (Mezuk & Kendler, 2012).

To reduce respondent burden, the full CIDI depression module is only administered to respondents endorse either of the two cardinal depressive symptoms—sadness or anhedonia—at sufficiently intensity. This skip-pattern is shown by Supplementary Figure 1. Respondents are first asked whether there was a period of more than 2 weeks when they had depressed mood or anhedonia over the past 12 months. If either item was endorsed, the respondent was further asked about the intensity and duration of the symptom(s) (Steffick, 2000). Respondents whose symptoms lasted most/all of the day and almost every/every day were then “screened in” and completed the full CIDI module.

We constructed a summary score of the CIDI-SF as follows. For respondents who were screened out of the CIDI-SF (i.e., endorsed neither sadness nor dysphoria at sufficient intensity), a score of zero symptoms was assigned. For respondents who were screened in, a summary score was generated by counting the number of endorsed symptoms. This score ranged from one to eight, with higher score indicating more depressive symptoms. Respondents with a summary score of ≥5 were considered “cases” of past-year MDE, in accordance with DSM criteria (Steffick, 2000).

The Center for Epidemiologic Studies Depression Scale 8

The Center for Epidemiologic Studies Depression Scale 8 (CESD-8), in its entirety, was administered to all respondents. Respondents were asked about whether they experienced any of the eight symptoms listed in Supplementary Table 2 (e.g., felt depressed, felt everything was an effort) “much of the time during the past week,” each recorded as yes or no (Steffick, 2000). The last two items were reverse coded to generate a summary score (range: 0–8) with higher scores indicating more depressive symptoms.

Consistent with its widespread use as a case-identification tool, these CESD-8 summary scores were then dichotomized to identify “cases” of depression syndrome. However, there is not a consistent threshold score that is universally applied to indicate case status using the CESD-8. A score of ≥3 has been used in most studies of this instrument (Steffick, 2000). However, recent a study by Polsky and colleagues, using the HRS, applied a more stringent threshold of ≥5 (Polsky et al., 2005). Other studies also supported a use of more conservative thresholds for older adults (Dozeman et al., 2011; Lyness et al., 1997; Moon et al., 2017; Schein & Koenig, 1997; Schulberg et al., 1985; Steffick, 2000; Zich et al., 1990). Prior work in the HRS has identified that a threshold of ≥4 corresponds to the commonly used cutoff score of 16 in the original 20-item CESD (Steffick, 2000). Therefore, in this study, we explored three thresholds (≥3, ≥4, and ≥5) to indicate depression case status using the CESD-8.

Other variables

Other variables included demographic characteristics, memory, and history of psychiatric problems, all factors associated with the likelihood of having depression among older adults (Clark, Aneshensel, Frerichs, & Morgan, 1981; Cole & Dendukuri, 2003; Cole, Kawachi, Maller, & Berkman, 2000; Karim, Weisz, Bibi, & ur Rehman, 2015; La Rue, Swan, & Carmelli, 1995; Perreira et al., 2005; Rabbitt, Donlan, Watson, McInnes, & Bent, 1995; Yang & Jones, 2008). Demographic variables included age (categorized as ≤55 years, 56–64, and ≥65 years), sex (male, female), race/ethnicity (white/Caucasian, black/African American, other), marital status (never married, currently married/partnered, widowed, divorced/separated), educational attainment (less than high school, high school, more than high school), employment status (currently working, unemployed, retired, other), and total wealth (reported in U.S. dollars and categorized into quantiles). The categories for age, years of education, and wealth were informed by the distribution of these variables in the HRS sample and selected to ensure adequate sample size for subgroup comparisons.

Memory was assessed using both immediate and 5-min delayed recall of 10 words (Bugliari et al., 2016). Both were treated as continuous measures ranging from 0 to 10. History of psychiatric problems was based on self-report of ever having been told by a doctor that they had “emotional, nervous, or psychiatric problems” (Bugliari et al., 2016).

Statistical Analysis

Agreement on depression case status between the CIDI-SF and the CESD-8 was assessed by categorizing respondents into four mutually exclusive groups: (a) not a case on both instruments; (b) case according to CESD-8 but not CIDI-SF; (c) case according to CIDI-SF but not CESD-8; and (d) case on both instruments. We compared sample characteristics across these four groups, using chi-squared tests for categorical variables and Student’s t-tests for continuous variables, and calculated Kappa coefficients (κ) for the agreement on depression case status classification. We also compared the combined discordant groups (Groups b and c) with respondents classified as a case on both instruments (Group d).

We used cross-tabulations to calculate sensitivity, specificity, false positive rate, and false negative rate of the CESD-8 compared with the CIDI-SF, treating the latter as a temporary “gold standard.” We estimated the prevalence of depression syndrome based on the CIDI-SF and the CESD-8, using the cutoffs described above. We then examined the overall agreement on number of endorsed depressive symptoms between the CESD-8 and the CIDI-SF using cross tabulation, although given that these two instruments have different items (Supplementary Table 2) our main interpretation focuses on agreement in case identification rather than in number of symptoms. Finally, we conducted stratified analyses by age, sex, and race/ethnicity. All steps were repeated using the three cutoff scores of the CESD.

All analyses were conducted using SAS, version 9.4 (SAS Institute), accounting for the HRS survey design. All significance tests were evaluated two-sided at the level of p <.05.

Results

Approximately 76% of the 17,613 respondents were considered noncases of depression by both the CESD-8 and the CIDI-SF; 6% were categorized as a case using both instruments. Approximately 17% was classified as a case by CESD-8 only, and 2% was classified as a case by CIDI-SF only (Table 1). Compared with respondents classified as cases by both instruments, those identified by CESD-8 only (using a threshold of ≥3 symptoms to indicate case status) were older, more likely to be male, more likely to be currently employed, had higher total wealth, poorer memory, and were less likely to ever have history of psychiatric problems (all p < .01). Compared with respondents classified as cases by both instruments, those identified by CIDI-SF only were more likely to be non-Hispanic white, have higher education, be employed, had higher total wealth, had better memory, and were less likely to report a history of psychiatric problems (all p < .01). We repeated these analyses using CESD cutoff scores of 4 and 5 (Supplementary Tables 3 and 4). Overall, results were consistent with those using threshold of ≥3. The only exceptions are that with these higher thresholds respondents classified as having depression by CIDI-SF only were more likely to be married/partnered or widowed and that gender and wealth did not vary by depression case status.

Table 1.

Sample Characteristics by Depression Classification Based on the CIDI-SF and the CESD-8: Health and Retirement Study, 2014

		CIDI-SF
		Depressiona (N = 1,399)		No depression (N = 16,214)
Characteristics	Total	CESD-8 depressiona (N = 1,018)	CESD-8 no depression (N = 381)	CESD-8 depressiona (N = 2,917)	CESD-8 no depression (N = 13,297)
Age, N (weighted %)
≤55	2,215	192 (12.7)	72 (12.2)	359 (6.9)	1,592 (7.3)
56–64	5,885	465 (52.7)	142 (46.5)	1,005 (37.6)	4,273 (40.6)
≥65	9,513	361 (34.6)	167 (41.4)	1,553 (55.6)	7,432 (52.1)
p Value	p < .0001b	—	p a = .2c	p b < .0001d	p c < .0001e
Sex, N (weighted %)
Female	10,458	729 (67.3)	281 (67.5)	1,891 (60.8)	7,557 (52.5)
Male	7,151	289 (32.7)	100 (32.5)	1,025 (39.2)	5,737 (47.5)
p Value	p < .0001b	—	p a = 1.0c	p b = .009d	p c < .0001e
Race/ethnicity, N (weighted %)
White/Caucasian	12,533	673 (78.7)	297 (88.9)	1,844 (76.4)	9,719 (84.6)
Black/African American	3,483	212 (12.1)	56 (6.3)	751 (14.5)	2,464 (9.4)
Other	1,545	128 (9.2)	26 (4.8)	311 (9.1)	1,080 (6.0)
p Value	p < .0001b	—	p a = .003c	p b = .4d	p c = .002e
Education, N (weighted %)
Less than high school	3,462	290 (22.6)	61 (12.9)	930 (26.4)	2,181 (11.7)
High school	5,472	312 (32.0)	126 (31.3)	934 (31.3)	4,100 (29.4)
Other	8,592	410 (45.4)	193 (55.8)	1,039 (42.3)	6,950 (58.8)
p Value	p < .0001b	—	p a = .006c	p b = .2d	p c < .0001e
Marital status, N (weighted %)
Never married	883	66 (9.0)	22 (9.8)	195 (9.6)	600 (6.1)
Married/partnered	10,996	491 (48.6)	212 (57.3)	1,481 (49.6)	8,812 (67.9)
Widowed	3,181	213 (18.0)	82 (18.1)	705 (21.1)	2,181 (12.5)
Divorced/separated	2,550	247 (24.3)	64 (14.8)	536 (19.7)	1,703 (13.5)
p Value	p < .0001b	—	p a = .05c	p b = .1d	p c < .0001e
Employment status, N (weighted %)
Currently working	5,312	155 (14.8)	118 (35.0)	620 (23.7)	4,419 (37.2)
Unemployed	407	41 (3.8)	15 (2.1)	84 (2.2)	267 (1.7)
Retired	10,827	682 (67.1)	215 (55.1)	1,942 (67.4)	7,988 (57.3)
Other	1,067	140 (14.2)	33 (7.8)	271 (6.7)	623 (3.9)
p Value	p < .0001b	—	p a < .0001c	p b < .0001d	p c < .0001e
Total wealth, N (weighted %)
First quantile	4,341	482 (42.6)	95 (18.8)	1,130 (32.6)	2,634 (15.4)
Second quantile	4,410	264 (25.4)	106 (28.8)	750 (24.8)	3,290 (22.4)
Third quantile	4,431	160 (17.5)	94 (24.5)	602 (23.0)	3,575 (27.3)
Fourth quantile	4,431	112 (14.5)	86 (27.9)	435 (19.6)	3,798 (34.9)
p Value	p < .0001b	—	p a < .0001c	p b < .0001d	p c < .0001e
Cognitive measures, mean (SE)
No. words recall—immediate	5.32 (0.07)	5.85 (0.09)	5.00 (0.06)	5.69 (0.03)
p Value	p < .0001b	—	p a < .0001c	p b = .0005d	p c < .0001e
No. words recall—delayed		4.15 (0.08)	4.93 (0.10)	3.84 (0.06)	4.71 (0.03)
p Value	p < .0001b	—	p a < .0001c	p b = .002d	p c < .0001e
Psychiatric problems, N (weighted %)
Ever	3,531	724 (73.7)	183 (51.7)	983 (34.9)	1,641 (13.1)
p Value	p < .0001b	—	p a < .0001c	p b < .0001d	p c < .0001e
Current	306	58 (5.5)	21 (6.3)	105 (3.5)	122 (0.9)
p Value	p < .0001b	—	p a = .7c	p b = .09d	p c < .0001e

Note: CESD-8 = eight-item Center for Epidemiologic Studies Depression Scale; CIDI-SF = Composite International Diagnostic Interview—short form; SE = standard error.

^aDepression status defined as: CIDI-SF ≥ 5 or CESD-8 ≥ 3.

^b p Value for difference across four depression status groups.

^c p _a value for difference between respondents who met the depression criteria on both the CIDI-SF and CESD-8 versus only CIDI-SF criteria.

^d p _b value for difference between respondents who met the depression criteria on both the CIDI-SF and CESD-8 versus only CESD-8 criteria.

^e p _c value for difference between respondents who met the depression criteria on both the CIDI-SF and CESD-8 versus neither CIDI-SF nor CESD-8 criteria.

Agreement on Number of Depressive Symptoms

Table 2 presents the agreement between CESD-8 and CIDI-SF on number of depressive symptoms. The numbers on the diagonal represent the number of respondents who had the same number of symptoms on the two scales. Out of 17,613 respondents, 7,646 (43.4%) had zero symptom on both scales, and only 286 (1.6%) had exact agreement. Over 15% of respondents (n = 2,774) with three or more CESD-8 symptoms did not meet the level of symptom intensity necessary to screen into the CIDI-SF module. That is, these respondents were classified as cases of depression based on CESD-8 criteria, whereas they did not even screen into the full CIDI-SF module. In contrast, among respondents who screened into the full CIDI-SF module, only 381 (2.2%) reported five to eight symptoms in CIDI-SF (i.e., met criteria as a case of MDE), but reported <3 symptoms in CESD-8.

Table 2.

Agreement Between the CESD-8 and CIDI-SF in the Number of Symptoms Reported for Depressive Syndrome in the 2014 Health and Retirement Study

Number of depressive symptoms using CESD-8	Number of depressive symptoms using CIDI-SF, number of subjects
	0	1	2	3	4	5	6	7	8	Total
0	7,646	2	8	22	48	41	49	37	15	7,868
1	3,796	2	5	13	24	30	47	37	15	3,969
2	1,686	2	5	16	22	26	36	34	14	1,841
3	1,047	1	2	9	25	21	41	32	16	1,194
4	627	1	6	9	17	18	35	29	14	756
5	440	1	5	16	9	24	39	41	34	609
6	371	0	2	6	13	36	60	67	43	598
7	199	0	1	1	14	29	58	89	74	465
8	90	0	1	0	4	16	40	82	80	313
Total	15,902	9	35	92	176	241	405	448	305	17,613

Note: CESD-8 = 8-item Center for Epidemiologic Studies Depression Scale; CIDI-SF = Composite International Diagnostic Interview—short form.

Agreement on Depression Case Classification

Prevalence of depression syndrome estimated by the CESD-8 was higher than that estimated by the CIDI-SF: CIDI-SF: 7.7% versus CESD-8: 9.9% (threshold ≥ 5), 13.5% (threshold ≥ 4), and 19.5% (threshold ≥ 3). This is despite the fact that the CIDI-SF assesses symptoms over the past 12 months, whereas the CESD only asks about the past week. Temporary assuming MDE identified by the CIDI-SF as the “gold standard,” the CESD-8 had moderate sensitivity (ranging from 56.2% to 70.2%, depending on threshold) and high specificity (ranging from 84.7% to 94.0%, depending on threshold) at identifying cases. As expected, higher threshold scores resulted in lower sensitivity but higher specificity (Figure 1).

Figure 1.

Sensitivity and specificity for different cutoff scores of the CESD-8 to identify depression syndrome when compared with the CIDI-SF Major Depressive Episode, Health and Retirement Study, 2014. Sensitivity and specificity of the CESD-8 across three thresholds of symptom counts (≥3, ≥4, and ≥5) to define depression case status relative to MDE defined by the CIDI-SF.

As illustrated in Table 3, the chance-corrected agreement on depression case classification between the CIDI-SF and the CESD-8 was in the poor to fair range (κ coefficients: 0.32–0.44). In addition, agreement between CIDI-SF screening status (a lower threshold than meeting full MDE criteria) and the CESD-8 indicated only fair agreement across the range of threshold scores (κ_{CES-D ≥ 3} = 0.32 [95% confidence level (CI): 0.30, 0.34]; κ_{CES-D ≥ 4} = 0.38 [95% CI: 0.36, 0.40]; κ_{CES-D ≥ 5} = 0.42 [95% CI: 0.40, 0.44]).

Table 3.

Agreement Between CESD-8 and CIDI-SF Depression Status: Overall and Across Subgroups

	CIDI-SF screening status		Depression syndrome weighted %	Agreement between CIDI-SF and CESD-8
Characteristics	Screened in N (weighted %)	Screened out N (weighted %)		Kappa unadjusted	95% CI	Kappa adjusteda	95% CI
CIDI-SF Scaleb
Mean score (SE)	5.94 (0.06)
Participants with score ≥ 5	1,399 (81.5)		7.7
CESD-8 Scalec
Mean score (SE)	4.12 (0.11)	1.07 (0.02)
Participants with score ≥ 3	1,161 (65.1)	2,774 (14.8)	19.5	0.30	0.28, 0.32	0.32	0.29, 0.35
Participants with score ≥ 4	1,014 (57.4)	1,727 (9.0)	13.5	0.37	0.35, 0.39	0.40	0.36, 0.43
Participants with score ≥ 5	885 (49.9)	1,100 (5.7)	9.9	0.43	0.40, 0.45	0.44	0.40, 0.48
CESD-8 Scale—by agec
≤55 years old
Mean score (SE)	4.36 (0.28)	1.06 (0.07)
Participants with score ≥ 3	208 (66.5)	343 (14.6)	21.9	0.37	0.32, 0.41	0.43	0.33, 0.53
Participants with score ≥ 4	187 (63.8)	231 (8.5)	16.3	0.43	0.38, 0.48	0.54	0.45, 0.64
Participants with score ≥ 5	166 (58.2)	144 (6.0)	13.4	0.48	0.43, 0.53	0.56	0.46, 0.65
56–64 years old
Mean score (SE)	4.31 (0.19)	1.02 (0.03)
Participants with score ≥ 3	510 (68.2)	960 (13.9)	20.0	0.35	0.33, 0.38	0.39	0.35, 0.44
Participants with score ≥ 4	450 (60.5)	610 (8.5)	14.3	0.42	0.39, 0.45	0.46	0.40, 0.51
Participants with score ≥ 5	398 (53.1)	412 (5.7)	11.0	0.46	0.43, 0.50	0.49	0.43, 0.55
≥65 years old
Mean score (SE)	3.82 (0.12)	1.10 (0.03)
Participants with score ≥ 3	443 (61.0)	1,471 (15.6)	18.9	0.23	0.21, 0.25	0.23	0.20, 0.26
Participants with score ≥ 4	377 (52.0)	886 (9.4)	12.6	0.30	0.27, 0.33	0.30	0.26, 0.34
Participants with score ≥ 5	321 (43.8)	544 (5.8)	8.6	0.36	0.33, 0.39	0.35	0.31, 0.40
CESD-8 Scale—by sexc
Female
Mean score (SE)	4.08 (0.11)	1.18 (0.02)
Participants with score ≥ 3	808 (64.8)	1,812 (16.9)	22.4	0.30	0.28, 0.33	0.33	0.30, 0.36
Participants with score ≥ 4	701 (56.7)	1,149 (10.4)	15.7	0.37	0.34, 0.39	0.40	0.36, 0.43
Participants with score ≥ 5	615 (49.6)	742 (6.8)	11.7	0.42	0.39, 0.44	0.43	0.40, 0.47
Male
Mean score (SE)	4.19 (0.23)	0.94 (0.03)
Participants with score ≥ 3	353 (65.8)	961 (12.3)	16.2	0.28	0.25, 0.31	0.30	0.25, 0.36
Participants with score ≥ 4	313 (58.9)	577 (7.4)	11.1	0.37	0.34, 0.41	0.40	0.33, 0.46
Participants with score ≥ 5	270 (50.5)	357 (4.5)	7.8	0.44	0.40, 0.48	0.45	0.38, 0.53
CESD-8 Scale—by racec
White/Caucasian
Mean score (SE)	3.97 (0.12)	0.98 (0.02)
Participants with score ≥ 3	783 (62.8)	1,734 (13.4)	18.1	0.31	0.29, 0.33	0.33	0.30, 0.36
Participants with score ≥ 4	671 (54.7)	1,025 (7.9)	12.3	0.38	0.36, 0.41	0.40	0.37, 0.44
Participants with score ≥ 5	588 (47.4)	630 (4.9)	8.9	0.44	0.41, 0.47	0.45	0.41, 0.49
Black/African Americans
Mean score (SE)	4.80 (0.26)	1.48 (0.05)
Participants with score ≥ 3	236 (77.6)	727 (21.4)	26.4	0.25	0.22, 0.29	0.29	0.23, 0.35
Participants with score ≥ 4	215 (73.1)	477 (14.2)	19.4	0.33	0.29, 0.37	0.38	0.30, 0.46
Participants with score ≥ 5	183 (61.8)	328 (9.5)	14.1	0.38	0.33, 0.42	0.42	0.33, 0.51
Other
Mean score (SE)	4.89 (0.32)	1.49 (0.08)
Participants with score ≥ 3	136 (76.2)	303 (21.5)	27.3	0.33	0.28, 0.38	0.32	0.23, 0.41
Participants with score ≥ 4	122 (69.1)	218 (14.7)	20.4	0.39	0.33, 0.44	0.39	0.29, 0.49
Participants with score ≥ 5	108 (62.8)	136 (10.7)	16.1	0.45	0.39, 0.51	0.43	0.33, 0.54

Note: CESD-8 = 8-item Center for Epidemiologic Studies Depression Scale; CIDI-SF = Composite International Diagnostic Interview—short form; CI = confidence interval; SE = standard error.

^aAdjusted for the HRS survey design and sample weights.

^bDepression syndrome based on CIDI-SF criteria.

^cDepression syndrome based on CESD-8 criteria.

Variation in Agreement by Age, Sex, and Race/Ethnicity

As shown in Figure 2, agreement on number of depressive symptoms between the two scales was poor/fair across age, sex, and race/ethnicity subgroups (κ: −0.0002 to 0.30). In general, among respondents with at least one symptom, Kappa increased as number of symptoms increased. Highest agreement was observed in respondents with eight symptoms (κ ranged from 0.23 to 0.30, depending on the subgroup). The agreement for one to five symptoms was poor for all subgroups (all κ < 0.1). In general, agreement tended to be higher among respondents who were male, younger (aged ≤55) and non-Hispanic white (Figure 2).

Figure 2.

Cohen’s kappa coefficient comparing agreement in number of depressive symptoms across the CESD-8 and CIDI-DF by age (A), gender (B), and race/ethnicity (C) in the 2014 Health and Retirement Survey.

Table 3 shows the agreement on depression case classification between the CESD-8 and the CIDI-SF was poor to fair for all subgroups (adjusted κ range: 0.23–0.56). The Kappa coefficients increased as the cutoff threshold for the CESD-8 increased. Agreement was similar across the subgroups defined by age, gender, and race/ethnicity, and was marginally higher for younger respondents, women, and non-Hispanic whites, regardless of the CESD-8 threshold applied.

Discussion

The primary finding of this study is that CESD-8 and CIDI-SF have a poor agreement in the case identification of depression syndrome in the general older adult population. Using the CIDI-SF as the criterion standard, the CESD-8 had only moderate sensitivity (ranging from 56.2% to 70.2%), regardless of the threshold score. Kappa coefficients for agreement on the number of symptoms and depression case status were consistently in the poor to fair range. There was greater agreement among respondents at the two extremes of the measures (i.e., agreement was highest for those with zero or ≥7 symptoms). These findings were similar across age, gender, and race/ethnicity, with slightly higher (but still relatively poor) agreement among respondents who were younger and male. Overall, our results indicate that depression case-identification measured by the CESD-8 is not interchangeable with case status measured by the CIDI-SF, and that this lack of concordance is not a function of demographic characteristics. This conclusion has implications for both descriptive and analytic epidemiology.

In terms of descriptive epidemiology, the discrepancies we identify here impact the credibility of estimates of the prevalence and distribution of depression in the population. Despite its widespread use as a case-identification tool, our findings provide little evidence that the CESD-8 is an appropriate metric of assessing depression burden in the general population. The CIDI-SF estimates of past-year depression are substantially lower than those estimated by the CESD-8, despite the fact that the CESD-8 only measures current symptomology. Assuming the CIDI-SF as the “gold standard,” the CESD had a high rate of false-positives, consistent with previously studies (Dozeman et al., 2011; Schein & Koenig, 1997; Schulberg et al., 1985; Zich et al., 1990). This makes it a poor stand-alone tool for case identification. If, however, the CESD were instead treated as a first-order screening instrument, one designed to identify individuals who would benefit from more intensive second-order diagnostic testing (say, with a clinician or a diagnostic instrument such as the CIDI-SF), this high false-positive rate would be seen as a strength (Kessler et al., 2002). Indeed, this latter interpretation is more in line with the original writings regarding the utility of the CESD:

Because depression can be characterized as a feeling, a reaction, a syndrome, and an illness, a study of the distribution of the symptoms common to all these variants will obviously suffer from lack of specificity … it seems reasonable to infer that persons with clinical depression are concentrated among persons with high levels of related symptoms, and that the epidemiologic pattern of clinical depression could be revealed, at least in broad outline, by the characteristics of persons with many or persistent symptoms of depression. Medical analogies are provided by hypertensive disease or diabetes mellitus. For both of these diseases, the epidemiologic pattern has been delineated not only by examining the distribution of persons with diagnosed disease but also by studying the distribution of one facet of disease, namely elevated blood pressure or blood sugar, respectively. (Comstock & Helsing, 1977, p. 559)

In sum, the intent and design of the measure (e.g., whether as a case-identification tool like the CIDI-SF or a screening assessment to capture population patterns of psychological distress) needs to be taken into consideration when interpreting the results of the descriptive epidemiology of depression assessments.

The implications for analytic epidemiology are complex. If the CESD-8 was simply a less precise measure of depression than the CIDI, we would expect that the effect estimates generated by studies that use it as a predictor to be biased toward the null. However, longitudinal studies examining depression as a predictor of subsequent health outcomes (e.g., diabetes, cardiovascular disease, mortality) generally report similar findings regardless of whether they use the CESD-8 or instruments like the CIDI (Demakakos, Pierce, & Hardy, 2010; Suglia et al., 2016; Vogelzangs et al., 2010; Womack et al., 2016). It is possible that the CESD is tapping into related constructs beyond depression, such as perceived stress or disengagement, which are also hypothesized to contribute to health outcomes in later life. Apathy, hopelessness, and social withdrawal are central to the concept of depression, particularly in later life, and are prominent in measures such as the Geriatric Depression Scale which includes several items to assess these thoughts and feelings (e.g., Do you feel that your life is empty? Do you often get bored? Have you dropped many of your activities and interests?; Adams, Matto, & Sanders, 2004). The CESD differs from the CIDI-SF in that it measures two positive elements of affect (i.e., felt happy, enjoyed life), which are then reverse-coded; however, positive and negative affect are not mutually exclusive (Diener & Emmons, 1984). While there are sound psychometric reasons for including reverse-coded items in scales, others have questioned this approach to measuring depression syndrome (Carleton et al., 2013). In sum, the CESD may be sufficient as a measure of “non-specific psychological distress” as a risk factor for subsequent outcomes, including depression syndrome. However, our findings indicate the CESD-8 is not appropriate for studies that seek to identify correlates of depression or how those correlates vary across subgroups. Finally, from an intervention and mental health services planning perspective, the clinical significance of depression case status as indexed by the CESD is simply unknown.

Findings should be interpreted considering study limitations. We acknowledge that even clinical assessments of depression status by psychiatrists do not agree completely, reflecting the challenge of case identification in psychiatry more generally (Blum, 1962; Eaton et al., 2007, 2012; Kessler, 2000; Williams et al., 1980). The CIDI-SF depression module has moderate agreement on MDE diagnosis compared with clinical psychiatric interview (Cooper, Peters, & Andrews, 1998; Eaton et al., 2000; Kessler et al., 1998; Wittchen, 1994), with Kappa coefficients in the range of 0.4–0.5 (Alegria et al., 2009; Brugha, Jenkins, Taub, Meltzer, & Bebbington, 2001; Kessler et al., 2004; Williams et al., 2007). Much of the discrepancy between instruments like the CIDI and psychiatric interviews comes from the latter identifying more symptomology (Eaton et al., 2000), that is, the CIDI is likely a conservative measure of MDE.

The study also has several strengths. First, this study examined the agreement of CESD-8 with the CIDI-SF specifically in the context of case identification in a general population sample. The CESD is the most frequently used, and often the only index of mental health included, in large population cohorts. Our findings therefore have implications for a large body of literature. Second, to the best of our knowledge, this is the largest study to compare the CESD and CIDI-SF in the same cohort. A recent meta-analysis study found only 28 validation studies with both the CESD and other diagnostic instruments; only three of these studies had a sample size >1,000, and none were larger than 2,500 (Vilagut et al., 2016). Third, we examined how the performance of the CESD varied across demographic subgroups. The U.S. older adult is becoming increasingly diverse, and it is important that our measures to assessment mental health are appropriate for this population.

Mental health is an important and understudied component of overall population health, particularly as populations age. In the time since the CESD was first proposed, multiple advances have been made in survey methods for assessing psychiatric disorders in the general population, and the widespread reliance on the CESD in population-based cohorts does not reflect those advances. Large population-based cohorts and surveillance surveys should include multiple measures of mental health in the same way they include multiple measures of physical health. These measures should reflect the most reliable and validated metrics available to have confidence in the inferences drawn from these efforts.

Funding

This study was supported by a grant from the National Institutes of Health (P30 AG015281), and the Michigan Center for Urban African American Aging Research.

Conflict of Interest

None reported.