Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Apr 1.
Published in final edited form as: Aging Ment Health. 2018 Feb 9;23(4):507–514. doi: 10.1080/13607863.2017.1423037

Optimal metrics for identifying long term patterns of depression in older HIV-infected and HIV-uninfected men who have sex with men

Nicole M Armstrong 1, Pamela J Surkan 2, Glenn J Treisman 3, Ned C Sacktor 4, Michael R Irwin 5, Linda A Teplin 6, Ron C Stall 7, Lisa P Jacobson 8, Alison G Abraham 9
PMCID: PMC6085148  NIHMSID: NIHMS945082  PMID: 29424569

Abstract

Objectives

Center of Epidemiologic Studies–Depression Scale (CES-D) provides a snapshot of symptom severity at a single point in time. However, the best way of using CES-D to classify long-term depression is unclear.

Method

To identify long-term depression among HIV-infected and HIV–uninfected 50+ year-old men who have sex with men (MSM) with at least 5 years of follow-up, we compared sensitivities and specificities of CES-D–based metrics (baseline CES-D; four consecutive CES-Ds; group-based trajectory models) thresholded at 16 and 20 to a clinician’s evaluation of depression phenotype based on all available data including CES-D history, depression treatment history, drug use history, HIV disease factors, and demographic characteristics.

Results

A positive depressive phenotype prevalence was common among HIV-infected (prevalence=33.1%) and HIV-uninfected MSM (prevalence=23.2%). Compared to the depressive phenotype, trajectory models of CES-D≥20 provided highest specificities among HIV-infected (specificity=99.9%, 95% Confidence Interval[CI]:99.4%–100.0%) and HIV-uninfected MSM (specificity=99.0%, 95% CI:97.4%–99.7%). Highest sensitivities resulted from classifying baseline CES-D≥16 among HIV-infected MSM (sensitivity=75.0%, 95% CI:67.3%–81.7%) and four consecutive CES-Ds≥16 among HIV-uninfected MSM (sensitivity=81.0%, 95% CI:73.7%–87.0%).

Conclusion

Choice of method should vary, depending on importance of false positive or negative rate for long-term depression in HIV-infected and HIV-uninfected MSM.

Keywords: depression, HIV infection, sensitivity, specificity, validity

Introduction

Questionnaires, such as the 9-item Patient Health Questionnaire (Kroenke, Spitzer, & Williams, 2001), the Centers for Epidemiologic Studies – Depression Scale (CES-D) (Radloff, 1977), the Beck Depression Inventory (Beck, Steer, & Brown, 1996), the Clinically Useful Depression Outcome Scale (Zimmerman, Chelminski, McGlinchey, & Posternak, 2008), and the Quick Inventory of Depressive Symptomatology (Rush et al., 2003), have been used to capture clinically relevant depressive symptoms in epidemiologic studies. These questionnaires and other similar scales consist of items related to criteria from the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition for the diagnosis of a spectrum of depressive disorders ranging from mild to severe (American Psychiatric Association, 2000). Most epidemiologic studies rely on questionnaires as screening, not diagnostic, tools to assess the presence of depressive symptoms at a single time point. However, a single assessment of depressive symptoms does not necessarily identify those at highest risk for long-term patterns of depression -- often the construct of interest (Kaup et al., 2016).

There are many challenges in trying to identify long-term depression risk based on a questionnaire. Depressive symptoms reported at a single time point may not represent clinically relevant depression, since depression is thought to be a state that is mutable over time rather than a trait that stays constant over time (Gotlib and Joormann, 2010). Questionnaire data may only be useful for identifying those with longer term patterns of depression when repeated assessments are considered (Kaup, et al., 2016). The appropriate threshold on a semi-continuous depressive symptom severity scale my not be clear for long-term risk; likely it varies across populations. A cut-off of 16 on the CES-D scale is typically used to define whether an individual has clinical relevant depressive symptoms (Radloff, 1977); yet a more stringent threshold of 20 could be more sensitive for detecting clinical depression in some older or diseased groups (Lyness et al., 1997). As a result of this uncertainty, it is unclear how best to use questionnaire data, like the CES-D, to ascertain a relevant long-term risk profile.

Regardless of the dearth of evidence on optimal metrics for identifying long-term depression risk, depression questionnaires and instruments continue to be used to identify depressed participants in research, as clinical interviews -- the gold standard for diagnosing clinical depression-- are often not feasible because of costs and participant burden. Thus, evaluating the performance of simple CES-D-based metrics for accurately identifying those at risk for clinically relevant recurrent depressive episodes would be important for assuring valid inferences in research studies of depression.

In this study, we sought to evaluate the discrimination performance of several CES-D based depression metrics to inform research practice. The context for our study was the Multicenter AIDS Cohort Study (MACS), a cohort of HIV-infected and HIV-uninfected men who have sex with men (MSM), as both MSM and HIV-infected individuals are at substantial risk for depression (Fendrich, Avci, Johnson, & Mackesy-Amiti, 2013; Perdue, Hagan, Thiede, & Valleroy, 2003; Reisner et al., 2009).Clinical depression has substantial health and quality of life implications in this population (Bhatia and Munjal, 2014). Assessment through clinical interviews were not available for the study; however, a clinical psychiatrist reviewed the complete longitudinal histories of relevant clinical indicators of depression to classify each participant as having a depressive phenotype or not. Then, CES-D-based metrics were compared to the depressive phenotype and evaluated for accuracy to provide guidance for researchers on the optimal metric for characterizing long-term patterns of depression.

Methods

Study sample

The MACS is a prospective study of the natural history of HIV infection among men who have sex with men (MSM) that began in 1984 (Kaslow et al., 1987). Study design, eligibility criteria, and recruitment are described elsewhere (Silvestre et al., 2006). Participants aged 50 years and older with at least five years of follow-up data (10 CES-D measures) on depressive symptoms collected semiannually (N=2,329) were included in the analysis. We excluded participants with an indeterminate depressive phenotype (N=762). The inclusion period was from January 15, 1985 to March 31, 2015. The sample was restricted to visits at or after age 50, as the risk of depression is higher among older HIV-infected individuals (McArthur and Brew, 2010), yielding higher rates of depression from which to estimate accuracy of classification. HIV status was defined by status at age 50 or the closest included visit and seroconverters (n=18) were excluded. The study protocol was approved at each institution’s IRB, and each participant provided written consent.

Classification of long term depression patterns based on clinical expertise

Instead of clinical interviews to assess clinical depression, the MACS collects semiannual measures of CES-D and extensive information on drug use patterns, behaviors, treatment history, HIV history and family history of depression -- all important and established indicators of depression risk. A psychiatrist specializing in HIV (GJT) and three investigators (NMA, PJS, AGA) reviewed all available longitudinal data (example shown in Figure 1) on self-reported depressive symptoms (CES-D), viral load, CD4 count, age, HIV serostatus, use of antiretroviral therapy (e.g., Efavirenz), cocaine use, intravenous drug use, use of antidepressants, age, and Hepatitis C status across all years under observation in the MACS study for each included participant. Fixed characteristics were also considered including race and nadir CD4 count. The variables that were considered in the assessment of the depression phenotype were based on the literature and clinical expertise, as summarized here: depression is a risk factor for both HIV viral load increase and CD4 T-cell count decline, indicating that depression may act to enhance viral replication or suppress immune function in PLWH (Horberg et al, 2008; Ironson et al, 2005). Efavirenz use is known to cause depressive symptoms (Panel on Antiretroviral Guidelines for Adults and Adolescents). HIV-infected individuals may have psychiatric and/or drug dependence disorders (Bing et al., 2001). Greater severity of depression is associated with greater frequency of injection drug use (Stein, Solomon, Herman, Anderson, & Miller, 2003). Use of antidepressant therapy can indicate if depression is controlled after clinical diagnosis (O’Connor, Whitlock, Gaynes, & Beil, 2009). Individuals co-infected with Hepatitis C and HIV report high prevalence of depressive symptoms (Marcellin et al., 2007).

Figure 1.

Figure 1

Open in a new tab

Clinically-identified Approach to Classifications of a Depressive Phenotype

Panels A, B, and C illustrate the way that the psychiatrist and investigators classified the depressive phenotype. The following criteria were applied to each panel: (1) more visits with CES-D scores ≥16 over the course of the study; (2) at least two episodes of depression, as characterized by substantial increase in CES-D score ≥16, lasting at least 1.5 years (≥ 3 CES-D measurements). Panel A shows a participant with a depressive phenotype. Panel B depicts the occurrence when there are multiple visits with a CES-D score between 16 and 20. The slight curves above CES-D scores of 16 are not substantial increases in CES-D score, so this participant was classified as not having the depressive phenotype. Panel C shows the depressive trajectories below CES-D scores under 16, so this participant was classified as not having the depressive phenotype. Time from HAART or time from baseline is measured in years.

Using graphical displays of clinical history, the following criteria were used to guide classification of the depressive phenotype: (1) more visits with CES-D scores ≥16 over the course of follow-up; (2) at least two episodes of depression, as characterized by substantial increase in CES-D score ≥16, lasting at least 1.5 years (≥3 CES-D measurements) (Figure 1). Other clinically relevant variables were considered in the classification of depressive phenotype, based on clinical judgment of a psychiatrist (GJT).

Participants were classified as having a depressive phenotype, a non-depressive phenotype or indeterminate phenotype by each investigator. Self-reported depression treatment alone was not considered sufficient for depressive phenotype classification. Disagreements in classification were adjudicated; if consensus could not be reached, status was considered indeterminate. In the present analysis, we included only those with a consensus classification of depressive or non-depressive phenotype. Indeterminant classifications also resulted from too few data points or gaps over a period of many years that made it impossible to determine the CES-D pattern over time. Final prevalence estimates from the included sample were compared to prevalence estimates from the literature.

CES-D-based metrics for classification of depressive phenotype

Classification using CES-D scores at a single time-point

The CES-D score at age 50 or the closest visit was used as one CES-D –based metric for determining depression status, representative of a standard cross-sectional definition. CES-D thresholds for classifying depression at a single time-point were based on prior studies (Radloff, 1977; Vilagut, Forero, Barbaglia, & Alonso, 2016). Established thresholds of 16 and 20 were both evaluated.

Classification using consecutive CES-D scores

A second CES-D-based metric was based on repeated assessment of CES-D scores over time, as depressive episode can last for two years or more (American Psychiatric Association, 2000). Among men meeting our inclusion criteria (e.g. with at least 10 observations over 5 years), we used four consecutive CES-D scores to define depression status, classifying as depressed those with all four scores above the threshold. Established thresholds of 16 and 20 were both evaluated.

Classification using longitudinal CES-D score trajectories

A final CES-D-based metric used group-based trajectory modeling to summarize the full CES-D observed history for each participant with at least 10 CES-D assessments. Group-based trajectory modeling allows for the use of all available longitudinal data (Murphy et al, 2015; Nagin and Odgers, 2010). First, depressive symptoms were considered present or absent based on established CES-D score thresholds of either 16 or 20. Depressive symptoms collected semiannually up to 10 years were then fit with univariate logit models, specifying a hypothesized number of depression trajectory groups. Nonlinear trends were fit using polynomial terms. We determined the best fitting model based on the average posterior probabilities of group membership being greater than 0.7 for all groups, presence of a minimum of 5% of participants per trajectory, and the odds of correct classification being greater than 5.0 (Nagin and Tremblay, 2001).

Statistical analysis

The depressive phenotype was considered the gold standard. Data driven approaches were compared to the gold standard. Sensitivity and specificity were estimated in the overall sample, among HIV-infected MSM, and among HIV-uninfected MSM. All analyses were performed in Stata 13.1 (StataCorp, 2013).

Results

Sample characteristics

Table 1 shows the overall sample characteristics and sample characteristics stratified by baseline HIV serostatus. There were 711 HIV-infected MSM and 838 HIV-uninfected MSM aged 50 years and older with at least 5 years of follow-up from January 15, 1985 to March 31, 2015. Of the total sample (N=1549), there were 429 (27.7%) MSM (235 HIV-infected and 194 HIV-uninfected) classified as having a depressive phenotype, 1120 MSM classified as having a non-depressive phenotype and 770 with an indeterminate phenotype. HIV-infected MSM had higher average CES-D scores and baseline CD4 counts, were younger, were more likely to report cocaine, intravenous drug or antidepressant use in the past, and more likely to have Hepatitis C than HIV-uninfected MSM (all p’s<0.01). HIV-infected MSM had a mean viral load of 17,284 copies/mL (Table 1).

Table 1.

Sample Characteristics from the Multicenter AIDS Cohort Study.

Sample Characteristics Overall (N=1549) HIV-Uninfected (N=838) HIV-Infected (N=711) p-value for difference
Depressive Phenotype, n(%) 429 (27.7) 194 (23.2) 235 (33.1) <0.01
Enrollment Year (Year 2000+), n(%) 1,070 (69.1) 665 (79.4) 405 (57.0) <0.01
Baseline CES-D score, mean (SD) 12.8 (9.6) 11.9 (8.9) 14.0 (10.3) <0.01
Baseline Age, mean (SD) 51.1 (2.5) 51.5 (2.8) 50.7 (2.1) <0.01
Baseline CD4 Count, cells/mm3, mean (SD) 788.8 (382.7) 993.4 (335.1) 547.6 (281.6) <0.01
Baseline Viral Load, cells/mm3, mean (SD) --- --- 17283.9 (61937.0) ---
Ever Cocaine Use, n(%) 286 (18.5) 118 (14.1) 168 (23.6) <0.01
Ever Intravenous Drug Use, n(%) 68 (4.4) 22 (2.6) 46 (6.5) <0.01
Ever Use of Antidepressants, n(%) 665 (42.9) 327 (39.0) 338 (47.5) <0.01
Ever Hepatitis C Status, n(%) 121 (7.8) 39 (4.7) 82 (11.5) <0.01
Open in a new tab

SD=standard deviation

Prevalence of clinically-identified depressive phenotype

Depression prevalence based on depression phenotype was estimated to be 28% (95% Confidence Interval, [CI]: 26%, 30%) overall, 33% (95% CI: 30%, 37%) in HIV-infected and 23% (95% CI: 20%, 26%) in HIV-uninfected MSM. Our estimates fall within the ranges of externally reported depression prevalences among overall MSM (Meyer, Dietrich, & Schwartz, 2008; Sandfort, de Graaf, Bijl, & Schnabel, 2001) and HIV-infected MSM (Atkinson et al., 2008).

Discrimination performance of a single time point CES-D –based metric

Using a single CES-D assessment with a threshold of 16, sensitivity was 76% (95% CI: 70%, 81%) in the overall sample, 75% (95% CI: 67%, 82%) among HIV-infected, and 77% (95% CI: 67%, 85%) among HIV-uninfected. Specificity was 90% (95% CI: 87%, 92%) in the overall sample, 88% (95% CI: 84%, 92%) among HIV-infected, and 92% (95% CI: 88%, 94%) among HIV-uninfected (Table 2).

Table 2.

Sensitivity and Specificity for Each Type of Depressive Phenotype Classification

OVERALL HIV-UNINFECTED HIV-INFECTED
Sens 95% CI Spec 95% CI Sens 95% CI Spec 95% CI Sens 95% CI Spec 95% CI
CES-D SCORE≥16
Single CES-D Score at Age 50 75.7 (69.9, 80.8) 89.9 (87.4, 92.1) 76.7 (67.3, 84.5) 91.5 (88.0, 94.2) 75.0 (67.3, 81.7) 88.3 (84.2, 91.6)
Four Consecutive CES-D Scores≥16 74.3 (69.4, 78.9) 89.8 (87.7, 91.6) 81.0 (73.7, 87.0) 90.3 (87.5, 92.6) 73.1 (66.1, 79.3) 89.1 (85.5, 92.0)
Chronic High† 53.1 (47.6, 58.4) 98.8 (97.9, 99.4) 62.4 (54.3, 70.0) 97.8 (96.3, 98.9) 74.7 (67.9, 80.8) 93.8 (90.9, 96.0)
CES-D SCORE≥20
Single CES-D Score at Age 50 59.6 (53.3, 65.7) 95.0 (93.0, 96.5) 59.2 (49.1, 68.8) 95.6 (92.8, 97.5) 59.9 (51.6, 67.7) 94.3 (91.1, 96.6)
Four Consecutive CES-D Scores≥20 57.1 (51.7, 62.4) 96.7 (95.4, 97.7) 58.0 (49.8, 65.8) 96.4 (94.5, 97.8) 71.9 (63.9, 79.0) 97.1 (94.9, 98.6)
Chronic High† 49.3 (43.9, 54.7) 99.4 (98.6, 99.8) 48.0 (42.0, 54.1) 99.9 (99.4, 100.0) 54.8 (47.4, 62.1) 99.0 (97.4, 99.7)
Open in a new tab

Sens=Sensitivity, Spec=Specificity, CI=Confidence Interval, CES-D=Centers for Epidemiologic Studies – Depression Scale

†

Class with highest probability of depressive symptoms across 10 years from group-based trajectory models. Reference group was all other classes from group-based trajectory models.

When the threshold was increased to a CES-D score of 20, sensitivity was 60% (95% CI: 53%, 66%) in the overall sample, 60% (95% CI: 52%, 68%) among HIV-infected, and 59% (95% CI: 49%, 69%) among HIV-uninfected (Table 2). Specificity was 95% (95% CI: 93%, 97%) in the overall sample, 94% (95% CI: 91%, 97%) among HIV-infected, 96% (95% CI: 93%, 98%) among HIV-uninfected (Table 2).

Discrimination performance of a consecutive assessment CES-D –based metric

Using four consecutive CES-D scores ≥ 16, the sensitivity was 74% (95% CI: 69%, 79%) in the overall sample, 73% (95% CI: 66%, 79%) among HIV-infected, and 81% (95% CI: 74%, 87%) among HIV-uninfected, when compared to our gold standard (Table 2). Specificity was 90% (95% CI: 88%, 92%) in the overall sample, 89% (95% CI: 86%, 92%) among HIV-infected, and 90% (95% CI: 88%, 93%) among HIV-uninfected (Table 2).

When using four consecutive CES-D scores ≥ 20, the sensitivity decreased to 57% (95% CI: 52%, 62%) in the overall sample and 58% (95% CI: 50%, 66%) among HIV-uninfected. The sensitivity slightly decreased to 72% (95% CI: 64%, 79%) among HIV-infected. Specificity increased to 97% (95% CI: 95%, 98%) in the overall sample, 97% (95% CI: 95%, 99%) among HIV-infected, and 96% (95% CI: 95%, 98%) among HIV-uninfected (Table 2).

Discrimination performance of a longitudinal trajectory CES-D –based metric

Final Trajectory Model

In the overall sample with depressive symptoms at a single visit defined as a CES-D≥16, the best fitting trajectory model identified four trajectory groups, as shown in Figure 2. This model met criteria for the best univariate model with optimal numbers of groups and corresponding polynomial functions (Table 3 and Supplemental Table 2). The average posterior probability for each group was above 0.7, and the odds of correct classification were above 5.0 for each group. All Wald tests were statistically significant for trends (P<0.05) (Table 3).

Figure 2.

Figure 2

Open in a new tab

Group-based Trajectories when using CES-D score thresholds of 16 or 20 for the overall sample

Table 3.

Model Diagnostics for Each Trajectory Group and Model

Model Group Overall HIV Positive HIV Negative
π AvePP OCC π AvePP OCC π AvePP OCC
Depressive Symptoms* 1 Rare/Never 0.50 0.92 11.94 0.51 0.95 17.42 0.63 0.97 16.65
2 Mild 0.21 0.81 16.13 --- --- --- --- --- ---
3 Periodically Depressed 0.14 0.83 29.27 0.27 0.93 33.51 0.21 0.91 39.12
4 Chronic High 0.15 0.92 63.34 0.21 0.96 101.33 0.16 0.96 125.95
Depressive Symptoms** 1 Rare/Never 0.63 0.96 14.49 0.54 0.94 14.35 0.68 0.97 16.03
2 Periodically Depressed 0.23 0.89 27.12 0.28 0.89 20.37 0.20 0.90 34.06
3 Chronic High 0.14 0.95 116.63 0.18 0.94 65.12 0.12 0.94 120.28
Open in a new tab

π is the proportion of the sample belonging to a particular group. All values are above 5%. AvePP is the average posterior probability of group membership. All values are above 0.7. The odds of correctly classifying the individual’s group membership are the OCC. All values are above 5.0.

*

Depressive symptoms trajectories are defined by a CES-D cutoff point of 16 or greater.

**

Depressive symptoms trajectories are defined by a CES-D cutoff point of 20 or greater.

From this model, 50% of the sample belonged to a rare/never group, with low probability of depressive symptoms longitudinally. 21% belonged to a mild group, with depressive symptoms occurring about twenty percent of the time. 14% belonged to a periodically depressed group, with depressive symptoms occurring about sixty percent of the time. 15% belonged to a chronic high group, with depressive symptoms occurring about ninety percent of the time. Using a CES-D threshold of 20 resulted in similar groupings with expected shifts in membership percentage towards higher membership in milder depression trajectory groups (Supplemental Table 2, Table 3, and Figure 2).

When stratifying by HIV serostatus, only three depressive symptom trajectory groups were identified: rare/never, periodically depressed and chronic high (Table 3). Group proportions were similar in HIV-infected and HIV-uninfected MSM (Figure 3).

Figure 3.

Figure 3

Open in a new tab

Group-based Trajectories when using CES-D score thresholds of 16 or 20 for HIV-Infected and HIV-Uninfected Men Who Have Sex with Men

Discrimination performance

To allow for comparison to our gold standard, we collapsed the rare/never, mildly depressed and periodically depressed trajectory groups into a single reference category. Comparing the chronic high class membership versus the combined reference group from group-based trajectory modeling of CES-D≥16 to our gold standard classifications, the sensitivity was 53% (95% CI: 48%, 58%) and specificity was 99% (95% CI: 98%, 99%) in the overall sample (Table 2). Among HIV-uninfected MSM, the sensitivity was 62.4% (54.3%, 70.0%) and specificity was 98% (96%, 100%). Among HIV-infected MSM, the sensitivity was 75% (68%, 81%) and the specificity was 94% (95% CI: 91%, 96%).

Comparing the chronic high class membership versus the combined reference group from group-based trajectory modeling of CES-D≥20 to our gold standard classifications, sensitivity decreased to 49% (95% CI: 44%, 55%) for the overall sample, 48% (95% CI: 42%, 54%) among HIV-uninfected MSM, and 55% (95% CI: 47%, 62%) among HIV-infected MSM (Table 3). Specificity increased slightly, but it was greater than 95% in all groups (Table 2).

Discussion

Many observational studies collect data on depressive symptoms through questionnaires, since the gold standard of a clinical interview for diagnosing clinical depression is costly, time-intensive, and burdensome to the participant. Given a lack of clinical diagnosis, prevalence estimates are derived from scores on questionnaires. Yet a single assessment from a questionnaire might not correctly classify long-term depression risk (Kaup, et al., 2016). Understanding how common single time point and sequential questionnaire-based assessments relate to long-term depression risk is critical for interpreting results from research studies that predominantly use such methods to make inferences about depression.

In the current study, we assessed long-term depression risk through intensive data review. Investigators including a clinical psychiatrist examined the available history of clinical indicators of depression and established criteria for agreement to classify participants in terms of their depressive phenotype. While the gold standard for diagnosing depression is a clinical interview, such detailed review of relevant clinical data was considered a surrogate. In fact, depressive phenotype classifications yielded prevalence estimates consistent with those reported in the literature from clinical interviews, providing face validity to our surrogate gold standard. The prevalence of clinical depression for the overall sample of MSM was 28%, as compared to reported prevalences between 26% and 29% in similar samples (Meyer, et al., 2008; Sandfort, et al., 2001). Among HIV-infected MSM, the prevalence of clinical depression was 33%, as compared to external estimates ranging from 27% to 33% (Atkinson, et al., 2008). The prevalence of clinical depression among HIV-uninfected MSM was 23%, which was lower than prevalence estimate of 33% for HIV-uninfected MSM (Atkinson et al., 2008) and prevalence estimates for MSM in general (Meyer, et al., 2008; Sandfort, et al., 2001).

When we compared each of three CES-D-based metrics (baseline single time point, four consecutive CES-D scores, and group-based trajectory methods) to the clinically-identified depressive phenotype, we found that specificity was highest when group-based trajectory methods were employed across the overall, HIV-infected, and HIV uninfected samples. However, across all metrics, specificity was generally high, ranging from 88% to 100%, meaning that even a single timepoint CES-D assessment may be adequate to conduct depression screenings among older MSM, as the high specificity would result in few false positives. In contrast, sensitivity was more modest across the CES-D-based metrics, ranging from 48% to 81%, with the highest sensitivity resulting from the aggregation of four consecutive CES-D scores among HIV-uninfected MSM. Lower sensitivity could result in missed cases of depression, which could dilute effect estimates in research studies and cause prevalence to be underestimated.

High specificity (reducing the number of false positives) and low sensitivity (resulting in classification of fewer depressed people) are important factors when considering use of the CES-D as a screening tool for depression. High specificity of the CES-D metrics will assure fewer false positives. However, the relatively low sensitivity of the metrics we evaluated suggests that fewer phenotypically depressed individuals will be appropriately classified, i.e. more false negatives. This could have implications for both research and screening, potentially attenuating effect estimates and delaying treatment for those who are missed. The cause of the low sensitivity could lie in the use of other information beyond CES-D for classifying depression phenotype in our gold standard, which could not be well-captured by CES-D alone. Alternatively, the low sensitivity may result from choices made in defining the CES-D based metrics. For example, we chose to consider only those in the chronically high depression trajectory class as phenotypically depressed; however, sensitivity would likely be improved if the periodically depressed trajectory class were also considered phenotypically depressed. Thus, CES-D metrics might need to be tailored, depending upon the relative cost of low sensitivity versus low specificity.

There were notable limitations to our study. The clinical review of available study data included the review of CES-D scores that were also used for the CES-D-based metrics. However, CES-D scores used for depressive phenotype were only one source of information and the entire long-term pattern was considered in the context of other clinically relevant factors rather than single or consecutive thresholds. Further, our gold standard classifications were based on the clinical expertise of an experienced psychiatrist and an expertise-driven process that was far removed from a purely data-driven approach typical of most research definitions of depression. The gold standard yielded prevalence estimates of depression among MSM that were very similar to those obtained from clinical interviews from external cohorts (Atkinson, et al., 2008; Fendrich, et al., 2013; Mills et al., 2004; Perdue, et al., 2003; Reisner, et al., 2009). While antidepressant use was considered an indicator of a depressive phenotype, it was not considered sufficient to classify an individual as having a depressive phenotype. Similarly, we did not use it to define depression in any of the data-driven approaches. We did not adjust models for antidepressant use as depression treatment may be both a cause and a consequence of depression status, and introduce bias (Armstrong et al., 2017). However, in studies where long-term data are unavailable, antidepressant use may serve as an important surrogate for past depressive episodes.

Our study benefited from the availability of long-term histories of depression indicators including CES-D measured semiannually over a ten-year period. Many studies have been limited by short duration of follow-up (Beekman et al., 2001; Penninx et al., 1999; Schoevers et al., 2003) or data availability at only two time points (Beekman, Deeg, Smit, & van Tilburg, 1995). As is standard in research studies, we developed depression metrics based only on CES-D scores and accepted thresholds to compare to our gold standard; two different thresholds were evaluated to assess whether there existed scaling differences in CES-D between HIV-infected and HIV–uninfected MSM. HIV-infected may report higher scores even in the absence of depression as a result of somatic symptoms or other HIV-related health issues (Kalichman, Rompa, & Cage, 2000).

Our results suggest that CES-D based metrics can be used to appropriately classify a depressive phenotype, with the highest specificity obtained when repeated assessments are available. Group-based trajectory methods can be used to accurately identify those at lowest risk of depression over time, which may be important in some research contexts. An easily implementable metric based on as few as four repeated CES-D measures may adequately identify a profile of depression in a cohort of MSM for most research purposes. Among MSM aged 50 years and older, even a single CES-D measure may be sufficient if investigators need a highly specific metric, and can tolerate lower sensitivity. Thus, the use of CES-D based metrics will depend on the research context, but as we have shown, good classification performance can be obtained if repeated assessments are available.

Supplementary Material

Supplemental Table 1
Supplemental Table 2

Acknowledgments

Role of Funding Source: This study was supported by National Institutes of Health (1R03MH10396-01). An additional source of support for MACS included the Center for AIDS Research, Johns Hopkins University (112547). NMA was supported by fellowship from the Epidemiology and Biostatistics of Aging Training Grant (5T32AG000247).

Data in this manuscript were collected by the Multicenter AIDS Cohort Study (MACS). MACS (Principal Investigators): Johns Hopkins University Bloomberg School of Public Health (Joseph Margolick), U01-AI35042; Northwestern University (Steven Wolinsky), U01-AI35039; University of California, Los Angeles (Roger Detels), U01-AI35040; University of Pittsburgh (Charles Rinaldo), U01-AI35041; the Center for Analysis and Management of MACS, Johns Hopkins University Bloomberg School of Public Health (Lisa Jacobson), UM1-AI35043. The MACS is funded primarily by the National Institute of Allergy and Infectious Diseases (NIAID), with additional co-funding from the National Cancer Institute (NCI), the National Institute on Drug Abuse (NIDA), and the National Institute of Mental Health (NIMH). Targeted supplemental funding for specific projects was also provided by the National Heart, Lung, and Blood Institute (NHLBI), and the National Institute on Deafness and Communication Disorders (NIDCD). MACS data collection is also supported by UL1-TR001079 (JHU ICTR) from the National Center for Advancing Translational Sciences (NCATS) a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research. The contents of this publication are solely the responsibility of the authors and do not represent the official views of the National Institutes of Health (NIH), Johns Hopkins ICTR, or NCATS. The MACS website is located at http://aidscohortstudy.org/.

We would like to thank the participants of the Multicenter AIDS Cohort Study.

Footnotes

Authors’ contributions: Armstrong wrote the manuscript. Surkan, Treisman, Sacktor, Jacobson, Armstrong, and Abraham made substantial contributions to the conception and design. Irwin, Stall, and Jacobson acquired the data. Armstrong, Surkan, Treisman, and Abraham analyzed the data, and all authors assisted with the interpretation of the data. All authors were involved in revising the manuscript critically for important intellectual content. All authors read and approved the final manuscript.

Conflicts of interest: none

Contributor Information

Nicole M. Armstrong, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA, Telephone: 410-955-0491

Pamela J. Surkan, Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA, Telephone: 410-502-7396

Glenn J. Treisman, Departments of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, USA, Telephone: 410-955-2343

Ned C. Sacktor, Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD, USA, Telephone: 410-550-5624

Michael R. Irwin, Cousins Center for Psychoneuroimmunology, UCLA Semel Institute for Neuroscience and Department of Psychiatry and Biobehavioral Sciences, UCLA David Geffen School of Medicine, Los Angeles, CA, USA, Telephone: 310-825-8281

Linda A. Teplin, Departments of Psychiatry and Behavioral Sciences and Medicine: Infectious Diseases, Feinberg School of Medicine, Chicago, IL, USA, Telephone: 312-503-3500

Ron C. Stall, Department of Behavioral and Community Health, University of Pittsburgh Medical Center, Pittsburgh, PA, USA, Telephone: (412) 624-7933

Lisa P. Jacobson, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA, Telephone: 410-955-4320

Alison G. Abraham, Department of Ophthalmology, Johns Hopkins University School of Medicine, Baltimore, MD, USA; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Telephone: 410-502-9763

References

  1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4. Washington, DC: American Psychiatric Association; 2000. text rev. ed. [Google Scholar]
  2. Armstrong NM, Surkan PJ, Treisman GJ, Sacktor NC, Irwin MR, Teplin LA, … Abraham AG. Association of long-term patterns of depressive symptoms and attention/executive function among older men with and without human immunodeficiency virus. Journal of Neurovirology. 2017;23(4):558–567. doi: 10.1007/s13365-017-0527-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Atkinson JH, Heaton RK, Patterson TL, Wolfson T, Deutsch R, Brown SJ, … Group H. Two-year prospective study of major depressive disorder in HIV-Infected men. Journal of Affective Disorders. 2008;108(3):225–234. doi: 10.1016/j.jad.2007.10.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation; 1996. [Google Scholar]
  5. Beekman A, Deeg D, Geerlings S, Schoevers R, Smit J, van Tilburg W. Emergence and persistence of late life depression: a 3-year follow-up of the Longitudinal Aging Study Amsterdam. Journal of Affective Disorders. 2001;65(2):131–138. doi: 10.1016/s0165-0327(00)00243-3. [DOI] [PubMed] [Google Scholar]
  6. Beekman A, Deeg D, Smit J, van Tilburg W. Predicting the course of depression in the older population: results from a community-based study in The Netherlands. Journal of Affective Disorders. 1995;34(1):41–49. doi: 10.1016/0165-0327(94)00103-g. [DOI] [PubMed] [Google Scholar]
  7. Bhatia M, Munjal S. Prevalence of depression in people living with HIV/AIDS undergoing ART and factors associated with it. Journal of Clinical and Diagnostic Research: JCDR. 2014;8(10):WC01–WC04. doi: 10.7860/JCDR/2014/7725.4927. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bing E, Burnam M, Longshore D, Fleishman J, Sherbourne C, London A, … Shapiro M. Psychiatric disorders and drug use among Human Immunodeficiency Virus–infected adults in the United States. Archives of General Psychiatry. 2001;58(8):721–728. doi: 10.1001/archpsyc.58.8.721. [DOI] [PubMed] [Google Scholar]
  9. Fendrich M, Avci O, Johnson T, Mackesy-Amiti M. Depression, substance use and HIV risk in a probability sample of men who have sex with men. Addictive Behaviors. 2013;38(3):1715–1718. doi: 10.1016/j.addbeh.2012.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Gotlib I, Joormann J. Cognition and depression: current status and future directions. Annual Review of Clinical Psychology. 2010;6:285–312. doi: 10.1146/annurev.clinpsy.121208.131305. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Horberg MA, Silverberg MJ, Hurley LB, Towner WJ, Klein DB, Bersoff-Matcha S, … Dodge WT. Effects of depression and selective serotonin reuptake inhibitor use on adherence to highly active antiretroviral therapy and on clinical outcomes in HIV-Infected patients. Journal of Acquired Immune Deficiency Syndromes. 2008;47(3):384–390. doi: 10.1097/QAI.0b013e318160d53e. [DOI] [PubMed] [Google Scholar]
  12. Ironson G, Balbin E, Stuetzle R, Fletcher MA, O’Cleirigh C, Laurenceau JP, … Solomon G. Dispositional optimism and the mechanisms by which it predicts slower disease progression in HIV: proactive behavior, avoidant coping, and depression. International Journal of Behavioral Medicine. 2005;12(2):86–97. doi: 10.1207/s15327558ijbm1202_6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Kalichman S, Rompa D, Cage M. Distinguishing between overlapping somatic symptoms of depression and HIV disease in people living with HIV-AIDS. The Journal of Nervous and Mental Disease. 2000;188(10):662–670. doi: 10.1097/00005053-200010000-00004. [DOI] [PubMed] [Google Scholar]
  14. Kaslow R, Ostrow D, Detels R, Phair J, Polk BCR., Jr The Multicenter AIDS Cohort Study: rationale, organization, and selected characteristics of the participants. American Journal of Epidemiology. 1987;126(2):310–318. doi: 10.1093/aje/126.2.310. [DOI] [PubMed] [Google Scholar]
  15. Kaup A, Byers A, Falvey C, Simonsick E, Satterfield S, Ayonayon H, … Yaffe K. Trajectories of depressive symptoms in older adults and risk of dementia. JAMA Psychiatry. 2016;73(5):525–531. doi: 10.1001/jamapsychiatry.2016.0004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lyness JM, Noel TK, Cox C, King DA, Conwell Y, Caine ED. Screening for depression in elderly primary care patients: a comparison of the Center for Epidemiologic Studies-Depression Scale and the Geriatric Depression Scale. Archives of Internal Medicine. 1997;157(4):449–454. [PubMed] [Google Scholar]
  18. Marcellin F, Préau M, Ravaux I, Dellamonica P, Spire B, Carrieri MP. Self-reported fatigue and depressive symptoms as main indicators of the quality of life (QOL) of patients living with HIV and Hepatitis C: implications for clinical management and future research. HIV Clinical Trials. 2007;8(5):320–327. doi: 10.1310/hct0805-320. [DOI] [PubMed] [Google Scholar]
  19. McArthur J, Brew B. HIV-associated neurocognitive disorders: is there a hidden epidemic? AIDS. 2010;24(9):1367–1370. doi: 10.1097/QAD.0b013e3283391d56. [DOI] [PubMed] [Google Scholar]
  20. Meyer I, Dietrich J, Schwartz S. Lifetime prevalence of mental disorders and suicide attempts in diverse lesbian, gay, and bisexual populations. American Journal of Public Health. 2008;98(6):1004–1006. doi: 10.2105/AJPH.2006.096826. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Mills T, Paul J, Stall R, Pollack L, Canchola J, Chang Y, … Catania J. Distress and depression in men who have sex with men: the Urban Men’s Health Study. American Journal of Psychiatry. 2004;161(2):278–285. doi: 10.1176/appi.ajp.161.2.278. [DOI] [PubMed] [Google Scholar]
  22. Murphy RA, Hagaman AK, Reinders I, Steeves J, Newman A, Rubin S … For the Health ABC Study. Depressive trajectories and risk of disability and mortality in older adults: longitudinal findings from the Health, Aging, and Body Composition Study. Journals of Gerontology, Series A, Biological Sciences and Medical Sciences. 2016;71(02):228–235. doi: 10.1093/gerona/glv139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Nagin D, Tremblay R. Analyzing developmentary trajectories of distinct but related behaviors: a group-based method. Psychological Methods. 2001;6:18–34. doi: 10.1037/1082-989x.6.1.18. [DOI] [PubMed] [Google Scholar]
  24. Nagin DS, Odgers CL. Group-based trajectory modeling in clinical research. Annual Review of Clinical Psychology. 2010;6(1):109–138. doi: 10.1146/annurev.clinpsy.121208.131413. [DOI] [PubMed] [Google Scholar]
  25. O’Connor E, Whitlock E, Gaynes B, Beil T. Screening for Depression in Adults and Older Adults in Primary Care: An Updated Systematic Review. Rockville, MD: Agency for Healthcare Research and Quality; 2009. [PubMed] [Google Scholar]
  26. Panel on Antiretroviral Guidelines for Adults and Adolescents. Guidelines for the Use of Antiretroviral Agents in Adults and Adolescents Living with HIV. Department of Health and Human Services; Available at http://www.aidsinfo.nih.gov/ContentFiles/AdultandAdolescentGL.pdf. [Google Scholar]
  27. Penninx B, Geerlings S, Deeg D, van Eijk J, van Tilburg W, Beekman A. Minor and major depression and the risk of death in older persons. Archives of General Psychiatry. 1999;56(10):889–895. doi: 10.1001/archpsyc.56.10.889. [DOI] [PubMed] [Google Scholar]
  28. Perdue T, Hagan H, Thiede H, Valleroy L. Depression and HIV risk behavior among Seattle-area injection drug users and young men who have sex with men. AIDS Education and Prevention. 2003;15(1):81–92. doi: 10.1521/aeap.15.1.81.23842. [DOI] [PubMed] [Google Scholar]
  29. Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement. 1977;1(3):385–401. [Google Scholar]
  30. Reisner SL, Mimiaga MJ, Skeer M, Bright D, Cranston K, Isenberg D, … Mayer KH. Clinically significant depressive symptoms as a risk factor for HIV infection among black MSM in Massachusetts. AIDS Behavior. 2009;13(4):798–810. doi: 10.1007/s10461-009-9571-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Rush AJ, Trivedi MH, Ibrahim HM, Carmody TJ, Arnow B, Klein DK, … Keller MB. The 16-item quick inventory of depressive symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biological Psychiatry. 2003;54(5):573–583. doi: 10.1016/s0006-3223(02)01866-8. [DOI] [PubMed] [Google Scholar]
  32. Sandfort T, de Graaf R, Bijl R, Schnabel P. Same-sex sexual behavior and psychiatric disorders: Findings from the Netherlands Mental Health Survey and Incidence Study (NEMESIS) Archives of General Psychiatry. 2001;58(1):85–91. doi: 10.1001/archpsyc.58.1.85. [DOI] [PubMed] [Google Scholar]
  33. Schoevers R, Beekman A, Deeg D, Hooijer C, Jonker C, van Tilburg W. The natural history of late-life depression: results from the Amsterdam Study of the Elderly (AMSTEL) Journal of Affective Disorders. 2003;76(1):5–14. doi: 10.1016/s0165-0327(02)00060-5. [DOI] [PubMed] [Google Scholar]
  34. Silvestre A, Hylton J, Johnson L, Houston C, Witt M, Jacobson L, Ostrow D. Recruiting minority men who have sex with men for HIV research: results from a 4-city campaign. American Journal of Public Health. 2006;96(6):1020–1027. doi: 10.2105/AJPH.2005.072801. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. StataCorp. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP; 2013. [Google Scholar]
  36. Stein M, Solomon D, Herman D, Anderson B, Miller I. Depression severity and drug injection HIV risk behaviors. The American Journal of Psychiatry. 2003;160(9):1659–1662. doi: 10.1176/appi.ajp.160.9.1659. [DOI] [PubMed] [Google Scholar]
  37. Vilagut G, Forero C, Barbaglia G, Alonso J. Screening for depression in the general population with the Center for Epidemiologic Studies Depression (CES-D): a systematic review with meta-analysis. PLoS One. 2016;11(5):e0155431. doi: 10.1371/journal.pone.0155431. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Zimmerman M, Chelminski I, McGlinchey JB, Posternak MA. A clinically useful depression outcome scale. Comprehensive Psychiatry. 2008;49:131–140. doi: 10.1016/j.comppsych.2007.10.006. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Table 1
NIHMS945082-supplement-Supplemental_Table_1.docx (15.9KB, docx)
Supplemental Table 2
NIHMS945082-supplement-Supplemental_Table_2.docx (22.8KB, docx)

RESOURCES