Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Dec 15.
Published in final edited form as: J Acquir Immune Defic Syndr. 2025 Dec 15;100(5):445–453. doi: 10.1097/QAI.0000000000003752

Depression improves the predictive accuracy of the VACS Index 2.0 for all-cause mortality among sexual minority men living with HIV in the Multicenter AIDS Cohort Study

Yuhang Qian a,b, Warren S Comulada c,d, Marco A Hidalgo e, Sung-Jae Lee a,c, M Reuel Friedman f, Frank J Palella g, Michael W Plankey h, Gypsyamber D’Souza i, Roger Detels a,e, Matthew J Mimiaga a,b,c
PMCID: PMC12433150  NIHMSID: NIHMS2109749  PMID: 40857055

Abstract

Background:

The Veterans Aging Cohort Study (VACS) Index 2.0 accurately predicts mortality using age and clinical biomarkers, but adding behavioral and psychosocial factors that are common among sexual minority men (SMM) may improve its predictive accuracy. We examined whether adding these factors would improve mortality prediction among SMM living with HIV.

Methods:

We included 1,438 SMM in the Multicenter AIDS Cohort Study (MACS) who initiated highly active antiretroviral therapy (HAART) for at least one year between January 1996 and September 2022. We divided the sample into development (70%) and validation (30%) sets. We used Cox proportional hazards models to develop new indices in the development set by adding binary behavioral and psychosocial factors (depression, cigarette smoking, heavy alcohol use, polydrug use) or the total number of these factors in the VACS Index 2.0 and estimated mortality using Weibull survival models. We compared accuracy using C-statistics and calibration curves in the validation set and within subgroups (age, race, CD4 count, and viral suppression).

Results:

Among the 1,438 SMM, 83 (5.8%) died within 5 years of follow-up. Depression significantly predicted 5-year mortality after adjusting for the VACS Index 2.0 and resulted in a 70% increased risk of death (aHR=1.70, 95% CI=1.10–2.63) compared to men without depression. The addition of depression improved C-statistics from 0.818 to 0.851 in the development set. Results were robust in all subgroups.

Conclusions:

Including depression improved the VACS Index 2.0 in predicting mortality. Screening and treating depression could improve health and reduce mortality among SMM living with HIV.

Keywords: Behavioral and psychosocial factors, VACS Index, Mortality, HIV, Sexual minority men

INTRODUCTION

In the highly active antiretroviral therapy (HAART) era, people living with HIV (PLWH) typically achieve viral suppression and have longer life expectancies. However, as they age, they face a higher risk of non-AIDS-defining comorbidities, including cardiovascular diseases, chronic kidney or liver diseases, and non-AIDS-defining cancer, compared with demographically similar individuals without HIV, and disparities in life expectancy persist.1–5 Aging PLWH are also at higher risk for depression, which increases with comorbidities.6

Traditional HIV biomarkers, including CD4 cell count and HIV RNA, have limited capabilities in predicting mortality.7 To characterize the shift from AIDS to non-AIDS morbidity among PLWH, the Veterans Aging Cohort Study (VACS) Index was developed to provide a summary of the overall disease burden using the Veterans Health Administration (VHA) electronic health record (EHR) data.7–10 It incorporates factors that reflect general health [age, albumin, body mass index (BMI), hemoglobin, Fibrosis-4 Index for Liver Fibrosis (FIB-4), estimated glomerular filtration rate (eGFR), white blood cell count (WBC), hepatitis C virus (HCV)] and HIV-specific factors (CD4 cell count and HIV RNA),8 and its accuracy and utility in predicting mortality have been validated in various samples representative of PLWH in North America and Europe.9,11,12 The VACS Index can be calculated online,13 and it has been increasingly used in a variety of research (e.g., risk adjustment in observational studies),14,15 public health (e.g., surveillance of the burden of diseases among PLWH),9 and clinical settings (e.g., used as a tool for patient management within EHR systems).9 Previous research has indicated that the VACS Index has high accuracy in predicting various outcomes including cardiovascular disease,16 neurocognitive dysfunction,17,18 inflammation,19 frailty, and hospitalization,20–22 and it has high predictive accuracy among age, sex, and race subgroups.9,10

Though widely used, the biomarkers in the VACS Index reflect physiologic injury from comorbidities but the clinical harm from behavioral and psychosocial factors may not operate through the biomarkers that are already included in the index. PLWH with heavy alcohol use, drug abuse, or cigarette smoking have higher VACS Index scores, but behavioral and psychosocial factors may account for additional unexplained mortality.23,24 One previous study revealed that the addition of depression and transactional sex significantly improved the performance of the VACS Index in predicting mortality among women living with HIV.25 However, whether the addition of behavioral and psychosocial factors in the VACS Index improves the prediction of mortality among sexual minority men (SMM) living with HIV remains unknown. Compared with heterosexual men, SMM living with HIV have a higher prevalence of behavioral and psychosocial factors, and several large cohorts of SMM in the US have revealed that the prevalence of these factors was high: 26–47% with depression,26–28 11–25% with heavy alcohol use,26,27,29 24–36% with current smoking,30–34 and 8–18% with polydrug use.26–29

We aimed to: 1) determine whether adding behavioral and psychosocial factors (depression, heavy alcohol use, cigarette smoking, polydrug use) or the number of these factors would improve the VACS Index (2.0) in predicting mortality among SMM living with HIV in the Multicenter AIDS Cohort Study (MACS); 2) examine whether the new model performs better among particular subgroups defined by age, race, CD4 count, and viral suppression.

METHODS

Study Population

The MACS, now part of the MACS/WIHS Combined Cohort Study (MWCCS), has been described in detail.35 It is an ongoing representative cohort study of SMM living with or without HIV in four US cities: Baltimore, MD; Chicago, IL; Los Angeles, CA; and Pittsburgh, PA. Participants returned every 6 months for a follow-up visit, which included a health interview, psychosocial and behavioral assessments, physical examination, and collection of biospecimens for laboratory testing.35 More information about the MACS can be found at: https://statepi.jhsph.edu/mwccs/.

For this analysis, eligible participants were SMM living with HIV who had initiated HAART for at least one year and completed at least two assessments between January 1996 and September 2022. We included participants with one year or more of HAART use because deaths within the first year of HAART initiation were mostly due to PLWH presenting to care with advanced disease.9 We further excluded participants with missing year of death and those who did not have at least one visit with non-missing components of the VACS Index 2.0 (age, CD4 cell count, HIV RNA, hemoglobin, FIB-4, eGFR, hepatitis C virus status, albumin, white blood cell count, body mass index) and behavioral and psychosocial factors (depression, cigarette smoking, heavy alcohol use, polydrug use) between 1996 and 2022. One participant with missing month and day of death had his last visit in 2021 and was reported deceased in 2022, and we assumed the death date as January 1, 2022. Our final analytic sample was 1,438. The first available visit after the one-year anniversary of HAART initiation without missing VACS Index 2.0 components and behavioral and psychosocial factors was the anchoring visit at which the VACS Index 2.0 was calculated and follow-up began.

Veterans Aging Cohort Study (VACS) Index 2.0

VACS Index 2.0 was calculated using age, traditional HIV indicators (CD4 cell count, HIV RNA), organ system injury indicators (hemoglobin, FIB-4, eGFR, albumin, white blood cell count), BMI, and hepatitis C virus co-infection status. We calculated validated composite biomarkers of liver and renal injury: FIB-4 was calculated using aspartate aminotransferase (AST), alanine aminotransferase (ALT), platelets, and age36; eGFR was calculated via the CKD-EPI equation using serum creatinine, sex, race, and age.37 Hepatitis C virus status was defined as positive if the participant had an acute (HCV RNA positive <1 year) or chronic infection (HCV RNA positive ≥1 year) or had viral clearance at the anchoring visit. The weights of the predictors in the VACS Index 2.0 were the regression coefficients from the Cox models developed in the Veterans Aging Cohort Study. The development and internal validity of the VACS Index 2.0 have been described in detail.9

Behavioral and Psychosocial Factors

Four behavioral and psychosocial factors were analyzed as binary indicator variables: 1) Clinically significant depressive symptoms were assessed using the Center for Epidemiological Studies Depression Scale (CES-D), and individuals with a score of 16 or greater over the past week were indicative of having depression38; 2) Cigarette smoking was defined as smoking any cigarettes at the time of study visits; 3) Heavy alcohol use was defined as having more than 14 drinks per week since their last study visit (or the previous 6 months)39; and 4) Polydrug use was defined as using ≥ 3 drugs [marijuana/hash, poppers, crack/cocaine, stimulants (including crystal methamphetamine, speed, ice, and MDMA), heroin/speedball, PCP] since last study visit (or the previous 6 months).

Number of Additive Behavioral and Psychosocial Factors

We calculated a count score (0–4) based on the number of behavioral and psychosocial factors each participant endorsed, collapsing 3 and 4 into a “3+” category due to low counts. Dummy variables were created, with 0 as the reference group.

Mortality Data

Time to all-cause mortality was our primary outcome, and all-cause mortality (binary) was ascertained by the researchers in the MACS via periodic death registry searches on all participants, including those lost to follow-up.40 Date of death was obtained either electronically from the National Death Index (NDI) or death certificates.40 The risk of 5-year all-cause mortality (range 0–100%) was produced from the VACS Index scores.

Statistical Analysis

Before building and cross-validating new models, we used Cox proportional hazards models to assess associations between primary predictors [the VACS Index 2.0 (per 5-point increment) and behavioral and psychosocial factors] and 5-year all-cause mortality in the overall sample of 1,438 participants. We assessed the relationships between behavioral and psychosocial factors and 5-year mortality before and after adjusting for the continuous VACS Index 2.0. Person-years were calculated from the anchoring visit to death, the last available MACS visit, 5 years after the anchoring visit, or September 2022. We repeated the analyses using 10-year all-cause mortality as the outcome.

Cross-validation assesses models’ ability to generalize to new data by evaluating performance, reducing overfitting, and ensuring reliable results.41 We split the 1,438 participants into development (n=1,014) and validation (n=424) sets representing 70% and 30% of the total sample separately via simple random sampling stratified by categorical age (≤35, 35–45, 45–55, ≤55) and race/ethnicity (non-Hispanic white, non-Hispanic Black, Hispanic/Latino, other/unknown) variables to maintain similar sociodemographic distributions.

Development and Validation of the New VACS Index

We used Cox models to predict 5-year mortality and develop the new index. We kept the ten components of the VACS Index 2.0 because they were validated as predictors of mortality in over 10,000 PLWH in North America.9,12

Given the demographic differences between our sample and the veterans’ sample used in developing the VACS Index 2.0, we re-evaluated the associations between predictors and mortality using Cox regression coefficients from our development set as weights. Additional candidate predictors included four behavioral and psychosocial factors and the number of these factors, and we developed a series of prognostic models to predict 5-year all-cause mortality with predictors as follows: 1) VACS Index 2.0 as published; 2) VACS Index 2.0 after reweighting in our sample; 3) VACS Index 2.0 components and depression; 4) VACS Index 2.0 components and cigarette smoking; 5) VACS Index 2.0 components and heavy alcohol use; 6) VACS Index 2.0 components and polydrug use; 7) VACS Index 2.0 components, depression, cigarette smoking, heavy alcohol use, and polydrug use; 8) VACS Index 2.0 components and the number of behavioral and psychosocial factors. We truncated continuous predictors at the 1st and 99th percentiles and centered them at the median. We used martingale residuals to determine the functional form of the predictors and applied quadratic, cubic, and logarithmic transformations when linearity assumptions were violated. We conducted the supremum test for the proportionality assumption.

For candidate models listed above (models 3–8, which range from the model incorporating the VACS Index 2.0 components and depression to the model incorporating the VACS Index 2.0 components and the number of behavioral and psychosocial factors), we multiplied the weights and the values of the predictors for each participant, summed the products to create scores, and rescaled the scores within 0–100. We used Akaike’s information criterion (AIC, lower is better) to assess model fit. The predictive accuracy of the models included discrimination (the extent to which the model predicts a higher probability of death among those who died compared to those who did not die) and calibration (the extent to which the model correctly estimates the mortality risk). We assessed the model discrimination using Harrell’s C-statistics [higher is better (0.7–0.8 is good; >0.8 is excellent), range 0.5–1.0] and the model calibration by comparing the predicted mortality to the observed mortality. Observed mortality was calculated using the Kaplan-Meier (KM) method, and predicted mortality was calculated using Weibull survival models. We used Weibull models to calculate predicted mortality because Cox models estimate hazard ratios, whereas Weibull models allow for the direct calculation of mortality probabilities over time. We compared the performance of the candidate models (models 3–8) to the VACS Index 2.0 as published and the VACS Index 2.0 after reweighting in our sample in the development and validation sets to examine whether the addition of behavioral and psychosocial factors or the number of these factors improved the predictive accuracy of the original index. We summarized the risk scores derived from the final model (model 3) and named it the VACS-Depression Index.

Subgroup Analyses

We evaluated the performance of the VACS Index 2.0, the VACS Index 2.0 after reweighting, and the VACS-Depression Index in subgroups [age<45 years vs. age≥45 years; non-Hispanic white vs. racial/ethnic minority (non-Hispanic Black, Hispanic/Latino, other/unknown); CD4 count of <500 cells/μl vs. CD4 count of ≥500 cells/μl; virally suppressed with an HIV RNA<200 copies/ml vs. non-virally suppressed with an HIV RNA≥200 copies/ml]. Calibration curves and Harrell’s C-statistics were used to compare the predictive accuracy of the three parameterizations of the Index. Analyses were conducted using SAS 9.4 (SAS Institute Inc., Cary, North Carolina).

RESULTS

Participant Socio-demographics

We excluded about 15% of otherwise eligible participants who did not have any visits with non-missing VACS Index 2.0 components and behavioral and psychosocial factors, and this proportion was similar to previous VACS studies9,10; 75% of the excluded participants lacked FIB-4, eGFR, albumin, or BMI data between 1996 and 2022. Demographics and laboratory values were similar between the development (n=1,014) and validation (n=424) sets at baseline (Table 1). Nearly half of all participants were racial/ethnic minorities (28.7% non-Hispanic Black, 15.8% Hispanic/Latino, 3.3% other/unknown), and 50.8% initiated HAART between 1996 and 2000. 28% of the participants were not virally suppressed (HIV RNA≥200 copies/ml) at baseline; 10.5% were anemic (hemoglobin level<13.0 g/dL); 2.5% had advanced liver fibrosis (FIB-4>3.25); 5.5% had renal disease (eGFR<60ml/min); and 12.4% had HCV co-infection. Compared to the validation set, the development set had lower proportions of depression and heavy alcohol use but a higher proportion of cigarette smoking. Among the 1,438 participants, 83 died within 5 years and 144 within 10 years (see Table 1, Supplemental Digital Content). The 5-year and 10-year survival probabilities were 94% and 90%, respectively, in both the development and validation sets.

Table 1.

Characteristics of participants at baseline between 1996 and 2022, after a minimum of 1 year of highly active antiretroviral therapy, in the development and validation samples in the Multicenter AIDS Cohort Study (n=1,438).

Overall
(n=1438)		 Development
(n=1014)		 Validation
(n=424)
HAART initiation, N (%)
 1996–2000 730 (50.8) 511 (50.4) 219 (51.7)
 2001–2005 308 (21.4) 221 (21.8) 87 (20.5)
 2006–2010 130 (9.0) 92 (9.1) 38 (9.0)
 2011–2017 270 (18.8) 190 (18.7) 80 (18.9)
Years on HAART
 Median (IQR) 2.3(1.4–5.6) 2.4(1.4–5.5) 2.2(1.4–5.7)
Race/ethnicity, N (%)
 Non-Hispanic white 751 (52.2) 527 (52.0) 224 (52.8)
 Non-Hispanic Black 413 (28.7) 292 (28.8) 121 (28.5)
 Hispanic/Latino 227 (15.8) 161 (15.9) 66 (15.6)
 Other/unknown 47 (3.3) 34 (3.4) 13 (3.1)
VACS Index 2.0 risk score
 Median (IQR) 36 (27–46) 36 (28–46) 36 (26–46)
Components of the VACS Index 2.0 risk score
Age (years)
 Median (IQR) 44 (38–51) 44 (38–51) 44 (38–50)
CD4 cell count (cells/μl)
 Median (IQR) 529 (355–721) 529 (355–718) 528 (356–722)
HIV RNA≥200 copies/ml, N (%) 403 (28.0) 292 (28.8) 111 (26.2)
Hemoglobin (g/dl)
 Median (IQR) 14.7 (13.8–15.6) 14.7 (13.8–15.6) 14.6 (13.7–15.5)
FIB-4
 <1.45 1080 (75.1) 768 (75.7) 312 (73.6)
 1.45–3.25 322 (22.4) 223 (22.0) 99 (23.3)
 >3.25 36 (2.5) 23 (2.3) 13 (3.1)
eGFR (ml/min)
 Median (IQR) 95.2 (81.6–107.8) 95.5 (82.0–108.1) 94.2 (80.4–107.2)
Hepatitis C infection, N (%) 178 (12.4) 129 (12.7) 49 (11.6)
Albumin (g/dl)
 Median (IQR) 4.4 (4.2–4.6) 4.4 (4.2–4.7) 4.4 (4.2–4.6)
White blood cell count (k/ml)
 Median (IQR) 5.4 (4.4–6.4) 5.4 (4.4–6.5) 5.3 (4.3–6.4)
Body mass index (kg/m2)
 Median (IQR) 24.9 (22.8–27.6) 24.9 (22.9–27.3) 24.9 (22.7–28.7)
Addition of behavioral and psychosocial factors, N (%)
Number of behavioral and psychosocial factors
 0 632 (43.9) 440 (43.4) 192 (45.3)
 1 521 (36.2) 369 (36.4) 152 (35.8)
 2 235 (16.3) 173 (17.1) 62 (14.6)
 3+ 50 (3.5) 32 (3.2) 18 (4.2)
  3 46 (3.2) 29 (2.9) 17 (4.0)
  4 4 (0.3) 3 (0.3) 1 (0.2)
Depression * 433 (30.1) 289 (28.5) 144 (34.0)
Cigarette smoking * 512 (35.6) 387 (38.2) 125 (29.5)
Heavy alcohol use * 79 (5.5) 47 (4.6) 32 (7.5)
Polydrug use 121 (8.4) 91 (9.0) 30 (7.1)
Open in a new tab

Note.

+

includes individuals with 3 or 4 behavioral and psychosocial factors.

*

P-values for Wilcoxon rank-sum or Chi-squared tests < 0.05.

Associations Between Candidate Predictors and Mortality

Table 2 showed associations between the VACS Index 2.0, behavioral and psychosocial factors, and 5-year mortality risk. The VACS Index 2.0 [hazard ratio (HR) for 5-point increment=1.50, 95% CI=1.41–1.59], depression (HR=2.31, 95% CI=1.50–3.55), cigarette smoking (HR=1.57, 95% CI=1.02–2.42), and having two behavioral and psychosocial factors (HR=2.79, 95% CI=1.60–4.86) significantly predicted 5-year mortality risk. Depression [adjusted hazard ratio (aHR)=1.70, 95% CI=1.10–2.63] remained significant after adjusting for the continuous VACS Index 2.0. Associations between the predictors and 10-year mortality were similar to the estimates observed for 5-year mortality.

Table 2.

Univariate and VACS-adjusted hazard ratios for predictors of 5-year and 10-year mortality in the overall sample (n=1,438).

5-year mortality 10-year mortality
Univariate models
HR (95% CI)		 VACS-adjusted models
HR (95% CI)		 Univariate models
HR (95% CI)		 VACS-adjusted models
HR (95% CI)
VACS Index 2.0, per 5 points increment 1.50 (1.41, 1.59) --- 1.46 (1.39, 1.54) ---
Depression 2.31 (1.50, 3.55) 1.70 (1.10, 2.63) 2.13 (1.53, 2.95) 1.63 (1.17, 2.27)
Cigarette smoking 1.57 (1.02, 2.42) 1.03 (0.66, 1.60) 1.62 (1.17, 2.25) 1.11 (0.79, 1.56)
Heavy alcohol use 1.52 (0.70, 3.30) 1.63 (0.75, 3.55) 1.15 (0.59, 2.27) 1.15 (0.59, 2.27)
Polydrug use 1.19 (0.57, 2.46) 1.21 (0.58, 2.50) 1.42 (0.84, 2.38) 1.31 (0.78, 2.20)
Number of behavioral and psychosocial factors
 0 --- --- --- ---
 1 1.38 (0.81, 2.37) 1.08 (0.63, 1.86) 1.76 (1.18, 2.62) 1.36 (0.91, 2.04)
 2 2.79 (1.60, 4.86) 1.50 (0.85, 2.64) 2.75 (1.77, 4.28) 1.55 (0.98, 2.44)
 3+ 2.58 (0.99, 6.75) 2.27 (0.87, 5.94) 2.65 (1.24, 5.64) 2.28 (1.07, 4.86)
Open in a new tab

Note. HR = hazard ratio. In univariate models, we included each of the variables as the only predictor; in VACS-adjusted models, we included each of the variables (except the VACS Index 2.0) as the predictor and adjusted for continuous VACS Index 2.0. CI = confidence interval.

Model Development in the Development Set

Compared to the VACS Index 2.0 as published (C-statistic=0.818), the VACS Index 2.0 after reweighting the components (C-statistic=0.834), the VACS Index 2.0 components plus depression (C-statistic=0.851), the VACS Index 2.0 components plus smoking (C-statistic=0.834), the VACS Index 2.0 components plus alcohol (C-statistic=0.839), the VACS Index 2.0 components plus polydrug (C-statistic=0.837), the VACS Index 2.0 components plus all the four behavioral and psychosocial factors (C-statistic=0.852), and the VACS Index 2.0 components plus the number of behavioral and psychosocial factors (C-statistic=0.849) predicted 5-year mortality better in the development set (Table 3). The final VACS-Depression model (see Table 2, Supplemental Digital Content) had a higher C-statistic (C-statistic=0.851), and the AIC dropped from 696.076 to 673.511 compared to the VACS Index 2.0 as published. Plots of mortality versus the VACS Index 2.0 as published, the VACS Index 2.0 after reweighting, and the VACS-Depression Index revealed that observed mortality was generally congruent with predicted mortality for all the three parameterizations of the VACS Index (Figure 1).

Table 3.

Model fit (AIC) and discrimination (C-statistic) from models for 5-year all-cause mortality using Cox regression in the development sample (n=1,014).

Model C-statistic (95% CI) AIC
VACS Index 2.0 as published 0.818 (0.770, 0.866) 696.076
VACS Index 2.0 after reweighting 0.834 (0.787, 0.882) 678.128
VACS Index 2.0 components, depression (Final model) 0.851 (0.809, 0.894) 673.511
VACS Index 2.0 components, smoking 0.834 (0.786, 0.881) 678.320
VACS Index 2.0 components, alcohol 0.839 (0.792, 0.885) 675.062
VACS Index 2.0 components, polydrug 0.837 (0.791, 0.884) 677.366
VACS Index 2.0 components, depression, smoking, alcohol, polydrug 0.852 (0.808, 0.897) 669.337
VACS Index 2.0 components, number of behavioral and psychosocial factors 0.849 (0.806, 0.893) 672.146
Open in a new tab

Note. AIC = Akaike information criterion. The smaller the AIC, the better the predictive performance of the model. C-statistic = concordance statistic. The higher the C-statistic, the better the model can discriminate between participants who died and participants who did not die. CI = confidence interval.

Figure 1.

Figure 1.

Open in a new tab

Observed (open circles) and predicted (solid line) 5-year all-cause mortality as a function of VACS Index 2.0 risk scores as published, VACS Index 2.0 risk scores after reweighting, and VACS-Depression Index risk scores in the development (n=1,014) and validation (n=424) samples.

Note. The closer the predicted and observed mortality overlap, the better the model calibration.

Model Validation in the Validation Set

Compared to the VACS Index 2.0 as published (C-statistic=0.826), model discrimination for predicting 5-year mortality in the validation set was better for the VACS Index 2.0 after reweighting (C-statistic=0.831), the VACS-Depression Index (C-statistic=0.830), the VACS Index 2.0 components plus smoking (C-statistic=0.831), the VACS Index 2.0 components plus polydrug (C-statistic=0.831), and the VACS Index 2.0 components plus the number of behavioral and psychosocial factors (C-statistic=0.827) in Table 4. Figure 1 revealed that the VACS Index 2.0 as published underestimated mortality when the score was between 60 and 70 but the VACS Index 2.0 after reweighting and the VACS-Depression Index overestimated the risk of mortality when the score was between 80 and 100 in the validation set.

Table 4.

Discrimination (C-statistic) from models for 5-year all-cause mortality using Cox regression in the validation sample (n=424).

Model C-statistic (95% CI)
VACS Index 2.0 as published 0.826 (0.738, 0.914)
VACS Index 2.0 after reweighting 0.831 (0.756, 0.906)
VACS Index 2.0 components, depression (Final model) 0.830 (0.751, 0.909)
VACS Index 2.0 components, smoking 0.831 (0.755, 0.906)
VACS Index 2.0 components, alcohol 0.824 (0.749, 0.899)
VACS Index 2.0 components, polydrug 0.831 (0.755, 0.907)
VACS Index 2.0 components, depression, smoking, alcohol, polydrug 0.819 (0.735, 0.903)
VACS Index 2.0 components, number of behavioral and psychosocial factors 0.827 (0.753, 0.902)
Open in a new tab

Note. C-statistic = concordance statistic. The higher the C-statistic, the better the model can discriminate between participants who died and participants who did not die. CI = confidence interval.

We also validated the VACS-Depression Index in the overall sample when the outcome was 10-year mortality. Compared to the VACS Index as published, the C-statistic for the VACS-Depression model increased from 0.798 to 0.804, and the calibration curves of these two parameterizations were similar.

Performance Across Subgroups

Across subgroups, 5-year mortality ranged from 2.9 to 11.7%, lowest among SMM with CD4 count ≥500 cells/μl and highest among those with HIV RNA≥200 copies/ml. Model discrimination for predicting 5-year mortality was better for the VACS-Depression Index than the VACS Index 2.0 as published and the VACS Index 2.0 after reweighting among all the subgroups (see Table 3, Supplemental Digital Content). Observed mortality was generally congruent with predicted mortality among subgroups (see Figure 1, Supplemental Digital Content), and the VACS-Depression Index had similar model calibration with the VACS Index 2.0 as published and the VACS Index 2.0 after re-weighting (graphs not presented).

DISCUSSION

In this study, the VACS Index 2.0 performed similarly to previous studies,9,12 indicated by the similar C-statistics (range approximately 0.8 or higher) and its strong association with mortality. The newly developed VACS-Depression Index improved mortality discrimination in the development sample and subgroups. In the validation sample, the C-statistic showed little improvement, possibly due to the small validation set and the limited sensitivity of the C-statistic to detect improvement, given that the VACS Index 2.0 already had a C-statistic greater than 0.8.42,43 However, a 70% increased risk of death among participants with depression, after adjusting for the VACS Index 2.0, indicated that depression affects mortality via distinct pathways compared to the predictors in the index.

Depression increases mortality among PLWH through several mechanisms. A meta-analysis in 2001 found that PLWH had twice the risk of developing depression compared to people without HIV.44 Depression often emerges from the psychological stress of HIV diagnosis, and it is associated with stigma, trauma, isolation, poverty, and poor social support, leading to low HAART adherence and poor quality of life.45,46 Depression also contributes to behavioral risk factors, including unhealthy diet and lifestyle.47–49 Depression in patients with cardiovascular disease and cancer also increases toxic inflammatory reactions and reduces the production of anti-inflammatory cytokines, which promotes disease progression and increases mortality risk.50–53 Therefore, implementing targeted interventions to address depression may not only improve mental health outcomes but also mitigate inflammation and its associated health risks. Psychotherapies (e.g., interpersonal psychotherapy, behavior therapy), antidepressants, exercise, lifestyle changes, and social support may effectively reduce depression, lower depression-related mortality, and improve the quality of life among SMM living with HIV.54

Having an accurate understanding of mortality risk may improve medical decision-making and resource allocation in real-world clinical practice among SMM living with HIV, such as building it into EHR systems and prompting more intensive disease management, including further follow-ups among patients with high risk of comorbidities and mortality.9,10 The VACS-Depression Index may enhance the VACS Index for use in HIV clinical research, surveillance, and treatment.9 Online calculators of the index13 can integrate with EHRs to automatically calculate the risk scores of patients. The index can be used for risk adjustment in observational studies,14,15 identifying eligible participants in clinical trials based on their risk profiles,55 and monitoring morbidity and mortality risk among PLWH.9,56 Healthcare providers may use it to guide treatment, allocate resources, and support end-of-life planning.9

Our study has limitations. First, missing VACS Index 2.0 data was common in early post-HAART MACS visits, especially for FIB-4, eGFR, albumin, and BMI, which were mainly measured after 2000. Because over half of the deaths occurred before 2008 (the start of the modern ART era with integrase strand transfer inhibitor (INSTI)-based regimens), we could not examine time (pre- vs. post-modern ART era) as a moderator. Second, depression was measured using the CES-D scale, which measures clinically significant depressive symptoms instead of diagnoses of clinical depression using the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5); however, the validity of CES-D in predicting clinical depression is high.57 Third, psychosocial factors such as childhood sexual abuse, intimate partner violence, and other mental health conditions, including anxiety, were not measured among all the participants in the MACS, and including these factors may further improve the predictive accuracy of the VACS Index 2.0. Fourth, the MACS cohort used a nonrandom convenience sample and was restricted to middle-aged or older SMM in four metropolitan regions, and our findings may not extend to all SMM living with HIV in the US. The VACS-Depression Index requires validation outside the MACS before external application. Fifth, time-dependent confounding may exist, whereby changes in health status influence subsequent changes in exposures (e.g., the “sick quitter” effect in polydrug use). Sixth, we did not include cumulative measures of behavioral and psychosocial factors or interactions among these factors in this analysis due to statistical power constraints; future studies with larger samples could explore these aspects in more detail. In addition, we used calibration curves and C-statistics to assess predictive accuracy rather than other methods such as the Integrated Brier Score (IBS). Although the IBS provides an overall measure of prediction accuracy, it lacks clinical interpretability and does not indicate whether poor performance is due to inadequate discrimination (ranking) or poor calibration (probability estimates).58

In conclusion, the VACS Index 2.0 performed well in the MACS, and including depression improved its predictive capabilities among SMM living with HIV. Our findings support depression screenings in routine clinical care visits, providing opportunities to treat depression and related inflammation and reduce mortality.25,59,60 Having a predictive model integrating both biological and psychological factors will help clinicians better determine prognosis and provide depression treatment to reduce mortality among SMM.

Supplementary Material

Supplemental Digital Content

Supplemental Digital Content.pdf

Conflicts of Interest and Source of Funding:

The authors have no conflicts of interest to disclose. Data in this manuscript were collected by the Multicenter AIDS Cohort Study (MACS), now the MACS/WIHS Combined Cohort Study (MWCCS). The contents of this publication are solely the responsibility of the authors and do not represent the official views of the National Institutes of Health (NIH). MWCCS (Principal Investigators): Atlanta CRS (Ighovwerha Ofotokun, Anandi Sheth, and Gina Wingood), U01-HL146241; Baltimore CRS (Todd Brown and Joseph Margolick), U01-HL146201; Bronx CRS (Kathryn Anastos, David Hanna, and Anjali Sharma), U01-HL146204; Brooklyn CRS (Deborah Gustafson and Tracey Wilson), U01-HL146202; Data Analysis and Coordination Center (Gypsyamber D’Souza, Stephen Gange and Elizabeth Topper), U01-HL146193; Chicago-Cook County CRS (Mardge Cohen, Audrey French, and Ryan Ross), U01-HL146245; Chicago-Northwestern CRS (Steven Wolinsky, Frank Palella, and Valentina Stosor), U01-HL146240; Northern California CRS (Bradley Aouizerat, Jennifer Price, and Phyllis Tien), U01-HL146242; Los Angeles CRS (Roger Detels and Matthew Mimiaga), U01-HL146333; Metropolitan Washington CRS (Seble Kassaye and Daniel Merenstein), U01-HL146205; Miami CRS (Maria Alcaide, Margaret Fischl, and Deborah Jones), U01-HL146203; Pittsburgh CRS (Jeremy Martinson and Charles Rinaldo), U01-HL146208; UAB-MS CRS (Mirjam-Colette Kempf, James B. Brock, Emily Levitan, and Deborah Konkle-Parker), U01-HL146192; UNC CRS (M. Bradley Drummond and Michelle Floris-Moore), U01-HL146194. The MWCCS is funded primarily by the National Heart, Lung, and Blood Institute (NHLBI), with additional co-funding from the Eunice Kennedy Shriver National Institute Of Child Health & Human Development (NICHD), National Institute On Aging (NIA), National Institute Of Dental & Craniofacial Research (NIDCR), National Institute Of Allergy And Infectious Diseases (NIAID), National Institute Of Neurological Disorders And Stroke (NINDS), National Institute Of Mental Health (NIMH), National Institute On Drug Abuse (NIDA), National Institute Of Nursing Research (NINR), National Cancer Institute (NCI), National Institute on Alcohol Abuse and Alcoholism (NIAAA), National Institute on Deafness and Other Communication Disorders (NIDCD), National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), National Institute on Minority Health and Health Disparities (NIMHD), and in coordination and alignment with the research priorities of the National Institutes of Health, Office of AIDS Research (OAR). MWCCS data collection is also supported by UL1-TR000004 (UCSF CTSA), UL1-TR003098 (JHU ICTR), UL1-TR001881 (UCLA CTSI), P30-AI-050409 (Atlanta CFAR), P30-AI-073961 (Miami CFAR), P30-AI-050410 (UNC CFAR), P30-AI-027767 (UAB CFAR), P30-MH-116867 (Miami CHARM), UL1-TR001409 (DC CTSA), KL2-TR001432 (DC CTSA), and TL1-TR001431 (DC CTSA). The authors gratefully acknowledge the contributions of the study participants and dedication of the staff at the MWCCS sites.

REFERENCE

  • 1.Althoff KN, Chandran A, Zhang J, et al. Life-expectancy disparities among adults with HIV in the United States and Canada: the impact of a reduction in drug- and alcohol-related deaths using the lives saved simulation model. Am J Epidemiol. 2019;188:2097–2109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Wong C, Gange SJ, Moore RD, et al. Multimorbidity among persons living with human immunodeficiency virus in the United States. Clin Infect Dis. 2018;66:1230–1238. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Hogg RS, Eyawo O, Collins AB, et al. Health-adjusted life expectancy in HIV-positive and HIV-negative men and women in British Columbia, Canada: a population-based observational cohort study. Lancet HIV. 2017;4:e270–e276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Park LS, Tate JP, Sigel K, et al. Association of viral suppression with lower AIDS-defining and non-AIDS-defining cancer incidence in HIV-infected veterans: a prospective cohort study. Ann Intern Med. 2018;169:87–96. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Althoff KN, McGinnis KA, Wyatt CM, et al. Comparison of risk and age at diagnosis of myocardial infarction, end-stage renal disease, and non-AIDS-defining cancer in HIV-infected versus uninfected adults. Clin Infect Dis. 2015;60:627–638. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Havlik RJ, Brennan M, Karpiak SE. Comorbidities and depression in older adults with HIV. Sex Health. 2011;8:551–559. [DOI] [PubMed] [Google Scholar]
  • 7.Tate JP, Justice AC, Hughes MD, et al. An internationally generalizable risk index for mortality after one year of antiretroviral therapy. AIDS. 2013;27:563–572. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Justice AC, McGinnis KA, Skanderson M, et al. Towards a combined prognostic index for survival in HIV infection: the role of “non-HIV” biomarkers. HIV Med. 2010;11:143–151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Tate JP, Sterne JAC, Justice AC, et al. Albumin, white blood cell count, and body mass index improve discrimination of mortality in HIV-positive individuals. AIDS. 2019;33:903–912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.McGinnis KA, Justice AC, Marconi VC, et al. Combining Charlson comorbidity and VACS indices improves prognostic accuracy for all-cause mortality for patients with and without HIV in the Veterans Health Administration. Front Med. 2024;10:1342466. [Google Scholar]
  • 11.Justice AC, Modur SP, Tate JP, et al. Predictive accuracy of the Veterans Aging Cohort Study Index for mortality with HIV infection: a North American cross-cohort analysis. J Acquir Immune Defic Syndr. 2013;62:149–163. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.McGinnis KA, Justice AC, Moore RD, et al. Discrimination and calibration of the Veterans Aging Cohort Study Index 2.0 for predicting mortality among people with human immunodeficiency virus in North America. Clin Infect Dis. 2022;75:297–304. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Veterans Aging Cohort Study (VACS) 2.0 Index. MDCalc. Available at: https://www.mdcalc.com/calc/10402/veterans-aging-cohort-study-vacs-2.0-index. [Google Scholar]
  • 14.Yuh B, Tate J, Butt AA, et al. Weight change after antiretroviral therapy and mortality. Clin Infect Dis. 2015;60:1852–1859. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Justice AC, Gordon KS, Skanderson M, et al. Nonantiretroviral polypharmacy and adverse health outcomes among HIV-infected and uninfected individuals. AIDS. 2018;32:739–749. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Salinas JL, Rentsch C, Marconi VC, et al. Baseline, time-updated, and cumulative HIV care metrics for predicting acute myocardial infarction and all-cause mortality. Clin Infect Dis. 2016;63:1423–1430. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Marquine MJ, Montoya JL, Umlauf A, et al. The Veterans Aging Cohort Study (VACS) Index and neurocognitive change: a longitudinal study. Clin Infect Dis. 2016;63:694–702. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Marquine MJ, Umlauf A, Rooney A, et al. The Veterans Aging Cohort Study (VACS) Index is associated with concurrent risk for neurocognitive impairment. J Acquir Immune Defic Syndr. 2014;65:190–197. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Justice AC, Freiberg MS, Tracy R, et al. Does an index composed of clinical data reflect effects of inflammation, coagulation, and monocyte activation on mortality among those aging with HIV? Clin Infect Dis. 2012;54:984–994. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Akgün KM, Gordon K, Pisani M, et al. Risk factors for hospitalization and medical intensive care unit (MICU) admission among HIV-infected veterans. J Acquir Immune Defic Syndr. 2013;62:52–59. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Akgün KM, Tate JP, Crothers K, et al. An adapted frailty-related phenotype and the VACS Index as predictors of hospitalization and mortality in HIV-infected and uninfected individuals. J Acquir Immune Defic Syndr. 2014;67:397–404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Qian Y, Moore RD, Coburn SB, et al. Association of the VACS Index with hospitalization among people with HIV in the NA-ACCORD. J Acquir Immune Defic Syndr. 2022;89:9–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Justice AC, McGinnis KA, Tate JP, et al. Risk of mortality and physiologic injury evident with lower alcohol exposure among HIV-infected compared with uninfected men. Drug Alcohol Depend. 2016;161:95–103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Williams EC, McGinnis KA, Tate JP, et al. HIV disease severity is sensitive to temporal changes in alcohol use: a national study of VA patients with HIV. J Acquir Immune Defic Syndr. 2019;81:448. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Cohen MH, Hotton AL, Hershow RC, et al. Gender-related risk factors improve mortality predictive ability of VACS Index among HIV-infected women. J Acquir Immune Defic Syndr. 2015;70:538–544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Mimiaga MJ, OʼCleirigh C, Biello KB, et al. The effect of psychosocial syndemic production on 4-year HIV incidence and risk behavior in a large cohort of sexually active men who have sex with men. J Acquir Immune Defic Syndr. 2015;68:329–336. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Satyanarayana S, Safren SA, Rogers BG, et al. Estimating HIV transmissions in a large U.S. clinic-based sample: effects of time and syndemic conditions. J Int AIDS Soc. 2021;24:e25679. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Parsons JT, Millar BM, Moody RL, et al. Syndemic conditions and HIV transmission risk behavior among HIV-negative gay and bisexual men in a U.S. national sample. Health Psychol. 2017;36:695–703. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Scheer JR, Clark KA, Maiolatesi AJ, et al. Syndemic profiles and sexual minority men’s HIV-risk behavior: a latent class analysis. Arch Sex Behav. 2021;50:2825–2841. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Li J, Berg CJ, Weber AA, et al. Tobacco use at the intersection of sex and sexual identity in the U.S., 2007–2020: a meta-analysis. Am J Prev Med. 2021;60:415–424. [DOI] [PubMed] [Google Scholar]
  • 31.Storholm ED, Halkitis PN, Siconolfi DE, et al. Cigarette smoking as part of a syndemic among young men who have sex with men ages 13–29 in New York City. J Urban Health. 2011;88:663–676. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Greenwood GL, Paul JP, Pollack LM, et al. Tobacco use and cessation among a household-based sample of US urban men who have sex with men. Am J Public Health. 2005;95:145–151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Lee JGL, Griffin GK, Melvin CL. Tobacco use among sexual minorities in the USA, 1987 to May 2007: a systematic review. Tob Control. 2009;18:275–282. [DOI] [PubMed] [Google Scholar]
  • 34.Tobacco Use in LGBT Communities. Truth Initiative, June 23, 2021. Available at: https://truthinitiative.org/research-resources/targeted-communities/tobacco-use-lgbt-communities.
  • 35.Kaslow RA, Ostrow DG, Detels R, et al. The Multicenter AIDS Cohort Study: rationale, organization, and selected characteristics of the participants. Am J Epidemiol. 1987;126:310–318. [DOI] [PubMed] [Google Scholar]
  • 36.Sterling RK, Lissen E, Clumeck N, et al. Development of a simple noninvasive index to predict significant fibrosis in patients with HIV/HCV coinfection. Hepatology. 2006;43:1317–1325. [DOI] [PubMed] [Google Scholar]
  • 37.Levey AS, Stevens LA, Schmid CH, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med. 2009;150:604–612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Lewinsohn PM, Seeley JR, Roberts RE, et al. Center for Epidemiologic Studies Depression Scale (CES-D) as a screening instrument for depression among community-residing older adults. Psychol Aging. 1997;12:277–287. [DOI] [PubMed] [Google Scholar]
  • 39.Alcohol Glossary. National Center for Health Statistics. Centers for Disease Control and Prevention, March 6, 2018. Available at: https://www.cdc.gov/nchs/nhis/alcohol/alcohol_glossary.htm.
  • 40.Hessol NA, Kalinowski A, Benning L, et al. Mortality among participants in the Multicenter AIDS Cohort Study and the Women’s Interagency HIV Study. Clin Infect Dis. 2007;44:287–294. [DOI] [PubMed] [Google Scholar]
  • 41.Arlot S, Celisse A. A survey of cross-validation procedures for model selection. Stat Surv. 2010;4:40–79. [Google Scholar]
  • 42.Greenland P, O’Malley PG. When is a new prediction marker useful? A consideration of lipoprotein-associated phospholipase A2 and C-reactive protein for stroke risk. Arch Intern Med. 2005;165:2454–2456. [DOI] [PubMed] [Google Scholar]
  • 43.Pencina MJ, D’Agostino RB, D’Agostino RB, et al. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27:157–172; discussion 207–212. [DOI] [PubMed] [Google Scholar]
  • 44.Ciesla JA, Roberts JE. Meta-analysis of the relationship between HIV infection and risk for depressive disorders. Am J Psychiatry. 2001;158:725–730. [DOI] [PubMed] [Google Scholar]
  • 45.Arseniou S, Arvaniti A, Samakouri M. HIV infection and depression. Psychiatry Clin Neurosci. 2014;68:96–109. [DOI] [PubMed] [Google Scholar]
  • 46.Tao J, Vermund SH, Qian HZ. Association between depression and antiretroviral therapy use among people living with HIV: a meta-analysis. AIDS Behav. 2018;22:1542–1550. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Roshanaei-Moghaddam B, Katon WJ, Russo J. The longitudinal effects of depression on physical activity. Gen Hosp Psychiatry. 2009;31:306–315. [DOI] [PubMed] [Google Scholar]
  • 48.Kingsbury M, Dupuis G, Jacka F, et al. Associations between fruit and vegetable consumption and depressive symptoms: evidence from a national Canadian longitudinal survey. J Epidemiol Community Health. 2016;70:155–161. [DOI] [PubMed] [Google Scholar]
  • 49.Appelhans BM, Whited MC, Schneider KL, et al. Depression severity, diet quality, and physical activity in women with obesity and depression. J Acad Nutr Diet. 2012;112:693–698. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Carney RM, Freedland KE, Miller GE, et al. Depression as a risk factor for cardiac mortality and morbidity: a review of potential mechanisms. J Psychosom Res. 2002;53:897–902. [DOI] [PubMed] [Google Scholar]
  • 51.Sotelo JL, Musselman D, Nemeroff C. The biology of depression in cancer and the relationship between depression and cancer progression. Int Rev Psychiatry. 2014;26:16–30. [DOI] [PubMed] [Google Scholar]
  • 52.Cohen L, Cole SW, Sood AK, et al. Depressive symptoms and cortisol rhythmicity predict survival in patients with renal cell carcinoma: role of inflammatory signaling. PLoS One. 2012;7:e42324. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Gustad LT, Laugsand LE, Janszky I, et al. Symptoms of anxiety and depression and risk of heart failure: the HUNT Study. Eur J Heart Fail. 2014;16:861–870. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Renn BN, Areán PA. Psychosocial treatment options for major depressive disorder in older adults. Curr Treat Options Psychiatry. 2017;4:1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Tashima K, Smeaton L, Klingman K, et al. Mortality among HIV+ Participants Randomized to Omit NRTIs vs. Add NRTIs in OPTIONS (ACTG A5241). Presented at: Conference on Retroviruses and Opportunistic Infections; 2014; Boston. [Google Scholar]
  • 56.Justice AC, Tate JP. Strengths and limitations of the Veterans Aging Cohort Study Index as a measure of physiologic frailty. AIDS Res Hum Retroviruses. 2019;35:1023–1033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Cosco TD, Prina M, Stubbs B, et al. Reliability and validity of the Center for Epidemiologic Studies Depression Scale in a population-based cohort of middle-aged U.S. adults. J Nurs Meas. 2017;25:476–485. [DOI] [PubMed] [Google Scholar]
  • 58.Wang X, Claggett BL, Tian L, et al. Quantifying and interpreting the prediction accuracy of models for the time of a cardiovascular event - moving beyond the C statistic. JAMA Cardiol. 2023;8(3):290–295. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Bengtson AM, Pence BW, O’Donnell J, et al. Improvements in depression and changes in quality of life among HIV-infected adults. AIDS Care. 2015;27:47–53. [Google Scholar]
  • 60.Gallo JJ, Morales KH, Bogner HR, et al. Long-term effect of depression care management on mortality in older adults: follow-up of cluster randomized clinical trial in primary care. BMJ. 2013;346:f2570. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Digital Content
NIHMS2109749-supplement-Supplemental_Digital_Content.pdf (176.1KB, pdf)

RESOURCES