Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Feb 15.
Published in final edited form as: J Affect Disord. 2020 Dec 9;281:208–215. doi: 10.1016/j.jad.2020.12.024

Depression, antiretroviral therapy initiation, and HIV viral suppression among people who inject drugs in Vietnam

Sara N LEVINTOW 1, Brian W PENCE 1, Kimberly A POWERS 1, Alexander BRESKIN 2, Teerada SRIPAIPAN 3, Tran Viet HA 4, Viet Anh CHU 4, Vu Minh QUAN 5, Carl A LATKIN 6, Vivian F GO 3
PMCID: PMC7855445  NIHMSID: NIHMS1655256  PMID: 33333474

Abstract

Background.

The burden of depression is high among people who inject drugs (PWID) and may contribute to the spread of HIV through poor treatment engagement and persistent viremia. We estimated the effects of depression on antiretroviral therapy (ART) initiation and viral suppression among PWID living with HIV.

Methods.

Longitudinal data were collected from 455 PWID living with HIV in Vietnam during 2009–2013. We estimated the 6- and 12-month cumulative incidence of ART initiation and viral suppression, accounting for time-varying confounding, competing events, and missing data. The cumulative incidence difference (CID) contrasted the incidence of each outcome had participants always vs. never experienced severe depressive symptoms across study visits to date.

Results.

Severe depressive symptoms decreased the cumulative incidence of ART initiation, with CID values comparing always vs. never having severe depressive symptoms of −7.5 percentage points (95% CI: −17.2, 2.2) at 6 months and −7.1 (95% CI: −17.9, 3.7) at 12 months. There was no appreciable difference in the cumulative incidence of viral suppression at 6 months (CID = 0.3, 95% CI: −11.3, 11.9) or 12 months (CID = 2.0, 95% CI: −21.8, 25.8).

Limitations.

Discrepancies between the ART initiation and viral suppression outcomes could be due to under-reporting of ART use and missing data on viral load.

Conclusions.

Future work probing the seemingly antagonistic effect of depression on treatment uptake - but not viral suppression - will inform the design of interventions promoting HIV clinical outcomes and reducing onward transmission among PWID.

Keywords: Depression, people who inject drugs, HIV, antiretroviral therapy, viral suppression

INTRODUCTION

Injection drug use is a key driver of the HIV epidemic, particularly in Asia and eastern Europe (DeHovitz et al., 2014; El-Bassel et al., 2014). Sharing injection equipment is one of the most efficient means of HIV acquisition and transmission (Patel et al., 2014; Thomas et al., 2014). Antiretroviral therapy (ART) in people living with HIV can improve clinical outcomes and reduce onward transmission risk through viral suppression and a corresponding reduction in infectiousness (Cohen et al., 2011). However, despite successful scale-up of HIV treatment services for people who inject drugs (PWID) in some high-resource settings (Des Jarlais et al., 2016), ART use is insufficient for most PWID globally (Boltaev et al., 2013; LaMonaca et al., 2019; Larney et al., 2017). In Vietnam, a setting where the HIV epidemic is concentrated among PWID (Lancaster et al., 2018; Quan et al., 2011), expanded ART use specifically among PWID could substantially reduce new infections (Kato et al., 2013); however, treatment is typically initiated at a late stage of infection in this population (Kato et al., 2014), hindering the impact of “treatment as prevention” on HIV transmission.

Optimizing HIV treatment as prevention among PWID may require addressing depression as an underlying cause of low treatment engagement. Up to 50% of PWID suffer from severe depressive symptoms (Anagnostopoulos et al., 2015; Bouhnik et al., 2005; Levintow et al., 2018; Li et al., 2015, 2014), and though not focused on PWID, a large body of research has linked depression to poor HIV treatment outcomes (Bengtson et al., 2019; Bing et al., 2001; Leserman, 2008; Lesko et al., 2017; Nanni et al., 2015; Pence et al., 2018, 2007; Todd et al., 2017; Treisman and Angelino, 2007). In recent work in Vietnam, we observed a high burden of depression among PWID living with HIV (Levintow et al., 2018) and found that depression increased the risk of sharing injection equipment (Levintow et al., 2020). If depression concomitantly inhibits ART initiation and/or viral suppression, then the corresponding increase in biological transmission risk resulting from depression could be amplified by these co-occurring injection risk behaviors.

In this study, we sought to understand the role of depression in ART initiation and viral suppression among male PWID living with HIV in Vietnam. We hypothesized that severe depressive symptoms would decrease the incidence of both ART initiation and viral suppression over 6 and 12 months. Given the disproportionate impact of both HIV and depression among PWID, it is critical to better understand depression as a potential underlying cause of poor clinical outcomes and increased transmission risk in this population.

METHODS

Parent Trial Design and Population

This study was nested within a randomized controlled trial of a multi-level HIV stigma and risk reduction intervention that enrolled PWID living with HIV in Thai Nguyen, Vietnam from 2009 to 2013 (Go et al., 2015). Thai Nguyen is a province in northeastern Vietnam with an estimated HIV prevalence of 34% among its approximately 6,000 PWID (Ministry of Health of Vietnam, 2011; Socialist Republic of Viet Nam, 2014; Thai Nguyen Provincial AIDS Center, 2007). Participants were recruited via snowball sampling from the 32 sub-districts of Thai Nguyen with the most PWID. The trial enrolled 455 participants who met the following eligibility criteria: 1) HIV-positive (confirmed through study testing), 2) male (because 97% of PWID in Thai Nguyen are male, and females would require a different intervention), 3) age ≥18 years, 4) reported having sex and injecting drugs in the prior six months, and 5) planned to live in Thai Nguyen for the next 24 months.

In a two-stage process, the 32 sub-districts where participants lived were randomized to a structural intervention, and the 455 enrolled participants across sub-districts were randomized to an individual-level intervention (Go et al., 2015). This process resulted in four trial arms to which participants could belong: 1) control (standard of care), 2) individual (individual-level intervention only), 3) sub-district (sub-district-level structural intervention only), and 4) combined (structural and individual interventions). The structural intervention aimed to reduce community-level HIV and injection drug use stigma, and the individual intervention provided support to participants in coping with HIV and reducing transmission risk behaviors. As previously reported (Go et al., 2015), no differences in injecting or sexual behaviors were observed across trial arms.

Measures

Questionnaire and laboratory data were collected at study visits every six months during the two-year trial (baseline, 6, 12, 18, 24 months). The questionnaire was administered in a face-to-face interview and collected information on demographics, general health, injection drug use and other substance use, depressive symptoms, HIV transmission risk behaviors, history of HIV testing, and ART use. Blood specimens were collected to confirm HIV infection at baseline and to measure CD4 cell count at all visits. In addition to self-reported ART use (as part of the questionnaire), participants were seen by the study physician who also ascertained current ART use. Participants were referred to ART clinics during HIV testing and counseling at trial baseline. National eligibility criteria for ART were CD4 cell count ≤200 cells/μl in 2009 and had moved to CD4 cell count ≤350 cells/μl by 2013 (Kato et al., 2014; Nguyen et al., 2013). Although this study did not ascertain participants’ ART regimens, a national evaluation of ART in Vietnam found that most patients were prescribed non-nucleoside reverse transcriptase inhibitor-based regimens that included nevirapine or efavirenz, consistent with recommendations from national guidelines during the study period (Nguyen et al., 2013).

The exposure of interest was severe depressive symptoms over the past week, as assessed with the 20-item Center for Epidemiologic Studies Depression Scale (CES-D), which has been validated for measuring depressive symptoms in Vietnam (Radloff, 1977; Thai et al., 2016). Consistent with past work including studies in Vietnam (Huynh et al., 2017; Levintow et al., 2018; Radloff, 1977; Thai et al., 2016), we defined severe depressive symptoms as CES-D scores ≥23 and no or mild symptoms as scores <23.

We specified two outcomes of interest: ART initiation and viral suppression. A participant was considered to have initiated ART in the prior six months as of the first visit at which the study physician reported participant ART use. If the physician’s report of ART use was missing (occurring for 17% of all study visits), we used the participant’s self-report of ART use. In prior work in this population, we found 92% concordance between self- and physician-reported ART use (Zelaya et al., 2016). Because viral load was not measured in the parent trial, we performed HIV RNA testing on stored blood plasma specimens using COBAS® AmpliPrep/COBAS® TaqMan® HIV-1 Test platform (Roche Diagnostics GmbH) with a lower limit of detection of 20 copies/mL. Participants were considered to have experienced the viral suppression outcome as of the first visit with viral load <400 copies/mL (Attia et al., 2009). Because missing viral load data resulted from insufficient sample volume (<1mL) for some participants, particularly for stored specimens at later follow-up visits, we evaluated both outcomes of interest only at the two earliest follow-up visits (6 and 12 months).

We identified potential confounders a priori as questionnaire or laboratory variables that could plausibly affect depression, ART initiation, and viral suppression. To prevent inclusion of variables mediating the relationship between depression and a given outcome, potential confounder values corresponded to the time period prior to exposure assessment (Robins, 1989; Robins et al., 2000). Confounders included demographics, drug use, and HIV disease progression, and the causal diagram summarizing these hypothesized relationships is provided in Supplemental Figure S1.

Study outreach workers attempted to trace participants who missed a visit using contact information collected at baseline. During tracing, outreach workers asked participant contacts if the participant had died or was incarcerated. Death and incarceration were treated as competing events for the outcomes of interest.

Statistical Analysis

We used a semi-parametric inverse probability-weighted estimator of the cumulative incidence of ART initiation and viral suppression (analyzed separately), accounting for the time-varying nature of depression and confounders and the occurrence of the competing events of death and incarceration (Hernán et al., 2000; Hubbard et al., 2000). We estimated the cumulative incidence of each outcome at 6 and 12 months in three scenarios: 1) Natural course: the incidence of the outcome in the observed study population, 2) Never depressed: the incidence of the outcome if participants never had severe depressive symptoms (i.e., symptoms were mild or absent, defined as CES-D score <23, at all study visits up to and including the visit prior to outcome ascertainment), and 3) Always depressed: the incidence of the outcome if participants always had severe depressive symptoms (CES-D ≥23 at all study visits up to and including the visit prior to outcome ascertainment). Given that depressive symptoms varied over time, these two extreme conditions were intended to capture the largest possible difference: the risk of the outcome if participants always had severe symptoms vs. the risk of the outcome if participants never had severe symptoms. Importantly, all analyses were structured such that the depression assessment preceded ART initiation and viral suppression in order to avoid possible reverse causation (e.g., mental health side effects of efavirenz use). Further details on the inverse-probability weighted estimators are provided in the Supplemental Text.

Application of the weights to the study population removes the association between the hypothesized confounding variables (included in the weights models) and depressive symptoms, permitting estimation of a causal effect between depression and our outcomes of interest under key assumptions of consistency, conditional exchangeability, positivity, and no measurement error (Cole and Hernan, 2008; Hernan and Robins, 2006) (see Discussion). In the weighted study population, we estimated the cumulative incidence of each outcome at 6 and 12 months and the cumulative incidence difference (CID) contrasting the never-depressed and always-depressed scenarios. The CID corresponds to the difference in 6- and 12-month cumulative incidence of the outcome if participants always had severe depressive symptoms, compared with the risk if participants never had severe depressive symptoms.

In our main analyses, the cumulative incidence of each outcome was estimated for participants who had not experienced that outcome at baseline, resulting in slightly different analytic samples for the two outcomes (e.g., participants who reported ART at baseline, but were not virally suppressed, were included in the viral suppression analysis, but not in the ART initiation analysis). In a second set of analyses, we restricted analysis of both outcomes to the same sample (participants with neither ART use nor viral suppression at baseline) to facilitate more direct comparison of estimates across outcomes.

There was substantial missing data due to insufficient sample volume for measuring viral load (31% at 6 months, 59% at 12 months) in addition to intermittent missing data on other study variables not due to death or incarceration (<15% across all variables at all visits). To address missing data, we used multiple imputation by chained equations (MICE) (Rubin, 1987; van Buuren and Groothuis-Oudshoorn, 2011), analyzing 50 complete datasets with values imputed for depression, ART initiation, viral suppression, and all hypothesized confounders. We used Rubin’s formula to pool results from the 50 imputed datasets and calculate variance accounting for within- and between-imputation variability (Rubin, 1987).

All analyses were conducted using R Version 3.4.3 (R Core Team, 2017).

Ethics

The parent trial and this analysis were approved by the ethical review committees at the Thai Nguyen Center for Preventive Medicine, the Johns Hopkins Bloomberg School of Public Health, and the University of North Carolina at Chapel Hill. Written informed consent was obtained from all participants.

RESULTS

Among the 455 participants at baseline, the median age was 35 years (interquartile range [IQR]: 30, 39), nearly half (47%) were married or cohabitating, and most (69%) were employed full-time. Most participants (73%) reported sharing injection drug use equipment with injecting partners over the past three months, and one-quarter (24%) reported sex without a condom in the prior three months. Prior to HIV testing at baseline, 74% of participants reported no previous HIV diagnosis. Table 1 shows additional characteristics of the study population.

Table 1.

Time-fixed and time-varying characteristics of 455 HIV-positive male PWID in Thai Nguyen, Vietnam at study baseline, 6 months, and 12 months.

Time-fixed characteristics Baseline Median (IQR) or N (%)
Age in years (range 19–60) 35 (30, 39)
Married or cohabitating 215 (47)
Full-time employment 315 (69)
Number of days (in past 90 days) injected heroin 70 (15, 90)
Any sharing of injection equipment (past 90 days) 332 (73)
Any sex without a condom (past 90 days) 108 (24)
Self-rated health as poor 136 (30)
History of overdose 84 (18)
Any alcohol use 307 (67)
No prior HIV diagnosis 336 (74)
Time-varying characteristics Baseline Median (IQR) or N (%) 6 Months Median (IQR) or N (%)i 12 Months Median (IQR) or N (%)ii
Severe depressive symptoms (CES-D≥23)iii 201 (44) 169 (37) 116 (25)
ART useiv 58 (13) 148 (33) 165 (36)
CD4 cell count (cells/μl)v 241 (126, 370) 251 (154, 387) 260 (151, 382)
HIV viral load (log10 copies/mL)vi 4.3 (2.6, 4.9) 3.6 (1.8, 4.5) 3.9 (3.2, 4.5)
Open in a new tab
i

Percentages are based on the total sample size of 455 participants. By 6 months, 55 (12%) participants had experienced competing events of death or incarceration.

ii

Percentages are based on the total sample size of 455 participants. By 12 months, 101 (22%) participants had experienced competing events of death or incarceration.

iii

There was missing data on CES-D scores (not due to death or incarceration) for 1 participant at baseline, 23 participants at 6 months, and 27 participants at 12 months.

iv

There was missing data on ART use (not due to death or incarceration) for 6 participants at baseline, 23 participants at 6 months, and 27 participants at 12 months.

v

There was missing data on CD4 cell count (not due to death or incarceration) for 9 participants at baseline, 43 participants at 6 months, and 50 participants at 12 months.

vi

HIV viral load was fully observed at baseline, but missing data (not due to death or incarceration) occurred for 162 participants at 6 months and 295 participants at 12 months.

The percentage of participants with severe depressive symptoms (CES-D ≥23) was 44% at baseline, decreasing to 37% at 6 months and 25% at 12 months (Table 1). This decreasing prevalence of depression over time was likely due in part to disproportionately higher risks of competing events over follow-up among participants who had severe depressive symptoms at baseline (Table 2, Supplemental Figure S2). At baseline, only 13% of participants reported current ART use; this percentage increased to 33% and 36%, at 6 and 12 months, respectively. There was a slight increase in CD4 cell count from baseline (median 241 cells/μL) to 6 months (median 251 cells/μL) and 12 months (260 cells/μL). The median HIV viral load was 4.3 log10 copies/mL at baseline, decreasing to 3.6 log10 copies/mL at 6 months, and then increasing to 3.9 log10 copies/mL at 12 months.

Table 2.

Distribution of depression, competing events, and missing data over follow-up, by baseline depression.

Severe depressive symptoms, N = 201 n (%)i No severe symptoms, N = 253 n (%)i
Severe depressive symptoms
 At 6 months 103 (51%) 65 (26%)
 At 12 months 73 (36%) 43 (17%)
Incarceration
 At 6 months 9 (4%) 5 (2%)
 At 12 months 18 (9%) 11 (4%)
Death
 During 0–6 months 30 (15%) 11 (4%)
 During 6–12 months 14 (7%) 16 (6%)
Missing dataii on ART
 At 6 months 5 (2%) 18 (7%)
 At 12 months 11 (5%) 16 (6%)
Missing dataii on viral load
 At 6 months 70 (35%) 91 (36%)
 At 12 months 119 (59%) 176 (70%)
Open in a new tab
i

Percentages are based on the sample sizes of 201 participants with severe depressive symptoms at baseline and 253 participants with no or mild depressive symptoms at baseline. One participant had a missing CES-D score at baseline.

ii

Frequencies and percentages correspond to participants who were missing data not due to death or incarceration.

Among 397 participants who had not initiated ART at baseline, the cumulative incidence of ART initiation in the natural course scenario (unweighted study population) was 24% (95% CI: 20%, 29%) at 6 months and 35% (95% CI: 30%, 40%) at 12 months (Table 3, Figure 1). In the never-depressed scenario (weighted incidence if participants never had severe depressive symptoms), the cumulative incidence of ART initiation was slightly higher: 27% (95% CI: 20%, 34%) at 6 months and 37% (95% CI: 30%, 44%) at 12 months. In the always-depressed scenario (weighted risk of outcome if participants always had severe depressive symptoms), the cumulative incidence of ART initiation was somewhat lower: 20% (95% CI: 13%, 27%) at 6 months and 30% (95% CI: 22%, 38%) at 12 months.

Table 3. Estimates of 6-month and 12-month cumulative incidence of ART initiation and viral suppression by depression scenario and the cumulative incidence difference (CID) contrasting the always-depressed and never-depressed scenarios.

Cumulative incidence estimates for the natural course (observed study population) are provided for reference. The study sample for the ART initiation outcome was n=397 participants without ART use at baseline, and the study sample for the viral suppression outcome was n=342 participants without viral suppression at baseline.

6-month cumulative incidence (95 CI%) 6-month CID by depression (95% CI) 12-month cumulative incidence (95 CI%) 12-month CID by depression (95% CI)
ART Initiation
Always Depressed 0.20 (0.13, 0.27) −0.08 (−0.17, 0.02) 0.30 (0.22, 0.38) −0.07 (−0.18, 0.04)
Natural Course 0.24 (0.20, 0.29) 0.35 (0.30, 0.40)
Never Depressed 0.27 (0.20, 0.34) 0.37 (0.30, 0.44)
Viral Suppression
Always Depressed 0.20 (0.11, 0.28) 0.00 (−0.11, 0.12) 0.48 (0.29, 0.68) 0.02 (−0.22, 0.26)
Natural Course 0.20 (0.14, 0.25) 0.44 (0.34, 0.54)
Never Depressed 0.19 (0.11, 0.27) 0.46 (0.31, 0.62)
Open in a new tab

Figure 1. Estimates of 6-month and 12-month cumulative incidence of ART initiation and viral suppression.

Figure 1.

Open in a new tab

The study sample for the ART initiation outcome was n=397 participants without ART use at baseline, and the study sample for the viral suppression outcome was n=342 participants without viral suppression at baseline.

Among 342 participants who were not virally suppressed at baseline (HIV RNA ≥400 copies/mL), the cumulative incidence of viral suppression in the natural course was 20% (95% CI: 14%, 25%) at 6 months and 44% (95% CI: 34%, 54%) at 12 months (Figure 1, Table 3). Estimates of cumulative incidence were similar in both of the hypothetical depression scenarios. In the never-depressed scenario, the cumulative incidence of viral suppression was 19% (95% CI: 11%, 27%) at 6 months and 46% (95% CI: 31%, 62%) at 12 months. In the always-depressed scenario, the cumulative incidence of viral suppression was 20% (95% CI: 11%, 28%) at 6 months and 48% (95% CI: 29%, 68%) at 12 months.

Severe depressive symptoms (compared with no severe symptoms) decreased the cumulative incidence of ART initiation, with CID values comparing always vs. never having severe depressive symptoms of −7.5 (95% CI: −17.2, 2.2) percentage points at 6 months and −7.1 (95% CI: −17.9, 3.7) percentage points at 12 months (Figure 2, Table 3). There were no appreciable differences in the cumulative incidence of viral suppression at 6 months (CID = 0.3 percentage points, 95% CI: −11.3, 11.9) or 12 months (CID = 2.0 percentage points, 95% CI: −21.8, 25.8).

Figure 2. Estimates of 6-month and 12-month cumulative incidence differences for ART initiation and viral suppression, contrasting the always-depressed scenario with the never-depressed scenario.

Figure 2.

Open in a new tab

The study sample for the ART initiation outcome was n=397 participants without ART use at baseline, and the study sample for the viral suppression outcome was n=342 participants without viral suppression at baseline.

Findings were largely unchanged in analyses where we estimated both outcomes in the set of participants who had not experienced either outcome at baseline (Supplemental Figure S3).

DISCUSSION

Using longitudinal data on PWID living with HIV in Vietnam, we found that severe depressive symptoms decreased the cumulative incidence of ART initiation, but not viral suppression, over one year. We used a rigorous methodological approach that accounted for the episodic nature of depression, time-varying confounding, substantial missing data, and common competing events of death and incarceration. By focusing on a population disproportionately impacted by both depression and HIV, we sought to better understand depression as a cause of phenomena that could lead to poor health outcomes and potential onward transmission. Although an inverse relationship between depression and HIV treatment-related outcomes has been established, prior work has largely focused on HIV patients who were engaged in clinical care (Bengtson et al., 2019; Bing et al., 2001; Leserman, 2008; Lesko et al., 2017; Nanni et al., 2015; Pence et al., 2018, 2007; Treisman and Angelino, 2007). As the majority of our study population was newly diagnosed and/or not receiving ART at the start of follow-up, this analysis provides insights into the association between depression and HIV treatment-related outcomes outside of well-established clinical cohorts.

We contrasted two extreme conditions intended to capture the largest possible difference: the risk of the outcome if participants always had severe symptoms vs. the risk of the outcome if participants never had severe symptoms. For the ART initiation outcome, the CID for the always-vs. never-depressed scenarios was approximately −7 percentage points at both 6 and 12 months. These effect estimates relied on small sample sizes (i.e., the subsets of participants who were always or never depressed, respectively), resulting in wide confidence intervals; that is, a CID ranging from a large decrease in ART initiation (−17 to −18 percentage points) to a slight increase in ART incidence (2 to 4 percentage points) is compatible with the data. Given that the cumulative incidence of ART initiation in the observed study population (natural course) was 24% at 6 months and 35% at 12 months, the 7-percentage-point decrease in cumulative incidence represented by the point estimate is substantively meaningful. Despite this relationship between depression and ART initiation, we did not observe an effect of depression on viral suppression: the CIDs at 6 and 12 months were both near-zero with extremely wide confidence intervals.

Our seemingly disparate findings for the ART initiation and viral suppression outcomes should be interpreted in light of potential measurement error. We observed discrepancies between measurements of ART use and viral load across all study visits. Higher-than-expected numbers of participants did not report ART use but had very low or undetectable viral load (e.g., 14% of the study population did not report any ART use at study baseline, but had viral loads <400 copies/mL). As a result, the estimated incidence of viral suppression exceeded that of ART initiation at 12 months, even when assessment of both outcomes was restricted to the same set of participants with neither ART use nor viral suppression at baseline. Similar discrepancies between reported ART use and viral load measurements have been observed in previous research and have been considered to result from a combination of elite suppressors and ART reporting bias (Chen et al., 2014; Fogel et al., 2019; Marzinke et al., 2014). Given that elite suppression is rare (Madec et al., 2005), we assume that discrepancies between ART use and viral load in our study more commonly resulted from ART reporting bias. Of particular relevance, in a similar population of PWID living with HIV in Vietnam (enrolled in HIV Prevention Trials Network [HPTN] 074), 13% of participants who reported no prior ART had antiretroviral drugs detected through study screening (Fogel et al., 2019). Although no antiretroviral screening could be performed in our study, we used the physician-reported ART variable (in addition to participant reports) to reduce possible reporting bias. However, the reporting physician only saw participants as part of the study and may have been unaware of ART use overseen by external providers if not disclosed by the participant.

In addition to the possibility of elite suppressors and under-reported ART use, apparent discrepancies in our study findings may also result from uncertainty in the viral suppression outcome. There was substantial missing data on viral load due to insufficient volume of many collected samples. Because missingness was thought to occur at random, we used multiple imputation to impute viral suppression outcomes for the 31% and 59% of participants with missing data at 6 and 12 months, respectively. There was considerable variability across imputations, resulting in imprecise estimates of cumulative viral suppression incidence and corresponding risk differences. Compared to cumulative incidence estimates for the ART initiation outcome across imputed datasets (Supplemental Figure S4), estimates for the viral suppression outcome showed greater variability, particularly at 12 months (Supplemental Figure S5), with this source of uncertainty resulting in the wider CID confidence intervals for this outcome vs. the ART outcome (Supplemental Figures S7 vs. S6).

Limitations:

While there may be biological or behavioral mechanisms through which depression decreases ART initiation but not viral suppression, we cannot rule out missing data and measurement error as factors contributing to this apparently paradoxical finding in our study. Other study limitations were possible violations of the assumptions (Cole and Hernan, 2008; Hernan and Robins, 2006) required for valid interpretation of our CIDs as representing causal effects. First of these assumptions is sequential conditional exchangeability, which in our case means that participants who never have depression and who always have depression are exchangeable, conditional on measured confounders. We controlled for a variety of confounders through the use of inverse probability weights, but it is possible that unmeasured confounders led to imbalances in depression groups, thereby biasing estimates of the effect of depression on our outcomes. The second assumption is positivity, which requires that there were participants who were never depressed and who were always depressed in all confounder-defined subsets of the study population. We used model diagnostics to verify this assumption. Finally, the assumption of consistency, or treatment version irrelevance, holds that any variability in treatment (i.e., multiple versions of treatment) is irrelevant to the effect of treatment on the outcome. Here, we did not model a specific treatment or intervention on depression, and our results should only be interpreted as the hypothetical effect of eliminating severe depressive symptoms, without specifying the precise treatment or intervention used for elimination.

Our findings are specific to this study sample, which may not be representative of all PWID living with HIV. Although snowball sampling enabled the study team to access a hard-to-reach population, the non-random nature of our study sample potentially limits generalizability of findings to PWID with HIV in Vietnam who were not recruited. In addition, while men who inject drugs are a key population in the Vietnamese HIV epidemic, our findings may not be applicable to women who inject drugs, a key population for HIV prevention efforts in other parts of the world (El-Bassel and Strathdee, 2015). Importantly, this study focused on one specific challenge to HIV treatment as prevention at the individual level (depressive symptoms), but did not consider other related individual-level factors (e.g., other mental illness, perceived stigma) or structural barriers to ART initiation and viral suppression (e.g., access to ART providers, costs associated with medications, discrimination towards PWID in healthcare settings). Further research on the high burden of depression in PWID, the potentially overlapping experiences of stigma and other mental illness, and the structural challenges to engagement in HIV care faced by this population could inform the design of future intervention.

In conclusion, we found that severe depressive symptoms may have hindered ART initiation but not viral suppression among PWID living with HIV in Vietnam. During the study period (2009–2013), there were extremely limited treatment services for depression in Vietnam, but in recent years, mental health services have received growing prioritization and funding (Murphy et al., 2018; Vuong et al., 2011). Screening and treating depressive symptoms among PWID present opportunities not only to improve mental health and drug abuse outcomes, but also to increase uptake of lifesaving HIV treatment. Future work probing the seemingly antagonistic effect of depression on treatment uptake - but not viral suppression - may help to elucidate the relationships between these two outcomes and inform the design of interventions promoting HIV care engagement, treatment initiation, and viral suppression among PWID.

Supplementary Material

1

HIGHLIGHTS.

  • There is a high burden of depression and HIV among people who inject drugs.

  • Severe depressive symptoms led to decreased initiation of antiretroviral therapy.

  • There were no differences in viral suppression by depressive symptoms.

  • Discrepancies in outcomes may be due to reporting bias and missing data.

ACKNOWLEDGEMENTS

We thank Dr. Steven R. Meshnick for helpful discussions on the conception and design of the study. We thank Dr. Stephen R. Cole for thoughtful feedback on drafts of this manuscript.

ROLE OF FUNDING SOURCE

This study and the parent trial were funded by National Institute on Drug Abuse (R36 DA045569, R01 DA022962). Doctoral training support for Sara N. Levintow was provided by NIAID (T32 AI070114) and ViiV Healthcare (pre-doctoral fellowship). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding sources had no role in study design, data collection and analysis, interpretation of results, or preparation of the manuscript for publication.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

CONFLICTS OF INTEREST

No authors have any actual or potential conflicts of interest to disclose.

References

  1. Anagnostopoulos A, Ledergerber B, Jaccard R, Shaw SA, Stoeckle M, Bernasconi E, Barth J, Calmy A, Berney A, Jenewein J, Weber R, 2015. Frequency of and risk factors for depression among participants in the swiss HIV Cohort Study (SHCS). PLoS One 10, 1–17. 10.1371/journal.pone.0140943 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Attia S, Egger M, Müller M, Zwahlen M, Low N, 2009. Sexual transmission of HIV according to viral load and antiretroviral therapy: systematic review and meta-analysis. AIDS 23, 1397–1404. 10.1097/QAD.0b013e32832b7dca [DOI] [PubMed] [Google Scholar]
  3. Bengtson AM, Pence BW, Mimiaga MJ, Gaynes BN, Moore R, Christopoulos K, O’Cleirigh C, Grelotti D, Napravnik S, Crane H, Mugavero M, 2019. Depressive Symptoms and Engagement in Human Immunodeficiency Virus Care Following Antiretroviral Therapy Initiation. Clin. Infect. Dis 68, 475–481. 10.1093/cid/ciy496 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bing EG, Burnam M. a, Longshore D, Fleishman J. a, Sherbourne CD, London a S., Turner BJ, Eggan F, Beckman R, Vitiello B, Morton SC, Orlando M, Bozzette S. a, Ortiz-Barron L, Shapiro M, 2001. Psychiatric disorders and drug use among human immunodeficiency virus-infected adults in the United States. Arch. Gen. Psychiatry 58, 721–8. 10.1001/archpsyc.58.8.721 [DOI] [PubMed] [Google Scholar]
  5. Boltaev AA, El-Bassel N, Deryabina AP, Terlikbaeva A, Gilbert L, Hunt T, Primbetova S, Strathdee SA, 2013. Scaling up HIV prevention efforts targeting people who inject drugs in Central Asia: a review of key challenges and ways forward. Drug Alcohol Depend. 132 Suppl 1, S41–7. 10.1016/j.drugalcdep.2013.07.033 [DOI] [PubMed] [Google Scholar]
  6. Bouhnik A-D, Preau M, Vincent E, Carrieri MP, Gallais H, Lepeu G, Gastaut J-A, Moatti J-P, Spire B, 2005. Depression and clinical progression in HIV-infected drug users treated with highly active antiretroviral therapy. Antivir. Ther 10, 53–61. [PubMed] [Google Scholar]
  7. Chen I, Cummings V, Fogel JM, Marzinke MA, Clarke W, Connor MB, Griffith S, Buchbinder S, Shoptaw S, del Rio C, Magnus M, Mannheimer S, Wheeler DP, Mayer KH, Koblin BA, Eshleman SH, 2014. Low-Level Viremia Early in HIV Infection. JAIDS J. Acquir. Immune Defic. Syndr 67, 405–408. 10.1097/QAI.0000000000000298 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, Hakim JG, Kumwenda J, Grinsztejn B, Pilotto JHS, Godbole SV, Mehendale S, Chariyalertsak S, Santos BR, Mayer KH, Hoffman IF, Eshleman SH, Piwowar-Manning E, Wang L, Makhema J, Mills LA, de Bruyn G, Sanne I, Eron J, Gallant J, Havlir D, Swindells S, Ribaudo H, Elharrar V, Burns D, Taha TE, Nielsen-Saines K, Celentano D, Essex M, Fleming TR, HPTN 052 Study Team, 2011. Prevention of HIV-1 Infection with Early Antiretroviral Therapy. N. Engl. J. Med 365, 493–505. 10.1056/NEJMoa1105243 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cole SR, Hernan MA, 2008. Constructing Inverse Probability Weights for Marginal Structural Models. Am. J. Epidemiol 168, 656–664. 10.1093/aje/kwn164 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. DeHovitz J, Uuskula A, El-Bassel N, 2014. The HIV Epidemic in Eastern Europe and Central Asia. Curr. HIV/AIDS Rep 11, 168–176. 10.1007/s11904-014-0202-3 [DOI] [PubMed] [Google Scholar]
  11. Des Jarlais DC, Kerr T, Carrieri P, Feelemyer J, Arasteh K, 2016. HIV infection among persons who inject drugs. AIDS 30, 815–826. 10.1097/QAD.0000000000001039 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. El-Bassel N, Shaw SA, Dasgupta A, Strathdee SA, 2014. Drug use as a driver of HIV risks. Curr. Opin. HIV AIDS 9, 150–155. 10.1097/COH.0000000000000035 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. El-Bassel N, Strathdee SA, 2015. Women who use or inject drugs: An action agenda for women-specific, multilevel, and combination HIV prevention and research. J. Acquir. Immune Defic. Syndr 69, S182–S190. 10.1097/QAI.0000000000000628 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Fogel JM, Zhang Y, Palumbo PJ, Guo X, Clarke W, Breaud A, Richardson P, Piwowar-Manning E, Hamilton EL, Ha TV, Dumchev K, Djoerban Z, Hoffman I, Hanscom B, Miller WC, Eshleman SH, 2019. Use of Antiretroviral Drug Testing to Assess the Accuracy of Self-reported Data from HIV-Infected People Who Inject Drugs. AIDS Behav. 23, 2101–2108. 10.1007/s10461-018-2379-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Go VF, Frangakis C, Minh N. Le, Latkin C, Ha TV, Mo TT, Sripaipan T, Davis WW, Zelaya C, Vu PT, Celentano DD, Quan VM, 2015. Efficacy of a Multi-level Intervention to Reduce Injecting and Sexual Risk Behaviors among HIV-Infected People Who Inject Drugs in Vietnam: A Four-Arm Randomized Controlled Trial. PLoS One 10, 1–19. 10.1371/journal.pone.0125909 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hernán MÁ, Brumback B, Robins JM, 2000. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology 11, 561–570. 10.1097/00001648-200009000-00012 [DOI] [PubMed] [Google Scholar]
  17. Hernan MA, Robins JM, 2006. Estimating causal effects from epidemiological data. J. Epidemiol. Community Heal 60, 578–586. 10.1136/jech.2004.029496 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hubbard AE, van der Laan MJ, Robins JM, 2000. Nonparametric locally efficient estimation of the Treatment Specific Survival distribution with right Censored Data and Covariates in Observational Studies. Springer, New York, NY, pp. 135–177. 10.1007/978-1-4612-1284-3_3 [DOI] [Google Scholar]
  19. Huynh V-AN, To KG, Do D. Van, To QG, Nguyen MTH, 2017. Changes in depressive symptoms and correlates in HIV+ people at An Hoa Clinic in Ho Chi Minh City, Vietnam. BMC Psychiatry 17, 35 10.1186/s12888-016-1170-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kato M, Granich R, Bui DD, Tran HV, Nadol P, Jacka D, Sabin K, Suthar AB, Mesquita F, Lo YR, Williams B, 2013. The Potential Impact of Expanding Antiretroviral Therapy and Combination Prevention in Vietnam. JAIDS J. Acquir. Immune Defic. Syndr 63, e142–e149. 10.1097/QAI.0b013e31829b535b [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Kato M, Long NH, Duong BD, Nhan DT, Nguyen TT Van, Hai NH, Giang LM, Hoa DM, Van NT, Suthar AB, Fontaine C, Nadol P, Lo Y-R, McConnell MS, 2014. Enhancing the benefits of antiretroviral therapy in Vietnam: towards ending AIDS. Curr. HIV/AIDS Rep 11, 487–95. 10.1007/s11904-014-0235-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. LaMonaca K, Dumchev K, Dvoriak S, Azbel L, Morozova O, Altice FL, 2019. HIV, Drug Injection, and Harm Reduction Trends in Eastern Europe and Central Asia: Implications for International and Domestic Policy. Curr. Psychiatry Rep 21, 47 10.1007/s11920-019-1038-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lancaster KE, Hoffman IF, Hanscom B, Ha TV, Dumchev K, Susami H, Rose S, Go VF, Reifeis SA, Mollan KR, Hudgens MG, Piwowar-Manning EM, Richardson P, Dvoriak S, Djoerban Z, Kiriazova T, Zeziulin O, Djauzi S, Ahn CV, Latkin C, Metzger D, Burns DN, Sugarman J, Strathdee SA, Eshleman SH, Clarke W, Donnell D, Emel L, Sunner LE, McKinstry L, Sista N, Hamilton EL, Lucas JP, Duong BD, Van Vuong N, Sarasvita R, Miller WC, HPTN 074 Study Team, the H. 074 S., 2018. Regional differences between people who inject drugs in an HIV prevention trial integrating treatment and prevention (HPTN 074): a baseline analysis. J. Int. AIDS Soc 21, e25195 10.1002/jia2.25195 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Larney S, Peacock A, Leung J, Colledge S, Hickman M, Vickerman P, Grebely J, Dumchev KV, Griffiths P, Hines L, Cunningham EB, Mattick RP, Lynskey M, Marsden J, Strang J, Degenhardt L, 2017. Global, regional, and country-level coverage of interventions to prevent and manage HIV and hepatitis C among people who inject drugs: a systematic review. Lancet Glob. Heal 5, e1208–e1220. 10.1016/S2214-109X(17)30373-X [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Leserman J, 2008. Role of depression, stress, and trauma in HIV disease progression. Psychosom. Med 70, 539–545. 10.1097/PSY.0b013e3181777a5f [DOI] [PubMed] [Google Scholar]
  26. Lesko CR, Todd JV, Cole SR, Edmonds A, Pence BW, Edwards JK, Mack WJ, Bacchetti P, Rubtsova A, Gange SJ, Adimora AA, Saag M, Kempf M-C, Konkle-Parker D, Ofotokun I, Wingood G, Anastos K, Minkoff H, Gustafson D, Cohen M, French A, Young M, Kassaye S, Fischl M, Metsch L, Adimora A, Greenblatt R, Aouizerat B, Tien P, Gange S, Golub E, Milam J, 2017. Mortality under plausible interventions on antiretroviral treatment and depression in HIV-infected women: an application of the parametric g-formula. Ann. Epidemiol 27, 783–789.e2. 10.1016/j.annepidem.2017.08.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Levintow SN, Pence BW, Powers KA, Sripaipan T, Ha TV, Chu VA, Quan VM, Latkin CA, & Go VF, 2020. Estimating the effect of depression on HIV transmission risk behaviors among people who inject drugs in Vietnam: A causal approach. AIDS Behav. 2020 Aug 24:1–9. Online ahead of print. 10.1007/s10461-020-03007-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Levintow SN, Pence BW, Ha TV, Minh N. Le, Sripaipan T, Latkin CA, Vu PT, Quan VM, Frangakis C, Go VF, 2018. Prevalence and predictors of depressive symptoms among HIV-positive men who inject drugs in Vietnam. PLoS One 13, e0191548 10.1371/journal.pone.0191548 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Li J, Gu J, Lau JTF, Chen H, Mo PKH, Tang M, 2015. Prevalence of depressive symptoms and associated factors among people who inject drugs in China. Drug Alcohol Depend. 151, 228–235. 10.1016/j.drugalcdep.2015.03.028 [DOI] [PubMed] [Google Scholar]
  30. Li Y, Hershow R, Praptoraharjo I, Setiawan M, Levy J, 2014. Factors associated with symptoms of depression among injection drug users receiving antiretroviral treatment in Indonesia. J AIDS Clin Res 5, 303-. 10.4172/2155-6113.1000303.Factors [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Madec Y, Boufassa F, Porter K, Meyer L, CASCADE Collaboration, 2005. Spontaneous control of viral load and CD4 cell count progression among HIV-1 seroconverters. AIDS 19, 2001–7. 10.1097/01.aids.0000194134.28135.cd [DOI] [PubMed] [Google Scholar]
  32. Marzinke MA, Clarke W, Wang L, Cummings V, Liu T-Y, Piwowar-Manning E, Breaud A, Griffith S, Buchbinder S, Shoptaw S, del Rio C, Magnus M, Mannheimer S, Fields SD, Mayer KH, Wheeler DP, Koblin BA, Eshleman SH, Fogel JM, 2014. Nondisclosure of HIV status in a clinical trial setting: antiretroviral drug screening can help distinguish between newly diagnosed and previously diagnosed HIV infection. Clin. Infect. Dis 58, 117–20. 10.1093/cid/cit672 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Ministry of Health of Vietnam, 2011. Results from the HIV/STI integrated biological and behavioral surveillance (IBBS) in Vietnam, Round II, 2009. [Google Scholar]
  34. Murphy J, Corbett KK, Linh DT, Oanh PT, Nguyen VC, 2018. Barriers and facilitators to the integration of depression services in primary care in Vietnam: a mixed methods study. BMC Health Serv. Res 18, 641 10.1186/s12913-018-3416-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Nanni MG, Caruso R, Mitchell AJ, Meggiolaro E, Grassi L, 2015. Depression in HIV Infected Patients: A Review. Curr. Psychiatry Rep 17, 1–11. 10.1007/s11920-014-0530-4 [DOI] [PubMed] [Google Scholar]
  36. Nguyen DB, Do NT, Shiraishi RW, Le YN, Tran QH, Huu Nguyen H, Medland N, Nguyen LT, Struminger BB, 2013. Outcomes of Antiretroviral Therapy in Vietnam: Results from a National Evaluation. PLoS One 8 10.1371/journal.pone.0055750 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Patel P, Borkowf CB, Brooks JT, Lasry A, Lansky A, Mermin J, 2014. Estimating per-act HIV transmission risk. AIDS 28, 1509–1519. 10.1097/QAD.0000000000000298 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Pence BW, Miller WC, Gaynes BN, Eron JJ, 2007. Psychiatric illness and virologic response in patients initiating highly active antiretroviral therapy. J. Acquir. Immune Defic. Syndr 44, 159–166. 10.1097/QAI.0b013e31802c2f51 [DOI] [PubMed] [Google Scholar]
  39. Pence BW, Mills JC, Bengtson AM, Gaynes BN, Breger TL, Cook RL, Moore RD, Grelotti DJ, O’Cleirigh C, Mugavero MJ, 2018. Association of Increased Chronicity of Depression With HIV Appointment Attendance, Treatment Failure, and Mortality Among HIV-Infected Adults in the United States. JAMA psychiatry 75, 379–385. 10.1001/jamapsychiatry.2017.4726 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Quan VM, Minh N. Le, Ha TV, Ngoc NP, Vu PT, Celentano DD, Mo TT, Go VF, 2011. Mortality and HIV transmission among male Vietnamese injection drug users. Addiction 106, 583–9. 10.1111/j.1360-0443.2010.03175.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Radloff LS, 1977. A Self-Report Depression Scale for Research in the General Population. Appl. Psychol. Meas 1, 385–401. 10.1177/014662167700100306 [DOI] [Google Scholar]
  42. Robins J, 1989. The control of confounding by intermediate variables. Stat. Med 8, 679–701. [DOI] [PubMed] [Google Scholar]
  43. Robins JM, Hernán MA, Brumback B, 2000. Marginal structural models and causal inference in epidemiology. Epidemiology 11, 550–60. [DOI] [PubMed] [Google Scholar]
  44. Rubin D, 1987. Multiple imputation for nonresponse in surveys. Wiley, New York. [Google Scholar]
  45. Socialist Republic of Viet Nam, 2014. Vietnam AIDS response progress report 2014, following up the 2011 political declaration on HIV/AIDS, reporting period: January 2013-December 2013. Hanoi, Vietnam. [Google Scholar]
  46. Team RC, 2017. R: A language and environment for statistical computing. [Google Scholar]
  47. Thai Nguyen Provincial AIDS Center, D. of S.E.C. and P., 2007. Quarterly Report of the Statistics Office of Thai Nguyen Province, Thai Nguyen Provincial AIDS Center and the Division of Social Evils Control and Prevention, Department of Labor, Invalids and Social Affairs of Thai Nguyen, Vietnam. [Google Scholar]
  48. Thai TT, Jones MK, Harris LM, Heard RC, 2016. Screening value of the Center for epidemiologic studies - depression scale among people living with HIV/AIDS in Ho Chi Minh City, Vietnam: a validation study. BMC Psychiatry 16, 145 10.1186/s12888-016-0860-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Thomas M, Sahu D, Raj Y, Pandey A, 2014. A Probability Model for Estimating the Force of Transmission of HIV Infection and Its Application. Am. J. Math. Stat 4, 171–177. [Google Scholar]
  50. Todd JV, Cole SR, Pence BW, Lesko CR, Bacchetti P, Cohen MH, Feaster DJ, Gange S, Griswold ME, Mack W, Rubtsova A, Wang C, Weedon J, Anastos K, Adimora AA, 2017. Effects of Antiretroviral Therapy and Depressive Symptoms on All-Cause Mortality Among HIV-Infected Women. Am. J. Epidemiol 185, 869–878. 10.1093/aje/kww192 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Treisman G, Angelino A, 2007. Interrelation between psychiatric disorders and the prevention and treatment of HIV infection. Clin. Infect. Dis 45 Suppl 4, S313–S317. 10.1086/522556 [DOI] [PubMed] [Google Scholar]
  52. van Buuren S, Groothuis-Oudshoorn K, 2011. mice: Multivariate Imputation by Chained Equations in R. J. Stat. Softw 45, 1–67. [Google Scholar]
  53. Vuong DA, Van Ginneken E, Morris J, Ha ST, Busse R, 2011. Mental health in Vietnam: Burden of disease and availability of services. Asian J. Psychiatr 4, 65–70. 10.1016/j.ajp.2011.01.005 [DOI] [PubMed] [Google Scholar]
  54. Zelaya CE, Le Minh N, Lau B, Latkin CA, Viet Ha T, Minh Quan V, Mo TT, Sripaipan T, Davis WW, Celentano DD, Frangakis C, Go VF, 2016. The Effect of a Multi-Level Intervention on the Initiation of Antiretroviral Therapy (ART) among HIV-Infected Men Who Inject Drugs and Were Diagnosed Late in Thai Nguyen, Vietnam. PLoS One 11, e0161718 10.1371/journal.pone.0161718 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1655256-supplement-1.docx (987.8KB, docx)

RESOURCES