Abstract
Objective
The prevalence of clinically significant depressive symptoms is 3 times higher in people living with HIV (PLWH) than in the general population. While studies have shown that depression predicts worse course with HIV, few have investigated its relationship with mortality, and none have had a 17-year follow-up period and been conducted entirely during the time since the advent of protease inhibitors.
Methods
We followed a diverse sample of HIV positive people (n=177) in the mid-range of illness for a study on stress and coping. Participants were assessed every six months (for 12 years) via blood draw, questionnaires and interview. Depression was measured using the Beck Depression Inventory (BDI-I). The study began in March, 1997 and mortality was assessed in April, 2014.
Results
In the primary analysis depression, analyzed as a continuous variable, significantly predicted all-cause mortality (Hazard Ratio (HR) = 1.038, 95%CI = 1.008 – 1.068). With BDI scores dichotomized, the HR was 2.044 (95%CI = 1.176 - 3.550). Furthermore, this result was moderated by race and educational attainment such that depression only predicted worse survival for non-African Americans and those with a college education or above.
Conclusions
Depression measured during the first year and a half predicted worse survival in a diverse sample of PLWH followed for 17 years after accounting for initial disease status, antiretroviral medications and age. More research is needed to identify psychological risk factors for long-term outcomes in African Americans and those who are not college educated. Interventions targeting depression may improve well-being and potentially also long-term survival in individuals with HIV.
Keywords: HIV, Depression, Survival, African American, Education
Introduction
Depression is highly prevalent among those with medical illnesses, including HIV. Depending on the medical illness, depression can be anywhere from around 3 to 5 times more prevalent in the medically ill when compared to the general population (1). For example, in those with coronary heart disease, rates of depression range from 20-31%, in diabetes the rates range from 18-32% and in HIV the rates range from 20-37%. These rates compare to a 6.7% prevalence in the general population (1). These alarming prevalence rates not only have implications for quality of life for the medically ill but also for the course of their medical illness. Though we know that prognosis for medical illness can be adversely affected by the presence of depression, less has been done relating depression to survival in HIV-positive populations.
Although there are many studies showing depression predicts faster disease progression (2, 3, 4, 5), there have been fewer studies which have extended the analysis to examine depression predicting mortality. Although the role of depression in HIV was challenged by a few studies conducted prior to the advent of combination anti-retroviral therapy (cART) which showed no relationship between depression and survival (6, 7), the majority of studies, including many conducted after the widespread availability of cART have indeed shown a significant relationship (8-12). One well-known study examining the impact of depression on survival in people living with HIV was conducted by Ickovics et al. (9). Researchers followed 765 women for 5 years and found that, compared to women with little or no depression or intermittent depression, women with chronic depression were twice as likely to die from AIDS adjusting for baseline CD4/viral load, and antiretroviral medication use (9). The Women's Interagency HIV study (13) followed 1,716 women from 5 U.S. cities over 7.5 years. Over the course of the study, those women with chronic depressive symptoms were twice as likely to die compared to those with few or no depressive symptoms. While these studies focused exclusively on women living with HIV, Leserman et al. (10) followed 490 men and women with HIV for 41 months. Researchers found that each standard deviation increase in depressive symptoms increased the risk of AIDS mortality by 49% adjusting for demographics, CD4, viral load, and antiretroviral medication use.
While much of the previous work done in this area after the advent of cART has focused on women, the present study will employ a diverse sample of men and women. Additionally, while Leserman et al. (10) did use a diverse sample, the present study will extend the findings to 17 years, which is the longest follow-up period of any study we know of examining the impact of depression on survival in HIV. In addition, no studies of PLWH have analyzed depression and mortality by race. Of note, the present study was also conducted entirely during the era of cART for HIV.
Method
Participants
Participants were recruited through specialty clinics, physician offices, hospitals and service organizations. Inclusion criteria included HIV-positive individuals over the age of 18 in the mid-range of illness (defined as CD4 cells between 150 and 500). Exclusion criteria included lifetime AIDS-defining (Category C) symptom, lifetime CD4 below 75, presence of another life threatening illness (e.g., cancer), or active psychosis, suicidality, dementia, current alcohol or drug dependence or current I.V. drug use as assessed using the Structured Clinical Interview for DSM-III-R (14). See (15) for details.
Design
The current study employed a prospective longitudinal design in which participants were assessed every 6 months for a period of 12 years (see 5, 15). The study lasted from 1997 to 2010 with an accrual period of 2.5 years.
Procedure
At baseline, participants completed questionnaires including a Beck Depression Inventory (BDI; 16) as well as an interview on stress and coping. Participants also completed a blood draw in order to assess for CD4 and viral load. These assessments were repeated every six months. All study procedures were approved by the University of Miami IRB and subjects provided informed consent and were compensated $50 for each assessment.
Measures
Demographics
Participants completed a demographic questionnaire assessing for age, sex, race, and educational attainment. Educational attainment was used as a proxy for socioeconomic status in lieu of income as income is likely to be confounded with health as the HIV infection progresses. The educational attainment variable contained five categories: below high school, high school diploma/GED, some college, college graduate, and masters/professional degree.
Medical Status and Antiretroviral Medication
CD4 cell counts (CD3+CD4+) were assessed via whole-blood 4-color direct immunofluorescence using a coulter XL-MCL flow cytometer. The percentage values given by the flow cytometer were converted into whole numbers by applying the percentage against a total lymphocyte count determined by a MaxM electronic hematology analyzer. Viral load was measured by using the Cobas Amplicor HIV-1 Monitor RT/PCR assay sensitive to 400 copies/mL. Participants also self-reported whether they were prescribed antiretroviral medications. Participants were classified in one of three groups: no medication, one antiretroviral without protease inhibitors, more than one antiretroviral without protease inhibitors or combination therapy with protease inhibitors.
Survival was assessed by using the Death Master File (DMF; 17) through the Social Security Administration (SSA) which reflects deaths reported to the SSA. The DMF includes deaths from 1935 to the present and is updated on a weekly basis. In order to find any deaths that may not have reported to SSA, obituaries were searched on the internet using participant first and last names and birthdays. This analysis includes deaths that occurred before May 1, 2014.
Depression
The current study used the BDI-I (16) to measure depressive symptoms. Prior research has shown that the somatic symptoms on the BDI might be confounded with symptoms of HIV or side effects of antiretroviral medication (18), thus, all analyses were done with both the full BDI and the cognitive/affective subscale. Additionally, our measure was an average measure of depression over the first four time points (1.5 years) in order to have a representative, somewhat chronic, and reliable measure using the least number of timepoints. The earliest timepoints were chosen in order to have a true prediction with the largest sample size with the most complete data. Four timepoints were necessary to achieve good reliability: Cronbach's α = .841 with four timepoints, α = .745 for three timepoints, and α = .672 for only two timepoints. The use of four timepoints is also consistent with our previous research (5). The depression variable was dichotomized using the suggested clinical cutoff of 10 (16). Those with an average score of greater than or equal to 10 over the first four timepoints were considered as higher in depression, and those with an average less than 10 were considered low in depression.
Statistical Analysis
SPSS 22® was used to perform descriptive and survival analyses. Specifically, for the primary analysis, Cox proportional hazards regression was used to examine whether depression predicted survival in our sample. Baseline CD4 and viral load, age, and use of antiretroviral medications were entered in block 1 as covariates. Our average depression variable (described earlier) was entered in block 2 as a continuous variable to allow us to derive a hazard ratio (HR) for the effect of depression on survival, adjusting for a priori covariates in block 1. Past research with this sample (5, 15) has shown that education, race, and sex predict disease progression and thus may be hypothesized to affect survival. These variables were examined independently by computing separate Cox regression models. Additionally, separate Kaplan-Meier plots were used to illustrate differences in survival among various levels of the moderators. For the purposes of creating these graphs, depression, education and race were dichotomized. Depression was dichotomized at the suggested clinical BDI cutoff of 10 which coincidentally was close to the median (9.50) in our sample. Since there was no clinical cutoff for the affective subscale, post-hoc analyses maintained it as continuous. Education was dichotomized at those receiving a college degree or higher vs. those without a college degree as some evidence suggests that college graduates have higher incomes throughout the lifetime (19). Race was dichotomized into African Americans vs. non-African Americans. Interactions for the potential moderator variables were tested by centering variables and then testing the cross-product term between the moderator variable and depression above the biomedical covariates, the moderator variable (race or education), and depression. Finally, secondary analyses were done to determine the role of adherence. Adherence was measured using the interviewer-administered AIDS Clinical Trial Group adherence measure (20) using the proportion of missed doses over three pre-assessment days. To match the depression measure, an average measure was calculated over the first four time points.
Results
The sample was sociodemographically diverse with respect to age (M = 37.49, SD = 8.88), sex (70% male), sexual orientation (55% gay or bisexual), and ethnicity (36% African American, 30% Caucasian, 28% Hispanic, and 5% other). In terms of socioeconomic status, over three-quarters of the sample had educational attainment above high school (18.1% had some high school or less, 13.7% had a high school diploma, 40.7% had attended trade school or some college, 18.7% were college graduates, and 8.8% had a graduate degree). Most of the participants were either unemployed or on disability (18.6% had full-time employment, 15.3% had part-time employment, 15.3% were unemployed, and 42.2% were on disability). A majority of the sample (61.6%) had an income at or below $10,000 per year.
The sample sizes during the prediction period (i.e., the first four timepoints) were 177, 165, 148, and 132. By 2004, 15% of participants had died, by 2010, 32% of participants had died, and by 2014, 34% of participants had died. The sample size in the analysis for survival includes the full cohort of 177 because death records were used.
Although the sample had satisfactory access to treatment, treatment success was limited. Most of the participants (134/177, 76%) were prescribed antiretroviral medication at baseline and, at two years into the study, 90% were prescribed antiretroviral medication. Additionally, only 26% of participants reported less than 95% antiretroviral adherence. However, only 29% reached undetectable viral load at the two year follow-up (55% at the five year follow-up).
Before adding depression in the model, lower CD4/mL at baseline was significantly related to mortality (HR = 0.996, 95% CI = 0.994-0.999, p = .003). Higher Viral load was also associated with mortality (HR = 1.000, 95%CI = 1.000-1.000, p = .013). Older age was associated with mortality (HR = 1.046, 95%CI = 1.02-1.08, p = .001). Additionally, not being prescribed antiretroviral medications was associated with greater mortality (HR = 0.644, 95%CI = 0.428-0.968, p =.034).
Table 1 reports the Cox regression for the BDI scores predicting mortality adjusting for age, baseline CD4, baseline viral load, and antiretroviral medication. Those classified as higher on depression had significantly greater odds of mortality than those who were low on depression (HR = 2.044, 95%CI = 1.176 – 3.550, p = .011; Figure 1). Similar results were obtained for the cognitive/affective subscale (p = .042).
Table 1.
B | SE | Wald | df | p | Exp(B) | |
---|---|---|---|---|---|---|
Baseline CD4 | -.005 | .001 | 11.959 | 1 | .001 ** | .995 |
Baseline VL | .000 | .000 | 1.331 | 1 | .249 | 1.000 |
Age | .046 | .015 | 9.660 | 1 | .002 ** | 1.047 |
Antiretrovirala | -.629 | .223 | 7.989 | 1 | .005 ** | .533 |
Depressionb | .715 | .282 | 6.436 | 1 | .011 * | 2.044 |
p < .05,
p <.01
Participants were classified in one of three groups: no medication, one antiretroviral without protease inhibitors, more than one antiretroviral without protease inhibitors or combination therapy with protease inhibitors
Results remained significant when removing somatic symptoms ( p = .042)
When adding additional adjustments for sex, race, and education, depression as measured by the BDI (or the cognitive/affective scale alone) was no longer a significant predictor of mortality. Race and education were responsible for this loss of effect. Additional analyses revealed that these covariates were confounded with depression. While no significant sex differences in depression existed in our sample and sex was not related to mortality (HR = 1.248, 95%CI = 0.658 – 2.368, p = .50), we did find that African Americans were significantly more depressed than whites on the BDI, (r = .191, p = .011, n = 177) even when adjusting for somatic symptoms (r = .173, p = .021, n = 177). Additionally, those with lower education were more likely to be depressed than those with higher education (r = -.165, p = .029, n = 176). Again, similar results were obtained when removing somatic symptoms of depression (r = -.191, p = .022, n = 176). Furthermore, race and education have been shown to be significant predictors of disease progression in this sample (5, 15). Thus, the statistical assumption that the relationship between depression and survival would be the same at different levels of the covariates would be violated. Therefore, the analysis of the relationship between depression and survival was done separately for different levels of each predictor (i.e., African Americans vs. non-African Americans and higher education vs. lower education), and moderation effects were tested as well by entering the interactions of race by depression and education by depression. Table 2 reports descriptive statistics for our predictor of interest for the overall sample as well as for various subgroups of interest.
Table 2.
BDI (M, SD) | BDI Affective (M, SD) | ||||
---|---|---|---|---|---|
M | SD | M | SD | ||
Overall Sample (N = 177) | 10.048 | 7.336 | 5.300 | 4.890 | |
African Americans (n = 66) | 11.864 | 8.809 | 6.395 | 5.926 | |
Non-African Americans (n = 111) | 8.968 | 6.091 | 4.649 | 4.045 | |
College Graduates (n = 48) | 9.051 | 5.930 | 4.600 | 3.504 | |
Non-College Graduates (n = 128) | 10.412 | 7.813 | 5.539 | 5.318 |
Depression in African Americans vs. Non-African Americans
Figure S1.a (Supplemental Digital Content 1) shows the survival plots for non-African Americans with separate lines for participants with higher depression scores versus lower depression scores. Among non-African Americans, those with higher depression according to the BDI (n = 69) had significantly greater odds of mortality at 17 years relative to those with low depression (n = 42; HR = 3.386, 95%CI = 1.396 -8.210, p = .007). Results were marginally significant when restricting depression to the BDI affective subscale (p =.052). Figure S1.b (Supplemental Digital Content 1) shows survival plots for African Americans separated by depression status. Among African Americans there was no significant difference in odds of mortality between those with higher depression (n = 31) and those with low depression (n = 35; HR = 1.009, 95%CI = .973-1.046, p = .62). Results were similar for higher depression versus low depression in African Americans when removing somatic symptoms (HR (p = .15).
Thus, the Kaplan-Meier plots showed that depression predicted survival in non-African Americans but not in African Americans. The test of the moderation effect of race (the interaction of race × depression) was significant for the BDI (Wald = 5.911, p = .015) but was non-significant for the affective subscale (Wald = 1.571, p = .21), although, results did show a significant race by affective depression difference when analyzed separately by subgroups above.
Depression in College Graduates vs. Non-College-Graduates
Figure S2.a (Supplemental Digital Content 1) shows the survival plots for those with higher education with separate lines for participants with higher depression scores versus those with low depression scores. Among those with higher education, participants with higher depression scores (n = 18) had significantly greater odds of mortality than those with low depression scores (n = 30; HR = 32.732, 95%CI = 4.059 - 263.916, p = .001). Similar results were obtained when removing somatic symptoms among participants with higher education (p = .002). In contrast, among those participants with lower education (i.e. below a college graduate; Figure S2.b, Supplemental Digital Content 1), there was no significant difference in odds of mortality between those with higher depression (n = 54) and those with low depression (n = 74; HR = 1.417, 95%CI = .757-2.651, p = .276). Similar results were obtained when removing somatic symptoms among participants with lower education (p = .20).
Thus, as pictured in the Kaplan-Meier plots, the results showed that depression predicted survival in those with higher education, but not in those with lower education. The test of the interaction of education by depression was significant for the BDI (Wald = 6.409, p = .011), as well as for the affective subscale (Wald = 5.181, p = .023).
Post-hoc analyses were run to determine whether moderated effects of depression on survival for African Americans and those with lower education could be attributed to differences in antiretroviral medications prescribed. First, independent sample t-tests were run to determine if antiretroviral medications prescribed differed as a function of race (African American vs. Non-African American) or education (higher vs. lower). No differences in antiretroviral medications prescribed were found between African Americans and Non-African Americans, t(174) = 1.36, p = .18, d = 0.206 or between college graduates and non-college graduates, t(174) = -.019, p = .99, d = -0.003. It should also be noted that restriction of range with antiretroviral medications prescribed did not affect these results as Levene's Test for Equality of Variances was non-significant. In the second set of post-hoc analyses, the Cox regression models for African Americans and non-college graduates were rerun without adjusting for antiretroviral medications prescribed to determine if depression would still be significant. If depression became significant in either of these models, this would suggest a potential mediational role of antiretroviral medications prescribed. However, antiretroviral medications prescribed did not explain differences in the relationship between depression and survival for African Americans versus non-African Americans nor higher education (i.e., college graduates) versus lower education (i.e., non-college graduates). Depression remained non-significant for both African Americans (Wald = 0.24, p = .63) and those with lower education (Wald = 0.60, p = .44).
Additional analyses were run in order to determine whether the moderated effects by race or education could be due to adherence. Depression continued to predict survival in both the higher education group and the non-African American group even after adjusting for adherence (Wald = 9.722, p = .002; Wald = 5.258, p = .022) or when utilizing the affective subscale alone after adjusting for adherence (Wald = 8.997, p = .003; Wald = 4.421, p = .036). Similarly, depression remained a non-significant predictor of survival in both the lower education group and the African American group after adjusting for adherence (Wald = 1.258, p = .26; Wald = .004. p = .95), and when restricting the BDI to the affective scale (Wald = .859, p = .35; Wald = .002, p = .96). A formal test of the race by depression interaction after adjusting for adherence showed that the significance of the interaction did not change. The race by depression interaction was still significant after adjusting for adherence (Wald = 5.836, p = .016). The education by depression interaction was also still significant after adjusting for adherence (Wald = 4.335, p = .037).
Given that African American participants in our sample had lower educational attainment, post-hoc analyses were also run in order to determine whether education could explain the moderator effects of depression on survival by race. Education did not account for these effects. The race by depression interaction remained significant (Wald = 5.780, p = .016) even after education was added into the model.
Discussion
The current study demonstrates a 17-year impact of depression on survival in a diverse sample of people living with HIV in the mid-stage of illness. Results show that higher levels of depression predict over two times greater risk of mortality over 17 years above and beyond the effects of age, antiretrovirals, and initial disease status. This is consistent with much of the extant literature on depression and survival in PLWH. However, this is the first study in PLWH to demonstrate moderated effects of depression on survival; specifically, depression predicted survival only in non-African Americans and those with higher education.
The overall differences in survival combined with the moderated effects highlight the fact that the effect of depression on survival is not the same for all PLWH. Different groups of people may deal with unique sets of social/contextual factors influencing their survival above and beyond depression. A seropositive status is associated with a host of other chronic stressors and instability in other areas of life such as problems with relationships, healthcare, housing, finances, employment, and also crime (21). For example, some research has focused on the effect of food insecurity (i.e., inability or uncertainty in ability to be able to provide for one's basic nutrition needs in socially acceptable ways; 22) on treatment outcomes in PLWH. A significant portion of PLWH experience food insecurity and it has been linked to depression and drug use in PLWH which in turn affects treatment utilization, adherence, viral suppression, and survival (23-26). Similarly, other research has focused on homelessness and has demonstrated that often times the need to address basic subsistence concerns like food, shelter, and clothing can interfere with regular treatment visits and adherence to medication (27, 28). Stigma and discrimination have also received attention in the literature. Negative attitudes held by the public may be sensed or experienced by PLWH and they may consequently be internalized and accepted. This internalized stigma has been shown to affect treatment adherence which can then affect mortality (29). More pertinent to the moderation effects in this study, research shows that people from PLWH who also have membership in other marginalized groups (e.g., African Americans or those with low education) may be at higher risk for food insecurity (25, 30, 31), homelessness (32), and stigma/discrimination (33). There is a social context that comes with being seropositive as an African American or as a person with low education that is fundamentally different from someone without those identities (34). We may not have seen a survival difference as a function of depression for these groups because perhaps, for African Americans and those with lower educational attainment, other factors might be more potent than depression, suggesting that examining depression alone in these populations may not be sufficient as a sole predictor of mortality.
A widely recognized syndemic is the “SAVA” constellation of substance abuse, violence, and AIDS. It is often present in areas where poverty, homelessness, malnutrition, and lack of health care are prevalent (35, 36). Finally, in addition to depression and substance use, conditions that are co-morbid with HIV at rates much greater than the general population include PTSD, domestic violence, and a history of childhood sexual abuse (1). All of these must be considered as part of the picture in considering treatment beyond depression.
Depression may also interfere with health behaviors which may affect survival. For example, depression has been related to worsened adherence which is critical for control of viral load (37). Depression is also related to greater levels of substance use (38, 39). Substance use may not only negatively affect adherence to anti-retroviral medications but also has been shown to interfere with the efficacy of these drugs (40). Furthermore, some evidence suggests depression may also promote risky sexual behavior (41); however, the evidence is mixed (42).
In addition, it is well-established that negative affect can influence biological pathways important for survival in people living with HIV. Negative affect such as anxiety and depression has been shown to be related to higher norepinephrine reactivity in response to stressors and abnormal cortisol release patterns (43). In those living with HIV, norepinephrine has been thought to accelerate disease progression through facilitation of viral entry into cells (43). Additionally, stress and norepinephrine may negatively affect the efficacy of antiretroviral medications (44). Some evidence suggests that cortisol, a glucocorticoid, might predict faster disease progression (45). However, this evidence is mixed (43) as in-vitro virology studies show minimal impact of glucocorticoids on viral replication rates (46, 47).
Depression has also been shown to be a strong predictor of cognitive complaints among PLWH (48). Indeed, some work has suggested that PLWH who are cognitively-impaired show a higher prevalence of clinically significant depressive symptoms compared with those who are not (49). Cognitive complaints among PLWH have been shown to be related to poorer neuropsychological performance which is a central theme in the diagnosis of HIV-associated neurocognitive disorders (HAND). HAND is a critical public health concern among PLWH across the continuum of care as it has been shown to be an independent risk factor for mortality among PLWH (50). However, the temporal relationship between depression and HAND and how these two conditions may operate synergistically to influence mortality remains inconclusive.
Finally, stress and negative affect also have been shown to have deleterious effects on the immune system. Depression has long been known to be linked to poorer innate immunity; specifically, it is associated with decreased natural killer cell activity in PLWH (3, 51). Reducing depression is likely to increase natural killer cell activity in PLWH (52). In turn, NK cells have been protective of health in HIV (53) .These immunological challenges may also trigger dysregulation of the balanced response between pro-inflammatory cytokines, (e.g., IL-6), and anti-inflammatory cytokines. That is, major depression is associated with an increased pro-inflammatory response characterized by increases in cytokines (IL-6 and IL-1), C-reactive protein, and TNF-α (54, 55).
Limitations
Although our study predicted mortality in a diverse sample over a considerable length of time there are several limitations. First, we were only able to predict to all-cause mortality and not HIV specific mortality because death records do not give the cause of death, and even if they did, death due to AIDS would likely be reported inconsistently. Second, although our sample was diverse, it was not large enough to explore other subgroupings (such as Hispanics) with adequate power especially after dividing the sample into those higher and lower on depression (African Americans were the largest subgroup). Third, we were not able to explore the role of other possible mediators of the depression effect either because they weren't measured or because of the inclusion/exclusion criteria. For example, we excluded people who were drug dependent or who were I.V. drug users and therefore we were not able to determine the role of substance use without bias. On a related note, unmeasured variables (e.g., health status, medication changes, health behaviors during follow-up or HIV-associated neurocognitive disorder and other co-morbid diagnoses) could potentially be confounding variables. Fourth, in addition to substance use, other syndemic factors that are of prime relevance to minority groups beyond HIV including but not limited to food insecurity, discrimination/stigma are among the factors that should be explored further. Fifth, the generalizability is also limited as the sample was a volunteer sample, and it may be expected that people who volunteer for, and come to, a study will be different from those who do not participate. While we did not systematically collect information on why people took part in the study, several told us it was to help others with HIV, several said it was for the money, and several said it was because we were one of the few people who they could be open with about their HIV.
Future studies should further explore the mechanisms of action by which depression might predict survival. Although our data provided preliminary evidence that adherence was not the cause of the moderator effects, future studies should include an examination of syndemic factors noted above including but not limited to food insecurity and discrimination/stigma. Substance use, post-traumatic stress disorder, violence and other contextual issues are important to address. Studies could also examine health behaviors linked to depression that might also affect survival such as risky sexual behavior and substance use. Negative styles of coping such as denial or behavioral disengagement might be another potential mediator of the link between depression and survival. While it was too much to explore in this manuscript we will be analyzing stress and coping in a future manuscript with these data. Future studies might also examine biological mechanisms of action such as increased stress hormones including cortisol and norepinephrine as well as increased pro-inflammatory responses. Finally, with such a novel moderated finding, results should be taken as preliminary rather than conclusive until replicated by other researchers.
Clinical Implications
Results not only highlight the impact that depressive symptoms can have on HIV but they also suggest that this impact may not be uniform for everyone. Fortunately, there are numerous trials showing that we can effectively treat depression in HIV using psychological (56, 57) or pharmacological treatments (58, 59, 60, 61). Similarly, use of mental health services was related to lower mortality in women followed for 7.5 years (13). Treatment of depression with cognitive-behavioral therapy has resulted not only in decreases in depression but also improvements in viral load (54) and adherence (55). However, the evidence for the efficacy of treatments to improve depression, adherence, and virological outcomes remains inconclusive (1). Additionally, research shows that many other issues beyond depression may interfere with adherence (21, 62). Furthermore, those who are homeless or marginally housed are often excluded from these trials due to co-occurring mental health /substance use which reduces heavily the knowledge base on effective interventions for this population (63). Thus, more research is needed on the ability of depression treatments to improve adherence and virological outcomes and further examination of more potent factors important for survival particularly in marginalized populations.
Conclusion
Depression predicts greater mortality in persons living with HIV by a factor of slightly greater than 2. This both confirms and extends previous findings to a longer period of time using a more diverse sample, but introduces a caveat. Depression did not predict survival in African Americans or those with low education. The mechanism of action for these effects remains unknown and should be explored further. Contextual syndemic factors including, but not limited to food insecurity, discrimination/stigma are ripe areas for further exploration. Clinically, efficacious pharmacological and psychological treatments do exist for treating depression in HIV. However, for African Americans and those with low education, more will need to be done to affect survival.
Supplementary Material
Acknowledgments
The authors wish to thank the participants who gave their time and energy to help us understand coping with HIV. We also thank Annie George and Elizabeth Balbin, project directors, Aurelie Lucette for assisting with mortality records, Jonathan Atwood and Kelly Detz for data management, and other graduate students and staff who were part of the Positive Survivors office.
Sources of funding: This work was supported by the National Institute of Mental Health grants (R01MH53791 and R01MH066697) to Dr. Gail Ironson and National Institute of Mental Health grant (1F31MH107315) to Calvin Fitch.
Abbreviations
- HIV
Human Immunodeficiency Virus
- PLWH
People Living with HIV
- cART
Combination Antiretroviral Therapy
- AIDS
Acquired Immunodeficiency Syndrome
- IV
Intravenous
- SAVA
Substance Abuse, Violence, and AIDS
- HR
Hazard Ratio
Footnotes
Conflicts of Interest: None of the authors have any conflicts of interest to declare.
References
- 1.Ironson G, Fitch C. Mental Health, Medical Illness, and Treatment with a Focus on Depression and Anxiety. In: Friedman Howard S., editor. Encyclopedia of Mental Health. 2nd. Vol. 3. Waltham, MA: Academic Press; 2016. pp. 107–118. [Google Scholar]
- 2.Chida Y, Vedhara K. Adverse psychosocial factors predict poorer prognosis in HIV disease: a meta-analytic review of prospective investigations. Brain, Behavior, and Immunity. 2009;23(4):434–445. doi: 10.1016/j.bbi.2009.01.013. [DOI] [PubMed] [Google Scholar]
- 3.Evans DL, Ten Have TR, Douglas SD, Gettes DR, Morrison M, Chiappini MS, Brinker-Spence P, Job C, Mercer DE, Wang YL, Cruess D, Dube B, Dalen E, Brown T, Bauer R, Petitto JM. Association of depression with viral load, CD8 T lymphocytes, and natural killer cells in women with HIV infection. American Journal of Psychiatry. 2002 doi: 10.1176/appi.ajp.159.10.1752. [DOI] [PubMed] [Google Scholar]
- 4.Leserman J. Role of depression, stress, and trauma in HIV disease progression. Psychosomatic Medicine. 2008;70(5):539–545. doi: 10.1097/PSY.0b013e3181777a5f. [DOI] [PubMed] [Google Scholar]
- 5.Ironson G, O'Cleirigh C, Kumar M, Kaplan L, Balbin E, Kelsch CB, Fletcher MA, Schneiderman N. Psychosocial and neurohormonal predictors of HIV disease progression (CD4 cells and viral load): A 4 year prospective study. AIDS and Behavior. 2014:1–10. doi: 10.1007/s10461-014-0877-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lyketsos CG, Hoover DR, Guccione M, Senterfitt W, Dew MA, Wesch J, VanRaden MJ, Treisman GJ, Morgenstern H, Saah AJ, Palenicek J, Armenian H, Farzadegan H, Graham N, Margolick J, McArthur J, Phair JP, Chmiel JS, Cohen B, O'Gorman M, Variakojis D, Wesch J, Wolinsky SM, Detels R, Visscher BR, Chen ISY, Dudley J, Fahey J, Giorgi JV, Lee M, Martinez-Mara O, Miller EN, Nishanian N, Taylor J, Zack J, Rinaldor CR, Jr, Kingsley L, Becker JT, Gupta P, Ho M, Munoz A, Jacobson LP, Beaty T, Galai N, Epstein L, Guccione M, Hoover DR, Meinert C, Nelson K, Piantadosi S, Su S, Schrager L, Vermund SH, Kaslow RA, Seminara D. Depressive symptoms as predictors of medical outcomes in HIV infection. JAMA. 1993;270(21):2563–2567. [PubMed] [Google Scholar]
- 7.Vedhara K, Schifitto G, McDermott M. Disease progression in HIV-positive women with moderate to severe immunosuppression: The role of depression. Behavioral Medicine. 1999;25(1):43–47. doi: 10.1080/08964289909596738. [DOI] [PubMed] [Google Scholar]
- 8.Antelman G, Kaaya S, Wei R, Mbwambo J, Msamanga GI, Fawzi WW, Fawzi MCS. Depressive symptoms increase risk of HIV disease progression and mortality among women in Tanzania. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2007;44(4):470–477. doi: 10.1097/QAI.0b013e31802f1318. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ickovics JR, Hamburger ME, Vlahov D, Schoenbaum EE, Schuman P, Boland RJ, Moore J, HIV Epidemiology Research Study Group Mortality, CD4 cell count decline, and depressive symptoms among HIV-seropositive women: longitudinal analysis from the HIV Epidemiology Research Study. JAMA. 2001;285(11):1466–1474. doi: 10.1001/jama.285.11.1466. [DOI] [PubMed] [Google Scholar]
- 10.Leserman J, Pence BW, Whetten K, Mugavero MJ, Thielman NM, Swartz MS, Stangl D. Relation of lifetime trauma and depressive symptoms to mortality in HIV. American Journal of Psychiatry. 2007;164(11):1707–1713. doi: 10.1176/appi.ajp.2007.06111775. [DOI] [PubMed] [Google Scholar]
- 11.Mayne TJ, Vittinghoff E, Chesney MA, Barrett DC, Coates TJ. Depressive affect and survival among gay and bisexual men infected with HIV. Archives of Internal Medicine. 1996;156(19):2233–2238. [PubMed] [Google Scholar]
- 12.Patterson TL, Shaw WS, Semple SJ, Cherner M, McCutchan JA, Atkinson JH, Grant I, Nannis E, HIV Neurobehavioral Research Center (HNRC) Group Relationship of psychosocial factors to HIV disease progression. Annals of Behavioral Medicine. 1996;18(1):30–39. doi: 10.1007/BF02903937. [DOI] [PubMed] [Google Scholar]
- 13.Cook JA, Grey D, Burke J, Cohen MH, Gurtman AC, Richardson JL, Wilson TE, Young MA, Hessol NA. Depressive symptoms and AIDS-related mortality among a multisite cohort of HIV-positive women. American Journal of Public Health. 2004;94(7):1133–1140. doi: 10.2105/ajph.94.7.1133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Spitzer RL, First MB, Gibbon M, Williams JB. Structured clinical interview for DSM-III-R. American Psychiatric Press; 1990. [DOI] [PubMed] [Google Scholar]
- 15.Ironson G, O'Cleirigh C, Fletcher MA, Laurenceau JP, Balbin E, Klimas N, Schneiderman N, Solomon G. Psychosocial factors predict CD4 and viral load change in men and women with human immunodeficiency virus in the era of highly active antiretroviral treatment. Psychosomatic Medicine. 2005;67(6):1013. doi: 10.1097/01.psy.0000188569.58998.c8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Archives of General Psychiatry. 1961;4:561–571. doi: 10.1001/archpsyc.1961.01710120031004. [DOI] [PubMed] [Google Scholar]
- 17.US, Social Security Death Index, 1935-Current. Provo, UT, USA: Ancestry.com Operations Inc; 2011. [Internet] Available from www.ancestry.com. [Google Scholar]
- 18.Kalichman SC, Sikkema KJ, Somlai A. Assessing persons with human immunodeficiency virus (HIV) infection using the Beck Depression Inventory: disease processes and other potential confounds. Journal of Personality Assessment. 1995;64(1):86–100. doi: 10.1207/s15327752jpa6401_5. [DOI] [PubMed] [Google Scholar]
- 19.Mitchell LL, Syed M. Does college matter for emerging adulthood? Comparing developmental trajectories of educational groups. Journal of Youth and Adolescence. 2015;44(11):2012–2027. doi: 10.1007/s10964-015-0330-0. [DOI] [PubMed] [Google Scholar]
- 20.Chesney MA, Ickovics JR, Chambers DB, Gifford AL, Neidig J, Zwickl B, Wu AW, Patient care committee & adherence working group of the outcomes committee of the adult AIDS clinical trials group (AACTG) Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: the AACTG adherence instruments. AIDS Care. 2000;12(3):255–266. doi: 10.1080/09540120050042891. [DOI] [PubMed] [Google Scholar]
- 21.Gurung RA, Taylor SE, Kemeny M, Myers H. “HIV is not my biggest problem”: The impact of HIV and chronic burden on depression in women at risk for AIDS. Journal of Social and Clinical Psychology. 2004;23(4):490–511. [Google Scholar]
- 22.Anderson SA. Core indicators of nutritional state for difficult-to-sample populations. The Journal of Nutrition. 1990;120(11):1557–1599. doi: 10.1093/jn/120.suppl_11.1555. [DOI] [PubMed] [Google Scholar]
- 23.Kalichman SC, Cherry C, Amaral C, White D, Kalichman MO, Pope H, Swetsze C, Jones M, Macy R. Health and treatment implications of food insufficiency among people living with HIV/AIDS, Atlanta, Georgia. Journal of Urban Health. 2010;87(4):631–641. doi: 10.1007/s11524-010-9446-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Normén L, Chan K, Braitstein P, Anema A, Bondy G, Montaner JS, Hogg RS. Food insecurity and hunger are prevalent among HIV-positive individuals in British Columbia, Canada. The Journal of Nutrition. 2005;135(4):820–825. doi: 10.1093/jn/135.4.820. [DOI] [PubMed] [Google Scholar]
- 25.Weiser SD, Young SL, Cohen CR, Kushel MB, Tsai AC, Tien PC, Hatcher AM, Frongillo EA, Bangsberg DR. Conceptual framework for understanding the bidirectional links between food insecurity and HIV/AIDS. The American Journal of Clinical Nutrition. 2011;94(6):1729S–1739S. doi: 10.3945/ajcn.111.012070. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Weiser SD, Fernandes KA, Brandson EK, Lima VD, Anema A, Bangsberg DR, Montaner JS, Hogg RS. The association between food insecurity and mortality among HIV-infected individuals on HAART. Journal of Acquired Immune Deficiency Syndromes. 2009;52(3):342. doi: 10.1097/QAI.0b013e3181b627c2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Riley ED, Moore K, Sorensen JL, Tulsky JP, Bangsberg DR, Neilands TB. Basic subsistence needs and overall health among human immunodeficiency virus-infected homeless and unstably housed women. American Journal of Epidemiology. 2011;174(5):515–522. doi: 10.1093/aje/kwr209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Young S, Wheeler AC, McCoy SI, Weiser SD. A review of the role of food insecurity in adherence to care and treatment among adult and pediatric populations living with HIV and AIDS. AIDS and Behavior. 2014;18(5):505–515. doi: 10.1007/s10461-013-0547-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Katz IT, Ryu AE, Onuegbu AG, Psaros C, Weiser SD, Bangsberg DR, Tsai AC. Impact of HIV-related stigma on treatment adherence: systematic review and meta-synthesis. Journal of the International AIDS Society. 2013;16(3) doi: 10.7448/IAS.16.3.18640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Kushel MB, Gupta R, Gee L, Haas JS. Housing instability and food insecurity as barriers to health care among low-income Americans. Journal of General Internal Medicine. 2006;21(1):71–77. doi: 10.1111/j.1525-1497.2005.00278.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Nord M, Andrews M, Carlson S. Household food security in the United States, 2004. USDA-ERS Economic Research Report. 2005;(11) [Google Scholar]
- 32.Folsom DP, Hawthorne W, Lindamer L, Gilmer T, Bailey A, Golshan S, Garcia P, Unutzer J, Hough R, Jeste DV. Prevalence and risk factors for homelessness and utilization of mental health services among 10,340 patients with serious mental illness in a large public mental health system. American Journal of Psychiatry. 2005;162(2):370–376. doi: 10.1176/appi.ajp.162.2.370. [DOI] [PubMed] [Google Scholar]
- 33.Rao D, Pryor JB, Gaddist BW, Mayer R. Stigma, secrecy, and discrimination: ethnic/racial differences in the concerns of people living with HIV/AIDS. AIDS and Behavior. 2008;12(2):265–271. doi: 10.1007/s10461-007-9268-x. [DOI] [PubMed] [Google Scholar]
- 34.Aral SO, Adimora AA, Fenton KA. Understanding and responding to disparities in HIV and other sexually transmitted infections in African Americans. The Lancet. 2008;372(9635):337–340. doi: 10.1016/S0140-6736(08)61118-6. [DOI] [PubMed] [Google Scholar]
- 35.Meyer JP, Springer SA, Altice FL. Substance abuse, violence, and HIV in women: a literature review of the syndemic. Journal of Women's Health. 2011;20(7):991–1006. doi: 10.1089/jwh.2010.2328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Sullivan KA, Messer LC, Quinlivan EB. Substance abuse, violence, and HIV/AIDS (SAVA) syndemic effects on viral suppression among HIV positive women of color. AIDS Patient Care and STDs. 2015;29(S1):S42–S48. doi: 10.1089/apc.2014.0278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Gonzalez JS, Batchelder AW, Psaros C, Safren SA. Depression and HIV/AIDS treatment nonadherence: a review and meta-analysis. Journal of Acquired Immune Deficiency Syndromes. 2011;58(2) doi: 10.1097/QAI.0b013e31822d490a. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Latkin CA, Mandell W. Depression as an antecedent of frequency of intravenous drug use in an urban, nontreatment sample. Substance Use & Misuse. 1993;28(14):1601–1612. doi: 10.3109/10826089309062202. [DOI] [PubMed] [Google Scholar]
- 39.Pilowsky DJ, Wu LT, Burchett B, Blazer DG, Ling W. Depressive symptoms, substance use, and HIV-related high-risk behaviors among opioid-dependent individuals: results from the Clinical Trials Network. Substance Use & Misuse. 2011;46(14):1716–1725. doi: 10.3109/10826084.2011.611960. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Hendershot CS, Stoner SA, Pantalone DW, Simoni JM. Alcohol use and antiretroviral adherence: review and meta-analysis. Journal of Acquired Immune Deficiency Syndromes. 2009;52(2):180. doi: 10.1097/QAI.0b013e3181b18b6e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Alvy LM, McKirnan DJ, Mansergh G, Koblin B, Colfax GN, Flores SA, Hudson S, Project MIX Study Group Depression is associated with sexual risk among men who have sex with men, but is mediated by cognitive escape and self-efficacy. AIDS and Behavior. 2011;15(6):1171–1179. doi: 10.1007/s10461-010-9678-z. [DOI] [PubMed] [Google Scholar]
- 42.Crepaz N, Marks G. Are negative affective states associated with HIV sexual risk behaviors? A meta-analytic review. Health Psychology. 2001;20(4):291. doi: 10.1037//0278-6133.20.4.291. [DOI] [PubMed] [Google Scholar]
- 43.Cole SW. Psychosocial influences on HIV-1 disease progression: Neural, endocrine, and virologic mechanisms. Psychosomatic Medicine. 2008;70(5):562–568. doi: 10.1097/PSY.0b013e3181773bbd. [DOI] [PubMed] [Google Scholar]
- 44.Ironson GH, Hayward HS. Do positive psychosocial factors predict disease progression in HIV-1? A review of the evidence. Psychosomatic Medicine. 2008;70(5):546. doi: 10.1097/PSY.0b013e318177216c. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Leserman J, Petitto JM, Gu H, Gaynes BN, Barroso J, Golden RN, Perkins DO, Evans DL. Progression to AIDS, a clinical AIDS condition and mortality: psychosocial and physiological predictors. Psychological Medicine. 2002;32(06):1059–1073. doi: 10.1017/s0033291702005949. [DOI] [PubMed] [Google Scholar]
- 46.Markham PD, Salahuddin SZ, Veren K, Orndorff S, Gallo RC. Hydrocortisone and some other hormones enhance the expression of HTLV-III. International Journal of Cancer. 1986;37(1):67–72. doi: 10.1002/ijc.2910370112. [DOI] [PubMed] [Google Scholar]
- 47.Kino T, Kopp JB, Chrousos GP. Glucocorticoids suppress human immunodeficiency virus type-1 long terminal repeat activity in a cell type-specific, glucocorticoid receptor-mediated fashion: direct protective effects at variance with clinical phenomenology. The Journal of Steroid Biochemistry and Molecular Biology. 2000;75(4):283–290. doi: 10.1016/s0960-0760(00)00187-4. [DOI] [PubMed] [Google Scholar]
- 48.Bassel C, Rourke SB, Halman MH, Smith ML. Working memory performance predicts subjective cognitive complaints in HIV infection. Neuropsychology. 2002;16(3):400. doi: 10.1037//0894-4105.16.3.400. [DOI] [PubMed] [Google Scholar]
- 49.Heaton RK, Franklin DR, Ellis RJ, McCutchan JA, Letendre SL, LeBlanc S, Corkran SH, Duarte NA, Clifford DB, Woods SP, Collier AC, Marra CM, Morgello, Rivera Mindt M, Taylor MJ, Marcotte TD, Hampton-Atkinson J, Wolfson T, Gelman BB, McArthur JC, Simpson DM, Abramson I, Gamst A, Fennema-Notestine C, Jernigan TL, Wong J, Grant I, CHARTER and HNRC groups HIV-associated neurocognitive disorders before and during the era of combination antiretroviral therapy: differences in rates, nature, and predictors. Journal of Neurovirology. 2011;17(1):3–16. doi: 10.1007/s13365-010-0006-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Ellis RJ, Deutsch R, Heaton RK, Marcotte TD, McCutchan JA, Nelson JA, Abramson I, Thal LJ, Hampton Atkinson J, Wallace MR, Grant I. Neurocognitive impairment is an independent risk factor for death in HIV infection. Archives of neurology. 1997;54(4):416–424. doi: 10.1001/archneur.1997.00550160054016. [DOI] [PubMed] [Google Scholar]
- 51.Cruess DG, Petitto JM, Leserman J, Douglas SD, Gettes DR, Ten Have TR, Evans DL. Depression and HIV infection: impact on immune function and disease progression. CNS Spectrums. 2003;8(01):52–58. doi: 10.1017/s1092852900023452. [DOI] [PubMed] [Google Scholar]
- 52.Cruess DG, Douglas SD, Petitto JM, Have TT, Gettes D, Dubé B, Cary M, Evans DL. Association of resolution of major depression with increased natural killer cell activity among HIV-seropositive women. American Journal of Psychiatry. 2005;162(11):2125–2130. doi: 10.1176/appi.ajp.162.11.2125. [DOI] [PubMed] [Google Scholar]
- 53.Ironson G, Balbin E, Solomon G, Fahey J, Klimas N, Schneiderman N, Fletcher MA. Relative preservation of natural killer cell cytotoxicity and number in healthy AIDS patients with low CD4 counts. AIDS. 2001;15:2065–73. doi: 10.1097/00002030-200111090-00001. [DOI] [PubMed] [Google Scholar]
- 54.Dinan TG. Inflammatory markers in depression. Current Opinion in Psychiatry. 2009;22(1):32–36. doi: 10.1097/YCO.0b013e328315a561. [DOI] [PubMed] [Google Scholar]
- 55.Howren MB, Lamkin DM, Suls J. Associations of depression with C-reactive protein, IL-1, and IL-6: a meta-analysis. Psychosomatic Medicine. 2009;71(2):171–186. doi: 10.1097/PSY.0b013e3181907c1b. [DOI] [PubMed] [Google Scholar]
- 56.Antoni MH, Carrico AW, Durán RE, Spitzer S, Penedo F, Ironson G, Fletcher MA, Klimas N, Schneiderman N. Randomized clinical trial of cognitive behavioral stress management on human immunodeficiency virus viral load in gay men treated with highly active antiretroviral therapy. Psychosomatic Medicine. 2006;68(1):143–151. doi: 10.1097/01.psy.0000195749.60049.63. [DOI] [PubMed] [Google Scholar]
- 57.Safren SA, O'Cleirigh C, Tan JY, Raminani SR, Reilly LC, Otto MW, Mayer KH. A randomized controlled trial of cognitive behavioral therapy for adherence and depression (CBT-AD) in HIV-infected individuals. Health Psychology. 2009;28(1):1. doi: 10.1037/a0012715. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Ferrando SJ, Freyberg Z. Treatment of depression in HIV positive individuals: a critical review. International Review of Psychiatry. 2008;20(1):61–71. doi: 10.1080/09540260701862060. [DOI] [PubMed] [Google Scholar]
- 59.Olatunji BO, Mimiaga MJ, O Cleirigh C, Safren SA. A review of treatment studies of depression in HIV. Topics in HIV Medicine. 2006;14(3):112. [PubMed] [Google Scholar]
- 60.Tsai AC, Karasic DH, Hammer GP, Charlebois ED, Ragland K, Moss AR, Sorensen JL, Dilley JW, Bangsberg DR. Directly observed antidepressant medication treatment and HIV outcomes among homeless and marginally housed HIV-positive adults: a randomized controlled trial. American Journal of Public Health. 2013;103(2):308–315. doi: 10.2105/AJPH.2011.300422. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Tsai AC, Weiser SD, Petersen ML, Ragland K, Kushel MB, Bangsberg DR. A marginal structural model to estimate the causal effect of antidepressant medication treatment on viral suppression among homeless and marginally housed persons with HIV. Archives of General Psychiatry. 2010;67(12):1282–1290. doi: 10.1001/archgenpsychiatry.2010.160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Mills EJ, Nachega JB, Bangsberg DR, Singh S, Rachlis B, Wu P, Wilson K, Buchan I, Gill CJ, Cooper C. Adherence to HAART: a systematic review of developed and developing nation patient-reported barriers and facilitators. PLoS Med. 2006;3(11):e438. doi: 10.1371/journal.pmed.0030438. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Posternak MA, Zimmerman M, Keitner GI, Miller IW. A reevaluation of the exclusion criteria used in antidepressant efficacy trials. American Journal of Psychiatry. 2002;159(2):191–200. doi: 10.1176/appi.ajp.159.2.191. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.