Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Sep 1.
Published in final edited form as: Health Psychol. 2015 Feb 2;34(9):951–959. doi: 10.1037/hea0000189

Latent class profiles of internalizing and externalizing psychosocial health indicators are differentially associated with sexual transmission risk: Findings from the CFAR Network of Integrated Clinical Systems (CNICS) cohort study of HIV-infected men engaged in primary care in the United States

Matthew J Mimiaga 1,2,3, Katie Biello 2,3, Sari L Reisner 3,4, Heidi M Crane 5, Johannes Wilson 1, Chris Grasso 3, Mari M Kitahata 5, Wm Christopher Mathews 6, Kenneth H Mayer 3,7, Steven A Safren 1,3
PMCID: PMC4522398  NIHMSID: NIHMS659440  PMID: 25642839

Abstract

Objective

To examine whether latent class indicators of negative affect and substance use emerged as distinct psychosocial risk profiles among HIV-infected men, and if these latent classes were associated with high-risk sexual behaviors that may transmit HIV.

Methods

Data were from HIV-infected men who reported having anal intercourse in the past six months and received routine clinical care at four U.S. sites in the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) cohort (n=1,210). Latent class membership was estimated using binary indicators for: anxiety, depression, alcohol and/or drug use during sex, and polydrug use. Generalized estimating equations modeled whether latent class membership was associated with HIV sexual transmission risk in the past six months.

Results

Three latent classes of psychosocial indicators emerged: (1) internalizing (15.3%) (high probability of anxiety and major depression); (2) externalizing (17.8%) (high probability of alcohol and/or drug use during sex and polydrug use); (3) low psychosocial distress (67.0%) (low probability of all psychosocial factors examined). Internalizing and externalizing latent class membership were associated with HIV sexual transmission risk, compared to low psychosocial class membership; externalizing class membership was also associated with higher sexual transmission risk compared to internalizing class membership.

Conclusions

Distinct patterns of psychosocial health characterize this sexually active HIV-infected male patient population and are strongly associated with HIV sexual transmission risk. Public health intervention efforts targeting HIV sexual risk transmission may benefit from considering symptom clusters that share internalizing or externalizing properties.

Keywords: HIV, sexual risk, internalizing, externalizing, men who have sex with men

HIV remains a pressing public health issue in the United States today. In 2010, the Centers for Disease Control and Prevention reported that there are over one million Americans currently living with HIV, with an estimated 56,300 incident infections occurring each year. Over the past five years, 75% or more of the people infected with HIV annually were men, and a majority of these infections occurred in men who have sex with men (MSM) (Prejean et al., 2011). HIV infection is associated with a number of co-occurring psychosocial factors, including substance use and negative affect (defined as a general dimension of subjective distress and negative or aversive mood states, including depression and anxiety as well as symptoms of depressive and anxious distress; Brown, Chorpita, & Barlow, 1998). These psychosocial factors can have adverse health consequences for people living with HIV, including suboptimal medication adherence and unprotected sex (DeLorenze, Satre, Quesenberry, Tsai, & Weisner, 2010; DeLorenze, Weisner, Tsai, Satre, & Quesenberry, 2011; Newville & Haller, 2010; Wagner et al., 2011).

Although at least half of new HIV sexual transmissions occur from people who are unaware of their HIV serostatus, research conducted over the past two decades indicates that some HIV-infected individuals may continue to engage in risky sex after learning they are infected, which places uninfected sexual partners at risk of HIV (CDC, 2012; Crepaz et al., 2009; Kalichman, Kelly, & Rompa, 1997; Kalichman, 2000; Kozal et al., 2004; Mansergh et al., 2010; Mayer & Mimiaga, 2011; Schiltz & Sandfort, 2000; Weinhardt et al., 2004). In a study of over 4,000 HIV-infected patients in clinical settings across 15 different cities in the U.S., 20% of heterosexual men and 40% of MSM reported engaging in unprotected anal or vaginal intercourse within the past six months (Morin et al., 2007). These data warrant further understanding of the factors that promote HIV sexual risk taking among HIV-infected men in order to inform future secondary prevention interventions with this population. Research is also needed that explicitly addresses the biological transmissibility of HIV infection and takes into account detectable viral load alongside efficiency of sexual transmission risk behaviors (Dosekun & Fox, 2010; Jin et al., 2010).

At-risk alcohol and illicit drug use is common among HIV-infected men and is associated with unprotected sex (Bing et al., 2001; Morin et al., 2007; Shuper, Joharchi, Irving, & Rehm, 2009; Weinhardt & Carey, 2000). A nationally representative sample of 2,864 HIV-infected adults found nearly 20% of participants reported heavy alcohol use, and nearly 40% of participants reported using non-prescription drugs, such as cocaine, amphetamines, and heroin, over the past year (Bing et al., 2001). Substance use can lead to impaired decision-making and behavioral disinhibition, which can increase the risk of engaging in unprotected vaginal or anal sex (Berg, Michelson, & Safren, 2007; McGowan et al., 2004; Morin et al., 2007; Schiltz & Sandfort, 2000; Weinhardt et al., 2004). Substance use in the context of sexual behavior increases the risk of HIV transmission (Beckett, Burnam, Collins, Kanouse, & Beckman, 2003; Calsyn et al., 2010; Cheng et al., 2009). For example, a meta-analysis of 38 studies over the past decade showed that HIV-infected individuals who reported alcohol use in sexual contexts were almost twice as likely to engage in unprotected sex as HIV-infected individuals who did not drink alcohol (Shuper et al., 2009).

Studies with HIV-infected patients typically treat depression and anxiety diagnoses as separate or discrete constructs or conditions (Crepaz & Marks, 2001). However, in the general psychiatric literature, an “internalizing-externalizing liability model” has been used to understand and explain psychiatric co-morbidities, focusing on identifying unifying latent dimensions of liabilities to experience multiple internalizing (mood and anxiety) or externalizing (antisocial and substance use) disorders and symptoms (Brown et al., 1998; Eaton et al., 2011; Eaton, South, & Krueger, 2010; Krueger, 1999; Slade & Watson, 2006). The internalizing dimension (negative affect) consists of two factors: distress (major depression, dysthymia, generalized anxiety) and fear (panic disorder, social phobia, specific phobia) factors (Eaton et al., 2013a; Eaton et al., 2013b). Given that both depressive and anxious symptoms have been shown to load onto a single dimension of negative affect (Eaton et al., 2013b), examining general internalizing negative affect alongside externalizing substance use indicators may have implications for understanding sexual risk, including targets for future secondary HIV prevention and mental health intervention efforts.

Prior research on the association between negative affective states and HIV sexual risk behavior is complex (Crepaz & Marks, 2001; Kalichman & Weinhardt, 2001). A meta-analysis by Crepaz and Marks (2001) did not find general support for the consistent association between negative affect and condomless sex; however, the research synthesis covered diverse populations and was not restricted to HIV-infected men, which may have contributed to heterogeneity of effects. Indeed, Kalichman & Weinhardt’s (2001) comment on the meta-analysis pointed out the glaring methodological shortcomings of the affect-sexual risk behavior studies reviewed and concluded: “the jury on the association between negative affect and sexual risk remains out” (page 301). Methodological innovations and new approaches are needed in studies of affect-sexual risk behaviors.

Methodologically, a weakness of HIV research is that studies typically treat depression, anxiety, and substance use as separate independent variables, for example using regression modeling with depression and anxiety as separate statistical predictors of sexual risk behaviors. Such an analytic approach fails to account for the dependency and overlap of these psychosocial health issues. Other statistical procedures, such as latent class techniques (Masyn, Henderson, & Greenbaum, 2010; McCutcheon, 1987), are capable of modeling the dependency among psychosocial conditions and offer an important methodological step forward for empirically considering the inter-relationships between mental health factors and their joint influence on sexual risk behaviors among HIV-infected men. This approach is also consistent with National Institute of Mental Health (NIMH) statements about Research Domain Criteria (RDoC) regarding the use of dimensions to describe mental health problems, rather than static classifications (Insel et al., 2010; Cuthbert & Insel, 2010; Cuthbert & Insel, 2013; Cuthbert & Kozak, 2013).

We sought to evaluate, in a large cohort study of HIV-infected men in routine clinical care, whether negative affect and substance use indicators were distinct latent classes of psychosocial risks, or whether unifying latent dimensions of liabilities emerged. We further examined if these latent classes differentially were associated with sexual behaviors defined by distinct levels of biological risk of HIV transmission.

Methods

Study setting

This observational study examined 1,210 HIV-infected men who reported anal intercourse in the past six months, and were receiving primary care as part of the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) cohort. CNICS is a longitudinal observational study of HIV-infected patients enrolled in care at 8 clinical sites from January 1995 to the present (Kitahata et al., 2008). Patients were approached at their routine primary care visit to participate in CNICS. No reimbursement was offered.

Study Subjects

All HIV-infected men eighteen years of age or older who completed a clinical assessment of patient reported outcomes (PROs) as part of a routine clinical care visit between January 2007 and April 2010, and who reported engaging in anal intercourse in the past six months were included in this analysis. The clinical assessment was integrated into clinical care between 2005 and 2008 at four participating CNICS sites: Fenway Health in Boston (Fenway), the University of Alabama at Birmingham (UAB) 1917 HIV/AIDS Clinic, the University of Washington (UW) Harborview Medical Center HIV Clinic in Seattle, and the University of California San Diego HIV Clinic (UCSD). Patients completed the assessment every 4–6 months on average. For those patients who completed the assessment multiple times, only their first clinical assessment was included in this analysis. Patients who were medically unstable, appeared intoxicated, had a cognitive impairment, or did not speak English or Spanish did not complete the assessment at that visit. The study was approved by Institutional Review Boards at each participating site.

Data Sources

The CNICS data repository captures longitudinal data on the CNICS cohort (Kitahata et al., 2008). It integrates comprehensive clinical data from outpatient and inpatient encounters including standardized HIV-related information collected at enrollment (initial clinic visit), demographic, clinical, medication, laboratory, and sociodemographic data obtained from each site’s electronic medical record and other institutional data sources. Medication data are entered into the electronic medical records by clinicians or prescription fill/refill data are uploaded directly from Pharmacy Systems and verified through medical record review.

The CNICS data repository also integrates PRO clinical assessment data. Patients used touch-screen tablets or touch-screen desktop computers to complete the PRO assessment including measures of alcohol or drug use during sex in the past six months, substance use in the last 3 months, sexual risk behaviors in the prior six months, including unprotected anal intercourse (Metraux, Metzger, & Culhane, 2004; Rosenberg et al., 2001), depressive symptoms (Patient Health Questionnaire [PHQ-9] from the PRIME-MD) (Kroenke, Spitzer, & Williams, 2001; Spitzer, Kroenke, & Williams, 1999), symptoms of anxiety (Spitzer et al., 1999), current use of HIV medications. Viral load is captured from laboratory data in patient electronic medical records. Assessments are described further below.

Analytic Approach

Psychosocial indicators

Using established cut-points, four binary psychosocial indicators were constructed: (1) anxiety (PHQ-5, a five item validated anxiety module of the Patent Health Questionnaire; PHQ-5 ≥ 3 indicates anxiety symptoms vs PHQ-5 < 3; Cronbach’s alpha in our sample =0.59) (Kroenke et al., 2010; Lowe et al., 2010); (2) depression (PHQ-9, a nine item validated depression module of the Patient Health Questionnaire; PHQ-9 ≥ ten indicates depressive symptoms or PHQ-9 < ten; Cronbach’s alpha in our sample=0.91) (Kroenke et al., 2001); (3) alcohol and/or drug use during intercourse, last six months; (4) polysubstance use during the past three months (self-reported use of two or more substances: marijuana, crack, amphetamines, opiates or injection drug use).

Sexual behavior, last six months

Participants were queried about their sexual behaviors in the last six months, including whether they had engaged in unprotected anal intercourse during this timeframe, how often they used condoms (using a likert scale ranging from always to never) and the number of sexual partners. We focused on anal intercourse because it is the most common means of HIV transmission in most of the developed world and has a high per-contact HIV transmission probability (Grulich & Zablotska, 2010; Dosekun & Fox, 2010). Three distinct binary outcome variables were operationalized with increasingly stringent definitions: (1) any unprotected anal intercourse (UAI) in last six months (vs. no UAI); (2) any UAI in last six months and having ≥ two anal intercourse partners during this timeframe (vs. no UAI and/or <two anal intercourse partners); (3) any UAI in last six months, having ≥ two anal intercourse partners during this timeframe, and having a detectable viral load (vs. no UAI and/or <two anal intercourse partners and/or undetectable viral load). We operationalized sexual transmission risk in three distinct ways for anal intercourse; these were not mutually exclusive and were not meant to be as we sought to determine whether psychosocial risk factors statistically predicted varying degrees of risk. This novel operationalization of sexual risk was implemented in order to contribute methodologically to the existing body of research on the topic of psychological health and HIV transmission by examining potentially distinct risk factors for varying degrees of sexual transmission risk.

Covariates

Geographical location (San Diego CA, Seattle WA, Birmingham AL, Boston MA) was included as a design covariate in all statistical analyses to adjust for clustering. Other demographic factors controlled for included age (continuous) and race/ethnicity (White, Hispanic/Latino, Black/African American, and Other Race/Ethnicity). Current HIV medication use and viral load (detectable vs. undetectable) were also included.

General Statistical Procedures

SAS® version 9.1.3 statistical software was used for analyses, where statistical significance was determined at the alpha <0.05 level (SAS Institute, Inc., 2003). There was minimal missing data (<2%) across all variables. As a result, for continuous covariate and psychosocial measures, mean imputation was used, and for categorical or binary variables, modal imputation was used to assign missing data to the most common category. Missing data (<2%) on sexual risk outcomes was not imputed, and therefore only those who responded to the sexual behaviors questions are included. Descriptive statistics were obtained for the outcomes and each variable included in analyses.

A latent class analysis (LCA) was implemented in SAS using a special add-on procedure (PROC LCA) developed by the Methodology Center at the Pennsylvania State University (PROC LCA & LTA, 2013). The goal of LCA analysis is to use a set of two or more observed variables to estimate a single “latent” variable that “explains” the associations between the observed variables and therefore represents the “true” source of the originally observed covariation between the variables (McCutcheon, 1987). The latent variable is defined by discrete, mutually exclusive classes of individuals based on their responses to the set of observed binary, ordered-category and Likert-scale, or nominal data variables (Collins & Lanza, 2010; Lanza, Collins, Lemmon, & Schafer, 2007). An important use of latent class analysis is to determine typologies or response profiles according to the set of observed variables. Accordingly, in this analysis, we estimate the response patterns or profiles of participants on the four psychosocial binary (e.g., yes/no) indicators simultaneously, given these are not independent. We utilized binary variables to capitalize on the clinical relevance of findings. For example, in the literature, established and validated internalizing distress cut-off scores are often used to assess clinically significant depression and anxiety in clinical settings (Kroenke, Spitzer, & Williams, 2001; Kroenke et al., 2010; Lowe et al., 2010). Two sets of parameters were estimated in LCA analyses: class membership probabilities (i.e., the proportion of the sample “assigned” to each class) and item-response probabilities conditional on class membership (i.e., the probability of answering yes vs. no for each observed variable given assignment to a class). We used goodness of fit statistics (i.e., G2 and AIC) to determine the number of classes that fit in the best solution. We then used the maximum-probability assignment rule to assign individuals to the latent class in which they had the highest posterior probability of membership. Finally, we named the classes based on how data empirically grouped in the LCA.

Due to the multi-site study design, all statistical procedures adjusted for clustering by geographic location. First, because the latent psychosocial measure had three classes, an unordered multinomial regression model was fit using PROC SURVEYLOGISTIC (glogit link function) with study site as a cluster variable to examine whether age, race/ethnicity, HIV medication use, and viral load were associated with latent class membership (Allison, 2005). Next, to model sexual risk behavior as a function of latent class membership, a series of multivariable generalized estimating equation (GEE) models were specified using PROC GENMOD (logit link function and independent variance-covariance structure), with study site as a design covariate in all models (Liang & Zeger, 1986; Zeger, Liang, & Albert, 1988). The models are as follows: Model 1: Regressed any unprotected anal intercourse (vs. no UAI) on latent class membership, adjusting for age, race/ethnicity, HIV medication use, and viral load. Model 2: Regressed any unprotected anal intercourse and ≥ two anal intercourse partners (vs. no UAI and/or <two anal intercourse partners) on latent class membership, adjusting for age, race/ethnicity, HIV medication use, and viral load. Model 3: Regressed unprotected anal intercourse, ≥ two anal intercourse partners, and detectable viral load (vs. no UAI and/or <two anal intercourse partners, and/or undetectable viral load) on age, race/ethnicity, and HIV medication use. Sensitivity analyses were conducted, adjusted all models for year of clinical visit (4 year data collection period, between 2007 and 2010). There were no meaningful differences in the results; therefore, final models did not adjust for year of data capture.

Results

Descriptive characteristics of HIV-infected men included in the data analytic sample at the time of their clinical assessment (N=1,210) are presented in Table 1. The mean age was 42; 60% were White/Caucasian, 19% Black/African American, 15% Latino/Hispanic, 6% other race/ethnicity. Overall, 30% reported anxiety, 19% reported depression, 49% used alcohol and/or drugs before sex in the last six months, and 16% reported polydrug use in the last three months. Sixty one percent reported unprotected anal intercourse and 28% reported unprotected anal intercourse in the setting of a detectable viral load (data not shown).

Table 1.

Characteristics of Sexually Active (via anal sex) HIV-Infected Male Patients (n=1,210).

Sociodemographic Characteristics
Age in Years [Mean (Standard Deviation)] 41.9 (9.4)
n (%)
Race/Ethnicity
 White/Caucasian 730 (60.3)
 Black/African American 235 (19.4)
 Hispanic/Latino 175 (14.5)
 OtherRace/Ethnicity 70 (5.8)
Geographic Location
 San Diego, CA 318 (26.3)
 Seattle, WA 304 (25.1)
 Birmingham, AL 375 (31.0)
 Boston, MA 213 (17.6)
Psychosocial Risk
Anxiety, Past 2 Weeks 367 (30.3)
Major Depression, Past 2 Weeks 231 (19.1)
Polydrug Use, Last 3 Monthsa 196 (16.2)
Marijuana 435 (36.0)
Crack 115 (9.5)
Amphetamines 190 (15.7)
Opiates 27 (2.2)
Injection Drug Use (IDU) 54 (4.5)
Alcohol and/or Drug use Before Having Sex, Last 6 Months 591 (48.8)
Sexual Risk Measures
Anal Intercourse, Last 6 Months
 Any Unprotected Anal Intercourse (UAI) 725 (60.5)
 ≥ 2Anal Intercourse Partners 609 (50.3)
HIV Medication Use and Viral Load
Currently Taking HIV Medications 923 (76.3)
Detectable Viral Load 545 (45.0)
HIV Transmission Sexual Risk Outcomesb
Any UAI 725 (60.5)
UAI + ≥ 2 Anal Intercourse Partners 436 (36.4)
UAI + ≥ 2 Anal Intercourse Partners + Detectable Viral Load 190 (15.9)
Open in a new tab
a

Polydrug use in the past 3 months was operationalized as self-reported use of two or more substances (marijuana, crack-cocaine, amphetamines, opiates, or injection drug use).

b

HIV transmission sexual risk outcomes are distinct measures of sexual transmission risk, and do not represent mutually exclusive categories.

Latent class analysis: three-class solution

We examined whether a set of psychosocial indicators resulted in distinct latent classes. Based on fit statistics (data not shown), a three-class solution was determined. Latent class analysis results are in Table 2. Three mutually exclusive latent classes emerged: (1) internalizing (15.3%) (high probability of anxiety and major depression); (2) externalizing (17.8%) (high probability of alcohol and/or drug use during sex and polysubstance use); (3) low psychosocial distress (67.0%) (low probability of all psychosocial factors examined). Figure 1 visually depicts the estimated item response probabilities by latent class membership. Note the dominant role that anxiety plays in the internalizing class and the high item-response probability of alcohol and/or substance use during sex in the externalizing class. Overall, covariates that significantly predicted latent class membership (internalizing and externalizing class vs. low psychosocial risk) were younger age and identifying as Latino (Table 3). However, in separate comparisons, identifying as Latino did not differentiate those in the low psychosocial risk class from the internalizing and externalizing classes.

Table 2.

Latent Class Analysis: 3-Class Solution (n=1,210)

Class 1 Internalizing Dimension Class 2 Externalizing Dimension Class 3 Low Psychosocial
Class membership probabilities: Gamma estimates (standard errors)
 Latent Class Prevalence 0.1528 (0.0520) 0.1776 (0.0827) 0.6695 (0.0642)
Item Response Probabilities: Rho estimates (standard errors)
Alcohol and/or Drug Use During Sex 0.4429 (0.1035) 1.0000 (0.0000) 0.3631 (0.0199)
Anxiety 1.000 (0.0000) 0.4930 (0.0889) 0.0940 (0.0739)
Major Depression 0.5573 (0.1442) 0.2855 (0.0823) 0.0822 (0.0146)
Polydrug Use 0.1255 (0.0708) 0.7258 (0.1568) 0.0207 (0.0048)
Open in a new tab

Notes. Classes were named as follows: (1) Class 1: Internalizing (high probability of having anxiety and major depression); (2) Class 2: Externalizing (high probability of alcohol and/or drug use during sex and polydrug use); (3) Class 3: Low psychosocial (low probability of the four psychosocial health indicators). Item response probabilities ≥ 0.50 are bolded for ease of interpretation.

Figure 1.

Figure 1

Open in a new tab

Latent class analysis: Item response probabilities by latent class membership (n=1,210).

Table 3.

Comparing Latent Class Membership by Sociodemographic (Age and Race/Ethnicity) and HIV Disease Characteristics (HIV Medication Use and Viral Load).

Internalizing vs. Low Psychosocial Risk Externalizing vs. Low Psychosocial Risk
OR (95% CI) p-value OR (95% CI) p-value Overall p-value
Covariates
Age in Years 0.99 (0.97, 1.00) 0.044 0.97 (0.97, 0.98) <0.0001 <0.0001
Race/Ethnicity (White is the Referent)
 Black/African American 0.43 (0.16, 1.15) 0.0915 0.57 (0.22, 1.45) 0.235 0.236
 Latino/Hispanic 0.89 (0.70, 1.14) 0.354 1.35 (0.65, 2.78) 0.422 0.0004
 Other Race/Ethnicity 1.29 (0.89, 1.87) 0.181 1.31 (0.40, 4.28) 0.656 0.404
HIV Medications
 Yes vs No 0.90 (0.40, 2.03) 0.798 0.72 (0.36, 1.42) 0.338 0.629
Viral Load
 Detectable vs Undetectable 1.10 (0.53, 2.32) 0.794 1.19 (0.64, 2.21) 0.578 0.803
Open in a new tab

Notes. Results shown are from an unordered multinomial logistic regression model with class membership as the outcome. Low psychosocial was the referent class. OR = Odds Ratio. 95% CI = 95% Confidence Interval.

Association between psychosocial latent class membership and sexual transmission risk behaviors

The majority of patients (60.5%) who engaged in anal intercourse in the last six months reported UAI. Multivariable GEE model findings examining whether latent class membership differentially predicted HIV risk-related sexual behaviors are in Table 4 for UAI (Models 1–3). Compared to membership in the low psychosocial distress latent class, internalizing and externalizing latent class membership were each associated with an increased odds of each measure of anal intercourse transmission risk. When externalizing was compared to internalizing latent class membership, membership in the externalizing dimension of psychosocial health was associated with an increased odds of each measure of anal intercourse transmission risk.

Table 4.

Modeling Unprotected Anal Intercourse Algorithms (n=1,210).

Model 1: Any UAIa Model 2: Any UAI + ≥2 Partners b Model 3: Any UAI + ≥2 Partners + Detectable Viral Load c
OR (95% CI) p-value OR (95% CI) p-value OR (95% CI) p-value
Latent Class Membership
Class 1 vs 3 (internalizing vs low psychosocial risk) 1.70 (1.12, 1.60) 0.014 1.54 (1.19, 2.01) 0.001 1.74 (1.27, 2.39) 0.001
Class 2 vs 3 (externalizing vs low psychosocial risk) 4.15 (2.91, 5.92) <0.0001 4.81 (3.19, 7.25) <0.0001 3.02 (2.53, 3.62) <0.0001
Class 2 vs 1 (externalizing vs internalizing) 2.44 (1.93, 3.08) <0.0001 3.11 (2.01, 4.81) <0.0001 1.73 (1.12, 2.68) 0.013
Covariates
Age in Years 1.00 (1.00, 1.01) 0.124 0.99 (0.99, 1.00) 0.005 0.99 (0.98, 1.00) 0.182
Race/Ethnicity (White is the Referent)
 Black/African American 0.51 (0.40, 0.65) <0.0001 0.56 (0.43, 0.72) <0.0001 0.84 (0.67, 1.06) 0.139
 Latino/Hispanic 0.71 (0.45, 1.13) 0.147 0.97 (0.69, 1.36) 0.839 0.90 (0.50, 1.63) 0.739
 Other Race/Ethnicity 1.07 (0.65, 1.78) 0.788 0.95 (0.77, 1.17) 0.621 0.65 (0.47, 0.92) 0.013
HIV Medications
 Yes vs No 0.63 (0.45, 0.87) 0.005 0.58 (0.45, 0.74) <0.0001 0.16 (0.07, 0.34) <0.0001
Viral Load
 Detectable vs Undetectable 0.91 (0.80, 1.04) 0.170 0.68 (0.56, 0.83) 0.0001 -- --
Open in a new tab

Notes. Results shown are from Multivariable Binomial Generalized Estimating Equation (GEE) Models, adjusted for age, race/ethnicity, HIV medications, and viral load). OR = Odds Ratio. 95% CI = 95% Confidence Interval.

a

Model 1: Modeling any UAI vs. No UAI

b

Model 2: Modeling any UAI + ≥ 2 Anal Intercourse Partners vs. No UAI and/or <2 Anal Intercourse Partners

c

Model 3: Modeling any UAI + ≥ 2 Anal Intercourse Partners + Detectable Viral Load vs. No UAI and/or <2 Anal Intercourse Partners and/or Undetectable Viral Load

Discussion

Findings from this study support the hypothesis that internalizing negative affect (anxiety and depression) and externalizing (substance use) dimensions are unique risk factors for sexual risk behaviors for sexually active HIV-infected men in care. It was found that membership in either the internalizing or externalizing latent classes, relative to low psychosocial distress class membership, was associated with sexual transmission risk behaviors. Specifically, internalizing class membership (i.e., high probability of depression and anxiety) was associated with 74% higher odds and externalizing class membership (i.e., high probability of substance use during sex and poly substance use) was associated with 202% higher odds of having any UAI, ≥two partners and detectable viral load, a potentially high-transmission risk. Consistent with an internalizing-externalizing liability model of psychopathology (Eaton et al., 2013b; Eaton et al., 2011; Eaton et al., 2010; Krueger, 1999; Slade & Watson, 2006), findings suggest unique psychosocial profiles are associated with specific sexual risk behaviors, especially for HIV transmission via anal intercourse. Future studies would benefit from implementing latent growth modeling approaches, such as latent class growth analysis or latent trajectory analyses, to examine longitudinal patterns of class membership prospectively over time, including temporal associations with sexual risk behaviors (Brown & Barlow, 2005).

Review articles suggest that substance use may synergistically interact with negative affect to increase the risk of unprotected intercourse (Kalichman, 2000; Mayer & Mimiaga, 2011; Sikkema et al., 2010; Stall et al., 2003). Given high prevalence of both substance use and negative affect among HIV-infected persons seen across studies (Bing et al., 2001), the current analysis provides a better understanding of the interrelationship among these variables and how they relate to sexual risk-taking and extends prior research methodologically. For example, in a daily diary study with 125 HIV-infected men and women, Barta, Tennen, and Kiene (2010) found that negative affect moderates the association between sexual risk and alcohol consumption prior to sex. However, this finding has limitations given the small sample size and the fact that negative affect was not assessed with widely validated clinical assessment instruments. Findings have been mixed to date, varying by the study design (e.g., cross-sectional, cohort, longitudinal), sampling frame (i.e., community-based convenience samples, clinic/patient populations), and assessment tools/instruments used to measure psychological health and wellbeing (e.g., screening instruments, diagnostic clinical interviews). The current study is the first we know of to look at these relationships among HIV-infected men engaged in HIV care with validated scales assessing psychological impairment, and using a latent class approach.

Given this study’s finding that externalizing class membership was associated with the highest probability of HIV transmission risk, results support regular screening for non-parenteral and parenteral drug use in primary care settings treating HIV-infected patients. Specifically, our results support US Public Health Service guidelines (HIVMA working group) that describe the need for comprehensive HIV-related care that incorporates screening for substance use and mental health, as well as referrals to tertiary services. Integrating mental health treatment and substance use interventions into routine HIV primary care and prevention efforts in order to reduce both negative affect and substance use, will likely effect reductions in HIV transmission risk behavior among this group.

Study limitations should be noted. First, there is always uncertainty associated with assigning latent class membership, unless the probability of latent class membership is one for an individual (see Lanza et al., 2007 for discussion). Thus, it is important to recognize that latent class findings are probabilistic and not deterministic. Second, these CNICS data were cross-sectional which did not allow for temporal ordering. Moreover, our measures were of sexual risk in the past six months, anxiety and depression in the last two weeks, substance use during sex in the last six months, and polysubstance use in the last three months. As a result, results should be interpreted as associations only. However, our data analytic strategy and use of latent class analysis did not utilize a causal framework – rather, we utilized an empirically-driven person-oriented approach (Collins & Lanza, 2010) to look at subtypes of individuals that exhibit similar patterns of psychosocial characteristics. Additionally, this was a secondary data analysis and we were limited in terms of the data collected. As a result, we could not include measure of frequency of intercourse, insertive vs. receptive anal intercourse, gender of partners, HIV serostatus of partners, and other potentially important sexual network-related factors. We used number of sexual partners as a means to estimate risk beyond condom use. Finally, findings are limited to sexually active HIV-infected men reporting anal intercourse in the last six months given the inclusion criteria for this analysis.

This study has implications for public health research, intervention, and practice. First, this application of latent class techniques to psychosocial comorbidities is consistent with NIMH’s approach to use dimensions to describe mental health problems, rather than static classifications (Insel et al., 2010; Cuthbert & Insel, 2010; Cuthbert & Insel, 2013; Cuthbert & Kozak, 2013). Future research using this approach is warranted in HIV research. Second, the co-occurrence of HIV transmission risk behaviors and presenting psychological symptoms, alongside substance use behaviors, could attenuate the public health benefits of “test, treat, and linkage to care” strategies (El-Sadr, Affrunti, Gamble, & Zerbe, 2010; Vermund et al., 2010), which are predicated on the assumption that if more HIV-infected people are tested and enrolled in care, treatment will decrease individual infectiousness and population-level transmission (known as “community viral load”) (Das et al., 2010; Granich, Gilks, Dye, De Cock, & Williams, 2009; Montaner et al., 2006). Findings suggest the need for increased emphasis on pre- and post-test counseling to accompany HIV testing, including referrals, assessment, and treatment for internalizing and externalizing problems. The current findings underscore the need to continue to develop and implement prevention programs that address the combination of substance use, mental health, and unprotected intercourse in order to decrease the transmission of HIV by patients enrolled in primary care to their sexual partners, as well as improve virologic outcomes for HIV-infected patients.

Acknowledgments

This work was supported in part by R34DA031028 (Mimiaga) and the CNICS grant R24AI067039 (Saag). Patient Reported Outcome (PRO) collection in CNICS is also supported in part by the PROMIS HIV grant U01AR057954 (Crane). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr. Safren is supported by K24 MH094214.

References

  1. Allison PD. Fixed Effects Regression Methods for Longitudinal Data: Using SAS. SAS Institute; 2005. [Google Scholar]
  2. Barta WD, Tennen H, Kiene SM. Alcohol-involved sexual risk behavior among heavy drinkers living with HIV/AIDS: Negative affect, self-efficacy, and sexual craving. Psychology of Addictive Behaviors. 2010;24:563–570. doi: 10.1037/a0021414. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Beckett M, Burnam A, Collins RL, Kanouse DE, Beckman R. Substance use and high-risk sex among people with HIV: a comparison across exposure groups. AIDS and Behavior. 2003;7:209–219. doi: 10.1023/A:1023906610710. [DOI] [PubMed] [Google Scholar]
  4. Berg CJ, Michelson SE, Safren SA. Behavioral aspects of HIV care: Adherence, depression, substance use, and HIV-transmission behaviors. Infectious Disease Clinics of North America. 2007;21:181–200. x. doi: 10.1016/j.idc.2007.01.005. [DOI] [PubMed] [Google Scholar]
  5. Bing EG, Burnam MA, Longshore D, Fleishman JA, Sherbourne CD, London AS, Shapiro M. Psychiatric disorders and drug use among human immunodeficiency virus-infected adults in the United States. Archives of General Psychiatry. 2001;58:721–728. doi: 10.1001/archpsyc.58.8.721. [DOI] [PubMed] [Google Scholar]
  6. Brown TA, Barlow DH. Dimensional versus categorical classification of mental disorders in the fifth edition of the Diagnostic and statistical manual of mental disorders and beyond: Comment on the special section. Journal of abnormal psychology. 2005;114(4):551–556. doi: 10.1037/0021-843X.114.4.551. [DOI] [PubMed] [Google Scholar]
  7. Brown TA, Chorpita BF, Barlow DH. Structural relationships among dimensions of the DSM-IV anxiety and mood disorders and dimensions of negative affect, positive affect, and autonomic arousal. Journal of Abnormal Psychology. 1998;107:179–192. doi: 10.1037/0021-843X.107.2.179. [DOI] [PubMed] [Google Scholar]
  8. Calsyn DA, Cousins SJ, Hatch-Maillette MA, Forcehimes A, Mandler R, Doyle SR, Woody G. Sex under the influence of drugs or alcohol: common for men in substance abuse treatment and associated with high-risk sexual behavior. The American Journal on Addictions. 2010;19:119–127. doi: 10.1111/j.1521-0391.2009.00022.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Centers for Disease Control and Prevention. HIV among gay and bisexual men. Centers for Disease Control and Prevention; 2012. Retrieved from http://www.cdc.gov/hiv/topics/msm/pdf/msm.pdf. [Google Scholar]
  10. Cheng WS, Garfein RS, Semple SJ, Strathdee SA, Zians JK, Patterson TL. Differences in sexual risk behaviors among male and female HIV-seronegative heterosexual methamphetamine users. The American Journal of Drug and Alcohol Abuse. 2009;35:295–300. doi: 10.1080/00952990902968585. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Collins LM, Lanza ST. Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. John Wiley & Sons; 2010. [Google Scholar]
  12. Crepaz N, Marks G. Are negative affective states associated with HIV sexual risk behaviors? A meta-analytic review. Health Psychol. 2001;20(4):291–299. doi: 10.1037//0278-6133.20.4.291. [DOI] [PubMed] [Google Scholar]
  13. Crepaz N, Marks G, Liau A, Mullins MM, Aupont LW, Marshall KJ, Wolitski RJ. Prevalence of unprotected anal intercourse among HIV-diagnosed MSM in the United States: A meta-analysis. AIDS. 2009;23:1617–1629. doi: 10.1097/QAD.0b013e32832effae. [DOI] [PubMed] [Google Scholar]
  14. Cuthbert B, Insel T. The data of diagnosis: new approaches to psychiatric classification. Psychiatry. 2010;73(4):311–314. doi: 10.1521/psyc.2010.73.4.311. [DOI] [PubMed] [Google Scholar]
  15. Cuthbert BN, Insel TR. Toward the future of psychiatric diagnosis: the seven pillars of RDoC. BMC Med. 2013;11:126. doi: 10.1186/1741-7015-11-126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Cuthbert BN, Kozak MJ. Constructing constructs for psychopathology: the NIMH research domain criteria. J Abnorm Psychol. 2013;122(3):928–937. doi: 10.1037/a0034028. [DOI] [PubMed] [Google Scholar]
  17. Das M, Chu PL, Santos GM, Scheer S, Vittinghoff E, McFarland W, Colfax GN. Decreases in community viral load are accompanied by reductions in new HIV infections in San Francisco. PloS One. 2010;5:e11068. doi: 10.1371/journal.pone.0011068. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. DeLorenze GN, Satre DD, Quesenberry CP, Tsai AL, Weisner CM. Mortality after diagnosis of psychiatric disorders and co-occurring substance use disorders among HIV-infected patients. AIDS Patient Care and STDs. 2010;24:705–712. doi: 10.1089/apc.2010.0139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. DeLorenze GN, Weisner C, Tsai AL, Satre DD, Quesenberry CP., Jr Excess mortality among HIV-infected patients diagnosed with substance use dependence or abuse receiving care in a fully integrated medical care program. Alcoholism, Clinical and Experimental Research. 2011;35:203–210. doi: 10.1111/j.1530-0277.2010.01335.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Dosekun O, Fox J. An overview of the relative risks of different sexual behaviours on HIV transmission. Curr Opin HIV AIDS. 2010;5(4):291–297. doi: 10.1097/COH.0b013e32833a88a3. [DOI] [PubMed] [Google Scholar]
  21. Eaton, Nicholas R, South SC, Krueger RF. The meaning of comorbidity among common mental disorders. In: Millon T, Krueger RF, Simonsen E, editors. Contemporary directions in psychopathology: Scientific foundations of the DSM-V and ICD-11. New York, NY US: Guilford Press; 2010. pp. 223–241. [Google Scholar]
  22. Eaton NR, Krueger RF, Keyes KM, Skodol AE, Markon KE, Grant BF, Hasin DS. Borderline personality disorder co-morbidity: relationship to the internalizing-externalizing structure of common mental disorders. Psychological Medicine. 2011;41:1041–1050. doi: 10.1017/S0033291710001662. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Eaton NR, et al. Ethnicity and psychiatric comorbidity in a national sample: evidence for latent comorbidity factor invariance and connections with disorder prevalence. Social psychiatry and psychiatric epidemiology. 2013a;48(5):701–710. doi: 10.1007/s00127-012-0595-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Eaton NR, et al. The structure and predictive validity of the internalizing disorders. Journal of abnormal psychology. 2013b;122(1):86. doi: 10.1037/a0029598. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. El-Sadr WM, Affrunti M, Gamble T, Zerbe A. Antiretroviral therapy: a promising HIV prevention strategy? Journal of Acquired Immune Deficiency Syndromes. 2010;55(Suppl 2):S116–121. doi: 10.1097/QAI.0b013e3181fbca6e. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Granich RM, Gilks CF, Dye C, De Cock KM, Williams BG. Universal voluntary HIV testing with immediate antiretroviral therapy as a strategy for elimination of HIV transmission: a mathematical model. Lancet. 2009;373:48–57. doi: 10.1016/S0140-6736(08)61697-9. [DOI] [PubMed] [Google Scholar]
  27. Grulich AE, Zablotska I. Commentary: probability of HIV transmission through anal intercourse. Int J Epidemiol. 2010;39(4):1064–1065. doi: 10.1093/ije/dyq101. [DOI] [PubMed] [Google Scholar]
  28. Insel T, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry. 2010;167(7):748–751. doi: 10.1176/appi.ajp.2010.09091379. [DOI] [PubMed] [Google Scholar]
  29. Jin F, et al. Per-contact probability of HIV transmission in homosexual men in Sydney in the era of HAART. AIDS. 2010;24(6):907–913. doi: 10.1097/QAD.0b013e3283372d90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kalichman SC, Kelly JA, Rompa D. Continued high-risk sex among HIV seropositive gay and bisexual men seeking HIV prevention services. Health Psychology. 1997;16:369–373. doi: 10.1037/0278-6133.16.4.369. [DOI] [PubMed] [Google Scholar]
  31. Kalichman Seth C. HIV transmission risk behaviors of men and women living with HIV-AIDS: Prevalence, predictors, and emerging clinical interventions. Clinical Psychology: Science and Practice. 2000;7:32–47. doi: 10.1093/clipsy.7.1.32. [DOI] [Google Scholar]
  32. Kalichman SC, Weinhardt L. Negative affect and sexual risk behavior: Comment on Crepaz and Marks (2001) Health Psychology. 2001;20(4):300–301. doi: 10.1037/0278-6133.20.4.300. [DOI] [PubMed] [Google Scholar]
  33. Kitahata MM, Rodriguez B, Haubrich R, Boswell S, Mathews WC, Lederman MM, Saag MS. Cohort profile: the Centers for AIDS Research Network of Integrated Clinical Systems. International Journal of Epidemiology. 2008;37:948–955. doi: 10.1093/ije/dym231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Kozal MJ, Amico KR, Chiarella J, Schreibman T, Cornman D, Fisher W, Friedland G. Antiretroviral resistance and high-risk transmission behavior among HIV-positive patients in clinical care. AIDS. 2004;18:2185–2189. doi: 10.1097/00002030-200411050-00011. [DOI] [PubMed] [Google Scholar]
  35. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9. Journal of General Internal Medicine. 2001;16:606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kroenke K, et al. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. Gen Hosp Psychiatry. 2010;32(4):345–359. doi: 10.1016/j.genhosppsych.2010.03.006. [DOI] [PubMed] [Google Scholar]
  37. Krueger RF. The structure of common mental disorders. Archives of General Psychiatry. 1999;56:921–926. doi: 10.1001/archpsyc.56.10.921. [DOI] [PubMed] [Google Scholar]
  38. Lanza ST, Collins LM, Lemmon DR, Schafer JL. PROC LCA: A SAS procedure for latent class analysis. Structural Equation Modeling: A Multidisciplinary Journal. 2007;14:671–694. doi: 10.1080/10705510701575602. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73:13–22. doi: 10.1093/biomet/73.1.13. [DOI] [Google Scholar]
  40. Lowe B, et al. A 4-item measure of depression and anxiety: validation and standardization of the Patient Health Questionnaire-4 (PHQ-4) in the general population. J Affect Disord. 2010;122(1–2):86–95. doi: 10.1016/j.jad.2009.06.019. [DOI] [PubMed] [Google Scholar]
  41. Mansergh G, McKirnan DJ, Flores SA, Hudson SM, Koblin BA, Purcell DW, Colfax GN. HIV-related attitudes and intentions for high-risk, substance-using men who have sex with men: Associations and clinical implications for HIV-positive and HIV-negative MSM. Journal of Cognitive Psychotherapy. 2010;24:281–293. doi: 10.1891/0889-8391.24.4.281. [DOI] [Google Scholar]
  42. Masyn KE, Henderson CE, Greenbaum PE. Exploring the latent structures of psychological constructs in social development using the dimensional–categorical spectrum. Social Development. 2010;19:470–493. doi: 10.1111/j.1467-9507.2009.00573.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Mayer KH, Mimiaga MJ. Past as prologue: the refractory and evolving HIV epidemic among men who have sex with men. Clinical Infectious Diseases. 2011;52:1371–1373. doi: 10.1093/cid/cir206. [DOI] [PubMed] [Google Scholar]
  44. McCutcheon AL. Quantitative applications in the social sciences series no. 64. Thousand Oaks (CA): Sage Publications; 1987. Latent class analysis. [Google Scholar]
  45. McGowan JP, Shah SS, Ganea CE, Blum S, Ernst JA, Irwin KL, Weidle PJ. Risk behavior for transmission of human immunodeficiency virus (HIV) among HIV-seropositive individuals in an urban setting. Clinical Infectious Diseases. 2004;38:122–127. doi: 10.1086/380128. [DOI] [PubMed] [Google Scholar]
  46. Metraux S, Metzger DS, Culhane DP. Homelessness and HIV risk behaviors among injection drug users. Journal of Urban Health. 2004;81:618–629. doi: 10.1093/jurban/jth145. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Montaner JSG, Hogg R, Wood E, Kerr T, Tyndall M, Levy AR, Harrigan PR. The case for expanding access to highly active antiretroviral therapy to curb the growth of the HIV epidemic. Lancet. 2006;368:531–536. doi: 10.1016/S0140-6736(06)69162-9. [DOI] [PubMed] [Google Scholar]
  48. Morin SF, Myers JJ, Shade SB, Koester K, Maiorana A, Rose CD. Predicting HIV transmission risk among HIV-infected patients seen in clinical settings. AIDS and Behavior. 2007;11:6–16. doi: 10.1007/s10461-007-9253-4. [DOI] [PubMed] [Google Scholar]
  49. Newville H, Haller DL. Psychopathology and transmission risk behaviors in patients with HIV/AIDS. AIDS Care. 2010;22:1259–1268. doi: 10.1080/09540121003615111. [DOI] [PubMed] [Google Scholar]
  50. Prejean J, Song R, Hernandez A, Ziebell R, Green T, Walker F, Hall HI. Estimated HIV incidence in the United States, 2006–2009. PLoS One. 2011;6 doi: 10.1371/journal.pone.0017502. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. PROC LCA & PROC LTA. Version 1.3.0. University Park: The Methodology Center, Penn State, PA; 2013. [Google Scholar]
  52. Rosenberg SD, Trumbetta SL, Mueser KT, Goodman LA, Osher FC, Vidaver RM, Metzger DS. Determinants of risk behavior for human immunodeficiency virus/acquired immunodeficiency syndrome in people with severe mental illness. Comprehensive Psychiatry. 2001;42:263–271. doi: 10.1053/comp.2001.24576. [DOI] [PubMed] [Google Scholar]
  53. SAS Institute, Inc. SAS® Version 9.1.3. Cary, NC: 2003. [Google Scholar]
  54. Schiltz MA, Sandfort TG. HIV-positive people, risk and sexual behaviour. Social Science & Medicine. 2000;50:1571–1588. doi: 10.1016/S0277-9536(99)00466-9. [DOI] [PubMed] [Google Scholar]
  55. Shuper PA, Joharchi N, Irving H, Rehm J. Alcohol as a correlate of unprotected sexual behavior among people living with HIV/AIDS: Review and meta-analysis. AIDS and Behavior. 2009;13:1021–1036. doi: 10.1007/s10461-009-9589-z. [DOI] [PubMed] [Google Scholar]
  56. Sikkema KJ, Watt MH, Drabkin AS, Meade CS, Hansen NB, Pence BW. Mental health treatment to reduce HIV transmission risk behavior: A positive prevention model. AIDS and Behavior. 2010;14:252–262. doi: 10.1007/s10461-009-9650-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Slade T, Watson D. The structure of common DSM-IV and ICD-10 mental disorders in the Australian general population. Psychological Medicine. 2006;36:1593–1600. doi: 10.1017/S0033291706008452. [DOI] [PubMed] [Google Scholar]
  58. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Journal of the American Medical Association. 1999;282:1737–1744. doi: 10.1001/jama.282.18.1737. [DOI] [PubMed] [Google Scholar]
  59. Stall R, Mills TC, Williamson J, Hart T, Greenwood G, Paul J, Catania JA. Association of co-occurring psychosocial health problems and increased vulnerability to HIV/AIDS among urban men who have sex with men. American Journal of Public Health. 2003;93:939–942. doi: 10.2105/AJPH.93.6.939. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Vermund SH, Hodder SL, Justman JE, Koblin BA, Mastro TD, Mayer KH, El-Sadr WM. Addressing research priorities for prevention of HIV infection in the United States. Clinical Infectious Diseases. 2010;50:S149–155. doi: 10.1086/651485. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Wagner GJ, Goggin K, Remien RH, Rosen MI, Simoni J, Bangsberg DR, Liu H. A closer look at depression and its relationship to HIV antiretroviral adherence. Annals of Behavioral Medicine. 2011;42:352–360. doi: 10.1007/s12160-011-9295-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Weinhardt LS, Carey MP. Does alcohol lead to sexual risk behavior? Findings from event-level research. Annual Review of Sex Research. 2000;11:125–157. [PMC free article] [PubMed] [Google Scholar]
  63. Weinhardt Lance S, Kelly JA, Brondino MJ, Rotheram-Borus MJ, Kirshenbaum SB, Chesney MA, Gore-Felton C. HIV transmission risk behavior among men and women living with HIV in 4 cities in the United States. Journal of Acquired Immune Deficiency Syndromes. 2004;36:1057–1066. doi: 10.1097/00126334-200408150-00009. [DOI] [PubMed] [Google Scholar]
  64. Zeger SL, Liang KY, Albert PS. Models for longitudinal data: a generalized estimating equation approach. Biometrics. 1988;44:1049–1060. doi: 10.2307/3315715. [DOI] [PubMed] [Google Scholar]

RESOURCES