Abstract
Sexual orientation stigma stems from discriminatory social contexts and may ultimately impact the behavioral health of stigmatized individuals through stress-related pathways. Sexual minority stigma is of particular concern in Europe given the diversity of social contexts on the continent and sexual minority men’s rapidly increasing risk of HIV infection, especially in Central and Eastern Europe, potentially rooted in stigma. This study assesses whether stigma in the ubiquitous social contexts surrounding sexual minority men (e.g., family, workplace, government) may place them at higher risk for HIV contraction across six countries. We utilized a large cross-sectional survey sample of HIV-negative sexual minority men (N = 2087; mean age = 31.6, SD = 9.7) from six European countries to test whether those who reported sexual orientation stigma also engaged in more HIV risk-related behaviors, including condomless sex with casual partners (in the absence of PrEP) and substance use before and during sex. Regression analyses were performed in Mplus. We found that a one standard deviation increase in reported sexual orientation stigma was significantly associated with the following during the last sexual encounter: a 19% increase in odds of sex under the influence of alcohol, 27% increase in odds of sex under the influence of cannabis, 49% increase in odds of sex under the influence of illicit drugs, an 11% increase in odds of condomless sex with casual partners in the past 6 months, and a 26% increase in odds of knowing where to receive an HIV test. Sexual minority men who reported perceiving greater sexual orientation-related stigma within their ubiquitous social contexts were significantly more likely to report sexual risk and alcohol and drug use during their last sexual encounter, yet reported more knowledge of preventive services. Contextual stigma might serve as a precursor to behavioral risks of HIV infection, generating maladaptive stress responses capable of being modified through individually-focused interventions. Structural interventions are also needed to ultimately reduce stigma at its source.
Keywords: Sexual minority men, HIV risk, Sexual orientation-related stigma, Drug use, Alcohol use
Introduction
Stigma toward sexual minorities varies widely across geographic regions, even on the same continent [1]. For instance, while Western European countries promote the equal treatment of sexual minority citizens through affirmative and non-discriminatory laws, policies, and national attitudes, many countries in Central and Eastern Europe lack equal legislative protections for sexual minorities and actively discriminate against them through legislation that promotes inequality and homophobic attitudes [1]. Globally, sexual orientation stigma has been shown to be associated with increased vulnerability to health risks [2, 3] including HIV transmission-risk behavior and substance use [4–8].
After political shifts in Central and Eastern Europe over the past two decades, gay, bisexual, and other men who have sex with men (referred hereafter as sexual minority men) in this region gained unprecedented freedoms of association (with preferred social, sexual and romantic partners) and travel. Consequently, Central and Eastern European sexual minority men who were previously insulated due to strict international travel prohibitions and censure against sexual minorities became newly exposed to the high HIV and substance use prevalence among sexual minority men living in Western Europe and other nearby regions. At the same time of this new exposure, sexual minority men living in Central and Eastern Europe have also been under-equipped with HIV prevention and transmission knowledge. In fact, Eastern Europe represents one of the highest-priority regions in the world for HIV prevention given that the HIV rate there has increased significantly in the past decade, with a particularly alarming increase in transmissions among sexual minority men [9]. For example, in one of the countries included in the current study, Romania, HIV prevalence among sexual minority was less than 10% in 2009, but close to 20% in 2014, based on best available evidence [10, 11].
While lesbian, gay, bisexual and transgender (LGBT) civil rights movements followed democratic reform eastward upon communism’s fall [12], with Central and Eastern European sexual minority men’s greater visibility came greater vulnerability in terms of increased legal and attitudinal stigma and discrimination [12–19] as a conservative response, primarily driven by religious dogma, to the increasingly overt diversity in sexual identities [20]. Stigma is often defined as an attribute or characteristic that is devalued in certain social contexts [21, 22]. But rather than residing within the stigmatized individuals themselves, stigma is ultimately propagated by social institutions [23]. For instance, social institutions such as workplaces, churches, families, communities, and government can perpetuate stigma toward sexual minorities, thereby thwarting their belongingness to these institutions and driving unhealthy coping reactions [24–27], including HIV-transmission risk behavior and substance use, which can co-occur with sexual behavior to generate HIV risk. Thus, identifying and intervening upon the source of stigma in these social contexts and institutions represents an important public health priority, especially in high-stigma locales [28, 29].
In this study, we examined the relationship between sexual minority stigma present in ubiquitous social contexts and HIV-transmission risk behaviors in a geographically diverse sample of European sexual minority men. Across Europe, sexual minority men represent the highest-risk group for HIV infection [9, 30, 31]. While various forms of stigma, including national laws and policies toward sexual minorities, and self-stigma (i.e., internalized homophobia), have been shown to be associated with European sexual minority men’s HIV risk [32–34], no research has examined HIV risk as a function of perceptions of social contextual stigma among geographically diverse European sexual minority men, especially those living in Central and Eastern Europe. We define social contextual stigma to consist of attitudes (or the degree of prejudice) towards sexual minorities (homosexuals or bisexuals) perceived across several life domains: work/school; parents; friends/acquaintances; political; social; and religious. Therefore, we sought to establish the association between social contextual stigma and HIV-transmission risk behavior among HIV-negative sexual minority men living in six European contexts [35, 36]. Europe offers an important geographic context in which to test these hypotheses given the wide variation in national contexts surrounding sexual minority men and the relatively high-quality data available across these contexts.
Method
Study Design
This manuscript is based on data collected through a multicenter biological and behavioral cross-sectional survey of sexual minority men implemented in the context of the SIALON (saliva in Ancient Greek) project, co-funded by the European Commission under the Public Health Programme 2003–2008 (GA 2007309) [37]. The survey was co-funded by the EU and implemented in six European cities: Barcelona (Spain), Bratislava (Slovakia), Bucharest (Romania), Ljubljana (Slovenia), Prague (Czech Republic), and Verona (Italy). Study sites were selected to provide comparative geographic contexts, including high-and low-stigma contexts.
The following eligibility criteria were applied: having had any kind of sex at least once with another man during the previous year; having signed the survey’s informed consent, agreeing to answer the study questionnaire and donate an oral fluid sample. Oral fluid data are not included in the current analyses. Exclusion criteria were adopted as follows: being younger than 18 years old; active injecting drugs; and having already participated in the study. Study bio-behavioral data collection procedures are described in detail elsewhere [38, 39].
Ethics
Prior to data collection, informed consent was obtained for each participant. To guarantee anonymity and confidentiality, a barcode was assigned to each enrolled participant to maintain their anonymity. The unique barcode was assigned to each participant when he agreed to take part in the study. The same barcode was used to link all the materials and documents pertaining to the subject (informed consent, questionnaire, biological sample, card to pick up the result). Requiring a card for participants to retrieve results (using the unique barcode) was explicitly requested by the ethics committee. This approach and the related procedure were approved by the ethics committee in each participating site. Study protocols were approved by each study site’s ethics committee.
Sampling
Sexual minority men were recruited using time-location sampling (TLS), a venue-based sampling method widely adopted for bio-behavioral surveys among sexual minority men [40]. In line with the main TLS principles, formative research including a gay-venue mapping exercise was conducted in each participating city. A list of venue-day-time (VDT) units was prepared according to the formative research results. A data collection calendar was calculated according to TLS principles, creating VDT units. VDT units were then categorized according to the median attendance on the basis of two categories: “high attendance” (15 sexual minority men) and “low attendance” (5 sexual minority men). A training was delivered to field workers at local LGBT associations, who were in charge of approaching potential participants in the specific VDT unit and distributing questionnaires.
On the basis of the type of venues (e.g., disco, bar, sauna, cruising setting, sex shop/party), VDT units were randomly selected within homogeneous clusters of venues in order to ensure their comprehensive inclusion in the first sampling stage and therefore improve the heterogeneity of the final sample of sexual minority men. Finally, data collectors were selected and trained to distribute anonymous pen-and-paper questionnaires, and an info-pack containing prevention materials and information. Participants competed the questionnaires in the venues where they were recruited, and oral fluid samples were also collected in these locations (via Oracle device), in a pre-identified private area within the venue. Oral fluid, rather than blood, was collected to ensure participants’ safety and increase acceptability among participants by preventing the need for venipuncture.
Questionnaire
A preliminary version of the self-administered pen-and-paper questionnaire was prepared with specific reference to the Global AIDS Response Progress Reporting (GARPR) guidelines [39]. A preliminary version of the questionnaire was then piloted among sexual minority men attending gay venues. Feedback regarding layout, wording, and comprehension was collected and included in the final version of the questionnaire. Translation procedures have been previously described elsewhere [37].
The self-administered questionnaire assessed a range of variables, of which we included the following in the current analyses: age, education, relationship status, behavioral practices (including sexual and substance use behaviors), types of sex partners (gender; steady and/or casual), and perceived stigma towards sexual minority men. Sexual orientation was assessed by asking “At present, how would you best define your sexual orientation?” with response options being “gay/homosexual,” “bisexual,” “heterosexual,” or “other.” We describe in detail the subset of variables we included in our analyses below.
Dependent Variables
We selected the following dependent variables because they have been shown in previous literature to have significant associations with HIV risk [41–46].
Sexual Behavior, Including HIV-Transmission Risk Behavior
We included two count variables indicating the number of (1) casual partners over the past 6 months and (2) instances of receptive and insertive sex (anal or vaginal) without a condom with casual partners reported over the past 6 months. Further, we included several dichotomous (yes/no) variables: (3) any HIV status-unknown casual partner at the last sexual encounter and (4) whether or not participants were under the influence of alcohol and/or recreational drugs (marijuana or illicit drugs such as cocaine, amphetamine, and/or ecstasy) prior to or during the last sexual encounter.
HIV Testing
We examined two factors related to HIV testing as dependent variables: (1) whether or not participants had an HIV test in the past 12 months (yes/no), and (2) whether they knew where to receive an HIV test (yes/no).
Predictor Variable
Sexual Orientation Stigma
Participants reported stigma toward sexual minorities in their daily contexts as a response to the following question [35, 36]: “In your experience, what is people’s attitude towards homosexuals or bisexuals in the following contexts: work/school; parents; friends/acquaintances; political; social; and religious.” Response options ranged from 1 = very negative to 5 = very positive, and scale reliability was 0.76. This variable was z-scored for analyses.
Analytic Strategy
Analyses were conducted using Mplus version 8.0. Given the nesting within countries, we initially intended to analyze the data using multilevel modeling. However, the number of level 2 units (i.e., countries) was only six, power to detect any level 2 effects as well as to examine random between-country variability was substantially reduced. In unconditional multilevel models, we found that there was not significant between-country (i.e., random) variability on the majority of the outcomes—subsequent models that attempted to include predictors failed to converge due to difficulties estimation of random variances with a limited number of level 2 units. The lack of between-country variability also suggested that a single-level model that did not account for nesting within country was statistically appropriate. As such, we took the most parsimonious (and more simplistic) approach that was capable of handling the level of complexity observed in these data.
Specifically, we analyzed the eight outcome variables within a series of simultaneous regressions based on the types of analyses we conducted for each of them. The following six outcomes were analyzed using logistic regression given their dichotomous nature: sex under the influence of alcohol, sex under the influence of cannabis, sex under the influence of illicit drugs, knowing where to receive an HIV test, receiving an HIV test in the past 12 months, and having had any HIV-status-unknown casual partners at the last sexual encounter. The following two outcomes were analyzed using Poisson regression given their count nature: number of casual partners and frequency of condomless sex with casual partners in the past 6 months. We utilized robust maximum likelihood (i.e., MLR) estimation, which is known to be the optimal method for analyses with missing data, with the requisite Montecarlo integration. By default, Mplus will include participants who have not reported an outcome variable of interest, but will not include participants who have not reported on predictor variables of interest. In order to include participants who were missing predictor variables within the models, we utilized the option to estimate the mean and variance of each predictor variable within the model. The model also included dummy-coded dichotomous indicators for whether participants were missing any of the demographic covariates (1 = yes, 0 = no) and whether they were missing a score on the stigma variable (1 = yes, 0 = no). Upon preliminary examination of associations, the following demographic variables showed significant associations with stigma scores, and were therefore included in all models to adjust for their potential confounding influence: age (in continuous form), education (basic/up to 12 grades vs. above), sexual orientation (gay/homosexual vs. bisexual/heterosexual/other), size of residence (small vs. large, with the latter including settlements with over 100,000 inhabitants), employment (employed vs. unemployed), and relationship status (having had a steady partner in the past 6 months vs. not). We included in analyses only participants whose HIV-negative serostatus was confirmed by this project, using Oracol oral fluid collection kits.
Results
Of the 2424 surveys in the dataset, we excluded 181 HIV-positive participants and 139 without HIV test results given our focus on predicting HIV-transmission risk behavior among HIV-negative participants. Finally, 17 cases missing all of the outcome variables were excluded from analyses, resulting in a final analytic sample of 2087 HIV-negative sexual minority men. Table 1 shows the prevalence of missing data on each of the covariates, which ranged from 3.4 to 6.6%. In total, 4.5% of the sample was missing data for the stigma variable (not shown) and prevalence of missing data on the eight outcomes ranged from 0 to 11%. Across all variables within the model, covariance coverage ranged from 86 to 100%.
Table 1.
n | % | M (SD) | Test statistica | |
---|---|---|---|---|
City, Country | ||||
Prague, Poland | 336 | 16.9 | 0.42 (0.81) | F(5, 1988) = 31.1, p < 0.001 |
Bratislava, Slovakia | 349 | 17.5 | 0.00 (0.77) | |
Barcelona, Spain | 312 | 15.6 | 0.47 (0.83) | |
Bucharest, Romania | 305 | 15.3 | −0.19 (0.93) | |
Ljubljana, Slovenia | 354 | 17.8 | 0.14 (0.91) | |
Verona, Italy | 338 | 16.9 | −0.04 (0.86) | |
Education | ||||
Basic | 1151 | 57.7 | 0.14 (0.91) | F(2, 1991) = 0.42, p = 0.66 |
University | 774 | 38.8 | 0.13 (0.84) | |
Missing | 69 | 3.4 | 0.04 (0.96) | |
Employment status | ||||
Employed | 1481 | 74.3 | 0.14 (0.89) | F(2, 1991) = 0.62, p = 0.54 |
Unemployed | 433 | 21.7 | 0.13 (0.86) | |
Missing | 80 | 4.0 | 0.02 (0.94) | |
Residential area | ||||
Small city (population < 100,000) | 639 | 32.0 | 0.40 (0.90) | F(2, 1991) = 4.96, p < 0.01 |
City (population > 100,000) | 1339 | 67.1 | 0.17 (0.87) | |
Missing | 16 | 0.80 | 0.03 (1.11) | |
Sexual identity | ||||
Gay/homosexual | 1628 | 81.6 | 0.18 (0.85) | F(2, 1991) = 11.5, p < 0.001 |
Bisexual or heterosexual | 343 | 17.2 | −0.07 (0.98) | |
Missing | 23 | 1.15 | −0.003 (0.96) | |
Steady partner in past 6 months | ||||
No | 596 | 29.9 | 0.12 (0.86) | F(2, 1991) = 1.57, p = 0.21 |
Yes | 1353 | 67.9 | 0.14 (0.89) | |
Missing | 45 | 2.25 | −0.90 (1.0) | |
M | SD | Range | ||
Age (Mdn = 30.0; valid n = 2000) | 31.6 | 9.7 | 18–76 | r = −0.021, p = 0.37 |
ANOVA tests were used to test for differences across groups
Participant Characteristics
Table 1 reports the socio-demographic characteristics of the sample. By design, the sample was nearly evenly divided among the five cities included within the present analyses. The majority of participants reported basic education or less (57.9%), being employed (73.9%), living within a city with population size exceeding 100,000 (66.5%), a gay sexual identity (80.2%), and having had a steady partner within the past 6 months (67.5%). The average age was 31.6 (SD = 9.7).
Associations of Demographic Characteristics and HIV Risk Behavior
Tables 2 and 3 report the results of the eight simultaneous regression analyses performed. With regard to demographic covariates, age was significantly associated with six of the outcome variables, education was significantly associated with three, employment was significantly associated with two, sexual orientation was significantly associated with five, location of primary dwelling was associated with four, and having a steady/main partner was associated with five.
Table 2.
Last sex: under the influence of alcohol |
Last sex: under the influence of cannabis |
Last sex: under the influence of club drugs |
Knowing where to get an HIV test |
||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
B | AOR | 95% CI | B | AOR | 95% CI | B | AOR | 95% CI | B | AOR | 95% CI | ||||
Age | −0.02 | 0.98** | [0.97, 0.99] | −0.01 | 0.99 | [0.97, 1.01] | −0.01 | 0.99 | [0.97, 1.01] | 0.04 | 1.04** | [1.01, 1.06] | |||
Missing covariates (ref. = no) | 0.21 | 1.24 | [− 0.07, 0.49] | 0.52 | 1.68* | [1.05, 2.69] | 0.16 | 1.18 | [0.71, 1.95] | −0.52 | 0.59* | [0.39, 0.90] | |||
Missing stigma (ref. = no) | 1.06 | 2.88*** | [0.47, 1.65] | 0.46 | 1.58 | [0.60, 4.17] | 0.66 | 1.93 | [0.75, 4.94] | −1.02 | 0.36** | [0.19, 0.68] | |||
University education (ref. = basic) | −0.13 | 0.88 | [0.72, 1.07] | −0.43 | 0.65* | [0.45, 0.96] | 0.04 | 1.04 | [0.71, 1.53] | 0.29 | 1.33 | [0.93, 1.92] | |||
Employed (ref. = unemployed) | −0.29 | 0.75* | [0.59, 0.96] | −0.07 | 0.93 | [0.59, 1.47] | −0.32 | 0.73 | [0.45, 1.19] | 0.03 | 1.03 | [0.67, 1.58] | |||
Bisexual orientation (ref. = gay) | 0.44 | 1.55*** | [1.22, 1.98] | 0.55 | 1.74** | [1.14, 2.64] | 0.83 | 2.30*** | [1.52, 3.47] | −0.25 | 0.78 | [0.52, 1.17] | |||
City dweller (ref. = small city/town) | 0.24 | 1.27* | [1.03, 1.55] | 0.31 | 1.37 | [0.91, 2.05] | 0.23 | 1.25 | [0.82, 1.92] | 0.40 | 1.50* | [1.06, 2.12] | |||
Had a steady partner (ref. = no) | −0.26 | 0.77* | [0.63, 0.95] | −0.34 | 0.71 | [0.49, 1.04] | −0.28 | 0.76 | [0.52, 1.11] | 0.23 | 1.26 | [0.88, 1.79] | |||
Sexual orientation stigma (z-score) | 0.18 | 1.19*** | [1.07, 1.33] | 0.24 | 1.27* | [1.03, 1.56] | 0.40 | 1.49*** | [1.22, 1.82] | 0.23 | 1.26* | [1.01, 1.59] |
All models were adjusted for demographic characteristics significantly associated with sexual orientation stigma
B unadjusted coefficient, AOR adjusted coefficient
N = 2087.
p < 0.05
p < 0.01
p < 0.001
Table 3.
HIV tested past 12 months |
Any HIV status-unknown casual partners |
Number of casual partnersa |
Frequency of condomless sex with casual partnersa |
||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
B | AOR | 95% CI | B | AOR | 95% CI | B | AOR | 95% CI | B | AOR | 95% CI | ||||
Age | 0.02 | 1.02*** | [1.01, 1.03] | 0.03 | 1.03*** | [1.02, 1.04] | 0.02 | 1.02*** | [1.01, 1.03] | −0.02 | 0.98*** | [0.97, 0.99] | |||
Missing covariates (ref. = no) | −0.19 | 0.83 | [0.64, 1.08] | −0.26 | 0.77 | [0.59, 1.01] | 0.45 | 1.56 | [0.65, 2.47] | 0.04 | 1.04 | [0.79, 1.29] | |||
Missing stigma (ref. = no) | −0.30 | 0.74 | [0.45, 1.21] | −0.24 | 0.79 | [0.49, 1.25] | 0.03 | 1.03 | [0.63, 1.43] | 0.33 | 1.39 | [0.91, 1.86] | |||
University education (ref. = basic) | 0.18 | 1.20 | [0.99, 1.44] | 0.15 | 1.16 | [0.96, 1.41] | 0.42 | 1.53* | [1.10, 2.11] | −0.39 | 0.68*** | [0.56, 0.82] | |||
Employed (ref. = unemployed) | 0.00 | 1.00 | [0.80, 1.26] | −0.04 | 0.96 | [0.76, 1.21] | 0.38 | 1.46* | [1.00, 2.13] | 0.05 | 1.05 | [0.84, 1.32] | |||
Bisexual orientation (ref. = gay) | −0.07 | 0.93 | [0.74, 1.18] | −0.31 | 0.73* | [0.58, 0.93] | 0.31 | 1.37 | [0.92, 2.02] | 0.10 | 1.11 | [0.92, 1.33] | |||
City dweller (ref. = small city/town) | 0.24 | 1.27* | [1.05, 1.54] | 0.03 | 1.03 | [0.85, 1.25] | 0.39 | 1.48*** | [1.22, 1.79] | −0.04 | 0.96 | [0.81, 1.14] | |||
Had a steady partner (ref. = no) | 0.29 | 1.34** | [1.10, 1.62] | −0.67 | 0.51*** | [0.42, 0.63] | −0.47 | 0.63*** | [0.53, 0.74] | 0.20 | 1.22* | [1.02, 1.45] | |||
Sexual orientation stigma (z-score) | 0.08 | 1.08 | [0.97, 1.20] | −0.09 | 0.92 | [0.83, 1.02] | 0.07 | 1.08 | [0.91, 1.27] | 0.11 | 1.11* | [1.01, 1.23] |
All models were adjusted for demographic characteristics significantly associated with sexual orientation stigma
B unadjusted coefficient, AOR adjusted coefficient
N = 2087.
p < 0.05
p < 0.01
p < 0.001
Modeled using Poisson distribution
Associations of Sexual Minority Stigma and Alcohol/Drug Use at Last Sexual Encounter
Stigma was significantly associated with five of the out-comes (Table 2). For the three substance-related outcomes, a one standard deviation increase in stigma was associated with a 19% increase in the odds of sex under the influence of alcohol at the last sexual encounter, a 27% increase in the odds of sex under the influence of cannabis at the last sexual encounter, and a 49% increase in the odds of sex under the influence of illicit drugs at the last sexual encounter.
Associations of Sexual Minority Stigma and Condomless Sex and HIV Testing
Table 3 presents results indicating that stigma was positively associated with one HIV risk outcome in the expected direction. Namely, a one standard deviation increase in stigma was associated with an 11% increase in the odds of condomless sex with casual partners in the past 6 months. Conversely, a one standard deviation increase in stigma was associated with a 26% increase in the odds of knowing where to receive an HIV test, though not associated with actual testing in the prior 12 months. Stigma was also not associated with having HIV status-unknown casual partners or with the total number of casual partners in the past 6 months.
Discussion
In this large sample of sexual minority men across six diverse European contexts, those who perceived greater sexual orientation stigma in their daily social institutions were significantly more likely to report sexual risk behavior with casual partners, as well as alcohol, marijuana, and illicit drug use (including cocaine, amphetamine, and MDMA) immediately prior to or during their most recent sexual encounter. While alcohol and drug use are established predictors of HIV-risk behavior [47, 48], the results of this study suggest that perceived high sexual orientation stigma from one’s social environment (from family to government), as reported by sexual minority men, might serve as a precursor to these risks, confirming previous findings [49]. At the same time, the impact of stigma on sexual minority men’s HIV-related risk is capable of being modified through interventions that support stigmatized individuals’ acquisition of skills to ameliorate the effects of stigma on health, and to view stigma as a problematic societal phenomenon, external to the individual, in need of change [50–52]. Below, we highlight how individually-focused interventions can improve stigma coping to reduce sexual minority men’s HIV risk alongside structural interventions to reduce stigma at its source in discriminatory social institutions.
According to minority stress theory, sexual minority men living in social contexts marked by prevalent sexual minority stigma likely experience increased stress and associated coping strategies, including maladaptive strategies, for managing that stress [53, 54]. Our results suggest that alcohol and drug use might serve as a prominent mechanism for coping with stigma-related stress among European sexual minority men who frequently encounter social contextual stigma. Our findings show that stigma is particularly associated with alcohol and drug use in the context of sexual activity with other men. Perhaps sex with men invokes shame and guilt through making the stigma of one’s sexual orientation particularly salient, with substance use during sex being one way to cope with this salience. Indeed, sexual minority men living in stigmatizing national contexts report high degrees of internalized homonegativity [55–57], which, though unmeasured here, potentially drives distress or cognitive dissonance during same-sex sexual behaviors [58]. Internalized homophobia, in turn, has shown consistent associations with substance use across several studies [49, 59]. These findings suggest that HIV-prevention strategies should build not only knowledge, motivation, and skills for engaging in HIV-prevention behavior, but also build adaptive forms of stigma coping as an important goal to reduce HIV-risk behavior [50, 60].
Our analyses paradoxically showed that those reporting higher levels of social contextual stigma were more likely to know where they could test for HIV, which may be explained by successful efforts by local advocates and NGOs to disseminate health-related information to those who most need it. Although stigma appeared to play a protective role in the sense that higher levels predicted better knowledge regarding HIV testing locations, it was not associated with individuals actually testing more frequently. This finding echoes literature indicating that knowledge is not sufficient to lead to action, and that building behavioral skills for self-efficacy is a necessary step alongside acquiring information [61]. Future research will need to determine the mechanisms through which greater stigma predicts greater knowledge of local testing resources. Results further suggest the need for prevention resources to motivate sexual minority men who perceive high levels of stigma in their daily lives to translate their knowledge of preventive resources into action by increasing their HIV testing frequency.
Finally, one’s number of casual partners reported over the past 6 months and not knowing the HIV status of these casual partners had no associations with perceptions of social contextual stigma. However, perceptions of social contextual stigma were associated with a small increase in odds of engaging in condomless sex with casual partners over the past 6 months. This finding supports previous research suggesting that stigma serves as a determinant of sexual minority men’s HIV risk [8, 29, 62, 63], but suggests that this risk might only extend to condom use, rather than selection of casual partners, lack of communication with partners about status, or number of partners. Future research might determine the mechanisms, both environmental (e.g., condom availability) and psychological (e.g., condom use self-assertion), as mechanisms linking perceived stigma to lack of condom use with casual partners.
As suggested above, interventions that promote stigma coping by promoting self-assertion and reducing social or emotional avoidance, might be employed fruitfully to increase sexual minority men’s condom use, HIV testing, and reduce alcohol and drug use. Evidence suggests that such interventions show preliminary promise [50, 64]. Furthermore, mobile health (mHealth) interventions, which possess acceptability, feasibility, as well as efficacy in reducing HIV risk across various cultural contexts [64–66] may be a particularly useful modality of reaching large numbers of vulnerable sexual minority men who are likely to conceal their identities in highly stigmatizing contexts and prefer to access preventive resources in virtual settings where they feel safest [65, 67–69]. These types of interventions address several barriers to prevention by directly delivering LGBT-affirmative HIV risk-reduction interventions to vulnerable populations [70], especially those who are otherwise out-of-reach of public health campaigns because of geographical constraints, lack of brick-and-mortar establishments, and/or stigma itself.
Limitations
These findings should be interpreted in the light of several limitations. First, our study is limited by several aspects of our measurement approach. Measuring stigma is complex given the multiple levels of the ecosystem on which it manifests itself (e.g., from structural to individuals). In fact, in choosing to focus on social contextual stigma, we did not capture other forms of this multifaceted construct, including self-stigma (e.g., internalized stigma) and structural stigma (e.g., national laws and policies toward sexual minorities) [24]. Furthermore, we examined prejudice from different sources as an aggregate of contextual stigma, however, stigma experienced from family vs. coworkers or religious institutions may have differential impact on various outcomes. Therefore, future research would benefit from examining these relationships individually, rather than as a cumulative independent variable on outcomes of sexual risk. Still, evidence suggests that these diverse forms of stigma are related to the construct assessed here [27, 62, 71]. Another measurement limitation involves reliance on self-reported stigma, introducing potential confounds between our stress-related outcomes and stigma [72]. The inclusion of an interviewer-based calendar review of sexual behavior and alcohol/drug use history would have allowed for more fine-grained analyses that would include, for example, alcohol and drug use modelling independent of sexual encounters, or for day-level sexual risk behavior. Relatedly, the data were obtained through self-report, which could be considered to be a limitation in an epidemiological study. Lastly, more recent data would provide a clearer picture of the immediate context; yet associations among the variables examined here are likely to persist across time, even if their overall prevalence changes.
Although a small proportion of respondents was missing data, we utilized one of the most widely accepted techniques for handling such data that also allowed us to include them within models. Patterns of missing data were significantly associated with some but not all outcomes, though these effects were accounted for within the models; nonetheless, findings for those models should be interpreted with caution. Sensitivity analyses suggested that our approach of using missing data indicators along with robust maximum likelihood estimation reduced effect sizes for stigma and widened confidence intervals, making it more difficult to detect significant effects, compared to models without such indicators. As such, despite missing data, this conservative approach likely underestimated rather than overestimating the effect sizes and, to the extent it may have been biased, was biased toward non-significance. Because the dataset contained individuals from a diverse group of European countries, we initially hoped to take the nesting of the data into account and consider the role of country-level structural factors. However, with only six countries, the number of Level 2 units within the multilevel model was small, undermining any power to analyze Level 2 predictors and compromising the ability to estimate the Level 2 variance of the eight categorical and count outcomes (i.e., models would not converge due to the combination of model complexity and insufficient Level 2 units of analysis). Future studies with more variability across locations that can also incorporate higher-level predictors are warranted.
Conclusion
Our findings suggest that perceptions of social contextual stigma toward one’s sexual orientation serve as an important risk factor for HIV infection. These findings help identify a clear pathway that might jeopardize the health of sexual minority men who live in ubiquitously stigmatizing environments. While our findings suggest that interventions designed to support sexual minority men’s stigma coping abilities can protect against risk [50, 51, 64–66, 73–77], concomitant structural changes capable of reducing stigma at its source in stigmatizing political, legal, and institutional structures are also needed. Such changes are especially needed in countries where these structures perpetuate conditions in which sexual minorities are likely to perceive high levels of social contextual stigma and therefore be at greater risk of adverse behavioral health.
Acknowledgements
We gratefully acknowledge all the participants and project staff who have dedicated their time to this research.
Funding This article is based on the findings from the SIALON project, co-funded by the European Commission under the Public Health Programme 2003–2008 (GA 2007309). The sole responsibility lies with the author of the publication, and the Commission is not responsible for any use that may be made of the information contained therein.
Footnotes
Compliance with Ethical Standards
Conflict of interest The authors have no conflicts of interest to declare.
Research Involving Human Subjects Informed consent was obtained from all individual participants included in the study. Study protocols were approved by each study site’s ethics committee, and are in compliance with the 1964 Declaration of Helsinki for ethical principles for conducting research with human subjects.
References
- 1.ILGA Europe. Rainbow Europe package. Annual Review and Rainbow Europe Map https://rainbow-europe.org/country-ranking. Accessed 19 July 2016.
- 2.Beyrer C, Baral SD, editors. MSM, HIV, and the law: the case of gay, bisexual and other men who have sex with men (MSM) Working paper for the Third Meeting of the Technical Advisory Group of the Global Commission on HIV and the Law. July 7–9, 2011. [Google Scholar]
- 3.Mahajan AP, Sayles JN, Patel VA, Remien RH, Ortiz D, Szekeres G, et al. Stigma in the HIV/AIDS epidemic: a review of the literature and recommendations for the way forward. AIDS 2008;22(Suppl 2):S67–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Millett GA, Peterson JL, Wolitski RJ, Stall R. Greater risk for HIV infection of black men who have sex with men: a critical literature review. J Inf 2006;96(6):1007–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Millett GA, Peterson JL, Flores SA, Hart TA, Jeffries WL, Wilson PA, et al. Comparisons of disparities and risks of HIV infection in black and other men who have sex with men in Canada, UK, and USA: a meta-analysis. Lancet 2012;380(9839):341–8. [DOI] [PubMed] [Google Scholar]
- 6.Jeffries WL IV, Marks G, Lauby J, Murrill CS, Millett GA. Homophobia is associated with sexual behavior that increases risk of acquiring and transmitting HIV infection among black men who have sex with men. AIDS Behav 2012;17(4):1442–53. [DOI] [PubMed] [Google Scholar]
- 7.Oster AM, Wiegand RE, Sionean C, Miles IJ, Thomas PE, Melendez-Morales L, et al. Understanding disparities in HIV infection between black and white MSM in the United States. AIDS 2011;25(8):1103–12. [DOI] [PubMed] [Google Scholar]
- 8.Lelutiu-Weinberger C, Pachankis JE, Golub SA, Walker JJ, Bamonte A, Parsons JT. Age cohort differences in the effects of gay-related stigma, anxiety and identification with the gay community on sexual risk and substance use. AIDS Behav 2013;17(1):340–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.European Center for Disease Prevention and Control/WHO Regional Office for Europe. HIV/AIDS surveillance in Europe 2015. https://ecdc.europa.eu/sites/portal/files/media/en/publications/Publications/HIV-AIDS-surveillance-Europe-2015.pdf. Accessed 15 Jan 2017.
- 10.SIALON II. City Profiles. Recommendations on HIV prevention gaps and opportunities for health promotion among men who have sex with men 2015. http://www.sialon.eu/data2/file/133_Sialon%20II_City%20Profiles_Recommendations%20on%20HIV%20prevention%20gaps%20and%20opportunities%20for%20Health%20Promotion%20among%20Men%20who%20have%20Sex%20with%20Men.pdf. Accessed 12 Jul 2016.
- 11.Ferrer L, Furegato M, Foschia J-P, Folch C, González V, Ramarli D, et al. Undiagnosed HIV infection in a population of MSM from six European cities: Results from the Sialon project. Eur J Public Health 2015;25(3):494–500. [DOI] [PubMed] [Google Scholar]
- 12.ACCEPT Association of Romania. ILGA Europe. Documentation of discrimination in the field of LGBT heath in Romania. General overview, legal framework, finding and recommendations 2014. http://www.ilga-europe.org/sites/default/files/Attachments/romania.pdf. Accessed 3 March 2017.
- 13.Nachescu V Hierarchies of difference: National identity, gay and lesbian rights, and the church in postcommunist Romania. In: Coleman E, Sandfort T, editors. Sexuality and gender in postcommunist Eastern Europe and Russia New York: Haworth Press; 2005. p. 57–78. [Google Scholar]
- 14.Turcescu L, Stan L. Religion, politics and sexuality in Romania. Eur-Asia Stud 2005;57(2):291–310. [Google Scholar]
- 15.Krishnamoorthy N. Romanian activists call for ban on same-sex marriages in the country. International Business Times; http://www.ibtimes.co.uk/romanian-activists-call-ban-same-sex-marriages-country-1561657. Accessed 14 July 2016. [Google Scholar]
- 16.Morgan J. Three million people want to ban same-sex marriage in Romania. Gay Star News; http://www.gaystarnews.com/article/three-million-people-want-ban-sex-marriage-romania/#gs.EsgV2U. Accessed 14 July 2016. [Google Scholar]
- 17.Moraru A Social perception of homosexuality in Romania. Proc Soc Behav Sci 2010;2(2):5726–30. [Google Scholar]
- 18.Holzhacker R State-sponsored homophobia and the denial of the right of assembly in Central and Eastern Europe: the “boomerang” and the “ricochet” between European organizations and civil society to uphold human rights. Law Policy 2013;35(1–2):1–28. [Google Scholar]
- 19.Takács J, Kuhar R, editors. Beyond the pink curtain. Everyday life of LGBT people in Eastern Europe Ljubljana: The Peace Institute of Contemporary Social and Political Studies; 2007. [Google Scholar]
- 20.Van den Akker H, Van der Ploeg R, Scheepers P. Disapproval of homosexuality: comparative research on individual and national determinants of disapproval of homosexuality in 20 European countries. Int J Public Opin Res 2013;25(1):64–86. [Google Scholar]
- 21.Crocker J, Major B, Steele C. Social stigma: the psychology of marked relationships. Handbook Soc Psychol 1998;2:504–53. [Google Scholar]
- 22.Pachankis JE, Hatzenbuehler ML, Wang K, Burton CL, Crawford FW, Phelan JC, et al. The burden of stigma on health and well-being: A taxonomy of concealment, course, disruptiveness, aesthetics, origin, and peril across 93 stigmas. Pers Soc Psychol Bull 2018;44(4):451–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Link BG, Phelan JC. Conceptualizing stigma. Annu Rev Sociol 2001;27(1):363–85. [Google Scholar]
- 24.Bos AE, Pryor JB, Reeder GD, Stutterheim SE. Stigma: advances in theory and research. Basic Appl Soc Psychol 2013;35(1):1–9. [Google Scholar]
- 25.Ragins BR, Cornwell JM. Pink triangles: antecedents and consequences of perceived workplace discrimination against gay and lesbian employees. J Appl Psychol 2001;86(6):124412–61. [DOI] [PubMed] [Google Scholar]
- 26.Ryan C, Huebner D, Diaz RM, Sanchez J. Family rejection as a predictor of negative health outcomes in white and Latino lesbian, gay, and bisexual young adults. Pediatrics 2009;123(1):346–52. [DOI] [PubMed] [Google Scholar]
- 27.Pachankis JE, Bränström R. Hidden from happiness: structural stigma, concealment, and life satisfaction among sexual minorities across 28 countries. J Consult Clin Psychol 2018;86:403–15. [DOI] [PubMed] [Google Scholar]
- 28.Leluțiu-Weinberger C, Pachankis JE. Acceptability and preliminary efficacy of an LGBT-affirmative mental health practice training in a homophobic national context. LGBT Health 2017;4(5):360–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Pachankis JE, Hatzenbuehler ML, Hickson F, Weatherburn P, Berg RC, Marcus U, et al. Hidden from health: structural stigma, sexual orientation concealment, and HIV across 38 countries in the European MSM internet survey. AIDS 2015;29(10):1239–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Centers for Disease Control and Prevention. HIV surveillance report. Diagnoses of HIV infection in the United States and dependent areas, 2015 https://www.cdc.gov/hiv/pdf/library/reports/surveillance/cdc-hiv-surveillance-report-2015-vol-27.pdf. Accessed 15 Nov 2016.
- 31.European Center for Disease Prevention and Control. EMIS 2010: The European men-who-have-sex-with-men internet survey: findings from 38 countries http://ecdc.europa.eu/en/publications/Publications/EMIS-2010-european-men-who-have-sex-with-men-survey.pdf. Accessed 28 Aug 2016.
- 32.Mayer KH, Beyrer C. HIV epidemiology update and transmission factors: risks and risk contexts—16th International AIDS Conference epidemiology plenary. Clin Infect Dis 2007;44(7):981–7. [DOI] [PubMed] [Google Scholar]
- 33.Folch C, Muñoz R, Zaragoza K, Casabona J, Pujol UGT. Sexual risk behaviour and its determinants among men who have sex with men in Catalonia, Spain. Euro Surveill 2009;14(47):24–30. [DOI] [PubMed] [Google Scholar]
- 34.Amirkhanian YA, Kelly JA, Takacs J, Kuznetsova AV, DiFranceisco WJ, Mocsonaki L, et al. HIV/STD prevalence, risk behavior, and substance use patterns and predictors in Russian and Hungarian sociocentric social networks of men who have sex with men. AIDS Edu Prev 2009;21(3):266–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Mirandola M, Gios L, Davis J, Furegato M, Breveglieri M, Folch C, et al. Socio-demographic factors predicting HIV test seeking behaviour among MSM in 6 EU cities. Eur J Public Health 2017;27(2):313–8. [DOI] [PubMed] [Google Scholar]
- 36.Mirandola M, Gios L, Sherriff N, Pachankis J, Toskin I, Ferrer L, et al. Socio-demographic characteristics, sexual and test-seeking behaviours amongst men who have sex with both men and women: Results from a bio-behavioural survey in 13 European cities. AIDS Behav 2017;21(10):3013–25. [DOI] [PubMed] [Google Scholar]
- 37.Mirandola M, Folch TC, Krampac I, Nita I, Stanekova D, Stehlikova D, et al. HIV bio-behavioural survey among men who have sex with men in Barcelona, Bratislava, Bucharest, Ljubljana, Prague and Verona, 2008–2009. Eur Commun Dis Bull 2008;14(48):1424–7. [DOI] [PubMed] [Google Scholar]
- 38.Stanekova D, Mirandola M, Gios L, Botsi C, Habekova M, Gonzales-Soler V, et al. Validation study of a conventional enzyme immunoassay to detect HIV antibodies in oral fluid. Bratisl Lek Listy 2016;117(1):19–21. [DOI] [PubMed] [Google Scholar]
- 39.UNAIDS Joint United Nations Programme on HIV/AIDS. Global AIDS response progress reporting 2014: construction of core indicators for monitoring the 2011 UN political declaration on HIV/AIDS. Joint United Nations Programme on HIV/AIDS (UNAIDS), WHO Library Cataloguing-in-Publication Data; Geneva: Switzerland; 2014. [Google Scholar]
- 40.Williamson LM, Hart GJ. HIV prevalence and undiagnosed infection among a community sample of gay men in Scotland. J Acquir Immune Defic Syndr 2007;45(2):224–30. [DOI] [PubMed] [Google Scholar]
- 41.Halkitis P, Parsons J, Stirratt M. A double epidemic: Crystal methamphetamine drug use in relation to HIV transmission among gay men. J Homosex 2001;41(2):17–36. [DOI] [PubMed] [Google Scholar]
- 42.Colfax G, Mansergh G, Guzman R, Vittinghoff E, Marks G, Rader M, et al. Drug use and sexual risk behavior among gay and bisexual men who attend circuit parties: a venue-based comparison. J Acquir Immune Defic Syndr 2001;28(4):373–9. [DOI] [PubMed] [Google Scholar]
- 43.Clatts M, Goldsamt L, Yi H. Club drug use among young men who have sex with men in NYC: a preliminary epidemiological profile. Subst Use Misuse 2005;40(9):1317–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Celentano DD, Valleroy LA, Sifakis F, MacKellar DA, Hylton J, Thiede H, et al. Associations between substance use and sexual risk among very young men who have sex with men. Sex Transm Dis 2006;33(4):265–71. [DOI] [PubMed] [Google Scholar]
- 45.Wilton L Correlates of substance abuse use in relation to sexual behavior in black gay and bisexual men: Implications for HIV prevention. J Black Psychol 2008;34(1):70–93. [Google Scholar]
- 46.Sarkar S, Panda S, Sarkar K, Hangzo CZ, Bijaya L, Singh NY, et al. A cross-sectional study on factors including HIV testing and counselling determining unsafe injecting practices among injecting drug users of Manipur. Indian J Public Health 1995;39(3):86–92. [PubMed] [Google Scholar]
- 47.Ackers M-L, Greenberg AE, Lin CY, Bartholow BN, Goodman AH, Longhi M, et al. High and persistent HIV seroincidence in men who have sex with men across 47 US cities. PLoS ONE 2012;7(4):e34972. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Sander PM, Cole SR, Stall RD, Jacobson LP, Eron JJ, Napravnik S, et al. Joint effects of alcohol consumption and high-risk sexual behavior on HIV seroconversion among men who have sex with men. AIDS 2013;27(5):815–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Hatzenbuehler ML, Nolen-Hoeksema S, Erickson SJ. Minority stress predictors of HIV risk behavior, substance use, and depressive symptoms: results from a prospective study of bereaved gay men. Health Psychol 2008;27(4):455–62. [DOI] [PubMed] [Google Scholar]
- 50.Pachankis JE, Hatzenbuehler ML, Rendina HJ, Safren SA, Parsons JT. LGB-affirmative cognitive behavioral therapy for young adult gay and bisexual men: a randomized controlled trial of a transdiagnostic minority stress approach. J Consult Clin Psychol 2015;83(5):875–89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Pachankis JE. A transdiagnostic minority stress treatment approach for gay and bisexual men’s syndemic health conditions. Arch Sex Behav 2015;44(7):1843–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Hatzenbuehler ML. How does sexual minority stigma “get under the skin”? A psychological mediation framework. Psychol Bull 2009;135(5):707–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Meyer IH. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence. Psychol Bull 2003;129(5):674–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Hatzenbuehler ML, McLaughlin KA, Keyes KM, Hasin DS. The impact of institutional discrimination on psychiatric disorders in lesbian, gay, and bisexual populations: a prospective study. Am J Public Health 2010;100(3):452–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Herek GM, Gillis JR, Cogan JC. Internalized stigma among sexual minority adults: insights from a social psychological perspective. J Couns Psychol 2009;56(1):32–43. [Google Scholar]
- 56.Ross MW, Rosser BS, Neumaier ER. The relationship of internalized homonegativity to unsafe sexual behavior in HIV seropositive men who have sex with men. AIDS Educ Prev 2008;20(6):547–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Huebner DM, Davis MC, Nemeroff CJ, Aiken LS. The impact of internalized homophobia on HIV preventive interventions. Am J Community Psychol 2002;30(3):327–48. [DOI] [PubMed] [Google Scholar]
- 58.Wells BE, Golub SA, Parsons JT. An integrated theoretical approach to substance use and risky sexual behavior among men who have sex with men. AIDS Behav 2011;15(3):509–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Weber G Using to numb the pain: Substance use and abuse among lesbian, gay, and bisexual individuals. J Mental Health Couns 2008;30(1):31–48. [Google Scholar]
- 60.Pachankis JE. Uncovering clinical principles and techniques to address minority stress, mental health, and related health risks among gay and bisexual men. Clin Psychol 2014;21:313–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Miller W, Rollnick S. Motivational interviewing: preparing people for change New York: Guilford Press; 2002. [Google Scholar]
- 62.Hatzenbuehler ML, Phelan JC, Link BG. Stigma as a fundamental cause of population health inequalities. Am J Public Health 2013;103(5):813–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Bruce D, Ramirez-Valles J, Campbell RT. Stigmatization, substance use, and sexual risk behavior among Latino gay and bisexual men and transgender persons. J Drug Issues 2008;38(1):235–60. [Google Scholar]
- 64.Lelutiu-Weinberger C, Manu M, Lăscut F, Dogaru B, Kovacs T, Dorobănțescu C, et al. An mHealth intervention to improve young gay and bisexual men’s sexual behavioral and mental health in a structurally stigmatizing national context. J Med Internet Res 2018. 10.2196/mhealth.9283. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Muessig KE, Nekkanti M, Bauermeister J, Bull S, Hightow-Weidman LB. A systematic review of recent smartphone, internet and web 2.0 interventions to address the HIV continuum of care. Curr HIV/AIDS Rep 2015;12(1):173–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Lelutiu-Weinberger C, Pachankis JE, Gamarel K, Surace A, Golub SA, Parsons JT. Feasibility, acceptability and preliminary efficacy of a live-chat intervention to reduce HIV risk among young men who have sex with men. AIDS Behav 2015;19(7):1214–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Schnall R, Travers J, Rojas M, Carballo-Diéguez A. eHealth interventions for HIV prevention in high-risk men who have sex with men: a systematic review. J Med Internet Res 2014;16(5):e134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Sullivan PS, Grey JA, Rosser BRS. Emerging technologies for HIV prevention for MSM: What we have learned, and ways forward. J Acquir Immune Syndr 2013;63:S102–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Donker T, Petrie K, Proudfoot J, Clarke J, Birch M-R, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013;15(11):e247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Muessig KE, Pike EC, Fowler B, LeGrand S, Parsons JT, Bull SS, et al. Putting prevention in their pockets: developing mobile phone-based HIV interventions for black men who have sex with men. AIDS Patient Care STDs 2013;27(4):211–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Pachankis JE, Eldahan AI, Golub SA. New to New York: ecological and psychological predictors of health among recently arrived young adult gay and bisexual urban migrants. Ann Behav Med 2016;50(5):692–703. [DOI] [PubMed] [Google Scholar]
- 72.Meyer IH. Prejudice as stress: conceptual and measurement problems. Am J Public Health 2003;93(2):262–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Lyles CM, Kay LS, Crepaz N, Herbst JH, Passin WF, Kim AS, et al. Best-evidence interventions: findings from a systematic review of HIV behavioral interventions for US populations at high risk, 2000–2004. Am J Public Health 2007;97(1):133–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Grossman CI, Forsyth A, Purcell DW, Allison S, Toledo C, Gordon CM. Advancing novel HIV prevention intervention research with MSM—meeting report. Public Health Rep 2011;126(4):472–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Johnson WD, Diaz RM, Flanders WD, Goodman M, Hill AN, Holtgrave D, et al. Behavioral interventions to reduce risk for sexual transmission of HIV among men who have sex with men. Cochrane Database Syst Rev 2008;3:1230. [DOI] [PubMed] [Google Scholar]
- 76.Bull SS, Levine DK, Black SR, Schmiege SJ, Santelli J. Social media-delivered sexual health intervention: a cluster randomized controlled trial. Am J Prev Med 2012;43(5):467–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Parsons JT, Lelutiu-Weinberger C, Botsko M, Golub SA. A randomized controlled trial utilizing motivational interviewing to reduce HIV risk and drug use in young gay and bisexual men. J Consult Clin Psychol 2014;82(1):9–18. [DOI] [PMC free article] [PubMed] [Google Scholar]