Internalized Homophobia and Perceived Stigma: A Validation Study of Stigma Measures in a Sample of Young Men who Have Sex with Men

Abstract

Young men who have sex with men (YMSM) experience minority stressors that impact their mental health, substance use, and sexual risk behaviors. Internalized homophobia (IH) and perceived stigma represent two of these minority stressors, and there has been limited research empirically validating measures of these constructs. We validated measures of IH and perceived stigma with a sample of 450 YMSM (mean age=18.9) and a sample of 370 YMSM (mean age=22.9). Results from exploratory and confirmatory factor analyses supported modifications to the IH and perceived stigma scales, ultimately revealing a three factor and one factor structure, respectively. Convergent and discriminant validity were examined utilizing correlations between IH, perceived stigma, and other variables related to minority stress (e.g., victimization). We evaluated predictive validity by examining relations with mental health, substance use, and risky sexual behaviors measured 12-months from baseline. There were mixed findings for IH, with subscales varying in their relations to mental health, drinking, and sexual risk variables. Perceived stigma was not related to mental health or substance use, but was associated with greater prevalence of STIs. Findings supported the use of these modified scales with YMSM and highlight the need for further measurement studies.

Lesbian, gay, and bisexual (LGB) individuals have elevated rates of depression, anxiety, and suicide attempts when compared to heterosexual samples (King et al., 2008; Marshal et al., 2011; McLaughlin, Hatzenbuehler, Xuan, & Conron, 2012), and it is widely-accepted that these elevated mental health problems are explained (at least in part) by the unique stressors associated with being a sexual minority (Hatzenbuehler, 2009; Meyer, 1995; 2003). In his minority stress model, Meyer (1995, 2003) described distal and proximal experiences of minority stress that are hypothesized to account for elevated levels of mental health issues in LGB populations. The distal components included experiencing prejudiced events such as harassment, discrimination, and victimization. The proximal components included awareness or expectations of rejection (or perceived stigma), concealment of one’s sexual orientation, and internalized homophobia (IH). Previous authors have noted a continued need for empirically validated measures of IH and perceived stigma (Grey, Robinson, Coleman, & Bockting, 2013; Herek, Gillis, & Cogan, 2009), although arguably there continues to be a lack of research on measurement despite significant interest in the underlying constructs. As such, we present a validation study of IH and perceived stigma measures in samples of racially-diverse young men who have sex with men (YMSM; ages 16–20 and 16–29 years old in our samples), a population that is often not included within existing validation studies of these constructs.

Operationalization and Measurement of Internalized Homophobia

IH is defined as when a sexual minority person has negative feelings and homophobic attitudes towards themselves and others who are sexual minorities, as a product of social bias (Shidlo, 1994). IH has received many names in the literature, including internalized heterosexism (Szymanski, 2004), internalized homonegativity (Mayfield, 2001), and homonegating processes (Russell & Bohan, 2006), and several authors have reviewed distinctions in this terminology including Shidlo (1994) and Szymanski, Kashubeck-West, and Meyer (2008). IH has been a popular topic within research, especially work focused on its connections with mental health for LGB individuals (e.g., Newcomb & Mustanski, 2010); however, limited research has empirically evaluated the reliability and validity of measures of IH and many studies may be impacted by issues related to the operationalization of the construct.

One central critique of the operationalization and measurement of IH is that some researchers have included correlates and outcomes of IH in the items assessing this construct rather than focusing purely on IH (Shidlo, 1994). While this concern was raised over two decades ago by Shidlo, its consequences can still be seen in contemporary measurement of IH. For example, many measures include items assessing perceptions of stigma (Ross & Rosser, 1996) and connections with other sexual minorities (Szymanski & Chung, 2001). The resulting varied definitions of IH have led to differences in scale development, as well as potentially influencing findings across studies. In order for the field to achieve a cohesive understanding of IH and its impact on health outcomes, measures of IH need to more closely map onto the original definition of this construct. Additionally, most scales have not been validated for use with YMSM, so the degree to which many previous scales are useful with this population is unknown.

Furthermore, assessing the reliability and validity of IH measures with current samples is important given the great social shifts that have occurred since some of the early measures were developed in the 1980s and 1990s (for a review of IH measures used with sexual minority men, see Grey, Robinson, Coleman, & Bockting, 2013). For example, one of the most widely used measures, the Internalized Homophobia Scale (Martin & Dean, 1987), was developed based on the diagnosis of ego-dystonic homosexuality in the Diagnostic and Statistical Manual of Mental Disorders – III, which is inherently connected with the clinical marginalization of sexual minorities. Further, recent samples have shown low rates of endorsing items on some IH scales. Herek, Gillis, and Cogan (2009) found that 77.5% of gay participants did not agree with any of the items on their IH scale. This could reflect either lack of precision in the measurement of IH or global decreases in the experience of IH in the study population. We sought to provide an evaluation of a commonly used measure of IH in light of the limited literature on validation of scales that considers these issues of conceptualization and adherence to the definition of the construct of IH, particularly with more recent samples of YMSM.

Operationalization and Measurement of Perceived Stigma

As Herek (2004) described, sexual orientation based stigma refers to the knowledge that individuals hold negative societal attitudes about sexual minorities or individuals who engage in same-sex sexual behaviors. This includes an awareness of the marginalized status of anyone with a non-heterosexual identity or who has same-sex relationships. This construct also has gone by many names in the literature, with a few highlighted by Herek (2004), such as sexual stigma or erotic stigma (from the 70s and 80s), with more recent writers using terms such as perceived stigma or stigma consciousness (Pinel, 1999) or expectations of rejection (Meyer, 2003).

Although both IH and perceived stigma are inherently connected to external experiences of marginalization as without a heterosexist society, neither would exist (Herek et al., 2009), there are several ways in which perceived stigma differs from IH. In defining these constructs, IH is a personal endorsement of the stigmatization or beliefs that one deserves or should be stigmatized for being a sexual minority. In contrast, perceived stigma reflects general beliefs or awareness that others are engaged in heterosexist acts or hold heterosexist belief systems. This awareness of stigma does not necessarily reflect a personal belief in those stigmatizing views – for example, some gay men may know that other people hold heterosexist beliefs that gay men should not be fathers (representative of perceived stigma), but that does not indicate that they personally believe this, which would be an indication of IH. The areas that appear to differentiate these constructs are the differences in the locus of blame and the ways in which individuals relate to or are sensitive to this marginalization.

By far, the most commonly used measure of perceived stigma is Pinel’s (1999) Stigma Consciousness Scale (SCS). The SCS was originally developed to measure stigma experienced by women and was then validated and extended to several other marginalized groups, such as gay men, lesbians, and racial minorities (Pinel, 1999). These extensions of the SCS assume a global experience of stigma that applies to all marginalized groups in the same way. Since the early development of this scale, the scientific community has scarcely seen empirical investigations of this perceived stigma scale or others that can be used in current research.

Additionally, there are sampling limitations in the available research on perceived stigma. Most of the research on perceived stigma has been conducted with primarily White samples (e.g., Velez, Moradi, & Brewster, 2013). In addition, although research has included various subgroups of sexual minorities, including bisexuals (Bostwick, 2012; Brewster, Moradi, DeBlaere, & Velez, 2013), sexual minority mothers (Puckett, Horne, Levitt, & Reeves, 2011), lesbians (Lewis, Derlega, Clarke, & Kuang, 2006), and gay men (Kimmel & Mahalik, 2005), there is a dearth of research on perceived stigma with YMSM. This provides a broad literature on the usefulness of perceived stigma measures across the LGB population, but more work is needed to validate measures of perceived stigma with YMSM samples, particularly among YMSM of color.

Due to a lack of many validated measures, some researchers have used measures of experiences of enacted stigma to assess perceived stigma (Talley & Bettencourt, 2011). This conflates experienced or enacted stigma (e.g., direct marginalization, like being verbally harassed) with perceived stigma and further highlights why more measurement development and validation is needed in this area. Although these are related experiences and constructs, they are distinct in their conceptualizations and thus require separate measurement in research.

Implications of Stigma on Mental Health, Substance Use, and HIV Risk Behaviors

Internalized Homophobia and Negative Outcomes

Minority stressors, including distal and proximal, have broad implications for the mental and physical health of sexual minorities (Meyer, 1995; 2003). There is an abundance of research showing that IH can impact LGB people in a variety of ways, including stunted identity development for gay men (Rowen & Malcolm, 2002), feelings of guilt for lesbians and gay men (Moradi, van den Berg, & Epting, 2009), low self-esteem for LGB people (Herek et al., 2009), and suicidal ideation for LGB older adults (D’Augelli, Grossman, Hershberger, & O’ Connell, 2001). In addition, a meta-analysis found that higher IH was related to greater anxiety and, even more so, depression for LGB individuals (Newcomb & Mustanski, 2010).

Although there is some theoretical support that there may be a relationship between IH and substance use (Brubaker, Garrett, & Dew, 2009), the empirical literature in this area is more limited. For example, the relationship between IH and drinking may only be present for certain levels of drinking. Amadio (2006) found that moderate drinking was associated with greater IH for sexual minority women but this relationship was not present when examining higher levels of drinking. In contrast, Weber (2008) found that there were significant relations between IH, alcohol use disorders, and substance abuse in a sample of LGB adults. The research examining these relations with YMSM has been even more limited, with some researchers finding that there is not a significant relation between IH, drinking, or substance use in this group (e.g., Dudley, Rostosky, Korfhage, & Zimmerman, 2004). Given the mixed research findings with LGB adults and the lack of many empirical studies with YMSM, it is important to further explore this area.

Similar to the research on substance use, research has not shown consistent relations between IH and risky sexual behaviors. In their meta-analysis of 16 studies, Newcomb and Mustanski (2011) found a small relation between IH and sexual risk-taking for MSM, with significant variability across studies and with the association decreasing in more recently published studies. The meta-analysis and investigation of previous studies revealed several aspects that may contribute to these inconsistent findings, including issues related to measurement of IH and the use of outdated scales. This suggests the possibility that with more accurate measurement of IH using scales that are adequately validated, it could be that we would find more consistent relations with sexual risk-taking. In addition, even though there may be mixed findings on the direct association between IH and risky sexual behaviors, IH could be a moderator of relations between other variables, such as the association between enacted stigma and risky behaviors.

Perceived Stigma and Negative Outcomes

Perceived stigma has similarly been related to higher psychological distress for LGB, transgender, and queer youth (Kelleher, 2009; Velez et al., 2013), depression for LGB individuals (Berghe, Dewaele, Cox, & Vincke, 2010; Lewis, Derlega, Griffin, & Krowinski, 2003), more negative mood states and intrusive thoughts for lesbians (Lewis et al., 2006), and higher levels of anxiety for gay and bisexual men (Lelutiu-Weinberger et al., 2013). In addition, perceived stigma also has been found to be related to worse physical wellbeing for lesbians (Lewis et al., 2006) and impairments in interpersonal relationships, such as being a perpetrator of intimate partner violence among gay men and lesbians (Carvalho, Lewis, Derlega, Winstead, & Viggiano, 2011).

There has been a limited amount of research on the relation between perceived stigma and substance use or sexual risk taking. Even so, some researchers have found that higher levels of awareness of stigma are related to a greater number of days where drugs were used, as well as more high risk sex acts under the influence of substances for gay and bisexual men (Lelutiu-Weinberger et al., 2013). Additionally, sexual orientation based stigma has been associated with higher levels of HIV-related stigma for MSM (Wohl et al., 2013). This relation has important implications for health outcomes given that HIV-related stigma may decrease medication adherence (Rao et al., 2012) and increase concerns about receiving ART (Soto, Komaie, Neilands, & Johnson, 2013). Therefore, further establishing adequate measurement of perceived sexual orientation based stigma may be important to both mental and physical health outcomes for YMSM.

Current Study: Validation of Stigma Measures with YMSM

In sum, there have been various issues related to the measurement of IH and perceived stigma in research with YMSM. As mentioned above, these include conflating these constructs with other minority stressors, limited validation studies and assessments of reliability, the use of potentially outdated measures without evaluating validity in more recent samples, and a lack of assessments of the applicability and validity of scales to more racially diverse samples. Given the issues regarding measures of IH and perceived stigma, we sought to validate modified versions of existing measures of these constructs specifically with a sample of YMSM. With these evaluations of modified measures, we hope to provide scales that can be used in studies specifically with YMSM as well as with other populations when validated. We examined concurrent and discriminant validity of the measures of IH and perceived stigma by evaluating associations with related constructs (i.e., victimization, gay community connectedness), as well as predictive validity in regards to mental health, substance use, and sexual risk behavior outcomes in two samples of YMSM.

Methods

Participants and Procedures

For the present analysis, data was taken from two samples: Crew 450 and eDAPT. Crew 450 is a longitudinal community-based study examining the development of a syndemic of psychosocial stressors associated with HIV in a cohort of 450 YMSM in the Chicago area (Garofalo, Hotton, Kuhns, Gratzer, & Mustanski, 2016). The sample was recruited beginning in 2011, when participants were aged 16–20. Three waves of data were included in these analyses, collected across one year (baseline, 6-, and 12-month follow-up), with 85.8% and 80.7% retention at 6- and 12-month follow-up, respectively. Computer-assisted self-interview (CASI) technology was used for data capture via an online survey platform during in-person visits. To be included in the study, participants met the following criteria: 1) were between 16 and 20 years of age at baseline; 2) assigned male at birth; 3) spoke English; 4) had a previous sexual encounter with a man or identified as gay or bisexual; and 5) were available for follow-up for 2 years.

Participants were recruited using a modified form of respondent driven sampling (Heckathorn, 1997). There was an initial convenience sample (i.e., “seeds”; N = 172; 38.2%) who were recruited from community outreach efforts at venues frequented by YMSM, as well as school and organizational outreach, flyers posted in community settings, and through geo-social network applications. The study was approved by the Institutional Review Boards of the primary investigators’ institutions with a waiver of parental permission under 45 CFR 46.408(c) (Mustanski, 2011). In order to participate, youth provided their consent/assent. Participants were compensated for participating and were given $70 for completing the baseline surveys, which were spread across two visits, and an additional $45 at each follow-up wave of data collection.

We also utilized data from the baseline assessment of another independent sample, eDAPT, a cohort of 370 YMSM who participated in a two-month behavioral diary study of sexual behavior and substance use designed to study behavioral reactivity in this methodology (Newcomb et al., In press). CASI technology was used for data capture via an online survey platform. In order to participate, individuals had to: (1) be between the ages of 16 and 29 at baseline; (2) be assigned male at birth; (3) speak English; (4) identify as gay or bisexual; (5) report that they were HIV negative or unaware of their HIV status; and (6) report at least one instance of heavy episodic drinking or illicit drug use in the previous six months. Participants were recruited through national Facebook advertisements beginning in 2014 and they were compensated $15 for completing the baseline assessment and up to an additional $60 for completing subsequent assessments.

Measures

Demographics

In both the Crew 450 and eDAPT samples, participants reported basic demographic information, including their age, sex assigned at birth, and race/ethnicity.

Victimization

In the Crew 450 sample, at baseline, participants answered a series of 24 questions to assess their experiences of victimization. This measure was adapted from previous research (Kuhns, Vazquez, & Ramirez-Valles, 2008; Ramirez-Valles, Kuhns, Campbell, & Diaz, 2010) and included items about rejection, unfair treatment, and victimization from friends, peers, family members, and other individuals. Participants reported how often they had each experience over the previous 6 months (1 = Never, 2 = Once or twice, 3 = A few times, 4 = Many times) and the mean of their responses was computed. Cronbach’s alpha was .86 at baseline.

Internalized homophobia

In both the Crew 450 and eDAPT samples, participants were asked a series of 22 items to assess experiences of IH. This investigator-adapted scale first used five items from the Homosexual Attitudes Inventory (Nungesser, 1983), which were adapted to be more interpretable for a youth population. This is a measure frequently used to assess IH (Grey et al., 2013) and this scale has been highly correlated with other measures of IH, including one that shows post-intervention decreases in IH (Lin & Istael, 2012). This scale included items such as, “Sometimes I wish I were not gay” and “Sometimes I feel ashamed of my sexual orientation.” Next, the investigative team added 17 items to the scale in order to capture a broader conceptualization of IH, including items that tapped into experiences of shame, self-blame, normative masculinity and desire to change sexual orientation (Ramirez-Valles, Kuhns, Campbell, & Diaz, 2010). Participants indicated how much they agreed with each statement on a 4-point Likert scale from Strongly Disagree to Strongly Agree. It was administered at each time point in the Crew 450 sample and at baseline in the eDAPT sample. Responses were averaged, so that higher scores indicated greater IH. Additional information on scale construction and reliability is subsequently presented.

Perceived stigma

In both the Crew 450 and eDAPT samples, participants were asked a series of questions to assess perceived stigma. This scale was developed based on prior research with Latino gay and bisexual men (Kuhns et al., 2008). We used 11 items from the original scale that assessed non-racially-specific experiences of perceived stigma (e.g., experiences not unique to the Latino community). Some changes also were made to the wording of the items for clarity and to generalize the scale to YMSM. It was administered at baseline and the 6-month follow-up in the Crew 450 sample and at baseline in the eDAPT sample. Additional information on scale construction and reliability is subsequently presented.

Community connection

In the Crew 450 sample, a modified version of the Identification and Involvement with the Gay Community Scale (Vanable, McKirnan, & Stokes, 1998) was used to measure feelings of connectedness to the sexual minority community. This scale consisted of 11 items; however, one item assessing how often participants frequented a “gay bar” was dropped since it was likely not relevant to our underage sample. The remaining items assessed participants’ engagement with the gay community, personal connection to other lesbian, gay, or bisexual individuals, and personal importance of connection to other gay or bisexual individuals. Cronbach’s alpha in the current sample was .70 at baseline.

Alcohol use

In the Crew 450 sample, participants’ past 6-month alcohol use was assessed at baseline and the 12-month follow-up by multiplying the frequency of drinking by the quantity of drinks consumed on a typical day. This method of assessing alcohol use has been found to closely reflect daily alcohol intake in previous research (Poikolainen, Podkletnova, & Alho, 2002). Frequency and quantity of alcohol use were assessed on a 9- and 10-point ordinal scale, respectively, using the following items: “How often did you usually have any kind of drink containing alcohol?” and “How many alcoholic drinks did you have on a typical day when you drank?” Valid responses for past 6-month alcohol use ranged from 0 to 90.

Drug use

In the Crew 450 sample, to assess use of illicit substances across the lifetime and past 6 months at baseline and the 12-month follow-up, participants were asked the following: “Have you ever used [insert name of substance]?”, and “During the past 6 months, how many times did you use [insert name of substance]?” These questions were repeated with the following substances: marijuana, cocaine, methamphetamines, prescription stimulants, prescription depressants, heroin, other opiates (e.g., morphine, codeine, Demerol), MDMA (ecstasy), psychedelics (e.g., PCP, LSD, mescaline, mushrooms), gamma hydroxbutyrate (GHB), ketamine, poppers, and other inhalants (e.g., glues, spray paint, cleaning fluids). A dichotomous variable indicating regular marijuana use was defined as anyone reporting the number of times they used marijuana to be at least weekly over the past 6 months. Separately, an illicit drug use variable was created that indicated if a participant used any other illicit drug previously listed, besides marijuana, in the past 6 months (1 = any illicit drug use; 0 = no drug use).

Sexual risk taking

In the Crew 450 sample, sexual risk taking was assessed through the use of a modified version of the HIV-Risk Assessment for Sexual Partnerships (H-RASP; Mustanski, Starks, & Newcomb, 2013) at baseline and the 12-month follow-up. The H-RASP is a computerized self-administered interview designed to assess sexual behavior and associated situational/contextual variables at the level of the sexual partnership. The sexual risk behavior outcome variables used in analyses were: (1) total number of male sexual partners at each wave, and (2) total number of condomless anal sex acts with male partners at each wave. Unlike the original version of the H-RASP, this second outcome had to be derived from two separate items. This second outcome was calculated by multiplying the total number of sex acts by the frequency of condom use (always = 0%, more than half the time = 25%, half the time = 50 %, less than half the time = 75 %, never = 100%) with each reported sexual partner and then summed across the three most recent partners. All outcomes were winsorized at three SDs from the mean to reduce the effects of outliers, which is a commonly used approach for sexual behavior count data.

STI and HIV prevalence

In the Crew 450 sample, STI and HIV testing was conducted at baseline and the 12-month follow-up. Urine specimens were collected and nucleic acid amplification testing was performed to detect the presence of Neisseria gonorrhea (NG) and Chlamydia trachomatis (CT). We tested for these STIs because they are the most common among YMSM (CDC, 2009; O’Hara et al., 2012). STI prevalence, for the purpose of these analyses, was defined by the presence of either NG or CT at the 12-month follow-up visit. In addition, HIV status was self-reported and confirmed through OraQuick oral fluid tests to identify the presence of HIV antibodies at either the 6-month or 12-month follow-up visits. Participants who reported an HIV positive status at their initial visit had their status confirmed through an antiretroviral therapy (ART) prescription or a release of their medical record.

Mental health

In the Crew 450 sample, the internalizing and externalizing scales of the Achenbach System of Empirically Based Assessment–Youth Self-Report (YSR; ages 11–17) and Adult Self-Report (ASR; ages 18–59; Achenbach, 2009) were used at baseline and the 12-month follow-up to measure internalizing and externalizing mental health problems in the prior 6 months. For example, the internalizing problems scale measured such symptoms as anxiety, depression, withdrawal, and somatic complaints. The externalizing problems scale measured such symptoms as aggressive behavior and rule-breaking. For each item, participants indicated how well it described their behaviors (1 = Not True, 3 = Very True or Often True), where higher scores indicated greater levels of internalizing or externalizing problems. T scores were computed for the total scores on each scale, which allowed for comparison across the youth and adult self-report scales. For the internalizing problems scale, Cronbach’s alpha for the ASR was .94 for both the baseline and 12-month follow-up visits and YSR were .89 and .91 respectively. Similarly, for the externalizing problems scale, Cronbach’s alpha for the ASR was .91 for both the baseline and 12-month follow-up visits and YSR were .89 and .88 respectively.

Analyses

Using the Crew 450 sample, exploratory factor analysis (EFA) using oblique rotation (geomin) was conducted to examine the factor structure of the IH and perceived stigma scales. Model fit was assessed by the chi-square statistic (χ²), root mean square error of approximation (RMSEA), Comparative Fit Index (CFI), and the standardized root mean square residual (SRMR) or weighted root mean square residual (WRMR) using cutoff criteria recommended for categorical data (Schreiber, Nora, Stage, Barlow, & King, 2006). Follow-up confirmatory factor analysis was performed using the eDAPT sample to verify model fit. Both EFA and CFA utilized listwise deletion and a weighted least squares mean- and variance-adjusted estimator (WLSMV). Subsequent analyses were conducted, using the Crew 450 sample, to test for demographic differences in IH and perceived stigma scale scores, as well as to examine correlations between study variables to assess concurrent and discriminant validity. Linear, negative binomial, and logistic regression analyses were conducted to examine the ability of IH, perceived stigma, and victimization measures assessed at baseline to predict mental health, substance use, and risky sexual behavior outcomes at the 12-month follow-up visit, which provided evidence of predictive validity. Age and race were included as covariates in these multivariate regression analyses. EFA and CFA were conducted using Mplus while all other analyses were conducted using SPSS.

Results

Sample Characteristics

Crew 450

The mean age of participants at baseline was 18.9 years old (SD = 1.3), with 116 participants (25.8%) less than 18 years old. The racial/ethnic composition of the sample was: 53.3% Black/African Americans, 20.0% Hispanic/Latinos, 18.0% White and 8.7% who identified as some other race/ethnicity. Participants ranged in the sexual orientations they reported at baseline, with about half of the sample (50.2%) identifying as only gay/homosexual, and an additional 22.9% identifying as mostly gay/homosexual, 21.3% bisexual, 2.4% mostly heterosexual, 0.7% only heterosexual, and 2.4% who chose “other” as their sexual orientation.

eDAPT

The mean age of participants at baseline was 22.9 years old (SD = 3.2), with 15 participants (4.1%) less than 18 years old. The racial/ethnic composition of the sample was: 45.7% White, 23.0% Hispanic/Latinos, 19.5% Black/African Americans, and 11.9% who identified as some other race/ethnicity. The sample consisted of participants who identified as gay (87.8%), bisexual (11.4%) or as some other sexual orientation (0.8%).

Internalized Homophobia – Factor Analysis

The factor structure for the 22-item IH scale was first parsed by EFA using WLSMV estimation with an oblique rotation (geomin) with the Crew 450 sample (see Table 1). Initially, a five factor solution was identified by extracting factors with eigenvalues greater than one resulting in a solution with strong fit (χ² (131) = 231.2, p < .001; RMSEA = .041; CFI = .973; SRMR = .024). However, prior to performing CFA, the fourth and fifth factors were dropped because only two items loaded highly on each of these factors and the items were not determined to be face valid. Additional items were dropped either due to a low factor loading (“I try to look masculine in order to avoid people’s rejection.”), a low communality along with poor representation of the latent construct (“Gay people are promiscuous.”), or because they cross loaded (loading difference < .10) on multiple factors (“I would like to get professional help in order to change my sexual orientation.”). In evaluating these items, we were cognizant of the need to establish a measure that more closely related to the definition of IH rather than correlates (Shidlo, 1994). The remaining 15-items formed three factors with 60.0% of the total variance explained: Desire to be Heterosexual (subscale 1), Fear of Coming Out (subscale 2), and Fear of Stereotypical Perception (subscale 3). The three-factor structure for the remaining items was verified by conducting an unrotated CFA with WLSMV estimating using the eDAPT sample and examining model fit statistics (χ² (87) = 272.7, p < .001; RMSEA = .077; CFI = .975; WRMR = 1.046). A scale score was derived for each factor by calculating the mean of the individual items.

Table 1.

Internalized Homophobia Exploratory and Confirmatory Factor Analyses

Subscale	Item	EFA					CFA
		F1	F2	F3	F4	F5	F1	F2	F3
Desire to be Hetero-sexual	1. Sometimes I wish I were not gay.	0.82	−0.05	0.04	−0.04	−0.06	0.88
	2. Most of the time I am glad to be gay.	0.41	0.16	−0.06	0.11	−0.02	0.77
	4. If people my age knew my sexual orientation, I’m afraid that many would not want to be my friends.	0.86	−0.12	−0.01	0.00	−0.02	0.87
	5. If there were a pill to make me straight I would take it.	0.73	0.00	−0.04	0.09	0.06	0.90
	6. I have tried to stop being attracted to men.	0.59	0.03	0.13	−0.18	0.18	0.61
	8. I’d like to get professional help in order to change my sexual orientation.	0.32	0.27	0.02	0.18	0.04	--
	9. Sometimes I wish I could become more sexually attracted to women.	0.69	0.14	0.02	0.02	−0.03	0.81
	12. I feel that being gay is a shortcoming for me.	0.36	0.12	−0.01	0.23	0.17	0.76
	13. Sometimes I feel ashamed of my sexual orientation.	0.54	0.18	0.02	0.05	0.07	0.85
Fear of Coming Out	3. If people my age knew my sexual orientation, I’m afraid that many would not want to be my friends.	0.07	0.49	0.29	−0.03	0.00		0.71
	14. If my straight friends knew about my sexual orientation I would feel uncomfortable.	0.02	0.63	0.22	0.13	0.02		0.92
	15. I’m afraid my family and friends will find out about my sexual orientation.	0.02	0.52	0.20	0.11	0.01		0.77
	16. Most of the time I’m comfortable with people knowing about my sexual orientation.	−0.03	0.52	−0.02	−0.08	−0.03		0.67
	17. Gay people are promiscuous.	0.03	−0.38	0.12	0.30	0.24		--
Fear of Stereo. Percept.	7. Whenever I tell my straight friends about my sexual orientation, I worry they will try to remember things about me that appear to fit the stereotype of a homosexual.	0.15	−0.02	0.62	−0.01	−0.07			0.82
	18. When I think about coming out to a straight friend, I worry that she or he might watch me to see if I do things that are stereotypically homosexual.	−0.02	−0.02	0.81	0.01	−0.01			0.87
	21. I try to look masculine in order to avoid people’s rejection.	0.11	0.17	0.28	0.11	0.16			--
	22. When I think about coming out to a straight friend, I’m afraid they will pay more attention to my body movements and voice than to me, the person.	−0.04	0.08	0.64	0.04	0.18			0.92
	10. Men who look or act too effeminate make me feel uncomfortable.	0.10	−0.01	0.07	0.52	0.02
	11. It is important for men to look and behave in a masculine way.	0.00	0.07	0.03	0.69	−0.01
	19. Being gay means you will be alone when you grow old.	0.06	0.22	0.07	−0.11	0.38
	20. Gay people are to blame for society’s attitudes towards us.	−0.01	−0.06	−0.06	0.03	0.78

Perceived Stigma – Factor Analysis

EFA was performed to determine the factor structure of the 11-item perceived stigma measure using the Crew 450 sample (see Table 2). Prior to analysis, the item “Gay men are not welcome in most churches or religions” was determined to not measure the construct of interest and thus was removed. A two factor solution was initially extracted based on the number of factors with eigenvalues greater than one. EFA results indicated that a two factor solution was a better fitting model compared to a one factor solution (χ² (9) = 147.5, p < .001); however, since investigators did not theoretically anticipate this construct to have multiple latent variables, nor were the two factors interpretable, a one factor solution was further examined. The model fit of this one factor solution using 10 items was inadequate (χ² (35) = 308.1, p < .001; RMSEA = .132; CFI = .927; SRMR = .066), so items that loaded onto a second factor (“Many people believe that gay men have too much sex,” and “Many people believe that gay men abuse drugs and alcohol”) were dropped and a second iteration of an EFA was performed that resulted in a model with moderate fit (χ² (20) = 118.6, p < .001; RMSEA = .105; CFI = .972; SRMR = .043). In an effort to increase model fit, modification indices were examined and a single item was dropped (“Many people have negative attitudes toward homosexuality.”) resulting in an improved model (χ² (14) = 40.4, p < .001; RMSEA = .065; CFI = .991; SRMR = .029) with 60.2% of the total variance explained. This factor structure was verified by CFA using the eDAPT sample and examining model fit statistics (χ² (14) = 49.0, p < .001; RMSEA = .083; CFI = .987; WRMR = 0.784). A scale score was derived by calculating the mean of the individual items.

Table 2.

Perceived Gay-Related Stigma Exploratory and Confirmatory Factor Analyses

Item	EFA		CFA
	F1	F2
1. Many people believe that gay men have psychological problems.	0.45	0.19	0.53
2. Many people have negative attitudes toward homosexuality.	0.85	−0.16	-
3. Many people do not see gay men as real men.	0.78	−0.01	0.70
4. Gay men are not welcome in most churches or religions.	-	-	-
5. Most families would be disappointed to have a gay son.	0.62	0.03	0.65
6. Many people think that gay men have HIV and will die of AIDS.	0.61	0.18	0.61
7. Many people believe that gay men have too much sex.	0.12	0.67	-
8. Many people believe that gay men abuse drugs and alcohol.	−0.01	0.74	-
9. Many people do not accept same-sex male couples.	0.86	−0.02	0.81
10. Many people believe that gay men should not raise children.	0.73	0.17	0.91
11. Many people believe that gay men should not hug, hold hands, or kiss in public.	0.71	0.19	0.80

Demographic Differences

Racial/ethnic differences were found for the Desire to be Heterosexual [F(3, 446) = 3.26, p = .021], Fear of Coming Out [F(3, 446) = 3.61, p = .013], and Fear of Stereotypical Perception [F(3, 446) = 4.89, p = .002] subscales of the IH measure, as well as the perceived stigma scale [F(3, 446) = 4.01, p = .008]. Tukey’s post-hoc analyses indicated that White participants had significantly lower scores on the perceived stigma scale compared to the Black (p = .016) and Other (p = .018) racial/ethnic groups. In addition, White participants reported significantly lower scores on the Fear of Coming Out subscale (p = .018) compared to Black participants. And Latino participants reported significantly lower scores on the Fear of Stereotypical Perception subscale compared to the White (p = .032) and Other (p = .004) racial/ethnic groups. Neither Tukey’s nor any other post-hoc analysis identified any significant differences among the racial/ethnic groups for the Desire to be Heterosexual subscale. Also, no significant correlations were found between age and the Desire to be Heterosexual IH subscale [r(448) = 0.04, p = .41], Fear of Coming Out IH subscale [r448) = −0.04, p = .42], Fear of Stereotypical Perception IH subscale [r(448) = −0.06, p = .19], or the perceived stigma scale [r(448) = 0.01, p = .98].

Reliability of IH and Perceived Stigma Scales

The reliability of the IH subscales was assessed at baseline, 6-month, and 12-month follow up in the Crew 450 sample. These analyses indicated fairly stable Cronbach’s alphas across the time points, particularly for the Desire to be Heterosexual subscale, which had alphas of .88, .90, and .89 across the three measurements. The second subscale, Fear of Coming Out, Stereotypical Perception, had alpha levels of .77, .80, and .81 across the three time points. The reliability of the perceived stigma scale was .85 at both baseline and 6-month follow-up in the Crew 450 sample. Test-retest reliability was assessed by calculating a one-way random effects intraclass correlation coefficient (ICC) with single rating estimates presented. ICCs indicate moderate test-retest reliability for the Desire to be Heterosexual IH subscale (ICC=.71), Fear of Coming Out IH subscale (ICC=.60), Fear of Stereotypical Perception IH subscale (ICC=.56), and perceived stigma scale (ICC=.41).

Convergent and Discriminant Validity

Correlations of the study constructs at baseline were examined to assess convergent and discriminant validity of the scales measuring IH and perceived stigma (see Table 3). Convergent validity refers to the degree to which the scales measure similar constructs, whereas discriminant validity refers to the ability of the scales to distinguish between theoretically distinct constructs (Campbell & Fiske, 1959; Cronbach & Meehl, 1955). In assessing validity, we utilized Cohen’s recommendations to describe the size of the effect – with correlations of .10 indicating a small effect size, .30 indicating a medium effect size, and .50 indicating a large effect size (Cohen, 1992).

Table 3.

Correlation Matrix and Descriptive Statistics of Study Variables at Baseline

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
1. Desire to be Heterosexual	--
2. Fear of Coming Out	.54**	--
3. Fear of Stereo. Perception	.48**	.52**	--
4. Perceived Stigma	.19**	.07	.18**	--
5. Victimization	.19**	.22**	.27**	.34**	--
6. Gay Community Scale	−.32**	−.25**	−.07	.02	.14**	--
7. Internalizing Problems	.17**	.04	.11*	.06	.20**	.01	--
8. Externalizing Problems	.08	−.03	.03	.07	.20**	.11*	.66**	--
9. Male Sex Partners	−.04	−.01	.01	−.04	.16**	.16**	.03	.16**	--
10. Condomless Anal Sex Acts	−.10*	−.10*	−.12*	.00	−.04	.05	.04	−.01	.08	--
11. STI Prevalence	−.05	−.02	−.06	.02	.00	.09	.06	.07	.09	.00	--
12. HIV Status	−.02	−.01	−.07	−.03	−.10*	−.02	−.04	−.06	.05	.03	.04	--
13. Alcohol Use	.00	−.08	−.02	.03	.03	.07	.04	.27**	.16**	.03	.02	−.10*	--
14. Marijuana Use	.04	−.03	−.03	−.04	.06	.05	.06	.21**	.10*	.05	.08	.07	.31**	--
15. Hard Drug Use	.01	−.04	−.03	−.07	.02	.03	.12**	.19**	.14**	.09	.09	−.05	.31**	.28**	--
Mean	1.93	1.77	2.20	3.08	1.59	3.02	54.62	54.73	3.93	5.86	--	--	10.00	--	--
SD	0.69	0.70	0.80	0.60	0.41	0.66	12.18	10.63	5.17	13.87	--	--	12.37	--	--
Proportion	--	--	--	--	--	--	--	--	--	--	8.2	7.6	--	25.6	18.0

Note. IH = Internalized Homophobia;

^*p <.05;

^**p < .01

To assess convergent validity, we first examined the intercorrelations of the three subscales of the IH measure as these were developed to measure different aspects of the same construct of IH. The three IH subscales were strongly intercorrelated, ranging from r (448) = .48 to .54 (p < .01), and these large effects reflected good convergent validity. We then evaluated the relations between these subscales and perceived stigma, as conceptually these variables all relate one’s internal experience of holding a stigmatized identity as a sexual minority. The Desire to be Heterosexual subscale and the Fear of Stereotypical Perceptions subscale were related to perceived stigma, r(448) = .19 and .18, respectively, p < .05. In contrast, there was not a significant association between the Fear of Coming Out subscale and perceived stigma.

After evaluating convergent validity, we examined discriminant validity by assessing the relations between IH and perceived stigma with victimization and gay community connection. Victimization and gay community connection are constructs that are unique to sexual minorities but do not represent processes by which sexual minorities interpret stigma. The findings indicated that victimization was associated with greater IH across the three subscales (r(448) = .19 to .27), as well as perceived stigma (r(448) = .34). The Desire to be Heterosexual and Fear of Coming Out subscales were the only IH subscales significantly associated with gay community connectedness (r(448) = -.32 and -.25, respectively). Perceived stigma was not significantly associated with gay community connectedness. These mixed findings reflect that our measures of IH and perceived stigma tap into different constructs that show differential associations with related variables.

Predictive Validity Analyses

Predictive validity was assessed through the use of multivariate regression. Baseline IH and perceived stigma were examined for their ability to predict mental health, substance use, and sexual risk taking outcomes at the 12-month follow-up, while controlling for baseline age, race, and experiences of victimization. For mental health outcomes, multivariate linear regression was performed. The Desire to be Heterosexual IH subscale (at baseline) was related to greater internalizing mental health problems at the 12-month follow-up (B = 3.81; p < .01). Neither of the other subscales of the IH measure or the perceived stigma measure at baseline were associated with internalizing problems at the 12-month follow-up. This provided some evidence of predictive validity for the Desire to be Heterosexual IH subscale when examining mental health outcomes. When examining the ability of the IH and perceived stigma scales to predict externalizing mental health problems, neither emerged as a significant predictor. However, more experiences of victimization at baseline were related to more externalizing problems at the 12-month follow-up (B = 3.36; p < .05). The regression coefficients and confidence intervals for these analyses are provided in Table 4.

Table 4.

Multivariate Linear Regressions for Mental Health and Substance Use Outcomes

	ASR/YSR Internalizing Problems A			ASR/YSR Externalizing Problems A			Alcohol Use A			Marijuana Use B			Hard Drug Use B
	Unstan. Coeff.	95% CI		Unstan. Coeff.	95% CI		Unstan. Coeff.	95% CI		OR	95% CI		OR	95% CI
		Lower	Upper		Lower	Upper		Lower	Upper		Lower	Upper		Lower	Upper
Age	0.32	−0.74	1.37	−0.14	−1.01	0.74	0.44	−0.35	1.24	0.91	0.75	1.10	1.07	0.85	1.35
Race (ref=Black)
White	5.84**	2.17	9.51	5.77**	2.73	8.81	10.52**	7.73	13.31	1.94*	1.03	3.66	12.61**	5.80	27.43
Latino	4.65**	1.29	8.01	2.30	−0.49	5.08	4.31**	1.76	6.86	0.93	0.49	1.77	4.45**	2.05	9.66
Other	7.91**	3.18	12.63	4.45*	0.53	8.36	5.55**	2.00	9.11	1.57	0.70	3.54	3.14*	1.14	8.69
Desire to be Heterosexual	3.81**	1.42	6.19	1.39	−0.59	3.37	1.34	−0.46	3.15	1.32	0.85	2.03	1.25	0.75	2.07
Fear of Coming Out	−0.15	−2.68	2.39	−0.34	−2.44	1.76	−1.99*	−3.91	−0.08	0.77	0.48	1.23	0.93	0.54	1.60
Fear of Stereo. Perception	−0.44	−2.54	1.66	−0.90	−2.64	0.84	1.39	−0.17	2.96	1.01	0.70	1.47	1.17	0.75	1.83
Perceived Stigma	−0.37	−2.83	2.09	−0.91	−2.95	1.13	−0.53	−2.37	1.32	0.92	0.59	1.44	1.22	0.69	2.15
Victimization	2.02	−1.49	5.52	3.36*	0.46	6.27	−1.31	−3.92	1.30	1.10	0.59	2.06	0.87	0.40	1.90

Note.

^*p < .05,

^**p < .01

OR = Odds Ratio. A = Multivariate linear regressions. B = Logistic Regressions. Independent variables were measured at baseline and the dependent variables were measured at the 12-month follow up.

For substance use outcomes, linear regression and logistic regression were performed. The Fear of Coming Out subscale of the IH measure at baseline was related to decreased frequency of alcohol use at the 12-month follow-up (B = −1.99; p < .05). No other subscales of the IH measure were related to alcohol use quantity-frequency. In addition, neither the IH nor the perceived stigma scale was related to marijuana or hard drug use. The regression coefficients, odds ratios, and confidence intervals for these analyses are provided in Table 4.

For sexual risk taking outcomes, negative binomial regression and logistic regression were performed. The IH and perceived stigma scales at baseline were not related to the total number of male sex partners at 12-month follow-up. The Fear of Stereotypical Perceptions subscale of the IH measure at baseline was related to fewer condomless anal sex acts at the 12-month follow-up (B = .57; p < .01). Consistent with the effect for behavior, the Fear of Stereotypical perceptions subscale of the IH measure was associated with STI prevalence at the 12-month follow-up (B = 0.36; p < .01), such that higher fear was associated with a lower probability of having an STI. Perceived stigma displayed the opposite pattern, with higher perceived stigma at baseline associated with a higher probability of having an STI at the 12-month follow-up (B = 2.64; p < .05). The IH and perceived stigma scales at baseline were not related to HIV status at the 12-month follow-up. The regression coefficients and confidence intervals for these analyses are provided in Table 5.

Table 5.

Multivariate Negative Binomial Regression and Logistic Regression for Sex Risk and HIV/STI Outcomes

	Total Male Sex Partners A			Male Condomless Anal Acts A			HIV Status B			STI B
	Exp(B)	95% CI		Exp(B)	95% CI		Exp(B)	95% CI		Exp(B)	95% CI
		Lower	Upper		Lower	Upper		Lower	Upper		Lower	Upper
Age	1.12*	1.02	1.24	1.25*	1.02	1.53	0.95	0.64	1.41	0.97	0.70	1.36
Race (ref=Black)
White	1.64**	1.19	2.26	1.42	0.75	2.68	0.00	0.00	-	0.28	0.04	2.22
Latino	1.51*	1.09	2.09	0.96	0.53	1.73	0.42	0.11	1.58	0.73	0.25	2.16
Other	1.11	0.71	1.74	2.20	0.83	5.81	0.71	0.14	3.47	2.44	0.76	7.87
Desire to be Heterosexual	1.12	0.90	1.39	1.29	0.76	2.20	1.33	0.56	3.15	1.42	0.70	2.88
Fear of Coming Out	0.97	0.75	1.24	0.91	0.57	1.46	0.63	0.24	1.69	1.71	0.77	3.81
Fear of Stereo. Perception	1.17	0.97	1.42	0.57**	0.39	0.83	1.38	0.63	3.01	0.36**	0.17	0.76
Perceived Stigma	0.86	0.67	1.12	1.13	0.72	1.78	1.23	0.47	3.21	2.64*	1.10	6.31
Victimization	0.90	0.63	1.29	1.13	0.53	2.42	0.17*	0.03	0.99	1.12	0.40	3.11

Note.

^*p < .05,

^**p < .01

A = Negative Binomial Regression. B = Logistic Regressions. Independent variables were measured at baseline and the dependent variables were measured at the 12-month follow up.

Discussion

There are various levels of stigma experienced by sexual minorities, including enacted stigma, perceived stigma, and internalized stigma/IH (Herek et al., 2009). Research related to the validity of perceived stigma and IH measures has been limited due to the varied operationalization of these constructs and the use of measures developed for different sub-populations which are then applied to all sexual minorities without further validation of these applications. In addition, many of the commonly used measures of IH provided evaluation of these scales, but during periods of drastically different social times and further investigating their continued validity and reliability could bolster support for their continued use. This study provides validation of adapted IH and perceived stigma measures for use with YMSM, as well as further documents some of the negative outcomes of these minority stressors in longitudinal analyses.

Validation of the Internalized Homophobia Measure

The validity analyses revealed that the three subscales of the IH measure were strongly correlated, suggesting that they measure related experiences. However, compared to the other factors, the Desire to be Heterosexual subscale of the IH measure most closely reflected the definition of IH as negative feelings and homophobic attitudes towards one’s self as a product of social bias against sexual minorities (Shidlo, 1994). This subscale captured the internalization of negative attitudes most effectively, and included items such as “I feel that being gay is a shortcoming for me” and “Sometimes I feel ashamed of my sexual orientation.” In contrast, subscales 2 and 3 more closely reflected related but distinct experiences of minority stressors. Subscale 2 reflected fears of coming out to others and included items such as “If my straight friends knew about my sexual orientation I would feel uncomfortable.” Subscale 3 measured worries about stereotypes based on sexual orientation. For example, this factor included items such as “When I think about coming out to a straight friend, I worry that she or he might watch me to see if I do things that are stereotypically homosexual.” In addition, the Desire to be Heterosexual subscale was the only subscale associated with mental health both at baseline and in the longitudinal analyses, further supporting this as a measure of IH in comparison to the remaining two subscales.

Based on the current analyses, we recommend only using the Desire to be Heterosexual subscale as a measure of IH (see Appendix A). With regard to subscales 2 and 3, we recommend that they be explored further for other purposes – namely to measure fears of coming out and fears of being stereotyped, respectively. This suggestion to only use the Desire to be Heterosexual subscale as a measure of IH also is supported by the validity analyses, which revealed different associations among the subscales and other study constructs (e.g., Desire to be Heterosexual was associated with perceived stigma, whereas Fear of Coming Out was not).

Although fears of rejection can be related to experiencing IH, this is more accurately conceptualized as a correlate instead of capturing IH specifically. The items could reflect actual risks of rejection in participants’ social circles instead of manifestations of IH. For example, it could be that someone scores high on these subscales because he has experienced heterosexist comments from others in the past which may make him worry about how others will react or perceive him. In this situation, a higher score does not necessarily mean that the individual himself feels negatively about being attracted to the same-sex, but rather his life experiences have resulted in higher levels of worry about others’ reactions. In contrast, even though the same experience may relate to a greater desire to be heterosexual, this type of increase would be indicative of a personal discomfort with one’s self (IH) rather than a fear of persecution by others. In addition, the Fear of Coming Out subscale had lower alpha levels across the study (ranging from .62 to .74) and this could be indicative of a need to further evaluate this subscale for consistency of measurement. However, this might be due to the changing nature of outness as youth were likely more out about their sexual orientations over time and this would impact their fear or apprehension about coming out and their responses on this subscale.

Validation of the Perceived Stigma Measure

The factor analysis of the perceived stigma scale revealed a single factor structure that supported retaining most of the items (see Appendix B). This scale’s items included statements such as “Many people do not see gay men as real men” and “Many people do not accept same-sex couples.” This scale mapped on well to the construct of perceived stigma as representing an awareness of negative attitudes that are held towards sexual minorities. This scale also included items that were similar in nature to those on the Stigma Consciousness Questionnaire (Pinel, 1999; e.g., “Most heterosexuals have a problem viewing homosexuals as equals.”), which further supports the use of our scale in measuring this construct. In addition, this study supported the use of this scale with a sample of YMSM, significantly expanding the literature beyond only males who identify as gay. Future research evaluating the effects of perceived stigma may benefit from using this scale, which represents a singular construct of stigma consciousness. Future research is needed evaluating this scale in relation to Pinel’s Stigma Consciousness Scale to further support the use of both measures as evaluations of perceived stigma.

Mental Health and Stigma

This study provides mixed support for the relations between IH and perceived stigma with mental health. For example, the Desire to be Heterosexual subscale was related to internalizing problems on the ASR/YSR, consistent with previous literature supporting the relation between IH and internalizing mental health problems for LGB individuals (Newcomb & Mustanski, 2010). We believe this to be indicative of this being a more accurate subscale to measure IH than the remaining subscales, as there was not a significant association between mental health and fear of coming out or fears of stereotypical perceptions in the longitudinal analyses. Theoretically, if all three subscales were to measure the underlying construct of IH, similar associations should hold across subscales over time. As mentioned, it is likely that the subscales assessing fears of coming out and fears of stereotypical perceptions are assessing correlates of IH that may be more in flux and thus have different longitudinal findings. For example, it could be that participants became more out about their sexual orientations over the course of the year and thus there are different findings for this subscale.

With regard to perceived stigma, there were no significant relations with internalizing or externalizing problems, which contrasts previous cross-sectional findings of the association between perceived stigma and greater psychological distress (e.g., Kelleher, 2009; Velez et al., 2013). This could be for a number of reasons. For example, in contrast to IH, simply being aware of stigma may not have as detrimental of an outcome. For individuals who are aware of stigma, this does not indicate that they also internalize these stigmatizing messages (Herek, Gillis, & Cogan, 2009), which may be the critical process that impacts mental health. For some individuals even, being aware of the larger negative sociopolitical context that they are living may help them to detach from negative self-judgments or help them to take action to make positive changes in their environments. This is similar to the development of a critical consciousness that marginalized groups can develop in the face of oppression (Freire, 1970). It could be that this builds a consciousness of stigma that allows some individuals to externalize blame and not develop a negative self-view that would be related to more psychological distress.

Alcohol/Drug Use and Stigma

Our findings about the relations between IH, perceived stigma, and alcohol/drug use were mixed, which is consistent with prior literature (e.g., Amadio, 2006; Dudley, Rostosky, Korfhage, & Zimmerman, 2004). We found that the Fear of Coming Out subscale of the IH measure was related to lower levels of drinking, whereas the remaining subscales were not associated with drinking and no subscales were related to drug use. These findings could reflect that individuals who are less out (and therefore would be more likely to score higher on fears of coming out) are also less likely to be connected to a gay community (Frost & Meyer, 2009) and subsequently the associated bar cultures which have been associated with community connectedness (Holloway et al., 2012).

With regard to perceived stigma, this scale was not related to alcohol or drug use, inconsistent with some previous findings (e.g., Lelutiu-Weinberger et al., 2013). It is interesting to note this lack of a significant association as previous research has shown that living within more oppressive climates (e.g., states with gay marriage bans), which arguably would be locations where individuals hold a greater stigma consciousness, is associated with greater odds of having an alcohol use disorder (Hatzenbuehler, McLaughlin, Keyes, & Hasin, 2009). It is possible that there are differences in the magnitude and saliency of one’s stigma consciousness depending on the type of context one is living within that should be taken into account in future research assessing alcohol and drug use for YMSM.

Research is needed to continue exploring potential associations between alcohol use and drug use with IH and perceived stigma. As previous research has shown, it could be that there are significant relations but that these only exist at certain levels of drinking or drug use (Amadio, 2006). It also could be that IH and perceived stigma are important moderators of other relations, even though they may not have a direct effect on alcohol use or drug use. For example, it could be that victimization is related to increased alcohol or drug use when participants also have high levels of IH. Given the limited literature in this area and the mixed findings, future research is needed that examines some of the nuances of how IH and perceived stigma may play a role in alcohol and drug use.

Sexual Risk Taking and Stigma

The relations between stigma and sexual risk taking also were mixed. After controlling for demographic variables, neither IH nor perceived stigma was related to the participants’ total number of sex partners during the prior six months or HIV status. However, lower scores on the Fear of Stereotypical Perceptions subscale and higher scores on perceived stigma were related to greater prevalence of STIs. The association between only this subscale of the IH measure and STI prevalence may shed some light on why there have been mixed findings in previous work on the relations between IH and sexual health outcomes (Newcomb & Mustanski, 2011). It could be that because some scales include what we see as correlates of IH (such as fear of others’ reactions or perceptions), these studies may find different results than those that may measure more of the core construct of IH, which we propose is measured more closely by the Desire to be Heterosexual subscale and which was not associated with sexual risk taking. This further supports the need for more consistency in the measurement of IH to improve the literature on predictors of sexual risk taking.

Limitations

This study provides empirical support for measures of IH and perceived stigma with YMSM, including a factor analysis of the scales and evidence of convergent and discriminant validity. Although empirically validated measures of these minority stress concepts is needed, this study is not without limitations. For one, even though our validation study provides evidence for the use of these scales, we recommend that future research more closely evaluate their convergent validity with other established measures of IH and perceived stigma. Additionally, our study evaluated the modification of existing scales of IH and perceived stigma and it may be useful for future work to evaluate the creation of new scales following guidelines for scale development (e.g., Clark & Watson, 1995; John & Benet-Martinez, 2000).

Our samples, while diverse, also had limitations. Our young samples, with one being from an urban area, may not represent all YMSM in the population. Additionally, our samples did not include females and instead focused on the experiences of males. Because these scales were specific to YMSM, they may not be applicable to sexual minority women (cisgender and transgender women), transgender men, or other gender minorities (e.g., genderqueer individuals). The samples across the two studies also differed in their ages and in the likelihood that they identified as gay, and replication of the results with more similar samples would further support our findings.

Most importantly though, there were some limitations with our findings. It is essential to note that our lack of significant associations between baseline assessments of two of the IH subscales and perceived stigma with several of the outcomes (e.g., mental health and substance use) may have been a function of time as these outcomes were assessed 12 months later. Due to this, these findings need to be interpreted with caution. Lastly, our study could only define sexual risk taking as the absence of condom use during anal sex. However, since Crew 450 began, both PrEP use and treatment as prevention strategies have become more prevalent among the MSM population. Due to a lack of data, we were unable to account for these newer safer sex practices in these analyses; however, future studies would benefit by incorporating this type of information when calculating sexual risk taking.

Future Directions

There are a number of future research directions supported by this study. First and foremost, more work is needed to establish cohesiveness in the literature on IH and perceived stigma. As highlighted earlier, these constructs have varied widely in their measurement and at times have been conflated with other constructs. Also, it may be beneficial to focus on more nuanced analyses that examine how IH and perceived stigma relate to risky behaviors of substance use and risky sex. For example, IH may be related to greater sexual risk in the context of sex with casual partners and not when in a serious relationship. For perceived stigma, it could be that it is not an awareness of stigma that is especially important to predicting negative outcomes. Instead, it could be having expectations that one will actually experience that stigma that lead to negative mental health and engagement in risky behaviors. Similarly, individuals may have varying levels of sensitivity to their being a member of a marginalized group and constructs such as rejection sensitivity (e.g., Cohen, Feinstein, Rodriguez-Seijas, Taylor, & Newman, 2016) may be promising avenues for future research. This type of distinction should be made by further developing macro- and micro-level measures of perceived stigma that will assess perceptions of broader societal beliefs, as well as expectations that individuals will directly encounter stigma in their lives.

Appendix A: Measure of Internalized Homophobia

Instructions

We are interested in how you feel about the following statements. Read each statement carefully and indicate how strongly you agree or disagree with the following statements.

Response options

1 = Strongly Disagree, 2 = Disagree, 3 = Agree, 4 = Strongly Agree

1. Sometimes I wish I were not gay.

2. Most of the time, I am glad to be gay.

3. Sometimes I think that if I were straight, I would probably be happier.

4. If there were a pill to make me straight I would take it.

5. I have tried to stop being attracted to men.

6. Sometimes I wish I could become more sexually attracted to women.

7. I feel that being gay is a shortcoming for me.

8. Sometimes I feel ashamed of my sexual orientation.

Appendix B: Measure of Perceived Stigma

Instructions

In this section we want to know what you think about OTHER people’s attitudes and beliefs. Using the response scale, please answer according to your point of view, feelings and experiences.

Response options

1 = Strongly Disagree, 2 = Disagree, 3 = Agree, 4 = Strongly Agree

1. Many people believe that gay men have psychological problems.

2. Many people do not see gay men as real men.

3. Most families would be disappointed to have a gay son.

4. Many people think that gay men have HIV and will die of AIDS.

5. Many people do not accept same-sex male couples.

6. Many people believe that gay men should not raise children.

7. Many people believe that gay men should not hug, hold hands, or kiss in public.