Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Nov 9.
Published in final edited form as: Arch Sex Behav. 2015 Aug 21;45(5):1269–1277. doi: 10.1007/s10508-015-0605-2

Correlates of a Single-Item Indicator Versus a Multi-Item Scale of Outness About Same-Sex Attraction

J Michael Wilkerson 1,✉, Syed W Noor 2, Dylan L Galos 3, B R Simon Rosser 3
PMCID: PMC5679701  NIHMSID: NIHMS916987  PMID: 26292840

Abstract

In this study, we investigated if a single-item indicator measured the degree to which people were open about their same-sex attraction (“out”) as accurately as a multi-item scale. For the multi-item scale, we used the Outness Inventory, which includes three subscales: family, world, and religion. We examined correlations between the single- and multi-item measures; between the single-item indicator and the subscales of the multi-item scale; and between the measures and internalized homonegativity, social attitudes towards homosexuality, and depressive symptoms. In addition, we calculated Tjur’s R2 as a measure of predictive power of the single-item indicator, multi-item scale, and subscales of the multi-item scale in predicting two health-related outcomes: depressive symptoms and condomless anal sex with multiple partners. There was a strong correlation between the single- and multi-item measures (r = 0.73). Furthermore, there were strong correlations between the single-item indicator and each subscale of the multi-item scale: family (r = 0.70), world (r = 0.77), and religion (r = 0.50). In addition, the correlations between the single-item indicator and internalized homonegativity (r = −0.63), social attitudes towards homosexuality (r = −0.38), and depression (r = −0.14) were higher than those between the multi-item scale and internalized homonegativity (r = −0.55), social attitudes towards homosexuality (r = −0.21), and depression (r = −0.13). Contrary to the premise that multi-item measures are superior to single-item measures, our collective findings indicate that the single-item indicator of outness performs better than the multi-item scale of outness.

Keywords: Male homosexuality, Outness, Sexual orientation, Internalized homophobia, Self-acceptance

Introduction

Researchers studying lesbian, gay, or bisexual populations frequently wish to assess the extent to which study participants are open about their same-sex attraction (“out”) to other people. However, the number of articles discussing how to measure outness or examining the psychometric properties of scales attempting to measure this construct is limited. Measures of outness tend to ask participants about the degree to which they are open about their same-sex attraction (out), and it is common for outness to be assessed using a single item. Like other researchers (Koblin et al., 2011), we have used a single-item indicator of outness in our research (Rosser, Bockting, Ross, Miner, & Coleman, 2008; Wilkerson, Smolenski, Brady, & Rosser, 2012). However, for some time, there have been concerns about the validity of this single-item indicator, specifically whether it accurately captures the critical factors conceptualized as being a part of the outness construct or whether it should be abandoned in favor of a multi-item scale.

There are good reasons to prefer a multi-item scale over a single-item indicator. In the psychometric literature, it is almost a maxim that the more items in self-report measures, the better the measure (Gardner, Cummings, Dunham, & Pierce, 1998). For example, Nunnally (1978) concludes that “other things being equal, a long test is a good test” (p. 243), while Diamantopoulos, Sarstedt, Fuchs, Wilczynski, and Kaiser (2012) caution against using single-item measures in empirical research whenever possible. To test this assumption of good practice, researchers need to conduct comparative studies to validate measures. Against the preference for longer surveys, Internet researchers, in particular, face strong demands for shorter surveys and concerns about subject burden (Konstan & West, 2010; Pequegnat et al., 2007). As Gardner et al. (1998) note, the recommendation for longer scales is ultimately based on the domain-sampling model of measurement error, which assumes that any given test is “composed of a random sample of items from a hypothetical domain of all items that measure the construct of interest” (p. 899). However, they note that researchers seldom construct scales from a random sample of items; instead, they usually create items based directly on their understanding and operationalization of the construct under study. One “good” item might perform better than many “poorer” items; hence, the need for reliability and validity studies.

Outness

In studying outness, another challenge is language. Historically, “coming out” was a term denoting when a young woman was formally presented at court as a debutante, thus becoming eligible to date, marry, and participate in society. In the nineteenth century, Urlichs applied the concept to homosexuality, arguing that homosexual people needed to reveal their same-sex attractions to “come out” into gay society and to challenge and change public hostility (Kennedy, 1997). “Outness,” “openness,” and “disclosure” are all similar terms that have been used in the sexuality literature. While different disciplines might use these terms in distinct ways (e.g., the mental health literature may prefer to use “open” to discuss subjective states; the political discourse, “out” to address political activism; and the legal literature, “disclosure” to describe the act of explicitly revealing such information), we use “out” as the term most commonly used by the target population, men who have sex with men (MSM).

The academic literature includes several theoretical models of the coming out process. Hencken and O’Dowd (1976) offered one of the first explanations of the coming out process by suggesting that it comprises three stages: awareness, behavioral acceptance, and identification. Soon after, Cass (1979) proposed five stages: identity confusion, comparison, tolerance, acceptance, and pride. Later, Coleman (1982) also suggested five stages: pre-coming out, coming out, exploration, first relationships, and acculturation. Like Coleman, Troiden (1989) spoke of coming out in terms of changing identity formation. According to Troiden, a man began by suspecting that he might be homosexual, claiming a gay identity, becoming involved in the gay community, and having his first same-sex relationship. The difference between Coleman’s and Troiden’s theories and those of Hencken and O’Dowd and Cass is that Coleman and Troiden discuss coming out in terms of social identity construction, whereas Hencken and O’Dowd and Cass discuss coming out in terms of a change in personal identity formation and psychological adaptation by accepting one’s newly acknowledged gay or bisexual identity. Unlike the others, Martin (1991) proposed six issues that a person must confront when coming out: feeling different, self-acknowledgement, disclosure to others, acceptance of identity, experimentation and exploration, and identity consolidation. He stressed that these six issues do not necessarily occur sequentially. Similarly, Savin-Williams and Diamond (2000) suggested that it was inaccurate to conceptualize coming out as a sequential process, favoring instead the consideration of gender differences in the context, timing, and spacing of first same-sex attractions, self-labeling, same-sex sexual contact, and disclosure. Common to all the coming out models is the recognition that the development of a personal homosexual identity includes decisions around revealing that identity to others (i.e., coming out). Thus, across all models, some degree of outness is essential to homosexual or gay identity formation.

Research indicates that the community influences an individual’s extent of outness and internalization of prevailing social attitudes toward homosexuality (Meyer & Dean, 1998; Ross et al., 2013). Higher levels of internalized homonegativity have been associated with decreased outness (Rosser et al., 2008), increased depression (Ross, Rosser, Neumeier, & the Positive Connections Team, 2008; Rosser et al., 2008), increased drug use (Ross et al., 2001; Shoptaw et al., 2009), less HIV-serostatus disclosure (Noor, Rampalli, & Rosser, 2014), and increased unsafe sexual activity (Ross et al., 2008). Thus, outness and internalized homonegativity are of interest to researchers studying the health of lesbian, gay, or bisexual populations. This includes HIV prevention researchers focused on studying how individual factors might influence sexual risk (Kubicek et al., 2009; Newcomb & Mustanski, 2011; Ross etal., 2001; Rostosky, Danner, & Riggle, 2007; Shoptaw et al., 2009), as well as some developmental researchers. There is some evidence that outness increases with age among MSM (Ross & Rosser, 1996; Ross et al., 2008).

Multi-Item Measures of Outness

We reviewed the published literature to identify articles that used two or more items to assess outness and found three multiitem measures of the construct. The first measure, the National Lesbian Health Care Survey, included four items that asked participants the proportion of family, gay and lesbian friends, heterosexual friends, and coworkers who knew the participant was a lesbian (Bradford, Ryan, & Rothblum, 1994). The second measure, the Lesbian Wellness Survey, included six items: three items from the National Lesbian Health Care Survey that asked about outness to heterosexual friends, family, and coworkers; two items that assessed participants’ attitudes toward being out (e.g., how important it was to be out and how much they worried about the consequences of being out); and one item that asked participants to enter their zip code (Morris, Waldo, & Rothblum, 2001). The zip code might not be immediately intuitive as a measure of outness. It appears to be based on Harry’s (1993) research where he compared outness of gay men across zip codes. He found that men were less likely to provide a zip code if they were less open about their same-sex attraction. Thus, the Lesbian Wellness Survey used the entering of a zip code as a behavioral indication of outness. The third measure, the Outness Inventory, included 11 items across three subscales: family, world, and religion (Mohr & Fassinger, 2000). This inventory measures the extent to which other people knew about the participant’s same-sex attraction and how frequently they talked about it. As the most comprehensive multi-item measure of outness, it was the logical choice to include in this comparative study.

Current Study

In this study, we examined if the single-item indicator of outness and the multi-item scale of outness similarly captured the degree to which someone was out about their same-sex attraction. A priori, based on the premise that multi-item measures perform better than single-item measures, we hypothesized that the Outness Inventory, a multi-item scale, would perform better than the single-item indicator.

Method

Participants

In 2008, we began a four-wave, prospective, matched-sample study of MSM in 16 U.S. metropolitan statistical areas. The purpose of the Structural Interventions to Lower Alcohol-related STI/HIV Risk (SILAS) study was to assess the extent to which legislation about homosexuality influenced alcohol use and risky sexual behavior among MSM. Data for this analysis came from the fourth wave of data collection. Additional details about the SILAS study are provided elsewhere (Jones-Webb, Smolenski, Brady, Wilkerson, & Rosser, 2013; Noor, Rampalli, & Rosser, 2014).

Participants were recruited into Wave 4 between May 20, 2011 and December 26, 2011, with geo-targeted banner advertisements to adult male members of Facebook (http://www.facebook.com) who self-identified as attracted to men and to men who frequented websites affiliated with the Gay Ad Network (Quantcast Corporation, 2010). Banner advertisements and e-mails directed interested persons to a webpage hosted on a dedicated university server with appropriate encryption to ensure data security. Eligibility criteria included being a man (i.e., a biological male), having prior sexual experience with a man, being 18 years or older, and reporting a residential zip code in a metropolitan statistical area under study. A Certificate of Confidentiality was obtained from the National Institutes of Health. The study was conducted under the oversight of the institutional review boards of the researchers’ home institutions.

Participants were asked a varying number of items depending on responses to skip patterns (maximum of 123 questions). A refuse-to-answer response option allowed participants to opt out of answering any item. The mean survey completion time was 71 min. Participants were compensated $30 for their time. A total of5047 MSM completed the survey; however, not every participant answered every outness item. Participants had the option of skipping an item if it was not applicable to them. A total of 5019 participants answered the 1-item indicator, 3974 answered the 5-item world subscale, 3682 answered the 4-item family subscale, and 2167 answered the 2-item religion subscale items. Because we were interested in comparing the 1-item indicator and the 11-item scale, we took a conservative approach and restricted this analysis to the 1475 MSM who answered all the items of interest.

Measures

Single-Item Outness Indicator

Participants were asked to respond to one Likert-type item, “I would say that I am open (out) as gay, bisexual, or a man attracted to other men.” The item included five response options, ranging from 1 = not at all open (out) to 5 = open (out) to all or most people I know. A higher score indicated greater outness.

Multi-Item Outness Scale

The Outness Inventory (Mohr & Fassinger, 2000) consists of 11 Likert-type items that ask participants to indicate how open they are about their same-sex attraction to the following people: mother, father, siblings, extended family, work peers, work supervisors, new acquaintances, new straight friends, old straight friends, members of religious community, and leaders of religious community. The original scale included seven response options to assess differences in the degree of disclosure and discussion, ranging from 1 = definitely does not know about your sexual orientation status to 7 = definitely know(s) about your sexual orientation status, and it is openly talked about. We added one response to the original scale: “Not applicable to your situation; there is no such person or group of people in your life.” In addition to calculating an overall outness score, we also calculated scores for three subscales: family (mother, father, siblings, and extended family); world (work peers, work supervisors, new acquaintance, new straight friends, and old straight friends); and religion (members of religious community, and leaders of religious community). In our sample, internal consistency was high for the entire scale (Cronbach’s alpha = 0.94), as well as for the three subscales (0.91 for family, 0.91 for world, and 0.96 for religion). We calculated four mean scores: a combined mean of all 11 responses and a mean score for each of the three subscales. A higher score indicated greater outness.

Internalized Homonegativity

To measure internalized homonegativity, we used the revised Reactions to Homosexuality 7-item scale (Smolenski, Diamond, Ross, & Rosser, 2010), which includes three constructs: personal comfort with being gay (three items), public identification as gay (two items), and social comfort with gay men (two items). Response options to 7-point Likert-type questions ranged from 1 = strongly disagree to 7 = strongly agree. Sample items include “I feel comfortable being seen in public with an obviously gay person” and “Even if I could change my sexual orientation, I wouldn’t.” In our sample, the Cronbach’s alpha for the entire scale was 0.78. A higher score indicated greater internalized homonegativity.

Social Attitudes Towards Homosexuality

We created a composite score for social attitudes towards homosexuality using responses to five questions related to rights for lesbian, gay, bisexual, and transgender populations. First, we asked, “Do you favor or oppose allowing gay and lesbian couples to get legally married?” with response options being “Strongly favor, ” “Favor,” “Oppose,” “Strongly oppose,” and “I don’t know.” We coded “I don’t know” as missing, and dichotomized this variable as favor versus oppose. Second, we asked, “Do you favor or oppose allowing gay and lesbian couples to enter into private legal agreements with each other that would give them many of the same rights as married couples (e.g., wills, trusts, contracts, powers of attorney)?” with response options being “Strongly favor,” “Favor,” “Oppose,” “Strongly oppose,” and “I don’t know.” We coded “I don’t know” as missing, and dichotomized this variable as favor versus oppose. Third, we asked, “Do you think amending the U.S. Constitution to ban gay marriage is a good idea or a bad idea?” with response options being “Bad idea,” “Good idea,” and “I don’t know.” We coded “I don’t know” as missing, and dichotomized this variable as good versus bad. Lastly, we asked participants the following two questions: “Do you favor or oppose gay or lesbians serving openly in the military?” and “Do you favor or oppose allowing gay or lesbians to adopt children?” with response options being “Strongly favor,” “Favor,” “Oppose,” “Strongly oppose,” and “I don’t know.” We coded “I don’t know” as missing, and dichotomized these two variables as favor versus oppose. Using the responses to these five items, we created a composite score, ranging from 0 to 5. A higher score indicated greater negative attitudes towards homosexuality.

Depressive Symptoms

To assess depressive symptoms, we used the 10-item Center for Epidemiologic Studies Depression Scale (CES-D 10; Zhang et al., 2012). Response options to 4-point Likert-type questions ranged from 0 = Rarely or none of the time to 3 = All of the time. A CES-D score of 10 or greater indicates presence of depressive symptoms. In our sample, the Cronbach’s alpha for this scale was 0.85.

Sexual Risk Behavior

A sexual behavior battery investigated sexual risk behavior in the past 90 days with partners met online, at a gay bar/club, and at any other location(s). If participants indicated that they engaged in anal sex in past 3 months with apartner(s) met online or offline, they were asked to report the number of partner(s) with whom they had anal sex with and without a condom, estimated separately. Using these estimates, we created a sexual risk variable: condomless anal sex with multiple partners (CASMP; 0–2 partners/2+ partners). We adopted a conservative approach and used the cut point of 0–2 partners versus 2 or more partners. Since there is universal agreement that recent condomless anal sex with 2 or men is high-risk behavior for sexually transmitted infection including HIV, the measure enabled a comparison between men clearly engaging in high-risk behavior with men at no-to-low risk.

Demographic Characteristics

Participants were asked their age, education, race/ethnicity, and sexual identity. For this analysis, we categorized age as 18–24, 25–29, 30–34, 35–39, and 40 and older. Because most of the participants identified as gay/homosexual, we placed participants reporting another sexual identity in an “other” category.

Data Analysis

Summary statistics were used to describe the study participants. Because the purpose of this analysis was to compare the 1-item outness indicator with the 11-item outness scale, we calculated correlations between the single- and multi-item measures; between the single-item indicator and the three subscales of the multi-item scale; and between the measures and internalized homonegativity, social attitudes towards homosexuality, and depressive symptoms. Finally, we fita series of log-binomial models with a sandwich estimator predicting (1) presence of depressive symptoms (a score of 10 or greater vs. less than 10 on CES-D 10) and (2) condomless anal sex with multiple partners (more than 2 CASMP vs. 2 or less CASMP), using the single-item indicator and the multi-item scale, as well as the three subscales of the multi-item scale. All models were adjusted for age, race/ethnicity, education, and sexual orientation. In addition to Akaike information criterion (AIC) and Bayesian information criterion (BIC), we calculated Tjur’s R2 (2009) as a measure of predictive power of the single-item indicator, the multi-item scale, and the three subscales of the multi-item scale. Tjur’s R2 is the difference in the average of the event probabilities between the groups of observations with observed events and nonevents. Tjur’s R2 value ranges from 0 to 1, with a higher score indicating greater discriminatory power. All statistical tests were two-tailed and all analyses were conducted using Stata, version 13.1 (StataCorp LP, 2013).

Results

Participant characteristics are shown in Table 1. Most participants were under 40 years of age (75.5 %), white (73.1 %), had more than a high school education (88.8 %), and self-identified as gay (88.8 %). In addition, most participants were supportive of same-sex marriage (93.7 %), ensuring same-sex couples have the same legal rights as married couples (95.6 %), serving openly in the military (95.9 %), and adopting children (95.7 %), but they were against amending the U.S. constitution to ban same-sex marriage (81.7%). On average, participants had low levels of internalized homonegativity (M = 2.30, SD = 1.33) and depression (M = 9.91, SD = 6.29).

Table 1.

Participant characteristics (N = 1475)

Characteristic n (%)
Age
 18–24   478 (33.3)
 25–29   226 (15.7)
 30–34   177 (12.3)
 35–39   159 (11.1)
 40+   394 (27.5)
Race/ethnicity
 Black     60 (4.1)
 Hispanic   172 (11.7)
 White 1078 (73.1)
 Othera   165 (11.2)
Education
 Less than high school or high school degree   165 (11.2)
 Some college   533 (36.2)
 College degree   532 (36.1)
 Graduate degree   244 (16.6)
Sexual identity
 Gay/homosexual 1308 (88.8)
 Otherb   165 (11.2)
Legalize same-sex marriage
 Favor 1341 (93.7)
 Oppose     91 (6.4)
Ensure same legal rights as married couples
 Favor 1372 (95.6)
 Oppose     63 (4.4)
Amend U.S. Constitution to ban same-sex marriage
 Bad idea 1162 (81.7)
 Good idea   260 (18.3)
Allow gays and lesbians to serve openly in the military
 Favor 1373 (95.9)
 Oppose     59 (4.1)
Allow gays and lesbians to adopt children
 Favor 1387 (95.7)
 Oppose     63 (4.3)
Condomless anal sex with multiple partners, past 90 days
 2 or less   580 (66.7)
 More than 2   290 (33.3)
 Internalized homonegativity, M (SD)  2.30 (1.3)
 Depressive symptoms (CES-D 10 score), M (SD)  9.91 (6.3)
Open in a new tab

Differences in values are the result of missing data

CES-D 10 Center for Epidemiologic Studies Short Depression Scale

a

“Other” race/ethnicity category included participants identifying as Asian or Pacific Islander (3.6 %), Native American (2.1 %), multi-racial (4.7 %), or did not report a specific race/ethnicity (0.9 %)

b

“Other” sexual identity category included persons identifying as bisexual (9.1 %) and heterosexual (1.3 %), as well as persons who selected “other” from the list of response options (0.8 %)

The correlation coefficients and summary statistics of the single- and multi-item measures of outness are shown in Table 2. There was a strong correlation (r = 0.73) between the 1-item indicator and the 11-item scale, indicating that participants who scored high on the 1-item indicator tended to score high on the 11-item scale. The coefficient of determination between the 1-item indicator and the 11-item scale was 0.53 (i.e., 0.732 = 0.53), meaning that 53 % of the variance was shared between the two measures.

Table 2.

Correlations between measures of outness, internalized homonegativity, social attitudes towards homosexuality, and depressive symptoms

1 2 3 4 5 6 7 8
1. 1-item outness score 1.00
2. 11-item combined outness score 0.73*** 1.00
3. 4-item family score 0.70*** 0.91*** 1.00
4. 5-item world score 0.71*** 0.93*** 0.75*** 1.00
5. 2-item religion score 0.50*** 0.83*** 0.67*** 0.69*** 1.00
6. Internalized homonegativity −0.63*** −0.55*** −0.51*** −0.53*** −0.42*** 1.00
7. Social attitudes towards homosexuality score −0.38*** −0.21*** −0.24*** −0.20*** −0.07*** 0.40*** 1.00
8. Depressive symptoms (CES-D 10 score) −0.14*** −0.13*** −0.13*** −0.11*** −0.12*** 0.28*** 0.24*** 1.00
N 1475 1475 1475 1475 1475 1475 1468 1475
M (SD) 4.06 (1.22) 4.92 (1.73) 5.15 (1. 84) 4.96 (1.83) 4.42 (2.32) 2.32 (1.13) 0.36 (0.80) 9.90 (6.29)
Median (IQR) 5.00 (3.00, 5.00) 5.36 (3.72, 6.27) 5.75 (4.00, 6.75) 5.40 (3.60, 6.60) 5.00 (2.00, 7.00) 2.00 (1.43, 3.00) 0 (0, 0) 9 (5, 15)
Open in a new tab

CES-D 10 Center for Epidemiologic Studies Short Depression Scale, IQR inter-quartile range

***

p<.001

The correlation coefficients between internalized homonegativity, social attitudes towards homosexuality, and depressive symptoms were higher for the 1-item indicator than for the 11-item scale. There was an inverse correlation between internalized homonegativity and the 1-item indicator (r = −0.63) and the 11-item scale (r = −0.55). More variance was shared between internalized homonegativity and the 1-item indicator (40 %) than between internalized homonegativity and the 11-item scale (30 %). There was also an inverse correlation between social attitudes towards homosexuality and the 1-item indicator (r = −0.38) and the 11-item scale (r = −0.21). Again, more variance was shared between social attitudes towards homosexuality and the 1-item indicator (14 %) than between social attitudes towards homosexuality and the 11-item scale (4 %). A similar pattern was observed for depressive symptoms: the 1-item indicator shared more variance (1.9 %) than the 11-item scale (1.6 %).

In terms of correlations between the single-item indicator and the three subscales of the multi-item scale, the correlations were stronger between the single-item indicator and the scores for the family (r = 0.70) and world (r = 0.71) subscales than the score for the religion subscale (r = 0.50). When we examined the median scores, we found that most participants selected the maximum value for the 1-item indicator but did not select the maximum values when responding to questions in the 11-item scale, although median responses for the items were in the upper range.

Fit indices of the regression models predicting depressive symptoms and CASMP are shown in Table 3. Models with the 1-item outness indicator had lower AIC and BIC values than those for models with the 11-item outness scale and each of its subscales. In addition, Tjur’s R2 values were higher for models with the 1-item indicator than those for the models with the 11-item scale and each of its subscales.

Table 3.

Fit indices of regression models predicting depressive symptoms and condomless anal sex with multiple partners using 1-item and 11-item measures of outness

Depressive symptomsa (present vs. absent)
CASMP (more than 2 vs. 2 or less)
Model with: AIC BIC Tjur’s R2b AIC BIC Tjur’s R2b
1-item outness indicator 2021.55 2053.15 0.044 1012.97 1041.28 0.072
11-item combined outness scale 2031.34 2062.94 0.037 1051.18 1079.49 0.026
4-item family subscale 2032.11 2063.71 0.036 1050.22 1078.54 0.028
5-item world subscale 2034.44 2066.04 0.035 1049.50 1077.81 0.029
2-item religion subscale 2037.80 2069.39 0.032 1042.94 1071.25 0.035
Open in a new tab

All models were adjusted for age, race/ethnicity, education, and sexual orientation

AIC Akaike information criterion, BIC Bayesian information criterion, CASMP condomless anal sex with multiple partners in past 90 days

a

Presence of depressive symptoms was defined as a score of 10 or greater on the Center for Epidemiologic Studies Short Depression Scale (CES-D10)

b

Tjur’s R2 was calculated as the difference between the means of two predicted probabilities of an event among cases with the event and cases without the event

Discussion

Contrary to the premise that multi-item measures are generally superior to single-item measures, and contrary to our hypothesis, we found that the single-item indicator of outness performed better than the multi-item scale of outness. The single-item indicator was strongly correlated with the multi-item scale, and with both the family and world subscales of the multiitem scale. In addition, models with the single-item indicator had higher discriminatory power than models with the multi-item scale in predicting depressive symptoms and CASMP. Because the single-item indicator is validated, performs better, while also reducing subject burden, our results support the use of the singleitem indicator as the standard measure of outness.

Consistent with Gardner et al.’s (1998) explanation, question relevance likely explains the better performance of the single-item indicator compared with the multi-item scale. For instance, the poorest predictor of internalized homonegativity was the religion subscale of the Outness Inventory. Because agnosticism and atheism are substantially higher in gay and bisexual men than the general public (Wilkerson, Smolenski, Brady, & Rosser, 2013), the questions about outness to their religious community may be irrelevant to more participants and, thus, may be poorer at discriminating on any factor, including outness. Similarly, the questions on family outness assume that participants have a living and known father, mother, siblings and extended family. However, given the changing family structures in the United States and elsewhere, measures relying on participants to have contact with both nuclear and extended family members may become increasingly irrelevant. While most MSM may have some combination of work peers, work supervisors, new acquaintances, new straight friends, and/or old straight friends, these questions assume employment and a social situation where participants meet new people, which may not be true for a segment of the gay and bisexual men in times of high unemployment or who are otherwise unemployed. In this regard, students, who frequently compose a significant proportion of MSM study participants, may not identify as having work “peers” or “supervisors.”

To explore this explanation of question relevance, we compared the “refuse-to-answer” rate (0.36 %) on the single-item indicator with the “refuse-to-answer” rates (0.71–2.48 %) and “not applicable” rates (2.83–53.22 %) on the multi-item scale. The higher combined rate stemming from the additional “not applicable” rates supports the explanation that more participants found the questions irrelevant, decreasing the items’ discriminatory power. This suggests that the Outness Inventory scale, as well as similar multi-item scales that measure outness to specific persons, is likely to be have this important limitation.

A second explanation why the single-item indicator performed better than the multi-item scale is that the two scales might measure slightly different constructs. When comparing the two measures, we note that the single-item indicator appears to be more subjective or internal, measuring an overall internal sense of being out, whereas the multi-item scale, by anchoring outness to specific persons and frequencies of conversation, could be considered to be more objective or external. However, because these differences are somewhat subtle, we caution against attaching undue importance to them.

A third explanation for the better performance of the single item indicator is that the Outness Inventory scale is unusual in that it uses dual-response anchor points (e.g., 2 = person might know about your sexual orientation status, but it is never talked about; 6 = person definitely knows about your sexual orientation status, and it is sometimes talked about). Although these response items have the advantage of high face validity, they have two weaknesses. The wording violates the principle of avoiding multi-barreled questions (or, in this case, responses), and it does not cover the domain of all possible response options. For example, there is no response option for a person who definitely knows about the participant’s sexual orientation status, but it is never talked about. When participants have no option available to denote their circumstance, they are forced to choose from the available options, which is likely to weaken the overall predictive power of the measure.

Despite the better performance of the single-item indicator, the multi-item scale remains the better choice for researchers interested in measuring how participants differ in their outness based on finer differentiation of dimensions of outness, particularly outness to religious leaders/congregants and family members. A multi-item scale may provide more nuance that would allow for detection of associations between different types of outness and health outcomes. For example, religious outness may be particularly salient for some individuals and, thus, may be associated with higher rates of specific health outcomes (e.g., depression). We caution researchers considering using the Outness Inventory scale, or another multi-item scale, to provide the additional category of “not relevant” to distinguish between the higher number of participants with no religious affiliation, living family members, work colleagues, or social opportunities.

The current study had several limitations and offers some directions for future research. Although we have used the single item indicator of outness in our research, which is similar to that used by other researchers, to our knowledge, this is the first study to directly compare the psychometric properties of the single item indicator with those of a multi-item scale. Thus, additional validation studies are needed to test the reliability and generalizability of our findings. To validate outness measures, we used internalized homonegativity (revised Reactions to Homosexuality 7-item scale), social attitudes towards homosexuality, and depression (CES-D 10) scores. Although measures of internalized homonegativity seem reasonable as a validation criterion for outness, because they are conceptually inversely correlated, other studies may use different validation measures. We considered conducting additional construct validity tests by measuring additional outcomes reliably associated with outness. Because such an approach inflates statistical probability, may invoke circular argument, and introduces potential problems of interpretation if findings are discrepant, we decided against further comparisons. We relied on self-report data and we did not assess social desirability, requiring us to assume participants responded to all items accurately. Because data were collected from a non-random online sample of MSM, the generalizability of our findings to other online samples, offline samples of MSM, or all MSM should not be assumed. In addition, our sample was largely white, identified as gay, and out. In light of research showing that outness varies across race, ethnicity, gender, and generation (Grov, Bimbi, Nanin, & Parsons, 2006), future research should include validating measures in racial/ethnic minorities and intergenerational samples. Finally, our study was restricted to gay or bisexual men and other MSM. Although the explanations of question relevance and subjectivity predict similar findings should be found in studies of outness among lesbian, bisexual, and other women who have sex with women, as well as in studies of outness as transgender among transgender persons, empirical work is needed to confirm these relationships.

For at least two generations, coming and being out as a gay or bisexual man has been a defining characteristic of gay identity in western countries and a key predictor of interest to researchers studying sexual and mental health. As equal rights for sexual minorities are achieved, there may be shifts in what it means to be out, what outness predicts, and even what the stages of the coming out process are. Qualitative research comparing different age cohorts of sexual minorities to understand how different cohorts experience coming out, and how participants assess their overall outness, could be valuable to identify what outness means in contemporary gay culture—definitions and sequence within developmental trajectories might be shifting over time. Quantitative research promoting standard measure items would enable the comparison of results across studies, strengthening this growing body of knowledge.

Acknowledgments

The Structural Interventions to Lower Alcohol-related STI/HIV Risk (SILAS) Study was funded by the National Institute on Alcohol Abuse and Alcoholism, funding number R01AA016270-01A1. All research was conducted with the approval of the University of Minnesota Institutional Review Board.

Footnotes

Compliance with Ethical Standards

Conflict of Interest The authors declare that they have no conflicts of interest.

References

  1. Bradford J, Ryan C, Rothblum ED. National Lesbian Health Care Survey: Implications for mental health care. Journal of Consulting and Clinical Psychology. 1994;62:228–242. doi: 10.1037/0022-006X.62.2.228. [DOI] [PubMed] [Google Scholar]
  2. Cass VC. Homosexual identity formation: A theoretical model. Journal of Homosexuality. 1979;4:219–235. doi: 10.1300/J082v04n03_01. [DOI] [PubMed] [Google Scholar]
  3. Coleman E. Developmental stages of the coming-out process. American Behavioral Scientist. 1982;25:469–482. doi: 10.1300/J082v07n02_06. [DOI] [Google Scholar]
  4. Diamantopoulos A, Sarstedt M, Fuchs C, Wilczynski P, Kaiser S. Guildelines for choosing between multi-item and single item scales for construct measure: A predictive validity perspective. Journal of the Academy of Marketing Science. 2012;40:434–449. [Google Scholar]
  5. Gardner DG, Cummings LL, Dunham RB, Pierce JL. Single-item versus multiple-item measurement scales: An empirical comparison. Educational and Psychological Measurement. 1998;58:898–915. [Google Scholar]
  6. Grov C, Bimbi DS, Nanín JE, Parsons JT. Race, ethnicity, gender, and generational factors associated with the coming-out process among gay, lesbian, and bisexual individuals. Journal of Sex Research. 2006;43:115–121. doi: 10.1080/00224490609552306. [DOI] [PubMed] [Google Scholar]
  7. Harry J. Being out: A general model. Journal of Homosexuality. 1993;26:25–39. doi: 10.1300/J082v26n01_03. [DOI] [PubMed] [Google Scholar]
  8. Hencken JD, O’Dowd WT. Coming out as an aspect of identity formation. Gai Saber. 1976;1:18–26. [Google Scholar]
  9. Jones-Webb R, Smolenski D, Brady S, Wilkerson M, Rosser BRS. Drinking settings, alcohol consumption, and sexual risk behavior among gay men. Addictive Behaviors. 2013;38:18241830. doi: 10.1016/j.addbeh.2012.11.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Kennedy H. Karl Heinrich Ulrichs: First theorist of homosexuality. In: Rosario V, editor. Science and homosexualities. New York: Routledge; 1997. p. 2645. [Google Scholar]
  11. Koblin BA, Mansergh G, Frye V, Tieu HV, Hoover DR, Bonner S, Colfax GN. Condom-use decision making in the context of hypothetical pre-exposure prophylaxis efficacy among substance-using men who have sex with men: Project MIX. Journal of Acquired Immune Deficiency Syndromes. 2011;58:319–327. doi: 10.1097/QAI.0b013e31822b76d2. [DOI] [PubMed] [Google Scholar]
  12. Konstan J, West W. Using technology and the Internet in research. In: Pequegnat W, Stover E, editors. How to write a successful research grant application. Bethesda, MD: National Institutes of Health; 2010. pp. 189–206. [Google Scholar]
  13. Kubicek K, McDavitt B, Carpineto J, Weiss G, Iverson EF, Kipke MD. “God made me gay for a reason”: Young men who have sex with men’s resiliency inresolving iinternalized homophobia from religious sources. Journal of Adolescent Research. 2009;24:601–633. doi: 10.1177/0743558409341078. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Martin HP. The coming-out process for homosexuals. Hospital & Community Psychiatry. 1991;42:158–162. doi: 10.1176/ps.42.2.158. [DOI] [PubMed] [Google Scholar]
  15. Meyer IH, Dean L. Internalized homophobia, intimacy, and sexual behavior among gay and bisexual men. In: Herek GM, editor. Stigma and sexual orientation: Understanding prejudice against lesbians, gay men, and bisexuals. Thousand Oaks, CA: Sage Publications; 1998. pp. 160–186. [Google Scholar]
  16. Mohr J, Fassinger R. Measuring dimensions of lesbian and gay male experience. Measurement & Evaluation in Counseling & Development. 2000;33:66–90. [Google Scholar]
  17. Morris JF, Waldo CR, Rothblum ED. A model of predictors and outcomes of outness among lesbian and bisexual women. American Journal of Orthopsychiatry. 2001;71:61–71. doi: 10.1037/0002-9432.71.1.61. [DOI] [PubMed] [Google Scholar]
  18. Newcomb ME, Mustanski B. Moderators of the relationship between internalized homophobia and risky sexual behavior in men who have sex with men: A meta-analysis. Archives of Sexual Behavior. 2011;40:189–199. doi: 10.1007/s10508-009-9573-8. [DOI] [PubMed] [Google Scholar]
  19. Noor SW, Rampalli K, Rosser BRS. Factors influencing HIV serodisclosure among men who have sex with men in the US: An examination of online versus offline meeting environments and risk behaviors. AIDS and Behavior. 2014;18:1638–1650. doi: 10.1007/s10461-014-0774-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Nunnally JC. Psychometric theory. New York: McGraw-Hill; 1978. [Google Scholar]
  21. Pequegnat W, Rosser BR, Bowen AM, Bull SS, DiClemente RJ, Bockting WO, Zimmerman R. Conducting Internet-based HIV/STD prevention survey research: Considerations in design and evaluation. AIDS and Behavior. 2007;11:505–521. doi: 10.1007/s10461-006-9172-9. [DOI] [PubMed] [Google Scholar]
  22. Quantcast Corporation. Gay ad network. 2010 Retrieved from www.quantcast.com.
  23. Ross MW, Berg RC, Schmidt AJ, Hospers HJ, Breveglieri M, Furegato M, European MSM Internet Survey (EMIS) Network Internalised homonegativity predicts HIV-associated risk behaviour in European men who have sex with men in a 38-country cross-sectional study: Some public health implications of homophobia. BMJ Open. 2013;3:e001928. doi: 10.n36/bmjopen-2012-001928. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Ross MW, Rosser BR. Measurement and correlates of internalized homophobia: A factor analytic study. Journal of Clinical Psychology. 1996;52:15–21. doi: 10.1002/(SICI)1097-4679(199601)52:1<15::AID-JCLP2>3.0.CO;2-V. [DOI] [PubMed] [Google Scholar]
  25. Ross MW, Rosser BR, Neumaier ER, Team, Positive Connections The relationship of internalized homonegativity to unsafe sexual behavior in HIV-seropositive men who have sex with men. AIDS Education and Prevention. 2008;20:547–557. doi: 10.1521/aeap.2008.20.6.547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Ross MW, Simon Rosser BR, Bauer GR, Bockting WO, Robinson BE, Rugg DL, Coleman E. Drug use, unsafe sexual behavior, and internalized homonegativity in men who have sex with men. AIDS and Behavior. 2001;5:97–103. doi: 10.1023/A:1009567707294. [DOI] [Google Scholar]
  27. Rosser BR, Bockting WO, Ross MW, Miner MH, Coleman E. The relationship between homosexuality, internalized homo-negativity, and mental health in men who have sex with men. Journal of Homosexuality. 2008;55:185–203. doi: 10.1080/00918360802129394. [DOI] [PubMed] [Google Scholar]
  28. Rostosky SS, Danner F, Riggle ED. Is religiosity a protective factor against substance use in young adulthood? Only if you’re straight! Journal of Adolescent Health. 2007;40:440–447. doi: 10.1016/j.jadohealth.2006.11.144. [DOI] [PubMed] [Google Scholar]
  29. Savin-Williams RC, Diamond LM. Sexual identity trajectories among sexual-minority youths: Gender comparisons. Archives of Sexual Behavior. 2000;29:607–627. doi: 10.1023/a:1002058505138. [DOI] [PubMed] [Google Scholar]
  30. Shoptaw S, Weiss R, Munjas B, Hucks-Ortiz C, Young S, Larkins S, Gorbach P. Homonegativity, substance use, sexual risk behaviors, and HIV status in poor and ethnic men who have sex with men in Los Angeles. Journal of Urban Health. 2009;86:77–92. doi: 10.1007/s11524-009-9372-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Smolenski DJ, Diamond PM, Ross MW, Rosser BRS. Revision, criterion validity, and multigroup assessment of the Reactions to Homosexuality Scale. Journal of Personality Assessment. 2010;92:568–576. doi: 10.1080/00223891.2010.513300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. StataCorp, L. P. STATA-SE (Version 13.1) College Station, TX: Author; 2013. [Google Scholar]
  33. Tjur T. Coefficients of determination in logistic regression models—A new proposal: The coefficient of discrimination. American Statistician. 2009;63:366–372. [Google Scholar]
  34. Troiden RR. The formation of homosexual identities. Journal of Homosexuality. 1989;17:43–73. doi: 10.1300/J082v17n01_02. [DOI] [PubMed] [Google Scholar]
  35. Wilkerson JM, Smolenski DJ, Brady SS, Rosser BR. Religiosity, internalized homonegativity, and outness in Christian men who have sex with men. Sexual and Relationship Therapy. 2012;27:122–132. doi: 10.1080/14681994.2012.698259. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Wilkerson JM, Smolenski DJ, Brady SS, Rosser BR. Performance of the Duke Religion Index and the Spiritual Well-Being Scale in online samples of men who have sex with men. Journal of Religion and Health. 2013;52:610–621. doi: 10.1007/s10943-012-9594-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Zhang W, O’Brien N, Forrest JI, Salters KA, Patterson TL, Montaner JS, Lima VD. Validating a shortened depression scale (10 item CES-D) among HIV-positive people in British Columbia, Canada. PLoS One. 2012;7:e40793. doi: 10.1371/journal.pone.0040793. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES