Abstract
Despite increasing rates of HIV infection among young men who have sex with men (YMSM), only a minority participate in formal HIV prevention efforts. Semi-structured mixed-methods interviews were conducted in a diverse sample of YMSM (N = 100, Mage = 25.0 years) in Los Angeles, California, to identify facilitators and barriers to participation in HIV prevention programs. Summative content analyses were used to evaluate transcribed field notes from these interviews. Results showed that 28.0% of all participants had previously attended an HIV prevention program, and that 21.3% of those who were also asked if they had ever participated in any research pertaining to HIV prevention had done so. A significantly higher percentage of those who had participated in HIV prevention programs had been tested for HIV in the past 6 months compared to those who had not (p < .05). The most frequently mentioned barriers to participation in such a program were being too busy to attend (12.0%), not perceiving themselves to be at risk for HIV infection (14.0%), and believing that they already knew everything they needed to know about HIV transmission (23.0%). YMSM suggested that future interventions should use technology (e.g., the Internet, mobile devices), engage their social networks, and highlight HIV prevention as a means for community connection. Collectively, these results provide some explanations for why YMSM account for a minority of HIV prevention program participants and offer possible directions for future HIV prevention efforts that target YMSM.
Keywords: YMSM, HIV prevention, barriers to participation
Introduction
Despite continued efforts to develop and implement behavioral HIV prevention programs, incidence rates for HIV among men who have sex with men (MSM) continue to rise (Centers for Disease Control and Prevention [CDC], 2009; HIV Epidemiology Program, Los Angeles County Department of Public Health, 2010; Hall et al., 2008; Jaffe, Valdiserri, & De Cock, 2007). This is especially true for young MSM (YMSM), who may be less informed than their older counterparts about HIV (CDC, 2010). Moreover, some research suggests that YMSM do not currently access HIV prevention programs and services as much as older MSM. For example, surveillance data show that among YMSM, those who self-identify as African American, those who are less affluent, and those who are older are over-represented in HIV prevention programs (Iguchi et al., 2009; Koblin et al., 2003; Orellana et al., 2006). According to survey data collected in Los Angeles County, although 60% of HIV/AIDS cases among MSM are in those aged 20–39 years (Bingham, 2009), the average age of those who participate in HIV prevention programs is between 40 and 49 years (Bingham, 2009; Shoptaw et al., 2009). Another study conducted in the Pacific Northwest demonstrated that MSM aged 20–24 years were over three times more likely not to enroll in an HIV prevention program than those aged 40 and over (Orellana et al., 2006).
Limited research has investigated why YMSM are less likely than their older counterparts to participate in HIV prevention programs. Some researchers have suggested that YMSM may feel less vulnerable to HIV, weakening their commitment to changing HIV risk behavior (Rutledge et al., 2002). Others have suggested that YMSM who reached sexual maturity in the age of highly active antiretroviral therapies may be complacent about sexual risk behavior because HIV can now be managed as many other chronic diseases (Valdisseri, 2004). These epidemiological and clinical observations shaped the objective of this study: To collect contemporary impressions about HIV prevention approaches and understand facilitators and barriers to participation in HIV prevention programs in a diverse sample of YMSM.
Methods
This study had a single-case quasi-experimental design (Cook & Campbell, 1979). Although this study design does not control for risks of biases and does not allow inference of causality, it can be used to compile exploratory evidence that is useful for developing further confirmatory study designs. Thus, the data collected were intended to provide qualitative descriptions of observed reluctance by YMSM to engage in HIV prevention programs.
Sampling and Procedures
A purposive sample of YMSM was recruited between July and August 2010 in West Hollywood, California, an area that includes a large population of YMSM and a variety of social venues that cater to middle- and upper–middle-class YMSM. Potential participants were approached on the street and asked to complete a brief screening interview to determine their eligibility for the study. YMSM were eligible to participate in the study if they (1) were between the ages of 18 and 30 years; and (2) either self-identified as gay or bisexual or reported having had sex with a man in their lifetime. YMSM who met the inclusion criteria completed an anonymous structured interview, which lasted approximately 5–10 minutes, and received a $5 gift card for their participation. This study was approved by the institutional review boards at the University of California, Los Angeles and the University of Southern California.
Measures
Semi-structured interviews began with a series of questions to obtain information about demographics (i.e., age, race/ethnicity, employment status, school attendance, highest level of education, place of residence), perceived HIV risk, and HIV testing history (i.e., ever tested, date of most recent test, testing experiences, and serostatus). Then, participants were asked a series of questions related to three content areas: (1) engagement in and impressions of HIV prevention activities; (2) impressions of HIV prevention advertisements; and (3) facilitators and barriers to future participation in HIV prevention activities.
Engagement in and impressions of HIV prevention activities
Participants were asked whether or not they had ever had the opportunity to participate in an HIV prevention program, defined as “a class or workshop outside of primary or secondary school sex education”; whether they had decided to participate; and if so, what were their impressions of that program. Participants were also asked whether or not they had ever had the opportunity to participate in HIV prevention research; whether they had decided to participate; and if so, what were their impressions of the research.
Impressions of HIV prevention advertisements
Participants were asked whether they had ever seen advertisements on HIV prevention. If participants had seen such advertisements, they were asked to name the medium (e.g., billboard, radio, television) and the sponsoring agency. Participants were also asked to give their impressions of these advertisements and their opinion about whether or not these advertisements were effective in changing sexual behavior.
Facilitators and barriers to future participation in HIV prevention activities
Regardless of whether or not they had previously participated in a formal HIV prevention program, participants were asked what they believed would encourage them to participate in such a program in the future (facilitators) and what would discourage or keep them from doing so (barriers).
Responses to all interview questions were hand-recorded by interviewers on paper field note templates. Immediately after data collection, interviewers typed these notes into individual word processing documents for subsequent abstraction and analysis.
Data Analysis
Data on demographics and HIV testing history were abstracted from field notes, recoded into numeric data, and entered into statistical software for analysis. Quantitative bivariate analyses were conducted to identify demographic and HIV testing variables associated with previous participation in a formal HIV prevention program. Associations were determined using independent sample t-tests for continuous variables and chi-square or Fisher’s exact tests for categorical variables. Since these analyses were exploratory, an a priori alpha level of .10 was used to determine statistical significance. All analyses were conducted using SPSS version 19.0 (IBM, 2010).
Semi-structured interview data on the three content areas described above were abstracted from field notes and analyzed using directed content analysis (Krippendorff, 2004) in order to be able to draw inferences from the text about both the message itself and the senders of the message (Weber, 1990). Data analysis began by reducing (i.e., selecting, simplifying, and transforming) the interview data (Miles & Huberman, 1999). Key concepts were used as initial coding categories based on the interview questions (Hsieh & Shannon, 2005). Coding was done with predetermined codes related to the research questions, not to specific words.
Based on previous research (Creswell, 2007; Lombard, Snyder-Duch, & Bracken, 2002; Sandelowski, 1993; Whittemore, Chase, & Mandle, 2001), several procedures were used to ensure that all analyses were rigorous. For instance, two doctoral-level coders with extensive research experience in YMSM HIV risk behavior analyzed all data. Once all field notes were initially reviewed, the coders created an audit trail to document decisions related to coding schema. This trail was maintained throughout the coding and analysis process; inter-coder agreement (Lombard et al., 2002; Tinsley & Weiss, 2000) was based on a comparison of a selection of interviews (1/5th; 20%); simple agreement (percentage of agreement) was employed (Lombard et al., 2002). A 95% agreement rate was achieved between the two coders. Any disagreements in coding were reviewed, resolved, and recoded.
Results
Demographics
Of 162 men who were approached, 100 completed the screening and semistructured interviews (61.7%). The primary reason for declining to participate in this study was being busy with another activity (e.g., on the way to meet friends). Mean age of study participants was 25 years (SD = 3.0). The vast majority had an education level beyond high school (89.9%), with many having completed college or higher (39.4%). Participants were from diverse racial/ethnic backgrounds: Latino (29.0%), White (29.0%), Asian (21.0%), Black (17.0%), and Other (4.0%).
Previous HIV Prevention Program Participation
Less than one-third (28.0%) of all participants reported having previously attended an HIV prevention program apart from a general sex education course received in primary or secondary school (Table). Of these, a significantly higher percentage had a racial/ethnic minority background compared to Whites (85.8% vs 14.3%; p < .05).
Table.
HIV Prevention Participation | |||
---|---|---|---|
Variable | Total (N = 100) |
No (n = 72) |
Yes (n = 28) |
No. or Mean (% or SD) |
No. or Mean (% or SD) |
No. or Mean (% or SD) |
|
Mean age, years (range, 19–30) | 25.01 (3.00) | 25.01 (2.82) | 25.00 (3.48) |
Race/ethnicitya** | |||
White | 29 (29.0) | 25 (34.7) | 4 (14.3) |
Black | 17 (17.0) | 12 (16.7) | 5 (17.9) |
Latino | 29 (29.0) | 17 (23.6) | 12 (42.9) |
Asian | 21 (21.0) | 15 (20.8) | 6 (21.4) |
Other | 4 (4.0) | 3 (4.2) | 1 (3.6) |
Highest level of educationb | |||
Less than high school | 2 (2.0) | 1 (1.4) | 1 (3.7) |
High school graduate | 8 (8.1) | 8 (11.1) | 0 (0.0) |
Associates degree, Junior College, Trade school | 8 (8.1) | 7 (9.7) | 1 (3.7) |
Some college | 42 (42.4) | 30 (41.7) | 12 (44.4) |
College graduate | 30 (30.3) | 20 (27.8) | 10 (37.0) |
More than college | 9 (9.1) | 6 (8.3) | 3 (11.1) |
In schoolc | 37 (37.4) | 25 (34.7) | 12 (44.4) |
Currently working | 79 (79.0) | 59 (81.9) | 20 (71.4) |
California resident | 93 (93.0) | 69 (95.8) | 24 (85.7) |
Tested for HIV | 91 (91.0) | 63 (87.5) | 28 (100.0) |
Tested for HIV in the past 6 monthsd** | 46 (56.8) | 27 (49.1) | 19 (73.1) |
HIV serostatuse | |||
Negative | 82 (83.7) | 58 (80.6) | 24 (92.3) |
Positive | 6 (6.1) | 4 (5.6) | 2 (7.7) |
Unknown | 10 (10.2) | 10 (13.9) | 0 (0.0) |
Perceived HIV risk (range, 1–10)f* | 2.64 (1.96) | 2.28 (1.52) | 3.56 (2.62) |
Note. Independent sample t-test used for continuous variables, and chi-square or Fisher’s exact test for categorical variables. Variables with category cell sizes less than 5 were not compared unless otherwise noted.
Comparison based on racial/ethnic minority vs. White.
Comparison based on college graduate vs. other (n = 99).
n = 99.
Among those ever tested for HIV (n = 81).
Comparison based on HIV-negative vs. other (n = 98).
n = 64. Higher scores indicate greater perceived HIV risk.
p < .10.
p < .05.
HIV Testing
Almost all participants (91.0%) reported having been previously tested for HIV, with over half (56.8%) having been tested within the past 6 months. A significantly higher percentage of participants who had attended a formal HIV prevention program had been tested for HIV in the past 6 months compared to those who had not (73.1% vs 49.1%; p < .05). Participants who reported being HIV-negative perceived their HIV risk to be low (M = 2.64; SD = 1.96). There was a marginally statistically significant difference in perceived HIV risk between those who had attended a formal HIV prevention program (M = 3.56; SD = 2.62) and those who had not (M = 2.28; SD = 1.52), with those who had attended a formal HIV prevention program perceiving themselves at greater risk for HIV (p < .10). Of those who had been previously tested for HIV, 83.7% reported having an HIV-negative result at last test (Table).
Impressions of HIV Prevention Advertisements
All but 5 participants reported having previously seen HIV prevention ads. Among the 95 respondents who had seen HIV prevention ads, the most frequently mentioned medium was billboards (68.4%), followed by television (26.3%), the Internet (15.8%), magazines (15.8%), and posters on public transportation (12.6%). Other places where HIV prevention ads had been seen included t-shirts, gay pride events, and posters/fliers.
Of the 86 participants who commented on the effectiveness of the ads they had seen, 53 (61.6%) felt that they were effective in promoting awareness about HIV. Among those who did not believe that the HIV prevention ads were effective (n = 33; 38.4%), the main complaint was that these ads minimized the severity of HIV. For example, one participant referred to pharmaceutical ads, which he believed “sugar coated” their messages. When commenting on a series of ads that highlighted the link between HIV infection and alcohol use, one participant remarked, “It should be a meth pipe.” Another participant believed ads should be more explicit about the various negative consequences of being HIV-positive, such as potential for skin ailments, burdensome medication regimens, and financial/insurance costs, rather than showing “buff men” who appear unconcerned about being HIV-positive. Several participants also voiced their concern that HIV prevention ads further stigmatized gay men and racial/ethnic minority men by being disproportionately placed in West Hollywood and by featuring African American men and alluding to the “down low.”
Engagement in and impressions of HIV prevention activities
Of the 94 participants who were also asked whether they had ever participated in any research pertaining to HIV prevention, 20 (21.3%) indicated that they had. Of the 19 participants who said they had received HIV education during high school, only 6 (31.6%) had attended a subsequent HIV prevention program.
Facilitators and Barriers to HIV Prevention Engagement
Among those who had previously attended an HIV prevention program, the most frequently mentioned barriers to participation were being “too busy” (12.0%), not perceiving themselves to be at “high risk” (14.0%), and believing that they already knew “the basics” so they did not need further education (23.0%). Two other barriers mentioned multiple times were HIV prevention programs not being a priority (6.0%) and these programs not being well advertised (6.0%).
When asked what would encourage them to participate in an HIV prevention program in the future, several participants (15.0%) stated “I don’t know” or “Nothing.” Several participants (14.0%) spoke about a desire to help prevent HIV in the larger gay community and that a program that explicitly emphasized the altruistic nature of participating in HIV prevention activities would encourage them to attend such a program. Other respondents indicated that a program would need to be “quick” (5.0%) and “convenient” (15.0%). They noted that HIV prevention materials often contained “stale” information, and stated that they would rather be given the latest information and skills about HIV prevention through an intervention effort that did not place an excessive burden on their time.
Several participants (14.0%) emphasized a desire to receive HIV prevention information and skills in social contexts that they were already attending with friends. Many indicated that they had done their own research about HIV on the Internet and that they liked the ability to access information through technology (e.g., mobile devices). These participants also stated a desire to receive HIV prevention messages through their social networks rather than through formal workshops. Some (12.0%) reported that they would need a “scare” (i.e., they or one of their friends becoming HIV-positive) to motivate them to participate in an HIV prevention program. Lastly, a few participants (10.0%) reported that incentives were important to them.
Discussion
Since the onset of HIV, tremendous advances have been achieved in HIV prevention among MSM. However, recent research indicates the need to focus on the unique desires and experiences of YMSM in order to increase their participation in HIV prevention programs (Orellana et al., 2006; Seal et al., 2000). In our study, less than one-third of all YMSM had attended a formal HIV prevention program other than primary school sex education, yet a greater percentage of those who had attended such a program reported having been tested for HIV within the past 6 months compared to those who had not. In addition, there was higher HIV risk perception among YMSM who had attended a formal HIV intervention program compared to those who had not. One plausible explanation for these findings is that attendance at such prevention programs increases the awareness of HIV as a health risk, which, in turn, may increase positive health behaviors and ultimately decrease HIV transmission risk. Although definitive conclusions regarding the reasons for the continued rise in HIV infection among YMSM cannot be drawn from our findings, they do suggest that this reality may be related to the inability to engage these men in HIV prevention programs. This hypothesis should be tested in future research with larger samples of YMSM.
Our study included YMSM who are disproportionately under-enrolled in HIV prevention research conducted in Los Angeles County. The discrepancy between our sample (young and currently employed) and those of previous studies (older and predominantly econdomically disadvantaged and/or unemployed) may suggest vastly different motivations for HIV prevention efforts that could explain the under-representation of higher-income and younger MSM in HIV prevention research and programs (Shoptaw et al., 2009). The YMSM in our study perceived themselves to be at low risk for HIV infection, which is consistent with previous explanations for low participation in HIV prevention programs among YMSM (Seal et al., 2000; Rutledge et al., 2002). Furthermore, they believed that current HIV prevention programs do not warrant allocation of time toward the effort, because they felt that they already knew what they needed to know about HIV prevention and could locate information quickly online. This finding is consistent with that from larger studies of MSM, which indicate that the majority are seeking health information online, using electronic resources such as search engines, GLBT websites, and online HIV expert postings (Wilkerson et al., 2010).
The expressed desire of YMSM to receive HIV prevention information quickly and easily, coupled with previous qualitative research on YMSM’s desire to have HIV prevention information delivered in a way that is both fun and confidential (Seal et al., 2000), may indicate the utility of online and social network–based HIV prevention for YMSM. Some YMSM in our study also indicated that they would be interested in an HIV prevention program that appealed to their desire to care for the larger gay community. These results may indicate a desire for community connection and altruism, both of which have been postulated as effective strategies for individual and community HIV risk reduction (Ramirez-Valles, 2002). Further research on the applicability of innovative HIV prevention approaches that pay particular attention to YMSM community-level factors is warranted.
The findings of this exploratory study cannot be used to establish an association between participants’ responses and low engagement in existing HIV prevention programs among YMSM. While a counting method for recruiting participants was intended in the original study design to control for selection bias, difficulty in recruiting potential participants in bars and clubs, coupled with limited study personnel, required modifying our recruitment strategy and resulted in a convenience sampling approach. Thus, our findings cannot be generalized to the larger population of YMSM in West Hollywood. However, the semi-structured interviews that were conducted provide several promising directions for HIV prevention development in the near future.
We found that participation in formal HIV prevention programs differed by race/ethnicity. Specifically, we found that higher percentages of racial/ethnic minority YMSM participated in HIV prevention programs than their White counterparts. These results stand in contrast to those of two previous studies, which found no racial/ethnic differences in HIV prevention program participation among MSM (Orellana et al., 2006; Rutledge et al., 2002). While our results may be indicative of successful HIV prevention outreach within racial/ethnic minority communities, they should be interpreted with caution due to the small sample size and data collection strategy. For example, it may be that racial/ethnic minority YMSM attending social venues in West Hollywood are more likely to be engaged in HIV prevention than those who do not attend gay-affiliated venues or who socialize elsewhere. Moreover, surveillance data indicate that African American YMSM remain at particularly high risk for HIV infection (CDC, 2010; HIV Epidemiology Program, Los Angeles County Department of Public Health, 2010), and previous studies have shown that existing HIV prevention programs are more effective in White populations (Johnson et al., 2008), highlighting the need for particular attention to more effective, tailored HIV prevention efforts for affluent YMSM of color. Future research should seek to further elucidate HIV prevention program participation by race/ethnicity.
Conclusions
YMSM interviewed for this study believe that they already have the information they need regarding HIV and their sexual health. Moreover, they use the Internet to access online resources to answer their questions about HIV or sexual risk behaviors. These findings are concerning in light of increasing rates of HIV infection among MSM in general and YMSM in particular, which may signal concepts such as “exceptionalism” when this group of YMSM engages in sexual risk behaviors. They suggest that HIV preventionists must continue to devote attention to elucidating the reasons that YMSM do not participate in HIV prevention programs. While our results indicate that successful approaches to HIV prevention with YMSM may benefit from delivery either online or through social network technology, further exploration of the utility and acceptability of these approaches is needed.
Contributor Information
Ian W. Holloway, Department of Social Welfare, Luskin School of Public Affairs, University of California, Los Angeles.
Julie A. Cederbaum, School of Social Work, University of Southern California, Los Angeles, CA.
Antonette Ajayi, David Geffen School of Medicine, University of California, Los Angeles.
Steven Shoptaw, Department of Family Medicine, David Geffen School of Medicine, University of California, Los Angeles.
References
- Bingham T. State of the HIV epidemic among MSM in Los Angeles. The Body. 2009 Sep; Retrieved from: http://www.thebody.com/content/art54525.html#3/
- Centers for Disease Control and Prevention. HIV/AIDS surveillance report, 2007 (Report No. 19) Atlanta, U.S.: Department of Health and Human Services, Centers for Disease Control and Prevention; 2009. [Google Scholar]
- Centers for Disease Control and Prevention. Prevalence and awareness of HIV infection among men who have sex with men—21 Cities, United States, 2008. Morbidity and Mortality Weekly Report. 2010;59:1201–1207. [PubMed] [Google Scholar]
- Cook TD, Campbell DT. Quasi-experimentation: Design and analysis for field settings. Chicago: Rand McNally; 1979. [Google Scholar]
- Creswell JW. Qualitative inquiry & research design: Choosing among five approaches. Thousand Oaks, CA: Sage Publications, Inc.; 2007. [Google Scholar]
- Hall HI, Song R, Rhodes P, Prejean J, An Q, Lee LM, McKenna MT. Estimation of HIV incidence in the United States. JAMA: The Journal of the American Medical Association. 2008;300(5):520–529. doi: 10.1001/jama.300.5.520. [DOI] [PMC free article] [PubMed] [Google Scholar]
- HIV Epidemiology Program, Los Angeles County Department of Public Health. HIV/AIDS surveillance summary. Los Angeles, CA: HIV Epidemiology Program, Los Angeles County Department of Public Health; 2010. [Google Scholar]
- Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qualitative Health Research. 2005;15(9):1277–1288. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
- IBM. Statistical Package for the Social Sciences (Version 19.0) [Computer software] Chicago, IL: SPSS; 2010. [Google Scholar]
- Iguchi MY, Ober AJ, Berry SH, Fain T, Heckathorn DD, Gorbach PM, Shoptaw S. Simultaneous recruitment of drug users and men who have sex with men in the United States and Russia using respondent-driven sampling: Sampling methods and implications. Journal of Urban Health. 2009;86(1):5–31. doi: 10.1007/s11524-009-9365-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jaffe HW, Valdiserri RO, De Cock KM. The reemerging HIV/AIDS epidemic in men who have sex with men. JAMA: The Journal of the American Medical Association. 2007;298(20):2412. doi: 10.1001/jama.298.20.2412. [DOI] [PubMed] [Google Scholar]
- Johnson W, Diaz R, Flanders W, Goodman M, Hill A, Holtgrave D, McClellan W. Behavioral interventions to reduce risk for sexual transmission of HIV among men who have sex with men. Cochrane Database of Systematic Reviews. 2008;3:CD001230. doi: 10.1002/14651858.CD001230.pub2. [DOI] [PubMed] [Google Scholar]
- Koblin BA, Chesney MA, Husnik MJ, Bozeman S, Celum CL, Buchbinder S, Huang Y. High-risk behaviors among men who have sex with men in 6 US cities: Baseline data from the EXPLORE study. American Journal of Public Health. 2003;93(6):926–932. doi: 10.2105/ajph.93.6.926. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Krippendorff K. Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage Publications, Inc.; 2004. [Google Scholar]
- Lombard M, Snyder-Duch J, Bracken CC. Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human Communication Research. 2002;28(4):587–604. [Google Scholar]
- Miles MB, Huberman AM. Qualitative data analysis. 2nd ed. Newbury Park, CA: Sage Publications; 1999. [Google Scholar]
- Orellana ER, Picciano JF, Roffman RA, Swanson F, Kalichman SC. Correlates of nonparticipation in an HIV prevention program for MSM. AIDS Education and Prevention. 2006;18(4):348–361. doi: 10.1521/aeap.2006.18.4.348. [DOI] [PubMed] [Google Scholar]
- Ramirez-Valles J. The protective effects of community involvement for HIV risk behavior: A conceptual framework. Health Education Research. 17(4):389–403. doi: 10.1093/her/17.4.389. [DOI] [PubMed] [Google Scholar]
- Rutledge SE, Roffman RA, Picciano JF, Kalichman SC, Berghuis JP. HIV prevention and attrition: Challenges and opportunities. AIDS and Behavior. 2002;6(1):69–82. [Google Scholar]
- Sandelowski M. Rigor or rigor mortis: The problem of rigor in qualitative research revisited. Advances in Nursing Science. 1993;16(2):1–8. doi: 10.1097/00012272-199312000-00002. [DOI] [PubMed] [Google Scholar]
- Seal DW, Kelly JA, Bloom FR, Stevenson BI, Broyles LA The Medical College of Wisconsin CITY Project Research Team. HIV prevention with young men who have sex with men: What young men themselves say is needed. AIDS Care. 2000;12(1):5–26. doi: 10.1080/09540120047431. [DOI] [PubMed] [Google Scholar]
- Shoptaw S, Weiss RE, Munjas B, Hucks-Ortiz C, Young SD, Larkins S, Gorbach PM. Homonegativity, substance use, sexual risk behaviors, and HIV status in poor and ethnic men who have sex with men in Los Angeles. Journal of Urban Health. 2009;86(Suppl 1):77–92. doi: 10.1007/s11524-009-9372-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tinsley HEA, Weiss DJ. Interrater reliability and agreement. In: Tinsley HEA, Brown SD, editors. Handbook of applied multivariate statistics and mathematical modeling. San Diego, CA: Academic Press; 2000. pp. 95–124. [Google Scholar]
- Valdiserri RO. Mapping the roots of HIV/AIDS complacency: Implications for program and policy development. AIDS Education and Prevention. 2004;16:426–439. doi: 10.1521/aeap.16.5.426.48738. [DOI] [PubMed] [Google Scholar]
- Weber RP. Basic content analysis. 2nd ed. Newbury Park, CA: Sage Publications; 1990. [Google Scholar]
- Whittemore R, Chase SK, Mandle CL. Validity in qualitative research. Qualitative Health Research. 2001;11(4):522–537. doi: 10.1177/104973201129119299. [DOI] [PubMed] [Google Scholar]
- Wilkerson JM, Smolenski DJ, Horvath KJ, Danilenko GP, Rosser BR. Online and offline sexual health seeking patterns of HIV-negative men who have sex with men. AIDS and Behavior. 2010;14:1362–1370. doi: 10.1007/s10461-010-9794-9. [DOI] [PMC free article] [PubMed] [Google Scholar]