Abstract
Background
Young sexual minority men in the United States have a high incidence rate of HIV infection. Early intervention among this group, that is timed to precede or coincide with sexual initiation, is of critical importance to prevent HIV infection. Despite this, there are very few published randomized controlled efficacy trials testing interventions to reduce sexual vulnerability for HIV acquisition among racially/ethnically diverse, very young, sexual minority men (aged ≤18 years). This paper describes the design of a mobile app-based intervention trial to reduce sexual risk for HIV acquisition and promote health protection in this group.
Methods
This study is a randomized controlled trial of an mHealth-based HIV prevention intervention, MyPEEPS Mobile, among diverse sexual minority cisgender young men, aged 13–18 years. The mobile intervention was adapted from a prior group-based intervention curriculum with evidence of efficacy, designed to be specific to the risk contexts and realities of young sexual minority men, and to include psychoeducational and skill-building components with interactive games and activities. Participants are recruited locally within four regional hubs (Birmingham, AL, Chicago, IL, New York City, NY, Seattle, WA) and nationwide via the Internet, enrolled in-person or remotely (via videoconference), and randomized (1:1) to either the MyPEEPS Mobile intervention or delayed intervention condition. Post-hoc stratification by age, race/ethnicity, and urban/suburban vs. rural statuses is used to ensure diversity in the sample. The primary outcomes are number of male anal sex partners and frequency of sexual acts with male partners (with and without condoms), sex under the influence of substances, and uptake of pre-and post-exposure prophylaxis, as well as testing for HIV and other sexually transmitted infections at 3-, 6- and 9-month follow-up.
Discussion
Behavioral interventions for very young sexual minority men are needed to prevent sexual risk early in their sexual development and maturation. This study will provide evidence to determine feasibility and efficacy of a mobile app-based HIV prevention intervention to reduce sexual risk among this very young group.
Trial registration
ClinicalTrials.gov number, NCT03167606, registered May 30, 2017.
Keywords: HIV prevention, Young men, Sexual minority, Intervention
Background
Young men who have sex (YMSM) with men are vulnerability to HIV infection, particularly racial/ethnic minority YMSM. In the United States in 2017, YMSM made up 93% of all new cases of HIV infection among youth age 13–24 years, with YMSM of color (Black or Latinx) comprising the vast majority of those cases (76%) [1]. Psychosocial factors, such as bullying and other forms of violence and related feelings of isolation; contextual factors (e.g., family, peer and partner relationships); as well as high number of partners, low rates of condom use, and low rates of testing for HIV and other sexually transmitted infections (STIs), are contributing factors [2]. Intervention among very young sexual minority men, prior to or at the time of sexual initiation, is an important strategy to increase sexual health education and skill-building; indeed, NIH’s Strategic Plan for HIV highlights the importance of addressing HIV prevention with key populations at-risk for HIV, including youth as young as 13 [3]. Currently, however, there are no interventions in the Center for Disease Control and Prevention’s Compendium of Evidence-based Interventions that target diverse sexual minority men 18 years of age and younger [4].
Evidence suggests that mHealth-based intervention approaches may be particularly salient for technology savvy youth, but also a promising method to increase reach to key populations with educational information, digital media, and/or game-based learning, aimed at reducing HIV risk behaviors [5, 6]. In a review of mHealth interventions for high risk MSM, Schnall and colleagues found that web-based videos and education modules reduced HIV risk behavior and promoted HIV testing [7]. Among youth ages 13–29, evidence suggests that web-based interactive and educational approaches are efficacious for delaying sexual initiation [8, 9], increasing knowledge of HIV/STIs, and promoting condom self-efficacy [10]. MyPEEPS Mobile is an intervention adapted from a group-based HIV prevention curriculum, developed via formative research [11–13], for diverse YMSM, ages 16–20. The group-based intervention demonstrated evidence of efficacy to reduce sexual risk behavior in this population [14]. We adapted the group-based intervention curriculum to mobile app for a younger (≤18 years of age) and more diverse group (i.e., to include Native Americans, Asian Americans) through a user-centered and iterative design process and tested for feasibility, acceptability, and usability in a recent set of studies [15–18].
Study objective
The primary objective of this study is to test the efficacy of the MyPEEPS Mobile intervention to reduce sexual risk for HIV acquisition and promote health behavior among young sexual minority men, aged 13 to 18 years.
Methods/design
Design
This study is a nationwide two-arm randomized controlled trial among racially and ethnically diverse sexual minority young men, aged 13–18 years. Participants are randomized to either the MyPEEPS Mobile intervention or a delayed intervention comparison condition and followed at 3-, 6-, and 9-months post-baseline. The intervention is delivered in the period between baseline and the 3-month follow-up visit. The comparison group is crossed-over to intervention at the 9-month visit and followed to 12-months post-baseline (Fig. 1).
Fig. 1.
MyPEEPS CONSORT Diagram
Identification and recruitment of participants
Participants are recruited locally within four regional hubs (Birmingham, AL, Chicago, IL, New York City, NY, Seattle, WA) and nationwide via the Internet. Local outreach occurs actively in youth- and sexual minority-focused community organizations and events, as well as passively via posted flyers. Internet-based outreach occurs primarily via paid and targeted ads on platforms frequented by adolescents (e.g., Instagram, Snapchat). Eligibility criteria include: (1) ages 13–18 years; (2) assigned male sex at birth and, at the time of enrollment, self-identify as male; (3) understand and read English; (4) live in the United States; and (5) own a smartphone (6) report sexual interest in other men (7) has either kissed another man or plan on having sex with a man in the next year and (7) self-reported HIV-negative or unknown HIV status. Participants are excluded if investigators determine that participation may be detrimental to them or the study (e.g., severe cognitive deficit) or they participated in the pilot phase of the study.
Our goal is to enroll 700 participants overall, and at least 70 participants of each year of age (i.e., aged 13, aged 14, etc.) in each of six racial/ethnic groups [i.e., American Indian or Alaskan native, Asian or Asian American, Black or African American, Latinx (of any race), Native Hawaiian or other Pacific Islander, and White]; and among youth living in rural-designated areas.
Randomization
On a rolling basis, participants are randomized in blocks of four with a 1:1 ratio to either MyPEEPS Mobile or a delayed intervention condition. The random assignments are generated by a computerized random number generator by the Principal Investigator (RS). Random assignments are concealed from both participants and study staff until they are revealed at the point of randomization after baseline data collection has been completed. The statistician will be blinded to condition for analysis to reduce potential bias during statistical analyses.
Description of the intervention: MyPEEPS Mobile
MyPEEPS Mobile is based on the Social-Personal Framework [19], which builds on Social Learning Theory [20] by adding important psychosocial (e.g., affect dysregulation) and contextual risk factors (e.g., family, peer, and partner relationships) related to youth vulnerability to HIV risk. MyPEEPS Mobile provides educational information about HIV and STIs among YMSM, raises awareness about minority stress (e.g., due to sexual identity), and builds skills for condom use, emotion regulation, and negotiating interpersonal and substance-related risk. The learning process is facilitated through the stories of four “peeps” (Philip, Nico, Artemio, and Tommy), who are composites of YMSM who participated in the formative phase of the original MyPEEPS intervention development process [11, 12]. A running theme throughout the intervention is sexual risk reduction and goal-setting through an activity called, “BottomLine,” in which participants are challenged to articulate how much risk they are willing to accept for different sexual acts (e.g., anal sex, oral sex) and to continually re-consider these limits after exposure to the intervention activities (i.e., building knowledge, self-awareness, as well as self-efficacy). Using a responsive web design, the conventional web site is viewable on small screens and usable with touch screens. The content is delivered through a series of games, scenarios and role-plays within 21 mobile activities that are divided into four sequential modules or “PEEPScapades,” which are targeted to younger and less sexually experienced sexual minority young men [18]. All content is accessible for the period between randomization and the 3-month follow-up visit (i.e., content does not expire and can be re-visited), and must be completed in a linear manner. Movement through the app is encouraged with both in-app “trophies” and monetary incentives for the time it takes to complete the activities. Privacy is protected via log-in and password credentials and automatic log-off of the app after 20 min of inactivity.
Delayed intervention condition
Participants in the delayed intervention condition are provided with access to the MyPEEPS Mobile app at the 9-month visit with log-in and password credentials. Procedures for app access and incentives for completion are the same as for the intervention condition. Access is provided through the 12-month study visit.
Study assessments
Participants are enrolled in-person or remotely (via videoconference with written electronic assent or consent) by study staff, then complete study assessments at baseline and follow-up via computer-assisted self-interviewing (CASI; either in-person or remotely via a web link). Follow-up assessments are conducted at 3-, 6- and 9-month follow-up visits, with an additional follow-up visit at 12-months for the delayed intervention group. Participants are required to show their ID, and the face on their ID are matched to their face on the videoconferencing screen. Each participant was given a survey link matched to their study ID upon confirmation of their identity. All study data are securely stored at the primary study site in a limited access database by study ID. All hard copy participant information (e.g., study checklists, consent forms) are securely stored at each study site in locked file cabinets with limited access.
Primary outcome
The primary outcome is number of male sex partners and frequency of anal sex acts with male partners (with and without condoms), sex under the influence of substances, and uptake of pre-and post-exposure prophylaxis (PrEP, PEP), as well as testing for HIV and other STIs (e.g., chlamydia, gonorrhea, hepatitis, HPV/genital warts, syphilis). Items assessing sexual behavior are self-administered and adapted for sexual minority men from the AIDS Risk Behavior Assessment (ARBA) [21]. Items assess sexual behavior in the prior 30 days and 3 months. The basis for construction of the primary outcome is a set of sequential questions asking the participant to estimate the number of sex partners they had in the recall period and the number of condomless sex acts by type of sex (anal, oral, vaginal) with these partners. Items to assess uptake of PrEP, PEP, and HIV testing history are based on those used in prior studies of YMSM [22, 23].
Statistical analysis
All multivariate analyses will be preceded by standard bivariate analyses to describe key variables and relationships among them. These analyses will include means, frequency tables, histograms, and examination of distributions to promote data quality. All statistical tests will be two-sided tests with the level of significance at 0.05. We propose to use generalized linear mixed models (GLMM) to analyze both count and binary outcomes to determine efficacy of the intervention. For the missing values at the baseline or partial baseline collected data, we will use a multiple imputation (MI) approach [24]. Models will also be run on the non-imputed data with full-information maximum likelihood (FIML) estimation as an alternative for the MI method [25]. Rates of reduction will be calculated controlling for all other covariates in the multivariable model. Models will be calculated by using the GLIMMIX and MIANALYZE procedures in SAS, version 9.4, and model fit will be evaluated by diagnostic statistics and residual plots.
Sample size calculation
We target enrollment of 700 participants overall, and at least 70 participants of each subgroups of age, racial/ethnicity, and rural-designated areas. We estimated the statistical power for the main outcome (recent number of condomless anal sex acts with male partners) based on two scenarios: (1) to examine overall effect with total subjects; and (2) to conduct stratified analysis to examine the effects in some subgroups (age, racial/ethnicity, and in rural areas). The following assumptions are used for the power estimation: (1) an 80% retention rate (analytic sample = 560 for total, and 56 for subgroups); (2) a conservative and high intra-cluster correlation (ICC) of 0.8, (3) mean number of recent condomless anal sex acts with male partners at baseline is 1.2 [14] and (4) all power estimations are based on α = 0.05 and 2-sided tests. Findings from the prior MyPEEPS study indicate that the post-intervention number of condomless anal sex acts decreased by 63%, or a relative risk (RR) of 0.37 [14]. However, the large effect was not statistically significant. Because the estimated effect size of the intervention was unreliable, instead, we use RR = 0.73, one standard error over the estimated RR of 0.37. This provides a conservative estimation of the minimum sample size need. For the subgroup analyses, we use the effect size of RR = 0.37. To examine overall effect with all participants, we will have 97% power to detect a relative risk of 0.73 with analytic sample size of 560. Secondly, for the stratified analyses, we will have 92% power to detect a relative risk of 0.37 in subgroups with analytic sample size of 56.
Primary study data will be analyzed as soon as possible after the end of data collection, with study findings disseminated in peer-review public health journals.
Discussion
We describe herein the design of the MyPEEPS Mobile study, a randomized controlled efficacy trial of an mHealth intervention with educational and gaming components to reduce HIV risk behavior and promote protective health behaviors in young sexual minority men aged 13–18. The intervention is based on a prior group-based intervention with evidence of efficacy [14], as well as both theory and empirical evidence for the challenges faced by sexual minority men to protect their own sexual health [11, 13, 22, 26–29]. The design of this study has several strengths, including its focus on very young sexual minority men, prior to or at the point of sexual initiation; testing a mobile-adapted evidence-based intervention informed by the experiences of YMSM; and the rigorous evaluation design, sample size and analysis by age, race/ethnicity, and rural-based subgroups.
The defining characteristics of this intervention are that it focuses on important psychosocial (e.g., affect dysregulation) and contextual factors (e.g., family, peer and partner relationships) related to youth vulnerability to HIV risk. It provides educational information about HIV and STIs among YMSM, raises awareness about minority stress, and build skills for condom use, emotion regulation, and negotiating interpersonal and substance-related risk. Given the vulnerability of young sexual minority men to HIV infection and the current plateau in progress in HIV prevention [30], sexuality education that is specific to their needs and builds skills to manage their own sexual health is of critical importance and is consistent with current recommendations for an empowerment approach to sexuality education [31]. Empowerment education, critical thinking, and communication skill-building provide the foundation for protective sexual health decision-making.
Early intervention is critical, yet there are no evidence-based interventions developed specifically for very young sexual minority men. Their vulnerability increases dramatically during the period of adolescence and young adulthood. In 2017, less than 1% of youth who received an HIV diagnosis were aged 13 to 14, 21% were aged 15 to 19, and 79% were aged 20 to 24 [1]. This study provides important information and skills prior to or coincident with sexual initiation, to inform sexual decision-making and directly address this developmental trajectory of vulnerability. Furthermore, MyPEEPS Mobile is delivered in a mobile app-format aimed at increasing both uptake and scalability. Our web-based outreach approach and mobile app format increase the potential reach of this intervention to this young group.
Finally, this intervention is well powered to detect effects in a randomized trial design, which increases scientific rigor. In addition, it is focused on gathering a diverse sample by age, race/ethnicity, and rural representation in sufficient size for testing of subgroup effects, which increases generalizability. If this trial of MyPEEPS Mobile demonstrates evidence of efficacy, this approach has potential for broad public health impact.
Acknowledgements
Not applicable.
Abbreviations
- ARBA
AIDS Risk Behavior Assessment
- CASI
Computer Assisted Self Interviewing
- FIML
Full Information Maximum Likelihood Estimation
- GLMM
Generalized Linear Mixed Models
- HIV
Human Immunodeficiency Virus
- ICC
Intra-Cluster Correlation
- mHealth
Mobile Health
- MI
Multiple Imputation
- PEP
Post-Exposure Prophylaxis
- PrEP
Pre-Exposure Prophylaxis
- RR
Relative Risk
- STI
Sexually Transmitted Infection
- YMSM
Young Men Who Have Sex with Men
Authors’ contributions
LK, RG and RS contributed to the design of all aspects of the study. MH, SH, and CP contributed to the proposed study design. HJ designed the proposed data analysis. LK, RG and RS drafted the manuscript. The manuscript has been read and approved by all authors.
Authors’ information
Not applicable.
Funding
This study is funded by the National Institute on Minority Health and Health Disparities of the National Institutes of Health under Award Number U01MD011279. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The funding source has no role in the original design of this study, analysis and interpretation of data, or decision to submit results.
Availability of data and materials
Not applicable.
Ethics approval and consent to participate
This protocol has been approved by the Institutional Review Board of Columbia University as the single IRB (IRB-AAAQ6500) with a waiver of parental permission for participation of minors (aged 13–17). Study participants complete a written informed e-assent or e-consent process through a web portal prior to participation in research activities. Current protocol version: September 4, 2019. Any modifications to the protocol are submitted to and approved by the IRB of Record prior to implementation. Spontaneously reported adverse events and unintended effects of the trial are tracked by the study Principal Investigators and reported to the IRB of Record. This study is monitored by a Data Safety and Monitoring Board (DSMB), the composition of which may be obtained from the study principal investigator at the lead institution. No criteria for discontinuing or modifying the intervention or trial stopping rules were defined for this study.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Centers for Disease Control and Prevention (CDC) HIV Surveillance Report. 2018. Diagnoses of HIV infection in the United States and dependent areas, 2017. [Google Scholar]
- 2.Centers for Disease Control and Prevention (CDC) HIV and youth. 2019. [Google Scholar]
- 3.National Institutes of Health, Office of AIDS Research . NIH strategic plan for HIV and HIV-related research. Bethesda: Office of AIDS Research, US Department of Health and Human Services; 2019. [Google Scholar]
- 4.Centers for Disease Control and Prevention (CDC) Compendium of evidence-based interventions and best practices for HIV prevention. 2019. [Google Scholar]
- 5.Cordova D, Mendoza Lua F, Ovadje L, Hong E, Castillo B, Salas-Wright CP. Randomized controlled trials of technology-based HIV/STI and drug abuse preventive interventions for african american and hispanic youth: systematic review. JMIR Public Health Surveill. 2017;3(4):e96. doi: 10.2196/publichealth.7129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Hightow-Weidman LB, Muessig KE, Bauermeister JA, LeGrand S, Fiellin LE. The future of digital games for HIV prevention and care. Curr Opin HIV AIDS. 2017;12(5):501–507. doi: 10.1097/COH.0000000000000399. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Schnall R, Travers J, Rojas M, Carballo-Dieguez A. eHealth interventions for HIV prevention in high-risk men who have sex with men: a systematic review. J Med Internet Res. 2014;16(5):e134. doi: 10.2196/jmir.3393. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Bull S, Pratte K, Whitesell N, Rietmeijer C, McFarlane M. Effects of an internet-based intervention for HIV prevention: the youthnet trials. AIDS Behav. 2009;13(3):474–487. doi: 10.1007/s10461-008-9487-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Markham CM, Shegog R, Leonard AD, Bui TC, Paul ME. +CLICK: harnessing web-based training to reduce secondary transmission among HIV-positive youth. AIDS Care. 2009;21(5):622–631. doi: 10.1080/09540120802385637. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Guse K, Levine D, Martins S, Lira A, Gaarde J, Westmorland W, Gilliam M. Interventions using new digital media to improve adolescent sexual health: a systematic review. J Adolesc Health. 2012;51(6):535–543. doi: 10.1016/j.jadohealth.2012.03.014. [DOI] [PubMed] [Google Scholar]
- 11.Bird JD, LaSala MC, Hidalgo MA, Kuhns LM, Garofalo R. “I had to go to the streets to get love”: pathways from parental rejection to HIV risk among young gay and bisexual men. J Homosex. 2017;64(3):321–342. doi: 10.1080/00918369.2016.1179039. [DOI] [PubMed] [Google Scholar]
- 12.Hidalgo MA, Cotten C, Johnson A, Kuhns LM, Garofalo R. ‘Yes, I am more than just that,’:Gay/bisexual young men residing in the United States discuss the influenceof minority stress on their sexual risk behavior prior to HIV infection. Int J Sex Health. 2013;25(4):291–304. doi: 10.1080/19317611.2013.818086. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Lyons Thomas, Johnson Amy K., Garofalo Robert. “What Could Have Been Different”: A Qualitative Study of Syndemic Theory and HIV Prevention Among Young Men Who Have Sex With Men. Journal of HIV/AIDS & Social Services. 2013;12(3-4):368–383. doi: 10.1080/15381501.2013.816211. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hidalgo MA, Kuhns LM, Hotton AL, Johnson AK, Mustanski B, Garofalo R. The MyPEEPS randomized controlled trial: a pilot of preliminary efficacy, feasibility, and acceptability of a group-level, HIV risk reduction intervention for young men who have sex with men. Arch Sex Behav. 2015;44(2):475–485. doi: 10.1007/s10508-014-0347-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Cho H, Powell D, Pichon A, Thai J, Bruce J, Kuhns LM, Garofalo R, Schnall R. A mobile health intervention for HIV prevention among racially and ethnically diverse young men: usability evaluation. JMIR Mhealth Uhealth. 2018;6(9):e11450. doi: 10.2196/11450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ignacio M, Garofalo R, Pearson C, Kuhns LM, Bruce J, Scott Batey D, Radix A, Belkind U, Hidalgo MA, Hirshfield S, et al. Pilot feasibility trial of the MyPEEPS mobile app to reduce sexual risk among young men in 4 cities. JAMIA Open. 2019;2(2):272–279. doi: 10.1093/jamiaopen/ooz008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Schnall R, Kuhns L, Hidalgo M, Hirshfield S, Pearson C, Radix A, Belkind U, Bruce J, Batey DS, Garofalo R. Development of MyPEEPS mobile: a behavioral health intervention for young men. Stud Health Technol Inform. 2018;250:31. [PMC free article] [PubMed] [Google Scholar]
- 18.Schnall R, Kuhns LM, Hidalgo MA, Powell D, Thai J, Hirshfield S, Pearson C, Ignacio M, Bruce J, Batey DS, et al. Adaptation of a group-based HIV risk reduction intervention to a mobile app for young sexual minority men. AIDS Educ Prev. 2018;30(6):449–462. doi: 10.1521/aeap.2018.30.6.449. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Donenberg G, Pao M. Youths and HIV/AIDS: Psychiatry's role in a changing epidemic. J Am Acad Child Adolesc Psychiatry. 2005;44(8):728–747. doi: 10.1097/01.chi.0000166381.68392.02. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Bandura A. Social learning theory. Englewood Cliffs: Prentice-Hall; 1977. [Google Scholar]
- 21.Donenberg GR, Emerson E, Bryant FB, Wilson H, Weber-Shifrin E. Understanding AIDS-risk behavior among adolescents in psychiatric care: links to psychopathology and peer relationships. J Am Acad Child Adolesc Psychiatry. 2001;40(6):642–653. doi: 10.1097/00004583-200106000-00008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Garofalo R, Hotton AL, Kuhns LM, Gratzer B, Mustanski B. Incidence of HIV infection and sexually transmitted infections and related risk factors among very young men who have sex with men. J Acquir Immune Defic Syndr. 2016;72(1):79–86. doi: 10.1097/QAI.0000000000000933. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Kuhns LM, Hotton AL, Schneider J, Garofalo R, Fujimoto K. Use of pre-exposure prophylaxis (PrEP) in young men who have sex with men is associated with race, sexual risk behavior and peer network size. AIDS Behav. 2017;21(5):1376–1382. doi: 10.1007/s10461-017-1739-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Rubin DB. Multiple imputation for nonresponse surveys. New York: Wiley; 1987. [Google Scholar]
- 25.Enders CK. A primer on maximum likelihood algorithms available for use with missing data. Struct Equ Model Multidiscip J. 2001;8(1):128–141. doi: 10.1207/S15328007SEM0801_7. [DOI] [Google Scholar]
- 26.Dowshen N, Binns HJ, Garofalo R. Experiences of HIV-related stigma among young men who have sex with men. AIDS Patient Care STDs. 2009;23(5):371–376. doi: 10.1089/apc.2008.0256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Garofalo R, Gayles T, Bottone PD, Ryan D, Kuhns LM, Mustanski B. Racial/ethnic difference in HIV-related knowledge among young men who have sex with men and their association with condom errors. Health Educ J. 2015;74(5):518–530. doi: 10.1177/0017896914549485. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Garofalo R, Herrick A, Mustanski BS, Donenberg GR. Tip of the iceberg: young men who have sex with men, the internet, and HIV risk. Am J Public Health. 2007;97(6):1113–1117. doi: 10.2105/AJPH.2005.075630. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Garofalo R, Mustanski B, Johnson A, Emerson E. Exploring factors that underlie racial/ethnic disparities in HIV risk among young men who have sex with men. J Urban Health. 2010;87(2):318–323. doi: 10.1007/s11524-009-9430-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Harris NSJA, Huang YA, et al. Vital signs: status of human immunodeficiency virus testing, viral suppression, and HIV preexposure prophylaxis — United States, 2013–2018. MMWR Morb Mortal Wkly Rep. 2019;68:1117–1123. doi: 10.15585/mmwr.mm6848e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Haberland N, Rogow D. Sexuality education: emerging trends in evidence and practice. J Adolesc Health. 2015;56(1 Suppl):S15–S21. doi: 10.1016/j.jadohealth.2014.08.013. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Not applicable.