Acceptability of a Mobile Smartphone Application Intervention to Improve Access to HIV Prevention and Care Services for Black Men Who Have Sex with Men in the District of Columbia

Abstract

Eliminating racial HIV disparities among men who have sex with men (MSM) will require a greater uptake of HIV prevention and care interventions among Black MSM (BMSM), yet such strategies generally require meaningful engagement in a health care system that often does not meet the unique needs of BMSM. This study assessed the acceptability of, and correlates of having favorable perceptions of, a mobile smartphone application (app) intervention for BMSM that aims to remove structural barriers and improve access to culturally relevant HIV prevention and care services. An Internet-based sample of 93 BMSM completed an online survey on their perceptions of the app using 14 items measured on a 100-point visual analogue scale that were validated in exploratory factor analysis (alpha=0.95). Among the sample, perceptions of two sample app modules were generally favorable and most BMSM agreed that they would use the modules (81.2% and 87.1%). Correlates of having favorable perceptions included trusting medical advice from social networks, lacking private health insurance, and not having accessed a primary care physician in the last year. Our findings warrant the further development of this app and point to subgroups of BMSM for which it may have the greatest impact.

Introduction

Current HIV prevention and treatment modalities are failing to control the HIV/AIDS epidemic among Black men who have sex with men (BMSM) in the United States (US) (Rosenberg, Millett, Sullivan, del Rio, & Curran, 2014). An estimated 3.0 to 5.1% of BMSM become infected with HIV each year (Balaji et al., 2013; Koblin et al., 2013) and BMSM are infected at a rate six times that of White men who have sex with men (MSM) (Purcell et al., 2012). Locally, in the District of Columbia (DC), BMSM represent 26% of people living with HIV and 25% of newly diagnosed HIV cases (DC DOH, 2013). The racial disparity in HIV incidence, however, is not adequately explained by differences in individual-level risk behaviors (Millett et al., 2012), and is better attributed to poor health outcomes of the HIV care continuum (Rosenberg et al., 2014). Indeed, BMSM living with HIV are less likely than other HIV-infected MSM to receive an HIV diagnosis (CDC, 2011; Millett, Peterson, Wolitski, & Stall, 2006), be linked and retained in care, and achieve and maintain viral suppression (Millett et al., 2012; Oster et al., 2011). These gaps in HIV care contribute to greater rates of HIV transmission among sexual networks of BMSM (Marks, Crepaz, & Janssen, 2006; Vernazza, Eron, Fiscus, & Cohen, 1999). In addition, recent research suggests that the disparity in HIV incidence between BMSM and White MSM can also be largely explained by differences in dyadic level characteristics (i.e., the extent to which men report partners from groups that likely have higher HIV prevalence) and in the possession of health insurance (Sullivan et al., 2015). Thus, it is important to maximize access to HIV prevention, care, and treatment services for BMSM in settings where partner pool risk is a driver of high incidence (Sullivan et al., 2015).

Eliminating racial disparities in HIV incidence and care outcomes will require a greater uptake of HIV prevention and care interventions among BMSM (Rosenberg et al., 2014). Most HIV prevention interventions and treatment approaches – behavioral sexual risk reduction counseling, pre-exposure prophylaxis (PrEP), treatment as prevention (TasP), linkage and retention in care, and adherence to antiretroviral therapy (ART) – require interaction with health care infrastructure, yet it is often difficult for BMSM to access culturally appropriate health care services due to known structural barriers (Levy et al., 2014). Such barriers include experiences of racism and homophobia during health care visits, discomfort and distrust associated with disclosing sexual behavior to health care providers for fear of ramifications, low cultural competency of providers for working with BMSM of diverse sexual identities, stigmatization of voluntary HIV testing, and low density of HIV prevention services in areas where BMSM live (Levy et al., 2014). On the whole, prevention and treatment strategies are failing to slow the epidemic because they require meaningful engagement in a health care system that often does not meet the unique health needs of BMSM. Novel interventions are urgently needed to reduce and/or eliminate these barriers to services for BMSM.

Smartphone applications (apps) provide an important opportunity to reach and engage BMSM in primary and secondary HIV prevention services across the HIV care continuum. For those with existing barriers to care, such apps may represent a critical link to enter and remain in the health care system. Smartphones not only offer standard features such as text and voice communication, but also advanced computing and communication capability, including Internet access and geo-positioning systems (Boulos, Wheeler, Tavares, & Jones, 2011). Among all US adults, non-Hispanic Blacks are more likely than Whites to own a smartphone (70% vs. 61%), and younger adults are the age group most likely to own one (85% of those aged 18–29, 79% of those aged 30–49, 54% of those aged 50–64, and 27% of those aged ≥65) (Smith, 2015). Among MSM, one study found that non-White men were nearly six times more likely than White men to report wanting to receive sexual health information via an app (Sun, Stowers, Miller, Bachmann, & Rhodes, 2014). In another study conducted among young BMSM, mobile technologies were a widely used and acceptable means for an HIV intervention (Muessig et al., 2013).

In fact, researchers and practitioners are increasingly using smartphone interventions for HIV prevention and care. In a systematic review, Muessig, Nekkanti, Bauermeister, Bull, and Hightow-Weidman recently identified the use of smartphone-based HIV interventions in four published studies and 14 ongoing projects (2015). These smartphone interventions address various stages of the HIV care continuum, including primary prevention, testing, linkage to care, retention in care, initiation of ART, ART adherence, and secondary prevention (Muessig et al., 2015). Eight ongoing smartphone-based or mobile-optimized projects specifically focus on HIV prevention and care among MSM in the US, including HIV-negative MSM, HIV-positive MSM, young MSM, stimulant-using MSM, and BMSM. Of these ongoing projects, two Internet-based interventions that are optimized for mobile devices specifically focus on HIV prevention and care among BMSM (Muessig et al., 2015). One mobile-optimized website, healthMpowerment, aims to reduce risky sexual behaviors and build community among HIV-positive and negative young BMSM and transgender women in North Carolina (Hightow-Weidman et al., 2011; LeGrand, Muessig, Pike, Baltierra, & Hightow-Weidman, 2014; Muessig, Baltierra, Pike, LeGrand, & Hightow-Weidman, 2014; Muessig et al., 2015). A second mobile-optimized website provides young BMSM in New York City with a tailored recommendation of their optimal HIV testing approach (Muessig et al., 2015).

Despite these emerging and innovative websites and apps focusing on critical health needs of MSM and BMSM, researchers have yet to fully harness the utility of smartphone apps and other technological advances to address structurally rooted reasons why BMSM frequently cannot access culturally relevant HIV services across the HIV continuum of care (Muessig et al., 2015). More specifically, there is a great need for an app that increases access to culturally relevant HIV prevention and care services for BMSM and improves the user’s self-efficacy in initiating communication about sexual and other health needs with health care providers. In this study, we surveyed BMSM in DC metropolitan statistical areas to investigate the potential utility of a smartphone app that aims to eliminate barriers to HIV prevention and care services among this population. The aim of this study was to assess the acceptability of such a smartphone app specifically designed to meet the needs of BMSM in DC. We also explored the psychometric properties of the survey items used to assess participants’ perceptions of the app and subsequently identified correlates of having favorable perceptions of the app’s utility.

Methods

Data Collection

We collected the data reported in this paper during an Internet-based study for the PRISM (Pursuing Real and Innovative Ideas to Remove Structural Barriers for Men) project conducted among BMSM in the DC metropolitan statistical areas. An online survey was used to evaluate the acceptability of a mobile smartphone app intervention that would aim to improve access to culturally relevant HIV prevention and care services and facilitate communication about sexual and other health needs between BMSM and providers.

We conceptualized the design of the smartphone app based on formative research conducted during an earlier phase of this project among a community-based sample of 100 BMSM. In brief, the objective of this formative phase was to identify and characterize socio-cultural and structural barriers to accessing HIV prevention and care services for BMSM. From July 2012 to February 2013, an ethnographer conducted in-depth individual interviews with 25 of these men to collect qualitative data on structural dimensions based on the dynamic social systems model developed by Latkin, Weeks, Glasman, Galletly, and Albarracin (2010). Subsequent transcription and team coding of these data informed the development of a mixed-methods instrument that was administered to 75 men between October 2013 and June 2014. At these study visits, BMSM who had experienced at least one barrier to engagement in health care provided self-reported data on their experiences with health care infrastructure during a computer-assisted quantitative self-interview and an in-depth qualitative interview that was digitally audio-recorded and transcribed verbatim by a professional transcription service. In this initial phase, we found that BMSM placed great importance on developing trusting relationships with nonjudgmental health care providers and expressed discomfort with initiating conversations about sexual behavior and HIV. The data from this parent study demonstrated a critical need for targeted and culturally appropriate interventions that would address the complex reasons why BMSM often do not access HIV services.

For this current study, using online recruitment methods, we invited BMSM to complete an anonymous, Internet-based survey on their perceptions of the potential utility of a smartphone app intervention. We distributed Internet-based recruitment materials that included a hyperlink to the survey via Facebook and email communication with professional networks and community-based organizations in DC that serve populations inclusive of BMSM (Figure 1). We used REDCap electronic data capture tools hosted at the Clinical and Translational Science Institute (CTSI) at Children’s National Health System to manage survey data collected between September and November 2014. REDCap is a workflow methodology and software solution for designing clinical and translational research databases (Harris et al., 2009).

Figure 1.

Internet-based recruitment material.

The survey’s introductory screen described the study as “a study about Black men’s health needs… [that] can be done on a tablet, a smartphone, or a computer.” If men responded that they were interested in learning more, the next screen indicated that we would first ask questions to determine if they were eligible. In order to be eligible for the study, participants had to: (1) be 18 years of age or older; (2) identify as Black, African American, Caribbean Black, or Multiethnic Black; (3) be labeled male at birth; (4) identify as gay, same gender loving, homosexual, or bisexual, or have had sex with a male, transgender, or intersex individual in the last 12 months; and (5) live in the DC metropolitan statistical areas (Figure 2). Using conditional logic in REDCap, ineligible respondents were informed that they were not selected for the survey. Eligible participants were notified that the survey would take 15–20 minutes to complete and that their responses would be voluntary and anonymous. At the end of the survey, participants could access a link to a secure Google form to provide unlinked contact information and retrieve a $25 gift check by mail or at our office. The George Washington University (GWU) Institutional Review Board approved all study protocols and instruments.

Figure 2.

Screenshot of eligibility criteria in REDCap survey.

Measures

The REDCap survey collected data on perceptions of the potential utility of an app that would aim to eliminate barriers to accessing HIV prevention and care services and improve communication of sexual and other health needs between BMSM and providers. We created questions on perceptions that were in reference to an initial prototype of two app modules named “Talking with my Health Care Provider” and “Sexual Health.” These two sample modules were designed based on formative data to facilitate the user’s own understanding of the health concerns most important to him and improve his self-efficacy in seeking relevant resources and health services that may be difficult to access. After a description of the first module – “Many people don’t feel comfortable asking their medical providers for all the services they need during a visit, forget to ask all their questions, or can’t ask since there is not enough time during the visit. In this part of the app, we are trying to make it easier to tell your provider what you want out of the visit” – participants viewed a sample screenshot with selection options that corresponded with health concerns that participants may want to communicate to providers (Figure 3). The survey language explained that the user would be able to select specific concerns that he would want to discuss with his provider and the app would then populate relevant information, tools, and resources that could be saved, printed, and discussed with providers during visits. We then provided participants with a description of the second module: “The goal of this part of the app is to help you get what you need from your medical provider to support your sexual health, since sometimes it is hard to talk with providers about sexual health and prevention.” Similarly, this module would allow the user to select specific sexual health concerns that would populate relevant information, tools, and resources to be saved, printed, and discussed with providers during visits (Figure 4).

Figure 3.

Screenshot of sample smartphone app module on communicating with health care providers.

Figure 4.

Screenshot of sample smartphone app module on sexual health.

Based on descriptions of the two modules, we asked participants to respond to 14 items – seven on the module pertaining to communicating with providers (numbered C1–C7) and seven on the module pertaining to sexual health (numbered S1–S7). These items were measured using a 100-point visual analogue scale by which participants could electronically select a response (from 0 to 100) consistent with the extent to which they agreed with each item (listed in Table IV) (Miller & Ferris, 1993). We explained to participants that they could drag the ‘slider’ – the position on the visual analogue scale representing their response – from the default midway point (i.e., 50 out of 100) to the desired point, consistent with the extent to which they agreed with each statement. Participants were also asked an open-ended question: “Do you have any ideas about how to make it easier for people to get the care and services they need, including HIV prevention services?” In addition, participants responded to questions on demographics, smartphone characteristics, sexual behaviors, health seeking behaviors, health knowledge, health care experiences, and health care beliefs that were adopted from previous pilot work with BMSM in DC during HIV Prevention Trials Network (HPTN) 061 (Koblin et al., 2013; Mayer et al., 2014), National HIV Behavioral Surveillance (NHBS) (Magnus et al., 2010), and PRISM studies (these items and their response options are listed in Table I).

Table IV.

Factor loadings, item-subscale correlations, and item-total correlations for items measuring perceptions of a smartphone application intervention among Black men who have sex with men in the District of Columbia Metropolitan Statistical Areas, 2014 (N=93)^a.

Itemsb	Factor Loadings		Correlation Coefficients
	Factor 1	Factor 2	Item-Total	Item-Subscale
Factor 1: Utility for Accessing health services (Access)c
C1. I would use this app if it were free	0.95	−0.14	0.68	0.82
C2. It would be easier to get the medical services I need if I had this app	0.89	−0.04	0.73	0.83
C3. I would be more likely to get support in getting health insurance	0.69	0.22	0.79	0.78
S1. I would use this app if it were free	0.91	−0.06	0.72	0.85
S2. It would be easier to get the sexual health services I need if I had this app	0.61	0.35	0.83	0.76
Factor 2: Ease in Communicating health needs (Communication)d
C4. It would be easier to remember to tell the doctor what I need at my visit	−0.15	0.86	0.61	0.66
C5. It would be easier to ask for an HIV test if I had this app	−0.11	0.92	0.70	0.78
C6. It would be easier to ask the doctor for routine preventive care (like flu shots)	0.14	0.75	0.78	0.76
S4. It would be easier to remember to tell the doctor what I need at my visit	0.29	0.58	0.75	0.70
S5. I would be more likely to ask about PrEP if I had this app	0.26	0.59	0.75	0.74
S6. I would be more likely to ask about condoms if I had this app	0.27	0.60	0.76	0.72
S7. I would be more likely to get an HIV test if I had this app	0.12	0.77	0.78	0.81
Items removed
C7. I would be more comfortable talking with my health care provider	0.58	0.32	0.79	NA
S3. It would be easier to talk about sex with my medical provider if I had this app	0.60	0.34	0.82	NA

^aParticipants included 91 self-identified males, one self-identified transgender woman, and one self-identified intersex individual.

^bItems numbered using “C” are in reference to the hypothetical mobile app module on communicating with health care providers and items numbered using “S” are in reference to the hypothetical mobile app module on sexual health.

^cFactor 1 was named for the general utility of the app for accessing health insurance and health services (Access).

^dFactor 2 was named for the app increasing ease in communicating health needs to the health care provider (Communication).

Table I.

Characteristics of an Internet-based sample of Black men who have sex with men in the District of Columbia Metropolitan Statistical Areas, 2014 (N=93)^a.

Demographic Characteristics	n (%)
Age
21–29	40 (43.0)
30–39	31 (33.3)
40–49	14 (15.1)
≥50	8 (8.6)
Sexual Orientationb
Gay, homosexual, or same gender loving	84 (90.3)
Bisexual or bi	12 (12.9)
Straight or heterosexual	1 (1.1)
Has tested HIV-positive	24 (25.8)
Owns a smartphone	88 (94.6)
Length of time lived in the DC metropolitan area
<1 year	7 (7.5)
1–2 years	8 (8.6)
≥3 years	78 (83.9)
Moved in the last year	36 (38.7)
Has health insurance	89 (95.7)
Type of health insurance (among those who have it)b
Private	67 (75.3)
Medicaid or Medicare	10 (11.2)
Student or Parent’s	5 (5.4)
Other	6 (6.7)
Had a period of time in the last year that he did not have health insurance	14 (15.1)
Was in jail or prison in the last year	2 (2.2)
Sexual Behaviors
Whom he had sex with in the last yearb
Men	93 (100.0)
Women	1 (1.1)
Transgender	1 (1.1)
Number of men he had oral or anal sex with in the last year
0	1 (1.1)
1 to 2	17 (18.3)
3 to 4	24 (25.8)
5 to 9	25 (26.9)
10 or more	23 (24.7)
Some, but not sure how many	3 (3.2)
Used a condom the last time he had anal sex with a man (among those who have had anal sex)c	49 (53.9)
Number of women he had oral, vaginal, or anal sex with in the last year
0	89 (95.7)
1 to 2	3 (3.2)
3 to 4	1 (1.1)
Gave money, drugs, or other goods to a male sexual partner in exchange for oral or anal sex in the last year	7 (7.5)
Received money, drugs, or other goods from a male sexual partner in exchange for oral or anal sex in the last year	9 (9.7)
Health Seeking Behaviors and Health Care Experiences
Health care settings accessed in the last yearb
Community-based clinic	57 (61.3)
Primary care doctor	46 (49.5)
Acute care setting	39 (41.9)
Dentist	34 (36.6)
Mental health care setting	8 (8.6)
Ever been tested for HIV	93 (100.0)
Ever heard of PrEP	79 (85.0)
Ever heard of PEP	69 (74.2)
Health care services received in the last yearb
HIV testing (among those HIV-negative)	61 (88.4)
Testing for another sexually transmitted infection	58 (62.4)
HIV counseling	21 (22.6)
PrEP (among those HIV-negative)	21 (30.4)
PEP (among those HIV-negative)	5 (7.3)
Free condoms	52 (55.9)
Was offered an HIV test the last time he saw a health provider for any reason (among those HIV-negative)	40 (58.0)
Was unable to get medical care (any type) that he felt he needed in the last year	9 (9.7)
Health Care Beliefsd
Trusts the health care provider he usually sees	79 (92.9)
Believes the health care provider he usually sees is competent	83 (96.5)
Believes it is easy to talk with the health care provider he usually sees	79 (94.0)
Believes it is easy to get all of his needs met when he sees a healthcare provider	78 (91.8)
Does not believe that health care providers think less of him because he is a man who has sex with men	70 (85.4)
Does not believe that health care providers think less of him because he is Black	70 (84.3)
Trusts medical advice from his social network about health issues	56 (65.9)
Trusts medical advice from apps about health issues	51 (63.8)

^aParticipants included 91 self-identified males, one self-identified transgender woman, and one self-identified intersex individual.

^bParticipants could provide more than one value for a response.

^cOne participant did not provide a response for this item on condom use.

^dEndorsement of health care beliefs is defined as a response ≥51 on a visual analogue scale from 0 to 100. The sum of response frequencies for questions on health care beliefs is less than 93 due to missing values in this component of the survey.

Statistical Analysis

We completed statistical analyses using SAS, Version 9.3 (Cary, NC). Descriptive statistics (frequencies and percentages) were calculated for each variable. For the 14 items that measured participants’ perceptions of the app using a 100-point visual analogue scale (C1–C7 and S1–S7), descriptive statistics were reported using a cutoff of ≥51 (representing agreement with items) as well as using the median and interquartile range (IQR). Differences in responses by self-reported HIV status were assessed using the Mann-Whitney U test to determine whether subsequent analyses should be stratified by HIV status. We performed exploratory factor analysis on this 14-item scale using the principal components method with promax rotation to assess its validity as a measure for perceptions of the app among BMSM in the sample and investigate its underlying factor structure. This approach has been effective for validating scales and creating subscales in other studies (Mutumba et al., 2015; Neufeld, Sikkema, Lee, Kochman, & Hansen, 2012) and recent simulation studies have indicated that small samples, even those with less than 50 participants, can produce reliable results (de Winter, Dodou, & Wieringa, 2009; Mundfrom, Shaw, & Ke, 2005). Cronbach’s alpha was calculated to assess the internal consistency of these 14 items and item analysis was performed by computing item-to-total correlation coefficients for the overall scale and for subscales that were created as the sum of items loading on each factor extracted in exploratory factor analysis. Using these subscales in bivariable analysis, we assessed correlates of having favorable perceptions of the utility of the app (i.e., having higher subscale scores) to generate medians with 95% confidence intervals (CIs), Mann-Whitney U test statistics, Spearman’s rank correlation coefficients with 95% CIs (for health care beliefs), and p-values. Key demographic characteristics, health care characteristics, and health care beliefs with sufficient variability in responses were selected for these analyses. Open-ended, qualitative responses were coded using thematic analysis, which is a method for identifying, analyzing, and reporting patterns within data (Braun & Clarke, 2006). After identifying patterned and related statements among open-ended responses, we named themes and used them as the basis for coding. Then, the data set was analyzed for the frequency with which different themes were mentioned in open-ended responses.

Results

Demographic, Behavioral, and Health Care Characteristics

A total of 93 BMSM completed the survey. Nearly half (43%) of participants were <30 years of age (mean=34; SD=10.3; range: 21–69) and 26% reported having previously tested positive for HIV (Table I). Almost all participants (95%) reported owning a smartphone. Participants reported having accessed community-based clinics (61%), primary care physicians (50%), acute care settings (42%), and dentists (37%) in the last year. Excluding dental care, only five participants had not accessed any medical care in the last year. Among participants who had not previously received a positive HIV test result, most (88%) reported having received HIV testing in the last year and nearly one-third (30%) had taken PrEP in the same time period. In terms of their beliefs about the health care providers they usually see, most participants trusted their providers (93%), believed they are competent (96%), agreed that it is easy to talk to them (94%), and agreed that it is easy to get their needs met (92%). Some participants believed their providers thought less of them because they have sex with other men (15%), because they are Black (16%), or for both of these reasons (7%).

Acceptability of the Mobile App Intervention

Based on perceptions of the smartphone app intervention, study participants on the whole supported its potential utility (Table II). After viewing the module on communicating with health care providers, most participants agreed that they would use the app if it were free (81%) and that it would be easier to remember to tell the doctor what they needed (88%) and get the medical services they needed (80%). After viewing the module on sexual health, most participants also agreed that they would use it if it were free (87%); they also believed that the app would make it easier to get the sexual health services they needed (87%), talk about sex with their medical providers (82%), and talk about PrEP (71%). There were no significant differences in perceptions of the app by self-reported HIV status; thus, subsequent analyses used pooled data from HIV-negative and HIV-positive participants.

Table II.

Perceptions of a smartphone application intervention to facilitate access to HIV prevention and care services among Black men who have sex with men in the District of Columbia Metropolitan Statistical Areas, 2014 (N=93)^a.

	n (%)b	median (IQR)c
Items Referring to the Mobile App Module on Communication with Health Care Providers
C1. I would use this app if it were free	69 (81.2)	91 (72 – 100)
C2. It would be easier to get the medical services I need if I had this app	66 (79.5)	79 (52 – 94)
C3. I would be more likely to get support in getting health insurance	53 (65.4)	68 (50 – 91)
C4. It would be easier to remember to tell the doctor what I need at my visit	75 (88.2)	87 (69 – 98)
C5. It would be easier to ask for an HIV test if I had this app	47 (58.8)	71 (40 – 93)
C6. It would be easier to ask the doctor for routine preventive care (like flu shots)	59 (73.8)	79 (50 – 92)
C7. I would be more comfortable talking with my health care provider	57 (70.4)	72 (50 – 94)
Items Referring to the Mobile App Module on Sexual Health
S1. I would use this app if it were free	74 (87.1)	88 (70 – 100)
S2. It would be easier to get the sexual health services I need if I had this app	73 (86.9)	77 (58 – 95.5)
S3. It would be easier to talk about sex with my medical provider if I had this app	69 (82.1)	73.5 (53 – 92)
S4. It would be easier to remember to tell the doctor what I need at my visit	73 (86.9)	83.5 (62.5 – 97.5)
S5. I would be more likely to ask about PrEP if I had this app	59 (71.1)	68 (50 – 94)
S6. I would be more likely to ask about condoms if I had this app	52 (62.7)	65 (47 – 90)
S7. I would be more likely to get an HIV test if I had this app	56 (68.3)	72 (49 – 90)

^aParticipants included 91 self-identified males, one self-identified transgender woman, and one self-identified intersex individual.

^bAgreement with each item is dichotomized as a response ≥51 on a visual analogue scale from 0 to 100. The sum of response frequencies is less than 93 due to missing values.

^cThe median and interquartile range are presented for each item measured on a visual analogue scale from 0 (i.e., no agreement) to 100 (i.e., complete agreement).

Forty-two men provided a response for the open-ended question on how to make it easier for people to get the care and services they need. Although this question did not directly ask about the app, many open-ended responses focused on the app and also demonstrated high endorsement of the utility of the app among participants (Table III). Men in the sample stated that the app would be easy to use and would help users identify culturally competent and accessible health care providers. They also recommended the app be visually appealing, track health seeking behaviors, use electronic notifications to remind them of behaviors like testing for HIV and taking PrEP, and incorporate tools for two-way communication between users and providers.

Table III.

Open-ended responses regarding ideas to improve access to HIV prevention services among Black men who have sex with men in the District of Columbia Metropolitan Statistical Areas, 2014 (N=93)^a.

Theme	Count	Sample Quotes
The app should allow users to identify culturally competent and affordable health care providers that are geographically accessible.	7	It would be ideal if the app could also identify ‘gay-friendly’ and culturally appropriate facilities based on my specific needs. If the app could use geo-location services to inform the user of nearby places they can receive free, confidential health services, that would help. A proximity feature that shows the closest clinics and medical facilities that offer services. Detailing whether the services are free or not. Maybe listing where local resources are so that people can access them. Including what LGBT-friendly resources are available. Ask them [users] who you are comfortable talkin to… such as a man or female… gay or straight… or black or white.
Community outreach and education efforts are imperative for HIV prevention and would also be important for operationalizing the app.	7	Creating a social media campaign to promote the app and its usages to help inform the potential users and health care providers. Schools need to redouble sex education classes and begin in middle school. Religious institutions need to teach sexual ethics to reduce the amount of exploitation young people are often the victims [of]. Increased community awareness must be expanded by the Department of Health especially among low-income groups. Parents need to be taught how to talk about safe sex with their children. Community outreach is also important. Posters or ads on public transportation tend to be really impactful to me at least. Anonymous (identification is not verified) counseling and services that are provided late into the evening and on weekends. Mobile vans that have routine locations would be great.
The app would be helpful and easy to use.	5	I think the web application is a great idea. I think having a phone app is a great idea. I think this app may work for individuals with low self-efficacy in this regard. It seems incredibly easy to me.
The app should track health seeking behaviors and use notifications to remind the user of behaviors such as HIV testing and taking PrEP.	4	I think it would be great to incorporate a system to remind individuals to get tested after certain marks. Essentially, adding a ‘notification center’ to the app to remind individuals to: get tested, take medication, etc. Set up within the app the ability for a person to log their health information, and their next hospital date. Provide a reminder for one’s PrEP treatment As it is built, I think it would be enhanced if it were to include a feature to set up reminders for people to make and/or follow up on appointments with a push notification
Health care providers need education to facilitate high cultural competency for working with patients who are Black men who have sex with men.	4	Medical providers who are culturally competent to African American MSM. I think it would be helpful if the doctors sometimes brought up medications or preventative treatment to the patients as ‘options.’ Many people do not understand Truvada, so having a doctor see a patient whom would be a good candidate for it, may allow them to offer them the treatment and explain its benefits/drawbacks. Then the patient can decide if they want the treatment or not!
The app should be clear and visually appealing.	3	I think for this quiz, it would be better to see a more designed layout of the app screen (what it would actually look like on an iPhone) in order to give better answers because design matters a lot in regards to if I use an app or not. If it looks intriguing, I’m invested. If not, I won’t use it and having the black and white copy of the screen is not appealing – even the font is a little boring. I think that you could gain a lot more insight if you had a better screengrab to show people taking this quiz. Make sure it is easy to understand
The app should allow users to engage in two-way communication with health care providers.	3	I think that an app that allow for real-time communication (i.e. an instant messenger) via an app would provide immediate education, support and information about HIV prevention services to those in need. Have the ability to email or call the health care facility directly from the app, or be able to ‘live chat’ with someone specifically about identifying a health care location
Health care providers should routinize and de-stigmatize HIV prevention services.	3	Make sure questions are direct and not make assumptions like I want to talk about HIV because I’ve had more than one sexual partner. Strongly encourage all health care providers to recommend HIV test for everyone that’s sexually active. Routinize prevention service questions; get comfortable with talking about to providers.

^aParticipants included 91 self-identified males, one self-identified transgender woman, and one self-identified intersex individual.

Psychometric Properties of the 14-Item Scale for Perceptions of the App

The exploratory factor analysis procedure supported the validity of using the 14 items to measure perceptions of the potential utility of the app intervention and informed the creation of two subscales (Table IV). The procedure extracted two factors that explained 78.0% and 11.6% of the total variance (eigenvalues of 8.54 and 1.27; KMO test=0.87; χ2 of Bartlett’s test of sphericity=997.92, p<0.0001). Based on the items that loaded more strongly on each factor, we named factor 1 for the general utility of the app for improving access to health insurance and health services (abbreviated as ‘Access’). We named factor 2 for the utility of the app for increasing ease in communicating health needs to health care providers (abbreviated as ‘Communication’). We recommended removing items C7 and S3 because they had two of the lowest loadings on factor 1 (0.58 and 0.60, respectively) and more closely represented the construct measured by items loading on factor 2 (Communication) – and had loadings >0.30 on factor 2 (0.32 and 0.34, respectively). Subsequent analyses excluded these two items. Further exploratory factor analysis on the other 12 items produced the same factor structure and explained 93.1% of the total variance. The Cronbach’s alpha values were 0.95 for the total scale, 0.92 for the Access subscale, and 0.91 for the Communication subscale. We then used these two newly created subscales to identify correlates of having favorable perceptions of the app.

Participant Characteristics Associated with Perceptions of the Utility of the App

In Table V, we report participant characteristics associated with higher scores on the two subscales that measured perceptions of the app. Variables significantly associated with higher scores on the Access subscale (i.e., having more favorable perceptions on the utility of the app for accessing health services) were not having private health insurance (U=981.5; p=0.0203), not having accessed a primary care physician in the last year (U=1748; p=0.0385), believing he could get his needs met when seeing a health care provider (rho=0.260; p=0.0234), trusting medical advice from social networks (rho=0.296; p=0.0090), and trusting medical advice from apps (rho=0.327; p=0.0039). Variables significantly associated with higher scores on the Communication subscale (i.e., having more favorable perceptions on the utility of the app for increasing ease in communicating health needs to providers) were believing that health care providers did not think less of him for being Black (rho=−0.292; p=0.0116), trusting medical advice from social networks (rho=0.231; p=0.0463), and trusting medical advice from apps (rho=0.372; p=0.0012).

Table V.

Associations between key participant characteristics and having favorable perceptions of a smartphone application intervention among Black men who have sex with men in the District of Columbia Metropolitan Statistical Areas, 2014 (N=93)^a.

Participant Characteristics	Subscale 1: Utility for Accessing health services (Access)b			Subscale 2: Ease in Communicating health needs (Communication)c
	Median (IQR)	Ud	p	Median (IQR)	Ud	p
Age		1548	0.2082		1326	0.8227
<30 years	423 (327 – 478)			475 (371 – 642)
≥30 years	381 (300 – 455)			497 (383 – 620)
Result of last HIV test		704	0.9338		629	0.4378
Positive	375 (330 – 464)			475 (288 – 573)
Negative	417 (301 – 464)			503 (378 – 642)
Has private health insurance		982	0.0203		881	0.0593
Yes	379 (285 – 452)			468 (372 – 588)
No, but has another type of insurance	435 (389 – 485)			574 (425 – 680)
Accessed a community-based clinic in the last year		1164	0.8332		1152	0.9746
Yes	423 (320 – 466)			475 (372 – 642)
No	383 (306 – 457)			503 (378 – 591)
Accessed a primary care physician in the last year		1748	0.0385		1310	0.2361
Yes	362 (233 – 455)			476 (371 – 585)
No	434 (346 – 470)			507 (383 – 658)
Accessed an acute care setting in the last year		1261	0.7169		1199	0.6439
Yes	399 (333 – 470)			504 (391 – 603)
No	399 (285 – 457)			468 (371 – 636)
Tested for HIV in the last year (among those HIV-negative)		735	0.8613		704	0.7415
Yes	399 (306 – 467)			499 (372 – 649)
No	399 (310 – 464)			476 (390 – 585)
Tested for another sexually transmitted infection in the last year		1071	0.3769		1104	0.2617
Yes	389 (302 – 455)			468 (371 – 603)
No	427 (338 – 470)			502 (392 – 652)
Received HIV counseling in the last year		757	0.5893		695	0.9903
Yes	443 (272 – 475)			499 (372 – 651)
No	394 (310 – 456)			494 (378 – 620)
Received PrEP in the last year (among those HIV-negative)		601	0.7385		548	0.7880
Yes	388 (306 – 480)			490 (371 – 669)
No	427 (300 – 452)			503 (387 – 612)
Was offered an HIV test the last time he saw a health provider for any reason (among those HIV-negative)		724	0.9039		698	0.4370
Yes	394 (294 – 473)			458 (372 – 647)
No	423 (322 – 454)			559 (425 – 636)
Unable to get medical care that he needed in last year		287	0.6387		277	0.5998
Was unable	389 (301 – 451)			440 (382 – 565)
Was not unable	402 (310 – 464)			500 (375 – 634)
	Spearman’s Rho (95% CI)e		p	Spearman’s Rho (95% CI)e		p
Trusts the health care provider he usually sees	0.162 (−0.066, 0.374)		0.1618	0.148 (−0.084, 0.364)		0.2095
Believes the health care provider he usually sees is competent	0.110 (−0.117, 0.326)		0.3426	0.097 (−0.133, 0.317)		0.4068
Believes it is easy to talk with the health care provider he usually sees	0.177 (−0.052, 0.388)		0.1295	0.043 (−0.189, 0.271)		0.7171
Believes it is easy to get all of his needs met when he sees a healthcare provider	0.260 (0.037, 0.458)		0.0234	0.117 (−0.114, 0.337)		0.3203
Believes that health care providers think less of him because he is a man who has sex with men	−0.002 (−0.231, 0.226)		0.9843	−0.059 (−0.285, 0.174)		0.6209
Believes that health care providers think less of him because he is Black	−0.224 (−0.428, 0.002)		0.0519	−0.292 (−0.488, −0.068)		0.0116
Trusts medical advice from his social network about health issues	0.296 (0.077, 0.487)		0.0090	0.231 (0.004, 0.435)		0.0463
Trusts medical advice from apps about health issues	0.327 (0.110, 0.515)		0.0039	0.372 (0.155, 0.555)		0.0012

^aParticipants included 91 self-identified males, one self-identified transgender woman, and one self-identified intersex individual.

^bSubscale 1 was determined using exploratory factor analysis of 14 items asking for participants’ beliefs on the app and was named for the general utility of the app for accessing health insurance and health services (Access). Values for subscale 1 range from 0 (completely negative perceptions) to 500 (completely favorable perceptions).

^cSubscale 2 was determined using exploratory factor analysis of 14 items asking for participants’ beliefs on the app and was named for the app increasing ease in communicating health needs to health care providers (Communication). Values for subscale 2 range from 0 (completely negative perceptions) to 700 (completely favorable perceptions).

^dP-values for categorical variables were obtained using the two-sided Mann-Whitney U test.

^ePositive values for Spearman’s Rho indicate positive correlations between the beliefs listed and the degree of agreement with subscale items. Conversely, negative values for Spearman’s Rho indicate negative correlations between the beliefs listed and the degree of agreement with subscale items.

Discussion

In this formative study of BMSM in DC, we evaluated the acceptability of, and correlates of having favorable perceptions of, a smartphone app that would aim to improve access to culturally competent HIV prevention and care services and facilitate communication about sexual and other health needs with providers. These men – almost all of whom owned a smartphone – were diverse in age and in HIV status. On the whole, they positively endorsed the utility of the app regardless of these and other participant characteristics. Surprisingly, despite known barriers to care, almost all participants reported having insurance and having accessed some form of health care in the last year. They also had generally positive perceptions of their health care providers. Despite their engagement in at least one form of health care, most participants agreed that they would use the app and supported its potential utility among BMSM. Quantitative and qualitative findings suggest that participants believed the app would facilitate more complete and culturally relevant utilization of the health care services they could access. In fact, previous formative research for this study found that the receipt of HIV prevention interventions differed by health care setting for BMSM in DC, with receipt of these services being less common in primary care settings than at community-based clinics. Given the complex known socio-cultural and structural barriers to HIV prevention and care services, access to health care does not guarantee access to HIV-specific services for BMSM (Levy et al., 2014). The high acceptability of the app could be attributed to its focus on improving the user’s self-efficacy in identifying and communicating specific sexual and other health needs to providers as opposed to solely focusing on promoting access to care.

Our findings are consistent with previous studies that have shown that Internet- and smartphone-based technologies are an acceptable and effective modality for an HIV-focused intervention among MSM and BMSM in the US. The high acceptability of these technologies for primary and secondary HIV prevention among MSM is demonstrated by research on a diverse set of interventions that are in development or that have already been created specifically for MSM: Internet- and smartphone-based interventions providing HIV prevention education (Holloway et al., 2014; Hooper, Rosser, Horvath, Oakes, & Danilenko, 2008; Mustanski, Lyons, & Garcia, 2011), geosocial and sexual networking apps for sexual health education and HIV/STI referrals (Sun et al., 2014), a live-chat intervention using Facebook to reduce HIV risk (Lelutiu-Weinberger et al., 2014), an online video intervention designed to decrease sexual risk behaviors and increase HIV disclosure (Chiasson, Shaw, Humberstone, Hirshfield, & Hartel, 2009), a chat room-based HIV prevention intervention designed to promote HIV testing (Rhodes et al., 2011), an Internet-based skills training and motivational intervention to reduce sexual risk (Carpenter, Stoner, Mikko, Dhanak, & Parsons, 2010), a text-messaging intervention to reduce HIV risk behaviors among methamphetamine-using MSM (Reback et al., 2012), and an Internet-based intervention to increase condom use among HIV-positive MSM (Miranda et al., 2013). Indeed, emerging Internet-based interventions appear to be the most promising approaches for HIV prevention among MSM (Rosser et al., 2011). To date, the only published Internet- or smartphone-based intervention developed specifically for BMSM, healthMpowerment, provides information, resources, and tailored feedback to reduce risky sexual behaviors and build community among young BMSM and transwomen. The feasibility and acceptability of the mobile-optimized website is supported by findings from pilot studies among BMSM, and its effectiveness is being assessed by a randomized controlled trial in North Carolina with multiple outcomes of interest: unprotected anal intercourse in the past three months (primary outcome), depression, social support, viral load/CD4, adherence, testing, HIV knowledge, and substance use (Hightow-Weidman et al., 2011; LeGrand et al., 2014; Muessig et al., 2014; Muessig et al., 2015; Muessig et al., 2013).

Although the design of the app’s sample modules used in this study were an initial prototype, other research has identified key characteristics and functionalities of smartphone-based HIV interventions that are important for user acceptability. In one study, young BMSM emphasized the importance that smartphone-based HIV interventions be fast-paced, useful, fun, efficient, user-friendly, and interactive without an overwhelmingly large amount of text. They also indicated a desire to control app features such as preferences for receiving app-related alerts and messages (Muessig et al., 2013). MSM in another study stated that an app’s design should facilitate interactive engagement whereby the user can input information using game-like functions and receive feedback, for example, on personal behaviors. They also preferred an app that incorporates social networking features yet also ensures privacy and discretion so that it feels safe and trustworthy (Goldenberg, McDougal, Sullivan, Stekler, & Stephenson, 2014). Moreover, guidelines for using mobile technologies in public health research call for interventions that integrate multiple communication devices, are scalable and sustainable, incorporate social network and/or geographic metrics, and take a community-based participatory approach to development and implementation (Young, Holloway, & Swendeman, 2014). Despite the app’s rudimentary design for the purpose of this study, app characteristics of known importance for user acceptability should further guide the development of its design and functionalities.

By creating subscales for participants’ perceptions of the app in exploratory factor analysis, we were also able to explore participant characteristics that were associated with having positive perceptions of the utility of the app. Both subscales were associated with trusting medical advice from apps, which provides support for the construct validity of these measures. Findings from bivariable analyses point to potential subgroups of BMSM for which the app may have the greatest impact. Participants who had not seen a primary care physician in the last year or who lacked private health insurance tended to more strongly support the general utility of the app for improving access to health insurance and health services. Research has found that BMSM (and the general population of Black men) often do not seek primary care due to barriers such as low cultural competency of providers, racial and sexual discrimination, mistrust of the medical establishment, low socioeconomic status, and lack of awareness of the need for care (Cheatham, Barksdale, & Rodgers, 2008; Malebranche, Peterson, Fullilove, & Stackhouse, 2004; Ravenell, Whitaker, & Johnson Jr, 2008). This app could provide an acceptable modality for these men to receive support in accessing comfortable primary care services. In addition, participants who did not believe that their providers thought less of them for being Black tended to more strongly agree that the app would increase ease in communicating health needs to providers. BMSM across multiple studies have indicated perceptions of racism and homophobia during visits with providers and were less likely to use HIV prevention services if they were unable to access nonjudgmental health care environments (Brooks, Etzel, Hinojos, Henry, & Perez, 2005; Dillon & Basu, 2014; Saleh, Operario, Smith, Arnold, & Kegeles, 2011). Experiences of discrimination and negative encounters with medical institutions have led to the inadequate use of health care, poor communication with providers, and poor ART adherence among BMSM living with HIV (Malebranche et al., 2004). Thus, some of these same barriers may inhibit men from using this app, yet the app might also help BMSM navigate a health care system that may not mitigate these barriers or meet their unique needs. Moreover, one study found that young BMSM were more likely to get HIV prevention information from social networks, the media, and the community rather than from medical providers (Voisin, Bird, Shiu, & Krieger, 2013). For BMSM who use smartphones, this app could provide an acceptable means to bridge these separate entities, with culturally appropriate HIV prevention information and tools immediately available to help them access the services they need.

Our findings are especially relevant to BMSM in the DC metropolitan statistical areas who are similar to the men in this study, yet these results also have applicability to other sub-populations of BMSM. Participants in this study were recruited via Internet-based materials and surveyed using Internet-based methods, and thus were admittedly a select group of BMSM who could and do regularly access the Internet on a laptop, tablet, or smartphone. An Internet-based sample was appropriate for this study because it consisted of men that would have the tools to use this app. However, given known barriers to engagement in health care experienced by BMSM, the health care characteristics of men in this study’s sample suggest that they were already more linked and engaged in care, on average, than the target users of the proposed app. Although recruitment methods included the distribution of materials via Facebook, we likely did not reach the group of BMSM who are the least engaged in care. Future outreach efforts and sampling methods regarding this app should attempt to reach more men who are not engaged in health care, since they might benefit from this app the most (although our sample endorsed the app’s utility despite most men having at least some engagement in health care). In addition, sub-populations of BMSM in other geographic regions of the US would also be likely to use this app. Smartphone apps are being developed for BMSM in other regions (Muessig et al., 2015) and evidence indicates that BMSM in other regions have experienced similar barriers to services (Levy et al., 2014).

This study has several limitations. First, participants’ perceptions of the app were based on an initial prototype, but based on the information provided to participants, they were able to conceptualize its general structure and purpose so that they could provide meaningful feedback on its acceptability and potential utility. Second, the items used to measure perceptions of the app had not been previously used or validated, but their psychometric properties in this study supported their validity. Third, analyses of this moderately sized sample used pooled data from HIV-negative and HIV-positive participants because, despite potential differences in health care experiences, there were no significant differences in perceptions of the app or health care beliefs by HIV status in this sample. Fourth, results could have been influenced by social desirability bias. To minimize bias, participants were able to anonymously take the survey on their own computer, tablet, or smartphone.

Conclusions

This study provides compelling formative evidence that BMSM in DC would use and may benefit from a targeted smartphone app that aims to improve their access to culturally competent HIV prevention and care services and facilitate communication about sexual and other health needs with providers. To our knowledge, this study is the first to report correlates of having favorable perceptions of a mobile app focused on HIV prevention and care for BMSM. Our findings warrant the further development of this app intervention and call for additional studies to evaluate its feasibility and effectiveness. By building on previous research on structural barriers to HIV prevention and care services among BMSM, this potential app intervention represents a step forward in efforts to eliminate racial disparities in care outcomes and HIV incidence among MSM in the US.

Biographies

Matthew Levy is a Research Associate and doctoral student in the Department of Epidemiology and Biostatistics at the George Washington University Milken Institute School of Public Health. He serves as the Study Coordinator and Data Quality Manager for multiple clinical trials of pre-exposure prophylaxis (PrEP) within the George Washington University HIV Prevention Trials Network (HPTN) Clinical Research Site. For his doctoral dissertation, he is investigating dyslipidemia and its management in a prospective cohort of people living with HIV who are engaged in care in the District of Columbia.

Contact: mattelevy@gwu.edu

Christopher Chauncey Watson, MHS, has studied health disparities affecting African Americans for over a decade. At the time of this paper’s conduct and analysis, Mr. Watson worked at The George Washington University Milken Institute School of Public Health. Mr. Watson’s professional efforts encompass multiple roles, including both domestic and global initiatives aimed to end the HIV epidemic and health disparities. Named a Champion of Change by a local organization, IMPACT, for his contributions to the global fight against HIV/AIDS, Mr. Watson works to influence both policy makers and community stakeholders about the importance of health inequalities.

Contact: cclwat@gwu.edu

Leo Wilton, PhD, is an Associate Professor in the Department of Human Development in the College of Community and Public Affairs (CCPA) at the State University of New York at Binghamton and a Senior Research Associate in the Faculty of Humanities at the University of Johannesburg in Johannesburg, South Africa. His primary research interests include health disparities (primary and secondary HIV prevention); community based research and evaluation; and Black psychological development and mental health. Dr. Wilton’s scholarly research on the AIDS epidemic focuses on the intersectionality of race, gender, and sexuality, as situated in macro- and micro-level inequalities in Black communities, both nationally and internationally. The overall objective of Dr. Wilton’s scholarly research program has been to focus on the impact of socio-cultural factors that influence sexual/drug-risk and protective behavior and mental health in Black communities. His scientific research examines socio-cultural factors that provide the basis for the development of culturally grounded HIV prevention interventions in Black communities.

Contact: lwilton@binghamton.edu

Vittoria Criss is a student pursuing a Master of Science in Public Health Microbiology with interests in HIV and tuberculosis. She has worked as a Research Assistant on HIV Prevention Trials Network (HPTN) clinical trials as well as other epidemiologic research studies focused on men who have sex with men and transgender women.

Contact: vcriss646@gwu.edu

Irene Kuo, PhD, MPH, is an Associate Research Professor in the Department of Epidemiology and Biostatistics in the Milken Institute School of Public Health at George Washington University (GWU). Dr. Kuo has nearly 15 years of experience conducting research and behavioral surveillance on infectious diseases (particularly HIV, HCV and HBV) in key populations at risk for HIV regarding substance use, risk behaviors, incarceration, linkage to HIV care and HIV prevention.

Contact: ikuo@gwu.edu

Sara Glick, PhD, MPH, is a Senior Research Scientist at the University of Washington. She has a BA in anthropology from Northwestern University, a Master of Public Health in epidemiology from the University of Illinois – Chicago, and a PhD in epidemiology from the University of Washington. Previously, Dr. Glick was on faculty at the George Washington University where she conducted HIV prevention research for men who have sex with men in collaboration with the co-authors of this paper. She currently is the site principle investigator for the National HIV Behavioral Surveillance project in Seattle.

Contact: snglick@gwu.edu

Russell Brewer, DrPH, has served as the Director of the HIV/STI Program at the Louisiana Public Health Institute in New Orleans, LA since August 2011. He is also a Visiting Professor at the University of San Francisco, Center for AIDS Prevention Studies. From 2010–2012, he was an HIV Prevention Trials Network (HPTN) Scholar conducting research to explore the relationship between incarceration and HIV among Black men who have sex with men (BMSM) enrolled in the HPTN 061 study. His research and programmatic efforts are focused on the needs of BMSM and persons living with HIV infection in the South.

Contact: rbrewer@lphi.org

Manya Magnus, PhD, MPH, is an Associate Professor in the Department of Epidemiology and Biostatistics at the George Washington University School of Public Health. Dr. Magnus is co-director of the School’s MPH Epidemiology Program. Dr. Magnus received her BA in psychology from the University of California, San Diego, and her MPH and PhD in epidemiology from Tulane University. Always interested in integrating research with clinical care, Dr. Magnus has collaborated on and directed a variety of epidemiologic studies, including clinical trials, cohort studies, and case-control studies. She applies epidemiologic methodology to conduct studies on the local, state, and national level, including NIH- and CDC-sponsored studies surrounding risk factors associated with HIV and use of innovative methods to improve HIV prevention services to at-risk individuals.

Contact: manyadm@gwu.edu