Abstract
Background:
We tested an adaptation of a mobile HIV prevention smartphone app, HealthMindr-PrEP, to assess its effectiveness to increase pre-exposure prophylaxis (PrEP) uptake among gay and bisexual men who have sex with men.
Setting:
This study was conducted online among gay and bisexual men who have sex with men living in Ending the HIV Epidemic priority jurisdictions, primarily in the southern United States.
Methods:
We conducted a randomized controlled trial in which participants were randomly assigned to receive the intervention app or an app that only allowed participants to track their study progress. The primary outcome was self-reported PrEP uptake. Cox proportional hazards models were used to compare the rate of PrEP uptake across study arms.
Results:
Although no results were statistically significant, the rate of PrEP initiation was higher among participants in the intervention arm compared with the control arm, particularly among participants with PrEP indications at baseline. Among those with PrEP indications at baseline, participants in the intervention arm had a 64% higher rate of PrEP initiation compared with the control arm (hazard ratio = 1.64, 95% CI: 0.89 to 3.03). Among those without PrEP indications, the same hazard ratio was 1.05 (95% CI: 0.67 to 1.66).
Conclusions:
Our results suggest that HealthMindr-PrEP increases the rate of PrEP uptake compared with a standard of care control condition; however, we had insufficient statistical power for stratified analyses that demonstrated that the strongest effect was among those who had PrEP indications at baseline.
Keywords: HIV, pre-exposure prophylaxis, men who have sex with men, mHealth, intervention, randomized controlled trial
INTRODUCTION
The HIV epidemic in the United States has long been concentrated among gay and bisexual men who have sex with men (GBMSM). Despite comprising approximately 2% of the population,1 GBMSM accounted for 67% of new HIV diagnoses in 2021.2 Since 2012, oral pre-exposure prophylaxis (PrEP) has been available as a biomedical HIV prevention tool. Oral PrEP is highly effective in preventing acquisition of HIV among GBMSM, providing near-complete protection against seroconversion following moderate adherence (at least 4 of 7 weekly doses).3 A long-acting injectable form of PrEP4 was approved for use in 2021.
Despite its effectiveness, there have been numerous challenges in PrEP implementation which have limited uptake, particularly among the groups most at risk of HIV. Despite being at increased risk of HIV,5 Hispanic and non-Hispanic Black MSM with PrEP indications are less likely than non-Hispanic White MSM to have used PrEP in the past year.6 Additional tools are needed to increase uptake of PrEP among GBMSM in general and among GBMSM of color specifically.
Mobile health (mHealth) tools have the potential to reduce barriers to uptake and empower users by reducing barriers to accessing health information and resources. We have developed an HIV prevention mHealth platform, Health-Mindr,7,8 that has been adapted for multiple populations.9-12 The original HealthMindr intervention was found to increase HIV testing, but not PrEP uptake, among GBMSM at high risk of HIV.13 Our new intervention adaptation sought to specifically target PrEP uptake by adding PrEP-specific content to increase interest in PrEP among participants, providing information about common barriers to initiation, and incorporating a PrEP provider locator (PrEPLocator.org).
The goal of this study was to test whether allocation to an adaptation of a HealthMindr app with enhanced PrEP content (HealthMindr-PrEP app) was associated with increased PrEP uptake among young GBMSM, compared with a standard of care control condition.
METHODS
Participants and Recruitment
We conducted a randomized controlled trial of the HealthMindr-PrEP app among young GBMSM with a primary outcome of self-reported PrEP initiation. The protocol for this study has been published.14 Participants were recruited from January 27, 2020, through March 8, 2022, through advertisements on social and sexual networking websites and apps (eg, Instagram, Jack’d). On general use apps, advertisements were targeted to be shown to users who were, based on their interests, likely to be GBMSM. On sexual networking apps, ads were targeted to all users within the geographic recruitment area. Advertising strategies were designed to over-sample GBMSM of color.
Interested participants were redirected to a secure online eligibility screening survey. At the outset of the study, eligible participants were aged 18–34, could read and understand English, were assigned male at birth, identified as male at the time of screening, lived within the geographic region of eligibility (described in more detail below), did not have plans to move in the next 12 months, used an Android or iOS smartphone, were willing to download a health-related study app to their phone, had anal sex with a man in the past 6 months, were not currently on PrEP, and had never received an HIV diagnosis.
Initially, participants were required to live in the Atlanta, GA; Jackson, MS; or Washington, DC, metropolitan areas to be eligible. To increase enrolment, changes were made to the eligibility criteria during the course of the study. We expanded the geographic eligibility to include other priority jurisdictions for the US Ending the HIV Epidemic initiative.15 Effective July 15, 2020, GBMSM in the metropolitan areas of Dallas, TX; Houston, TX; Miami, FL; Nashville, TN; and New Orleans, LA, were also eligible to participate. Effective April 28, 2021, GBMSM in Austin, TX; Baltimore, MD; Birmingham, AL; Charlotte, NC; Columbia, SC; Columbus, OH; Detroit, MI; Louisville, KY; Memphis, TN; Orlando, FL; and Tampa, FL, were eligible to participate. On July 9, 2021, the criterion to not have plans to leave the study area within 12 months was removed.
Those who were eligible were provided with informed consent information for the study, including a video summary of the informed consent document, which has been found to increase comprehension.16 Informed consent was recorded through electronic signature. Consenting participants were then able to download the study app, which allowed them to complete all study activities. This study was reviewed and approved by the Emory University Institutional Review Board.
Enrolment
Eligible participants who consented to participate could then download the study app and register for a login. They then completed the mobile-optimized baseline survey. Next, enrolment was paused for fraud prevention procedures. These involved comparing answers to specific questions that were repeated across the eligibility and baseline surveys to identify any discrepancies. All participants were then contacted by telephone to discuss any questions about the study; those with inconsistent responses across the eligibility and baseline surveys were asked about the discrepancies on the call to try and resolve them. Participants who passed the fraud check were enrolled and randomized to a study arm. Random permuted block randomization, stratified by location, was used to randomize participants to the intervention and control arms in a 2:1 ratio.
Study App and Arms
This study used HealthMindr-PrEP, an adaptation of the base HealthMindr app.17 HealthMindr-PrEP is an HIV prevention smartphone app for GBMSM that includes information and resources about HIV and STI testing and treatment; Frequently Asked Questions about PrEP; a PrEP self-assessment; a PrEP provider locator; information about paying for PrEP; general information about options for HIV and STI prevention (eg, proper condom use); ordering of condoms, lubricant, and at-home HIV and STI self-test kits; and resources related to substance use and mental health. The version of the app used in this study also included a personalized study timeline.
On initial download, the study app only displayed the participant timeline. On randomization, if a participant was randomized to the intervention arm, then the HealthMindr-PrEP HIV prevention content described above became available. If a participant was randomized to the control arm, then they continued to only see the participant timeline for the duration of the study.
Follow-Up Schedule
Participants were followed for a total of 12 months and completed study surveys every 3 months. Quarterly surveys were compensated according to the following schedule: $50 for baseline, $40 for each of months 3 and 6, $50 for month 9, and $60 for month 12. Subsequent to enrolment, compensation amounts increased to encourage participant retention.
Participants were also prompted to complete optional monthly self-assessments. The monthly self-assessment for participants in the intervention arm was a PrEP eligibility screener that also asked whether they had initiated PrEP in the past month. The PrEP eligibility screener was based on CDC criteria that were in effect at the launch of the trial.18 For participants in the control arm, the monthly self-assessment was a nutrition and exercise screener that served as an attention control; participants in the control arm were asked whether they had started any new prescription medications in the past month and, if yes, to specify which medications they started. They were not asked about PrEP specifically. Participants were not compensated for completing the monthly self-assessments.
Measures
The primary outcome was time to self-reported PrEP uptake based on reporting receiving a PrEP prescription or taking any doses of PrEP in the previous 3 months on a quarterly follow-up survey. The PrEP prescription question asked participants whether they had received a new PrEP prescription in the past 3 months with the following answer options: Yes; No, but I started taking PrEP without a prescription; No, I have a prescription for PrEP from more than 3 months ago; No. Answering “Yes” or “No, but I started taking PrEP without a prescription” were counted as reporting PrEP uptake. We experienced a data quality issue with this question at the beginning of the study. From the beginning of enrolment (January 27, 2020) through November 17, 2020, the “No” option was not available. We describe sensitivity analyses below to address this issue.
For PrEP starters, time to self-reported PrEP uptake was calculated as the number of days between baseline survey completion and the midpoint between the follow-up survey where a participant first self-reported using PrEP and the prior survey. We did not capture the exact date of PrEP uptake, so the midpoint served as a surrogate. Although the monthly screeners provide more granular data, the PrEP uptake questions were asked differently across arms; thus, only quarterly survey data were used for the primary outcome analysis. Monthly survey data were used in a secondary analysis described below. Because some participants missed interim quarterly surveys, the 12-month survey included a question asking participants to report whether they had taken any PrEP doses in the past 12 months. Person-time for participants who reported PrEP doses in the past 12 months but who did not report PrEP initiation on a prior survey was calculated as the number of days between the baseline survey and the midpoint between the baseline survey and the 12-month survey. Person-time for those who did not start PrEP was the number of days between the baseline survey and last completed survey.
As a secondary outcome, we measured time to self-reported PrEP uptake at any time, inclusive of the monthly self-assessments. Thus, the primary outcome was measured only on compensated, quarterly surveys in which all participants were asked the same questions, and the secondary outcome also included self-reported PrEP uptake from uncompensated monthly self-assessments in which self-reported PrEP uptake was assessed differently across arms (see Follow-up Schedule above). Person-time was determined similarly as for the primary analysis described above. This resulted in a more granular time-to-event analysis because PrEP uptake was assessed monthly instead of quarterly for these analyses.
Participants also reported demographic and socioeconomic characteristics, sexual behavior, and HIV prevention services uptake on the baseline and quarterly surveys. Based on survey responses, participants were categorized as having PrEP indications or not, according to current criteria from the US Centers for Disease Control and Prevention.19 Specifically, men were considered to have PrEP indications if they reported sex with a partner living with HIV with a detectable or unknown viral load; sex with one or more partners of unknown HIV serostatus without 100% condom use; or a gonorrhea, chlamydia, or syphilis diagnosis in the past 6 months.
Among intervention participants, we used app paradata to assess the frequency and duration of HealthMindr PrEP use. App acceptability was measured using the systems usability scale (SUS),20 which was administered on the 12-month survey. We present the mean SUS score, with a value >68 indicating acceptability.20
Statistical Analysis
We compared selected sociodemographic and behavioral variables overall and by study arm assignment. Median and interquartile ranges (IQR) are presented for the number and duration of app use sessions per user.
The primary outcome analysis was conducted using Cox proportional hazards regression. We examined the unadjusted association between study arm assignment and PrEP initiation to determine the effect of the intervention. We estimated the model in multiple ways. First, because there were 2 different types of surveys that asked about PrEP initiation (incentivized quarterly surveys, nonincentivized monthly self-assessments), we estimated the regression model using data from quarterly surveys only (ie, the primary outcome) and using data from monthly and quarterly surveys (ie, secondary outcome).
Owing to the data quality issue described above that affected measurement of the primary outcome of PrEP uptake, we also estimated each model using all participant data and only data from participants for whom all follow-up visits were conducted after November 17, 2020.
Finally, during the time this study was being conducted, another adaptation of HealthMindr was found to increase HIV testing among MSM with high-risk sexual behavior but not those with lower risk sexual behavior.13 To determine whether a similar effect might be present in our current study, we stratified the models by PrEP indication at baseline.
Statistical Power
A priori power analyses were based on the following assumptions: 9% PrEP uptake in the intervention arm (based on data from the HealthMindr pilot), 2% PrEP uptake in the intervention arm (based on PrEP uptake data available at the time of grant proposal submission), 2:1 ratio of randomization to the intervention vs control arm, and alpha = 0.05. Under these assumptions, we had 90% power to detect a difference in PrEP uptake between the intervention and control arms with full retention and 83% power with 80% retention. Power was reduced for the sensitivity analyses we conducted, but was still approximately 70% when restricted to those who had follow-up after November 17, 2020.
RESULTS
Participant recruitment began on January 27, 2020, and ended on March 22, 2022. A total of 32,701 eligibility screeners were initiated, of which 15,561 (47.6%) were completed resulting in 2770 (8.4%) eligible participants (Fig. 1). Among the eligible participants, 839 (30.2%) completed the baseline survey and 660 (23.8%) passed the fraud prevention procedures and were randomized. Two participants who were randomized were later discovered to be duplicate enrolments, so the final analytic data set comprised 658 participants (221 control, 437 intervention). Overall, 83%, 77%, 75%, and 73% of participants completed the month 3, month 6, month 9, and month 12 surveys, respectively. Approximately half of participants completed the optional monthly check-in surveys, with similar completion rates in the intervention and control arm (data not shown).
FIGURE 1.
CONSORT diagram of study enrolment and retention in a randomized trial of a mobile prevention app to increase uptake of pre-exposure prophylaxis among gay and bisexual men who have sex with men in the United States, 2020–2023.
A plurality of participants was non-Hispanic White (n = 283, 43.0%); however, 23.1% (n = 152) identified as Hispanic and 21.6% (n = 142) identified as non-Hispanic Black (Table 1). Most had private health insurance (n = 446, 67.8%).
TABLE 1.
Demographic Characteristics of Gay and Bisexual Men Who Have Sex With Men Enrolled in a Randomized Controlled Trial of a Mobile HIV Prevention App
| Overall, N = 658, n (%) | Intervention, n = 437, n (%) | Control, n = 221, n (%) | |
|---|---|---|---|
| Age group | |||
| 18–24 yrs | 208 (31.6) | 141 (32.3) | 67 (30.3) |
| 25–29 yrs | 241 (36.6) | 165 (37.8) | 76 (34.4) |
| 30–34 yrs | 209 (31.8) | 131 (30.0) | 78 (35.3) |
| Combined race/ethnicity | |||
| Hispanic | 152 (23.1) | 100 (22.9) | 52 (23.5) |
| Non-Hispanic Black | 142 (21.6) | 92 (21.1) | 50 (22.6) |
| Non-Hispanic White | 283 (43.0) | 190 (43.5) | 93 (42.1) |
| Non-Hispanic other or multiple races | 81 (12.3) | 55 (12.6) | 26 (11.8) |
| Household income | |||
| $0–29,999/yr | 210 (32.0) | 135 (31.0) | 75 (34.1) |
| $30–74,999/yr | 278 (42.4) | 187 (42.9) | 91 (41.4) |
| $75,000+ | 133 (20.3) | 91 (20.9) | 42 (19.1) |
| Do not know | 35 (5.3) | 23 (5.3) | 12 (5.5) |
| Educational status | |||
| High school or less | 64 (9.7) | 42 (9.6) | 22 (10.0) |
| *Less than 4 yrs of college | 144 (21.9) | 96 (22.0) | 48 (21.7) |
| 4-yr college grad or higher | 450 (68.4) | 299 (68.4) | 151 (68.3) |
| Employment status | |||
| Not currently employed | 129 (19.6) | 82 (18.8) | 47 (21.3) |
| Employed full time | 417 (63.5) | 288 (66.1) | 129 (58.4) |
| Employed part time | 111 (16.9) | 66 (15.1) | 45 (20.4) |
| Health insurance status | |||
| Private | 446 (67.8) | 303 (69.3) | 143 (64.7) |
| Public | 60 (9.1) | 41 (9.4) | 19 (8.6) |
| Other/do not know | 16 (2.4) | 8 (1.8) | 8 (3.6) |
| No insurance | 136 (20.7) | 85 (19.5) | 51 (23.1) |
| Housing | |||
| Stable/permanent | 532 (81.3) | 358 (82.3) | 174 (79.5) |
| Transitional/temporary | 107 (16.4) | 68 (15.6) | 39 (17.8) |
| Homeless | 11 (1.7) | 7 (1.6) | 4 (1.8) |
| Other | 1 (0.2) | 1 (0.2) | 0 (0.0) |
| Do not know | 3 (0.5) | 1 (0.2) | 2 (0.9) |
| Food insecurity | |||
| Yes | 131 (20.0) | 83 (19.0) | 48 (22.1) |
| No | 505 (77.2) | 343 (78.5) | 162 (74.7) |
| Do not know | 5 (0.8) | 2 (0.5) | 3 (1.4) |
| Prefer not to answer | 13 (2.0) | 9 (2.1) | 4 (1.8) |
Includes technical or vocational school.
The proportion of participants initiating PrEP, mean time to initiation among those who started PrEP, and rate of PrEP initiation stratified by study arm, race and ethnicity, and age group are summarized in Table 2. Similar proportions of participants in each study arm started PrEP during the study; however, the time to PrEP initiation was shorter for intervention compared with control participants. The rate of PrEP initiation was consistently higher for the analysis that included the monthly surveys, reflecting the fact that many intervention participants only reported PrEP initiation on a monthly check-in but not a quarterly follow-up survey.
TABLE 2.
Number and Proportion Starting PrEP; Mean Time to PrEP Initiation Among Those Who Started PrEP; and Overall Rate of PrEP Initiation by Study Arm, Race, and Ethnicity; and Age Group in a Randomized Controlled Trial of a Mobile App to Increase PrEP Initiation in the United States, 2020–2023
| Control |
Intervention |
|||||||
|---|---|---|---|---|---|---|---|---|
| n | Started PrEP, n (%) |
Mean Time to PrEP Initiation (d)* |
Rate of PrEP Initiation† (95% CI) |
n | Started PrEP, n (%) |
Mean Time to PrEP Initiation (d)* |
Rate of PrEP Initiation† (95% CI) |
|
| Quarterly surveys only | ||||||||
| All participants | ||||||||
| Race and ethnicity | ||||||||
| Hispanic | 49 | 15 (31) | 100.5 | 43 (25 to 69) | 88 | 39 (44) | 89.0 | 72 (52 to 98) |
| Non-Hispanic Black | 44 | 19 (43) | 145.8 | 65 (40 to 99) | 80 | 30 (38) | 109.2 | 57 (39 to 80) |
| Non-Hispanic Black | 44 | 19 (43) | 145.8 | 65 (40 to 99) | 80 | 30 (38) | 109.2 | 57 (39 to 80) |
| Non-Hispanic White | 82 | 33 (40) | 119.7 | 62 (43 to 86) | 166 | 40 (24) | 110.8 | 32 (23 to 43) |
| Non-Hispanic other/multiple races | 25 | 6 (24) | 131.6 | 31 (13 to 65) | 51 | 15 (29) | 139.0 | 37 (22 to 60) |
| Age group | ||||||||
| 18–24 yrs | 60 | 23 (38) | 121.3 | 58 (38 to 86) | 125 | 37 (30) | 113.4 | 42 (30 to 57) |
| 25–29 yrs | 67 | 28 (42) | 122.4 | 66 (45 to 94) | 143 | 47 (33) | 96.1 | 47 (35 to 62) |
| 30–34 yrs | 73 | 22 (30) | 127.3 | 40 (26 to 60) | 117 | 40 (34) | 113.6 | 47 (34 to 63) |
| Follow-up after Nov 17, 2020 | ||||||||
| Race and ethnicity | ||||||||
| Hispanic | 31 | 7 (23) | 112.0 | 30 (13 to 59) | 60 | 25 (42) | 105.3 | 63 (42 to 91) |
| Non-Hispanic Black | 30 | 10 (33) | 194.5 | 45 (23 to 81) | 51 | 17 (33) | 132.1 | 44 (27 to 69) |
| Non-Hispanic White | 47 | 21 (45) | 133.6 | 73 (46 to 109) | 104 | 26 (25) | 112.9 | 33 (22 to 48) |
| Non-Hispanic other/multiple races | 18 | 3 (17) | 186.2 | 20 (5 to 55) | 33 | 11 (33) | 174.2 | 43 (23 to 75) |
| Age group | ||||||||
| 18–24 yrs | 37 | 11 (30) | 148.9 | 42 (22 to 73) | 86 | 26 (30) | 125.5 | 42 (28 to 61) |
| 25–29 yrs | 44 | 17 (39) | 127.1 | 63 (38 to 99) | 88 | 30 (34) | 115.7 | 47 (32 to 67) |
| 30–34 yrs | 45 | 13 (29) | 176.6 | 36 (20 to 60) | 74 | 23 (31) | 130.2 | 41 (26 to 60) |
| Quarterly and monthly surveys | ||||||||
| All participants | ||||||||
| Race and ethnicity | ||||||||
| Hispanic | 50 | 15 (30) | 106.0 | 41 (24 to 66) | 88 | 45 (51) | 95.8 | 88 (65 to 117) |
| Non-Hispanic Black | 45 | 19 (42) | 132.3 | 65 (41 to 100) | 84 | 35 (42) | 110.6 | 69 (49 to 95) |
| Non-Hispanic White | 84 | 33 (39) | 125.8 | 60 (42 to 84) | 172 | 47 (27) | 110.0 | 37 (28 to 49) |
| Non-Hispanic other/multiple races | 25 | 6 (24) | 158.3 | 30 (12 to 63) | 51 | 17 (33) | 128.1 | 44 (26 to 69) |
| Age group | ||||||||
| 18–24 yrs | 62 | 23 (37) | 129.2 | 56 (37 to 83) | 126 | 45 (36) | 115.1 | 53 (39 to 70) |
| 25–29 yrs | 69 | 28 (41) | 121.4 | 64 (43 to 91) | 149 | 52 (35) | 94.4 | 53 (40 to 69) |
| 30–34 yrs | 73 | 22 (30) | 128.8 | 40 (26 to 59) | 120 | 47 (39) | 115.8 | 56 (42 to 74) |
| Follow-up after Nov 17, 2020 | ||||||||
| Race and ethnicity | ||||||||
| Hispanic | 32 | 7 (22) | 111.6 | 28 (12 to 56) | 60 | 31 (52) | 106.4 | 85 (59 to 119) |
| Non-Hispanic Black | 31 | 10 (32) | 155.7 | 47 (24 to 83) | 51 | 22 (43) | 119.5 | 62 (40 to 93) |
| Non-Hispanic White | 49 | 21 (43) | 135.2 | 70 (44 to 105) | 109 | 30 (28) | 105.1 | 38 (26 to 54) |
| Non-Hispanic other/multiple races | 18 | 3 (17) | 208.5 | 20 (5 to 54) | 33 | 12 (36) | 128.2 | 50 (27 to 85) |
| Age group | ||||||||
| 18–24 yrs | 39 | 11 (28) | 152.6 | 41 (21 to 71) | 86 | 32 (37) | 115.5 | 55 (38 to 76) |
| 25–29 yrs | 46 | 17 (37) | 117.7 | 60 (36 to 95) | 90 | 34 (38) | 97.8 | 56 (40 to 78) |
| 30–34 yrs | 45 | 13 (29) | 163.3 | 36 (20 to 60) | 77 | 29 (38) | 124.1 | 53 (36 to 75) |
Among those initiating PrEP.
Per 100 person-years.
Among those who initiated PrEP and reported their dosing strategy, most reported taking PrEP daily (n = 123, 79%) followed by on-demand dosing (n = 30, 19%). One participant reported using injectable PrEP and one reported taking PrEP whenever he thought of it.
Among intervention participants (n = 437), there was a median of 31 (IQR: 17, 46) app use sessions over the course of the 12-month follow-up period. The median session duration was 1.48 minutes (IQR: 0.7, 3.3). The mean SUS score was 77.8, an above average score indicating that participants found the app to be acceptable.
The results of the unadjusted Cox proportional hazards model for the effect of study arm assignment are summarized in Table 3. When only analyzing data collected from the quarterly surveys and restricting to follow-up conducted after the PrEP initiation question was corrected, there was no difference in time to PrEP initiation comparing intervention with control participants [hazard ratio (HR) = 0.95, 95% confidence interval (95% CI): 0.65 to 1.39). When analyzing data from the quarterly and monthly surveys with the same restriction, intervention arm participants were found to initiate PrEP at moderately higher rates compared with control arm participants (HR = 1.23, 95% CI: 0.85 to 1.78). None of these results was statistically significant.
TABLE 3.
Unadjusted Hazard Ratios Comparing Uptake of PrEP Among Intervention With Control Participants in a Randomized Controlled Trial of a Mobile HIV Prevention App
| All Participants (n = 385 Intervention, n = 200 Control) |
Participants With Follow-Up Surveys After November 17, 2020 (n = 248 Intervention, n = 126 Control) |
|
|---|---|---|
| Quarterly surveys only | ||
| Unadjusted HR for PrEP initiation | ||
| Intervention | 0.86 (0.65, 1.15) | 0.95 (0.65, 1.39) |
| Control | Ref | Ref |
| All participants (n = 395 intervention, n = 200 control) |
Participants with follow-up surveys after Nov 17, 2020 (n = 253 intervention, n = 130 control) |
|
| Quarterly and monthly surveys | ||
| Unadjusted HR for PrEP initiation | ||
| Intervention | 1.05 (0.80, 1.40) | 1.23 (0.85, 1.78) |
| Control | Ref | Ref |
Participants were not able to indicate that they had not initiated PrEP on follow-up surveys until November 17, 2020, so data are presented from the entire study and restricted to participants whose follow-up was completed after that date.
The results of the Cox proportional hazards model stratified by baseline PrEP indications are summarized in Table 4. When analyzing data from quarterly surveys only and restricting to follow-up conducted after the PrEP initiation question was corrected, the intervention effect among those with PrEP indications at baseline (HR = 1.18, 95% CI: 0.62 to 2.24) was stronger and in the opposite direction of the intervention effect among those without PrEP indications (HR = 0.85, 95% CI: 0.53 to 1.36). When analyzing data from quarterly and monthly surveys, restricted to data collected after the PrEP initiation question was corrected, there was a moderate intervention effect among those with PrEP indications (HR = 1.64, 95% CI: 0.89 to 3.04) and no intervention effect among those with no PrEP indications at baseline (HR = 1.06, 95% CI: 0.67 to 1.67). None of these results was statistically significant.
TABLE 4.
Unadjusted Hazard Ratios, Stratified by PrEP Indications at Baseline, Comparing Uptake of PrEP Among Intervention With Control Participants in a Randomized Controlled Trial of a Mobile HIV Prevention App
| PrEP Indications at Baseline |
No PrEP Indications at Baseline |
|||
|---|---|---|---|---|
| All Participants (n = 112 Intervention, n = 66 Control) |
Participants With Follow-Up Surveys After November 17, 2020 (n = 81 Intervention, n = 47 Control) |
All Participants (n = 273 Intervention, n = 134 Control) |
Participants With Follow-Up Surveys After November 17, 2020 (n = 167 Intervention, n = 79 Control) |
|
| Quarterly surveys only | ||||
| HR for PrEP initiation | ||||
| Intervention | 0.96 (0.59, 1.56)a | 1.18 (0.63, 2.24)b | 0.82 (0.57, 1.18)a | 0.85 (0.53, 1.36)b |
| Control | Ref | Ref | Ref | Ref |
| All participants (n = 119 intervention, n = 67 control) |
Participants with follow-up surveys after November 17, 2020 (n = 83 intervention, n = 48 control) |
All participants (n = 276 intervention, n = 137 control) |
Participants with follow-up surveys after November 17, 2020 (n = 170 intervention, n = 82 control) |
|
| Quarterly and monthly surveys | ||||
| HR for PrEP initiation | ||||
| Intervention | 1.26 (0.79, 2.01)e | 1.64 (0.89, 3.03)f | 0.97 (0.68, 1.38)e | 1.05 (0.67, 1.66)f |
| Control | Ref | Ref | Ref | Ref |
Participants were not able to indicate that they had not initiated PrEP on follow-up surveys until November 17, 2020, so data are presented from the entire study and restricted to participants whose follow-up was completed after that date.
Chi-square test for interaction P-value = 0.6056a, 04132b, 03866c, 02555d.
DISCUSSION
In this randomized controlled trial of a HIV prevention app, HealthMindr-PrEP, we observed a moderate effect of the app on time to PrEP initiation among young GBMSM in the United States. In stratified analyses, the highest magnitude point estimates of the hazard ratio were observed among participants who had PrEP indications according to the 2021 US Centers for Disease Control and Prevention guidelines19 at the time they completed the baseline survey. Although none of the observed effects met traditional criteria of statistical significance (ie, P < 0.05), we note that the study was not powered to conduct stratified analyses. Thus, although compelling, the results of the stratified analysis should be considered preliminary.
This study adds to the body of evidence that mHealth solutions have the potential to drive uptake of HIV prevention services. In an uncontrolled pilot study of the HealthMindr platform conducted in 2015, 9% of app users initiated PrEP over a 4-month follow-up period. We saw much higher PrEP uptake in both arms of this study, likely reflecting the changing landscape regarding PrEP awareness and access since the HealthMindr pilot study was completed. In a separate study13 of a modification of the Healthmindr platform (“Mobile Messaging for Men,” or M3 app) that added HIV prevention messages in infographic and video formats, allocation to the study app was associated with a doubling of PrEP starts compared with an information-only control arm. In this study, the base Healthmindr app was modified to incorporate much more PrEP-related content including frequently asked questions, a HIV risk self-assessment with customized PrEP recommendations, and a PrEP provider locator. Unlike the trial of the previous M3 app, in the current trial, we also sought to address other barriers to PrEP uptake by including general information about health insurance, when and how to screen for HIV and other sexually transmitted infections, and resources related to substance use and mental health. Like this study, in the M3 study, there was also a larger intervention effect among men with PrEP indications at baseline. Another mHealth-based approach, HealthMpowerment, was found to decrease condomless anal sex among young, Black GBMSM in the short term, but the effect was not sustained over 12 months of follow-up.21 PrEP use was not assessed in the HealthMpowerment study. A preliminary efficacy trial for another app, myDEx,22 found evidence indicating that myDEx might result in increased condom use and PrEP uptake among young MSM aged 18–24.
PrEP is a key strategy for reducing HIV incidence and is a cornerstone of the Ending the HIV Epidemic initiative.23 PrEP uptake has not been equitable to date,6 with lower uptake of PrEP among Hispanic and non-Hispanic Black people with PrEP indications compared with non-Hispanic White people. There are a number of barriers to PrEP initiation including cost,24 stigma,25-27 geographic accessibility,28 and access to culturally competent health care.26 HIV prevention apps such as HealthMindr have the potential to reduce some of these barriers by addressing concerns about cost and providing easy access to PrEP provider locators. HealthMindr currently integrates PrEPLocator.org through29 an application programming interface to geolocate nearby PrEP providers; future iterations of the app could incorporate connections to online telePrEP providers and PrEP navigators to further reduce barriers and increase access to PrEP.
Our findings are complicated by some factors that affect data quality. First, we opted to assess PrEP uptake differently on monthly check-in surveys completed by the intervention and control participants. These were uncompensated surveys that, for the intervention participants, were prompts to re-do the PrEP self-assessment. For an attention control, participants in the control arm completed a health and nutrition screener. Intervention arm participants were asked directly on the monthly screeners whether they had recently started PrEP; to avoid intervening on the control participants, we opted to only ask about new prescriptions on the control monthly surveys. The assessment of PrEP uptake was identical on the compensated, quarterly surveys. There were n = 20 intervention participants who reported PrEP uptake on a monthly screener but not on a quarterly survey. This might reflect lack of persistent PrEP use; however, the quarterly survey asked about any PrEP use in the previous 3 months. Thus, even if a participant had discontinued PrEP, they should have responded affirmatively if they had used PrEP at all.
The second data quality issue was that there was not an answer option available on the follow-up surveys until November 17, 2020, for participants to report that they had not received a PrEP prescription. We addressed this by conducting analyses of the full study population and analyses restricted to those whose follow-up occurred after this date; these restricted analyses yielded results indicating a stronger intervention effect compared with the analyses using all survey data. However, these results had reduced sample size and, therefore, reduced statistical power due to the exclusion of 155 participants from these analyses.
We collected paradata to measure app use of participants in both arms of the trial. We have planned analyses to investigate the effect of app engagement on PrEP uptake among participants in the intervention arm. Because the participants in the control and intervention arms had distinctly different app experiences (ie, study timeline and survey access for the control arm; study timeline, survey access, and HIV prevention content for the intervention arm), we opted not to include paradata in the multivariable models for the current analysis.
Our study was affected by the timing of its enrolment during the COVID-19 pandemic, a period in which sexual activity among MSM and PrEP use outside of the study decreased.30-32 Although study activities were continued uninterrupted due to the remote study design, our results are likely affected by changes in sexual behavior and reduced access to health care providers due to physical distancing protocols and shelter-in-place orders.33-35 Ecological analyses have substantiated that PrEP use declined during the COVID-19 pandemic.30-32 We relied on self-reported measures to assess lack of PrEP use on study entry and to assess PrEP starts during the study. These reports are susceptible to misclassification, either because of lack of understanding of the study questions or because of obsequiousness bias.36 However, due to the remote study design and prohibitive cost, we were unable to independently verify PrEP use status for all participants. All participants who reported PrEP initiation were asked to submit a self-collected dried blood spot specimen. Sample collection and laboratory processing errors limit the utility of these data. Person-time calculations are not based on the exact date of PrEP initiation but on the date of the survey on which the participant reported starting PrEP.
Despite the challenges described above, our results indicate a moderate, although not statistically significant, effect of the HealthMindr-PrEP app in increasing PrEP uptake among young GBMSM. The biggest intervention effect was observed among GBMSM with PrEP indications; this suggests that the app is accomplishing the key goal of prompting GBMSM who are most in need of PrEP to use it. When taken together with other studies13,17 demonstrating efficacy of versions of the HealthMindr app that included prevention messaging and other prevention commodities, our results indicate that app-based prevention services to MSM, especially for younger men, like those included in this study, and men with PrEP indications, are a promising avenue to increase access to prevention services for such men. These results also indicate that the efficacy of the HealthMindr platform might depend on the inclusion of adjunctive interventions (eg, prevention messages) and other contextual factors (eg, the COVID-19 pandemic). In this study, we did not observe statistically significant effects, but that is likely in part because of the pandemic and the loss of statistical power due to measurement error in the PrEP uptake question.
Future work is needed to move HealthMindr and similar apps beyond the research environment and into real-world usage; implementation science designs will be invaluable in determining how to most effectively disseminate these interventions.
Acknowledgments
Supported by the National Institute On Drug Abuse of the National Institutes of Health under Award Number R01DA045612. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Footnotes
The authors have no funding or conflicts of interest to disclose.
REFERENCES
- 1.Purcell DW, Johnson CH, Lansky A, et al. Estimating the population size of men who have sex with men in the United States to obtain HIV and syphilis rates. Open AIDS J. 2012;6:98–107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Centers for Disease Control and Prevention. HIV Surveillance Report, 2021. Vol. 34; 2023. Available at: http://www.cdc.gov/hiv/library/reports/hiv-surveillance.html. Accessed October 4, 2023. [Google Scholar]
- 3.Grant RM, Anderson PL, McMahan V, et al. Uptake of pre-exposure prophylaxis, sexual practices, and HIV incidence in men and transgender women who have sex with men: a cohort study. Lancet Infect Dis. 2014;14: 820–829. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Landovitz RJ, Donnell D, Clement ME, et al. Cabotegravir for HIV prevention in cisgender men and transgender women. N Engl J Med. 2021;385:595–608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hess KL, Hu X, Lansky A, et al. Lifetime risk of a diagnosis of HIV infection in the United States. Ann Epidemiol. 2017;27:238–243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Centers for Disease Control and Prevention. Monitoring selected national HIV prevention and care objectives by using HIV surveillance data—United States and 6 dependent areas, 2021. May 2023. Available at: https://www.cdc.gov/hiv/library/reports/hiv-surveillance.html. Accessed July 13, 2023.
- 7.Goldenberg T, McDougal SJ, Sullivan PS, et al. Building a mobile HIV prevention app for men who have sex with men: an iterative and community-driven process. JMIR Public Health Surveill. 2015;1:e18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Goldenberg T, McDougal SJ, Sullivan PS, et al. Preferences for a Mobile HIV prevention app for men who have sex with men. JMIR Mhealth Uhealth. 2014;2:e47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Biello KB, Marrow E, Mimiaga MJ, et al. A mobile-based app (MyChoices) to increase uptake of HIV testing and pre-exposure prophylaxis by young men who have sex with men: protocol for a pilot randomized controlled trial. JMIR Res Protoc. 2019;8:e10694. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Jones J, Butler G, Woody M, et al. Adaptation of a HIV prevention mobile app for transmasculine people: a pilot acceptability and feasibility study. Transgender Health. 2024;9:573–581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Sullivan PS, Zahn RJ, Wiatrek S, et al. HIV prevention via mobile messaging for men who have sex with men (M-Cubed): protocol for a randomized controlled trial. JMIR Res Protoc. 2019;8:e16439. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Dana R, Sullivan S, MacGowan RJ, et al. Engaging Black or African American and Hispanic or Latino men who have sex with men for HIV testing and prevention services through technology: protocol for the iSTAMP comparative effectiveness trial. JMIR Res Protoc. 2023;12:e43414. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Sullivan PS, Stephenson R, Hirshfield S, et al. Behavioral efficacy of a sexual health mobile app for men who have sex with men: randomized controlled trial of Mobile Messaging for men. J Med Internet Res. 2022; 24:e34574. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Jones J, Dominguez K, Stephenson R, et al. A theoretically based mobile app to increase pre-exposure prophylaxis uptake among men who have sex with men: protocol for a randomized controlled trial. JMIR Res Protoc. 2020;9:e16231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.U.S. Department of Health and Human Services. Priority jurisdictions: phase I; 2020. Available at: https://www.hiv.gov/federal-response/ending-the-hiv-epidemic/jurisdictions/phase-one. Accessed June 26, 2020.
- 16.Hall EW, Sanchez TH, Stein AD, et al. Use of videos improves informed consent comprehension in web-based surveys among internet-using men who have sex with men: a randomized controlled trial. J Med Internet Res. 2017;19:e64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Sullivan PS, Driggers R, Stekler JD, et al. Usability and acceptability of a mobile comprehensive HIV prevention app for men who have sex with men: a pilot study. JMIR Mhealth Uhealth. 2017;5:e26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Centers for Disease Control and Prevention. US public health service: preexposure prophylaxis for the prevention of HIV infection in the United States—2017 Update: a clinical practice guideline; 2018. Available at: https://www.cdc.gov/hiv/pdf/risk/prep/cdc-hiv-prep-guidelines-2017.pdf.
- 19.Centers for Disease Control and Prevention: US Public Health Service. Preexposure prophylaxis for the prevention of HIV infection in the United States—2021 Update: a clinical practice guideline; 2021. Available at: https://www.cdc.gov/hiv/pdf/risk/prep/cdc-hiv-prep-guidelines-2021.pdf.
- 20.Brooke J. SUS: a “quick and dirty” usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, et al. , eds. Usability in Industry. Vol 189. Taylor and Francis; 1996:4–7. [Google Scholar]
- 21.Hightow-Weidman LB, LeGrand S, Muessig KE, et al. A randomized trial of an online risk reduction intervention for young black MSM. AIDS Behav. 2019;23:1166–1177. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Bauermeister JA, Tingler RC, Demers M, et al. Acceptability and preliminary efficacy of an online HIV prevention intervention for single young men who have sex with men seeking partners online: the myDEx project. AIDS Behav. 2019;23:3064–3077. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.U.S. Department of Health and Human Services. Ending the HIV epidemic: about ending the HIV epidemic: plan for America: overview; 2020. Available at: https://www.hiv.gov/federal-response/ending-the-hiv-epidemic/overview. Accessed January 14, 2021.
- 24.van Dijk M, de Wit JBF, Guadamuz TE, et al. Slow uptake of PrEP: behavioral predictors and the influence of price on PrEP uptake among MSM with a high interest in PrEP. AIDS Behav. 2021;2021;25:2382–2390. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Fitch C, Foley J, Klevens M, et al. Structural issues associated with pre-exposure prophylaxis use in men who have sex with men. Int J Behav Med. 2021;28:759–767. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Babel RA, Wang P, Alessi EJ, et al. Stigma, HIV risk, and access to HIV prevention and treatment services among men who have sex with men (MSM) in the United States: a scoping review. AIDS Behav. 2021;25: 3574–3604. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Rice WS, Stringer KL, Sohail M, et al. Accessing pre-exposure prophylaxis (PrEP): perceptions of current and potential PrEP users in Birmingham, Alabama. AIDS Behav. 2019;23:2966–2979. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Sharpe JD, Sanchez TH, Siegler AJ, et al. Association between the geographic accessibility of PrEP and PrEP use among MSM in nonurban areas. J Rural Health. 2022;38:948–959. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Siegler AJ, Wirtz S, Weber S, et al. Developing a web-based geolocated directory of HIV pre-exposure prophylaxis-providing clinics: the PrEP locator protocol and operating procedures. JMIR Public Health Surveill. 2017;3:e58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Fallahi M, Guadamuz JS, Shooshtari A, et al. Changes in HIV pre-exposure prophylaxis (PrEP) coverage at state and county level during the COVID-19 pandemic in the United States. AIDS Behav. 2024;28: 799–804. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Hong C, Horvath KJ, Stephenson R, et al. PrEP use and persistence among young sexual minority men 17-24 years old during the COVID-19 pandemic. AIDS Behav. 2022;26:631–638. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Fikslin RA, Goldberg AJ, Gesselman AN, et al. Changes in utilization of birth control and PrEP during COVID-19 in the USA: a mixed-method analysis. Arch Sex Behav. 2022;51:365–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Stephenson R, Sullivan SP, Pitter RA, et al. Are we still having sex? Results of round two of the love and sex in the time of COVID survey with gay, bisexual and other men who have sex with men. AIDS Behav. 2022;26:2338–2348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Sanchez TH, Zlotorzynska M, Rai M, et al. Characterizing the impact of COVID-19 on men who have sex with men across the United States in April, 2020. AIDS Behav. 2020;24:2024–2032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Mann LM, Sanchez T, Stephenson R, et al. The impact of the COVID-19 pandemic on sexual behavior and HIV prevention and treatment services among U.S. men who have sex with men in the post-lockdown era. Am J Mens Health. 2023;17:15579883231168602. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Sackett DL. Bias in analytic research. J Chronic Dis. 1979;32:51–63. [DOI] [PubMed] [Google Scholar]