Pilot randomized controlled trial of Game Plan for PrEP: A brief, web and text message intervention to help sexual minority men adhere to PrEP and reduce their alcohol use

Abstract

Suboptimal adherence to oral PrEP medications, particularly among younger sexual minority men (SMM), continues to be a key barrier to achieving more substantial declines in new HIV infections. Although variety of interventions, including web and text-message-based applications, have successfully addressed PrEP adherence, very few have addressed the potential influence of alcohol. This pilot study explored whether the Game Plan for PrEP, a brief, web-based and text messaging intervention, helped promote PrEP persistence and adherence and reduced condomless sex and alcohol use. Seventy-three heavy-drinking SMM on PrEP were recruited online from states with Ending the HIV Epidemic jurisdictions and randomly assigned 1:1 to receive either the Game Plan for PrEP intervention or an attention-matched control. We collected online surveys assessing primary outcomes at one, three, and six months post-enrollment. As secondary outcomes, we also collected dried blood spot samples at baseline, three, and six months to analyze for biomarkers of PrEP and alcohol use. Our results showed that the odds of stopping PrEP or experiencing a clinically meaningful lapse in PrEP adherence (≥4 consecutive missed doses) were not different across the two conditions. We also did not find evidence of any differences in condomless sex or drinking outcomes across conditions, although participants in both conditions reported drinking less often over time. These findings were consistent across both self-reported outcomes and biomarkers. Overall, we did not find evidence that our brief, web and text messaging intervention encouraged more optimal PrEP coverage or moderate their alcohol use.

Introduction

Although HIV incidence in the United States (US) has declined by about 8% in the last decade, sexual minority men (SMM) continue to be disproportionally affected [1,2]. In 2020, 68% of all new HIV infections among adults and adolescents in the US were attributed to male-to-male sexual contact [1]. Medications that prevent HIV infection, such as pre-exposure prophylaxis (PrEP), could help achieve a sustained decline in new infections, particularly for high-risk groups such as young SMM [3,4]. The first PrEP medication, tenofovir disoproxil fumarate (TDF) and emtricitabine (FTC), was approved by the US Food and Drug Administration in 2012, followed by the approval of tenofovir alafenamide (TAF) and FTC in 2019 [5]. Since then, the US Centers for Disease Control and Prevention (CDC) has integrated PrEP as a key part of its strategy to end the HIV epidemic in the US [6]. According to recent CDC data, about 25% of the 1.2 million people in the US for whom PrEP was recommended were prescribed it, indicating an 8-fold increase between 2015 and 2020 [7].

Despite growth in PrEP use over the past decade, current PrEP utilization is likely too low to reach the US’s goal of reducing new HIV infections by 90% by 2030 [2]. Furthermore, PrEP’s proven efficacy in preventing 99% of HIV transmissions strongly depends on optimal adherence [8,9]. The two most widely prescribed PrEP drugs, TDF/FTC and TAF/FTC, are approved as taking one pill by mouth per day. Past research suggests that five or more doses a week provides >99% protection against HIV infection [8]. However, adherence rates among SMM are often below this threshold, particularly in younger age groups [10,11], conferring continued risk of HIV infection.

A variety of interventions have been developed to support optimal PrEP adherence and persistence among SMM. For example, a study of 50 SMM showed that participants who received a six-session, nurse-led, face-to-face counseling intervention based on the principles of cognitive behavioral therapy called Life Steps for PrEP had significantly higher mean levels of tenofovir diphosphate, a biomarker of recent PrEP adherence [12]. Web and text-message-based interventions have also been developed and have shown promise. In a synthesis of available research findings, the Community Preventive Services Task Force (CPSTF) reviewed seven trials of digital interventions and found that the median percentage of participants who took ≥4 PrEP doses per week was 10% higher among those who received digital interventions compared to those who received standard of care [13]. Therefore, digital interventions are recommended, but their overall impact is modest.

One potential barrier to optimal PrEP adherence that is under-addressed in interventions developed so far is alcohol use. Limited data to date does not suggest a strong role of alcohol use in disrupting day-to-day PrEP adherence [8,14,15], although at least one small study has shown that those with heavier patterns of drinking may also be generally less adherent to PrEP [16]. It is also well-established that alcohol use interferes with a variety of other HIV prevention and treatment behaviors, including safer sex practices and adherence to HIV treatment medications, resulting in poorer health outcomes, increased transmission risk, and higher incidence of other sexually-transmitted infections (STIs) [17–20]. Given their higher risk for HIV, encouraging more heavy drinking SMM to start PrEP, take it consistently, and continue it for as long as they are at risk is an important public health priority.

Meta-analyses have supported the efficacy of brief interventions, such as personalized feedback interventions (PFIs), in reducing alcohol use and related outcomes across a wide variety of settings and populations [21,22]. PFIs generally involve providing feedback to recipients about their drinking and their personal health risks and how these compare with others in a relevant reference group [22]. Partly because of their simplicity, PFIs have been delivered using a variety of digital tools, such as websites, text messages, smartphone apps, and others [23]. Meta-analyses have found that these digitally-delivered interventions are generally as efficacious as those delivered face-to-face [21,23]. However, few studies have explored whether these interventions can help heavy-drinking SMM reduce their alcohol use. In a pilot study of 40 heavy-drinking SMM who were not on PrEP, we showed that those who received a digital PFI after being tested for HIV reported 24% fewer heavy drinking days and 17% fewer alcohol-related problems over three months relative to those who received standard post-test counseling alone [24]. Interventions like these that encourage SMM who are on PrEP to reduce their drinking could help support more optimal PrEP adherence.

In our ongoing work, we developed Game Plan for PrEP, a web application designed to help SMM adhere to PrEP more optimally and reduce their alcohol use. Its content is described in detail elsewhere [25]. Briefly, the intervention (1) addresses risk perceptions by providing detailed feedback about users’ risk for HIV based on their sexual behavior and illustrates how much taking their PrEP consistently reduces that risk, (2) challenges common myths about PrEP to combat stigma, and (3) encourages them to create a plan to take their PrEP regularly. Game Plan for PrEP also (4) provides users feedback about their alcohol use, its health effects, and compares their level of use with relevant norms, and (5) enables them to set goals and create a plan to reduce or stop drinking. Users can then print or email their plans to themselves and elect to sign up for a weekly text messaging service that checks in about any progress toward the goals they set in the app.

In present study, we conducted a pilot test of whether the Game Plan for PrEP app and text messaging service helped promote optimal PrEP persistence and adherence and reduce alcohol use in a small sample of heavy-drinking SMM on PrEP who were recruited from states with Ending the HIV Epidemic jurisdictions. Participants were recruited online and randomly assigned 1:1 to receive either the Game Plan for PrEP intervention or an attention-matched control. Online surveys collected semi-quarterly over 6-months assessed primary outcomes, while dried blood spot (DBS) samples collected at baseline, three, and six months were analyzed for biomarkers of key outcomes to corroborate self-report.

Methods

Participants

Seventy-nine sexual minority men (SMM) were recruited from popular websites, social media platforms (e.g., Facebook, Instagram), and gay-oriented dating smartphone apps (e.g., Grindr, Jack’d) from selected US states with EHE Jurisdictions, primarily in the South, from June 2021 to February 2022. The EHE jurisdictions we focused our recruitment on are reported in Appendix A. We selected these areas due to their high HIV incidence and large populations of Black/African American and Hispanic/Latino residents. Although the EHE jurisdictions identified in these states were primarily specific counties in these states, we opted to recruit from whole states to avoid excluding rural residents. Eligible participants were: (1) 18 years of age or older, (2) assigned male sex at birth, (3) currently male gender, (4) able to speak and read English fluently, (5) currently prescribed and taking a medication for HIV PrEP, (6) currently resided in an identified state (Appendix A), and (7) currently received their PrEP medication from a provider in an identified state, (8) met criteria for “hazardous drinking” using NIAAA criteria, meaning that they reported consuming at least 14 drinks on an average week or drinking ≥ 5 drinks in a single sitting at least once in the past month [26], and (9) had access to a laptop or desktop computer that could be used privately. Participants were excluded if they: (1) had a history of complicated alcohol withdrawal, (2) were currently receiving medications or counseling to address alcohol or drug use, (3) reported injection drug use in the past year, or (4) scored a 3 or higher on the 10-item Drug Abuse Screening Test (DAST-10)[27,28]. As this was an exploratory RCT, a priori power analyses were not conducted to determine the final sample size, and the primary goal was to preliminarily explore the direction and magnitude of any between-group effects among a relatively small sample of participants (N≈50), rather than to be especially thoroughly confident that the results from our regression models identified an effect if one were indeed present. Examining basic descriptive statistics for each outcome across conditions was equally important in evaluating evidence for any benefit of intervention.

Procedures

Figure 1 shows the Consolidated Standards of Reporting Trials (CONSORT) flow diagram of participants through study milestones [29]. Advertisements placed on popular websites (e.g., Google Search), social media (e.g., Facebook, Instagram), and gay-oriented dating apps (e.g., Grindr, Jack’d) were set to display to users in each of the identified states (Appendix A). Ads were rotated frequently and used various combinations of images, video, animation, and graphics to continue attracting interest. Users who clicked on an ad were directed to a landing webpage that briefly reviewed key details of the study. Those interested then went on to complete an online survey that assessed basic eligibility criteria. Those who met basic eligibility criteria were then provided with further information about the study and were asked to complete a three-question, true/false “quiz” to assess their understanding of the study’s procedures. Interested prospective participants who answered these items correctly in three or fewer attempts were then shown the study’s full informed consent document and asked to indicate their consent to participate. Those eligible and who consented were then asked to provide contact information in a separate survey. Once completed, participants were notified that, as a final enrollment step, a member of the research staff would reach out via phone or videoconference to confirm their personal information. Those who completed this step were considered confirmed and formally enrolled, were randomized 1:1 to a study condition (Game Plan for PrEP vs. attention-matched control) by the study database using a random number sequence, and completed the online baseline survey. Once confirmed, research staff prepared a dried blood spot (DBS) self-collection kit for each participant, which was then shipped to the home address they provided at registration. These kits contained: (1) a Whatman 903 Protein Saver card (Cytiva; Marlborough, MA), (2) two alcohol prep pads (1.2 in. × 2.6 in.; McKesson, Irving, TX), (3) two single-use, high-flow, contact activated lancets (2 mm × 1.5 mm, BD Microtainer, Franklin Lakes, NJ), (4) two round spot sheer plastic bandages (McKesson), (5) two gauze pads (2 in. × 2 in.; Dukal Corporation, Ronkonkoma, NY), (6) a biohazard-marked plastic specimen bag (6 in. × 9 in., Lab-Loc^® from Elkay Plastics, Phoenixville, PA) prefilled with a (7) silica desiccant packet (1 g; Dry & Dry, Brea, CA). We also included detailed, illustrated instructions about how to collect the DBS sample, which contained a QR code and link to video instructions, as well. Kits also included a pre-addressed, postage-paid, tearproof bubble mailer marked with an “exempt human specimen sticker” for participants to return their DBS cards. Automated emails sent from a study database reminded participants to return their kits 14 and 18 days after kits were sent. Staff contacted participants by phone, text message, and email if they had not returned their kits 21 days after being sent to encourage them to do so.

Figure 1.

Trial recruitment and enrollment flow.

For participants in both conditions, an additional postcard was included in their DBS test kit that provided more information about their assigned condition and encouraged them to visit a personalized weblink or QR code to complete it. Emails automatically delivered two weeks after participants’ enrollment dates also provided this information, encouraged them to complete their assigned intervention, and provided them with personalized URLs enabling them to do so. Personalized URLs embedded participants’ unique study ID numbers, allowing us to collect any data produced through their interaction with either the experimental or control websites.

Participants assigned to the Game Plan for PrEP condition were asked to use the Game Plan for PrEP website, which has been described in detail elsewhere [25]. Briefly, the website collected basic demographic and behavioral information about users and provided feedback about their risk for HIV, both with and without PrEP, in order to highlight how much continuing to take PrEP consistently reduces their risk for HIV. It also provided feedback about participants’ sexual behavior generally and relative to age-associated norms among SMM in the US, as well as their risk for sexually transmitted infections (STIs) other than HIV, and challenged common myths about PrEP. It then assessed participants’ past-month drinking and provided both general and age-associated normative feedback, comparing users’ quantity of drinking to other US SMM in their age group. Based on assessed levels of motivation to change, participants could then complete a decisional balance to help them weigh the pros and cons of changing their sexual behavior. Finally, users who expressed at least some interest in exploring change were encouraged to consider making a plan to support their PrEP use, reduce their sexual behavior, and/or reduce their drinking. Users could check specific goals (e.g., “set a routine to take my PrEP every day” or “use condoms with all partners”), as well as specific motivations they might have for setting each goal (e.g., “to make me feel safer,” or “to improve my relationship with my partner”) and specific steps they could take to help with their goal (e.g., “keep my meds in a prominent place where they are within reach” or “set an alarm or calendar to remind me”). Any goals and steps participants selected were then displayed on a final page, and participants could elect to email their plans to themselves. On the final page, participants could also elect to sign up for a weekly text messaging service to check in on their goals. The text message service assessed weekly PrEP adherence, sexual behavior, and alcohol use, and gave feedback about how much any changes might have affected the level of HIV risk shown in their initial reports and/or reflected progress toward any goals they may have set.

Those in the control condition were encouraged to visit a website that guided them through educational videos on sleep hygiene and diet. These videos had a total duration of 23 minutes, and afterward, participants responded to general questions about their plans for changing their sleep and diet habits to assess their engagement.

Participants were asked to complete online surveys at baseline and months one, three, and six, and were asked to complete a DBS kit at baseline and months three and six. On the due date of each online survey, automated emails were sent to participants with their personalized link to the survey. Automated reminders were sent three, five, and seven days afterward. After seven days, staff made three further attempts to contact participants by phone, text, and/or email to encourage completion. On each DBS due date, DBS kits were sent to participants’ home addresses using the same procedures as those described for the baseline kit. An automated email was also sent to prompt participants to watch for it, and reminder emails were sent on the same schedule as the baseline kit. Participants in this study were compensated $25 for each online survey they completed, $50 for each DBS kit they completed, and $20 for using their assigned intervention websites (including signing up for text messages, if they were assigned to the Game Plan condition), with a bonus of $50 for completing all study procedures, for a total possible compensation of $320. All procedures were approved by the Brown University Institutional Review Board (IRB) and were registered on ClinicalTrials.gov (NCT04973267).

Measures

PrEP persistence

PrEP persistence, or whether participants had continued to be prescribed and take PrEP in general, was assessed using single binary items collected at each follow-up: “since your last survey, did you decide to stop taking your HIV pre-exposure prophylaxis medication?” Those who responded “yes” to this item at any point during the study were considered to not to have persisted.

PrEP adherence lapses

PrEP adherence lapses were assessed by collecting self-reported PrEP adherence each day over the previous 30 days using an online Timeline Followback (TLFB) we created [30]. Participants first reported whether they had taken any PrEP over the last 30 days, and if so, were shown a calendar of the 30 day period prior to their survey’s due date. Participants were then asked to click the days on which they took their PrEP, and each day was then marked by a unique sticker. Consistent with past research showing that PrEP’s effectiveness is reduced when patients miss four or more doses per week [9], we then coded the number of days that were part of a lapse of four or more missed doses in a row over the past 30 days.

Condomless anal sex (CAS) events

Condomless anal sex (CAS) events were assessed using the same TLFB method and tool we used to assess PrEP adherence. On a calendar showing the past 30 days, participants were first asked to mark the days on which they had oral, anal, or vaginal sex, which were then marked by a unique sticker. A detail view then asked about specific partners and behaviors each day. Participants could report up to four partners each day, and for each partner, were asked whether this partner was “casual,” meaning they had not agreed to have sex with only each other, or “exclusive.” They were also asked whether they engaged in insertive or receptive anal sex with this partner, and for each act, whether they used a condom throughout each act. We coded CAS events as the total number of times that participants reported engaging in insertive or receptive anal sex without using a condom with partners that were “casual” over the past 30 days.

Alcohol use

Alcohol use was also assessed within this same TLFB. Like PrEP adherence and sex, participants were first shown a calendar of the previous 30 days and asked to mark the days on which they consumed alcohol, which were then marked by a unique sticker. A detail view then asked participants to report specifics about their drinking on each day. For each day they reported drinking, participants reported the number of standard drinks they consumed (1.5 oz. liquor, 5 oz. wine, 12 oz. beer) and the number of hours over which they drank. Using these data, we calculated the total number of drinking days and the average number of drinks per drinking day over the past 30 days. We also calculated the total number days on which participants reported drinking 5 or more drinks on a given day, or heavy drinking days, over the past 30 days.

Tenofovir diphosphate (TFV-DP)

Tenofovir diphosphate (TFV-DP) in DBS samples was analyzed by the Colorado Antiviral Pharmacology Lab (CAVP Lab; University of Colorado Anschutz Medical Campus, Aurora, Colorado). TFV-DP is a biomarker of adherence to PrEP [9,31,32] that has a 17-day half-life and provides a measure of cumulative dosing during a period up to 3 weeks before sampling [31]. We used guidance provided by the CAVP Lab to code values of TFV-DP that were suggestive of ≤ 3 doses per week (≤ 799 fmol/punch for Tenofovir disoproxil fumarate and ≤ 949 fmol/punch for tenofovir alafenamide) at each DBS interval to reflect suboptimal PrEP adherence.

Phosphatidylethanol (PEth)

Phosphatidylethanol (PEth) is a phospholipid that appears on red blood cell membranes in response to exposure to alcohol [33]. PEth has been used as a biomarker of recent alcohol use, since it has shown high sensitivity for detecting chronic and binge drinking episodes that occur within 1–2 weeks of sampling [34–36]. PEth quantities in DBS samples (ng/mL) were analyzed by US Drug Testing Laboratories (Des Plaines, IL). We used these reported quantities directly in PEth models. The limit of quantitation was 3.2 ng/mL [37,38]. Peth values that were reported but below this limit were adjusted to their maximum potential value of 3.2 ng/mL.

Data Analysis Plan

First, we calculated overall rates of online survey and DBS completion. We also calculated descriptive statistics on participants’ use of the website and SMS program among those assigned to the Game Plan condition. Primary outcomes of this study were (1) overall PrEP persistence after 6 months of follow-up, and the following outcomes at 1, 3, and 6 months: (2) any PrEP adherence lapse of ≥ 4 concurrent days in the previous 30 days, (3) the number of CAS events, (4) total number of drinking days, (5) average number of drinks per drinking day, and (6) total number of heavy drinking days over the previous 30 days (≥ 5 drinks). Secondary outcomes were (1) TFV-DP and (2) PEth. In all models, we adopted a per-protocol (PP) approach, which involved comparing outcomes across only those participants who successfully completed their respective conditions [39]. Although we considered adopting an intent-to-treat approach, we believe that a PP approach was both more consistent with the exploratory nature of this work and provided a clearer test of the true effects of exposure to intervention content. For those assigned to the Game Plan condition, we considered participants to have successfully completed their assigned condition if they finished using the Game Plan website, as confirmed by web analytics data. We also considered comparing those who used Game Plan and set outcome-related goals (i.e., those assigned to Game Plan that used it and that set an alcohol-related goal), but ultimately elected to keep with PP because part of the site’s goals were to encourage users to set change-related goals. Overall PrEP persistence was a participant-level, binary variable, so we estimated a logistic regression model for this outcome with a dummy variable representing intervention condition assignment as the focal predictor. Age and whether or not participants reported a lapse in the 30 days prior to their baseline were included as covariates in this model. We considered including a covariate reflecting whether participants had quantifiable TFV-DP at baseline, but only three total participants did not meet that criterion, so we opted to include past 30-day lapses instead. To evaluate PrEP adherence lapses, we originally planned to evaluate lapses in adherence within each follow-up interval, but less than 6.9% of all person-periods across the study period involved a lapse. As such, we opted to code a participant-level variable reflecting whether participants reported any such lapse across the entire follow-up period and estimated a similar cross-sectional logistic regression model for this outcome with the same predictors and covariates as the previous model. Since all other outcomes had more variability across follow-up periods, we estimated mixed models for the remaining primary and secondary outcomes. In each of these models, time (follow-up period dummy coded with baseline as the referent), condition, and their interaction were the focal predictors, and age and baseline values for each outcome were included as covariates. CAS, total drinking days, and heavy drinking days were all distributed as count outcomes, with positive skew and many low values or zeros. CAS showed considerable overdispersion, so a negative binomial model was used, while Poisson models were used with total drinking days and heavy drinking days. Log link functions were used in each case. Average drinks per drinking day was approximately normally distributed, so we used a linear model for this outcome. We used a linear model for PEth values as well, after log-transformation to correct positive skew. Finally, given the binary variable we created representing suboptimal adherence in TFV-DP analyses, we used a logistic mixed effects model for this outcome. In all models, we specified exchangeable correlation matrices and robust standard errors. We assumed data were missing at random, and initially estimated all models using all available data. However, since missing data were considerable at some timepoints, we also estimated the above models after multiple imputation using chained equations (MICE; Appendix B) [40]. Results did not differ across imputed and non-imputed models. All analyses were conducted in Stata SE version 14 (College Station, TX).

Results

See Figure 1 for the recruitment and enrollment flow. Although 105 participants consented to the study and registered, 79 were confirmed by staff and enrolled. Of these, a further six participants completed only the baseline online survey and were excluded from further analyses, leaving a final analyzed sample of 73 participants. Table 1 provides basic demographic and behavioral characteristics of the analyzed sample. Participants were largely non-Hispanic White, college educated, and employed, with most residing in the South and mid-Atlantic. Participants had been on PrEP for an average of about 2.5 years at enrollment, and slightly more than half were on TAF/FTC. Most had AUDIT scores that suggested the potential for alcohol use disorder. Online survey response rates and DBS kit return rates are shown in Figure 2. Although online survey response rates were generally high through six months, DBS response rates were more modest, especially by month six.

TABLE 1.

Demographic and behavioral characteristics of the analyzed sample (N=73) by condition assignment

Characteristics	Control (N=36)	Intervention (N=37)
	Mean (SD) or N (%)	Mean (SD) or N (%)
Age (Range: 21 – 59)	33.4 (7.4)	37.4 (10.0)
Ethnicity: Hispanic or Latino	9 (25.0)	9 (24.3)
Race
White	30 (83.3)	30 (81.1)
Black or African American	4 (11.1)	4 (10.8)
Asian	0 (0.0)	1 (2.7)
American Indian/Alaska Native	0 (0.0)	0 (0.0)
Multiracial	1 (2.8)	2 (5.4)
Chose not to respond	1 (1.4)	0 (0.0)
College degree	32 (88.9)	32 (86.5)
Low income1	5 (13.9)	5 (13.5)
Unemployed	1 (2.8)	2 (5.4)
In a committed relationship	20 (55.6)	13 (35.1)
Sexual identity
Gay	35 (97.2)	33 (89.2)
Bisexual	1 (2.8)	4 (10.8)
Region
South	15 (41.7)	21 (56.8)
Mid-Atlantic	11 (30.6)	11 (29.7)
West	10 (27.8)	5 (13.5)
Avg. drinks per week	14.3 (8.5)	13.3 (8.0)
High AUDIT2 (>16)	26 (72.2)	23 (62.2)
DAST total score	0.94 (1.0)	0.81 (1.2)
PrEP medication
Tenofovir disoproxil fumarate (TDF)	16 (44.4)	16 (43.2)
Tenofovir alafenamide (TAF)	19 (52.8)	21 (56.8)
Switched from TDF to TAF	1 (2.8)	0 (0.0)
Years on PrEP	2.7 (2.2)	2.5 (2.2)

Note.

¹Represents those with a household income of <$30,000/year.

²Alcohol Use Disorders Identification Test.

³Drug Abuse Screening Test.

Figure 2.

Online survey response rates and dried blood spot sample return rates over the course of the 6-month study period.

Website and text messaging use

Of those assigned to the Game Plan for PrEP condition, 95% (N = 35) used the website, and all of those who used the website viewed all of its content. Of those who used the website, 61% set a goal relevant to improving their PrEP adherence. Sixty-one percent also set at least one goal to reduce their sexual risk, but only half of these were goals that involved using condoms more often. Fifty-eight percent of those who used the site set a goal to reduce their alcohol use, with 17% electing to focus on stopping their drinking entirely and the remaining hoping to reduce their drinking. Of the 35 participants who completed the site, 75% elected to email their change plans to themselves, and 75% opted to sign up for text messages to check in on their plans. Sixty-seven percent went on to complete all 12 weeks of the text messaging portion. Of those assigned to the control condition, 92% (N = 33) viewed the assigned videos.

PrEP outcomes

Only 8.2% of participants (N=6) reported ever having a lapse in PrEP adherence of four or more days throughout the 6-month follow-up period; 8.3% of participants in the control group reported such a lapse, versus 8.1% of the intervention group. As such, we coded a variable reflecting whether participants had reported any lapse of ≥4 days across the study period and estimated cross-sectional logistic regression models for this outcome. Results are shown in Table 2. The odds of reporting any lapse in PrEP adherence did not significantly differ across intervention condition. Similarly, only 16% of participants (N=12) reported having explicitly stopped taking PrEP at some point during the study period; 19% in the control condition (N=7) and 13% in the intervention condition (N=5). In a cross-sectional logistic regression model, the odds of reporting having stopped taking PrEP at any time during the study period did not significantly differ across intervention condition (Table 2).

TABLE 2.

Cross-sectional logistic regression models of PrEP adherence outcomes

Variable	Any PrEP adherence lapse1				PrEP persistence2
	OR	SE	p	95% CI	OR	SE	p	95% CI
Age	0.94	0.07	.375	0.81–1.08	0.98	0.05	.709	0.88–1.09
BL PrEP adherence (30d)	1.00	0.19	.987	0.70–1.44	0.95	0.13	.713	0.72–1.25
Intervention condition3	0.67	0.64	.678	0.10–4.40	0.60	0.50	.544	0.12–3.09

Note.

¹Reflects whether participants self-reported a lapse in PrEP adherence of ≥4 concurrent days at any time during the 6-month study period.

²Reflects whether participants self-reported having stopped taking PrEP at any time during the 6-month study period.

³Intervention condition effects were analyzed in a per-protocol manner.

Condomless anal sex (CAS) events

Participants reported an average of 2.6 CAS events (SD=2.9) with non-sexually-exclusive partners in the 30 days period to enrollment, but this declined to an average of 1.7 events at Month 1 (SD=2.4), 1.4 in Month 3 (SD=2.4), and 1.4 in Month 6 (SD=2.4). In the negative binomial mixed model of this outcome (Table 3), the overall test of the interaction between intervention condition and follow-up period was not significant (Wald X²[2]=1.11, p=.574). Similarly, although effects were in the expected directions, the main effect of intervention condition was not statistically significant, nor were contrasts between specific follow-up periods (e.g., 3 month, 6 month) and baseline. See Figure 3 for model-predicted marginal means by study condition and follow-up period.

TABLE 3.

Negative binomial mixed model of condomless anal sex (CAS) events with non-exclusive partners

Variable	IRR	SE	p	95% CI
Age	0.99	0.02	.698	0.96–1.03
BL CAS events (30 d)	1.22	0.06	<.001	1.12–1.34
Time (vs. month 1)
Month 3	0.86	0.22	.550	0.51–1.42
Month 6	0.63	0.20	.140	0.35–1.16
Intervention condition	0.70	0.25	.311	0.35–1.40
Intervention*Month 3	0.94	0.36	.870	0.44–2.00
Intervention*Month 6	1.32	0.56	.516	0.57–3.02

Figure 3.

Model-predicted CAS events with non-exclusive partners by intervention condition and follow-up period

Alcohol use outcomes

Participants reported an average of 11.8 drinking days (SD=7.5) in the 30 days prior to enrollment, which decreased to 9.5 at Month 1 (SD=7.3), 8.2 at Month 3 (SD=6.8) and 6.0 at Month 6 (SD=5.2). In the Poisson mixed model (Table 4), the overall interaction between intervention condition and follow-up period was not significant (Wald X²[2]=0.34, p=.842). Likewise, the main effect of condition assignment was not statistically significant. However, participants in both conditions reported significantly fewer drinking days at both 3 and 6 months when compared to Month 1, independent of intervention condition assignment (see Figure 4). Participants reported consuming an average of 3.9 drinks (SD=2.3) when they drank in the 30 days prior to enrollment, which stayed relatively stable at 3.7 drinks (SD=2.4) at Month 1, 4.3 drinks at Month 3 (SD=2.1), and 3.9 (SD=2.2). In the linear mixed model (Table 4), the overall interaction between intervention condition and follow-up period was again not significant (Wald X²[2]=0.56, p=.757). Likewise, neither the main effects for intervention condition nor follow-up period were significant. Finally, participants reported an average of 3.9 heavy drinking days (SD=4.7) in the 30 days prior to enrollment, which declined to 2.7 heavy drinking days at Month 1 (SD=3.4), 2.6 days at Month 3 (SD=3.1), and 2.1 days at Month 6 (SD=2.5). In the full Poisson mixed model, the overall interaction between intervention and condition was not significant (Wald X²[2]=1.14, p=.565). Although the overall main effect of intervention condition was in the expected direction, there were no statistically significant differences in heavy drinking days in the intervention group compared to the control group. The main effect of follow-up period was also not significant.

TABLE 4.

Poisson & linear mixed models of alcohol outcomes

Variable	Total drinking days				Avg. drink/drinking day				Heavy drinking days1
	IRR	SE	p	95% CI	β	SE	p	95% CI	IRR	SE	p	95% CI
Age	1.01	0.01	.261	0.99–1.03	−0.03	0.03	.194	−0.08–0.02	0.98	0.02	.243	0.95–1.01
BL outcome variable2	1.05	0.01	<.001	1.03–1.07	0.42	0.09	<.001	0.24–0.60	1.13	0.04	<.001	1.06–1.20
Follow-up period (vs. M1)
Month 3	0.83	0.08	.050	0.68–0.99	0.76	0.40	.056	−0.02–1.53	1.04	0.17	.801	0.76–1.42
Month 6	0.69	0.10	.012	0.52–0.92	0.48	0.41	.238	−0.32–1.27	0.75	0.13	.112	0.53–1.07
Intervention condition3	1.01	0.17	.940	0.72–1.41	0.24	0.51	.639	−0.76–1.24	0.83	0.23	.508	0.48–1.44
Intervention*M3	1.01	0.13	.911	0.78–1.32	−0.16	0.55	.774	−1.25–0.93	0.87	0.21	.573	0.54–1.40
Intervention*M6	0.93	0.14	.629	0.69–1.24	−0.42	0.56	.458	−1.53–0.69	1.16	0.30	.568	0.70–1.93

Note.

¹Reflects whether participants reported drinking 5 or more drinks on a given day.

²Reflects the baseline value of the outcome variable.

³Intervention condition effects were analyzed in a per-protocol manner.

Figure 4.

Model-predicted alcohol outcomes by intervention condition and follow-up period

Secondary outcomes - TFV-DP and PEth

At baseline, 15.7% of participants who returned their DBS samples had TFV-DP levels that were consistent with having taken fewer than four doses per week in the weeks prior to collection. This percentage increased to 24.6% at Month 3, and 25.5% at Month 6. Spearman’s rank correlations showed that the category of adherence level reflected in participants’ TFV-DP levels were generally not correlated with their levels of self-reported adherence at the same timepoint (ρ=0.15, p=0.13). In a logistic mixed model (Table 6), self-reported PrEP doses taken at baseline was not associated with suboptimal TFV-DP levels at a given time point. Baseline TFV-DP levels were also not associated with suboptimal TFV-DP levels at later timepoints, although the magnitude of associations was large and in the expected direction. However, neither the two-way interaction between intervention condition and follow-up period, nor the main effects of intervention condition and follow-up period were significantly associated with suboptimal TFV-DP levels. See Figure 5, left panel.

TABLE 6.

Logistic and linear mixed models of tenofovir-diphosphate (TFV-DP) levels and phosphatidylethanol (PEth)

Variable	Tenofovir-diphosphate1				Variable	Phosphatidylethanol
	OR	SE	p	95% CI		β	SE	p	95% CI
Age	1.20	0.13	.086	0.94–1.48	Age	0.31	0.53	.556	−0.73–1.36
BL self-reported PrEP adherence	0.90	0.22	.663	0.56–1.44	BL total drinking days	0.58	0.63	.360	−0.66–1.81
BL TFV-DP	6.37	11.12	.288	0.21–194	BL PEth quantity	0.84	0.06	<.001	0.71–0.97
Follow-up (M6 vs. M3)	1.18	0.80	.811	0.31–4.47	Follow-up (M6 vs. M3)	18.11	12.4	.144	−6.18–42.41
Intervention2	0.15	0.24	.231	0.01–3.32	Intervention2	18.61	12.1	.123	−5.04–42.25
Intervention*follow-up	9.13	14.78	.171	0.38–217	Intervention*follow-up	−41.53	17.12	.015	−75.10–−8.00

Note.

¹Reflected whether participants had TFV-DP levels that were consistent with having taken > 4 doses per week.

²Intervention condition effects were analyzed in a per-protocol manner.

Figure 5.

Model-predicted biomarker outcomes, TFV-DP and PEth, by intervention condition and follow-up period

Participants’ median PEth quantities were 73.8 ng/mL at baseline (IQR=91.0), which decreased slightly to 58.3 ng/mL at Month 3 (IQR=105.4), but increased again to 71.7 ng/mL at Month 6 (IQR=89.2). In the full linear mixed model, participants’ baseline PEth quantities were significantly associated with their PEth quantity at each follow-up timepoint. The two-way interaction between intervention condition and follow-up timepoint was also statistically significant (Wald X²[2]=74.22, p<.001). However, follow-up contrasts showed that this effect was primarily driven by higher PEth quantities among intervention participants at month 3 (versus control) decreasing to lower PEth at month 6, relative to control participants. That is, the significant interaction term was driven partly by higher PEth values among intervention participants at the shorter-term follow-up. See Figure 5, right panel.

Discussion

In this pilot study, we tested whether brief, web-based content and weekly text messages addressing key determinants of PrEP and alcohol use would help encourage PrEP adherence and reductions in drinking among PrEP-experienced SMM recruited from EHE areas. Overall, our results did not provide strong evidence that using these tools improved outcomes in this sample, whether those outcomes were assessed via self-report or via biomarkers. Our finding that this intervention did not appear to reduce clinically-meaningful lapses in PrEP adherence (≥3 days) or encourage more participants to persist with PrEP contrast with findings from a recent research synthesis conducted by the CPSTF showing that, across seven published studies, text messaging programs appeared to increase the percentage of participants with good adherence to PrEP [13]. Several differences between the studies included in this synthesis and the current study could explain these contrasting findings. First, the majority of studies included in the CPSTF synthesis enrolled participants with less PrEP experience than this study, which followed participants who had been on PrEP for an average of over two years prior to enrolling. SMM with more PrEP experience may have already established adherence routines that are more difficult to change than those who with less experience. Given that this sample was also very adherent in general, these factors may have made it more difficult to observe changes. Second, most of the studies included in the synthesis also provided daily text messages over longer periods of time (e.g., 9–12 months) [41,42] than our approach, which used weekly text messages over a much shorter period of time (12 weeks). Although we chose this briefer, less frequent approach to explore a strategy that might be more feasible for users to engage with in clinical contexts, providing messages daily over many months may be key to the effectiveness of text messaging in supporting adherence. However, third, it is also possible that the evidence supporting text messaging in general could be less conclusive than the CPSTF’s summary suggests. Of the seven studies included in the synthesis, three evaluated interventions that included content other than text messages, such as apps or games that had other content or interventions that involved other programming. Of those focusing on text messaging specifically, only one that had a large sample size and used a rigorous design (e.g., used randomization, included a control group) provided compelling evidence of benefit, with 15% more participants having TFV-DP levels that were consistent with good PrEP adherence in the text messaging condition versus the control condition [41]. Other such studies, like this study, were small [43,44], showed little/no benefit [42,45], or showed that text messaging mostly appeared to increase adherence among those who already had adequate adherence, with limited effects in encouraging those with inadequate adherence achieve more adequate adherence. Findings from this study, as well as several other ongoing trials, will help to clarify these effects. Despite this uncertainty, though, text messaging is a relatively low-cost, low-burden intervention to implement and maintain, and so, may still be worth implementing in settings that provide PrEP care to large numbers of patients.

Our findings also provide limited evidence that this intervention helped SMM reduce their drinking. Specifically, results suggested that heavy drinking SMM in both conditions reported drinking less often over time, such that participants overall drank about 45% less often after 6-months relative to baseline. However, participants did not report significant changes in the average number of standard drinks they consumed per drinking episode over time, and there were no differences across intervention condition. There was some evidence that participants in the intervention condition may have reported slightly fewer heavy drinking days (≥ 5 standard drinks) at earlier follow-ups (e.g., 1, 3 months), but these differences were not statistically significant and the two groups had converged by six months. This pattern of findings contrasts with previous studies of brief, web-based interventions for alcohol that involve personalized feedback, which have shown that these interventions are effective in a wide variety of populations [21]. These results also contrast with findings from a similar pilot trial we conducted in a small sample of heavy drinking SMM who were not on PrEP, which found that SMM who completed an earlier version of the website after being tested for HIV reported 27% fewer drinking days, 24% fewer heavy drinking days, and 17% fewer alcohol-related problems than those who received only standard post-test counseling alone [24]. One potential explanation for these convergent findings is that SMM who are on PrEP may not be as open to considering the impact of drinking on their HIV risk as those who are not on PrEP. Specifically, there is evidence that a primary motivation for using PrEP among SMM is to manage the HIV risks involved in ongoing sexual behavior [46,47], which alcohol may play a key role in. That is, some SMM may use PrEP in part to avoid needing to take other steps to reduce their risk, including reducing their alcohol use. However, the specific links and mechanisms whereby alcohol affects PrEP use are not as clear as the mechanisms in which alcohol affects condom use [16,19], so another potential explanation for these contrasting findings could be that SMM on PrEP were not as convinced of the need to change their drinking as SMM who were not on PrEP were. Future research should explore whether SMM’s response to interventions like these is indeed moderated by PrEP use and whether these mechanisms might explain this variability in response. This is particularly important to examine given that trials to date have been small scale studies that were not powered to detect significant effects.

Beyond these primary outcomes, our findings add to the growing body of research suggesting that it is feasible to guide clinical trial participants in collecting biospecimens at home [48,49], so that biomarkers can be used as key endpoints, even in the context of virtual, siteless, or “limited interaction” clinical trials. Our results suggest that, with procedures in place to support participant’s adherence to these procedures, many participants recruited into a pilot, virtual clinical trial will return dried blood spot samples. Although our return rate at six months was affected by a malfunction in our study database that resulted in some automated reminders to return DBS samples not being sent on schedule, response rates at other timepoints were generally stronger and mostly support the feasibility of this approach. Our findings also support the large body of research showing that patients often over-report their adherence to PrEP when asked to self-report, compared to biomarkers of adherence like TFV-DP [50–53]. Together, these results lend further support for the need to incorporate biomarkers into the primary outcomes of studies evaluating PrEP interventions, and suggest that doing so should be feasible even in more efficient study designs, like virtual and pragmatic clinical trials.

Limitations

Although this study has important strengths, a number of key limitations should also be noted. First, the SMM enrolled in this study had substantially more PrEP experience than those recruited in past studies of similar interventions, which mostly focused on SMM and other populations who had initiated PrEP relatively recently [CPSTF, 13]. Although we had initially planned to follow suit, COVID-19 restrictions at the clinical sites we had initially planned to partner with made recruitment difficult. We further believed that web-based and text messaging interventions might also be beneficial to those who had been on PrEP for longer, but who still struggled to use it consistently. However, it is possible that interventions like these may provide more benefit for those who are earlier on in PrEP care, when they are establishing habits and routines around PrEP. Second, a substantial number of participants enrolled in this study also had high scores on the Alcohol Use Disorder Identification Test (AUDIT, >16), meaning that many were likely at high risk for alcohol use disorders. These individuals may have required more extensive intervention than brief technology-facilitated programs like these can provide and may partly explain the lack of response we observed in drinking outcomes. Likewise, roughly 10% more participants in the intervention group had high scores on the AUDIT, which could have made it difficult to observe changes relative to a control group with fewer at-risk participants. Our finding that participants in both groups generally reported reductions across some alcohol outcomes also raises the possibility that this pattern could have occurred because of reactivity to the online TLFB assessment. That is, participants may have reported fewer alcohol use events in order to shorten the length of the survey. However, because participants in both conditions completed the same procedure, we are fairly confident in our conclusion that the results reported do not provide strong evidence in support of the intervention’s effects on these outcomes. Finally, the majority of the sample we recruited were white, non-Hispanic men, so our findings may not generalize to other groups and populations. Future studies should also examine digital interventions for other populations with relatively high HIV burden such transgender women.

Summary

In sum, we did not find evidence that a brief, web and text messaging intervention helped SMM optimally adhere and persist with PrEP or reduce their alcohol use relative to an attention-matched control. These findings extended both to outcomes collected via self-report and in biomarkers relevant to PrEP and alcohol use. Our findings contrast with several recent studies demonstrating that similar online and text messaging interventions improve PrEP use among SMM and reduce drinking in other populations.

Data, Materials, and/or Code availability

Data used in this project will not be deposited in a data repository or archive.