Prepared, Protected, EmPowered (P3): Primary Results of a Randomized Controlled Trial Using a Social Networking, Gamification, and Coaching App to Promote Pre-exposure Prophylaxis (PrEP) Adherence for Sexual and Gender Minority (SGM) Youth Living in the United States

Abstract

The population-level reductions in HIV incidence attributed to pre-exposure prophylaxis (PrEP) have not been fully realized among sexual, gender, and racial/ethnic minority youth. P3 (Prepared, Protected, emPowered) is a comprehensive mobile application (app) intervention developed to support PrEP adherence through gamification, medication tracking, and social engagement. A randomized controlled trial was conducted with 246 young men who have sex with men (YMSM) and young transgender women who have sex with men (YTWMSM) currently on or planning to begin PrEP. Participants were randomized in a 1:1:1 ratio to: standard of care (SOC), P3 app (P3), or P3 app plus in-app adherence coaching (P3+). Adherence was measured at 3- and 6- months post enrollment by emtricitabine-triphosphate (FTC-TP) and tenofovir-diphosphate (TFV-DP) levels in dried blood spots consistent with PrEP use > 4 days/week. The primary outcome was the difference in the proportion adherent comparing P3/P3+ to SOC. P3/P3+ was associated with a higher proportion adherent compared to SOC for both outcome measures. At 3 months, the estimated increase in the proportion adherent was 0.13 (95% CI: 0.00, 0.27, p: 0.05) and 0.12 (95% CI: −0.03, 0.26, p: 0.11) for TFV-DP and FTC-TP, respectively. Estimated adherence was higher, but not statistically different, in P3+ compared to P3, for both TFV-DP and FTC-TP. Receipt of P3 (P3 or P3+) is associated with an increase in PrEP adherence among YMSM and YTWSM at 3 months. Additional analyses to discern the role of app usage and sociodemographic and behavioral factors on intervention effects are warranted.

Introduction

While new HIV infections are declining in the United States (US), gay, bisexual and other men who have sex with men (MSM) continue to bear the heaviest HIV burden, accounting for 71% of new infections in 2021 [1]. Most new HIV diagnoses are among adolescents and young adults aged 13–34 [2]. Transgender persons account for ~ 2% of new HIV diagnoses in the US, and young transgender women (YTW) account for 92% of these new HIV diagnoses among transgender persons [2]. To meet the Ending the HIV Epidemic in the US (EHE) target of decreasing new HIV diagnoses by 90% by 2030, young men who have sex with men (YMSM) and YTW need improved access to proven HIV prevention strategies [3].

Pre-exposure prophylaxis (PrEP) is an HIV prevention strategy included in the EHE plan [3]. Daily, oral PrEP is effective against HIV acquisition with sufficient adherence at ≥ 4 doses per week; [4, 5] however, adherence to daily PrEP among YMSM and YTW who have sex with men (YTWSM) remains low [6–15], suggesting a need for PrEP interventions tailored to the priorities of young sexual and gender minorities. Digital health interventions via smartphone apps have increased substantially in recent years given widespread use of these devices, ability to deliver interventions in real time, and low implementation costs after initial development investments [16–21]. Given the high uptake and use of smartphones among young people [22], a smartphone app is a suitable intervention platform to support PrEP adherence.

Building on findings from studies aimed to improve antiretroviral adherence among people with HIV, smartphone apps can be tailored to meet the needs of YMSM and YTWSM seeking HIV prevention services [23–27]. Thus, our team developed and tested P3 (Prepared, Protected, emPowered), a smartphone app designed for YMSM and YTWSM who are initiating or currently taking PrEP. P3 includes social engagement, support from peers through a social discussion board, and medication tracking. App components such personalized medication reminders, educational content (e.g., informational resources about PrEP, HIV prevention, sexual health and wellness, etc.) and gamification features were developed through usability and field testing with YMSM and YTWSM [28]. Usability testing demonstrated sustained usage over the testing period and high acceptance [29]. A three-arm randomized controlled trial (RCT) was conducted to test the efficacy of the P3 app (P3 arm) and the P3 app combined with individual text-based counselling with an adherence coach (P3 + arm) compared with standard of care (SOC) adherence messages (control arm). The aim of this paper is to report the efficacy of the P3 app in increasing PrEP adherence among YMSM and YTWSM.

Methods

Study Design and Population

We conducted a three-arm RCT from March 2019 to September 2021 and followed each participant for six months. Participants were recruited through the Adolescent Medicine Trials Network for HIV Interventions (ATN) from nine subject recruitment venues (SRVs) in Bronx, New York; Chicago, Illinois; Atlanta, Georgia; Houston, Texas; Boston, Massachusetts; Philadelphia, Pennsylvania; Tampa, Florida; Charlotte, North Carolina; and Chapel Hill, North Carolina. Recruitment methods included in-person, venuebased (including recruitment from local PrEP clinics or providers) and web-based recruitment on social media and networking platforms (paid advertisements on Facebook, Instagram, Grindr, and Jack’d). Eligible participants were (1) aged 16–24 years; (2) assigned male sex at birth; (3) report sex with or intentions to have sex with men or transgender women (TW); (4) had reliable daily access to an Android or iOS smartphone with a data plan (to be able to access the P3 app); (5) able to speak and read English; (6) self-reported HIV-negative; and (7) were not currently on PrEP but plan to initiate in the next 7 days and had an active PrEP prescription (prescription confirmed by study staff) OR on PrEP and have an active PrEP prescription (prescription confirmed by study staff).

Study Procedures

After providing informed consent either in-person or electronically, eligible participants provided a baseline dried blood spot (DBS) sample for tenofovir-diphosphate (TFV-DP) and emtricitabine-triphosphate (FTC-TP) and completed a baseline computer assisted survey self-interview (CASI). Participants were then randomized in a 1:1:1 ratio to one of three treatment arms: SOC, P3, or P3+. Those randomized to SOC, were informed of services available at their provider’s clinic and provided written materials on PrEP adherence. Those randomized to P3 received the P3 app, which included features meant to build self-efficacy while establishing and reinforcing positive health habits. Participants randomized to P3 + received the P3 app plus access within the app to trained adherence coaches. After randomization, study staff guided P3 and P3 + participants through app download and provided an overview of key features. Participants randomized to the P3 + arm also received an overview of the coaching component. Those randomized to the P3 or P3 + study arm were incentivized to use the app during the first 3 months of study follow-up. Specifically, participants in the P3 or P3 + condition started with a baseline amount of USD in a virtual bank account ($90) and gained (+$0.50) or lost (-$1.00) money from this initial amount each day corresponding to use and non-use of the app respectively. The final amount in the virtual bank account (total possible ranged from $0.00, representing no app use - to $135, representing perfect daily app use) was paid out as part of the study remuneration at the 3-month study follow up.

Three-month and 6-month study follow-up included collection of DBS sample for TFV-DP and FTC-TP and follow-up CASI. Participants were compensated with US $50 upon completion of their enrollment visit, and US $50 each at the 3- and 6-month visits (US $150 total). TFV-DP and FTC-TP were quantified in DBS as previously published using fully validated LC-MS/MS methods [30]. Additional details about study procedures and development of the P3 app were reported in a prior publication [28].

Outcomes

The primary outcome in the study was a protective level of PrEP, which was quantified in two ways: TFV-DP levels in the blood consistent with ≥ 4 pills per week and FTC-DP levels in the blood consistent with ≥ 4 pills per week as previously defined [31]. Each of these outcomes was analyzed separately at both 3-and 6-months for adherence. Adherence as measured by self-reported PrEP use was a secondary outcome. This was measured in two ways, self-reported PrEP use in the past week (number of days in the last 7 days that PrEP was taken) and in the past month (percent of time in the past month where PrEP was taken from 0 to 100% using intervals of 10%); these outcomes were analyzed separately.

Sample Size and Power

Sample size was determined using a simulation-based approach, with sample size selected to ensure greater than 80% power to detect differences in adherence of approximately 14% between the comparison groups assuming 10% missing data by month 6. For additional details, see supplementary material of protocol paper [28].

Statistical Analyses

Descriptive Statistics

Descriptive statistics were calculated for important baseline variables (Table 1), for all participants in the study population and stratified by treatment arm. Median and inter-quartile range (IQR) were calculated for continuous variables and count and percentage were calculated for categorical variables.

Table 1.

Baseline participant characteristics overall and by treatment group.

	Overall (N=246)		SOC (N=83)		P3 (N=82)		P3+ (N=81)
	N	%	N	%	N	%	N	%
Age
Median, IQR	22.0	20,23	22.0	21,23	22.0	20,23	22.0	20,23
Hispanic/Latino
Yes	71	28.9	27	32.5	26	31.7	18	22.2
No	175	71.1	56	67.5	56	68.3	63	77.8
Race
Black	51	20.7	16	19.3	16	19.5	19	23.5
Asian	16	6.5	4	4.8	7	8.5	5	6.2
White	137	55.7	42	50.6	49	59.8	46	56.8
American Indian or Alaska Native	3	1.2	1	1.2	1	1.2	1	1.2
Multiple races	16	6.5	9	10.8	3	3.7	4	4.9
Other race	1	0.4	1	1.2	0	0.0	0	0.0
Missing/decline to answer	22	8.9	10	12.0	6	7.3	6	7.4
Sexual orientation
Gay	197	80.1	63	75.9	70	85.4	64	79.0
Bisexual	27	11.0	14	16.9	6	7.3	7	8.6
Queer	18	7.3	4	4.8	6	7.3	8	9.9
Straight	2	0.8	1	1.2	0	0.0	1	1.2
Pansexual	2	0.8	1	1.2	0	0.0	1	1.2
Gender identity
Cis-man	216	87.8	74	89.2	71	86.6	71	87.7
Non-binary/gender-queer	9	3.7	4	4.8	3	3.7	2	2.5
Trans-woman/trans-feminine	5	2.0	2	2.4	0	0.0	3	3.7
Multiple genders reported	14	5.7	3	3.6	8	9.8	3	3.7
Missing/decline to answer	2	0.8	0	0.0	0	0.0	2	2.5
Currently employed
Yes	183	74.4	62	74.7	62	75.6	59	72.8
No	62	25.2	21	25.3	20	24.4	21	25.9
Missing/decline to answer	1	0.4	0	0.0	0	0.0	1	1.2
Self-reported income
Low income	126	51.2	46	55.4	40	48.8	40	49.4
Middle income	95	38.6	26	31.3	36	43.9	33	40.7
High income	7	2.8	5	6.0	1	1.2	1	1.2
Missing/decline to answer	18	7.3	6	7.2	5	6.1	7	8.6
Relationship status
In a relationship	42	17.1	13	15.7	13	15.9	16	19.8
Married or other legal commitment	1.0	0.4	1	1.2	0	0.0	0	0.0
Single – casually dating	103	41.9	37	44.6	39	47.6	27	33.3
Single – not currently dating	98	39.8	32	38.6	30	36.6	36	44.4
Missing/decline to answer	2	0.8	0	0.0	0	0.0	2	2.5
Ever Used PrEP
Yes, current	225	91.5	75	90.4	76	92.7	74	91.4
Yes, restarting	10	4.1	3	3.6	2	2.4	5	6.2
No	11	4.5	5	6.0	4	4.9	2	2.5

Exploratory Analysis of the Relationship between Baseline Survey Response and DBS Data

Dried blood-related biomarkers provide an objective and biochemical measure of drug levels, reflecting actual medication intake over a specific period [32, 33]. Self-reported adherence, on the other hand, offers information about participants’ perceived adherence and behaviors, but it may be susceptible to social desirability bias, recall inaccuracies, and reporting errors [34, 35]. Thus, to determine the extent to which self-reported adherence aligns with biomarker data and to assess the accuracy and reliability of these measures, the relationship between participant survey responses and adherence measured by FTC-TP and TFV-DP via DBS biomarker was examined. First, summary statistics for the survey responses were calculated, stratified by blood adherence measures, to understand differences in reporting between those who were adherent and non-adherent. Next, Spearman’s correlation coefficients between DBS and self-report adherence (weekly and monthly) measures at three time points (i.e., baseline, 3-, and 6-months) were calculated for those who had both measures.

Treatment Effect Measures

From a causal inference perspective, the average treatment effect was the quantity of interest, i.e., the expected difference in each outcome of interest if everyone were assigned to an intervention of interest (e.g., P3/P3+) versus if everyone were assigned to SOC. P3 and P3 + were combined in the primary analysis and examined separately in secondary analyses according to a pre-specified analysis plan. When the outcome is binary, the average treatment effect is interpreted as a difference in proportions. For example, when the outcome is PrEP adherence, the average treatment effect quantifies the difference between the proportion of participants adherent to PrEP if everyone was assigned to P3/P3 + compared to if everyone was assigned to SOC.

Targeted Maximum Likelihood Estimation (TMLE) for Estimating Treatment Effects

Targeted maximum likelihood estimation (TMLE) was used to estimate the average treatment effect at both 3 months and 6 months, comparing P3/P3 + to SOC in terms of both the primary and secondary outcome measures of interest. TMLE required two types of regressions to be fit in the estimation process, referred to as propensity scores and outcome regressions. The required propensity scores were the probability of intervention assignment (known) and the probability of missingness in the outcome given treatment and baseline covariates (unknown). The outcome regression models the expected outcome given treatment and baseline covariates.

Since treatment was randomized, the probability of treatment was calculated with a sample proportion. Super learning was used to estimate the probability of missingness in the outcome given treatment and baseline covariates. Super learning is a machine learning method that, when given a set of algorithms, uses cross-validation to determine the optimal convex combination of the algorithms [36]. Similarly, the outcome regression was estimated with super learning.

Many DBS records were missing due to clinic lockdowns and study pause during the height of the COVID-19 pandemic. A high level of missing outcome data can decrease the power for detecting treatment effects. When estimating treatment effects on the primary (DBS) outcomes, self-report data was included in outcome regressions as it was presumed to be predictive of the missing DBS data. This was done by adding another outcome regression layer into the TMLE estimation framework, where the DBS data at a given follow-up time point (3 or 6 months) was regressed on self-reported adherence collected at that same time point (3 or 6 months), treatment, and relevant baseline variables. The self-reported adherence measures used in this regression were number of days in the last 7 days that the participant reported taking PrEP and percent of time in the last month that PrEP was taken as prescribed. To incorporate this information into TMLE, the propensity score for missing survey data was also required and was built with super learning [36]. For additional details on the TMLE method and assumptions used please see the supplement.

There is the potential that patients who did not turn in their DBS samples had self-report responses that were systematically different than patients who turned in their DBS samples. For example, patients who did not turn in their DBS samples may tend to over-inflate their self-reported adherence or persistence, which could bias results. As a sensitivity analysis, we also analyzed the DBS outcomes without incorporating self-report information into TMLE. DBS outcomes are a more objective measure of adherence and persistence, so this analysis may be considered more reliable but also may lack statistical power due to high levels of missing DBS data.

Reporting and Hypothesis Testing

Point estimates and 95% Wald confidence intervals (CI) were reported for all treatment effects of interest in the study and the corresponding p-value from a two-sided Wald Hypothesis test was also reported. Variance of the estimates was calculated using sample variance of the estimated influence curve, an estimate methodologically similar to the use of “robust standard errors” in standard regression analysis. When analyzing the self-reported PrEP use measures, only point estimates and 95% Wald CIs are reported.

Missing Data

For simplicity, baseline variables with minimal missingness were singly imputed and missing data indicators were included in the covariate adjustment set, an approach that is valid in the context of a randomized trial [37]. The primary and secondary analyses of outcome data (e.g., DBS or self-report outcomes) utilized TMLE methodology, which is specifically built to handle missingness in the outcome of interest. The TMLE approach is valid under the assumption that follow-up data are missing at random, an assumption that stipulates differences between observed and missing outcomes can be explained by measured covariates [38]. The missing at random assumption was deemed plausible, as the primary driver of missing outcomes data was attributable to COVID-19-related study interruptions.

Exploratory Analysis Comparing P3 + to P3

As an exploratory objective, we also estimated the average treatment effect comparing P3 + to P3 at both 3- and 6-months, for both the primary and secondary outcomes of interest. Average treatment effects were estimated using TMLE, as described above.

Software

The analysis was conducted using R version 4.0.5, R version 4.2.0, and Statistical Package for the Social Sciences (SPSS) version 29.

Ethics Statement

The ATN P3 protocol, informed assent/consent documents, and related modifications were reviewed and approved by the University of North Carolina at Chapel Hill Institutional Review Board (IRB, #17-1951). This study is registered at ClinicalTrials.gov (NCT03320512).

Results

Baseline Sample Characteristics

We enrolled 246 participants. Median age was 22 years. Most participants identified as male (87.8%) and White (55.7%) or Black (20.7%). The region with the largest proportion of participants was the South (52.4%) and most were currently employed (74.4%). Table 1 includes baseline sample characteristics for the sample and by treatment group. Most participants (n = 225, 91.5%) were currently on PrEP at baseline.

Among participants currently on PrEP at baseline, baseline adherence as quantified by TFV-DP was similar across study arms (Table 2). Conversely, when assessed by FTC-TP levels, baseline adherence was higher among those in the P3 arm (81.1%) compared to 68.5% in the SOC arm and 72.6% in the P3 + arm. By self-report, a higher percentage of those in the P3 arm reported adherence consistent with ≥ 4 doses per week at baseline (93.4%) compared to those in the SOC or P3 + arm (84.0% and 83.3%), respectively.

Table 2.

Adherence measures, at each time point overall and stratified by treatment arm. Baseline metrics were calculated among participants who were on PrEP at baseline. Metrics at 3- and 6-months are from all participants with available measures.

	SOC			P3			P3+
	BL	3-month	6-month	BL	3-month	6-month	BL	3-month	6-month
Primary
TFV	52/74 (70.3%)	33/50 (66.0%)	31/51 (60.8%)	53/74 (71.6%)	39/50 (78.0%)	31/51 (60.8%)	51/73 (69.9%)	41/53 (77.4%)	34/58 (58.6%)
FTC	50/73 (68.5%)	29/51 (56.9%)	24/48 (50.0%)	60/74 (81.1%)	36/50 (72.0%)	31/50 (62.0%)	53/73 (72.6%)	38/54 (70.4%)	31/56 (55.4%)
Secondary (self-report)
≥4 days, last 7 days	63/75 (84.0%)	51/69 (73.9%)	47/68 (69.1%)	71/76 (93.4%)	58/68 (85.3%)	49/66 (74.2%)	60/72 (83.3%)	59/70 (84.3%)	48/71 (67.6%)
>60%, last month	69/76 (90.8%)	54/70 (77.1%)	52/68 (76.5%)	71 /75 (94.7%)	59/69 (85.5%)	47/66 (71.2%)	66/72 (91.7%)	61/70 (87.1%)	49/71 (69.0%)

Missing DBS outcomes were higher than expected owing in part to COVID-19-based study interruptions (Fig. 1). For example, only 51/83 (61.4%), 51/82 (62.2%), and 54/81 (66.7%) of participants in the SOC, P3, and P3 + arms respectively had DBS available at the month 3 visit.

Fig. 1.

Consort

*Baseline laboratory retention numbers describe the count of participants who were on PrEP at baseline and had laboratory samples available for analysis

Relationships between Self-Reported Adherence to PrEP and DBS Data

We calculated Spearman’s correlation coefficients to examine the relationship between self-reported adherence to PrEP and adherence measured using DBS for TFV-DP and FTC-TP at three different time points: baseline, 3-, and 6-months, among those who had both of these measures (Table 3). The self-reported adherence measures included patient reports of PrEP use in the past week (0–7 days) and in the past month (0–100%), while the DBS adherence measures were represented by binary indicator variables for TFV-DP and FTC-TP, indicating consistent use of ≥ 4 doses per week (0 = No, 1 = Yes). As shown in Table 3, each DBS measure at three time points was significantly correlated with the self-reported adherence measures for each corresponding time point. This suggests a relatively strong agreement between these two methods of assessing PrEP adherence.

Table 3:

Spearman’s correlation coefficients between adherence measures.

Adherence Measures		FTC-TP/TFV-DP at baseline	FTC-TP/TFV-DP at 3 months	FTC-TP/TFV-DP at 6 months
Self Reported Weekly PrEP adherence
Baseline	r	.33/.36
	p	.000/.000
	N	211/212
3 months	r		.40/.45
	p		.000/.000
	N		133/131
6 months	r			.26/.22
	p			.006/.014
	N			111/118
Self Reported Monthly PrEP adherence
Baseline	r	.26/.17
	p	.000/.009
	N	226/228
3 months	r		.36/.46
	p		.000/.000
	N		147/145
6 months	r			.39/.51
	p			.000/.000
				148/156

TFV-DP Treatment Effects

Of participants who provided a blood sample that could be analyzed for TFV-DP at 3 months (n = 153), 66.0%, 78.0% and 77.4% were adherent in the SOC, P3 and P3 + arms respectively (Table 2). Table 4 contains the adjusted estimates for treatment effect when the outcome is measured by TFV-DP levels. The analysis was adjusted for age, whether the participant was on PrEP at baseline, baseline TFV-DP indicator, study site, and race/ethnicity. At 3 months, estimated proportion adherent for P3/P3 + is 0.77 (CI: 0.70, 0.85) versus 0.64 (CI: 0.52, 0.76) in the SOC arm. This difference in proportion adherent between P3/P3 + and SOC of 0.13 is statistically significant (p-value: 0.05). Of those participants who provided a blood sample analyzable for TFV-DP at 6 months (n = 160), adherence was similar (58.6–60.8%) across all three arms. At 6 months, for P3/P3+, the estimated proportion persistent is 0.57 (CI: 0.48, 0.66) versus 0.62 (CI: 0.50, 0.73) for SOC. The difference in proportion persistent between P3/P3 + and SOC is −0.05 (p-value: 0.50) (Fig. 2).

Table 4:

Adjusted estimates for treatment effects as measured by TFV-DP and FTC-TP levels, with the corresponding 95% confidence intervals and p-values.

DBS Measure (TFV-DP)	Estimate	95% Lower Bound	95% Upper Bound	p-value
P3/P3+ vs. SOC
3 month	0.13	0.00	0.27	0.05
6 month	-0.05	-0.19	0.09	0.50
DBS Measure (FTC-TP)	Estimate	95% Lower Bound	95% Upper Bound	p-value
P3/P3+ vs. SOC
3 month	0.12	-0.03	0.26	0.11
6 month	0.07	-0.07	0.22	0.31

Fig. 2.

Bar plots of estimated proportion adherent and persistent according to TFV-DP levels with 95% confidence intervals bands

FTC-TP Treatment Effects

Of participants who provided a blood sample that could be analyzed for FTC-TP at 3 months (n = 155), 56.9%, 72.0% and 70.4% were adherent in the SOC, P3 and P3 + arms respectively (Table 2). Table 4 contains the adjusted estimates for treatment effect when the outcome is measured by FTC-TP levels in the blood. The analysis was adjusted for age, whether the participant was on PrEP at baseline, baseline FTC-TP indicator, study site, and race/ethnicity. At 3 months, proportion adherent for P3/P3 + is estimated to be 0.68 (CI: 0.60, 0.76) as compared to 0.57 (CI: 0.44, 0.69) for SOC. P3/P3 + is associated with an increase in proportion adherent of 0.12 (CI: −0.03, 0.26, p-value: 0.11) compared to SOC. Of those participants who provided a blood sample analyzable for FTC-TP (n = 154) at 6-months, adherence was similar among those in the SOC and P3 + arms (50% and 55.4%, respectively) but higher (62.0%) in the P3 arm. Proportion adherent for P3/P3 + is estimated to be 0.57 (CI: 0.49, 0.66) versus 0.50 (CI: 0.38, 0.62) for SOC. The increase for P3/P3 + compared to SOC in proportion adherent is 0.07 (CI: −0.07, 0.22, p-value: 0.31) (Fig. 3). Treatment effect estimates for FTC-TP and TFV-DP when self-report data was not incorporated into estimation (Supplementary Table 1) are similar to the results from the primary analysis (Table 4).

Fig. 3.

Bar plots of estimated proportion adherent and persistent according to FTC-TP levels with 95% confidence interval bands

Self-Reported Adherence Treatment Effects

The adjusted treatment effect estimates comparing P3/P3 + to SOC, in terms of self-reported PrEP use in the past week ≥ 4 days and percent of time in the last month the participant reported taking PrEP, are displayed in Table 5. At 3 months, the adjusted treatment effect estimate for adherence measured by self-reported PrEP use in the past week was 0.09 (CI: −0.01, 0.20). At 3 months, the adjusted treatment effect estimate for adherence measured by percentage of time in the past month the participant reported taking PrEP was 8.95 (CI: 0.07, 17.83). At 6 months, the adjusted treatment effect estimate for adherence measured by self-reported PrEP use in the past week was − 0.01 (CI: −0.13, 0.12). At 6 months, the adjusted treatment effect estimate for adherence measured by percentage of time in the past month the participant reported taking PrEP was − 3.69 (CI: −13.33, 5.95).

Table 5:

Adjusted treatment effect estimates when adherence/persistence is measured by self-reported PrEP use in the past week (where PrEP adherence/persistence is measured by whether or not the participant reports taking PrEP > 4 days in the past week) and past month (where PrEP use is measured as the percentage of time in the past month the participant reports taking PrEP).

	Estimate	95% Lower Bound	95% Upper Bound
P3/P3+ vs. SOC
Self Reported Weekly PrEP
3 month	0.09	-0.01	0.20
6 month	-0.01	-0.13	0.12
Self Reported Monthly PrEP
3 month	8.95	0.07	17.83
6 month	-3.69	-13.33	5.95

Treatment Effects Comparing P3 + to P3

The adjusted treatment effect estimates comparing P3 + to P3 were generally small, for both the primary and secondary outcome measures (Supplementary Table 2). For example, at 3-months, we estimate a slightly higher adherence proportion under the P3 + intervention with a difference in proportion adherent of 0.02 (95% CI: −0.12, 0.16) when adherence is measured by TFV-DP and 0.03 (95% CI: −0.12, 0.18) when adherence is measured by FTC-TP.

Discussion

In this national sample of YMSM and YTWSM, we found improvements in PrEP adherence at 3-months as assessed by TFV-DP levels among participants randomized to receive the P3 intervention. Adherence as assessed by FTC-TP was also higher among those in the intervention groups compared to control (SOC) at 3-months but was not statistically significant. While no differences in adherence among the groups were noted at 6-months, overall PrEP persistence remained high as compared to prior studies with similar populations [7, 39–41]. Findings suggest that YMSM and YTWSM can adhere to PrEP when warranted and that perhaps, the well-documented phenomenon of young people routinely discontinuing PrEP may be related to factors yet to be elucidated. For example, although not approved by the US Food and Drug Administration, the model of PrEP on demand (also referred to as 2-1-1) holds popularity among young people who may engage in sexual behaviors intermittently, and therefore take PrEP in a way that would be considered as non-adherent as compared to daily dosing. While our findings indicate the value of digital health interventions for HIV prevention with younger people, they also highlight that (1) DBS testing is acceptable to youth, (2) youth can be adherent to daily dosing PrEP and perhaps benefit from the additional social supports that are in-built into a research study, and that (3) the design of the P3 intervention which included elements tailored to the experiences and priorities of youth is promising and should be incorporated into other PrEP smartphone app-based interventions.

To our knowledge, this is one of the first theory-based mobile app interventions to be tested nationally with MSM and TGW youth, a stigmatized, hard-to-reach due to medical distrust, and often underserved population. P3 is one of the few app interventions to show some improvement in PrEP adherence in YMSM and YTWSM, however, given the lack of sustained response, it is not clear if making the extension of this intervention – including the repackaging of the research version of P3 into an easily accessible and widely available app – is warranted. Other mHealth approaches utilizing SMS and gaming have also shown promise, such as PrEPMate and Viral Combat. PrEPMate, a multi-component mHealth intervention that delivers interactive weekly “check-in” messages between patients and PrEP navigators, found that MSM assigned to the intervention were significantly more likely than controls to have TFV-DP levels consistent with ≥ 4 doses/week (OR 2.05, 95% CI 1.06–3.94) [42]. A RCT among 69 YMSM using Viral Combat, an iPhone game, found significant improvements in TFV-DP levels among intervention participants compared to controls [43]. The growing literature affirms the potential for digital health interventions to have a meaningful impact on PrEP adherence among groups disproportionately impacted by structural barriers that impede their ongoing engagement with PrEP care.

While estimates of adherence were similar between the two intervention arms, the P3 + arm did have higher levels of estimated adherence proportions at 3-months. Future analyses that factor in engagement by arm as well as costing measures will be important to parse out whether P3 is as effective as P3+, or nearly as effective. Further, we did note slight differences between outcomes between TFV-DP and FTC-TP, which likely reflects the different half-lives of the metabolites in red blood cells. The half-life of TFV-DP in whole blood is approximately 17 days, thus concentrations reflect the average number of doses taken over the last 6–12 weeks [44], while the half-life of FTC-TP is shorter (approximately 62 h), thus reflecting doses taken over the past week [31]. At both time points, we noted higher protective levels of TFV-DP compared to FTC-TP, which argues against the sample engaging in more recent dosing (e.g., “white coat” adherence).

Another important contribution from this study is that we found strong concurrency between self-report and biologic measures of adherence. While in this study we found alignment in biomarker and self-report data, in ATN studies 110 and 113, YMSM over-reported their PrEP adherence as compared to biomarker data at a difference of over 30% [14, 35]. The P3 intervention included a medication tracker and this may have served to help youth better remember and subsequently report on their actual medication taking, thus decreasing recall bias. Inclusion of medication trackers in new interventions could thus serve as a strategy of promoting and collecting adherence data. A related ATN study examining concurrence between self-report and biomarker alcohol use found similar results to our work, which is that intervention strategies, components within an intervention, can potentially improve the quality of self-report data particularly among younger people [45]. Given both the logistical and financial challenges associated with collection of home-based biologic samples, the concurrency between measures could be used to justify exclusively collecting self-reported outcomes in trials that include intervention strategies for ongoing engagement and medication reminders, or using biospecimen sampling more sparingly.

Future investments in digital health interventions like P3 are needed to ensure their usability in current contexts, including enhancements to support injectable PrEP and 2–1-1 dosing of oral PrEP. Further, considering the well-documented challenges experienced by YMSM and YTWSM, P3 can be expanded to include modules on stigma reduction, mental health, and information on healthy relationships, all topics highly relevant to the emerging adulthood development period.

Limitations

This study should be interpreted considering limitations. First, this study was conducted during the COVID-19 pandemic resulting in more missing biologic data than anticipated. However, the strong agreement between DBS results and self-report in this sample for assessing PrEP adherence provides confidence in the findings. To reduce recall and social desirability biases associated with the self-report measures, we incorporated strategies like using validated questionnaires and ensuring anonymity by using a CASI. Second, this was an intent-to-treat analysis, so it does not account for use of the app in either the P3 or the P3 + arm. An exploratory secondary analysis focused on app engagement is underway for future dissemination. Additional analyses to discern the role of app usage and sociodemographic and behavioral factors (e.g., PrEP stigma, sexual behaviors, substance use) on intervention effects are warranted. Thirdly, hypothesis testing results for the primary outcome measures should be interpreted with caution due to multiple testing. More data is warranted to support or refute the evidence provided by this study.

The generalizability of the study findings is subject to certain limitations due to the specific population and settings involved. The study focused on YMSM and YTWSM, which may limit the applicability of the results to other demographic groups. Additionally, the recruitment methods, which may have relied on specific community networks or digital platforms, could introduce selection bias, further affecting generalizability. Despite these constraints, the findings provide valuable insights into PrEP adherence in these targeted populations, though caution should be exercised when extrapolating the results to broader or different populations. Future research should aim to include a more diverse participant pool and varied settings to enhance the applicability of the findings. Finally, participants in the study were incentivized for app use which may have produced a higher level of engagement. However, use of financial incentives is an established technique in behavioral economics for achieving a desired behavior [46–48]. Healthcare programs reflect “real-world” use of behavioral economic incentives to nudge desired behaviors around tobacco cessation and weight management, suggesting that these kind of initiatives may be cost-saving or cost-neutral at the health system level [49]. Using the modest financial incentive presents a relatively small overall cost ($0 $150 per person over the course of 6 months), however additional costing analyses would be useful to evaluate future interventions.

Conclusion

The P3 intervention was associated with an increase in PrEP adherence among YMSM and YTWSM, suggesting that digital health interventions could be beneficial in reaching and supporting adolescents and young adults on PrEP. Additional studies that elucidate how intervention components individually impact adherence, explore the best time to introduce adherence interventions into treatment plans, and study the longer-term effects of digital health interventions aimed at increasing PrEP adherence are warranted.

Supplementary Material

The online version contains supplementary material available at https://doi.org/10.1007/s10461-024-04547-0.