Supplemental digital content is available in the text.
In a study on preexposure prophylaxis in Amsterdam, a mobile app for reporting adherence and sexual behavior was used frequently, especially by participants on daily preexposure prophylaxis or recently started.
Abstract
Background
We studied the use of a mobile application (app) to measure human immunodeficiency virus (HIV) preexposure prophylaxis (PrEP) adherence and sexual behavior, assessed determinants of app use, and we compared data in app and questionnaires.
Methods
Men who have sex with men participating in the Amsterdam PrEP project (AMPrEP) on daily or event-driven PrEP at the Public Health Service of Amsterdam completed the data on sexual risk behavior and PrEP adherence through a standard questionnaire every 3 months and on a daily basis using the project's app. Regression analyses were used to assess factors associated with app use. Among those who reported 90% or greater of data in the app, the number of PrEP pills taken and number of unknown casual sex partners were compared between the app and the questionnaires by Wilcoxon signed-rank test.
Results
Of all participants (n = 374), 94% (352 of 374) reported data in the app at least once; 72% (261 of 362) reported data ≥90% of the days in the sixth month and 62% (222 of 359) in the 12th month following PrEP initiation.
Factors associated with reporting data in the app were using daily PrEP and recent initiation of PrEP. The reported numbers of pills taken and unknown sexual partners were comparable between app and questionnaires.
Conclusions
The AMPrEP app was used frequently, especially by those using a daily PrEP regimen. Data collected by app regarding adherence and sexual risk behavior were consistent with questionnaire data among those who used the app consistently. An app is a promising tool to measure PrEP adherence and sexual risk behavior.
The use of oral tenofovir disoproxil fumarate combined with emtricitabine as preexposure prophylaxis (PrEP) effectively protects men who have sex with men (MSM) against human immunodeficiency virus (HIV) infection.1 Both daily and event-driven PrEP are highly effective.1,2 Adherence is the strongest determinant of effectiveness.1 For daily PrEP, pill counts and drug level measurements are used to measure adherence, but event-driven use requires more advanced monitoring addressing correspondence of PrEP use and sexual activity.
The collection of data using electronic tools is an accepted strategy in many areas of health research and is reported to improve speed and scalability of data collection while reducing costs.3 Computer-based technology may be an acceptable and effective way to deliver HIV (prevention) interventions4,5 and support adherence.6 Several commercially available mobile applications (apps) allow tracking of sexual activity.7 However, studies on the feasibility of collecting data on sexual behavior and PrEP adherence via an app are lacking, and the validity of those data has not been established. The aim of this study was to investigate the use of an app to gather data on PrEP adherence and sexual risk behavior, to determine factors associated with reporting data in the app, and to compare sexual behavior and PrEP adherence data collected with this app with data from questionnaires.
MATERIALS AND METHODS
Study Design Amsterdam PrEP Project
The Amsterdam PrEP project (AMPrEP) is an open-label demonstration project conducted by the Public Health Service of Amsterdam which investigates the uptake, acceptability and feasibility of PrEP for HIV prevention, including adherence and sexual behavior. Study procedures were published previously.8 In short, enrolment took place at the Public Health Service of Amsterdam between August 17, 2015, and June 24, 2016. Men who have sex with men and transgender people were eligible if they were at least 18 years old, HIV negative, at higher risk of acquiring HIV, and had no contraindications for PrEP. Participants made a choice between daily (one pill every day) or event-driven PrEP (2 pills between 24 and 2 hours before, and 1 pill every 24 hours until 48 hours after the last sexual encounter). Switching was allowed at each study visit. At 3-months study visits, participants completed a questionnaire.
Informed consent was obtained from all participants. Ethical approval for AMPrEP was obtained from the Medical Research Ethics Committee of the Amsterdam Medical Center (NL49504.018.14). The study protocol is registered at the Dutch Trial Register.9
Measurement Instruments
The AMPrEP App
The app was developed as a tool to collect health information and as a reminder service to take PrEP for participants of the AMPrEP project. The app was designed for Android and iOS. At the PrEP initiation visit, participants were requested to daily report PrEP intake and sexual behavior in the app during study participation. Access to the app was restricted to study participants through the use of a personal registration code and data were saved at a protected server of our institution using a unique study identifier. Each participant was asked to complete 2 to 8 daily questions: (1) “Did you take the pill today?”, to be answered by “yes” or “no”; and: (2) “Did you have anal sex today?”, to be answered by “yes” or “no”; and if “yes”: (3) “With an unknown partner?”; (4) “With a known partner?”; and (5) “With a steady partner?”, each to be answered by “yes” or “no.” If “yes” to questions 3, 4 or 5, the number of partners and condom use per category was asked.
A randomized controlled trial (RCT) on adherence support for daily PrEP users through an extended version of the app, is embedded in the AMPrEP project and is registered in the Dutch Trial Register, number NTR5741. The extended app provided a visualization of self-reported pill use and sexual activity because use of the app Interfaces of the standard and the extended app are shown in Supplementary Figures 1 and 2, http://links.lww.com/OLQ/A367, http://links.lww.com/OLQ/A368.
Three Monthly Questionnaires
Participants completed a self-administered computer-assisted questionnaires about sexual risk behavior and adherence every 3 months, which contained questions on the same topics as the app but formulated differently. Questions pertinent to this analysis were: “How many pills did you take in the past 30 days?” and “How many unknown casual partners have you had anal sex with in the past 3 months?”
Outcomes
Baseline Characteristics
Characteristics that were recorded were: chosen PrEP strategy, age, ethnicity, residence, sexual preference, educational level, employment, monthly net income, current steady relationship, living situation, sexual risk behavior, alcohol use and drug use. To assess presence of alcohol and drug use disorders, the Alcohol Use Disorders Identification Test (AUDIT)10 and Drug Use Disorders Identification Test (DUDIT)11 were used. The AUDIT and DUDIT are validated questionnaires with, respectively, 10 and 11 items and with resulting scores ranging from 0 to 40 and from 0 to 44. An AUDIT score of 8 or more suggests harmful alcohol use. A DUDIT score of 8 or more suggests harmful drug use.
Use of the App Over Time
Data of the first 12 study months of participation were used. Days in the dataset were converted into 30-day periods (“study months”). The first study month started at the PrEP initiation visit.
Using the app was defined as reporting data on at least 1 record (day). The proportion of participants who reported data into the app over time was calculated per study month. The proportions were categorized as 0 days (0%), 1 to 14 days (>0% to < 50%), 15 to 26 days (≥50% to < 90%), and 27 to 30 days (≥90%) per study month.
Sexual Behavior and PrEP Adherence
To compare questionnaire and app data, we assessed the reported number of unknown casual partners. Regarding adherence, participants could indicate whether they had taken a pill on a particular day, but not the number of pills. In the analysis, we assumed 2 pills were taken whenever an event-driven PrEP user started a PrEP course. Inadvertently, pills taken reported in the questionnaire at the 12-month visit for daily PrEP users were not registered.
Statistical Analysis
Transgender people were excluded from analysis because numbers were small (n = 2). Baseline characteristics are presented as means and standard deviation for normally distributed continuous data and total number with percentage for categorical data. Differences in baseline characteristics between daily and event-driven PrEP users were compared with 2-sided t tests and χ2 tests.
The first time participants stopped reporting data in the app for a prolonged period of time was analyzed with a Kaplan-Meier curve. Time was presented as study months and an event was defined as not reporting data in the app during at least 30 consecutive days.
To determine factors associated with reporting data on a given day in the app, Poisson regression using generalized estimating equations was done, because each participant provided multiple data points (up to 360 days), resulting in a ratio of count means. Factors associated with a P value of <0.2 in univariable regression were included in multivariable analysis; a stepwise backward procedure was used to select significant determinants of daily reporting in the app. For participants who switched PrEP strategy, the strategy used at the day of reporting data in the app was used for Poisson regression. For this analysis, data from those randomized to the intervention arm of the adherence support RCT were censored at enrolment in the RCT.
We conducted a negative binomial regression to determine characteristics associated with being a consistent app user. Participants who completed ≥90% of the days in each month for the first 6 study months were defined as consistent app users. Factors associated at P < 0.2 in univariable regression were included in multivariable analysis; through a stepwise backward procedure we obtained a parsimonious model with adjusted prevalence ratios.
Wilcoxon signed-rank tests were used to compare the distributions of the number of pills taken and of casual sexual partners among consistent app users, gathered through app or questionnaire. Wilcoxon rank-sum tests were used to compare the distributions of the number of pills taken according to the questionnaire between consistent and inconsistent app users. Consistent app users reported data in the app ≥90% of the days preceding the questionnaire (ie, ≥27 days for pill count and ≥81 days for sexual partner count).
Data collected from PrEP initiation until November 28, 2017, were used. Analyses were done with STATA 13.1 (STATA Corporation, College Station, TX).12 The significance threshold was set at P less than 0.05.
RESULTS
Study Population
All 374 MSM participating in the AMPrEP project were included in this analysis. Three hundred fifty-six (92.5%) of these completed 12 months of follow-up; 18 (7.5%) had stopped participation or were lost-to-follow-up. Baseline characteristics of the full cohort were published previously8; characteristics of the population used for this study are presented in Supplementary Table 1, http://links.lww.com/OLQ/A369. Mean age was 40.6 years (standard deviation [SD], 11.6). Most participants were white (85.1%) and living in Amsterdam (61.2%). The majority was highly educated (76.6%) and employed (78.4%). Over one third (40.4%) had a monthly net income between €1701 and €2950. At PrEP initiation, 272 participants opted for daily and 102 participants for event-driven PrEP. Daily PrEP users were significantly younger than event-driven PrEP users (mean age, 39.2 vs. 44.2). Compared with event-driven PrEP users, daily users were less likely to be employed and more likely to be a student; a smaller proportion of daily users lived alone and a larger proportion lived together with other people.
AMPrEP App Use Over Time
Figure 1 shows the proportions of days on which data were reported in the app per study month. The percentage of participants reporting data for 27 to 30 days (≥90%) per study month decreased over time (P < 0.001). The percentage of participants reporting data for zero days per study month increased. However, over 60% of the participants reported data in the app for 27 to 30 days (≥90%) in the twelfth study month.
Figure 1.
AMPrEP app use over time among MSM participating in AMPrEP, Amsterdam, 2015–2017: the proportions of days on which data were reported in the app, per study month.
A Kaplan-Meier curve of app use over time is depicted in Figure 2. All participants (n = 374) were at risk of not reporting data in the app for 30 consecutive days. In total 10.2% of participants did not use the app in their first month in AMPrEP. The use of the app decreased gradually during time spent in the study. After 12 months, 37.7% had a 30-day period of non–app use.
Figure 2.
Kaplan-Meier curve of time to 30 consecutive days of non-use of app among MSM participating in AMPrEP, Amsterdam, 2015–2017.
Characteristics Associated With Reporting Data in the AMPrEP App
Table 1 shows the variables associated with reporting data in the app. In univariable analysis, earlier study month, higher income, with a possible threshold effect at 1700 euro, and potentially harmful drug use were significantly associated with reporting data in the app on a given day.
TABLE 1.
Univariable and Multivariable Poisson Regression Analyses Using GEE to Determine Factors Associated With Reporting Data per Day in the App for MSM Participating in AMPrEP, Amsterdam 2015–2017
In multivariable analysis, earlier study month and event-driven PrEP strategy were significantly associated with less frequent reporting of data in the app (adjusted ratio [aR] of count means, 0.98 [95% confidence interval (CI), 0.97–0.99] per additional month and aR for using daily PrEP aR 0.86 [95% CI, 0.74–1.00]).
In sensitivity analysis, using daily PrEP and having a middle or high income were significantly associated with being a consistent app user (Supplementary Table 2, http://links.lww.com/OLQ/A370).
Comparison of App Data to Questionnaire Data
In the daily PrEP group, PrEP adherence by app data and questionnaire data was comparable: the median of the monthly number of pills taken was 30 at 3, 6, and 9 months (Table 2). Among event-driven PrEP users, the median number of pills taken according to the app tended to be lower than the number reported in the questionnaire; this difference was only significant in the ninth month (P = 0.009).
TABLE 2.
Comparison of Self-Reported Number of Pills Taken and Self-Reported Number of Unknown Casual Partners by MSM With Recent Consistent App-Use in AMPrEP, Between App and Questionnaire, Amsterdam 2015–2017
Sensitivity analysis showed that the reported number of pills taken (questionnaire) was similar for consistent and inconsistent app users; nevertheless the difference was significant among daily users (Supplementary Tables 3 and 4, http://links.lww.com/OLQ/A371).
The number of unknown casual partners reported by both daily and event-driven PrEP users was comparable between the app and questionnaire assessment, with the exception of the twelfth month among event-driven PrEP users, where the number reported in the app was significantly higher (P = 0.016).
DISCUSSION
In this study among MSM in the Netherlands, the use of an app to collect daily data on adherence of PrEP and sexual behavior among MSM was high in the first year. Factors associated with reporting data in the app were study month and having chosen a daily PrEP strategy. Among those who reported recent data in the app, app data on pill intake and number of unknown casual sex partners were comparable to data collected with questionnaires.
We observed a decrease in the number of days that participants contributed data to the app over time. In the first study month, approximately 75% of study participants reported data in the app on 90% or greater of days; this decreased to 62% in the 12th month. This decrease over time is in line with other studies that investigated the use of apps to improve health containing daily diaries.13,14 Similarly, we found an association of being in the study for a shorter period of time with reporting data in the app. In addition, daily PrEP was associated with completing daily app data. These factors are non-modifiable and thus should be accounted for when designing an intervention, for example, regular addition of new features to a daily data collection app.
Adherence data collected with the app were similar to those collected with questionnaires for daily and event-driven PrEP users for those using the app consistently. However, in the ninth month the data were variant for event-driven PrEP users. There are 3 possible explanations. First, when adherence among daily users is maximal, 30 pills taken in 30 days, this is relatively easy to remember. For event-driven use, it may be harder to remember how many pills were taken, and hence a small variation between daily and quarterly data collection may have occurred. Second, participants who chose a daily strategy may be different from event-driven users in a way that causes them to be more consistent in data reporting. In the IPERGAY study,2 where adherence was measured through pill counts, event-driven PrEP users took a median number of 15 pills per month, which was comparable to their questionnaire data. Measurement of adherence to PrEP is challenging, especially for event-driven PrEP use. No gold standard exists and it is difficult to assess which method best approximates the truth.15 An additional benefit of daily data collection is that PrEP coverage of sex acts can be assessed. Third, this could be caused by a type 1 error, due to the number of comparisons being made. However, observed changes between app and questionnaire data were small, indicating that app data collection on pill use may replace questionnaire data collection.
Regarding sexual behavior, a gold standard for data collection is not available. We showed that data on sexual behavior collected daily via an app were comparable to aggregated quarterly data collected by questionnaire among consistent app users. This is reassuring in 2 ways: it suggests apps may be used for daily data collection on sexual behavior, and that app-based data collection is reliable, as it is comparable to the most common way of eliciting sexual behavior data, by questionnaires.
Strengths
This study has several strengths. This is the first study describing the use of an app to collect data related to PrEP adherence and sexual behavior over time. We showed reliability of data when collected daily via an app as compared to questionnaire data collected every 3 months. As e-health technology is only just starting to take off and many health apps may be developed in the coming years,16 this proof of feasibility of an app to collect reliable adherence and sexual behavior data is very helpful for researchers and mobile technology developers.
Furthermore, we collected data on PrEP adherence related to sexual exposure, which may allow for accurate assessment of coverage of sex acts and accurate adherence. Event-driven PrEP is not used widely, although a varying proportion of PrEP users is interested in this regimen in several settings.2,8 Information on how event-driven PrEP is used is needed. Because individualized prevention is preferable over a blanket strategy, data from this study may aid to effectively inform and support those who want to use PrEP, resulting in optimal HIV transmission prevention.
Limitations
This study has several limitations. First, highly educated people were over-represented in our study and this group may adopt innovative health technology more easily than those with less education.17,18 Transgender people were excluded as their number was too small to derive meaningful conclusions. Whether an app designed to collect data on adherence and sexual behavior will be used by the wider MSM and transgender community needs to be investigated in future PrEP roll-out studies.
Second, for the comparison of app data to questionnaire data, only participants were included who reported data in the app 90% or greater of the 30 days before the questionnaire for pill count and 90% or greater of the 90 days before the questionnaire for unknown casual partners. The conclusion of comparability between app data and questionnaire data may thus only apply to participants who provided data in the app on 90% or greater of days. More data are also needed on app use in event-driven PrEP users, because sample size in this study was relatively small. Additionally, the app did not track the number of doses of PrEP event-driven PrEP users took.
Third, we did not investigate the reasons for not entering data in the app; qualitative research may guide designing interventions for app engagement in the long haul.
A final limitation is that participants may have used the app when completing the standard questionnaire to help them remember the number of pills they took and casual partners they had.
The AMPrEP app can successfully monitor PrEP adherence and collect data on sexual behavior. App use was high with a decrease over time. The majority of the data collected with the app were similar to questionnaire data among those using the app consistently. The use of an app aimed at collecting research data and reminding people to use PrEP is a very promising tool to assist men in being adherent and to collect detailed data on sensitive topics.
Footnotes
Acknowledgements: The authors wish to thank the participants, and the following persons for their invaluable support to this study: Gerben Rienk Visser, Linda May, Anders Boyd, Kees de Jong, Ilya Peters, Myra van Leeuwen, Princella Felipe, and the teams of the STI outpatient clinic and infectious diseases, research and prevention department of GGD Amsterdam. Furthermore, we wish to thank the members of the advisory board and the community engagement group of AMPrEP, and all of those who contributed to the H-team (annex 1).
E.H., R.A., M.P., H.dV., M.S.vdL., and U.D. participated in the study concept and design. R.F., E.H., R.A., M.S.vdL., and E.M. participated in the acquisition, analysis or interpretation of the data. R.F., E.H. participated in the drafting of the article. E.H., M.S.vdL., E.M. participated in the critical revision of the article. R.F., E.M., and M.S.vd.L. participated in the statistical analysis. E.H., M.P., M.S.vdL. obtained funding. All authors read and approved the final version.
Sources of Funding: The AMPrEP project in the H-team initiative received funding from ZonMw (grant 522002003), the National Institute for Public Health and the Environment and GGD research funds. The study drug is provided by Gilead Sciences. The H-TEAM initiative is being supported by Aidsfonds Netherlands (grant 2013169), Amsterdam Dinner Foundation, Gilead Sciences Europe Ltd (grant number: PA-HIV-PREP-16-0024), Gilead Sciences (protocol numbers: CO-NL-276-4222, CO-US-276-1712), Janssen Pharmaceuticals (reference number: PHNL/JAN/0714/0005b/1912fde), M.A.C AIDS Fund, and ViiV Healthcare (PO numbers: 3000268822, 3000747780).
Potential conflicts of interest: We received the study drug for the Amsterdam PrEP study from Gilead Sciences. EH: my institute received financial reimbursement for time I spent serving on advisory boards of Gilead Sciences, and a speaker fee from Janssen-Cilag.
Supplemental digital content is available for this article. Direct URL citations appear in the printed text, and links to the digital files are provided in the HTML text of this article on the journal’s Web site (http://www.stdjournal.com).
Contributor Information
Collaborators: on behalf of the Amsterdam PrEP Project team in the HIV Transmission Elimination AMsterdam Initiative (H-TEAM)
REFERENCES
- 1.Fonner VA, Dalglish SL, Kennedy CE, et al. Effectiveness and safety of oral HIV preexposure prophylaxis for all populations. AIDS 2016; 30:1973–1983. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Molina JM, Capitant C, Spire B, et al. On-demand preexposure prophylaxis in men at high risk for HIV-1 infection. N Engl J Med 2015; 373:2237–2246. [DOI] [PubMed] [Google Scholar]
- 3.Marcano Belisario JS, Huckvale K, Saje A, et al. Comparison of self administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database Syst Rev 2014; 2014:1–115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Noar SM, Black HG, Pierce LB. Efficacy of computer technology-based HIV prevention interventions: A meta-analysis. AIDS 2009; 23:107–115. [DOI] [PubMed] [Google Scholar]
- 5.Daher J, Vijh R, Linthwaite B, et al. Do digital innovations for HIV and sexually transmitted infections work? Results from a systematic review (1996–2017). BMJ Open 2017; 7:e017604. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Muessig KE, LeGrand S, Horvath KJ, et al. Recent mobile health interventions to support medication adherence among HIV-positive MSM. Curr Opin HIV AIDS 2017; 12:432–441. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Stalgaitis C, Glick SN. The use of web-based diaries in sexual risk behaviour research: A systematic review. Sex Transm Infect 2014; 90:374–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hoornenborg E, Achterbergh RC, van der Loeff MFS, et al. Men who have sex with men more often chose daily than event-driven use of pre-exposure prophylaxis: Baseline analysis of a demonstration study in Amsterdam. J Int AIDS Soc 2018; 21:e25105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Nederlands Trial Register. Trial Info AMPrEP (Amsterdam PrEP project). Available at: http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=5411. Accessed on October 15, 2018.
- 10.Babor TF, de la Fuente JR, Saunders J, et al. The alcohol use disorders identification test: Guidelines for use in primary health care. Geneva: World Health Organization. Gen Dent, 1992. [Google Scholar]
- 11.Berman AH, Bergman H, Palmstierna T, et al. Evaluation of the Drug Use Disorders Identification Test (DUDIT) in criminal justice and detoxification settings and in a Swedish population sample. Eur Addict Res 2005; 11:22–31. [DOI] [PubMed] [Google Scholar]
- 12.Stata Statistical Software [Computer program]. Version 13.1. College Station, TX: StataCorp, 2013. [Google Scholar]
- 13.Herrmann LK, Kim J. The fitness of apps: A theory-based examination of mobile fitness app usage over 5 months. mHealth 2017; 3:2–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Goh G, Tan NC, Malhotra R, et al. Short-term trajectories of use of a caloric-monitoring mobile phone app among patients with type 2 diabetes mellitus in a primary care setting. J Med Internet Res 2015; 17:e33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Haberer JE, Bangsberg DR, Baeten JM, et al. Defining success with HIV pre-exposure prophylaxis: A prevention-effective adherence paradigm. AIDS 2015; 29:1277–1285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Wyatt JC. How can clinicians, specialty societies and others evaluate and improve the quality of apps for patient use? BMC Med 2018; 16:225. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Rai A, Chen L, Pye J, et al. Understanding determinants of consumer mobile health usage intentions, assimilation, and channel preferences. J Med Internet Res 2013; 15:e149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Or CK, Karsh BT. A systematic review of patient acceptance of consumer health information technology. J Am Med Inform Assoc 2009; 16:550–560. [DOI] [PMC free article] [PubMed] [Google Scholar]