WYZ: a pilot study protocol for designing and developing a mobile health application for engagement in HIV care and medication adherence in youth and young adults living with HIV

Abstract

Introduction

Youth and young adults bear a disproportionate share of the HIV burden and there is a critical need for interventions to curb health disparities experienced among these age groups. The purpose of our research is to build on our theory-guided model and formative research to develop a mobile health application, called WYZ, for improved engagement in HIV care and antiretroviral therapy adherence, and pilot test it among youth and young adults living with HIV (YLWH). In this paper, we explain the design and development of WYZ for YLWH, describe the design of a forthcoming pilot trial for evaluating the feasibility and acceptability of WYZ and compare WYZ with other mobile health applications being developed to improve engagement in HIV care and antiretroviral medication adherence.

Methods and analysis

We used an agile methodology, shown to be useful in software development, and elicited feedback during beta testing to develop WYZ. WYZ is a modular, adaptive and personalised intervention delivered via a mobile phone. It is grounded in the information, motivation, behaviouralskills model which has been valuable for understanding and guiding the development of interventions for complex health behaviours. WYZ was created in collaboration with YLWH aged 18–29 years using a human-centred design approach that emphasises understanding the perspective of the users of the technology. WYZ is intended to improve engagement in HIV care by: (1) enhancing medication adherence self-efficacy, (2) increasing awareness and use of community resources, (3) reducing barriers to communication between youth and their healthcare team, and (4) providing a secure platform for the formation of a private online community of YLWH. We will conduct a 6-month single-arm pilot study to examine feasibility and acceptability of WYZ among 76 YLWH who live or receive care in the San Francisco Bay Area. All study activities, including recruitment, screening, enrolment, study assessments, provision of incentives and exit interviews, will be conducted remotely. We will explore feasibility and acceptability outcomes of the intervention using quantitative and qualitative methods.

Ethics and dissemination

Study staff will obtain written consent for study participation from all participants. This study and its protocols have been approved by the University of California San Francisco (UCSF) Institutional Review Board. Study staff will work with the UCSF Center for AIDS Prevention Studies’ Community Engagement Core and the Youth Advisory Panel to disseminate results to the participants and the community using presentations, community forums, journal publications and/or social media.

Trial registration number

NCT03587857; Pre-results.

Strengths and limitations of this study.

• The use of the human-centred design and the agile methodology to iteratively and incrementally design and develop the mobile health application, and make adjustments based on the changing requirements dynamically, increases the study’s potential impact and acceptability among participants.

• The intervention is significant because it is grounded in a theoretical framework and it uses tailored approaches to target individual, dyadic and community-level factors to address engagement in HIV care and antiretroviral therapy adherence.

• The use of a mobile health application with youth and young adults living with HIV is innovative, significant and ‘youth-friendly’ because it provides an intervention using tools commonly used by this population and it challenges the current delivery of healthcare.

• The intervention does not support HIV testing and linkage to care.

• The mobile health application was designed and developed with the guidance of youth living with HIV in the San Francisco Bay Area; therefore, findings may not be generalisable to youth in other geographic locations.

Background

Globally, youth and young adults are recognised as being particularly vulnerable to HIV and bear a disproportionate share of the HIV burden, with HIV being the second leading cause of death.¹ In the USA, youth and young adults living with HIV (YLWH) have the lowest likelihood of linking to care, being retained in care, adhering to antiretroviral therapy (ART) and achieving viral suppression, compared with other age groups.^2–4 One review estimated that among all YLWH in the USA, fewer than 6% are virally suppressed.⁵ The personal and public health consequences of suboptimal adherence among YLWH include poor health outcomes, increased drug resistance, reduced efficacy of existing treatment options, widening health disparities and potential transmission of drug-resistant virus. Efforts to adapt healthcare services for YLWH are critical, lest we risk a future generation of adults who are more susceptible to developing AIDS.

While barriers to engagement in HIV care and ART adherence are generally similar across settings and populations, YLWH are more likely to report barriers such as forgetting, being away from home, change in daily routine, depression and alcohol and substance use compared with other age groups.⁶ Additionally, studies have shown that YLWH face many challenges across the HIV care continuum, and clinical outcomes are worse in this group compared with adults.⁷ The combination of these health disparities and developmental challenges in this transitional period may further hinder YLWH’s ability to maintain optimal ART adherence and sustain virological suppression.^{1 7} The link between younger age and lower ART adherence and higher risk of virological failure has been reported in numerous studies.^8–11 However, there are few studies to inform the design of youth-friendly healthcare services and interventions.¹²

The widespread use of personal mobile phones presents an opportunity to develop novel interventions capable of bridging structural and systemic gaps in service and improving the equity of healthcare delivery. Mobile health (mHealth), the use of mobile technology for health, has the potential to reduce barriers to healthcare access, increase use of healthcare services and improve health outcomes among underserved populations.¹³ Reducing barriers like travel to healthcare settings, mHealth interventions hold the potential to increase accessibility for underserved populations and allow for more efficient use of provider resources.¹⁴ Additionally, technology-based interventions have the capacity to consistently deliver interventions with high fidelity and to complement and integrate with other healthcare services and technologies. As the largest consumers of mobile technology and internet use, youth and young adults hold the greatest potential to benefit from such technology-based interventions.¹⁵ In the USA, among 18–29 year-olds, greater than 77% download mobile applications and over 90% are users of social media.^{16 17}

There is a rapidly growing body of evidence demonstrating efficacy of mHealth interventions among various populations and for a variety of conditions, yet the field is nascent and confronting challenges related to limited generalisability,¹⁸ lack of rigorous long-term evaluations and demonstration projects,¹³ and complexities in evaluating efficacious theory-driven components.¹⁹ The purpose of our research is to build on our theory-guided model and formative research to develop an mHealth application called WYZ for improved engagement in HIV care and ART adherence, and pilot test it among 18–29 year-olds living with HIV. In this paper, we describe our approach to design and develop an mHealth intervention for YLWH, outline the design of a forthcoming trial to evaluate the feasibility and acceptability of this intervention and compare and contrast our mHealth application with other applications (ie, SMART,²⁰ Care4Today,²¹ PositiveLinks,²² Epic Allies,²³ Battle Viro²⁴ and AllyQuest²⁵) also being developed for engagement in HIV care and ART adherence.

Intervention

Design and development

To improve engagement in HIV care and ART adherence is an mHealth intervention designed and developed in collaboration with YLWH to improve ART adherence and engagement in HIV care among this age group. WYZ design began with individual qualitative interviews with healthcare providers²⁶ and YLWH and focus group discussions with YLWH.²⁷ This formative research, in conjunction with the information, motivation, behavioural skills (IMB) model for understanding complex health behaviours, guided the initial WYZ design. The IMB model is based on the premise that relevant information, motivation and behavioural skills interact in dynamic ways that influence an outcome, such as improved retention in care.²⁸ The IMB model was selected to serve as a guiding framework because it has been applied extensively in research regarding ART adherence,^29–32 used successfully with youth^{33 34} and the components were supported by our formative research with YLWH. This conceptual design was translated into visual schematics (ie, wireframes) which were used to build the prototype (ie, WYZ V.1.0). We then used data collected from field testing this prototype to develop a beta version (WYZ V.2.0) for pilot testing.

Patient and public involvement

The iterative method used for the development of WYZ is commonly known as human-centred design (HCD).^{35 36} HCD focuses on creating approaches and delivering solutions to problems based on a concerted effort to understand the specific needs and perspectives of the users. In other words, one seeks to gain insight into the needs of the potential beneficiaries of an innovation and creates approaches and delivers solutions to meet their needs. HCD was selected because it allows for iterative development and refinement of a customised and tailored solution, and may increase the likelihood of user engagement with the tool. WYZ design, user testing and ongoing refinement have involved a multistep process of systematic review of the literature³⁷; one-on-one qualitative interviews, focus groups²⁷ and quantitative surveys with YLWH; and qualitative interviews with healthcare providers serving YLWH.²⁶ These formative studies involved a multidisciplinary team consisting of a panel of YLWH, healthcare technology consultants, mobile developers and the research co-investigators.

We additionally used an agile methodology shown to be useful in development of software.³⁸ In the agile methodology, the project uses incremental, iterative cycles of development commonly known as sprints and adjusts to changing requirements dynamically. At the end of each sprint, work completed during the sprint is demonstrated and submitted by the developer to the team for approval. This approach was selected because it complemented our iterative design approach, allowed the team to make immediate alterations to the application and created multiple opportunities to provide feedback throughout the course of development. The combination of HCD and an Agile software development is commonly used in industry and has recently gained traction in government and healthcare environments.³⁹

We will continue to work with our study population to recruit and conduct the pilot study (see the Pilot study protocol section) and disseminate findings through the University of California San Francisco (UCSF) Center for AIDS Prevention Studies’ (CAPS) Community Engagement Core and the Youth Advisory Panel using presentations, community forums, journal publications and/or social media.

Architecture

We chose to build two native mobile applications (ie, iOS and Android) as opposed to one hybrid application due to improved performance and long-term viability. Native applications allow one to take advantage of existing tools available on mobile phones (eg, geolocation, camera and fingerprint log-in) without major development costs. These features allow the application to remain current and relevant in spite of changes in technologies and devices.

WYZ is composed of three primary modules, and each module contains multiple features that work together to target specific needs of the user in different situations. Table 1 provides further details of the features of WYZ.

Table 1.

Description of the WYZ features

Module	Target barriers	Feature	Description
My Health	Busy, forgot, changes to routine, ran out of pills, unstructured lifestyles	My meds	Name of medication, directions for use, refill date and pharmacy contact information Schedule medication alerts and refill reminders
		My adherence	Track, monitor and visualise adherence
		My labs	Track, monitor and visualise CD4+ cell count, and HIV viral load
My Team	Less mobility, travel/distance to clinic, depressions, substance use, mental health, inconsistent access to care	Appointments	Contact for front desk staff*
		Health	Contact for primary care provider*
		Housing and food	Contact for social worker*, listing of local agencies and organisations
		Insurance	Contact for benefits counsellor*
		Jobs and legal services	Listing of local agencies and organisations
		Meds	Contact for pharmacists*
		Mental health and substance use	Contact for counsellor/therapist*, listing of local agencies and organisations
		Wellness and exercise	Listing of local agencies and organisations
My Community	Low health literacy, social isolation, stigma, lack of community support, medical mistrust	Chats	Anonymised and moderated discussion forum
		News	Latest HIV and other relevant medical news
		Calendar	Community calendar: listing of free or low-cost events related to health, housing, food, employment, education, finance and wellness Personal calendar: scheduling appointments with healthcare team or other meetings not wished to be listed in native phone calendar

*In cases where a user does not have access to this service at their clinic, they see a listing of local community organisations that offer these services for free or at low cost.

My Health

This module was conceptualised based on formative research indicating the importance of user’s desires for keeping track of their medication information, visualising their adherence and laboratory data and understanding their health.^{20 23 27} My Health provides medication and refill reminders, medication adherence self-monitoring tool and visualisation of HIV viral load and CD4+ cell count test results. The medication adherence alerts can be personalised by selecting the time, the frequency and the message content. Refill reminders are set based on the last refill date and the quantity received. Users receive refill reminders starting 5 days before their next refill. This feature also provides encouraging guidance based on the user’s self-reported adherence and HIV viral load. For example, if the user has taken 85% of their doses over the past week, they will see a message such as ‘Looking good! Keep it up!’ or if the user has no recent laboratory results, the HIV viral load message will state, ‘No viral load results in the past 6 months. Check-in with your team.’

My Team

The basis of this module was formative research demonstrating that providing resources and facilitation of communication with healthcare team members can improve retention in care.²² My Team facilitates communication between users and healthcare team members and enhances users’ ability to access community organisations and resources. It allows users and clinical team members to connect more efficiently via a Health Insurance Portability and Accountability Act complaint platform, which allows for communication by phone, email or text message without sharing the healthcare team member’s mobile phone number with the user. This feature also provides users with the phone number, email address and website of local agencies that serve YLWH with regard to housing, food, insurance, jobs, legal services, medications, mental health, substance use and wellness.

My Community

This module was created based on formative research showing that social support from peers, connecting with community resources and staying up-to-date on health-related news from the research community are important for YLWH.²⁷ My Community enables users to interact with each other through moderated discussion forums, where they can anonymously share their stories and thoughts, comment on posts, share experiences and make recommendations. It seeks to overcome social isolation by fostering a sense of community among YLWH and providing opportunities for social engagement. These factors have been shown to be important in other similarly relevant domains such as sexual risk behaviours of young men who have sex with men.⁴⁰ A member of the research team will approve chat topics suggested by users and monitor activities for inappropriate content. This module also lists free or low-cost events via a community calendar, and allows one to keep track of personal meetings via a private in-app calendar. Lastly, this module offers access to a curated newsfeed featuring developments in HIV cure, treatment and research; mental health, nutrition, sexual health and substance use.

Security and privacy

We prioritised user privacy and security in the creation of WYZ. We are leveraging a customised Salesforce Dot Com instance to serve as the backend service and storing sensitive data in a secure cloud-based database. To download the application, the user is required to satisfy a two-step authentication process: provision of a password and entry of a security code sent to the user’s registered email. WYZ is password protected and the password is required with each log-in. In case of theft, loss or misuse, study staff can remotely revoke application access. All messages or communications within the app expire after 30 days. All users use an alias and emoji instead of their name when interacting with other users within the application. An independent review conducted by the Information Technology group at the UCSF certified that the application has implemented appropriate protocols and strategies to protect users’ privacy and health information, and mitigate known security risks.

Downloading and provisioning WYZ for new users

The Hockey application, a platform that facilitates the release of application updates, is used to distribute non-publicly available applications to users, who are sent an email from the study staff containing a link for application download. This method of distribution ensures that WYZ is downloaded only on the intended devices. At first log-in, the user is prompted to set a custom passcode which is required to gain access to WYZ. In case a user forgets their passcode, the Salesforce service can be used to securely authenticate the user’s identity via their email address. To verify their identity, users are asked to enter a temporary access code sent to the user’s email.

Field testing and iterative refinement

Field testing is an important component of the iterative HCD approach. We field tested WYZ (V.1.0) with two study team members and eight YLWH who were members of the study advisory panel from October through December 2018. We asked users to log into the application at least once per week and report any bugs and application crashes, as well as suggest modifications to increase engagement. During this time frame, we checked in with users on a weekly basis and, in December 2018, we conducted a focus group with users to further discuss application refinements. We also conducted brief individual qualitative interviews with two primary care providers who provided healthcare for some of the panel members to examine the ideal experience that they would want in interacting with application users. The results of this field test were used to identify potential areas for refinements. Based on these data, an updated version of WYZ (ie, V.2.0) will be created for the forthcoming pilot study.

Pilot study protocol

Study design

We will conduct a 6-month single-arm pilot study to examine WYZ feasibility and acceptability among 76 YLWH (18–29 years old) who live or receive care in the San Francisco Bay Area. The objective of this forthcoming phase is to refine the design and development and pilot test WYZ. In line with evolving guidelines from the National Institutes of Health and in the literature regarding instability of effect sizes from pilot studies and concomitant recommendations for pilot studies to eschew hypothesis testing, preliminary clinical impacts will not be examined.^{41 42} All recruitment and study procedures have been reviewed and approved by the UCSF Institutional Review Board. Here, we describe the pilot trial design of WYZ among 76 YLWH that will be conducted remotely.

Study settings

The study will be conducted in the San Francisco Bay Area. The research team includes a clinical pharmacist, physician, biostatistician, psychologist and a research coordinator from the UCSF CAPS. Technological support and design elements are provided by UCSF School of Medicine Technology Services and the development team consists of developers from Wolox, a technology solutions company. All study activities, including recruitment, screening, enrolment, study assessments, provision of incentives and exit interviews, will be conducted remotely.

Study participants

The potential participants will be recruited from local clinics, community-based organisations and prior research participants who have consented to be contacted for future research. The study sample will consist of 76 individuals aged 18–29 years, living with HIV and who live or receive medical care in the San Francisco Bay Area. Other inclusion criteria include ability to speak and understand English, willingness and ability to provide informed consent and access to an Android or iOS smartphone. Those with evidence of severe cognitive impairment or active psychosis that impedes their ability to provide informed consent will be excluded. Primary care providers, nurses, social workers and pharmacists who provide care to participating individuals will also be asked to participate in study activities.

Sample size justification

The sample size (n=76) was chosen to provide sufficient variability while remaining feasible in scope for a pilot study. We used NCSS PASS V.16 (Kaysville, Utah: NCSS Statistical Software) to compute (A) the distance from the mean to the 95% confidence limit for continuous feasibility and acceptability measures assuming an SD of 1.00, and (B) the exact Clopper-Pearson confidence limits and width from a sample proportion for binary feasibility and acceptability measures. The binary measure scenario further assumed a 70% feasibility/acceptability measure proportion. Assuming 20% anticipated attrition, the estimated distance from the mean to the limit is 0.26. For binary measures, the confidence limits are 0.57 and 0.81, yielding an actual width of 0.24.

Recruitment methods

A combination of recruitment strategies will be used to help achieve the study’s enrolment goals. These include strategies such as posting flyers at clinics and community-based organisations, participant submission of contact information through the study website (wyz.ucsf.edu) and contacting healthcare team members to refer application users. We will also work with a city-wide consortium of youth service providers and the CAPS Youth Advisory Panel to disseminate study information to the target population. Enrolled participants will be encouraged to refer eligible peers and receive US$10 incentive for the referee and the referred individual who participates. Lastly, participants from the principal investigator’s prior studies who had expressed interest and consented to be contacted about future research will be given the opportunity to participate, if still eligible.

Eligibility screening

Interested individuals will be provided a brief overview of the study goals and given the chance to ask questions. Individuals wishing to enrol in the study will be screened for eligibility. To confirm an individual’s age, we will require a photo identification showing their date of birth. To confirm an individual’s HIV serostatus, we will require a clinician’s letter of HIV diagnosis, copy of laboratory test results (for HIV antibody or HIV viral load), or the individual’s ART prescription showing their name. These documents can be sent as a text message to the encrypted and secure study phone for verification by study staff.

Consent and enrolment

To enrol in the study, individuals will review and sign an electronic consent form sent to them via Qualtrics (Provo, UT, USA; version March 2017). All participants will be emailed a copy of the Experimental Subject’s Bill of Rights as mandated by the State of California and a signed copy of their consent. If a participant does not have an email address, the study staff will assist them in signing up for one, as it will be necessary for gaining access to WYZ.

Participant retention

A number of steps will be taken to engage and retain participants throughout the study period. First, participants will be asked for multiple forms of contact information (including emergency contacts, clinical contacts and social media contacts) at the initial visit. They will receive monthly follow-up text messages to confirm that their contact information has not changed. Finally, participants will receive reminder text messages 24 hours before scheduled study activities.

Participant incentives

Participants can receive up to a total of US$215 incentive for completing all study activities. They will receive US$40 for the baseline session, which includes consent, application installation and baseline survey. They will then receive US$10, US$10, US$15, US$15, US$20, US$20 and US$25 for participating in check-ins at weeks 1, 2, 4, 8, 12, 16 and 20, respectively. For the final exit survey at week 24, participants will receive US$40, and those who are invited to participate in the exit qualitative interview at this time will receive an additional US$20. Part of this incentive is aimed at offsetting any data and text messaging fees associated with the application use. We estimate that participants will spend a total of 4–4.5 hours for study activities. Participants will be given a ClinCard, a reloadable debit card, which will be used for the remote provision of study incentives. Finally, we will offer US$50 to each healthcare provider for participating in a one-time qualitative interview.

Outcome measures

The main outcomes of the pilot trial include feasibility and acceptability as indicated by meeting or exceeding the benchmarks presented in table 2. Feasibility will be assessed via user metrics by examining the participant’s interactions with WYZ via a mobile analytics service called Flurry and backend (Salesforce) reporting tools (table 2). Acceptability will be examined during monthly check-ins to examine satisfaction with application use, the system usability scale (SUS),⁴³ a study-developed acceptability survey and an exit qualitative interview. HIV viral load and CD4+ cell count will be collected to determine feasibility of collecting these measures at both baseline and study completion.

Table 2.

Feasibility metrics and threshold for determining feasibility

Measure	Threshold (per user)
General
Recruitment	Recruit at least 55 participants (ie, 70% of target n)
Number of log-ins	Mean of 1 log-in per week
Mean number of minutes in application	Mean of 15 min in application per week
My Health
Use of refill feature	Use of feature once monthly (if receiving 30-day ART supply)
Use of ART adherence tracking	Track meds at least three times per week
Access of laboratory data	Review of laboratory data at least one time per month
My Team
Communications with clinical team member	Mean of one communication per month with clinical team member
My Community
Postchat or response to chats	Mean of one post or response to a post per week
Calendar and news	Minimum access once per week
Administrative
Time for participant onboarding	Mean of 30 min to onboard (including downloading app and reviewing app features)
Time to maintain and support application	<1 hour per participant per week (excludes first visit)
Mean number of app crashes reported	Fewer than one self-reported app crash per week

ART, antiretroviral therapy.

Given that application use engagement is a complex and multifactorial concept, we aim to move beyond simple metrics. Therefore, in addition to feasibility measures detailed in table 2, we will examine an engagement index (EI) that has been detailed in other mHealth interventions.⁴⁴ The EI includes the following subindices: (1) click depth (number of pages a user views per session), (2) loyalty (measures how frequently users access the application during the study period), (3) recency (the time difference between each session the user accessed the application), (4) interaction (number of push notifications opened from those sent through the application), and (5) feedback (subjective measure of participants’ satisfaction with the application).

Data analysis plan

We have established minimum thresholds for determining the feasibility and acceptability of the intervention. Table 2 details these thresholds for the feasibility metrics. For acceptability, SUS scores range from 0 to 100 and scores >68 are considered above average. We will use a threshold of 70% or greater satisfaction to determine acceptability. Means and proportions will be computed at the study endpoint to enable descriptive evaluation of feasibility and acceptability. For EI, we will add all five subindices and examine cut-off points based on the distribution of the total samples’ EI scores and categorise participants as either poorly, moderately or highly engaged.

On completion of the study, up to 20 participants will be invited to complete a one-time, audio-recorded semistructured qualitative exit interview with study staff by phone to further assess the acceptability of WYZ. We will select participants based on their EI category (ie, poor, moderate or high engagement), and interviews will occur until data saturation or until 20 participants have been interviewed. The interviews will focus on the participant’s (1) attitudes towards exiting features; (2) attitudes and perceptions of privacy and information security; (3) features that they would modify, add or eliminate; and (4) suggested modifications to increase engagement and sustained use of WYZ.

Finally, we will conduct in-depth qualitative interviews with clinical team members whose patients participated in the pilot study. The interview will focus on perceived (1) burden or ease of study procedures; (2) attitudes and beliefs regarding impact of the intervention on users’ engagement in HIV care and ART adherence; (3) sustainability and scalability of the application to other youth in the clinic; (4) challenges of integrating an application into clinical work; and (5) suggested modifications to improve usability. All interviews will be done by phone, digitally recorded, transcribed verbatim and deidentified. Transcriptions will be analysed using thematic and content analysis frameworks. This combination is useful in identifying implicit and explicit ideas within the data while also allowing for quantification of qualitative data.⁴⁵

Discussion

In this paper, we document the development of WYZ, an mHealth application for engagement in HIV care and ART adherence among YLWH, report on a 10-week field test and describe the design of a forthcoming pilot trial for evaluating the feasibility and acceptability of WYZ. This work is part of a larger effort in mHealth scholarship to escalate the rigour of methods used to demonstrate the efficacy of mHealth interventions.

We anticipate that the findings of our pilot study will provide critical data informing whether WYZ is acceptable and feasible among YLWH. We expect that the findings from this study will provide information about the use of mobile application technologies to support the HIV care goals and needs of YLWH. The results of this pilot study will allow us to refine WYZ to create V.3.0 and test its efficacy in a randomised controlled trial among a larger cohort of YLWH.

To date, several technology-based interventions to reach, engage, retain and promote ART adherence among YLWH have been examined. The earlier versions of these interventions focused exclusively on medication management and provided electronic reminders at the time of dosing delivered via text messaging,⁴⁶ the internet (via a computer or mobile phone)^47–51 and social media.^52–54 These investigations suggest technology-based interventions can improve ART medication management with nearly 65% of all mHealth studies published between 2009 and 2012 reporting positive adherence outcomes.¹⁹ While these results are encouraging, comparisons across studies are difficult owing to differences in study design, intervention and outcome measurements,¹⁹ and underscore the need for more standardised ways of evaluating mHealth interventions.¹⁴

There are a growing number of application-based interventions described in the literature that address engagement in HIV care and ART adherence.^20–25 Here we review how our work compares with other mHealth applications (ie, SMART,²⁰ Care4Today,²¹ PositiveLinks,²² Epic Allies,²³ Battle Viro²⁴ and AllyQuest²⁵) that are in various phases of development. In this section, we compare the differences and similarities across various key factors and summarise this information in table 3. Overall limitations of many mHealth applications thus far include limited generalisability of study samples, short study duration (median time=18 months) and small sample sizes (median n=28).

Table 3.

Comparison of mHealth applications for engagement in HIV care

	WYZ	AllyQuest25	Battle Viro24	EPIC Allies23	PositiveLinks22	Care4Today21	SMART20
Phase of development	Pilot study	Pilot study	Formative study	Efficacy testing	Pilot study	Observational-descriptive	Proof of principle
Intervention type	Individual, dyadic and community level	Individual level with some peer-to-peer support	Individual level with some peer-to-peer support	Individual level with some social support	Individual and dyadic level	Individual level	Individual level
Framework	IMB model	Social cognitive theory, narrative communication and persuasive technology	IMB model	IMB model	Grounded in past evidence, no specific theory implicated	Technology acceptance model and theory of planned behaviour	Common sense model of self-regulation
Study design	Single-arm trial	Single-arm trial	Cross sectional	Randomised controlled trial	Single-arm trial	Prospective cohort	Randomised controlled trial
Methods	Survey; mobile analytics; chart review; interviews	Survey; mobile analytics; interviews	Survey; interviews	Survey; chart review; mobile analytics; interviews	Survey; mobile analytics; chart review	Focus groups	Survey; mobile analytics; chart review
Outcome measures	Feasibility; acceptability; usability	Feasibility; acceptability; usability	Feasibility; acceptability; usability	Efficacy; acceptability; perceived impact; sustainability	Acceptability; feasibility; visit attendance; viral suppression	Feasibility; acceptability; perceived utility	Adherence; viral load; HIV and ART attitudes; perceived utility
Sample size	n=76 YLWH n=10 HCP	n=20	n=20	n=146	n=77	n=23	n=28
Follow-up	6 months	4 weeks	45–60 min	26 weeks	12 months	3 months	3 months
Population of interest	YLWH (18–29 years old)	MSM (16–24 years old)	YLWH (14–26 years old)	MSM and TWSM (16–24 years old)	Adults living with HIV with high-acuity care needs (≥18 years old)	Adults living with HIV (≥18 years old)	Adults living with HIV (≥18 years old)
Study location	USA	USA	USA	USA	USA	USA	New Zealand
HIV care targets	Engagement in care and viral suppression	Viral suppression	Viral suppression	Engagement in care and viral suppression	Linkage to care, engagement in care and viral suppression	Viral suppression	Viral suppression
Intervention delivery	Mobile app	Mobile app	Prototype+compatible pill monitoring organiser	Mobile app	Mobile phone+cell service+mobile app	Mobile app	Mobile app
Platform accessibility	Android and iOS	iOS	iOS	Android and iOS	Android	Android and iOS	Android

ART, antiretroviral therapy; HCP, healthcare provider; IMB, information, motivation, behavioural skills; MSM, men who have sex with men; TWSM, transgender women who have sex with men; YLWH, youth and young adults living with HIV.

Approximately half of these mHealth studies tracked and reported usage data to demonstrate the level of application engagement. Epic Allies²³ proposed to track application usage data (ie, number of log-ins, use of application features and application progress) as a proxy for engagement. AllyQuest²⁵ and PositiveLinks²² both tracked and reported usage data; however, neither specified thresholds that would be used to determine whether application use was feasible and acceptable. In the evaluation of WYZ, we propose to measure and report on application engagement using a combination of metrics and thresholds including feasibility (table 2) and acceptability metrics, as well as a novel EI used in prior mHealth application development.⁴⁴

In addition to engagement with application use, user retention is another major challenge for mHealth applications. Among the prior mHealth applications addressing engagement in HIV care and ART adherence, only two studies reported retention rates. AllyQuest²⁵ noted that application use declined over the 4-week evaluation period reaching an average 2.8 days of use out of 7 days by the fourth week. PositiveLinks²² also reported a sharp decline in application use over time with approximately 40% of its users using the application after a 12-month period. These rates are comparable to rates seen in commercially available applications. Average mobile applications report retention rates of approximately 30% after 90 days, with approximately 21% of users abandoning the application after a single use.⁵⁵

Advances in technology and the proliferation of mobile phones have the potential to redefine current methods for the delivery of healthcare and help resolve many healthcare challenges. Engaging youth and young adults in their own healthcare is one such challenge. This is more pronounced for YLWH as evidenced by the disproportionate rate of suboptimal health outcomes and health disparities among this age group.^{1 7} Yet, limited evidence-based guidance is available to support the need for youth-friendly healthcare systems and services.²⁶ As such, there is a critical need for research to inform the development and evaluation of interventions and services that respond to the unique and complex psychosocial and physical health needs of HIV-related outcomes in this age group.

This study has limitations that are important to note. The intervention does not support HIV testing and linkage to care. The application was built to address needs that arise after a user forms a relationship with a clinic or healthcare team. We aim to explore future opportunities for parallel work focused on ways to improve testing and linkage to care. Additionally, we are developing WYZ with YLWH in the San Francisco Bay Area; therefore, our findings may not be generalisable to youth in other geographic locations. We are mainly interested in the feasibility and acceptability of WYZ; therefore, we will not be randomising participants nor testing clinical impact in this pilot study. A subsequent formal randomised controlled trial will incorporate refinements to WYZ based on the findings from this pilot study and will evaluate clinical outcomes in a larger sample of YLWH.

The disparate rate of HIV infection and HIV-associated morbidity and mortality among YLWH, in conjunction with the exponential growth of mobile phone use in this age group, suggests potential benefits of using mobile phones as a treatment delivery and/or support system to overcome persistent health disparities. As noted above, we plan to test the efficacy of WYZ in a subsequent large-scale randomised trial. If found to be feasible and acceptable, WYZ may be one of the first mHealth applications using individual, dyadic and community-level approaches for improved engagement in HIV care and ART adherence, grounded in a theoretical framework and created using an HCD approach.