ACCEPTABILITY, FEASIBILITY, AND PRELIMINARY EFFICACY OF A THEORY-BASED RELATIONAL EMBODIED CONVERSATIONAL AGENT MOBILE PHONE INTERVENTION TO PROMOTE HIV MEDICATION ADHERENCE IN YOUNG HIV-POSITIVE AFRICAN AMERICAN MSM

Abstract

An embodied conversational agent can serve as a relational agent and provide information, motivation, and behavioral skills. To evaluate the feasibility, acceptability, and preliminary efficacy of My Personal Health Guide, a theory-based mobile-delivered embodied conversational agent intervention to improve adherence to antiretroviral therapy in young African American men who have sex with men, we conducted this prospective pilot study using a 3-month pre–post design. Outcome measures included adherence, acceptability, feasibility, pre versus post health literacy, and pre versus post self-efficacy. There were 43 participants. Pill count adherence > 80% improved from 62% at baseline to 88% at follow-up (p = .05). The acceptability of the app was high. Feasibility issues identified included loss of usage data from unplanned participant app deletion. Health literacy improved whereas self-efficacy was high at baseline and follow-up. This pilot study of My Personal Health Guide demonstrated acceptability and preliminary efficacy in improving adherence in this important population.

Although avatars and embodied conversational agents are commonly employed in computer games, their application to the promotion of human health is new and actively emerging (Fox, Bailenson, & Tricase, 2013). Embodied conversational agents may appear as a human, an animal, or possibly some other character form and are based on three elements: (1) an interface through which the user can communicate with the agent; (2) a computer model that allows the semblance of mental capability that, e.g., through a decision tree, might lead to logical response or perception of empathy; and (3) a body that may be used to communicate not just through audio but also visually as with facial or body movement (Bickmore, Caruso, Clough-Gorr, & Heeren, 2005; Provoost, Lau, Ruwaard, & Riper, 2017). As a result, they may serve as a customizable relational agent that can educate, motivate healthy behavior change, and augment user engagement by employing multiple modalities such as audio, graphics, animation, and text. A dynamic user experience with a relational agent might encourage repeated use and establish trust and therapeutic alliance that could promote healthy behavior (Bickmore et al., 2010).

Young African American men who have sex with men (AAMSM) are a priority population for intervention to improve antiretroviral therapy (ART) adherence, viral suppression, and retention in care. MSM account for 82% of new HIV diagnoses among men (Centers for Disease Control and Prevention [CDC], 2017b). The largest subgroup within this population are AAMSM. In a national study of HIV-positive MSM reported by the CDC, both young MSM (ages 18–34 years) and AAMSM had the lowest viral suppression and retention in care compared to MSM in other age or racial/ethnic groups (Singh et al., 2010). Although African Americans represent 13% of the US population, they account for more than 50% of deaths from HIV/ AIDS (CDC, 2017a).

There is scientific premise for the use of embodied conversational agents to effectively promote healthy behavior in young African American MSM, although this application to HIV care is new. First, mobile phone interventions for HIV care have shown promise improving ART adherence (Muessig, Pike, Legrand, & Hightow-Weidman, 2013; Ybarra & Bull, 2007). Studies of smartphone ownership in MSM reveal they are higher than the national average and smartphone ownership among African Americans is as common as among whites (72% and 72%, respectively; Community Marketing Inc., 2014; Pew Research Center, 2015, 2017). Since mobile phones are commonly kept with their user most or all of the day and often used for information gathering and entertainment, especially in adolescents and young adults, placing the intervention on a mobile phone makes it more readily accessible than if it were Internet- or clinic-based.

Second, several health studies have demonstrated that an avatar or embodied conversational agent may be acceptable and useful in the health care setting. For example, ethnic minority patients with low health literacy reported greater comprehension of hospital discharge information from an avatar rather than a doctor or nurse (Bickmore, Pfeifer, & Jack, 2009). When they were asked, “How did you feel about a computer character giving you health information?” there was a mean level of comfort of 6.3 on a 1 to 7 scale where 7 was completely comfortable. Patients with lower health literacy reported feeling more cared for by the agent than those with higher health literacy. Since low health literacy is a factor associated with ART adherence and retention in care (Jones, Cook, Rodriguez, & Waldrop-Valverde, 2013; Kalichman, Ramachandran, & Catz, 1999; Kalichman et al., 2008; Kalichman & Rompa, 2000; Osborn, Paasche-Orlow, Davis, & Wolf, 2007; Wolf, Davis, Tilson, Bass, & Parker, 2006), interventions employing an embodied conversational agent like this virtual nurse may be useful to promote healthy HIV-related behaviors. In a study of treatment of chronic pain and depression, participants interacting with an avatar reported 100% compliance with the avatar’s suggestions to reduce stress and 44% preferred the avatar to interacting with a clinician (McCue et al., 2015). Other interventions using avatars or embodied agents have been designed to increase walking and influence other clinical and health issues including posttraumatic stress disorder, addictions, body image disturbance, binge eating disorder, weight loss, and reduction of sexual risk behavior in MSM (Bordnick, Carter, & Traylor, 2011; Bordnick et al., 2004; Gabarron, Serrano, Wynn, & Armayones, 2012; Girard, Turcotte, Bouchard, & Girard, 2009; Napolitano et al., 2013; Ready, Pollack, Rothbaum, & Alarcon, 2006; Riva, Bacchetta, Baruffi, & Molinari, 2001; Riva, Bacchetta, Cesa, Conti, & Molinari, 2003; Rosser et al., 2012).

Third, avatars are highly acceptable to young MSM (LeGrand et al., 2016). An avatar or an embodied conversational agent can serve as a relational agent (promoting a social-emotional relationship with the user) and provide information, motivation, and behavioral skills. AAMSM express a desire for role models who are believable and credible in terms of language, age, and dress, and typically feel that relevant Internet home pages are not culturally sensitive to them (Hightow-Weidman et al., 2011).

We developed a theory-based mobile-delivered, realistic talking human avatar embodied conversational agent intervention (My Personal Health Guide) to address adherence and retention in HIV care with the overarching goal of improving viral suppression. We previously reported development of the app, informed by iterative focus groups with young African American MSM living with HIV (Dworkin et al., 2018). We report here the results of a pilot study exploring acceptability, feasibility, and preliminary efficacy of the effect of My Personal Health Guide on adherence.

METHODS

PARTICIPANTS

Participants were young AAMSM, ages 18–34 years, who self-reported that they were HIV-positive, on ART for at least 3 months, owned an Android smartphone, and had at least one scheduled blood draw and HIV care-related visit during the 3 months after baseline assessment. Individuals who contacted the study were screened for eligibility using these criteria. Recruitment began in January 2017 and continued through December 2017 in Chicago. Recruitment consisted of fliers and word of mouth at the University of Illinois at Chicago (UIC) HIV clinic, four UIC Community Outreach Intervention Project (COIP) sites located in different high HIV-incidence areas of the city, and one adult clinic of the Ruth M. Rothstein CORE Center.

STUDY DESIGN

All procedures were approved by the UIC Institutional Review Board. This study employed a convenience sample of patients who were followed prospectively for 3 months. After informed consent was obtained, a baseline questionnaire was administered that included demographics, literacy (measured with the REALM-SF tool; Agency for Healthcare Research and Quality, 2016), HIV information (baseline knowledge of information taught by the app), self-efficacy, duration of HIV positivity, whether the participants had > 2 routine HIV medical care visits in the past 12 months at least 3 months apart, if they had > 2 CD4 tests in the preceding 12 months, the names of their ART (collected from their pill bottles) and duration of ART, self-reported 4-week adherence (number of doses missed), and contact information. After the survey, a trained project staff member downloaded the mobile phone app from a portable laptop into the participant’s phone (Dworkin et al., 2018). They then demonstrated the app functions, oversaw setting reminder functions, encouraged use of the app, and answered questions. A check-in phone call was made by the project staff twice (the end of months 1 and 2) to troubleshoot for any technical problems. After 3 months, participants returned for a follow-up survey that included repeating the knowledge and self-efficacy questions.

App usage data including the number of days used, mean use per participant, and mean duration of app use was collected by the app and downloaded from phones at the follow-up visit.

INTERVENTION

App development was guided by the Information, Motivation, and Behavioral Skills Model of Adherence to antiretroviral therapy (Fisher, Fisher, Amico, & Harman, 2006). Therefore, the app provided substantial information relevant to adherence to ART such as side effects for each medication. Motivation to adhere was provided by the embodied agent, acting like a social support person by blending motivational statements into educational information and by offering motivational audio-recorded messages (described below) of real stakeholders of HIV care. Behavioral skills for adherence were built in as app functions such as phone calling ability directly to relevant health care personnel (doctor’s office, pharmacy, and case manager), reminder alerts, monitoring adherence, and tracking side effects. Adherence and side effect data, as well as self-entered CD4 count and viral load data, were displayable to allow self or health care provider interpretation and to potentially motivate behavior change (such as after observing a graphically declining CD4 trend).

Privacy features included password-protection, default log out after 5 minutes of nonuse, and tap screen disguisability (changing the screen image). Upon activation, the agent introduces himself as the user’s personal health guide and then provides an audiovisual orientation to the home screen and its available functions. The primary functions are Let Me Explain, Medicine Manager, and Settings.

The Let Me Explain function displays the avatar and a scroll bar of 26 educational questions, many that contain rationales for healthy behavior, such as “Why do I need my blood drawn?” and “What can happen if I get AIDS?” There are also questions that acknowledge concerns voiced in focus groups performed in the developmental phase of this project (Dworkin, Chakraborty, Lee, et al., 2018) and respond with motivation such as “Are you afraid to take your meds?” Many responses to questions include motivation, relation (such as empathy, empowerment, or other dialogue that might elicit an emotional response), and/or interaction where the agent provokes the user to think by asking them a question (Table 1). Ten of the questions include images or simple animation to augment explanations.

TABLE 1.

Examples of Motivation, Relation, and Interaction Within the Let Me Explain Function of My Personal Health Guide.

Question	Motivational statement(s)	Relational statement(s)	Interaction
	(encouragement)	(empathy, empowerment, or other dialogue that might have an emotional response)	(asking the user a question)
How many times a year should I see a health care provider?	“I hope you understand that it’s really important to keep these appointments and to try hard to be on time for these appointments.”	“Some people even say they feel empowered from their appointments, because they don’t have a lot of people in their life they can talk to about having HIV.”
	“Checking things out regularly with your health care provider, you’re going to do well!
Why do I need my blood drawn?	“Do you know when you’re supposed to get your next blood draw? If you don’t, now is a good time to check on that.”*		“Have you heard of ADAP or CHIC? The AIDS Drug Assistance Plan or CHIC Premium assistance.”
What are the blood tests my health care provider is checking on me?	“Take advantage of it. Get your blood tested when they order it for you!”		“Have you ever heard of low blood or anemia?”
What is a CD4 count?		“This is a really good question!”
What’s a viral load?	“So you want the doctor to tell you that your viral load is low.”	“Alright! That’s great you want to know!”	“You don’t want to carry around a big load of this virus, right?”
	“Lower your load, by taking the HIV medication every day.”
	“Taking the medication every day can do this. And remember, lower your viral load by taking your medicine every day.”
	“So take your medication and imagine the virus going away.”
Is AIDS the same thing as HIV?	“If you don’t know if you have AIDS, or just HIV, be sure to ask your health care provider.”	“Listen—just keeping it real—since AIDS can make you suffer, you want to focus on what you can do to prevent it.”
	“HIV and AIDS is not the end of the world. Things get better.”
	“Take your medication every day.”
	“Whether you have HIV or AIDS, taking your medication is still really important for staying healthy.”
What is the benefit to me of taking the HIV medicine?	“Keep control of the HIV virus by taking your medicine and keeping your appointments.”	“Good luck with your personal goals!”
	“This is how you get a full life. And it’s how you are going to get to achieve your personal goals.”	“Enjoy that full life. You’ll get there!”
	“Stay positive. Take your medication so you can make that viral load get undetectable and build up your T cells.”
What happens to my viral load and CD4 count when I take HIV medicine?	“You can control HIV so you get a full life.”	“When your T cells go up—‘cause you took the medicine—that backs the car up, back to the road you want to be on, going to where you want to go with your life, not where HIV would take you.”
	“Now that’s medicine I would want to take every day!”
If my viral load is undetectable, can I have unprotected sex?	“So even if your viral load is undetectably low, it’s still a good idea to use condoms.”	“If your viral load is undetectable, that’s great!”
	“Protect yourself when you have sex.”
Are you afraid to take your meds?	“This medicine can save your life. This medicine is what gets the HIV to stop, and let you be healthy. To treat HIV, you need to take the medication every day for a lifetime.”	“I get that. Yeah, like, people are telling you that you got to take medicine for the rest of your life? And it’s like, can’t these make me SICK?”	“But let’s keep it real, okay?”
	“So don’t be afraid to take your meds. These meds can really save your life.”	“And while thinking about taking HIV medication might make you feel a little afraid think about this too. HIV should be afraid of you! You are the one who’s got the HIV medication that can keep you healthy. You are the one who can stop HIV from reproducing itself. You are the one who really has the ability to control this for your own benefit.”
Can I just stop taking my meds for a while?	“And if the meds make you feel sick, then you just go back to the doctor and ask if you should switch to different meds or are these side effects gonna wear off in time.” “It can get back to its work of destroying things.” “It can be bad to stop taking your meds.” “If you can’t stop thinking about this, go talk to your doctor before you give the virus a chance.”	“If your meds are working, you know, you’re viral load is really low, like undetectable, sometimes you get to thinking, ‘Hey, I can just take a break for a while.’ I get that!”
Can I take an extra dose if I missed my meds yesterday?	“So don’t take an extra dose.”	“I know how it can be. You have things going on in your life. You got stuff to think about. And sometimes it’s hard to take the medication. You just lay down, and you’re like, ‘I’ll take it in a little while.’ And then you fall asleep. Or you’re going out and you don’t bring the medication with you. When this happens, it’s better to take it later that same day, than to completely miss it.”
	“If you are having trouble taking your medicine, talk to your health care provider or pharmacist to see if they have ideas that can help.”
Can I take my medicine with alcohol?	“Try not to take your pills with alcohol but definitely don’t miss taking your pills. If you drink, drink moderately. And talk to your health care provider if you need help cutting back.”	“You know, this is one of those questions you should really ask your pharmacist.”
What do I do if I have side effects after taking the medicine for a while?	“Don’t just stop taking your medicine and avoid the doctor.”
	“Be your own advocate.”
	“Tell your health care provider what’s going on and ask them to help you.”
	“Don’t be afraid to tell them something’s wrong and you want help.”
Why are my medicines different than other people who are HIV-positive?	“If you’re not sure if your medicines are the best ones for you, talk to your health care provider.”	“That’s a good question.”
	“Ask them if your medicines are still a good choice or is there any reason to think about any changes.”
What are some things I can do if I don’t take my medicine every day?	“If you don’t know how to set a phone alarm, someone you know or your health care provider or case manager might be able to help you.”		“Do you have someone in your life you trust? Like your sister, your brother, your mother or father, a lover, or a good friend?”
	“If you are struggling with taking your medication every day, don’t just accept it.”
	“Do something about it. It’s your life!”
What if I can’t understand everything my health care provider says to me?	“Also, I want to encourage you to ask questions when you don’t understand.”	“You’re not the only one with this problem. Sometimes the health care provider talks too fast, right? Or says things with words not everybody knows what they mean. And it’s kind of uncomfortable to ask them to explain it. And maybe they seem to be in a hurry too.”	“Do you have someone you could bring to an appointment? Someone you trust, who’s on your side? Like a sister, a brother, a trusted friend, a parent?”
	“You deserve to understand everything you’re told. You really do! Just say, ‘Can you explain that a different way?’ Or, ‘Can you write that down so I can take a look at it after we’re done?’ But do make sure you do something to get all the information you need. You’re worth the extra time and energy.”
What if I need to find someone for my mental health?	“You can ask them to refer you. If you don’t, but you live in the Chicago area, try calling the AIDS Foundation of Chicago and ask for help getting a case manager. Their number is 312 922 2322. If things are really bad for you, seek help at an emergency room.”	“If you need to find a mental health referral, first of all, that’s good that you’re going to check this out!”	“Do you have a case manager or a health care provider?”
How do I know if I might have syphilis?	“There’s treatment for syphilis so it’s important for you to know if you have it so you can get treated.”		“You do hear me, right? You don’t have to have those nasty symptoms to have syphilis.”
	“Check with your health care provider to ask them, ‘Are you checking me for syphilis?’”
How do you get syphilis?	“It’s really important to use a condom during sex, including oral sex, to help protect you from this disease.”
What is thrush?	“If you think you might have thrush, see your health care provider. They do have treatment for it. And be sure to take your medicine every day.”
	“That’s the best way to fight HIV and keep thrush away.”
What can happen if I get AIDS?	“This is why I keep emphasizing taking your medication and keeping your appointments. Because when you control the virus, it gives you a lot of protection from AIDS.”
	“Talk with your health care provider to learn more.”
What else can happen if I get AIDS?	“But it’s one you can avoid by taking your medicine every day to fight HIV. And talk to your health care provider if you are having this problem.”
	“And talk to your health care provider if you are having this problem. AIDS can affect your sex life.”	“Just keeping it real. You need to know about AIDS and how it can also affect your sex life.”
What is PrEP?	“That’s one reason why wearing a condom, even if using PrEP, is the safer thing to do.”
What is hepatitis A, B and C?	“But there is treatment for hepatitis B and C, so if you have either of these infections, talk to your health care provider to see if you can receive the treatment.”	“That’s a lot of information, so feel free to replay this question to hear me tell you again.”

Note.

^*Motivational statements may also be interactional, as in this example.

The Medication Manager contains several functions. Users can track viral load and CD4 count trends over time. Entry of undetectable viral load triggers a positive image (chosen by the user) and auditory feedback (a ta-da horn sound) while entry of other viral load values produces a warning sign that flashes briefly stating “Viral load too high. Discuss with provider.” Users can record common side effects. They may then display a summary of side effect frequency over the tracked period of time. They can also set reminders for medication taking, doctor’s appointments, blood draws, or other scheduled events. The Medication Manager also allows the user to select his medication(s) to display his regimen and offers a spoken overview of each medication’s common side effects, number of doses per day, whether or not it should be taken with food, and, for some medications, additional information (e.g., if antacids should not be taken at the same time). In addition, the Medication Manager asks daily if the user took his medication and records his responses and reasons for missing to produce a visual display of summary adherence data. Alternatively, a user may manually enter his adherence data in this function. This summary data displays as a calendar that a user may show his health care provider so that patterns may be recognized. Summary data of frequencies of reasons missing is also provided.

In Settings, users can call their case manager, doctor’s office, or pharmacy and create reminder message alerts to use the app. They may customize the app by selecting the agent’s clothing and glasses, screen background, reward images for controlled viral load, and a screen image that appears if they tap the upper left corner to disguise the screen from others. Users may also select their name if it is among the 95 names that the agent can speak (during several responses and greetings). There is also a function to read a description of the app’s goals and its theory base (About the App), submit a bug report, and replay the orientation (How to Use).

To promote self-efficacy and provide a relational experience, when a user comes to the home tab or when the app is restarted the agent asks the user, “Do you want to hear what a friend of mine says?” or a similar question, “Do you know what I think?” If he clicks yes, he hears one of 13 motivational empathetic messages where either the agent or a young HIV-positive AAMSM, doctor, nurse, medical assistant, mental health counselor, or a pharmacist speaks, reflects on his own experience, and encourages healthy behavior. By having the avatar directly question the user about hearing what a friend says, the app becomes more interactional and less user directed. This decision was also based on feasibility because it is more complex and costly to pilot and update multiple platforms. An overview of the app is available for viewing at https://youtu.be/1Wa69MwfhUI

OUTCOME MEASURES

To assess preliminary efficacy, the primary outcome was ART adherence as measured by percent of app users with >80% adherence at baseline versus after 3 months of exposure to the app as measured by pill count adherence ratio (Bezabhe, Chalmers, Bereznicki, & Peterson, 2016; Vishwanathan et al., 2015). Adherence was measured by baseline and follow-up pill count. In addition, they were asked how many doses of ART were missed during the 4 weeks before the baseline and follow-up surveys. This included determining if one dose of a twice daily regimen was not taken.

To measure acceptability, the follow-up survey included questions based on the Technology Acceptance Model including if the app was easy to use and how willing they are to continue using it (Holden & Karsh, 2010). We also asked to what extent they valued each of the app’s functions, if the app embarrassed them or made them uncomfortable, to what extent they felt the app made them feel more in control of their health, to what extent they felt the avatar cared about them, and if they would recommend the app to an HIV-infected friend. Responses were collected using a Likert scale. Acceptability variables included the extent of app use after the baseline visit, which functions were used more frequently, if the app was easy to use, to what extent participants valued each of the app’s functions, if the app embarrassed them or made them uncomfortable, if they felt more in control of the their health, if they were willing to continue app usage, and if they would recommend the app to an HIV-infected friend. Other variables of interest included pre versus post self-efficacy and pre versus post health literacy (measured as baseline versus follow-up knowledge of information that was taught in the app). We explored self-efficacy with two questions from the Coping Self-efficacy Scale (if they find something good in a negative situation and if they receive emotional support) as well an overarching adherence question (if they are confident that they can take their medication consistently; Chesney, Neilands, Chambers, Taylor, & Folkman, 2006).

Feasibility was examined by determining retention in the study, responsiveness to study check in calls, how many phones experienced technical difficulties or were lost, how many participants deleted the app before the follow-up visit, and proportion of participants with completed surveys and baseline and follow-up pill counts at 3 months.

Participants were reimbursed for their time including $50 for the baseline survey visit and $50 for the follow-up visit. If a participant lost his phone and a study phone was available as a loaner or they deleted the app, they received $15 for returning to a study site to resolve the issue.

STATISTICAL ANALYSES

Descriptive analyses were performed to describe participants. Baseline characteristics of the participants who completed the study and those who were lost to follow-up were compared. Wilcoxon rank sum test was used to compare median of continuous variables and Fisher’s exact test was used to compare the distribution of categorical variables between completers and those who were lost to follow-up. McNemar’s test was used for pre-post comparison of adherence and health literacy. Wilcoxon signed rank test was used to compare pre-post self-efficacy with a significance level set at p < .05 (two-tailed test).

RESULTS

Ninety-five persons contacted the study and 43 were screened eligible and enrolled, completing the baseline interview. Eleven participants were lost to follow-up. Six had technical difficulties with downloading the app after the baseline interview including two who were part of the 11 lost to follow-up because these difficulties were not overcome. Participants ranged from age 18 to 34 (median age 29). There was no statistically significant difference in baseline characteristics between those who completed the study and those who were lost to follow-up (Table 2).

TABLE 2.

Baseline Characteristics of Young African American Men Who Have Sex With Men Study Participants (N = 43)

Characteristics	All patients interviewed at baseline (N = 43)	Patients with both baseline and follow-up interviews (n = 32)	Patients with only baseline interviews (n = 11)	p value
	n (%)	n (%)	n (%)
Age in years, median (range)	29.0 (18.0–34.0)	30.0 (18.0–34.0)	27.0 (21.0–31.0)	.11
REALM-SF Score, median (range)	5.0 (0.0–7.0)	5.0 (0.0–7.0)	5.0 (2.0–7.0)	.44
Number of doses missed, median (range)	2.0 (0.0–7.0)	2.0 (0.0–7.0)	3.0 (1.0–7.0)	.10
Self-reported ability to read
Fair	9 (20.9)	7 (21.9)	2 (18.2)	.99
Good	15 (34.9)	11 (34.4)	4 (36.4)
Excellent	19 (44.2)	14 (43.8)	5 (45.5)
Pill count based adherence (n = 32)
>80% adherent to all medications	20 (46.5)	16 (50.0)	4 (36.4)	.35
>80% adherent to no medications	14 (32.6)	11 (34.4)	3 (27.3)
Unknown	9 (20.9)	5 (15.6)	4 (36.4)
Used any drug in past 6 months (including marijuana)
Yes	37 (86.0)	27 (84.4)	10 (90.9)	.99
No	6 (14.0)	5 (15.6)	1 (9.1)
Used any drug in past 6 months (excluding marijuana)
Yes	11 (25.6)	9 (28.1)	2 (18.2)	.70
No	32 (74.4)	23 (71.9)	9 (81.8)
Binge drinking in past 3 months
Yes	13 (30.2)	9 (28.1)	4 (36.4)	.71
No	30 (69.8)	23 (71.9)	13 (63.6)
Uses a pillbox
Yes	16 (37.2)	14 (43.8)	2 (18.2)	.17
No	27 (62.8)	18 (56.3)	9 (81.8)
Highest level of education
Less than high school diploma	14 (32.6)	12 (37.5)	2 (18.2)	.29
College or more	29 (67.4)	20 (61.5)	9 (81.8)
Number of years since first tested HIV positive, median (range)	6.0 (0.0–28.0)	6.0 (0.0–28.0)	5.0 (0.0–12.0)	.53
Number of years since first started taking HIV medication, median (range)	4.0 (0.0–21.0)	5.0 (0.0–21.0)	2.0 (0.0–10.0)	.24
Number of years since started taking current HIV medication, median (range)	0.8 (0.0–10.0)	0.7 (0.0–7.0)	1.5 (0.2–10.0)	.21
At least 2 medical care visits in past 12 months that were at least 3 months apart
Yes	41 (95.4)	30 (93.8)	11 (100.0)	.99
No	2 (4.7)	2 (6.3)	0 (0.0)
At least 2 blood draws in past 12 months that were at least 3 months apart
Yes	39 (90.7)	28 (87.5)	11 (100.0)	.56
No	4 (9.3)	4 (12.5)	0 (0.0)

Note. REALM-SF: Rapid Estimate of Adult Literacy in Medicine-Short Form.

FEASIBILITY AND ACCEPTABILITY

Thirty-two (74%) participants were retained in the study (had baseline and follow-up interviews). Among the cumulative 474 attempts to reach participants during the follow time to check-in, 217 (46%) were successful. Among the 43 participants interviewed at baseline, we were able to install the app in the phone of 37 (86%). Five of the six phones that did not install were types of LG phones (X401, K7, and Tribute 5) and most had an operating system that was more than 7 years old. Among the six participants in whose phones we were unable to install the app, we were able to provide four a loaner phone. Two of them lost the loaner phone during follow-up. Of the 32 participants who completed a follow-up interview, 21 (64%) phones had app usage data available for analysis. Among these 21, four were participants who had required reinstallation of the app because three (14%) lost or changed their phone and one (5%) experienced technical difficulties. Among participants who completed a follow-up interview but did not have app usage data (n = 11), nine (82%) either lost or changed the phone and two (18%) deleted the app before the follow-up visit. At the conclusion of the study, 26 (60%) participants had baseline and follow-up surveys and pill counts.

Nineteen (90%) participants who returned for follow-up and had app utilization data available had used the app at least once. Nineteen (100%) had used the app at least once during each of the 3 months. The mean number of days that participants used the app was 17.3 days (median 8). The total number of times the app was used was 397 times (mean per participant 20.9 times). The mean duration of each app use was approximately 2 minutes and varied greatly by app function. For example, the Let Me Explain function takes much more time to listen to an explanation whereas entering medication adherence takes less time. The functions used the most often (expressed as mean number) were entering medication adherence (48.05 times), customizing the agent’s appearance (11.58 times), responding to being asked if the user took his medicine today (9.58 times), playing a Let Me Explain question/answer (6.63 times) and playing one of the audio recordings of the motivational messages (5.37 times). The functions used the least often were submitting a bug report (never), About the App (never), How to Use (0.11 times), entering side effects (0.16 times), entering a reminder to use the app (0.16 times), and entering contact information for a health care team member (0.21 times). At least one participant used the following functions more than 10 times: entering adherence data (13 participants, maximum 123 times), responding to if he took his medicine today (4 participants, maximum 64 times), customizing the agent (7 participants, maximum 44 times), playing a Let Me Explain question/answer (5 participants, maximum 43 times), listening to a motivational message (3 participants, maximum 30 times), setting a reminder (1 participant, maximum 14 times), and concealing the screen (1 participant, maximum 11 times).

Thirty-one (97%) would recommend the app to a friend who is HIV-positive and 21 (66%) would recommend the app to someone else (e.g., someone newly diagnosed with HIV, a partner, or people who are not HIV positive). Five (16%) felt the app embarrassed them or made them feel uncomfortable at any time. When asked to explain, one stated another person “was staring at my phone,” and another offered to replace the image of a pill (that signified dose taken in the adherence calendar) with a smiley face. The remaining explanations concerned disliking the lock out function (after nonuse), the “responses were too much textbook type,” and the conspicuousness of the study loaner phone he received because it was different from his own phone. Eleven (34%) listened to every one of the Let Me Explain questions. Thirteen (41%) listened to the agent answer the same question more than once. Table 3 provides results of other acceptability measures. The median score for all the acceptability questions (such as to what extent the app made you feel in control of your health) was 10. Scores of 9 or 10 were recorded for at least 75% of participants for all acceptability questions.

TABLE 3.

Measures of Acceptability of My Personal Health Guide Avatar-Based Mobile Phone App

Question	Likert scale score from 1 (least) to 10 (most)
	10	9	8	7	6	5	4	3	2	1
	n (%)	n (%)	n (%)	n (%)	n (%)	n (%)	n (%)	n (%)	n (%)	n (%)
How easy or difficult the app is to use?	22 (69)	4 (13)	1 (3)	2 (6)	1 (3)	1 (3)	1 (3)	0 (0)	0 (0)	0 (0)
How valuable the app’s Let Me Explain function is for you?	21 (66)	4 (13)	1 (3)	3 (9)	1 (3)	0 (0)	0 (0)	1 (3)	1 (3)	0 (0)
How valuable the app’s medication function is that has a photo of medication and explains the medication name, its side effects, and whether it is taken with food?	26 (81)	3 (9)	1 (3)	0 (0)	1 (3)	0 (0)	1 (3)	0 (0)	0 (0)	0 (0)
How valuable the app’s function is that asked you if you took your medication and provided a monthly view of doses missed and taken? (n = 31)	25 (81)	2 (7)	2 (7)	1 (3)	1 (3)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)
How valuable the app’s function is that collects and displays CD4 and viral load results? (n = 29)	18 (62)	5 (17)	2 (7)	0 (0)	3 (10)	1 (3)	0(0)	0(0)	0 (0)	0 (0)
How valuable the app’s function was that allowed you to change the screen to disguise the app?	26 (81)	1 (3)	2 (6)	1 (3)	0 (0)	1 (3)	0(0)	1 (3)	0 (0)	0 (0)
To what extent the app made you feel in control of your health?	19 (59)	5 (16)	2 (6)	2 (6)	0 (0)	3 (9)	0(0)	0 (0)	1 (3)	0 (0)
How willing you are to continue using the app?	19 (59)	6 (19)	1 (3)	2 (6)	0 (0)	1 (3)	1 (3)	0 (0)	0 (0)	2 (6)
To what extent do you think the avatar cared about you?	20 (63)	4 (13)	0(0)	1 (3)	2 (6)	2 (6)	1 (3)	0 (0)	1 (3)	1 (3)
To what extent do you think the avatar understands the issues you face?	23 (72)	1 (3)	2 (6)	1 (3)	2 (6)	2 (6)	0 (0)	0 (0)	1 (3)	0 (0)

Note. n = 32 unless specified otherwise.

Concerning when asked which function they liked the most and the least, the most agreement for most liked was 11 (34%) for the reminder function and the most agreement for least liked was three (9%) for the lock out of the app.

PRELIMINARY EFFICACY

Pill count adherence > 80% improved from 62% at baseline to 88% at follow-up (p = .05) (Table 4). Limiting the analysis only to participants with utilization data confirming they used the app at least once after the baseline visit, pill count adherence >80% improved from 59% to 88% at follow-up (n = 17) (p = .10).

TABLE 4.

Baseline Versus Follow-Up Knowledge and Efficacy Among 32 Participants Who Downloaded the My Personal Health Guide Mobile Phone App

Knowledge item	Baseline	Follow-up	p value
	n (%)	n (%)
Why do persons taking HIV medicine need their blood drawn?
Correct: To check the CD4 or to check if the medicine is working	29 (90.6)	31 (96.9)	.07
Incorrect	3 (9.4)	1 (3.1)
What blood tests does your health care provider order to check up on how you are doing?
Correct: CD4, viral load, kidney function, anemia, platelets, electrolytes (or any component or synonym), sexually transmitted diseases like gonorrhea, chlamydia, or syphilis
3 or more correct responses	15 (46.9)	22 (68.8)	.11
1–2 correct responses	8 (25.0)	7 (21.9)
No correct response	9 (28.1)	3 (9.4)
What is a viral load?
Correct: The amount of virus in your blood.	19 (59.4)	23 (71.9)	.05
Incorrect	13 (40.6)	9 (28.1)
What is a CD4 count?
Correct: Amount of T cells/CD4 cells in your blood.	11 (34.4)	21 (65.6)	< .01
Incorrect	21 (65.6)	11 (34.4)
Is AIDS the same thing as HIV?
Correct: No	31 (100)	32 (100)
Incorrect: Yes	0 (0)	0 (0)	.99
If your viral load is undetectable, can you have unprotected sex?
Correct: Noa	28 (87.5)	30 (93.8)	.32
Incorrect: Yes	4 (12.5)	2 (6.3)
If you miss a dose of your HIV medication when it is due, should you wait until the next one is due or take the missed dose at a later time the same day you missed it?
Correct: Later time the same day	20 (62.5)	25 (78.1)	.26
Incorrect: Wait until the next one is due	8 (25.0)	5 (15.6)
Not sure/Don’t know	4 (12.5)	2 (6.3)
What are some things that can happen when someone has AIDS?; median number of diseases listed (range)b	1 (range 0–5)	1 (range 0–5)	.15
Do you know if your HIV medication has to be taken with or without food?c
Correct for all the participant’s ART	17 (53.1)	29 (90.6)	< .01
Incorrect for at least one ART	15 (46.9)	3 (9.4)
Adherence
Self-reported
>90% adherent	22 (68.8)	26 (81.3)	.29
100% adherent	8 (25.0)	15 (46.9)	.03
Pill count (n = 26)
>80% adherent	16 (61.5)	23 (88.5)	.05
>80% adherent among participants with app usage data (n = 17)	10 (58.8)	15 (88.2)	.10

Note.

^aThe app teaches about risk for STDs and possible infection with a resistant strain of HIV as reasons why unprotected sex is risky;

^bthe app teaches about Pneumocystis pneumonia, thrush, erectile dysfunction; however, any correct answer was accepted;

^cvalidated by checking participants’ medication.

Self-reported adherence results were not substantially or significantly different for a cut-off of > 80% adherence, however for > 90% adherence the percent of participants rose from 69% to 81% (p = .29) and for 100% adherence it rose from 25% to 47% (p = .03).

SELF-EFFICACY

Using a Likert scale where strongly disagree is 1 and strongly agree is 5, the median baseline and follow-up responses to the statement, “I am confident that I can take my medication consistently” were 5 and 5, respectively (p = .99). Similarly, the median responses for, “I can find something good in a negative situation” (p = .99) and for, “I receive emotional support from my friends and/or family” were 4 and 4, respectively (p = .61).

HEALTH LITERACY

Baseline versus follow-up knowledge analysis was performed for each of the following: why persons taking HIV need their blood drawn, what blood tests are checked to see how they are doing, what is a viral load, what is a CD4 count, is AIDS the same thing as HIV, if they appreciate the potential risks of unprotected sex (for example STDs even when the viral load is undetectable), whether a dose can be taken later the same day if they miss a dose of HIV medication when it is due, knowledge of complications of AIDS, and knowledge of if medication has to be taken with food (Table 4). Baseline knowledge was high (> 90%) for why a blood draw is needed and if AIDS is the same thing as HIV. Statistically significant change in knowledge was observed for knowing what is a viral load (increasing more than 10% from baseline), what is a CD4 count (increasing more than 30% from baseline), and if their medication has to be taken with food (increasing nearly 40% from baseline and rising to > 90%).

DISCUSSION

We report the first pilot study of a theory-based mobile-delivered, realistic talking human avatar-like embodied conversational agent intervention (My Personal Health Guide) to address ART adherence. This pilot study demonstrated evidence of acceptability and preliminary efficacy of My Personal Health Guide in young AAMSM living with HIV. It also revealed several feasibility issues that will be used to inform refinement of the app and future study.

Since smartphone interventions are experienced in a location of the user’s choice, their success may be less influenced by stigma or other privacy concerns related to clinical settings than other interventions. AAMSM prefer interventions that allow for anonymity (Senn, Braksmajer, Coury-Doniger, Urban, & Carey, 2017). Feedback from participants in the iterative focus groups informed My Personal Health Guide’s development, including the importance of building several features that respected privacy in order to promote acceptability. These features included a bland app icon that gives no indication it is for personal health, password protection, timed default logout, disguise screen ability, and user-created reminder alerts.

My Personal Health Guide takes a multi-faceted approach to improving health behavior. By using an embodied conversational agent whose appearance was influenced by focus groups with young AAMSM living with HIV, the app provides an attractive credible source for ART adherence information, motivation, and behavioral skills. Motivation and relational expressions are embedded in the app’s informational interaction to promote memory and to augment efficacy of the provision of information. The agent also advocates to the user to use functions and offers behavioral skills that are not part of the app (such as encouraging use of a weekly pill organizer). In addition, the app includes the opportunity to hear the voices and motivational thoughts of peers and health care supportive personnel through audio recordings that provide a level of reality and credibility beyond the agent itself.

The median REALM-SF score for our participants was 5. This value is equivalent to a seventh to eighth grade range. Persons at this level are expected to struggle with most educational materials (Agency for Healthcare Research and Quality, 2016). Low health literacy has been associated with poor ART adherence and retention in care (Jones et al., 2013; Kalichman & Rompa, 2000; Kalichman et al., 1999, 2008; Osborn et al., 2007; Wolf et al., 2006). Our embodied conversational agent provides information in an audiovisual format that allows for instruction; this is complementary to outpatient settings where providers may have limited time for instruction, patients may be afraid to ask for explanation, and medical jargon is used and not understood. Also, for patients for whom distraction or emotion interfere with comprehension, in My Personal Health Guide the agent’s explanations are both replayable and readable. Health literacy, as measured by information taught in the Let Me Explain function, improved in our study population. The largest improvements were observed for understanding of a CD4 count and whether their medication should be taken with food.

In addition to information, motivation was an important component of the theory-base of this app. Confidence that they could take their medication as indicated was already high in this population (> 90%) but did show further improvement to 100% at follow-up. However, although the app was motivational, measurements of coping self-efficacy did not demonstrate substantial improvement. Given the high baseline confidence, self-efficacy, and social support of our study participants, our ability to show improvement was limited.

There was high acceptability of My Personal Health Guide by participants in our pilot study. Generally, they found the app easy to use and valued its functions. Most felt the app helped them feel in control of their health and would continue using it. However, usage of functions was variable and informative. Behavioral skills, as provided with medication adherence and HIV outcome measure monitoring functions (CD4 count and viral load), were used variably. The usage data suggested that for many participants the monitoring medication function was valued based on its repeated use. However, the follow-up time was only 3 months and so we did not expect nor did we observe substantial use of the HIV outcome measure tracking functions. The high use of agent customization also demonstrated the popularity of this function, which may be an important location for the agent to interact further in a future app version. For example, we may build new dialogue from the agent, encouraging use of app features and/or delivering education (possibly by having the agent offer an educational question and answer) while the user is engaged in this function. Few of the participants entered contact information of their health care provider, although studies in this population suggest this ability is valued (Dworkin et al., 2018).

The Let Me Explain questions and answers and the audio recordings of motivational messages received moderate use. Refinement of the app might include increased attention to the user experience of the Let Me Explain function to make it more of a draw or a repeated draw. This function could be made more visually appealing graphically and it could be associated with a game feature. The game could be educational in nature where users are tested on information from the Let Me Explain answers and their game score could be improvable by engaging or reengaging with the Let Me Explain questions. Gamification has been used in medicine and public health to motivate behavior change (Kawachi, 2017). Such a modification to this app might augment efficacy as it promotes interaction with educational information that the app teaches but the user avoided because possibly they did not think they would learn something new or did not care to learn it. Gamification is an emerging area in health apps (Brull, Finlayson, Kostelec, MacDonald, & Krenzischeck, 2017; Hightow-Weidman, Muessig, Bauermeister, LeGrand, & Fiellin, 2017; King, Greaves, Exeter, & Darzi, 2013). We expect to build into the education game the ability to unlock prizes (conditional rewards, equivalent to achieving additional levels), which appeals to the human tendency of loss aversion and assigning value to things they hold (endowment effect; King et al., 2013).

Performing a mobile phone study in young AAMSM living with HIV revealed several feasibility issues. We successfully recruited 43 participants. We expected to lose approximately 15% and lost approximately 25%. Loss (e.g., theft) or change of phone was the main problem and reflects the difficult conditions many of these urban young men live with due to problems such as unemployment and homelessness. Technical difficulties were partially recoverable, such as identification of incompatibilities with certain phones that used very old operating systems. Troubleshooting to overcome these problems is ongoing. However, our attempts to stay in contact with the participants were only successful about half the time. We offered no compensation for responsiveness to check-in calls, which in retrospect might have helped. We also had many participants who did not bring their pill bottles to their follow-up appointment despite their instruction to do so. Again, linking bringing pill bottles to compensation would have likely helped improve these numbers.

There were several limitations of our study. Our pilot had no control group. We did not require participants to have a detectable viral load or to report struggling with adherence, which may mean the pilot underestimated the efficacy of the app since those who began with perfect adherence could not have demonstrated improvement. We relied on pill count adherence. Although our budget could not afford it, electronic adherence monitoring with adjudication may have been more accurate. We also did not collect biomarkers of adherence. We also had substantial loss of usage data due to app deletion and phone loss. Deletion of the app deleted all its usage data. Although this was undesirable, it was necessary given the relatively short funded timeline of development and pilot. Finally, our sample size was small giving the study low statistical power to detect significant differences and our study was performed in one city, which limits generalizability.

Many young HIV-positive AAMSM in care struggle with adherence and experience uncontrolled viral load. Our pilot study of a theory-based embodied conversational agent mobile phone app demonstrated acceptability and preliminary efficacy in improving adherence in this important population. The pilot also provided useful feasibility data to inform refinement of the app and performance of a randomized clinical trial. We believe this app has the potential to augment clinical care and case management for patients starting ART and for those struggling with adherence. We envision the app being recommended by case managers and health care providers as a supplement to care and potentially employed in combination with other adherence interventions such as adherence counseling. Future refinement of the app should include modifications such as collecting usage data via network while the app is running so that loss of phone and app deletion do not interfere with its measurement up to the time of loss, wireless adherence monitoring, and gamification. Compensation during follow-up to increase retention should be employed in future research, which may also include modification of the app for use on tablets or personal computers and determining its acceptability and suitability to other populations living with HIV such as older MSM, other men, adolescents younger than 18 years, and women.

FIGURE 1.

My Personal Health Guide theory-based mobile phone app home screen.

FIGURE 2.

Four views illustrating Medication Manager functions. 2a: Calendar illustrating a hypothetical adherence summary. 2b: Alternative example of adherence data. 2c: Example of side effect tracker data. 2d: Example of viral load trends if most recent entry was above 200 copies/mL.