Understanding the predisposing, enabling, and reinforcing factors influencing the use of a mobile-based HIV management app: A real-world usability evaluation

Abstract

Objective:

To conduct an in-depth analysis of users’ experiences using an HIV self-management app.

Materials and Methods:

We conducted four follow-up focus groups at the end of a 3-month randomized feasibility trial. All focus group sessions were audio-recorded and transcribed. A thematic analysis was conducted to explore emerging themes. All of the themes were categorized into three factors of the PRECEDE component of the PRECEDE-PROCEED framework. For a finer granularity of analysis, the codes of each theme were broken into positive, negative, and neutral codes by study group.

Results:

36 participants, including 24 from the intervention group and 12 from the control group, participated in the focus group sessions. A total of 14 themes organized by the PRECEDE factors were identified from focus group transcripts. Five themes related to predisposing factors were: 1) ease of app use; 2) user-friendly functionality; 3) self-efficacy for symptom management; 4) design preference of illustrated strategies with videos; and 5) user-control (convenience vs. security). Four themes related to enabling factors were: 1) information needs of symptom management; 2) symptom-tracking; 3) fit in lifestyle/schedule/living conditions; and 4) additional languages (e.g., Spanish). Five themes related to reinforcing factors were: 1) communication with healthcare providers; 2) individual-tailored information visualization; 3) social networking; 4) individual-tailored information quality; and 5) improvement in quality of life.

Discussion and Conclusion:

Usability evaluation in a real-world setting enabled us to measure users’ actual experiences when interacting with the app during their everyday lives. Our work highlights the importance of using mobile technology for persons living with HIV, specifically those with low income/housing instability, and adds to the body of literature on the potential for implementation and dissemination of symptom self-management strategies through a mobile platform.

1. INTRODUCTION

77% of Americans own a smartphone in 2018, up from 35% in 2011.[1] Concurrent with the increase in smartphone users, the use of mobile technologies in healthcare has expanded.[2, 3] Approximately 165,000 mobile health (mHealth) applications (apps) are now available in the Apple iTunes and Android app stores in the United States (US).[4] mHealth apps have successfully been used for self-management by persons living with chronic conditions and have successfully improved outcomes.[5, 6] For example, of the 27 randomized controlled trials (RCTs) that measured the effect of mHealth interventions on the adherence of patients to chronic disease management, 15 studies (56%) among patients with diabetes, cardiovascular diseases, and chronic lung diseases showed a statistically significant improvement in adherence behaviors (e.g., medication adherence, engagement in healthy behaviors, and frequency of symptom monitoring) between groups.[7]

Despite the increasing availability of self-management apps for people living with chronic diseases, usability factors remain one of the major obstacles to adoption of health information technologies.[8–10] User acceptance of mobile technology, specifically how people use technology, is closely related to usability factors and is further linked to the success or failure of the technology.[11] Adoption of mHealth tools and their subsequent use is highly related to the usability of the system.[12] Usability is often evaluated in a laboratory setting, which may have limited relevance to a real-world setting.[13–15] Therefore, it is essential to assess the usability of the system in persons’ everyday settings to ensure that the mHealth tool works properly and supports intended end-users to achieve their goals in real-world situations.[16, 17] The purpose of this study was to conduct an in-depth analysis of users’ perceptions of the factors influencing the use of a mHealth app and its acceptance in a real-world setting. The mHealth app is intended to help persons living with HIV (PLWH), who are confronted with persistent symptoms related to the disease, medication side effects and comorbidities making symptom management essential for improving quality of life,[18] manage their symptom experience through evidence-based self-care strategies.

2. METHODS

Study Context:

This study was conducted as a follow-up to a larger trial. In summary, we used participatory methods with end-users to design a mobile app for symptom self-management, entitled mVIP, for PLWH.[19, 20] The mVIP app was comprised of 143 self-care strategies for 13 different symptoms (i.e., anxiety, cough/shortness of breath, depression, diarrhea, difficulty falling/staying asleep, difficulty remembering, dizziness, fatigue, fever/chills/sweats, nausea/vomiting, neuropathy, skin problems, and weight loss/wasting). Once users log into the mVIP app, an avatar guides them through a series of questions for 13 HIV-related symptoms. Based on the nature and severity of their symptoms (e.g., did you have fatigue in the past seven days? If yes; how much did it bother you?), the avatar recommends self-care strategies, for ameliorating their symptoms, with videos to participants in the intervention group, whereas the avatar did not recommend any of the strategies/videos to participants in the control group. Following an iterative development process,[21] we tested the efficacy of the app in improving symptom outcomes in PLWH in a randomized feasibility trial.[22] In the 12-week feasibility trial, the app was found to be efficacious in improving patient reported outcomes such as symptom burden and medication adherence in PLWH.[22] At the end of the trial, we invited a sub-sample of our study population to participate in follow-up focus group sessions to describe their experiences using the app in their everyday settings and to inform the future dissemination and implementation of the mVIP app.

2.1. Sampling and Recruitment

The sampling frame, inclusion criteria and recruitment strategies are reported in the trial paper.[22] Following completion of the trial, we recruited participants for the follow-up focus groups using convenience sampling [23] where the study sample meets certain practical criteria, such as availability at a given time (focus group sessions took place on four different dates), and willingness to participate in the focus groups. Given that it is recommended each focus group consists of 6 to 12 participants,[24] we over-recruited to accommodate for cancellations and no-shows, scheduling 12–14 participants for each focus group session.[25] Evidence suggests that three focus groups are enough to identify the prevalent themes within data set,[24] therefore we included three focus groups among participants assigned to the intervention group in addition to one focus group session with those assigned to the control group. While the inclusion of control group participants in the follow-up focus groups can produce insights regarding usability of the app, more extensive focus group sessions with intervention group participants is of greater value.[26, 27] Of the 76 participants who completed the trial between December 2016 and June 2017, 36 (24 from the intervention group and 12 from the control group) participated in the follow-up focus group sessions. There were 7 to 12 participants per focus group session.

2.2. Study Procedures

Following completion of the informed consent process, all focus group sessions were audio-recorded with two digital recorders to safeguard against mechanical failure. Focus group sessions were moderated by a member of the research team using a semi-structured discussion guide (Table 1). Guided by the focus group guide, participants were encouraged to discuss issues regarding their experience using the app during the trial period. Each focus group was 60–90 min in length. Participants were given $40 as a token of appreciation for their time. Data collection continued until saturation of themes was reached.

Table 1.

Focus Group Discussion Guide

During the past 3 months of mVIP use:
General perceptions of mVIP
•  Tell us your perceptions of the app.
○  Probe: What was your first impression of the app? Did it behave as you expected?
•  How comfortable were you using the mVIP app in social settings?
○  Probe: where and when used it mostly?
•  How long did it feel like it took you to complete each app session?
○  Probe: how this fit into your lifestyle & schedule?
•  (Only intervention group) What are your thoughts about the videos displayed in the app?
○  Probe: watching videos vs. reading the content; videos with sounds
Use of mVIP
•  After your first app use, how difficult was it for you to use in follow up uses?
•  Describe your experience with any technical problems.
○  Probe: crash/error/back button/Continue button
•  Tell us about your experience with the instructions provided through the app.
○ Probes: how our app works, reminder/email, error messages
•  Describe your experience navigating the app pages and remembering the basic structure of the app.
•  (Only intervention group) What was your experience using the app to review your symptoms and strategies (=Your History)?
•  What are your thoughts about the design of mVIP?
○  Probe: main logo/avatars; font/color; progress bar; Save ID&PW; Log-in Help; Continue button
○ (Only intervention group) helpfulness response options of Yes/No/Didn’t Try
Impact of mVIP
•  How did this app help you gain information about relieving your symptoms?
•  How confident are you in your ability to self-manage your HIV-related symptoms?
•  How do you think that your HIV-related symptoms improved after using the app for 3 months?
•  Tell us how the app did/can change your current personal, professional or healthcare provider relationships.
•  Explain how the app did/can change your quality of life.

2.3. Data Analysis

After each focus group session, the research team conducted peer debriefing. Audio-recordings from the focus groups were transcribed. The transcripts and field notes were reviewed and analyzed. Two research team members coded the transcripts after reading each transcript at least twice; each research team member independently generated a set of codes based on a line-by-line analysis. A thematic analysis was conducted to explore emerging themes with similar patterns across focus groups. A codebook was then developed, and free text excerpted from the transcripts was entered followed by each of the themes into the codebook. The Predisposing, Reinforcing, and Enabling Constructs in Evaluation (PRECEDE) portion of the PRECEDE-PROCEED Model of health program planning and evaluation [28] was used to guide the coding framework. All themes in the codebook were categorized into the three factors of the PRECEDE Model: predisposing, reinforcing, and enabling.[29] Predisposing factors, which include knowledge, beliefs, values, attitudes, and self-efficacy, appeal to people’s motives for behavior change. Enabling factors, which include health-related skills, and resources (e.g., training), facilitate a behavior’s occurrence. Reinforcing factors occur after a behavior, and include the incentives and rewards, with the increased probability that the behavior will recur at the next opportunity.[30] For a finer granularity of analysis, the codes of each theme were broken into positive with a plus sign (+), negative with a minus sign (−), and neutral codes without sign. Discrepancies in the themes/codes were discussed until consensus was achieved. A third reviewer resolved any discrepancies throughout the coding.

Descriptive statistics were used to analyze demographic information using SPSS version 24.0 (IBM Corp, 2015). Counts (N) and percentages were reported for categorical variables; continuous variables were expressed as means and standard deviations (SDs). We compared normally-distributed continuous variables using the two-sample t test between study groups. Level of significance was set at α = 0.05.

3. RESULTS

3.1. Sample

Characteristics of the study sample are reported in Table 2. The majority (71% in the intervention; 92% in the control) of focus groups participants reported their race as African American/Black, and half (54% in the intervention; 42% in the control) of the participants had an annual median income of less than $10,000. The mean age for participants was 50.54 years (SD = 13.02; range = 23–72 years of age) for the intervention and 52.25 years (SD = 6.64; range = 39–63 years of age) in the control group. There were no statistically significant differences between study groups.

Table 2.

Characteristics of Focus Group Participants

Characteristics	Main Trial Study Group; N (%)
	Intervention=24	Control=12
Gender
Male	7 (29)	7 (58)
Female	17 (71)	5 (42)
Race
African American / Black	17 (71)	11 (92)
White	3 (12)	0
Other	4 (17)	1 (8)
Ethnicity
Hispanic / Latino	4 (17)	2 (17)
Marital Status
Single	13 (54)	9 (75)
Married/ In a relationship	6 (25)	2 (17)
Divorced	3 (12)	1 (8)
Widowed	1 (4)	0
Education Level
Elementary school	1 (4)	0
Some high school	5 (21)	2 (17)
High school graduate/ Got GED	7 (29)	4 (33)
Some college	6 (25)	2 (17)
Associate’s degree	2 (8)	0
College graduate (4 years)	1 (4)	4 (33)
Professional or graduate degree	2 (8)	0
Current Employment Status
Employed full-time	1 (4)	2 (17)
Employed part-time	1 (4)	0
Unemployed looking for work	7 (29)	1 (8)
Unemployed, not looking for work	2 (8)	2 (17)
Disabled	9 (38)	5 (42)
Annual Income
Less than $10,000	13 (54)	5 (42)
$10,000-$19,999	6 (25)	2 (17)
$20,000-$39,999	0	3 (25)
$40,000-$59,999	1 (4)	0
Don’t know	2 (8)	1 (8)
Health Insurance Provider
Public (e.g. Medicare, Medicaid, Ryan White)	23 (96)	11 (92)
Private (e.g. through employer/relative’s employer)	1 (4)	1 (8)
Age
Years of age (SD)	50.54 (13.02)	52.25 (6.64)

3.2. Findings

A total of 14 themes were identified from the focus group transcripts. We organized the themes according to the three factors of the PRECEDE [29], component of the PRECEDE-PROCEED framework [28]: 1) Predisposing, 2) Enabling, and 3) Reinforcing Factors.

3.2.1. Predisposing factors

The five major themes related to predisposing factors, were: 1) ease of app use; 2) user-friendly functionality; 3) self-efficacy for symptom management; 4) design preference of illustrated strategies with videos; and 5) user-control (convenience vs. security). These predisposing factors and representative quotes are presented in Table 3. Codes were organized by positive, negative and neutral signs, and study group (i.e., intervention vs. control).

Table 3.

Themes and Quotes of Predisposing Factors

Factor	Code	Quotes
P-1. Ease of app use
Intervention group	Positive	It wasn’t hard at all. Once I got to the part, where it was easy to get to start, it was very easy for me.
	Negative	No comment
	Neutral	No comment
Control group	Positive	Once I got to the part where, you know, it was easy to get to start. Very easy for me.
		No comment
	Negative
		No comment
	Neutral
P-2. User-friendly functionality
Intervention group	Positive	You guys give us a reminder. One day I did forget on Monday. It was Tuesday, Reminder said “You have to answer your questions.” And, I could get right on top of it.
	Negative	I wanted to be able to go back and review my history just like when you go on the internet and it says, “Your History” and you go back, before you delete everything, you can go back and see it. That’s what I thought it was going to do, but it didn’t do that.
	Neutral	More choices (of Avatars) might be helpful? I guess maybe… I really didn’t bother me either way.
Control group	Positive	The email was awesome because you guys remind me that I need to answer that.
	Negative	The avatar could be much more fun and be more related to people who like to see and hear things. Not just see it, but if the avatar could actually talk or move its mouth.
	Neutral	The cartoon, it was okay, because I had to do something, so basically I had to get used to learning how to work it and I did that in a couple of days after she taught me.
P-3. Self-efficacy for symptom management
Intervention group	Positive	I got several symptoms like depression or not sleeping well and rush. And, I got all recommendations from the app and tried. It works!
	Negative	No comment
	Neutral	No meds. Just vitamins, working out, eating lots of fruits and whatever plan God has for me because he’s in this, too. You understand what I’m saying? I respect the app, you know, but everything is not for everybody.
Control group	Positive	I’m able to know what’s really going on. I constantly know how often this is going on, so I already got it (symptom) down and it’s in the app.
	Negative	No comment
	Neutral	No comment
P-4. Design preference of illustrated strategies with videos
Intervention group	Positive	Attractive videos. Exactly. All of them were cute. And, easy to understand (the strategies).
	Negative	I felt like I would just see the air bubbles so I never wanted to press play again because I didn’t hear anything. I wanted audio.
	Neutral	No comment
Control group	Positive	No comment
	Negative	No comment
	Neutral	No comment
P-5. User-control (convenience vs. security)
Intervention group	Positive	I didn’t have a problem with password. I saved it. Every time I would go into Chrome and mVIP had already that password and it had the number (ID) and all I had to do was only sign in. That’s it.
	Negative	No comment
	Neutral	No comment
Control group	Positive	9 out of 10 times, I forgot the password. The app had a little button that says: remember me. Therefore, I didn’t have to remember my password. No need to remember. (convenience)
	Negative	For me, I never do that…If you have that “remember me” and somebody accesses you phone…let’s say you have it on your bank account. They can immediately see what your bank account level is, they have access to your HIV thing. (security)
	Neutral	No comment

Participants in both intervention and control groups perceived that mVIP was easy to use. A participant in the intervention group stated that, “The questions (in the app) were all pretty much straightforward, easy to answer. Actually, the app was kind of simple to use, very simple. Totally easy…”

Several participants in both groups thought that the app had user-friendly functionalities such as avatar selection, reminder emails, app instructions, and error messages. One participant stated that, “I would get an email when I totally forgot. And, I’m like, “Oh, wow. They reminded me.” I felt so good.” Meanwhile, a participant highlighted his expectation that the mVIP app would include additional functionality which would allow end-users to upload their own pictures instead of selecting an avatar, describing that, “I didn’t like the avatar… I’m interested in really looking at their face and not the avatar. I want to see my face on my phone. I didn’t want to see this cartoon.”

While both the intervention and control groups received symptom and intensity questions (e.g., did you have fatigue? If yes; how much did it bother you?), only the intervention group was provided with self-management strategies for the symptoms that bothered the participants. Participants in the intervention group expressed self-efficacy for managing HIV-related symptoms via the mVIP app. Relatedly, participants in the control group expressed their self-efficacy for symptom management by monitoring their symptoms through the use of the app. One participant stated that, “My experience while using this easy app, it gave me the power to control and monitor what was really going on with my body.”

The app for the intervention group provided strategies using text and short animated videos. Several participants expressed that the animated videos were attractive and helpful for their better understandings of the strategies. Meanwhile, some participants suggested adding sound into the videos. One participant stated that, “The video didn’t have any sound. So, I would just look at them and I’m like, “Why aren’t they talking but they’re moving.” I want to hear something. I want to hear someone talking.”

Participants in both groups appreciated having the option of saving or not saving their password in accordance with an individual’s preference (convenience vs. security). A control group participant expressed a preference for saving a password stating that, “9 out of 10 times, I forgot the password. The app had a little button that says: remember me. Therefore, I didn’t have to remember my password. No need to remember.” On the contrary, however, another participant raised concerns about the security of the password, reflected on his experience when his smartphone was stolen, “For me, I never do that…If you have that “remember me” and somebody accesses you phone…let’s say you have it on your bank account. They can immediately see what your bank account level is, they have access to your HIV thing.”

3.2.2. Enabling factors

The four themes related to enabling factors influenced use of the app were: 1) information needs of symptom management; 2) symptom-tracking; 3) fit in lifestyle/schedule/living conditions; and 4) additional languages (e.g., Spanish). The enabling factors and representative quotations are presented in Table 4.

Table 4.

Themes and Quotes of Enabling Factors

Factor	Code	Quotes
E-1. Information needs of symptom management
Intervention group	Positive	The suggestions they’re telling us to do is helping us. And, those are minor issues, but it can affect us a lot. Like, moving my bowels. I had a problem with that.
	Negative	But, after the first week of viewing everything, nothing changed. It was the same questions. It was the same video. I could have used a lot of other scenarios or more information to show me how to do things differently. Everything, after a while, everything it seems like it doesn’t work anymore… But, after doing it for 3 months it gets boring.
	Neutral	Every time I use it, sometimes I hit different things just to see what they would say in the symptoms or results.
Control group	Positive	And, my doctor that took my sample, they didn’t tell me nothing. But, I think it was telling me it’s what I’m eating. Doctor didn’t tell me that.
	Negative	What if there was a version of mVIP where it went into a forum where you could talk to therapists or have referrals based on what your symptoms are and can actually address it.
	Neutral	No comment
E-2. Symptom-tracking
Intervention group	Positive	The good thing about it (app) was that you can really go back to whatever you had, your progress was, and see what you did throughout the week.
	Negative	It’s just like, I mean, I don’t know, what I wanted to see more was like a calendar or like the frequency of how often I was experiencing certain symptoms rather than just like, these are the symptoms you’ve had in the past.
	Neutral	No comment
Control group	Positive	I think the app, for me, made me pay more attention to my over health. The app made me think of my overall health and how to really monitor my health and what I’m doing on the outside.
	Negative	No comment
	Neutral	No comment
E-3. Fit in my lifestyle/schedule/living conditions
Intervention group	Positive	I can use it even at home because I’m usually in the house all the time. I had a particular time, “OK, let me sit down and do the app.”
	Negative	There’s a lot of places you didn’t have service. Like even here there’s no service. So I should always use it at home…
	Neutral	I only use it at home…when I get bored I just pick up the phone and go on anything. Just use it at home in privacy.
Control group	Positive	I use the app anywhere…I don’t care who knows that I’m HIV positive. You either like me or you don’t.
	Negative	No comment
	Neutral	It didn’t make a difference to me. I didn’t do it the same time, but I tried to do it basically the majority of the times when my alarm went off for me to take my medicine in the morning.
E-4. Additional languages (e.g., Spanish)
Intervention group	Positive	No comment
	Negative	There is a huge Latin community in New York City living with HIV. And, a lot of them don’t speak English very well. They stick with their language. Spanish is very important.
	Neutral	No comment
Control group	Positive	No comment
	Negative	No comment
	Neutral	No comment

Participants in the intervention group thought the mVIP app provided useful self-care strategies which were helpful for self-managing their HIV-related symptoms. The control group participants also reported needing information about symptom management, and they pointed out a lack of information on how to manage the symptom reported since the app provided to them did not suggest any strategies. One participant in the control group stated that, “I think the app lacks referral services. When you have a question with somebody experiencing depression and they could be experiencing it 5 days or more. That could be serious…You should have something there where they say, “If this is going on for the past 4 days maybe you need to seek medical attention or call 911” or something.” Nonetheless, the control group participants perceived the app to be useful because participants could self-monitor their symptoms using the app. One participant stated that, “I found the app to help in terms of weekly stress because stress has very affected on everybody, including myself. So, it was important to be able to pinpoint the different type of stress that were happening. You should know what issues may occur. To be alert. To be aware.”

Several participants in both groups discussed the convenience of using the app at a place and time they preferred. An intervention group participant described that, “It’s very convenient because I can use it almost anywhere. While I’m in public transportation, on the buses, at the clinic, at home…everywhere.” On the other hand, a participant in the intervention group disclosed housing instability as a barrier related to the app use stating that, “I only used app sometime…I couldn’t use it because I had to move again and again. I didn’t have a place to stay…”

Moreover, having additional languages was identified as one of the enabling factors. A participant suggested that the app be offered in additional languages such as Spanish, where the participant specifically questioned, “That is only for the study? Or, you’re going to try to introduce that to the people? You know why? Because I suggest to use 2 languages.”

3.2.3. Reinforcing factors

The five themes related to reinforcing factors were: 1) communication with healthcare providers; 2) individual-tailored information visualization; 3) social networking; 4) individual-tailored information quality; and 5) improvement in quality of life. The themes and quotes of reinforcing factors are reported in Table 5.

Table 5.

Themes and Quotes of Reinforcing Factors

Factor	Code	Quotes
R-1. Communication with healthcare providers
Intervention group	Positive	I told my therapist/my psychiatrist. She thought it was good. I told her it was helping me with my anxiety and she was curious about it (my symptom).
	Negative	No comment
	Neutral	I was discussing with my doctor about the app because, coincidentally, my visit to my doctor was during the study and she was just impressed about the app. That’s it.
Control group	Positive	It would be great if the doctors had access to see how often things are going on. I can show her this has been going on for a whole month, such and such. And she will see that he or she is responsible about taking care of themselves and what’s going on.
	Negative	No comment
	Neutral	No comment
R-2. Individual-tailored information visualization
Intervention group	Positive	No comment
	Negative	No comment
	Neutral	No comment
Control group	Positive	No comment
	Negative	We were in a study for 3 months, and there could be the graphic information of what we pressed every day or every week, I could bring that to my doctor…that’s just my suggestion.
	Neutral	No comment
R-3. Social networking
Intervention group	Positive	I showed a couple of my friends and I even gave them a couple of suggestions that the app gave me. They seemed to like it. It worked for them, so it was cool.
	Negative	No comment
	Neutral	I definitely showed it to my cousin. I was doing it there and she’s like, “That’s too many questions.” I was just showing her what I got, you know, showed her it and stuff and she said, “That’s too many questions.”
Control group	Positive	No comment
	Negative	I wanted to relate to other people on the app without the avatar. I wanted to send them something back like, “I was feeling the same.”
	Neutral	No comment
R-4. Individual-tailored information quality
Intervention group	Positive	I think it was pretty cool. The way it got me to switch out my routine a little bit. My sleeping routine…It kind of changed my routine up a little bit each time I logged in and I did the app. It was like a refresher to myself. It helped me.
	Negative	The app was less personal. It was just repetitive and I knew what was coming. We’re all different and we may have the same status but we all have our own things that we’re dealing with in life.
	Neutral	I take medication for other areas, but as far as the app, I basically just took what I needed from it and the rest that didn’t apply.
Control group	Positive	I like the app because there were some things that were listed in the app that I had no idea, you know, that were related to my HIV. So, it caused me to listen more closely to what’s going on with me. So, I find the app very helpful.
	Negative	But, there’s other health issues, too, that you all can put up there…The only thing I saw about diabetes was the tingling. There are other symptoms from diabetes, like you said, the blurred vision. I did it really quick. It was sort of too short.
	Neutral	I like collaborating on things that were said through others that took the app. I relate what you’re saying. The app has things that each of us go through.
R-5. Improvement in quality of life
Intervention group	Positive	No comment
	Negative	No comment
	Neutral	No comment
Control group	Positive	What I like about the app is that, say when it got to the symptoms like what bothers me a lot…this app made me more aware in life. You know, we have a lot to deal with HIV.
	Negative	No comment
	Neutral	No comment

Several participants in both groups thought that the mVIP app could help them to better interact with their healthcare providers if they could share the information about their symptom status with a review function within the app. Users in the intervention group were able to review all of the self-care strategies when they used the app. This was noted as being useful since participants reported being able to better communicate their symptom experience to their healthcare providers. One participant described that, “I would discuss it with my doctor about that situation. I say, ‘Look at the app. Look at what happened to me. Look about my sleeping problem. Maybe we can change something in my medication.’” Control group participants would also like to share their symptom status information with their healthcare providers suggesting individual-tailored information visualization such as a symptom report/summary. One participant stated that, “The one thing I want to say…about the graphic report of your symptoms, that would be great because then you can bring that to your doctor to share…”

In addition to communication with healthcare providers, participants in both groups mentioned the potential of the app for use as a social networking tool. Particularly, several participants in the control group thought that additional functionality of social networking to share feelings with other PLWH would be helpful. One participant said that, “I think you should create a network where we can network among each other with the app. You know you can respond to someone and you can say, ‘You know what? I’m feeling the same exact way today.’ I think that’s what the app needs for me.”

Regarding the quality of information provided within the app, several participants in the intervention group mentioned that it would be important for the app to be tailored to patient groups of varying years of diagnosis and to be tailored to more sensitive individuals. As HIV is considered a chronic disease, PLWH have been living with the common symptoms in their daily lives and have already tried various self-care strategies. They highlighted their desires for mVIP to include more symptoms and self-management strategies. One participant elucidated that, “If you just got diagnosed with HIV between 1–5 years, this (strategy) is perfect. I’ve been diagnosed since 1989. There were a lot of suggestions that did work, but we can’t just stick with one. More helpful hints. More quantity of suggestions and strategies for us (who were diagnosed with HIV a long time ago).” In addition, participants expressed their expectations for a more individualized self-tailoring symptom management app stating that, “I think it should be more personalized, like we said before because we all are different. I mean, you know. I may have something that they don’t have. They may have something I don’t have. Even though we have the same status, we go through daily different things.”

Participants in the control group reported improvements in their quality of life through use of the app. A participant mentioned that, “The question, ‘have you had difficulty falling asleep’ that was consistently true almost 3 or 4 days out of the week. One of the things that made me think, looking at my behavioral patterns, and now I literally will shut down my computer, I’ll turn off the TV at least 15 or 20 min before going to bed, and now I haven’t had that problem at all. Better life!”

4. DISCUSSION

Despite the rapid proliferation and widespread uptake of mHealth apps, there is little research assessing user engagement of apps in their everyday lives. To understand the usability of an HIV self-management app in a real-world setting, we conducted four follow-up focus groups at the end of a 3-month feasibility trial. We examined the factors influencing app use and its acceptance by the end-users. Guided by the PRECEDE component of the PRECEDE-PROCEED framework, five themes related to predisposing factors were identified as ease of app use, user-friendly functionality, self-efficacy for symptom management, design preference of illustrated strategies with videos, and user-control; four themes related to enabling factors were identified as information needs of symptom management, symptom-tracking, fit in lifestyle/schedule/living conditions, and additional languages; and five themes related to reinforcing factors were identified as communication with healthcare providers, individual-tailored information visualization, social networking, individual-tailored information quality, and improvement in quality of life.

Usability of a system is closely linked to the interaction of users performing tasks in the system within a specified environment. A change in any of the components of user, task, system, and environment may change the entire interaction and influence the usability of the system.[31] It is imperative to take usability problems into consideration throughout the process of system development and deployment;[32] however, studies on mHealth apps mostly evaluate their usability in a laboratory setting. This approach may overlook usability issues related to the actual interaction between user, task, system, and environment.[16] In this paper, usability evaluation in a real-world setting enabled us to measure users’ actual experiences when interacting with the app during their everyday lives. This is an important strength of our study.

In our study, being a good fit in users’ lifestyle/schedule/living conditions was one of the enabling factors influencing user’s routine app use. The living conditions (e.g., housing instability) and low-income status are relevant issues that may make it more difficult for these persons to self-manage their health. Poverty-related barriers have been cited as barriers to health-related outcomes in PLWH. For example, lack of transportation and food insecurity have been shown to influence the extent to which PLWH adhere to their HIV medications.[33, 34] mHealth apps can be a flexible tool that easily fit into a persons’ lifestyle/schedule/living conditions which may facilitate access to services.[35] This has important, real-world implications, highlighting the usefulness of mobile technologies for improving health outcomes in low-income persons. Additional implications of this work suggest the need for accommodating structural barriers in the lives of low-income persons and conducting pragmatic trials [36] which allows for flexibility in scheduling study visits, providing nutritious snacks, and offering appropriate social referral information when needed or requested.

There are a number of critical factors that may influence technology use. Research suggests two key constructs are especially important: perceived usefulness and perceived ease of use.[37] First, people tend to use technology to the extent they perceive it to be useful. Second, potential users may believe that the technology is useful, but perceive that it is too hard to use and that the performance benefits of usage are outweighed by the effort of technology use. In our focus groups, themes identified as enabling factors were more likely to relate to perceived app’s usefulness, and those identified as predisposing factors were more like to relate to perceived ease of the app use. In addition to the two constructs, themes identified as reinforcing factors related to users’ expectations and desired functionality of the app. Findings from the focus groups contribute to the literature of users’ technology acceptance in healthcare, and importantly, suggest that end-users’ expectations and desired functionality of the app in real-world situations be considered as a factor influencing technology use in future research.

Participants perceived the app to be useful for self-managing their symptoms in the intervention group and facilitate self-awareness of symptoms in the control group. Given that the control group participants were not provided with self-care strategies, it was not surprising that the control group participants pointed out the lack of information related to symptom management. These findings from our focus group sessions indicate the identification of information needs for self-management related to symptoms in PLWH. Furthermore, the findings highlight the importance of the app’s usability to identify users’ information needs in terms of a mobile intervention delivering health information.

Key findings identified as reinforcing factors were users’ suggestions of a summary of self-reported symptoms, specifically, visualization using graphs or charts. Participants wanted symptom charts to share with their providers in an effort to improve their communication with their providers. Future research investigating end-user’s abilities and preferences regarding the best visual format to present symptom reports is recommended.

4.1. Limitations

The generalizability of the results may be limited by the study sample. While more than 75% of PLWH are male in the US,[38] the study sample in this study predominantly included females (71% in the intervention; 42% in the control), which needs future research with inclusion of males. Also, in the US a substantial number of underserved PLWH are primary-Spanish speakers [39, 40] and we unfortunately could not include them in this trial because the mVIP app is currently only available in English. There might be unique aspects of HIV management in other groups that limits the generalizability of our findings. Our targeted population was underserved PLWH, specifically racial and ethnic minorities and those from low-socioeconomic groups who have low income, and low level of education. Of particular relevance to our study population, use of mobile technology can improve health information delivery to the underserved communities in a real-world setting. Ownership of a mobile device is equally as common among Blacks (94%), Whites (98%) and Hispanics (97%).[41] 90% of Americans with less than high school graduate and 92% of Americans with an income less than $30,000 own a mobile device. Using mobile tools at high rates across racial/ethnic and low-socioeconomic groups supports the use of mHealth apps for bridging some of the current disparities in health outcomes.

5. CONCLUSIONS

In this paper, we identified the predisposing, enabling, and reinforcing factors influencing app use and its uptake by conducting an in-depth analysis of PLWH’s experience using the app during their everyday lives. Our work highlights the importance of using mobile technology for PLWH, specifically for those persons from low-income groups, and adds to the body of literature on the implementation and dissemination of mobile technology interventions for improving outcomes in PLWH.

Summary Points:

• mHealth apps have successfully been used for self-management by persons living with chronic conditions and have successfully improved health outcomes.

• Usability of mHealth technology should be assessed not only in a laboratory setting but also in a real-world setting.

• Usability evaluation in a real-world setting enabled us to measure users’ actual experiences when interacting with the app during their everyday lives.

• mHealth has the potential for implementation and dissemination of symptom self-management strategies through a mobile platform for persons living with HIV, specifically those with low income/housing instability.