mHealth Intervention Elements and User Characteristics Determine Utility: A Mixed-Methods Analysis

Abstract

Background: Mobile health (mHealth) interventions are improving the medication adherence of adults with type 2 diabetes mellitus (T2DM), but few studies examine how users experience these interventions. Therefore, we used a mixed-methods approach to understand how T2DM users experience a text messaging and interactive voice response (IVR)-delivered medication adherence intervention called MEssaging for Diabetes (MED).

Methods: Adults with T2DM used MED as part of a 3-month pilot study. MED sends daily tailored text messages addressing adherence barriers, daily assessment text messages asking about adherence, and weekly tailored IVR calls providing adherence feedback, encouragement, and questions to facilitate problem solving. Sixty participants completed feedback interviews. We used a mixed-methods approach to understand their experience, examining associations between participants' characteristics and their feedback.

Results: Participants who completed feedback interviews were on average 50.0 ± 10.1 years old; 65% female, 62% non-white; 15% had less than a high school education, 70% had annual incomes less than $20K; and average hemoglobin A1c was 8.0% ± 1.9%. Participants rated each intervention element favorably; common reasons for MED's helpfulness included receiving novel information about diabetes medications, emotional support, and reminders to take medication. People who were younger and more recently diagnosed with T2DM had more favorable experiences using MED. In general, users valued text messages more than IVR calls.

Conclusions: Consideration of the user experience is critical for developing engaging mHealth interventions. User feedback reveals what mHealth elements have the most value and why, which users to target, and how to optimize an intervention's utility and appeal.

Introduction

Alarge percentage of adults with type 2 diabetes mellitus (T2DM) have difficulty adhering to medications.¹ Medication nonadherence is associated with having suboptimal glycemic control.² Mobile health (mHealth) interventions using basic cell phone technology (i.e., text messages and voice communications) may improve the medication adherence and glycemic control of people with diabetes.^3,4 However, we need to understand users' acceptance and opinions of mHealth interventions to optimize benefit.^5,6

Users' opinions of mHealth interventions can reveal why an intervention or certain parts of it are helpful. For example, when Nundy et al. collected user feedback on an automated text messaging intervention, reasons for helpfulness went beyond being reminded to perform self-care.⁷ Users said the intervention made them more aware of their self-care behaviors, helped them accept the seriousness of their diabetes, and provided emotional support.⁷ In addition, users of Burner et al.'s text messaging intervention said messages cueing a behavioral action were most helpful because the prompts were concrete and motivating.⁸ Such feedback tells us how an intervention personally affects users and prioritizes what to target in future interventions.

Collecting user feedback can also inform why people did not use an intervention,⁹ and what might interfere with an intervention's efficacy.¹⁰ Lyles et al. interviewed T2DM users about their experience with a mobile- and web-based collaborative care intervention.¹¹ Although users valued the ease of uploading blood glucose readings from their meter to their mobile phone, they experienced technical difficulties with the smartphones they were given for the study.¹¹ Without learning about and addressing this type of feedback, technical issues may prevent users from experiencing an intervention's value.

Certain types of users may find an intervention or its distinct parts more or less beneficial. Understanding who likes what is the first step to better meet the needs of users, facilitating better engagement, and, ultimately, efficacy. Quantitative studies report that being younger,¹² more educated,¹² and male¹³ is associated with being more engaged with a diabetes mHealth intervention. However, this research does not account for nuanced reasons for varied engagement, such as why users who have more positive opinions of an intervention choose to use it more. Qualitative studies linking user characteristics to a favorable user experience have been underrepresented in the literature.¹⁴

To design efficacious mHealth interventions, we first need to understand users' experiences with these interventions and what would be more or less useful among specific types of users. To date, the majority of studies examining diabetes users' opinions of mHealth interventions have assessed helpfulness and satisfaction quantitatively with close-ended self-report scales.^14,15 Although some studies have collected qualitative feedback, study samples have been small^8,11,16 or based on a short intervention duration (i.e., 2–4 weeks).^7,17 Therefore, we conducted interviews using both quantitative and qualitative questions to understand T2DM users' experiences with and opinions of a 3-month text messaging and interactive voice response (IVR) intervention called MEssaging for Diabetes (MED).

Research Design and Methods

MED intervention

MED uses the SuperEgo mobile communications platform¹⁸ to tailor and deliver medication adherence promotion text messages and IVR calls to people with diabetes.^17,19 Users receive a daily tailored text message at a random time each day addressing 1 of their 3 highest ranked barriers to adherence (out of 17 assessed). Barriers are reassessed monthly to update the user experience. Depending on what barriers rank highest each month, a user can receive messages addressing three new barriers or one or more of the same barriers previously addressed. Users reporting less than three barriers have barriers randomly selected. The daily tailored text messages addressing adherence barriers were created by two behavioral psychologists and three clinicians, including a diabetes educator and practitioner.

MED users also receive a daily assessment text message at the end of each day asking, “On [day of week, date], did you take ALL your diabetes meds? Reply with ‘YES’ or ‘NO’.” Finally, users receive a weekly IVR call delivering tailored adherence feedback based on “YES” replies to the daily assessment text messages (e.g., “This week you told us you took your diabetes medications as prescribed on four days. Last week you told us you took your diabetes medications as prescribed on three days.”). This feedback is followed by an encouraging message (e.g., “Based on what you told us, you've taken your diabetes medications as prescribed on more days this week. Keep up the good work!”).

To facilitate problem solving, the IVR call also asks users to describe a day when they were successful and a day when they were unsuccessful taking their medications and if they experienced any problems receiving or responding to text messages. If users miss this call, they can call back before midnight to receive their feedback and encouraging message, and respond to the problem-solving questions. This call information is no longer available after midnight on the day a call is made. For a full description of how MED was developed, including system and content development, see Osborn and Mulvaney.¹⁷

Participants and recruitment

We recruited 80 adults with a T2DM diagnosis from a Federally Qualified Health Center (FQHC) in Nashville, TN.¹⁹ We used a combination of advertisements, and worked with clinic staff to identify eligible persons who were at least 18 years of age, were English speaking, were prescribed diabetes medication(s), and had a cell phone with text message capabilities. Exclusion criteria included a preexisting diagnosis of dementia, auditory limitations, and the inability to see and respond to text messages. Trained research assistants (RAs) met with interested persons to describe the study and screen for eligibility. This included sending a test text message to each potential participant to make sure they could see and read the message, and successfully respond to it. The Vanderbilt University Institutional Review Board approved all study procedures before participant enrollment.

Procedures

After completing informed consent, RAs collected participants' self-reported information, including a measure assessing barriers to medication adherence.¹⁷ A clinic phlebotomist performed a blood drawn hemoglobin A1c (HbA1c) test, and RAs reviewed participants' medical records to collect relevant clinical data. Following this appointment, participants received the MED intervention for 3 months. RAs called participants at one and 2 month(s) to reassess barriers to adherence for the purpose of updating the daily tailored text messages and to solicit feedback on participants' experience with MED thus far. Feedback questions were quantitative and qualitative. At 3 months, RAs met with participants in person to obtain this feedback. Interviews were 30–45 minutes in duration.

Measures

Demographic characteristics

At baseline, we collected self-reported age, gender, race, ethnicity, income, education (i.e., years in school), insurance status, and asked participants if they were comfortable using their cell phone.

Clinical characteristics

At baseline, participants self-reported diabetes duration (i.e., years/months since a diabetes diagnosis) and the number of diabetes medications prescribed, including insulin. RAs reviewed each participant's medical record to confirm a T2DM diagnosis and the quantity of prescribed medications.

Health literacy

The three-item Brief Health Literacy Screen (BHLS) assessed participants' health literacy.²⁰ Two items asked, “How often do you have someone help you read hospital materials?” and, “How often do you have problems learning about your medical condition because of difficulty understanding written information?” Response options ranged from 1 = “never” to 5 = “always.” The third item asked, “How confident are you filling out medical forms by yourself?” Response options ranged from 1 = “not at all” to 5 = “extremely”. Responses to the first two items were reverse scored, and all three responses were summed to create a score ranging from 3 to 15, with higher scores indicating better health literacy.

Quantitative feedback

As shown in Table 1, we included close-ended, yes/no items assessing MED's ease of use, users' acceptance of MED, and users' understanding of the text message and IVR content. Likert items assessed the helpfulness of the daily tailored text messages and weekly IVR calls in getting users to take their diabetes medications. Response options ranged from 1 = “not at all” to 10 = “a lot.”

Table 1.

Quantitative Feedback Interview Items

Item format	Item content
Close ended (yes/no)
Daily tailored text messages	Did you read the messages every day?
	Were the messages easy to understand?
	Did it seem like the messages were directed at issues that were relevant to you?
Daily assessment text messages	Would you continue responding to these messages for longer than 3 months?
Weekly IVR calls	Were the choices in the recordings easy to understand?
	Did you understand the statement telling you about how often you took your diabetes medication?
	Did you have trouble understanding the questions on the call?
	Did the information help you to get insight about how often you actually take your medication?
Likert itemsa
Daily tailored text messages	How much did the personal text messages help you in taking your diabetes medication?
Weekly IVR calls	How much did the recorded phone call help you in taking your diabetes medication?

^aResponse options for Likert items ranged from 1 = “not at all” to 10 = “a lot.”

IVR, interactive voice response.

Qualitative feedback

Open-ended items asked participants to discuss and describe what they liked and did not like about MED, explain how and why a specific aspect of MED was or was not helpful, and provide suggestions for improving MED. All interviews were semistructured, and, therefore, semiemergent. RAs took notes on participants' responses.

Analyses

Sixty participants provided MED feedback, and 20 did not. We report participants' feedback at their longest exposure to MED (i.e., 4 at 1 month, 4 at 2 months, and 52 at 3 months). All statistical tests were performed using SPSS version 23. Descriptive statistics characterized the sample, and Mann–Whitney U tests and chi-square tests examined differences between participants who completed and did not complete a follow-up interview.

We transcribed participants' responses to the interview questions and used inductive content analysis to identify patterns and categories from our data.^21,22 First, while reading through responses, we identified key words, phrases, or concepts used by participants. Once responses were coded into recurring concepts, we sorted and grouped categories of similar content to create higher order categories. Using an iterative process, we went back and forth between our data and our codes to review and refine emergent categories. To ensure the reliability of our analysis, a second rater performed these same steps, and we compared categories between raters. We reconciled discrepancies through discussion until consensus was reached.

We tested associations between participants' characteristics and our categories using Spearman's rho correlation coefficients (ρ), Mann–Whitney U tests, or chi-square tests, as appropriate. We also used frequencies and percentages to summarize participants' comments and subsequent categories.

Results

We previously reported baseline sample characteristics for the 80 participants who received MED.¹⁹ Table 2 reports these characteristics by participants who did and did not complete a follow-up interview. Of the 60 participants who completed the interview, the average age was 50.0 ± 10.1 years old; 65% were female and 62% were non-white (95% African-American); 15% had less than a high school degree or equivalent; 70% had annual household incomes less than $20K; and 83% were either uninsured or had public insurance. Over half of the sample (63%) was prescribed insulin and the average baseline HbA1c was 8.0% ± 1.9%.

Table 2.

Patient Characteristics (n = 80)

	M ± SD or n (%) or IQR
Variable	Completed interview (n = 60)	Did not complete interview (n = 20)	P-value
Age, years	49.95 ± 10.08	50.45 ± 12.05	0.56
Gender			0.58a
Male	21 (35)	5 (25)
Female	39 (65)	15 (75)
Race			<0.05a
Caucasian/white	23 (38.3)	2 (10)
Non-Caucasian/non-white	37 (61.7)	18 (90)
Education, years	13.22 ± 2.35	12.10 ± 1.92	0.05
Annual household income (in $US)			0.24
<10,000	19 (31.7)	10 (50)
10,000–20,000	23 (38.3)	4 (20)
>20,000	18 (30)	6 (30)
Insurance status			0.74
Private	10 (16.7)	4 (20)
Public	30 (50)	8 (40)
None	20 (33.3)	8 (40)
Comfortable with using cell phone	60 (100)	19 (95)	0.56
Diabetes duration, years	9.87 ± 6.66	8.70 ± 5.74	0.37
Number of prescribed diabetes medications	1.9 ± 0.75	2.0 ± 0.79	0.62
One	20 (33.3)	6 (30)
Two	26 (43.3)	8 (40)
Three	14 (23.3)	6 (30)
Insulin status, taking insulin	38 (63.3)	13 (65)	1.00a
Glycemic control (HbA1c, %)	7.99 ± 1.87	9.02 ± 2.20	0.06
Optimal (<7.0%)	21 (35)	5 (25)
Suboptimal (≥7.0%)	39 (65)	15 (75)
Health literacy (BHLS)	9.88 ± 2.53	7.80 ± 3.33	<0.01

Non-Caucasian/non-white participants were majority (94.6%) African American.

^aYates' Correction for Continuity.

BHLS, Brief Health Literacy Screen; HbA1c, hemoglobin A1c.

Participants who did not provide feedback on MED (n = 20) did not differ from those who did (n = 60) on age, gender, income, education, insurance status, cell phone comfort, insulin use, number of prescribed diabetes medications, diabetes duration, or HbA1c. However, white participants were more likely to provide feedback than non-whites, χ² (1) = 4.36, P < 0.05, and participants who provided feedback had higher health literacy (U = 360.50, z = −2.74, P = 0.006) than participants who did not provide feedback (Table 2).

In the follow sections, we organize and present participants' opinions of MED by intervention component: (1) daily tailored text messages, (2) daily assessment text messages, and (3) weekly tailored IVR calls. We also present associations between participants' characteristics and their comments.

Daily tailored text messages

MED sends users one unique daily tailored text message addressing user-specific barriers to medication adherence. The vast majority of participants (90%) read this message every day and 98.3% said these messages were easy to understand. The majority (82.1%) reported that these text messages were personally relevant. On average, participants rated these text messages a 7.65 ± 3.23 out of 10 in helping them take their diabetes medications. Both younger age (P < 0.08) and less years of education (P < 0.07) were marginally associated with rating these messages more helpful.

To understand the type of messages participants found most useful, we asked them to tell us about one helpful message they received. One quarter of participants (25%) said that messages addressing the cost of medications were helpful, including this participant who commented on the message's practical advice:

[One text message was]: If you're having trouble paying [for medication], set aside money. [So] I had a jar [for] saving money that I used for my medications. (51-year-old, Caucasian/white, female)

In addition, 33.3% of participants said messages suggesting they talk with a doctor or pharmacist about their medications were helpful. These participants often said such messages encouraged them to take initiative, ask questions, and seek professional guidance with medication-related questions.

[Another text message was]: Talk to your doctor if you're not sure about something. [This] made me think to write questions down before my appointment. (52-year-old, Caucasian/white, male)

Similarly, 13.3% of participants said helpful messages offered strategies to make medication taking easier.

The text that helped me remember to take my meds [said to] put a reminder on my mirror. Another one advised [me] to check if where I buy groceries has a pharmacy. (37-year-old, African American/black, female)

Participants also appreciated text messages that increased their knowledge of diabetes medications, encouraged them to talk with family and friends about their diabetes, and helped them overcome embarrassment associated with taking medications in public. This participant shared how a message made her feel more comfortable taking insulin around others:

I have always been apprehensive about taking insulin in front of friends and family, but the text [messages] encouraged me to talk to a friend to help me if I have a problem. (60-year-old, Caucasian/white, female)

When asked what made certain messages more or less helpful, participants (31.7%) said helpful messages delivered novel information about medications. For instance, one participant said such messages helped him acclimate to taking insulin:

[The texts] were especially helpful because taking insulin was new to me. (44-year-old, Caucasian/white, male)

Some participants (15%) said tailored messages helped reinforce the importance of taking medications. For example, participants said messages helped with overcoming burnout from taking medications and were a constant reminder of why they should take their medications.

[The text messages had] very good information. [They] helped on days I was tired of taking my meds. (48-year-old, African American/black, female)

[The messages] helped me become more aware of why I should take [my] meds. (58-year-old, Caucasian/white, male)

Participants (26.7%) also said messages were uplifting and provided emotional support for adherence. Participants described struggling with regularly taking their medications and said messages helped inspire them to be more adherent:

Some days I am disgusted with my diabetes, but when the messages came through, they gave me a positive outlook. (52-year-old, Caucasian/white, female)

[The messages were] very helpful, informative, and encouraging. It felt like I had my own personal cheerleader. (34-year-old, Caucasian/white, male)

When we asked participants to describe an unhelpful message, 16.7% said all messages were helpful, with the majority saying no messages were unhelpful (60%). However, 23.3% said at least one message was unhelpful. Being more educated (U = 177.5, P < 0.05) and having diabetes longer (U = 179.0, P < 0.05) were associated with reporting at least one unhelpful message. Unhelpful messages were about affording medications, getting to the pharmacy, and not forgetting to pick up refills. In several cases, participants said unhelpful messages simply did not apply to them.

Texts about costs were not helpful because I am on Medicare and everything was covered. (69-year-old, Caucasian/white, female)

[The texts were] not helpful to me because I've had diabetes for so long, but [they] could be very helpful to those just diagnosed. (64-year-old, Caucasian/white, female)

Participants suggested ways to improve tailored text messages. Many said these messages became redundant and requested future messages focus on other self-care behaviors, in addition to medication taking. They asked for messages promoting healthful eating and physical activity.

Tips on nutrition and other ways to manage diabetes would be helpful. (29-year-old, African American/black, female)

They also requested messages that inspired, empowered, and motivated them to take care of themselves.

I wanted more variation in messages and to sometimes get messages not only [intended] for barriers. I needed more encouragement rather than repeating the same facts. (54-year-old, Caucasian/white, male)

Participants diagnosed with diabetes for a long time also wanted messages to provide more cutting-edge news (e.g., new tips on how to make taking their medication easier). Others wanted text messages to refer them to websites (e.g., diabetes websites with more comprehensive information).

Texts would be more helpful if they had links to resources, like [where] to get medications cheaper instead of just saying “talk to your doctor.” (35-year-old, Hispanic, female)

Daily assessment text messages

MED sends users one assessment text message at the end of each day asking if they took all of their diabetes medications that day, and to reply with “YES” or “NO.” Responses to this question were aggregated weekly and delivered as feedback during the weekly IVR call. We asked participants if daily assessment messages helped them take their medications and found four common opinions. The majority of participants (66.7%) said these messages were an extra reminder to take their medications. If they had not yet taken an evening dose of medication, this message prompted participants to take it. Message timing facilitated that.

When [those text messages] came, I would be getting ready to take my bedtime dose, so it helped me remember [to take it], if I was already sleepy. (36-year-old, African American/black, female)

[If I was] sitting around watching TV, [I] didn't realize what time it was, [and the] message came through that I needed to take my meds, [so I did]. (25-year-old, African American/black, female)

Some participants (15%) said it was important for them to respond “YES” to this message because it felt like they had failed if they replied “NO.”

It was a goal that I would be able to answer yes when the text was coming, and I always knew it was coming. (54-year-old, Caucasian/white, male)

It made me remember because I thought it was coming and I didn't want to lie, so I took [my] meds every day. (69-year-old, Caucasian/white, female)

[I] got tired and ashamed of saying no. (53-year-old, African American/black, female)

Other participants (15%) said assessment text messages acted like a “check-in,” keeping them on track and accountable, and made them feel like someone cared.

At night, if I forgot, I would take it when I got the text. It's like AA [i.e., referring to Alcoholics Anonymous], even though I didn't see anybody, I felt a part of a program for diabetics. (52-year-old, “Other” race, male)

Finally, participants (15%) said assessment text messages were not particularly helpful because they had an established habit of taking all of their medications every day. One participant suggested these messages might be more helpful to people newly diagnosed with diabetes or newly prescribed medications:

I had a routine, so it didn't hurt, but I would have taken it regardless. Perfect for “newbies” who aren't used to taking their medication on a routine or when you get a new prescription. (52-year-old, Caucasian/white male)

The majority of participants (89.3%) would continue responding to the daily assessment text messages every day for longer than 3 months. However, four participants said it became tedious to respond to these messages every day, and it would have been more helpful if a weekly message was sent instead of a daily one.

Weekly IVR calls

MED called users (through IVR) at the end of each week at a user-specified time to provide adherence feedback based on the user's responses to the daily assessment text messages. Following this feedback, participants heard an encouraging message and were asked to respond to problem-solving questions described above. The majority of participants (96.7%) knew how to participate on the call, 98.3% understood the adherence feedback, and 91.2% understood the problem-solving questions. On average, participants rated the IVR calls a 7.96 ± 2.95 out of 10 in helping them take their diabetes medications. In addition, being younger (ρ = 0.27, P < 0.05) and newer to diabetes (ρ = −0.29, P < 0.05) were each associated with rating the IVR call more favorably.

When participants were asked to discuss why the calls were helpful or unhelpful, favorable and unfavorable opinions emerged. Participants (31%) said the information helped them be more aware of and reflect on how often they took their medications.

[The feedback] helped me to stop and think about my medication routine. (61-year-old, African American/black, male)

[The feedback] made me think. [It] kept me on schedule [and] more focused. (52-year-old, Caucasian/white, male)

Yes, it was useful. [The feedback] helped me realize how often I was missing my medication. (51-year-old, African American/black, female)

Others said the call motivated them. After participants (31%) heard adherence feedback, they were motivated to do better the next week, and the IVR's subsequent encouraging message provided additional support.

Yes, [the call] gave me [the] number of days I took meds, [and I] tried to improve each week. (47-year-old, Caucasian/white, female)

It was encouraging. I wanted that pat on the back. Hearing [the feedback] is exciting. (37-year-old, African American/black, female)

I enjoyed the reward part of it, the encouragement. As diabetic patients, we always hear when we're not doing [what's] right. [The calls] told me when I was doing a good job. (34-year-old, African American/black, female)

Finally, some participants said the IVR call was not helpful to them. When asked if the call's information helped them know how often they took their medications, 23.7% of participants said it did not. These participants were older than participants who said the calls did help them know this information (U = 173.0, P < 0.05). There appeared to be four primary reasons why participants felt IVR feedback was not helpful. Some participants said they already knew how often they took their medications, so getting adherence feedback was uninformative.

[I already] know when I do and when I don't take [my] medications, so it [didn't] really make a difference. (58-year-old, African American/black, female)

[The calls were] not really helpful because I take [my] meds every day. (64-year-old, Caucasian/white, female)

Some participants said the adherence feedback was inaccurate and did not correspond with the amount of days they had actually taken their medications.

For the last month [the feedback] was incorrect and said I had not taken my medications all seven days even though I texted back every day [that I had]. (58-year-old, Caucasian/white, female)

[The call wasn't helpful] because it was incorrect. I was keeping track myself though. (69-year-old, Caucasian/white, female)

Other participants said the calls became redundant and bothersome because they were also being asked to respond to daily assessment text messages. In other words, although the call provided different information from the daily assessment text message (e.g., adherence feedback, an encouraging message), both requested participants to respond to questions concerning their medication-taking behavior.

[I] already take [my] medications. [I] anticipated the text, but [the] phone call was like answering twice. (50-year-old, African American/black, female)

Last, participants reported issues receiving their feedback, either because they did not receive the call or had issues with calling back when they missed the call. Many of these issues were due to technical problems with the IVR system.

[I] don't think I got [a call] every week. [I] could have missed some. (52-year-old, African American/black, male)

[I was] always busy when I got the call. (51-year-old, African American/black, female)

I tried to call [it] back and it didn't work. (48-year-old, African American/black, female)

I called back one time and [the system] said “You don't have any surveys at this time.” (54-year-old, Caucasian/white, male)

Discussion

mHealth interventions are improving the medication adherence of patients with chronic health conditions, including T2DM.^6,23 To optimize their utility (i.e., their value and/or helpfulness), we need to understand patients' opinions of mHealth interventions. However, there is limited qualitative research involving both large samples and interventions longer than 1 month. Therefore, we interviewed patients with T2DM about their experience using the MED text messaging and IVR-delivered medication adherence intervention. Participants rated MED favorably, particularly patients who were younger and more recently diagnosed with T2DM. Overall, users gave different reasons for why certain parts of the intervention were more or less helpful, which helps us to optimize the utility, and, in turn, efficacy of future mHealth interventions.

Users said the two different types of MED-delivered text messages were helpful. Daily tailored text messages addressing barriers to adherence gave new medication information, reinforced the importance of medication taking, and provided emotional support. The purpose of the daily assessment text message was to obtain adherence data that could be aggregated and delivered as feedback in the weekly IVR call. However, most MED users (66.7%) said this daily text message served as a reminder to take medications, promoting adherence in and of itself. Dick et al. interviewed diabetes patients on their opinions of SMS-DMCare, a text messaging program with self-care adherence questions.²⁴ Similar to MED users, users of SMS-DMCare said receiving messages at the same time every day helped them maintain a routine for taking their medications²⁴; in addition, SMS-DMCare users said they began anticipating messages and this motivated them to respond in an affirmative way.²⁴

Although users rated MED favorably overall, they liked daily text messages more than weekly IVR calls. Some users said IVR calls increased their awareness of how often they took medications and their motivation to improve, but many users said these calls were not helpful largely due to technical issues. Technical problems can discourage continued use of an mHealth intervention, preventing potential benefits from being realized.^25,26 MED participants who missed an IVR call were frustrated when they called the system back and the 24-hour window to participate in the call expired. In contrast, if participants were busy when a text message came in, they were able to read the message at any time. To preserve the helpfulness of adherence feedback, this same information could be delivered through text messages in future mHealth interventions.

Participants who were younger and more recently diagnosed with T2DM rated MED more favorably. Dobson and Hall reported younger age is associated with more positive attitudes toward and intentions of using diabetes self-care technologies.²⁷ However, there is a paucity of intervention research linking other patient factors to an intervention's perceived value. It may be that younger adults enjoy mHealth interventions more in general because they are more likely to own a cell phone compared to older adults.²⁸ Similarly, people who are more recently diagnosed with diabetes may value medication tips and reminders more because they are still learning about T2DM.²⁹ Future mHealth studies should compare user opinions by age and duration of diabetes to determine the ideal target audience.

Participants also provided suggestions for improving MED's text messages. Many wanted tailored text messages to be more varied, and new messages to be added addressing other self-care behaviors such as diet and exercise. Although MED is a medication adherence intervention, more varied content may improve engagement. In addition, some participants suggested sending assessment messages less often. In a pilot study for a text message-based diabetes self-care program, participants felt it was necessary to send texts more frequently when starting the program, but less frequently later on.²⁴ To improve engagement and outcomes, mHealth interventions must tailor content and design to user preferences.^30,31

There are several limitations to our study. We could not reach 20 participants at follow-up; therefore, not everyone who experienced the intervention provided feedback on their experience. The difficulty with contacting these participants is likely due to the common challenge of retention in intervention trials³²; furthermore, recruiting participants from FQHCs may have made retention more difficult.³³ Interventions are often demanding and require sustained interest and time; if participants already experience many demands and stressors in their daily lives, the intervention may cause further distress and lead to withdrawal.³² Although white participants and higher health literate participants were more likely to provide feedback than non-white participants and lower literate participants, respectively, user feedback did not differ on any other characteristic, including education.

As an additional limitation, conducting the majority (86.7%) of follow-up interviews in person may have led to biased responses relative to phone interviews;³⁴ however, in-person interviews are superior for building rapport that is often beneficial for generating rich qualitative data.³⁵ In addition, our quantitative findings are limited to a mixed-methods approach. Other study designs that can more robustly link user characteristics to the mHealth intervention experience are needed. Finally, because we conducted our study at a single clinic site and feedback was based on one intervention, our findings may not generalize to other patient populations or interventions.

Conclusion

Despite evidence of mHealth interventions being effective for improving diabetes self-care, there is limited information on patients' opinions of these interventions. Using mixed methods to reveal nuanced information on patients' experiences can help design interventions that appeal to patients' needs and preferences.

Although MED is a text messaging and IVR-delivered intervention, our methods and findings are also applicable to the design of mobile applications (apps) for diabetes medication adherence. Mobile apps are growing in use and offer features that can further facilitate adherence³⁶; however, opinions of and engagement with an app may vary based on its specific features and users. When we determine which parts of interventions users find most useful, we can focus on developing and delivering those parts. When we determine what user characteristics determine value, we can focus on delivering our interventions to specific user groups most likely to benefit. To sustain user interest, diabetes mHealth interventions and apps should consider both targeting people with less experience caring for their diabetes and being inclusive of all self-care behaviors. Understanding user opinions of an intervention and linking user characteristics to these opinions can help design interventions and apps with the greatest potential to improve outcomes.

Acknowledgments

The authors thank Cecilia C. Quintero and Dr. Lindsay S. Mayberry for their role in data collection, and the Vine Hill Community Clinic personnel and the participants for their contributions to this research. We also thank Lauren LeStourgeon for her help with coding transcripts. The contents of this article are solely the responsibility of the authors and do not necessarily represent official views of the funding entities. This research was supported by a McKesson Foundation mHealth Award to Drs. Osborn and Mulvaney. Dr. Osborn was also supported by a career development award NIH/NIDDK K01-DK087894 and the Vanderbilt Center for Diabetes Translational Research P30-DK092986. Drs. Osborn and Nelson were both supported by NIH/NIDDK R01-DK100694.

Authors' Contributions

L.A.N. coded data, conducted analyses, and wrote the article. S.A.M. codesigned the study, oversaw data collection, administered the intervention platform, and edited the article. K.B.J. edited the article. C.Y.O. codesigned the study, oversaw data collection and analyses, and cowrote and edited the article.

Author Disclosure Statement

No competing financial interests exist.