Content Analysis: First-Time Patient User Challenges with Top-Rated Commercial Diabetes Apps

Abstract

Background/Introduction: Using a mobile application (app) may improve diabetes self-management. However, the use of diabetes apps is low, possibly due to design and usability issues. The purpose of this study was to identify barriers to app use among adult patients with diabetes who were testing diabetes apps for the first time.

Materials and Methods: We conducted a content analysis of observation notes and patient comments collected during the testing of two top commercially available diabetes apps as part of a crossover randomized trial. Participants were adult patients with type 1 or type 2 diabetes on insulin therapy. We analyzed field notes and transcriptions of audio recordings. Open coding derived categories of usability issues, which then were grouped into themes and subthemes on usability problem types.

Results: A total of 92 adult Android smartphone users were recruited online (e.g., Facebook) and in-person postings. Three major themes described problems with data input, app report display and presentation, and self-learning options. Data entry modes were problematic because of overcrowded app screens, complicated “save data” steps, and a lack of data entry confirmation. The app icons, wording, entry headings, and analysis reports were not intuitive to understand. Participants wanted self-learning options (e.g., pop-up messages) during app use.

Conclusions: Patient testing of top commercially available diabetes apps revealed key usability design issues in data entry, app report, and self-help learning options. Good app training for patients is necessary for both initial use and long-term use of diabetes apps to support self-management.

Introduction

According to the 2017 National Diabetes Statistics Report by the Centers for Disease Control and Prevention, 34.1 million adult Americans or 13.0% of all U.S. adults have type 1 or type 2 diabetes.¹ By 2021, a projected 40.3 million Americans will have diabetes, which will increase health care spending for diabetes from $206 to $512 billion.² Self-management is essential for improving patient outcomes,^3,4 reducing complications,⁵ and lowering the costs of the disease. According to the American Association of Diabetes Educators, 7 self-management behaviors improve diabetic care: healthy eating, being active, monitoring, taking medication, problem-solving, reducing risks, and healthy coping.⁶ Mobile health (m-health) apps⁷ are one way to help patients manage their diabetes and have been suggested as a promising strategy to improve self-management of diabetes for patients and their caregivers.^8–10 m-Health technology has been increasing in popularity with 318,000 m-health apps currently available.¹¹ One in 5 smartphone users downloads a health app.¹² Previous studies utilizing m-health technology to improve health behavior and outcomes have mixed results.^13,14

Development of health applications without attentiveness to usability reduces the efficacy of m-health apps.¹⁵ Studies reported a wide range of diabetes app usability rating.^16–18 Few studies have assessed the usability of current m-health apps, particularly from a patient end-user perspective.^16,19–21 Studies are needed to better understand patient engagement in using diabetes apps. Prior work on usability evaluation by experts²² and end users²³ using the System Usability Scale²⁴ indicated that the apps had substantial room for improvement in the end-user experience.

To address this gap between the ideal and the experienced usability, we undertook this study to identify barriers to successful app use and describe key app usability design important to patients in supporting diabetes self-management activities such as monitoring blood glucose (BG), carbohydrate (carb), and insulin dose.

Materials and Methods

DESIGN

This study is part of a parent quantitative study, a randomized crossover trial with two age-based strata, adults age ≥56 years and adults <56 years, that tested the usability of two Android apps conducted in 2017 (Ref.^23,25). The tested apps (OnTrack and mySugr) were listed as “the Best Diabetes Apps 2016” by Healthline, an online health forum.²⁶ This within-subject design (one group of patients to test two apps) allowed collection of two sets of usability observations. Testing was observed in person. Each participant tested both apps on an Android phone in a randomized testing order that a statistician randomly assigned using a computer software program. The Android platform was chosen because it was most frequently used (52.7%) in the market at that time.²⁷ The principal investigator (H.N.C.F.) took field notes describing user experiences and recorded participant comments during app use. User experience measuring usability with objective measurements by time, success, and accuracy rates was reported in the parent study publication.²³ For this study, we analyzed the field observations to identify key usability challenges and designs presenting as barriers during first-time app use. We aim to describe a phenomenon, the reaction, and experience of patients using diabetes apps for the first time.

PARTICIPANTS

Participants (n = 92) were recruited from flyer postings at a federal health qualifying health center, a veterans affairs (VA) clinic, a university campus, community bulletin boards, a diabetes support group meeting, and websites (Craigslist and Facebook) from July 2017 to November 2017. We screened interested individuals by phone and included them if they (1) were age 18 years or older, (2) had type 1 or type 2 diabetes, (3) have used an Android phone for 6 months or longer, (4) used insulin therapy for 6 months or longer, (5) have adequate English, and (6) are proficient with a smartphone and they use the device for more than just phone calls, e-mail, texting, or taking pictures. Individuals were excluded if they were unable to read or speak English, had used either app in this study (OnTrack or mySugr) previously, or had used another diabetes app in the past 6 months. The University of Minnesota Institutional Review Board approved the study. Each participant signed an informed consent, and they received a $50 gift card upon study completion.

PROCEDURES

Testing locations were public locations such as private meeting rooms inside a public library or building. Each participant first learned how to use the apps by watching a YouTube video posted by each app developer. Afterward, the participant followed a checklist protocol to practice app use: (1) enter a carb intake, (2) enter an exercise activity, (3) enter an insulin dose, (4) enter a BG reading, (5) locate a BG report for days of the week, (6) locate a BG report for each meal, and (7) e-mail a BG report. They tested one app at a time following the checklist protocol with the same tasks but in a different order. The principal investigator (H.N.C.F.) recorded participant reactions and comments during app practice and app testing as field notes, which also included participant responses or decisions to download and use the tested apps in the future. Handwritten field notes were transcribed in a Microsoft Word document and transferred to Excel for coding.

DATA ANALYSIS

Transcripts were analyzed inductively by conventional content analysis to describe the phenomenon of first-time patient user experience to identify key usability problems with the diabetes apps.²⁸ This is inductive category development,²⁹ in which researchers are themselves in the data (statements and reactions observed from patients) to allow new insights to emerge³⁰ on patient user experience. One researcher (D.J.) did open coding and confirmed codes with another researcher (T.J.A.) to create categories of usability problem scenarios and patient difficulty in app use. The categories were then grouped into major themes with subthemes organized by usability problem type. Next, the research team met, discussed, and reached a consensus on the thematic analysis for the types of app difficulty experiences encountered by patient users.

Results

Diverse recruitment sites yielded 92 participants from Facebook (50%), participant referrals (9%), a community clinic (8%), a university (6.5%), public housing (6.5%), Craigslist (5%), VA clinic (4%), diabetes support groups (3%), and miscellaneous sites (8%). The sample included 23 African Americans (25%), 23 Native Americans (25%), and two Asians (2%) (Table 1). Education levels included 33 participants with high school or less (33%), 31 participants with 2 years of college (34%), 19 graduates of 4 years of college (21%), and 11 postgraduates (12%). The majority of participants were female (59%) with a mean age of 54 years, had type 2 diabetes (70%), and had lived with diabetes for an average of 17 years. Most participants were on insulin injections (85%), and some used an insulin pump (15%). After app testing, the decision to use tested apps is mixed (Table 2). Overall, almost half of participants are interested (48%): mySugr (7%), OnTrack (37%), and both apps (4%). About quarter of participants reported they might use either mySugr or OnTrack (n = 20), but four participants would not use apps and 24 participants did not decide. At the end of the study session, 16% of participants (n = 15) downloaded the tested apps. Conventional content analysis of diabetes app experiences identified three major areas of app use difficulty: (1) data input, (2) presentation and report display, and (3) self-learning options.

Table 1.

Participants' Characteristics (n = 92)

CHARACTERISTICS	TOTAL
Age, years (SD)	54 (13)
Men, n (%)	38 (41)
Race, n (%)
White	57 (62)
Black/African American	23 (25)
Native American	10 (11)
Asians	2 (2)
Highest completed education, n (%)
Elementary	4 (4)
High school or equivalent	27 (29)
2 Years of college	31 (34)
4 Years of college	19 (21)
Graduate school	11 (12)
Device brand, n (%)
Samsung	44 (48)
LG	19 (20)
iPhone	8 (9)
ZTE	7 (8)
Motorola	6 (6)
Other	8 (9)
Smartphone comfort level, n (%)
Very uncomfortable	23 (25)
Neither	12 (13)
Comfortable	33 (36)
Very comfortable	24 (26)
Diabetes types, n (%)
Type 1	28 (30)
Type 2	64 (70)
Insulin use types, n (%)
Insulin pump	14 (15)
Long- and short-acting injection	46 (50)
Long-acting injection	28 (30)
Short-acting injection	2 (2)
None (stopped use)	2 (2)
BG testing prescribed per day	3.8 (1.8)
BG testing per day	6.2 (1.4)
Daily or less, n (%)	19 (21)
2 Times a day, n (%)	34 (37)
4 Times a day, n (%)	21 (23)
>4 Times a day, n (%)	18 (19)

BG, blood glucose; SD, standard deviation.

Table 2.

App Use Intention

APP USE DECISION, N (%)	(N = 92)
Use overall	44 (48)
Use mySugr®	6
Use OnTrack®	34
Use both mySugr and OnTrack	4
Might use	20 (22)
No for mySugr or OnTrack	4 (4)
Unknown	24 (26)
Downloaded overall	15 (16)
mySugr	2
OnTrack	11
Both mySugr and OnTrack	2

PROBLEMATIC DATA INPUT

This first theme is defined as “problems participants faced that increased the difficulty to enter correct data into the app.” Data errors occurred because of problematic data entry mode. Participants had difficulty in finding data entry icons or buttons. They first attempted to click task names (e.g., “activity”) to enter data, which were only headers and not buttons. In other instances, they could not find icons that represented the information to be entered. Many participants also accidentally clicked elsewhere in the screen than intended, called the “fat finger phenomenon,” because it is easy to mistakenly tap small buttons on the phone leading to incorrect data input. The lack of immediate data entry confirmation made participants feel uncertain on what they entered. For example, if the time was changed from AM to PM, participants were unsure if the change was updated because there was no notification unless the user was exiting the screen. Another common error was related to programmed default values that required participants to override them by saving data entries. Some participants saved by accident (fat finger phenomenon) before they finished entering all data, which led to premature closeout of the app. Some participants missed the save button entirely and lost the data.

PROBLEMATIC PRESENTATION AND REPORT DISPLAY

This theme is derived from participant complaints about problems understanding what is presented inside the app: medication list, icons, words, and app report. In mySugr, the medication list was unfamiliar to most participants (e.g., “corresponding” insulin dose). They were unsure which option to use for entering insulin dose. Participants were also unfamiliar with certain icons or words (e.g., “airplane” icon to send an e-mail). A 64-year-old female reported not understanding “export” in mySugr “That sounds like input to me, bring it in, bring it in.” A 64-year-old female reported “Oh, I expected it to say e-mail, where is the send button using the email app?” when she was trying to e-mail a report from mySugr. Participants had difficulty understanding the apps' BG reports. They thought the phone screen may be too small to read a report (limited visibility), whereas others could see the report but could not understand it. One 69-year-old male asked, “Does it [mySugr] give you a visual?” and asked for help to interpret the BG report displayed in the app. Participants also gave mixed opinions on the function of the “monster” mascot in mySugr that makes a sound after each task was finished. Some found the monster to be personal, motivating, and reassuring, but others thought the sound was distracting and confusing. A 69-year-old female reacted to it by asking, “What did I do wrong?” when she heard the sound. Some participants turned down the volume in response to the sound.

INSUFFICIENT SELF-LEARNING OPTIONS

Although each app was prefaced with a YouTube app training video, participants felt the videos were not enough to teach them how to use the app. Many participants believed the apps would need to have more tutorial options or training help for them to learn to use the app on their own. Multiple participants also felt the mySugr app was not intended for adults. A 29-year-old female stated, “This [mySugr] is more for teenagers” and she had no patience for it. A 69-year-old male asked about built-in help options: “The app [mySugr] is not too intuitive, do they have a help screen?” One 79-year-old male reported he lacked the confidence to retain information presented in mySugr's training video.

Discussion

This study analyzed patient experience with using two commercially available diabetes apps for the first time. A strength of our study was the sample size of 92 with a diverse population (38% nonwhite) given that most m-health usability evaluations have fewer than 30 participants and limited recruitment sites.^16–18 Overall, this study supports the findings from previous studies, suggesting that poor usability is a large barrier for patients to use diabetes apps. Common usability themes of app use barriers and recommendations for app design improvement are listed in Table 3.

Table 3.

Common Usability Themes During App Use by First-Time Patient Users

USABILITY THEMES	APP USE BARRIERS	RECOMMENDATION
Data entry burden	Accidental tapping of incorrect area (i.e., fat finger phenomenon)	Reduce data entry steps and increase spacing between buttons and data fields
	Hard to find correct icon	Easy app navigation to locate items to enter
	Programmed default values override user entry	Avoid manual data entry and allow users to automatically override default
	No confirmation if data are entered	Error prevention (e.g., confirm value)
	Forgets to tap “save”	Automatic data saving
Intuitive app display and presentation	Mixed opinions on “monster” mascot in mySugr	User customization of text/icons and define how many self-management tasks to monitor
	Hard to read glucose reports	Avoid clutter on app screen to allow clear visibility
	Do not know what “export” means	Use words that are patient-friendly
	Trouble viewing the reports and reading report detail	Good color contrast and improve readability on a smartphone
	Cannot interpret reports	Simple analysis reports of BG readings, carb intake, exercise, and insulin use with key legends or pop-up message to explain
Self-learning options available within the app	Desire more tutorials and/or training options	Offer self-learning resources easily accessible within the app
	Hard to remember training video	Resource index that is easy to search
	Refresher on how to use app	Pop-up messages built in during app use
	Cannot read text in the app	Offer alternative training in video or audio with app screenshots

carb, carbohydrate.

DATA ENTRY BURDEN

One benefit of diabetes app use is the ability to track data passively through internal app functionality or by integration with another system (e.g., continuous glucose monitoring system and food scanner). App navigation needs to be more user-friendly. Patients feel frustrated when they are unable to find the right icon to record the information they intended. They need to enter multiple items such as BG readings, carb intake, exercise, and insulin dose, among others. The data entry process was also prone to error. A lack of error prevention is consistent with prior findings that duplicate data and missed data are contributory to poor usability.³¹ Patients reported confusion about how to save data and how to recover data from erroneous entries in heart failure m-health apps.³¹ Reducing errors and data entry steps, making navigation easy, and automatic data saving options in apps are crucial.

INTUITIVE APP DISPLAY AND PRESENTATION

Patient difficulty in viewing and understanding small text and icons is consistent with other studies that found text size, screen visibility, and app buttons and symbols were not intuitive enough for older users.³² Furthermore, older diabetic patients criticized font size, color, contrast, and readability of health apps.³³ Some participants in our study commented that the mySugr “monster” icon is personal, motivating, and reassuring, whereas other participants found the sounds made by this icon to be distracting and confusing. Customization in m-health apps has been suggested in the literature because patients have divided preferences in app functionalities.³⁴ Patient difficulty in reading and understanding analysis reports of BG, carb intake, exercise, and insulin dose is consistent with prior reports of patients experiencing problems in understanding data patterns, graphs, and reports in apps.³¹ Data visualizations and reports in apps should be simplified, and the app should provide tutorials on how to interpret them (e.g., key legend or pop-up message to explain the analysis) (Table 3).

Patients with type 2 diabetes are usually older adults and may have comorbidities such as retinopathy and neuropathy that could cause difficulties when using apps. One participant found the screen difficult to see because of glaucoma. Many participants had visibility issues and difficulty reading the phone screen accurately. Furthermore, patients with diabetes may have multiple chronic conditions such as heart failure and neuropathy. Generally healthy individuals may not find health information management as daunting, but patients with greater comorbid conditions have described data management as time-consuming and tiring.³⁵ This is supported by the concept of “illness work,” which is described as any activity involved with managing an illness.³⁶ Patients with diabetes already have a high amount of “illness work,” so diabetes apps should not become another mound of “work” in managing diabetes.

SELF-LEARNING OPTIONS AVAILABLE WITHIN THE APP

Another recurring theme was insufficient learning resources within the app. OnTrack and mySugr offered tutorial videos in YouTube, which are accessible outside the app, but there were no self-learning resources within the app (e.g., help search options or tutorial message pop-ups) to help new users. Patients cannot retain all information at one time. Adapting to a technology or gaining confidence to use the app typically occurs in stages. Also, each individual learns at a different pace. An integral part of user adaptation in health apps is providing additional intermittent short app training to prevent early technology rejection.³⁷ Making self-learning options more accessible through a variety of formats, including video and audio tutorials for patients with decreased vision, would be helpful.

LIMITATION

The data were observational field notes from the principal investigator (H.N.C.F.). Each session was audio recorded to validate the field notes; however, future studies would benefit from video and eye-tracking software. A second observer would also be beneficial to compare notes. Only 2 apps were tested and may not be representative of the diabetes apps on the market. The phone platform was limited to Android, which may not be representative of the larger population²⁷ because we did not include iOS users. Only free versions of the apps were tested. App functionality that required payment was not included in this study. Some paid apps allow for connections with a glucometer, which could increase usability. Requiring payment of an m-health app increases patient engagement because patients will use the app to get a return on the money they spend.³⁴ A potential limitation of generalizability to this study is the overall representation of females. This is not surprising as women spend more time than men on smartphones.^38,39 Women also use social media apps such as Facebook, Instagram, and Pinterest more often than men (83% versus 75%).⁴⁰

FUTURE STUDIES

Paid apps were more likely to use plain language and display content clearly than free apps.⁴¹ Future studies should test paid app functionality and include those offering paired or automatic upload data from a glucometer and other health monitoring devices. Future studies should compare a suite of unbundled, small apps versus a single, bundled app for m-health management. Single-functionality apps may allow participants to tailor app use to their health needs. Patients may want separate app features to manage chronic conditions other than diabetes. Conversely, not all patients with diabetes find all features in a diabetic app useful to their own needs.³¹ Patients also emphasized the benefit of app use only if it was useful for their clinician.⁴² App usability is also affected when health apps do not integrate directly into a clinician's workflow.⁴³ Future research should include app integration with the clinical care process and clinician views of app usability. Another research consideration is to include caregivers. Family members are often the primary choice to provide technical support, especially for patients aged 50 years or older with chronic disease.³¹

Conclusions

Commercially available free diabetes apps have the potential to assist patients with diabetes self-management. However, patient testing of two top-rated apps showed challenges in their usability. Manual data entry was prone to error because of overcrowded app screens, the fat finger phenomenon, lack of entry confirmation, and app programmed default value logic requiring manual override for every entry. Another major challenge was the app display and presentations. Wording or icons in the app were not intuitive to understand. App analysis reports on monitored data such as BG readings and insulin dose were hard to understand or interpret for patients. Additional self-learning options within the app are important to provide supplemental learning to help patients gain confidence and proficiency in app use. Otherwise, patients with diabetes who already have a chronic disease burden may not begin to use a diabetes app or continue use over the long term.

Authors' Contributions

H.N.C.F. and D.J. wrote the article and researched the data. T.J.A. researched the data, contributed to the general discussion before writing, and reviewed and edited the article.

Disclosure Statement

No competing financial interests exist.

Funding Information

This study was supported by the Robert Wood Johnson Foundation Future of Nursing and Sigma Theta Tau International—Zeta Chapter.