Rapid Evidence Review of Mobile Applications for Self-management of Diabetes

Abstract

Background

Patients with diabetes lack information on which commercially available applications (apps) improve diabetes-related outcomes. We conducted a rapid evidence review to examine features, clinical efficacy, and usability of apps for self-management of type 1 and type 2 diabetes in adults.

Methods

Ovid/Medline and the Cochrane Database of Systematic Reviews were searched for systematic reviews and technology assessments. Reference lists of relevant systematic reviews were examined for primary studies. Additional searches for primary studies were conducted online, through Ovid/Medline, Embase, CINAHL, and ClinicalTrials.gov. Studies were evaluated for eligibility based on predetermined criteria, data were extracted, study quality was assessed using a risk of bias tool, information on app features was collected, and app usability was assessed. Results are summarized qualitatively.

Results

Fifteen articles evaluating 11 apps were identified: six apps for type 1 and five apps for type 2 diabetes. Common features of apps included setting reminders and tracking blood glucose and hemoglobin A1c (HbA1c), medication use, physical activity, and weight. Compared with controls, use of eight apps, when paired with support from a healthcare provider or study staff, improved at least one outcome, most often HbA1c. Patients did not experience improvements in quality of life, blood pressure, or weight, regardless of app used or type of diabetes. Study quality was variable. Of the eight apps available for usability testing, two were scored “acceptable,” three were “marginal,” and three were “not acceptable.”

Discussion

Limited evidence suggests that use of some commercially available apps, when combined with additional support from a healthcare provider or study staff, may improve some short-term diabetes-related outcomes. The impact of these apps on longer-term outcomes is unclear. More rigorous and longer-term studies of apps are needed.

Registration

This review was funded by the Agency for Healthcare Research and Quality (AHRQ). The protocol is available at: http://www.effectivehealthcare.ahrq.gov/topics/diabetes-mobile-devices/research-protocol.

Electronic supplementary material

The online version of this article (10.1007/s11606-018-4410-1) contains supplementary material, which is available to authorized users.

BACKGROUND

More than 30 million Americans have some form of diabetes mellitus (diabetes), with 1.5 million people diagnosed each year.¹ Diabetes self-management—which includes healthy eating, physical activity, taking medications on time, tracking health data and adjusting medication and behaviors accordingly—is believed to play an important role in preventing microvascular and macrovascular complications.² Increasingly, patients have started to use mobile applications (apps) to assist with their diabetes self-management.^3–5 In a 2015 survey, 58% of smartphone users had downloaded a health-related app, and the majority said that they downloaded the app to help track physical activity and eating.⁶ Apps that track diabetes-related health information, provide education, and connect patients to support systems could potentially facilitate patients’ self-management and improve diabetes-related outcomes.

Although numerous apps for diabetes self-management are commercially available, there is little information on which apps are effective in improving diabetes-related outcomes. This rapid evidence review aims to help patients and clinicians make informed decisions by (1) evaluating evidence on the efficacy of mobile apps in improving outcomes for adult patients with type 1 and type 2 diabetes; (2) summarizing up-to-date information on app features, platforms, cost, and privacy/security information; and (3) evaluating the usability of apps.

METHODS

The protocol for this rapid evidence review is available on the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care Program website.⁷ The full technical brief, which includes detailed information on individual apps and the evidence supporting them, will be made available on that site.

Topic Development

Reviewers at the Scientific Resource Center consulted with national experts in diabetes care and mobile health evaluation to identify the populations, interventions, comparators, and outcomes of interest. These eligibility criteria are presented in Table 1.

Table 1.

Eligibility Criteria for Primary Studies

Study designs	Include: Randomized controlled trials, nonrandomized controlled trials, or other observational study with a comparator; a subgroup analysis of these studies or a registry study Exclude: Pre-post studies without a comparator
Populations	Include: Entire included population are adults (18+ years old) diagnosed with type 1 or type 2 diabetes; or effects for this population can be distinguished (i.e., through a subgroup analysis) Exclude: Children, adolescents, pregnant women, those with prediabetes or risk factors for diabetes, or gestational diabetes
Interventions	Include: Commercially available website, program, or application delivered through a mobile device (i.e., phone, tablet, or watch) for the purpose of diabetes self-management. Interventions must include at least one of the following components: 1. Education 2. Data tracking 3. User-provider communication 4. Social support/social media 5. Reminders (with the exception of appointment reminders) Exclude: Mobile health programs not available through an app (e.g., glucose meters, texting programs); artificial pancreas
Comparators	Include: Usual care or other mobile or nonmobile program for diabetes self-management; no comparator but part of a registry study
Outcomes	Include: All patient outcomes Exclude: Provider outcomes, healthcare system outcomes, technology performance outcomes (e.g., bugs and crash statistics)
Timing/setting	Include: Any setting; any study length; only studies published 2008 or later
Language	Include: English

Search Strategy

We searched Ovid/Medline and Cochrane Database of Systematic Reviews for systematic reviews and technology assessments from January 2008 to June 2017. Because we identified systematic reviews that contained sufficient information to address our research questions, we relied on reference lists to identify primary studies. We also conducted primary literature searches online, through Ovid/Medline, Embase, CINAHL, and ClinicialTrials.gov, to identify recently published studies on commercially available apps from January 2016 to July 2017. We conducted updated searches in December 2017. We also solicited information through the Federal Register.⁸ For all included apps, we contacted either the app developer or primary study author to request additional information, including free trials of apps that required payment, subscription, access code, or password. Search strategies are available in ESM.

Study Selection

Systematic reviews were chosen if they addressed our research questions⁷ and met three additional criteria: searched one or more citation databases, applied prespecified inclusion and exclusion criteria, and assessed the quality or risk of bias of primary studies. Primary studies were chosen based on criteria described in Table 1. Titles, abstracts, and full-text studies for all systematic reviews and primary studies were reviewed by a single reviewer.

Study Data Extraction and Quality Assessment

We extracted data on each of the eligibility criteria (Table 1) and results using piloted data extraction forms. One reviewer extracted data and another checked for accuracy. We modified a study-level risk of bias tool which was based on AHRQ guidance⁹ and used this to rate the overall quality of each study (see ESM). For each study (excluding subgroup analyses), we used the nine criteria to determine whether the study was at low, unclear, or high risk of bias. Because the patients in our included studies knew whether they were using an app, and no sham controls were used, we did not include masking of participants or healthcare providers as a risk of bias criterion. We took this lack of masking into consideration in our overall risk of bias ratings.

We used a study’s overall risk of bias as a proxy measure for quality: low risk of bias meant a study was likely high quality, a moderate risk of bias meant a study was likely moderate quality, and a high risk of bias meant a study was likely low quality. One reviewer rated the study and another checked for accuracy. Disagreements on data extraction and risk of bias assessment were resolved through discussion.

App Feature Extraction and Usability Testing

Using the most recent release of the app, we collected the following data: available to download on Apple and Android platforms, available devices (tablet, phone, etc.), cost, privacy/security information, and features. Three reviewers rated each available app using the System Usability Scale (SUS) which includes ten Likert-like items assessing the usability of a system or service.¹⁰ We used guidance from Bangor et al. to interpret SUS ratings (≥ 70 “acceptable,” 50–69 “marginal,” < 50 “not acceptable”).¹¹

Data Synthesis

We summarized results qualitatively and grouped results by type of diabetes. For all outcomes, statistical efficacy was determined by whether use of the app was associated with a statistically significant improvement (p < 0.05) in an outcome compared with control. For HbA1c, we determined clinical efficacy by whether the use of the app was associated with a decrease of at least 0.5%. We did not rate bodies of evidence using Grading of Recommendations Assessment, Development, and Evaluation (GRADE) or another system because each app was associated with sparse data.

RESULTS

Table 2 contains selected characteristics of included apps for type 1 and type 2 diabetes. ESM summarizes risk of bias assessments within and across studies. Online Appendices 4 and 5 contains detailed information on the identified studies.

Table 2.

Features, Usability, and Significant Outcomes for Apps for Type 1 and Type 2 Diabetes

The “Patients who used the app saw improvement in which outcomes?” column shows significant between-group outcomes and study-reported satisfaction/usability only

BG blood glucose, C/F carbohydrates/food, EMR electronic medical record, Ex exercise, HbA1c Hemoglobin A1c, Rx prescriptions/medication, NA not available for usability testing, Wt weight

*Not currently supported by IOS 11

Literature Flow

Our search identified 143 systematic reviews and technology assessments, five of which were eligible and pertinent to our research questions (Fig. 1).^46–50 These reviews included 34 unique primary research studies that we evaluated for inclusion. Primary literature searches identified 372 additional articles. We included a total of 15 articles^{14, 17, 20, 22, 24, 25, 28, 31, 32, 35, 40, 43, 45, 51, 52} that evaluated 11 unique apps (six for type 1 and five for type 2). Two of these apps (one for type 1 and one for type 2) had two tiers of access (free and paid), resulting in the assessment of 13 total unique apps. We reviewed studies submitted to the Federal Register notice, but none were eligible for inclusion. After public and peer review of the AHRQ draft technical brief, we re-classified one study of the BlueStar Diabetes app as eligible for inclusion and evaluated features and usability after receiving an access code from the developer.

Fig. 1.

Literature flowchart. .

Apps for Type 1 Diabetes

Study and Participant Characteristics

We identified eight articles^{14, 17, 20, 22, 24, 25, 28, 51} from seven studies evaluating six commercially available mobile applications for type 1 diabetes (Glucose Buddy,^{12, 13, 14} Diabetes Manager,^{16, 17} Diabetes Diary,^{21, 22} Dbees,^18–20 Diabetes Interactive Diary,^23–25 and Diabeo Telesage^{26–28, 51}). Among the eight articles we identified, seven were RCTs^{14, 17, 20, 22, 24, 25, 28} and one was a subgroup analysis of an included RCT.⁵¹ Study size ranged from 30 to 180 participants. Participants ranged in mean age from 33 to 40 years old and had diabetes for an average of 16 to 25 years. Baseline HbA1c ranged from 7.8 to 8.78%. Study length and length of time that participants used apps ranged from 8 weeks to 6 months.

For most studies, the intervention group used the app with additional support from a clinician or diabetes educator. The control group typically consisted of usual care, standard education, or use of a paper diary, although in the trial of Diabetes Diary²² the comparison group used the app without a feedback module.

Features

Common features of apps for type 1 diabetes included the ability to track health data such as blood glucose, carbohydrates/food, prescriptions, and exercise; patient feedback such as reminders to take medication or measure blood glucose; and diabetes education.

Efficacy

Use of three apps (Glucose Buddy, Diabetes Manager, and Diabeo Telesage) demonstrated a statistical improvement in HbA1c, while use of only two apps (Glucose Buddy and Diabeo Telesage) demonstrated a clinically meaningful reduction in HbA1c. Use of one app (Diabetes Diary) demonstrated an improvement in out-of-range hypoglycemic and hyperglycemic episodes, and use of one app (Diabetes Interactive Diary) demonstrated an improvement in triglycerides, treatment satisfaction, and severe hypoglycemic episodes, as compared with controls. None of the participants in the six studies experienced improvements in quality of life, weight, blood pressure, lipids other than triglycerides, diabetes-related self-efficacy, diabetes treatment satisfaction, or self-care behaviors.

Usability

Of the five apps we could access, only one (Glucose Buddy) scored in the acceptable range. Three other apps were marginal (Glucose Buddy Pro, Diabetes Manager, and Dbees [Android]) and one was not acceptable (Diabetes Diary). Diabetes Interactive Diary and Diabeo Telesage were unavailable to download in the USA. We were unable to use Dbees [Apple] due to an error that did not allow login credentials to be entered.

Privacy and Security

We found privacy policies for only two of the six apps for type 1 diabetes (Glucose Buddy and Dbees). Glucose Buddy (Apple) requires that the user create an account or give the app access to the user’s Facebook account. The privacy policy states that the developer can use personal data, de-identified data, and anonymous data for processing. This app integrates with a third-party platform that purports to provide reasonable security, but data security cannot be guaranteed. Dbees has a vague privacy policy. This app will not sell users’ information and will use personal information for internal purposes to keep the app functioning. Diabetes Diary had no privacy policy, and we could not determine whether Diabeo Telesage, Diabetes Interactive Diary, or Diabetes Manager had privacy policies.

Study Quality

Studies of these apps ranged from low quality (Diabetes Diary, Diabetes Manager) to moderate quality (Glucose Buddy, Dbees, Diabetes Interactive Diary, Diabeo Telesage). Common methodological issues included inconsistent reporting of randomization and allocation concealment and considerably more potential for interaction with healthcare providers in intervention versus control groups. See ESM for detailed risk of bias ratings.

Apps for Type 2 Diabetes

Study and Participant Characteristics

We identified seven articles^{31, 32, 35, 40, 43, 45, 52} of six studies evaluating five commercially available apps for type 2 diabetes (BlueStar Diabetes [BlueStar],^{29, 30, 31, 32, 52} WellTang,^{44, 45} NextJ Connected Wellness Platform Health Coach + [NextJ],^{38, 39, 40} Gather Health,^{41, 42, 43} and mDiab^{33, 34, 35, 36, 37}). Of the seven publications, six were RCTs^{31, 32, 35, 40, 43, 45} and one was a subgroup analysis of an included RCT.⁵² Study size ranged from 30 to 163 participants. Participants in the six studies ranged in mean age from 48 to 55 years old, had had diabetes for an average of 6.63 to 11 years, and had baseline HbA1c ranging from 8.59 to 9.86%, depending on the study group. Study length and length of time that participants used the apps ranged from 2 to 12 months.

Similar to studies for type 1 diabetes, participants who used apps for type 2 diabetes typically received support from a healthcare provider, health coach, or the study team, while the comparison group typically received usual care. The exception was the study of NextJ, where the control group received health coach support without an app. The 2011 study of BlueStar also had multiple comparison groups with varying levels of support, although the primary comparison was the most intensive support versus usual care.

Features

Apps for type 2 diabetes had similar features as those for type 1 diabetes, although they were more likely to include a feature to alert patients when blood glucose was out of an acceptable range.

Efficacy

Use of three apps (Gather Health, BlueStar, WellTang) demonstrated both a statistical and clinically meaningful improvement in HbA1c compared with controls. Use of WellTang also demonstrated an improvement in fasting blood glucose, 2-hour post-breakfast blood glucose, diabetes knowledge, and self-care behaviors compared with controls. Participants who used BlueStar in the 2008 study were more likely than controls to experience an increase in medication dosage, app-identified errors in self-entered medications, and be satisfied with care. None of the participants in the six studies experienced improvements in blood pressure, lipids, depression, weight, body mass index, waist circumference, hip circumference, diabetes distress, or self-reported diabetes symptoms.

Usability

BlueStar was scored as acceptable and was the highest-rated of all of the apps. Two apps (mDiab and mDiab Lite) were not acceptable. While we were able to download NexJ, Gather Health, and WellTang, they require a prescription (or in the case of WellTang, were only available in Mandarin) so we were unable to test usability.

Privacy and Security

Three apps (NexJ, Gather Health, and BlueStar) have privacy policies that suggest that while the apps track user data, the data are generally secure and safeguards exist to prevent third party access without permission. Gather Health is the only app that explicitly asserts a right to share, sell, transfer, license, and/or covey information collected for any reason whatsoever, without disclosing the user’s identity or contact details. mDiab does not have a privacy policy. Because WellTang was only available in Mandarin, we were unable to determine if it has a privacy policy.

Study Quality

Studies of these apps ranged from low quality (mDiab, Gather Health, BlueStar) to moderate quality (NexJ, WellTang). Methodological issues were similar to those seen in studies for type 1 diabetes, including inconsistent reporting of randomization and allocation concealment and greater potential for interaction with healthcare providers or research personnel. See ESM for detailed risk of bias ratings.

DISCUSSION

Summary of Evidence

To our knowledge, this is the first evidence review to provide both an examination of the evidence supporting exclusively commercially available apps for diabetes self-management and a detailed assessment of app features, privacy/security, and usability. While there are numerous studies examining apps for diabetes, the literature on commercially available apps is limited. In addition, these available studies have considerable methodological issues, which is consistent with the findings of other researchers.^{46, 47}

Despite methodological limitations, the evidence indicates that the use of some mobile apps with additional support from a healthcare provider or study staff may be useful in improving short-term outcomes, especially HbA1c, compared with controls for both type 1 and type 2 diabetes. Use of six apps (three for type 1 and three for type 2) were associated with a statistically significant improvement in HbA1c. Even with a stricter criterion for clinical improvement in HbA1c (reduction of 0.5% or more), use of five apps (two for type 1 and three for type 2) were associated with improvements. The evidence does not indicate that the use of apps improves other important outcomes such as quality of life, blood pressure, weight, or body mass index (BMI). Further, diabetes-related complications such as neuropathy, retinopathy, or hypertension were not measured, so we could not determine if the use of apps reduced their incidence or severity.

We found app usability to be variable, which is consistent with other research findings.^{53, 54} Of the five apps available for usability testing for type 1 diabetes, one was acceptable, three were marginal, and one was not acceptable. We rated three apps for type 2 diabetes—one as acceptable and two as not acceptable. Our results suggest that patients may have a difficult time using some of these apps.

We also found variation in the privacy and security policies of these apps. Of the eight apps we could download and use, five had clear privacy policies (two of which included stipulations that the app company could share data with third parties), one had a vague privacy policy, and two had no privacy policies.

Findings from our identified studies are likely generalizable to broader type 1 or type 2 diabetes adult populations. Participants with type 1 diabetes were on average 33 to 40 years old with a diabetes diagnosis for 16 to 25 years, which is comparable to the typical adult with type 1 diabetes.⁵⁵ Multiple studies involved participants on complex management regimens, including multi-day injections or insulin pumps, which may have piqued interest in using an app for self-care management. Participants with type 2 diabetes were on average 48 to 55 years old, which falls within range of highest rates of diabetes diagnoses (age 45 to 64 years).¹ Participants with type 2 diabetes were also in their first 11 years of diagnosis which may have made them more interested in using an app for self-care management.

Limitations

Limitations of this review resulted from methodological issues in the available research, limitations from the rapid review methodology we used, lack of access to some commercially available apps, and limitations in how the SUS was administered.

The most important methodological shortcoming in the available research was that control and intervention group participants did not have equal potential for healthcar e provider support. In most studies, the intervention group had the ability to message providers, health coaches, or study staff and receives an immediate response while control participants were limited to standard patient procedures such as phone calls and monthly appointments. For these studies, the control group did not receive a sufficient degree of attention control so we could not determine whether it was the use of the app or the extra provider attention that caused the effect on outcomes. As a result, it is difficult to apply findings to other healthcare contexts where patients may not have as much support. Another shortcoming was that studies were relatively brief (2 months to 1 year) compared with the lifelong duration of diabetes. Longer studies could more adequately assess the effects on important outcomes such as quality of life. Longer studies could also help answer questions about whether apps continue to engage patients or if interest naturally drops off over time.

We took strategic shortcuts to complete this review on a rapid timeline, and as a result, we may have missed studies and did not analyze all the important characteristics of apps. Although researchers have begun to use rapid review methodology to complete evidence syntheses on short time frames, there is no consensus on which shortcuts to take.⁵⁶ In this review, we identified our list of potentially relevant studies from five recently published systematic reviews and a search for recently published primary studies, rather than through a full systematic review search. The protocol was posted on the Federal Register, and we contacted app developers and study authors to capture missing studies, but it is possible that some were missed. Additionally, due to the rapid timeline, we did not examine all characteristics of apps that are important to patients. Most importantly, we did not examine technology performance outcomes such as malfunctions and crash statistics. The extent to which an app reliably works is an important consideration in patient decision-making that we did not address in this review.

Due to limited funds, for paid apps we only evaluated usability on one platform (Apple iPad mini). For free apps, we evaluated usability on all available platforms. We could not download or log into some of the apps because the interface was only available in a non-English language, the app was not available in the USA, or the app was only available through a doctor’s prescription, access code, or through a membership. Although we contacted app companies requesting access to apps so we could test their usability, we only received access to one proprietary app.

Last, there were a few key differences between how the SUS is typically used and how we used it in this review. The SUS is meant to be administered to the users of apps, preferably in large numbers. None of our reviewers had diabetes; therefore, they may not have accurately represented the needs and preferences of people with diabetes. Our reviewers were also homogenous in terms of gender and education, so they might not represent the diversity of people with diabetes. Although we averaged the scores of three reviewers, the SUS is typically administered to much larger groups of people. We believe that our SUS assessment provides useful information that is not consistently captured in primary studies, but our results should be interpreted with caution.

Conclusion and Next Steps

Our rapid evidence review found that the use of some mobile apps may improve short-term diabetes-related outcomes, especially HbA1c, when linked to additional support from a healthcare provider or coach. The impact of these apps on longer-term outcomes such as quality of life, neuropathy, retinopathy, or hypertension is unclear. More rigorous and longer-term studies are needed that carefully consider the potential for interaction with healthcare providers and study staff in each group. App developers should consider issues of usability, privacy, and security in addition to app features. Finally, newer apps that patients are downloading and using should be evaluated for their impact on short- and long-term diabetes-related outcomes.

Funding Information

This project was funded under Contract Nos. HHSA29020120004C and HHSA290201700003C from the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services (HHS). Dr. Ivlev was supported by National Library of Medicine Biomedical Informatics Training Grant #T15LM007088.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.