Abstract
Background
Advancements in smartphone technology coupled with the proliferation of data connectivity has resulted in increased interest and unprecedented growth in mobile applications for diabetes self-management. The objective of this article is to determine, in a systematic review, whether diabetes applications have been helping patients with type 1 or type 2 diabetes self-manage their condition and to identify issues necessary for large-scale adoption of such interventions.
Methods
The review covers commercial applications available on the Apple App Store (as a representative of commercially available applications) and articles published in relevant databases covering a period from January 1995 to August 2012. The review included all applications supporting any diabetes self-management task where the patient is the primary actor.
Results
Available applications support self-management tasks such as physical exercise, insulin dosage or medication, blood glucose testing, and diet. Other support tasks considered include decision support, notification/alert, tagging of input data, and integration with social media. The review points to the potential for mobile applications to have a positive impact on diabetes self-management. Analysis indicates that application usage is associated with improved attitudes favorable to diabetes self-management. Limitations of the applications include lack of personalized feedback; usability issues, particularly the ease of data entry; and integration with patients and electronic health records.
Conclusions
Research into the adoption and use of user-centered and sociotechnical design principles is needed to improve usability, perceived usefulness, and, ultimately, adoption of the technology. Proliferation and efficacy of interventions involving mobile applications will benefit from a holistic approach that takes into account patients’ expectations and providers’ needs.
Keywords: glucose monitoring, mobile applications, self-care, telemedicine
Introduction
Diabetes is a chronic illness that requires continuing medical care and ongoing patient self-management education and support to reduce the risk of long-term disability and prevent complications.1 In this review, self-management refers to “tasks that an individual must undertake to live well with one or more chronic conditions. These tasks include gaining confidence to deal with medical management, role management, and emotional management.”2 Self-management processes are inherently data intensive, requiring acquisition, storage, and analysis of large amounts of data on a regular basis.
Advances in smartphone technology and wireless networks have resulted in increased adoption and enhanced capability, leading to opportunities for improved diabetes self-management. Arsand and coauthors3 reviewed tools for management of diabetes as well as other chronic diseases that were reported in the literature or publicly available on the Internet. The study listed the tools but did not provide a detailed discussion of their functionality and how they compared with clinical guidelines. Chomutare and coauthors4 reviewed the functionality of diabetes applications available commercially as well as reported in the literature. Whereas, Martin and coauthors5 reviewed the diabetes applications available on the Apple App Store (AAS), with a particular emphasis on usability.
This article complements existing reviews by evaluating functional, nonfunctional requirements together with related issues necessary for large-scale adoption of such interventions. Specifically, we review mobile applications available commercially from the AAS (as a representative of commercially available mobile applications) and/or reported in peer-reviewed articles. The objective is to determine, in a systematic review, whether diabetes applications have been helping patients with diabetes self-manage their condition. From a practical perspective, the results of the analysis highlight key features and requirements for diabetes self-management applications. From a theoretical perspective, the results identify research gaps and help outline an agenda for research related to various factors for improving the adoption, usability, and integration of diabetes self-management applications in the patients’ daily routine and in the context of the larger health care organization.
Methods
Data Source
Our search is based on the AAS (as a representative of commercially available applications) and journal databases. We scanned the AAS during August 2012 for a diabetes self-management application with keyword “diabetes.” We identified peer-reviewed articles by searching in PubMed, Web of Science, IEEE, and ACM databases using the keywords (Mobile OR PDA OR smartphone) AND (diabetes OR insulin OR blood glucose), covering a period from January 1995 to August 2012. Next, we reviewed the title, abstract, and full text (for peer-reviewed articles) using the following criteria.
Inclusion and Exclusion Criteria
To be included in the review, an application had to exhibit the following characteristics:
Support for blood glucose monitoring as a minimum requirement
The patient as the intended primary user of the application, and
The application to be used as an enabler for diabetes self-management by supporting one or more of the self-management tasks.
Applications excluded from the review exhibited any of the following characteristics:
Duplicate applications
Applications where the sole purpose is to educate the patient about the disease
Applications without an English-language user interface, and
Applications intended exclusively for health care professionals.
Evaluation Approach
For the purpose of evaluation, a set of 50 applications were randomly selected and independently reviewed by the authors with a joint probability of agreement of 0.98. We resolved any conflict by discoursing about the disputed applications and by revisiting the aforementioned criteria of inclusion and exclusion. We then independently reviewed another set of 50 randomly selected applications with a joint probability of agreement of 1.0. The remaining apps were reviewed by one of the authors; however, for each application review, another author did random crosschecking and validation.
Result
Figure 1 outlines the article selection process and results. Overall, 71 applications and 16 articles were included in the final review. A total of 39% (27/71) of the commercially available applications supported self-management tasks such as physical exercise, timely insulin dosage or medication, frequent blood glucose testing, and suitable diet. Out of 16 articles, 38% (6/16) supported all four tasks. Additional self-management tasks reported in the literature include self-management education and personalized feedback, weight management, blood pressure monitoring, and communication and patient monitoring by the clinicians.1,4,6,7 Other support tasks/features considered include decision support, notification/alert, tagging of input data, and integration with social media such as Facebook, twitter, MySpace, and Security. Secondary features and characteristics include average consumer rating (as reported on the AAS), latest version, reported research studies about the application, and cost. Figure 2 illustrates the primary features, secondary features, and characteristics of the applications. Table 1 reports the percentage of applications supporting each of the diabetes self-management features, while Tables 2 and 3 list all applications and their respective functionalities and characteristics. The following subsection further describes our findings with respect to a variety of features representing patients’ expectations and provider needs.
Figure 1.
Applications selection process and result. APP, application.
Figure 2.
Application features for diabetes self-management. APP, application; BMI, body mass index; IOS, iPhone operating system.
Table 1.
Percentage of Applications Supporting Various Diabetes Self-Management Tasks
| Function | AAS (%) | Literature (%) |
| Blood glucose | 100 | 88 |
| Medication | 76 | 38 |
| Diet | 68 | 75 |
| Physical exercise | 41 | 50 |
| Education | 18 | 31 |
| Weight | 25 | 6 |
| Blood pressure | 23 | 16 |
| Communication | 83 | 69 |
| PHR | 21 | 63 |
| Decision support | 17 | 19 |
| Data entry automation | 4 | 57 |
| Security | 11 | 38 |
| Social networking | 7 | Not discussed |
Table 2.
Data Abstraction for the Apple App Storea
| Application | Version | Blood glucose | Exercise | Diet | Medication | Blood pressure | Weight | Education | Communication | Decision support | PHR | Automatic data entry | Cost | Average rating |
| bant: a diabetes application for the ePatient | 1.4.3 | Y | N | N | N | N | N | General | N | Microsoft HealthVault | N | Free | 2+(All) | |
| Diagnosis Management of Gestational Diabetes | 1.1 | Y | N | N | N | N | N | General | N | Calculation of body mass index, calorie intake, and dosage | N | N | Free | NA |
| Diabetes Life Harmonized | 1 | Y | Y | N | N | Y | Y | General | N | N | N | N | Free | NA |
| Glucose Buddy: Diabetes Helper | 3.6.5 | Y | Y | Y | Y | N | N | N | N | glucosebuddy.com | N | Free | 4+ | |
| Continued→ | ||||||||||||||
| Track3: Diabetes Planner and Carbohydrate Counter | 4 | Y | Y | Y | Y | N | Y | Personalized | Email; spreadsheet | N | Y | N | 4.9 | 4+ |
| Diabetes Buddy Lite: Control Your Blood Sugar | 1.4 | Y | Y | Y | Y | N | Y | General | N | N | N | Free | 4 | |
| Wave Sense Diabetes Manager | 1.3 | Y | N | Y | Y | N | N | N | N | N | N | Free | 3+ | |
| Diabetes Companion | 1.0.8 | Y | Y | Y | Y | N | N | General | N | N | N | Free | 3+ | |
| Diabetes Pilot | 5.1.1 | Y | Y | Y | Y | Y | Y | N | Email; sync with PC/Mac | Estimated HbA1c | N | N | 14.99 | 3+ ( All) |
| Islet-Diabetes Assistant | 2.2.3 | Y | Y | Y | Y | N | N | N | Email; CSV | N | N | N | 0.99 | 3+ |
| Blood Sugar Diabetes Control | 1.2 | Y | Y | Y | Y | N | N | N | Screen shoot; email | N | N | N | 0.99 | 3 |
| Diabetes Log | 0.97.5 | Y | N | Y | Y | N | N | N | Email; CSV | N | N | N | Free | 3 |
| Diabetes Personal Calculator | 1.2 | Y | N | Y | Y | N | N | N | Insulin dose | N | N | Free | 3+(All) | |
| Diabetes Glucose Tracker Application (iDiabetes) | 3.1.0 | Y | Y | Y | Y | N | N | N | N | N | N | 1.99 | 2+(All) | |
| Diabetes Pal by Telcare: Blood Glucose Manager | 1.1.4 | Y | N | Y | Y | N | N | N | Sync with Mac/PC | N | mytelcare.com | BG | Free | 4.5 |
| Vree For Diabetes | 2.1.4 | Y | Y | Y | Y | Y | Y | General | N | N | N | Free | 3(All) | |
| Diabetes Diary | 1.8.2 | Y | N | Y | Y | N | N | N | Email; CSV | N | N | N | 2.99 | 3.5 |
| GlucoMo | 1.0.4 | Y | Y | Y | Y | Y | Y | N | Forum | N | N | N | 0.99 | 3 |
| LogFrog DB-A Leap in Diabetes | 1.7 | Y | Y | Y | Y | Y | Y | N | Export spreadsheet | N | N | N | 2.99 | 4+(all) |
| Diabetes in Pregnancy | 1 | Y | N | Y | Y | Y | N | N | N | N | N | 5.99 | NA | |
| Easy Diabetes | 1.0.2 | Y | N | Y | Y | N | N | N | Export CSV | N | N | N | Free | 4 |
| Diabetes Manager | 1.3 | Y | N | Y | Y | N | N | N | Export CSV | Insulin dose | N | N | 4.99 | 3(All) |
| dbees.com: diabetes under | 0.9.5 | Y | Y | Y | Y | Y | N | N | Online | N | Sync online | N | Free | 5(All) |
| Diamedic | 3.1.1 | Y | Y | Y | Y | Y | Y | N | N | N | N | 5.99 | 3+(All) | |
| Diabetes UK Tracker | 1.4 | Y | N | Y | Y | Y | Y | N | N | N | N | Free | ND | |
| Glooko Logbook | 1.4.4 | Y | Y | Y | Y | N | N | General | N | N | Blood glucose | Free | 4 | |
| Onsync Diabetes Manager | 1 | Y | N | Y | Y | N | N | N | N | N | N | Free | ND | |
| Sugars: A Diabetes Blood Glucose | 2.1 | Y | N | Y | Y | N | N | N | Email; short message service | N | N | N | 3.99 | ND |
| Gestational Diabetes Management | 1 | Y | Y | Y | Y | N | Y | Y | N | N | N | 2.99 | NA | |
| myDiabetes | 1.0.2 | Y | Y | Y | Y | Y | N | N | N | N | N | Free | NA | |
| RapidCalc Insulin Dose | 1.4.1 | Y | Y | Y | Y | N | N | N | Insulin dose | N | N | 7.99 | ND | |
| Glucose Tracker: Simple and Complete Application to track Diabetes Mellitus | 1 | Y | N | N | N | N | N | Y | N | N | N | Free | NA | |
| Blood Sugar Monitor | 1 | Y | N | N | N | N | N | N | N | N | N | 5.99 | NA | |
| Simple Diabetes | 1 | Y | N | N | Y | N | N | N | CSV | N | N | N | Free | NA |
| IBGSTAR Diabetes Manager | 2.1 | Y | N | Y | Y | N | N | N | Email; CSV | N | N | BG | Free | 4 |
| Diabe | 1 | Y | N | N | N | Y | Y | N | Sync online | N | Y | N | Free | NA |
| DF Diary | 2.4.1 | Y | N | Y | Y | N | N | N | Insulin dose | diabeticfriend.co.uk | N | 1.99 | NA | |
| Diabetes-Plus | 2.3 | Y | Y | Y | Y | Y | Y | N | Email; CSV | N | N | 4.99 | 4(All) | |
| SiDiary | 1.0.9 | Y | N | Y | Y | Y | Y | N | Sync SiDiary online | N | N | 5.99 | NA | |
| Gluco-Record | 1.1 | Y | Y | Y | Y | N | N | N | Email; CSV | N | N | 2.99 | 2 | |
| BlueLoop | 1.1 | Y | N | Y | Y | N | N | N | Sync Online | N | N | Free | NA | |
| Diabetes-Diary | 1.8.2 | Y | N | Y | Y | N | N | N | N | N | 2.99 | 3+(all) | ||
| Diabetes Teams Lite | 1 | Y | Y | Y | Y | N | N | N | N | N | Free | NA | ||
| Your Diabetes Guardian | 1 | Y | Y | Y | Y | N | N | N | Parent’s phone | N | N | 4.99 | NA | |
| Insulin Dose Advisor | 2.61 | Y | Y | Y | Y | N | N | N | Sync online | Insulin dose | N | 2.99 | NA | |
| Glucose Pro | 1.0.4 | Y | Y | Y | Y | N | Y | N | Sync Mac/PC; CSV | N | N | €2.99 | NA | |
| Blood Glucose (Sugar) Tracker | 3.1 | Y | Y | Y | Y | N | N | N | N | N | Free | NA | ||
| DAFNE Online | 1.6.2 | Y | N | Y | Y | N | N | N | Sync online | Insulin dose suggestion | N | Free | 4+(All) | |
| GlucoPluse | 1 | Y | N | Y | Y | N | N | General | Google Health | N | N | 3.99 | NA | |
| Glucose Tracker: Log and Moni | 1.1 | Y | N | N | Y | N | N | General | N | N | N | 0.99 | 3 | |
| Diabetes Office Visit | 1.5 | Y | N | N | N | N | N | General | N | N | N | 3.99 | 3(All) | |
| Blood Sugar Tracker by healthyclouding | 1.3 | Y | N | N | N | N | N | N | CSV | N | N | Free | 4+ | |
| Diabetic Tracker Unlimited: Track | 5 | Y | N | N | N | N | N | N | N | N | 1.99 | 2(All) | ||
| Glucose | 0.7.3 | Y | N | N | Y | N | N | N | Spreadsheet | N | N | 0.99 | 2+(All) | |
| Glucose Companion Free | 1.6 | Y | N | N | N | N | Y | N | Email; CSV; PDF | N | N | Free | 4+(All) | |
| Diabetes Tracker | 1 | Y | N | N | N | N | N | N | N | N | N | 1.99 | NA | |
| Diabetes 360 | 1.5.1 | Y | Y | Y | Y | N | N | N | Email; CSV | Insulin calculator | N | 1.99 | 4+(All) | |
| Diabetes Grapher Free | 1.1 | Y | N | N | Y | N | N | N | Graph | N | N | Free | 2 | |
| Blood Pal Free: Glucose Tracker | 1.2 | Y | N | N | Y | N | N | N | N | N | Free | 3+ | ||
| Glucose Monsters | 1.0.1 | Y | N | N | N | Y | Y | N | N | N | N | Free | 2 | |
| AT&T mHealth Solutions Presents DiabetesManager | 1.3.1 | Y | Y | Y | Y | N | N | N | Y | Y | N | Free | NA | |
| Diabetes Management | 1 | Y | N | Y | N | N | N | N | N | N | N | 4.99 | NA | |
| MyBloodWorks | 1.3 | Y | N | N | Y | Y | Y | N | N | N | N | 3.99 | 5(All) | |
| iTeam-Agenda Diabetes | 1.0.1 | Y | N | N | N | N | N | N | N | N | N | 0.99 | NA | |
| Glucose Manager Patients | 1 | Y | N | Y | N | N | N | N | N | N | 3.99 | NA | ||
| Dia-Log | 1.4 | Y | Y | Y | Y | N | N | N | Email; CSV | N | N | Free | NA | |
| GliControl | 3.2 | Y | N | N | Y | N | N | N | Email; CSV | Insulin calculator | N | Free | NA | |
| Diabetophone | 1.1 | Y | N | N | Y | N | N | N | N | N | N | 2.99 | NA | |
| myBolus | 2 | Y | N | Y | Y | N | N | N | Sync iCloud | Insulin calculator | N | 1.99 | NA | |
| DM graph | 1 | Y | N | N | N | N | N | N | N | N | N | 0.99 | NA | |
| Health Garden: Diabetic Edition | 1 | Y | Y | N | N | Y | Y | N | N | N | 0.99 | NA | ||
Y, yes; N, no; NA, not available; CSV, comma-separated values.
Table 3.
Data Abstraction for Literaturea
| Author | Application | Medication | Exercise | Blood glucose | Diet | Blood pressure | Weight | PHR | Education | Automatic data entry | Decision support | Metrics | Outcome |
| Preuveneers8 | ND | Y | Y | Y | Y | N | N | N | N | N | Drug dose | Satisfaction; usability | Improvement |
| Arsand9 | Few Touch | N | Y | Y | Y | N | N | N | N | BG; exercise | N | Usability; healthy behavior | Usability, 84% |
| Forjuoh10 | Diabetes Pilot | Y | Y | Y | Y | N | N | N | Food information | N | N | HbA1c | -1.7 |
| Farmer11 | ND | Y | Y | Y | Y | N | N | Y | N | BG | N | HbA1c | -0.7 |
| Harris12 | HealthReachMobile | N | N | Y | N | N | N | Y | N | BG | N | App functionalities | Automated upload, good; graphical feedback, good; automated feed-back, not useful |
| Istepanian13 | ND | N | N | Y | N | Y | N | Y | N | BG; BP | N | Blood pressure; glucose level | Decrease 7.76% in intervention versus 8.4% in control group |
| Lyles14 | N | N | Y | N | N | N | Y | N | G | N | Satisfaction | Smartphone is frustrating | |
| Ma15 | Glycemic index meal planner | N | N | N | Y | N | N | Y | Food information | N | N | HbA1c; satisfaction; carbohydrate intake | HbA1c, -0.5%; easy, 95%; useful, 63%; carbohydrate intake, -40 g/day |
| Mohan16 | Window Mobile 6 | N | N | Y | N | Y | N | Y | N | BG; BP | N | Planned for future testing | |
| Quinn17 | WellDoc DiabetesManager | N | N | Y | Y | N | N | Y | Specific | BG | Y | HbA1c | (-)2.03% compared with (-)0.68 in control group |
| Quinn18 | ND | N | N | Y | Y | N | N | Y | Specific | BG | Y | HbA1c | (-)1.9% compared with (-)0.7% in control group |
| Kollmann19 | Diab-Mwmory | Y | Y | Y | Y | N | N | Y | N | N | N | HbA1c | (-)0.6% |
| Sultan20 | ND | N | Y | Y | Y | Y | Y | N | N | N | N | Satisfaction | 95.2% of patients explained they would use such system |
| Nes21 | ND | Y | Y | Y | Y | N | N | N | N | N | N | Lifestyle | Improvement |
| Frøisland22 | Diamob | Y | Y | Y | Y | N | N | N | Y | BG | N | HbA1c; satisfaction | HbA1c no improvement; knowledge of patient increases; patient favor application over short message service |
| Rollo23 | Nutricam | N | N | N | Y | N | N | Y | N | N | N | Number of food record | Energy intake was under-recorded by 649 kJ (standard deviation 810) using the mobile phone method |
ND, not discussed; Y, yes; N, no; BG, blood glucose; BP, blood pressure.
Additional Support Features
Data Entry Automation
Data entry automation has been employed primarily for uploading blood glucose levels, blood pressure levels, and physical exercise.9,16 When comparing manual to automated data entry, automated is more often desirable and is likely to result in more data captured.24 Moreover, eliminating the extra step of transferring data from the measurement device to the phone would also increase patients’ satisfaction.25,26 It simplifies self-management tasks for elderly or non-tech-savvy patients. A total of 4% (3/71) of commercial applications and 57% (9/16) of research articles employed some type of data entry automation. Commercial applications available on the AAS showed that data entry automation has been employed only for blood glucose level transmission.27,28 A custom-developed chipset,27,28 Bluetooth technology (wireless), and a special cable are used to connect the meter to the iPhone.
Automated entry of physical activity was found only in a study by Arsand and coauthors.9 This was accomplished by using a physical activity sensor (step counter) that communicates wirelessly with the mobile phone using Bluetooth technology. Although patients were fairly satisfied with the system, they complained about the large size of the sensor and the fact that the sensor could not record activities such as skiing, cycling, and swimming. The use of the step counter resulted in a 20% increase in the number of steps from the first week to the last week of use.9
Decision Support
In addition to serving as an electronic logbook, automated analysis and rule-based interpretation of data on an individual basis would be helpful.19 Primarily, these tools are used to process large and complex data and provide feedback relating to the insulin dosage, carbohydrate intake, and healthy behavior. In all, 19% (3/16) of research articles employed some form of decision support.8,17,18 The decision support tool may be located in the patient’s mobile device or in the provider’s electronic medical record (EMR) to process data and provide feedback to the patient. For example, Preuveneers8 reports a study where a patient entered data relating to carbohydrate intake, physical activities, and blood glucose level. An algorithm analyzed the data and provided insulin dosage suggestions. With respect to commercial applications, 17% (12/71) have decision support capabilities. All decision support functionality relate only to insulin dosage suggestion (as opposed to lifestyle change, exercise, and diet). For example, RapidCalc Diabetes Manager29 provides insulin dosage suggestion based on blood glucose reading, carbohydrate intake, and insulin on board. The application can also adjust the insulin dosage based on the planned physical exercise on that day. Another application suggests insulin dose based on the glucose level, activities, carbohydrate intake, and carbohydrate/insulin ratio.30 Unfortunately, customer reviews seem to indicate problems with the application (bugs), a serious issue considering the potential ramifications of faulty or inaccurate decision support recommendations. Indeed, it is an issue that is prompting government guidelines and/or regulations. For example, in order to provide safety and assurance, the United States Food and Drug Administration (FDA) plans to provide regulation for standalone software, including applications for remote use of medical devices, software that uses mobile platforms as medical devices, and applications that process patient information and generate patient recommendations.31 Of the reviewed applications, only one app had FDA clearance, AT&T mHealth Solutions Presents DiabetesManager.32
Communication
Mobile applications may offer some advantages to patients upgrading from a paper logbook to an electronic system. Data stored in mobile devices can be emailed to clinicians or certified diabetes educators for review. Providers should be able to respond to patients easily, manually, and through automated alerts to assist patients in managing their condition.33 Communication functionalities primarily allow patients to share their data with clinicians, peers, and families.
The authors classified any application as supporting patient–provider communication if the application consists of one of the following features: data can be emailed, data can be synchronized with patient’s other devices such as a laptop or computer, data can be exported to portable document format, comma-separated values, or some other standard format that can then be transferred to other devices, and/or data can be transferred and integrated with a personal health record. For commercial applications, 83% (59/71) included at least one of the aforementioned communication functions. For example, Your Diabetes Guardian application allows a child to use his/her phone to record daily measurements as instructed on the schedule. The information is communicated in real time to the parent’s phone. Failure by the child to execute a scheduled event will trigger an audible or visual alert to the parent’s phone.34
Integration with Patient Health Record/Web
A Patient Health Record system (PHR) is an electronic application that enables patients to aggregate, integrate, and share data across multiple health providers and self-manage their health better.35
Overall, 21% (15/71) of commercial applications support synchronization of data with PHRs. Generally, PHRs included in an application are online accounts that are specific to the application.36 However, commonly available personal health records such as Microsoft Health Vault were also reported.27 With respect to research articles, 63% (10/16) of included studies have data synchronization with an online system. Five studies have integrated PHR with EMR and provide both patient and physician Web portal,8,11,12,14,19 whereas the remaining studies include either a patient view or a clinician view of the EMR.
Education
While decision supports aims to provide explicit support for a particular diabetes self-management task, e.g., insulin dosage, diabetes education is the ongoing procedure of supporting patients by enhancing their knowledge, skill, and ability necessary to self-manage diabetes.37 Insofar, research has provided adequate proof that self-management education has a positive impact on clinical outcome.38 Of commercial applications, 18% (13/71) have some component of self-management education (in addition to supporting self-management tasks). The majority of applications provide general diabetes education, e.g., guideline for diabetes care, food information, videos about diabetes, and expert questions and answers.39–41 However, only one application provides personalized education.42 Personalization is defined as modification of the nature of the activity that a user is involved in or desires to be involved in, the unique characteristics of the user, and the circumstance within which the user currently resides.43 Five (31%) research articles included an education component, of which three have provided personalized education.16–18 We can personalize the education based on the patient’s profile, the patient context and education, and the goals of the medical treatment process.16 The mobile software of Quinn and coauthors18 allows patients to enter diabetes self-care data on a mobile phone and receive automated, real-time educational, behavioral, and motivational messaging particular to the patient’s condition.
Security and Privacy
The FDA projects that, by 2015, 500 million people will use mobile health applications.44 However, mobile devices used to access health records raised several security- and privacy-related issues, such as vulnerability to loss due to small size, authentication, encryption of data, and use of unsecure public Wi-Fi. The Health Insurance Portability and Accountability Act (HIPAA) includes a Privacy Rule protecting the privacy of individually identifiable health information. The act also includes a security rule, setting national standards for the security of electronic protected health information.45 Most commercial applications do not explain their security features. Out of 71 applications, only 1 application (MyCareConnect LLC) has indicated HIPAA compliance.46 Five applications are passcode protected,47–51 and 2 applications have secure data transmission.52,53 On the other hand, out of 16 applications reported in the literature, 5 indicate HIPAA compliance. Overall, our analysis indicates that further attention is warranted regarding securing and protecting patient information. Applications need to be explicit regarding the extent of HIPAA compliance.
Evaluation
Fifteen out of 16 articles experimentally tested their application. Two studies found no added benefit provided by the use of the application,14,23 whereas the remaining articles reported some type of added benefit. Quinn and coauthors17 showed the highest improvement in hemoglobin A1c (HbA1c; -2.03%). They recruited 30 patients with type 2 diabetes for 3 months and evenly randomized the set. In their study, patient-entered data was sent to the provider where it was analyzed by computational algorithms as well as by a health care team. Feedback was transmitted back to the patient’s mobile application.17 The application is now available commercially in iTunes as AT&T mHealth Solutions Presents Diabetes Manager.32 In another study, where the application was not integrated with the provider, Forjuoh and coauthors10 reported a 1.7% improvement in HbA1c. In this study, the application had the capability to record medication, blood glucose measurement, meals, exercise, test results, and other notes.10 Research studies have shown that use of mobile applications for self-management has positive impacts on developing health behavior through active monitoring of diet intake over time.9,15 However, not all studies report positive outcomes. Lyles and coauthors14 reported that some study participants found smartphones frustrating to use. Two out of eight participants mentioned that they spent hours learning the basic features of the phone, and they continued to launch unwanted programs and end calls accidentally.14 Despite receiving a smartphone and a data plan for free as an incentive, two participants returned the smartphone and signed off from the program.14 This is a concern for large-scale adoption, particularly where such incentives may not even be possible.
Discussion
Overall, the review indicates that mobile applications can be viable tools for diabetes self-management. Mobile appli-cations are generally preferred to Web- or computer-based systems when it comes to usability.18,19 Regimen adherence can be managed through the use of mobile applications, which affects self-efficacy and other enabling factors. The review also seems to point out that diabetes self-management applications are useful to patients as well as providers. Application usage was reported to result in improved health habits such as following a healthy diet, increased physical activities, and increased blood glucose testing.9 Applications proved particularly useful for patients with very high blood sugar or for those having difficulty controlling it. Interestingly, some applications reported use by patients with prediabetes (blood glucose levels that are higher than normal but not yet high enough to be diagnosed as diabetes, i.e., HbA1c 5.7–6.4%1).
Nevertheless, the review also identified a number of issues and limitations with the current state of the art. For example, while clinical guidelines emphasize the role of self-management education as an integral part of diabetes care,1 most diabetes self-management applications that aim to support self-management tasks do not integrate educational information. When available, such information is often generic and is not personalized to the individual patient. This is more prominent in commercial applications. Furthermore, patients tend to perceive data entry automation as an important driver for ease of use and, ultimately, for satisfaction with the technology. Yet among 71 included applications from the AAS, only 3 applications were featured with automatic data upload. Another important missing element in many applications was integration with the personal health record. This may be due to the fact that patients are resistant to pay for a PHR or possibly due to the lack of integration with the EMRs of patients (creating a disincentive to patients and providers alike). Further work is needed to integrate the PHR with EMRs.54 Applications, in many cases, appeared as an electronic version of paper-based logbook systems, reducing the perceived usefulness to patients (particularly less tech-savvy ones). There is a distinct need to include value-added features such as decision support capabilities. Current capabilities are limited to insulin dosage suggestions. The rapid growth in smartphone computing capabilities coupled with cloud computing provide unprecedented opportunities to expand decision support and analytics capabilities into the hands of the end user.55 Such capabilities can be employed to provide personalized education, intelligent feedback based on the input data, medication suggestions based on the multiparameter analysis, and motivational tips based on the patient’s condition. It was not clear whether the applications were designed based on the behavioral science theory; no applications mentioned any specific detail of behavioral theory model in their design. While there is no sufficient evidence on recommended theory or intervention models, the transtheoretical model,56 the social cognitive theory,57 self-efficacy,58 and the health belief model59 were all found to deliver positive outcome.
There are also nonfunctional requirements to consider. One of the major issues in diabetes applications is security. We found only one application that is HIPAA compliant.46 Usability is another consideration, e.g., limited data entry options.60 This is consistent with the findings of Martin and coauthors,5 emphasizing not just data entry, but also usability issues with adjusting personal settings along with learnability, aesthetics, and error management. There is adistinct need to employ a user-centered design that will take into account the needs and characteristics of the individual patient. The usability of mobile applications can be further enhanced as implantable sensors become common, accurate, and safe to use.3
Conclusion
In conclusion, the review attests to the value and potential for such applications to improve diabetes self-management. However, it is important to note that mobile applications-based self-management is not a “silver bullet,” and it is critical to understand that its effect is based on strong behavioral change theory. Such interventions may not be suitable for all patients with diabetes.61 Some patients abandon use due to technical problems,14 and others cannot afford the cost (phone and associated data plans). Work by Torsi and coauthors62 further highlights the need for a user-centered63,64 design approach. Along these lines, the application of sociotechnical design principles65–68 can significantly improve the adoption and wide-scale use of such technology.
Glossary
- (AAS)
Apple App Store
- (EMR)
electronic medical record
- (FDA)
Food and Drug Administration
- (HbA1c)
hemoglobin A1c
- (HIPAA)
Health Insurance Portability and Accountability Act
- (PHR)
Patient Health Record
References
- 1.American Diabetes Association. Standards of Medical Care in Diabetes--2012. Diabetes Care. 2012;35(Suppl 1):S11–63. doi: 10.2337/dc12-s011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Adams KG, Greiner AC, Corrigan JM, editors. Washington DC: National Academies Press; 2004. Report of a summit. The 1st annual crossing the quality chasm summit: a focus on communities. Jan 6-7 2004. [PubMed] [Google Scholar]
- 3.Arsand E, Tatara N, Hartvigsen G. Hershey: IGI Global; 2011. Wirelss and mobile technologies inproving diabets self-management. In: Handbook of research on mobility and computing: evolving technologies and ubiquitous impacts; pp. 136–56. [Google Scholar]
- 4.Chomutare T, Fernandez-Luque L, Arsand E, Hartvigsen G. Features of mobile diabetes applications: review of the literature and analysis of current applications compared against evidence-based guidelines. J Med Internet Res. 2011;13(3):e65. doi: 10.2196/jmir.1874. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Martin C, Flood D, Sutton D, Aldea A, Harrison R, Waite M. A systematic evaluation of mobile application for diabetes management. INTERACT’11; Proceedings of the 13th IFIP TC 13 International Conference on Human-computer interaction, Volume IV; 2011. [Google Scholar]
- 6.Paulweber B, Valensi P, Lindström J, Lalic NM, Greaves CJ, McKee M, Kissimova-Skarbek K, Liatis S, Cosson E, Szendroedi J, Sheppard KE, Charlesworth K, Felton AM, Hall M, Rissanen A, Tuomilehto J, Schwarz PE, Roden M, Paulweber M, Stadlmayr A, Kedenko L, Katsilambros N, Makrilakis K, Kamenov Z, Evans P, Gilis-Januszewska A, Lalic K, Jotic A, Djordevic P, Dimitrijevic-Sreckovic V, Hühmer U, Kulzer B, Puhl S, Lee-Barkey YH, AlKerwi A, Abraham C, Hardeman W, Acosta T, Adler M, AlKerwi A, Barengo N, Barengo R, Boavida JM, Charlesworth K, Christov V, Claussen B, Cos X, Cosson E, Deceukelier S, Dimitrijevic-Sreckovic V, Djordjevic P, Evans P, Felton AM, Fischer M, Gabriel-Sanchez R, Gilis-Januszewska A, Goldfracht M, Gomez JL, Greaves CJ, Hall M, Handke U, Hauner H, Herbst J, Hermanns N, Herrebrugh L, Huber C, Hühmer U, Huttunen J, Jotic A, Kamenov Z, Karadeniz S, Katsilambros N, Khalangot M, Kissimova-Skarbek K, Köhler D, Kopp V, Kronsbein P, Kulzer B, Kyne-Grzebalski D, Lalic K, Lalic N, Landgraf R, Lee-Barkey YH, Liatis S, Lindström J, Makrilakis K, McIntosh C, McKee M, Mesquita AC, Misina D, Muylle F, Neumann A, Paiva AC, Pajunen P, Paulweber B, Peltonen M, Perrenoud L, Pfeiffer A, Pölönen A, Puhl S, Raposo F, Reinehr T, Rissanen A, Robinson C, Roden M, Rothe U, Saaristo T, Scholl J, Schwarz PE, Sheppard KE, Spiers S, Stemper T, Stratmann B, Szendroedi J, Szybinski Z, Tankova T, Telle-Hjellset V, Terry G, Tolks D, Toti F, Tuomilehto J, Undeutsch A, Valadas C, Valensi P, Velickiene D, Vermunt P, Weiss R, Wens J, Yilmaz T. A European evidence-based guideline for the prevention of type 2 diabetes. Horm Metab Res. 2010;42(Suppl 1):S3–36. doi: 10.1055/s-0029-1240928. [DOI] [PubMed] [Google Scholar]
- 7.Sibal L, Home PD. Management of type 2 diabetes: NICE guidelines. Clin Med. 2009;9(4):353–7. doi: 10.7861/clinmedicine.9-4-353. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Preuveneers D, Berbers Y. Mobile phones assisting with health self-care: a diabetes case study; Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services; Amsterdam, The Netherlands: 2008. Sep 2_5, [Google Scholar]
- 9.Arsand E, Tatara N, Østengen G, Hartvigsen G. Mobile phone-based self-management tools for type 2 diabetes: the few touch application. J Diabetes Sci Technol. 2010;4(2):328–36. doi: 10.1177/193229681000400213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Forjuoh SN, Reis MD, Couchman GR, Ory MG. Improving diabetes self-care with a PDA in ambulatory care. Telemed J E Health. 2008;14(3):273–9. doi: 10.1089/tmj.2007.0053. [DOI] [PubMed] [Google Scholar]
- 11.Farmer A, Gibson O, Hayton P, Bryden K, Dudley C, Neil A, Tarassenko L. A real-time, mobile phone-based telemedicine system to support young adults with type 1 diabetes. Inform Prim Care. 2005;13(3):171–7. doi: 10.14236/jhi.v13i3.594. [DOI] [PubMed] [Google Scholar]
- 12.Harris LT, Tufano J, Le T, Rees C, Lewis GA, Evert AB, Flowers J, Collins C, Hoath J, Hirsch IB, Goldberg HI, Ralston JD. Designing mobile support for glycemic control in patients with diabetes. J Biomed Inform. 2010;43(5 Suppl):S37–40. doi: 10.1016/j.jbi.2010.05.004. [DOI] [PubMed] [Google Scholar]
- 13.Istepanian RS, Zitouni K, Harry D, Moutosammy N, Sungoor A, Tang B, Earle KA. Evaluation of a mobile phone telemonitoring system for glycaemic control in patients with diabetes. J Telemed Telecare. 2009;15(3):125–8. doi: 10.1258/jtt.2009.003006. [DOI] [PubMed] [Google Scholar]
- 14.Lyles CR, Harris LT, Le T, Flowers J, Tufano J, Britt D, Hoath J, Hirsch IB, Goldberg HI, Ralston JD. Qualitative evaluation of a mobile phone and web-based collaborative care intervention for patients with type 2 diabetes. Diabetes Technol Ther. 2011;13(5):563–9. doi: 10.1089/dia.2010.0200. [DOI] [PubMed] [Google Scholar]
- 15.Ma Y, Olendzki BC, Chiriboga D, Rosal M, Sinagra E, Crawford S, Hafner AR, Pagoto SL, Magner RP, Ockene IS. PDA-assisted low glycemic index dietary intervention for type II diabetes: a pilot study. Eur J Clin Nutr. 2006;60(10):1235–43. doi: 10.1038/sj.ejcn.1602443. [DOI] [PubMed] [Google Scholar]
- 16.Mohan P, Marin D, Sultan S, Deen A. MediNet: personalizing the self-care process for patients with diabetes and cardiovascular disease using mobile telephony. Conf Proc IEEE Eng Med Biol Soc. 2008;2008:755–8. doi: 10.1109/IEMBS.2008.4649262. [DOI] [PubMed] [Google Scholar]
- 17.Quinn CC, Clough SS, Minor JM, Lender D, Okafor MC, Gruber-Baldini A. WellDoc mobile diabetes management randomized controlled trial: change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther. 2008;10(3):160–8. doi: 10.1089/dia.2008.0283. [DOI] [PubMed] [Google Scholar]
- 18.Quinn CC, Shardell MD, Terrin ML, Barr EA, Ballew SH, Gruber-Baldini AL. Cluster-randomized trial of a mobile phone personalized behavioral intervention for blood glucose control. Diabetes Care. 2011;34(9):1934–42. doi: 10.2337/dc11-0366. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Kollmann A, Riedl M, Kastner P, Schreier G, Ludvik B. Feasibility of a mobile phone-based data service for functional insulin treatment of type 1 diabetes mellitus patients. J Med Internet Res. 2007;9(5):e36. doi: 10.2196/jmir.9.5.e36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Sultan S, Mohan P. Self-regulated learning in a mobile health initiative for diabetic patients; 2012 IEEE 12th International Conference on Advanced Learning Technologies; 2012. pp. 706–7. [Google Scholar]
- 21.Nes AA, van Dulmen S, Eide E, Finset A, Kristjánsdóttir OB, Steen IS, Eide H. The development and feasibility of a web-based intervention with diaries and situational feedback via smartphone to support self-management in patients with diabetes type 2. Diabetes Res Clin Pract. 2012;97(3):385–93. doi: 10.1016/j.diabres.2012.04.019. [DOI] [PubMed] [Google Scholar]
- 22.Frøisland DH, Arsand E, Skårderud F. Improving diabetes care for young people with type 1 diabetes through visual learning on mobile phones: mixed-methods study. J Med Internet Res. 2012;14(4):e111. doi: 10.2196/jmir.2155. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Rollo ME, Ash S, Lyons-Wall P, Russell A. Trial of a mobile phone method for recording dietary intake in adults with type 2 diabetes: evaluation and implications for future applications. J Telemed Telecare. 2011;17(6):318–23. doi: 10.1258/jtt.2011.100906. [DOI] [PubMed] [Google Scholar]
- 24.Tasker AP, Gibson L, Franklin V, Gregor P, Greene S. What is the frequency of symptomatic mild hypoglycemia in type 1 diabetes in the young?: assessment by novel mobile phone technology and computer-based interviewing. Pediatr Diabetes. 2007;8(1):15–20. doi: 10.1111/j.1399-5448.2006.00220.x. [DOI] [PubMed] [Google Scholar]
- 25.Carroll AE, Marrero DG, Downs SM. The HealthPia GlucoPack Diabetes phone: a usability study. Diabetes Technol Ther. 2007;9(2):158–64. doi: 10.1089/dia.2006.0002. [DOI] [PubMed] [Google Scholar]
- 26.Chun H, Kang J, Kim KJ, Park KS, Kim HC. IT-based diagnostic instrumentation systems for personalized healthcare services. Stud Health Technol Inform. 2005;117:180–90. [PubMed] [Google Scholar]
- 27.Telcare Inc. Diabetes Pal Star Blood Glucose Manager. http://itunes.apple.com/us/app/mytelcare-diabetes-pal-5-star/id495245376?mt=8Accessed April 24, 2012. [Google Scholar]
- 28.sanofi-aventis U.S. LLC. iBGStar Diabetes Manager. https://itunes.apple.com/us/app/ibgstar-diabetes-manager/id506018173?mt=8. Accessed September 7, 2012. [Google Scholar]
- 29.Gilport Enterprises. RapidCalc Diabetes Manager. http://itunes.apple.com/us/app/rapidcalc-diabetes-manager/id400469609?mt=8. Accessed April 25, 2012. [Google Scholar]
- 30.Knowledge is Power Solutions. Insulin Dose Advisor ControlWizard. http://itunes.apple.com/us/app/insulin-dose-advisor-controlwizard/id405859720?mt=8. Accessed April 29, 2012. [Google Scholar]
- 31.Food and Drug Administration. Draft guidance for industry and Food and Drug Administration staff - mobile medical applications. http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm263280.htm. Accessed November 1, 2012. [Google Scholar]
- 32.AT&T Services Inc. AT&T mHealth Solutions Presents DiabetesManager. https://itunes.apple.com/us/app/at-t-mhealth-solutions-presents/id518413489?mt=8. Accessed August 15, 2012. [Google Scholar]
- 33.Ciemins E, Coon P, Sorli C. An analysis of data management tools for diabetes self-management: can smart phone technology keep up? J Diabetes Sci Technol. 2010;4(4):958–60. doi: 10.1177/193229681000400427. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.WWW Machealth Pty Ltd. Your Diabetes Guardian. http://itunes.apple.com/us/app/your-diabetes-guardian/id394280782?mt=8. Accessed April 25, 2012. [Google Scholar]
- 35.Marshall P. Personal health records their benefits and the role of standards. http://www.ncvhs.hhs.gov/020724p5.pdf. Accessed 1 May, 2012. [Google Scholar]
- 36.Azumio Inc. Glucose Buddy. http://itunes.apple.com/us/app/glucose-buddy-diabetes-helper/id294754639?mt=8. Accessed April 20, 2012.
- 37.Funnell MM, Brown TL, Childs BP, Haas LB, Hosey GM, Jensen B, Maryniuk M, Peyrot M, Piette JD, Reader D, Siminerio LM, Weinger K, Weiss MA. National standards for diabetes self-management education. Diabetes Care. 2012;35(Suppl 1):S101–8. doi: 10.2337/dc12-s101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Norris SL, Lau J, Smith SJ, Schmid CH, Engelgau MM. Self-management education for adults with type 2 diabetes: a meta-analysis of the effect on glycemic control. Diabetes Care. 2002;25(7):1159–71. doi: 10.2337/diacare.25.7.1159. [DOI] [PubMed] [Google Scholar]
- 39.BHI Technologies Inc. Diabetes App Lite. http://itunes.apple.com/us/app/diabetes-buddy-lite-control/id387337850?mt=8.
- 40.Fetal Diagnostic Center. Diagnosis and Management of Gestational Diabetes. http://itunes.apple.com/gb/app/diagnosis-management-gestational/id454529365?mt=8.
- 41.dLife. Diabetes Companion. http://itunes.apple.com/us/app/diabetes-companion/id360403719?mt=8. Accessed April 15, 2012.
- 42.Coheso Inc. Track3. http://itunes.apple.com/us/app/track3-diabetes-planner-carb/id318622618?mt=8.
- 43.Marin D, Mohan P. Personalisation in mobile learning. Int J Mob Learn Organ. 2009;3(1):25–43. [Google Scholar]
- 44.Patel B. Food and Drug Administration. Mobile Medical Apps. http://www.fda.gov/downloads/MedicalDevices/NewsEvents/WorkshopsConferences/UCM271893.pdf. Accessed November 1, 2012.
- 45.HIPAA Definitions for Electronic Protected Health Information. http://hipaanews.org/. Accessed May 7, 2012.
- 46.BlueLoop LLC. BlueLoop. http://itunes.apple.com/us/app/blueloop/id457845904?mt=8. Accessed April 15, 2012.
- 47.Deltaworks. Blood Pal Free - Glucose Tracker. https://itunes.apple.com/us/app/blood-pal-free-glucose-tracker/id381244630?mt=8. Accessed August 15, 2012.
- 48.Diabetic Friend Ltd. DF Diary. https://itunes.apple.com/us/app/df-diary/id378775922?l=es&mt=8. Accessed Auguest 1, 2012.
- 49.websmc. Diabetes Management. https://itunes.apple.com/us/app/diabetes-management/id480557891?mt=8. Accessed August 15, 2012.
- 50.Maxwell Software. Glucose Companion Free. https://itunes.apple.com/us/app/glucose-companion-free/id464241333?mt=8. Accessed August 15, 2012.
- 51.WWW Machealth Pty Ltd. Your Diabetes Guardian. https://itunes.apple.com/lk/app/your-diabetes-guardian/id394280782?mt=8. Accessed August 12, 2012.
- 52.softbycloud.com. GliControl. https://itunes.apple.com/us/app/glicontrol/id453603829?mt=8. Accessed August 15, 2012.
- 53.Enemra Inc. GlucoPulse. https://itunes.apple.com/us/app/glucopulse/id441554963?mt=8. Accessed August 15, 2012. [Google Scholar]
- 54.Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc. 2006;13(2):121–6. doi: 10.1197/jamia.M2025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Nkosi MT, Mekuria F. Cloud Computing for Enhanced Mobile Health Applications; 2nd IEEE International Conference on Cloud Computing Technology and Science (CloudCom 2010); 2010. pp. 629–33. [Google Scholar]
- 56.Kim CJ, Kang DH. Utility of a Web-based intervention for individuals with type 2 diabetes: the impact on physical activity levels and glycemic control. Comput Inform Nurs. 2006;24(6):337–45. doi: 10.1097/00024665-200611000-00008. [DOI] [PubMed] [Google Scholar]
- 57.Liebreich T, Plotnikoff RC, Courneya KS, Boule N. Diabetes NetPLAY: A physical activity website and linked email counselling randomized intervention for individuals with type 2 diabetes. Int J Behav Nutr Phys Act. 2009;6:18. doi: 10.1186/1479-5868-6-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Glasgow RE, Boles SM, McKay HG, Feil EG, Barrera M., Jr The D-Net diabetes self-management program: long-term implementation, outcomes, and generalization results. Prev Med. 2003;36(4):410–9. doi: 10.1016/s0091-7435(02)00056-7. [DOI] [PubMed] [Google Scholar]
- 59.Richardson CR, Mehari KS, McIntyre LG, Janney AW, Fortlage LA, Sen A, Strecher VJ, Piette JD. A randomized trial comparing structured and lifestyle goals in an internet-mediated walking program for people with type 2 diabetes. Int J Behav Nutr Phys Act. 2007;4:59. doi: 10.1186/1479-5868-4-59. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.David Tessitore. Diabetic Tracker Unlimited. http://itunes.apple.com/us/app/diabetic-tracker-unlimited/id306198686?mt=8. Accessed April 25, 2012. [Google Scholar]
- 61.Liang X, Wang Q, Yang X, Cao J, Chen J, Mo X, Huang J, Wang L, Gu D. Effect of mobile phone intervention for diabetes on glycaemic control: a meta-analysis. Diabet Med. 2011;28(4):455–63. doi: 10.1111/j.1464-5491.2010.03180.x. [DOI] [PubMed] [Google Scholar]
- 62.Torsi S, Nasr N, Wright P. Proceedings of the 2nd International Conference on Pervasive Technologies Related to Assistive Environments, PETRA 2009. Corfu, Greece: 2009. Jun 9-13, User-centered design for supporting the self-management of chronic illnesses: an interdisciplinary approach. [Google Scholar]
- 63.Baek EO, Cagiltay K, Boling E, Frick T. User-centered design and development. Handbook of research on educational communications and technology. In: Spector JM, Merrill MD, van Merrienboer JJ, Driscoll MF, editors. Mahwah: Lawrence Erlbaum Associates; 2007. [Google Scholar]
- 64.Silva da Silva T, Martin A, Maurer F, Silveira M. User-centered design and agile methods: a systematic review. 2011 AGILE Conference. 2011:77–86. [Google Scholar]
- 65.Cherns A. Principles of socio-technical design. Human Relations. 1976;29(8):783–92. [Google Scholar]
- 66.Cherns A. Principles of sociotechnical design revisted. Human Relations. 1987;40(3):153–61. [Google Scholar]
- 67.Clegg CW. Sociotechnical principles for system design. Appl Ergon. 2000;31(5):463–77. doi: 10.1016/s0003-6870(00)00009-0. [DOI] [PubMed] [Google Scholar]
- 68.Carayon P. Human factors of complex sociotechnical systems. Appl Ergon. 2006;37(4):525–35. doi: 10.1016/j.apergo.2006.04.011. [DOI] [PubMed] [Google Scholar]