Abstract
Background:
An important global public health problem in many economically developed countries, in particular in Kazakhstan, is the leading cause of morbidity and mortality from cardiovascular disease (CVD), both in urban and rural areas.
Objective:
This study aims to develop the design of a mobile application for smartphones for cardiac patients, taking into account the specifics of outpatient care.
Methods:
This methodological study includes identified educational and prevention content for CVD patients to develop an education and monitoring tool using a smartphone app. The application was developed according to the stages of analysis, design, development, implementation and evaluation, which is a systematic training model of design. The levels of satisfaction with the developed application for smartphones among 65 outpatients with CVD were assessed using a questionnaire during their visit to a medical organization.
Results:
The «My Heart» smartphone app was developed based on a review of the literature related to educating patients with CVD, consulting specialists, monitoring and searching for medical smartphone applications that are already available. The content of education and registration includes three main sections, containing basic questions on registration and monitoring of the condition, materials on education and prevention of CVD, such as a daily questionnaire and nutritional advice. After modification based on expert feedback, the application was finally developed and evaluated by patients who reported being satisfied with the usefulness of the application.
Conclusions:
This application is developed as a research tool to further conduct a study in CVD patients. Current evaluation was a pilot testing wherein this application showed promising results.
Keywords: Mobile Applications, Ambulatory Care, Cardiovascular Diseases, Patients, Chronic Disease
1. BACKGROUND
Cardiovascular disease (CVD) continues to be the leading cause of death and disability worldwide, as well as leading in socioeconomic damage (1). Practically, all health systems regularly adopt special research and social programs aimed at combating CVDs and develop various innovative platforms for CVD prevention during a pandemic (2). There is research that could provide evidence for the use of mobile phones and web services for mentoring and self-management in a home care model aimed at sustainable behavioral change in cardiac rehabilitation (CR) patients (3, 4). The ever-increasing penetration of smartphones, coupled with embedded sensors and modern communication technologies, makes them an attractive technology for providing continuous and remote monitoring of human health and well-being with little additional cost (5, 6).
In the formation of the innovative policy of a medical organization, the selection and implementation of tools that make it possible to significantly improve the quality of medical care, taking into account some risks, acquire the main importance (6, 7). In this regard, it becomes necessary to obtain recommendations in relation to specific quality management technologies, the most valuable of which are those implemented in practice and proven to be highly effective.
Similar research in the field of mobile health has been carried out in many countries, research and implementation, especially in the field of prevention, CR, and education. Mobile health includes simple strategies such as short messaging (SMS) or text messaging in successful short-term smoking cessation, weight loss, and diabetes management programs (8-12).
Smartphone-based technologies along with broadband connectivity are changing the way modern cardiology is practiced. The ever-broadening connectivity and -increasing capabilities of smartphone- based technologies can better monitor, diagnose, and prevent CVD (13). Mobile health delivery has the potential to improve access to CR and heart failure management for patients unable to attend traditional center-based CR programs. Feasibility testing of culturally appropriate mHealth delivery for CR and heart failure management is required in rural and remote settings with subsequent implementation and evaluation into local health-care services (14). These applications will be very convenient to use for heart patients also living in remote areas that are very far from the city. At the same time, when developing such applications, it is necessary to take into accounts the characteristics and behavioral skills of the population.
Patients with heart disease have various lifestyle risk factors, therefore, there is a need to increase motivation by acquiring knowledge about disease management and lifestyle improvement to prevent CVD recurrence after discharge. Overall, many studies have shown that the development of smartphone applications for CVD patients that enable self-supervised learning and relearning at home and at work will increase the availability of information. Against this backdrop, much research has been directed toward the development of an educational smartphone app as a self-guided learning tool for CVD patients (3, 15, 16).
Taking into account the conducted scientific research in the field of developing mobile applications for cardiac patients, it showed us what the rationale for their development became: post-discharge care (CR), self-management and self-learning (self-control), and distance between cities and towns (remote areas). Furthermore, each country, when developing a mobile application, should take into account the health-care system, features of tradition, lifestyle, preventive measures, geographical location, and more. Therefore, we cannot use one developed mobile application for cardiac patients all over the world.
2. OBJECTIVE
This study aims to develop the design of a mobile application for smartphones for cardiac patients, taking into account the specifics of outpatient care.
3. MATERIAL AND METHODS
Analysis and design phase
The design of the mobile application was developed according to a systematic instructional design model. The prerequisites for the development were the analysis of the electronic database of medical organizations, the questioning of cardiological patients and healthcare workers, and the study of international experience in the development of mobile devices in the field of health care.
The electronic resources of medical organizations providing outpatient care for 2 years (2019 and 2020) were used, the data were taken from the electronic register of dispensary patients (ERDP), which are used to record cardiology outpatients of Kazakhstan (Akmola region). The ERDP portal allows employees of medical organizations to automate dispensary registration and observation of patients, storage and formation of a register of dispensary patients, processing, and provision of statistical and analytical data. Dispensary observation of patients with chronic non-communicable diseases of various nosologies is carried out throughout the republic in all state organizations providing primary health care. To justify the development of a mobile application, it was necessary to analyze the system for monitoring patients at the dispensary (diagnoses according to the International Classification of Diseases: I25.0-I25.9). We studied the dynamics for 2 years in Akmola region of Kazakhstan and paid attention to patients registered in the dispensary, both in the city and in the countryside.
We analyzed the quality of medical services for cardiological patients and over the past 2 years, a trend towards an increase in health indicators was revealed. For this, ERDP was analyzed (number of patients in 2019, n = 3660, in 2020, n = 7014).
In general, we wanted to show the peculiarity of providing medical care to cardiac patients in our country on an outpatient basis and a general scheme for collecting this data into electronic resources, starting with a patient’s visit to the clinic and entering data to the final result (Republican Center for Electronic Health of the Ministry Health Care of the Republic of Kazakhstan) (Figure 1).
Figure 1. Block diagram of electronic register of dispensary patient for taking a patient for dispensary registration.
Participants
All the interviewed patients were recruited from the CVD outpatients’ dispensary registration, with whom the primary care doctors talked at the reception. We included patients selected by their primary care physician who were over 18 years of age, spoke Kazakh and Russian, had a diagnosis of CVD, and reported regular smartphone use. We excluded patients under 18 years of age and registered at the dispensary for other chronic diseases, also with cognitive impairments.
The user satisfaction survey is conducted in institutions that provide outpatient care to both urban and rural populations of two regions (urban and rural).
Procedure and ethical considerations
Reviewed, discussed and approved by the Local Ethics Committee at Astana Medical University (Kazakhstan, Astana city). Protocol No. 4 of 20.02.2020
Development phase
Based on the goals and structure of education and prevention, an educational application was developed during the analysis and design phase. The training and monitoring content were organized after considering the objectives and scope of training based on the training menu flowchart, and the first version of the educational application was developed by a professional web developer. All content has been designed to be easy to view, allowing users to zoom in and out on the screen. After several consultations on the adequacy of the first version of the application and after finalization by cardiologists and nurses of the final version of the application, the educational and preventive application “My Heart” is finally developed.
The appropriateness of the program was determined by assessing the effectiveness and efficiency of the program in the implementation and evaluation phases. The “My Heart” app was developed to run in the Android phone environments.
The questionnaire was designed based on earlier work done by (17) to evaluate the level of satisfaction with multimedia learning content. The evaluation questionnaire included 11 items assessed on a 5 point scale (1=Poor/Strongly Disagree to 5 = Excellent/ Strongly Agree) on the overall experience, quality of content, usefulness, and mobile application design and functionality.
4. RESULTS
Results of the analysis of indicators of ambulatory care
In general, the number of CVD is growing from year to year, this is confirmed by the data from the ERDP. The number of cardiological patients registered with the dispensary has sharply doubled from 3660 in 2019 to 7014 in 2020 (+2.03; p <0.05). We have analyzed the data structure of cardiac patients at the dispensary registration (Table 1).
Table 1. The social structure of cardiac patients in the dispensary.
| Item | Cardiological patients registered with dispensaries | |||
|---|---|---|---|---|
| 2019 year, n | 2020 year, n | +/- | p-value | |
| Gender | ||||
| male | 2670 (73%) | 5230 (75%) | +1.95 | <0.05 |
| female | 990 (28%) | 1784 (25%) | +1.80 | <0.05 |
| Age groups (years) | ||||
| 18-34 | 298 (8%) | 546 (8%) | +1.83 | <0.05 |
| 35-49 | 876 (24%) | 2110 (30%) | +2.4 | <0.05 |
| 50-64 | 1179 (32%) | 1814 (25%) | +1.53 | <0.05 |
| 65+ | 1307 (36%) | 2544 (36%) | +1.94 | <0.05 |
| Social status | ||||
| students | 121 (3%) | 204 (3%) | +1.68 | <0.05 |
| workers | 419 (11%) | 1165 (17%) | +2.78 | <0.05 |
| state employee | 532 (15%) | 1132 (16%) | +2.1 | <0.05 |
| unemployed | 638 (17%) | 1877 (27%) | +2.94 | <0.05 |
| retiree | 1950 (53%) | 2614 (37%) | +1.3 | <0.05 |
| others | 0 | 22 (0) | 0 | _ |
| Total: | n=3660 | n=7014 | +2.03 | <0.05 |
According to the table, gender, men have predominance over women in 2019 and in 2020, 75% of the share of heart diseases is occupied by men. Ages data are similar in both years 65 and older occupy 35% and 50–64 age 25–30%. The social status of cardiac patients registered with dispensaries showed that retirees occupy about 55% in 2019, and in 2020, it decreases to 38%, and a sharp increase in cardiac patients is unemployed (2019 – 17.4% and 2020 – 26.8%). There was a statistically significant difference between the age groups (p < 0.05).
This shows that it will be more convenient for elderly people and their relatives to enter data into a mobile application, rather than go to the clinic every time or wait for a medical worker to visit their homes and for medical workers to conduct remote monitoring in dynamics in the office and make a decision regarding each patient.
Mobile application design
In Kazakhstan, there are currently no active mobile applications for individual disease nosologies. We have the «DamuMed» mobile application on the market, which is used by the population and medical workers. The «DamuMed» mobile application is constantly being improved. Now, there is application for the hospital and for the clinic separately. In general, this application is designed for general access to outpatient services, but not for individual groups of diseases, it is convenient to use an appointment with a general practitioner or nurse practitioner.
In our design and functionality of the mobile application, the following main menus were covered three parts. The first part – Registration of Patient Data at the dispensary account (“D” account) as the information about daily well-being (as in the form of a questionnaire), such symptoms as headache, chest pain, shortness of breath, swelling, blood pressure, as well as information on medication, diet, and bad habits, results of laboratory tests, and an emergency button for an alarm call. The second part is an Assessment of Your Health by Zones and the third part – Materials on the Training Menu (Figure 2).
Figure 2. Mobile application for patients on the “D” account.
You can to enter your data (identification number, age, height, and weight) and register for further use of the mobile application.
There is a section on the results of laboratory tests, which the patient passes on according to the schedule, it is necessary for monitoring the dynamics of the state of health. The main data are entered by a medical staff, and the test results are automated and sent to the doctor. If the patient receives data on their hands, they can check their data in the mobile application. It was especially important to turn on the alarm button for cardiac patients, the exacerbation of this disease is often and unexpected. In this regard, we decided to include the telephone number of relatives, except for emergency medical services, for a quick response.
The next section is an assessment of one’s health using certain zones, which was developed by the Republican Center for Health Development of the Ministry of Health of the Republic of Kazakhstan “Chronic Heart Failure: Clinical Guidelines for Nurses on Patient Management (PHC and Hospital), adapted clinical guidelines.” From this document, data were taken on the assessment of your health in three zones: Green zone – all symptoms are under control (no shortness of breath and no weight gain) or there is a weight loss of about 500–900 g (these changes can occur within a few days), no edema feet, ankles, legs, or abdomen, there are no chest pains), yellow zone – call your GP or nurse (you are gaining 700–1200 g per day or you recorded a weight gain of 2 kg or more per week, you feel an increase in shortness of breath, you have more swelling in your feet, ankles, legs, or abdomen, you feel more tired, no strength, you have a dry, hacking cough, you feel dizzy, and you feel anxious, it becomes more difficult for you to breathe while lying down, you are forced to sleep sitting in a chair), and red zone – call an ambulance or panic button (it is difficult for you to breathe, persistent shortness of breath, you sit motionless in a chair/in a chair, and chest pain). This card was borrowed from the booklet heart failure.
The last section includes educational materials, such as information about diseases of the circulatory system, risk factors, and recommendations on diet, physical activity, and healthy foods. Materials will be included in the form of slides, presentations, and videos. Patients need to get acquainted daily and study the necessary materials for the prevention and maintenance of their health, in addition to taking medications. Such materials on the usefulness and harmfulness of products, as well as the necessary light physical activity to some extent, can affect the improvement of health.
Results of User Satisfaction Evaluation
Forty-five randomly selected CVD patients who consented to participate in the survey were given the test link for application installation. They downloaded and used application for 2 weeks; followed by providing feedback using user satisfaction survey questionnaire. The mean age of users was 52, of which 42.86% were female and 57.14% were male. A total of 67.14% belonged to urban region while 32.86% patients were from rural area. More than half of the patients (51.23%) were graduate or above.
Overall, average of satisfied users (who rated application 4 and above) accounts to 70%. More than 85% of users were satisfied with the usefulness, 57% were satisfied with the design, layout and functionality and almost 70% were satisfied with the quality of content. Table 2 summarizes mean values with standard deviation and percentage of users who rated the mobile application 4 and above for various individual questions asked in the questionnaire.
Table 2. User’s satisfaction analysis for “My Heart” mobile application (n = 65).
| Questionnaire Category | Mean ± SD | No. of Users Rated >4(%) |
||
|---|---|---|---|---|
| urban (n=35) | rural (n=30) | urban (n=35) | rural (n=30) | |
| 1. Composition of contents | ||||
| 1.1 Content quality (design, color, font size, access to content, etc.) | 4.09±0.7 | 4.01±0.54 | 58.0 (87) | 49.0 (80) |
| 1.2 The structure of the content is suitable for your medical examination (medical record, online tests, materials on the prevention of cardiovascular diseases and others) | 4.30±0.7 | 4.15±0.58 | 61.0 (91) | 45.0 (74) |
| 1.3 Information provided is clear and understandable | 4.46±0,7 | 4.22±0.42 | 56.0 (84) | 52.0 (85) |
| 2. Usefulness | ||||
| 2.1. Usefulness of use | 4.55±0,7 | 4.30±0.55 | 62.0 (93) | 59.0 (97) |
| 2.2 Usefulness for health monitoring | 4.66±0.8 | 4.42±0.64 | 64.0 (96) | 58.0 (95) |
| 2.3 Useful for managing your own health | ||||
| 2.4 target recommendations for loads and other information | ||||
| 3. Application design and functionality | ||||
| 3.1 General content and sections of the application | 4.41±0,7 | 4.55±0.67 | 55.0 (82) | 48.0 (79) |
| 3.2 Application Availability | 4.56±0.6 | 4.42±0.34 | 62.0 (93) | 54.0 (88) |
| 3.3 Easy to download and apply | 4.25±0,3 | 4.24±0.48 | 57.0 (85) | 53.0 (87) |
| 3.4 Content logic (registration, profile, health indicators, calendar for recording, survey results, training content, analyzes) | 4.63±0,4 | 4.53±0.64 | 62.0 (93) | 55.0 (90) |
5. DISCUSSION
The main goal of our study was to analyze the situation with cardiac patients in an outpatient setting and to offer one of the tools to improve the situation with CR.
Such studies have been carried out in many countries, taking into account the peculiarities of the provision of medical care to cardiac patients (19).
In general, the advantages of such technologies are obvious: On the one hand, they can make health care more accessible, convenient, and understandable for the patient, on the other hand, they can provide the doctor with comprehensive technical and advisory support, which will ultimately lead to an improvement in the quality of medical care (18). It has been proven that the success of therapy is determined by the effectiveness of the interaction between the doctor and the patient, their equal responsibility for the result. The active introduction of mobile technologies into everyday clinical practice improves this interaction and brings it to a qualitatively new level.
The results showed that medical professionals are generally willing to work with a mobile application for remote monitoring of patients who suffer from various chronic diseases. Patients want to see the content of applications in terms of monitoring the dynamics of their condition, online consultations on taking medications, and recommendations of various types depending on the current state of the patient. Along with this, the population will be able to be mobile in terms of using online applications, both for themselves and for loved ones who suffer from various chronic diseases (18, 19).
According to the questionnaires, in general, the population is ready to use mobile devices, since almost everyone has a mobile phone. The experience of other countries shows that a survey of the population is a necessary part in conducting a study of this kind (20-23). Since the issue of preference and inclusion of the necessary moments in a mobile application are being studied. An important issue remains after the development and implementation of such technologies, to study the satisfaction of the population with this tool.
6. CONCLUSION
According to the analysis, it was revealed how much the service of medical services has improved using medical information systems. In general, the need for medical applications in Kazakhstan is no less than abroad. The studied materials showed a trend towards an increase in the incidence of CVD both in the city and in the countryside.
The developed design of the mobile application requires further improvement, taking into account the peculiarities of dispensary registration. In our study, we were able to identify the target group of patients with chronic diseases who are ready to use medical mobile applications. At the same time, the high prevalence of chronic non-communicable diseases determines the relevance of the development and widespread implementation of specialized mobile applications for patients.
Patient Consent Form:
All participants were informed about subject of the study.
Acknowledgment:
The authors express their gratitude to the doctors of clinics and specialists of statistic offices who helped in collecting materials, cardiologists who advised us.
Author’s contribution:
AKT, ShK, AT gave a substantial contribution to the conception and design of the work. ShK, MZh gave a substantial contribution of data. AKT, ShK, AT, gave a substantial contribution to the acquisition, analysis, or interpretation of data for the work. ZhS, MZh, had a part in article preparing for drafting or revising it critically for important intellectual content. All authors gave final approval of the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Conflicts of interest:
There are no conflicts of interest.
Financial support and sponsorship:
None.
REFERENCES
- 1.Boytsov S.A, Balanova Yu.A, Shalnova S.A, et al. Arterial hypertension among persons 25-64 years old: Prevalence, awareness, treatment and control. Cardiovascular Therapy and Prevention. 2014;13(4):4–14. doi: 10.15829/1728-8800-2014-4-4-14. 2014 (In Russ.) [DOI] [Google Scholar]
- 2.Boukhris M, Hillani A, Moroni F, Annabi MS, et al. Cardiovascular implications of the COVID-19 Pandemic: A global perspective. Can J Cardiol. 2020;36(7):1068–1080. doi: 10.1016/j.cjca.2020.05.018.2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Cho MJ, Sim JL, Hwang SY. Development of smartphone educational application for patients with coronary artery disease. Healthc Inform Res. 2014;20(2):117–124. doi: 10.4258/hir.2014.20.2.117. 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Jain YS, Garg A, Jhamb DK, Jain P, Karar A. Preparing India to leverage power of mobile technology: Development of a bilingual mobile health tool for heart patients. Cardiovasc Hematol Agents Med Chem. 2019;17(2):125–134. doi: 10.2174/18715.2019. [DOI] [PubMed] [Google Scholar]
- 5.Lv Q, Jiang Y, Qi J, Zhang Y, Zhang X, Fang L, et al. Using mobile apps for health management: A new health care mode in China. JMIR Mhealth Uhealth. 2019;7(6):e10299. doi: 10.2196/10299. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Guo Y, Chen Y, Lane DA, Liu L, Wang Y, Lip GY. Mobile health technology for atrial fibrillation management integrating decision support, education, and patient involvement: Maf app trial. Am J Med. 2017;130(12):1388–1396e6. doi: 10.1016/j.am.2017. [DOI] [PubMed] [Google Scholar]
- 7.Aldabergenova G, Turgambayeva A, Brusati LG. Problems of the Organization of the Labor Process That Affect the Effectiveness of the Work of Kazakhstani Doctors. Iran J Public Health. 2021 Oct;50(10):2144–2145. doi: 10.18502/ijph.v50i10.7520. 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Medvedev N.V, Egorov K.N. Effectiveness of Management of Patients with Chronic Heart Failure at the Ambulatory Stage. In: Effective Clinical Practice: Challenges and Opportunities for; Materials of the International Scientific and Practical Conference.2017. [Google Scholar]
- 9.Langford A.T, Solid C.A, Scott E, et al. Mobile Phone Ownership, Health Apps, and Tablet Use in US Adults With a Self-Reported History of Hypertension: Cross- Sectional Study. JMIR MHealth Uhealth. 2019;7(1):e12228. doi: 10.2196/12228. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Boulos MN, Wheeler S, Tavares C, Jones R. How smartphones are changing the face of mobile and participatory healthcare: An overview, with example from eCAALYX. BioMed Eng Online. 2011;10:24. doi: 10.1186/1475-925x-10-24. 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Lewis TL, Wyatt JC. mHealth and mobile medical apps: A framework to assess risk and promote safer use. J Med Internet Res. 2014;16(9):e210. doi: 10.2196/jmir.3133. 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Asvall’s.WHO, J. E. 1. Vol. 13. International Journal of Occupational Medicine and Environmental Health; 2000. Health for all in the 21st century a policy framework for the WHO European Region/February 2000; pp. 5–13. [PubMed] [Google Scholar]
- 13.Nguyen HH, Silva JN. Use of smartphone technology in cardiology. Trends Cardiovasc Med. 2016;26(4):376–186. doi: 10.1016/j.tcm.2015.11.002. 2016. [DOI] [PubMed] [Google Scholar]
- 14.Hamilton SJ, Mills B, Birch EM, Thompson SC. Smartphones in the secondary prevention of cardiovascular disease: A systematic review. BMC Cardiovasc Disord. 2018;18(1):25. doi: 10.1186/s12872-018-0764-x. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kim YG, Lee DH. Smartphone application market trend analysis of the country domestic and other countries. KIISC Rev. 2011;21(1):26–37. [Google Scholar]
- 16.Lönnberg L, Ekblom-Bak E, Damberg M. Improved unhealthy lifestyle habits in patients with high cardiovascular risk: Results from a structured lifestyle programme in primary care. Ups J Med Sci. 2019;124(2):94–104. doi: 10.1080/03009734.201. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kim EJ, Hwang SY. Development of Web-based learn¬ing program on cardiopulmonary emergency care fo¬cused on clinical scenarios. J Korean Acad Adult Nurs. 2010;22(1):70–79. doi: 10.4258/hir.2014.20.2.117. 2010. [DOI] [Google Scholar]
- 18.Misra S, Lewis TL, Aungst TD. Medical application use and the need for further research and assessment for clinical practice: Creation and integration of standards for best practice to alleviate poor application design. JAMA Dermatol. 2013;149(6):661–2013. doi: 10.1001/jamadermatol.2013.606. [DOI] [PubMed] [Google Scholar]
- 19.Widmer RJ, Collins NM, Collins CS, West CP, Lerman LO, Lerman L. Digital health interventions for the prevention of cardiovascular disease: A systematic review and meta-analysis. Mayo Clin Proc. 2015;90(4):469–480. doi: 10.1016/j.mayocp.2014. 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Honeyman E, Ding H, Varnfield M, Karunanithi M. Mobile health applications in cardiac care. Interv Cardiol. 2014;6(2):227–240. doi: 10.2217/ica.14.4. 2014. [DOI] [Google Scholar]
- 21.Feinberg L, Menon J, Smith R, Rajeev JG, Kumar RK, Banerjee A. Potential for mobile health (mHealth) prevention of cardiovascular diseases in Kerala: A population-based survey. Indian Heart J. 2017;69(2):182–199. doi: 10.1016/j.ihj.2016.11. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Breil B, Kremer L, Hennemann S, Apolinário-Hagen J. Acceptance of mHealth apps for self-management among people with hypertension. Stud Health Technol Inform. 2019;267:282–288. doi: 10.2196/preprints.15373. 2019. [DOI] [PubMed] [Google Scholar]
- 23.Boudreaux ED, Waring ME, Hayes RB, Sadasivam RS, Mullen S, Pagoto S. Evaluating and selecting mobile health apps: Strategies for healthcare providers and healthcare organizations. Transl Behav Med. 2014;4(4):363–271. doi: 10.1007/s13142-014. 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]