Evaluation of mHealth Applications Related to Cardiovascular Diseases: a Systematic Review

Abstract

Introduction:

Currently, with the widespread penetration of mobile devices with Internet access, including smartphones, they can allow specific and/or complementary activities in the health field as well as in other commercial sectors.

Aim:

This systematic review examined the impact of mHealth-based for cardiovascular research. The specific aims of the systematic review are to (1) classification of the studies according to the type of research (scientific articles and master’s and doctoral theses) and (2) relationship of studies with topics associated with cardiovascular diseases.

Results:

This review corresponds to information in scientific journals of high impact. This review intends to respond to the following question: How these research works have evaluated the performance of health mobile applications, with a special interest in cardiac issues? This review of these searches corresponds to an analysis by 14 categories, being these: 1) Scientific paper; 2) Doctoral Thesis; 3) Master thesis; 4) Telemedicine; 5) m-sssshealth, e-health; 6) cardiovascular, coronary diseases, heart failures, cardiopulmonary, cardiac rehabilitation; 7) rural health; 8) prevention and control, protection; 9) wearables; 10) mobile, web applications, app, smartphone, software, platform; 11) mhealth education, training, promotion, formative process; 12) self-management; 13) Multiple Vital Sign Monitoring, medical device, heart rate measurement, health care information systems; 14) health regulatory. It provides evidence of how some apps have been evaluated, and in some cases, the effectiveness of the estimated accuracy is not in line with the real situation.

Conclusion:

The analysis of these studies allows us to locate the sources of the development of mobile health projects. It also guides us to discover some needs that require new technology implementations.

1. INTRODUCTION

Mobile health is part of e-health and ICT such as computers, smartphones, mobile health applications, patient monitoring of services. All this information generated is used with the purpose to positively increase access to health information and increase healthier behavior (1). Many studies have been conducted in the hope of providing a more comfortable life for the patient and allowing the patient to continue carrying out his/her daily activities. It is no longer an exclusive activity of the medical and health field. Nowadays, it has become a field of multidisciplinary research. New proposals have been initiated to change how health services are provided to patients. The evolution of the different technological devices has been integrated into the treatment and monitoring of diseases. Broad integration of mobile devices has emerged that allows a wide range of activities, including greater functionality in comparison with the original purposes for which they were initially developed. Here, our interest is presented as a proposed methodology to verify the impact of studies of medical evaluations in which cardiovascular applications or studies have been evaluated. When we refer to normal heart rate, the normal heart rate at rest is 60 to 100 beats per minute. It is usually measured by the electrocardiogram (ECG). The ECG measures the electrical activity of the heart (2).

Currently, it is a fact the wide penetration of mobile devices with Internet access, including the smartphones. They can allow specific and/or complementary activities in the health field. On the other hand, advances in sensors, mobile, and built-in devices have made it possible to monitor the medical condition of the patient, have provided medical treatments and other healthcare assistance based on a communication platform that allows the safe transmission of medical data (3).

New technological innovations can bring benefits to healthcare professionals. Smartphone applications prove to be powerful tools to improve education in the health sector. However, by deeply analyzing the medical implications of Apps, it is neither easy nor feasible to do so transparently or objectively because of the lack of reliable comparative data. Despite the benefits offered by APPs in health care, the outcome of the patient, end-user, and education of beginning professionals, they are not appreciated so easily in some cases of medical specialties. In addition, we raise more disturbing questions about the regulations of these applications and how sensitive patient data are protected (4).

Currently, there are wearables, refers to the set of electronic devices that are incorporated in some parts of our body interacting continuously with the user and other devices to perform specific functions (5). For example, smartwatches, GPS shoes incorporated bracelets that monitor our health and other devices, etc. Currently, people have access to a mobile device connected to the internet, according to Nielsen (6), 74% of global respondents appreciate being connected anywhere and anytime. In addition, 70% of the global respondents mention that their mobile devices have made their lives better. The people access from their smartphone to find information and use the app to solve a specific activity (7).

In this sense, many mobile applications or apps can give us some information about the heart rate. Here, at this point, it is important to know the scope of these applications, how reliable can be their results, and in which field moves. For example, we can say that many of these applications range occur from companies that must do with body care or training (for example sports brands: Nike, and others) until the apps used by medical bodies. Within this range, there are some applications verified by studies with certain criteria that allow the user chooses any with references to tests.

This paper is structured to present a general review of the use of smartphones for cardiovascular diseases. First, we present the systematic review methodology used in the paper. Then, the results are showed in 14 categories, which are: Scientific paper, Doctoral Thesis, Master thesis and in a field as Telemedicine, m-health, e-health, cardiovascular, coronary diseases, heart failures, cardiopulmonary, cardiac rehabilitation, rural health, prevention and control, protection wearables, mobile, web applications, app, smartphone, software, platform, m-health education, training, promotion, formative process, self-management, Multiple Vital Sign Monitoring, medical device, heart rate measurement, health care information systems, health regulatory. Finally, the main conclusions are presented.

2. SYSTEMATIC REVIEW

2.1. Searching Terms

The goal of this systematic review is to identify the main Cardiovascular studies based on mobile development, techniques, or methods applied to. we have selected the terms “cardiovascular” or “cv” and “app” or “m-health” as mandatory search words and, at least, one of the following terms: “heard and diseases”, “self-management” and “vital sign monitoring” AND (law regulatory OR law health).

2.2. Eligibility Criteria

The systematic review aims to identify recent high-quality research in English. For the analysis and evaluation of each of the related works, we have defined a set of criteria:

Type of publication: The searches of the research works have been defined by the following types: Scientific paper, Doctoral Thesis, Master thesis.

For each type the various areas related to Telemedicine, m-health, e-health, Cardiovascular, Coronary diseases, Heart failures, Cardiopulmonary, Cardiac rehabilitation, Rural health, Prevention and control, Protection, Wearables, Mobile, Web applications, App, Smartphone, Software, Platform, m-health education, Training, Promotion, Formative process, Self-management, Multiple Vital Sign Monitoring, Medical device, Heart rate measurement, Healthcare information systems, Health regulatory, Laws.

Publication year: We have limited the search for publications between 1994 and 2018.

Language: Only research papers written in English have been considered for this review and were listed in the types of publications mentioned above.

2.3. Document Collection

The review corresponds to scientific journals of high impact, intending to verify how they have evaluated the performance of mobile applications related to health topics with a special interest in cardiac issues. The main scientific databases used are IEEE Xplore, Web of Science, Science Direct and Scopus and Google Scholar.

Multiple information has been found by some professionals, who evaluate their applications. However, there is little information related to the evaluation and comparisons of apps in the health sector. In our preliminary research (Figure 1), 46 articles have been found, which presented interesting information on cardiovascular disease studies. So, the analysis of 46 articles selected in this methodology represents our main objective. It is to verify scientific articles about app or studies evaluations to be able to consider them.

Figure 1. Data obtained in the review.

3. RESULTS

In the field of health, innovation, and new products are developed that can surprise the consumer, people with the need to take care or wellness. For example, some companies sell watches that can measure the pace cardiac or discharge the app that indicates its performance. It is important to mention that personal training is part of this sector. The selected scientific articles have been grouped according to the distribution proposed in the methodology (Figure 1).

As shown in Table 1, on the authors’ side, the state of the art has been classified according to 14 categories, being these: 1) Scientific paper; 2) Doctoral Thesis; 3) Master thesis; 4) Telemedicine; 5) m-health, e-health; 6) cardiovascular, coronary diseases, heart failures, cardiopulmonary, cardiac rehabilitation; 7) rural health; 8) prevention and control, protection; 9) wearables; 10) mobile, web applications, app, smartphone, software, platform; 11) m-health education, training, promotion, formative process; 12) self-management; 13) Multiple Vital Sign Monitoring, medical device, heart rate measurement, healthcare information systems; 14) health regulatory. The information search strategy has been structured as follows:

Table 1. Research classification by categories.

1. Scientific paper 2. Doctoral Thesis, 3. Master thesis, 4. Telemedicine, 5. m-health, e-health, 6. Cardiovascular, Coronary diseases, Heart failures, Cardiopulmonary, Cardiac rehabilitation, 7. Rural health, 8. Prevention and control, Protection 9. Wearables 10. Mobile, Web applications, App, Smartphone, Software, Platform, 11. m-health education, Training, Promotion, Formative process 12. Self-management, 13. Multiple Vital Sign Monitoring, Medical device, Heart rate measurement, Healthcare information systems 14. Health regulatory, Laws
Authors	Year	1	2	3	4	5	6	7	8	9	10	11	12	13	14
Ahn et al., [8]	2016	X					x				X	x
Alluhaidan et al., [9]	2015	X				x	x
Broderick et al., [10]	2013	X									X
Bruining et al., [11]	2014	X				x					X			x
Cajita et al., [12]	2013	X				x	x
Chow et al., [13]	2016	X				x	x				X
Clement et al., [14]	2018	X				x					X		x
Coppetti et al., [15]	2017	X				x					X			x
De Castro et al., [16]	2018	X				x	x		X		x
Dicianno et al., [17]	2015	X				x	x							x
Duff [18]	2017		x				x				x			x
Duff[19]	2016	X					x				x	x
Duff et al., [20]	2018	X			x	x	x				x
Feinberg et al., [21]	2017	X			x	x	x	x	X
Hervás et al., [22]	2013	X				x			X	X
Gallagher et al., [23]	2017	X				x	x				X
Hilbel et al., [24]	2016	X				x				X				x
Jia et al., [25]	2015	X				x							x
Kitsiou et al., [26]	2017	X				x	x				X			x
Lobelo et al., [27]	2016	X				x	x			X
Lupthon[28]	2013	X				x						x
Villarreal et al., [29]	2009	X				x
Metelmann et al., [30]	2018	X				x					x			x
Montovani et al., [31]	2013	X				x			X		x
Morey et al., [32]	2017	X				x	x				x
Neubeck et al., [33]	2016	X				x	x				x
Nguyen et al., [34]	2017	X				x	x		X			x
Odhiambo [35]	2017			x		x	x	x
Osmani et al., [36]	2017	X				x					x
Park et al., [37]	2016	X				x	x				x
Rawstorn[38]	1994	X				x	x				x
Santo et al., [39]	2015	X				x	x		X
Bravo et al., [40]	2008	X				x	x					x
Sharpe [41]	2017	X	x			x	x				x
Silva et al., [42]	2018	X				x	x
Silberman[43]	2012	X			x	x					x			x	x
Sorber et al., [44]	2012	X				x				X
Thangam et al., [45]	2018	x					x				x
Unal et al., [46]	2018	X				x	x		X		x
Vickey et al., [47]	2011	X									x
Woods et al.,[48]	2017	X				x	x						x
Woods et al.,[49]	2018	X				x					x
Wyatt et al., [50]	2015	X				x					x
Yetisen et al., [51]	2014	x				x					x				x

According to Figure 2, the 95.45% (42/44) corresponds to category - the first: Scientific Publications in Congresses and Indexed Journals. The second category corresponds to category 5, that is, articles with the keyword: m-Health. Category 5 corresponds to 86.36% (38/44) of the documents examined. The third-place corresponds to categories 10 with 61.36% (27/44). Category 10 corresponds to articles with keywords or combinations of these: Mobile, Web Applications, App, Smartphone, Software, Platform. The fourth-place corresponds to category 6 with 59.09% (26/44). Category 6 corresponds to articles with keywords such as cardiovascular, coronary diseases, heart failures.

Figure 2. Classification by categories for each studies or cardiovascular application obtained.

Some relevant studies are: Ahn et al. (8) present a mixed-method, sequential explanatory study to assess CPR training apps downloaded on two apps stores in South Korea. The authors define the inclusion criteria as follows, Korean-language instruction, training features, and emergency supports for real-life incidents and analyzed with two tests. In the evaluation section, 15 medical experts evaluated the apps’ contents according to current Basic Life Support guidelines in conformity test, and 15 nonmedical individuals examined the apps using System Usability Scale (SUS) in the learnability/usability test.

Alluhaidan et al., (9) present the MyHeart as a telehealth system, which we have designed for CHF patients to bridge the current gap in the Congestive Heart Failure care continuum when the patient moves to the home environment. The system uses wireless health devices and a mobile application to collect patients’ data.

Broderick et al., (10) offer a blend of health literacy– and usability-improving strategies that can help developers build health literate apps. They present a case study in mHealth design from a 2012 mobile app challenge, organized by the U.S. Department of Health and Human Services’ Office of Disease Prevention and Health Promotion (ODPHP), to demonstrate how these strategies can be applied throughout the development process.

Bruining et al., (11) present a review that focuses on the acquisition and analysis of smartphones of three important vital signs in the cardiovascular and respiratory field as well as in rehabilitation i.e. heart or pulse rate, blood pressure, and blood oxygenation. The potential, pitfalls, and perspectives on mobile devices and smartphone apps for health management by patients and healthy individuals are discussed.

Cajita et al., (12) present a study to examine factors that influence the intention to use mHealth among older adults with Heart Failure (HF). Some conclusion in this study explains that the researchers should consider using the participatory approach in developing their interventions to ensure that their mHealth-based interventions will not only address the patient’s HF self-management needs but also be easy enough to use even for those who are less technology savvy.

Chow et al., (13) presented a study where Mobile health (m-Health) has been defined as medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, and personal digital assistants. The author explains that mHealth includes simple strategies, such as the use of short message service (SMS) or text messages in successful short-term smoking-cessation, weight loss, and diabetes management programs. mHealth can also involve more complex strategies, such as smartphone applications (apps), global positioning systems (GPS), and Bluetooth technologies. There is now a need to rethink traditional health service structures and bioengineering capacity, to ensure mHealth systems are also safe, secure and robust.

According to Clement et al., (14) promising first results for Kaia, a mobile app digitalizing multidisciplinary rehabilitation for low back pain, were recently published. It remains unclear whether the implementation of user feedback in an updated version of this app leads to desired effects in terms of increased app usage and clinical outcomes. The aim is to elucidate the effect on user retention and clinical outcomes of an updated version of the Kaia app where user feedback was included during development.

According to Coppett et al., (15) present a study aimed to test the diagnostic accuracy of such heart rate measuring apps in clinical practice. They found substantial performance differences between the four studied heart rate measuring apps. The two contact photoplethysmography-based apps had higher feasibility and better accuracy for heart rate measurement than the two non-contact photoplethysmography-based apps.

De Castro et al., (16) present a study about the Nurse practitioners (NPs) that demonstrate the main role of nursing in Cardiovascular Diseases (CVD) prevention and must employ technological innovations to advance practice. This manuscript educated and informed NPs to m-Health interventions and its potential implications to their practice and patients.

Dicianno et al., (17) present a review for the evolution of m-Health technologies and presents insights as to how this evolution informed the development of a novel mHealth system, iMHere (interactive mobile health and rehabilitation), and other technologies, including those used by the Veterans Administration. This article also describes how mHealth can be used to improve patient satisfaction and delivery of care and to promote health and wellness.

Duff (18, 19) present a paper to outlines the formative research process to develop a mobile-application for cardiovascular rehabilitation. The main purpose of cardiac rehabilitation is to prevent a further cardiac event and improve the person’s quality of life. mHealth technologies may tackle some of the issues relating to poor uptake and low adherence, such as accessibility and affordability.

Duff et al., (20) present an application called MedFit. This app is designed to facilitate the participation of people with cardiovascular disease (CVD) in an exercise-based rehabilitation program remotely. The author developed an application for improvement of behavior change, theoretically informed exercise rehabilitation mobile app for adults with CVD by following the early stages of the formative research: development and feasibility testing. This work will guide future research aiming to develop mobile apps by incorporating a best practice framework for m-Health intervention development and a user-centered design approach.

Feinberg et al., (21) present a study to explore mobile phone usage patterns in rural Kerala (Ernakulam), the acceptability of mHealth delivery of health promotion, and Cardiovascular Diseases (CVD) prevention. Some results in the obtained are the majority of this population approve mHealth interventions. While the further investigation of mHealth as a health education tool is warranted, SMS interventions may fail to maximize equity and penetration across all patient groups.

Hervás et al., (22) developed an end-to-end software application for patients and physicians and a rule-based reasoning engine. We have also proposed a conceptual module to integrate recommendations to patients in their daily activities based on information proactively inferred through reasoning techniques and context-awareness. To evaluate the platform, we carried out usability experiments and performance benchmarks.

Gallagher et al., (23) explain that emerging evidence indicates mobile technology-based strategies may improve access to secondary prevention and reduce risk factors in cardiac patients. The author explains that little is known about the use of mobile technology by cardiac patients, particularly for health reasons and whether the use varies according to the demographics of the patient.

Hilbel et al., (24) present a wireless data logger system that allows seamless 24/7 monitoring of relevant vital sign parameters. CL covers the entire period of acute point of care inside the hospital and the recovery period, when first mobility is achieved and when the patient is released into an ambulatory or home care environment.

Jia et al., (25) present an implementation of a set of ease of use principles in the mHealth design and employed the quantitative Fogg Behavior Model to enhance users’ execution ability. The framework consists of medical apparatuses, mobile applications, and a health management server. The system can monitor the physiological status in an unconstrained manner with simplified operations, while supervising the healthcare plan.

Kitsiou et al., (26) in their articles presented the main features and components of iCardia–an innovative mHealth platform designed to support remote monitoring and health coaching of cardiac rehabilitation (CR) patients, through Fitbit wearable sensor devices, smartphones, and personalized SMS text messages.

For Lobelo et al., (27) present paper that reviews the validity, utility, and feasibility of implementing mHealth technology in clinical settings and proposes an organizational framework to support Physical Activity (PA) assessment, counseling and referrals to community resources for CVD risk reduction interventions.

Lupthon (28) presents apps and social media tools that offer new ways of monitoring, measuring, and representing the human body. This article adopts a critical sociological perspective to identify some of the social and cultural meanings of self-tracking practices via digital devices.

Villarreal et al., (29) present a proposal for Patients’ Mobile Monitoring framework. This framework enables the definition and generation of profiles, modules and communication structures between each of the measuring devices and the mobile phone depending on the kind of condition and the measuring values of the patient.

Metelmann et al., (30) as the aim of the study written to examine mobile phone apps offering real-time instructions in German or English in case of a cardiac arrest, to evaluate their adherence to current resuscitation guidelines, and to test their usability.

Montovani et al., (31) explores these problems and their implications for the development of mHealth. In conclusion, the authors suggest potential approaches that may be able to resolve such problems.

Morey et al., (32) conducted a 3-phase assessment of human factors issues for common mHealth apps designed for managing congestive heart failure. They report design issues identified in the apps that limit usability by older adults.

Neubeck et al., (33) inform the development of a web-based application integrated with the primary care electronic health record, they undertook a collaborative user-centered design process to develop a consumer-focused e-health tool for cardiovascular disease risk reduction.

Nguyen et al., (34) present a sudden-death case secondary to a right atrial rupture, occurring as a result of myocardial infarction with very subtle clinical and electrocardiographically signs. They review other potential complications which, in clinical practice, are not always attributed to their true origin.

Odhiambo (35) exploring how mHealth is being used in the improvement of healthcare as well as in the reduction of costs in Rural Kenya. An integrative review of literature as a method was used to analyze and critique existing literature in line with the topic of this study.

Osmani et al., (36) describe an innovative framework for prescription of personalized health apps by integrating Personal Health Records (PHR) with disease-specific mobile applications for managing medical conditions and the communication with clinical professionals.

Park et al., (37) present a quantitative systematic review. The selected studies were critically evaluated to extract and summarize pertinent characteristics and outcomes. A large majority of studies (22 of 28, 79%) demonstrated text messaging, mobile applications, and telemonitoring via mobile phones were effective in improving outcomes. Overall, text messaging appears more effective than smartphone-based interventions.

Rawstorn (38) present an application to allow Exercise-based cardiac rehabilitation (exCR) has multifactorial secondary prevention benefits for people with coronary heart disease (CHD). The development and evaluation of a mobile health (mHealth) exCR delivery model that combines clinical exercise specialists’ expertise with enhanced access.

Santo et al., (39) present a review, they discuss the potential use of mHealth in a variety of medical conditions and we highlight some promising applications in Coronary heart disease (CHD) prevention. Many patients with CHD fail to receive and adhere to the guideline recommendations, including lifestyle advice and evidence-based cardiovascular medication.

Bravo et al., (40) present a proposal that seeks to adapt to Near Field Communication technology (NFC) for nursing care. This consists of a combination of RFID and mobile phones. We propose that information management problems be solved with a single interaction by contact.

Sharpe (41) focused on the use of mobile health and wellness applications (apps) in chronic disease management. An exploratory mixed methods design was used to gain an understanding of patient and provider perspectives and experiences. The study was conducted in a cardiac rehabilitation program in Ontario, Canada.

Silva et al., (42) present MOVIDA.eros. It is an ongoing project aims to bring more people into cardiac rehabilitation programs and help physicians monitor to better serve patients with different heart conditions.

Silberman (43) present a study about M-health that includes consumer- and provider-oriented medical applications (apps), such as weight monitoring apps, and medical devices, such as glucose meters, that send health information back to the provider.

Sorber et al., (44) propose Amulet, a mHealth architecture that provides strong security and privacy guarantees while remaining easy to use and outline the research and engineering challenges required to realize the Amulet vision.

Thangam et al., (45) present a Cardiac Health application developed on Android. It is an awareness app for preliminary assessment of heart diseases which was designed to discriminate cardiac associated problems or other ailments. This is a unique app to spread awareness of cardiac risks using mobile technology to both public and medical professionals.

Unal et al., (46) present a study is to identify, retrieve, critically appraise and synthesize information regarding existing mobile phone text messaging interventions that have been done for secondary prevention of cardiovascular disease (CVD).

Vickey et al., (47) purpose a study was to research the new emerging technology of mobile health, the use of mobile fitness apps to share one’s workout with their Twitter social network, the workout tweets and the individualities of the Tweeters.

Woods et al., (48, 49) present the process established to co-design a mHealth application in support of heart-failure self-management. For this development, an interdisciplinary team systematically proceeds through the phases of Stanford University’s Design Thinking process; empathize, define, ideate, prototype, and test with a user-centered philosophy. Using this clinician-led heart failure app research as a case study, they describe a sequence of procedures to engage with local patients, carers, software developers, eHealth experts, and clinical colleagues to foster rigorously developed and locally relevant patient-facing mHealth solutions. This paper reports the participatory, user-centered co-design process of the conceptual design and iterative development of the application.

Wyatt et al., (50) have developed and piloted an 18-item checklist to help clinicians assess the structure, functions, and impact of medical apps. Use of this checklist should help clinicians to feel more confident about using medical apps themselves, about recommending them to their staff or prescribing them for patients.

Yetisen et al., (51) explain the essence of this Food and Drug Administration (FDA) guidance by providing examples and evaluating the impact on academia, industry and other key stakeholders, such as patients and clinicians.

4. DISCUSSION

In health issues, especially those related to the detection of cardiovascular diseases, heart rate measurement turns out to be a good start and mobile applications on smartphones are an available means that can save costs and shorten the distance. However, it is important to verify the accuracy of the results of different APPs that are available on iOS and Android. The confidence generated by these applications can benefit users who use it, either by improving their health monitoring or simply verifying their body condition in a workout in the gym or daily life. The motivations are many, both for those who develop APPs and for those who use them.

This work provides scientific evidence of how some Apps or studies have been evaluated, and in some cases, the effectiveness of the estimated accuracy does not correspond to the real one. This makes us reflect on the real implications that approach us in technological innovations, with a world so changing and globalized, information and communication technologies allow us to use mobile applications that by placing fingers on the smartphone we can measure many characteristics of the environment and the position using the sensors. There are many ways to measure a person’s heart rate. One of them can be by using mobile phones. All that is necessary for heart rate measurement is the mobile phone with a camera equipped with flash, for example.

The most relevant research on the development of mobile applications for topics related to cardiovascular diseases originate from scientific publications, research products that arise from research nuclei. These results are shown in indexed journals, which allows us to compare the solutions of each country. Within those publications, most of the results end in the development of Web Applications, App, Smartphone, Software, and Platform. Of those applications generated most of it manage data of patients or people without cardiovascular disease, coronary diseases, heart failures.

5. CONCLUSION

In this field, future health professionals should rely on mobile applications to help the patient. Mobile apps can generate alarms for medications, education treatments, or procedures to patients during their consultations, among other uses that can be offered with apps. It is important to be able to differentiate between applications for personal use and those for professional medical use. Both need specific education for the integration of mobile technologies in the patient solutions.

Author contributions:

Conceptualization, V.V.; Methodology V.V., A.B.; Investigation V.V.; Writing-Review & Editing V.V., A.B. Final proof reading was made by the first author.

Conflicts of Interest:

The authors declare no conflict of interest.

Financial suppoet and sponsorship:

This article is presented as part of a project ITE15-001 financed by SENACYT and development by Technological University of Panama.