Table 1.
Comprehensibility and perceived importance of the criterion by stakeholders.
| Category and criterion | Comprehension, n (%) | Perceived importance (0-10) | |||||
|
|
|
Patients (N=7) | Clinicians (N=6) | Engineers (N=5) | Patients | Clinicians | Engineers |
| Usability | |||||||
|
|
The app has been tested by potential users before being made available to the public. | 7 (100) | 5 (83) | 5 (100) | 8.9 | 8.6 | 9.4 |
|
|
It has instructions or some kind of assistance for use. | 6 (86) | 6 (100) | 5 (100) | 10 | 8.5 | 7 |
|
|
It is easy to use (ie, navigation is intuitive). | 7 (100) | 6 (100) | 5 (100) | 9.6 | 9.3 | 9 |
|
|
It follows the recommendations, patterns, and directives in the official manuals of the different operating systems (Android, iOS, or others). | 7 (100) | 5.5 (92) | 4 (80) | 7.5 | 7 | 7.9 |
|
|
The interface design follows the same pattern. That is, all graphic elements (typographies, icons, and buttons) have a consistent appearance. The function of each element (navigation menu, lists, and photo gallery) is clearly identified. | 7 (100) | 6 (100) | 5 (100) | 9 | 8.3 | 8.8 |
|
|
The functionality is adapted to the purpose of the app. | 7 (100) | 5.5 (92) | 5 (100) | 9.6 | 8.5 | 8.8 |
|
|
The information of the app must be able to be accessed in the shortest possible time. All users must be able to access all resources regardless of their capabilities. | 7 (100) | 5.5 (92) | 5 (100) | 8.3 | 8.6 | 8.4 |
|
|
The app can be consulted in more than one language. All languages adapt appropriately to the content interface. | 7 (100) | 5.5 (92) | 5 (100) | 8.3 | 7.4 | 7.2 |
| Privacy | |||||||
|
|
The app gives information about the terms and conditions of purchases in the app and personal data recorded. | 7 (100) | 6 (100) | 5 (100) | 9.5 | 8.5 | 9 |
|
|
It gives information about the kind of user data to be collected and the reason (the app must only ask for user data that is essential for the app to operate). It gives information about access policies and data treatment and ensures the right of access to recorded information. It describes the maintenance policy and the data erasure procedure. It gives information about possible commercial agreements with third parties. | 7 (100) | 5.5 (92) | 5 (100) | 9.6 | 8.8 | 9.4 |
|
|
It guarantees the privacy of the information recorded. It requires users to give their express consent. It warns of the risks of using the app. | 7 (100) | 6 (100) | 5 (100) | 9.9 | 9.2 | 9 |
|
|
It tells users when it accesses other resources of the device, such as their accounts or their social network profile. | 7 (100) | 6 (100) | 5 (100) | 9.3 | 8.3 | 9.4 |
|
|
It takes measures to protect minors in accordance with the current legislation. | 7 (100) | 5.5 (92) | 5 (100) | 8.7 | 9.7 | 8.2 |
|
|
Confidential user data are protected and anonymized, and there is a privacy mechanism so that users can control their data. | 7 (100) | 6 (100) | 5 (100) | 9.6 | 9.2 | 9.4 |
| Security | |||||||
|
|
The app has encryption mechanisms for storing, collecting, and exchanging information. It has password management mechanisms. | 7 (100) | 6 (100) | 5 (100) | 9.9 | 8.3 | 8.6 |
|
|
The cloud services used have the relevant security measures. It states the terms and conditions of cloud services. | 7 (100) | 6 (100) | 5 (100) | 9.5 | 8.2 | 8.4 |
|
|
The authorization and authentication mechanisms protect the users’ credentials and gives access to their data. It limits access to data that is only necessary for the user. | 7 (100) | 6 (100) | 5 (100) | 9.6 | 7.5 | 9.6 |
|
|
It detects and identifies cybersecurity vulnerabilities, possible threats, and the risk of being exploited. It applies the appropriate security measures to cybersecurity vulnerabilities in the face of possible threats. | 7 (100) | 5.5 (92) | 5 (100) | 9.3 | 8.3 | 8.4 |
| Appropriateness and suitability | |||||||
|
|
The end users for whom the app is designed are explicitly indicated or actually intuitable (the name identifies the app) to the audience to whom it is set out. | 7 (100) | 6 (100) | 5 (100) | 8.2 | 7.7 | 8.4 |
|
|
The benefits and advantages of using the app are explained. | 7 (100) | 6 (100) | 5 (100) | 9 | 7 | 7.5 |
|
|
The app has been validated or created by experts (eg, a group of specialized professionals, a health organization, or a scientific society). | 6.5 (93) | 5.5 (92) | 5 (100) | 8 | 9.7 | 9 |
| Transparency and content | |||||||
|
|
The app identifies the authors of the content and their professional qualifications. | 7 (100) | 6 (100) | 5 (100) | 8 | 8.5 | 8.4 |
|
|
It gives transparent information about the owners’ identity and location. | 7 (100) | 6 (100) | 5 (100) | 7.2 | 8.2 | 8.4 |
|
|
It gives information about its sources of funding, promotion and sponsorship, and possible conflicts of interests. Any third parties or organizations who have contributed to the app development are clearly identified. | 7 (100) | 6 (100) | 5 (100) | 7.5 | 7 | 7 |
|
|
It uses scientific evidence to guarantee the quality of the content. It is based on ethical principles and values. | 7 (100) | 6 (100) | 5 (100) | 10 | 9.3 | 9 |
|
|
The sources of the information are indicated. Concise information is given about the procedure used to select the content. | 7 (100) | 6 (100) | 5 (100) | 8.2 | 7 | 8 |
| Safety | |||||||
|
|
The possible risks to users are identified. Users are warned that the app does not intend to replace the services provided by a professional. | 7 (100) | 6 (100) | 5 (100) | 9.5 | 8.9 | 8.6 |
|
|
Potential risks for users caused by bad usage or possible adverse effects are explained. | 7 (100) | 6 (100) | 5 (100) | 8.5 | 8.5 | 8.6 |
| Technical support and updates | |||||||
|
|
It gives a warning if updates modify or affect how the app functions. It gives a warning if updates can influence insensitive data. | 7 (100) | 6 (100) | 5 (100) | 8.5 | 7.2 | 7 |
|
|
Frequent security updates are guaranteed. Every time an update of a third-party component is published, the change is inspected, and the risk evaluated. | 7 (100) | 5.5 (92) | 4.5 (90) | 8.2 | 8.2 | 7 |
|
|
The frequency with which the content of the app is revised or updated is shown. | 7 (100) | 6 (100) | 4.5 (90) | 7.7 | 7 | 7 |
|
|
Users have support mechanisms (email, phone, and contact form) for solving doubts, problems, or issues related to the health content, and technical support. | 7 (100) | 6 (100) | 5 (100) | 9.2 | 9 | 8.4 |
| Technology | |||||||
|
|
It works correctly. It does not fail during use (eg, blocks). Functions are correctly retrieved after context changes (eg, switch to another app and return), external interruptions (eg, incoming calls or messages), and switching off the terminal. | 7 (100) | 6 (100) | 5 (100) | 9.5 | 8.5 | 9 |
|
|
It does not waste resources excessively: battery, central processing unit, memory, data, or network. | 7 (100) | 6 (100) | 5 (100) | 8.9 | 7.8 | 8.2 |
|
|
It can work in flight mode and deal with network delays and any loss of connection. | 7 (100) | 5.5 (92) | 5 (100) | 7.9 | 7.2 | 7 |
| It supports multiple versions of data structures or formats (eg, to support different operating systems). | 6.5 (93) | 5.5 (92) | 4.5 (90) | 8 | 7 | 7 | |