Skip to main content
. 2017 Sep 25;20(4):e20. doi: 10.1136/eb-2017-102751

Table 1.

ORCHA-24 app assessment criteria and related standards

App assessment criteria Related standards
Data governance
 (1) Does the app state that no data will be shared with other parties without explicit user consent?

  • The Data Protection Act 1998: Principle 113

  • Data Protection Directive: Article 1914

 (2) Does the app outline a process for managing data confidentiality breaches?

  • Information Commissioner’s Office: Guidance on data security breach management. Version 2.1. The Data Protection Act15

  • The Charter of Fundamental Rights of the European Union: Article 724

 (3) Is there a data privacy policy, either within the app itself or on a website?

  • PAS-277: 2015 6.2.3 .f (project documentation)11

  • GSMA: Mobile and Privacy. Privacy Design Guidelines for Mobile Application Development (2012) (TCC2, TCC3, DRS4)16

 (4) Does the data privacy policy, or statement, provide detail about what data is collected by the app?

  • PAS-277: 2015 6.3.e(2)11:

  • Data Protection Directive: Article 1014

 (5) Does the data privacy policy, or statement, provide detail about what that data is used for by the app?

  • The Data Protection Act 1998: Principle 213

  • Data Protection Directive: Article 1014

 (6) Does the data privacy policy, or statement, state whether personal data are stored using recognised secure data storage technologies?

  • The Data Protection Act 1998: Principles 6 and 713

  • The Data Protection Directive: Article 1714

 (7) Does the data privacy policy, or statement, state that all personally identifiable data will be encrypted in transit between the device and any developer host storage? (eg, using FTP protocol)

  • The Data Protection Directive: Article 17.14

  • GSMA: Mobile and Privacy. Privacy Design Guidelines for Mobile Application Development (2012) (DRS2)16

  • The Data Protection Act 1998: Principle 713

 (8) Does the data privacy policy state that only the minimum data items necessary for the app to function will be collected?

  • The Data Protection Act 1998: Principle 313

  • The Data Protection Directive: Article 6 (data minimisation)14

Clinical efficacy and assurance
 (9) Is there a statement within the app itself, or the app store, about user feedback during design, development or testing?

  • PAS-277: 2015 6.2.3 (c), 6.2.4, 6.511

 (10) Is there a statement either in the app or store about user involvement in testing?

  • PAS-277: 2015 6.711

 (11) Is there a statement within the app that it has been tested and shown to be beneficial to someone with the relevant condition?

  • Misrepresentation Act 196720

  • PAS 277: 2015 6.2.3(c) (Project documentation)12

 (12) Is there a statement within the app, or app store, about the app having been through a clinical trial, or other form of testing to show real world effectiveness, and has received positive feedback?

  • PAS-277: 2015, 6.2.3, 6.511

 (13) Is there a statement about how frequently any advice, guidance or content will be reviewed to ensure accuracy and clinical relevance?

  • PAS-277: 2015, 6.711

 (14) Is there a statement within the app that it has been positively evaluated or validated by a clinical or other relevant expert?

  • PAS-277: 2015, 6.2.3(b)11

 (15) Is there any evidence within the app that the developer has attempted to validate any guidance or recommendations with academic expertise?

  • No specific guidance. Result of Delphi panel discussion
 (16) Is there a statement within the app identifying a list of review or accrediting bodies or individuals?

  • No specific guidance. Result of Delphi panel discussion
User experience and engagement
  (17) Does the app provide support options for users with visual impairment? Including changing font sizes or colour?

  • W3C: Accessibility Requirements for People with Low Vision Editor’s Draft 6 June 2016, clause 3.3.1.18

  • App Quality Alliance (AQuA): Accessibility Testing Criteria for Android Applications version 1.2: July 2015, sections 1.4.1 and 1.5.317

  (18) Does the app provide support options for users with hearing difficulties?

  • AQuA: Accessibility Testing Criteria for Android Applications version 1.2: July 2015, section 317

 (19) Does the app contain a ‘?/HELP/ABOUT’ function to aid user understanding?

  • AQuA: Best Practice Guidelines for producing high-quality mobile applications version 2.3 – June 2013 page 23.30

 (20) If clinical or technical terms are used, are they explained clearly to the user? (either within the content of the app or via a glossary)

  • No specific guidance. Result of Delphi panel discussion
  (21) Is there any statement within the app about how to report issues, bugs or errors to the developers?

  • PAS-2772015 - Clause 10 (transparency)11

  (22) Does the app set goals for users or allow them to set goals for themselves?

  • No specific guidance. Result of Delphi panel discussion
  (23) Is there a statement within the app about the developer’s commitment to addressing problems reported to them? (eg, timescales to respond, commitment to eradicate reported bugs and faults)

  • PAS-277: 2015–6.7 Accountability11

 (24) Are there opportunities to link with other users of the app, including buddying, forums or group education?

  • No specific guidance. Result of Delphi panel discussion

FTP, File Transfer Protocol; GSMA, Groupe Speciale Mobile Association; PAS, Publicly Available Standard.