Table 1.
Existing Resources
| Existing resource/framework | History | Organizational membership | Focus | Reference | Areas scored | Methodology provided? | Ranking Criteria Defined? | Scoring transparent? | Standards-based assessment? |
|---|---|---|---|---|---|---|---|---|---|
| NHS | Apps Library launched in 2017 with 46 Apps | Open to all app developers to submit apps | Government sponsored validation effort using commercial company Our Mobile Health | Clincial effecdtiveness, regulatory approval, clincal safety, prvacy & confidentiality, security, usability & accessibility, interoperability, technical stability, change management | Yes | Apps are either NHS Approved, NHS under test; presumabily pass/fail | Somewhat | Yes | |
| PCHA | Formed in 2014 as a spinoff of HIMSS, and merger with Continua Alliance | 240 companies, including health providers, payers, pharma, medical device vendors | Connected devices and mobile platform interoperability and data standards; expanding into FHIR, direct ot cloud, cyber security | www.pchalliance.org | Health device interface, service interface, and healthcare information system interface | Yes | No ranking, certification only | Yes | Yes |
| Xcertia | Founded in 2016 by AHA, AMA, HIMSS, and DHX Group; Strategic alliance with Consumer Technology Association | Approximately 40 healthcare industry member companies, and liaison relationships with ATA, CTA, IEEE, EHNAC, PCHA. | Broad criteria for mobile app curation. Work groups created for security, privacy, content (evidence based), operability, and advocacy | http://xcertia.org/ | Operability, content, security, and privacy | No; planned release in 2019 | No | No | Yes, where standards exist. |
| RankedHealth | Run by the Hacking Medicine Insititute, a non profit spun off MITs Hacking Medicine program | N/A; Crowd Sourced medical and technology professionals perform peer reviews | Apps related to managing and monitoring chronic conditions and issues affecting broader populations, including mental health, heart disease, diabetes, obesity, sleep quality, fitness, medication adhernce, sumpton tracking, emergency care, pregnancy, and reproductive health | www.rankedhealth.com |
Effectiveness: clincial relevance, credibility, evidence-based Functionality: features, data sharing, integration with other apps, HealthKit, or EMR Usability: user interface, user experience, ease of use, look and feeel |
Peer review | No | No | No |
| NODE.Health | The Network for Digital Evidence in Health was founded in 2016 | 20 member health systems from across the health ecosystem; Health systems, Trade Organizations, corporations, accelerators, start-ups, payors | Vision of creating Evidence Based Medicine (EBM) for digital health solutions, ending “death by pilot” | www.nodehealth.org/ | Clincial efficacy, and Usability (UX), as a necessary predicate to positive outcomes | Developed per solution as part of study design | Expected to be developed over time using data from multiple studies | Future vision | Yes, where standards exist. |
| Wellocracy | Sponsored by Partners Connected Health, founded by Brigham and Women’s Hospital and Massachusetts General Hospital, teaching affiliates of Harvard Medical School | Launched by Partners Center for Connected Health in 2013 to promote “Self-Health” consumer adoption of wellness apps. | Consumer fitness apps, nutrition, sleep, food and calorie, healthy habit and heart health apps and devics. | http://www.wellocracy.com/ | Heuristic reviews of apps in each category, including consumer reports like relative comparisons. Main areas are Fun Features, “Which it Had”, comaptibility (device, iOS vs. Android), and consumer erviews. | Qualitative only | Yes; heuristic review by experts at Partners | No | No |
| Digital Therapeutics Alliance | Founded in October 2017 by Akili Interactive, Propeller Health, Voluntis, and Welldoc. | 19 companies including primarily Pharma and pure play digital health solutions. 10 formal strategic advisors from industry and academia. | Digital health solutions that prevent, diagnose, or treat a medical disease or disorder or optimized medication. Predominantly in the regulated space. | https://www.dtxalliance.org/ | Not a scoring mechamism but a framework with principles to be adapted, including product development practices, clinical validation, security and privacy, and promoting appropriate regulatory oversight of market claims and risks. | Framework of Principles only | No | No | No |
This table provides representative examples and details of existing resources that aim to address various aspects of digital health evaluation