To the editor: New approaches toward actionable mobile health evaluation

Hensher et al¹ offer an impressive review of the app evaluation space. The scope and depth of their review is a boon to the literature, with great applicability across diverse health domains. However, one concern is that the authors may fall prey to the same issues that they seek to caution readers about. The authors note in the discussion that outcomes of app evaluation systems without scoring “often seemed to be subjective” in a clear subjective statement for which they offer no supporting results or citations. The authors cite our team’s 2018 review article to support their claim that “consistent with previous studies, which highlight the importance of point-scale approach,” which is odd, as this is neither the message nor the conclusion that we wrote in that article cited. Rather, we noted that “perhaps point-based evaluation will be more useful in the near future” after reviewing a lack of consensus and convergence in these point based approaches.² In this same cited article, we illustrate the merits of a pyramid-based framework developed in conjunction with the American Psychiatric Association,^3,4 which the authors suggest might “reduce the credibility and validity of the evaluation process” for reasons that are not explained. There are many approaches to app evaluation, and each holds merit, but without a discussion based on facts there cannot be progress.

To illustrate the changing landscape of subjective and point-based systems, we highlight 2 updates in the field that occurred after the April 2020 search limit of Hensher et al’s article. First, the authors highlight the Mobile App Rating Scale (MARS) as a paradigm of point-based ratings, but a 2021 article that sought to adapt MARS for mobile phone apps and e-tools noted that “whilst the A-MARS is a useful tool to guide health professionals as they explore available apps and e- tools for potential clinical use, the training, time, and skill required to use it effectively may be prohibitive.”⁵ From our team, we have adapted the pyramid framework labeled as subjective by the authors into the MIND (M-Health Index and Navigation Database) database, which is publicly accessible online (apps.digitalpsych.org). MIND offers clear and objective criteria, and is derived from a series of questions that involved careful selection.⁶ The 105 questions that comprise the database are all objective and derive from the American Psychiatric Association framework but do not order priority domains so as to allow users to prioritize the domains as they see fit. Thus, in these 2 examples, we see a point-based framework praised by Hensher et al now facing limitations and a subjective system critiqued now informing new efforts that are more objective in spirit. The title of our recent article, “Assessing Mental Health Apps Marketplaces with Objective Metrics from 29 190 Data Points from 278 Apps,” underscores the potential of this synergy.⁷ Each approach has merits, and recognizing how each can inform the others will enable continued progress in app evaluation.

FUNDING

There are no funders to report for this submission.

AUTHOR CONTRIBUTIONS

Both authors contributed equally.

DATA AVAILABILITY

No new data were generated or analyzed in support of this research.

CONFLICT OF INTEREST STATEMENT

JT chairs the American Psychiatric Association Health IT committee which helped develop one of the frameworks discussed here. JT and SL developed the MIND database referred, which is openly and freely shared.