Abstract
Background:
Mental health (MH) apps are growing in popularity. While MH apps may be helpful, less is known about how crises such as suicidal ideation are addressed in apps.
Aims:
We examined the proportion of MH apps that contained language mentioning suicide or suicidal ideation and how apps communicated these policies and directed users to MH resources through app content, terms of services, and privacy policies.
Method:
We chose apps using an Internet search of “top mental health apps,” similar to how a user might find an app, and extracted information about how crisis language was presented in these apps.
Results:
We found that crisis language was inconsistent among apps. Overall, 35% of apps provided crisis-specific resources in their app interface and 10.5% contained crisis language in terms of service or privacy policies.
Limitations:
This study employed a nonsystematic approach to sampling apps, and therefore the findings may not broadly represent apps for MH.
Conclusion:
To address the inconsistency of crisis resources, crisis language should be included as part of app evaluation frameworks, and internationally accessible, vetted resources should be provided to app users.
Keywords: mobile health, suicidal ideation, self-help, mental health treatment, telehealth, digital health ethics
Although an estimated 970 million people are diagnosed with a mental health (MH) condition worldwide (James et al., 2018), access to MH treatment remains limited. Mobile apps can provide access to resources, support, and connections to care for people experiencing MH problems (Aguirre et al., 2013) and may allow greater access to care for a larger population of individuals compared to in-person treatment. Although the effectiveness of MH apps remains a topic of ongoing research (Weisel et al., 2019), approximately one third of outpatients with MH conditions have downloaded an app for their MH (Torous et al., 2018). While MH apps hold promise in helping people with MH needs, less is known about how crises, such as the presence of suicidal ideation, are addressed within apps.
Evidence supporting the effectiveness of apps targeting suicide and self-harm prevention is preliminary, showing some positive effects of the apps on suicide-related outcomes but requiring more robust trials (Melia et al., 2020). Similarly, other studies have concluded that few suicide- or depression-focused MH apps offered evidence-based clinical support (Martinengo et al., 2019). Troublingly, fewer than 10% of these suicide specific apps met standards for evidence-based intervention for suicide prevention and some apps provided incorrect crisis helpline numbers (Martinengo et al., 2019).
Crises may emerge at any time among people with MH problems not specifically seeking apps for suicide prevention. Studies have shown higher rates of self-disclosure of suicidal ideation in an app-based version of a depression screening tool compared with clinician-administered assessments, indicating that users may feel more comfortable disclosing suicidal ideation in the context of an app (Torous et al., 2015). Thus, language surrounding MH crisis is important for all MH apps (not only those related to depression and suicide) to inform the user of what will happen should they disclose suicidal thoughts or to provide resources. Yet, crisis statements may not often be in the app content but instead frequently embedded in the terms of service document or privacy policy. Generally, a terms of service document states the conditions a user must agree to in order to use the app, where a privacy policy gives the user information about how data are collected, stored, and shared. However, between 31% and 69.5% of MH apps do not contain a privacy policy (Huckvale et al., 2019; Sunyaev et al., 2015), highlighting the importance of understanding crisis language in both documents. Some apps may contain an end-user licensing agreement (EULA) instead of a terms of service (TermsFeed, 2020), which specifies rights to use, reproduce, and modify software.
There is a paucity of research about crisis language within apps, in the terms of service/EULA, and in privacy policies. The aim of this study was to examine the proportion of apps that contained language mentioning suicide or suicidal ideation and the approach taken to communicate these policies and to direct users to MH resources through app content, terms of service/EULA, and privacy policies.
Method
App Selection
Since many MH app users will search for apps on app stores, Google searches, and online forums (Schueller et al., 2018), we strove to understand how the most readily available and accessible apps addressed suicidal crisis by using an Internet search strategy (similar to Jimenez et al., 2019). On September 11, 2019, one of the authors (EMP) searched for the term “best mental health apps” on Google Chrome using the incognito mode. Incognito mode ensures that the search would not be affected by previous search history and targeted advertisements. The top three popular-press Internet articles on MH apps were identified (Anxiety and Depression Association of America, 2019; Gerry, 2019; Nichols, 2018). In total, 42 apps were presented in a list format from these three sources, with information and recommendations. Apps from these lists were recorded and duplicates removed. There were no country restrictions, and all apps returned were in English.
App Assessment
We created an app evaluation tool to extract information from the terms of service, privacy policy, and the app itself. The tool was created based on the consensus of all authors. First, we collected data available about the app from the Google Play Store (Android) and Apple App Store, including developer country, cost to download, and target population. Presence of crisis language was separated into two categories: one defined by language directing users in event of medical emergencies (no suicide specific language), and the other containing presence of language in the event of user-reported suicidal/homicidal ideation or suicidal/homicidal behavior. Further information collected included presence of MH resources, defined as general support for any MH issues, including resources that are not suicide specific (e.g., links to MH organizations). The proportion of apps that included crisis (or suicide-specific resources) within the app was examined, which we define as a national suicide support hotline, a national emergency number, or safety planning. Additionally, we examined whether the terms of service/EULA asserted no liability. Information was extracted from the app itself, terms of service/EULA, and privacy policies. While terms of service and EULA serve slightly different purposes, apps typically have one or the other, and therefore in this paper we will refer to them as “terms of service.” Author TFF downloaded all available terms of service and privacy policy documents and extracted data from these documents. Author TFF also downloaded the app itself on either an iPhone 8 with iOS 13.3 or Android Samsung Galaxy 8 with Android Version 9, depending on app availability. Apps requiring up-front payments were excluded.
Data Extraction and Analysis
Author EMP performed initial coding of the first three apps’ app interface, terms of service, and privacy policy listed on the app data extraction spreadsheet. Then, author TFF received training on this spreadsheet by independently extracting data from the components of these three apps. She obtained the same results as author EMP, establishing 100% reliability of the coded data. The remaining apps were downloaded separately by TFF and coded. Author TFF consulted with the entire research group throughout the data extraction process if obscurities arose that influenced final coding.
Quantitative and qualitative data were collected using Microsoft Excel. App information and policy documents were coded for presence of crisis language (yes/no), including: whether users were told to call a national emergency number in the event of an emergency (general medical or suicide specific coded separately), and whether there was language in the terms of service or privacy policy specifically mentioning suicide, suicidal/homicidal ideation, or harm to self. Other information collected included general MH resources and suicide-specific in-app resources, and whether the organization asserted no corporate liability for harm toward a person. If there was any other notable feature, it was noted in the data extraction spreadsheet, along with where the terms of service and privacy policy documents were stored in the app. Descriptive statistics and frequencies were used to analyze the data.
Results
Description of Apps
There were 42 apps identified from these three top articles of our Google search of “best MH apps.” Of this list, 10 apps were for iPhone only, 10 for Android only, and 31 available for both iPhone and Android. Of these apps, three could not be found and were excluded from the dataset. One app was the same by two names and therefore the app was only counted as one. The final sample was 38. See Figure 1 for full flowchart.
Figure 1.
Flowchart of included apps. EULA = end-user licensing agreement; PP = privacy policy; TOS = terms of service.
Of these apps, 31 (81.6%) were free to download; whereas the remaining seven required an upfront payment to download; 73.7% of apps were developed primarily in the United States (n = 28), but 10 (26.3%) were from other countries. The apps were sorted according to the main feature of the app, for example, primarily online chat (see Table A1 in the Appendix under “Content”). The apps targeted a variety of MH problems, as indicated under the app description on the Apple App Store/Google Play Store, which included anxiety (n = 24), depression (n = 12), and posttraumatic stress disorder (n = 8; see Table A1 in the Appendix).
Within-App Resources
One app, 10% Happier, was initially downloaded by author TFF but then later could not be re-downloaded during the time of analysis, suggesting that perhaps the app was no longer available. It was excluded from the in-app resources analysis. Six apps required upfront payment. Therefore, 31 apps were downloaded and included in the in-app resources analysis. Of these 31 apps, 14 (45.2%) included MH resources for app users. However, not all of these resources were crisis specific. Seven of the 31 apps (22.5%) included advice to call a national crisis number in the event of an emergency. Three (9.7%) provided suicide-specific safety planning resources and 10 (32%) included a suicide hotline number. Overall, 11 apps (35.5%) contained a crisis-specific resource (see Table A2 in the Appendix).
The placement of resources varied between apps. In some apps, they were located several pages away from the home page. For example, in Happify, a user must navigate to the home page of the app, click on the “more” button on the bottom task bar, press the top right “settings” button, and then press “resources” to find a national suicide prevention hotline. In other apps such as Talkspace, a national suicide prevention hotline is easily accessible as a banner on the bottom of the homepage. Moreover, one app developed in Australia (Smiling Mind) contained in-app crisis support resources linked to an Australian number that was subject to international data rates when calling from the United States.
Terms of Service
In total, 30 apps (78.9%) had a terms of service agreement or EULA located on the Internet, the developer’s website, or a hyperlink on the Apple App Store/Google Play Store. These apps made it possible for a potential user to review the terms of service before agreeing to download. Two free apps listed the terms of service only upon initial signing into the app, not before downloading. Within the apps, 21 of 32 downloaded apps (65.6%) had the terms of service accessible by the user at any time during use. Overall, there were 32 apps (84.2%) with an accessible terms or service (n = 25) or EULA (n = 7) available to the user.
Four of these 32 apps (12.5%) included crisis language in the terms of service. For example, Moodpath’s terms of service states: “If you have suicidal thoughts or need medical help for other reasons, please consult a local doctor or therapist, or in urgent cases an emergency ambulance” (Moodpath, 2019). Only one of the terms of service mentioned that a suicide hotline number was included in the app itself. Thirteen (40.6%) of the terms of service told users to call a national emergency number. However, the majority of terms of service did not encourage users to call for help for suicidal ideation, but rather “in the event of a medical emergency” (75% or nine apps). For example, Anxiety Reliever’s terms of service states: “If you think you are having a medical emergency, call 911 or seek emergency medical treatment. Anxiety Reliever does not provide any medical advice” (AnxietyReliever App, 2016). Only four apps (30.8%) encouraged users to contact a national emergency number for suicidal ideation or self-harm. Furthermore, 26 (81.3%) of the terms of service included language asserting a lack of corporate liability in case of any harm or damages as a consequence of or in any relation to using the app (see Table A2 in the Appendix for full data).
Privacy Policy
The terms of service and privacy policy are usually separate, therefore there may be some discrepancy between the number of terms of services and privacy policies. A total of 34 apps (94.4%) had an accessible privacy policy document, separate from the terms of service. However, like the terms of service documents, one of these apps was only accessible after initial download of the app and could not be found elsewhere. The remaining 33 apps had a privacy policy found either on the developer’s website or on the Google Play Store and/or the Apple App Store. Of the total 32 downloaded apps, 23 (71.9%) had the privacy policy accessible from within the app, for the user’s review at any time. Two apps, Breathe2Relax and T2 Mood Tracker, contained a link to the privacy policy or website on the Apple App Store and/or the Google Play Store yet the link did not load and the authors were unable to confirm the presence of a privacy policy. Only one (2.9%) of these 34 privacy policies included any sort of crisis language. The privacy policy of 7 Cups states that a user’s information may be revealed while using the “therapist” function if “they or someone else is in imminent danger” (7 Cups, 2018; see Table A2 in the Appendix).
Finally, we examined crisis language in either the terms of service, privacy policy, or within the app to determine whether inclusion of this language differed by app content (see Table A1 in the Appendix). Both the online chat (N = 2) and forum type (N = 1) apps were most likely to contain this crisis language (100%). Informational (9/13; 69.2%), activity (8/12; 66.7%), and mood tracking apps (5/9; 55.6%) followed, with the highest percentage of crisis language. Both general mental wellbeing (2/17; 11.8%) and productivity apps (0/5; 0%) were least likely to contain crisis language.
Discussion
Our results show that language discussing suicidal ideation/suicidal behavior is lacking, with only 35% providing in-app crisis resources. Additionally, our results highlight inconsistency in how MH apps offer crisis management plans and privacy protections, as well as inconsistency in crisis language provided in the terms of service and privacy policies. These results highlight the need to create an evidence-based and standardized approach to crisis management for MH apps.
Implications of this work include recommending that app evaluation frameworks should include presence of crisis language in order to ensure that this feature is implemented in apps, and its presence (or lack) is known when people make informed decisions in selecting that app. Crisis language is not currently a criterion used to evaluate apps in any app evaluation framework (American Psychiatric Association, 2020; Henson et al., 2019; Nebeker et al., 2019). Second, international resources such as the International Association for Suicide Prevention’s (IASP) crisis resource search tool (IASP, 2020) are useful to direct MH app users, especially considering apps are often used by international audiences. Third, we propose that “best practices” be developed, tested, and disseminated in order to be used as a centralized resource for crisis language in MH apps. A best practice may be to place a crisis banner on the bottom of every screen or include a crisis resources document readily available in the app. Fourth, although some research has highlighted that app users want emotional support from apps (de Alva et al., 2015), further research should be conducted to understand the salient features in an app’s management of suicidal ideation from a user standpoint. Last, it is important to note that risk may vary depending on app content and specific use of the app (e.g., mindfulness vs. suicide prevention).
Limitations
A key limitation of this work is that it examines potential risks and not harms experienced nor the potential benefit of including more robust crisis resources. We also did not comprehensively sample MH apps available nor search for included apps using the Apple App Store/Google Play Store. Rather, we approached our app selection as an app user might – by consulting popular-press Internet articles readily available on Google (Schueller et al., 2018). It is important to note that we are not evaluating the apps themselves in this paper, but rather the presence of crisis language for the apps. Despite these limitations, our findings speak to a lack of standards around crisis-related resources, terms of service, and privacy policies among commonly used MH apps.
Conclusion
MH apps are easily accessible, but lack a coherent approach to directing users to care when in crisis. The majority of apps do not direct users to resources if in crisis and those that do provide resources may direct users to inappropriate crisis support numbers. Clinicians have a role to play in advising consumers who might be interested in trying an app and need to be aware of how the apps may direct or not direct individuals in crisis. App developers and those creating the terms of service, along with users, may help identify best practices in shaping crisis language to direct users to needed care. Given the alarming suicide rate and increasing uptake of MH apps, it seems critical that MH apps be developed with consideration that persons in crisis are a substantial proportion of their users.
Acknowledgments
We would like to thank Megha Majumder for her contributions to early versions of this project.
Conflict of Interest
Dr. J. Torous receives unrelated research support from Otsuka Pharmaceutical, a role that does not represent a conflict of interest with this study. Dr. R. C. Moore is a co-founder of KeyWise, Inc., a role that does not represent a conflict of interest with this study. All other authors have no disclosures.
Funding
This work was supported by the National Institutes of Health (R. C. Moore, grant numbers NIMH R21MH116104, NIA R01AG062387; C. A. Depp NIMH MH116902, NIMH MH113613).
Biography
Emma M. Parrish, BS, is a graduate student in the San Diego State University/University of California San Diego Joint Doctoral Program in Clinical Psychology, CA. Her research interests lie in real-time interventions and assessments through technology for people with serious mental illness, with a focus on suicide prevention, functioning, and cognition.
Tess F. Filip, BA, is a project coordinator at the University of California San Diego, CA. She has also worked as a case manager and as a research coordinator in a clinical pharmaceutical trial setting. Her interests lie in social cognition in severe mental illness, particularly schizophrenia.
John Torous, MD, MBI, is a dually board-certified psychiatrist and clinical informaticist who is an assistant professor at Harvard Medical School, Boston, MA. He runs a digital mental health clinic serving patients with serious mental illness and researches smartphone and sensors to improve and augment mental health care.
Camille Nebeker, EdD, MS, is a research ethicist and associate professor in the Herbert Wertheim School of Public Health, UC San Diego, CA. Her research focuses on two intersecting areas: community research capacity building (e.g., citizen science, community engaged research) and digital health research ethics (e.g., consent, access, privacy expectations, data management).
Raeanne C. Moore, PhD, is associate professor of psychiatry at the University of California, San Diego (UCSD), CA. She researches the use of digital health technologies to improve the assessment of cognitive functioning, as well as machine learning and AI to identify digital biomarkers predictive of cognitive decline.
Colin A. Depp, PhD, is professor of psychiatry at UC San Diego, CA, and a clinical staff psychologist at the San Diego Veterans Affairs Hospital. Dr. Depp’s research focuses on technology-based longitudinal assessment and real-time interventions in mental health – particularly in serious mental illnesses.
Appendix
Table A1.
App descriptions
| App name | iPhone, Android, or both? | Free to download? | Date downloaded | Country developed | App store category | Content | Target population |
|---|---|---|---|---|---|---|---|
| 10% Happier: Meditation | Both | Free | October 20, 2019 | USA | Health and fitness | General mental wellbeing | |
| 7 Cups | Both | Free | October 20, 2019 | USA | Health and fitness | Online chat | Anxiety and depression |
| Anxiety Relief Hypnosis | Android | Free | November 21, 2019 | USA | Medical | General mental wellbeing | Anxiety disorders, depression, OCD, PTSD, and panic disorder |
| Anxiety Reliever | Both | Free | October 20, 2019 | USA | Health and fitness | General mental wellbeing | Anxiety and depression |
| AnxietyCoach | iPhone | Free | November 21, 2019 | USA | Health and fitness | Informational/mood tracking | Anxiety disorders |
| Aura | Both | Free | November 18, 2019 | USA | Health and fitness | General mental wellbeing | Anxiety |
| Brain.fm | Both | Free | November 18, 2019 | USA | Health and fitness | General mental wellbeing | |
| Breathe2Relax | Both | Free | December 6, 2019 | USA | Health and fitness | General mental wellbeing | Anxiety |
| Buddhify | Both | Paid: $4.99 | November 21, 2019 | UK | Health and fitness | General mental wellbeing | Anxiety |
| Calm | Both | Free | November 21, 2019 | USA | Health and fitness | General mental wellbeing | Anxiety |
| CPT Coach | Both | Free | December 19, 2019 | USA | Health and fitness | Informational/activities | PTSD |
| HabitBull | Both | Free | November 21, 2019 | USA | Productivity | Productivity | |
| Happify | Both | Free | January 28, 2020 | USA | Health and fitness | Activities/forum | |
| Headspace | Both | Free | November 20, 2019 | USA | Health and fitness | General mental wellbeing | Anxiety |
| INSCAPE | iPhone | Free | November 21, 2019 | USA | Health and fitness | General mental wellbeing | Anxiety and depression |
| Insight Timer | Both | Free | November 21, 2019 | USA | Health and fitness | General mental wellbeing | Anxiety |
| Live OCD Free | iPhone | Paid: $24.99 | USA | Health and fitness | Informational/activities | OCD | |
| Mindshift | Both | Free | December 6, 2019 | Canada | Health and fitness | Informational/activities/general mental wellbeing | Anxiety disorders, panic disorder, social anxiety, and phobias |
| MoodKit | iPhone | Paid: $6.99 | USA | Health and fitness | Informational/mood tracking | ||
| Moodnotes | iPhone | Paid: $4.99 | USA | Health and fitness | Informational/mood tracking | ||
| Moodpath | Both | Free | January 28, 2020 | Germany | Medical | Informational/mood tracking | Anxiety and depression |
| MoodTools | Both | Free | December 6, 2019 | USA | Medical | Informational/activities | Anxiety disorders, depression, PTSD, bipolar disorder, schizophrenia, seasonal affective disorder, dysthymia, OCD, panic disorder, and GAD |
| Motivate Me | Both | Free | December 2, 2019 | France | Health and fitness | Productivity/general mental wellbeing/activities | Anxiety |
| Sanvello | Both | Free | November 21, 2019 | USA | Health and fitness | Informational/mood tracking/activities | Anxiety and depression |
| Panic Relief | Both | Paid: $0.99 | Denmark | Medical | Informational/activities | Panic disorder and anxiety disorders | |
| PE Coach 2 | Both | Free | November 21, 2019 | USA | Health and fitness | Informational/activities | PTSD |
| Productive | Both | Free | November 21, 2019 | USA | Productivity | Productivity | |
| PTSD Coach | Both | Free | November 21, 2019 | USA | Health and fitness | Informational/activities | PTSD |
| Self-Help Anxiety Management | Both | Free | November 21, 2019 | UK | Health and fitness | Informational/mood tracking | Anxiety |
| Smiling Mind | Both | Free | December 6, 2019 | Australia | Health and fitness | General mental wellbeing | |
| Stop, Breathe, Think | Both | Free | January 30, 2020 | USA | Health and fitness | General mental wellbeing | Anxiety and depression |
| Streaks | iPhone | Paid: $4.99 | Australia | Health and fitness | Productivity | ||
| SuperBetter | Both | Free | November 21, 2019 | USA | Health and fitness | Activities | Anxiety, depression, and PTSD |
| T2 Mood Tracker | Both | Free | December 6, 2019 | USA | Health and fitness | Mood tracking | Anxiety, depression, PTSD, and TBI |
| Talkspace | Both | Free | December 6, 2019 | USA | Health and fitness | Online chat | Depression, anxiety, PTSD, and substance use |
| The Mindfulness App | Both | Free | January 28, 2020 | Sweden | Health and fitness | General mental wellbeing | |
| What’s up? | Both | Free | November 22, 2019 | Australia | Health and fitness | General mental wellbeing/mood tracking/activities | Depression and anxiety |
| WorryWatch | iPhone | Paid: $3.99 | USA | Health and fitness | Mood tracking/productivity | Anxiety |
Note. GAD = generalized anxiety disorder; OCD = obsessive compulsive disorder; PTSD = posttraumatic stress disorder; TBI = traumatic brain injury.
Table A2.
Terms of service/end-user license agreement, privacy policy, and in-app crisis language and mental health resources
| name | In-app mental health resources | Resources description | TOS/EULA present | Suicide-specific crisis language in TOS/EULA | TOS/EULA in app | PP present | Suicide-specific crisis language in PP | PP in app |
|---|---|---|---|---|---|---|---|---|
| 10% Happier: Meditation | N/A | X | X | X | X | |||
| 7 Cups | X | Online chat/forum | X | X | b | X | X | |
| Anxiety Relief Hypnosis | N/A | X | X | X | ||||
| Anxiety Reliever | N/A | X | X | X | X | |||
| AnxietyCoach | X | Psychoeducation | Xa | X | b | Xa | ||
| Aura | N/A | X | X | X | X | |||
| Brain.fm | N/A | X | X | X | X | |||
| Breathe2Relax | N/A | X | b | c | c | |||
| Buddhify | N/A | X | X | X | X | |||
| Calm | N/A | X | X | X | X | |||
| CPT Coach | X | Psychoeducation; national emergency number (911) | X | b | X | X | ||
| HabitBull | N/A | X | X | X | X | |||
| Happify | X | Online chat/forum; national MH organizations; hotline; resources for finding MH care | X | X | X | X | ||
| Headspace | N/A | X | X | X | X | |||
| INSCAPE | N/A | X | X | X | X | |||
| Insight Timer | N/A | X | b | X | ||||
| Live OCD Free | N/A | N/A | N/A | N/A | N/A | N/A | ||
| Mindshift | X | National emergency number (911); hotline; local and national MH organizations; psychoeducation | X | X | X | X | ||
| MoodKit | N/A | N/A | N/A | N/A | X | N/A | ||
| Moodnotes | N/A | N/A | N/A | N/A | X | N/A | ||
| Moodpath | X | Online chat/forum; hotline; psychoeducation | X | X | X | X | ||
| MoodTools | X | Psychoeducation; safety planning; resources for finding MH care; hotline; national emergency number (911) | X | X | ||||
| Motivate Me | N/A | X | X | X | X | |||
| Sanvello | X | National emergency number (911); hotline; online chat/forum | X | X | X | X | ||
| Panic Relief | N/A | N/A | N/A | N/A | N/A | N/A | ||
| PE Coach 2 | X | National emergency number (911); hotline, psychoeducation; safety planning | Xa | b | X | X | ||
| Productive | N/A | X | X | X | X | |||
| PTSD Coach | X | National emergency number (911); hotline; safety planning; national MH organizations; resources for finding MH care; psychoeducation | X | b | X | X | ||
| Self-Help Anxiety Management | X | National MH organizations; hotline | X | X | X | |||
| Smiling Mind | N/A | X | X | X | X | |||
| Stop, Breathe, Think | N/A | X | X | X | X | |||
| Streaks | N/A | N/A | N/A | N/A | X | N/A | ||
| SuperBetter | N/A | X | X | X | X | |||
| T2 Mood Tracker | X | National MH organizations | X | b | c | c | ||
| Talkspace | X | National emergency number (911); hotline; online chat/forum | X | X | X | X | X | |
| The Mindfulness App | N/A | X | X | X | X | |||
| What’s up? | X | Hotline; national MH organizations; online chat/ forum; psychoeducation | N/A | X | ||||
| WorryWatch | N/A | N/A | X | N/A | X | N/A | ||
| Number apps with criteria/total no. of apps | 14/31 | 14 | 32/38 | 4/32 | 21/32 | 34/36 | 1/34 | 23/32 |
| Percentage of apps | 45.2% | 45.2% | 84.2% | 12.5% | 65.6% | 94.4% | 2.9% | 71.9% |
Note. EULA = end user licensing agreement; MH = mental health; N/A = not applicable; PP = privacy policy; TOS = terms of service.
EULA/TOS or PP only available after downloading app before sign in, not online or app store.
EULA/TOS or PP only shown either as hyperlink or review before initial sign up, and not accessible in-app.
Hyperlink to PP or website, but website will not load and unable to determine.
References
- 7 Cups. (2018). 7 Cups privacy policy. https://www.7cups.com/Documents/PrivacyPolicy/
- Aguirre RTP, McCoy MK, & Roan M (2013). Development guidelines from a study of suicide prevention mobile applications (apps). Journal of Technology in Human Services, 31(3), 269–293. 10.1080/15228835.2013.814750 [DOI] [Google Scholar]
- American Psychiatric Association. (2020). App evaluation model. https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/app-evaluation-model
- Anxiety and Depression Association of America. (2019). ADAA Reviewed Mental Health Apps. https://adaa.org/finding-help/mobile-apps
- AnxietyReliever App. (2016). Terms of service. https://anxietyrelieverapp.com/terms
- de Alva FEM, Wadley G, & Lederman R (2015). It feels different from real life: Users’ opinions of mobile applications for mental health. Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction. 10.1145/2838739.2838806 [DOI] [Google Scholar]
- Gerry LM (2019). 23 Mental health apps for stress, anxiety, and moreVeryWell Mind. https://www.verywellmind.com/best-mental-health-apps-4588479
- Henson P, David G, Albright K, & Torous J (2019). Deriving a practical framework for the evaluation of health apps. The Lancet Digital Health, 1(2), e52–e54. 10.1016/S2589-7500(19)30013-5 [DOI] [PubMed] [Google Scholar]
- Huckvale K, Torous J, & Larsen ME (2019). Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Network Open, 2(4), e192542. 10.1001/jamanetworkopen.2019.2542 [DOI] [PMC free article] [PubMed] [Google Scholar]
- International Association for Suicide Prevention (IASP). (2020). International Association for Suicide Prevention – Resources: Crisis centers. https://www.iasp.info/resources/Crisis_Centres/
- James SL, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, Abbastabar H, Abd-Allah F, Abdela J, Abdelalim A, Abdollahpour I, Abdulkader RS, Abebe Z, Abera SF, Abil OZ, Abraha HN, Abu-Raddad LJ, Accrombessi MMK, Acharya D, … Abdelalim A (2018). Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: A systematic analysis for the Global Burden of Disease Study 2017. The Lancet, 392(10159), 1789–1858. 10.1016/S0140-6736(18)32279-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jimenez G, Lum E, & Car J (2019). Examining diabetes management apps recommended from a Google search: Content analysis. JMIR mHealth and uHealth, 7(1), e11848. 10.2196/11848 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Martinengo L, Van Galen L, Lum E, Kowalski M, Subramaniam M, & Car J (2019). Suicide prevention and depression apps’ suicide risk assessment and management: A systematic assessment of adherence to clinical guidelines. BMC Medicine, 17(1), 1–12. 10.1186/s12916-019-1461-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- Melia R, Francis K, Hickey E, Bogue J, Duggan J, O’Sullivan M, & Young K (2020). Mobile health technology interventions for suicide prevention: Systematic review. JMIR mHealth and uHealth, 8(1), e12516. 10.2196/12516 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moodpath. (2019). General terms and conditions. https://mymoodpath.com/en/terms-services/
- Nebeker C, Bartlett Ellis RJ, & Torous J (2019). Development of a decision-making checklist tool to support technology selection in digital health research. Translational Behavioral Medicine, 10(4), 1004–1015. 10.1093/tbm/ibz074 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nichols H (2018, January 8). The top 10 mental health apps. Medical News Today. https://www.medicalnewstoday.com/articles/320557
- Schueller SM, Neary M, O’Loughlin K, & Adkins EC (2018). Discovery of and interest in health apps among those with mental health needs: Survey and focus group study. Journal of Medical Internet Research, 20(6), e10141. 10.2196/10141 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sunyaev A, Dehling T, Taylor PL, & Mandl KD (2015). Availability and quality of mobile health app privacy policies. Journal of the American Medical Informatics Association, 22(e1), e28–e33. 10.1136/amiajnl-2013-002605 [DOI] [PMC free article] [PubMed] [Google Scholar]
- TermsFeed. (2020, December 19). EULA vs. terms of use for a mobile app. https://www.termsfeed.com/blog/eula-vs-terms-of-use-mobile-app/#:~:text=Another%20difference%20is%20that%20an,browse%20to%20agree%20to%20it
- Torous J, Staples P, Shanahan M, Lin C, Peck P, Keshavan M, & Onnela J-P (2015). Utilizing a personal smartphone custom app to assess the patient health questionnaire-9 (PHQ-9) depressive symptoms in patients with major depressive disorder. JMIR Mental Health, 2(1), e8. 10.2196/mental.3889 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Torous J, Wisniewski H, Liu G, & Keshavan M (2018). Mental health mobile phone app usage, concerns, and benefits among psychiatric outpatients: Comparative survey study. JMIR Mental Health, 5(4), e11715. 10.2196/11715 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weisel KK, Fuhrmann LM, Berking M, Baumeister H, Cuijpers P, & Ebert DD (2019). Standalone smartphone apps for mental health – a systematic review and meta-analysis. npj Digital Medicine, 2(1), 118. 10.1038/s41746-019-0188-8 [DOI] [PMC free article] [PubMed] [Google Scholar]