Abstract
The expanding domain of digital mental health is transitioning beyond traditional telehealth to incorporate smartphone apps, virtual reality, and generative artificial intelligence, including large language models. While industry setbacks and methodological critiques have highlighted gaps in evidence and challenges in scaling these technologies, emerging solutions rooted in co‐design, rigorous evaluation, and implementation science offer promising pathways forward. This paper underscores the dual necessity of advancing the scientific foundations of digital mental health and increasing its real‐world applicability through five themes. First, we discuss recent technological advances in digital phenotyping, virtual reality, and generative artificial intelligence. Progress in this latter area, specifically designed to create new outputs such as conversations and images, holds unique potential for the mental health field. Given the spread of smartphone apps, we then evaluate the evidence supporting their utility across various mental health contexts, including well‐being, depression, anxiety, schizophrenia, eating disorders, and substance use disorders. This broad view of the field highlights the need for a new generation of more rigorous, placebo‐controlled, and real‐world studies. We subsequently explore engagement challenges that hamper all digital mental health tools, and propose solutions, including human support, digital navigators, just‐in‐time adaptive interventions, and personalized approaches. We then analyze implementation issues, emphasizing clinician engagement, service integration, and scalable delivery models. We finally consider the need to ensure that innovations work for all people and thus can bridge digital health disparities, reviewing the evidence on tailoring digital tools for historically marginalized populations and low‐ and middle‐income countries. Regarding digital mental health innovations as tools to augment and extend care, we conclude that smartphone apps, virtual reality, and large language models can positively impact mental health care if deployed correctly.
Keywords: Digital mental health, smartphone apps, virtual reality, generative artificial intelligence, large language models, engagement, implementation science, depression, anxiety, schizophrenia, eating disorders, substance use disorders
The surge in telehealth related to the COVID‐19 pandemic has transformed the behavioral health field 1 , 2 , 3 , yet the nature of the emerging domain remains in flux. Synchronous telehealth (video visits) rapidly expanded access to care during the pandemic, and psychiatry recorded the highest use of these visits compared to other medical specialties 3 . However, the reliance of traditional telehealth on clinician availability limited scalability, and growth is already contracting. Recent data indicate that telehealth visits in 2024 were less than 50% of their COVID‐19 peak 4 . Few clinics today offer fully virtual practices, with the majority instead providing a blend of online and in‐person care options 5 , 6 . These changes are partly driven by unstable telehealth legislation 7 , with many of the regulations that permitted the rapid move to telehealth during the pandemic now expired or in flux. But they also reflect a deeper concern that telehealth alone is insufficient to substantially increase access to care and quality of mental health services.
Asynchronous digital health – such as the use of smartphone apps, virtual reality, and generative artificial intelligence, including large language models (LLMs) – offers unique opportunities to scale care delivery. Unlike traditional telehealth, these tools can function as self‐help, coach‐guided, or clinician‐led interventions, providing flexibility and accessibility outside of immediate clinician interactions 8 . While initial enthusiasm for these technologies remains high, a notable gap in robust, real‐world evidence continues to preclude their integration into routine care 9 . Despite significant advancements since our early review published in this journal 1 , recent industry failures and research critiques have highlighted the need for more rigorous approaches, including use of digital placebos in controlled trials, generalizable and pre‐registered models, and greater transparency in data sharing 10 , 11 , 12 . Much of recent research has focused on how a particular app or artificial intelligence program might work, but has not produced mechanistic and generalizable evidence that the field can utilize to build a strong scientific base. These setbacks, however, lay the groundwork for a new generation of evidence‐based digital innovations.
Hybrid models that utilize both traditional telehealth and asynchronous digital health reflect the latest evidence and represent a promising approach to increase access and quality of care. However, blending the use of novel technologies into care requires careful consideration. Emerging usage of digital navigators (technology coaches) 13 to supplement digital mental health interventions and support patients has gained attention with the growing recognition that self‐help tools offer limited effectiveness without some degree of human support. The optimal dose and balance of human and digital support, delivered in new hybrid or blended formats, presents a new frontier. It also broadens the concept of digital health from tools or products to care‐delivery platforms. Already successful models have integrated dedicated digital mental health services in Australia 14 , 15 , Denmark 16 , Sweden 17 , Norway 18 , the US 19 , and Canada 20 , among others. While it is possible that artificial intelligence and LLMs could soon serve some of digital navigator features, the evidence remains scant today, and the importance of the human connection in mental health care should not be underestimated.
To explore the evolution of digital mental health and how this new generation of tools will work with humans to create superior outcomes, this review focuses on five key areas. First, it examines recent advances in smartphones, virtual reality, generative artificial intelligence, and LLMs. Second, it evaluates clinical outcomes for smartphone apps across common mental health conditions. Third, it explores engagement challenges hampering all digital mental health tools, and proposes several solutions. Fourth, it analyzes implementation strategies to support real‐world adoption and scalability. Fifth, it addresses how current tools often fail to meet the needs of overlooked populations, including cultural minorities and those living in low‐resource settings 21 , 22 , and explores possible ways forward. Common threads across all five themes – around scientific rigor, real‐world engagement, community partnerships, and blended‐care models – reinforce the transformation of the field from creating tools to improving care.
Looking ahead, innovation in engagement strategies and implementation science will play pivotal roles in advancing the next generation of digital tools. Just‐in‐time adaptive interventions, digital phenotyping, and personalized approaches are gaining renewed attention for their potential to address long‐standing challenges in adherence and effectiveness. This paper offers an optimistic perspective on the field's evolution, and a well‐defined roadmap for the years to come.
RECENT TECHNOLOGICAL ADVANCES
Smartphone apps and digital phenotyping
Smartphone apps serving as therapeutics have gained traction, and several of them have been cleared as medical devices by the US Food and Drug Administration (FDA). However, the actual effectiveness of these apps in real‐world conditions remains uncertain. To address this complex topic, a subsequent section of this paper will critically review the current evidence available across a range of mental health conditions.
The interest in smartphones extends beyond their potential to deliver apps and interventions. These same devices are also capable of surveying patients in real time, enabling ecological momentary assessment for the vast majority of the population. In addition, data from smartphone sensors can generate behavioral metrics (e.g., sleep patterns, sedentary periods) and information on environmental exposures (e.g., local temperature, light exposure, greenspace) that can provide personalized contexts and temporal trajectories for how individuals experience mental illness. Often referred to as digital phenotyping 23 , recent evidence on this approach in youth 24 and adults 25 , 26 provides promising signals with clinical validity.
A recent review exploring machine learning applied to digital phenotyping noted that mood disorders, anxiety disorders, and schizophrenia spectrum disorders are the three most studied conditions across all health care, even beyond mental health 27 . Relapse detection in schizophrenia and symptom prediction in mood disorders have strong pilot results with replication and external validation 28 , 29 , 30 which provide promising generalizable clinical signals. The vast number of pilot studies suggest that digital phenotyping research should now move towards validation to determine clinical relevance, with larger sample sizes and longer duration studies 31 . Such ongoing efforts include the US National Institutes of Health's Accelerating Medicine Partnership Schizophrenia Study, capturing smartphone digital phenotyping data from over 40 sites around the world in people at clinical high risk for psychosis for up to 12 months 32 .
While early evidence suggests that digital phenotyping is feasible and acceptable, key barriers in the field remain a lack of standards for data collection, data processing, and feature creation, with variable data streams derived from different models/brands of smartphones. For example, a recent review of digital phenotyping across mental health found that, even when generating seemingly uncontroversial behavioral features such as sleep duration, each study used a different combination of sensors and processing pipelines, so that comparison of results and generalizability of outcomes were challenging 33 . Efforts to externally validate digital phenotyping work thus remain limited 34 .
While research using wearable devices is sometimes labeled as digital phenotyping, this work is best categorized as actigraphy and considered in the framing of that unique field. Core differences include additional needed hardware and results often unique to that hardware and supporting software. While each approach has its merits, digital phenotyping utilizes patients’ existing smartphone devices, so it represents a scalable and low‐cost method limited primarily by device variance and missing data. Wearable studies offer the benefit of devices with often superior sensors, but are more limited in terms of scalability and longer‐term engagement. As smartphone technology and sensors improve, the two fields may continue to blend. Even today, several studies can simultaneously apply both digital phenotyping and wearable devices through Android's Health Connect and Apple HealthKit/SensorKit features.
Given that successful digital phenotyping can support a myriad of other digital health developments, ranging from just‐in‐time adaptive interventions to precision‐guided medication selection, success here will benefit the entire field. A focus on standards around both data collection and data processing in the mental health field, mirroring advances in accelerometry studies generally 35 , can generate better science and more synergistic advances. Likewise, standards around protecting privacy and data governance in this sensitive area can engender trust and patient interest in sharing their personal data for research.
Virtual reality
Virtual reality is emerging as a significant innovation in the field of mental health treatment 36 . In using immersive simulations, it addresses a key limitation of traditional mental health interventions, which are often restricted to clinical settings and rely on patients recalling experiences and subsequently applying therapeutic techniques in their daily lives 37 . A recent review of the field 36 found that a growing body of research supports the efficacy of virtual reality‐based interventions across different mental health conditions.
The unique capacity of virtual reality to recreate real‐world environments has been particularly effective in augmenting cognitive‐behavioral therapy (CBT), otherwise known as VR‐CBT 38 . The majority of randomized controlled trials (RCTs) of VR‐CBT approaches have been conducted in anxiety disorders, with a recent meta‐analysis finding that they were superior to waiting lists or psychoeducation controls 39 . However, significant heterogeneity between effect sizes was evident, and active comparisons yielded non‐significant differences. A meta‐analysis of VR‐CBT for social anxiety disorder also demonstrated that it had superior effects compared to waitlist controls for anxiety symptoms and avoidance behaviors 40 . These findings parallel results from studies in other conditions, such as psychosis, post‐traumatic stress disorder (PTSD) and specific phobias, which indicate that VR‐CBT is generally as effective as traditional CBT 36 , 41 , 42 , 43 .
Virtual reality treatments have also been developed to support psychosocial and functional recovery, with the majority of evidence in mental disorders where routine functioning is challenging, such as schizophrenia, autism, and attention‐deficit/hyperactivity disorder (ADHD) 41 , 42 , 44 , 45 . In these conditions, virtual reality has been found to enhance everyday living skills, including social and vocational tasks, by providing a safe space to learn and practice in relevant scenarios 42 , 45 . However, results have been mixed: a recent RCT of a virtual reality treatment targeting social cognition in psychosis found no significant difference compared to an active virtual reality relaxation condition 46 . This mirrors the findings of another trial comparing VR‐CBT targeting social behaviors with virtual reality relaxation in a psychosis sample, which also found no difference between these conditions 47 .
Emerging evidence suggests that virtual reality can be effectively integrated into various therapeutic modalities by leveraging its capacity to represent visual stimuli and influence affective states within virtual environments. A recent systematic review 48 found that virtual reality‐based relaxation interventions are equally or more effective than non‐virtual reality approaches in reducing short‐term stress and anxiety, with the added benefit of being more resource‐efficient to deliver. Virtual reality has also shown promise in enhancing “third wave” CBT approaches such as mindfulness, acceptance and commitment therapy, and dialectical behavioral therapy (DBT), which have a focus on separating the self from mental events. Similarly, virtual reality‐enhanced DBT has shown the potential to help individuals manage emotional dysregulation more effectively by practicing distress tolerance skills in immersive, controlled environments 49 .
Despite extensive research supporting the efficacy of virtual reality treatments for various mental health conditions, there remain few consumer‐ready applications available on the market. One of the primary challenges is scaling virtual reality interventions to reach a broader population cost‐effectively. Although the technology is becoming more accessible, the expenses associated with high‐quality hardware, software development, and clinician training remain significant barriers 50 . Additionally, strategies to make these interventions accessible in under‐resourced areas are critical 21 , 51 .
Generative artificial intelligence
Few innovations have garnered so much interest in mental health as generative artificial intelligence. This is a unique subset of artificial intelligence in that it can create novel content, such as conversations or images, based on data and patterns on which it has been trained. The public release of ChatGPT 3.5 unleashed interest in the topic and gave rise to a rush for mental health use.
A new generation of artificial intelligence‐driven chatbots is becoming increasingly prevalent in digital mental health, evolving from early rule‐based chatbots. However, these latter chatbots are still common, and a 2022 review suggested that, across all of health care, 96% of chatbots were driven by decision‐tree‐like logic and not actual artificial intelligence 52 . Those earlier systems, which relied on predefined scripts and decision trees, were helpful in controlled environments, but faced limitations in handling complex, real‐world interactions. Their inability to process free‐text inputs or maintain context in multi‐turn conversations raised concerns about their broader applicability 1 , 52 . Examples include psychotherapy chatbots such as versions of Wysa and Woebot, which, despite their limitations, offered the advantage of predictability and reduced risk of errors. The importance of such reduced risk was highlighted in 2023, when a generative artificial intelligence code embedded in an eating disorder chatbot led it to make harmful statements to users, prompting its removal within days of its public release 53 . The underlying issues of bias, subtle errors, and more overt errors (often labeled as “hallucinations”) must be considered in framing the potential of generative artificial intelligence models and assessing the evolving risks that must be weighed with the expanding benefits.
Recent advancements have shifted toward machine learning‐powered models, particularly LLMs. These models, trained on vast datasets from the Internet and other sources, address many of the limitations of rule‐based systems. Their ability to generate human‐like responses has made them valuable not only as tools but also as virtual companions. Users appreciate their capacity to handle diverse inputs, exhibit personality traits, and respond empathetically, which makes them more effective for personalized mental health support. Research shows that LLMs can demonstrate consistent behavior across the Big Five personality traits 54 , and even outperform humans in certain tasks, such as recognizing irony and false beliefs 55 . Furthermore, the multimodal capabilities of modern LLMs enable them to process not just text but also voice and image inputs 56 , 57 , expanding their versatility in digital mental health.
Preliminary research has demonstrated the potential of LLMs across various stages of mental health care. While much of this work has not been replicated, these pilot studies underscore the broad range of applications. For prevention, LLMs can offer low‐risk, personalized psychoeducation, effectively raising mental health awareness by utilizing high‐quality resources 58 , 59 . For relapse or onset detection, LLMs show promise in risk prediction, with studies indicating that models such as GPT‐4 can approach clinical accuracy in identifying suicidal ideation and other crisis indicators, though additional safety measures and bias mitigation are necessary 60 , 61 , 62 . In diagnosis, LLMs can facilitate data‐driven assessments of mental health conditions, sometimes matching clinicians’ ability, for instance in predicting depression scores based on clinical data 63 . For treatment optimization, LLMs can assist in medication selection and therapeutic interventions by leveraging patient‐specific data to help clinicians make informed decisions 64 , 65 . In high‐risk situations, such as crisis intervention, LLMs can provide elements of crisis counseling, although this use carries a higher risk of harm 60 , 66 . Finally, LLMs have been applied to deliver ongoing therapy and counseling, enhancing access to routine mental health services by analyzing past therapy outcomes to improve care 67 , 68 .
Despite the growing popularity of LLM‐powered chatbots for mental health support, this field remains underexplored at its current stage, particularly related to the lack of transparency in training data, explainability of models, and standardized evaluation methods 69 . All base models for LLMs have been trained, at least partially, on social media data. This is understandable given that the newest models need billions, and likely trillions, of parameters (data points) to learn from. But, in learning about mental health mainly from social media, these models have also learned about stigma and bias. This point was well illustrated in a 2022 paper showing a range of stigmatizing images generated in response to prompts around schizophrenia 70 .
Studies have also pointed out that, while these models can perform some theory‐of‐mind tasks, they still struggle with more complex social reasoning, highlighting the gap between artificial intelligence‐driven reasoning and human cognition 71 . Finally, while many models have been proposed to evaluate LLM chatbots on criteria ranging from ethics to efficiency, none are well utilized today, and no standard has emerged 72 . Thus, comparison between chatbots, let alone evaluation of evolving chatbots, remains a challenge.
While LLMs have shown promise in providing human‐like companionship, their unpredictability remains a major challenge. LLMs highlight the therapeutic potential of conversation and the rule‐based nature of human language, meaning that they can produce convincing conversations. However, psychiatry is less rule‐based, with debates about nosology and etiology ongoing today. Thus, LLMs will continue to face challenges as they confront a relative dearth of high‐quality training data. In the meantime, debates on the delineation between conversation vs. therapy and companionship vs. care will continue to shift. Anyway, regardless of where the line is drawn, it is clear that some people are already finding benefits in talking with LLMs.
The current uncertainty around the patient‐facing use of LLMs contrasts with their rapidly evolving use around clinical documentation. While the subject of less media attention and research, the transformative potential of clinician‐facing artificial intelligence tools should not be underestimated. There is already enthusiasm for nascent efforts to utilize LLMs to document clinical encounters 73 , 74 , likely saving clinicians’ hours per day of note‐writing. Other efforts to use artificial intelligence in upskilling of non‐clinicians, in training of clinicians, and in offering clinical decision support are also evolving 75 , 76 , 77 , and could represent a paradigm shift in workforce and training while facilitating evidence‐ and measurement‐based care.
With so many use cases and such rapid progress, LLMs have the potential to drive research and care trends in mental health, if the field can unify such work under clear standards and safety procedures. We have already seen the emergence of ethical issues calling international attention in relation to LLMs, including the eating disorder chatbot case 53 and a case with help‐seeking people explicitly told that they were interacting with a human while it was actually a LLM 78 . Without such standards and safety considerations, impressive technical achievements by LLMs may find a limited role in clinical care beyond documentation.
SMARTPHONE APP INTERVENTIONS
Access to the Internet is now more common via smartphones than computers 79 . The number of smartphone mental health apps has been estimated at 10,000 80 and remains a dynamic landscape, with new apps frequently introduced and others disappearing from the marketplace 81 . Some apps, such as PTSD Coach 82 , are still functional after several years, but most are far less stable. Since most research in the digital mental health field focuses on smartphone apps, we cover this issue in detail in this section.
While the clinical outcomes of studies are important to consider, any benefits, including those discussed below, must be considered along with risks. Adverse events are often not well reported in digital health studies, despite calls to change this 83 , 84 , 85 , 86 , 87 . In some cases, assumed adverse events – such as technology making people with schizophrenia paranoid or delusional – have been disproven through specific studies 85 . Of course, negative effects are not unique to apps, but have also been reported for Internet interventions 88 , face‐to‐face psychotherapies 89 , and virtual reality treatments 90 .
Negative effects from apps can range from mild (e.g., frustration with glitches, boredom) to severe (e.g., symptom deterioration, onset of new symptoms, suicidal ideation). Concerns have been raised that current marketplace offerings have the potential to induce negative effects because many publicly available apps provide content that is either inaccurate or not grounded in evidence‐based treatments 91 . It is difficult to quantify the extent of negative effects, given heterogeneous study designs, sample characteristics, and types of apps delivered. Yet, recent clinical trials in people with a severe mental illness have reported rates of negative effects to be as high as 20% 92 , 93 , 94 , 95 .
This state of research on adverse events makes it difficult to integrate apps into clinical practice safely 83 , 96 . In particular, the degree to which adverse events, such as deterioration, are caused by the use of the smartphone device itself or other external factors may be difficult to understand 87 , 97 . Researchers involved in future clinical trials of apps should plan from the outset to build data‐driven risk prediction models, because this would help ensure that relevant data are collected, enabling better opportunities to match patients to appropriate treatments safely.
Well‐being enhancement apps
A significant proportion of people who download a mental health app report doing so to acquire adaptive psychological skills useful to improve their overall well‐being 98 . Many well‐being apps include meditation, especially mindfulness, practices as a prominent element. For example, during the COVID‐19 pandemic, one app curation service reported that searches for mindfulness apps rose by nearly 2,500% compared to the 156% increase observed for depression‐specific apps 99 . Investment in this space has flourished, with well over 99% of publicly available mental health‐related apps marketing themselves as well‐being and not health devices 100 .
Several RCTs have reported positive effects of self‐guided well‐being apps on various adaptive psychological attributes, including emotion regulation 101 , mindful awareness 102 , psychological flexibility 103 , subjective well‐being 104 , social functioning 105 , and self‐esteem 106 . However, recent meta‐analyses have found that these apps produce modest improvements relative to control conditions on subjective quality of life, positive affect, general well‐being, mindful awareness, psychological flexibility, and self‐compassion 107 , 108 , 109 . Additional high‐quality RCTs are needed to confirm the utility of well‐being apps, as concerns about the quality of existing research have been raised around small sample sizes, inadequate control groups, high risk of bias, high attrition, and low adherence, which likely explain the different published findings 107 , 109 .
Given that as few as 2% of publicly available well‐being apps have scientific evidence supporting their feasibility and efficacy 110 , research partnerships could quickly transform this crowded space. Research focusing on mechanisms of action could also be useful. There is evidence that some well‐being apps may exert their effects through enhanced mindful awareness 102 . However, a similar degree of evidence has been reported for other possible mediators (e.g., purpose in life, cognitive defusion) of effects on psychological distress 111 . Since multiple mechanisms are likely to be at work, tailoring choice to individual users based on such potential mechanisms may usher in a new era of more rational use of these apps.
Depression and anxiety self‐management
Depression and anxiety, at both diagnostic and sub‐threshold levels, are the most prevalent mental health conditions, and are linked with significant impairments in psychological, social and occupational functioning 112 . Since few people with depression or anxiety have access to specialized psychological treatments 113 , apps that are grounded in an evidence‐based framework and offer credible skills, resources or tips have the potential to represent an accessible, cost‐effective and viable option for users to manage their symptoms.
Alongside the proliferation of commercially available depression and anxiety apps 114 , 115 , the number of RCTs evaluating these apps has grown exponentially in recent years. The largest and most recent available meta‐analysis 114 identified 176 RCTs of standalone mental health apps for depressive or anxiety symptoms, 67% of which have been published since 2020. This meta‐analysis found significant although small effects for mental health apps over control conditions in reducing depressive and generalized anxiety symptoms, which corroborates the findings of recent but more narrow meta‐analyses on the effects of apps on these symptoms in specific contexts (e.g., mindfulness meditation apps only 116 , clinically diagnosed depressed patients 117 , the perinatal period 118 ). The large meta‐analysis 114 also found evidence from a smaller number of trials that apps may be beneficial for reducing social anxiety, obsessive‐compulsive, post‐traumatic stress, and acrophobia symptoms, although the findings were considered preliminary due to the small sample sizes and high risk of bias.
Research has recently sought to understand the characteristics of apps that make them more or less effective for depression and anxiety. Knowledge of the mechanisms involved and of “active ingredients” is critical for producing more efficient apps. Such mechanisms and components can be prioritized, added or refined, while the ineffective or redundant components can be discarded 119 . The above‐mentioned recent meta‐analysis 114 showed that effects were larger in trials that delivered apps based on CBT principles (compared to mindfulness or cognitive training) or containing chatbot technology or mood monitoring features. These components could offer greater personalization or foster emotional self‐awareness, resulting in more significant clinical benefit.
One methodological design that can help identify effective components of an app is the factorial trial, in which participants are randomly assigned to the presence or absence of a particular treatment component. A factorial trial was recently conducted 120 to test the efficacy of five CBT skills (self‐monitoring, cognitive restructuring, assertiveness training, behavioral activation, and problem‐solving) delivered through the Resilience Training app in 1,093 university students with sub‐threshold depression. The authors could not identify whether one CBT skill was more effective than another, as reductions in depressive symptoms were observed for all participants, regardless of the presence or absence of the five CBT skills. However, this trial is noteworthy, and it is encouraging to see further factorial trials on depression apps underway 121 . These trials will ideally shed light on active ingredients, generate hypotheses for future research, and inform the development of more effective self‐management apps.
A consistent trend observed in recent research is that the provision of human guidance augments the effect sizes found for depression and anxiety apps 122 . This finding may be due to human support increasing app engagement, offering additional therapy, or mediating/moderating outcomes through the benefits of therapeutic alliance. The involvement of digital navigators may be useful in this respect. Moreover, the next generation of chatbots that can better personalize recommendations and simulate emotional and empathic responses may offer a novel and complementary approach to increase the efficacy of digital health tools 123 .
Overall, the effects of depression and anxiety self‐management apps are now established on the basis of nearly 200 trials, underscoring the need to use ongoing research opportunities for further advancements. Carefully designed studies focusing on mechanisms of change, the impact of engagement on clinical outcomes, the use of automated support systems, and integration into real‐world settings will likely prove more valuable than additional trials confirming already available results.
Clinical management of mood disorders
Existing research has mostly focused on the effects of apps as a standalone, low‐intensity intervention option among community or student samples screened for mild‐to‐moderate symptoms of depression. Less is known about the utility of apps in severe mood disorders.
New meta‐analytic evidence suggests that apps may enhance the efficacy of conventional treatments for major depressive disorder. A systematic review 124 recently located five RCTs that assessed the added value of integrating apps into standard treatment for this disorder. From seven comparisons, a small but significant effect was found in favor of app‐augmented treatment arms, which was robust after removing trials with high risk of bias. Although preliminary, these findings are promising and suggest that apps may offer an incremental benefit to standard care for major depression. Additional research is needed to identify the optimal timing, dosage and content to combine these interventions effectively with established approaches for maximum benefit.
The fluctuations in mood, cognition and behavior that characterize bipolar disorder support the use of data continuously collected through real‐time approaches such as digital phenotyping, and indicate a possible value of apps to provide tailored treatment strategies. However, the clinical benefit of apps in the management of this disorder is currently unclear. A recent meta‐analysis 125 identified seven RCTs that integrated monitoring apps in the treatment of bipolar disorder, concluding that there was no evidence that they assist in reducing the severity of depressive and manic symptoms. In fact, individual trials have found that, in some cases, monitoring apps may even increase the risk of depressive episodes 126 or be associated with an escalation in manic symptoms 127 .
These findings led to recommendations from the International Society for Bipolar Disorders Big Data Task Force that future trials of monitoring apps should consider using more sensitive outcomes, such as mood instability, in addition to relapses and psychiatric hospitalizations 125 . Indeed, a more recent trial evaluating the LiveWell self‐management app in 205 patients with bipolar disorder 128 found no difference in reduction of relapse risk for those assigned to the app relative to treatment as usual, but did detect positive effects on depressive symptoms and relational quality of life. Overall, while there are some promising trends in the use of monitoring apps for the clinical management of bipolar disorder 129 , there is a clear need for further research aiming to better understand how, for whom, and under what set of circumstances these apps can be safely integrated into the clinical management of the disorder.
While the clinical focus of apps for management of mood disorders requires their integration into ordinary care, a core issue today is clinicians’ hesitancy and limited awareness. For example, survey research 130 shows that two‐thirds of health care providers have little to no knowledge about apps available for bipolar disorder, and only 10% of clinicians surveyed in another study perceived apps to be helpful for patients with severe depression 131 , despite the above‐mentioned empirical evidence. Investment in workshops and educational videos that provide trustworthy, up‐to‐date information about apps could increase provider confidence 132 , 133 , 134 .
Schizophrenia/psychosis
Smartphone technology represents a potential tool to increase access to care of people with schizophrenia, and has been studied as such for over a decade 135 . Concerns that app‐assisted monitoring tools and interventions could increase paranoia and delusions are refuted by clear data suggesting that people with schizophrenia are receptive and eager to use smartphone technology as part of their treatment plan 85 . As a consequence, research on apps for early diagnosis, real‐time monitoring, psychoeducation, lifestyle, relapse prevention, and intervention among people with schizophrenia has rapidly expanded in the last ten years 136 .
Several RCTs of app‐supported interventions in individuals with schizophrenia have found positive effects on important clinical outcomes, including reduced fear of relapse 137 , and improvement of psychotic symptoms 138 , cognitive functioning 139 , 140 , depressive symptoms 141 , and medication adherence 139 . However, not all trials have reported favorable results. For instance, incorporating an app that offered a toolbox of behavioral and cognitive skills (PEAR‐004) conferred no added clinical benefit on symptom scores relative to a non‐specific digital sham control among 112 patients with schizophrenia receiving antipsychotic medication 142 . Similarly, the delivery of the self‐guided Temstem app, designed to provide coping skills to deal with voice hearing, was not superior to a placebo monitoring app in reducing voice hearing distress, and in increasing social functioning and control over voices, among 89 patients with severe mental illness 143 . Likewise, the CBT‐informed Actissist app study reported no difference in outcomes for people with schizophrenia when compared to a mood‐tracking app 144 . The SlowMo 145 and Horyzons 146 blended interventions also reported null primary results, but some effects on secondary outcomes were promising, underscoring both the potential of blended approaches and the need for more rigorous research.
A recent systematic review and meta‐analysis of 26 RCTs considering smartphones and other digital technologies in people with schizophrenia reported minimal effects, but found that these may increase when the technology is paired with human support 147 . The critical role of human support was highlighted in another recent review paper 148 .
The potential for smartphone apps to address the significant physical health inequalities among people with schizophrenia is emerging as a new direction. While digital innovations for physical health have hitherto been neglected in this population, there are signs of renewed interest 149 . Encouragingly, findings of a recent review indicated that digital health behavior change interventions, including apps, were broadly feasible and acceptable to people with severe mental illness 150 .
Even if apps are to be integrated into clinical care for schizophrenia spectrum disorders, concerns with their current accessibility and availability have been highlighted. A review of the marketplace 151 identified 25 apps aimed to support people with psychosis. Crucially, 19 of these apps were either non‐functional, inaccessible without an access code, or contained outdated, stigmatizing or harmful information. Of the six easily accessible, appropriate and psychosis‐specific marketplace apps, five exclusively provided psychoeducation content, while only one offered therapeutic and monitoring features. These findings suggest an urgent need for better translation of apps from research to the marketplace.
Eating disorders
As less than one‐quarter of people with eating disorders have access to specialized treatment 152 , our early review on digital mental health in this journal 1 highlighted the potential clinical value of apps for these conditions, as evidenced by a handful of RCTs finding CBT apps to be efficacious as either a standalone intervention or as an adjunct to traditional treatment services. Since then, research on apps for eating disorders has been relatively limited, which is surprising, given that these apps are in high demand and are met with great enthusiasm among this clinical population 153 , 154 .
The need for more rigorous research on eating disorder apps has been highlighted recently 154 . A review of the marketplace identified 65 apps aimed to support the treatment of these disorders 154 , whose quality was suboptimal, with 92% omitting key in‐app features, and only 7% having any research support. Several RCTs evaluating eating disorder apps have emerged since that review. One trial delivered a blended CBT digital intervention for binge‐eating disorder, comprised of a web program supported by a mobile app that enabled users to practice homework skills in daily life. The intervention group reported greater reductions in eating disorder symptoms and psychological distress than the control group 155 . Another trial 156 investigated whether a monitoring app enhanced the efficacy of a CBT web program in a symptomatic sample of 293 participants. While no between‐group differences emerged on key symptoms, those allocated to the app‐augmented intervention were less likely to drop out, suggesting that monitoring apps could help retain users for longer periods in this context.
Recent efforts have explored whether app‐enabled micro‐intervention prompts in high‐risk settings could have therapeutic value in eating disorders. Juarascio et al 157 developed a just‐in‐time adaptive intervention that provided personalized skill recommendations in real time based on data recorded through digital monitoring mechanisms. In a small pilot trial 158 , they compared the presence versus absence of the intervention among 56 patients with bulimia nervosa who were receiving standard CBT. The intervention demonstrated feasibility and acceptability, but did not produce a greater rate of symptom change, possibly because the study was underpowered. It is encouraging to see larger trials of just‐in‐time adaptive interventions for eating disorders in progress 159 , which will ideally shed more light on whether these interventions offer clinical benefit to a population who find it difficult to forecast warning signs of symptom escalation and relapse.
Substance use disorders
People with a substance use disorder are typically reluctant to seek professional help, are prone to relapse, and find it difficult to anticipate those events that trigger cravings 160 , 161 . This, coupled with the fact that smartphone ownership is as high as 92% among people with these disorders 162 , indicates the potential for apps to enhance treatment seeking, mental health literacy, and therapeutic outcomes in this clinical population.
While some of the earliest FDA clearances for apps were around substance use disorders, a 2020 report from the Institute for Clinical and Economic Review suggested that the underlying evidence for these early apps was poor 163 . However, since then, the empirical research exploring the role of apps for these disorders has been continually evolving. In the context of smoking, a recent meta‐analysis 124 identified ten RCTs which tested whether apps can increase the efficacy of conventional treatment, and reported a significant moderate effect in favor of augmented treatment conditions. Another recent meta‐analysis 164 examined the efficacy of apps as either a standalone or adjunctive intervention on smoking abstinence rates, finding no significant between‐group difference from nine RCTs. However, follow‐up analysis showed that apps produced higher rates of smoking cessation than control conditions when paired with pharmacotherapy, further demonstrating the potential for apps to augment conventional treatment approaches.
Comparatively, less research has been conducted on apps for other substance use disorders. A systematic review 165 of mobile interventions identified three pilot studies that focused on cannabis use, which all reported positive treatment effects. In contrast, some systematic reviews have synthesized evidence for smartphone interventions targeting risky alcohol use across distinct population groups, concluding that the evidence for their effectiveness is uncertain 166 . The clinical benefit of app‐based just‐in‐time adaptive interventions specifically designed to target illicit substance use was recently summarized in a systematic review 167 , which concluded that the evidence for their therapeutic value is mixed and that adequately powered efficacy trials are lacking.
Summary for apps across all conditions
From the rapidly evolving evidence base available, it emerges that app‐based interventions have an established efficacy in the self‐management of depression and anxiety, while the evidence is mixed concerning their role in well‐being enhancement, clinical management of mood disorders, schizophrenia/psychosis, eating disorders, and substance use disorders.
Further research is obviously needed in order to study these interventions with more rigorous methods, such as digital placebos and factorial trial designs; to investigate the working mechanisms of these devices; to explore innovative ways to embed these technologies into practice to ensure that they meet their potential as scalable tools; and to build clinical prediction models helping to select the best available treatment approach for each patient given his/her profile and ongoing progress.
CHALLENGES IN ENGAGEMENT
One of the most widely cited challenges to the utilization of mental health digital tools is low engagement, which refers to a lack of uptake and/or poor adherence to interventions in service users 168 , 169 . Even among individuals who consent to participate in a study on a mental health app, as many as 50% never download the app 170 . Furthermore, those who download the app are unlikely to use it for more than a few days, and even fewer complete the entire treatment program. For example, one study found that nearly half of the participants allocated to the popular Headspace and Smiling Mind apps reported never using the app again after ten days 171 . Another study on Headspace found that only 2% of stressed employees completed all of the prescribed meditation sessions 172 . Engagement issues in mental health app trials appear to be a problem not localized to specific settings, populations or clinical groups 173 .
Poor engagement is an even greater problem in real‐world settings. Investigation of real‐world objective data on user engagement with 93 popular mental health apps showed a median daily open rate of 4%, with around a 3% retention rate over a 30‐day period 174 . A recent naturalistic evaluation 175 of the HeadGear depression app showed that, while there were over 26,000 new downloads over the study period, there were only 90 average active daily users, and less than 6% of those who commenced the 30‐day challenge component of the app completed it in its entirety. Another recent study 176 examined objective engagement data from 158,930 individuals who downloaded the publicly available MoodTools app. Analyses showed that nearly 50% never logged into the app a second time, one‐third of active sessions lasted between 0 and 10 seconds, and less than 1% of sessions occurred following a 3‐month to 1‐year period of inactivity.
Knowing the causes of low engagement is necessary for developing solutions. Factors likely contributing to low engagement include poor usability, lack of user‐centric design, concerns about privacy, skepticism about benefits/usefulness, limited digital literacy skills, and lack of personalization features 169 , 177 , 178 .
Addressing engagement through personalized fit and integration into daily life
Systematic reviews report that customizable, personalized content which aligns with users' values and culture supports better engagement 177 , 179 , while one‐size‐fits‐all approaches are less engaging 179 , 180 . Digital mental health interventions need to better align with users' needs and expectations 168 , 180 , and be tailored to be inclusive for minority groups 181 , 182 and by age 183 . Customizable reminders and assessments are reported as beneficial to enhancing engagement 184 , 185 . It has been suggested 186 that personalized coaching could enable digital mental health interventions to better align with end users’ needs.
Time constraints are often mentioned as considerable barriers to engagement with digital mental health interventions 179 , 180 . For example, a study 187 provided the following end‐user perspective: “I assume a lot of people who are in my situation are in a crazy schedule… and not always have appointments booked for you is good”. People often report forgetting about the digital intervention or struggling to engage with it, particularly during periods of stress, indicating some challenges in integrating these tools into daily life 185 , 188 . End‐users are more likely to engage with digital mental health interventions when these are flexible and can be integrated into their daily routines 177 , 179 , 180 , 181 , 184 , 189 .
Addressing engagement through inclusion and trust
Issues about safety continue to be raised as critical factors influencing end‐user engagement with digital mental health interventions. Barriers include concerns over privacy 181 , 185 , unauthorized access 181 , 183 , data security and protection 181 , 183 , and lack of confidentiality 179 , 181 , 182 , 183 , 184 . For example, an older person reported 190 : “Websites being hacked, people's personal details being hacked, y'know it's nothing, nothing is safe. Nothing is secure – and I know that nothing on the web is 100% safe, it can't be”. Greater trust in digital tools can be fostered by providing secure ways to record information 181 , and assuring strong data protection measures 183 and clear communication of privacy settings 181 , 184 .
Recent efforts to address these engagement barriers have shown encouraging results. Using co‐design principles by gathering the target population's needs, preferences and feedback has generated mental health apps with higher ratings of usability, satisfaction and adherence 177 , 179 , 188 . Delivery of acceptance‐facilitating interventions – such as training or brief educational videos that provide trustworthy information about the role of digital interventions and address common concerns and misconceptions – has been shown to enhance motivation, positive attitudes and self‐efficacy, and reduce digital anxiety, security concerns, and skepticism 133 , 191 , 192 , which appears to translate into greater uptake and adherence 193 .
Addressing engagement through human support: from technical support to coaching
Technical issues – including bugs, usability and accessibility challenges – are major barriers hindering engagement 177 , 180 , 182 , 189 . One young person highlighted 194 : “So yeah, because I'm not a technical person at all… that's the only downside of it, if it doesn't work okay, then it has quite an impact”. Barriers to engagement include problems with Internet connection (“Because if Internet connection is not great and you click next then it takes a while for the next page to come up and you know it gets frustrating” 176 ) and digital mental health interventions not being accessible on a smartphone (“The pages weren't phone friendly – lots of scrolling left to right” 195 ). Rural participants face unique barriers in terms of limited Internet access and poor connectivity 177 , 184 .
Multiple reviews report that professional guidance – whether from therapists, coaches, counselors, or other health professionals – is crucial for user engagement and adherence 177 , 179 , 182 , 183 , 186 , 188 . End‐users consistently prefer digital mental health interventions that include professional support, finding these more engaging and safer than unguided or self‐guided interventions, which could be viewed as impersonal or distressing 177 , 183 , 184 , 185 , 189 , 196 . Some reviews report that end‐users prefer a digital mental health intervention as a complement to existing, in‐person therapy rather than a replacement 177 , 185 . For instance, a qualitative study with veterans highlighted that most participants who had used a digital intervention (PTSD Coach Australia) had done so as an adjunct to therapy, as it was “more helpful if you are seeing a psychologist or psychiatrist” 197 . Importantly, however, negative attitudes from health care providers could diminish end‐user engagement 181 .
Including human support in digital mental health interventions may be flexible. For example, therapists could be directly involved in facilitating interventions or providing reminders 177 . Some end‐users report satisfaction with instantaneous support through digital channels such as chat or email 179 , while others emphasize the value of structured interactions with coaches 186 . Infrequent or delayed responses from professionals are reported barriers to engagement 179 . Regular interactions and personalization of feedback from professionals during delivery of digital mental health interventions are found by end‐users to be essential for maintaining engagement and feeling supported 179 , 180 , 183 , 186 , 188 . This reinforces the potential of roles such as that of digital navigators 11 , 13 in improving engagement rates and therapeutic outcomes 198 .
Addressing engagement through just‐in‐time adaptive interventions
Just‐in‐time adaptive interventions are an innovative approach that leverages mobile devices to collect real‐time data from sensors or user input, allowing them to deliver brief, tailored “micro‐interventions” at precise moments when individuals are most in need or receptive to support 199 . For example, an intervention of this kind was designed to support smoking cessation by delivering brief mindfulness exercises when individuals reported increased negative affect or smoking behaviors 200 . By aligning support with the user's immediate needs, just‐in‐time adaptive interventions may enhance the effectiveness and engagement of treatment through increased personalization.
Although numerous trials have been conducted of just‐in‐time adaptive interventions for health conditions 201 , a significant research gap exists in the development and testing of these interventions for mental health problems. There has been some progress in mental health conditions where behavioral patterns are more discrete and measurable, such as eating disorders 202 , suicide prevention 203 , and addictive behaviors 204 . For example, a just‐in‐time adaptive intervention targeting opioid addiction in chronic pain prompted mindfulness exercises when stress was detected via a smartwatch 205 . A just‐in‐time adaptive intervention for youth depression and anxiety (Mello 206 ), targeting repetitive negative thinking (rumination and worry), showed moderate to large effects over six weeks in a pilot RCT. Another pilot RCT of a just‐in‐time adaptive intervention for depressive rumination in adults found that the intervention showed greater improvement in rumination relative to a control condition 207 . Finally, a pilot trial of a just‐in‐time adaptive intervention targeting sleep in veterans found that using the app in conjunction with clinical support improved sleep outcomes 208 .
While early results are promising, more development is needed alongside RCTs to thoroughly assess the efficacy of just‐in‐time adaptive interventions across a range of mental health conditions, and empirically determine whether they can increase engagement.
Addressing engagement through digital literacy
The effectiveness of digital mental health interventions has been closely tied to factors such as digital literacy, familiarity with technology, and availability of training 177 . Limited technological skills and low digital literacy among users are substantial barriers to effective usage of digital interventions 177 , 181 , 183 , 184 , 185 , 188 , and these issues are compounded when end‐users are confronted with technological barriers. Programs designed to teach digital literacy to people with serious mental illness have shown promising pilot results 209 , 210 and offer a tangible solution that should be expanded.
Positive beliefs about technology 177 , 188 and understanding its benefits 177 increase engagement, while low self‐efficacy concerning using technology poses a challenge 183 . Increased exposure to technology improved the comfort of young First Nations people and their families over time with using digital mental health interventions 182 . Training and support are essential in improving digital skills, confidence and overall engagement with digital interventions, particularly for users initially struggling with technology 181 , 184 .
Addressing engagement through social influence
Social influence is reported to play a critical role in end‐user engagement with digital mental health interventions. Positive influence from peers and family consistently emerges as a factor encouraging initial adoption and sustained engagement 177 , 185 , 188 . Family involvement increases over time with repeated use of technology 182 , and sharing tasks with family or friends supports end‐users’ regular practice 180 . However, the presence of family or caregivers may inhibit open discussion in older adults 183 and in community forums 179 .
Social connectedness‐related features in digital mental health interventions – such as peer support, community forums, and family involvement – are valued across studies and found to contribute to user retention 177 , 181 , 186 . However, these features might instead give rise to feelings of isolation and disengagement 179 . For example, one young person in a qualitative study commented 212 : “I felt even if I had something to say, I didn't feel comfortable saying it. I wasn't sure if I wrote something it'd make it worse, or I'm not sure how to feel about giving other people advice”.
CHALLENGES IN IMPLEMENTATION
Translating evidence‐based practices into real‐world use is an increasingly recognized challenge in mental health research 213 . Notoriously, fewer than 50% of clinical innovations are adopted in practice, and those that are adopted often take up to 17‐20 years to do so 214 . This challenge is particularly critical in digital mental health, where the perceived benefits of accessibility, reach and scalability are key drivers of interest, funding and innovation, but few examples of implementation success exist 215 , 216 . This underscores the need for systematic approaches to bridge the knowledge‐practice gap in digital mental health. Implementation science provides these approaches, employing a variety of key theories, models and frameworks 217 .
Barriers and facilitators in digital tool implementation
A growing body of research has identified barriers and facilitators at multiple levels in implementation of digital mental health interventions, aligning with the domains outlined in key frameworks such as the Consolidated Framework for Implementation Research 218 .
Practitioner level
Clinical staff of mental health services can play a critical role in translating digital mental health interventions into routine care. Factors influencing clinicians' motivation, capability and opportunity are key to successful implementation 219 , 220 , 221 .
Despite high interest in digital support generated by the COVID‐19 pandemic 1 , 220 , negative perceptions – such as concerns about the quality of digital interventions compared to face‐to‐face care, and privacy issues – remain significant barriers 181 . Moreover, poor digital literacy and lack of confidence in using digital tools 221 , 222 can impede clinician adoption, which is compounded by concerns about safety and risk management in digital spaces 223 , 224 . Further, concerns about the potential impact on the therapeutic relationship, with digital interventions often perceived as impersonal or “cold”, also act as barriers 223 , 224 , 225 , 226 .
Like many professionals with established competencies, clinicians can resist changes to practice, creating an additional obstacle 20 , 224 , 227 . This highlights the importance of training in the digital space, which has not kept pace with the rapid development of digital tools and their growing evidence base, especially around generative artificial intelligence and LLMs 228 . While training is among the most cited facilitators for digital implementation at the clinician level 181 , recent research reveals a lack of content concerning digital mental health interventions within clinical training programs 229 . Moreover, the high‐stress, high‐demand nature of many mental health services, and the high burden placed on clinicians, must be acknowledged, which may limit opportunities to learn, understand and integrate digital interventions 20 , 223 , 224 , further emphasizing the need for digital competency training to occur before clinicians are recruited in mental health services. The above factors also affect motivation, capability and opportunity at the leadership level, where a lack of active support or resources for digital implementation can create significant barriers within services 222 , 224 , 230 .
Practitioner‐level facilitators include training, co‐design and co‐production with clinicians 226 , and a belief in a digital future for mental health care 181 . Involving clinicians in the design process increases their buy‐in and ensures that digital tools enhance their work meaningfully. Motivated, innovation‐minded leaders who demonstrate, model and are accountable are also critical to driving implementation 20 . In addition, integrating digital mental health into curricula for trainee clinicians would build digital competencies earlier and help shift expectations around future clinical roles, addressing some of the barriers to digital adoption. Together, these strategies can address individual‐level barriers and create a workforce better equipped to embrace digital transformation in mental health care.
Service level
Several service level determinants relate to both the characteristics of the setting and the factors involved in implementation delivery. Significant barriers are the lack of alignment between digital mental health interventions and existing workflows, the compatibility between digital and face‐to‐face care 181 , and the perceived priority of digital interventions compared to critical clinical tasks, such as responding to emergencies 20 , 222 , 224 . Although interoperability and integration with existing technology systems are proposed as solutions, real‐world examples remain scarce, due to the high complexity and variability of technological systems used in care settings.
The value of co‐designing digital tools with service stakeholders is clear. In the same way that the field now recognizes the importance of user‐centered design and participatory research in the development and evaluation of digital mental health interventions, an early understanding of contextual factors is key to scalability models that include integration and implementation. The involvement of service stakeholders can better support a fit between technology and practice, and ensure that digital tools enhance and complement, rather than compete with, clinical tasks.
The availability and reliability of technology in mental health services – especially in remote or low‐resource areas – is a further barrier 222 . Moreover, staff shortage and personnel turnover also hinder implementation and sustainability, as those who have developed expertise and confidence in digital mental health interventions may be replaced by less experienced professionals 20 , 224 , underscoring the key role of training programs conducted before clinicians are recruited in services 229 .
Service setting implementation facilitators include adequate resourcing, infrastructure and staffing 181 , as well as collaboration and communication between team members or service staff 221 , 225 . New roles such as that of digital navigators, discussed above, may also help alleviate the staffing issues when clinicians need support to utilize digital mental health technologies in care optimally.
System level
System level barriers include policy, regulatory and financial issues. Regulatory and reimbursement frameworks remain an ongoing challenge. Unclear, restrictive and not fit‐for‐purpose regulations pose barriers to digital mental health implementation 223 , 226 . Conversely, regulations that mandate privacy and data protection can shape the digital health ecosystem, influencing confidence in digital mental health interventions among both clinicians and users.
The certification or endorsement of specific interventions can facilitate their implementation, alongside their integration in clinical guidelines 226 . In many Western countries, such as the US and Australia, regulatory agencies do not actively enforce regulatory or certification requirements for digital interventions that fall under the wellness or low‐risk sphere. However, the regulatory landscape is evolving rapidly. For example, the UK National Health Service and the Australian government have introduced frameworks such as the Digital Technology Assessment Criteria 230 and the National Safety and Quality Digital Mental Health Standards 231 , respectively, with significant implications for future government funding.
In Germany, the Digital Health Application (DiGA) system has linked regulatory approval with reimbursement 232 . The US FDA has recently issued several new guidances 233 , 234 for digital health technologies that are likely to preview future regulation and enforcement. Decisions are being made regarding whether the recently announced US Medicare billing codes to reimburse digital mental health interventions will stipulate that these interventions must be FDA‐cleared to qualify for reimbursement 235 . Thus, the regulatory and reimbursement space is dynamic, with frequent consultation and revision, indicating that, while progress is being made, keeping abreast of developments requires close monitoring of relevant policies and processes.
New developments
The above‐mentioned barriers and facilitators have focused on the implementation of digital mental health interventions into mental health care settings, reflecting the predominant focus of current research and scoping efforts. However, a nascent literature also exists exploring the integration of these interventions across schools and educational environments, workplaces, and community services 236 . Given the flexibility and adaptability of digital interventions to “meet people where they are”, it is crucial for implementation research to extend beyond traditional care settings.
Further, most progress in digital mental health implementation research has been in identifying and understanding barriers and facilitators. Less is known about which implementation strategies may facilitate real‐world use, and under what conditions 215 , 237 . A recent notable exception is the ImpleMentAll trial, which tested a tailored implementation toolkit for Internet‐based CBT (iCBT) against “implementation as usual” across Europe and Australia 216 . Results indicated that the toolkit had a small but statistically significant effect on the degree to which iCBT is considered a normal part of work within the context. While this study provides valuable evidence and resources for tailored implementation, detailed insights into how the tailored strategy differed from “implementation as usual” has yet to be published.
To support research on implementation strategies and outcomes in the digital space, methods of the traditional research pipeline should be replaced by methods that develop and test digital mental health interventions within the real‐world contexts in which they will be implemented or scaled 213 . One such design is the hybrid implementation‐effectiveness trial, which evaluates both effectiveness and implementation to varying degrees 238 . By adopting these and other novel methodologies and involving multidisciplinary teams – including key stakeholders and implementation scientists – the next generation of digital mental health interventions, particularly in expanding areas such as artificial intelligence and virtual reality, can have a more solid foundation for implementation and impact at scale.
DIGITAL MENTAL HEALTH FOR MINORITIES AND LOW‐RESOURCE CONTEXTS
The potential of digital mental health to increase access to care is often discussed around serving the unmet needs for care in historically marginalized communities, cultural minorities, and low‐resource settings. However, digital approaches could have unintended effects of exclusion without a concomitant focus on digital access and literacy. This section reviews how digital mental health is evolving to meet these important needs and ensure that no patients are left behind.
Historically marginalized communities and cultural minorities
Communities historically affected by discrimination, marginalization and stigma – e.g., racial and ethnic minorities; lesbian, gay, bisexual, transgender, queer/questioning (LGBTQ+) populations; and low‐income communities – experience disparities in mental health access and treatment. For instance, although these minorities tend to have a higher prevalence of common mental health problems, they are less likely to seek professional mental health treatment and more likely to prematurely drop out when they are in care, as well as to experience persistent symptoms 239 , 240 , 241 , 242 . Blacks, Asians and Hispanics/Latinos are also more likely to receive a diagnosis of severe mental disorder when they seek mental health services 242 . If implemented skillfully, the rise of digital mental health may help reduce such disparities by increasing access to care, reducing cost, removing the stigma associated with seeking mental health care in in‐person settings, and engaging underserved communities 243 , 244 .
Research on mental health apps not customized for these populations suggests that they have not fulfilled the promise of broadening access and utilization. A recent study on a freely available meditation app found that African Americans were much less likely to access and utilize the app 245 . As evidence from psychotherapy research consistently shows, culturally tailored interventions are more efficacious than non‐tailored ones 246 . Effectively tailored efforts that appeal to and engage people from underserved communities will be critical in digital mental health as well. A recent systematic review and meta‐analysis 247 found that culturally adapted digital mental health interventions for racial and ethnic minorities produced a large and significant effect across outcomes (g=0.90) compared to waitlist and treatment‐as‐usual conditions, although the average attrition was somewhat high (42%). A lack of research with Black and Indigenous populations was highlighted. Tailored digital mental health programs have also been piloted in sexual and gender minorities, a population heavily affected by discrimination, stigma, early life adversity, and more prevalent mental health concerns compared to their heterosexual counterparts 248 , 249 , 250 .
Improving access and engagement is one of the most important challenges for digital mental health program designers when tailoring such efforts to underserved communities. To this end, employing participatory designs, such as the community‐based participatory research method, may be fruitful in developing community‐specific, culturally relevant content and improving delivery design (e.g., frequency, dosage) and engagement strategies (e.g., reminders, peer interventions, personalized messages) 251 . This may be particularly pertinent to historically marginalized communities to overcome potential barriers such as mistrust of traditional health care systems and medical research, stigma in seeking and receiving mental health services, and literacy and language‐related issues.
The development and evaluation of digital mental health interventions for historically marginalized populations are still in their infancy, and much work is needed to understand the best approaches to digital mental health for subgroups of minority populations and outcomes. With the advancement and growth of technology, using artificial intelligence and machine learning in digital mental health with marginalized individuals and groups has been an area of both promise and caution, and requires deeper and purposeful research. For instance, a recent study evaluated a personalized artificial intelligence‐facilitated self‐referral chatbot in the UK, and found that increases in referrals were particularly pronounced among gender non‐binary and ethnic minority individuals, with the participants’ need for treatment as well as the chatbot's human‐free nature (thus reducing the likelihood of stigmatizing interactions with a provider) being potential contributors 252 .
Despite the promise of artificial intelligence and machine learning for personalized interventions, they could exacerbate health disparities by displaying biases against marginalized and underserved groups due to algorithms and predictions built on data that reflect social biases 253 . For example, diagnostic algorithms built on historical data might be more likely to suggest diagnoses of severe mental disorders for Black/African Americans despite displaying similar symptoms to their non‐Hispanic White peers. Validating computational models with minority populations, training artificial intelligence to overcome social biases, and exploring how artificial intelligence and machine learning may facilitate or hinder addressing factors that lead to mental health disparities, including structural inequalities, are important issues for this area in the future.
Low‐ and middle‐income countries and low‐resource settings
Globally, there are significant disparities in access to mental health care, and the World Health Organization (WHO) has called for action to address this inequality as a major global health challenge 254 . It is estimated that 76‐85% of severe mental health cases in low‐ and middle‐income countries (LMICs) do not receive treatment due to the scarcity of resources and trained professionals 255 . Today, people in LMICs increasingly own smartphones and want to use them towards health 256 . However, research in digital mental health remains relatively scarce. A systematic review conducted in 2020 found 22 controlled studies that employed mobile/Internet‐based psychological interventions in LMIC settings, with the majority conducted in Asia (59%) and focusing on adults with elevated depression, anxiety, PTSD or substance use symptoms 257 .
Another systematic review 258 uncovered 55 studies (including those with only qualitative reports) that conducted cultural adaptation of Internet‐ and mobile‐based mental health interventions in LMIC settings, or with migrants and Indigenous people in high‐income countries. The study provided a taxonomy of 17 components of cultural adaptation that range from content to methodological (e.g., functions, aesthetics) and procedural (e.g., who is involved, how information is gathered) aspects, and highlighted the complexity in situating and tailoring digital interventions in new cultural contexts. Further evidence from more diverse regions and populations is needed.
In addition to digital mental health tools in direct clinical roles, there is also great potential for digital interventions to address training needs, reduce the burden of care for providers and mental health care systems, and build local capacity in low‐resource LMIC settings. This may include engaging lay providers (i.e., digital navigators) as part of the prevention‐to‐treatment spectrum of care 259 . For example, to address the surge of suicide among adolescents in China 260 and the lack of resources in rural school systems in that country, a localized gatekeeper for teachers program was developed and delivered via digital training 261 . In India, the EMPOWER study 262 uses digital tools for training and supervising non‐specialist providers. A 2024 review of digital psychiatry in LMIC countries offers further examples 256 .
Another relevant area for digital mental health globally is its application for people and communities affected by the growing number of wars and conflicts 263 . To this end, digital mental health may offer scalable solutions to address accessibility issues and provide needed real‐time support. Examples of this work include a recent RCT of a WHO‐guided digital health intervention for depression (Step by Step) for Syrian refugees in Lebanon 264 , and an RCT with refugees in Germany 265 which found that a hybrid approach (combining digital treatment with in‐person intervention based on symptom severity) was sustainable and cost‐effective for depression. A recent review of digital mental health interventions for children and adolescents affected by war found that most interventions suffered from gaps, including that most programs were not culturally or linguistically adapted to their contexts 266 . Appropriate contextual tailoring often takes time and resources, and how to best adapt an evidence‐based digital health intervention for conflict‐affected communities in times of need remains a challenge. Evaluating and ensuring that such interventions are not only efficacious but also scalable is critical, given the large refugee and migrant populations in need of care.
Given the significant need for addressing mental health needs in LMIC settings and the limited resources, digital mental health efforts in these contexts should adopt a population health approach and expand beyond an individual patient focus. This may involve public education to raise awareness of mental health and reduce stigma, as well as engaging key people in communities, schools, and family and work settings 267 . Employing implementation science perspectives and engaging with policy makers early in the research process could also be beneficial to scaling up effective programs, increasing impact, and fostering translation from empirical evidence to practice 268 , 269 .
CONCLUSIONS
The digital mental health space is rapidly growing far beyond traditional telehealth visits. New tools such as LLMs have rapidly emerged, while relatively older ones such as smartphone apps and virtual reality have quickly expanded. While each tool has offered evidence of clinical impact, broad real‐world impact remains aloof for all. This paper has highlighted many of the factors involved and proposed actionable solutions. While it is impossible to summarize such a vast and evolving space neatly, two key points around the scientific nature of digital health research and real‐world engagement must be highlighted.
First, the vast majority of research reviewed in this paper focused on individual products, particular apps, unique virtual reality programs, and specific LLM models. This focus on digital health as a tool instead of the generalizable principles behind the tools has hindered scientific progress. Clinical research requires synergies, which remain limited in the digital mental health field because of a lack of common metrics and, in part, a lack of shared tools/softwares.
Second, results of our review underscore that the human connection supporting any of these technologies is critical for real‐world impact outside of research studies. The next generation of digital tools must be better co‐designed, personalized, and responsive to patient needs. These tools must also be studied, beyond efficacy trials, through methods prioritizing external validity and generalizability, such as hybrid research designs guided by implementation science frameworks. With this approach, new tools may better engage clinicians and achieve integration into complex clinical systems.
There are many pathways to improved clinical research and real‐world use of digital mental health tools. The extent to which future digital mental health interventions will be genuinely beneficial for people with mental health conditions will directly reflect the intersection of progress across those two domains.
ACKNOWLEDGEMENTS
S.B. Goldberg and S. Sun were partially supported by the US National Center for Complementary and Integrative Health (grants nos. K23AT010879, R24AT012845 and K23AT011173).
REFERENCES
- 1. Torous J, Bucci S, Bell IH et al. The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry 2021;20:318‐35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. McBain RK, Schuler MS, Qureshi N et al. Expansion of telehealth availability for mental health care after state‐level policy changes from 2019 to 2022. JAMA Netw Open 2023;6:e2318045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Yellowlees P, Nakagawa K, Pakyurek M et al. Rapid conversion of an outpatient psychiatric clinic to a 100% virtual telepsychiatry clinic in response to COVID‐19. Psychiatr Serv 2020;71:749‐52. [DOI] [PubMed] [Google Scholar]
- 4. Bartelt K, Piff A, Allen S et al. Telehealth utilization higher than pre‐pandemic levels, but down from pandemic highs. https://epicresearch.org.
- 5. Worthen A, Torous J, Khan S et al. Telepsychiatry current practice and implications for future trends: a 2023 American Psychiatric Association member survey. Telemed J E Health 2024;30:11. [DOI] [PubMed] [Google Scholar]
- 6. Chen PV, Helm A, Caloudas SG et al. Evidence of phone vs video‐conferencing for mental health treatments: a review of the literature. Curr Psychiatry Rep 2022;24:529‐39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Kinoshita S, Cortright K, Crawford A et al. Changes in telepsychiatry regulations during the COVID‐19 pandemic: 17 countries and regions' approaches to an evolving healthcare landscape. Psychol Med 2022;52:2606‐13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Belz FF, Vega Potler NJ, Johnson IN et al. Lessons from low‐ and middle‐income countries: alleviating the behavioral health workforce shortage in the United States. Psychiatr Serv 2024;75:699‐702. [DOI] [PubMed] [Google Scholar]
- 9. Goldberg SB, Sun S, Carlbring P et al. Selecting and describing control conditions in mobile health randomized controlled trials: a proposed typology. NPJ Digit Med 2023;6:181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Torous J, Firth J, Goldberg SB. Digital mental health's unstable dichotomy – wellness and health. JAMA Psychiatry 2024;81:539‐40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Chekroud AM, Hawrilenko M, Loho H et al. Illusory generalizability of clinical prediction models. Science 2024;383:164‐7. [DOI] [PubMed] [Google Scholar]
- 12. Drazen JM, Haug CJ. Trials of AI interventions must be preregistered. NEJM AI 2024;1:4. [Google Scholar]
- 13. Perret S, Alon N, Carpenter‐Song E et al. Standardising the role of a digital navigator in behavioural health: a systematic review. Lancet Digit Health 2023;5:e925‐32. [DOI] [PubMed] [Google Scholar]
- 14. Alvarez‐Jimenez M, Nicholas J, Valentine L et al. A national evaluation of a multi‐modal, blended, digital intervention integrated within Australian youth mental health services. Acta Psychiatr Scand 2025;151:317‐31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Titov N, Dear B, Nielssen O et al. ICBT in routine care: a descriptive analysis of successful clinics in five countries. Internet Interv 2018;13:108‐15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Mathiasen K, Riper H, Andersen TE et al. Guided internet‐based cognitive behavioral therapy for adult depression and anxiety in routine secondary care: observational study. J Med Internet Res 2018;20:e10927. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Hedman E, Ljótsson B, Kaldo V et al. Effectiveness of Internet‐based cognitive behaviour therapy for depression in routine psychiatric care. J Affect Disord 2014;155:49‐58. [DOI] [PubMed] [Google Scholar]
- 18. Nordgreen T, Gjestad R, Andersson G et al. The implementation of guided Internet‐based cognitive behaviour therapy for panic disorder in a routine‐care setting: effectiveness and implementation efforts. Cogn Behav Ther 2018;47:62‐75. [DOI] [PubMed] [Google Scholar]
- 19. Macrynikola N, Nguyen N, Lane E et al. The digital clinic: an innovative mental health care delivery model utilizing hybrid synchronous and asynchronous treatment. NEJM Catal Innov Care Deliv 2023;4:9. [Google Scholar]
- 20. Hadjistavropoulos HD, Nugent MM, Dirkse D et al. Implementation of internet‐delivered cognitive behavior therapy within community mental health clinics: a process evaluation using the consolidated framework for implementation research. BMC Psychiatry 2017;17:1‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Naderbagi A, Loblay V, Zahed IU et al. Cultural and contextual adaptation of digital health interventions: narrative review. J Med Internet Res 2024;26:e55130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Whitehead L, Talevski J, Fatehi F et al. Barriers to and facilitators of digital health among culturally and linguistically diverse populations: qualitative systematic review. J Med Internet Res 2023;25:e42719. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Torous J, Kiang MV, Lorme J et al. New tools for new research in psychiatry: a scalable and customizable platform to empower data driven smartphone research. JMIR Ment Health 2016;3:e5165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Beames JR, Han J, Shvetcov A et al. Use of smartphone sensor data in detecting and predicting depression and anxiety in young people (12‐25 years): a scoping review. Heliyon 2024;10:e35472. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Moura I, Teles A, Viana D et al. Digital phenotyping of mental health using multimodal sensing of multiple situations of interest: a systematic literature review. J Biomed Inform 2023;138:104278. [DOI] [PubMed] [Google Scholar]
- 26. Choi A, Ooi A, Lottridge D. Digital phenotyping for stress, anxiety, and mild depression: systematic literature review. JMIR mHealth uHealth 2024;12:e40689. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. dos Santos MP, Heckler WF, Bavaresco RS et al. Machine learning applied to digital phenotyping: a systematic literature review and taxonomy. Comput Human Behav 2024;161:108422. [Google Scholar]
- 28. Cohen A, Naslund JA, Chang S et al. Relapse prediction in schizophrenia with smartphone digital phenotyping during COVID‐19: a prospective, three‐site, two‐country, longitudinal study. Schizophrenia 2023;9:6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Cohen A, Naslund J, Lane E et al. Digital phenotyping data and anomaly detection methods to assess changes in mood and anxiety symptoms across a transdiagnostic clinical sample. Acta Psychiatr Scand 2025;151:388‐400. [DOI] [PubMed] [Google Scholar]
- 30. Currey D, Torous J. Digital phenotyping data to predict symptom improvement and app personalization: protocol for a prospective study. JMIR Res Protoc 2022;11:e37954. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Ortiz A, Mulsant BH. Beyond step count: are we ready to use digital phenotyping to make actionable individual predictions in psychiatry? J Med Internet Res 2024;26:e59826. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Brady LS, Larrauri CA, Steering Committee AMP SCZ. Accelerating Medicines Partnership® Schizophrenia (AMP® SCZ): developing tools to enable early intervention in the psychosis high risk state. World Psychiatry 2023;22:42‐3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Chen K, Huang JJ, Torous J. Hybrid care in mental health: a framework for understanding care, research, and future opportunities. NPP Digit Psychiatry Neurosci 2024;2:16. [Google Scholar]
- 34. Currey D, Torous J. Digital phenotyping correlations in larger mental health samples: analysis and replication. BJPsych Open 2022;8:e106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Pulsford RM, Brocklebank L, Fenton SA et al. The impact of selected methodological factors on data collection outcomes in observational studies of device‐measured physical behaviour in adults: a systematic review. Int J Behav Nutr Phys Act 2023;20:26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Bell IH, Pot‐Kolder R, Rizzo A et al. Advances in the use of virtual reality to treat mental health conditions. Nat Rev Psychol 2024;3:552‐67. [Google Scholar]
- 37. Kazdin AE. Addressing the treatment gap: a key challenge for extending evidence‐based psychosocial interventions. Behav Res Ther 2017;88:7‐18. [DOI] [PubMed] [Google Scholar]
- 38. Lindner P. Better, virtually: the past, present, and future of virtual reality cognitive behavior therapy. Int J Cogn Ther 2021;14:23‐46. [Google Scholar]
- 39. Schröder D, Wrona KJ, Müller F et al. Impact of virtual reality applications in the treatment of anxiety disorders: a systematic review and meta‐analysis of randomized‐controlled trials. J Behav Ther Exp Psychiatry 2023;81:101893. [DOI] [PubMed] [Google Scholar]
- 40. Wong KP, Lai CY, Qin J. Systematic review and meta‐analysis of randomised controlled trials for evaluating the effectiveness of virtual reality therapy for social anxiety disorder. J Affect Disord 2023;333:353‐64. [DOI] [PubMed] [Google Scholar]
- 41. Dellazizzo L, Potvin S, Luigi M et al. Evidence on virtual reality‐based therapies for psychiatric disorders: meta‐review of meta‐analyses. J Med Internet Res 2020;22:e20889. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Schroeder AH, Bogie BJ, Rahman TT et al. Feasibility and efficacy of virtual reality interventions to improve psychosocial functioning in psychosis: systematic review. JMIR Ment Health 2022;9:e28502. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. van Loenen I, Scholten W, Muntingh A et al. The effectiveness of virtual reality exposure‐based cognitive behavioral therapy for severe anxiety disorders, obsessive‐compulsive disorder, and posttraumatic stress disorder: meta‐analysis. J Med Internet Res 2022;24:e26736. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Novo A, Fonseca J, Barroso B et al. Virtual reality rehabilitation's impact on negative symptoms and psychosocial rehabilitation in schizophrenia spectrum disorder: a systematic review. Healthcare 2021;9:1429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Tan BL, Shi J, Yang S et al. The use of virtual reality and augmented reality in psychosocial rehabilitation for adults with neurodevelopmental disorders: a systematic review. Front Psychiatry 2022;13:1055204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Nijman SA, Pijnenborg GH, Vermeer RR et al. Dynamic interactive social cognition training in virtual reality (DiSCoVR) versus virtual reality relaxation (VRelax) for people with a psychotic disorder: a single‐blind multicenter randomized controlled trial. Schizophr Bull 2023;49:518‐30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Freeman D, Lister R, Waite F et al. Automated virtual reality cognitive therapy versus virtual reality mental relaxation therapy for the treatment of persistent persecutory delusions in patients with psychosis (THRIVE): a parallel‐group, single‐blind, randomised controlled trial in England with mediation analyses. Lancet Psychiatry 2023;10:836‐47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Riches S, Jeyarajaguru P, Taylor L et al. Virtual reality relaxation for people with mental health conditions: a systematic review. Soc Psychiatry Psychiatr Epidemiol 2023;58:989‐1007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Lamb R, Crowe A, Stone J et al. Virtual reality enhanced dialectical behavioural therapy. Br J Guid Couns 2023;51:491‐512. [Google Scholar]
- 50. Mishkind MC, Norr AM, Katz AC et al. Review of virtual reality treatment in psychiatry: evidence versus current diffusion and use. Curr Psychiatry Rep 2017;19:1‐8. [DOI] [PubMed] [Google Scholar]
- 51. Kuhn E, Saleem M, Klein T et al. Interdisciplinary perspectives on digital technologies for global mental health. PLoS Glob Public Health 2024;4:e0002867. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Parmar P, Ryu J, Pandya S et al. Health‐focused conversational agents in person‐centered care: a review of apps. NPJ Digit Med 2022;5:1‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Sharp G, Torous J, West ML. Ethical challenges in AI approaches to eating disorders. J Med Internet Res 2023;25:e50696. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54. Jiang G, Xu M, Zhu SC et al. Evaluating and inducing personality in pre‐trained language models. Adv Neural Inf Process Syst 2024;13:36. [Google Scholar]
- 55. Strachan JW, Albergo D, Borghini G et al. Testing theory of mind in large language models and humans. Nat Hum Behav 2024;8:1285‐95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Maples B, Cerit M, Vishwanath A et al. Loneliness and suicide mitigation for students using GPT3‐enabled chatbots. Npj Ment Health Res 2024;3:4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57. Llanes‐Jurado J, Gómez‐Zaragozá L, Minissi ME et al. Developing conversational virtual humans for social emotion elicitation based on large language models. Expert Syst Appl 2024;246:123261. [Google Scholar]
- 58. Mármol‐Romero AM, García‐Vega M, García‐Cumbreras MÁ et al. An empathic GPT‐based chatbot to talk about mental disorders with Spanish teenagers. Int J Hum Comput Interact 2024;1:1‐17. [Google Scholar]
- 59. Holderried F, Stegemann‐Philipps C, Herschbach L et al. A generative pretrained transformer (GPT)‐powered chatbot as a simulated patient to practice history taking: prospective, mixed methods study. JMIR Med Educ 2024;10:e53961. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60. Lee C, Mohebbi M, O'Callaghan E et al. Large language models versus expert clinicians in crisis prediction among telemental health patients: comparative study. JMIR Ment Health 2024;11:e58129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61. Perlis RH, Goldberg JF, Ostacher MJ et al. Clinical decision support for bipolar depression using large language models. Neuropsychopharmacology 2024;49:1412‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62. Lai T, Shi Y, Du Z et al. Supporting the demand on mental health services with AI‐based conversational large language models (LLMs). BioMedInformatics 2023;4:8‐33. [Google Scholar]
- 63. Galatzer‐Levy IR, McDuff D, Natarajan V et al. The capability of large language models to measure psychiatric functioning. arXiv 2023;2308.01834.
- 64. Spiegel BM, Liran O, Clark A et al. Feasibility of combining spatial computing and AI for mental health support in anxiety and depression. NPJ Digit Med 2024;7:22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65. Berrezueta‐Guzman S, Kandil M, Martín‐Ruiz ML et al. Future of ADHD care: evaluating the efficacy of ChatGPT in therapy enhancement. Healthcare 2024;12:683. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66. Lawrence HR, Schneider RA, Rubin SB et al. The opportunities and risks of large language models in mental health. JMIR Ment Health 2024;11:e59479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67. Vowels LM, Francois‐Walcott RR, Darwiche J. AI in relationship counselling: evaluating ChatGPT's therapeutic capabilities in providing relationship advice. Comput Hum Behav Artif Humans 2024;2:100078. [Google Scholar]
- 68. Grabb D. The impact of prompt engineering in large language model performance: a psychiatric example. J Med Artif Intell 2023;6:20. [Google Scholar]
- 69. Hua Y, Na H, Li Z et al. Applying and evaluating large language models in mental health care: a scoping review of human‐assessed generative tasks. arXiv 2024;2408.11288.
- 70. King M. Harmful biases in artificial intelligence. Lancet Psychiatry 2022;9:e48. [DOI] [PubMed] [Google Scholar]
- 71. Kosinski M. Evaluating large language models in theory of mind tasks. Proc Natl Acad Sci USA 2024;121:e2405460121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72. Hua Y, Xia W, Bates DW et al. Standardizing and scaffolding healthcare AI‐chatbot evaluation. medRxiv 2024;2024.07.21.24310774.
- 73. Baxter SL, Longhurst CA, Millen M et al. Generative artificial intelligence responses to patient messages in the electronic health record: early lessons learned. JAMIA Open 2024;7:ooae028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74. Blease C, Worthen A, Torous J. Psychiatrists’ experiences and opinions of generative artificial intelligence in mental healthcare: an online mixed methods survey. Psychiatry Res 2024;333:115724. [DOI] [PubMed] [Google Scholar]
- 75. Raile P. The usefulness of ChatGPT for psychotherapists and patients. Humanit Soc Sci Commun 2024;11:1‐8. [Google Scholar]
- 76. Harder N. Using ChatGPT in simulation design: what can (or should) it do for you? Clin Simul Nurs 2023;87:101487. [Google Scholar]
- 77. Maurya RK, Montesinos S, Bogomaz M et al. Assessing the use of ChatGPT as a psychoeducational tool for mental health practice. Couns Psychother Res 2024;25:e12759. [Google Scholar]
- 78. Ingram D. A mental health tech company ran an AI experiment on real users. Nothing's stopping apps from conducting more. NBC News, January 14, 2023. [Google Scholar]
- 79. Gelles‐Watnick R. Americans’ use of mobile technology and home broadband. Washington: Pew Research Center, 2024. [Google Scholar]
- 80. Torous J, Myrick K, Aguilera A. The need for a new generation of digital mental health tools to support more accessible, effective and equitable care. World Psychiatry 2023;22:1‐2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81. Camacho E, Cohen A, Torous J. Assessment of mental health services available through smartphone apps. JAMA Network Open 2022;5:e2248784. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82. Kuhn E, van der Meer C, Owen JE et al. PTSD Coach around the world. Mhealth 2018;4:15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83. Lopez‐Campos G, Gabarron E, Martin‐Sanchez F et al. Digital interventions and their unexpected outcomes – time for digitalovigilance? Stud Health Technol Inform 2024;310:479‐83. [DOI] [PubMed] [Google Scholar]
- 84. Taher R, Hall CL, Bergin AD et al. Developing a process for assessing the safety of a digital mental health intervention and gaining regulatory approval: a case study and academic's guide. Trials 2024;25:604. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85. Bird M, O'Neill E, Riches S. Digitally enhanced psychological assessment and treatment of paranoia: a systematic review. Clin Psychol Psychother 2024;31:e3019. [DOI] [PubMed] [Google Scholar]
- 86. Eisner E, Richardson C, Thomas N et al. Measurement of adverse events in studies of digital health interventions for psychosis: guidance and recommendations based on a literature search and framework analysis of standard operating procedures. Schizophr Bull 2024;50:1456‐70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 87. Allan S, Ward T, Eisner E et al. Adverse events reporting in digital interventions evaluations for psychosis: a systematic literature search and individual level content analysis of adverse event reports. Schizophr Bull 2024;50:1436‐55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 88. Rozental A, Castonguay L, Dimidjian S et al. Negative effects in psychotherapy: commentary and recommendations for future research and clinical practice. BJPsych Open 2018;4:307‐12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 89. Linden M, Schermuly‐Haupt ML. Definition, assessment and rate of psychotherapy side effects. World Psychiatry 2014;13:306‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 90. Fernández‐Álvarez J, Rozental A, Carlbring P et al. Deterioration rates in virtual reality therapy: an individual patient data level meta‐analysis. J Anxiety Disord 2019;61:3‐17. [DOI] [PubMed] [Google Scholar]
- 91. Huckvale K, Nicholas J, Torous J et al. Smartphone apps for the treatment of mental health conditions: status and considerations. Curr Opin Psychol 2020;36:65‐70. [DOI] [PubMed] [Google Scholar]
- 92. Hensler I, Sveen J, Cernvall M et al. Efficacy, benefits, and harms of a self‐management app in a Swedish trauma‐exposed community sample (PTSD Coach): randomized controlled trial. J Med Internet Res 2022;24:e31419. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 93. Kerber A, Beintner I, Burchert S et al. Effects of a self‐guided transdiagnostic smartphone app on patient empowerment and mental health: randomized controlled trial. JMIR Ment Health 2023;10:e45068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 94. Araya R, Menezes PR, Claro HG et al. Effect of a digital intervention on depressive symptoms in patients with comorbid hypertension or diabetes in Brazil and Peru: two randomized clinical trials. JAMA 2021;325:1852‐62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 95. Dahne J, Wahlquist AE, Kustanowitz J et al. Behavioral activation‐based digital smoking cessation intervention for individuals with depressive symptoms: randomized clinical trial. J Med Internet Res 2023;25:e49809. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 96. Foulkes L, Andrews JL, Reardon T et al. Research recommendations for assessing potential harm from universal school‐based mental health interventions. Nat Mental Health 2024;2:270‐7. [Google Scholar]
- 97. Moitra M, Owens S, Hailemariam M et al. Global mental health: where we are and where we are going. Curr Psychiatry Rep 2023;25:301‐11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 98. Eisenstadt M, Liverpool S, Infanti E et al. Mobile apps that promote emotion regulation, positive mental health, and well‐being in the general population: systematic review and meta‐analysis. JMIR Ment Health 2021;8:e31170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 99. ORCHA . Digital & mental health recovery action plans. https://orchahealth.com.
- 100. Kahane K, François J, Torous J. The digital health app policy landscape: regulatory gaps and choices through the lens of mental health. J Ment Health Policy Econ 2021;24:101‐8. [PubMed] [Google Scholar]
- 101. Al‐Refae M, Al‐Refae A, Munroe M et al. A self‐compassion and mindfulness‐based cognitive mobile intervention (Serene) for depression, anxiety, and stress: promoting adaptive emotional regulation and wisdom. Front Psychol 2021;12:648087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 102. Goldberg SB, Imhoff‐Smith T, Bolt DM et al. Testing the efficacy of a multicomponent, self‐guided, smartphone‐based meditation app: three‐armed randomized controlled trial. JMIR Ment Health 2020;7:e23825. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 103. Levin ME, Haeger J, Cruz RA. Tailoring acceptance and commitment therapy skill coaching in the moment through smartphones: results from a randomized controlled trial. Mindfulness 2019;10:689‐99. [Google Scholar]
- 104. Gnanapragasam SN, Tinch‐Taylor R, Scott HR et al. Multicentre, England‐wide randomised controlled trial of the ‘Foundations’ smartphone application in improving mental health and well‐being in a healthcare worker population. Br J Psychiatry 2023;222:58‐66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 105. Børøsund E, Ehlers SL, Varsi C et al. Results from a randomized controlled trial testing StressProffen; an application‐based stress‐management intervention for cancer survivors. Cancer Med 2020;9:3775‐85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 106. Bruhns A, Lüdtke T, Moritz S et al. A mobile‐based intervention to increase self‐esteem in students with depressive symptoms: randomized controlled trial. JMIR mHealth uHealth 2021;9:e26498. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 107. Gál É, ștefan S, Cristea IA. The efficacy of mindfulness meditation apps in enhancing users’ well‐being and mental health related outcomes: a meta‐analysis of randomized controlled trials. J Affect Disord 2021;279:131‐42. [DOI] [PubMed] [Google Scholar]
- 108. Chen B, Yang T, Xiao L et al. Effects of mobile mindfulness meditation on the mental health of university students: systematic review and meta‐analysis. J Med Internet Res 2023;25:e39128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 109. Linardon J. Can acceptance, mindfulness, and self‐compassion be learned by smartphone apps? A systematic and meta‐analytic review of randomized controlled trials. Behav Ther 2020;51:646‐58. [DOI] [PubMed] [Google Scholar]
- 110. Lau N, O'Daffer A, Colt S et al. Android and iPhone mobile apps for psychosocial wellness and stress management: systematic search in app stores and literature review. JMIR mHealth uHealth 2020;8:e17798. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 111. Hirshberg MJ, Dahl CJ, Bolt D et al. Psychological mediators of reduced distress: preregistered analyses from a randomized controlled trial of a smartphone‐based well‐being training. Clin Psychol Sci 2025;13:146‐59. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 112. Santomauro DF, Herrera AM, Shadid J et al. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID‐19 pandemic. Lancet 2021;398:1700‐12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 113. Gulliver A, Griffiths KM, Christensen H et al. A systematic review of help‐seeking interventions for depression, anxiety and general psychological distress. BMC Psychiatry 2012;12:1‐2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 114. Linardon J, Torous J, Firth J et al. Current evidence on the efficacy of mental health smartphone apps for symptoms of depression and anxiety. A meta‐analysis of 176 randomized controlled trials. World Psychiatry 2024;23:139‐49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115. Wasil AR, Gillespie S, Patel R et al. Reassessing evidence‐based content in popular smartphone apps for depression and anxiety: developing and applying user‐adjusted analyses. J Consult Clin Psychol 2020;88:983‐93. [DOI] [PubMed] [Google Scholar]
- 116. Linardon J, Messer M, Goldberg SB et al. The efficacy of mindfulness apps on symptoms of depression and anxiety: an updated meta‐analysis of randomized controlled trials. Clin Psychol Rev 2023;107:102370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 117. Serrano‐Ripoll MJ, Zamanillo‐Campos R, Fiol‐DeRoque MA et al. Impact of smartphone app‐based psychological interventions for reducing depressive symptoms in people with depression: systematic literature review and meta‐analysis of randomized controlled trials. JMIR mHealth uHealth 2022;10:e29621. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 118. Tsai Z, Kiss A, Nadeem S et al. Evaluating the effectiveness and quality of mobile applications for perinatal depression and anxiety: a systematic review and meta‐analysis. J Affect Disord 2022;296:443‐53. [DOI] [PubMed] [Google Scholar]
- 119. Deady M, Glozier N, Calvo R et al. Preventing depression using a smartphone app: a randomized controlled trial. Psychol Med 2022;52:457‐66. [DOI] [PubMed] [Google Scholar]
- 120. Sakata M, Toyomoto R, Yoshida K et al. Components of smartphone cognitive‐behavioural therapy for subthreshold depression among 1093 university students: a factorial trial. BMJ Ment Health 2022;25:e18‐25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 121. Furukawa TA, Tajika A, Sakata M et al. Four 2×2 factorial trials of smartphone CBT to reduce subthreshold depression and to prevent new depressive episodes among adults in the community – RESiLIENT trial (Resilience Enhancement with Smartphone in LIving ENvironmenTs): a master protocol. BMJ Open 2023;13:e067850. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 122. Linardon J, Cuijpers P, Carlbring P et al. The efficacy of app‐supported smartphone interventions for mental health problems: a meta‐analysis of randomized controlled trials. World Psychiatry 2019;18:325‐36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 123. He Y, Yang L, Qian C et al. Conversational agent interventions for mental health problems: systematic review and meta‐analysis of randomized controlled trials. J Med Internet Res 2023;25:e43862. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 124. Fuhrmann LM, Weisel KK, Harrer M et al. Additive effects of adjunctive app‐based interventions for mental disorders – A systematic review and meta‐analysis of randomised controlled trials. Internet Interv 2023;35:100703. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 125. Anmella G, Faurholt‐Jepsen M, Hidalgo‐Mazzei D et al. Smartphone‐based interventions in bipolar disorder: systematic review and meta‐analyses of efficacy. A position paper from the International Society for Bipolar Disorders (ISBD) Big Data Task Force. Bipolar Disord 2022;24:580‐614. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 126. Faurholt‐Jepsen M, Frost M, Christensen EM et al. The effect of smartphone‐based monitoring on illness activity in bipolar disorder: the MONARCA II randomized controlled single‐blinded trial. Psychol Med 2020;50:838‐48. [DOI] [PubMed] [Google Scholar]
- 127. Faurholt‐Jepsen M, Lindbjerg Tønning M, Fros M et al. Reducing the rate of psychiatric re‐admissions in bipolar disorder using smartphones – The RADMIS trial. Acta Psychiatr Scand 2021;143:453‐65. [DOI] [PubMed] [Google Scholar]
- 128. Goulding EH, Dopke CA, Rossom R et al. Effects of a smartphone‐based self‐management intervention for individuals with bipolar disorder on relapse, symptom burden, and quality of life: a randomized clinical trial. JAMA Psychiatry 2023;80:109‐18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 129. Faurholt‐Jepsen M, Vinberg M, Frost M et al. Smartphone data as an electronic biomarker of illness activity in bipolar disorder. Bipolar Disord 2015;17:715‐28. [DOI] [PubMed] [Google Scholar]
- 130. Morton E, Torous J, Murray G et al. Using apps for bipolar disorder – An online survey of healthcare provider perspectives and practices. J Psychiatr Res 2021;137:22‐8. [DOI] [PubMed] [Google Scholar]
- 131. Kerst A, Zielasek J, Gaebel W. Smartphone applications for depression: a systematic literature review and a survey of health care professionals’ attitudes towards their use in clinical practice. Eur Arch Psychiatry Clin Neurosci 2020;270:139‐52. [DOI] [PubMed] [Google Scholar]
- 132. Armstrong CM, Ciulla RP, Edwards‐Stewart A et al. Best practices of mobile health in clinical care: the development and evaluation of a competency‐based provider training program. Prof Psychol Res Pract 2018;49:355‐63. [Google Scholar]
- 133. Linardon J, Anderson C, Chapneviss T et al. Effects of an acceptance‐facilitating intervention on acceptance and usage of digital interventions for binge eating. Psychiatr Serv 2022;73:1173‐6. [DOI] [PubMed] [Google Scholar]
- 134. Baumeister H, Terhorst Y, Grässle C et al. Impact of an acceptance facilitating intervention on psychotherapists’ acceptance of blended therapy. PLoS One 2020;15:e0236995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 135. Firth J, Torous J. Smartphone apps for schizophrenia: a systematic review. JMIR mHealth uHealth 2015;3:e102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 136. Firth J, Cotter J, Torous J et al. Mobile phone ownership and endorsement of “mHealth” among people with psychosis: a meta‐analysis of cross‐sectional studies. Schizophr Bull 2016;42:448‐55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 137. Gumley AI, Bradstreet S, Ainsworth J et al. The EMPOWER blended digital intervention for relapse prevention in schizophrenia: a feasibility cluster randomised controlled trial in Scotland and Australia. Lancet Psychiatry 2022;9:477‐86. [DOI] [PubMed] [Google Scholar]
- 138. Lewis S, Ainsworth J, Sanders C et al. Smartphone‐enhanced symptom management in psychosis: open, randomized controlled trial. J Med Internet Res 2020;22:e17019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 139. Chen HH, Hsu HT, Lin PC et al. Efficacy of a smartphone app in enhancing medication adherence and accuracy in individuals with schizophrenia during the COVID‐19 pandemic: randomized controlled trial. JMIR Ment Health 2023;10:e50806. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 140. Krzystanek M, Krysta K, Borkowski M et al. The effect of smartphone‐based cognitive training on the functional/cognitive markers of schizophrenia: a one‐year randomized study. J Clin Med 2020;9:3681. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 141. Schlosser DA, Campellone TR, Truong B et al. Efficacy of PRIME, a mobile app intervention designed to improve motivation in young people with schizophrenia. Schizophr Bull 2018;44:1010‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 142. Ghaemi SN, Sverdlov O, van Dam J et al. A smartphone‐based intervention as an adjunct to standard‐of‐care treatment for schizophrenia: randomized controlled trial. JMIR Form Res 2022;6:e29154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 143. Jongeneel A, Delespaul P, Tromp N et al. Effects on voice hearing distress and social functioning of unguided application of a smartphone app – A randomized controlled trial. Internet Interv 2024;35:100717. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 144. Bucci S, Berry N, Ainsworth J et al. Effects of Actissist, a digital health intervention for early psychosis: a randomized clinical trial. Psychiatry Res 2024;339:116025. [DOI] [PubMed] [Google Scholar]
- 145. Garety P, Ward T, Emsley R et al. Effects of SlowMo, a blended digital therapy targeting reasoning, on paranoia among people with psychosis: a randomized clinical trial. JAMA Psychiatry 2021;78:714‐25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 146. Alvarez‐Jimenez M, Koval P, Schmaal L et al. The Horyzons project: a randomized controlled trial of a novel online social therapy to maintain treatment effects from specialist first‐episode psychosis services. World Psychiatry 2021;20:233‐43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 147. Arnautovska U, Trott M, Vitangcol KJ et al. Efficacy of user self‐led and human‐supported digital health interventions for people with schizophrenia: a systematic review and meta‐analysis. Schizophr Bull 2024; doi: 10.1093/schbul/sbae143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 148. Arnautovska U, Milton A, Trott M et al. The role of human involvement and support in digital mental health interventions for people with schizophrenia spectrum disorders: a critical review. Curr Opin Psychiatry 2024;37:356‐62. [DOI] [PubMed] [Google Scholar]
- 149. Wasserman D. Mental health for all: fostering healthy lifestyles. World Psychiatry 2023;22:343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 150. Sawyer C, McKeon G, Hassan L et al. Digital health behaviour change interventions in severe mental illness: a systematic review. Psychol Med 2023;53:6965‐7005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 151. Kwon S, Firth J, Joshi D et al. Accessibility and availability of smartphone apps for schizophrenia. Schizophrenia 2022;8:98. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 152. Solmi M, Monaco F, Højlund M et al. Outcomes in people with eating disorders: a transdiagnostic and disorder‐specific systematic review, meta‐analysis and multivariable meta‐regression analysis. World Psychiatry 2024;23:124‐38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 153. Linardon J, Messer M, Lee S et al. Perspectives of e‐health interventions for treating and preventing eating disorders: descriptive study of perceived advantages and barriers, help‐seeking intentions, and preferred functionality. Eat Weight Disord 2021;26:1097‐109. [DOI] [PubMed] [Google Scholar]
- 154. O'Leary T, Torous J. Smartphone apps for eating disorders: an overview of the marketplace and research trends. Int J Eating Disord 2022;55:625‐32. [DOI] [PubMed] [Google Scholar]
- 155. Linardon J, Shatte A, McClure Z et al. A broad v. focused digital intervention for recurrent binge eating: a randomized controlled non‐inferiority trial. Psychol Med 2023;53:4580‐91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 156. Linardon J, Messer M, Shatte A et al. Does the method of content delivery matter? Randomized controlled comparison of an internet‐based intervention for eating disorder symptoms with and without interactive functionality. Behav Ther 2022;53:508‐20. [DOI] [PubMed] [Google Scholar]
- 157. Juarascio A, Srivastava P, Presseller E et al. A clinician‐controlled just‐in‐time adaptive intervention system (CBT+) designed to promote acquisition and utilization of cognitive behavioral therapy skills in bulimia nervosa: development and preliminary evaluation study. JMIR Form Res 2021;5:e18261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 158. Juarascio AS, Presseller EK, Srivastava P et al. A randomized controlled trial of CBT+: a clinician‐controlled, just‐in‐time, adjunctive intervention for bulimia‐spectrum disorders. Behav Modif 2023;47:551‐72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 159. Juarascio AS, Presseller EK, Trainor C et al. Optimizing digital health technologies to improve therapeutic skill use and acquisition alongside enhanced cognitive‐behavior therapy for binge‐spectrum eating disorders: protocol for a randomized controlled trial. Int J Eating Disord 2023;56:470‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 160. Mojtabai R, Olfson M, Mechanic D. Perceived need and help‐seeking in adults with mood, anxiety, or substance use disorders. Arch Gen Psychiatry 2002;59:77‐84. [DOI] [PubMed] [Google Scholar]
- 161. Bradizza CM, Stasiewicz PR, Paas ND. Relapse to alcohol and drug use among individuals diagnosed with co‐occurring mental health and substance use disorders: a review. Clin Psychol Rev 2006;26:162‐78. [DOI] [PubMed] [Google Scholar]
- 162. Hsu M, Martin B, Ahmed S et al. Smartphone ownership, smartphone utilization, and interest in using mental health apps to address substance use disorders: literature review and cross‐sectional survey study across two sites. JMIR Form Res 2022;6:e38684. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 163. Institute for Clinical and Economic Review. Opioid epidemic: digital health technologies. https://icer.org/assessment/opioids‐digital‐apps‐2020.
- 164. Guo YQ, Chen Y, Dabbs AD et al. The effectiveness of smartphone app‐based interventions for assisting smoking cessation: systematic review and meta‐analysis. J Med Internet Res 2023;25:e43242. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 165. Carreiro S, Newcomb M, Leach R et al. Current reporting of usability and impact of mHealth interventions for substance use disorder: a systematic review. Drug Alcohol Depend 2020;215:108201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 166. Colbert S, Thornton L, Richmond R. Smartphone apps for managing alcohol consumption: a literature review. Addict Sci Clin Pract 2020;15:1‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 167. Perski O, Hébert ET, Naughton F et al. Technology‐mediated just‐in‐time adaptive interventions (JITAIs) to reduce harmful substance use: a systematic review. Addiction 2022;117:1220‐41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 168. Nwosu A, Boardman S, Husain MM et al. Digital therapeutics for mental health: is attrition the Achilles heel? Front Psychiatry 2022;13:900615. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 169. Torous J, Nicholas J, Larsen ME et al. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. BMJ Ment Health 2018;21:116‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 170. Arean PA, Hallgren KA, Jordan JT et al. The use and effectiveness of mobile apps for depression: results from a fully remote clinical trial. J Med Internet Res 2016;18:e330. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 171. Flett JA, Hayne H, Riordan BC et al. Mobile mindfulness meditation: a randomised controlled trial of the effect of two popular apps on mental health. Mindfulness 2019;10:863‐76. [Google Scholar]
- 172. Bostock S, Crosswell AD, Prather AA et al. Mindfulness on‐the‐go: effects of a mindfulness meditation app on work stress and well‐being. J Occup Health Psychol 2019;24:127‐38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 173. Linardon J, Fuller‐Tyszkiewicz M. Attrition and adherence in smartphone‐delivered interventions for mental health problems: a systematic and meta‐analytic review. J Consult Clin Psychol 2020;88:1‐13. [DOI] [PubMed] [Google Scholar]
- 174. Baumel A, Muench F, Edan S et al. Objective user engagement with mental health apps: systematic search and panel‐based usage analysis. J Med Internet Res 2019;21:e14567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 175. Deady M, Collins DA, Glozier N et al. Naturalistic evaluation of HeadGear: a smartphone app to reduce depressive symptoms in workers. Behav Ther 2024; doi: 10.1016/j.beth.2024.01.001. [DOI] [PubMed] [Google Scholar]
- 176. Su L, Anderson PL. User behavior of a publicly available, free‐to‐use, self‐guided mHealth app for depression: observational study in a global sample. JMIR Form Res 2022;6:e35538. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 177. Borghouts J, Eikey E, Mark G et al. Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. J Med Internet Res 2021;23:e24387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 178. Melcher J, Camacho E, Lagan S et al. College student engagement with mental health apps: analysis of barriers to sustained use. J Am Coll Health 2022;70:1819‐25. [DOI] [PubMed] [Google Scholar]
- 179. Ho TQ, Le LK, Engel L et al. Barriers to and facilitators of user engagement with web‐based mental health interventions in young people: a systematic review. Eur Child Adolesc Psychiatry 2024;14:1‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 180. Osborne EL, Ainsworth B, Hooper N et al. Experiences of using digital mindfulness‐based interventions: rapid scoping review and thematic synthesis. J Med Internet Res 2023;25:e44220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 181. Berardi C, Antonini M, Jordan Z et al. Barriers and facilitators to the implementation of digital technologies in mental health systems: a qualitative systematic review to inform a policy framework. BMC Health Serv Res 2024;24:243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 182. Toombs E, Kowatch KR, Dalicandro L et al. A systematic review of electronic mental health interventions for Indigenous youth: results and recommendations. J Telemed Telecare 2021;27:539‐52. [DOI] [PubMed] [Google Scholar]
- 183. Peng R, Li X, Guo Y et al. Barriers and facilitators to acceptance and implementation of eMental‐health intervention among older adults: a qualitative systematic review. Digit Health 2024;10:20552076241234628. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 184. Dennard S, Patel R, Garety P et al. A systematic review of users experiences of using digital interventions within psychosis: a thematic synthesis of qualitative research. Soc Psychiatry Psychiatr Epidemiol 2025;60:275‐303. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 185. Corthésy‐Blondin L, Lemyre A, Poitras M et al. Mobile applications for individuals affected by a traumatic event: a systematic review of qualitative findings. Techn Mind Behav 2023;4:3. [Google Scholar]
- 186. Bernstein EE, Weingarden H, Wolfe EC et al. Human support in app‐based cognitive behavioral therapies for emotional disorders: scoping review. J Med Internet Res 2022;24:e33307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 187. Eccles H, Nannarone M, Lashewicz B et al. Perceived effectiveness and motivations for the use of web‐based mental health programs: qualitative study. J Med Internet Res 2020;22:e16961. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 188. Bear HA, Ayala Nunes L, DeJesus J et al. Determination of markers of successful implementation of mental health apps for young people: systematic review. J Med Internet Res 2022;24:e40347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 189. Drews‐Windeck E, Greenwood K, Cavanagh K. A systematic review and meta‐analysis of digital interventions targeted at individuals with borderline personality disorder (BPD), emotionally unstable personality disorder (EUPD), and related symptoms. J Clin Psychol 2023;79:2155‐85. [DOI] [PubMed] [Google Scholar]
- 190. Pywell J, Vijaykumar S, Dodd A et al. Barriers to older adults’ uptake of mobile‐based mental health interventions. Digit Health 2020;6:2055207620905422. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 191. Kopka M, Camacho E, Kwon S et al. Exploring how informed mental health app selection may impact user engagement and satisfaction. PLoS Digit Health 2023;2:e0000219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 192. Ebert DD, Berking M, Cuijpers P et al. Increasing the acceptance of internet‐based mental health interventions in primary care patients with depressive symptoms. A randomized controlled trial. J Affect Disord 2015;176:9‐17. [DOI] [PubMed] [Google Scholar]
- 193. Bosbach K, Schoenenberg K, Martin A. Development and evaluation of an acceptance‐facilitating intervention for an internet‐based cognitive behavioral self‐esteem training for adults with body dysmorphic symptoms. J Obs‐Compuls Relat Disord 2023;37:100798. [Google Scholar]
- 194. Taylor KM, Bradley J, Cella M. A novel smartphone‐based intervention targeting sleep difficulties in individuals experiencing psychosis: a feasibility and acceptability evaluation. Psychol Psychother: Theory Res Pract 2022;95:717‐37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 195. Ashford MT, Olander EK, Rowe H et al. Feasibility and acceptability of a web‐based treatment with telephone support for postpartum women with anxiety: randomized controlled trial. JMIR Ment Health 2018;5:e9106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 196. Liu M, Schueller SM. Integrating digital therapeutics with mental healthcare delivery. J Health Serv Psychol 2024;50:77‐85. [Google Scholar]
- 197. Shakespeare‐Finch J, Alichniewicz KK, Strodl E et al. Experiences of serving and ex‐serving members with the PTSD Coach Australia app: mixed methods study. J Med Internet Res 2020;22:e18447. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 198. Camacho E, Chang SM, Currey D et al. The impact of guided versus supportive coaching on mental health app engagement and clinical outcomes. Health Informatics J 2023;29:14604582231215872. [DOI] [PubMed] [Google Scholar]
- 199. Nahum‐Shani I, Smith SN, Spring BJ et al. Just‐in‐time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med 2018;8:1‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 200. Yang MJ, Sutton SK, Hernandez LM et al. A just‐in‐time adaptive intervention (JITAI) for smoking cessation: feasibility and acceptability findings. Addict Behav 2023;136:107467. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 201. Hardeman W, Houghton J, Lane K et al. A systematic review of just‐in‐time adaptive interventions (JITAIs) to promote physical activity. Int J Behav Nutr Phys Act 2019;16:1‐21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 202. Juarascio AS, Parker MN, Lagacey MA et al. Just‐in‐time adaptive interventions: a novel approach for enhancing skill utilization and acquisition in cognitive behavioral therapy for eating disorders. Int J Eat Disord 2018;51:826‐30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 203. Coppersmith DD, Dempsey W, Kleiman EM et al. Just‐in‐time adaptive interventions for suicide prevention: promise, challenges, and future directions. Psychiatry 2022;85:317‐33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 204. Goldstein SP, Evans BC, Flack D et al. Return of the JITAI: applying a just‐in‐time adaptive intervention framework to the development of m‐health solutions for addictive behaviors. Int J Behav Med 2017;24:673‐82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 205. Garland EL, Gullapalli BT, Prince KC et al. Zoom‐based mindfulness‐oriented recovery enhancement plus just‐in‐time mindfulness practice triggered by wearable sensors for opioid craving and chronic pain. Mindfulness 2023;14:1329‐45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 206. Bell I, Arnold C, Gilbertson T et al. A personalized, transdiagnostic smartphone intervention (Mello) targeting repetitive negative thinking in young people with depression and anxiety: pilot randomized controlled trial. J Med Internet Res 2023;25:e47860. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 207. Wang L, Miller L. Assessment and disruption of ruminative episodes to enhance mobile cognitive behavioral therapy just‐in‐time adaptive interventions in clinical depression: pilot randomized controlled trial. JMIR Form Res 2023;7:e37270. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 208. Pulantara IW, Parmanto B, Germain A. Clinical feasibility of a just‐in‐time adaptive intervention app (iREST) as a behavioral sleep treatment in a military population: feasibility comparative effectiveness study. J Med Internet Res 2018;20:e10124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 209. Camacho E, Torous J. Impact of digital literacy training on outcomes for people with serious mental illness in community and inpatient settings. Psychiatr Serv 2023;74:534‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 210. Alon N, Perret S, Cohen A et al. Digital navigator training to increase access to mental health care in community‐based organizations. Psychiatr Serv 2024;75:608‐11. [DOI] [PubMed] [Google Scholar]
- 211. Austin SF, Frøsig A, Buus N et al. Service user experiences of integrating a mobile solution (IMPACHS) into clinical treatment for psychosis. Qual Health Res 2021;31:942‐54. [DOI] [PubMed] [Google Scholar]
- 212. Bilello D, Townsend E, Broome MR et al. Friendships and peer relationships and self‐harm ideation and behaviour among young people: a systematic review and narrative synthesis. Lancet Psychiatry 2024;11:633‐57. [DOI] [PubMed] [Google Scholar]
- 213. McGinty EE, Alegria M, Beidas RS et al. The Lancet Psychiatry Commission: transforming mental health implementation research. Lancet Psychiatry 2024;11:368‐96. [DOI] [PubMed] [Google Scholar]
- 214. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 2011;104:510‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 215. Graham AK, Lattie EG, Powell BJ et al. Implementation strategies for digital mental health interventions in health care settings. Am Psychol 2020;75:1080‐92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 216. Vis C, Schuurmans J, Aouizerate B et al. Effectiveness of self‐guided tailored implementation strategies in integrating and embedding internet‐based cognitive behavioral therapy in routine mental health care: results of a multicenter stepped‐wedge cluster randomized trial. J Med Internet Res 2023;25:e41532. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 217. Nilsen P. Making sense of implementation theories, models, and frameworks. Implement Sci 2020;10:53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 218. Damschroder LJ, Reardon CM, Widerquist MA et al. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci 2022;17:75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 219. Michie S, Atkins L, West R. The behaviour change wheel. A guide to designing interventions. Sutton: Silverback, 2014. [Google Scholar]
- 220. Bell IH, Thompson A, Valentine L et al. Ownership, use of, and interest in digital mental health technologies among clinicians and young people across a spectrum of clinical care needs: cross‐sectional survey. JMIR Ment Health 2022;9:e30716. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 221. LaMonica HM, Milton A, Braunstein K et al. Technology‐enabled solutions for Australian mental health services reform: impact evaluation. JMIR Form Res 2020;4:e18759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 222. Orlowski S, Lawn S, Matthews B et al. The promise and the reality: a mental health workforce perspective on technology‐enhanced youth mental health service delivery. BMC Health Serv Res 2016;16:1‐2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 223. Lattie EG, Nicholas J, Knapp AA et al. Opportunities for and tensions surrounding the use of technology‐enabled mental health services in community mental health care. Adm Policy Ment Health 2020;47:138‐49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 224. Town R, Midgley N, Ellis L et al. A qualitative investigation of staff's practical, personal and philosophical barriers to the implementation of a web‐based platform in a child mental health setting. Couns Psychother Res 2017;17:218‐26. [Google Scholar]
- 225. Folker AP, Mathiasen K, Lauridsen SM et al. Implementing internet‐delivered cognitive behavior therapy for common mental health disorders: a comparative case study of implementation challenges perceived by therapists and managers in five European internet services. Internet Interv 2018;11:60‐70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 226. Gaebel W, Lukies R, Kerst A et al. Upscaling e‐mental health in Europe: a six‐country qualitative analysis and policy recommendations from the eMEN project. Eur Arch Psychiatry Clin Neurosci 2021;271:1005‐16. [DOI] [PubMed] [Google Scholar]
- 227. Ridout SJ, Ridout KK, Lin TY et al. Clinical use of mental health digital therapeutics in a large health care delivery system: retrospective patient cohort study and provider survey. JMIR Ment Health 2024;11:e56574. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 228. Torous J, Greenberg W. Large language models and artificial intelligence in psychiatry medical education: augmenting but not replacing best practices. Acad Psychiatry 2025;49:22‐4. [DOI] [PubMed] [Google Scholar]
- 229. Pote H, Rees A, Holloway‐Biddle C et al. Workforce challenges in digital health implementation: how are clinical psychology training programmes developing digital competences? Digit Health 2021;7:2055207620985396. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 230. National Health Service England . Digital technology assessment criteria. https://transform.england.nhs.uk.
- 231. Australian Commission on Safety and Quality in Health Care . National safety and quality digital mental health standards. Sydney: Australian Commission on Safety and Quality in Health Care, 2020. [Google Scholar]
- 232. Stern AD, Brönneke J, Debatin JF et al. Advancing digital health applications: priorities for innovation in real‐world evidence generation. Lancet Digit Health 2022;4:e200‐6. [DOI] [PubMed] [Google Scholar]
- 233. US Food and Drug Administration . Digital health technologies for remote data acquisition in clinical investigations. www.fda.gov.
- 234. US Food and Drug Administration . Framework for the use of digital health technologies in drug and biological product development. Silver Spring: US Food and Drug Administration, 2023. [Google Scholar]
- 235. Centers for Medicare & Medicaid Services. Medicare and Medicaid Programs; CY 2025. payment policies under the physician fee schedule and other changes to part B payment and coverage policies; Medicare shared savings program requirements; Medicare prescription drug inflation rebate program; and medicare overpayments. https://www.federalregister.gov. [Google Scholar]
- 236. Beames JR, Johnston L, O'Dea B et al. Factors that help and hinder the implementation of digital depression prevention programs: school‐based cross‐sectional study. J Med Internet Res 2021;23:e26223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 237. Schueller SM, Torous J. Scaling evidence‐based treatments through digital mental health. Am Psychol 2020;75:1093‐104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 238. Curran GM, Bauer M, Mittman B et al. Effectiveness‐implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012;50:217‐26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 239. Cook BL, Hou SS, Lee‐Tauler SY et al. A review of mental health and mental health care disparities research: 2011‐2014. Med Care Res Rev 2019;76:683‐710. [DOI] [PubMed] [Google Scholar]
- 240. Rodgers CR, Flores MW, Bassey O et al. Racial/ethnic disparity trends in children's mental health care access and expenditures from 2010‐2017: disparities remain despite sweeping policy reform. J Am Acad Child Adolesc Psychiatry 2022;61:915‐25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 241. Sun S, Hoyt WT, Brockberg D et al. Acculturation and enculturation as predictors of psychological help‐seeking attitudes (HSAs) among racial and ethnic minorities: a meta‐analytic investigation. J Couns Psychol 2016;63:617‐32. [DOI] [PubMed] [Google Scholar]
- 242. Maura J, Weisman de Mamani A. Mental health disparities, treatment engagement, and attrition among racial/ethnic minorities with severe mental illness: a review. J Clin Psychol Med Settings 2017;24:187‐210. [DOI] [PubMed] [Google Scholar]
- 243. Ralston AL, Andrews AR III, Hope DA. Fulfilling the promise of mental health technology to reduce public health disparities: review and research agenda. Clin Psychol 2019;26:e12277. [Google Scholar]
- 244. Ramos G, Chavira DA. Use of technology to provide mental health care for racial and ethnic minorities: evidence, promise, and challenges. Cogn Behav Pract 2022;29:15‐40. [Google Scholar]
- 245. Jiwani Z, Tatar R, Dahl CJ et al. Examining equity in access and utilization of a freely available meditation app. NPJ Ment Health Res 2023;2:5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 246. Anik E, West RM, Cardno AG et al. Culturally adapted psychotherapies for depressed adults: a systematic review and meta‐analysis. J Affect Disord 2021;278:296‐310. [DOI] [PubMed] [Google Scholar]
- 247. Ellis DM, Draheim AA, Anderson PL. Culturally adapted digital mental health interventions for ethnic/racial minorities: a systematic review and meta‐analysis. J Consult Clin Psychol 2022;90:717‐33. [DOI] [PubMed] [Google Scholar]
- 248. Bauermeister J, Choi SK, Bruehlman‐Senecal E et al. An identity‐affirming web application to help sexual and gender minority youth cope with minority stress: pilot randomized controlled trial. J Med Internet Res 2022;24:e39094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 249. Pachankis JE, Soulliard ZA, Layland EK et al. Guided LGBTQ‐affirmative internet cognitive‐behavioral therapy for sexual minority youth's mental health: a randomized controlled trial of a minority stress treatment approach. Behav Res Ther 2023;169:104403. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 250. Sun S, Nardi W, Murphy M et al. Mindfulness‐based mobile health to address unhealthy eating among middle‐aged sexual minority women with early life adversity: mixed methods feasibility trial. J Med Internet Res 2023;25:e46310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 251. Gonzalez C, Early J, Gordon‐Dseagu V et al. Promoting culturally tailored mHealth: a scoping review of mobile health interventions in Latinx communities. J Immigr Minor Health 2021;23:1065‐77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 252. Habicht J, Viswanathan S, Carrington B et al. Closing the accessibility gap to mental health treatment with a personalized self‐referral chatbot. Nat Med 2024;30:595‐602. [DOI] [PubMed] [Google Scholar]
- 253. Timmons AC, Duong JB, Simo Fiallo N et al. A call to action on assessing and mitigating bias in artificial intelligence applications for mental health. Perspect Psychol Sci 2023;18:1062‐96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 254. World Health Organization . World mental health report: transforming mental health for all. Geneva: World Health Organization, 2022. [Google Scholar]
- 255. The WHO World Mental Health Survey Consortium . Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA 2004;291:2581‐90. [DOI] [PubMed] [Google Scholar]
- 256. Chakrabarti S. Digital psychiatry in low‐and‐middle‐income countries: new developments and the way forward. World J Psychiatry 2024;14:350‐61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 257. Fu Z, Burger H, Arjadi R et al. Effectiveness of digital psychological interventions for mental health problems in low‐income and middle‐income countries: a systematic review and meta‐analysis. Lancet Psychiatry 2020;7:851‐64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 258. Spanhel K, Balci S, Feldhahn F et al. Cultural adaptation of internet‐ and mobile‐based interventions for mental disorders: a systematic review. NPJ Digit Med 2021;4:128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 259. Rodriguez‐Villa E, Naslund J, Keshavan M et al. Making mental health more accessible in light of COVID‐19: scalable digital health with digital navigators in low and middle‐income countries. Asian J Psychiatr 2020;54:102433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 260. Zhao M, Li L, Rao Z et al. Suicide mortality by place, gender, and age group – China, 2010‐2021. China CDC Weekly 2023;5:559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 261. Cai C, Qu D, Liu D et al. Effectiveness of a localised and systematically developed gatekeeper training program in preventing suicide among Chinese adolescents. Asian J Psychiatr 2023;89:103755. [DOI] [PubMed] [Google Scholar]
- 262. Patel V, Naslund JA, Wood S et al. EMPOWER: toward the global dissemination of psychosocial interventions. Focus 2022;20:301‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 263. Institute for Economics & Peace . Global Peace Index 2023: measuring peace in a complex world. http://visionofhumanity.org/resources.
- 264. Cuijpers P, Heim E, Abi Ramia J et al. Effects of a WHO‐guided digital health intervention for depression in Syrian refugees in Lebanon: a randomized controlled trial. PLoS Med 2022;19:e1004025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 265. Böge K, Karnouk C, Hoell A et al. Effectiveness and cost‐effectiveness for the treatment of depressive symptoms in refugees and asylum seekers: a multi‐centred randomized controlled trial. Lancet Reg Health Eur 2022;19:100413. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 266. Danese A, Martsenkovskyi D, Remberk B et al. Scoping review: Digital mental health interventions for children and adolescents affected by war. J Am Acad Child Adolesc Psychiatry 2025;64:226‐48. [DOI] [PubMed] [Google Scholar]
- 267. Kirkbride J, Anglin DM, Colman J et al. The social determinants of mental health and disorder: evidence, prevention and recommendations. World Psychiatry 2024;23:58‐90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 268. Betancourt TS, Chambers DA. Optimizing an era of global mental health implementation science. JAMA Psychiatry 2016;73:99‐100. [DOI] [PubMed] [Google Scholar]
- 269. De Silva MJ, Ryan G. Global mental health in 2015: 95% implementation. Lancet Psychiatry 2016;3:15‐7. [DOI] [PubMed] [Google Scholar]
