Skip to main content
World Psychiatry logoLink to World Psychiatry
. 2021 Sep 9;20(3):318–335. doi: 10.1002/wps.20883

The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality

John Torous 1,2, Sandra Bucci 3,4, Imogen H Bell 5,6, Lars V Kessing 7,8, Maria Faurholt‐Jepsen 7,8, Pauline Whelan 3,4, Andre F Carvalho 9,10,11, Matcheri Keshavan 1,2, Jake Linardon 12, Joseph Firth 13,14
PMCID: PMC8429349  PMID: 34505369

Abstract

As the COVID‐19 pandemic has largely increased the utilization of telehealth, mobile mental health technologies – such as smartphone apps, vir­tual reality, chatbots, and social media – have also gained attention. These digital health technologies offer the potential of accessible and scalable interventions that can augment traditional care. In this paper, we provide a comprehensive update on the overall field of digital psychiatry, covering three areas. First, we outline the relevance of recent technological advances to mental health research and care, by detailing how smartphones, social media, artificial intelligence and virtual reality present new opportunities for “digital phenotyping” and remote intervention. Second, we review the current evidence for the use of these new technological approaches across different mental health contexts, covering their emerging efficacy in self‐management of psychological well‐being and early intervention, along with more nascent research supporting their use in clinical management of long‐term psychiatric conditions – including major depression; anxiety, bipolar and psychotic disorders; and eating and substance use disorders – as well as in child and adolescent mental health care. Third, we discuss the most pressing challenges and opportunities towards real‐world implementation, using the Integrated Promoting Action on Research Implementation in Health Services (i‐PARIHS) framework to explain how the innovations themselves, the recipients of these innovations, and the context surrounding innovations all must be considered to facilitate their adoption and use in mental health care systems. We conclude that the new technological capabilities of smartphones, artificial intelligence, social media and virtual reality are already changing mental health care in unforeseen and exciting ways, each accompanied by an early but promising evidence base. We point out that further efforts towards strengthening implementation are needed, and detail the key issues at the patient, provider and policy levels which must now be addressed for digital health technologies to truly improve mental health research and treatment in the future.

Keywords: mHealth, digital health, psychiatry, mental health, smartphone apps, virtual reality, social media, chatbots, digital phenotyping, implementation


Mental health problems impact over one billion people world­wide annually1, with depression representing the leading cause of disability across the globe2. The World Health Organization’s Mental Health Gap Action Program (mhGAP) outlines evidence‐based interventions to address this global crisis, yet acknowledges that barriers include lack of available services and funding3.

The extent of these barriers, even for high‐income countries, is highlighted in a December 2020 report from the US government, which indicates that offering evidence‐based mental health care in the US alone would require an additional 4 million trained professionals4. On a global scale, it is simply not feasible to propose that practices based entirely on in‐person care will ever be able to meet this demand. Thus, even before the emergence of the COVID‐19 pandemic, there was growing interest in the possible role of new technologies to extend care.

The rapid international growth in access to and capabilities of digital health technologies (DHTs) presents a feasible route towards augmenting traditional mental health care and bridging the gap between the need for treatment and the capacity to deliver it. In this paper, we consider DHTs to be innovations beyond electronic medical records or classical telepsychiatry, to instead focus on the recent developments in smartphone apps, virtual reality, social media, and chatbots.

While the integration of these DHTs into mental health care began somewhat slowly, restrictions driven by the COVID‐19 pandemic have sparked a paradigm shift as assumptions, interest and utilization of digital health have undergone a fundamental transformation. Although there has been variability in the response of health care services to the unmet needs raised by the pandemic, a recent study encompassing 17 different countries reported an overall increased use of digital health in mental health care settings, as well as a renewed support for facilitating uptake during the pandemic5. This increased uptake in response to the pandemic is related not only to DHTs’ ability to connect people to care while social distancing regulations are in place, but also to recent innovations in these technologies that enable them to deliver scalable, affordable and accessible mental health care solutions6, 7.

In this state‐of‐the‐art review, we explore the technologies, the available research evidence and the implementation issues most relevant to integrating digital psychiatry within mental health care. In the first section, we discuss technology mediums of smartphones, social media, virtual reality and chatbots as innovations in the digital psychiatry revolution. The second section critically discusses recent research informing the clinical evidence‐based uses of DHTs, with a focus on smartphone studies, covering their use across multiple contexts, from the promotion of public mental health and well‐being, to the management of long‐term psychiatric conditions. The third section identifies the forefront challenges towards implementation, and discusses potential solutions for improving the use and facilitating evidence‐based adoption of DHTs into mental health care across the world.

TOOLS AND TECHNOLOGIES

The aptly titled 2012 New York Times article The Therapist May See You Anytime, Anywhere 8 highlights that the use of smartphone devices in mental health care has been discussed and anticipated for nearly a decade. Smartphones have quickly become a driving force of digital health, due to special properties defining both the hardware and software of these devices.

From a hardware perspective, they are compact and wireless, with low purchasing and running costs, making them the first devices to provide ubiquitous connectivity/Internet access for a sizeable proportion of the global population. The sensors on these devices allow for new data capture and graphical/computing power for delivery of individualized interventions.

According to 2018 survey data, 76% of people in advanced economies and 45% in emerging economies owned a smartphone9, with recent data from the US showing that ownership rates may be as high as 70% even among people with severe mental illness10, 11, 12. While a digital divide still does exist, it is feasible to envisage that, in the near future, the majority of the world will have access to some form of a smartphone device.

From a software perspective, the relative ease of building new smartphone programs (termed applications or “apps”), combined with the centralized online platforms for finding, sharing and downloading these (i.e., the “app store”), creates an almost infinite potential for any new idea to quickly become an “app”, which can in turn be readily proliferated across any number of users, potentially reaching billions of people across the world. Further, smartphones can serve as a digital “hub” for integration of novel devices such as wearables and other sensors.

In the context of mental health, the clearest result of this focus on smartphones has been the massive influx of apps aiming to provide therapeutic interventions for virtually all known mental health problems13. Alongside app‐based therapeutic interventions, smartphone devices also hold the potential for bolstering mental health care in a number of other ways, including: a) capturing longitudinal, dense and multimodal mental health data for use in diagnosis and monitoring; b) analyzing data, increasingly via machine learning paradigms, to generate clinically individual‐level actionable insights and predictions; and c) offering a wide range of interventions often outside of the app itself, through facilitating connections to clinical care, peer support, personalized resources, emergency care, and even novel therapies. Below, we explain in more detail the evidence behind the multifaceted and large‐scale applications of smartphones.

Smartphone sensor data and digital phenotyping

Until recently, a large portion of the understanding around the determinants of the onset, relapse or temporal variation in mental disorders was primarily based on data from large prospective studies. Although useful, the broad insights gained from such data may fail to capture individual differences or the more fine‐grained temporal relationships between causes and consequences of mental ill‐health. Across the entire field of health care, smartphones are providing a plethora of data enabling new insights into various conditions, through combining their increasingly detailed streams of longitudinal, multimodal and temporally dense data collection. To better clarify the nature and clinical utility of these data, the concepts of “active” and “passive” data are useful.

Active data typically refers to smartphone‐based surveys – i.e., active symptom monitoring or ecological momentary assessment – which can be completed by the user either spontaneously or in response to a prompt, and then stored while crucially time‐stamped (a digital record of the date and time when an item was completed) onto the collecting app. Active data capture offers a new means to characterize a patient’s clinical course.

While most clinical assessment scales have not been validated for deployment on mobile devices, strong correlations between traditional in‐clinic metrics and their often‐simplified mobile versions suggest adequate face validity14. The evolution of these assessments to focus on non‐traditional metrics such as perception of self, functioning and social life (which research has shown to be particularly important to patients15) provides new opportunities for furthering the potential of active data collection. The use of smartphones for cognitive assessment16 and for remotely monitoring symptoms17 also appears feasible, with promising results even for severe mental illness such as schizophrenia18, 19.

While concern is often raised around using mental health apps for monitoring suicidal thoughts and urges, or even eliciting an increase in symptoms through reactivity to monitoring, research shows that actively collecting data on suicidal thoughts and urges does not elicit adverse effects20.

Passive data are obtained automatically through sensors, either on the smartphone or via a wearable device, ranging from simple device use metrics to accelerometry, global positioning system (GPS), and even now voice tone (via microphone) or facial expression (via camera) data. These automatically collected data offer a means to reduce patient burden typically related to active data collection, while also capturing novel digital markers of behavior.

Often referred to as “digital phenotyping”21 within the emerging framework of precision psychiatry, the multimodal nature of passive data obtained from consumer grade devices offers a means to understand the lived experiences of mental health in context22. For example, GPS data have recently offered insights into the relationship between reduced mobility and poorer mental health during the COVID‐19 pandemic23. Passive data from smartphones have also been shown to correlate with outcomes such as social functioning and loneliness24, 25. An important trend emerging from passive data studies in various conditions is that variance, or measures of entropy or deviation from a personal mean, appear of more value than absolute measurements from any sensors23, 24, 25.

Recent reviews suggest that “most studies still only scratch the surface of advanced smartphone capabilities”26, and less than 2% of apps on the commercial marketplaces appear to leverage digital phenotyping potential27. Still, recent studies are employing digital phenotyping methods across diverse mental disorders28, 29, 30, and research interest in this field is expanding at a rapid pace.

The density and complexity of passive data31 is far greater than current clinical assessments, which continue to rely on static scales that ask a patient to recall symptoms over a defined period of time – e.g., a two week period in the case of the ubiquitous depression assessment by the Patient Health Questionnaire‐9 (PHQ‐9)32. However, the depth and diversity of passive data (which already typically combine measures such as step counts from wearables, text analytics from social media, metadata from electronic medical records, or green‐space exposure from geolocation) require new techniques in data science, such as artificial intelligence and machine learning, to meaningfully combine and utilize such “big data” to inform mental health care33.

Advances in artificial intelligence and machine learning will likely represent a prominent bridge for translating new data into clinically relevant digital biomarkers34, 35, 36. Like all biomarkers, though, impact will be determined not only by statistical significance but also by clinical utility. A case in point refers to digital markers of self‐harm and suicide, which, according to a recent review, possess high classification accuracy yet near zero accuracy for predicting future events37.

However, other approaches to digital phenotyping for different conditions/outcomes are beginning to show some promise. For instance, relapse risk in schizophrenia may be foreseen by “anomaly detection”, which involves the use of smartphone sensor data to monitor divergences of an individual’s behavioral patterns compared to his/her personal baseline. Preliminary studies in small samples have found reasonable sensitivity and specificity from applying this approach to date30.

Overall, while active and passive data have the potential to make smartphones crucial elements for the development and implementation of precision psychiatry38, the validity of the measures, how the data can be meaningfully represented, and the potential for ethical and effective uses in treatment delivery have all yet to be established.

Smartphone technologies for closed loop interventions

A rich legacy of Internet‐delivered and computerized therapy research and experience39 is now in the process of being translated into new smartphone‐based interventions, with promising results as well as challenges. These app‐based interventions often utilize established aspects of cognitive and behavioral therapies to offer patients “on demand” access to evidence‐based care tools. Examples abound of studies targeting mental health problems such as depression and anxiety40, 41, 42, 43, and early psychosis and schizophrenia44, 45, that have been the subject of previous reviews46, 47, 48, 49, 50, 51. The existing clinical evidence for digital health interventions across specific disorders is reviewed in more detail in the second section of this paper.

The potential for more personalized digital health interventions is bright. Known as a just‐in‐time‐adaptive‐intervention (JITAI), active and passive symptom data capture may aid in the development of personalized and real‐time intervention strategies52, 53. For example, the smartphone may be able to infer low mood in the context of social isolation and offer a relevant intervention, whilst, in another circumstance, it may infer low mood in the context of poor sleep and recommend an alternative intervention. Although in its infancy, using JITAIs to offer “closed loop” mental health interventions is a promising area for future research.

Nevertheless, app marketplaces rarely reflect evidence from recent studies, or otherwise take advantage of the unique potential of app interventions54. For instance, just one percent of marketplace apps support use of sensors55, suggesting that concepts of digital phenotyping to support JITAI or behavioral interventions via apps are largely not incorporated into existing commercial technologies. Rather, even when considering more static interventions that do not take advantage of advanced smartphone features, the evidence base for widely proliferated apps remains poor54. For example, one review suggests that only ~2% of commercially available mental health apps are supported by original research evidence27. As we explore more details of app interventions in later sections, it is useful to consider that integration with sensors and digital phenotyping will likely soon transform this space.

Social media

The relationship between social media and mental health has received much attention from not only the academic literature, but also the traditional media and general public56. Frequently accessed via smartphone apps and connecting people from their own devices to global networks of friends, information, and health resources, social media can represent both a means to quantify mental health as well as a source of both positive and negative interactions.

Increasingly, research suggests that absolute screen time or exposure itself is not strongly associated with adverse mental health outcomes57. This is in marked contrast to the more popular conception that screen time and social media use is detrimental to mental health. In part, this view gained ascendence from the older literature, which was largely based on self‐reported usage and cross‐sectional analysis, thus offering limited evidence in this regard. Recent studies, however, based on objective screen use and social media engagement measurements, prospective cohorts, and new scales to assess problematic Internet use, are painting a more nuanced picture of social media and mental health58, 59. For example, during the COVID‐19 pandemic, social media have been a source of social support for many who have been socially isolated and lonely.

While excessive use of social media and screen time is likely not beneficial for mental health (in the same manner that excessive use of any activity or behavior is often associated with deleterious outcomes), the quality of screen time and social media interactions appears to be more important than the quantity60. It is interesting that in recent years social media companies such as Facebook and Pinterest have undertaken new efforts to flag content that may be related to self‐harm or suicide56. Nevertheless, it is currently difficult to determine the results of such interventions. The impact of social media on the developing brain also remains an unresolved61 yet frequently discussed topic, especially as the pandemic has forced increasing reliance on technology to connect people.

Patterns of social media use may represent a means to detect worsening of mental health symptoms. For example, changes in the content and style of social media posts may offer an early warning sign of relapse in schizophrenia62. Social media, combined with natural language processing methods, also offer a practical means to understand population‐level mental health trends. For example, an analysis of 60 million Twitter posts in March‐May 2020, as compared to one year prior, was able to detect pandemic‐related increases in coping mechanisms63. These methods have also been employed in studies exploring psychosocial reactions to the COVID‐19 pandemic64, 65, as well as the effects of psychiatric medications66.

While currently available work has largely focused on text‐based natural language processing methods, the increasingly voice‐ and video‐based nature of newer social media content has sparked interest in emotion recognition67. For example, early studies identified relationships between negative mood and posting pictures with darker colors68, although such relationships are now known to be more nuanced, thus highlighting inherent challenges in assessing mental health without a broader context.

Social media can also be used as a therapeutic tool. Novel research using carefully curated and monitored social networks as interventions has shown promise in youth with diverse mental health needs69, 70, 71. For example, the PRIME app72 is designed to help people with schizophrenia through the promotion of functional recovery and the mitigation of negative symptoms (e.g., amotivation) through a supportive and personalized network. The Moderated Online Social Therapy (MOST) platform is another example of an innovation that offers personalized therapy combined with social connections among other features71, 73.

It is noteworthy that social media are not without risk. Disinformation74 and stigma on social media are forces that cannot be ignored. Stigma on social media is common75, although efforts are also underway to challenge and reverse this trend76. Using social media for mental health work also remains a catalyst for ethical tensions, and a recent review offers a practical taxonomy of these tensions as well as guidance for navigating through these ongoing challenges77.

Chatbots

Conversational agents, such as Apple’s Siri or Amazon's Alexa, have become common in the digital marketplace. Termed “chatbots”, the use of these conversational style interfaces offers an intelligent, automated system for detecting and responding to immediate mental health needs. Chatbots have the look and feeling of interacting with a human, despite being run by an automated software program. Thus, chatbots or “robot therapists” have become a galvanizing force for those seeking to automate therapy where software programs listen and respond to people's mental health needs. While the words “robot therapist” conjure images of a physical robot, most are actually text based, although animated video and even physical robot versions have been researched78, 79.

One ongoing challenge in chatbot work is seeking to offer emotional support from inherently inanimate computer code. There is some evidence that people can develop therapeutic relationships with digital technologies (referred to as “digital therapeutic alliance”80). As therapeutic alliance with an in‐person therapist is related to more positive outcomes in mental health treatment81, harnessing the digital therapeutic alliance through human‐style interactions with a chatbot might promote change without the need of human support82. Research has found that some people feel more comfortable conversing anonymously with a chatbot83, and that this may open up the possibility to improve detection of distress and in turn provide momentary interventions to those who feel less comfortable with face‐to‐face contact84.

Chatbot interfaces have become a key feature of many commercially available mental health apps. However, their evidence base is not well established85. Across two recent systematic reviews, 24 studies investigating chatbots for health care were identified85, 86. Of the 11 trials targeting mental health problems, most were for depression, with a smaller number targeting anxiety, schizophrenia, post‐traumatic stress disorder (PTSD), and autism spectrum disorder. Only two randomized controlled trials were included, and, while some mental health benefits from chatbot interventions were indicated, the types of benefits observed were not consistent across studies, which were further limited by small sample sizes, short duration, and a lack of follow‐up data.

While the development and implementation of more complex interactive systems is inevitable, current chatbots are limited in their ability to deliver appropriate contextual responses to complex language inputs, presenting important safety concerns. One study of commercial chatbots such as Siri found that they often failed to recognize serious mental health concerns and provide appropriate responses such as referral to a support service87. For example, chatbots were found to not recognize when suicidal ideation was being discussed, and these devices also seemed to ignore domestic violence problems. Further, surveys of consumer attitudes reveal concerns about the privacy of chatbots as well as their potential to replace human care. Nevertheless, satisfaction ratings in the limited number of pilot and feasibility studies tend to be high, and rates of adverse events low88. Given the evidence and governance in place at this time, chatbots are best used only as a supportive tool in the context of a broader treatment plan.

Virtual reality

Virtual reality involves an immersion in an interactive, computer‐simulated environment via a headset. The ability to create and control exposure to real‐world environments presents important opportunities for mental health assessment and treatment89, 90, 91. Standard psychological assessments are limited by a lack of real‐world validity and overreliance on subjective ratings92. Virtual reality allows precise, real‐time data capture of responses to stimuli within controlled virtual environments, and hence provides critical insight into the way in which clinically relevant phenomena develop in real world89, 93.

Controlled exposure to anxiety‐inducing stimuli within a virtual environment offers a safe, convenient and accessible medium to deliver exposure‐based behavioral treatments. The benefit of virtual reality treatment lies in the repeated exposure to feared stimuli, enabling the individual to adapt to triggers and develop healthy responses in a safe and controlled therapeutic platform94. For example, randomized controlled trials have shown that learning to engage in virtual social interactions can reduce paranoia in people experiencing psychosis95, 96.

A recent meta‐review of 11 meta‐analyses, covering predominantly anxiety disorders and PTSD, found that effect sizes for virtual reality exposure treatments were overall moderate to large, and were typically maintained at follow‐up97. A smaller number of trials have been conducted for other psychiatric disorders, with emerging evidence that virtual reality treatment may be effective for depression, schizophrenia97 and eating disorders98. However, in the studies that have compared virtual reality to traditional treatment, there was little evidence for superior efficacy. Further, the quality of evidence is overall low to moderate, due to the predominance of studies with small sample sizes, the relatively limited number of randomized controlled trials, and issues around publication bias.

Fewer studies have explored virtual reality treatments beyond exposure therapy, with the exception of skills training, which has also demonstrated positive results99. Pilot studies have also shown that virtual reality applications can guide people to learn therapeutic skills such as mindfulness100, 101, 102, relaxation103 and self‐compassion104, 105. Using virtual reality as a vehicle to deliver experiences that help people develop skills to manage mental health difficulties may increase treatment engagement and efficacy.

Virtual worlds offer a compelling solution to increased demand for technology platforms that can deliver personal clinical care remotely106. Virtual worlds enable users to meet within virtual environments, represented as personalized avatars, and interact with other users in real time. Whilst few studies have been conducted in mental health, there have been promising early results especially in psychosis107. Delivering therapy via virtual worlds has the clear potential of offering highly accessible care within personally tailored, engaging therapeutic environments that provide a safe and comfortable medium for social interactions.

Whilst commercial growth in virtual reality is occurring rapidly, with an estimated growth of $54 billion over the next 7 years108, the technology remains unfamiliar and inaccessible to many users, presenting a barrier to implementation89. As costs decrease and virtual reality becomes more mainstream (partly due to the increased capacity to deliver it via smartphones), there is a need for further research and subsequent provision of evidence‐based treatments and protocols, with adequate training for relevant workforces to enable their implementation.

EVIDENCE FOR DIGITAL PSYCHIATRY WITHIN SPECIFIC CONTEXTS

The research base on the efficacy and acceptability of the various types of DHTs is rapidly expanding. In this section, we explore recent and notable findings from empirical studies of the DHTs described above, with a focus on smartphones, across four specific contexts of mental health care: self‐management of depression and anxiety; clinical management of major mood disorders; remote monitoring and interventions for psychosis, eating disorders and substance use disorders; and child and adolescent mental health.

Self‐management of depression and anxiety

Depression and anxiety disorders are among the most common types of mental health conditions in the world109, and many more individuals experience subthreshold albeit disabling symptoms. Due to the high demand for self‐management strategies for depression and anxiety, smartphone apps claiming to help with these problems are widely available on app marketplaces110, 111.

A recent large‐scale meta‐analysis of 66 randomized controlled trials explored the efficacy of smartphone apps for mental health problems including depression and anxiety across clinical and non‐clinical populations112. For depressive symptoms, this meta‐analysis found that smartphone apps outperformed control conditions, with larger effect sizes found when waitlist or educational resources (health tips, information) were used compared to attention/placebo controls (e.g., gaming apps)112. Smartphone apps also outperformed control conditions for generalized anxiety and social anxiety symptoms112. App interventions for anxiety did not differ significantly from face‐to‐face or other computer‐based interventions in terms of outcomes, although only a small number of studies were used in these comparisons. For both depression and anxiety, studies which provided professional support alongside the smartphone app (e.g., through supportive phone calls or personalized therapist feedback) produced larger effect sizes compared to studies which did not.

A common criticism of smartphone apps for depression and anxiety is that they lack an underlying evidence‐based framework111, 113. A review of 293 commercially‐available apps for anx­iety and/or depression found that just over half (55.3%) included a reference to an evidence‐based framework in their app store descriptions111. When a reference was included, a range of therapeutic frameworks were mentioned, including cognitive behavioral therapy techniques (30.0%), mindfulness (15.7%), positive psychology (9.2%), dialectical behavior therapy (3.4%), acceptance and commitment therapy (1.7%), and other techniques (6.8%). However, of the 162 apps that claimed to use a theoretical framework, only 6.2% had published evidence supporting their efficacy111.

The selective adoption of self‐management apps for depression and anxiety has also been explored. A consumer data‐driven review highlighted that the proliferation of depression and anxiety apps on the marketplace is in contrast with the relatively small number of apps which are regularly downloaded and used. The review reported that just three apps (Headspace, Youper and Wysa) accounted for about 90% of app downloads for depression. Similarly, three apps (Headspace, Calm and Youper) accounted for approximately 90% of downloads and daily active users of anxiety apps114. Moreover, most apps for depression (63%) and anxiety (56%) had no active users for the one‐month period under review114. While commercial app companies do not publish engagement data, it is clear that downloads do not automatically translate into active use. For example, the popular (and free) COVID Coach app designed to address stress during the pandemic reported over 140,000 downloads, but only 1.56% of individuals who have downloaded the app recorded at least two weeks of use115.

There are several areas in which improvements can be made for apps dedicated to depression and anxiety. They include: ensuring substantive involvement of relevant health care professionals in the development of the apps110; embedding apps within local health care settings116; more robust testing of apps, specifically more well‐designed randomized controlled trials to assess their efficacy114; understanding engagement techniques to ensure optimal use114; and using validated treatment techniques/interventions within the apps116. Further evaluation of anxiety and depression apps is clearly warranted114, including the need for additional research into the efficacy of app‐delivered interventions compared with face‐to‐face “care as usual”116. Further research is also needed to understand the long‐term engagement, as well as to examine any possible deleterious effects related to app usage111.

The evidence to date suggests that smartphone apps could provide an accessible, scalable and low‐cost mechanism to deliver effective self‐management interventions for symptoms of depression and anxiety, particularly to non‐clinical populations and those who cannot access face‐to‐face services110, 116. However, the promise of apps to increase low‐cost access to evidence‐based treatment for depression and anxiety has not yet been fully realized. Efficacy trial data are needed to support many anxiety and depression apps available on the marketplace. Most of such apps have no clear evidence of efficacy47, 51, 117.

Clinical management of major mood disorders

Despite the growing evidence base described above on the use of DHTs for self‐management of depression and anxiety, much of the existing research has been conducted in general population samples or people with mild‐to‐moderate symptoms. Thus, the current applicability of such research in the actual clinical management of severe mood disorders, such as bipolar disorder and major depressive disorder, remains unclear.

A recent systematic review and meta‐analysis concerning the efficacy of digital interventions in bipolar disorder found positive effects on both depressive and manic symptoms118, but only four of the ten included studies were randomized controlled trials45, 119, 120, 121. As to unipolar depressive disorder, while an increasing number of randomized controlled trials of apps with psychotherapy‐related content have been published26, several of them have shown no evidence that delivering psychological interventions via smartphone confers a significant advantage beyond control conditions122, 123, 124. However, randomized controlled trials which have used app‐based interventions alongside human coaching to bolster their usage in community patients with depression have produced more robust evidence42, suggesting that human engagement in supporting app‐based interventions is critical. New roles such as digital navigators to support app use in mental health care may provide one solution to offer human support without overburdening the clinician125, 126, 127, 128 (see below).

The fact that bipolar disorder and major depressive disorder are characterized by episodic fluctuations in mood and behavior may suggest that smartphone‐based interventions which provide fine‐grained monitoring and real‐time treatment (including JITAIs) may improve outcomes, either by fostering early identification of deterioration and/or by providing means for flexible and timely treatment interventions. Preliminary evidence in patients with major depression indicates that smartphones do indeed represent an available platform for real‐time monitoring of patient‐reported symptoms, such as changes in mood and activity, through ecological momentary assessments129, 130, 131, and that this can feasibly be supported through collection of sensor‐based data such as the number of incoming and outgoing calls and text messages, or location information which may reflect changes in behavior and psychomotor activity. Similarly, in bipolar disorder, several recent studies have shown that smartphone‐based active and passive data reflect digital markers of symptoms132, 133, 134, and classifications of affective states135, 136 and affective traits28, 137 have been published. Collectively, these studies suggest that such digital data could provide important real‐time information reflecting the psychopathological status of patients with major mood disorders.

An important consideration is that patient‐reported symptoms collected in clinical encounters have an inherent risk of recall bias45. On the other hand, establishing the extent to which patient‐reported mood ratings collected via smartphones are consistent with clinical symptom ratings in patients with severe mood disorders is imperative for determining the role of such technologies in the clinical landscape. Studies examining this issue have largely indicated that smartphone‐based mood assessments represent promising alternatives or adjuvants to traditional clinical measures, while acknowledging the methodological limitations in the existing evidence base, including that the overwhelming majority of trials and observational studies to date have enrolled small samples138, 139, 140.

To determine how worthwhile these new approaches could be in routine practice, it is also crucial to examine whether the use of monitoring technologies as an adjuvant ongoing evidence‐based tool for major mood disorders would result in an improvement of outcomes. In keeping with this view, two recent pragmatic randomized controlled trials have examined the effect of smartphone‐based monitoring and treatment in patients with bipolar disorder139 and unipolar depressive disorder140 in real‐world settings. These trials found no effect on primary or secondary outcome measures, including rates of rehospitalization or severity of depressive or manic symptoms, whilst showing higher levels of patient‐reported recovery, compared to the control condition.

Overall, there are several promising trends in the use of smartphones for treatment and monitoring141, 142, 143, 144 in clinical samples with bipolar disorder and major depressive disorder. Continuous data analysis (potentially paired with machine learning models) could support prediction of relapse and use of smartphone‐based interventions in real‐time within the context of precision psychiatry. However, validating the measures used, establishing clinically useful interventions, and ensuring that patients are indeed able to engage with these long‐term interventions, are all key steps to be undertaken by researchers prior to the evidence‐based implementation of these novel technologies in routine clinical practice.

Psychosis/schizophrenia

While those outside of the mental health field at time wonder if smartphones and digital technology could induce paranoid delusions in people with schizophrenia spectrum disorders, the reality is quite the opposite. People with psychosis/schizophrenia are interested and eager to use innovative tools to possibly augment their care and ultimate recovery. Adverse events related to paranoia are nearly non‐existent. Research in this area features innovative works around both remote monitoring and app‐based interventions.

Remote monitoring is of interest in psychotic disorders, especially to augment self‐reported information when cognition may be overly impaired. Real‐time and in‐context patient‐generated symptom data, obtained through remote‐monitoring platform technology, have the potential to timely warn clinicians about the need for intervention, improve treatment decisions by providing a clearer picture of changing patterns of symptoms, and support scheduling of health care contacts based on need145.

Research groups around the world have started to explore how integrating this active data collection with passive remote monitoring can both predict relapse and allow delivery of time‐sensitive intervention strategies. To date, these data streams have been predominantly used in small (N<100) studies in selected populations, with promising results. A systematic review146 of studies conducted in samples with psychotic disorders identified 17 active monitoring apps. App use duration ranged from 1 week to 14 months, with self‐assessment prompts ranging from multiple times per day to weekly. People typically adapted their response strategy to less frequent active data collection over time. App assessments were well tolerated, with 69% to 88% assessments completed. All studies showed that people found this active data collection acceptable and useful, despite some negative effects reported (e.g., increased awareness of symptoms).

Sensors on the smartphone or a wearable device have emerged as tools to assess behavioral patterns in a range of populations, and have been utilized to both reduce the burden associated with active symptomatic monitoring and to obtain additional objective behavioral data. A systematic review of studies146 identified four passive monitoring studies, with usage ranging from 5 to 365 days in sample sizes ranging from 5 to 62 participants. Two studies found that passive monitoring was largely acceptable, although 20% of participants reported privacy concerns and 20% felt upset by it.

More specifically, Barnett et al147 followed 17 patients with psychosis using a passive monitoring app installed on their smartphone for up to three months, and identified anomalies in mobility patterns and social behavior in the two weeks prior to relapse. A further study in 83 patients with psychosis using digital markers found similar results30. This was also observed in a study (N=60) using a neural network approach148.

Ben‐Zeev et al149 identified sensor data changes – including physical activity, geolocation, phone unlock duration, and speech frequency and duration – in participants with psychosis in the days leading up to a relapse. Wisniewski et al150 also noted high variability in behavioral patterns observed through passive monitoring in individuals who were deemed to be at clinical high risk for psychosis. However, the utility of passive monitoring in predicting conversion to psychosis among these individuals remains unclear.

Although in their infancy, passive monitoring studies have shown that most patients with psychotic disorders are comfortable, able and willing to use wearable devices to monitor outcomes in their daily life151, with emerging evidence supporting identification of an impending relapse through changes in passively collected behavioral data. However, robust safety data are needed to understand the utility of this approach more clearly152.

Beyond monitoring, DHTs have also played a significant role in delivering intervention strategies and support for psychosis. A recent systematic review identified 21 DHTs for psychosis published in the peer‐reviewed literature, incorporating a mixture of computerized, avatar, and app‐based approaches. The studies included a total of over 1,500 participants, and were mostly conducted in Europe and North America152.

Whilst it is difficult to compare these studies, given the different technologies used and outcomes measured, there is emerging evidence that DHTs can improve symptoms, as well as cognitive and other clinical outcomes, in people with psychosis152. For example, the Actissist app targets negative symptoms (e.g., reduced socialization), general psychotic symptoms, mood, and cannabis misuse through offering tools to help with cognitive appraisals, belief conviction, emotions and associated behaviors153. Another app (SlowMo) targets paranoia through offering tools to help with jumping to conclusions and belief inflexibility as part of blended care154. A study of 361 patients with a diagnosis of schizophrenia, randomized to receive either SlowMo therapy or usual care, found no significant difference between groups on the primary outcomes related to paranoia at 24 weeks (the primary end‐point), though significant effects between groups were apparent post‐treatment154.

While these approaches are promising, further well‐powered efficacy trials are needed to appraise their full potential. Co‐design of the technology with the actual end users is vital in ensuring that engagement to DHT is maximized155. Furthermore, with some exceptions153, 156, trials have not robustly measured adverse events, which is needed when determining the safety of DHTs not only in people with severe mental health problems but in health care more broadly.

There is, of course, the challenge of implementing these intervention approaches, should they prove to be effective and cost‐effective. There are very few examples of successful implementation of DHTs into routine clinical services, though research groups have proposed frameworks to support implementation from the outset of digital health program development157, 158, 159.

Eating disorders

The interest in smartphone app technologies for eating disorders is growing, either as a standalone intervention or an adjunct to traditional treatment services.

People with eating disorders are a clinical group that could be well suited to app‐based interventions, as the ego‐syntonic nature of these conditions usually results in treatment refusal, ambivalence to change, or low motivation to engage in the therapeutic process160, 161. The ability for apps to allow an individual to approach treatment at his/her own pace may address these concerns and could help individuals feel more in control of their treatment. Similarly, tailored reminders and motivational messages to practice key therapeutic skills may help increase these patients’ motivation and adherence to the treatment program.

Furthermore, their scalability, flexibility, and cost advantages over traditional face‐to‐face services indicate that smartphone app technologies could offer a potential solution to many existing help‐seeking barriers and the widespread treatment gap reported in this clinical group162. Importantly, recent survey data show that a significant proportion of individuals with an eating disorder report a preference for, and willingness to use, smartphone apps and other DHTs163, 164, indicating that their demand is high.

The quality of information in publicly available eating disorder apps has been widely discussed. Existing eating disorder apps tend to serve one or more of four broad functions: delivery of information, self‐assessment, self‐monitoring, and provision of advice or treatment. Two earlier systematic appraisals of the quality of commercially available eating disorder apps concluded that very few of them incorporated components of evidence‐based treatments, with some even providing potentially harmful information165, 166. However, four commercially available eating disorder apps (Mental Health Tests, Recovery Road, Rise Up, and Psychiatry Pro‐Diagnosis, Info, Treatment, CBT & DBT) — each of which are grounded in an evidence‐based framework — account for 96% of monthly active users according to a recent review167, indicating that those resorting to apps to help manage their eating disorder are likely exposed to credible information.

Limited research has been conducted on the efficacy of smartphone apps as a standalone intervention approach for eating disorders. The most up‐to‐date meta‐analysis of self‐ or minimally‐guided DHTs for the treatment and prevention of these disorders did not locate any published randomized controlled trials of standalone app‐based interventions up until June 2020168. One randomized controlled trial has since been published, finding preliminary support for the short‐term efficacy of a transdiagnostic cognitive‐behavioral app (Break Binge Eating) on numerous symptom measures among individuals with a threshold or subthreshold binge eating‐type disorder169. Although these results are promising, additional evidence from randomized controlled trials with longer follow‐ups is needed to determine whether smartphone app technologies are an appropriate stand­alone intervention modality or first step in the treatment and management of eating disorders.

More attention has been devoted towards understanding the role of smartphone apps as an adjunct to traditional face‐to‐face services. In light of evidence demonstrating a robust relationship between low skills utilization and poor treatment outcomes among individuals with eating disorders170, 171, app technologies have been proposed to augment face‐to‐face treatments by allowing patients to more regularly practice essential homework tasks between sessions172.

Indeed, evidence from existing randomized controlled trials indicates that the addition of smartphone app technology to traditional face‐to‐face services may lead to greater treatment adherence and skills utilization, and quicker symptom improvements in adults with binge eating173, 174. However, whether these benefits persist in the longer term, and for whom specifically app technology offers an added benefit, remains unclear.

Overall, although significant interest has been generated towards understanding what role apps might play for the treatment and management of eating disorders, further rigorous trial designs with longer follow‐up assessments across different diagnostic categories (e.g., anorexia nervosa) are needed.

Substance use disorders

Interest in the clinical utility of smartphone app technologies for substance use disorders is also growing. Inbuilt app features such as machine learning algorithms, that automatically adjust in response to active and passive data, can facilitate the delivery of highly specific, tailored intervention strategies in moments of need52. This functionality is especially applicable for substance use disorders, as affected individuals often find it difficult to anticipate upcoming internal or external events that trigger a relapse175.

Although an increasing number of apps for substance use disorders are commercially available, their focus is largely restricted to targeting smoking or alcohol consumption, with few apps specifically designed to address other costly and debilitating disorders, such as cocaine or methamphetamine use176.

Apps to address opioid use disorder have recently become available and received Food and Drug Administration (FDA)’s approval in the US. However, a 2020 report examining the economic benefit of these apps for opioid use disorders noted: “At current… pricing, and given available evidence, these potential cost offsets and clinical gains were not enough to generate incremental cost‐effectiveness estimates beneath commonly cited cost‐effectiveness thresholds”177.

Empirical research investigating the efficacy of app‐based interventions for substance use disorders remains limited, but is rapidly expanding. In the context of smoking, one recent meta‐analysis51 of three randomized controlled trials comparing standalone apps to control conditions observed a significant although small effect size for reduced smoking frequency in favor of the app conditions (g=0.39, 95% CI: 0.21‐0.57). In contrast, a recent Cochrane review178 of five randomized controlled trials found no significant differences in rates of smoking cessation between apps and non‐app smoking cessation support conditions (risk ratio, RR=1.00, 95% CI: 0.66‐1.52). For alcohol use, the above meta‐analysis51 identified three randomized controlled trials comparing a standalone app to a control condition, reporting a small and non‐significant pooled effect size (g=–0.03, 95% CI: –0.22 to 0.17). Other recent qualitative reviews have not been able to identify any randomized controlled trials of app‐based interventions for other substance use disorders179, 180.

Additional research is needed to better understand what role smartphone app technologies could play towards the treatment, prevention or management of substance use disorders. Although the quality of commercially available apps for these disorders is suboptimal, it is promising to see research teams from around the globe beginning to develop smartphone apps in this area that have a clear underlying evidence‐based framework, capitalize on latest advancements in technology (e.g., gamification, conversational agents), are routinely tested for their usability, involve feedback from end users, and are registered for evaluation in prospective clinical trials181, 182, 183, 184, 185.

Child and adolescent mental health

Child and adolescent mental health is a public health priority, with a prevalence of up to 20% of mental disorders across child and adolescent populations worldwide186. The increasing ubiquity of smartphone use among these populations suggests that smartphones could be an ideal mode of delivery for mental health interventions187. A systematic and meta‐review of DHTs for children and young people identified anxiety and depression as the most common mental health problems targeted, with many other areas (e.g., psychosis) being relatively under‐researched188.

The strongest evidence of effectiveness of DHTs for children and young people is reported for approaches using computerized or Internet cognitive behavioral therapy (iCBT)49, 188. A meta‐analysis of 34 randomized controlled trials for depression or anxiety in child and adolescent populations supported the effectiveness of iCBT‐based interventions in comparison to waitlist controls189. More favorable outcomes were achieved when the treatment was therapist‐ or parent‐supported189.

Alongside the growing evidence for digital therapies in young people, passive sensing technology is likely to be used in future research. While currently in its infancy, there is an increasing body of research suggesting that passive data collected through DHTs may aid in the understanding of how behavior relates to mood and anxiety in children and young people190.

Overall, most reviews in the area of digital mental health for children and young people recognize the need for further research into the effectiveness of DHTs, but highlight the promise of smartphone apps49, 188. A core challenge in this research is the additional privacy issues inherent to working with young people, as well as continued screen time concerns (as noted in above sections). Still, the future is promising, and particular progress has been made on iCBT‐based interventions for anxiety in children190. It is to be acknowledged that many of the technologies tested in young people have been first developed for adult populations, rather than being designed and co‐produced by young people themselves.

IMPLEMENTATION: CHALLENGES AND OPPORTUNITIES

The potential and evidence around digital mental health must be considered in the context of real‐world use, given that much of the interest and excitement around DHTs stems from perceived feasibility and scalability of real‐world implementation. In actuality, however, implementing DHTs has proven challenging. Even the relatively simple task of translating effective face‐to‐face interventions directly into digital versions is often more complex than once thought191, 192.

There are numerous implementation science frameworks. In this section, we utilize the Integrated Promoting Action on Research Implementation in Health Services (i‐PARIHS)193 framework, which focuses on three elements: the innovation itself, the recipients of that innovation, and the context surrounding the innovation. While the prior two sections of this paper have focused on innovations across DHTs and clinical use cases, here we focus on recipients (patient/clinician implementation) and context (health care factors, including regulatory, market, ethical, and global mental health forces). Only when the three elements – innovative DHTs, recipients primed to utilize DHTs, and contextual forces that support and sustain DHT use – are all aligned, can the full potential be realized. Despite rapidly advancing work around DHT innovation, the latter two elements have not received equal attention.

Recipients of innovation: patient factors

While smartphone ownership is above 80% in the general population in the US, US‐based Medicare data from 2018 suggests only 61% of beneficiaries have access to a smartphone with a wireless plan, and that access is more likely to be lacking in those who are older, less educated, and Black or Hispanic194.

While disparities in ownership must be acknowledged and addressed today, they are projected to diminish as technology becomes more affordable. Thus, a larger threat to access may be a new digital divide around technology literacy. If DHTs become a part of routine practice within mental health services, will the most vulnerable patients be able to navigate these technologies and access care? Formal data on digital literacy in mental health is scarce, but there are mounting calls for new resources and tools to help ensure that patients have the skills, training and confidence to utilize DHTs195, 196, 197. Training programs designed to teach patients how to utilize DHTs are becoming available. An example is the Digital Outreach for Obtaining Resources and Skills (DOORS) program198, that offers a suite of in‐person and online training resources.

Many DHTs rely on end‐user engagement in offering monitoring or interventions, yet engagement remains a core challenge199, 200, both in and outside the context of research trials. Without standard measurements for evaluating or comparing engagement across DHTs201, 202, progress towards improved engagement has been fragmented. Extracting engagement data from apps and other technologies, especially outside of academic research efforts, is often impossible, except through market research companies that can only offer general population‐based samples.

Using this type of data, a 2020 study examined engagement rates of popular (over 100,000 downloads) mental health apps on the app stores/marketplace, and found that 90% of users abandon apps within 10 days200. Actual data on app engagement from over 100,000 participants in different studies across various health conditions showed that the median participant retention was just 5.5 days203. As mentioned before, this drop‐off in use was also found in a 2021 report of the stress app COVID Coach, which reported that only 1.56% of users remained engaged for at least 14 days115. Research data also do not confirm commonly held assumptions that older adults will engage less than younger people203, although other reviews suggest the opposite197.

Studies suggest that human support alongside app use offers the strongest contribution towards improving engagement112. However, human facilitation of app‐based tools limits the scalability and underlying potential of many apps to expand access to care. An increasing attention is now devoted to co‐designing and co‐producing digital mental health tools with end users and all stakeholders at the outset, in the hope that digital solutions will reflect the actual needs and preferences of those they are designed to serve204, 205.

DHTs hold an unique potential to extend access to care in middle‐ and lower‐income countries, where there is less investment or infrastructure around mental health206. Yet, a recent review on this topic found only 37 relevant studies published in those countries between 2016 and 2020, with the majority reporting feasibility and accessibility outcomes instead of efficacy, cost‐effectiveness or implementation207. Yet, smartphone use is common and rapidly expanding in those countries, and thus represents a promising tool to reduce the mental health gap.

Research in low‐resource countries has focused to date on medication and appointment adherence as well as relapse and rehospitalization prevention208, which offer important targets with transdiagnostic potential. Leaders in the field have called for a focus on data science, task sharing by empowering community health workers, and early interventions as promising leads209. The untapped potential for global mental health is to adapt current digital health tools with strong efficacy data, and devote resources towards establishing local effectiveness and routes towards implementation. While this has not frequently occurred to date, recent research in implementation science holds lessons for this translation, suggesting that context, culture, and personal connections cannot be ignored when deploying an app in a novel setting158, 210, 211.

Recipients of innovation: clinical and clinician factors

While clinicians are aware of the potential benefits of DHTs, they are also concerned about several factors, ranging from safety of apps to therapeutic alliance rupture212, 213. Furthermore, the rapidity of developments of digital mental health technologies represents a challenge to clinicians. Medical education programs often do not cover digital mental health, and many clinicians are left without the resources to utilize the newest innovations. Educational efforts focusing on the clinical workforce are now emerging126.

On the other hand, it may be necessary to consider in this area a new team member analogous to the radiology or pathology technologist, a “digital navigator” who is able to support both the clinician and the patient in implementing digital technology into clinical care127. The role of this digital navigator will include helping to match patients to the useful apps, helping set up and trouble shoot technology, assisting the patient with customizing the technology based on clinical needs, offering support for continued use, and summarizing data for presentation to both the clinician and the patient. Another version of this concept is the “coach”, who is more patient‐facing and often employed to drive engagement214.

Alongside workforce considerations, the positioning of DHTs within the clinical workflow must also be determined. While downloading an app onto a smartphone is relatively simple, recent reviews highlight many challenges in implementing an app into the clinical workflow in this way191, 215. Solving these issues is a high priority and significant challenge216. New workflow considerations were critical to designing a digital psychiatry clinic in Boston in order to ensure that apps were a core part of treatment217.

In the US, the Kaiser Permanente Health system reported that, through offering training to clinicians on using apps in the context of care relationships, the number of referrals to using a mental health app doubled from 20,000 in January 2020 to over 44,000 in May 2020218. The US Veterans Administration has outlined best practices for use of apps and offered training and resources to help integrate its suite of apps into care settings219.

As so few DHTs discussed in this paper have been implemented into real‐world care, workflow considerations remain among the least explored but most needed factors towards facilitating implementation. DHTs offering immediate feedback for patients, medical record integration for clinicians, and data portals for administrators hold potential for better fitting population health needs.

Contextual factors

The COVID‐19 pandemic has already transformed the context for telehealth and DHTs. While the various governments across the world have opted for different approaches, common aspects have been temporary increases of reimbursement for telepsychiatry, reduction in some licensing requirements, and waiving of certain liability concerns. It remains to be seen whether these regulatory changes will become permanent, and the extent to which this recently increased use of telepsychiatry can extend to DHTs such as smartphone apps, chatbots, virtual reality, and social media. Taking examples and adopting models from other areas of health care may speed up the process of building appropriate regulatory frameworks. For example, remote monitoring in cardiology is well established and regulated already, with appropriate reimbursement models in place220.

One area where the context for DHTs has made less recent progress is that of trust. Trust from both patients and clinicians around DHTs remains low, especially with respect to sharing data with companies221. Lax privacy regulations as well as their limited enforcement222, 223 further limit trust. A 2021 report by the magazine Consumer Reports highlighted numerous privacy policy flaws in popular mental health apps224, underscoring that progress around privacy enforcement/legislation remains lacking.

Compounding this, the amount of misinformation about apps continues to present challenges to both clinicians and patients in evaluating their risks as well as benefits225. Transforming the image of DHTs from the current “lawless wild west” will require advancing evidence but also censuring false claims that obscure actual evidence‐based tools which patients and clinicians should feel comfortable using today. While evolving regulatory approaches will help bring order and trust, an important step will be general education about risks and benefits for both clinicians and patients. Such education programs are emerging127, 198, 226 and will continue to evolve.

Not all regulatory hurdles have been reduced by the COVID‐19 pandemic. Many DHTs continue to live outside of any effective regulation by declaring themselves a wellness device (rather than a medical device). With so many thousands of apps and emerging virtual reality, artificial intelligence, and chatbot programs, it is clear that new regulatory approaches are necessary. In the US, there is an ongoing pilot testing of a novel regulatory framework called “pre‐certification”, which would move much of the regulatory burden towards self‐certification by the technology developers. Such a system is not without critics, and its utility in the mental health field remains unclear227. Already some groups have raised concerns about regulations for new apps – such as those around substance use disorders that are approved for use only in conjunction with medication assisted treatment, but not psychotherapy – which exclude non‐prescribing mental health professionals228. Other countries are also looking for new ways to regulate DHTs, with developing policies from Australia, the UK, and the European Union229.

The app marketplaces themselves serve as another source of informal regulation around DHTs. Today the commercial app stores, namely the Apple iTunes and Android Google Play stores, have a role in advertising claims, privacy protections, and payment models around app‐based DHTs. Entrepreneurial investing and startups have also become a proving ground for new DHTs. The aptly entitled paper published in this journal in 2016, Tech Giants Enter Mental Health 230, was a harbinger for increasing investment from venture capital, entrepreneurs, and a wave of startups in digital mental health. However, understanding the value of DHTs, and their related companies, remains elusive, as the necessary data around engagement, effect sizes, necessary dose, and duration of effect remain unknown for almost all DHTs231.

Still, funding has continued to grow, with the marketplace of investors now focused on DHTs that can offer a sustainable business model. This has fueled trends in DHTs that focus on the needs of employees (for employer payers) or offer traditional telehealth services, such as Internet‐delivered therapies, which have been found to be viable and cost‐effective for improving several mental health outcomes232. Understanding the cost‐effectiveness of DHTs will likely become the new point of competition for companies, as the markets begin to saturate with new product offerings.

The technical integration of DHT data presents a final challenge related to contextual factors. The digital information that is eponymous to DHTs is only as useful as it can integrate across devices, networks, and health care settings. Yet, most apps today do not draw on data from existing medical records, and predictive models based on social media or app data (active or passive) are not routinely integrated into the clinical visit or history, largely because of technical integration challenges in sharing data between devices and systems. The Substitutable Medical Applications and Reusable Technologies (SMART) on Fast Healthcare Interoperability Resources (FHIR), often called SMART on FHIR, has emerged as the likely standard that can ensure privacy as well as interoperability.

A related challenge is that, even with such standards in place, utilization cannot be assumed. Barriers must be overcome at the level of patients, clinicians and systems, with one recent study showing that only 1% of patients at a large hospital chose to link their app data with their medical record233. In April 2021, new rules to limit information blocking have taken effect in the US, and even mental health notes must now be electronically shared with patients.

As more mental health data become easily available, interoperability will be even more critical in the DHT ecosystem. Creating new DHTs that are able to comply with and interface with different data systems is an important first step towards building the next generation of useful technologies.

Recommendations around implementation

Considering all the issues discussed in this section, it becomes clear that the main limiting factors of digital psychiatry are not the technologies or innovation themselves, but rather the challenges related to priming the recipients (i.e., patients/clinicians) and the context of health care delivery (e.g., regulation). Therefore, the most immediate benefits in the field could be realized through making effective and ethical use of existing technologies in real‐world settings. While it is unlikely that there will be a single solution to these implementation challenges, various options can be considered depending upon local conditions and clinical needs.

Using a recent app evaluation model7, 234 as a scaffold, we put forward the following recommendations around high‐priority opportunities for advancing the field:

  • Privacy and security. Without a renewed focus on privacy and protecting users’ data, DHTs will lack the trust necessary for uptake. Across all conditions and technologies, ensuring privacy remains critical. Co‐producing digital solutions with end users, starting with the fundamentals around data use, is critical.

  • Efficacy. Increasing evidence shows that DHTs are feasible and acceptable to those with mental health problems. Likewise, efficacy studies suggest that, under ideal research conditions, DHTs can offer benefit and have clinical utility. As DHTs seek reimbursement or addition into national formularies, the need for high‐quality effectiveness data can no longer be ignored. High‐quality studies that compare DHTs to active control or placebo groups are required to support this.

  • Engagement. Downloads are a poor proxy for app engagement. Available data suggest that engagement remains limited across all apps. Augmenting app use with human support appears to offer one solution to sustain engagement, though this detracts from apps’ potential to offer scalable and affordable solutions to health care access. Research on why people use DHTs, and how best to encourage sufficient engagement, is necessary.

  • Clinical integration. Integration of DHTs into clinical practice is feasible, but remains cumbersome. Creating new “digital” clinical services and rethinking care models is necessary to realize the full benefit of DHTs. Advances in digital health standards, policies and regulation are more feasible in the post‐COVID‐19 era, and the field must be prepared to offer viable solutions.

These recommendations apply across all DHTs, but there are special considerations for each technology that are summarized in Table 1. Understanding the future potential, key issues, and priority actions is most productive in the light of the above discussion of challenges concerning recipients and context.

Table 1.

Summary points related to common digital health technologies in mental health

Technology Main uses Future potential Key issues Priority actions
Digitally‐delivered ­psychological therapies Self‐management of ­symptoms of depression and anxiety Precision interventions; ­preventive treatments Lack of engagement; saturated consumer marketplace; claims outpacing clinical evidence Establishing evidence base for use in people with diagnosed mental disorders
Smartphone data (active + passive) Tracking mood and lifestyle in people with major ­depression, bipolar ­disorder and psychosis Machine learning towards ­individualized risk prediction and delivery of targeted “just in time” interventions Lack of validation across ­studies; establishing trust around data usage Data standards for interoperability and validation; industry‐academic partnerships around access
Social media Population level monitoring of mood and anxiety Real time monitoring of mental health state; ­accessible peer support Sampling bias; access to data from social media companies; privacy Industry‐academic partnerships and privacy standards
Virtual reality Exposure therapies Higher engagement and potentially higher efficacy than apps Increased accessibility Low‐cost headsets; expanded ­clinical targets
Chatbots Increased access to care Limited range of appropriate responses Establishing evidence base for use in people with diagnosed mental disorders

CONCLUSIONS

The role of the Internet and digital technologies in providing wider access to psychological interventions and mental health care has long been discussed. However, only in recent years have the abilities, affordability and accessibility of ubiquitous Internet devices (particularly smartphones) advanced to such a point as to allow digital psychiatry to move from a theoretical concept to a realistic option for augmenting traditional mental health care globally.

The development of related technologies, including artificial intelligence and machine learning algorithms, chatbots, and virtual reality, alongside empirical research on the utility of each within mental health contexts, has presented a number of promising avenues. The uptake of this has further been accelerated by the COVID‐19 pandemic, which has highlighted how digital approaches can offer some level of adaptive care under circumstances where access to in‐person services is precluded.

In terms of DHTs for the clinical management of long‐term mental health problems, there are several lines of research emerging, in multiple different conditions, to support the use of DHTs for individuals to self‐manage their symptoms, as an adjunctive to usual care. Alongside digitally delivered therapies, there has also been progress towards the use of smartphone‐collected data for predicting clinical outcomes or risk of relapse. Future research should aim to combine these two areas, in order to harness the available data to provide timely and targeted remote interventions, termed JITAIs, to prevent relapse and other adverse outcomes235. Finally, the considerable interest and investment in the application of DHTs within child and adolescent mental health should aim to take advantage of young people’s apparent proclivity towards new technologies.

All of the aforementioned advances in both the technologies themselves and research supporting them, however, are not enough to ensure that their potential is realized in real‐world settings. Instead, a number of pitfalls and possibilities surrounding implementation must now be addressed. At the patient level, a better understanding of user engagement with these technologies, and how this relates to benefits observed, is required. At the provider level, improved training for “prescribing” DHTs by the mental health care workforce, clearer expectations of where DHTs should sit within the clinical workflow, and improvements of interoperability for new DHTs within existing systems are all necessary if integration is to be at all possible. At the policy level, further action is required to ensure that clinical regulations are suitably flexible to allow for innovation to be effectively adopted within health care services, while stricter regulations for commercial settings may be needed to protect the public and increase their confidence in these new approaches.

Each of these implementation issues must also be considered and actioned with an understanding of the complex ethical issues surrounding DHTs, and their related data. Overall, it now seems inevitable that digital technologies will change the face of mental health research and treatment. The extent to which these changes are genuinely beneficial for those with mental disorders will depend on equitable access, robust research, and ethical, evidence‐based implementation of these new technologies within global mental health care.

ACKNOWLEDGEMENTS

J. Firth is supported by a University of Manchester Presidential Fellowship (P123958) and a UK Research and Innovation Future Leaders Fellowship (MR/T021780/1). J. Torous and M. Keshavan receive support from the Natalie Mental Health Foundation. J. Torous receives support from the US National Institute of Mental Health (K23MH116130) and the American Psychiatric Association Foundation. S. Bucci is supported by a National Institute for Health Research (NIHR) Research Professorship.

REFERENCES

  • 1.Rehm J, Shield KD. Global burden of disease and the impact of mental and addictive disorders. Curr Psychiatry Rep 2019;21:10. [DOI] [PubMed] [Google Scholar]
  • 2.World Health Organization . Depression. Geneva: World Health Organization, 2020. [Google Scholar]
  • 3.Keynejad RC, Dua T, Barbui C et al. WHO Mental Health Gap Action Programme (mhGAP) Intervention Guide: a systematic review of evidence from low and middle‐income countries. Evid Based Ment Health 2018;21:30‐4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.US Substance Abuse and Mental Health Services Administration . Behavioral health workforce report. Rockville: US Substance Abuse and Mental Health Services Administration, 2020. [Google Scholar]
  • 5.Kinoshita S, Cortright K, Crawford A et al. Changes in telepsychiatry regulations during the COVID‐19 pandemic: 17 countries and regions’ approaches to an evolving healthcare landscape. Psychol Med (in press). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Insel TR. Digital phenotyping: a global tool for psychiatry. World Psychiatry 2018;17:276‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Torous J, Andersson G, Bertagnoli A et al. Towards a consensus around standards for smartphone apps and digital mental health. World Psychiatry 2019;18:97‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Carey B.The therapist may see you anytime, anywhere. New York Times, February 13, 2012. [Google Scholar]
  • 9.Silver L.Smartphone ownership is growing rapidly around the world, but not always equally. Pew Research Center, February 5, 2019. [Google Scholar]
  • 10.Young AS, Cohen AN, Niv N et al. Mobile phone and smartphone use by people with serious mental illness. Psychiatr Serv 2020;71:280‐3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Luther L, Buck BE, Fischer MA et al. Examining potential barriers to mHealth implementation and engagement in schizophrenia: phone ownership and symptom severity. J Technol Behav Sci (in press). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Torous J, Wisniewski H, Liu G et al. Mental health mobile phone app usage, concerns, and benefits among psychiatric outpatients: comparative survey study. JMIR Mental Health 2018;5:e11715. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Lecomte T, Potvin S, Corbière M et al. Mobile apps for mental health issues: meta‐review of meta‐analyses. JMIR mHealth and uHealth 2020;8:e17458. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Targum SD, Sauder C, Evans M et al. Ecological momentary assessment as a measurement tool in depression trials. J Psychiatr Res 2021;136:256‐64. [DOI] [PubMed] [Google Scholar]
  • 15.Chevance A, Ravaud P, Tomlinson A et al. Identifying outcomes for depression that matter to patients, informal caregivers, and health‐care professionals: qualitative content analysis of a large international online survey. Lancet Psychiatry 2020;7:692‐702. [DOI] [PubMed] [Google Scholar]
  • 16.Weizenbaum E, Torous J, Fulford D. Cognition in context: understanding the everyday predictors of cognitive performance in a new era of measurement. JMIR mHealth and uHealth 2020;8:e14328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Lewis S, Ainsworth J, Sanders C et al. Smartphone‐enhanced symptom management in psychosis: open, randomized controlled trial. J Med Int Res 2020;22:e17019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Liu G, Henson P, Keshavan M et al. Assessing the potential of longitudinal smartphone based cognitive assessment in schizophrenia: a naturalistic pilot study. Schizophr Res Cogn 2019;17:100144. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Parrish EM, Kamarsu S, Harvey PD et al. Remote ecological momentary testing of learning and memory in adults with serious mental illness. Schizophr Bull 2021;47:740‐50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Glenn CR, Kleiman EM, Kearns JC et al. Feasibility and acceptability of ecological momentary assessment with high‐risk suicidal adolescents following acute psychiatric care. J Clin Child Adolesc Psychol 2020;3:1‐7. [DOI] [PubMed] [Google Scholar]
  • 21.Torous J, Kiang MV, Lorme J et al. New tools for new research in psychiatry: a scalable and customizable platform to empower data driven smartphone research. JMIR Ment Health 2016;3:e16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Cohen AS, Cox CR, Masucci MD et al. Digital phenotyping using multimodal data. Curr Behav Neurosci Rep 2020;7:212‐20. [Google Scholar]
  • 23.Huckins JF, DaSilva AW, Wang W et al. Mental health and behavior of college students during the early phases of the COVID‐19 pandemic: longitudinal smartphone and ecological momentary assessment study. J Med Int Res 2020;22:e20185. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Wang W, Mirjafari S, Harari G et al. Social sensing: assessing social functioning of patients living with schizophrenia using mobile phone sensing. Presented at the CHI Conference on Human Factors in Computing Systems, Honolulu, April 2020.
  • 25.Fulford D, Mote J, Gonzalez R et al. Smartphone sensing of social interactions in people with and without schizophrenia. J Psychiatr Res 2021;137:613‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Miralles I, Granell C, Díaz‐Sanahuja L et al. Smartphone apps for the treatment of mental disorders: systematic review. JMIR mHealth and uHealth 2020;8:e14897. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Baxter C, Carroll JA, Keogh B et al. Assessment of mobile health apps using built‐in smartphone sensors for diagnosis and treatment: systematic survey of apps listed in international curated health app libraries. JMIR mHealth and uHealth 2020;8:e16741. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Faurholt‐Jepsen M, Busk J, Vinberg M et al. Daily mobility patterns in patients with bipolar disorder and healthy individuals. J Affect Disord 2020;278:413‐22. [DOI] [PubMed] [Google Scholar]
  • 29.Jongs N, Jagesar R, van Haren NE et al. A framework for assessing neuropsychiatric phenotypes by using smartphone‐based location data. Transl Psychiatry 2020;10:211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Henson P, D’Mello R, Vaidyam A et al. Anomaly detection to predict relapse risk in schizophrenia. Transl Psychiatry 2021;11:28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Gillan CM, Rutledge RB. Smartphones and the neuroscience of mental health. Annu Rev Neurosci (in press). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Torous J, Staples P, Shanahan M et al. Utilizing a personal smartphone custom app to assess the Patient Health Questionnaire‐9 (PHQ‐9) depressive symptoms in patients with major depressive disorder. JMIR Mental Health 2015;2:e8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Barnett I, Torous J, Staples P et al. Beyond smartphones and sensors: choosing appropriate statistical methods for the analysis of longitudinal data. J Am Med Inform Assoc 2018;25:1669‐74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Koppe G, Meyer‐Lindenberg A, Durstewitz D. Deep learning for small and big data in psychiatry. Neuropsychopharmacology 2021;46:176‐90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Shatte AB, Hutchinson DM, Teague SJ. Machine learning in mental health: a scoping review of methods and applications. Psychol Med 2019;49:1426‐48. [DOI] [PubMed] [Google Scholar]
  • 36.Thieme A, Belgrave D, Doherty G. Machine learning in mental health: a systematic review of the HCI literature to support the development of effective and implementable ML systems. ACM Trans Comput Hum Interact 2020;27:34. [Google Scholar]
  • 37.Belsher BE, Smolenski DJ, Pruitt LD et al. Prediction models for suicide attempts and deaths: a systematic review and simulation. JAMA Psychiatry 2019;76:642‐51. [DOI] [PubMed] [Google Scholar]
  • 38.Goodday SM, Friend S. Unlocking stress and forecasting its consequences with digital technology. NPJ Digit Med 2019;2:75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Andersson G, Titov N, Dear BF et al. Internet‐delivered psychological treatments: from innovation to implementation. World Psychiatry 2019;18:20‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Mohr DC, Schueller SM, Tomasino KN et al. Comparison of the effects of coaching and receipt of app recommendations on depression, anxiety, and engagement in the IntelliCare platform: factorial randomized controlled trial. J Med Int Res 2019;21:e13609. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Graham AK, Greene CJ, Kwasny MJ et al. Coached mobile app platform for the treatment of depression and anxiety among primary care patients: a randomized clinical trial. JAMA Psychiatry 2020;77:906‐14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Arean PA, Hallgren KA, Jordan JT et al. The use and effectiveness of mobile apps for depression: results from a fully remote clinical trial. J Med Int Res 2016;18:e330. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.de Girolamo G, Barattieri di San Pietro C, Bulgari V et al. The acceptability of real‐time health monitoring among community participants with depression: a systematic review and meta‐analysis of the literature. Depress Anxiety 2020;37:885‐97. [Google Scholar]
  • 44.Kidd SA, Feldcamp L, Adler A et al. Feasibility and outcomes of a multi‐function mobile health approach for the schizophrenia spectrum: App4Independence (A4i). PLoS One 2019;14:e0219491. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Ben‐Zeev D, Brian RM, Jonathan G et al. Mobile health (mHealth) versus clinic‐based group intervention for people with serious mental illness: a randomized controlled trial. Psychiatr Serv 2018;69:978‐85. [DOI] [PubMed] [Google Scholar]
  • 46.Porras‐Segovia A, Díaz‐Oliván I, Gutiérrez‐Rojas L et al. Apps for depression: are they ready to work? Curr Psychiatry Rep 2020;22:11. [DOI] [PubMed] [Google Scholar]
  • 47.Marshall JM, Dunstan DA, Bartik W. Clinical or gimmickal: the use and effectiveness of mobile mental health apps for treating anxiety and depression. Aust N Z J Psychiatry 2020;54:20‐8. [DOI] [PubMed] [Google Scholar]
  • 48.Nunes A, Castro SL, Limpo T. A review of mindfulness‐based apps for children. Mindfulness 2020;11:2089‐101. [Google Scholar]
  • 49.Temkin AB, Schild J, Falk A et al. Mobile apps for youth anxiety disorders: a review of the evidence and forecast of future innovations. Prof Psychol Res Pract 2020;51:400‐13. [Google Scholar]
  • 50.Goreis A, Felnhofer A, Kafka JX et al. Efficacy of self‐management smartphone‐based apps for post‐traumatic stress disorder symptoms: a systematic review and meta‐analysis. Front Neurosci 2020;14:3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Weisel KK, Fuhrmann LM, Berking M et al. Standalone smartphone apps for mental health – a systematic review and meta‐analysis. NPJ Digit Med 2019;2:118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Nahum‐Shani I, Smith SN, Spring BJ et al. Just‐in‐time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med 2018;52:446‐62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Wang L, Miller LC. Just‐in‐the‐moment adaptive interventions (JITAI): a meta‐analytical review. Health Commun 2020;35:1531‐44. [DOI] [PubMed] [Google Scholar]
  • 54.Baumel A, Torous J, Edan S et al. There is a non‐evidence‐based app for that: a systematic review and mixed methods analysis of depression‐ and anxiety‐related apps that incorporate unrecognized techniques. J Affect Disord 2020;273:410‐21. [DOI] [PubMed] [Google Scholar]
  • 55.Lau N, O’Daffer A, Colt S et al. Android and iPhone mobile apps for psychosocial wellness and stress management: systematic search in app stores and literature review. JMIR mHealth and uHealth 2020;8:e17798. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Singer N.In screening for suicide risk, Facebook takes on tricky public health role. New York Times, December 31, 2018. [Google Scholar]
  • 57.Aschbrenner KA, Naslund JA, Tomlinson EF et al. Adolescents’ use of digital technologies and preferences for mobile health coaching in public mental health settings. Front Publ Health 2019;7:178. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Davidson BI, Shaw H, Ellis DA. Fuzzy constructs: the overlap between mental health and technology “use”. PsyArXiv 2020;10.31234. [Google Scholar]
  • 59.Jensen M, George MJ, Russell MR et al. Young adolescents’ digital technology use and mental health symptoms: little evidence of longitudinal or daily linkages. Clin Psychol Sci 2019;7:1416‐33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Vogel L.Quality of kids’ screen time matters as much as quantity. CMAJ 2019;191:E721. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Firth J, Torous J, Stubbs B et al. The “online brain”: how the Internet may be changing our cognition. World Psychiatry 2019;18:119‐29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Birnbaum ML, Ernala SK, Rizvi AF et al. Detecting relapse in youth with psychotic disorders utilizing patient‐generated and patient‐contributed digital data from Facebook. NPJ Schizophr 2019;5:17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Saha K, Torous J, Caine ED et al. Psychosocial effects of the COVID‐19 pandemic: large‐scale quasi‐experimental study on social media. J Med Int Res 2020;22:e22600. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Sarker A, Lakamana S, Hogg‐Bremer W et al. Self‐reported COVID‐19 symptoms on Twitter: an analysis and a research resource. J Am Med Inform Assoc 2020;27:1310‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Low DM, Rumker L, Talkar T et al. Natural language processing reveals vulnerable mental health support groups and heightened health anxiety on reddit during COVID‐19: observational study. J Med Int Res 2020;22:e22635. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Saha K, Sugar B, Torous J et al. A social media study on the effects of psychiatric medication use. Presented at the International AAAI Conference on Web and Social Media, Munich, July 2019. [PMC free article] [PubMed]
  • 67.Baker JT, Pennant L, Baltrušaitis T et al. Toward expert systems in mental health assessment: a computational approach to the face and voice in dyadic patient‐doctor interactions. Iproceedings 2016;2:e6136. [Google Scholar]
  • 68.Reece AG, Danforth CM. Instagram photos reveal predictive markers of depression. EPJ Data Sci 2017;6:1‐2.32355601 [Google Scholar]
  • 69.Odgers CL, Jensen MR. Annual research review: Adolescent mental health in the digital age: facts, fears, and future directions. J Child Psychol Psychiatry 2020;61:336‐48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Abi‐Jaoude E, Naylor KT, Pignatiello A. Smartphones, social media use and youth mental health. CMAJ 2020;192:E136‐41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Alvarez‐Jimenez M, Rice S, D’Alfonso S et al. A novel multimodal digital service (Moderated Online Social Therapy+) for help‐seeking young people experiencing mental ill‐health: pilot evaluation within a national youth e‐mental health service. J Med Int Res 2020;22:e17155. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Schlosser DA, Campellone TR, Truong B et al. Efficacy of PRIME, a mobile app intervention designed to improve motivation in young people with schizophrenia. Schizophr Bull 2018;44:1010‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.D’Alfonso S, Phillips J, Valentine L et al. Moderated online social therapy: viewpoint on the ethics and design principles of a web‐based therapy system. JMIR Ment Health 2019;6:e14866. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Hao K.Nearly half of Twitter accounts pushing to reopen America may be bots. MIT Technol Rev 2020:6‐25. [Google Scholar]
  • 75.Robinson P, Turk D, Jilka S et al. Measuring attitudes towards mental health using social media: investigating stigma and trivialisation. Soc Psychiatry Psychiatr Epidemiol 2019;54:51‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Parrott S, Billings AC, Hakim SD et al. From #endthestigma to #realman: stigma‐challenging social media responses to NBA players’ mental health disclosures. Commun Rep 2020;33:148‐60. [Google Scholar]
  • 77.Chancellor S, Birnbaum ML, Caine ED et al. A taxonomy of ethical tensions in inferring mental health states from social media. Presented at the Conference on Fairness, Accountability, and Transparency, Atlanta, January 2019.
  • 78.Fiske A, Henningsen P, Buyx A. Your robot therapist will see you now: ethical implications of embodied artificial intelligence in psychiatry, psychology, and psychotherapy. J Med Int Res 2019;21:e13216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Abd‐Alrazaq AA, Alajlani M, Ali N et al. Perceptions and opinions of patients about mental health chatbots: scoping review. J Med Int Res 2021;23:e17828. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Henson P, Wisniewski H, Hollis C et al. Digital mental health apps and the therapeutic alliance: initial review. BJPsych Open 2019;5:e15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81.Tremain H, McEnery C, Fletcher K et al. The therapeutic alliance in digital mental health interventions for serious mental illnesses: narrative review. JMIR Ment Health 2020;7:e17204. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Frank AF, Gunderson JG. The role of the therapeutic alliance in the treatment of schizophrenia. Relationship to course and outcome. Arch Gen Psychiatry 1990;47:228‐36. [DOI] [PubMed] [Google Scholar]
  • 83.Lucas GM, Gratch J, King A et al. It’s only a computer: virtual humans increase willingness to disclose. Comp Hum Behav 2014;37:94‐100. [Google Scholar]
  • 84.Martínez‐Miranda J, Martínez A, Ramos R et al. Assessment of users’ acceptability of a mobile‐based embodied conversational agent for the prevention and detection of suicidal behaviour. J Med Syst 2019;43:1‐8. [DOI] [PubMed] [Google Scholar]
  • 85.Laranjo L, Dunn AG, Tong HL et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc 2018;25:1248‐58. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 86.Vaidyam AN, Linggonegoro D, Torous J. Changes to the psychiatric chatbot landscape: a systematic review of conversational agents in serious mental illness. Can J Psychiatry 2021;66:339‐48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.Miner AS, Milstein A, Schueller S et al. Smartphone‐based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA Intern Med 2016;176:619‐25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.Vaidyam AN, Wisniewski H, Halamka JD et al. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry 2019;64:456‐64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 89.Bell IH, Nicholas J, Alvarez‐Jimenez M et al. Virtual reality as a clinical tool in mental health research and practice. Dialog Clin Neurosci 2020;22:169‐77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 90.Freeman D, Reeve S, Robinson A et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol Med 2017;47:2393‐400. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 91.Valmaggia LR, Latif L, Kempton MJ et al. Virtual reality in the psychological treatment for mental health problems: a systematic review of recent evidence. Psychiatry Res 2016;236:189‐95. [DOI] [PubMed] [Google Scholar]
  • 92.Kessler R, Wittchen H, Abelson J et al. Methodological issues in assessing psychiatric disorders with self‐reports. In: Stone AA, Turkkan JS, Bachrach CA et al (eds). The science of self‐report: implications for research and practice. Mahwah: Erlbaum, 2020:229‐55. [Google Scholar]
  • 93.Parsons TD. Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front Hum Neu­rosci 2015;9:660. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 94.Maples‐Keller JL, Bunnell BE, Kim SJ et al. The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders. Harv Rev Psychiatry 2017;25:103‐13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 95.Freeman D, Bradley J, Antley A et al. Virtual reality in the treatment of persecutory delusions: randomised controlled experimental study testing how to reduce delusional conviction. Br J Psychiatry 2016;209:62‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 96.Pot‐Kolder RM, Geraets CN, Veling W et al. Virtual‐reality‐based cognitive behavioural therapy versus waiting list control for paranoid ideation and social avoidance in patients with psychotic disorders: a single‐blind randomised controlled trial. Lancet Psychiatry 2018;5:217‐26. [DOI] [PubMed] [Google Scholar]
  • 97.Dellazizzo L, Potvin S, Luigi M et al. A. Evidence on virtual reality‐based therapies for psychiatric disorders: meta‐review of meta‐analyses. J Med Int Res 2020;22:e20889. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 98.De Carvalho MR, Dias TR, Duchesne M et al. Virtual reality as a promising strategy in the assessment and treatment of bulimia nervosa and binge eating disorder: a systematic review. Behav Sci 2017;7:43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 99.Howard MC, Gutworth MB. A meta‐analysis of virtual reality training programs for social skill development. Comp Educ 2020;144:103707. [Google Scholar]
  • 100.Chandrasiri A, Collett J, Fassbender E et al. A virtual reality approach to mindfulness skills training. Virtual Real 2020;24:143‐9. [Google Scholar]
  • 101.Navarro‐Haro MV, Modrego‐Alarcón M, Hoffman HG et al. Evaluation of a mindfulness‐based intervention with and without virtual reality dialectical behavior therapy® mindfulness skills training for the treatment of generalized anxiety disorder in primary care: a pilot study. Front Psychol 2019;10:55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 102.Seabrook E, Kelly R, Foley F et al. Understanding how virtual reality can sup­port mindfulness practice: mixed methods study. J Med Int Res 2020;22:e16106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 103.Veling W, Lestestuiver B, Jongma M et al. Virtual reality relaxation for patients with a psychiatric disorder: crossover randomized controlled trial. J Med Int Res 2021;23:e17233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 104.Brown P, Waite F, Rovira A et al. Virtual reality clinical‐experimental tests of compassion treatment techniques to reduce paranoia. Sci Rep 2020;10:8547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 105.Falconer CJ, Rovira A, King JA et al. Embodying self‐compassion within virtual reality and its effects on patients with depression. BJPsych Open 2016;2:74‐80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 106.Realpe A, Elahi F, Bucci S et al. Co‐designing a virtual world with young people to deliver social cognition therapy in early psychosis. Early Interv Psychiatry 2020;14:37‐43. [DOI] [PubMed] [Google Scholar]
  • 107.Thompson A, Elahi F, Realpe A et al. A feasibility and acceptability trial of social cognitive therapy in early psychosis delivered through a virtual world: the VEEP study. Front Psychiatry 2020; 25;11:219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 108.Fortune Business Insights . Virtual reality market size, share & industry analysis. https://www.fortunebusinessinsights.com.
  • 109.Vigo D, Thornicroft G, Atun R. Estimating the true global burden of mental illness. Lancet Psychiatry 2016;3:171‐8. [DOI] [PubMed] [Google Scholar]
  • 110.Drissi N, Ouhbi S, Janati Idrissi MA et al. An analysis on self‐management and treatment‐related functionality and characteristics of highly rated anxiety apps. Int J Med Inform 2020;141:104243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 111.Marshall J, Dunstan D, Bartik W. Apps with maps – anxiety and depression mobile apps with evidence‐based frameworks: systematic search of major app stores. JMIR Mental Health 2020;7:e16525. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 112.Linardon J, Cuijpers P, Carlbring P et al. The efficacy of app‐supported smartphone interventions for mental health problems: a meta‐analysis of ran­domized controlled trials. World Psychiatry 2019;18:325‐36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 113.Bakker D, Kazantzis N, Rickwood D et al. Mental health smartphone apps: review and evidence‐based recommendations for future developments. JMIR Mental Health 2016;3:e4984. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 114.Wasil AR, Gillespie S, Shingleton R et al. Examining the reach of smartphone apps for depression and anxiety. Am J Psychiatry 2020;177:464‐5. [DOI] [PubMed] [Google Scholar]
  • 115.Jaworski BK, Taylor K, Ramsey KM et al. Exploring usage of COVID Coach, a public mental health app designed for the COVID‐19 pandemic: evaluation of analytics data. J Med Internet Res 2021;23:e26559. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 116.Firth J, Torous J, Nicholas J et al. Can smartphone mental health interventions reduce symptoms of anxiety? A meta‐analysis of randomized controlled trials. J Affect Disord 2017;218:15‐22. [DOI] [PubMed] [Google Scholar]
  • 117.Wasil AR, Venturo‐Conerly KE, Shingleton RM et al. A review of popular smartphone apps for depression and anxiety: assessing the inclusion of evidence‐based content. Behav Res Ther 2019;123:103498. [DOI] [PubMed] [Google Scholar]
  • 118.Liu JY, Xu KK, Zhu GL et al. Effects of smartphone‐based interventions and monitoring on bipolar disorder: a systematic review and meta‐analysis. World J Psychiatry 2020;10:272‐285. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 119.Depp CA, Ceglowski J, Wang VC et al. Augmenting psychoeducation with a mobile intervention for bipolar disorder: a randomized controlled trial. J Affect Disord 2015;174:23‐30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 120.Faurholt‐Jepsen M, Frost M, Ritz C et al. Daily electronic self‐monitoring in bipolar disorder using smartphones – the MONARCA I trial: a ran­domized, placebo‐controlled, single‐blind, parallel group trial. Psychol Med 2015;45:2691‐704. [DOI] [PubMed] [Google Scholar]
  • 121.Faurholt‐Jepsen M, Frost M, Christensen EM et al. The effect of smartphone‐based monitoring on illness activity in bipolar disorder: the MONARCA II randomized controlled single‐blinded trial. Psychol Med 2020;50:838‐48. [DOI] [PubMed] [Google Scholar]
  • 122.Ly KH, Trüschel A, Jarl L et al. Behavioural activation versus mindfulness‐based guided self‐help treatment administered through a smartphone application: a randomised controlled trial. BMJ Open 2014;4:e003440. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 123.Ly KH, Topooco N, Cederlund H et al. Smartphone‐supported versus full behavioural activation for depression: a randomised controlled trial. PLoS One 2015;10:e0126559. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 124.Watts S, Mackenzie A, Thomas C et al. CBT for depression: a pilot RCT comparing mobile phone vs. computer. BMC Psychiatry 2013;13:1‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 125.Roberts AE, Davenport TA, Wong T et al. Evaluating the quality and safety of health‐related apps and e‐tools: adapting the Mobile App Rating Scale and developing a quality assurance protocol. Internet Interv 2021;24:100379. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 126.Wisniewski H, Gorrindo T, Rauseo‐Ricupero N et al. The role of digital navigators in promoting clinical care and technology integration into practice. Digit Biomark 2020;4(Suppl. 1):119‐35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 127.Wisniewski H, Torous J. Digital navigators to implement smartphone and digital tools in care. Acta Psychiatr Scand 2020;141:350‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 128.Rauseo‐Ricupero N, Henson P, Agate‐Mays M et al. Case studies from the digital clinic: integrating digital phenotyping and clinical practice into today’s world. Int Rev Psychiatry (in press). [DOI] [PubMed] [Google Scholar]
  • 129.Colombo D, Palacios AG, Alvarez JF et al. Current state and future directions of technology‐based ecological momentary assessments and interventions for major depressive disorder: protocol for a systematic review. Syst Rev 2018;7:233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 130.Dogan E, Sander C, Wagner X et al. Smartphone‐based monitoring of objective and subjective data in affective disorders: where are we and where are we going? Systematic review. J Med Internet Res 2017;19:e262. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 131.Beiwinkel T, Kindermann S, Maier A et al. Using smartphones to monitor bipolar disorder symptoms: a pilot study. JMIR Mental Health 2016;3:e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 132.Faurholt‐Jepsen M, Vinberg M, Frost M et al. Behavioral activities collected through smartphones and the association with illness activity in bipolar disorder. Int J Methods Psychiatr Res 2016;25:309‐23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 133.Faurholt‐Jepsen M, Vinberg M, Frost M et al. Smartphone data as an electronic biomarker of illness activity in bipolar disorder. Bipolar Disord 2015;17:715‐28. [DOI] [PubMed] [Google Scholar]
  • 134.Faurholt‐Jepsen M, Busk J, Frost M et al. Voice analysis as an objective state marker in bipolar disorder. Transl Psychiatry 2016;6:e856. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 135.Antosik‐Wójcińska AZ, Dominiak M, Chojnacka M et al. Smartphone as a monitoring tool for bipolar disorder: a systematic review including data analysis, machine learning algorithms and predictive modelling. Int J Med Inform 2020;138:104131. [DOI] [PubMed] [Google Scholar]
  • 136.Stanislaus S, Faurholt‐Jepsen M, Vinberg M et al. Mood instability in patients with newly diagnosed bipolar disorder, unaffected relatives, and healthy control individuals measured daily using smartphones. J Affect Disord 2020;271:336‐44. [DOI] [PubMed] [Google Scholar]
  • 137.Yim SJ, Lui LM, Lee Y et al. The utility of smartphone‐based, ecological momentary assessment for depressive symptoms. J Affect Disord 2020;274:602‐9. [DOI] [PubMed] [Google Scholar]
  • 138.Burns MN, Begale M, Duffecy J et al. Harnessing context sensing to develop a mobile intervention for depression. J Med Internet Res 2011;13:e55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 139.Faurholt‐Jepsen M, Tønning ML, Frost M et al. Reducing the rate of psychiatric Re‐ADMISsions in bipolar disorder using smartphones – The RADMIS trial. Acta Psychiatr Scand 2021;143:453‐65. [DOI] [PubMed] [Google Scholar]
  • 140.Tønning ML, Faurholt‐Jepsen M, Frost M et al. The effect of smartphone‐based monitoring and treatment on the rate and duration of psychiatric readmission in patients with unipolar depressive disorder: the RADMIS randomized controlled trial. J Affect Disord 2021;282:354‐63. [DOI] [PubMed] [Google Scholar]
  • 141.Hidalgo‐Mazzei D, Mateu A, Reinares M et al. Self‐monitoring and psychoeducation in bipolar patients with a smart‐phone application (SIMPLe) project: design, development and studies protocols. BMC Psychiatry 2015;15:1‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 142.Ritter PS, Bermpohl F, Gruber O et al. Aims and structure of the German Research Consortium BipoLife for the study of bipolar disorder. Int J Bipolar Disord 2016;4:26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 143.Saunders KE, Cipriani A, Rendell J et al. Oxford Lithium Trial (OxLith) of the early affective, cognitive, neural and biochemical effects of lithium carbonate in bipolar disorder: study protocol for a randomised controlled trial. Trials 2016;17:1‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 144.Kessing LV, Munkholm K, Faurholt‐Jepsen J et al. The Bipolar Illness Onset study: research protocol for the BIO cohort study. BMJ Open 2017;7:e015462. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 145.Bucci S, Schwannauer M, Berry N. The digital revolution and its impact on mental health care. Psychol Psychother 2019;92:277‐97. [DOI] [PubMed] [Google Scholar]
  • 146.Chivilgina O, Wangmo T, Elger BS et al. mHealth for schizophrenia spectrum disorders management: a systematic review. Int J Soc Psychiatry 2020;66:642‐65. [DOI] [PubMed] [Google Scholar]
  • 147.Barnett I, Torous J, Staples P et al. Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology 2018;43:1660‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 148.Adler DA, Ben‐Zeev D, Tseng VW et al. Predicting early warning signs of psychotic relapse from passive sensing data: an approach using encoder‐decoder neural networks. JMIR Mhealth and Uhealth 2020;8:e19962. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 149.Ben‐Zeev D, Brian R, Wang R et al. CrossCheck: integrating self‐report, behavioral sensing, and smartphone use to identify digital indicators of psychotic relapse. Psychiatr Rehabil J 2017;40:266‐75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 150.Wisniewski H, Henson P, Torous J. Using a smartphone app to identify clinically relevant behavior trends via symptom report, cognition scores, and exercise levels: a case series. Front Psychiatry 2019;10:652. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 151.Cella M, He Z, Killikelly C et al. Blending active and passive digital technology methods to improve symptom monitoring in early psychosis. Early Interv Psychiatry 2019;13:1271‐5. [DOI] [PubMed] [Google Scholar]
  • 152.Clarke S, Hanna D, Mulholland C et al. A systematic review and meta‐analysis of digital health technologies effects on psychotic symptoms in adults with psychosis. Psychosis 2019;11:362‐73. [Google Scholar]
  • 153.Bucci S, Barrowclough C, Ainsworth J et al. Actissist: proof‐of‐concept trial of a theory‐driven digital intervention for psychosis. Schizophr Bull 2018;44:1070‐80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 154.Garety P, Ward T, Emsley R et al. Effects of SlowMo, a blended digital therapy targeting reasoning, on paranoia among people with psychosis: a ran­domized clinical trial. JAMA Psychiatry (in press). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 155.Berry N, Machin M, Ainsworth J et al. Developing a theory‐informed smartphone app for early psychosis: learning points from a multidisciplinary collaboration. Front Psychiatry 2020;11:602861. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 156.Gumley A, Bradstreet S, Ainsworth J et al. Early signs monitoring to prevent relapse in psychosis and promote well‐being, engagement, and recovery: protocol for a feasibility cluster randomized controlled trial harnessing mobile phone technology blended with peer support. JMIR Res Protoc 2020;9:e15058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 157.Mohr DC, Lyon AR, Lattie EG et al. Accelerating digital mental health research from early design and creation to successful implementation and sustainment. J Med Internet Res 2017;19:e153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 158.Greenhalgh T, Wherton J, Papoutsi C et al. Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework. BMC Med 2018;16:66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 159.Camacho E, Levin L, Torous J. Smartphone apps to support coordinated specialty care for prodromal and early course schizophrenia disorders: systematic review. J Med Internet Res 2019;21:e16393. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 160.Bardone‐Cone AM, Thompson KA, Miller AJ. The self and eating disorders. J Pers 2020;88:59‐75. [DOI] [PubMed] [Google Scholar]
  • 161.Halmi KA. Perplexities of treatment resistence in eating disorders. BMC Psychiatry 2013;13:292. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 162.Weissman RS, Rosselli F. Reducing the burden of suffering from eating disorders: unmet treatment needs, cost of illness, and the quest for cost‐effectiveness. Behav Res Ther 2017;88:49‐64. [DOI] [PubMed] [Google Scholar]
  • 163.Linardon J, Shatte A, Tepper H et al. A survey study of attitudes toward, and preferences for, e‐therapy interventions for eating disorder psychopathology. Int J Eat Disord 2020;53:907‐16. [DOI] [PubMed] [Google Scholar]
  • 164.Linardon J, Messer M, Lee S et al. Perspectives of e‐health interventions for treating and preventing eating disorders: descriptive study of perceived advantages and barriers, help‐seeking intentions, and preferred functionality. Eat Weight Disord 2021;26:1097‐109. [DOI] [PubMed] [Google Scholar]
  • 165.Juarascio AS, Manasse SM, Goldstein SP et al. Review of smartphone applications for the treatment of eating disorders. Eur Eat Disord Rev 2015;23:1‐11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 166.Fairburn CG, Rothwell ER. Apps and eating disorders: a systematic clinical appraisal. Int J Eat Disord 2015;48:1038‐46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 167.Wasil AR, Patel R, Cho JY et al. Smartphone apps for eating disorders: a systematic review of evidence‐based content and application of user‐adjusted analyses. Int J Eat Disord 2021;54:690‐700. [DOI] [PubMed] [Google Scholar]
  • 168.Linardon J, Shatte A, Messer M et al. E‐mental health interventions for the treatment and prevention of eating disorders: an updated systematic review and meta‐analysis. J Consult Clin Psychol 2020;88:994‐1007. [DOI] [PubMed] [Google Scholar]
  • 169.Linardon J, Shatte A, Rosato J et al. Efficacy of a transdiagnostic cognitive‐behavioral intervention for eating disorder psychopathology delivered through a smartphone app: a randomized controlled trial. Psychol Med (in press). [DOI] [PubMed] [Google Scholar]
  • 170.Wilson GT, Fairburn CC, Agras WS et al. Cognitive‐behavioral therapy for bulimia nervosa: time course and mechanisms of change. J Consult Clin Psychol 2002;70:267‐74. [PubMed] [Google Scholar]
  • 171.Sivyer K, Allen E, Cooper Z et al. Mediators of change in cognitive behavior therapy and interpersonal psychotherapy for eating disorders: a secondary analysis of a transdiagnostic randomized controlled trial. Int J Eat Disord 2020;53:1928‐40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 172.Juarascio AS, Parker MN, Lagacey MA et al. Just‐in‐time adaptive interventions: a novel approach for enhancing skill utilization and acquisition in cognitive behavioral therapy for eating disorders. Int J Eat Disord 2018;51:826‐30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 173.Hildebrandt T, Michaeledes A, Mayhew M et al. Randomized controlled trial comparing health coach‐delivered smartphone‐guided self‐help with standard care for adults with binge eating. Am J Psychiatry 2020;177:134‐42. [DOI] [PubMed] [Google Scholar]
  • 174.Hildebrandt T, Michaelides A, Mackinnon D et al. Randomized controlled trial comparing smartphone assisted versus traditional guided self‐help for adults with binge eating. Int J Eat Disord 2017;50:1313‐22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 175.Hendershot CS, Witkiewitz K, George WH et al. Relapse prevention for addictive behaviors. Subst Abuse Treat Prev Policy 2011;6:17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 176.Tofighi B, Chemi C, Ruiz‐Valcarcel J et al. Smartphone apps targeting alcohol and illicit substance use: systematic search in in commercial app stores and critical content analysis. JMIR mHealth and uHealth 2019;7:e11831. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 177.Institute for Clinical and Economic Review . Digital therapeutics as an adjunct to medication assisted therapy for opioid use disorder. https://icer.org.
  • 178.Whittaker R, McRobbie H, Bullen C et al. Mobile phone text messaging and app‐based interventions for smoking cessation. Cochrane Database Syst Rev 2019;10:CD006611. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 179.Staiger PK, O’Donnell R, Liknaitzky P et al. Mobile apps to reduce tobacco, alcohol, and illicit drug use: systematic review of the first decade. J Med Internet Res 2020;22:e17156. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 180.Nuamah J, Mehta R, Sasangohar F. Technologies for opioid use disorder management: mobile app search and scoping review. JMIR mHealth and uHealth 2020;8:e15752. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 181.Carreiro S, Newcomb M, Leach R et al. Current reporting of usability and impact of mHealth interventions for substance use disorder: a systematic review. Drug Alcohol Depend 2020;215:108201. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 182.Liao Y, Tang J. Protocol: Efficacy of cognitive behavioural therapy‐based smartphone app for smoking cessation in China: a study protocol of a randomised controlled trial. BMJ Open 2021;11:e041985. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 183.Coughlin LN, Nahum‐Shani I, Philyaw‐Kotov ML et al. Developing an adaptive mobile intervention to address risky substance use among adolescents and emerging adults: usability study. JMIR mHealth and uHealth 2021;9:e24424. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 184.Bricker JB, Watson NL, Mull KE et al. Efficacy of smartphone applications for smoking cessation: a randomized clinical trial. JAMA Intern Med 2020;180:1472‐80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 185.Manning V, Piercy H, Garfield JB et al. Personalized approach bias modification smartphone app (“SWIPE”) to reduce alcohol use among people drinking at hazardous or harmful levels: protocol for an open‐label feasibility study. JMIR Res Protoc 2020;9:e21278. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 186.World Health Organization . Caring for children and adolescents with mental disorders: setting WHO directions. Geneva: World Health Organization, 2003. [Google Scholar]
  • 187.Punukollu M, Marques M. Use of mobile apps and technologies in child and adolescent mental health: a systematic review. Evid Based Ment Health 2019;22:161‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 188.Hollis C, Falconer CJ, Martin JL et al. Annual research review: Digital health interventions for children and young people with mental health problems – a systematic and meta‐review. J Child Psychol Psychiatry 2017;58:474‐503. [DOI] [PubMed] [Google Scholar]
  • 189.Grist R, Croker A, Denne M et al. Technology delivered interventions for depression and anxiety in children and adolescents: a systematic review and meta‐analysis. Clin Child Fam Psychol Rev 2019;22:147‐71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 190.Hill C, Creswell C, Vigerland S et al. Navigating the development and dissemination of internet cognitive behavioral therapy (iCBT) for anxiety disorders in children and young people: a consensus statement with recommendations from the #iCBTLorentz Workshop Group. Internet Interv 2018;12:1‐10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 191.Peiris D, Miranda JJ, Mohr DC. Going beyond killer apps: building a better mHealth evidence base. BMJ Glob Health 2018;3:e000676. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 192.Mohr DC, Riper H, Schueller SM. A solution‐focused research approach to achieve an implementable revolution in digital mental health. JAMA Psychiatry 2018;75:113‐4. [DOI] [PubMed] [Google Scholar]
  • 193.Kitson AL, Rycroft‐Malone J, Harvey G et al. Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implement Sci 2008;3:1‐2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 194.Roberts ET, Mehrotra A. Assessment of disparities in digital access among Medicare beneficiaries and implications for telemedicine. JAMA Intern Med 2020;180:1386‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 195.Fischer SH, Ray KN, Mehrotra A et al. Prevalence and characteristics of tele­health utilization in the United States. JAMA Netw Open 2020;3:e2022302 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 196.Nouri S, Khoong EC, Lyles CR et al. Addressing equity in telemedicine for chronic disease management during the COVID‐19 pandemic. NEJM Catalyst Innovations in Care Delivery, May 2020. [Google Scholar]
  • 197.Szinay D, Jones A, Chadborn T et al. Influences on the uptake of and engagement with health and well‐being smartphone apps: systematic review. J Med Internet Res 2020;22:e17572. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 198.Hoffman L, Wisniewski H, Hays R et al. Digital Opportunities for Outcomes in Recovery Services (DOORS): pragmatic hands‐on group approach toward increasing digital health and smartphone competencies, autonomy, relatedness, and alliance for those with serious mental illness. J Psychiatr Pract 2020;26:80‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 199.Torous J, Lipschitz J, Ng M et al. Dropout rates in clinical trials of smartphone apps for depressive symptoms: systematic review and meta‐analysis. J Affect Disord 2020;263:413‐9. [DOI] [PubMed] [Google Scholar]
  • 200.Baumel A, Muench F, Edan S et al. Objective user engagement with mental health apps: systematic search and panel‐based usage analysis. J Med Internet Res 2020;22:e17572. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 201.Bradway M, Gabarron E, Johansen M et al. Methods and measures used to evaluate patient‐operated mobile health interventions: scoping literature review. JMIR mHealth and uHealth 2020;8:e16814. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 202.Ng MM, Firth J, Minen M et al. User engagement in mental health apps: a review of measurement, reporting, and validity. Psychiatr Serv 2019;70:538‐44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 203.Pratap A, Neto EC, Snyder P et al. Indicators of retention in remote digital health studies: a cross‐study evaluation of 100,000 participants. NPJ Digit Med 2020;3:21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 204.Morton E, Barnes SJ, Michalak EE. Participatory digital health research: a new paradigm for mHealth tool development. Gen Hosp Psychiatry 2020;66:67‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 205.Hetrick SE, Robinson J, Burge E et al. Youth codesign of a mobile phone app to facilitate self‐monitoring and management of mood symptoms in young people with major depression, suicidal ideation, and self‐harm. JMIR Ment Health 2018;5:e9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 206.Rudd BN, Beidas RS. Digital mental health: the answer to the global mental health crisis? JMIR Ment Health 2020;7:e18472. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 207.Carter H, Araya R, Anjur K et al. The emergence of digital mental health in low‐income and middle‐income countries: a review of recent advances and implications for the treatment and prevention of mental disorders. J Psychiatr Res 2021;133:223‐46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 208.Merchant R, Torous J, Rodriguez‐Villa E et al. Digital technology for management of severe mental disorders in low‐income and middle‐income countries. Curr Opin Psychiatry 2020;33:501‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 209.Naslund JA, Gonsalves PP, Gruebner O et al. Digital innovations for global mental health: opportunities for data science, task sharing, and early intervention. Curr Treat Options Psychiatry 2019;6:337‐51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 210.Connolly SL, Hogan TP, Shimada SL et al. Leveraging implementation science to understand factors influencing sustained use of mental health apps: a narrative review. J Technol Behav Sci 2020;10.1007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 211.Bird KA, Castleman BL, Denning JT et al. Nudging at scale: experimental evidence from FAFSA completion campaigns. J Econ Behav Organ 2019;183:105‐28. [Google Scholar]
  • 212.Lattie EG, Nicholas J, Knapp AA et al. Opportunities for and tensions surrounding the use of technology‐enabled mental health services in community mental health care. Adm Policy Ment Health 2020;47:138‐49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 213.Lederman R, D’Alfonso S. The digital therapeutic alliance: insights on the effectiveness of online therapy. Presented at the First Annual Symposium on the Digital Therapeutic Alliance, Melbourne, August 2019.
  • 214.Chikersal P, Belgrave D, Doherty G et al. Understanding client support strategies to improve clinical outcomes in an online mental health intervention. Presented at the CHI Conference on Human Factors in Computing Systems, Honolulu, April 2020.
  • 215.Graham AK, Lattie EG, Powell BJ et al. Implementation strategies for digital mental health interventions in health care settings. Am Psychol 2020;75:1080‐92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 216.Schueller SM, Torous J. Scaling evidence‐based treatments through digital mental health. Am Psychol 2020;75:1093‐104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 217.Rodriguez‐Villa E, Rauseo‐Ricupero N, Camacho E et al. The digital clinic: implementing technology and augmenting care for mental health. Gen Hosp Psychiatry 2020;66:59‐66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 218.Mordecai D, Histon T, Neuwirth E et al. How Kaiser Permanente created a mental health and wellness digital ecosystem. NEJM Catalyst Innovations in Care Delivery, January 2021. [Google Scholar]
  • 219.Owen JE, Kuhn E, Jaworski BK et al. VA mobile apps for PTSD and related problems: public health resources for veterans and those who care for them. Mhealth 2018;4:28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 220.Nielsen JC, Kautzner J, Casado‐Arroyo R et al. Remote monitoring of cardiac implanted electronic devices: legal requirements and ethical principles – ESC Regulatory Affairs Committee/EHRA joint task force report. Europace 2020;22:1742‐58. [DOI] [PubMed] [Google Scholar]
  • 221.Ghafur S, Van Dael J, Leis M et al. Public perceptions on data sharing: key insights from the UK and the USA. Lancet Digit Health 2020;2:e444‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 222.Parker L, Halter V, Karliychuk T et al. How private is your mental health app data? An empirical study of mental health app privacy policies and practices. Int J Law Psychiatry 2019;64:198‐204. [DOI] [PubMed] [Google Scholar]
  • 223.Stern AD, Gordon WJ, Landman AB et al. Cybersecurity features of digital medical devices: an analysis of FDA product summaries. BMJ Open 2019;9:e025374. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 224.Germain T.Mental health apps aren’t all as private as you may think. Consumer Reports, March 2, 2021. [Google Scholar]
  • 225.Larsen ME, Huckvale K, Nicholas J et al. Using science to sell apps: evaluation of mental health app store quality claims. NPJ Digit Med 2019;2:18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 226.Nebeker C, López‐Arenas A. Building Research Integrity and Capacity (BRIC): an educational initiative to increase research literacy among community health workers and promotores. J Microbiol Biol Educ 2016;17:41‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 227.Alon N, Stern AD, Torous J. Assessing Food and Drug Administration’s risk‐based framework for software precertification with top US health apps: quality improvement study. JMIR mHealth and uHealth 2020;8:e20482. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 228.Carl JR, Jones DJ, Lindhiem OJ et al. Regulating digital therapeutics for mental health: opportunities, challenges, and the essential role of psychologists. Br J Clin Psychol (in press). [DOI] [PubMed] [Google Scholar]
  • 229.Rodriguez‐Villa E, Torous J. Regulating digital health technologies with trans­parency: the case for dynamic and multi‐stakeholder evaluation. BMC Med 2019;17:226. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 230.Eyre HA, Singh AB, Reynolds C III. Tech giants enter mental health. World Psychiatry 2016;15:21‐2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 231.Powell A, Torous J. A patient‐centered framework for measuring the economic value of the clinical benefits of digital health apps: theoretical modeling. JMIR Ment Health 2020;7:e18812. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 232.Mitchell LM, Joshi U, Patel V et al. Economic evaluations of Internet‐based psychological interventions for anxiety disorders and depression: a systematic review. J Affect Disord 2021;284:157‐82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 233.Gordon WJ, Patel V, Thornhill W et al. Characteristics of patients using patient‐facing application programming interface technology at a US health care system. JAMA Netw Open 2020;3:e2022408. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 234.Lagan S, Emerson MR, King D et al. Mental health app evaluation: updating the American Psychiatric Association's framework through a stakeholder‐engaged workshop. Psychiatr Serv (in press). [DOI] [PubMed] [Google Scholar]
  • 235.Torous J, Choudhury T, Barnett I et al. Smartphone relapse prediction in serious mental illness: a pathway towards personalized preventive care. World Psychiatry 2020;19:308‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from World Psychiatry are provided here courtesy of The World Psychiatric Association

RESOURCES