Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Apr 14.
Published in final edited form as: Procedia Comput Sci. 2022 Sep 21;206:6–22. doi: 10.1016/j.procs.2022.09.081

Developing, Deploying, and Evaluating Digital Mental Health Interventions in Spaces of Online Help- and Information-Seeking

Kaylee P Kruzan a,*, Ellen E Fitzsimmons-Craft b, Mallory Dobias c, Jessica L Schleider c, Abhishek Pratap d,e,f,g,h
PMCID: PMC10104522  NIHMSID: NIHMS1838305  PMID: 37063642

Abstract

The internet is frequently the first point of contact for people seeking support for their mental health symptoms. Digital interventions designed to be deployed through the internet have significant promise to reach diverse populations who may not have access to, or are not yet engaged in, treatment and deliver evidence-based resources to address symptoms. The liminal nature of online interactions requires designing to prioritize needs detection, intervention potency, and efficiency. Real-world implementation, data privacy and safety are equally important and can involve transparent partnerships with stakeholders in industry and non-profit organizations. This commentary highlights challenges and opportunities for research in this space, grounded in learnings from multiple research projects and teams aligned with this effort.

Keywords: digital mental health, digital intervention, information-seeking, help-seeking, community partnership

1. Introduction

Although around 26% of the U.S. population meet criteria for a psychiatric disorder per year (1), it is estimated that less than 30% of those with clinically significant symptoms ever receive any mental health treatment (13). Without treatment, mental health symptoms can be prolonged or worsen with time, contributing to functional impairment and financial costs, in part due to symptom progression resulting in more acute needs (4). Since the start of the Covid-19 pandemic, multiple studies have shown increases in emergency room visits for issues like self-harm and overdose (5,6) and hospitalizations for eating disorders (EDs) have increased by 48% (7), adding urgency to the widely recognized need to get evidence-based resources to people who experience, or are at risk of, mental health challenges in a timely fashion.

The ability to address mental health through prevention and early intervention is constrained by at least two factors, however. First, there are not nearly enough mental healthcare providers to respond to those who already seek mental health treatment – resulting in long waitlists and preventing quality care (2,3). For example, in the National Comorbidity Survey-Replication study of over 9,000 individuals, only 21.5% with a psychiatric disorder received treatment from a mental health professional, with other studies reporting 70% of those needing mental health care not receiving it (8,9). Further, in a study replicating the experience of obtaining in-network care by way of telephone calls to psychiatrists listed on U.S. insurance websites, only 26% of the sample was able to secure an appointment, meaning that over 70% were unable to get care or did not receive any type of follow-up (10).

Second, many individuals who experience mental health symptoms are hesitant to utilize services due to attitudinal barriers such as low perceived need for treatment, concerns about judgment, low mental health literacy, and strong preferences for self-management (1113). Adolescents face additional barriers to treatment access, since caregivers typically act as ‘gatekeepers’ to their mental health support, and they often worry that caregivers will be notified of, or involved in, their treatment without personal consent (14,15). Financial barriers are also highly prevalent, with one study showing that in the U.S. showing that around 45% of psychiatrists do not accept private insurance or Medicare (16), and self-pay visits (which are disproportionately sought by white patients) have been trending upwards in the past decade (17). Similar rates of insurance acceptance and self-pay are not seen in other medical disciplines like primary care, demonstrating a lack of national investment in mental health, specifically (17). Digital technologies, including barrier-free options that are immediately, easily accessible to both adolescents and adults at no cost, have been recommended as a practical response to increase the scale for delivery of evidence-based interventions (1820).

1.1. Digital mental health interventions

Digital mental health interventions (DMHIs) are tools that use technologies, such as mobile phones, computers, tablets, and sensors, designed to intervene in, and improve, mental health symptoms (21,22). Large meta-analyses of randomized controlled trials (RCT) of DMHIs consistently show their effectiveness at reducing mental health symptoms, particularly for conditions such as depression, anxiety, and eating disorders (2325). Among the advantages of DMHIs are their reach and ability to extend treatment to individuals who may not otherwise have access to treatment (e.g., those from rural areas) (26). These interventions can have flexible formats (e.g., app-based, web-based, chatbot) and vary in their clinical approach and degree of human support (e.g., fully autonomous, guided, coached) (21,27). DMHIs can also be tailored for different groups’ needs, which may ultimately increase intervention uptake and engagement, and improve chances of improved outcomes (26).

Digital tools may also be more acceptable to people with stigmatized mental health conditions since they can reduce concerns around common barriers to treatment. For example, the sociotechnical affordances of online interactions, like the ability to engage in anonymous and asynchronous communication, can attenuate concerns around disclosure of these conditions, and may fit more seamlessly within their lifestyle (2830). Another advantage of DMHIs, like self-help or guided self-help apps or structured chatbots, is the ability to ensure that the treatment provided is evidence-based, which contrasts with other service settings. As an example, when individuals with EDs receive care, more often than not, it is not an evidence-based treatment (3133). Indeed, the number of ED specialist therapists who report adhering to evidence-based protocols is between 6 and 35%, with far more clinicians reporting using an eclectic mix of techniques derived from evidence-based techniques and some not even supported to that level (34). This lack of specialized training is not unique to ED but is also the case for other conditions. Even when clinicians say they are using an evidence-based treatment, they omit critical elements of these approaches (35,36). In sum, DMHIs have promise to address gaps in treatment due to their format, which may attenuate attitudinal and financial barriers, and their ability to deploy evidence-based interventions at scale and at a fraction of the cost of traditional methods.

1.2. Early information and help-seeking online

Internet-based technologies are pervasive with over 90% of U.S. adults reporting regular use of at least one of these technologies, and most U.S. adolescents having access to the Internet through a smartphone (88%) or desktop/laptop (95%) (37). As such, the internet has become a go-to resource for mental health information and support. People of all ages engage in informal help- and information-seeking online (3841). Some of the common ways individuals seek mental health information is through Internet search engines (42), where they may find informational websites and communities of peers with shared mental health experiences through social media. Young people and adults alike are not only comfortable with using the internet to learn about their mental health (43), but youths often prefer online help-seeking to in-person help-seeking because it supports autonomy (44). Given that DMHIs can be efficacious, scalable, and implemented within spaces of existing online help-seeking, efforts to design for these unique spaces are needed.

Developing and deploying DMHIs for non-traditional service settings, like spaces where people seek mental health information online, requires careful attention to (a) the unique contextual elements of the online environment that may impact a person’s receptivity, willingness, or desire to engage, (b) collaborations between various stakeholders, and (c) a strategic plan for deployment, sustainability along with data safety and privacy. The goal of this brief commentary is to describe the ongoing work of several research teams focused on the implementation of DMHIs within contexts where people seek mental health information online, including through popular search engines, online mental health screens, social media, and open-access websites. We then discuss the challenges and opportunities identified through this work to guide future research.

1.3. Internet Search Engines

Internet search engines are often an early point of contact for individuals seeking mental health information online. With over 50% of the global population using the internet daily and over 1.2 trillion Internet searches per year worldwide (45), the Internet represents a scalable and effective information retrieval medium that enables masses to seek health-related information within seconds. In the US, two out of three adults regularly make health queries online (46). Nearly 1 in 10 searches on the largest search engine, Google, are related to health (47). Internet users turn to the internet to look up symptoms they are experiencing, to seek treatment resources based on a recent diagnosis, or to view information on health and wellbeing (46).

Search engines like Google and Bing gather population level trends (48) as well as individualized search queries that could be mined for health-related patterns and warning-signs (48,49). For example, syndromic surveillance could detect individual mental health risk and subsequently facilitate the delivery of targeted interventions. In addition to detecting need for intervention, individualized search activity can be used to better understand real-world behaviors and risk factors that have historically been difficult to assess using episodic traditional clinical data—as has been the case with suicide. Indeed, suicide continues to be among the top 10 leading causes of death in the US (50), yet decades of research have not significantly improved our ability to predict when someone might be at the highest risk of self-harm or suicide (51,52). Similar methods have been used to predict other health issues with varying degrees of success (53).

Importantly, personalized online search history data provides an ongoing real-world data-stream enabling us to detect mental health risk, and deliver personalized and just-in-time intervention (54,55). This method of risk assessment can be: (1) individualized, (2) conducted in real-time, and (3) based on high-risk search queries that are independent of both an individual’s contact with the mental health system and their disclosures of self-injurious thoughts and behaviors and mental health symptoms to others. However, accessing individualized online search queries associated behaviour for research and real-world DMHI deployment can be complicated due to regulatory, data safety and privacy concerns (56). To develop robust and transparent data collection and natural language processing (NLP)-based deidentification mechanisms enabling detection and deployment of interventions at the point of search queries, it is crucial to work with multiple stakeholders from technology companies to study participants and their families to understand their concerns and willingness to share personal search data.

To this end, a team of researchers sought to examine the feasibility and acceptability of using personalized online information-seeking behavior in the identification of risk for suicide attempts through a recent pilot study (57). By mapping search queries to expert-curated risk factors of suicide using NLP, this pilot work provided evidence of the ability to detect changes in online search behavior proximal to suicide attempts up to 60 days before an attempt. Importantly, when participants were asked about their comfort with search data being used for prevention purposes, including delivering of non-invasive interventions, most participants felt this was acceptable. However, concerns were also raised about detection accuracy, privacy, and the potential for overly invasive intervention. This will be something future research will need to navigate.

In sum, findings from this early work suggest that using AI-based analytical approaches to assess real-world online information-seeking behavior could offer an unparalleled opportunity for researchers to understand the novel and proximal risk factors for mental health and potentially provide an early intervention opportunity uniquely situated within a common help-seeking process.

1.4. Online self-screenings, including through non-profit advocacy organizations

People also turn to mental health advocacy websites that host resources and self-screeners when seeking mental health information online. These websites aim to empower visitors to understand and act on their mental health symptoms by providing visitors with a breakdown of their symptom severity, recommendations for treatment, and links to psychoeducation materials. Screening sites see millions of individuals per year, many of whom are from underserved populations, making them a unique pathway to services. Despite the impressive reach of these screenings, research has shown that they do not consistently increase service use for the majority of screeners (40,58), necessitating a reconsideration of resource delivery at this unique point of contact. In what follows, we describe two research team efforts to develop DMHIs for deployment in these contexts.

Mental Health America (MHA) is the largest mental health patient advocacy group in the U.S. and hosts a variety of validated mental health screeners for community members experiencing mental health symptoms. Over 5 million people take MHA’s online screeners per year, with many screening above thresholds for a clinical diagnosis (59). Of those who took the depression screener (PHQ-9) in 2020, 71% reported experiencing frequent thoughts of suicide or self-harm and never received mental health care, for example.

As an early effort to identify ways to better support visitors in making the transition from self-screening to service use, a team of researchers sought to understand young adults’ experiences with MHA’s online depression and anxiety screeners. Qualitative research revealed that common catalysts to self-screening included visitors recognizing a change in their emotions or moods, and concerning shifts from a “personal normal” (60). Most visitors felt validated upon receiving screener results that were consistent with their experience, but despite a desire to do something to improve their mental health they were unsure of how to act on the feedback the screener provided. This work suggests that while self-screenings may satisfy early needs in help-seeking (e.g., awareness, validation) (61), they are not enough. As a result, the research team has partnered with MHA on several projects in formative stages to develop free, publicly available, autonomous digital interventions based on evidence-based practices for young adults with depression, anxiety (62) and nonsuicidal self-injury (63).

Similar efforts have been undertaken with National Eating Disorders Association (NEDA), the largest national non-profit organization dedicated to eating disorders. The NEDA website hosts a screener that reaches 200K respondents per year, with the vast majority screening as positive or at risk for an ED (85%+), and only 14% had previously received treatment and only 3% were currently receiving treatment (64). In a study of the website, only 16% of those screening positive for an ED, and not in treatment, initiated care following screen completion, with this figure likely being a gross overestimate given its basis on only those who provided follow-up data (40). These findings support the need for additional tools to increase service utilization following screening.

In response, a chatbot named “Alex” was developed. Alex delivered four theoretically-informed components to target service utilization: psychoeducation, motivational interviewing, personalized service recommendations, and repeated check-ins to check in on service seeking and offer support and troubleshooting. The chatbot underwent an extensive iterative, user-centered design and usability testing process, in order to ensure it would maximally meet the needs of its ultimate end-users. Overall, participants reflected positively on interactions with the initial version of Alex, and refinements between cycles of usability testing further improved user experiences. A randomized factorial trial is now underway to test which components may be effective in terms of promoting service utilization, so as to ultimately deploy an optimized version.

Related work from the same research team focused on an ED prevention program for those at high risk. In this study, women at high risk for an ED (N=700) were randomized to: 1) Tessa, an EDs prevention chatbot; or 2) waitlist control, and were followed for six months. Findings showed a significantly greater reduction in weight/shape concerns, for intervention vs control at 3- (d=−0.20; p=.03) and 6-month follow-up (d=-0.19; p=.04). Moreover, the odds of remaining non-clinical for EDs were significantly higher in intervention vs control at both 3- (OR = 2.37, 95% CI [1.37, 4.11]) and 6-month follow-ups (OR = 2.13, 95% CI [1.26, 3.59]) (65). Within one month of publication of these findings, the research team successfully collaborated with the industry partner, X2AI, who hosted Tessa and NEDA to deploy the chatbot, free of charge, through their online ED screen and other channels, with the potential to reach hundreds of thousands of individuals each year. In just the first 3 months, nearly 700 individuals accessed the chatbot, exchanging 21,000+ messages. Notably, users are most active between the hours of 8–10 pm ET, and 29% of messages are exchanged over the weekend, dovetailing with previous work on when individuals are most likely to use digital interventions (66,67), and providing support during hours when traditional treatment or even asynchronous support in coached apps is not available.

In sum, partnering with non-profits to deliver the DMHI provides a clear dissemination pathway and access to a diverse group of individuals who are not in treatment (68).

1.5. Social Media

Social media websites are another context where people engage in online information and help-seeking for mental health. Research has shown that people frequently share about their experiences of mental health symptoms like postpartum depression (69,70), depression (71,72), eating disorders (73,74), and self-harm (7578) and seek support from peers with similar experiences (39). Research has also shown that social media are used by many groups that are traditionally underserved in mental healthcare settings, including young people from racial and sexual minority groups (7981), making it a promising venue to both provide psychoeducation on available supports and to deliver evidence-based strategies to improve mental health symptoms.

Despite the potential for DMHI delivery through social media, few studies have actually leveraged popular social media platforms for this purpose (82). This could be due to several factors including difficulty controlling what happens on social media, as well as barriers to productive collaboration between social media or industry partners and academic researchers. Below we highlight two projects aimed at integrating brief DMHIs (single session interventions, SSIs) within popular social media platforms in collaboration with Koko—a non-profit, online mental health platform. Koko has established partnerships with social media companies which has enabled research teams to evaluate the dissemination of “in-the-moment” support to social media users.

First, a team of researcher embedded three SSIs within Tumblr, a popular social media platform that hosts more than 130 million monthly active users (83). Users searching for mental health-related topics on the platform were sent a direct message with links to crisis resources and SSIs, which were presented as Koko “minicourses.” Each of these three SSIs (ABC Project, Project SAVE, and REFRAME) were self-guided, 5–8 minutes in length, and targeted a core idea or skill (e.g., ABC: “taking action can help improve your mood”) derived from evidence-based therapeutic techniques (e.g., behavioral activation). Specifically, this project sought to evaluate acceptability and short-term utility of these SSIs by measuring changes in three mental health outcomes (hopelessness, self-hatred, and desire to discontinue self-harm) from pre- to post-SSI. Preliminary findings from this work have shown both high completion rates among those engaged in all three SSIs as well as high rates of acceptability. Among those who completed an SSI, improvements were detected across all outcomes (hopelessness, self-hate, desire to stop self-harm; ps < .0001) from pre- to post-SSI.

A similar collaboration with Koko involved designing and evaluating a single-session version of the Tessa ED prevention intervention (described earlier) for deployment on Tumblr. In a feasibility study, 1100 Tumblr users who used ED-related search terms were offered the intervention, and 525 users completed the program. This work showed significant improvement in body image pre- to post-intervention (d=.54, p<.001), and users found the program very enjoyable.

Taken together, these findings show that embedding brief interventions within online social networking platforms may provide one avenue toward increasing quick and free access to evidence-based mental health support. Further research will be needed to explore the longer-term efficacy of these interventions as well as the feasibility of deploying interventions of longer durations.

1.6. Open access websites and collaborations with local government

Providing access to low- or no-cost, brief, evidence-based digital supports and disseminating such resources within underserved communities through dedicated open-access websites might reflect a final promising path towards repairing need-to-access gaps in mental health support., A team of researchers recently developed and now maintains Project YES (www.schleiderlab.org/YES) which is a cost-free, anonymous website where young people can flexibly complete any of three evidence-based digital SSIs. Each SSI within Project YES has shown short- and long-term utility in reducing hopelessness, increasing agency, and mitigating depression and anxiety symptoms in young people, across randomized and open trials alike (8487). Within YES, youths are empowered to choose any SSI they view as relevant to their difficulties and needs; after completing their preferred 20-to-30 minute, self-guided intervention, youths are then invited to offer their “best, anonymous coping advice” for other young people dealing with depression, anxiety, or stress, which they may choose to post publicly on the YES “Advice Center” (85).

Importantly, despite the acceptability and utility of Project YES in the context of research studies, naturalistic community uptake has remained low (approximately 20–40 youth monthly)—as is common for freestanding, self-guided digital health tools. This trend suggested a need for community-partnered adaptation and implementation efforts, which prompted an academic-community partnership with the City of San Antonio, along with the University of Texas Health Science Center at San Antonio, to increase local cultural relevance, acceptability, and uptake of this barrier-free support in local youth. Specifically, with City government support, the Project YES website was (1) translated into Spanish, (2) culturally adapted via focus groups and co-design sessions with San Antonio youth with lived experience of depression and anxiety, and (3) disseminated throughout San Antonio, Texas via local outreach efforts to community organizations and schools. Given that this SSI was intended for adolescents, it was made accessible through advertisements on local organizations’ Instagram and Facebook pages. By adapting YES to fit local language and context, and thanks to community-led awareness-raising efforts, Project YES uptake increased substantially. During the 8-month project period, 1,801 San Antonio youth began and 894 completed a 30-minute, single-session intervention within Project YES (39.9% male, 53.4% female, 31.5% gender diverse; 82% Hispanic, 4.3% non-Hispanic white, 6.3% Black, 1.6% Asian, 5.16% other). This SSI completion rate (49.64%) surpassed those in prior studies (e.g., 34%) (85) when YES was disseminated via social media, in the absence of community-based promotion or adaptation. Not only did San Antonio youth rated Project YES as highly acceptable across all metrics, both in English and Spanish, but YES completers reported significant improvements in hopelessness (d = 0.33), self-hatred (d = 0.27), and perceived agency (d = 0.25), from pre- to post-program.

To increase the likelihood of broad access, these SSIs were also deployed within the community through provider websites, where people go to look at treatment options and book appointments. This context for SSIs is especially relevant for communities that are geographically isolated and/or lack service providers, since they can be offered to individuals when they book appointments and are on waitlists—a prospect that has demonstrated promise in initial pilot trials (88,89). Indeed, research has shown that SSIs delivered to adolescents on outpatient mental health wait lists can improve outcomes like symptoms of depression and anxiety, and self-reported distress (88,89).

This collaboration embedded free, potent, and evidence-based SSIs within the existing mental health infrastructure to provide access to individuals who otherwise may not receive services, or whose service needs were at present unmet. Local outreach efforts and formative work with community members were critical to the success of both the adaptation of YES as well as the implementation of YES within the community.

2. Challenges and opportunities for designing and implementing future digital mental health interventions

While the above projects are at different stages of exploration, evaluation and implementation, findings support the value of DMHIs deployed in spaces of mental health outreach to extend the reach of evidence-based treatment to people in need of services. At the same time, the projects collectively highlight common challenges to embedding and sustaining DMHIs in such spaces, including the need to build and maintain partnerships with multi-level stakeholders—from leaders in non-profit or government sectors to target end-users of DMHIs themselves. In what follows, we describe key challenges encountered in the process of executing these projects, as well as opportunities to move this research forward. These challenges are not novel but play a particularly impactful role in being able to make DMHIs available for users in non-traditional service settings online.

2.1. Challenge 1: The critical necessity of mutually-beneficial cross-sector collaborations

Clinical scientists, human-computer interaction researchers, software programmers, industry and non-profit partners, and governmental agencies, all played essential roles in the projects discussed in the paper – each bringing unique domain, methodological and technical expertise. Involving individuals with lived experience of mental illness in the early design and evaluation process for digital tools is also critical to ensure the tools meet the population’s needs, and to center them in conversations around how these tools should be appropriately, and ethically, deployed in the spaces they occupy. However, collaborations like this are difficult to navigate when there is not a shared language, timeline, vision, and incentive structure among stakeholders (21,90). Ensuring mutual benefit and equity across contributors cannot be an afterthought but must be planned for early on.

Complex issues, like those presented in the field of digital mental health, require multidisciplinary and cross-sector collaboration, however, for the most part, academic researchers do not receive training to equip them to engage in, and lead, successful collaboration across disciplines (91). As individuals progress in their academic journey, they become specialized in their research area, the methodologies that best support line of questioning, and, as a result, the research is produced from silos. For researchers working on pragmatic and applied projects this often means they must learn the necessary components of collaboration – including how to establish community-based partnerships – on the fly, slowing innovation and progress.

Opportunity.

Principles and practices from team science may provide a remedy to this issue, and consequently be of great benefit to researchers working in the space of digital mental health. Team science is a discipline that provides pragmatic guidance on how to bring individuals with different expertise together to work towards the same goal (92,93). At the core, team science challenges traditional ways of pursuing science, highlighting that researchers have to be trained and attuned to collaborative practices to arrive at innovated solutions to complex challenges (94). For example, Horowitz and colleagues describe that this requires researchers, or project leads, to establish a culture of inclusivity early on, with participatory goal setting and interdependent research tasks and incentive systems (92,94,95).

Across our projects, we learned that this went beyond having consensus on the overarching goal of a project (e.g., provide a mental health service for members of a particular community). While the pursuit of science is incentivised, and thus prioritized, among researchers situated within academic institutions, it was not a main priority to partners. This fact challenged us to find ways to support the diverse objectives of our collaborators on a timeline that fit their needs. For example, for some research teams this meant ensuring that information generated at all phases of the project was made public and available in pre-print formats so findings on community needs and preferences, and the feasibility and efficacy of interventions were not beholden to publication timelines and not behind paywalls.

In sum, a greater institutional investment in graduate and post-graduate training in team science and community-based collaboration, as well as new incentive structures for these types of collaborations, may lead to more innovative and mutually beneficial work in digital mental health moving forward.

2.2. Challenge 2: Few incentives, and sparse roadmaps, for closing research-to-practice gaps in digital mental health

In all our projects, there were no existing roadmaps to follow for implementing DMHIs in new, nontraditional service settings. As such, every method and decision-point felt like uncharted territory. There have historically been a lack of incentives and pathways for translation of evidence-based interventions from research to practice, and this challenge was especially salient in projects that required upfront time- and resource-investment to establish and nurture mutually productive partnerships. It can take upwards of 17 years for research discoveries and innovations to land in the hands of individuals who could benefit from them (96). In digital mental health this is especially problematic due to the fast pace of technological innovation, and the rate at which technologies become obsolete (97,98). This is complicated by the fact that most research grants fund discoveries and evaluation, with much less support for their translation or widespread dissemination. The successful development and dissemination of digital interventions requires both time, expertise, and resources, including: (1) adequate marketing, (2) budgeting for maintenance and technology iteration, and (3) funding for personnel for coaching. It is unrealistic for researchers to do this on their own. Partnership with industry can help address some of these challenges, but this type of work and time are not often incentivized by academic institutions. As a result, digital tools created in academic institutions risk being obsolete by the time they make it to public, at which point even carefully-considered, well-executed implementation and sustainment plans can fall short on impacting real-world outcomes.

Opportunity.

Models to accelerate the research process and narrow this gap can assist in bringing tools to the public more quickly—without compromising the rigorous methods necessary to ensure that they are safe and effective. For example, the Accelerated Creation-to-Sustainment Model (ACTS) (99) describes three phases of digital mental health research: (1) the create phase, wherein the digital tool, service delivery protocols, and strategic implementation plan are developed, (2) the trial phase, in which a trial is run to evaluate the efficacy of the tool and implementation strategy simultaneously, e.g., via an effectiveness-implementation trial (100), and (3) the sustainment phase, which involves a lessening of research support as the tool is fully embedded within the context for delivery with plans for sustainment. Importantly, the ACTS model emphasizes the need to consider, and plan for, implementation and sustainment as early as the first phase, focusing on components of the digital tool, implementation and sustainment procedures at each phase (99).

Models like this are becoming more widely used in digital mental health. However, without proper funding for research personnel and collaboration, these models can only go so far to address real change for population mental health on a timeline that is pragmatic and capable of meeting acute public health needs. Additionally, while there has been a fair about of attention on how to integrate DMHIs into traditional service settings, there has been less attention on how to embed DMHIs in non-traditional service settings, like points of online help-seeking. Future work must focus on the unique considerations and (partnerships) needed to reach individuals outside of these care settings where there may be less interest, oversight, or resources to sustain tools over time.

2.3. Challenge 3: Lack of funding and reimbursement to support access to digital mental health

Issues of sustainability surfaced across projects such that there were no clear paths to pay for the DMHIs we were implementing, beyond the project period. In several cases, partner organizations were willing to fund free access points to the DMHIs, but our ability to ensure that the technology would receive the continual care and maintenance required for proper functioning over time was limited. Despite the recognized need for new service models, policies supporting the integration of digital mental health services into clinical practice are not yet well-established. The U.S. is behind other developed countries in their integration of digital mental health into health care settings. The Covid-19 pandemic rapidly changed the landscape of healthcare delivery, and led to a broader adoption of remote- and telehealth care options (101). These changes required a relaxing of rules around remote healthcare, including increases in institutional support and new reimbursement models (102). However, similar support and reimbursements have not been put in place for most digital mental health interventions, outside of telehealth. Though studies have shown that clinicians are willing to use DMHIs in practice (103,104), rates of adoption are challenged by a lack of infrastructure for clinicians to integrate these tools for use by their patients, and a lack service providers and sufficient training on DMHIs.

Opportunity.

Several groups have convened experts to advocate for DMHI coverage, and make recommendations to address issues of broad adoption of DMHI in clinical spaces, including the Banbury Group, the American Medical association’s Digital Medicine Payment Advisory Initiative, and the newly founded Society for Digital Mental Health (https://www.societydmh.org). The Banbury statement was written by academics, researchers, payers, and insurance companies, and describes the need for reimbursement of both the cost of the DMHI as well as provider times to incentivize use (105). Academic centers are also playing a role in training health care organizations to develop reimbursement procedures (105), and could be a driving force in supporting the use of CPT codes, device codes, and evidence-based standards for DMHI (106). This work is utilizing a team science approach to advocate for important change. However, as acknowledged elsewhere (106), other steps to impact the treatment gap at a broad scale may include opening billing codes to non-mental health personnel for delivery, as well as relaxing licensure restrictions to allow for those trained in DMH to provide service across geographic boundaries.

2.4. Challenge 4: Disseminating tools to minors experiencing mental health symptoms

Youths are digital natives, and research has shown that DMHIs appeal to them since they can be easily integrated within their existing activities. Youths may be one of the groups most likely to benefit from digital interventions embedded within these spaces, due to their interest, developmental phase, and limitations in available, accessible mental health support options that do not require adult or caregiver disclosure. Yet, in the U.S., most research with minors requires parent or guardian consent – stunting our ability to enroll adolescents in research studies and possibly lessening our chances of developing tools that address their unique needs. In most U.S. states, adolescents 12 years of age and older can consent to mental and physical healthcare without parent or guardian approval. This increases the likelihood of young people getting the care they need. Applying similar logic to digital mental health, then, requiring guardian permission for adolescents to engage in online activities, like SSI, could inadvertently stop youth from receiving evidence-based support (107,108).

Opportunity.

A critical step forward may be to allow youths to participate in minimal risk online interventions without parental consent. A growing body of research suggests that adolescents can participate in online SSIs safely and that these interventions can positively impact shorter and longer-term mental health outcomes (87). This fact has opened doors to disseminating brief evidence-based interventions openly in online spaces like social media, without delaying, or preventing likelihood of access, due to the unique barriers presented to adolescents. If we are to make advancements in digital mental health for young people outside of traditional healthcare settings, we will need to remove barriers that prevent them from engaging in the research meant to develop resources highly attuned to their needs.

3. Conclusion

In this paper, we describe projects conducted by several research groups using internet search engines, social media, online mental health screens, and open-access websites to reach individuals who are seeking information and support for mental health symptoms. We have described the purpose of these projects, collaborator and stakeholder involvement, and preliminary findings which demonstrate the feasibility and potential efficacy of digital interventions embedded within online spaces where people seek information and help for their mental health.

Acknowledgements

We would like to acknowledge our research team and community partners who were integral to this work, including David C. Mohr, Madhu Reddy, Rachel Kornfield, Jonah Meyerhoff, Robert Morris (Koko), Akash Shroff, Chantelle Roulston, Julia Fassler, Nicole A. Dierschke, Jennifer San Pedro Todd, Ámbar Ríos-Herrera, Kristen Plastino, Theresa Nguyen, William W. Chan, Arielle C. Smith, Marie-Laure Firebaugh, Lauren A. Fowler, Biance DePietro, Naira Topooco, Denise E. Wilfley, C. Barr Taylor, Nicholas C. Jacobson, Laura D’Adamo, Olivia Laing, Lauren Smolar, Shiri Sadeh-Sharvit, Andrea Kass Graham, Gavin Rackoff, Michelle G. Newman, Linda Collins, Patricia A Areán, Honor Hsin, Tierney K Huppert, Karin E Hendricks, Patrick J Heagerty, Trevor Cohen, Courtney Bagge, Katherine Anne Comtois, the National Eating Disorders Association, & X2AI

Funding

EFC was supported by a National Eating Disorders Association Feeding Hope Fund Grant (K08 MH120341). KPK was supported by a grant from the National Institute for Mental Health (T32MH115882). JLS receives funding from NIH Office of the Director (DP5OD028123), NIMH (R43MH128075), NSF (2141710), HRSA (U3NHP45406-01-00), Upswing Fund for Adolescent Mental Health, Society for Clinical Child and Adolescent Psychology, and Klingenstein Third Generation Foundation.

Conflicts to acknowledge

EFC receives royalties from UpToDate. She receives consulting fees from PepsiCo and the Gatorade Sports Science Institute. JLS serves on the Scientific Advisory Board for Walden Wise and the Clinical Advisory Board for Koko; is Co-Founder and Co-Director of Single Session Support Solutions; and receives book royalties from New Harbinger, Oxford University Press, and Little Brown Book Group.

References

  • 1.Kessler RC, Demler O, Frank RG, Olfson M, Pincus HA, Walters EE, et al. Prevalence and Treatment of Mental Disorders, 1990 to 2003. The New England Journal of Medicine; Boston. 2005. Jun 16;352(24):2515–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Kazdin AE. Addressing the treatment gap: A key challenge for extending evidence-based psychosocial interventions. Behaviour Research and Therapy. 2017. Jan;88:7–18. [DOI] [PubMed] [Google Scholar]
  • 3.Kazdin AE, Fitzsimmons-Craft EE, Wilfley DE. Addressing critical gaps in the treatment of eating disorders: KAZDIN et al et al. Int J Eat Disord. 2017. Mar;50(3):170–89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Furukawa TA, Noma H, Caldwell DM, Honyashiki M, Shinohara K, Imai H, et al. Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis. Acta Psychiatr Scand. 2014. Sep;130(3):181–92. [DOI] [PubMed] [Google Scholar]
  • 5.Henry N, Parthiban S, Farroha A. The effect of COVID-19 lockdown on the incidence of deliberate self-harm injuries presenting to the emergency room. Int J Psychiatry Med. 2021. Jul;56(4):266–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Kang JH, Lee SW, Ji JG, Yu JK, Jang YD, Kim SJ, et al. Changes in the pattern of suicide attempters visiting the emergency room after COVID-19 pandemic: an observational cross sectional study. BMC Psychiatry. 2021. Dec;21(1):571. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Devoe D, Han A, Anderson A, Katzman DK, Patten SB, Soumbasis A, et al. The impact of the COVID -19 pandemic on eating disorders: A systematic review. Intl J Eating Disorders. 2022. Apr 5;eat.23704. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-Month Use of Mental Health Services in the United States: Results From the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005. Jun 1;62(6):629. [DOI] [PubMed] [Google Scholar]
  • 9.Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime Prevalence and Age-of-Onset Distributions of DSM-IV Disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005. Jun 1;62(6):593–602. [DOI] [PubMed] [Google Scholar]
  • 10.Malowney M, Keltz S, Fischer D, Boyd JW. Availability of Outpatient Care From Psychiatrists: A Simulated-Patient Study in Three U.S. Cities. PS. 2015. Jan;66(1):94–6. [DOI] [PubMed] [Google Scholar]
  • 11.Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. 2010. Dec;10(1):113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Mojtabai R, Olfson M, Sampson NA, Jin R, Druss B, Wang PS, et al. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med. 2011. Aug;41(8):1751–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Wilson CJ, Rickwood DJ, Bushnell JA, Caputi P, Thomas SJ. The effects of need for autonomy and preference for seeking help from informal sources on emerging adults’ intentions to access mental health services for common mental disorders and suicidal thoughts. Advances in Mental Health. 2011. Oct;10(1):29–38. [Google Scholar]
  • 14.Burke TA, Bettis AH, Barnicle SC, Wang SB, Fox KR. Disclosure of Self-Injurious Thoughts and Behaviors Across Sexual and Gender Identities. Pediatrics. 2021. Oct 1;148(4):e2021050255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Stiffman AR, Pescosolido B, Cabassa LJ. Building a Model to Understand Youth Service Access: The Gateway Provider Model. Ment Health Serv Res. 2004. Dec;6(4):189–98. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Bishop TF, Press MJ, Keyhani S, Pincus HA. Acceptance of Insurance by Psychiatrists and the Implications for Access to Mental Health Care. JAMA Psychiatry. 2014. Feb 1;71(2):176. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Benjenk I, Chen J. Trends in Self-payment for Outpatient Psychiatrist Visits. JAMA Psychiatry. 2020. Dec 1;77(12):1305. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Taylor CB, Fitzsimmons-Craft EE, Graham AK. Digital technology can revolutionize mental health services delivery: The COVID -19 crisis as a catalyst for change. Int J Eat Disord. 2020. May 25;eat.23300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, et al. WHO World Mental Health Surveys International College Student Project: Prevalence and distribution of mental disorders. Journal of Abnormal Psychology. 2018. Oct;127(7):623–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Ramos G, Chavira DA. Use of Technology to Provide Mental Health Care for Racial and Ethnic Minorities: Evidence, Promise, and Challenges. Cognitive and Behavioral Practice. 2022. Feb;29(1):15–40. [Google Scholar]
  • 21.Schueller SM, Torous J. Scaling evidence-based treatments through digital mental health. American Psychologist. 2020;75(8):1093–104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Mohr DC. Behavioral Intervention Technologies: Evidence review and recommendations for future research in mental health. 2013; [DOI] [PMC free article] [PubMed]
  • 23.Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry. 2017. Oct;16(3):287–98. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord. 2017. Aug 15;218:15–22. [DOI] [PubMed] [Google Scholar]
  • 25.Linardon J, Cuijpers P, Carlbring P, Messer M, Fuller-Tyszkiewicz M. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry. 2019. Oct;18(3):325–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Schueller SM, Hunter JF, Figueroa C, Aguilera A. Use of Digital Mental Health for Marginalized and Underserved Populations. Curr Treat Options Psych. 2019. Sep;6(3):243–55. [Google Scholar]
  • 27.Hermes ED, Lyon AR, Schueller SM, Glass JE. Measuring the Implementation of Behavioral Intervention Technologies: Recharacterization of Established Outcomes. Journal of Medical Internet Research. 2019. Jan 25;21(1):e11752. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Rains SA, Peterson EB, Wright KB. Communicating Social Support in Computer-mediated Contexts: A Meta-analytic Review of Content Analyses Examining Support Messages Shared Online among Individuals Coping with Illness. Communication Monographs. 2015. Oct 2;82(4):403–30. [Google Scholar]
  • 29.Walther JB. Selective self-presentation in computer-mediated communication: Hyperpersonal dimensions of technology, language, and cognition. Computers in Human Behavior. 2007. Sep;23(5):2538–57. [Google Scholar]
  • 30.Coulson NS, Bullock E, Rodham K. Exploring the Therapeutic Affordances of Self-Harm Online Support Communities: An Online Survey of Members. JMIR Mental Health. 2017. Oct 13;4(4):e44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Cooper Z, Bailey-Straebler S. Disseminating Evidence-Based Psychological Treatments for Eating Disorders. Curr Psychiatry Rep. 2015. Mar;17(3):12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Fairburn CG, Wilson GT. The dissemination and implementation of psychological treatments: Problems and solutions. Int J Eat Disord. 2013. Jul;46(5):516–21. [DOI] [PubMed] [Google Scholar]
  • 33.Lilienfeld SO, Ritschel LA, Lynn SJ, Cautin RL, Latzman RD. Why many clinical psychologists are resistant to evidence-based practice: Root causes and constructive remedies. Clinical Psychology Review. 2013. Nov;33(7):883–900. [DOI] [PubMed] [Google Scholar]
  • 34.Waller G Treatment Protocols for Eating Disorders: Clinicians’ Attitudes, Concerns, Adherence and Difficulties Delivering Evidence-Based Psychological Interventions. Curr Psychiatry Rep. 2016. Apr;18(4):36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Kosmerly S, Waller G, Robinson AL. Clinician adherence to guidelines in the delivery of family-based therapy for eating disorders: Clinician Adherence to Guidelines in the Delivery of FBT. Int J Eat Disord. 2015. Mar;48(2):223–9. [DOI] [PubMed] [Google Scholar]
  • 36.Von Ranson KM, Wallace LM, Stevenson A. Psychotherapies provided for eating disorders by community clinicians: Infrequent use of evidence-based treatment. Psychotherapy Research. 2013. May;23(3):333–43. [DOI] [PubMed] [Google Scholar]
  • 37.Mobile Technology and Home Broadband 2019 [Internet]. Pew Research Center: Internet, Science & Tech. 2019. [cited 2020 Nov 29]. Available from: https://www.pewresearch.org/internet/2019/06/13/mobile-technology-and-home-broadband-2019/ [Google Scholar]
  • 38.Kruzan KP, Meyerhoff J, Nguyen T, Reddy M, Mohr DC, Kornfield R. “I Wanted to See How Bad it Was”: Online Self-screening as a Critical Transition Point Among Young Adults with Common Mental Health Conditions. In: CHI Conference on Human Factors in Computing Systems [Internet]. New Orleans LA USA: ACM; 2022. [cited 2022 Jul 7]. p. 1–16. Available from: https://dl.acm.org/doi/10.1145/3491102.3501976 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Pretorius C, Chambers D, Coyle D. Young People’s Online Help-Seeking and Mental Health Difficulties: Systematic Narrative Review. J Med Internet Res. 2019. Nov 19;21(11):e13873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Fitzsimmons-Craft EE, Balantekin KN, Graham AK, DePietro B, Laing O, Firebaugh M, et al. Preliminary data on help-seeking intentions and behaviors of individuals completing a widely available online screen for eating disorders in the United States. Int J Eat Disord. 2020. Sep;53(9):1556–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Horgan Á, Sweeney J. Young students’ use of the Internet for mental health information and support. Journal of Psychiatric and Mental Health Nursing. 2010;17(2):117–23. [DOI] [PubMed] [Google Scholar]
  • 42.Rideout V, Fox S, Peebles A, Robb MB. Coping with Covid-19: How young people use digital media to manage their mental health. [Internet]. San Francisco, California: Common Sense and HopeLab; 2021. [cited 2022 Jun 5]. Available from: https://www.commonsensemedia.org/sites/default/files/research/report/2021-coping-with-covid19-full-report.pdf [Google Scholar]
  • 43.Pretorius C, Chambers D, Cowan B, Coyle D. Young People Seeking Help Online for Mental Health: Cross-Sectional Survey Study. JMIR Ment Health. 2019. Aug 26;6(8):e13524. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Pretorius C, McCashin D, Kavanagh N, Coyle D. Searching for Mental Health: A Mixed-Methods Study of Young People’s Online Help-seeking. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems [Internet]. New York, NY, USA: Association for Computing Machinery; 2020. [cited 2021 May 18]. p. 1–13. (CHI ‘20). Available from: 10.1145/3313831.3376328 [DOI] [Google Scholar]
  • 45.Chaffey D Search engine marketing statistics 2022 [Internet]. Smart Insights; 2022. Jan [cited 2022 Jun 5]. Available from: https://www.smartinsights.com/search-engine-marketing/search-engine-statistics/ [Google Scholar]
  • 46.Bach RL, Wenz A. Studying health-related internet and mobile device use using web logs and smartphone records. Borsci S, editor. PLoS ONE. 2020. Jun 12;15(6):e0234663. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Rawal A Google’s new health-search engine - The startup [Internet]. The Startup: Medium; 2020. Jan [cited 2022 Jun 5]. Available from: https://medium.com/swlh/googles-new-healthcare-data-search-engine9e6d824b3ccd [Google Scholar]
  • 48.Nuti SV, Wayda B, Ranasinghe I, Wang S, Dreyer RP, Chen SI, et al. The Use of Google Trends in Health Care Research: A Systematic Review. Voracek M, editor. PLoS ONE. 2014. Oct 22;9(10):e109583. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Vaidyanathan U, Sun Y, Shekel T, Chou K, Galea S, Gabrilovich E, et al. An evaluation of Internet searches as a marker of trends in population mental health in the US. Sci Rep. 2022. Dec;12(1):8946. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Hedegaard H Suicide Mortality in the United States, 1999–2017. 2018;(330):8. [PubMed] [Google Scholar]
  • 51.Walsh CG, Ribeiro JD, Franklin JC. Predicting Risk of Suicide Attempts Over Time Through Machine Learning. Clinical Psychological Science. 2017. May;5(3):457–69. [Google Scholar]
  • 52.Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, et al. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin. 2017;143(2):187–232. [DOI] [PubMed] [Google Scholar]
  • 53.Phillips CA, Hunt A, Salvesen-Quinn M, Guerra J, Schapira MM, Bailey LC, et al. Health-related Google searches performed by parents of pediatric oncology patients. Pediatr Blood Cancer [Internet]. 2019. Aug [cited 2022 Jun 6];66(8). Available from: https://onlinelibrary.wiley.com/doi/10.1002/pbc.27795 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Adler N, Cattuto C, Kalimeri K, Paolotti D, Tizzoni M, Verhulst S, et al. How Search Engine Data Enhance the Understanding of Determinants of Suicide in India and Inform Prevention: Observational Study. J Med Internet Res. 2019. Jan 4;21(1):e10179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Asch JM, Asch DA, Klinger EV, Marks J, Sadek N, Merchant RM. Google search histories of patients presenting to an emergency department: an observational study. BMJ Open. 2019. Feb;9(2):e024791. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Wies B, Landers C, Ienca M. Digital Mental Health for Young People: A Scoping Review of Ethical Promises and Challenges. Front Digit Health. 2021. Sep 6;3:697072. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Areán PA, Pratap A, Hsin H, Huppert TK, Hendricks KE, Heagerty PJ, et al. Perceived Utility and Characterization of Personal Google Search Histories to Detect Data Patterns Proximal to a Suicide Attempt in Individuals Who Previously Attempted Suicide: Pilot Cohort Study. J Med Internet Res. 2021. May 6;23(5):e27918. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Fitzsimmons-Craft EE, Eichen DM, Monterubio GE, Firebaugh ML, Goel NJ, Taylor CB, et al. Longer-term follow-up of college students screening positive for anorexia nervosa: psychopathology, help seeking, and barriers to treatment. Eating Disorders. 2020. Nov 1;28(5–6):549–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.About MHA Screening [Internet]. Mental Health America. Available from: https://mhanational.org/about-mha-screening#ScreeningReportsandResearch [Google Scholar]
  • 60.Kruzan KP, Meyerhoff J, Reddy M, Mohr DC, Kornfield R. “I wanted to see how bad it was:” Mental health self-screening as a critical transition point among young adults. In Proceedings of the 2022 ACM Conference on Human Factors in Computing Systems (CHI). In Press; [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Rickwood D, Thomas K. Conceptual measurement framework for help-seeking for mental health problems. Psychol Res Behav Manag. 2012. Dec 6;5:173–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Kornfield R, Meyerhoff J, Studd H, Bhattacharjee A, Williams JJ, Reddy M, et al. Meeting Users Where They Are: User-centered Design of an Automated Text Messaging Tool to Support the Mental Health of Young Adults. In: CHI Conference on Human Factors in Computing Systems [Internet]. New Orleans LA USA: ACM; 2022. [cited 2022 Jun 5]. p. 1–16. Available from: https://dl.acm.org/doi/10.1145/3491102.3502046 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Kruzan KP, Mohr DC, Reddy M. How Technologies Can Support Self-Injury Self-Management: Perspectives of Young Adults With Lived Experience of Nonsuicidal Self-Injury. Frontiers in Digital Health. 2022;4(913599). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Fitzsimmons-Craft EE, Balantekin KN, Graham AK, Smolar L, Park D, Mysko C, et al. Results of disseminating an online screen for eating disorders across the U.S.: Reach, respondent characteristics, and unmet treatment need. Int J Eat Disord. 2019. Jun;52(6):721–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Fitzsimmons-Craft EE, Chan WW, Smith AC, Firebaugh M, Fowler LA, Topooco N, et al. Effectiveness of a chatbot for eating disorders prevention: A randomized clinical trial. Intl J Eating Disorders. 2022. Mar;55(3):343–53. [DOI] [PubMed] [Google Scholar]
  • 66.Singla DR, Raviola G, Patel V. Scaling up psychological treatments for common mental disorders: a call to action: LETTERS TO THE EDITOR. World Psychiatry. 2018. Jun;17(2):226–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Renfrew ME, Morton DP, Northcote M, Morton JK, Hinze JS, Przybylko G. Participant Perceptions of Facilitators and Barriers to Adherence in a Digital Mental Health Intervention for a Nonclinical Cohort: Content Analysis. J Med Internet Res. 2021. Apr 14;23(4):e25358. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Reinert M, Fritze D, Nguyen T. The State of Mental Health in America 2022. Alexandria, VA: Mental Health America; 2021. Oct. [Google Scholar]
  • 69.Jaworska S ‘Bad’ mums tell the ‘untellable’: Narrative practices and agency in online stories about postnatal depression on Mumsnet. Discourse, Context & Media. 2018. Oct;25:25–33. [Google Scholar]
  • 70.De Choudhury M, Counts S, Horvitz EJ, Hoff A. Characterizing and predicting postpartum depression from shared facebook data. In: Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing [Internet]. ACM Press; 2014. [cited 2018 May 8]. p. 626–38. Available from: http://dl.acm.org/citation.cfm?doid=2531602.2531675 [Google Scholar]
  • 71.De Choudhury M, Gamon M, Counts S, Horvitz E. Predicting Depression via Social Media. In: Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media. Association for the Advancement of Artificial Intelligence; 2013. p. 10. [Google Scholar]
  • 72.Andalibi N, Ozturk P, Forte A. Sensitive Self-disclosures, Responses, and Social Support on Instagram: The Case of #Depression. In: Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing [Internet]. ACM Press; 2017. [cited 2018 May 8]. p. 1485–500. Available from: http://dl.acm.org/citation.cfm?doid=2998181.2998243 [Google Scholar]
  • 73.Fitzsimmons-Craft EE, Krauss MJ, Costello SJ, Floyd GM, Wilfley DE, Cavazos-Rehg PA. Adolescents and young adults engaged with pro-eating disorder social media: eating disorder and comorbid psychopathology, health care utilization, treatment barriers, and opinions on harnessing technology for treatment. Eat Weight Disord. 2020. Dec;25(6):1681–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Cavazos-Rehg PA, Fitzsimmons-Craft EE, Krauss MJ, Anako N, Xu C, Kasson E, et al. Examining the self-reported advantages and disadvantages of socially networking about body image and eating disorders. Int J Eat Disord. 2020. Jun;53(6):852–63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75.Kruzan KP, Bazarova NN, Whitlock J. Investigating Self-injury Support Solicitations and Responses on a Mobile Peer Support Application. Proc ACM Hum-Comput Interact. 2021. Oct 13;5(CSCW2):1–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Williams AJ, Nielsen E, Coulson NS. “They aren’t all like that”: Perceptions of clinical services, as told by self-harm online communities. Journal of Health Psychology. 2018. Jul 19;135910531878840. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Smithson J, Sharkey S, Hewis E, Jones R, Emmens T, Ford T, et al. Problem presentation and responses on an online forum for young people who self-harm. Discourse Studies. 2011. Aug;13(4):487–501. [Google Scholar]
  • 78.Kruzan KP, Whitlock J, Bazarova NN. Examining the Relationship Between the Use of a Mobile PeerSupport App and Self-Injury Outcomes: Longitudinal Mixed Methods Study. JMIR Ment Health. 2021. Jan 28;8(1):e21854. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Fish JN, McInroy LB, Paceley MS, Williams ND, Henderson S, Levine DS, et al. “I’m Kinda Stuck at Home With Unsupportive Parents Right Now”: LGBTQ Youths’ Experiences With COVID-19 and the Importance of Online Support. Journal of Adolescent Health. 2020. Sep;67(3):450–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.McInroy LB, Craig SL. “It’s like a safe haven fantasy world”: Online fandom communities and the identity development activities of sexual and gender minority youth. Psychology of Popular Media. 2020. Apr;9(2):236–46. [Google Scholar]
  • 81.Anderson M, Jiang J. Teens, Social Media & Technology 2018. :10.
  • 82.Ridout B, Campbell A. The Use of Social Networking Sites in Mental Health Interventions for Young People: Systematic Review. Journal of Medical Internet Research. 2018. Dec 18;20(12):e12244. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83.Dobias M, Morris R, Schleider J. Single-session interventions embedded within Tumblr: A test of acceptability and utility. JMIR Form Res. In Press; [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 84.Schleider JL, Weisz JR. Little Treatments, Promising Effects? Meta-Analysis of Single-Session Interventions for Youth Psychiatric Problems. Journal of the American Academy of Child & Adolescent Psychiatry. 2017. Feb 1;56(2):107–15. [DOI] [PubMed] [Google Scholar]
  • 85.Schleider JL, Burnette JL, Widman L, Hoyt C, Prinstein MJ. Randomized Trial of a Single-Session Growth Mind-Set Intervention for Rural Adolescents’ Internalizing and Externalizing Problems. Journal of Clinical Child & Adolescent Psychology. 2020. Sep 2;49(5):660–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 86.McDanal R, Rubin A, Fox K, Schleider JL. Associations of LGBTQ+ Identities with Acceptability and Response to Online Single-Session Youth Mental Health Interventions [Internet]. PsyArXiv; 2021. May [cited 2021 Jul 20]. Available from: https://osf.io/v5tgc [DOI] [PubMed] [Google Scholar]
  • 87.Schleider JL, Mullarkey MC, Fox KR, Dobias ML, Shroff A, Hart EA, et al. A randomized trial of online single-session interventions for adolescent depression during COVID-19. Nat Hum Behav. 2022. Feb;6(2):258–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.Ching BC, Bennett SD, Morant N, Heyman I, Schleider JL, Fifield K, et al. Growth mindset in young people awaiting treatment in a paediatric mental health service: A mixed methods pilot of a digital single-session intervention. Clin Child Psychol Psychiatry. 2022. Jun 1;135910452211051. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 89.Schleider JL, Sung J, Bianco A, Gonzalez A, Vivian D, Mullarkey MC. Open Pilot Trial of a Single-Session Consultation Service for Clients on Psychotherapy Wait-Lists [Internet]. PsyArXiv; 2020. May [cited 2022 Jun 5]. Available from: https://osf.io/fdwqk [Google Scholar]
  • 90.Dugstad J, Eide T, Nilsen ER, Eide H. Towards successful digital transformation through co-creation: a longitudinal study of a four-year implementation of digital monitoring technology in residential care for persons with dementia. BMC Health Serv Res. 2019. Dec;19(1):366. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 91.Nordgreen T, Rabbi F, Torresen J, Skar YS, Guribye F, Inal Y, et al. Challenges and possible solutions in cross-disciplinary and cross-sectorial research teams within the domain of e-mental health. JET. 2021. Oct 12;15(4):241–51. [Google Scholar]
  • 92.Falk-Krzesinski HJ, Börner K, Contractor N, Fiore SM, Hall KL, Keyton J, et al. Advancing the Science of Team Science. Clinical and Translational Science. 2010. Oct;3(5):263–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 93.Hall KL, Vogel AL, Huang GC, Serrano KJ, Rice EL, Tsakraklides SP, et al. The science of team science: A review of the empirical evidence and research gaps on collaboration in science. American Psychologist. 2018. May;73(4):532–48. [DOI] [PubMed] [Google Scholar]
  • 94.Horowitz C, Shameer K, Gabrilove J, Atreja A, Shepard P, Goytia C, et al. Accelerators: Sparking Innovation and Transdisciplinary Team Science in Disparities Research. IJERPH. 2017. Feb 23;14(3):225. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 95.Nash JM. Transdisciplinary Training. American Journal of Preventive Medicine. 2008. Aug;35(2):S133–40. [DOI] [PubMed] [Google Scholar]
  • 96.Balas EA, Boren SA. Managing Clinical Knowledge for Health Care Improvement. Yearb Med Inform. 2000. Aug;09(01):65–70. [PubMed] [Google Scholar]
  • 97.Lyon AR, Wasse JK, Ludwig K, Zachry M, Bruns EJ, Unützer J, et al. The Contextualized Technology Adaptation Process (CTAP): Optimizing Health Information Technology to Improve Mental Health Systems. Adm Policy Ment Health. 2016. May;43(3):394–409. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 98.Patrick K, Hekler EB, Estrin D, Mohr DC, Riper H, Crane D, et al. The Pace of Technologic Change. American Journal of Preventive Medicine. 2016. Nov;51(5):816–24. [DOI] [PubMed] [Google Scholar]
  • 99.Mohr DC, Lyon AR, Lattie EG, Reddy M, Schueller SM. Accelerating Digital Mental Health Research From Early Design and Creation to Successful Implementation and Sustainment. J Med Internet Res. 2017. May 10;19(5):e153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 100.Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation Hybrid Designs: Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact. Medical Care. 2012. Mar;50(3):217–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 101.Nouri S, Khoong EC, Lyles CR, Karliner L. Addressing Equity in Telemedicine for Chronic Disease Management During the Covid-19 Pandemic. 2020;13. [Google Scholar]
  • 102.Johansen SL, Olmert T, Chaudhary N, Vasan N, Aragam GG. Incorporating Digital Interventions into Mental Health Clinical Practice: a Pilot Survey of How Use Patterns, Barriers, and Opportunities Shifted for Clinicians in the COVID-19 Pandemic. J technol behav sci [Internet]. 2022. May 10 [cited 2022 Jun 3]; Available from: https://link.springer.com/10.1007/s41347-022-00260-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 103.Lattie EG, Nicholas J, Knapp AA, Skerl JJ, Kaiser SM, Mohr DC. Opportunities for and tensions surrounding the use of technology-enabled mental health services in community mental health care. Administration and Policy in Mental Health and Mental Health Services Research. 2020;47(1):138–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 104.Schueller SM, Washburn JJ, Price M. Exploring mental health providers’ interest in using web and mobilebased tools in their practices. Internet Interventions. 2016. May;4:145–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 105.Mohr DC, Azocar F, Bertagnolli A, Choudhury T, Chrisp P, Frank R, et al. Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment. PS. 2021. Jun;72(6):677–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 106.Graham AK, Weissman RS, Mohr DC. Resolving Key Barriers to Advancing Mental Health Equity in Rural Communities Using Digital Mental Health Interventions. JAMA Health Forum. 2021. Jun 11;2(6):e211149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 107.Wilson CJ, Deane FP. Brief report: Need for autonomy and other perceived barriers relating to adolescents’ intentions to seek professional mental health care. Journal of Adolescence. 2012. Feb;35(1):233–7. [DOI] [PubMed] [Google Scholar]
  • 108.Samargia LA. Foregone Mental Health Care and Self-Reported Access Barriers Among Adolescents. 2006;22(1):8. [DOI] [PubMed] [Google Scholar]

RESOURCES