Debate: Conversational AI and young people's mental health – friend or foe? – the illusion of a safe space for youth mental health

Abstract

Artificially intelligent (AI) chatbots are increasingly used for mental health support, yet their safety guidelines and ethical structures remain unclear, particularly for youth populations. We put forward that many AI‐based mental health chatbots create the illusion of a safe and accessible therapeutic space. Drawing on observations from 117 publicly available applications, we contend that claims of full accessibility and confidentiality are countered by both explicit and at times implicit requirements to fully engage with the chatbot, including entering personal information, agreeing to extensive data sharing, and making payments. Applications often overstate the professional expertise and therapeutic outcomes delivered, potentially misleading users to overestimate application capabilities. With the growing popularity of AI chatbots among youth, it is essential to raise awareness about how they create the illusion of a safe space and empower users to make safe and informed decisions about their use.

Artificial intelligence (AI) has introduced a new generation of mental health tools in the form of therapy chatbots that employ natural language processing to simulate conversation, empathy, and emotional support (Zimmerman, Janhonen, & Beer, 2024). These systems range from clinically oriented programs such as Wysa and Youper, which claim to use cognitive‐behavioral therapy principles (Farzan, Ebrahimi, Pourali, & Sabeti, 2025), to commercially available chatbots like Replika, marketed for companionship (Laestadius, Bishop, Gonzalez, Illenčík, & Campos‐Castillo, 2022; Skjuve, Følstad, Fostervold, & Brandtzæg, 2021). Such tools promise 24/7 availability, anonymity, and low cost to users with limited access to professional care (Haque & Rubya, 2023).

However, AI use in mental health raises significant ethical and safety concerns. Many applications are marketed toward youth (13–24 years) who may be particularly susceptible to emotional influence and unaware of privacy risks. With AI chatbots often presenting as approachable digital companions, the line between therapeutic aid and social interaction is commonly blurred as emotional bonding, mimicked but not genuine (Zimmerman et al., 2024). The absence of consistent safety guidelines and policy raises concerns about negative impacts related to misinformation and overreliance.

Here we discuss the current context of publicly available AI therapy applications accessible to Canadian youth and highlight the implications of their use in youth mental health. To inform our position, we sampled and analyzed 117 publicly available AI therapy apps and websites accessible through major search engines and app providers between May and August 2025. Evidence related to requirements for account creation (e.g., age requirements, consent processes, and costs) and scientific grounding of the apps (e.g., mentions of expert verification, disclaimers about AI, and claims of impact on mental health outcomes) was reviewed to guide the present perspective. Our characterization of AI chatbots for mental health reveals important tensions between their promises and their pitfalls, calling for urgent efforts to raise awareness about risks. While AI therapy applications often portray themselves as credible and therapeutic, our examination highlights many misleading and unfounded claims regarding their benefits, creating expectations of professional expertise that cannot be provided.

AI chatbots are often framed as uniquely capable of providing constant, low‐cost mental health support (Haque & Rubya, 2023), potentially benefitting adolescents who experience parental stigma or lack financial support. Yet the promise of accessibility is undercut in practice: although some free options exist, 56% of applications operated on freemium or paywall models, and 24% restricted use by limiting AI support using maximum message counts or time limits unless users upgrade to a premium plan. This deceptive practice raises concerns regarding the gap between perceived accessibility of mental health support and users' actual experiences.

Another fundamental concern is how applications position themselves as capable of addressing clinical mental health challenges (e.g., ‘users report significant improvements in their mental health […] whether it's reducing anxiety, coping with depression’) without the foundations to support such claims. Nearly a third (29.9%) of apps and websites explicitly promised improvements in mental health outcomes, referencing anxiety (34%), depression (14.5%), and coping strategies (10.2%), yet only one in five (20.5%) claimed any expert endorsements or affiliations, and those endorsements were rarely substantiated. Applications often use expertise language to create a false sense of clinical knowledge, for example ‘[AI tool] offers evidence‐based virtual cognitive behavioral therapy (CBT) interventions’, obscuring the difference between evidence‐based care and features added solely to project credibility. The result of this is a misleading sense of authority, and a false belief that guidance from AI therapy applications is comparable to professional care.

Many AI therapy tools present themselves as companions, confidants, or empathetic figures capable of understanding users' feelings (Gauk, Amir, Putri, & Salimah, 2025). In our sample, 39.3% claimed to provide companionship or reduce loneliness (e.g., ‘the friend you wish you had’), 31.6% offered emotional support (e.g., ‘[AI tool can] truly understand the complexity of human emotions’), and a further 26.5% promised to help users improve their lives or sense of self (e.g., ‘unlock your best self’). While comforting, this approach promotes relational closeness with systems that cannot reciprocate with human judgment (Dosso, Riminchan, & Robillard, 2023). When these applications do not clearly identify themselves as non‐substitutes for professional help, users may form overly dependent or parasocial attachments. Reports of users forming intense emotional bonds with AI chatbots such as Replika illustrate the dangers of these dynamics (Laestadius et al., 2022). Among these dangers is a greater willingness to disclose information (Skjuve et al., 2021), raising serious concerns about privacy for young people that are in an already vulnerable situation.

Paradoxically, many AI therapy tools advertise complete anonymity yet require users to create accounts and disclose personal information before accessing the application. In our sample, 57.3% required account creation, and 60% lacked any privacy policy or disclaimer. This inconsistency between perceived confidentiality and the reality of data collection and sharing is particularly concerning in a mental health context, where users may reveal sensitive information under the assumption that their conversations are protected (O'Loughlin, Neary, Adkins, & Schueller, 2019). Without clear data management practices, AI applications infringe upon users' informed consent, leaving users unaware of how their personal data may be used for commercial purposes beyond immediate therapeutic interaction.

A further concern involves apps that promise unrealistic outcomes or recovery timelines. Among the 13.7% of applications making specific outcome claims, language resembled marketing slogans more than therapeutic guidance, such as promises to make users ‘feel better in minutes, not months’, or statistics like ‘91% of people feel better after talking to me just once’. Claims that promise improvement within a fixed timeframe can worsen user wellbeing when experiences differ (Rief & Glombiewski, 2017), potentially producing feelings of personal failure rather than relief.

While AI therapy is novel, we can look to analogous applications to draw parallels and valuable lessons. This trend is reminiscent of brain training games, which generated sensationalized optimism around their potential to improve brain health and cognition. However, these expectations have not yet been realized, as there is little evidence that such brain training improves cognitive performance (Simons et al., 2016), contrary to public perception. This mismatch highlights the need for large‐scale evaluations of effectiveness as well as strong guidance from respected authorities in the field.

Taken together, AI therapy applications fail to clearly communicate the nature and capabilities of their services, creating the illusion of a safe space without the transparency, privacy, and accessibility promised (Gauk et al., 2025). Although AI therapy chatbots appear as accessible and reliable mental health tools, many fail to meet the ethical standards needed for safe use among youth. Current applications raise concerns by creating misleading expectations regarding accessibility, professional expertise, anonymity, and therapeutic outcomes. Increasing awareness of the gap between advertised claims and actual capabilities is essential to promote informed engagement with AI therapy.

The potential harms of AI therapy tools carry important clinical implications, as mental health care providers and users alike must recognize that AI therapy chatbots do not provide the confidential and accessible support advertised. Understanding the role these tools play in youth mental health experiences may guide clinicians to better assess patient risk, address misconceptions, and direct people toward secure, evidence‐based sources of support.

Author contributions

AY: data curation, formal analysis, methodology, investigation, writing – original draft, writing – review and editing; KAT: data curation, formal analysis, methodology, investigation, writing – original draft, writing – review and editing; AP: data curation, formal analysis, methodology, investigation, writing – original draft, writing – review and editing; JMR: conceptualization, funding acquisition, resources, methodology, project administration, supervision, writing – review and editing.

Funding information

BC Children's Hospital Foundation.

Conflict of interest statement

The authors report there are no competing interests to declare.

Ethics statement

No ethics approval was required for this debate article.

Data availability statement

Data is available upon request.