Skip to main content
NPJ Digital Medicine logoLink to NPJ Digital Medicine
. 2026 Jun 19;9:763. doi: 10.1038/s41746-026-02888-9

Young people’s perceptions and recommendations for conversational generative artificial intelligence in youth mental health

Adam Poulsen 1,✉, Ian B Hickie 1, Carla Gorban 1, Zsofi de Haan 1, William Capon 1, Ebenezer Eyeson-Annan 2, Jalal Radwan 2, Elizabeth M Scott 1, Frank Iorfino 1, Haley M LaMonica 1
PMCID: PMC13639063  PMID: 42321405

Abstract

Conversational generative artificial intelligence agents (or genAI chatbots) could benefit youth mental health, yet young people’s perspectives remain underexplored. We examined the Mental health Intelligence Agent (Mia), a genAI chatbot originally designed for professionals in Australian youth services. Following co-design, 32 young people participated in online workshops exploring their perceptions of genAI chatbots in youth mental health and to develop recommendations for reconceptualising Mia for consumers and integrating it into services. Four themes were developed using reflexive thematic analysis: (1) Humanising AI without dehumanising care, (2) I need to know what’s under the hood, (3) Right tool, right place, right time?, and (4) Making it mine on safe ground. This study offers insights into young people’s attitudes, needs, and requirements regarding genAI chatbots in youth mental health, with key implications for service integration. Additionally, by co-designing system requirements, this work informs the ethics, design, development, implementation, and governance of genAI chatbots in youth mental health contexts.

Subject terms: Information systems and information technology; Psychology; Psychology; Science, technology and society; Social sciences

Introduction

Youth mental health is in crisis1. Worldwide, young people aged 12-25 are experiencing high and, in most cases, rising rates of anxiety, depressive symptoms, psychological distress, self-harm, suicide, and other mood disorders2,3. Contributing factors span psychological, social, and economic domains, creating a multifaceted challenge4. The wide spectrum of mental illnesses in presenting populations adds to the challenge of addressing all clients’ needs, risking the exclusion of some from care access5. Socioeconomic, cultural, and environmental factors compound the issue, adding stressors such as social stigma, employment and housing insecurity, intergenerational inequality, especially in low- and middle-income countries1,2. Worsening the problem, services are overburdened1 and suffer substantial gaps in workforce, service data, regulation, service specificity, coverage, and coordination2,6. Prevention and early, effective, and high-quality intervention of emerging mental illnesses at their peak onset of 15 years is essential to positively change the daily lives and future life-course of young people1. Yet, the state of youth mental health often leads to delayed, reactive, and inappropriate treatment decisions5,7.

The digitisation of youth mental health shows promise in supporting services and stakeholders and alleviating the present issues8. This includes the integration of various digital mental health technologies (DMHTs), including smartphone apps, videoconferencing platforms, digital phenotyping, virtual and augmented reality, and artificial intelligence (AI) systems9. Principally, DMHTs may help to optimise services with an improved, flexible, technology-driven user experience, generate actionable and useful data for mental health service providers and professionals to deliver highly personalised and measurement-based care, and provide new avenues for prevention, delivery, and treatment support through DMHT interventions9. Research indicates that the DMHTs support shared and informed decision-making by enhancing the breadth of information considered to best treat the myriad mental illnesses in presenting populations in youth mental health services10,11.

Advances in large language models (LLMs) and machine learning techniques present an opportunity for introducing conversational generative AI agents (or genAI chatbots) in youth mental health to engage users, classify large volumes of text, answer questions, translate languages, and generate new content based on user input beyond standardised, untailored responses of conversational AI alone12, benefiting mental health professionals and clients13,14. For professionals, research highlights that conversational genAI in mental health is positioned to increase efficiency to clinician workflows by supporting assessment15, assisting in developing empathic clinician documentation and client-facing interactions, and reducing time spent on administrative clinician tasks, thus recovering critical time with clients14. For clients, research indicates that conversational genAI in this context improves mental health outcomes16,17, reduces symptoms18, benefits psychoeducation and self-adherence19, and user satisfaction19, and supports health information comprehension and informed decision-making for non-experts20. Going forward, conversational genAI is expected to play a significant role in supporting accessible, scaleable, and personalised mental healthcare, owing to its considerable context-driven adaptability and data collection and processing capabilities21–23. This aligns with the existing trend of young people turning to publicly accessible, general-purpose systems, such as OpenAI’s ChatGPT, for mental health support24. Yet, these tools raise key questions around trust, usability, effectiveness, engagement, risk of harm, and inclusion of vulnerable groups16,25–29. This calls for meaningful co-design with users and other stakeholders to develop a rich understanding of their needs and preferences, along with the associated ethical, design, development, implementation, and governance priorities.

As is typical in digital health research, much of the literature focuses on evaluating technologies for functionality and effectiveness, risking obfuscation of qualitative insights concerning user perceptions, user experience, and human-computer interaction and human-centred design considerations30. Furthermore, potential end users are often only involved in the final stages of technology development, excluding an account of their perceptions, needs, and characteristics in the early stages31. A scoping review by Balan et al.32 on conversational AI in youth mental health noted this trend, finding that input from potential end users during the design and development stages is often overlooked. While a growing body of formative research has begun to explore stakeholder perceptions of AI in youth mental health contexts33,34, fewer studies have generated recommendations relating to the features, scope, content, personality, ethics, and subsequent design considerations to inform the development of conversational AI (and genAI35) in this space32,36.

This paper contributes to this emerging research by reporting the co-design findings from the development of a genAI chatbot, the Mental health Intelligence Agent (Mia)37, designed for enhancing triage, clinical assessment, formulation, treatment planning, and self-management in youth mental health services and examines young people’s relevant perceptions.

Results

Participant characteristics

A total of 32 young people participated in online workshops exploring their perceptions and recommendations about genAI chatbots in youth mental health generally and Mia specifically. Among them, 19 also completed an optional follow-up survey reporting demographics after workshop participation (see Fig. 1). The demographic characteristics reported here, therefore, reflect this subset, offering descriptive context about the sample rather than representing all workshop participants. They ranged in age from 18 to 30 years (mean age 25 years) and first sought mental health support at ages 9-23 years (mean age 18 years). The gender distribution was women (n = 11), men (n = 5), non-binary (n = 1), agender (n = 1), and not disclosed (n = 1). Overall, participants reported identifying with one identity (n = 10), multiple or intersectional identities (n = 5), or did not indicate an identity group (n = 4). Identities reported included lesbian, gay, bisexual, transgender, gender diverse, intersex and queer (n = 8), culturally and linguistically diverse background (referring, relative to Australia, to participants who self-identify based on a range of aspects such as country of birth, ancestry, parental birthplace, language/s spoken, and religious affiliation38) (n = 8), belonging to a religious and/or spiritual group (n = 4), carer responsibility (n = 1), neurodivergent (n = 1), chronic illness (n = 1), and none (n = 4), with identities reported non-exclusively. The majority reported lived experience of mental ill-health that had impacted their everyday lives (n = 14), with others reporting no lived experience (n = 4) or choosing not to respond (n = 1). Just over half reported currently receiving treatment (n = 10), while others reported not currently receiving treatment (n = 6) or opted not to disclose their treatment status (n = 2), with one choosing not to respond.

Fig. 1.

Fig. 1

Participant demographics.

Reflexive thematic analysis

All 32 participants’ workshop contributions were included in the qualitative analysis. Following inductive reflexive thematic analysis39, four themes were developed by identifying patterns of shared meaning in the data without a priori coding or themes: (1) Humanising AI without dehumanising care, (2) I need to know what’s under the hood, (3) Right tool, right place, right time?, and (4) Making it mine on safe ground. To follow is a brief description of the iterative theme development, refinement, and finalisation process undertaken in this study. Initially, six initial themes were developed: Operationalising AI in the mental health ecosystem; It’s a question of trust, safety, and evidence; Can AI handle mental health realities?; Making it mine and yours/Choicemaxing; Searching for authenticity; and Speaking my language. In collaboration with the wider research team, the initial six themes were iteratively revised, refined, split, and merged with reference to the study focus. Principally, this generated two additional potential themes: AI fit in the help-seeking process and Understanding the user. Ultimately, four themes were finalised, uniting the initial themes based on patterns of shared meaning in the data. Figure 2 reports the combined initial and final thematic map39, illustrating the theme development, refinement, and finalisation. Figure 3 reports the final thematic map39, indicating the relationships between the themes. Table 1 details the theme descriptions.

Fig. 2. Combined initial and final thematic map.

Fig. 2

The figure illustrates the theme development, refinement, and finalisation process in this study. Relationships indicate the refinement from initial to final themes.

Fig. 3. Final thematic map.

Fig. 3

The figure indicates the four themes identified in this study. Relationships between themes are indicated by directional lines and labels.

Table 1.

Theme descriptions

Theme Description
Humanising AI without dehumanising care GenAI chatbots are framed as supportive tools within youth mental health, provided they augment rather than replace the relational and empathetic role of human care.
I need to know what’s under the hood Trust in genAI chatbots in youth mental health depends on system transparency and accuracy.
Right tool, right place, right time? The appropriateness of genAI chatbots in youth mental health depends on their role within care pathways and the timing of their use.
Making it mine on safe ground Sustained engagement with genAI chatbots depends on their ability to personalise interactions while maintaining a sense of safety and support in the digital environment.

Theme 1: Humanising AI without dehumanising care

This theme speaks to young people’s perceptions of introducing genAI chatbots into the fundamentally human mental healthcare environment, and the necessary harmonisation between using advanced systems while preserving the centrality of humans in the delivery of care. Young people discussed the need to develop genAI chatbots equipped with human-like qualities to echo empathy, while ensuring that these tools support rather than replace clinicians.

‘It can’t outright replace actual psychologists and psychiatrists, because the human connection is still important at the end of the day, it’s not something that can be replicated by AI or chatbots’ (YP15)

Young people’s central concern about preserving human connection was driven by doubts about whether genAI chatbots could truly comprehend the complexity of mental health experiences. The profoundly personal, contextual, and nuanced nature of mental health was felt to exceed what genAI chatbots could understand, raising apprehension about them replacing real care and creating a sense of alienation and loss of human touch.

‘I feel a lot of the problems that I experience as a young person with lived experience are a lot more complex than maybe the advice that a system like this might be able to offer’ (YP7)

These concerns were compounded by more fundamental fears about genAI chatbot capabilities and their potential to dehumanise care through unintended consequences. These included genAI hallucinations and poor sustainability of genAI, as well as existing problems in health care that are at risk of being exacerbated with the introduction of genAI, such as poor information recall, risk of bias in decision-making, fostering unhealthy dependence or inappropriate attachment, and perpetuation of existing social stigma, leading to feelings of judgement. In this way, young people conveyed that the perceived cost of humanising genAI chatbots must not come at the expense of eroding the human foundation of mental healthcare, or the human experience generally. Additionally, they emphasised the need for ongoing AI ethics engagement and for considering a diversity of stakeholder perspectives beyond lived experience of mental ill-health in development processes to ensure these tools serve all young people appropriately and best advance the tailoring of user experiences and delivery of personalised outputs rather than a generic approach, necessitating a broad and diverse training data set and application of ethical frameworks.

During user testing, which utilised a version of Mia originally designed for professionals (see Fig. 4), some young people tested these concerns by challenging the genAI chatbot with difficult queries, complex or vague mental health experiences and concerns, and non-English language inputs in an effort to break it. This scepticism, as well as comparisons made to other tools like ChatGPT, reaffirmed young people’s doubts about whether current genAI chatbots (either general or purpose-built) are meaningful and sophisticated enough for mental healthcare.

Fig. 4. Original Mental health Intelligence Agent (Mia) user interface intended for health professionals, utilised for early stages of user testing.

Fig. 4

a Home screen. b First user input screen. c Main screen with persistent chat panel (left) showing ongoing messaging between the user and system, and tabbed information panel (right) showing the structure of information to be collected and insights to be generated. d Information threshold reached and thus insights generated, showing recommendations and information organised by mental health dimensions (composite image due to the information panel being scrollable). e Timeline tab, showing a suggested event to add to the user’s information automatically derived from user input.

Yet, young people also recognised the importance of designing genAI chatbots to reflect human-like empathy and acknowledged the potential value in thoughtfully developed tools. This was especially important for building rapport, trust, and credibility, evidencing their value. Primarily, this was expected to be achieved through genAI chatbots being equipped with key language capabilities. Those included, having an emotionally resonant, supportive, and personable tone, using accessible language that communicates mental health information without sounding clinical, nuanced understanding of youth and mental health related language (including multilingual capabilities), ability to sensitively address confronting topics like suicide and death, adaptive communication tailored to individual users, and clear prompts to elicit key mental health information from users without pressuring them for responses. Beyond language, young people also emphasised that human-like empathy in genAI chatbots ought to be realised through high standards for information recall, case formulation accuracy, and intersectionality sensitivity.

‘Obviously you want the chatbot to be professional, but you also don’t want them to be lacking of empathy’ (YP19)

Theme 2: I need to know what’s under the hood

This theme captures young people’s expectations of system transparency and accuracy in their interactions with genAI chatbots in youth mental health. Young people expected transparency across multiple dimensions, including being able to inspect a genAI chatbot’s evidence base, examine its decision-making processes, and have clarity on how their data would be used and stored. Equally important were expectations about accuracy. Young people wanted genAI chatbots to interpret user inputs aptly, draw appropriate inferences from them, and provide reliable outputs grounded in evidence.

Based on user testing feedback, they valued functionality that provided insight into the inner workings of genAI chatbots and enabled traceability. Specifically, they appreciated that the genAI chatbot could explain its interpretations of user inputs plainly, identify its rationale for extracting and organising specific components of the user inputs, and detail the reasoning process from initial inferences about the user through to treatment, self-management, and service navigation recommendations, with each step linked to evidence-based knowledge from the research literature. With this information made visible, they felt positioned to make their own independent determinations about the accuracy of system interpretations, decisions, and outputs, rather than accepting the genAI chatbot’s conclusions at face value.

‘A lot of people say, “Oh, you can’t trust AI 100%”. And I’m sure you can’t really trust anything 100% but just to show that it bases its answers on reasonable things, I think it’s important to show its thought process’ (YP14)

Most young people noted that transparency and traceability are empowering, enabling greater agency over their mental health information, which in turn advances trust and credibility in genAI chatbots. However, some also identified a potential tension about being too transparent. That is, presenting AI-generated insights that suggest negative trajectories about one’s mental health might undermine empowerment by triggering negative feelings and disengagement from the tool and mental health services more broadly, regardless of accuracy. This was especially a concern in a hypothetical discussion about genAI chatbots detecting risk of immediate risk factors, such as imminent physical violence or high rates of suicidal thoughts and behaviours. Despite these concerns, the prevailing view was that withholding information (either positive or negative) from users by design and stifling system transparency is intrinsically disempowering and unethical.

‘I feel like information that’s available to clinicians…like the point of a clinician is to be explaining stuff to you, right? So, it doesn’t really make sense for the info [sic] to be gate-kept like that. I feel I would be a bit annoyed if I found out that there was a clinician’s view on this and I didn’t have access to all of it’ (YP28)

Young people offered key insights into how transparency and accuracy expectations could be better established within system design. To advance transparency, they recommended addressing these requirements at the onboarding stage through an explanatory introduction detailing the genAI chatbot’s functions, data practices, reasoning processes, and underlying architecture. Clear communication about system operations from initial contact was viewed as essential for setting appropriate expectations and establishing a foundation for trust. To enhance accuracy, they recommended scaling the knowledge base of genAI chatbots to incorporate global research insights and broader and more diverse data sources and variables. Figure 5 presents the revised Mia user interface intended for young people rather than professionals, reflecting changes made in response to young people’s feedback. Revisions shown include new introductory information with user disclaimers and system operations details and an information completeness visualisation, both introduced to address participant feedback calling for greater transparency around data practices, system operations, and system decision-making and reasoning processes.

Fig. 5. Revised Mental health Intelligence Agent (Mia) user interface updated for young people based on feedback, used for later stages of user testing.

Fig. 5

a Home screen, with added introductory panels featuring user disclaimers and system operations information. b First user input screen. c Main screen, with revised information panel (right) featuring an information completeness visualisation to communicate how much information the system has about the user and in-progress assessment insights listing what the system knows about the user, what topics it is currently exploring with the user, and the system’s working assumptions about the user (composite image). d Information threshold reached and thus insights generated, showing revised information panel featuring personal experiments (previously, recommendations), missing pieces, and service needs index visualisation (shown as a popout here due to space limitations). In-progress assessment insights are reframed as missing pieces. Information completeness hidden from user when threshold is reached (composite image).

Young people also assessed genAI chatbots against their existing experiences with DMHTs (e.g. mental health apps) and other technologies used for mental health support, including social media platforms and general-purpose AI tools (e.g. ChatGPT). These experiences shaped their expectations. They shared that these digital tools could offer accessibility and anonymity while also posing risks, including misinformation, echo chambers, and superficial engagement. In contrast, they expected genAI chatbots in youth mental health to meet higher standards of both transparency and accuracy than unregulated information and tools operating outside of mental health service environments,requiring any advantages in accuracy, evidence-grounding, and appropriate safeguards to be demonstrable and communicated directly to users.

‘In terms of accuracy, it’s pretty important to me that it is good. Otherwise, it’s pretty worthless […] Just the retention of information from an LLM is just not at the same level, or even anywhere close to, obviously, the clinician’s ability to interpret and understand and remember back to previous conversations and have context’ (YP12)

Theme 3: Right tool, right place, right time?

This theme addresses young people’s fundamental questions and reactions about the fit of genAI chatbots in youth mental health. It examines whether genAI chatbots represent the right tool for the job, positioned at the right touchpoints within the service ecosystem, and available at the right times in young people’s care journeys. Young people discussed how genAI chatbots might fit within the existing multi-stakeholder sociotechnical system of mental healthcare, what roles they should appropriately fill, where young people would encounter them, and what value those interactions would have. They also explored how these chatbots could be integrated into established services, practices, and relationships without being disruptive. Important framing here is that the discussions focused on service integration, scoping the conversation to explore genAI chatbots that are made accessible only to young people who were seeking care or actively engaged in care at a service, ranging from initial contact with the mental health system to being a regular client.

Much of the discussion centred on identifying the potential roles that genAI chatbots could play in youth mental health. First, genAI chatbots could serve as navigators to help young people identify, understand, and access the right mental health services. Here, providing access means providing service-approved contact information or website links to young people for services applicable to their needs. Some young people noted that these recommendations should be categorised by location and cost, enabling users to make decisions aligned with their preferences and needs, but without requiring additional potentially invasive personal data collection, such as location sharing. Second, they could function as assessment tools (or assessors) to iteratively interpret user input concerning a young person’s mental health information (e.g. previous and current symptoms, case formulation, support, and service engagement) and other associated contextual factors, elicit information relating to critical signs to assess for in young people, develop summaries detailing a young person’s needs, and generate highly personalised and context-driven treatment, self-management, and service navigation recommendations ongoingly. Third, genAI chatbots could serve as mental health educators to detail and compare different treatments, outline the steps involved in engaging with a service or treatment, inform individuals about what to expect at different service engagement stages, and communicate the research evidence about particular mental health conditions, treatments, and self-management strategies, ultimately benefiting mental health literacy and supporting informed decision-making. Other minor roles briefly mentioned included supporting the creation of personalised support plans and safety plans, serving as informal spaces where young people can vent or seek advice, and facilitating some administrative processes such as booking appointments.

‘I think it just provides a bit more continuity of care in terms of the young person feeling like they have somebody that they can touch base with that’s not just their clinician. And I think it could serve the purpose of checking in that the service is still the right fit for the young person […] just check in and say, “Hey, it’s been six weeks with your current service. I wanted to check in and see if you still feel as though this service is providing the right support and meeting your needs. Is there anything that you feel like you need more support in or that would be better suited to you?”’ (YP19)

Young people considered the touchpoints where users would encounter genAI chatbots within their care journeys. Key touchpoints included initial contact with a service (or pre-intake), intake processes, preparing for clinical sessions, situations requiring instant support, and ongoing service engagement (any point after intake). From these two dimensions (i.e. roles and touchpoints), specific use cases emerged that illustrated how navigation, assessment, and education could be enacted at different moments in young people’s engagement with genAI chatbots in youth mental health (see Table 2). While certain use cases are inherently linked to specific touchpoints, such as self-screening at pre-intake, completing an initial assessment during intake, or accessing emergency contacts when in crisis, some use cases are ongoing and not bound to particular touchpoints. Young people expressed that it was important for users to be able to access psychoeducational content, ask questions about treatment, self-management, and service navigation recommendations and ideate alternatives, receive updated recommendations based on current needs, review personal information, and use mental health tracking functions at any point after intake. Table 2 details the use cases organised by the touchpoints where they are most salient, while recognising that the utility of genAI chatbots extends beyond these discrete moments (see the ongoing service engagement touchpoint).

Table 2.

Use cases for genAI chatbots in youth mental health emerging from the workshop discussions, organised by touchpoints in the care journey

Touchpoint Use case
Pre-intake

• Complete self-screening to understand if help is needed (assessor)

• Explore service options based on needs without commitment (navigator and assessor)

• Explore care options based on needs without commitment (assessor and educator)

Intake

• Orient to a service based on needs (navigator and educator)

• Book initial appointment (navigator)

• Complete intake assessments and forms (e.g. service registration forms or consent for treatment) (assessor and navigator)

• Access content about the intake process and what information is needed (educator)

• Create initial recommended treatment plan based on needs (assessor)

• Create initial safety plan (assessor and navigator)

Preparing for clinical sessions

• Receive reminders about appointments (navigator)

• Receive reminders about therapy homework (navigator)

• Access pre-session preparation materials (educator and navigator)

Moments of crisis

• Orient to an appropriate level of care based on risk assessment (assessor and navigator)

• Receive assessment-based, distress management recommendations (e.g. self-care and coping strategies) (assessor and educator)

• Access emergency contacts or crisis service information (navigator)

• Document crisis episodes for later review with a clinician (assessor)

Ongoing service engagement

• Access psychoeducational content about mental health concerns (educator)

• Query about treatment, self-management, and service navigation recommendations (educator and navigator)

• Receive updated personalised recommendations based on current needs (assessor and educator)

• Review personal information (e.g. self-care recommendations, safety plans, timeline, tracked data) (assessor and educator)

• Track symptoms, mood, or other metrics (assessor)

Young people reflected on broader issues within the mental health system, including access, efficiency, and quality of care, and raised a key consideration about whether genAI chatbots could address these issues. They noted that genAI chatbots could potentially improve access to support and care by supporting intake processes, managing demand on overstretched services, reducing wait times, and enabling a soft entry into care for hesitant young people who may find traditional service engagement difficult for a variety of reasons, such as stigma, privacy, and power imbalance concerns. The option to interact with a non-judgmental system before encountering mental health professionals was viewed as particularly valuable for reducing barriers to help-seeking. Additionally, genAI chatbots could enhance care efficiency by automating routine tasks and improving user data retention and sharing, thereby reducing the burden of repeatedly retelling one’s historical and current mental health experiences and concerns across multiple service touchpoints, which is a common source of frustration and re-traumatisation for young people navigating fragmented care systems.

When discussing what would make interactions with genAI chatbots genuinely useful at various touchpoints, young people emphasised the importance of treatment, self-management, and service navigation recommendations. They wanted genAI chatbots to provide recommendations personalised to individual circumstances and actionable. Young people wanted to see both treatment and self-management options, i.e. those requiring implementation with a clinician (e.g. cognitive behavioural therapy) and those centred around self-care that young people could implement independently between formal care contacts (e.g. mindfulness). Moreover, they expected that both types of care options would be tailored to them based on their previous experiences with mental healthcare, offering alternatives to previously unsuccessful treatments. This requirement positioned genAI chatbots as tools that could extend the reach of professional care and empower young people to take active roles in managing their mental health. However, young people also identified the potential risk of conflicting recommendations between genAI chatbots and clinicians that could undermine trust in both the technology and professional care, highlighting the need for careful integration and alignment with clinical practice.

Theme 4: Making it mine on safe ground

This theme captures what enables young people to sustain engagement with genAI chatbots in youth mental health, with a particular emphasis on the dual requirements of choice and safety. Young people highlighted that meaningful engagement depends on the capacity to customise interactions to suit their individual preferences while feeling confident that they are protected, both in terms of data privacy and security and appropriate system safeguards to ensure safe interactions. Without choice, the genAI chatbots are at risk of feeling non-personalised, restrictive, overbearing, and inflexible. Without safety, they are susceptible to being perceived as risky and untrustworthy. Both elements were expressed as prerequisites for young people to invest in using these tools over time.

Young people emphasised the importance of maximising choices about how they interact with genAI chatbots wherever possible and appropriate. This included preferences about interaction modalities (e.g. text versus voice input), access points (e.g. mobile app versus web app), the structure of conversations (e.g. guided by prompts versus open-ended), and the pacing of engagement (e.g. quick check-ins versus extended interactions). This trend resonated throughout all the workshops, with young people making it clear that one size does not fit all. Individuals have varying and evolving communication preferences, access modalities, comfort levels, and needs at different touchpoints and with genAI chatbots serving different roles, so providing options to allow users to tailor their experience was suggested to increase the likelihood of sustained use.

Related to choice was the question of how much information genAI chatbots in youth mental health should provide to users. Young people had diverse preferences about the level of detail they wanted. Again, echoing the importance of choice, some appreciated comprehensive explanations while others found that too much information is overwhelming. The majority expressed a preference for an opt-in approach where concise responses are provided by default, and users can request more detailed information. This approach respects that information needs vary by individual, context, and topic, and empowers users to control the depth of engagement rather than having genAI chatbots (or designers) dictate it.

Young people also discussed practical questions about how genAI chatbots should be accessed technically. There was an emerging preference for access via mobile apps, recognising that young people primarily use smartphones for digital interactions and that apps provide a more seamless, familiar, and on-demand experience compared to web browser-based platforms. Additional questions arose regarding whether users need to register personal accounts with identifying information, whether genAI chatbots require internet connectivity, and how accessibility features should be incorporated for users with diverse needs. These technical access considerations impact whether young people can meaningfully engage with genAI chatbots in the way they want.

Safety was considered non-negotiable for engagement, with young people expressing concerns across two distinct but interconnected dimensions: data safety and interaction safety. On data safety, young people wanted clarity on what data would be collected, how it would be stored to maintain privacy and security, who would have access to it, how long it would be retained, and whether it would be used to retrain genAI chatbot models. Beyond organisational data security measures and privacy policies to ensure data safety, young people wanted user-side control over their own data and privacy. A critical aspect of this control concerned protecting the privacy of their raw data (e.g. user inputs). Young people emphasised that for the sake of promoting unfiltered interactions and protecting privacy and autonomy, their raw data should remain private and under their control. Specifically, they wanted to dynamically decide whether to share the raw data and with whom.

On interaction safety, young people emphasised that genAI chatbots should recognise and respond appropriately when a user is at risk. Young people expected these tools to detect when users discuss unsafe situations (e.g. sharing imminent plans of self-harm or violence) and when patterns suggest deteriorating mental health or elevated risk. When such risks are detected, young people suggested users should be directed to mental health professionals (e.g. online chat or local emergency crisis support services).

Data safety and interaction safety intersect in complex ways. Young people’s desire for data privacy (i.e. keeping raw data private) exists in tension with the need for interaction safety (i.e. ensuring clinicians can intervene when risk is detected). Young people resolved this tension by distinguishing between raw data and AI-generated insights. While raw conversational data should remain private, AI-generated insights developed from genAI chatbot interactions (i.e. summaries detailing a user’s needs, personalised treatment, self-management, and service navigation recommendations, and deterioration or risk detection) should be accessible to clinicians and services. This distinction allows genAI chatbots to fulfil protective functions (monitoring for risk, flagging concerning patterns, enabling timely intervention, supporting coordinated care, and upholding the duty of care) without compromising the privacy of users’ raw data. In this model, young people maintain a private space for authentic expression while clinicians receive the clinically relevant information needed to provide appropriate support and ensure safety.

‘So in the clinician’s view […] if they can also see the recommendations that the clients going to get, that would be helpful for not only safety, but also for the clinician to know what their clients kind of do as well’ (YP12)

Discussion

This study engaged 32 young people in workshops to explore their perceptions of genAI chatbots in youth mental health and to develop recommendations for transforming Mia (originally a tool for health professionals) into a solution for consumers and integrating it into youth mental health services. Four themes were developed that, together, describe how these technologies might meaningfully support mental health, realise potential benefits, navigate possible risks, and fit within the broader sociotechnical context. The themes address critical knowledge gaps regarding genAI chatbots for consumers in youth mental health. Namely, how to design human-like AI without devaluing human-delivered care (Theme 1), what transparency mechanisms are positioned to build trust in high-stakes mental health contexts (Theme 2), where and when genAI chatbots may add value across care journeys (Theme 3), and how to sustain engagement while balancing user choice and system safeguards through nuanced data governance (Theme 4).

Findings indicate that young people expected genAI chatbots in youth mental health to communicate empathetically while also fearing that human-like genAI might inappropriately reduce access to human clinicians or be used to justify reduced access. This tension reflects longstanding ethical concerns about technologies in care settings mimicking human qualities to substitute rather than supplement human relationships40. Furthermore, it echoes perspectives emphasising that human empathy in caregiving cannot be fully replicated by advanced systems, as they lack the understanding of lived human experience necessary for authentic care41,42. Scepticism about whether genAI chatbots could comprehend the profoundly personal, contextual nature of mental health experiences is grounded in phenomenological understandings that lived experience resists algorithmic reduction43. Still, the enduring expectation for empathetic genAI chatbots presents a conflict that requires careful balancing. To address it, high standards for youth-centric language adaptation, information recall, case formulation accuracy, and intersectionality sensitivity were proposed as acceptable advances toward human-like empathy in genAI chatbots.

Subsequently, another critical tension emerged around performance standards. Young people in this study held genAI chatbots to higher standards than human clinicians, expecting exceptional system comprehension while acknowledging that clinicians regularly make interpretive errors. This expectation asymmetry reflects how people apply the machine heuristic, i.e. the rule of thumb that machines are more objective, perform tasks with greater precision, and handle information more securely than humans, thereby holding systems to higher reliability standards44. Furthermore, if a user observes system imprecision, then algorithm aversion may arise, i.e. ‘the tendency to prefer human judgements over algorithmic decisions even when it is suboptimal’44. In mental health contexts, this takes on particular significance and may be explained by the fact that genAI chatbots lack the therapeutic relationship, which has consistently been shown to drive significant change in clinical outcomes45 and allows human errors to be tolerated and repaired relationally46. Ultimately, young people still considered human-delivered care more desirable and authentic despite recognising instances of negative clinical experiences, reflecting relational cultural theory’s insight that connection itself, not merely competent intervention, defines healing47.

Relatedly, young people needed transparency of system functioning, knowledge base, reasoning, data practices, and accuracy. In mental healthcare, where errors can have serious consequences, such scrutiny is appropriate and aligns with established principles about transparency and interpretability for developing trustworthy and safe AI48. The connection between transparency and safety concerns in the workshop discussions, particularly regarding data breaches and unauthorised access, reflects research showing privacy is foundational to mental health help-seeking, given persistent stigma49.

While transparency was generally perceived as enabling empowerment and agency, young people recognised scenarios in which complete disclosure of AI-generated insights to users might be counterproductive, i.e. negative trajectories becoming self-fulfilling or triggering disengagement. Although this concern has been identified in the broader health literature relating to transparent information presentation practices and the risk of overwhelming clients50, young people’s perspective here suggests more salient risk to mental health outcomes and advocates a context-dependent rather than universal transparency approach. Additionally, young people’s acknowledgement that decisions about risk detection implementation must be governed by clinical standards, not user preferences alone, reflects a nuanced understanding of when their agency should be steered by clinical responsibility51 and user safety35.

Through co-design, young people established Mia’s role as a guide, joining the navigator, assessor, and educator roles across care journey touchpoints (see Table 3). Importantly, young people distinguished the guide role from companionship or friendship, emphasising that a genAI chatbot functioning as a friend would be inappropriate in mental health contexts where therapeutic boundaries and clinical relationships are paramount. This conception aligns with highly personalised and measurement-based care, positioning digital tools as support across care pathways52. Framed as a guide, Mia should draw out and retain mental health histories, reducing the burden and re-traumatisation from retelling experiences in fragmented care systems53; actively elicit information for developing personalised recommendations, consistent with evidence that assessing a client’s strengths and resources is important for treatment planning54; and generate actionable self-management recommendations central to recovery54. Once a young person is linked to a service, Mia should gather updated information to revise recommendations, prepare users for sessions, conduct regular check-ins, and connect clients to immediate crisis support if needed. These functions address the persistent challenge of care continuity55 and adhere to best practice, including between-session support that improves outcomes56, immediate access to coping resources during distress that prevents escalation57, and high-quality, reliable information access, which is crucial to support young people to self-advocate and make informed decisions58.

Table 3.

Mia as a guide across care journey touchpoints

Touchpoint Navigator function Assessor function Educator function
Pre-intake Explore service options without commitment Complete self-screening and explore treatment options Access psychoeducational content about mental health concerns
Intake Orient to service and book appointments Complete assessments and forms, and create treatment and safety plans Learn about intake process and treatment options
Between sessions Reminders about appointments and homework Access personalised self-care recommendations Pre-session preparation materials
Crisis Direct to appropriate care level and provide emergency contacts Risk assessment and distress management recommendations Evidence-based coping strategies
Ongoing Query about service navigation Track symptoms and mood, and receive updated recommendations Query about treatments and build mental health literacy

Young people’s design requirements reflect broader youth digital mental health trends. Comprehensive user control mechanisms emphasise the importance of choice across modalities, consistent with evidence that personalisation improves engagement59. The distinction drawn between raw conversational data (private and user-controlled) and AI-generated insights (shareable for coordination) evidences an alternative data sharing approach for alleviating privacy-oversight tensions documented in youth mental health services, which typically follows a user-controlled, opt-in procedure60. Moreover, this approach appropriately balances young people’s confidentiality with the clinical duty of care expected in a client-centred and empowering approach to youth mental health61. The two-part safety framing, i.e. distinguishing data safety from interaction safety, offers a useful structure for protective design going forward. The interaction safety expectations identified align with crisis intervention best practices57 and user needs regarding genAI chatbots in mental health reported elsewhere35, though current solutions often lack sophistication in handling mental health crises appropriately34.

Successfully integrating genAI chatbots in youth mental health requires careful consideration and balancing of stakeholder needs and requirements, existing governance structures, and sociotechnical systems. Crucially, humanising genAI chatbots requires more than technical sophistication. Following responsible innovation principles62, services must clearly communicate the role, capabilities, limitations, and appropriate expectations of these tools. Without such framing, gaps between expected and actual performance may undermine trust and risk disengagement. Integration challenges around data access expectations from multiple stakeholders with legitimate interests in young clients’ wellbeing (e.g. guardians, educational institutions, and allied health services) that may conflict with youth privacy preferences reflect broader tensions in services63. Young people’s nuanced perspective on privacy and safety challenges narratives positioning them as either privacy-unconcerned or categorically opposed to monitoring64, underscoring the importance of considering these views to inform integration. Per sociotechnical systems theory, technology must fit operating contexts or risk disruption65, and thus, without clarity about usage parameters, workflow integration, and governance frameworks, poorly integrated genAI chatbots risk disrupting care66.

This work co-designed several concrete implications for the design, development, implementation, and governance of genAI chatbots in youth mental health. Design priorities include adaptive communication to manage expectations about system limitations, transparency and explainability in the presentation of potentially distressing information, granular privacy controls differentiating raw data from AI-generated insights, and potential risk response protocols governed by clinical standards. Development needs include diverse training data to reduce bias in AI-supported decision-making, robust user testing with young people to challenge systems, infrastructure supporting differential data access, and service system interoperability. Implementation requires transparent onboarding processes and positioning genAI chatbots as complementary to human-delivered care. And governance requires clinician intervention pathways during crises, transparency about data practices, multi-stakeholder governance frameworks, and mechanisms for sustained lived experience input.

While the findings above have been discussed primarily in terms of young people’s expectations and design requirements, it is critical to recognise these as substantive ethical concerns that transcend individual preferences and usability. Questions persist about whether genAI should be deployed in youth mental health, given the complexity of mental health, vulnerability of young people, and the fundamental role of human connection67. Young people’s conditional acceptance advances but does not settle fundamental questions about dehumanisation risks, algorithmic decision-making requiring empathy and relational understanding, or the moral status of human-AI relationships in care. Yet, these perceptions provide some insight into how these technologies can be designed to avoid dehumanising care, as well as delineating the circumstances where they should and should not be deployed. Equity considerations carry particular moral weight. That is, genAI risks creating stratified care systems in which resourced individuals receive human support, amid global limitations on specialised service availability, while marginalised populations receive technological substitutes68,69. This raises concerns about justice when institutional efficiency benefits accrue to service systems while vulnerable young people experience diminished relational care. Nevertheless, notwithstanding this injustice, those within marginalised populations who possess sufficient technical competency to utilise DMHTs are situated to gain huge advantages due to better quality care associated with use of advanced tools35. This points to a further underlying concern, namely that those lacking adequate digital skills and access will not equally benefit, calling for improved digital access and literacy among marginalised groups70. The transparency, privacy, and data governance requirements reported here are not merely preferences but ethical imperatives grounded in respect for autonomy, informed consent, and protection from harm51. Automated risk detection exemplifies complex ethical tensions between beneficence and autonomy in bioethics71. While young people in this study demonstrated sophisticated reasoning about proportionality (privacy for raw data and oversight for risk), implementing such a balance requires navigating competing values (i.e. protection versus liberty and a clinician’s duty of care versus a client’s self-determination) where stakeholders may disagree. Beyond being technical or design problems and solutions, these are ethical considerations that require an ethical framework, stakeholder deliberation, and ongoing reflection to address going forward.

Several limitations of this work should be noted. First, the study participants consisted of young people aged 18-30 who were willing to participate in mental health services research in the Australian context. As a result, the generalisability of the findings and conclusions drawn is inevitably shaped by local context, participant ages, and the absence of certain voices and perspectives in this study, particularly those most marginalised due to associated stigma or being younger than 18 years. Yet, drawing on the strengths of reflexive thematic analysis, the qualitative analysis engaged a diverse group of researchers (including lived experience researchers) at multiple points to contribute a range of perspectives and to develop a comprehensive examination of young people’s perceptions and recommendations. Second, all workshops were conducted and recorded remotely, and thus, in some instances, the quality and accuracy of the transcripts were affected by internet connectivity issues. Third, despite efforts to enhance co-design inclusiveness (i.e. enabling remote participation, engaging lived experience researchers, and purpose sampling), broaden interpretive perspectives (i.e. engaging multiple, diverse researchers in data collection and analysis), and prevent a reductionist framing (i.e. featuring direct participant quotations in reporting), such limitations cannot be fully eliminated. Fourth, the primary focus on a single genAI chatbot (Mia) in a single context limits transferability, though the identified principles and implications are likely broader in relevance. Finally, participants engaged with functioning prototypes at one time point rather than final products over an extended period of user testing, and thus, user experiences may differ from anticipated ones.

Overall, young people reported the value of genAI chatbots for consumers in youth mental health, contingent on humanising AI without dehumanising care, transparency, strategic positioning and service integration, and balanced control with safety for sustained engagement. Their nuanced perspectives and recommendations reflect the complexity of introducing genAI into domains defined by vulnerability, trust, and relationships, as well as a careful balancing of competing priorities among stakeholders. The challenge ahead for integrating genAI chatbots into youth mental health services extends beyond technical capability to encompass managing expectations, framing appropriate roles, and ensuring technological pursuits do not devalue human relationships defining meaningful care. Realising potential requires centring youth voices as ongoing partners in design, development, implementation and governance, honestly grappling with inherent tensions, and prioritising wellbeing over technological enthusiasm or cost savings. Co-designed insights reported here provide a foundation going forward, but translating principles into ethical, effective, and equitable systems that advance highly personalised and measurement-based youth mental healthcare requires ongoing, meaningful work to appreciate sociotechnical systems and embed lived experience engagement.

Methods

Conversational generative artificial intelligence agent development

We developed a prototype genAI chatbot and agentic system for youth mental health, the Mental health Intelligence Agent (Mia)37, and made it available to young people who participated in this study. Mia was prototyped with insights from internal stakeholders representing expertise in psychiatry, general and clinical psychology, youth mental health service management, technology development, and human-computer interaction37. In use, Mia interprets user input concerning a young person’s mental health information (e.g. previous and current symptoms, case formulation, support, and service engagement) and other associated, contextual factors; highlights missing critical information and produces structured summaries and timelines; elicits information relating to critical signs to assess for in young people, principled multidimensional assessment, and confidence threshold criteria for developing a treatment plan; generates prioritisation guidance; and presents highly personalised and context-driven guideline-linked suggestions for advising treatment options, future assessment, and service navigation. Following an iterative and responsive co-design approach, regular design sprints were conducted throughout the study to incorporate evolving insights from young people into subsequent Mia prototype versions, which were then shown to new participants. More details on specific features and functionality of Mia, as well as the user interface, are provided throughout the presentation of findings.

Mia is built on the UNCAPT Platform, a multi-stage agentic reasoning architecture comprising three core components: (1) a Living Knowledge Bank built from the Brain and Mind Centre Youth Model of Care literature54,72, clinical guidelines, and expert annotations; (2) a set of Stage Verifiers, i.e. fine-tuned evaluator models that govern each stage of the clinical reasoning workflow; and (3) a Domain Orchestrator that coordinates execution across these components via an OODA reasoning loop (Observe, Orient, Decide, Act). The system operates as a continuous intake loop, collecting information about a young person’s mental health (including previous and current symptoms, psychosocial factors, case formulation, treatment history, and service engagement) until both a confidence threshold and an information completeness threshold are met based on established frameworks (e.g. Brain and Mind Centre’s Youth Model of Care). During each loop, the system then proceeds through the following stages. First, the Orient stage, integrating observational evidence-based knowledge against the Brain and Mind Centre’s multidimensional framework (suicidal thoughts and behaviours; social and occupational function; alcohol and substance use; physical health; illness type, stage, and trajectory54). Then, the Decide stage, where specific criteria need to be met to warrant exit from the ‘intake’ loop, or potential next questions are evaluated and selected should more information be required. Then, the Act stage, for action generation, ranging from a question to a completed intake assessment output (e.g. IAR-DST score), assumptions, missing pieces by domain and more. At each stage of the OODA reasoning cycle, a Stage Verifier produces a continuous confidence score. Where confidence is insufficient, the system re-runs the stage with targeted improvement guidance and supplementary knowledge retrieval before re-evaluating. In future, should this stage continue to fail, the system will escalate to a domain expert only if the subsequent passes also fail to meet the threshold. The Living Knowledge Bank was built through a structured ingestion pipeline in which the same reasoning architecture parses documents to extract semantic layers (foundational axioms, domain concepts, and typed relationships) populating a versioned knowledge graph rather than a flat document index. Prior to retrieval, the system generates and optimises a structured search query based on the clinical context assembled during Observe and partial Orient stages, and enriches retrieved evidence with cross-references to related guidelines and prior expert annotations before passing it to the reasoning layer. The clinical assessment criteria, completeness thresholds, and output schema are maintained in a separate configuration layer that authorised domain experts can update independently of model retraining, enabling clinical governance to remain under expert control across the fine-tuning and training cycles. Mia’s training follows two phases: in the pre-deployment phase, domain experts reviewed and corrected the system’s reasoning chains at each OODA stage, creating branches of training that were used for training the Stage Verifiers on those corrections, and in the future production phase, cases permanently failing a Stage Verifier will be escalated to experts to generate new training signals for the relevant Stage Verifier as a by-product of clinical governance. Domain experts can also initiate proactive updates (i.e. adding new guidelines, refining assessment criteria, or updating treatment reasoning) independently of any system failure, with changes propagating through the knowledge graph and configuration layer without requiring stage-specific model retraining. All outputs are advisory and explainable, i.e. for each output, the inputs considered, confidence criteria applied at each reasoning stage, knowledge bank version queried, and underpinning guideline citations are shown in line with a full chain-of-reasoning audit trail. In clinical evaluation, Mia achieved 94% triage accuracy versus an expert panel, a 24/25 expert recall rate, 100% red-herring rejection, and 0% hallucinations.

Study design

This research was approved by the University of Sydney Human Research Ethics Committee (Protocol no: 2021/680). This study was performed in line with the principles of the Declaration of Helsinki and according to the guidelines of the National Statement on Ethical Conduct in Human Research and the Australian Code for the Responsible Conduct of Research. All study participants provided informed consent.

This study engaged young people in online workshops to inform the development of Mia following a co-design methodology, centring on transforming it from a health professional tool into a solution for consumers. To develop generalisable findings relevant to genAI chatbots in youth mental health, we further expanded the focus of the workshops to investigate participants’ perceptions and recommendations about genAI chatbots in youth mental health generally, in addition to Mia specifically. Co-design is frequently employed to collect qualitative insights and stakeholder design considerations regarding the introduction of various digital health technologies in different sociotechnical contexts (e.g. health and wellbeing apps73,74).

Participants, recruitment and sampling

Participants were recruited via snowball and purposive sampling, as part of a series of research projects addressing youth mental health. Participants comprised young people aged 18-30. Existing research and community-based networks were engaged to support the distribution of recruitment materials among their networks. Additionally, individuals who participated in prior or parallel research projects conducted by the wider research team and gave their informed consent to be contacted directly for future research conducted by the research team were sent recruitment materials for this study. Individuals responded directly to the research team if they wished to take part, and were thereafter sent a participant information sheet and consent form via an online form (via Qualtrics) to record their informed consent.

Data collection

Data were collected online via Zoom and Qualtrics between March and August 2025 using workshops and optional follow-up surveys. Two types of workshops were held. First, a series of five conceptual development and design workshops, focusing on exploring young people’s perceptions and developing recommendations regarding genAI chatbots in youth mental health generally and Mia specifically. Activities included a live, interactive demonstration of Mia in which the research team operated the tool according to participant direction, and a semi-structured discussion. Second, a set of four user testing workshops that involved a period of free-form user testing in which participants operated Mia independently with a concurrent think-aloud component, and a semi-structured discussion that incorporated brainstorming and voting activities. Group discussion in both types of workshops was guided by the research team using different, evolving semi-structured discussion guides. The guides were organised by key topics relevant to the workshop aims, each accompanied by open-ended questions to stimulate discussion. These were adapted in situ to ensure a natural conversational flow while maintaining focus on the workshop aims. The guides were revised after each workshop to reflect emerging insights and explore gaps in subsequent discussions. Topics included in the conceptual development and design workshop guide included perceived usefulness, purpose, value, risk, safety concerns, safeguards, access preferences, and personalisation needs regarding Mia and other genAI chatbots in youth mental health, as well as existing mental health information needs and DMHT use. Topics in the user testing workshop guide were perceived usability, acceptability, inclusivity, engagement, functionality, system requirements, user experience, user-friendliness, personalisation factors, access preferences, missing and unnecessary features, likes and dislikes, and language considerations regarding Mia and other genAI chatbots.

The optional follow-up survey consisted of demographic questions using free-text and multi-select response formats (including an ‘other’ option with free-text input), capturing participants’ age, gender, identity groups, help-seeking history, lived experience, and current mental health treatment status.

All workshops were audio-video recorded and transcribed verbatim with the assistance of Otter.ai, an automated transcription platform. All data were anonymised prior to analysis. Data was collected by AP, HML, and FI.

Data analysis

Data were analysed following inductive reflexive thematic analysis to develop and report themes based on patterns of shared meaning in the workshop data, without a pre-determined set of themes75,76. Following reflexive thematic analysis, six iterative analytic phases were completed: data familiarisation, generating initial codes, generating candidate themes, reviewing candidate themes and establishing final themes, defining and naming final themes, and producing the report39.

To start, AP and HML independently reviewed one workshop transcript and generated and shared initial codes. Thereafter, AP coded the remaining data. At the midway point, HML independently coded a second workshop transcript to contribute new, evolving insights and enrich the ongoing data analysis. To generate, review, and finalise the themes, the wider research team was engaged in discussion. This added key insights from expertise in lived experience, psychiatry, general and clinical psychology, and human-computer interaction. NVivo 15 was utilised to organise and code the workshop data. Descriptive statistics were employed to develop a frequency analysis of the survey responses, providing contextual insight into the characteristics of workshop participants who elected to complete the survey.

Acknowledgements

This work was supported by a philanthropic funding donor affected by mental health who wishes to remain anonymous. IBH was supported by an NHMRC L3 Investigator Grant (GNT2016346). WC was supported by the Australian Government Research Training Program (RTP) Scholarship. FI was supported by an NHMRC EL1 Investigator Grant (GNT2018157).

Author contributions

A.P.: Conceptualisation, methodology, formal analysis, investigation, data curation, writing–original draft, writing–review and editing, visualisation. I.B.H.: Resources, writing–review and editing, supervision, project administration, funding acquisition. C.G.: Formal analysis, writing–review and editing. Z.H.: Writing–review & editing. W.C.: Writing–review & editing. E.E-A.: Software, writing–review & editing. J.R.: Software, writing–review and editing. E.M.S.: Writing–review & editing. F.I.: Conceptualisation, methodology, formal analysis, investigation, resources, writing–original draft, writing–review and editing, supervision, project administration. H.M.L.: Conceptualisation, methodology, formal analysis, investigation, resources, writing–original draft, writing–review & editing, supervision, project administration. All authors read and approved the final manuscript.

Data availability

Raw data are not available to the public due to data protection and privacy reasons. Anonymised metadata (e.g. code frequency) that support the findings are available from the corresponding author upon reasonable request.

Competing interests

I.B.H. is a Professor of Psychiatry and the Co-Director of Health and Policy, Brain and Mind Centre, University of Sydney. He has led major public health and health service development in Australia, particularly focusing on early intervention for young people with depression, suicidal thoughts and behaviours and complex mood disorders. He is active in the development through codesign, implementation and continuous evaluation of new health information and personal monitoring technologies to drive highly-personalised and measurement-based care. He holds a 3.2% equity share in Innowell Pty Ltd that is focused on digital transformation of mental health services. E.M.S. is a Principal Research Fellow at the Brain and Mind Centre, University of Sydney, a Consultant Psychiatrist and Adjunct Clinical Professor at the School of Medicine, University of Notre Dame. She previously served as the Discipline Leader for Adult Mental Health at Notre Dame until January 2025. In addition, she is a member of Medibank’s Medical and Mental Health Reference Groups. E.M.S. has also delivered educational seminars on the clinical management of depressive disorders, receiving honoraria from pharmaceutical companies including Servier, Janssen, and Eli Lilly. Moreover, she has contributed to a national advisory board for Pfizer’s antidepressant Pristiq and served as the National Coordinator for an antidepressant trial sponsored by Servier. A.P., C.G., Z.H., W.C., E.E-A., J.R., E.M.S., F.I., and H.M.L. have nothing to disclose.

Footnotes

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

These authors jointly supervised this work: Frank Iorfino, Haley M LaMonica.

References

  • 1.McGorry, P. D. The reality of mental health care for young people, and the urgent need for solutions. Med. J. Aust.216, 78–79 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.McGorry, P. D. et al. The Lancet Psychiatry Commission on youth mental health. Lancet Psychiatry11, 731–774 (2024). [DOI] [PubMed] [Google Scholar]
  • 3.Weinrabe, A., Tran, J. & Hickie, I. B. Risk tolerance in youth with emerging mood disorders. Humanit. Soc. Sci. Commun.10, 882 (2023). [Google Scholar]
  • 4.World Health Organization. Mental Health of Adolescents, https://www.who.int/news-room/fact-sheets/detail/adolescent-mental-health (2025).
  • 5.McGorry, P. D. et al. Designing and scaling up integrated youth mental health care. World Psychiatry21, 61–76 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Hung, Y.-A. et al. Child and adolescent mental health burden, policies, and services in high-income countries and areas in the Western Pacific Region. Lancet Reg. Health West. Pac.62, 101678 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.McGorry, P. D. & Hickie, I. B. Clinical Staging in Psychiatry: Making Diagnosis Work for Research and Treatment (Cambridge University Press, 2019).
  • 8.Creswell, C. et al. Digitally augmented, parent-led CBT versus treatment as usual for child anxiety problems in child mental health services in England and Northern Ireland: a pragmatic, non-inferiority, clinical effectiveness and cost-effectiveness randomised controlled trial. Lancet Psychiatry11, 193–209 (2024). [DOI] [PubMed] [Google Scholar]
  • 9.Bond, R. R. et al. Digital transformation of mental health services. npj Ment. Health Res.2, 13 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Carpenter-Song, E. et al. Individualized intervention to support mental health recovery through implementation of digital tools into clinical care: feasibility study. Community Ment. Health J.58, 99–110 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Bassi, E. M. et al. Perceptions of mental health providers of the barriers and facilitators of using and engaging youth in digital mental-health-enabled measurement based care. Digit. Health10, 10.1177/20552076241253093 (2024). [DOI] [PMC free article] [PubMed]
  • 12.Pathak, H., Goyal, M., Das, P. & Santhanam, S. Impact of conversational and generative AI systems on libraries: a use case large language model (LLM). Sci. Technol. Libr.43, 1531–1539 (2023). [Google Scholar]
  • 13.Kanjee, Z., Crowe, B. & Rodman, A. Accuracy of a generative artificial intelligence model in a complex diagnostic challenge. JAMA330, 78 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Blease, C. & Torous, J. ChatGPT and mental healthcare: balancing benefits with risks of harms. BMJ Ment. Health26, e300884 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Elyoseph, Z. & Levkovich, I. Comparing the perspectives of generative AI, mental health experts, and the general public on schizophrenia recovery: case vignette study. JMIR Ment. Health11, e53043 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Li, H., Zhang, R., Lee, Y.-C., Kraut, R. E. & Mohr, D. C. Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being. npj Digit. Med.6, 236 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Kleinau, E. et al. Effectiveness of a chatbot in improving the mental wellbeing of health workers in Malawi during the COVID-19 pandemic: a randomized, controlled trial. PloS ONE19, e0303370 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Torous, J. & Topol, E. J. Assessing generative artificial intelligence for mental health. Lancet406, 683 (2025). [DOI] [PubMed] [Google Scholar]
  • 19.Vaidyam, A. N., Wisniewski, H., Halamka, J. D., Kashavan, M. S. & Torous, J. B. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can. J. Psychiatry64, 456–464 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Gómez, C., Yin, J., Huang, C.-M. & Unberath, M. How large language model-powered conversational agents influence decision making in domestic medical triage contexts. SSRN 10.2139/ssrn.4797707 (2024). [DOI]
  • 21.Sezgin, E. & McKay, I. Behavioral health and generative AI: a perspective on future of therapies and patient care. npj Ment. Health Res.3, 25 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Banerjee, S., Dunn, P., Conard, S. & Ali, A. Mental health applications of generative AI and large language modeling in the United States. Int. J. Environ. Res. Public Health21, 910 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Kvedaravičienė, G., Poulsen, A. & Žaliauskaitė, M. The impact of artificial intelligence on healthcare. In Future Law, Ethics, and Smart Technologies, John-Stewart Gordon. 10.1163/9789004682900_008 (2023). [DOI]
  • 24.Sobowale, K., Humphrey, D. K. & Zhao, S. Y. Evaluating generative AI psychotherapy chatbots used by youth: cross-sectional study. JMIR Ment. Health12, e79838 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Rajashekar, N. C. et al. in Proc.CHI Conference on Human Factors in Computing Systems (eds Mueller, F. F. et al.) 1–20 (ACM, 2024).
  • 26.Sun, X., Liu, Y., Wit, J. D., Bosch, J. A. & Li, Z. in Proc.Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (eds Mueller, F. F., Kyburz, P., Williamson, J. R., & Sas, C) 1–7 (ACM, 2024).
  • 27.Young, J. et al. in Proc. CHI Conference on Human Factors in Computing Systems (eds Mueller, F. F. et al.) 1–18 (ACM, 2024).
  • 28.De Freitas, J. & Cohen, I. G. The health risks of generative AI-based wellness apps. Nat. Med.30, 1269–1275 (2024). [DOI] [PubMed] [Google Scholar]
  • 29.Bragazzi, N. L., Crapanzano, A., Converti, M., Zerbetto, R. & Khamisy-Farah, R. The impact of generative conversational artificial intelligence on the lesbian, gay, bisexual, transgender, and queer community: scoping review. JMIR25, e52091 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Blanchard, M. User experience research in the development of digital health products: research letter. Health Policy Technol.12, 100753 (2023). [Google Scholar]
  • 31.Gilbert, R. M. Reimagining digital healthcare with a patient-centric approach: the role of user experience (UX) research. Front. Digit. Health4, 10.3389/fdgth.2022.899976 (2022). [DOI] [PMC free article] [PubMed]
  • 32.Balan, R., Dobrean, A. & Poetar, C. R. Use of automated conversational agents in improving young population mental health: a scoping review. npj Digit. Med.7, 75 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Horstkötter, D. et al. Decision-making on an AI-supported youth mental health app: a multilogue among ethicists, social scientists, AI-researchers, biomedical engineers, young experiential experts, and psychiatrists. J. Responsible Technol.22, 100119 (2025). [Google Scholar]
  • 34.Ding, X. & Barbic, S. Perception of AI use in youth mental health services: qualitative study. J. Particip. Med.17, e69449 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Siddals, S., Torous, J. & Coxon, A. It happened to be the perfect thing”: experiences of generative AI chatbots for mental health. npj Ment. Health Res.3, 48 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Park, J. K., Singh, V. K. & Wisniewski, P. Current landscape and future directions for mental health conversational agents for youth: scoping review. JMIR Med. Inf.13, e62758 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Poulsen, A. et al. in CHI EA ‘25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (eds Yamashita, N., Evers, V., Yatani, K., & Ding, X) 1–7 (ACM, 2025).
  • 38.Australian Institute of Health and Welfare. Culturally and linguistically diverse Australians, https://www.aihw.gov.au/reports-data/population-groups/cald-australians/glossary (2026).
  • 39.Braun, V. & Clarke, V. in APA Handbook of Research Methods in Psychology: Research Designs: Quantitative, Qualitative, Neuropsychological, and Biological (eds Cooper, H. et al.) 65–81 (American Psychological Association, 2023).
  • 40.Coeckelbergh, M. Health care, capabilities, and AI assistive technologies. Ethical Theory Moral Pract.13, 181–190 (2010). [Google Scholar]
  • 41.Vallor, S. Technology and the Virtues: A Philosophical Guide to a Future Worth Wanting (Oxford University Press, 2016).
  • 42.Chin, H., Baek, G., Cha, C. & Cha, M. Chatbots’ empathetic conversations and responses: a qualitative study of help‑seeking queries on depressive moods across 8 commercial conversational agents. JMIR Form. Res.9, e71538 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Ratcliffe, M. Real Hallucinations: Psychiatric Illness, Intentionality, and the Interpersonal World (The MIT Press, 2017).
  • 44.Sundar, S. S. Rise of machine agency: a framework for studying the psychology of human–AI interaction (HAII). J. Comput. Mediat. Commun.25, 74–88 (2020). [Google Scholar]
  • 45.Byrne, M. K. & Deane, F. P. Enhancing patient adherence: outcomes of medication alliance training on therapeutic alliance, insight, adherence, and psychopathology with mental health patients. Int. J. Ment. Health Nurs.20, 284–295 (2011). [DOI] [PubMed] [Google Scholar]
  • 46.Norcross, J. C. & Lambert, M. J. Psychotherapy relationships that work III. Psychotherapy55, 303–315 (2018). [DOI] [PubMed] [Google Scholar]
  • 47.Jordan, J. V. Relational–cultural therapy (American Psychological Association, 2018).
  • 48.Barredo Arrieta, A. et al. Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion58, 82–115 (2020). [Google Scholar]
  • 49.Wittevrongel, E. et al. A user perspective on youth mental health services: increasing help-seeking behaviour requires addressing service preferences and attitudinal barriers. Early Interv. Psychiatry. 19, 10.1111/eip.13584 (2025). [DOI] [PubMed]
  • 50.Hibbard, J. H. & Peters, E. Supporting informed consumer health care decisions: data presentation approaches that facilitate the use of information in choice. Annu. Rev. Public Health24, 413–433(2003). [DOI] [PubMed] [Google Scholar]
  • 51.Morley, J. et al. The ethics of AI in health care: a mapping review. Soc. Sci. Med.260, 113172 (2020). [DOI] [PubMed] [Google Scholar]
  • 52.Iorfino, F. et al. Using digital technologies to facilitate care coordination between youth mental health services: a guide for implementation. Front. Health Serv. 1, 10.3389/frhs.2021.745456 (2021). [DOI] [PMC free article] [PubMed]
  • 53.Orlowski, S. et al. People, processes, and systems: an observational study of the role of technology in rural youth mental health services. Int. J. Ment. Health Nurs.26, 259–272 (2017). [DOI] [PubMed] [Google Scholar]
  • 54.Hickie, I. B. et al. Right care, first time: a highly personalised and measurement-based care model to manage youth mental health. Med. J. Aust. 211, 10.5694/mja2.50383 (2019). [DOI] [PubMed]
  • 55.Naert, J., Roose, R., Rapp, R. C. & Vanderplasschen, W. Continuity of care in youth services: a systematic review. Child. Youth Serv. Rev.75, 116–126 (2017). [Google Scholar]
  • 56.Clough, B. A. & Casey, L. M. Technological adjuncts to increase adherence to therapy: a review. Clin. Psychol. Rev.31, 697–710 (2011). [DOI] [PubMed] [Google Scholar]
  • 57.Stanley, B. & Brown, G. K. Safety planning intervention: a brief intervention to mitigate suicide risk. Cogn. Behav. Pract.19, 256–264 (2012). [Google Scholar]
  • 58.Syan, A. et al. The Wellness Quest: A health literacy and self-advocacy tool developed by youth for youth mental health. Health Expect.24, 659–669 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Chong, M. K. et al. Personalized and collaborative use of digital measurement-based care tools enhances engagement among young adults: a mixed-methods study. BMC Health Serv. Res.25, 752 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Figueroa, C. A. et al. Advancing youth co-design of ethical guidelines for AI-powered digital mental health tools. Nat. Ment. Health3, 870–878 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Molyneaux, R., Mirembe, E., Leicester, S., Schley, C. & Alisic, E. Digital disclosures: exploring the role of online mental health services in supporting young people who disclose violence or maltreatment. Child Prot. Pract.1, 100016 (2024). [Google Scholar]
  • 62.Stilgoe, J., Owen, R. & Macnaghten, P. Developing a framework for responsible innovation. Res. Policy42, 1568–1580 (2013). [Google Scholar]
  • 63.Hoopes, A. J. et al. What do adolescents and their parents need from mental health integration in primary care? A qualitative exploration of design insights. J. Pediatr. Health Care36, 570–581(2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Farthing, R. et al. Online privacy, young people, and datafication: different perceptions about online privacy across Antigua & Barbuda, Australia, Ghana, and Slovenia. Soc. Media Soc. 10, 10.1177/20563051241298042 (2024). [DOI]
  • 65.Greenhalgh, T. et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J. Med Internet Res.19, e367 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Ross, J., Stevenson, F., Lau, R. & Murray, E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement. Sci.11, 146 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Fiske, A., Henningsen, P. & Buyx, A. Your robot therapist will see you now: ethical implications of embodied artificial intelligence in psychiatry, psychology, and psychotherapy. J. Med. Internet Res.21, e13216, 10.2196/13216 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Skinner, A. et al. Reshaping youth mental health care for optimal system-level outcomes: a dynamic modelling analysis. PLOS Ment. Health2, e0000232 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Latulippe, K., Hamel, C. & Giroux, D. Social Health Inequalities and eHealth: a literature review with qualitative synthesis of theoretical and empirical studies. J. Med Internet Res.19, e136 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.van Kessel, R., Wong, B. L. H., Clemens, T. & Brand, H. Digital health literacy as a super determinant of health: more than simply the sum of its parts. Internet Inter.27, 100500 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Beauchamp, T. L. & Childress, J. F. Principles of Biomedical Ethics 7th edn (Oxford University Press, 2013).
  • 72.McKenna, S. et al. Youth perspectives on ‘highly personalised and measurement-based care’: Qualitative co-design of education materials. Health Expect. 27, 10.1111/hex.14137 (2024). [DOI] [PMC free article] [PubMed]
  • 73.Zhang, Z. et al. in Proceedings of the 2025 ACM Designing Interactive Systems Conference (eds Nunes, N. J. et al.) 3132–3152 (ACM, 2025).
  • 74.Ortega, A. et al. Co-designing prediction data visualizations for a digital binge eating intervention. Transl. Behav. Med. 15, 10.1093/tbm/ibaf009 (2025). [DOI] [PMC free article] [PubMed]
  • 75.Braun, V. & Clarke, V. Reflecting on reflexive thematic analysis. Qual. Res. Sport Exerc. Health11, 589–597 (2019). [Google Scholar]
  • 76.Braun, V. & Clarke, V. One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qual. Res. Psychol.18, 328–352 (2021). [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Raw data are not available to the public due to data protection and privacy reasons. Anonymised metadata (e.g. code frequency) that support the findings are available from the corresponding author upon reasonable request.


Articles from NPJ Digital Medicine are provided here courtesy of Nature Publishing Group

RESOURCES