Developing standards for the implementation of stepped care in child and youth mental health service settings: protocol for a multi-method, delphi-based study

Abstract

Introduction

Canadian youth mental health (YMH) systems have the potential to urgently tackle the mental health treatment gap currently impacting young people, and stepped care (SC) is one model that can address this need. The adoption of SC models can guide the development of better-connected YMH systems by simplifying transitions and care pathways. To do so requires robust standards that are co-created across stakeholder groups, including with lived experience experts, to ensure the effective implementation of SC models.

Methods and analysis

This study aims to establish standards for implementing SC in Canadian child and YMH service settings by convening and developing a learning alliance (LA) of 65 individuals, translating guiding principles to standards via consensus methods (Delphi study), and operationalising and applying draft standards to three test ecosystems. Members of the LA will be recruited via snowball and purposive recruitment techniques to complete an e-Delphi study over three to four rounds until consensus is achieved. Participants will rank their agreement with including specific clause items in the final standard, and will be given opportunities to provide feedback and suggest revisions during each round. Comments will be analysed, scored and coded accordingly. Once consensus has been achieved, members of the LA will consider the application of these implementation standards in three settings that could benefit from SC. The protocol for this study was registered at Open Science Framework (https://doi.org/10.17605/OSF.IO/J5UNW).

Ethics and dissemination

The protocol has been approved by the Centre intégré universitaire de santé et de services sociaux (CIUSSS) de l'Ouest-de-l'Île-de-Montréal—Mental Health and Neuroscience subcommittee. As part of the ethics approval, informed consent forms for all Delphi participants were created and distributed to participants ahead of the Delphi. This includes parental consent forms for all LA members participating in the study who are under the age of 18. On completion, the project will ultimately support the implementation of SC in diverse service systems and guide the development of a robust and connected mental health delivery system in Canada. The final standard will be shared with relevant government bodies and health planners and disseminated via academic and other platforms.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The active engagement of lived experience experts (youth and family/carers) will ensure that this standard is co-created with those who have experienced the Canadian youth mental health (YMH) system.

• The final standard list will be considered in three distinct ecosystems that increasingly use system models such as stepped care.

• A number of additional service settings that could benefit from stepped care are outside the scope of this study; as such, we will not be able to state that our created standards are necessarily applicable across any YMH setting.

• The Delphi study will be limited to those who reside or work in Canada; this may restrict the applicability of our findings to mental health service systems outside of Canada.

Introduction

Mental illnesses tend to arise during adolescence, before or around the age of 25.¹ They affect, on average, one in five Canadian youth each year.² Tragically, only 20% of this group receive the support they need.³ When support is accessed, youth are often met with additional challenges within mental health delivery systems, such as complex and confusing pathways to care, unacceptably long waitlists and poor quality or inappropriate services.⁴ Further difficulties arise for those ‘ageing’ out of the youth mental health (YMH) system, which requires transitions between adolescent and adult services.⁵

As a response to these obstacles, many system planners and mental health experts have pointed to a need for delivery models that can coordinate and even reverse this fragmentation.^{4 6} Stepped care (SC) has recently gained traction as one such model. A defining feature of SC is the purposeful arrangement of interventions—including low-intensity, low-barrier ones alongside more intensive treatments—that can be ‘stepped’ up or down as needed along a continuum of care, such as guided self-help supports all the way to acute or inpatient services.⁷ As such, SC models hold the potential to (1) help match service users to evidence-informed interventions that are likely to meet their needs and preferences,^{8 9} (2) streamline initial access to community-based services and (3) ensure smooth transitions across a continuum of interventions.⁴

A recent examination of SC models identified that there are no consistently agreed-upon definitions of SC in practice.⁴ Models vary widely across various implementation settings; frequent differences include the number of steps provided along the continuum, available entry point(s) and selection of appropriate intervention(s) and provider types.^{8 10} Additionally, there is no explicit agreement regarding the degree to which community-based supports and more formalised healthcare providers (ie, hospitals, in-patient care and/or specialist services) are to be integrated with these models. Overall, these gaps demonstrate a clear need for agreement on what defines SC, not just in concept but also in practice.⁴

Five guiding principles theorised to underlie the implementation of SC models have been articulated to strengthen conceptual clarity⁷:

1. Providing a breadth of evidence-informed mental health supports and services along a continuum of care, including a range of different intensities.

2. Utilising diverse methods to ensure that clients can make informed decisions about their care, based on their readiness, goals and priorities.

3. Integrating interventions and services with each other to inform a cohesive system with functional connections for continuity and referral.

4. Using validated tools to assess the benefits of care provided, such that both service users and service providers can track outcomes.

5. Ensuring community-responsive definitions of services, providers and access points tailored to local needs and expertise.

These principles serve as a starting point for system integration and transformation; however, they also lack the explicit detail required by system planners to fully implement SC models in their communities. In response, standards would enhance consistency in how decision-makers interpret and apply SC, ultimately improving the utility and effective implementation of these models.

As part of a pan-Canadian effort to derive standards for child and YMH service domains funded by the Canadian Institutes of Health Research in partnership with Health Canada,¹¹ we designed a project to develop SC guiding principles into minimum standards and then apply these standards to community-based ecosystems where SC is currently being explored and employed. Importantly, this collaborative protocol requires bringing together individuals with expertise from a range of perspectives relevant to mental health: clinicians, managers, service planners, researchers, decision-makers and young people and their families.

Methods and analysis

Team and project overview

Our research team consists of experts in both the Canadian YMH system and patient-oriented knowledge creation, mobilisation and implementation. Critically, and in alignment with our commitment to patient-oriented research, the learning alliance (LA) includes participants who are recent and/or longstanding users of YMH services as well as carers and family members who have supported young people as they attempt to navigate services. These individuals will participate in all aspects of the project, articulated as the following three aims (each expanded on below):

1. Convening and developing an LA.

2. Translating guiding principles to standards via consensus methods (e-Delphi study).

3. Operationalising and applying standards to three exemplar ecosystems.

Youth engagement

In order to ensure the results of this study are relevant to and able to capture the expertise of those with lived experience, engagement activities have been woven throughout the duration of this project. All youth, family and community engagement activities, including knowledge-sharing sessions, will be hosted by two members of the research team with a combined 15 years of experience working in youth, family and community engagement in the mental health sector.

Aim 1: convening and developing an LA

This study will use LA methodology by convening a diverse network of community-based experts committed to improving knowledge on SC models and collaboratively furthering systems change.¹² Notably, LAs differ from traditional communities of practice in that, unlike the latter, they actively support and contribute capacity-building, documenting and disseminating innovative practices and fostering contextualised understandings of and responses to crises that a single stakeholder group or perspective cannot solve.^{13 14} As such, the LA inherently responds to the needs, values and objectives of its members and will actively ensure the development and ultimate application of the SC standard. This is especially important given the LA’s role in the Delphi study (aim 2) and in considering the application and implementation of the Delphi results (aim 3).

LA participants

To ensure varied representation in the LA, the research team identified a team of 65 Canadian experts in YMH care via purposive sampling and snowball recruitment. Individuals invited to join this network were asked to self-identify which stakeholder groups they represent (listed below) and denote the primary stakeholder group that they are representing within the Delphi process. This information allows us to ensure sufficient representation across stakeholder groups:

1. Clinicians

2. Researchers

3. Decision-makers

4. Service system experts

5. Lived experience experts

Exclusion criteria included non-residents of Canada or those who did not hold a valid study permit for Canada, individuals unable to communicate in English or French and youth lived experience experts under 15 or over 25 years of age as of the start of the study.

Online webinar series

The research team hosted a series of four online webinars after the LA membership was finalised. Webinars were delivered from June to August 2023 and were intended to introduce the core concepts of SC models and the need for standards. This was critical to building foundational knowledge of SC models among members of the LA, motivating participation in our Delphi study and ensuring that LA members could contribute actively to the project. Webinars were organised as follows: (1) introduction to stepped care and the need for standards; (2) creating standards and consensus methodology and (3) group discussion(s). Group discussions were intended to provide participants with an opportunity to contribute to, query, and synthesise the information presented. Webinars were recorded and made available to all LA members for future reference and consultation.

Aim 2: translating guiding principles to standards utilising consensus methods (Delphi study)

A multi-round electronic Delphi survey will be designed to elicit opinions across the aforementioned five stakeholder groups¹⁵; survey design will begin with a draft list of clauses based on the original guiding principles (expanded below).⁷ Delphi methodology was explicitly chosen as a consensus derivation technique to ensure that our draft items can be continually iterated across rounds, evolving and developing over time based on participant feedback without concern of judgement or unintentional bias.^{16 17} Given known power imbalances, our inclusion of lived experience experts (both youth and families/carers) also underscores the need for a multistakeholder method that allows for input from each participant group without influence from others.¹⁸

Draft standard

In preparation for the Delphi study, the research team will begin with an initial draft list of individual clause items believed to operationalise the five guiding principles referenced above. These clause items will form the basis for the first round of the Delphi. LA members will be asked to rate their agreement with each clause item on a 6-point Likert scale with anchors at 1 (‘disagree, strong objections’) and 6 (‘agree, as written’). An open text box will also be provided to capture suggested edits or questions. As part of this process, 29 clauses were drafted to be tested in round 1 of the Delphi study. These clauses will be uploaded to the online e-Delphi platform Calibrum.

Delphi participants

Delphi participants will comprise a subset of LA members who have agreed to participate in all rounds of the Delphi study. The research team will monitor key demographics, such as representation from our five key stakeholder categories, gender, age and province/territory of residence/work, to ensure sufficient representation from the LA in the Delphi study. Purposive sampling will be used to fill any gaps identified by the research team.

In addition to the Delphi, community outreach and engagement with youth and young people with lived experience expertise will be conducted. These targeted engagement sessions will include focus groups designed to solicit feedback on the draft list of standards (see the section Draft standard) prior to launch and may result in revisions or additions to the draft standard list. These same participants will be contacted at the conclusion of the Delphi study to provide feedback on the final standard list. This session will occur prior to the launch of Aim 3.

Delphi analysis plan

Consensus measures have been chosen as the primary indicator to be tracked during the duration of the study. Consensus measures determine the percentage required for an item to ‘pass’ and be considered agreed to.^{15 17} In keeping with prior Delphi studies, the research team determined that a 70% agreement threshold would be required across stakeholder groups as well as for each stakeholder subgroup.^{16 17} Items that do not meet this threshold will be retained for a subsequent round but revised by the research team based on participant feedback.

All votes, responses and comments received during the duration of the Delphi study will be anonymous. Comments responding to the Delphi clauses will be thematically analysed, coded and summarised by members of the research team. Results and comments will be examined in detail following each round, and high-level overviews will be shared with participants prior to the next round to ensure that a breadth of feedback is identified before revising a clause requiring attention.

It is anticipated that the Delphi will require three to four rounds to achieve consensus across clauses that operationalise each of the five principles. Data analysis will begin in December 2024 at the conclusion of the Delphi study; as part of this analysis, and in line with principles of patient-oriented research, particular attention will be paid to discrepancies between the opinions of professional (service provider/decision-maker) groups and those of service users (youth and caregivers), with the aim of ensuring that the latter group’s perspective is privileged.

Aim 3: operationalising and applying standards to three exemplar ecosystems

Simply reporting the standards will be insufficient to support the effective translation of our Delphi study into practice. Instead, it will be essential to test the utility of the standards by considering their application to settings in which SC is currently being or may be implemented. We will therefore consider how the final standard could be applied in three exemplar ecosystems representing key arenas where SC is strengthening or has the potential to strengthen YMH service systems in Canada: (1) postsecondary institutions, (2) e-mental health platforms and (3) integrated youth services. This aim will also highlight potential consistencies and divergence in the implementation of SC models and ultimately contribute to their evidence base.

We will therefore bring together members of our LA and other experts in Canadian YMH systems for a 2-day in-person meeting to review the applicability and feasibility of the newly developed standard (Aim 2) for implementation in these ecosystems. LA members will be provided with summaries of the Delphi results (i.e., the full set of approved SC implementation standards), followed by a detailed day-long consideration of how these standards apply in their ecosystems of practice. Specifically, participants will be asked to reflect on the following: (1) how will these standards strengthen the implementation of SC in your ecosystem?, (2) are any questions regarding SC implementation left unanswered via these standards? Following these discussions, the entire group will consider what resources could support dissemination and any additional next steps needed to support the implementation and uptake of these standards. Suggested revisions resulting from Aim 3 will be included in implementation guidance, which the research team aims to submit for publication by June 2025.

Ethics and dissemination

Ethics

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2013. All procedures involving human subjects/patients were approved by the Centre intégré universitaire de santé et de services sociaux (CIUSSS) de l'Ouest-de-l'Île-de-Montréal—Mental Health and Neuroscience subcommittee in March 2024 (REB #2024–958). Informed consent forms, including parental consent forms for participants under the age of 18, were created with approval from the research and ethics board and were distributed electronically via Calibrum to all participants ahead of their participation in the Delphi study.

Dissemination

Upon conclusion of the final phase (Aim 3) of the project, the information gained from discussions (including settings where the standards were more or less applicable) will generate implementation guidance that will be translated into a range of fora. First, the Aim 2 Delphi survey results and Aim 3 practice implications exercise with LA participants will together produce a final standard for implementing SC models for child and YMH service settings across Canada. The final standard will be shared with relevant bodies such as the Mental Health and Substance Use Standards Collaborative for public review and comment as well as the Canadian Institutes of Health Research, Health Canada and the Standards Council of Canada. Second, the results will be disseminated via academic platforms, including open-access peer-reviewed journals as well as conference presentations.

Third, the final standards will be compiled into an implementation support document for service designers and implementers to use as they work to streamline their services based on the created standard; this will include ecosystem-specific insights or considerations and comments generated during our Aim 3 in-person meeting. It will be accompanied by a plain-language implementation guide and resource to assist system planners in the application of SC models. Finally, we will generate a lay version of this implementation guidance for young people and family members to understand how their contributions to the study have helped to shape the evidence base and guidance on SC implementation. We aim to submit these guidelines for publication by June 2025.

To date, there is little agreement on what unites SC models nor guidance on how one should begin to build an SC system. This greatly restricts the ability of health systems planners, clinicians, policymakers and researchers to leverage the concept of SC to create more robust and streamlined mental health systems. We do not intend to create a rigid definition of SC models with a set number of steps or a list of mandatory interventions. Rather, we aim to create a minimum core set of implementation standards that will improve consistency across the Canadian YMH field while still allowing for flexibility depending on diverse needs and local contexts.