Evaluating Strategies to Promote Effective, Multidisciplinary Team Collaboration in School Mental Health

Abstract

Multidisciplinary teams of school- and community-employed mental health, health, and educational staff work together in schools to offer a full continuum of mental health promotion, prevention, early intervention, and treatment services and supports. Intentional teaming structures and practices are essential to ensure teams deliver effective, coordinated services and supports. The current study investigated the extent to which continuous quality improvement strategies improved school mental health team performance during a 15-month national learning collaborative for 24 school district teams. All teams significantly improved their average teaming performance from baseline to the end of the collaborative (t(20) = −5.20, p < .001). Plan-Do-Study-Act (PDSA) cycles allowed teams to rapidly evaluate specific quality improvement changes to improve their performance. Teams with the most improvement focused on increasing multidisciplinary team membership, avoiding duplication and promoting efficiency, and connecting to community mental health providers/resources.

Introduction

Schools continue to provide more mental health services to children, adolescents, and their families (e.g. 22.1%) than any other child-serving sector, including specialty outpatient mental health (Duong et al., 2020). Multidisciplinary teams of school-employed and community-employed mental health, health, and educational staff work together in schools to offer a full continuum of mental health promotion, prevention, early intervention, and treatment services and supports. The COVID-19 pandemic and the resulting impact of social isolation, economic hardship, grief and trauma is related to current and anticipated increases in mental health concerns for children and adolescents (Chegg et al., 2020; Loades et al., 2020). Thus, school mental health teams are facing unprecedented workflow demands to deliver efficient, effective mental health services and supports for a high volume of student mental health concerns. Intentional, coordinated teaming structures and practices are essential to ensure effective collaboration among multidisciplinary team members in schools (Mellin et al., 2011). However, wide variations in the use of teaming best practices present barriers to students accessing needed mental health care at school (Iachini et al., 2013). The current study investigated the extent to which school mental health teaming performance improved, during a 15-month national learning collaborative of 24 school district teams, when teams had access to a quality improvement structure, methods and resources to implement changes to their teaming practices. We also examined the types of changes used by the most successful teams across five different performance indicators of school mental health teaming quality.

Comprehensive school mental health systems (CSMHS)

The quality and comprehensiveness of school mental health systems vary greatly across the country based on numerous school and district characteristics (e.g. size, urbanicity, local staffing, nonfinancial and financial resources, and student needs and strengths). Comprehensive School Mental Health Systems (CSMHS) promote positive school climate, social and emotional learning, and mental health and well-being, while reducing the prevalence and severity of mental illness (Hoover et al., 2019). This is achieved via a full continuum of multi-tiered supports in which mental health promotion is provided to all students, with additional early intervention and treatment supports available for students as needed. Funding and sustainability of CSMHS require optimization of financial and non-financial resources, often by blending and braiding a variety of different sources to support various aspects of the system (e.g. third-party billing, district or state budget allocated funds, federal grants and/or block funding) (Hoover et al., 2019). CSMHS are built on a strong foundation of district and school professionals, in strategic collaboration with students, families, and community health and mental health partners. Therefore, multidisciplinary, effective collaborative teaming processes are a core feature of CSMHS. In reality, achieving optimal performance in all aspects of a CSMHS is aspirational; quality improvement processes are inherent in CSMHS development and maintenance as school and district teams have different strengths and areas of growth within their system.

Comprehensive school mental health systems team composition

CSMHS are made possible by multidisciplinary teams of school-based professionals, including school and community-employed personnel, who work together to support students and the school community. The configuration of CSMHS teams, including which providers or professionals are a part of the team and their roles, varies across the country, between states and within states and districts (Mellin et al., 2011). The variation of CSMHS team composition adds to the complexity of using teaming best practices and ensuring high quality teaming performance. Despite variation across the country, there are some general types of team members and roles that pertain to CSMHS widely. For instance, mental health promotion via activities such as social emotional learning and mental health literacy are often supported by school administrators and provided by educators. Most schools have some school-employed staff specifically dedicated to supporting the social, emotional, and behavioral success of their students. These school-employed professionals, also known as student instructional support professionals, include school psychologists, school social workers, school counselors, school nurses and other school health providers. These team members each have unique roles and responsibilities but together support the entire multi-tiered continuum of mental health supports for students. While school-employed professional staffing varies considerably, there is general consensus that student instructional support professionals-to-student ratios are not adequate in many of our nation’s schools (American School Counselor Association, 2021; Castillo et al., 2016; Gagnon & Mattingly, 2016; The National Association of Social Workers, 2012; Walcott et al., 2017). For instance, no states currently meet the recommended school social worker to student ratio of 1:250, only two states meet the recommended 1:500 school psychologist to student ratio and only three states meet the recommended 1:250 recommended school counselor to student ratio. Many schools also have community-employed mental health professionals who provide services and supports directly on school campus to augment existing school supports, particularly to deliver indicated mental health interventions to students in general education.

Comprehensive school mental health system team collaboration

CSMHS teams face numerous challenges to establishing effective collaboration (Splett et al., 2017; Weist et al., 2012). First, school-based professionals have competing priorities for their time and resources, including academic progress, supporting benchmark and standardized testing requirements and additional, non-mental health related school responsibilities such as bus or lunch duty (Splett et al., 2017; Weist et al., 2012). Second, while many school-employed professionals recognize the added value of partnering with community-employed mental health providers to deliver services on school grounds, role confusion and turf issues can arise with this model. Specifically, bringing community-employed providers in to support student mental health and wellbeing may imply that current school-employed staffing ratios are sufficient, or that the school professionals have been unable to meet student needs (Mellin & Weist, 2011; Weist et al., 2012). CSMHS team role clarity and definitions are particularly important challenges and targets for quality improvement. In a qualitative analysis assessing collaboration in an urban community, school mental health collaborators reported concerns about preserving their roles within a school and identifying enough difference in roles to justify each position (Mellin & Weist, 2011). Further, the different training and philosophical orientations, as well as familiarity with school language and culture, within and among health and mental health providers may pose challenges to effective collaboration (Mellin et al., 2011, Waxman et al., 1999). Thus, multidisciplinary school teams face numerous challenges such as varying goals and responsibilities among disciplines, differences in training and education, possible turf issues and limited clarity on differing and similar roles, time and logistical barriers, and differing priorities and agendas between school and community providers (Flaherty et al., 1998; Mellin & Weist, 2011; Splett et al., 2017; Weist et al., 2012).

Comprehensive school mental health systems team components and benefits

To assess and improve CSMHS performance, the components and best practices of a high functioning CSMHS team must be specified. One major focus of quality improvement for CSMHS teams is to optimize effective multidisciplinary collaboration strategies and teaming structures. Multidisciplinary collaboration has been characterized by five core components: (1) interdependence; (2) newly-created professional activities; (3) flexibility; (4) collective ownership of goals; and (5) reflection on process (Bronstein, 2003). Effective implementation of these five components has been associated with improved service coordination and reduction of competing demands for resources (Mellin et al., 2011). Multidisciplinary collaboration in school mental health can provide schools with additional resources and supports; provide services to a larger proportion of students when funding is limited; decrease service fragmentation; increase access to services and consistency of services for youth and families; and provide all tiers of mental health supports (Mellin & Weist, 2011).

However, current conceptualizations of CSMHS team performance are not limited to multidisciplinary collaboration, as other elements of high quality CSMHS teaming structures and practices have been identified. Specifically, the School Mental Health Quality Assessment (SMH-QAVersion 1; Hoover et al., 2015) includes Teaming as a foundational domain of comprehensive school mental health. The SMH-QA-Version 1 includes the 5 performance indicators of high quality school mental health teaming practices: (1) having a multidisciplinary team; (2) promoting efficiency; (3) using meeting best practices; (4) promoting data sharing; and (5) connecting to community resources. These indicators, and the best practices for each (see Figure 1) were developed using an iterative, four-stage, multi-method process involving broad stakeholder feedback (Connors et al., 2016). Many prior measures of school mental health quality also include items related to teaming, such as multidisciplinary collaboration and communication efforts within a school mental health team to coordinate referral processes, ensuring students receive needed services, and ensuring services are not duplicated (The National Assembly on School Based Health Care, 2008; Weist et al., 2006). Yet, prior measures of school mental health quality did not explicitly address structures and functions of a high performing team (Iachini et al., 2013; Weist et al., 2012) or overall quality improvement efforts to advance teaming performance (Meltzer et al., 2010; Pryor et al., 2009). The primary purpose of the SMH-QA is to provide a tool for CSMHS teams to self-assess the strengths and needs of their system to inform quality improvement priorities and action plans. Over time, this type of performance assessment and improvement can lead teams to more structured and sustainable performance improvements for their local CSMHS.

Figure 1.

School Mental health Quality assessment: Teaming indicators and Best practices. Note. This figure represents the Teaming domain (including the indicators and best practices) of the School Mental health Quality assessment (SMH-QA-Version 1; Hoover et al., 2015).

Teaming continuous quality improvement

School district teams can overcome barriers to effective teaming with the support of targeted guidance and continuous quality improvement (Iachini et al., 2013). Learning communities are one approach to provide this type of structure and support for quality improvement and are increasingly used to improve behavioral health systems (Hoge et al., 2020). Learning communities focus on specific quality improvement target(s) for participating teams to achieve via a set of established best practices; key components include prework (i.e. convening an expert panel and organizations demonstrating commitment), learning sessions, team calls, e-mail or web support, and Plan-Do-Study-Act (PDSA) cycles (Nadeem et al., 2014). PDSA cycles are small-scale, rapid tests of change that teams use to experiment with change ideas and applications of best practices to improve quality (e.g. reduce variation, improve efficiency). PDSAs are conducted in interactive cycles to monitor progress and predict outcomes of a change over time; the process is designed to implement changes to drive improvement (Taylor et al., 2014).

The current study took place in the context of the first national school mental health learning community in the United States (Connors et al., 2020). One focus of this learning community was to enable and speed up the application of teaming best practices from the research literature to the ‘real world’ operations of school mental health systems via a structured quality improvement process as described above. The study investigated novel strategies used by school districts through PDSAs to improve teaming and collaboration amongst multidisciplinary school mental health team members. This study investigated the following research questions:

• RQ1: Do school districts that tested changes to school mental health teaming during a 15-month school mental health learning collaborative report improved teaming performance as compared to districts that did not focus their quality improvement efforts on teaming?

• RQ2: Is the number of teaming quality improvement tests of change completed positively related to amount of improvement in overall teaming performance?

• RQ3: What elements of teaming were addressed via quality improvement tests of change for teams with the most teaming performance improvements over the course of the learning collaborative?

Methods

Participants

Twenty-four school mental health teams at the district level participated in one of two cohorts of 15-month learning collaboratives focused on improving school mental health system quality and sustainability and led by the National Center for School Mental Health. Twelve teams participated in the first cohort and 13 teams participated in the second cohort (one district team participated in both cohorts for a total of 24 unique district teams). Each of the participating teams was comprised of approximately six members (range = 4 to 8; M = 6.7). Teams included at least one representative from the following groups: 1) school district or community leader, 2) school building administrator (e.g. principal, assistant principal), 3) school-employed supervisor/director of mental health personnel (e.g. district school psychology supervisor), and 4) community-employed supervisor/director of mental health (e.g. clinical director of a community behavioral health organization that provides school mental health services). Four teams included a youth, family member, or advocate. Team leaders included district mental health coordinators, special education district directors, nurse coordinators, and directors of community mental health providers.

The National Center for School Mental Health selected district teams through a competitive application process; applicants submitted letters of intent describing their capacity and commitment to participate in the learning collaborative; provided an overview of their CSMHS (including number of participating schools, number and description of staff, types of programs and services, school-community partnerships, demographic information on population served); and described their current challenges, opportunities, goals, and commitment to programmatic change in their school mental health system. The National Center for School Mental Health selected teams based on team composition, geography, and readiness to participate to establish a diverse national learning collaborative. Teams participating in the learning collaborative represented a variety of school districts in terms of size and location (rural, suburban, and urban). The 24 participating districts served 1,446 total schools, with each serving between two to 680 schools. Participating districts, in total, served approximately 815,222 students, with each district serving between 600 to 386,261 students. Although the application requirements, process, and dissemination strategy were identical for both rounds of cohort recruitment, Cohort I teams represented primarily medium and large school districts while Cohort II represented primarily small and medium school districts. See Table 1 for participating district team characteristics.

Table 1.

Participating district characteristics

Team	Cohort	State	Schools	Student population	Urbanicitya	% families with children below poverty line
1	1	CA	8	4,100	2	41.30
2	1	MA	5	6,953	1	6.40
3	1	CA	32	21,800	1	9.60
4	1	CA	15	8,029	1	9.80
5	1	CT	20	15,715	2	11.60
6	1	MD	188	84,976	1	24.60
7	1	IL	680	386,261	1	25.20
8	1	TN	156	85,795	1	22.10
9	1	MN	76	35,356	1	1.70
10	1	IL	1	1,700	1	12.90
11	1	WI	2	1,175	3	15.60
12	1&2	KS	10	4,577	4	27.80
13	2	CA	19	30,000	1	16.30
14	2	NY	8	5,960	1	4.80
15	2	MN	2	907	6	8.00
16	2	NH	3	1,100	7	18.80
17	2	NH	3	1963	2	0.50
18	2	RI	42	24,000	1	30.40
19	2	CA	16	11,249	1	7.20
20	2	MN	8	3,000	4	10.60
21	2	NC	19	12,574	2	11.50
22	2	DC	113	48,439	1	23.30
23	2	CA	6	3,086	1	22.60
24	2	NH	4	520	7	13.90

^a1 Counties in metro areas of 1 million population or more

2 Counties in metro areas of 250,000 to 1 million population

3 Counties in metro areas of fewer than 250,000 population

4 Urban population of 20,000 or more, adjacent to a metro area

5 Urban population of 20,000 or more, not adjacent to a metro area

6 Urban population of 2,500 to 19,999, adjacent to a metro area

7 Urban population of 2,500 to 19,999, not adjacent to a metro area

8 Completely rural or less than 2,500 urban population, adjacent to a metro area

9 Completely rural or less than 2,500 urban population, not adjacent to a metro area

Design

The 15-month learning collaborative used a modified version of Institute for Healthcare Improvement Breakthrough Series Collaborative model and the associated Model for Improvement (Langley et al., 2009). The learning collaborative included an orientation webinar, a two-day, in-person learning session, monthly learning collaborative action period calls, and quarterly virtual learning sessions. The National Center for School Mental Health, in collaboration with the project’s national project advisory group, school mental health subject matter faculty, and quality improvement advisors, facilitated all components of the learning collaborative and provided individualized guidance to school district teams on their quality improvement work. Learning content was based on the best practices and performance indicators in the School Mental Health Quality Assessment-Version 1 (Hoover et al., 2015) and learning collaborative teams received didactic training and quality improvement resources on each domain. However, teams were encouraged to select 1–2 specific domains of most relevance for their own quality improvement efforts. As a result, only 15 of the 24 teams focused on quality improvements in teaming specifically, by submitting at least one Plan-Do-Study-Act (PDSA) cycle focused on teaming quality improvement.

Learning collaborative teams submitted performance measures and at least one PDSA each month, with 65% and 98% data submission, respectively, across all teams (Connors et al., 2020). PDSAs were used to document school district specific changes in school mental health quality. Action period calls focused on reviewing PDSAs and discussing observed changes in progress measures. Virtual learning sessions focused on best practices for achieving school mental health quality across all domains (including teaming) and were complemented by field examples and presentations from participating districts. Approximately 30 learning collaborative participants in each cohort (usually three members from each district team) attended monthly, 60-minute ‘action period’ calls and quarterly virtual learning sessions. Additional details about the learning collaborative design are available in Connors et al. (2020).

Teaming performance

Teaming performance was measured by the teaming domain of the School Mental Health-Quality Assessment-Version 1 (SMH-QA-Version 1; Hoover et al., 2015). Teams completed and submitted this performance assessment at the beginning and end of the learning collaborative. The teaming domain subscale included the following performance indicators: (1) having a multidisciplinary team; (2) promoting efficiency; (3) using meeting best practices; (4) promoting data sharing; and (5) connecting to community resources (see Figure 1). Teams rated their CSMHS on all indicators using a Likert-style scale of 1–6 to assess degree of implementation of best practices (1 = Never, reflecting lowest performance; 6 = Always, reflecting highest performance). Items ratings are averaged to produce a teaming performance score. Scores of 1.0–2.9 are classified as ‘Emerging’ areas, 3.0–4.9 are classified as ‘Progressing’ areas, and 5.0–6.0 are classified as areas of ‘Mastery.’

The SMH-QA is the first federally-funded set of national performance standards for school mental health quality for school and district teams throughout the United States and territories. The SMH-QA was developed based on prior team-based, self-assessment measures for school mental health quality, refined via a multi-stage process with stakeholder input and expands prior measures by including an explicit domain for teaming practices and structures. See Figure 1 for the five teaming performance indicators and best practices. Preliminary results suggest the SMH-QA has strong internal consistency (α = .88).

Quality improvement tests of change

Quality improvement tests of change were assessed by the number and type of PDSAs completed in the Teaming domain. District teams completed a PDSA worksheet documenting their quality improvement goal, the specific change idea that they used, results of that test when it was piloted, and intentions for the next PDSA (i.e. abandon the change if it didn’t work, adapt it to try a different variation or scale in the next PDSA, or adopt it into implementation). See the PDSA worksheet in Supplemental Material for an example. Learning collaborative teams could focus on several areas for improvement related to the teaming indicators and best practices, and were encouraged to consider innovative approaches to drive quality improvement in their CSMHS teaming efforts.

The total number of Teaming PDSAs that each team completed over the course of the learning collaborative, measured the amount of quality improvement tests of change. The National Center for School Mental Health coded PDSA content based on the Teaming indicators to measure quality improvement test of change content.

Data analytic plan

Descriptive statistics were used to describe the data and distribution including testing assumptions of normality, homogeneity of variance, and linearity. RQ1 (i.e. whether teams who submitted teaming PDSAs had better overall teaming performance than teams who did not) was examined using an ANOVA test to compare pre and post SMH-QA Teaming Domain subscale scores. RQ2 (i.e. whether amount of teaming PDSAs are related to overall teaming performance), the non-parametric Spearman’s rank correlation was used because the data violated parametric assumptions. RQ3 (i.e. elements of teaming that were addressed for improvement) was analyzed using a framework-guided rapid content analysis (Gale et al., 2019) of the completed Teaming PDSA worksheets. One coauthor completed the rapid content analysis of all Teaming PDSA worksheets; a second coauthor reviewed the content analysis and discrepancies were resolved through consensus. The study team used SPSS to conduct all data analyses.

Results

Of the 24 total learning collaborative teams, all 24 teams completed PDSAs. The number of Teaming PDSAs completed by each team ranged from 0 to 12 (mean = 2.67, median = 2). Fifteen teams had one or more Teaming PDSA (range = 1 to 12, mean = 4), and nine teams had three or more Teaming PDSAs (i.e. above the mean). Teams were not required to complete Teaming PDSAs and could select other areas of focus for their PDSAs during the learning collaborative. Of the nine teams that did not complete any Teaming PDSAs, three of these teams did not complete the SMH-QA at the end of the learning collaborative and thus had missing data for teaming performance and were unable to be included in teaming performance analyses. These three teams were not different than the other learning collaborative teams on geographic or demographic characteristics of their school districts, or in their participation in the learning collaborative.

Teaming quality improvement relation to teaming performance

For all teams, mean teaming performance at baseline was 3.08 (SD = 0.76) and increased to 4.08 (SD = 0.88) at the end of the learning collaborative (t(20) = −5.20, p <.001). This reflects a significant improvement for teams within the ‘emerging’ range of teaming performance. For the 15 districts that specifically worked on teaming, mean teaming performance at baseline was 2.90 (SD = .81) and increased to 4.00 (SD = 1.01) at the end of the learning collaborative (t(14) = −4.19, p <.001). The districts that did not submit a Teaming PDSA had a higher baseline but less improvements in teaming over the course of the learning collaborative (baseline: M = 3.53, SD = .37; end of LC: M = 4.27, SD = .45) but there was not a significant difference between the two groups (RQ1- F(20) = .73, p = .40; See Table 2).

Table 2.

Teaming performance.

	Baseline		End of LC
	M	SD	M	SD	t	df	p
Overall sample	3.08	.76	4.08	.88	−5.20	20	<.001
Districts that worked on teaming	2.90	.81	4.00	1.01	−4.19	14	<.001
Districts that did not work on teaming	3.53	.37	4.27	.45	−6.57	5	<.001

To answer RQ2, a Spearman’s rank correlation was used to assess the relation between the number of Teaming PDSAs completed by learning collaborative teams and the difference in scores between baseline and the end of the learning collaborative. For the 15 learning collaborative teams that completed a Teaming PDSA, the correlation between number of PDSAs and teaming performance change score was not significant (r_s (14) = .20, p = .50)

Elements of teaming quality improvement

Of the 15 districts that completed Teaming PDSAs, seven (50%) had a teaming performance change score above the average (i.e. a 1-point increase in score between baseline and the end of the learning collaborative). These seven districts reported teaming quality improvement in three areas: multidisciplinary teams, collaboration and teaming structures, and meeting structure best practices. Table 3 displays PDSA topics by team for each of the five quality indicators. However, in practice, specific quality improvement tests of change often overlapped several SMH-QA indicators. These are described under each theme, below. All seven districts reported engaging in teaming quality improvement for at least two of the teaming quality indicators and most districts engaged in quality improvement efforts based on three teaming quality indicators.

Table 3.

Team PDSA quality improvement content based on the SMHQA teaming indicators.

	Team PDSA quality improvement content based on the SMH-QA teaming indicators					Teaming performance score on SMH-QA
Team	Multidisciplinary team membership	Avoid duplication and promote efficiency	Use meeting best practices	Promote data sharing	Connect to community resources	Baseline	15-month follow up	Change Score
2	Increase the number and diversity of roles of CSMHS team members	Define roles of CSMHS team members and test effective case load models			Evaluate community partnerships and develop a standard MOU for district	2.2	5.4	3.2
6	Collaborate with school principals	Develop a district framework for collaboration				3.0	4.8	1.8
8	Invite more district team members in diverse roles to join CSMHS leadership	Conduct a survey to identify overlaps between district efforts and better coordinate initiatives	Schedule consistent meeting times and use meeting best practices			2.0	3.4	1.4
11	Expand roles included in CSMHS leadership team	Change in team structures to include subcommittees	Change meeting structure to promote efficiency			2.33	4.2	1.9
16	Include University partner in CSMHS team	Implement Tier 2 services and supports	Implement meeting best practices to ensure efficient use of time to cover topics	Use standardized process to share and select quality improvement efforts	Work with a University partner to develop training for Tier 2 teaming	3.8	5.2	1.4
14	Develop an inclusive district CSMHS team by adding community partners, students, and families	Identify district leadership staff across a MTSS			Expand leadership team to include community partners	2.8	4.4	1.6
12	Increase collaboration with community partners			Track behavioral health outcomes within academic tracking system	Collaborate with community mental health partners to build alternatives to suspension program	3.6	4.6	1.0

Note. Teams submitted Teaming PDSAs and had teaming performance change scores above the average (i.e. at least a 1-point increase in score between baseline and the end of the learning collaborative). Teaming performance was measured based on a Likert-style scale of 1–6 assessing degree of implementation of best practices (1 = Never; 6 = Always). The ‘Baseline’ scores represent the mean Teaming score for each district on the School Mental Health Quality Assessment at the start of the learning collaborative; the ‘End of LC’ scores represent the mean Teaming score for each district at the end of the learning collaborative. Composite Score of 1.0–2.9 are classified as ‘Emerging’ areas, 3.0–4.9 are classified as ‘Progressing’ areas, and 5.0–6.0 are classified as areas of ‘Mastery’.

Multidisciplinary teams

Most districts diversified their CSMHS team members at both the district and school levels. Many districts expanded district leadership teams as a starting point. For example, to inform the addition of team members and the collaboration of the larger district team, one school district conducted a survey of initiatives across the district. This survey enabled the team to identify overlap between district efforts and better coordinate initiatives. Several districts focused on diversifying both their district and school teams by creating youth and family councils. For example, one district created youth and family advisory councils that met monthly to collaborate in the development of new initiatives, refine existing programming, and bolster district communication with all students and families. Participants reported in their PSDA tracking sheets that these efforts resulted in increased support from multiple stakeholder groups for their CSMHS.

Collaboration and teaming structures

Many districts worked toward improving teaming structures at both district and school levels. These structures include staffing and workflows (to avoid duplication and promote efficiency), such as connection to community resources and providers, and processes and models for effective collaboration. At the district level, one district identified the lack of clarity across the district about the definition of Multi-tiered Systems of Support for Student Behavioral Health. They first worked at the district level to clarify their Multi-tiered Systems of Support leadership, functioning, and services included. This effort consisted of expanding their district CSMHS leadership team to include community partners and student and family representatives. They also invited the general school community to attend regularly scheduled CSMHS planning meetings.

Several districts focused on collaboration and teaming structures at the school level. Many looked at their school-level teaming structures and considered ways to ensure they were effectively using staff. Districts developed models for collaboration and roles and responsibilities matrices to help standardize their CSMHS teaming model within each school across the district. Districts also found utility in creating standardized memorandum of understanding for their relationships with community partners providing school-based services.

Meeting best practices

After addressing school and district collaboration and teaming structures, many districts then looked at specific strategies to improve their meeting efficiency, including having regularly scheduled meeting times and using meeting best practices (e.g. creating and using an agenda), focusing on making decisions, and using meeting time to follow up on the status of action items.

Discussion

Efficient and effective multidisciplinary school mental health teaming is necessary, given the wide array and variability of school- and community-employed staff who support student mental health in our nation’s schools, and the current and growing need for mental health supports and services related to the COVID-19 pandemic (Chegg et al., 2020; Loades et al., 2020). Although national best practice standards for school mental health teaming exist, many districts and schools are unaware of these best practices or face barriers to using them (Doll et al., 2005; Markle et al., 2013). Underuse of best practices can result in poor coordination, inefficiencies, and role confusion among disciplines that can hinder the overall access and quality of mental health care for students (Iachini et al., 2013; Mellin et al., 2011; Splett et al., 2017). The current study demonstrated that engagement in a continuous quality improvement learning collaborative process resulted in school district improvement in the implementation of school mental health teaming best practices. Specifically, 24 school districts participating in a 15-month national learning collaborative reported increased implementation of best practices in teaming, including expanding diversity of teams, engaging youth and families, improving collaboration and teaming structures, and using meeting best practices.

Notably, teams that demonstrated the most improvement in teaming reported change in at least two different areas of teaming, which suggests that improvement efforts will be more successful if teams focus more broadly on teaming rather than working intensively in one area. Additionally, specific examples of real-world teaming improvements implemented by participating districts provided case examples for other districts and schools to support their efforts to advance more effective multidisciplinary supports for students.

Among districts that chose to specifically focus on teaming as a targeted area for continuous quality improvement, most started teaming improvements at the district level, followed by parallel efforts at the school building level. For example, districts first delineated roles and responsibilities of Specialized Instructional Support Personnel supervisors and the intended role of community mental health partners in the district, and then replicated this process at the school building level with district roles and responsibilities as a guide. This process encouraged standardization of roles at the district level and simultaneously provided guidance to schools that could be tailored to their unique needs and staffing. This ‘district-first’ approach is consistent with prominent Multi-Tiered Systems of Support quality improvement efforts in the field such as Positive Behavioral Interventions and Supports and the Interconnected Systems Framework, both of which suggest starting implementation efforts at the district team level and then moving to school-level implementation (Eber et al., 2019).

District multidisciplinary teaming improvements also focused on developing partnerships with youth and family members. Youth and family partnership and involvement is a key quality improvement to diversify who is contributing to school mental health effort planning, implementation, and evaluation. Engaging a broader array of stakeholders in the system involves active partnership with youth and families for whom systems are designed to support but who are often not engaged beyond their role as service recipients. Youth and family engagement at all levels of system development and improvement can help better meet the needs of students and families by incorporating their perspectives on the needs and desires for support, and their ideas for system improvement (Weist et al., 2019).

Teams with the most improvements in school mental health teaming also focused their quality improvement efforts on concrete ways to improve their teaming processes by using meeting best practices such as using an agenda, taking notes and accountability for action items. Monitoring and improving meeting effectiveness can result in saved time and improved productivity (Allen et al., 2008; Delva et al., 2008), particularly in fastpaced settings, such as schools, where time, financial and staffing resources are consistently limited (Locke et al., 2015). Optimizing efficiency of workflows while maintaining high quality structures and services is a common quality improvement goal in learning collaboratives. Therefore, learning collaborative teams were able to tackle not just their overall teaming structures, but also the specific workflows and meeting processes needed to make meaningful improvements within school mental health teaming.

In the current study, learning collaborative teams used tests of change (PDSAs) that targeted 2–5 distinct School Mental Health Quality Assessment quality indicators at a time. It was not uncommon for teams to prioritize one teaming target indicator (e.g. avoid duplication and promote efficiency), which revealed the need to work on another related teaming indicator first or simultaneously (e.g. efficiency of teams often relies on how effective the meetings are, how multidisciplinary the team is in terms of stakeholders represented, ability to connect to community resources and share data). PDSAs are an optimal tool for teams to try their ideas for improvement on a small scale, then accelerate change by continuously monitoring the results and exchanging feedback and ideas with other teams in the learning community (Stafford et al., 2020). When used in succession, PDSAs offer an organized, tailored quality improvement method that school mental health teams often need to build on prior successes and lessons learned.

Resources for school mental health quality improvement

To spur learning beyond the participating learning collaborative teams, the National Center for School Mental Health developed tools that highlight effective practices and processes used by participating district teams to improve their overall school mental health quality. The federally-funded School Health Assessment and Performance Evaluation (SHAPE) System (www.theSHAPEsystem.com; National Center for School Mental Health (NCSMH), 2021) includes the updated version of the School Mental Health Quality Assessment (SMH-QA-Version 2; Hoover et al., 2019) that the National Center for School Mental Health refined since the current study through a multi-stage data driven process with broad stakeholder feedback.

The SHAPE System also includes School Mental Health Quality Guides for each quality domain, and offers individuals, schools, districts, and states best practices and strategies for improvement (National Center for School Mental Health (NCSMH), 2020). The Teaming School Mental Health Quality Guide on the SHAPE System includes findings and resources from the cohorts that participated in the learning collaborative and are highlighted as strategies for other districts and schools to leverage as they engage in their own continuous quality improvement. For example, participating districts developed matrices to delineate roles and responsibilities between professions and many developed comprehensive memoranda of understanding to improve partnerships with community mental health providers; these tools are available to the public at no cost on the SHAPE System.

Limitations

While findings from the current study are promising in terms of demonstrating the positive impact of engaging in a continuous quality improvement learning collaborative to enhance teaming practices and performance, the performance improvement metrics are based on a self-report assessment of implementation quality (i.e. the School Mental Health Quality Assessment). The value of a self-report instrument for continuous quality improvement is clear; it allows a team to reflect on their own relative strengths and weaknesses and determine priorities and strategies for quality improvement outside of the scrutiny or judgment of outside observers. This protects against socially desirable responding and results in a more meaningful quality improvement activity. However, teams may have inflated improvements over time to reflect the level of effort they had put into quality improvement rather than based on actual improvement in implementation quality. A more objective measure of best practice implementation in future research could be an outside observer’s report of the SMH-QA to validate the accuracy of self-report. Additionally, three teams did not complete the SMH-QA highlighting the potential difficulty that school district teams may have finding time, or seeing the utility in using such a measure.

Further, the relatively small sample size of 24 school districts limits our ability to generalize findings to the thousands of school districts across the United States. This learning collaborative study is consistent with others in size and did include diverse representation in terms of size and geography of districts, but could be augmented by a larger sample size and replication in future studies.

Future directions

Ultimately system and provider improvements targeted by a learning collaborative process such as the school mental health learning collaborative intend to have direct positive impact on system users or, in this case, students and families served by the CSMHS. While we can surmise that having a more efficient and coordinated school mental health teaming structure and process contributes to a more effective system of student support, future investigation of the impact of teaming improvements on student outcomes is warranted. Some prior research suggests that learning collaboratives have greater impact on provider-level versus patient-level outcomes (Nadeem et al., 2014), though this may have been an artifact of data examined before allowing sufficient time for patient-level outcomes to take effect.

While we await studies that better examine the impact of effective teaming structures and processes on student outcomes, it is reasonable to assert that the lessons and resources learned and developed in the current study may benefit districts and schools in their pursuit of more effective multidisciplinary support of students. In the spirit of the learning collaborative process, districts and schools can benefit from ‘sharing seamlessly and stealing shamelessly’ the lessons learned, practice innovations, and resources used and developed during the school mental health learning collaborative as they navigate their own teaming quality improvement. In the context of the global COVID-19 pandemic, schools are increasingly recognized as an essential hub for students to receive mental health support (Hoover & Bostic, 2021). Federal, state, and local investments and innovation are affording greater staffing, by both school-employed and community-partnered clinicians, to support student mental health. As such, teaming, collaborative learning structures, and rapid feedback cycles are critical to establish the strong networks needed for comprehensive mental health in schools to work effectively.