The implementation of a team training intervention for school mental health: Lessons learned

Abstract

Children obtain more mental health services through schools than through any other system. In urban, low-resource schools, mental health care often is provided by teams of contracted community mental health workers. Implementation of intended services may struggle in the context of challenges related to team functioning. Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) is an efficacious approach for improving team functioning in health care. In collaboration with stakeholders, we adapted TeamSTEPPS for school mental health teams, and pilot tested it in three schools participating in an ongoing implementation of cognitive behavioral therapy. Three teams randomized to receive TeamSTEPPS were compared with three teams who did not participate in TeamSTEPPS. Feasibility and acceptability of the adapted TeamSTEPPS and the impact on team skills and behavior was assessed through qualitative interviews and field notes, and quantitatively over the course of one school year. In this paper we describe the process of adapting and implementing TeamSTEPPS. In addition to providing the researchers’ perspective, we illustrate participant perspectives using qualitative data when possible. Key challenges included leader and staff turnover, logistical barriers (e.g., difficulty securing private space for qualitative interviews in schools), and navigating the protection of participant rights and autonomy given that prospective participants were employed by an agency with a vested interest in their participation. Concrete suggestions for overcoming challenges are provided to guide future research.

Schools have increasingly become the de-facto provider of children’s mental health services (New Freedom Commission on Mental Health, 2003). Few children receiving school-based mental health care receive evidence-based practices (EBPs; Flaspohler, Meehan, Maras, & Keller, 2012). EBP in psychology has been defined as, “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences” (APA Presidential Task Force on Evidence-Based Practice, 2006). In an effort to improve the quality of school-based mental health care, some systems have trained their school-based mental health providers in treatment approaches with strong empirical support, like cognitive-behavioral therapy (CBT). Training in efficacious, manualized interventions alone has not been shown to appreciably change clinician behaviors (Herschell, Kolko, Baumann, & Davis, 2010; Stark, Arora, & Funk, 2011), likely at least in part because most training strategies have neglected the critical role of the context in which the intervention is to be delivered (Beidas & Kendall, 2010). Improving the quality of mental health services students receive requires attending to these contextual influences (Damschroder et al., 2009). For example, a team’s culture, communication, and perceived implementation climate may all impact intervention implementation. These contextual factors may be important and potentially malleable targets in implementation research.

Many urban districts contract with community mental health agencies to send teams to schools to provide mental health care to students (Committee on School Health, 2004). This is the model that the School District of Philadelphia, the setting of the current study, uses. School-based mental health teams in Philadelphia typically consist of one-to-two masters’ level clinicians per school and various paraprofessionals, who assist the clinicians and provide classroom support to students. The services students receive include individual and group therapy, behavioral health support in the classroom, and crisis management. The Philadelphia Department of Behavioral Health and Intellectual disAbility Services (DBHIDS) organizes and pays for extensive training and consultation in CBT through the Beck Community Initiative (Creed et al., 2016; Creed et al., 2013; Stirman, Buchhofer, McLaulin, Evans, & Beck, 2009), including for many school-based clinicians. Challenges related to teamwork, such as the need for a strong leader, regular communication, and commitment from all team members, have been identified as key difficulties in school mental health (Weist, Grady Ambrose, & Lewis, 2006). We hypothesized that the addition of a team training intervention may have an important impact on factors driving implementation, which may in turn support practice change and improve the quality of care provided to students.

There is evidence from medicine that teamwork affects clinical care (Schmutz & Manser, 2013) and interventions designed to strengthen teams have been associated with improved clinical outcomes (Hughes et al., 2016; Neily et al., 2010). One such intervention, Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS; Agency for Healthcare Research and Quality, 2007), has been widely disseminated in hospital settings with favorable outcomes (Mahoney, Ellis, Garland, Palyo, & Greene, 2012; Mayer et al., 2011; Sawyer, Laubach, Hudak, Yamamura, & Pocrnich, 2013). TeamSTEPPS focuses on building competencies in leadership, situation monitoring, mutual support, and communication (King et al., 2008). Improvements in measures of teamwork and patient outcomes have been observed following TeamSTEPPS (Mahoney et al., 2012; Sheppard, Williams, & Klein, 2013; Stead et al., 2009).

Given the efficacy of TeamSTEPPS and our interest in supporting the context of school-based clinicians, we set out to adapt TeamSTEPPS for school mental health teams in partnership with stakeholders. Following adaptation, we pilot tested the adapted intervention in three schools participating in an ongoing DBHIDS-sponsored CBT implementation. The three teams randomized to receive TeamSTEPPS were compared with three teams who did not participate in TeamSTEPPS. Feasibility and acceptability of the adapted TeamSTEPPS and the impact on team skills and behavior was assessed qualitatively (i.e., interviews and field notes) and quantitatively over the course of one school year. The purpose of the present paper is to describe the process of adapting and implementing TeamSTEPPS. The main study outcomes have been detailed elsewhere (Wolk, Stewart, et al., unpublished manuscript). We illustrate participant perspectives and contextual observations using qualitative data augmented by our own experiences within the research team. We describe key challenges we faced during the study and provide concrete suggestions for overcoming these challenges to inform future research and implementations in this area.

Method

The TeamSTEPPS curriculum consists of an introductory module and four didactic modules targeting competencies in the areas of leadership, situation monitoring, mutual support, and communication (Agency for Healthcare Research and Quality, 2007; King et al., 2008). The training defines team skills, provides strategies for improving those skills, and tools for overcoming barriers (King et al., 2008). Vignettes and case examples are used to reinforce learning. See Table for additional details. Originally developed for medical settings, TeamSTEPPS required adaptation for school-based systems. Adaptations were developed in partnership with relevant stakeholders, including team members and administrators, and then pilot tested. The adaptation and pilot processes are described in detail in the sections that follow.

Table.

TeamSTEPPS Curriculum

Module	Rationale	Key Skills/Goals	Example from training
Introduction	Provide an overview of the TeamSTEPPS model and review the research on team training and TeamSTEPPS	Participants learn the rationale for TeamSTEPPS and why it was selected as the team training model for school mental health	Trainer discusses how the actions of the team impact clinical care and how when teams fail to work well together, children and families experience confusion, miscommunications, and uncoordinated care. Engaging in teamwork using proper strategies and skills can reduce mistakes and improve quality of care.
Leadership	Provide information on characteristics of effective leaders and practice optimal leadership strategies	Participants learn about and practice three strategies for assembling the team and facilitating team events: 1. Briefs (planning) 2. Huddles (problem solving) 3. Debriefs (process improvement)	Trainer sets the stage by explaining that Briefs are held for planning purposes; huddles are used for problem solving; and debriefs are for reflection and process improvement. Each is explored in greater detail. The team leader typically facilitates these types of team events but any team member can request a brief, huddle, or debrief at any time, which is an example of shared leadership.
Situation Monitoring	Situation monitoring, the process of actively scanning behaviors and actions to assess elements of the situation or environment, fosters mutual respect and team accountability. It is a skill that can be acquired, practiced, and strengthened.	Participants learn about and practice cross monitoring. This allows one to recognize risk or unfolding error and to interrupt or correct an action or event before the child’s care is impacted.	The trainer explains that monitoring enables team members to identify potential issues or challenges early enough so that they can correct and address them before they become a problem to the child. Examples of situation monitoring in school mental health include assessing the child’s mental status, noting problems in the environment, and being aware of workload spikes and stress levels among team members.
Mutual Support	Mutual support is the key principle of teamwork. It includes anticipating the needs of other team members through knowledge of their roles and protects team members from work overload.	Participants learn the importance of assisting team members in their tasks, strategies for providing effective feedback to one another, how to effectively advocate for children to team members, and conflict resolution strategies.	The trainer teaches that effective feedback includes the following—it is… Timely—Feedback is most effective when the behavior being discussed is fresh in the mind of the receiver. Respectful—Feedback should not be personal, it should be about behavior. Specific—Feedback should be related to a specific situation or task. Directed—Goals should be set for improvement Considerate—Be considerate of team members’ feelings when delivering feedback.
Communication	Communication is a process of clear and accurate information sharing. There is a large body of research supporting the efficacy of strong communication skills for effective teamwork.	Participants learn specific communication strategies for framing conversations using the SBAR approach, for transitioning care of a child to a team member, and the importance of employing closed-loop communication strategies.	The trainer teaches and participants practice the SBAR method, which stands for situation, background, assessment, recommendation. It is an easy-to-remember tool for framing conversations, especially critical ones requiring a clinician’s action or attention. SBAR can be used to standardize communication which is helpful in developing teamwork.

Agency for Healthcare Research and Quality (2007). TeamSTEPPS instructor guide. Rockville, MD: USUHS Publication

Agency for Healthcare Research and Quality (2017). TeamSTEPPS: National Implementation. Retrieved from http://teamstepps.ahrq.gov/

Our work adapting and implementing TeamSTEPPS was community-partnered (Jones & Wells, 2007) and guided by the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009). CFIR synthesizes the many frameworks in implementation science (Tabak, Khoong, Chambers, & Brownson, 2012) and contains five major domains: 1) intervention characteristics; 2) outer setting (i.e., the economic, political, and social context within which an implementing organization exists); 3) inner setting (i.e., characteristics of the organizational setting); 4) characteristics of individuals involved in the implementation; and 5) the implementation process. Characteristics of the five CFIR constructs have been associated with successful implementation (Damschroder et al., 2009).

Adaptation

Participants.

In developing the adaptations we partnered with a large community-based mental health agency serving youth and families in nine local public schools at the time the study was conducted. We established a community advisory board (CAB) which consisted of the PI (CBW), two leadership representatives from the agency, three clinical managers who are licensed clinicians and team leaders, two full-time clinician team members, one mobile therapist and one paraprofessional staff member.

Procedure.

We partnered closely with key stakeholders to adapt TeamSTEPPS for the school mental health context. This community-partnered approach was time consuming but important in ensuring the adaptation was appropriate to context and relevant to key stakeholders (Newman et al., 2011; Southam-Gerow, Hourigan, & Allin, 2009). A key mechanism of this partnership was the CAB, which was directed by the PI and which met at the agency (a more central location for the participants than the Pi’s research institution). To recruit for the CAB, leadership invited all school-based clinicians and paraprofessional staff members of the agency to participate during staff meetings. Those who expressed interest were asked to apply and agree in writing to participate to ensure they could commit for the duration of the research. We had initially anticipated a smaller CAB but agency leaders were concerned that excluding interested staff would negatively impact morale. In response to our community partners’ preferences, we invited all who applied to participate.

The PI was an equal partner in the CAB, but charged with executing the adaptation. The other stakeholders served as expert members of the research team, offering advice based on their understanding of contextual factors in school-based mental health. The first CAB meeting operated much like a focus group in that we assessed perceptions and opinions of these key stakeholders to guide the adaptation process. Goals included identifying barriers and facilitators to implementing a team training strategy. For example, we asked stakeholders to describe ideal team training approaches and barriers to implementing these ideal approaches. We also identified possible adaptations to TeamSTEPPS so that its applicability to teams would be enhanced. The CAB met monthly over a five month period and discussed challenges as the adaptation proceeded iteratively, module by module. Consensus on important points was determined by a 70% majority, consistent with the literature (Newman et al., 2011). CAB participants were compensated for their time at the rate of $35 per meeting. Leaders were offered compensation but declined. Refreshments were provided at all CAB meetings.

Piloting of the adapted TeamSTEPPS intervention

Participants (N = 27 individuals representing six school-based teams).

Teams varied in their compositions. Typically, they consisted of two masters’ level clinicians per school, one of whom was designated the team leader, several paraprofessional behavioral health workers, and a case manager who often worked with multiple school-based teams. We randomly selected six teams from the pool of nine teams in the agency. Per agency leadership’s report, all school-based clinicians (n = 6, 22%) had received either in person or web-based training in CBT through the DBHIDS-sponsored Beck Community Initiative (Creed et al., 2016) and were participating in ongoing consultation. Briefly, participation in the Beck Community Initiative involved 22 hours of didactic instruction followed by 6 months of weekly group consultation in transdiagnostic CBT. Case conceptualization, intervention, and personalized treatment planning were emphasized. See Creed and colleagues (2016) for additional details. No additional inclusion/exclusion criteria were applied.

Procedure.

All study procedures were approved by the University of Pennsylvania IRB, Philadelphia’s IRB, and the school district’s Office of Research and Evaluation. Informed consent was obtained prior to engagement in research activities. Teams were randomized to receive the adapted TeamSTEPPS training. Questionnaires were completed at pre-training and at one, five, and eight month post-training. Quantitative measures included the_TeamSTEPPS Teamwork Perceptions Questionnaires (T-TPQ; American Institutes for Research, 2010), a 35-item self-report measure of individual perceptions of group-level team skills and behavior; the TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ; Baker, Krokos, & Amodeo, 2008), a 30-item self-report measure of individual attitudes related to the core components of teamwork captured within TeamSTEPPS; the Evidence Based Practice Attitude Scale (EBPAS; Aarons, 2004), 15-item self-report measure of attitudes toward adoption of EBPs; and the Maslach Burnout Inventory Human Services Survey (MBI-HSS; Maslach, Jackson, & Leiter, 1996), a 22 item self-report measure of therapist burnout. The T-TPQ and T-TAQ have been previously used with school mental health teams (Wolk, Locke, et al., unpublished manuscript).

Additionally, participants who received the TeamSTEPPS intervention completed post-training measures and were invited to participate in a one-time, in-person, semi-structured interview (n = 7). A standardized interview guide was developed and consisted of three parts: general views about the feasibility of the adapted TeamSTEPPS (e.g., the extent to which TeamSTEPPS can be continued in their organization and extended to other similar programs), perceptions of the acceptability of the adapted TeamSTEPPS (e.g., was it agreeable, palatable and satisfactory), and a review of findings from the quantitative data and solicitation of participants reflections on the quantitative results. Interviews were digitally recorded, transcribed and organized for analysis in NVivo. Analysis followed an integrated approach (Bradley, Curry, & Devers, 2007). We identified both a priori attributes of interest (core components of TeamSTEPPS, acceptability, feasibility) and also used modified grounded theory (Charmaz, 2000; Glaser, & Strauss, 1967) to systematically and rigorously identifying emergent codes and themes. Two members of the research team (CBW and RES) separately coded a subset of transcripts and compared their application of the coding scheme to assess its reliability and robustness. Disagreements were resolved through discussion. The codebook was refined based on these discussions and the revised codebook was then applied to all transcripts. CBW coded all transcripts and RES separately coded 62%; reliability was excellent (κ = .93). Note that RES was not involved otherwise with conducting the study, developing the interview guide, or interviewing participants.

Finally, team skills were documented using a validated measure (Flin et al., 2003) adapted for the study (Wolk & Mandell, 2016) and with qualitative field notes in which we captured additional details about the context. Participants were compensated for their time in completing study measures and interviews at the rate of $50 per hour.

The results of the quantitative and qualitative findings regarding the acceptability, feasibility, and effectiveness of TeamSTEPPS are detailed elsewhere (Wolk, Stewart, et al., unpublished manuscript). In addition to the characteristics of our community participants, below we report the lessons learned as noted by our team throughout the implementation with illustrative quotes. We describe key challenges encountered during the study and provide suggestions for overcoming them in future research. The primary obstacles encountered spanned CFIR domains and included loss of agency champions, high staff turnover, logistical barriers (e.g., difficulty securing private space for qualitative interviews in schools), and challenges we faced in navigating protecting participant rights and autonomy given that prospective participants were employed by an agency with a vested interest in their participation. Our qualitative work informed our understanding of challenges and provides illustrative quotes from interviews and field notes, when applicable.

Results and Discussion

Participants reported a mean age of 36.7 (SD = 12.0) years and were predominately female (67%). They had completed their most advanced degree 8.0 (SD = 8.1) years earlier; 37% had earned a master’s degree, 52% a bachelor’s and 11% declined to report highest degree earned. Participants self-reported race/ethnicity was 19% white, 59% black, 4% Asian American, 4% Hispanic/Latino, 4% multiracial, and 4% indicated “Other.” Participants included 22% clinicians, 59% paraprofessional providers, 7% case managers, and 4% leadership. Over the course of the school year, 46% of team members in the research study left their school team or agency (i.e., turnover). Additionally, the two key leaders also left early in the pilot implementation year for other employment. These individuals had been key partners in the project, and the implications of their departure and the high staff turnover are discussed later in this manuscript.

Loss of Agency Champions

This project took place over the course of three years. At the time we began planning the study, two key agency leaders were very eager to participate and expressed that the study aims were an excellent match to their identified needs. These leaders secured approval from the agency’s senior leader, were instrumental in recruiting for the CAB that worked to adapt the intervention, and actively participated in the CAB, which met during the first year of study funding. Just prior to piloting the intervention at the beginning of year two, both leaders left the agency for external positions and their departures occurred in close proximity to one another. At the time, the agency was also encountering regulatory and fiscal challenges which may have impacted the culture in the organization. The new leadership was less involved in the project than the original champions and this was felt by the staff. For example, one participant noted that in order to enhance use of TeamSTEPPS in the agency, “I think it would start up at a higher management level.” Another noted, “If that program could be built into the framework of it [the agency], then I think it would be more helpful and beneficial like that. And not just something extra added on.”

In particular, recruitment for participation in research activities was likely hampered by the loss of these individuals who had previously played an important role in advertising and endorsing study participation. Additionally, turnover amongst school team members was high (described below) and this was likely at least partially influenced by the changes in leadership and larger challenges the agency was experiencing during the course of the study. In an effort to overcome this, we attempted to develop relationships with the new leaders. This was somewhat successful. We also worked with the outgoing leaders to schedule the TeamSTEPPS training before their departure and invited the incoming leadership to attend the training. One of the two incoming leaders attended (the other had not yet been hired) and this individual became a helpful partner in the study. On participant stated:

I think having the slides and the documents [from the TeamSTEPPS training] has been great as well as having contact with [the PI] and X [new leader] has sat through [the training]. So I think we have some top-down support, which I think is helpful for every situation, but particularly when you’re trying to implement a new strategy or change when you show that the agency as a whole is doing it. It’s easier to get the individuals to go along with it.

However, because this new leader was in a new role and working in an agency facing some obstacles, her capacity to participate was limited. One participant reported, “I think it’s slowly becoming implemented and we just possibly need leadership to push what they want to see.”

Because this was a small study we were only able to partner with one agency. Future researchers should strongly consider partnering with multiple agencies as this would provide more variation in leadership and organizational culture as well as some protection against the impact of agency-specific challenges on study procedures and outcomes.

Staff Turnover

As noted earlier, in addition to losing two key leaders from within the agency, 46% of team member study participants left their assigned school during the course of the school year. Some left employment in the agency all together while others were reassigned to new schools. This presented two key challenges. First, new staff members who replaced departing team members may not have participated in the TeamSTEPPS training if they were not previously employed in the agency at a school randomized to the intervention. Additionally, the change in team composition was described by some participants as disruptive:

Some of the challenges that I think are not necessarily unique to our situations that we’re facing this year is that there is a lot of turnover, there is a lot of transition… so a lot of the team this year is new… and getting together and getting that like meld and focus… people get into either stagnant of what they used to do and so getting people to maybe switch up how they’re thinking or trying a new way of doing things has been a challenge.

We did not have a mechanism for training new staff in TeamSTEPPS or providing booster trainings for existing staff. Future studies should plan for this in advance. One of our participants noted, “I would hope that there would be ongoing trainings either through you guys or through inside the agency itself just because the turnover is so high.” We agree with this sentiment. Second, as our participation rates diminished considerably over the course of the school year and our sample sizes at the latter two follow-up intervals were low and may have limited our ability to detect effects. To address this, we are focusing our reports of the data on earlier time points and the more illustrative qualitative work. In future studies we will consider adding a mechanism to enroll new staff in the research mid-school year and to provide ongoing trainings for new staff. Ideally, this would be done in partnership with the agency in which their own staff would become trainers (i.e., a train the trainer model) and trainings incorporated in to the onboarding of new staff to promote sustainment. In general, research should anticipate high rates of staff turnover in community settings and develop a priori plans for mitigating the impact of turnover on the research.

Logistical Challenges

Logistical barriers were present and mainly due to the under-resourced nature of the school setting and the fact that the mental health providers were contracted to work in the schools and not school employees. Limited resources are common in urban public schools and working in this sometimes high stress environment may have contributed to the high level of turnover we experienced. The chaotic nature of the setting may have contributed to teams feeling as though they had little time to focus on teamwork because they were so frequently responding to crises. Research from within this community mental health system corroborates high levels of provider turnover and that organizational factors are important predictors of turnover (Beidas et al., 2016).

Working on this project has highlighted for us that while intra-team collaboration is important, inter-professional collaboration between contracted mental health providers and school-employed personnel also is likely crucial to implementation success. Participants reported varying degrees of collaboration and integration with the schools. Unique implementation challenges likely exist in non-specialty behavioral health care, including expectations that personnel from various disciplines and systems (e.g., school personnel and contracted mental health providers) will know how to collaborate with one another and navigate potentially disparate missions and goals for care. Few implementation studies focus on inter-organizational and inter-professional factors (Stark et al., 2011). Specific, rigorous strategies to facilitate alignment between professionals from different disciplines and organizations have not been sufficiently explored and future research in this area is recommended.

One challenge of conducting research in urban school settings was that there was very little private space available. This not only impacts care provided to students, but it was particularly relevant for collecting qualitative interview data. This was frequently reflected in our field notes (e.g., “they don’t have another private space so we meet in the hallway and there is lots of background noise from students and teachers”). As a result, some interviews were conducted in public spaces with various degrees of background noise, which interfered with the transcription of audio recordings of interviews. In other cases we were able to find quiet spaces in empty school libraries or cafeterias. The advantage of the lack of private space was that teams spent a lot of time together and we were able to observe team members interact with one another and with students during our visits because most activities occurred in a shared space. To capitalize on this, we took detailed qualitative field notes during visits and we analyzed these along with our interview data.

Another challenge to conducting research in this setting was that the school-based team members were in and out of the office to work with students for individual therapy or inclassroom support. In some instances team leaders arranged for us to come and collect follow-up survey data at one time point and arranged for all team members to be present. In other teams this did not happen and we had to stay in the school for much of the school day in order to “catch” all our participants when they were in the team’s main room. This underscores the time-intensive nature of conducting research in the community and the need to be flexible. In general, we tried to create as little disruption to the teams’ workdays as possible by being very flexible and believe that served us well in that team leaders were very accommodating of our requests to schedule data collection visits.

Protecting Autonomy

One challenge in partnering with agency leadership for research, which is necessary in this type of collaborative work, is that a leader’s endorsement of a research project has the potential to impact participants’ feelings of autonomy regarding participation. It was important to us that no one feel coerced into participating and we took several steps to ensure this. First, when discussing participation in the project and consenting individuals we did it in the absence of senior leadership. We also emphasized that participation was voluntary and that we would not report back to leadership regarding who agreed or declined to participate in the research. Finally, we only reported aggregated data back to agency leadership, taking care not to identify individual participants or school teams as we disseminated results to them. It is important that researchers develop plans for ensuring individuals do not feel coerced into research participation, communicate those plans to agency leaders in advance, and take every step to ensure participant confidentiality as a breach of confidentiality in these situations could negatively impact employment.

Conclusion

Conducting research in real-world settings presents a number of challenges that may be new for those more accustomed to traditional efficacy research. The adaptation and implementation process can be time consuming and intensive. In our adaptation of TeamSTEPPS for school mental health, the intervention ultimately remained unchanged with regard to core components but was adapted so that the language (e.g., changing the word patient to child) and case examples were relevant to the school mental health context. This adaptation process, while time consuming, was important and ultimately led to very positive feedback from participants, such as, “I think it fit really well and I think that was one of the things that stood out for me. I was like, Oh wow, they really know STS [the school based services program].”‘ It is important for those embarking on community-partnered implementation research be prepared to invest this time and energy. While we could have predicted some of the adaptations based on our team’s extensive experience conducting research within this context, we could not have anticipated others. The intimate knowledge of the service context that our advisory board partners brought to the table undoubtedly enriched the project in ways that could not have been achieved without their input.

Based on our experience implementing a team training intervention with school-mental health teams we have highlighted some of the particular challenges we encountered, how we attempted to overcome them, and suggestions for future research. We hope that the lessons we have learned around navigating leadership and staff transitions, logistical barriers, and the importance of protecting autonomy and confidentiality will be useful to other researchers. Partnering closely with multiple individuals and organizations from the beginning can mitigate many of the challenges we described and may be the most important recommendation we can provide. Had we partnered with more than one agency for this project we would have been able to examine organizational-level predictors of implementation and we also may have been less impacted by staff and leader turnover. In the particular agency we partnered with, turnover may have been at least partially attributable to fiscal and regulatory challenges the agency encountered during the study period. Partnering with more than one agency, when feasible, may afford researchers a greater buffer against agency-specific challenges that may arise.

In addition, being flexible, proactive, and responsive to one’s community partners’ needs and wishes is essential. By working closely with our agency partners we were often able to determine when to schedule study visits to maximize staff presence and minimize disruption to the team. By doing so we avoided inadvertently burdening teams. For example, we avoided study visits during the time that the agency was engaged in a routine audit as this audit was quite time consuming for them. We were also able to schedule our TeamSTEPPS training during the period of summer when the agency scheduled other mandatory trainings. This led to high training attendance.

Additional research that moves beyond identifying barriers and facilitators of implementation, for example by studying the implementation process, are needed. Measures like Saldana’s Stages of Implementation Completion (Saldana, 2014) are helping to advance the field in this regard. Additionally, studies that utilize adaptive designs to address implementation challenges as they become apparent are also important for advancing the field.

Clinical Impact Statement.

Question:

How can we enhance the implementation of evidence-based practice with school mental health teams?

Findings:

Adapting and implementing practices in new settings is challenging, but community-partnered approaches and attention to contextual factors may increase success.

Meaning:

TeamSTEPPS can be implemented in school mental health teams with appropriate attention to contextual factors that otherwise may impede successful implementation.

Next Steps:

Future studies that move beyond identifying barriers and facilitators of implementation and utilize adaptive designs may be helpful in advancing team-based practices.