Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jun 1.
Published in final edited form as: J Psychol Couns Sch. 2019 Sep 5;30(2):172–184. doi: 10.1017/jgc.2019.18

An examination of the factor structure of TeamSTEPPS measures in school mental health teams

Courtney Benjamin Wolk 1,§, Jill Locke 2, Eduardo Salas 3, Ricardo Eiraldi 4, Peter F Cronholm 5, David Mandell 1
PMCID: PMC7995600  NIHMSID: NIHMS1524213  PMID: 33777407

Abstract

Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS™) is a tested strategy for improving communication and climate in hospitals. It is a promising but untested tool among school-based mental health teams. We examined the psychometric properties of the TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ) and Teamwork Attitudes Questionnaire (T-TAQ) among 167 school mental health team members. Team members worked for one of five agencies in 33 K-8 urban public schools. Exploratory factor analyses and descriptive data are presented. For both the T-TPQ and T-TAQ, a unitary factor structure best fit the data for this sample. The T-TPQ and T-TAQ were not significantly correlated with one another and total scores did not significantly differ by staff role. Agencies differed in T-TAQ results, and one agency had lower T-TAQ total scores relative to other agencies. Results suggest that the factor structures are different among school mental health teams than among other healthcare providers.

Keywords: Teams, TeamSTEPPS, school mental health

Children in the U.S. obtain more mental health services through schools than through any other public system (New Freedom Commission on Mental Health, 2003). Received services often are ineffective, particularly in under-resourced schools (Farahmand, Grant, Polo, Duffy, & DuBois, 2011; U.S. Public Health Service, 2000). Effective interventions exist for preventing and treating most common mental health problems of childhood (Kutash, Duchnowski, & Lynn, 2006). However, few interventions have been adopted for use in schools and knowledge about evidence-based practices can vary greatly among providers in under-resourced schools (Eiraldi, Wolk, Locke, & Beidas, 2015). Increasingly, mental health services in urban U.S. schools are provided by teams, which often include master’s level providers like professional counselors (Eiraldi et al., 2015). Counselors working in school-based or other interdisciplinary teams often have not received any specialized training in providing team-based care, despite growing recognition that this should be a training priority (Boat et al., 2016; National Academies of Sciences, Engineering, and Medicine, 2017). Counselors on school-based mental health teams, especially those in leadership roles, may benefit from a deeper understanding of teamwork and how to facilitate optimal team processes amongst the team members they oversee.

Teamwork is important because the organizational context of school-based mental health programs, including the ways in which mental health team members interact and support each other, likely affects the quality of delivered services (Beidas & Kendall, 2010). Team science research can inform efforts to improve school-based mental health teams by addressing an important aspect of their organizational context: teamwork. Teamwork affects clinical performance (Schmutz & Manser, 2013) and teamwork training can improve clinical outcomes (Neily et al., 2010). A growing body of research on teams and how to support team-based work (Gregory, Feitosa, Driskell, Salas, & Vessey, 2013) has been applied successfully in medical settings (Hughes et al., 2016; King et al., 2008; Wagner et al., 2001). This literature has yet to be extended to school mental health teams, despite the commonalities. For example, school-based mental health teams are required to coordinate care activities for students, much like medical teams coordinating patient care. They must work together efficiently with individuals from interdisciplinary backgrounds functioning in a variety of roles, which may or may not be clearly defined. While each member of the team is responsible for carrying out independent tasks, they share the common goal of improving child outcomes. Relative to other areas of healthcare, research on school-based mental health team functioning is scant (Mellin, 2009; Mellin et al., 2010; Weist et al., 2012). The literature on team training in healthcare demonstrates the robustness of interventions regardless of trainee composition and work environment (Hughes et al., 2016), supporting the potential for extension to school teams.

Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS; Agency for Healthcare Research and Quality, 2007) is an evidence-based approach to improving team functioning that builds competencies in leadership, situation monitoring, mutual support, and communication (King et al., 2008). The U.S. Department of Defense Patient Safety Program in collaboration with the Agency for Healthcare Research and Quality developed TeamSTEPPS, which builds upon a growing body of team science literature that emphasizes the importance of team knowledge, skills, and attitudes in healthcare (Baker, Salas, King, Battles, & Barach, 2005). TeamSTEPPS has been widely disseminated in healthcare settings with promising results (Mahoney, Ellis, Garland, Palyo, & Greene, 2012; Mayer et al., 2011; Sawyer, Laubach, Hudak, Yamamura, & Pocrnich, 2013) including improved teamwork and communication (Mahoney et al., 2012; Sheppard, Williams, & Klein, 2013) and patient outcomes, such as decreased seclusion in psychiatric hospitals (Stead et al., 2009). TeamSTEPPS has been applied across a range of healthcare settings including intensive care (Mayer et al., 2011), emergency medicine (Turner, 2012), neonatology (Sawyer et al., 2013), surgery (Sanfey, McDowell, Meier, & Dunnington, 2011), inpatient psychiatry (Mahoney et al., 2012; Stead et al., 2009), and military trauma (Deering et al., 2011). It also has been applied broadly across large healthcare networks (Brock et al., 2012; Sheppard et al., 2013).

Given that improvements in team functioning have been demonstrated in other areas of healthcare and associated with improvements in quality of care, it stands to reason that implementing TeamSTEPPS in school mental health teams may impact team skills and behaviors and, ultimately, student outcomes. Evidence-based practice implementation efforts have generally been insufficient in producing long-term clinician behavior change (Stark, Arora, & Funk, 2011). Training is the primary implementation strategy to improve clinician delivery of evidence-based practice, despite the growing recognition that training alone does not improve service delivery (Beidas & Kendall, 2010; Edmunds, Beidas, & Kendall, 2013; Forsetlund et al., 2009; Herschell, Kolko, Baumann, & Davis, 2010). Thus, a cost-effective and evidence-based approach like TeamSTEPPS has considerable potential when added to evidence-based practice training as usual.

The core dimensions of teamwork that comprise TeamSTEPPS are team structure, leadership, communication, mutual support, and situation monitoring (Agency for Healthcare Resarch and Quality, 2007). Team structure refers to identifying the components of a team system that must work together to ensure optimal care delivery. Leadership is the ability to optimize team member’s activities through ensuring adequate resources, information sharing, and clarity around team activities. Communication refers to the structured processes by which team members exchange information. Mutual support is the anticipation and support of team members’ needs achieved via a shared understanding of responsibilities and workload. Finally, situation monitoring is the process of scanning and assessing the environment to gain necessary information or maintain awareness of important situational elements to support teamwork.

The TeamSTEPPS program includes two measures, the TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ; Agency for Healthcare Research and Quality, 2014b; American Institutes for Research, 2010) and TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ; Agency for Healthcare Research and Quality, 2014a; Baker, Krokos, & Amodeo, 2008), which assess the core TeamSTEPPS dimensions. The T-TPQ captures teamwork skills and behavior while the T-TAQ focuses on attitudes about teamwork. These measures may be used as tools to evaluate readiness for or the impact of TeamSTEPPS. Little research has evaluated the psychometric properties of the T-TPQ and T-TAQ, and none has examined its properties outside of traditional healthcare settings. Therefore, it is unknown if these distinct dimensions of teamwork identified in healthcare are similarly distinct and relevant to school-based mental health teams and counselors working in team settings.

In Philadelphia, the setting of the current study, school-based mental healthcare often is provided by school therapeutic services (STS) teams. This care can include individual and group therapy, one-to-one behavioral health support in the classroom, or behavioral health crisis management. STS teams comprise clinicians employed by community mental health agencies and contracted to work in schools, one of several common school-based mental health models (Committee on School Health, 2004). Teams typically consist of 1-2 masters’ level mental health counselors or clinicians per school supported by paraprofessional behavioral health workers (i.e., persons trained to assist independent clinicians but who do not have professional licensure).

In an effort to improve STS services in Philadelphia, the city’s Department of Behavioral Health has initiated and funded training for clinicians employed by publicly-funded community mental health agencies in evidence-based practices, such as cognitive therapy (Creed et al., 2016; Creed et al., 2013). Typically, the clinicians on STS teams receive trainings, with the expectation that they will then train other members of their team and school staff as needed. To date, implementation efforts have not sufficiently taken into account that the functioning of the mental health team may affect clinicians’ ability to train and supervise other team members in care practices.

As part of a larger study adapting and piloting TeamSTEPPS for school mental health teams, we examined the factor structure of the TeamSTEPPS measures in a sample of school mental health team members. Given that TeamSTEPPS and its associated measures have not been used previously with school mental health teams, a necessary first step in this agenda involves examining the T-TPQ and T-TAQ measures in this population. Previous studies have confirmed that the T-TPQ’s factor structure aligns with the key TeamSTEPPS dimensions as designed among other healthcare professionals (e.g., physicians, nurses); therefore, we hypothesized that a five factor solution would best fit the data. Exploratory analyses also examined the association between T-TPQ and T-TAQ scores, which has not yet been reported in the literature. Finally, given that previous research with the T-TAQ has demonstrated differential scores by staff roles, we conducted additional exploratory analyses to examine differences in T-TPQ and T-TAQ scores by staff role and agency.

Method

Participants

Participants included 167 mental health team members, recruited from five of the 19 community mental health agencies contracted by the city to provide STS services in Philadelphia. These five agencies provide services in 36 of the 109 schools (33.0%) with STS in the district and vary in size and scope of STS service provision, supporting their representativeness. Approximately 80% of children in the school district are eligible for free or reduced price lunch. On average, 33.4 providers per agency participated (SD = 21.2; Range = 7 to 57). Participants were all in the mental health field and employed by agencies contracted to provide mental health services to students in one or more of 33 urban public schools in the same district. An average of 6.2 providers participated from each school (SD = 4.2; Range = 1 to 15). Seventeen (10.2%) individuals identified as team leaders (e.g., clinical managers). These individuals also were licensed master’s level mental health providers, including both professional counselors and social workers. Forty-three (25.7%) individuals identified as master’s level clinicians (i.e., professional counselors and social workers with or without a license and not in a team leadership role). Ninety eight (58.7%) identified as paraprofessional providers, whose role is largely to support the work of the team’s clinicians by providing in-class mental health support and care to students. Three (1.8%) identified as case managers, and two (1.2%) as having a primary leadership role in the agency (i.e., clinical director, school services director). Data regarding role on the team were missing for four (2.4%) individuals.

Measures

Participants completed a demographic form that included their role on the STS team and the school in which they provided services. Data collected about participants were limited so as to obtain a waiver of the required elements of consent from the institutional review board (IRB). Participants also completed two measures of teamwork, described below. These measures have been used in peer-reviewed studies in the broader healthcare literature (Baker, Amodeo, Krokos, Slonim, & Herrera, 2010; Keebler, 2014).

The TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ; Agency for Healthcare Research and Quality, 2014b; American Institutes for Research, 2010) is a self-report measure of individual perceptions of group-level team skills and behavior. Each of the five key TeamSTEPPS dimensions (i.e., team structure, leadership, communication, mutual support, and situation monitoring) is represented by seven questions, totaling 35 items. Items are rated on a five-point scale from “strongly agree” to “strongly disagree.” A total score was computed for each dimension, as well as an overall total score summing all 35 items. Cronbach’s alpha ranges from .88 to .95 and convergent validity has been shown to be adequate (American Institutes for Research, 2010).

Items for the T-TPQ were designed to map onto the five TeamSTEPPS dimensions. The initial items were chosen through cognitive interviews with healthcare providers, a small group trial to ascertain item agreement among team members, and a field test with 169 healthcare providers to assess subscale reliabilities and initial validation (American Institutes for Research, 2010). Cronbach’s alpha from this initial validation sample ranged from .88-.95 and intercorrelations between the five dimensions ranged from .57-.79. Preliminary evidence of convergent validity also was demonstrated (American Institutes for Research, 2010). Subsequently, a confirmatory factor analysis of the T-TPQ was conducted with a sample of 1700 healthcare professionals and support staff (Keebler, 2014). Internal consistency for the five teamwork dimensions was excellent (Cronbach’s alpha = .92-.96) and the five-factor model was shown to be a good fit with the data, indicating construct validity. Dimensions were correlated with one another but also showed evidence of independence.

A brief version of the T-TPQ also has been developed and its factor structure, reliability, and validity were evaluated in a sample of 456 nurses (Castner, 2012). Construct validity, discriminant validity (with the constructs of self-esteem and control over practice), and internal consistency (Cronbach’s alpha = .83-.94) were demonstrated and a three-factor solution was found to best fit the Brief T-TPQ data (Castner, 2012). Factors were conceptualized as representing peer, leadership, and bedside relationships.

The TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ; Agency for Healthcare Research and Quality, 2014a; Baker, Krokos, & Amodeo, 2008) is a self-report measure of individual attitudes related to teamwork. Six items measure each of the five core TeamSTEPPS teamwork dimensions, totaling 30 items. Items are rated on a five-point scale from “strongly agree” to “strongly disagree.” Four items are reverse-coded: three mutual support items and one communication item. A total score was computed for each dimension, as well as an overall total score summing all 30 items. Dimensions exhibit unique variance and the Cronbach’s alpha for each construct ranges from .70 to .83 (Baker et al., 2010; Baker et al., 2008).

The T-TAQ’s development followed a similar process to the T-TPQ in that a pool of items were developed to map onto TeamSTEPPS dimensions and refined through expert review and pilot testing (Baker et al., 2010; Baker et al., 2008). Pilot testing was conducted with a sample of 346 military healthcare providers and 149 civilians. Cronbach’s alpha across the five dimensions ranged from .70-.83. Intercorrelations between the five dimensions ranged from .36-.63. Overall teamwork attitudes in this sample were positive and few differences between groups of respondents were evidenced. Both physicians and dentists evidenced significantly lower overall T-TAQ scores compared to nurses, and nurses reported more positive ratings on the leadership dimension compared to physicians. T-TAQ scores have been shown to significantly improve following TeamSTEPPS training in interdisciplinary groups of healthcare professionals and students (Brock et al., 2012; Sawyer et al., 2013) and are associated with nurses’ attitudes about safety (Li, 2013).

Procedures

The IRB approved the study as well as a waiver of the required elements of consent. STS providers were recruited through leadership at their respective agencies. The first author identified participants by contacting agency leaders via phone and/or email and describing the study. Participants were invited to complete the paper-and-pencil TeamSTEPPS measures on one occasion at their agency or school, whichever agency leadership reported was more convenient for the team. The measures took approximately 15 minutes to complete. No identifying information was collected from participants in order to ensure anonymity and minimize the potential for social desirability. As an incentive, teams were offered lunch for completing the measures.

Data Analyses

All analyses were conducted using IBM SPSS Statistics software version 23. Exploratory principal axis factor analyses were conducted to determine the best factor solution for each measure. We used exploratory factor analysis because: (1) the application of TeamSTEPPS and the program’s associated measures to school mental health teams is novel; and (2) it has not been established via exploratory factor analysis that the intended five factor solution is the best fit; previous studies have relied on confirmatory factor analysis. We aimed to identify factor solutions that would meet the following criteria (Fabrigar, Wegener, MacCallum, & Strahan, 1999; Hayton, Allen, & Scarpello, 2004): a) satisfy Cattell’s (Cattell, 1966) scree test; b) retain three or more items per factor with salient loadings (≥ .40); c) yield high internal consistency for the scale when salient items were included (≥ .70); d) retain a maximum number of items while achieving a simple structure; e) produce a Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy > .50; and (f) yield a statistically significant Bartlett’s test of sphericity (p < .001).

Missing data were minimal (<4% per item). Mean imputation was used for descriptive analyses. We calculated the Pearson correlations between T-TPQ and T-TAQ total scores to determine if the instruments were indeed capturing distinct information. Finally, we conducted one-way ANOVA analyses with post-hoc Tukey’s honestly significant difference (HSD) to examine systematic within-agency and staff role differences in T-TPQ and T-TAQ total scores.

Results

Factor Analysis

Separate exploratory principal axis factor analyses were conducted to determine the latent structure of the T-TPQ and T-TAQ. KMO measures of sampling adequacy (0.94 and 0.92, respectively) and Bartlett’s tests of sphericity (4201.89 and 4033.70, p < .001, respectively) were significant for both the T-TPQ and T-TAQ, indicating that there are correlations in the data set that indicate appropriateness for factor analysis. The T-TPQ yielded a unitary factor that included all 35 items. Overall, 48% of the total variance was explained on the T-TPQ by the first factor.

For the T-TAQ, all but four of the 30 items loaded onto a single factor. Forty six percent of the total variance on the T-TAQ was explained by factor 1. A second factor included four cross-loading items assessing task assistance and monitoring of team members emotional and physical statuses, however these items loaded more strongly onto factor 1. Ten percent of the total variance on the T-TAQ was explained by factor 2. The four items that did not load on to factors 1and 2 were the four reverse-scored items, and these four items loaded on to a third factor. Nine percent of the total variance was explained by factor 3. The factor loadings of each item on the T-TPQ and T-TAQ and Cronbach’s alpha are presented in Tables 1 and 2.

Table 1.

Item-level factor analysis of the T-TPQ (N =167)

Items Factor 1
My unit operates at a high level of efficiency. .79
My unit has clearly articulated goals. .77
My supervisor/manager takes time to meet with staff to develop a plan for patient care. .77
My supervisor/manager resolves conflicts successfully. .77
Staff relay relevant information in a timely manner. .77
Staff follow a standardized method of sharing information when handing off patients. .76
Staff share information regarding potential complications (e.g., patient changes, bed availability). .74
When communicating with patients, staff allow enough time for questions. .73
Staff meets to reevaluate patient care goals when aspects of the situation have changed. .72
Staff within my unit share information that enables timely decision making by the direct patient care team. .72
My supervisor/manager ensures that staff are aware of any situations or changes that may affect patient care. .72
Staff exchange relevant information when it becomes available. .71
Staff continuously scan the environment for important information. .71
Staff verbally verify information that they receive from one another. .70
Feedback between staff is delivered in a way that promotes positive interactions and future change. .70
Staff seek information from all available sources. .69
My supervisor/manager models appropriate team behavior. .69
Staff understand their roles and responsibilities. .69
My supervisor/manager considers staff input when making decisions about patient care. .69
My supervisor/manager provides opportunities to discuss the unit’s performance after an event. .69
Information regarding patient care is explained to patients and their families in lay terms. .69
Staff are held accountable for their actions. .69
My unit makes efficient use of resources (e.g., staff supplies, equipment, information). .67
Staff assist fellow staff during high workload. .67
Staff use common terminology when communicating with each other. .67
My supervisor/manager ensures that adequate resources (e.g., staff, supplies, equipment, information) are available. .67
Staff correct each other’s mistakes to ensure that procedures are followed properly. .66
When staff have a concern about patient safety, they challenge others until they are sure the concern has been heard. .66
Staff request assistance from fellow staff when they feel overwhelmed. .63
Staff caution each other about potentially dangerous situations. .62
The skills of staff overlap sufficiently so that work can be shared when necessary. .61
Staff effectively anticipate each other’s needs. .61
Staff resolve their conflicts, even when the conflicts have become personal. .60
Staff advocate for patients even when their opinion conflicts with that of a senior member of the unit. .48
Staff monitor each other’s performance. .47
Alpha (α) .97
Open in a new tab

Table 2.

Item-level factor analysis of the T-TAQ (N =167)

Items Factor 1 Factor 2 Factor 3
Effective leaders view honest mistakes as meaningful learning opportunities. .82 −.32 .03
It is important for leaders to share information with team members. .83 −.39 .08
Monitoring patients provides an important contribution to effective team performance. .79 .25 −.04
It is important to ask patients and families for feedback regarding patient care. .80 −.29 .03
Leaders should create informal opportunities for team members to share information. .80 −.35 .03
Individuals can be taught how to scan the environment for important situational cues. .74 .21 −.03
It is important for leaders to take time to discuss with their team members plans for each patient. .84 −.34 .07
Team leaders should ensure that team members help each other out when necessary. .85 −.40 .07
Patients are a critical component of the care team. .84 −.29 .05
Team members who monitor their emotional and physical status on the job are more effective. .72 .40 .00
To be effective, team members should understand the work of their fellow team members. .74 .28 .02
It is a leader’s responsibility to model appropriate team behavior. .81 −.31 .01
It is important to monitor the emotional and physical status of other team members. .71 .42 −.07
It is appropriate for one team member to offer assistance to another who may be too tired or stressed to perform a task. .72 .40 .06
This facility’s administration influences the success of direct care teams. .72 −.29 −.08
Effective team members can anticipate the needs of other team members. .73 −.26 −.08
I prefer to work with team members who ask questions about information I provide. .52 .37 −.24
Poor communication is the most common cause of reported errors. .63 .26 −.08
It is appropriate to continue to assert a patient safety concern until you are certain that it has been heard. .62 .33 −.08
Teams that do not communicate effectively significantly increase their risk of committing errors. .66 .31 −.09
A team’s mission is of greater value than the goals of individual team members. .61 −.32 −.10
High-performing teams in health care share common characteristics with high-performing teams in other industries. .62 −.28 −.19
Offering to help a fellow team member with his/her individual work tasks is an effective tool for improving team performance. .52 .37 .08
Adverse events may be reduced by maintaining an information exchange with patients and their families. .67 .31 −.02
Even individuals who are not part of the direct care team should be encouraged to scan for and report changes in patient status. .46 .28 −.16
It is important to have a standardized method for sharing information when handing off patients. .62 .34 −.14
It is nearly impossible to train individuals how to be better communicators. .23 .00 .47
Asking for assistance from a team member is a sign that an individual does not know how to do his/her job effectively. .19 .08 .81
Providing assistance to team members is a sign that an individual does not have enough work to do. .20 .06 .84
Personal conflicts between team members do not affect patient safety. .04 .18 .68
Alpha (α) .95
Open in a new tab

Descriptive Data

Table 3 presents means and standard deviations of T-TPQ and T-TAQ dimensions and total scores. A Pearson correlation was conducted to examine the association between T-TPQ and T-TAQ total scores. Results were not statistically significant: r = −.003, p = .97. One-way ANOVAs comparing T-TPQ total score by staff role, F(4, 158) = 1.90, p = .11, and agency, F(4, 162) = .39, p = .82, were not statistically significant. A one-way ANOVA comparing T-TAQ total score by staff role was not statistically significant: F(4, 158) = 1.73, p = .15. Finally, a one-way ANOVA comparing T-TAQ total score by agency was statistically significant: F(4, 162) = 3.03, p = .02. Post-hoc Tukey’s HSD indicated that one agency had lower T-TAQ total scores compared to the other four agencies (mean difference ranged from 11.85 to 24.18).

Table 3.

Means and Standard Deviations (SD) (N =167)

Mean SD Possible Range
T-TPQ 140.86 20.49 35-175
 Team Structure 28.20 4.98 7-35
 Leadership 29.75 5.15 7-35
 Situation Monitoring 26.88 4.77 7-35
 Mutual Support 27.91 4.65 7-35
 Communication 28.11 4.61 7-35
T-TAQ 122.84 18.53 30-150
 Team Structure 23.92 5.17 6-30
 Leadership 26.08 5.53 6-30
 Situation Monitoring 24.95 4.12 6-30
 Mutual Support 23.94 4.30 6-30
 Communication 24.08 3.92 6-30
Open in a new tab

Note. T-TPQ = TeamSTEPPS Teamwork Perceptions Questionnaire; T-TAQ = TeamSTEPPS Teamwork Attitudes Questionnaire.

Discussion

A unitary factor structure best fit the data for both the T-TPQ and T-TAQ based on responses from school mental health team members. This means that, for school-based team members, the items on the TeamSTEPPS measures reflect a single teamwork construct and do not discriminate among the five TeamSTEPPS dimensions, as has been the case in healthcare settings. Our results were contrary to our hypothesis that the factor structures would align with the five key TeamSTEPPS dimensions of team structure, leadership, communication, mutual support, and situation monitoring. Both the T-TPQ and T-TAQ demonstrated excellent internal consistency and the two measures were not significantly correlated. This suggests that T-TPQ and T-TAQ provide unique information about perceptions of and attitudes toward teamwork, respectively.

We also examined differences in T-TPQ and T-TAQ scores by staff role and employer. Analyses comparing T-TPQ and T-TAQ total score by staff role were not statistically significant. T-TAQ total scores significantly differed by agency such that one agency had lower ratings of attitudes toward teamwork compared to the other agencies. It should be noted that T-TPQ and T-TAQ scores were quite positive overall, consistent with previous research (Baker et al., 2010). This suggests the measures may have a restricted range and this may have limited our ability to identify differences between groups.

The T-TPQ and T-TAQ were developed with the explicit goal of including an equal number of items representing each of the five TeamSTEPPS dimensions. The 35-item T-TPQ, as used in this study, was previously examined via confirmatory factor analysis in a large sample of healthcare providers and those results suggested the five-factor model was a good fit to the data (Keebler, 2014). In another study, however, when the 20-item Brief T-TPQ was examined in a sample of nurses, a three-factor solution in which the factors represented peer, leadership, and bedside relationships was a better fit. To our knowledge, the T-TAQ has not yet been studied using exploratory factor analysis, supporting the need for the present analyses in this sample. Our finding that a one-factor solution best fit the data for both the T-TPQ and T-TAQ suggests two possibilities. First, the factor structure of these measures may differ for school mental health teams than for other types of healthcare providers. Since counselors and other school mental health team members typically have not received training in teamwork (Boat et al., 2016; National Academies of Sciences, Engineering, and Medicine, 2017) and do not work within medical settings these distinct dimensions may be less relevant. Second, the use of a total score for these measures may be more appropriate for all groups than dimensions scores as the measure developers recommend. Future research in other team-based care settings is needed.

For the T-TPQ, all 35 items loaded onto the single factor. In this case a one-factor solution clearly best fit the data. For the T-TAQ, the four reverse-scored items did not load onto the first factor. Based on our a-priori criteria and the fact that these four items did not meaningfully align with one another (other than all being negatively worded), we determined that a one-factor solution was most appropriate for the T-TAQ as well. The fact that only the reverse-scored items did not load with the other items suggests that the four reverse-scored T-TAQ items may have been unclearly worded to the participants in this study.

Several study limitations should be noted. First, the sample size, while adequate (MacCallum, Widaman, Zhang, & Hong, 1999), was small relative to the number of items on the scales. Second, we were unable to examine convergent and divergent validity or retest reliability in our sample. Given the widespread reach of TeamSTEPPS (TeamSTEPPS National Implementation, 2013; Weaver et al., 2010) and importance of assessing readiness (Alexander, 2012) and evaluating outcomes of implementation (Proctor et al., 2011), further research with these measures is needed. For example, exploratory factor analyses in healthcare other samples are needed to clarify whether a five-factor or unitary factor structure is most appropriate.

Despite these limitations, there are important implications related to these findings. The value of interdisciplinary collaboration between school-employed personnel and community-employed mental health providers has been well described (Mellin et al., 2010; Weist et al., 2012). However, despite the vast research on teamwork in other domains of healthcare (e.g., Manser, 2009) and the evidence that team culture impacts efficiency and patient outcomes (Sacks et al., 2015), little research has explored teamwork within the counseling profession or in schools. The TeamSTEPPS measures are a promising tool in this regard. The present study adds to the limited literature on the psychometrics of TeamSTEPPS measures and extends the scope of TeamSTEPPS to school mental health teams and counselors for the first time, contributing to its utility and importance. A unitary factor structure was the best fit to the data for both measures in this sample. This underscores the importance of examining the psychometric properties of measures that were developed for other types of healthcare providers with school mental health providers before their use. All teams in healthcare are not created equal. Some of the TeamSTEPPS skills may not fit all teams all the time. Future research should examine which TeamSTEPPS skills best fit the school mental health context.

An extensive body of research on conducting and evaluating team training exists in healthcare, and existing approaches are effective and minimally burdensome (Gregory et al., 2013; Hughes et al., 2016; Salas & Rosen, 2013). The potential for extending these existing tools and approaches to mental health teams and teams of professionals in schools is exciting as it could provide a relatively cost-effective strategy for improving services in a variety of team-based care settings and bolstering existing evidence-based practice implementation efforts. Future research also should explore modifications to the TeamSTEPPS measures to more fully align them with school and mental health contexts. For example, substituting the word ‘student’ or ‘child’ for the word ‘patient’ may be preferable. For practicing counselors working in team settings, the TeamSTEPPS measures may be useful resources for understanding the attitudes and perceptions toward teamwork broadly on their teams. In this way, these measures may provide actionable data for those counselors in leadership roles on teams seeking to improve the quality of care their team provides. School mental health teams are one particular team-based care model. However, teams operate in a variety of health and educational settings and take many forms. This work may be applicable to those contexts as well. Future work that expands TeamSTEPPS to a range of team settings outside of healthcare is needed.

Acknowledgments.

This manuscript was supported by National Institute of Mental Health grants to CBW (F32 MH103955) and JL (K01 MH100199). Dr. Wolk is an investigator with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (5R25MH08091607) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI). Dr. Wolk is an associate fellow and Dr. Mandell and Dr. Cronholm are senior fellows with the Leonard Davis Institute of Health Economics, University of Pennsylvania.

Footnotes

Disclosure of interest. The authors report no conflicts of interest.

Contributor Information

Courtney Benjamin Wolk, Email: cbenja@upenn.edu.

Jill Locke, Email: jjlocke@uw.edu.

Eduardo Salas, Email: Eduardo.Salas@rice.edu.

Ricardo Eiraldi, Email: eiraldi@pennmedicine.upenn.edu.

Peter F. Cronholm, Email: peter.cronholm@uphs.upenn.edu.

David Mandell, Email: mandelld@upenn.edu.

References

  1. Agency for Healthcare Resarch and Quality. (2007). TeamSTEPPS instructor guide. Rockville, MD: USUHS Publication [Google Scholar]
  2. Agency for Healthcare Resarch and Quality (2014a). Teamwork Attitudes Questionnaire (T-TAQ). Retrieved from http://www.ahrq.gov/professionals/education/curriculum-tools/teamstepps/instructor/reference/teamattitude.html
  3. Agency for Healthcare Resarch and Quality. (2014b). Teamwork Perceptions Questionnaire (T-TPQ). Retrieved from http://www.ahrq.gov/professionals/education/curriculum-tools/teamstepps/instructor/reference/teampercept.html
  4. Alexander JA, & Hearld LR (2012). Methods and metrics challenges of delivery-system research. Implementation Science, 12, 7–15. doi: doi: 10.1186/1748-5908-7-15 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. American Institute for Research. (2010). TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ) Manual. Rockville, MD: Retrieved from http://teamstepps.ahrq.gov/abouttoolsmaterials.htm. [Google Scholar]
  6. Baker DP, Amodeo AM, Krokos KJ, Slonim A, & Herrera H (2010). Assessing teamwork attitudes in healthcare: Development of the TeamSTEPPS teamwork attitudes questionnaire. . Quality and Safety in Healthcare, 19, 1–4. doi: doi: 10.1136/qshc.2009.036129 [DOI] [PubMed] [Google Scholar]
  7. Baker DP, Krokos KJ, & Amodeo AM (2008). TeamSTEPPS Teamwork Attitudes Questionnaire Manual. Rockville, MD: Retrieved from http://teamstepps.ahrq.gov/abouttoolsmaterials.htm. [Google Scholar]
  8. Baker DP, Salas E, King H, Battles J, & Barach P (2005). The role of teamwork in the professional education of physicians: Current status and assessment recommendations. Joint Commission Journal on Quality and Patient Safety, 31, 185–202 [DOI] [PubMed] [Google Scholar]
  9. Beidas RS, & Kendall PC (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Boat TF, Land ML, Leslie LK, Hoagwood KE, Hawkins-Walsh E, McCabe MA, Fraser MW, de Saxe Zerden L, Lombardi BM, Fritz GK, Frogner BK, Hawkins JD, and Sweeney M. (2016). Workforce Development to Enhance the Cognitive, Affective, and Behavioral Health of Children and Youth: Opportunities and Barriers in Child Health Care Training. Discussion Paper, National Academy of Medicine, Washington, DC. https://nam.edu/wp-content/uploads/2016/11/Workforce-Development-to-Enhance-the-Cognitive-Affective-and-Behavioral-Health-of-Children-and-Youth.pdf. [Google Scholar]
  11. Brock D, Abu-Rish E, Chiu C, Hammer D, Wilson S, Vorvick L, & …Zierler B (2012). Interprofessional education in team communication: working together to improve patient safety. BMJ Quality and Safety, 22, 414–423 [DOI] [PubMed] [Google Scholar]
  12. Castner J (2012). Validity and reliability of the Brief TeamSTEPPS Teamwork Perceptions Questionnaire. Journal of Nursing Measurement 20(3), 186–198 [DOI] [PubMed] [Google Scholar]
  13. Cattell RB (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245–276 [DOI] [PubMed] [Google Scholar]
  14. Committee on School Health. (2004). School-Based Mental Health Services. Pediatrics 113(6), 1839–1845 [DOI] [PubMed] [Google Scholar]
  15. Creed TA, Frankel SA, German RE, Green KL, Jager-Hymen S, Taylor KP, … Beck AT (2016). Implementation of transdiagnostic Cognitive Therapy in community behavioral health: The Beck Community Initiative. Journal of Consulting and Clinical Psychology, 84(12), 1116–1126 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Creed TA, Jager-Hyman S, Pontoski K, Feinberg B, Rosenberg Z, Evans A, & …Beck AT (2013). The Beck Initiative: Training school-based mental health staff in Cognitive Therapy. The International Journal of Emotional Education, 5, 49–66 [Google Scholar]
  17. Deering S, Rosen MA, Ludi V, Munroe M, Pocrnich A, Laky C, & Napolitano PG (2011). On the front lines of patient safety: Implementation and evaluation of team training in Iraq. The Joint Commission Journal on Quality and Patient Safety, 37(8), 350–356 [DOI] [PubMed] [Google Scholar]
  18. Eiraldi R, Wolk CB, Locke J, & Beidas R (2015). Clearing Hurdles: The Challenges of Implementation of Mental Health Evidence-Based Practices in Under-resourced Schools. Advances in School Mental Health Promotion, 8(3), 124–145. 10.1080/1754730X.2015.1037848 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Fabrigar LR, Wegener DT, MacCallum RC, & Strahan EJ (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 3, 272–299 [Google Scholar]
  20. Farahmand FK, Grant KE, Polo AJ, Duffy SN, & DuBois DL (2011). School-based mental health and behavioral programs for low-income, urban youth: A systematic and meta-analytic review. Clinical Psychology Science and Practice, 18, 372–390. doi: doi: 10.1111/j.1468-2850.2011.01265.x [DOI] [Google Scholar]
  21. Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O’Brien MA, Wolf F, Oxman AD (2009). Continuing education meetings and workshops: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews. doi: 10.1002/14651858.CD003030.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Gregory ME, Feitosa J, Driskell T, Salas E, & Vessey WB (2013). Designing, Delivering, and Evaluating Team Training in Organizations: Principles that Work. In Salas E, Tannenbum S, Cohen D & Latham G (Eds.), Developing and Enhancing Teamwork in Organizations: Evidence-based practice guidelines. San Francisco, CA: John Wiley & Sons [Google Scholar]
  23. Hayton JC, Allen DG, & Scarpello V (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7, 191–205 [Google Scholar]
  24. Herschell AD, Kolko DJ, Baumann BL., & Davis AC (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30, 448–466. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hughes AM, Gregory ME, Joseph DL, Sonesh SC, Marlow SL, Lacerenza CN, … Salas E (2016). Saving lives: A meta-analysis of team training in healthcare. Journal of Applied Psychology, 101(9), 1266–1304 [DOI] [PubMed] [Google Scholar]
  26. Keebler JR, Dietz AS, Lazzara EH, Benishek LE, Almeida SA, Toor PA, Kind HB, & Salas E (2014). Validation of a teamwork perceptions measure to increase patient safety. BMJ Quality and Safety, 23(9), 718–726 [DOI] [PubMed] [Google Scholar]
  27. King HB, Battles J, Baker DP, Alonso A, Salas E, Webster J, & …Salisbury M (2008). TeamSTEPPS: Team strategies and tools to enhance performance and patient saftey. In Henriksen K, Battles JB, Keyes MA & Grady ML (Eds.), Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 3: Performance and Tools). Rockville, MD: Agency for Healthcare Research and Quality (US). [PubMed] [Google Scholar]
  28. Kutash K, Duchnowski AJ, & Lynn N (2006). School-based mental health: An empirical guide for decision-makers. Tampa, FL: University of South Florida, The Louis de la Parte Florida Mental Health Institute, Department of Child & Family Studies, Research and Training Center for Children’s Mental Health. [Google Scholar]
  29. Li A (2013). Teamwork climate and patient safety attitudes: Associations among nurses and comparison with physicians in Taiwan. Journal of Nursing Care Quality, 28(1), 60–67 [DOI] [PubMed] [Google Scholar]
  30. MacCallum RC, Widaman KF, Zhang S, & Hong S (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84–99 [Google Scholar]
  31. Mahoney JS, Ellis TE, Garland G, Palyo N, & Greene PK (2012). Supporting a Psychiatric Hospital Culture of Safety. Journal of the American Psychiatric Nurses Association 18(5), 299–306 [DOI] [PubMed] [Google Scholar]
  32. Manser T (2009). Teamwork and patient safety in dynamic domains of healthcare: A review of the literature. Acta Anastesiologica Scandinavica, 53, 143–151 [DOI] [PubMed] [Google Scholar]
  33. Mayer CM, Cluff L, Lin WT, Willis TS, Stafford RE, Williams C, & …Amoozegar JB (2011). Evaluating efforts to optimize TeamSTEPPS implementation in surgical and pediatric intensive care units. The Joint Commission Journal on Quality and Patient Safety, 37(8), 365–374 [DOI] [PubMed] [Google Scholar]
  34. Mellin EA (2009). Unpacking interdisciplinary collaboration in expanded school mental health: A conceptual model for developing the evidence base. Advances in School Mental Health Promotion, 2, 4–14. doi: doi: 10.1080/1754730X.2009.9715706 [DOI] [Google Scholar]
  35. Mellin EA, Bronstein L, Anderson-Butcher D, Amorose AJ, Ball A, & Green J (2010). Measuring interprofessional team collaboration in expanded school mental health: Model refinement and scale development. Journal of Interprofessional Care, 24, 514–523. doi: doi: 10.3109/13561821003624622 [DOI] [PubMed] [Google Scholar]
  36. Mellin EA, Bronstein L, Anderson-Butcher D, Amorose AJ, Ball A, & Green J (2010). Measuring interprofessional team collaboration in expanded school mental health: Model refinement and scale development. Journal of Interprofessional Care, 24(5), 514–523 [DOI] [PubMed] [Google Scholar]
  37. National Academies of Sciences, Engineering, and Medicine. (2017). Training the Future Child Health Care Workforce to Improve Behavioral Health Outcomes for Children, Youth, and Families: Proceedings of a Workshop-in Brief. Washington, DC: The National Academies Press. doi: 10.17226/24789. [DOI] [PubMed] [Google Scholar]
  38. Neily J, Mills PD, Young-Xu Y, Carney BT, West P, Berger DH, & …Bagian JP (2010). Association between implementation of a medical team training program and surgical mortality. Journal of the American Medical Association, 304(15), 1693–1700 [DOI] [PubMed] [Google Scholar]
  39. New Freedom Commission on Mental Health. Achieving the Promise: Transforming Mental Health Care in America. Final Report 2003: Rockville, MD [Google Scholar]
  40. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A… & Hensley M (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38, 65–76 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Sacks GD, Shannon EM, Dawes AJ, Rollo JC, Nguyen DK, Russell MM, … Maggard-Gibbons MA (2015). Teamwork, communication, and safety climate: a systematic review of interventions to improve surgical culture. BMJ Quality and Safety, 24, 458–467 [DOI] [PubMed] [Google Scholar]
  42. Salas E, & Rosen MA (2013). Building high reliability teams: Progress and some reflections on team training. Quality and Safety in Healthcare, 22, 369–373 [DOI] [PubMed] [Google Scholar]
  43. Sanfey H, McDowell C, Meier AH, & Dunnington GL (2011). Team training for surgical trainees. The Surgeon, Journal of the Royal Colleges of Surgeons of Edinburgh and Ireland, 9 Suppl(1), S32–S34 [DOI] [PubMed] [Google Scholar]
  44. Sawyer T, Laubach VA, Hudak J, Yamamura K, & Pocrnich A (2013). Improvements in teamwork during neonatal resuscitation after interprofessional TeamSTEPPS training. Neonatal Network, 32(1), 26–33 [DOI] [PubMed] [Google Scholar]
  45. Schmutz J, & Manser T (2013). Do team processes really have an effect on clinical performance? A systematic literature review. British Journal of Anesthesia, 110(4), 529–544 [DOI] [PubMed] [Google Scholar]
  46. Sheppard F, Williams M, & Klein VR (2013). TeamSTEPPS and patient safety in healthcare. American Society for Healthcare Risk Management 32(3), 5–10 [DOI] [PubMed] [Google Scholar]
  47. Stark KD, Arora P, & Funk CL (2011). Training in school psychologists to conduct evidence-based treatments for depression. Psychology in the Schools, 48(3), 272–282 [Google Scholar]
  48. Stead K, Kumar S, Schultz TJ, Tiver S, Pirone CJ, Adams RJ, & Wareham CA (2009). Teams communicating through STEPPS. The Medical Journal of Australia S11, S129–132 [DOI] [PubMed] [Google Scholar]
  49. TeamSTEPPS National Implementation. http://www.teamsteppsportal.org/. Accessed July 14 2013.
  50. Turner P (2012). Implementation of TeamSTEPPS in the emergency department. Crtical Care Nursing Quarterly 35(3), 208–212 [DOI] [PubMed] [Google Scholar]
  51. U.S. Public Health Service. Report of the Surgeon General’s Conference on Children’s Mental Health: A National Action Agenda. Department of Health and Human Services, Editor. 2000: Washington, DC: [PubMed] [Google Scholar]
  52. Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, & Bonomi A (2001). Improving chronic illness care: Translating evidence into action. Health Affairs 20(6), 64–78 [DOI] [PubMed] [Google Scholar]
  53. Weaver SJ, Rosen MA, DiazGranados D, Lazzara EH, Lyons R, Salas E, … King HB (2010). Does teamwork improve performance in the operating room? A multilevel evaluation. The Joint Commission Journal on Quality and Patient Safety, 36(3), 133–142 [DOI] [PubMed] [Google Scholar]
  54. Weist MD, Mellin EA, Chambers KL, Lever NA, Haber D, & Blaber C (2012). Challenges to collaboration in school mental health and strategies for overcoming them. Journal of School Health, 82, 97–105 [DOI] [PubMed] [Google Scholar]

RESOURCES