Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 May 1.
Published in final edited form as: Prev Sci. 2016 May;17(4):439–449. doi: 10.1007/s11121-016-0633-8

Understanding and Measuring Coach–Teacher Alliance: A Glimpse Inside the ‘Black Box’

Stacy R Johnson 1,✉, Elise T Pas 1, Catherine P Bradshaw 1,2
PMCID: PMC4840105  NIHMSID: NIHMS762885  PMID: 26872479

Abstract

Coaching models are increasingly used in schools to enhance fidelity and effectiveness of evidence-based interventions; yet, little is known about the relationship between the coach and teacher (i.e., coach–teacher alliance), which may indirectly enhance teacher and student outcomes through improved implementation quality. There is also limited research on measures of coach–teacher alliance, further hindering the field from understanding the active components for successful coaching. The current study examined the factor structure and psychometric characteristics of a measure of coach–teacher alliance as reported by both teachers and coaches and explored the extent to which teachers and coaches reliably rate their alliance. Data come from a sample of 147 teachers who received implementation support from one of four coaches; both the teacher and the coach completed an alliance questionnaire. Separate confirmatory factor analyses for each informant revealed four factors (relationship, process, investment, and perceived benefits) as well as an additional coach-rated factor (perceived teacher barriers). A series of analyses, including cross-rater correlations, intraclass correlation coefficients, and Kuder-Richardson reliability estimates suggested that teachers and coaches provide reliable, though not redundant, information about the alliance. Implications for future research and the utilization of the parallel coach–teacher alliance measures to increase the effectiveness of coaching are discussed.

Keywords: Teachers, Coaching, Alliance, Implementation of evidence-based interventions

Coaching is increasingly used as a professional development strategy to strengthen the fidelity and effectiveness of evidence-based interventions in schools (Domitrovich et al. 2010; Pas et al. 2014). Domitrovich et al. (2008a) conceptualized coaching as an implementation support, for which the quality, fidelity, and dosage of coaching delivery may vary. However, coaching as an implementation support has received limited attention within the literature, and thus little is known about variables affecting the success of coaching (Pas et al. 2014). As a result, coaching could be viewed as a ‘black box’, implying questions remain regarding underlying components and best practices (Nadeem et al. 2013; Pas et al. 2014). This highlights the need for measuring the coaching process to improve implementation and targeted outcomes. Coach–teacher alliance is one such attribute for which there is emerging evidence of an association with improved implementation (Wehby et al. 2012). The current study examined the psychometric properties and factor structure of parallel coach–teacher alliance measures, as completed by both teachers and coaches. This line of research aims to fill several gaps in the coaching literature regarding measurement and various dimensions of the coaching relationship, which in turn may inform our understanding of the coaching process and coaching as an implementation support.

Defining Coach–Teacher Alliance

Alliance is well documented as a crucial process variable in the therapeutic context (Horvath and Symonds 1991; Martin et al. 2000), consistently demonstrating a significant relationship with client behavior change (Ackerman and Hilsenroth 2003; Horvath and Greenberg 1989). Therapeutic alliance has been typically defined as the therapist–client relational bonding and agreement on goals and tasks (Bordin 1979). Considerably less literature has focused on the formation of alliance between teachers and support providers in schools, particularly within the context of implementation support. Similar to the relationship between a clinician and a patient, a coach and teacher develop a relational bond and need to agree upon the coaching goals and tasks. With regard to the alliance, however, coach–teacher alliance may include additional dimensions as shown in other fields of coaching (Hart et al. 2001) as well as in school-based implementation research (e.g., Han and Weiss 2005).

Dimensions of Coach–Teacher Alliance

An identified dimension of alliance across multiple fields is the relationship quality between the two stakeholders. This is of particular importance for collaborative coaching models, as opposed to traditional expert models of professional development (Burbank and Kauchak 2003), as the first step is relationship building (Frank and Kratochwill 2014). This relationship building recognizes that teachers who feel understood and have confidence and trust in the coach will be more successful (Allen and Graden 2002; Frank and Kratochwill 2014). Consultation research further emphasizes this in the focus on language-based processes (e.g., using the ‘we’ pronoun) to specifically encourage collaboration (Newman et al. 2015). In using such language, a coach’s willingness to work with the teacher and promote an authentic shared ownership is expressed and relates to improved teacher skills (Newman et al. 2015). Similarly, therapeutic research shows that therapists perceived as trustworthy, warm, interested, flexible, and honest form more positive alliances (Ackerman and Hilsenroth 2003).

Collaboration is not only reflected in the relationship quality but also underlies the agreed-upon tasks within the coaching implementation process. Traditional professional development is typically passive and lacks personalization to individual needs, and consequently, these models are often found to be ineffective in producing positive teacher change (Burbank and Kauchak 2003; Hamre et al. 2012). Collaborative coaching seeks to include the teacher in personalized goal setting and action planning, facilitating a sense of ownership and acceptance, and ultimately adoption of the skills. Similarly, in the therapeutic context, alliance is partially defined as agreement on the tasks needed to accomplish specific goals (Horvath and Greenberg 1989). Therefore, aligning with the available literature, there is likely value in considering the perceived implementation of the coaching process as a dimension of coach–teacher alliance.

A third alliance dimension is the teacher’s investment in working with the coach, which reflects their support of, cooperation with, and involvement in an intervention (Power et al. 2009). This dimension is important to consider, as the relationship and coaching process may not be sufficient in producing valuable results without teacher investment. Both levels of investment and exposure have been included as indicators for intervention fidelity and have been shown to predict teachers’ use of targeted skills (Reinke et al. 2013; Wanless et al. 2014). Greater teacher investment, as well as coaches’ perceptions of teachers’ investment, likely lead to increased dosage of coaching and, thus, improved program outcomes.

Perceptions of the benefits of an intervention have also been shown to relate to treatment success and integrity (Cowan and Sheridan 2003; Eckert and Hintze 2000; Elliott 1988). Perceived effectiveness reflects teachers’ beliefs that targeted behaviors and skills have moved in the desired direction and are often related to implementer perceptions of treatment acceptability (Von Brock and Elliott 1987). Positive attributions about a program’s effectiveness importantly related to motivation to continue applying skills and engaging in an intervention (Datnow and Castellano 2000; Han and Weiss 2005). It is possible that coaching models that incorporate performance feedback emphasizing positive gains may be related to perceived intervention gains (Han and Weiss 2005; Rose and Church 1998).

Finally, it may be relevant to consider potential barriers, which may hinder alliance formation and the success of coaching. Barriers to teachers’ implementation of interventions and engagement with professional development can range from burnout to organizational health issues and to personal mental health difficulties. Reasons for burnout are varied and include work-related problems, as well as difficulties in private life and mental health (Huberman 1993; Maslach et al. 2001). Such problems that may lead to burnout also are negatively associated with intervention implementation and may impede alliance (Domitrovich et al. 2015; Han and Weiss 2005; Ross et al. 2011; Wehby et al. 2012).

Measuring Coach–Teacher Alliance from Both Perspectives

There has been limited empirical work on how best to conceptualize, define, and thus measure alliance across such varied dimensions within school-based coaching. The availability of a comprehensive tool that is valid and reliable can inform the development, refinement, and evaluation of coaching models to support interventions. This work may also advance our understanding of the core dimensions of alliance that lead to improved or new teacher practices.

Given the transactional process that exists within coaching and the importance of collaboration and personalization in successful coaching models, it is advantageous to consider both teachers’ and coaches’ perspectives. Traditionally, research examining the behavioral ratings of teachers, parents, and students has shown low to moderate correlations between raters (Renk and Phares 2004; Wright and Torrey 2001). Similarly, teacher and coach ratings may also be only moderately correlated, indicating the possibility that the two raters apply different standards and conceptualizations about alliance (Renk and Phares 2004). It may be that one rater has better predictive validity for intervention outcomes or that each perspective contributes unique information. For example, coach ratings of the coaching process may operate akin to a dosage checklist, and teacher ratings may provide more information about quality that can, in turn, be used to provide feedback to coaches. Measurement development and research using the ratings of both teachers and coaches are needed to examine these possibilities. The inclusion of both raters may provide a more comprehensive and practical assessment of coach–teacher alliance.

Current Study

The current study aimed to advance our understanding of the core dimensions of coach–teacher alliance through the analysis of data from both coaches and teachers. These data came from a set of trials testing a culturally-responsive classroom management and student engagement preventive framework called Double Check (Bottiani et al. 2012; Bradshaw and Rosenberg 2016; Hershfeldt et al. 2009). To facilitate teachers’ implementation of positive behavior supports and cultural responsivity, Double Check used a previously-developed and tested coaching model called the Classroom Check-Up (Reinke et al. 2008). The Classroom Check-Up is a collaborative coaching model that uses motivational interviewing, data-informed coaching practices, and ongoing performance feedback to facilitate new teacher practices. Specifically, in the current study, we analyzed data from two trials of the Double Check framework, through which data were collected about the coach–teacher alliance via a self-reported measure adapted from Wehby et al. (2012) and other prior instruments (e.g., Domitrovich et al. 2008b, c). In this study, we examined the factor structure and psychometric characteristics of the two coach–teacher alliance measures, using confirmatory factor analysis and other related approaches to assess inter-rater reliability. Based on previous research on the variables related to alliance and successful coaching (Pas et al. 2014; Wehby et al. 2012), we sought to confirm the following four factors: working relationship, competent implementation of the coaching process, teacher investment in the process, and the benefits of coaching. We also anticipated a fifth factor in the coach-reported measure pertaining to barriers to coaching. Together, these analyses were intended to inform our understanding of both the measurement of alliance as well as its various dimensions, which are important considerations for the field of implementation science.

Method

Study Design and Procedures

Data for this study were collected as a part of a series of two trials of the Double Check framework (Bottiani et al. 2012; Bradshaw and Rosenberg 2016; Hershfeldt et al. 2009), in which teachers received coaching using the Classroom Check-Up (Reinke et al. 2008) to support their implementation of positive behavioral classroom management and culturally-responsive teaching practices. In addition to coaching, Double Check included five 1-hour professional development sessions provided by the coach to all teachers within a school. Coaches tracked all contacts and activities using an electronic log, which included detailed documentation that all five professional development sessions were delivered as intended across all schools. The alliance data were consistently collected across both of these consecutive trials of Double Check, the first of which used a quasi-experimental design (52 teachers in six schools, across three elementary and three middle schools) in which teachers provided pre-post data in reference to non-randomized comparison teachers. The second trial used a randomized controlled design (100 coached teachers in 12 schools, across six elementary and six middle schools) in which a random assignment process was used to assign teachers to receive coaching (vs. no coaching). As noted above, coaches tracked all contacts and activities using an electronic log, which included detailed documentation that each component of the coaching process was implemented with each teacher. None of the non-coached comparison/control teachers are included in this study, as they provided no data on coaching alliance. The Double Check framework, including coaching, was implemented consistently across both trials (only the use of random assignment was added in the second trial), and thus the data were pooled in the current study. The study took place in a large east coast public school district. Teacher participation in the intervention and data collection was voluntary and consent was provided. The Institutional Review Boards at John Hopkins University and the University of Virginia approved this study.

Participants

Coaching was completed with 147 teachers in grades K-8, of which 52 were in the first trial and 95 were in the second trial (i.e., of the 100 teachers in the second trial, five teachers were never coached, for reasons including leaving the teaching field, extended health-related leave); 37 % taught in elementary schools. The vast majority of teachers were female (i.e., 85 %) and white (i.e., 80 %), and 36 % were 30 years old or younger. Approximately 27 % were within the first 3 years in their role as teacher. Of the 147 teachers who were coached, 20 did not submit the alliance measure, resulting in teacher perception data from 127 teachers in 18 schools. Chi-square tests indicate no differences in trial, school level, gender, age, race, or career stage between teachers who did and did not submit the alliance measure. Coach-reported data were available for all 147 coached teachers. Two coaches provided support to the teachers in the first trial; the same two and two additional coaches provided support to the teachers in the second trial. All coaches were women; two were white and two were black. Two coaches had a master degree in education and the other two had a doctorate in school psychology. All coaches had at least two years of prior coaching experience and received training for the current project, including didactic instruction, role play, and viewing of videos, as well as ongoing supervision.

Overview of the Coaching Intervention

As noted above, Double Check implementation was supported by the Classroom Check-Up coaching model (Reinke et al. 2008), which incorporates motivational interviewing to promote and sustain effective classroom management practices (Reinke et al. 2011). Each coach served all teachers in three schools each year. The coaching process includes (a) an assessment of the teacher’s classroom practices through an interview with the teacher, direct observations in the classroom, and a classroom ecology checklist completed by the teacher; (b) a feedback session using information gathered during the assessment step; (c) the collaborative development of a menu of options for intervention; (d) the dyad selecting a target skill/outcome and creating an action plan; and (e) ongoing support and performance feedback from the coach to ensure progress toward the selected goal. On average, coaches completed 3.52 hours of face-to-face coaching with teachers and dedicated 8.61 hours to each coaching case.

Measures

Teacher Demographics

Teachers provided information regarding their demographic characteristics, including gender, age, race, and years teaching at the start of the school year.

Coach-Teacher Alliance Measures

At the conclusion of the coaching process (i.e., spring of the school year), both the teacher and coach completed parallel versions of a coach–teacher alliance measure called the Measure of Coach and Teacher Alliance–Teacher Report (Bradshaw et al. 2009a) and the Measure of Coach and Teacher Alliance–Coach Report (Bradshaw et al. 2009b), respectively. Both versions included 30 items, which assessed the same dimensions of the coach–teacher alliance from each person’s perspective. The coach version included an additional 12 items, most of which pertained to the coach’s perception of teacher barriers and emotional response to the process, discussed below. All items were rated on a 5-point Likert scale ranging from Never to Always. The core content of the measures was jointly developed and based in part on prior work by Domitrovich et al. (2008b, c) and Wehby et al. (2012). The Wehby et al. (2012) scale was a teacher self-report measure that assessed teachers’ perceptions of teacher–coach relationship quality and also included items assessing teacher perceptions of effectiveness, fit, and burden of an intervention (referred to by Wehby et al. as social validity; Cronbach’s alpha = 0.94). The Double Check project investigators, with input from key consultants (i.e., Domitrovich, Reinke, Herman, and Poduska), adapted the Wehby et al. measure by adding items and creating a parallel coach-completed measure; this expanded set of measures had not been previously psychometrically examined. The items were hypothesized to assess four areas of coach–teacher alliance: working relationship, competent implementation of the coaching process, teacher investment in the process, and the benefits of coaching. As noted earlier, there were 12 additional items included on the coach rating form, which were selected based on prior literature (e.g., Becker et al. 2013; Domitrovich et al. 2015; Rhodes and Beneicke 2002) and focus interviews, through which coaches described challenges (e.g., strong emotions; problems with teachers’ supervisor) that may impede coaching success. See Table 1 for a listing of items on the measures.

Table 1.

Item loadings by alliance factor for teacher and coach ratings CFA

Teacher perspective
Coach perspective
Factor Loading SE R2 Factor Loading SE R2
Working relationship Working relationship
 The coach and I agreed on what the most important goals for intervention were. 0.719 0.047 0.517  The teacher and I agreed on what the most important goals for intervention were. 0.866 0.028 0.750
 The coach and I trust one another. 0.840 0.061 0.705  The teacher and I trust one another. 0.906 0.017 0.821
 The coach was approachable. 0.882 0.047 0.779  The teacher was approachable. 0.879 0.025 0.772
 The coach and I worked together collaboratively. 0.903 0.040 0.815  The teacher and I worked together collaboratively. 0.936 0.028 0.877
 Overall, the coach showed a sincere desire to understand and improve my classroom. 0.824 0.072 0.679  Overall, the teacher showed a sincere desire to improve his/her classroom. 0.871 0.023 0.758
 The coach incorporated my views into the services provided. 0.956 0.018 0.915  The services I provided incorporated the teacher’s view. 0.857 0.053 0.735
 The teacher easily shared his/her concerns with me. 0.758 0.039 0.575
 I was able to empower the teacher to try new strategies. 0.824 0.053 0.678
Coaching process Coaching process
 The coach was knowledgeable. 0.844 0.050 0.712  I felt that I was knowledgeable when it came to working with this teacher. 0.544 0.069 0.296
 The coach communicated effectively. 0.867 0.038 0.751  I was able to communicate effectively with this teacher. 0.912 0.034 0.832
 The coach delivered support, recommendations, and technical assistance in a clear and concise manner. 0.933 0.027 0.870  I was able to deliver support, recommendations, and technical assistance in a clear and concise manner to this teacher. 0.782 0.059 0.612
 The coach made suggestions that were appropriate for my classroom culture. 0.957 0.015 0.916  I was able to provide suggestions that were appropriate for this teacher’s classroom culture. 0.906 0.037 0.822
 The coach provided support that matched the needs of me and my classroom. 0.946 0.009 0.894  I was able to provide support that matched the needs of this teacher and his/her classroom. 0.868 0.049 0.753
 I received an appropriate amount of feedback from the coach. 0.925 0.025 0.856  I felt I was able to provide the appropriate amount of feedback to this teacher. 0.860 0.056 0.740
 The coach provided me with practical and useful feedback and strategies. 0.973 0.012 0.946  The feedback I gave the teacher was practical and useful. 0.969 0.030 0.940
 The coach provided helpful information. 0.917 0.029 0.841
 The coach was accessible. 0.769 0.048 0.591
 The teacher agreed with the data presented and the weaknesses highlighted as areas to work on. 0.868 0.060 0.753
Investment Investment
 The time spent working with the coach was effective and productive. 0.919 0.036 0.845  The time spent working with the teacher was effective and productive. 0.913 0.016 0.833
 I had enough time available to participate in the coaching process. 0.769 0.035 0.591  There was enough time available for this teacher to participate in the coaching process. 0.704 0.045 0.496
 The work I did with the coach was important. 0.894 0.036 0.800  The work I did with the teacher was important. 0.638 0.066 0.407
 The coaching took too much of my time. 0.525 0.061 0.276  The coaching took too much time with this teacher. 0.619 0.029 0.384
 I will be able to effectively implement the strategies recommended by the coach in the future. 0.906 0.021 0.821  The teacher will be able to effectively implement the strategies recommended in the future. 0.805 0.039 0.648
 I would recommend the coaching to another teacher. 0.917 0.032 0.842  I think this teacher would recommend the coaching to another teacher. 0.891 0.018 0.793
 My overall reaction to the coaching was positive. 0.951 0.032 0.905  The teacher’s overall reaction to the coaching was positive. 0.833 0.022 0.693
 The teacher was accessible. 0.723 0.047 0.523
 The teacher found the information I provided to be helpful. 0.763 0.040 0.583
 The teacher was able to use the coaching as a way to answer his/her own questions. 0.850 0.026 0.723
Benefits of coaching Benefits of coaching
 My students benefitted from my work with the coach. 0.875 0.024 0.765  The students benefitted from my work with the teacher. 0.945 0.020 0.892
 The coach helped build my capacity to implement evidence-based strategies. 0.889 0.022 0.790  The teacher showed increased capacity to implement evidence-based strategies. 0.868 0.037 0.753
 The coach had a positive impact on my classroom. 0.962 0.025 0.925  The coaching had a positive impact on the teacher’s classroom. 0.952 0.029 0.905
 The coaching increased my knowledge of strategies to promote student engagement. 0.912 0.025 0.831  The coaching increased the teacher’s knowledge of strategies to promote student engagement. 0.809 0.033 0.655
 The coach increased my knowledge of classroom management strategies. 0.789 0.043 0.622  The teacher’s knowledge of classroom management strategies increased as a result of the coaching. 0.832 0.027 0.692
 The coach increased my knowledge of cultural proficiency. 0.801 0.044 0.641  The teacher’s knowledge of cultural proficiency was increased because of the coaching. 0.851 0.029 0.725
Barriers to coaching
 Displayed anger or hostility. 0.866 0.141 0.750
 Displayed sadness or depression. 0.824 0.065 0.679
 Reported a health or mental health problem. 0.626 0.127 0.392
 Reported other personal problem (e.g., financial, family). 0.489 0.141 0.239
 Reported other work related problem (e.g., problems with boss). 0.635 0.104 0.403
 Cried or became emotional during the session(s). 0.340 0.104 0.116
Open in a new tab

Note. CFA confirmatory factor analyses, Loading standardized factor loading, SE standard error

Analyses

Our first aim was to evaluate the factor structure of the coach–teacher alliance for each informant. Using Mplus 7.1, we conducted a separate confirmatory factor analysis (CFA) for each informant’s rating of the coach–teacher alliance using the four a priori dimensions. All alliance items had ordinal response categories and therefore we used weighted least squares mean-and variance-adjusted estimation. Factors were allowed to covary. All analyses accounted for the clustering of teachers within schools using the Huber-White adjustment of the standard errors (Muthén and Muthén 1998–2014); because there were just four coaches assigned to three schools each, clustering by coach was not accounted for. Model fit was assessed utilizing the comparative fit index (CFI), Tucker-Lewis index (TLI), and root mean-square error of approximation (RMSEA). A value of 0.90 or higher on the CFI or TLI is considered acceptable fit (Bentler and Bonett 1980), with values closer to 0.95 considered to indicate a well-fitting model (Hu and Bentler 1999). Values of less than 0.06 on the RMSEA indicate a good fit (Hu and Bentler 1999). While there were no missing data for coach-reported alliance, teacher-reported alliance data were missing for 13.6 % of the sample. All teachers who submitted the alliance measure were included in the analyses and full information maximum likelihood was used to account for any missing data within these teachers (93 % had complete data).

Using each factor’s items, scale scores were calculated by averaging the response value for each item. A Cronbach’s alpha (α) was calculated for each scale. We then conducted descriptive analyses of the dimensions, including examining mean differences for teacher characteristics and coach assignment as well as cross-informant correlations. Lastly, the intraclass correlation coefficients (ICC) and Kuder-Richardson 20 (KR-20) were examined for each scale to assess interrater reliability between coach and teacher ratings.

Results

Examining Factor Structure

CFAs were performed to evaluate the four (teacher) and five (coach) factor solutions developed a priori for the perceptions of alliance. Table 1 presents the items included for each factor. On the teacher form, two items were excluded, whereas four items were excluded on the coach form; these items were excluded because of poor conceptual and statistical fit with the model (e.g., item about need for additional follow-up and the process being stressful). For the teacher report on coach–teacher alliance, the CFA confirmed the four factors, with all fit indices indicating a well-fitting model (CFI = 0.994, TLI = 0.993, and RMSEA = 0.035). All factor loadings were above 0.50 (see Table 1). The working relationship and benefits of coaching factors each had six items; there were nine items on the competent process factor; and seven items on the investment factor. Cronbach’s α= 0.84 for relationship, 0.89 for benefits, 0.94 for process, and 0.87 for investment.

For the coach report on coach–teacher alliance, the CFA confirmed the five factors, with all fit indices indicating a well-fitting model (CFI = 0.969, TLI = 0.967, and RMSEA = 0.046). All factor loadings were again above 0.50. The working relationship and competent process factors each included eight items; the benefits of coaching and barriers to coaching factors each had six items; and ten items were on the investment factor. Cronbach’s α= 0.88 for relationship, 0.84 for process, 0.89 for investment, 0.92 for benefits, and 0.67 for barriers to coaching.

Descriptive Analyses

Descriptive and correlational data on the final scales are presented in Table 2. Both teachers and coaches, on average, rated the four overlapping alliance scales positively and in the often to always range (i.e., teacher Ms = 3.40 to 3.81; coach Ms = 3.00 to 3.70). Additionally, coaches rated that barriers were present never to seldom (i.e., M = 0.43; range = 0 to 2.67). All of the scales rated by teachers were highly correlated with one another (rs = 0.64 to 0.87). Likewise, all of the coach scales were highly correlated with another (rs = 0.64 to 0.86), with the exception of the barriers to coaching scale. The barriers scale was only weakly correlated scale with the other scales (rs = −0.26 to −0.35), such that more frequently observed barriers were related to less positive views of the other dimensions.

Table 2.

Correlations among the teacher and coach alliance factors

Teacher factors M (SD) M Range by coach Relationship Process Investment Benefits
 Relationship 3.81 (0.36) 3.75–3.88 (0.84)
 Process 3.69 (0.47) 3.59–3.81 0.80** (0.94)
 Investment 3.45 (0.57) 3.23–3.64 0.64** 0.81** (0.87)
 Benefits 3.40 (0.61) 3.33–3.54 0.69** 0.87** 0.77** (0.89)
Coach factors M (SD) M Range by coach Relationship Process Investment Benefits Barriers
 Relationship 3.57 (0.54) 3.06–3.74 (0.88)
 Process 3.70 (0.40) 3.45–3.76 0.64** (0.84)
 Investment 3.37 (0.59) 2.77–3.49 0.86** 0.75** (0.89)
 Benefits 3.00 (0.79) 2.16–3.25 0.78** 0.65** 0.85** (0.92)
 Barriers to coaching 0.43 (0.51) 0.21–0.62 −0.26** −0.35** −0.28** −0.27** (0.67)
Cross-informant correlations Coach rating
Teacher rating Relationship Process Investment Benefits Barriers
 Relationship 0.41** 0.24** 0.36** 0.28** −0.24**
 Process 0.42** 0.20* 0.39** 0.36** −0.21**
 Investment 0.39** 0.18* 0.39** 0.35** −0.18
 Benefits 0.32** 0.15 0.33** 0.34** −0.14
Open in a new tab

Note. Coach-Teacher Alliance coded on a scale of 0 (never) to 4 (always). Values in parentheses across the diagonal are coefficients of internal consistency (Cronbach’s alpha [α]) for each subscale).

*

p < .05,

**

p < .01

Independent samples t tests evaluated whether perceptions on the various alliance scales were different based on the school level (elementary vs. middle) as well as teacher characteristics. Middle school teachers perceived fewer benefits to coaching as compared to elementary school teachers (Ms = 3.29 and 3.58, respectively, t(125) = 2.68, p < 0.01) and reported lower levels of investment (Ms = 3.34 and 3.63, respectively, t(125) = 2.84, p < 0.01). Coaches did not report differences for middle versus elementary school teachers, but did perceive significantly fewer barriers to coaching for younger teachers as compared to older teachers (M = 0.29 versus 0.50, t(139) = 2.47, p < 0.05). No differences were found for trial, gender, race, or career stage in either rating. Post hoc ANOVAs indicated that teachers did not significantly vary in their ratings based on the coach they worked with; one coach tended to provide more conservative ratings than other coaches (see Table 2).

Inter-Rater Analyses

Cross-informant correlations were in the low to moderate range (i.e., rs = −0.18 to 0.42). Examining the correlations between the informants on the same dimension, coaches and teachers displayed moderate agreement across the four overlapping dimensions (relationship, r = 0.41; process, r = 0.20; investment, r = 0.39; benefits, r = 0.34). When either the teacher or coach rated the working relationship more positively, the other rater similarly rated the process as more competent (rs = 0.24 and 0.42), perceived greater investment (rs = 0.36 and 0.39), and saw greater benefits (rs = 0.28 and 0.32). When teachers viewed the process as more competently implemented and said they were more invested, coaches also viewed the process, investment, and benefits more positively (see Table 2). Coaches reported fewer barriers for teachers who rated the relationship and the process more positively (rs = −0.24 and −0.21, respectively).

A series of analyses further examined rater differences. Paired t tests revealed that teachers viewed the coaching relationship more positively than coaches, t(126) = −4.35, p < 0.01, and teachers perceived greater benefits than the coaches, t(125) = −4.64, p < 0.01. Both ICCs and KR-20 coefficients were calculated for each of the four overlapping dimensions. ICCs examined the similarity between how the coaches and teachers perceived the dimensions of alliance, providing insight into the degree to which coaches and teachers provided consistent and reliable information. The ICCs were weak to moderate (i.e., relationship = 0.38; process = 0.20; investment = 0.30; and benefits = 0.33). This indicated that there was moderate overlap in the reports provided by the raters. The KR-20 (Kuder and Richardson 1937) examined the reliability of items within a dimension, by assessing the extent to which teachers and coaches consistently agreed or disagreed with one another on their item responses. Therefore, a high score is possible when a teacher–coach pair disagreed about the ratings of a scale, as long as they consistently disagreed on each item. All dimensions showed an acceptable to high level of consistency and thus reliability between the raters (i.e., relationship = 0.75; process= 0.85; investment = 0.73; and benefits = 0.66). Thus, the two raters provided unique information about alliance (i.e., per the ICC findings); however, the KR-20 demonstrated that teachers and coaches responded relatively consistently across items on the four overlapping dimensions.

Discussion

There has been relatively limited attention to understanding and measuring the core attributes of coaching (Becker et al. 2013), making it a bit of a ‘black box’. This article aimed to fill gaps in the literature regarding the conceptualization and measurement of the coach–teacher alliance through the perspective of both coaches and teachers. Although there is a rich body of literature focused on the alliance between a therapist and client, there has been limited application of this work to school-based coaching as an implementation support. Our CFA results indicated support for the existence and interrelatedness of the following dimensions: working relationship, competent implementation of the coaching process, teacher investment, benefits of coaching, and barriers to coaching. Specifically, the items assessing the working relationship reflected the quality of the collaborative relationship between the teacher and coach as evidenced by trust, how approachable the coach was, and an understanding of the teacher’s goals and views. The competent process dimension included indicators of the perception that the coaching process was delivered (e.g., providing feedback in a way that was well-received) and tailored (e.g., matching recommendations to the teacher and classroom) appropriately for the specific teacher. These relationship and process dimensions are based on counseling and consultation research, which suggests that the relationship formed and agreement upon the tasks of working together are important components of alliance (Bordin 1979; Martin et al. 2000; Wehby et al. 2012). Investment in the coaching process comprised perceptions of support, cooperation, and involvement in the intervention, demonstrated by prioritizing and engaging with the coaching. Implementation research suggests that investment in coaching may be a necessary counterpart to the relationship and process dimensions of alliance (Power et al. 2009; Reinke et al. 2013). Additionally, the perceived benefits dimension reflected perceptions of change in the desired direction due to the intervention (e.g., increased teacher knowledge and improved student outcomes). The barriers to coaching encompassed several emotional and structural problems that may have impeded coaching. As hypothesized, perceived benefits and barriers emerged as robust scales, which is consistent with prior work on fidelity and effectiveness of coaching to support interventions (e.g., Han and Weiss 2005; Ross et al. 2011; Wehby et al. 2012).

An additional contribution of this study was to preliminarily examine these dimensions by assessing the degree to which they relate to one another, as well as to teacher characteristics and coach assignment. Within each informant, the four dimensions were interrelated, lending initial support for the broad conceptual model of coach–teacher alliance. There were, however, some differences based on teacher characteristics that should be considered when implementing coaching. Specifically, coaches perceived that older teachers had more barriers to coaching. Interestingly, coaches did not have differential reports on the other dimensions nor did older teachers themselves. Research using a different coaching model similarly showed that older teachers were more likely to receive a high number of coaching contacts (Pas et al. 2015); perhaps this was because more time and contacts were needed to overcome similar barriers as those perceived in this study. In addition, middle school teachers rated that they were less invested in the coaching and saw fewer benefits; perhaps this setting difference could be attributed to contact with a greater number of students, resulting in inconsistent classroom dynamics and increased burden. This implies that the formation of coach–teacher alliance may be context-specific and tailoring of the process may be needed for teachers within different settings to form a successful alliance; however, additional context-specific research is needed to confirm this.

Similar to work comparing ratings of parents, teachers, and students (e.g., Mascendaro et al. 2012; Renk and Phares 2004), coach and teacher ratings of alliance in the current study showed moderate correlations with one another. Interestingly, all of the alliance dimensions were rated highly by both teachers and coaches, and coaches perceived minimal barriers. However, teachers provided significantly higher ratings regarding the coaching relationship and perceived benefits than coaches. Teachers may be more apt to respond with socially desirable ratings or have less coaching experience upon which to contrast the current relationship. Although there appears to be a relatively high level of convergence in the coach and teacher ratings, there may also be some areas of divergence, suggesting the value of assessing both coach and teacher perspectives when examining alliance.

Limitations and Future Directions

Although the current sample of 147 teachers and four coaches is relatively small for factor analytic and psychometric work, there are few large scale studies of coaching and the dynamic coach–teacher relationship (Becker et al. 2013). Nevertheless, the relatively small sample may limit the interpretation and generalizability of the findings. Notably, while teachers did not significantly vary in their ratings based on the coach they worked with, there was some evidence that one coach was more conservative in her ratings than the other coaches. A larger sample of teachers and coaches would allow for further validation of this factor solution, as well as a more sophisticated examination of the relationships between factors and potential variation based on teacher and coach characteristics. All of the teachers volunteered to be in the coaching intervention and therefore they may differ from those who did not volunteer. This study was also conducted specifically with the Classroom Check-Up coaching model. Although this model is similar in structure to other collaborative and problem-solving coaching models, future research should be conducted using the coach–teacher alliance measures following other coaching models. Further, research examining the alliance measure in relation to other implementation outcomes is needed to establish predictive validity and perhaps inform whether one rater is more valid. Lastly, extant literature supports the importance of the five dimensions measured; it is possible that additional research in this area would indicate that other dimensions should be considered.

Conclusions and Implications

This study was intended to test the coach–teacher alliance measures, with the overarching goal of informing our understanding of the conceptualization and measurement of alliance in the context of a coaching implementation support model. The current coach–teacher alliance measures reflected multiple dimensions including working relationship, perceptions of a competently implemented coaching process, teacher investment, perceived benefits, and barriers to coaching. Each of these dimensions may differentially associate with the teacher and student outcomes targeted by coaching models; this is an area for further research. Importantly, assessing alliance from both the perspective of coaches and teachers may contribute unique information that is relevant to the shaping and improvement of coaching being delivered, teacher goal attainment, and ultimately teacher and student outcomes. While coaches bring greater experience with the coaching process, and therefore may have a broader perspective on how well the coaching proceeded, coaches may also struggle to objectively assess how well they delivered the model. Thus, alliance data could be shared with either coaches or teachers (although they were not in the current study). In doing so, future studies may investigate the extent to which this information can improve coaching. Both perspectives may provide helpful information in evaluating and improving the coaching process, including self-reflection on one’s o wn engagement in the process and the identification of areas of strength and weakness. Taken together, these findings suggest that the coach–teacher alliance measures hold promise for use in future studies, and in turn may inform efforts to improve our understanding of the ‘black box’ of coaching implementation supports.

Acknowledgments

We thank Celene Domitrovich, Wendy Reinke, Keith Herman, and Jeanne Poduska for their consultation on the alliance measures used and tested in the Double Check Project. Support for this work comes from the U.S. Department of Education, Institute of Education Sciences (R324A110107; R305A150221, R305A130701), Spencer Foundation, and the National Institute of Mental Health (T32 MH019545-23).

Footnotes

Compliance with Ethical Standards All procedures were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Conflict of Interest The authors declare that they have no conflict of interest.

Informed Consent Informed consent was obtained from all participants included in the study.

References

  1. Ackerman SJ, Hilsenroth MJ. A review of therapist characteristics and techniques positively impacting the therapeutic alliance. Clinical Psychology Review. 2003;23:1–33. doi: 10.1016/S0272-7358(02)00146-0. [DOI] [PubMed] [Google Scholar]
  2. Allen SJ, Graden JL. Best practices in collaborative problem solving for intervention design. In: Thomas A, Grimes J, editors. Best practices in school psychology. 4. Bethesda: National Association of School Psychologists; 2002. pp. 414–435. [Google Scholar]
  3. Becker KD, Darney D, Domitrovich CE, Keperling JP, Ialongo NS. Supporting universal prevention programs: A two-phased coaching model. Clinical Child and Family Psychology Review. 2013;16:213–228. doi: 10.1007/s10567-013-0134-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bentler PM, Bonett DG. Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin. 1980;88:588–606. doi: 10.1037/0033-2909.88.3.588. [DOI] [Google Scholar]
  5. Bordin ES. The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research, and Practice. 1979;16:252–260. doi: 10.1037/h0085885. [DOI] [Google Scholar]
  6. Bottiani JH, Bradshaw CP, Rosenberg MS, Hershfeldt PA, Pell KL, Debnam KJ. Applying Double Check to response to intervention: Culturally responsive practices for learning disabilities. Insight on Learning Disabilities: Prevailing Theories to Validated Practices. 2012;9:93–107. [Google Scholar]
  7. Bradshaw CP, Rosenberg MS. Supporting schools in cultural proficiency and student engagement. 2016. Unpublished book. [Google Scholar]
  8. Bradshaw CP, Pas ET, Domitrovich CE, Reinke WM, Herman K, Poduska JM. Measure of Coach and Teacher Alliance-Teacher Report. Unpublished Measure. Baltimore: Johns Hopkins University; 2009a. [Google Scholar]
  9. Bradshaw CP, Pas ET, Domitrovich CE, Reinke WM, Herman K, Poduska JM. Measure of Coach and Teacher Alliance-Coach Report. Unpublished Measure. Baltimore: Johns Hopkins University; 2009b. [Google Scholar]
  10. Burbank MD, Kauchak D. An alternative model for professional development: Investigations into effective collaboration. Teaching and Teacher Education. 2003;19:499–514. doi: 10.1016/S0742-051X(03)00048-9. [DOI] [Google Scholar]
  11. Cowan RJ, Sheridan SM. Investigating the acceptability of behavioral interventions in applied conjoint behavioral consultation: Moving from analog conditions to naturalistic settings. School Psychology Quarterly. 2003;18:1–21. doi: 10.1521/scpq.18.1.1.20877. [DOI] [Google Scholar]
  12. Datnow A, Castellano M. Teachers’ responses to success for all: How beliefs, experiences, and adaptations shape implementation. American Educational Research Journal. 2000;37:775–799. doi: 10.2307/1163489. [DOI] [Google Scholar]
  13. Domitrovich CE, Bradshaw CP, Poduska JM, Hoagwood K, Buckley JA, Olin S, … Ialongo NS. Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion. 2008;1:6–28. doi: 10.1080/1754730x.2008.9715730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Domitrovich CE, Poduska JM, Bradshaw CP. The Coach-Teacher Alliance Scale. Penn State University; 2008b. Unpublished technical report. [Google Scholar]
  15. Domitrovich CE, Poduska JM, Bradshaw CP. The Teacher-Coach Alliance Scale. Penn State University; 2008c. Unpublished technical report. [Google Scholar]
  16. Domitrovich CE, Bradshaw CP, Greenberg MT, Embry D, Poduska JM, Ialongo NS. Integrated models of school-based prevention: Logic and theory. Psychology in the Schools. 2010;47:71–88. doi: 10.1002/pits.20452. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Domitrovich CE, Pas ET, Bradshaw CP, Becker KD, Keperling JP, Embry DD, Ialongo N. Individual and school organizational factors that influence implementation of the PAX Good Behavior Game Intervention. Prevention Science. 2015 doi: 10.1007/s11121-015-0557-8. [DOI] [PubMed] [Google Scholar]
  18. Eckert TL, Hintze JM. Behavioral conceptions and applications of acceptability: Issues related to service delivery and research methodology. School Psychology Quarterly. 2000;15:123–148. doi: 10.1037/h0088782. [DOI] [Google Scholar]
  19. Elliott SN. Acceptability of behavioral treatments: Review of variables that influence treatment selection. Professional Psychology: Research and Practice. 1988;19:68–80. doi: 10.1037/0735-7028.19.1.68. [DOI] [Google Scholar]
  20. Frank JL, Kratochwill TR. School-based problem-solving consultation: Plotting a new course for evidence-based research and practice in consultation. In: Erchul WP, Sheridan SM, editors. Handbook of research in school consultation. 2. New York: Taylor and Francis; 2014. pp. 13–30. [Google Scholar]
  21. Hamre BK, Pianta RC, Burchinal M, Field S, LoCasale-Crouch J, Downer JT, … Scott-Little C. A course on effective teacher-child interactions: Effects on teacher beliefs, knowledge, and observed practice. American Educational Research Journal. 2012;49:88–123. doi: 10.3102/0002831211434596. [DOI] [Google Scholar]
  22. Han SS, Weiss B. Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology. 2005;33:665–679. doi: 10.1007/s10802-005-7646-2. [DOI] [PubMed] [Google Scholar]
  23. Hart V, Blattner J, Leipsic S. Coaching versus therapy: A perspective. Consulting Psychology Journal: Practice and Research. 2001;53:229–237. doi: 10.1037/1061-4087.53.4.229. [DOI] [Google Scholar]
  24. Hershfeldt PA, Sechrest R, Pell K, Rosenberg MS, Bradshaw CP, Leaf PJ. Double-Check: A process of cultural responsiveness applied to classroom behavior. Teaching Exceptional Children PLUS. 2009;6:2–18. [Google Scholar]
  25. Horvath AO, Greenberg LS. Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology. 1989;36:223–233. doi: 10.1037/0022-0167.36.2.223. [DOI] [Google Scholar]
  26. Horvath AO, Symonds BD. Relation between working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology. 1991;38:139–149. doi: 10.1037/0022-0167.38.2.139. [DOI] [Google Scholar]
  27. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1–55. doi: 10.1080/10705519909540118. [DOI] [Google Scholar]
  28. Huberman M. Burnout in teaching careers. European Education. 1993;25:47–69. doi: 10.2753/eue1056-4934250347. [DOI] [Google Scholar]
  29. Kuder GF, Richardson MW. The theory of the estimation of test reliability. Psychometrika. 1937;2:151–160. doi: 10.1007/BF02289147. [DOI] [PubMed] [Google Scholar]
  30. Martin DJ, Garske JP, Davis MK. Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology. 2000;68:438–450. doi: 10.1037/0022-006x.68.3.438. [DOI] [PubMed] [Google Scholar]
  31. Mascendaro PM, Herman KC, Webster-Stratton C. Parent discrepancies in ratings of young children’s co-occurring internalizing symptoms. School Psychology Quarterly. 2012;27:134–143. doi: 10.1037/a0029320. [DOI] [PubMed] [Google Scholar]
  32. Maslach C, Schaufeli WB, Leiter MP. Job burnout. Annual Review of Psychology. 2001;52:397–422. doi: 10.1146/annurev.psych.52.1.397. [DOI] [PubMed] [Google Scholar]
  33. Muthén LK, Muthén BO. Mplus User’s Guide. Vol. 7. Los Angeles, CA: Muthén & Muthén; 1998–2014. [Google Scholar]
  34. Nadeem E, Gleacher A, Beidas RS. Consultation as an implementation strategy for evidence-based practices across multiple contexts: Unpacking the black box. Administration and Policy in Mental Health and Mental Health Services Research. 2013;40:439–450. doi: 10.1007/s10488-013-0502-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Newman DS, Guiney MC, Barrett CA. Language use in consultation: Can Bwe^ help teachers and students? Consulting Psychology Journal: Practice and Research. 2015;67:48–64. doi: 10.1037/cpb0000028. [DOI] [Google Scholar]
  36. Pas ET, Bradshaw CP, Cash AH. Coaching classroom-based preventive interventions. In: Weist MD, Lever NA, Bradshaw CP, Owens JS, editors. Handbook of School Mental Health: Research, Training, Practice, and Policy. 2. New York: Springer; 2014. pp. 255–267. [Google Scholar]
  37. Pas ET, Bradshaw CP, Becker KD, Domitrovich CE, Berg J, Musci R, Ialongo NS. Identifying patterns of coaching to support the implementation of the Good Behavior Game: The role of teacher characteristics. School Mental Health. 2015;7:61–73. doi: 10.1007/s12310-015-9145-0. [DOI] [Google Scholar]
  38. Power TJ, Soffer SL, Mautone JA, Costigan TE, Jones HA, Clarke AT, Marshall SA. An analysis of teacher investment in the context of a family–school intervention for children with ADHD. School Mental Health. 2009;1:107–117. doi: 10.1007/s12310-009-9005-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Reinke WM, Lewis-Palmer T, Merrell K. The Classroom Check-up: A classwide teacher consultation model for increasing praise and decreasing disruptive behavior. School Psychology Review. 2008;37:315–332. [PMC free article] [PubMed] [Google Scholar]
  40. Reinke WM, Herman KC, Sprick R. Motivational interviewing for effective classroom management: The classroom check-up. New York: Guilford Press; 2011. [Google Scholar]
  41. Reinke WM, Herman KC, Stormont M, Newcomer L, David K. Illustrating the multiple facets and levels of fidelity of implementation to a teacher classroom management intervention. Administration and Policy in Mental Health and Mental Health Services Research. 2013;40:494–506. doi: 10.1007/s10488-013-0496-2. [DOI] [PubMed] [Google Scholar]
  42. Renk K, Phares V. Cross-informant ratings of social competence in children and adolescents. Clinical Psychology Review. 2004;24:239–254. doi: 10.1016/j.cpr.2004.01.004. [DOI] [PubMed] [Google Scholar]
  43. Rhodes C, Beneicke S. Coaching, mentoring and peer-networking: Challenges for the management of teacher professional development in schools. Journal of In-Service Education. 2002;28:297–310. doi: 10.1080/13674580200200184. [DOI] [Google Scholar]
  44. Rose DJ, Church RJ. Learning to teach: The acquisition and maintenance of teaching skills. Journal of Behavioral Education. 1998;8:5–35. doi: 10.2307/41824208. [DOI] [Google Scholar]
  45. Ross SW, Romer N, Horner RH. Teacher well-being and the implementation of Schoolwide Positive Behavior Interventions and Supports. Journal of Positive Behavior Interventions. 2011 doi: 10.1177/1098300711413820. [DOI] [Google Scholar]
  46. Von Brock MB, Elliott SN. Influence of treatment effectiveness information on the acceptability of classroom interventions. Journal of School Psychology. 1987;25:131–144. doi: 10.1016/0022-4405(87)90022-7. [DOI] [Google Scholar]
  47. Wanless SB, Rimm-Kaufman SE, Abry T, Larsen RA, Patton CL. Engagement in training as a mechanism to understanding fidelity of implementation of the Responsive Classroom Approach. Prevention Science. 2014:1–10. doi: 10.1007/s11121-014-0519-6. [DOI] [PubMed] [Google Scholar]
  48. Wehby JH, Maggin DM, Moore Partin TC, Robertson R. The impact of working alliance, social validity, and teacher burnout on implementation fidelity of the Good Behavior Game. School Mental Health. 2012;4:22–33. doi: 10.1007/s12310-011-9067-4. [DOI] [Google Scholar]
  49. Wright D, Torrey GK. A comparison of two peer-referenced assessment techniques with parent and teacher ratings of social skills and problem behaviors. Behavioral Disorders. 2001;26:173–182. doi: 10.2307/23888766. [DOI] [Google Scholar]

RESOURCES