Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Jul 1.
Published in final edited form as: Adm Policy Ment Health. 2017 Jul;44(4):480–491. doi: 10.1007/s10488-015-0672-7

Discrepancies in Leader and Follower Ratings of Transformational Leadership: Relationship with Organizational Culture in Mental Health

Gregory A Aarons 1,3,✉, Mark G Ehrhart 2, Lauren R Farahnak 1,3, Marisa Sklar 1,3, Jonathan Horowitz 1,3
PMCID: PMC4775440  NIHMSID: NIHMS707650  PMID: 26164567

Abstract

The role of leadership in the management and delivery of health and allied health services is often discussed but lacks empirical research. Discrepancies are often found between leaders’ self-ratings and followers’ ratings of the leader. To our knowledge no research has examined leader–follower discrepancies and their association with organizational culture in mental health clinics. The current study examines congruence, discrepancy, and directionality of discrepancy in relation to organizational culture in 38 mental health teams (N = 276). Supervisors and providers completed surveys including ratings of the supervisor transformational leadership and organizational culture. Polynomial regression and response surface analysis models were computed examining the associations of leadership discrepancy and defensive organizational culture and its subscales. Discrepancies between supervisor and provider reports of transformational leadership were associated with a more negative organizational culture. Culture suffered more where supervisors rated themselves more positively than providers, in contrast to supervisors rating themselves lower than the provider ratings of the supervisor. Leadership and leader discrepancy should be a consideration in improving organizational culture and for strategic initiatives such as quality of care and the implementation and sustainment of evidence-based practice.

Keywords: Leadership, Discrepancy, Organizations, Management, Public sector, Mental health

Introduction

Multi-source feedback about leadership is common in organizational research and applied practice (London and Smither 1995; Tornow 1993) and is thought to overcome biases and limitations inherent in using only self-ratings (Ashford 1993; Hanser and Muchinsky 1978; Podsakoff and Organ 1986). Although self-ratings can provide important insight into how leaders perceive their own abilities and behaviors, follower ratings are particularly useful in that they provide a different perspective on leader behavior. Follower ratings may be more reliable (greater number of data points) and may offer additional information that is not available to supervisors’ own supervisors (Riggio and Cole 1992). For example, follower reports may be less subject to self-perceptual biases that may influence leader self-report (Martinko and Gardner 1987; Yammarino and Atwater 1993).

Interpreting multi-source feedback can become complicated when discrepant ratings emerge. The authors’ anecdotal experience and other published studies suggest that discrepancies between self- and other-ratings of leaders are not uncommon (Ostroff et al. 2004) and can lead to strong emotional responses from those receiving discrepant feedback. Leaders tend to overrate their own leadership abilities relative to ratings provided by followers (Atwater and Yammarino 1992; Fleenor et al. 1996). Where discrepancies are present, leaders may express surprise, consternation, disbelief, or emotional distress upon learning of such discrepancies. In the present study, we use quantitative data to examine the degree to which such leader–follower discrepancies exist, the directionality of discrepancies (i.e., leader more positive ratings compared to followers or leader less positive ratings relative to followers), and the degree to which discrepancies are related to organizational culture.

Also consistent with our anecdotal experience, a number of previous investigations have found poor agreement between self- and other-ratings (e.g., Atwater and Yammarino 1992; Harris and Schaubroeck 1988; Mabe and West 1992; Ostroff et al. 2004). In a review of 22 studies, Thornton (1980) found that leader self-ratings did not correlate strongly with ratings from other sources, and that self-ratings tended to be more positive and less variable. As another example, Harris and Schaubroeck (1988) conducted a meta-analysis of discrepancy studies and found lower agreement on self-supervisor and self-peer rating comparisons than on peer-supervisor comparisons. Similarly, Mount (1984) found that agreement was higher for supervisor-follower (other–other) ratings than for self-other (self-supervisor or self-follower) ratings.

Several factors may give rise to self-other discrepancies. The formation of accurate self-perceptions is a delicate process, particularly for individuals in management or supervisory roles which are often ambiguous and fluid (London 2002). Bias in self-ratings may arise in several ways. First, leaders may fail to adequately receive or seek out feedback from followers thus limiting confirmation or disconfirmation. Second, some leaders may judge supervisee feedback to be relatively unimportant, particularly if their own performance evaluations do not depend on their followers’ perceptions. Third, although leaders may value follower feedback, they may fail to seek it out for fear that this would communicate uncertainty (Ashford 1986; Ashford and Tsui 1991). Fourth, leaders may arrive at inaccurate self-perceptions by gathering or attending to information from a biased sample. For example, because receiving negative feedback may be a threat to one’s self-perception, leaders may seek more feedback from followers they believe to hold a favorable opinion of them.

Even if leaders are able to obtain adequate and accurate information regarding their followers’ opinions on an ongoing basis, they may fail to effectively integrate this information and make the necessary behavioral changes where discrepancies exist. More general biases may also exist. For example, social psychological studies have found that when evaluating themselves on general abilities, individuals tend to rank themselves above the median, indicating that most people hold unrealistically positive views of themselves (Buunk and Van Yperen 1991; Taylor and Brown 1988). Consistent with research on self-enhancement (Sedikides and Gregg 2008), leaders are likely to have overly positive self-ratings at baseline (Podsakoff and Organ 1986), independent of feedback they may or may not have received from followers. Further, they are likely to discount negative feedback (Taylor and Brown 1988). For example, leaders may discount or reinterpret negative feedback in a way that is less threatening to their self-esteem. Consistent with self-verification theory, leaders are more likely to attend to feedback that is consistent with their self-opinion, whether that opinion is positive or negative (Swann and Read 1981). Leaders may acknowledge the validity of their followers’ feedback, but may rationalize it as the inevitable result of a difference in perspective.

Discrepancies in ratings may provide insight into both individual performance and organizational context. Several investigators have found that degree of discrepancy is related to job performance (Atwater et al. 1995; Bass and Yammarino 1991; Church 1997; Fleenor et al. 1996). One reason for this may be that accurate self-perceptions are important to social adjustment and functioning (Jourard and Landsman 1980; Schulz 1977). Self-awareness moderates the relationship between transformational leadership behavior and managerial performance (Sosik and Megerian 1999). Metacognitive skill (Kruger and Dunning 1999), the ability to recognize one’s own errors, may be a precursor to the development of a skill as complex as leadership. Individuals who are unable to identify their own weaknesses or incompetencies are unable to make the behavioral changes necessary to improve (Atwater et al. 2005).

Leadership is an important predictor of organizational culture and climate (Aarons and Sawitzky 2006a, b; Ashkanasy et al. 2000; Ehrhart 2004; Ehrhart et al. 2014; Litwin and Stringer 1968; Tsui et al. 2006; Zohar and Tenne-Gazit 2008). Although individual characteristics are likely to explain some of the variance in discrepancies, it is worth considering whether discrepancies are related to organizational characteristics. There is a small literature examining the relationship between culture and discrepancies in leadership ratings. For example, Atwater et al (2009) examined self- and follower-ratings of leadership in a sample of 964 managers from 21 countries. They found that the relationship between self- and follower-ratings was more strongly positive in countries that are marked by higher levels of assertiveness. In another example, Atwater et al. (2005) examined the relationship between self-other discrepancies and managerial performance in the U.S. and five European countries and found that the impact of self-other discrepancies varied across countries. In the U.S., consideration of both self- and other-ratings were important for the prediction of performance, whereas in Europe, only follower ratings were predictive of performance.

In this paper, we examine the relationship between leader–follower discrepancies in transformational leadership and culture. Summarizing Bass’s (1985) theory of transformational leadership, Antonakis and House (2002) defined the concept as follows: “Transformational leaders act as agents of change by arousing and transforming followers’ attitudes, beliefs, and motives from a lower to a higher level of arousal. They provide vision, develop emotional relationships with followers and make them aware of, and believe in, superordinate goals that go beyond self-interest” (p. 8). Leaders enact transformational leadership through behaviors that demonstrate inspirational motivation, individualized consideration, intellectual stimulation, and idealized influence (Bass and Avolio 1995). Organizational culture refers to the set of expectations and norms that govern behavior within an organization (Ehrhart et al. 2014; Glisson et al. 2008). Although there is some research on the role of national culture on discrepancies in leadership ratings (Atwater et al. 2005, 2009), we are aware of no prior research that has examined how leader–follower discrepancies in leadership ratings, and particularly transformational leadership ratings, are related to the culture that emerges in the unit.

Alignment between leader and follower perspectives on managerial effectiveness is indicative of better communication between leaders and their followers (Keyton 2010), which is likely to lead to more positive/less negative organizational cultures. As noted previously, there are many possible explanations for differences between leaders and their followers with regard to their opinions on managerial effectiveness, including lack of leader feedback seeking, lack of interest in follower opinions, or leaders avoiding threats to self-perceptions and only seeking favorable opinions. In all of these cases, the implications for the norms that develop in the unit are likely dysfunctional, resulting in a more negative organizational culture. Furthermore, the negative implications are likely more stark when the leaders’ ratings are higher than follower ratings. Follower ratings that are higher than leader ratings may indicate other dysfunction within the team, such as lack of leader self-confidence, but the implications are likely less negative than the opposite. Thus, for the present study we make the following hypotheses:

  • Hypothesis 1 There will be discrepancies in transformational leadership, such that leaders will rate their transformational leadership more positively than their followers.

  • Hypothesis 2 Leadership discrepancies will predict ratings of organizational culture.

  • Hypothesis 3 When discrepancies are present between leader and follower ratings of leadership styles, there will be less favorable ratings of organizational culture when follower ratings of leadership styles are lower than leader ratings than when follower ratings of leadership styles are greater than leader ratings.

Methods

Participants

Participants were recruited from public sector mental health clinics in San Diego County, California. Based on administrative data, 99 teams within mental health programs were initially identified. Of the 99 teams, 26 were considered ineligible because they provided residential treatment and/or information was not provided to identify their supervisor and thus could not be nested within teams for analysis (i.e., a single supervisor for the team was not identified). In addition, one team was excluded due to nonresponse despite repeated contact attempts, resulting in a total of 72 eligible and responsive teams. Seven of the remaining 72 teams declined participation, resulting in 68 participating teams (89 % response rate). Some remaining teams were excluded due to having no direct supervisor data, only supervisor data, only one provider, or incomplete measures. This resulted in a final analytic sample of 276 direct service providers (n = 236) and their supervisors (n = 40), which comprised 38 teams with a mean of 7.3 members in each team (range 3–15; SD = 3.0). Providers were clinicians or case-managers providing direct services to clients. Supervisors were program managers or team leaders that directly supervised providers. In the context of this study we identify providers as “followers” and supervisors as “leaders”. Table 1 shows participant demographics.

Table 1.

Demographic characteristics of clinicians and supervisors

Clinicians (n = 236) Supervisors (n = 40)
Age 35.85 41.26
Years of experience 7.68 13.95
Years in agency 2.80 5.80
Gender
 Male 20.1 % 32.5 %
 Female 79.9 % 67.5 %
Ethnicity
 Caucasian 51.1 % 70.0 %
 Hispanic 24.5 % 15.0 %
 African American 8.6 % 0.0 %
 Asian American 4.3 % 2.5 %
 Other 11.6 % 0.0 %
Educational level
 High school 1.3 % 0.0 %
 Some college 6.0 % 0.0 %
 Bachelor’s degree 14.1 % 0.0 %
 Some graduate work 7.7 % 0.0 %
 Master’s degree 62.8 % 87.5 %
 Doctoral degree 6.4 % 12.5 %
 Other 1.7 % 0.0 %
Major of highest degree
 Marriage/family therapy 46.0 % 50.0 %
 Social work 22.6 % 37.5 %
 Psychology 14.9 % 10.0 %
 Child development 3.8 % 0.0 %
 Human relations 2.1 % 0.0 %
 Other 10.6 % 2.5 %
Open in a new tab

N = 276

Procedure

Research staff conducted meetings with each of the participating teams in order to administer the surveys in paper format. This study was approved by the IRB of Children’s Hospital San Diego. Consent forms were signed by each participant and returned to research assistants prior to survey administration. Each meeting included the team’s entire clinical staff, unless team members were out of the office, refused to participate, or were on-call and needed to leave to address client issues. The surveys took an average of 60 min (range 45–180 min) to complete. Participants returned completed surveys to research assistants who then checked for completeness.

Measures

The Multifactor Leadership Questionnaire (Bass and Avolio 1995) 45-item form 5× was used to assess providers’ perceptions of the supervisors’ transformational leadership behaviors. Transformational leadership is comprised of the following four subscales: idealized influence (8 items, α = 0.87), inspirational motivation (four items, α = 0.91), intellectual stimulation (four items, α = 0.90), and individual consideration, (four items, α = 0.90). Our analyses focused on overall transformational leadership (20 items, α = 0.95).

Organizational Culture

The Children’s Services Survey (Glisson and James 2002) was used to assess organizational culture. The CSS has established excellent psychometric characteristics, and subscales are associated with work attitudes, staff turnover, and service quality in human service organizations (Glisson and James 2002). The organizational culture subscales were adapted for use in mental health services (Cooke and Rousseau 1988; Farmer et al. 2002; Glisson and James 2002). The subscales used in the current study assessed defensive culture including sub-scales of consensus (nine items, α = 0.85), conformity (seven items, α = 0.82) and subservience (four items, α = 0.82).

Data Analyses

The data analytic approach involved aggregating provider ratings to the team level (K = 38) in order to make team-level inferences about relationships among variables. Supervisor ratings of leadership were already at the team level. To justify aggregation of the -provider data to the team level, intraclass correlation coefficients (ICC(1)s) and within group agreement (rWG(j)) statistics were calculated; these are presented in Table 2 and support the aggregation of team ratings.

Table 2.

Subscale intraclass correlation coefficients (ICC) and within group agreement

Subscale # Items ICC 95 % CI for ICC Mean rWG(j)
Multifactor leadership questionnaire: leader self-report
 Transformational leadership 20 0.342 0.187, 0.496
Multifactor leadership questionnaire: clinician report
 Transformational leadership 20 0.342 0.187, 0.496 0.914
Organizational culture: clinician report
 Defensive cultures 20 0.12 0, 0.224 0.824
 Consensus 9 0.049 0, 0.146 0.795
 Conformity 7 0.092 0, 0.203 0.676
 Subservience 4 0.148 0.024, 0.272 0.598
Open in a new tab

Prior to conducting polynomial regression with response surface analysis for examining discrepancies, researchers have recommended inspecting the data to ensure that discrepancies indeed exist in the data being analyzed (Shanock et al. 2010). Consistent with Fleenor et al. (1996), transformational leadership scores were standardized and supervisor and provider ratings of transformational leadership that were discrepant by half a standard deviation or less were considered to be “in agreement.” This also provided information relevant to our first hypothesis regarding the presence and direction of discrepancies in supervisor-provider ratings. We then tested the relationship between discrepancies in leadership ratings and culture utilizing polynomial regression and response surface methodology (Edwards 2002; Shanock et al. 2013, 2010). First, the omnibus tests from four polynomial regression analyses were assessed to determine whether variance in organizational culture was predicted by the polynomial trends in and the interaction between supervisor self-ratings and provider ratings of supervisor transformational leadership. Next, we tested the slope and curvature along the x = y and x = −y axes of the response surface because they correspond directly to the substantive research questions of interest. The x = y axis is the axis along which provider and supervisor ratings are congruent, whereas the x = −y axis is the axis along which provider and supervisor ratings are incongruent. The relationship between organizational culture and either congruence or incongruence was then explored.

Results

Means, standard deviations, and correlations among the study variables included in the discrepancy analyses are presented in Table 3.

Table 3.

Means, standard deviations and correlations of study variables

Mean sd 1 2 3 4 5
Provider TL ratings 2.40 0.58 –
Supervisor TL ratings 2.50 0.48 0.36* –
OCM—consensus 1.66 0.33 −0.43** −0.08 –
OCM—conformity 1.56 0.40 0.60** −0.29 0.83** –
OCM—subservience 1.67 0.48 −0.62** −0.26 0.81** 0.93** –
Open in a new tab
*

p < 0.05;

**

p < 0.01

Supervisor-Provider Discrepancy in Leadership Ratings

Table 4 shows the means, proportions, and directionality of discrepancy between supervisor (self) and provider ratings of transformational leadership. Over 70 % of our sample had supervisor and provider ratings of overall transformational leadership that were different from each other in one direction or the other. Specifically, 44.7 % of teams consisted of supervisors who rated their transformational leadership higher than that of providers, and 28.9 % of supervisors rated their transformational leadership behaviors lower than that of providers. Because of the level of discrepancies present and because both directions of discrepancy were well-represented in the data, we could move forward with the polynomial regression analyses. This analysis also allowed provided information relevant for testing Hypothesis 1. Although more teams showed a pattern in which supervisors rated themselves higher than providers than vice versa, the mean of the supervisor self-ratings (M = 2.495) was not significantly greater than that of provider ratings (M = 2.398) of the supervisor’s transformational leadership (Mdiff = 0.097, SE = 0.098, t(37) = 0.994, p = 0.327). Thus, Hypothesis 1 was not supported.

Table 4.

Supervisor and provider discrepancy on the multifactor leadership questionnaire

Agreement groups N Percentage Mean supervisors Mean providers
Transformational leadership
 Supervisors >providers 17 44.74 2.65 2.01
 Supervisors = providers 10 26.32 2.69 2.68
 Supervisors <providers 11 28.95 2.09 2.75
Open in a new tab

Discrepancy in Leadership Ratings and Organizational Culture

We then tested our second hypothesis, that leadership discrepancies would predict ratings of organizational culture. Results of omnibus tests suggested that supervisor self-ratings and provider ratings of supervisor transformational leadership predicted variance in consensus culture (R2 = 0.306, p = 0.032), conformity culture (R2 = 0.413, p = 0.003), subservience culture (R2 = 0.454, p = 0.001), and overall defensive culture (R2 = 0.0413, p = 0.003). Response surface analyses were examined next to further test our second hypothesis that discrepancies between supervisor self-ratings and provider ratings of supervisor transformational leadership predict ratings of organizational culture. Surface tests were also examined to test our third hypothesis, that ratings of organizational culture would be more negative when provider ratings of transformational leadership are lower than supervisor ratings than when provider ratings of transformational leadership are greater than supervisor ratings. Table 5 shows results from all polynomial and response surface analyses. Graphical representations of response surfaces are presented in Figs. 1, 2, 3, 4.

Table 5.

Polynomial regression and response surface analyses

Consensus culture and overall transformational leadership
Variable regressed onto consensus culture b (se)
Polynomial regression analysis Constant 1.592**
Transformational leadership: supervisor 0.191 (0.124)
Transformational leadership: team −0.323 (0.099)*
Transformational leadership: sup squared 0.233 (0.211)
Transformational leadership: sup × team −0.681 (0.289)*
Transformational leadership: team squared 0.262 (0.206)
R2 0.306*
Response surface tests a1 −0.13 (0.12)
a2 −0.19 (0.28)
a3 0.51 (0.19)*
a4 1.18 (0.36)**
Conformity culture and overall transformational leadership
Variable regressed onto conformity culture b (se)
Polynomial regression analysis Constant 1.418**
Transformational leadership: supervisor 0.002 (0.138)
Transformational leadership: team −0.420 (0.110)**
Transformational leadership: sup squared 0.300 (0.236)
Transformational leadership: sup × team −0.327 (0.323)
Transformational leadership: team squared 0.334 (0.230)
R2 0.413**
Response surface tests a1 −0.42 (0.14)**
a2 0.31 (0.31)
a3 0.42 (0.21)*
a4 0.96 (0.40)*
Subservience culture and overall transformational leadership
Variable regressed onto subservience culture b (se)
Polynomial regression analysis Constant 1.494**
Transformational Leadership: supervisor 0.089 (0.158)
Transformational leadership: team −0.547 (0.126)**
Transformational leadership: sup squared 0.371 (0.270)
Transformational leadership: sup × team −0.618 (0.370)
Transformational leadership: team squared 0.461 (0.263)
R2 0.454**
Response surface tests a1 −0.46 (0.16)**
a2 0.21 (0.35)
a3 0.64 (0.24)*
a4 1.45 (0.46)**
Overall defensive culture and overall transformational leadership
Variable regressed onto defensive culture b (se)
Polynomial regression analysis Constant 1.501**
Transformational leadership: supervisor 0.094 (0.132)
Transformational leadership: team −0.430 (0.105)**
Transformational leadership: sup squared 0.301 (0.226)
Transformational leadership: sup × team −0.542 (0.309)
Transformational leadership: team squared 0.353 (0.220)
R2 0.413**
Overall defensive culture and overall transformational leadership
Variable regressed onto defensive culture b (se)
Response surface tests a1 −0.34 (0.13)*
a2 0.11 (0.29)
a3 0.52 (0.20)*
a4 1.20 (0.38)**
Open in a new tab

N = 38;

*

p <.05;

**

p < .01

Fig. 1.

Fig. 1

Open in a new tab

Consensus predicted by discrepancy between supervisor and provider ratings of Transformational Leadership (TL). a1: The line of perfect agreement (solid black line on floor) as related to consensus (CNS) has a non-significant slope. Thus, agreement between supervisors and provider’s TL ratings are not related to CNS. a4: Moving along the line of incongruence (dashed line on floor) away from the center of the graph to either the left or right shows how the degree of discrepancy between supervisors and provider’s TL has a significant curvilinear relationship with CNS. The graph shows that toward the left and right of the graph, where ratings become more and more discrepant, CNS increases. a3: The direction of discrepancy matters. At the bottom right corner of the graph (where provider ratings of TL is low combined with high supervisor ratings of TL), CNS is very high. At the bottom left corner of the graph (where provider ratings of TL is high combined with low supervisor ratings of TL), CNS is high, though not as high as when supervisor rating of TL are high and provider ratings are low

Fig. 2.

Fig. 2

Open in a new tab

Conformity predicted by discrepancy between supervisor and provider ratings of Transformational Leadership (TL). a1: The line of perfect agreement (solid black line on floor) as related to conformity (CON) has a negative slope. Thus, agreement between supervisors and provider’s TL ratings matters such that the greatest level of CON is at the front corner of the graph where supervisor and provider TL are both low, and decreases toward the back of the graph where both supervisors and provider’s TL are both in agreement and high. a4: Moving along the line of incongruence (dashed line on floor) away from the center of the graph to either the left or right shows how the degree of discrepancy between supervisors and provider’s TL has a significant curvilinear relationship with CON. The graph shows that toward the left and right of the graph, where ratings become more and more discrepant, CON increases. a3: The direction of discrepancy matters. At the bottom right corner of the graph (where provider ratings of TL is low combined with high supervisor ratings of TL), CON is very high. At the bottom left corner of the graph (where provider ratings of TL is high combined with low supervisor ratings of TL), CON is high, though not as high as when supervisor rating of TL are high and provider ratings are low

Fig. 3.

Fig. 3

Open in a new tab

Subservience predicted from discrepancy between supervisor and provider ratings of Transformational Leadership (TL). a1: The line of perfect agreement (solid black line on floor) as related to subservience (S) has a negative slope. Thus, agreement between supervisors and provider’s TL ratings matters such that the greatest level of S is at the front corner of the graph where supervisor and provider TL are both low, and decreases toward the back of the graph where both supervisors and provider’s TL are both in agreement and high. a4: Moving along the line of incongruence (dashed line on floor) away from the center of the graph to either the left or right shows how the degree of discrepancy between supervisors and provider’s TL has a significant curvilinear relationship with S. The graph shows that toward the left and right of the graph, where ratings become more and more discrepant, S increases. a3: The direction of discrepancy matters. At the bottom right corner of the graph (where provider ratings of TL is low combined with high supervisor ratings of TL), S is very high. At the bottom left corner of the graph (where provider ratings of TL is high combined with low supervisor ratings of TL), S is high, though not as high as when supervisor rating of TL are high and provider ratings are low

Fig. 4.

Fig. 4

Open in a new tab

Defensive culture predicted from discrepancy between supervisor and provider ratings of Transformational Leadership (TL). a1: The line of perfect agreement (solid black line on floor) as related to overall Defensive Culture (PDC) has a negative slope. Thus, agreement between supervisors and provider’s TL ratings matters such that the greatest level of PDC is at the front corner of the graph where supervisor and provider TL are both low, and decreases toward the back of the graph where both supervisors and provider’s TL are both in agreement and high. a4: Moving along the line of incongruence (dashed line on floor) away from the center of the graph to either the left or right shows how the degree of discrepancy between supervisors and provider’s TL had a significant curvilinear relationship with PDC. The graph shows that toward the left and right of the graph, where ratings become more and more discrepant, PDC increases. a3: The direction of discrepancy matters. At the bottom right corner of the graph (where provider ratings of TL is low combined with high supervisor ratings of TL), PDC is very high. At the bottom left corner of the graph (where provider ratings of TL is high combined with low supervisor ratings of TL), PDC is high, though not as high as when supervisor rating of TL are high and provider ratings are low

Surface tests assessing the relationship between congruence/incongruence between supervisor self-ratings and provider ratings of supervisor transformational leadership and consensus culture were examined first. Results suggested that agreement between supervisors and providers’ ratings of transformational leadership did not influence consensus (a1 = −0.13, t = −1.071, p = 0.291). However, incongruence between supervisors and providers’ ratings of transformational leadership had a significant curvilinear relationship with consensus (a4 = 1.18, t = 3.281, p = 0.002), such that as ratings between supervisors and providers became increasingly discrepant, ratings of a consensus culture increased. Further analysis revealed that the direction of discrepancy between supervisor and providers’ ratings of transformational leadership significantly influenced consensus culture (a3 = 0.51, t = 2.741, p = 0.010). Consistent with our third hypothesis, consensus culture was greater when providers’ ratings of transformational leadership were low and supervisor self-ratings of transformational leadership were high than when providers’ ratings of transformational leadership were high combined with low supervisor self-ratings of transformational leadership.

A mostly similar pattern emerged when assessing the relationship between congruence/incongruence between supervisor self-ratings and provider ratings of supervisor transformational leadership and conformity culture. Unlike the results regarding a consensus culture, results suggested that agreement between supervisors and clinicians’ ratings of transformational leadership significantly influenced conformity (a1 = −0.42, t = −3.021, p = 0.005). Specifically, the greatest level of conformity was when supervisor and providers’ similarly rated transformational leadership low. Conformity decreased as both supervisor and clinicians’ ratings of transformational leadership increased. Similar to the results with consensus culture, incongruence between supervisors and providers’ ratings of transformational leadership had a significant curvilinear relationship with conformity (a4 = 0.96, t = 2.381, p = 0.023), such that as ratings between supervisors and providers became increasingly discrepant, ratings of a conformity culture increased. Further analysis revealed that the direction of discrepancy between supervisor and providers’ ratings of transformational leadership significantly influenced conformity culture (a3 = 0.42, t = 2.032, p = 0.050). Consistent with our third hypothesis, conformity culture was greater when provides’ ratings of transformational leadership were low and supervisor self-ratings of transformational leadership were high than when providers’ ratings of transformational leadership were high combined with low supervisor self-ratings of transformational leadership.

The pattern of results assessing the relationship between congruence/incongruence between supervisor self-ratings and provider ratings of supervisor transformational leadership and subservience culture followed a similar pattern to that of conformity culture. Results demonstrated that agreement between supervisors and providers’ ratings of transformational leadership significantly influenced subservience (a1 = −0.46, t = −2.906, p = 0.006). Specifically, the greatest level of subservience was when supervisor and providers’ similarly rated transformational leadership low. Subservience decreased as both supervisor and providers’ ratings of transformational leadership increased. Incongruence between supervisors and providers’ ratings of transformational leadership had a significant curvilinear relationship with subservience (a4 = 1.45, t = 3.142, p = 0.003), such that as ratings between supervisors and providers became increasingly discrepant, ratings of a subservient culture increased. Further analysis revealed that the direction of discrepancy between supervisor and providers’ ratings of transformational leadership significantly influenced subservience culture (a3 = 0.62, t = 2.668, p = 0.011). Consistent with our third hypothesis, subservience culture was greater when providers’ ratings of transformational leadership were low and supervisor self-ratings of transformational leadership were high than when providers’ ratings of transformational leadership were high combined with low supervisor self-ratings of transformational leadership.

This same pattern of results emerged when assessing the relationship between congruence/incongruence between supervisor self-ratings and provider ratings of supervisor transformational leadership and overall defensive culture. Results suggested that agreement between supervisors and providers’ ratings of transformational leadership significantly influenced overall defensive culture (a1 = −0.34, t = −2.620, p = 0.013). Specifically, the greatest level of defensive culture was when supervisor and providers’ similarly rated transformational leadership low. Defensive culture decreased as both supervisor and providers’ ratings of transformational leadership increased. Incongruence between supervisors and providers’ ratings of transformational leadership had a significant curvilinear relationship with overall defensive culture (a4 = 1.20, t = 3.120, p = 0.004), such that as ratings between supervisors and providers became increasingly discrepant, ratings of overall defensive culture increased. Further analysis revealed that the direction of discrepancy between supervisor and providers’ ratings of transformational leadership influenced defensive culture (a3 = 0.52, t = 2.605, p = 0.013). Consistent with our third hypothesis, overall defensive culture was greater when providers’ ratings of transformational leadership were low and supervisor self-ratings of transformational leadership were high than when providers’ ratings of transformational leadership were high combined with low supervisor self-ratings of transformational leadership.

Discussion

Our primary findings were that supervisor and provider ratings of supervisors in the organizational context of public mental health service organizations do indeed differ. We also found that the magnitude and direction of this discrepancy were associated with decrements in organizational culture. That is, greater discrepancy between supervisors and providers was associated with a more defensive (i.e., more negative) organizational culture. This effect was more pronounced where supervisors rated themselves more positively than did providers as compared to supervisors rating themselves less positively than the providers they supervised. These findings are consistent with the literature and with our anecdotal experience in working with public sector mental health and social service organizations.

We found that discrepancy in either direction (i.e., supervisor rating themselves higher than did providers, or supervisors rating themselves lower than did providers) was associated with more negative organizational culture. Thus, helping leaders develop self-perceptions that are more consistent with followers perceptions of the leader, may help to mitigate negative effects of such discrepant perceptions on organizational culture. Consistent with our anecdotal experience and reports from mental health program supervisors, supervisors often ascend from providing clinical services to supervising services and other providers while receiving very little training on supervision, management and leadership (Aarons et al. 2015). Indeed, when management training does occur, it is often focused on day-to-day issues such as time management, billing requirements, and completing appropriate documentation for service contracts or insurance reimbursements. There is little to no training on effective leadership or how to appraise and develop one’s own leadership knowledge, skills, and abilities.

The finding that discrepancies between supervisor and provider ratings of transformational leadership were related to provider ratings of organizational culture raises some additional questions. First, if supervisors have difficulty accurately assessing their own behaviors, do they also have biases in perceptions of their providers or of organizational social context? For example, some supervisors may avoid accurate introspection, and may also avoid or be unable to make accurate assessments or interpretations of the organizational culture and climate in which they operate (Ashkanasy et al. 2000). Furthermore, their own behavior (based on bias or misperception) may contribute to less than optimal organizational context. Common-source bias could also account for this finding; however, a number of factors decrease the likelihood of this explanation and we discuss this issue in the limitations section.

The very act of seeking feedback from providers may lead to improved perceptions of a supervisor’s competence. Ashford and Tsui (1991) found that leaders willing to seek negative feedback have a better understanding of the criteria by which followers judge their work. Leaders who sought negative feedback tended to be more positively rated by followers, whereas leaders seeking positive feedback were rated more negatively. Thus, organizations might use feedback mechanisms to seek out and provide both positive and negative feedback in order to facilitate leader development and promote a more positive organizational culture.

Clinical and Organizational Practice Implications

It is important to note that supervisors/leaders often respond emotionally and with distress when their self-ratings of leadership are highly discrepant, particularly when their own ratings are more positive than their provider/supervisee ratings. This raises the issue of how supervisors respond to negatively discrepant feedback. Attributional biases may impact the degree to which supervisors experience distress or emotional stress related to negatively discrepant feedback. Anecdotally, supervisors generally appear to wonder about the causes of positively discrepant feedback, but do not dwell on it or experience negative emotionality or affect as a result. There may even be a sense of relief that their own moderate or negative self-evaluation is disconfirmed by relatively objective observers.

Limitations

Consistent with recommendations by Podsakoff et al. (2003), items that may potentially exhibit common source bias have proximal and methodological separation in that they are measured in different ways and in different substantive sections of the survey. In this study, one set of provider questions related to ratings of observable supervisor behavior, measured on a Likert-like type scale, and another set of questions assessed perceptions of organizational culture. Each set of questions were embedded in a series of questions with a different substantive focus (i.e., leadership, culture). We considered the use of analytical models to assess common source bias (i.e., latent variables constructed of all potentially biased items), but the approach proved untenable given the small sample size and differing variable types (i.e., categorical, count, continuous). Finally, to promote accurate and unbiased responses and minimize any social pressures or expectations, the survey was conducted voluntarily, confidentially, and online (Podsakoff et al. 2003).

Discrepancy may be an issue in other settings and it may also be that discrepancy in self-other ratings varies in different countries and cultures. For example Cullen et al. (2015) found that self-enhancement and self-diminishment were dependent on culture. The Global Leadership and Organizational Behavior Effectiveness (GLOBE) project has documented issues in leadership from 62 societies and 27 European societies (House et al. 2013). Studies in cross-cultural comparative management have documented multiple universal desirable and undesirable leadership characteristics, but also 35 that are contingent on culture. However, the GLOBE project focused on top management leadership. Future research should continue the examination of first level leadership (Petkevičiūtė and Giedraitis 2013; Priestland and Hanig 2005) across contexts and culture.

Although the GLOBE project looked at top management leadership, current work in first-level strategic leadership for implementation of evidence-based health and allied health is a new area of study. For example, the newly developed Implementation Leadership Scale assess four dimensions of leader characteristics including the degree to which the leader is knowledgeable, proactive, supportive, and perseverant in regard to a strategic initiative (Aarons et al. 2014). Future studies should examine the impact of both transformational and implementation leadership discrepancy on organizational and implementation climate, and evidence-based practice implementation and sustainment.

Conclusions

The role of discrepancy in organizational functioning, organizational change, leader–follower relationships, effective provider supervision, and effective implementation are critical areas not just for research but also for practice. Organizations should consider how discrepancy might support or derail change initiatives and effective functioning in general. Use of multi-source feedback can be used as a developmental tool to help support leaders and supervisors in understanding where they stand, and what biases or misperceptions they may hold. The proximal goals of this approach were to develop and support leaders. The more distal goals included improving organization relationships and functioning, and improving the bottom-line, whether that be quality improvement, organizational viability, or clinical care outcomes. Although leadership is often discussed, it is not often well-addressed in public sector service settings. The work presented here should contribute to an understanding of how to identify leadership discrepancy and its impact as a step toward effective organizational intervention and support for mental health and other allied health organizations.

Acknowledgments

This work was supported by National Institute of Mental Health Grants R01MH072961; R21MH082731; R21MH098124; K01MH001695. The authors thank the community stakeholders, providers, and supervisors who participated in this study.

References

  1. Aarons GA, Ehrhart MG, Farahnak LR. The Implementation Leadership Scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science. 2014;9(1):45. doi: 10.1186/1748-5908-9-45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science. 2015;10(1):11. doi: 10.1186/s13012-014-0192-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aarons GA, Sawitzky AC. Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Administration and Policy in Mental Health and Mental Health Services Research. 2006a;33(3):289–301. doi: 10.1007/s10488-006-0039-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aarons GA, Sawitzky AC. Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychological Services. 2006b;3(1):61–72. doi: 10.1037/1541-1559.3.1.61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Antonakis J, House RJ. An analysis of the full-range leadership theory: The way forward. In: Avolio BJ, Yammarino FJ, editors. Transformational and charismatic leadership: The road ahead. Amsterdam: JAI; 2002. pp. 3–34. [Google Scholar]
  6. Ashford SJ. Feedback seeking in individual adaptation: A resource perspective. Academy of Management Journal. 1986;29(3):465–487. [Google Scholar]
  7. Ashford SJ. The feedback environment: An exploratory study of cue use. Journal of Organizational Behavior. 1993;14:201–224. [Google Scholar]
  8. Ashford SJ, Tsui AS. Self-regulation for managerial effectiveness: The role of active feedback seeking. Academy of Management Journal. 1991;34(2):251–280. [Google Scholar]
  9. Ashkanasy NM, Wilderom CPM, Peterson MF. Handbook of organizational culture and climate. Thousand Oaks: Sage; 2000. [Google Scholar]
  10. Atwater LE, Roush P, Fischthal A. The influence of upward feedback on self and follower ratings of leadership. Personnel Psychology. 1995;48(1):35–59. [Google Scholar]
  11. Atwater LE, Waldman DA, Ostroff C, Robie C, Johnson KM. Self-other agreement: Comparing its relationship with performance in the U.S. and Europe. International Journal of Selection and Assessment. 2005;13(1):25–40. [Google Scholar]
  12. Atwater LE, Wang M, Smither JW, Fleenor JW. Are cultural characteristics associated with the relationship between self and others’ ratings of leadership. Journal of Applied Psychology. 2009;94(4):876–886. doi: 10.1037/a0014561. [DOI] [PubMed] [Google Scholar]
  13. Atwater LE, Yammarino FJ. Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions. Personnel Psychology. 1992;45(1):141–164. [Google Scholar]
  14. Bass BM. Leadership and performance beyond expectations. New York: Free Press; 1985. [Google Scholar]
  15. Bass BM, Avolio BJ. Technical Report. Binghamton University, NY: Center for Leadership Studies; 1995. MLQ: Multifactor leadership questionnaire. [Google Scholar]
  16. Bass BM, Yammarino FJ. Congruence of self and others’ leadership ratings of Naval officers for understanding successful performance. Applied Psychology: An International Review. 1991;40(4):437–454. [Google Scholar]
  17. Buunk BP, Van Yperen NW. Referential comparison, relational comparisons, and exchange orientation: Their relation to marital satisfaction. Personality and Social Psychology Bulletin. 1991;17(6):709–717. [Google Scholar]
  18. Church AH. Managerial self-awareness in high-performing individuals in organizations. Journal of Applied Psychology. 1997;82(2):281–292. doi: 10.1037/0021-9010.82.2.281. [DOI] [PubMed] [Google Scholar]
  19. Cooke RA, Rousseau DM. Behavioral norms and expectations: A quantitative approach to the assessment of organizational culture. Group Organization Management. 1988;13(3):245–273. [Google Scholar]
  20. Cullen KL, Gentry WA, Yammarino FJ. Biased self-perception tendencies: self-enhancement/self-diminishment and leader derailment in individualistic and collectivistic cultures. Applied Psychology. 2015;64(1):161–207. [Google Scholar]
  21. Edwards JR. Alternatives to difference scores: Polynomial regression and response surface methodology. In: Drasgow F, Schmitt NW, editors. Advances in measurement and data analysis. San Francisco: Jossey-Bass; 2002. pp. 350–400. [Google Scholar]
  22. Ehrhart MG. Leadership and procedural justice climate as antecedents of unit-level organizational citizenship behavior. Personnel Psychology. 2004;57(1):61–94. [Google Scholar]
  23. Ehrhart MG, Schneider B, Macey WH. Organizational climate and culture: An introduction to theory, research, and practice. New York: Routledge; 2014. [Google Scholar]
  24. Farmer EMZ, Compton SN, Burns JB, Robertson EB. Review of the evidence base for treatment of childhood psychopathology: Externalizing disorders. Journal of Consulting and Clinical Psychology. 2002;70(6):1267–1302. doi: 10.1037//0022-006x.70.6.1267. [DOI] [PubMed] [Google Scholar]
  25. Fleenor JW, McCauley CD, Brutus S. Self-other rating agreement and leader effectiveness. Leadership Quarterly. 1996;7(4):487–506. [Google Scholar]
  26. Glisson C, James L. The cross-level effects of culture and climate in human service teams. Journal of Organizational Behavior. 2002;23:767–794. [Google Scholar]
  27. Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood K, Mayberg S, Green P. Assessing the Organizational Social Context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research Special Issue: Improving mental health services. 2008;35(1–2):98–113. doi: 10.1007/s10488-007-0148-5. [DOI] [PubMed] [Google Scholar]
  28. Hanser LM, Muchinsky PM. Work as an information environment. Organizational Behavior and Human Performance. 1978;21:43–62. doi: 10.1016/0030-5073(78)90038-7. [DOI] [PubMed] [Google Scholar]
  29. Harris MM, Schaubroeck J. A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. Personnel Psychology. 1988;41(1):43–62. [Google Scholar]
  30. House RJ, Dorfman PW, Javidan M, Hanges PJ, de Luque MFS. Strategic leadership across cultures: GLOBE study of CEO leadership behavior and effectiveness in 24 countries. Thousand Oaks: Sage Publications; 2013. [Google Scholar]
  31. Jourard SM, Landsman T. Healthy personality: An approach from the viewpoint of humanistic psychology. 4. New York: Macmillan; 1980. [Google Scholar]
  32. Keyton J. Communication and organizational culture: A key to understanding work experiences. Thousand Oaks: Sage Publications; 2010. [Google Scholar]
  33. Kruger J, Dunning D. Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology. 1999;77(6):1121–1134. doi: 10.1037//0022-3514.77.6.1121. [DOI] [PubMed] [Google Scholar]
  34. Litwin G, Stringer R. Motivation and organizational climate. Cambridge: Harvard University Press; 1968. [Google Scholar]
  35. London M. Leadership development: Paths to self-insight and professional growth. Mahwah: Lawrence Erlbaum Associates Inc; 2002. [Google Scholar]
  36. London M, Smither JW. Can multi-source feedback change perceptions of goal accomplishment, self-evaluations, and performance-related outcomes? Theory-based applications and directions for research. Personnel Psychology. 1995;48(4):803–839. [Google Scholar]
  37. Mabe PA, West SG. Validity of self-evaluation of ability: A review and meta-analysis. Journal of Applied Psychology. 1992;67(3):280–296. [Google Scholar]
  38. Martinko MJ, Gardner WL. The leader/member attribution process. Academy of Management Review. 1987;12(2):235–249. [Google Scholar]
  39. Mount MK. Psychometric properties of subordinate ratings of managerial performance. Personnel Psychology. 1984;37(4):687–702. [Google Scholar]
  40. Ostroff C, Atwater LE, Feinberg BJ. Understanding self-other agreement: A look at rater and rate characteristics, context, and outcomes. Personnel Psychology. 2004;57(2):333–375. [Google Scholar]
  41. Petkevičiūte N, Giedraitis A. Leadership skills formation in workgroup of the first level managers in manufacturing companies. Organizaciju˛ Vadyba: Sisteminiai Tyrimai. 2013;67:69–82. [Google Scholar]
  42. Podsakoff PM, MacKenzie SB, Podsakoff NP. Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology. 2003;88(5):879–903. doi: 10.1037/0021-9010.88.5.879. [DOI] [PubMed] [Google Scholar]
  43. Podsakoff PM, Organ DW. Self-reports in organizational research: Problems and prospects. Journal of Management. 1986;12(4):531–544. [Google Scholar]
  44. Priestland A, Hanig R. Developing first-level leaders. Harvard Business Review. 2005;83(6):112–120. [PubMed] [Google Scholar]
  45. Riggio RE, Cole EJ. Agreement between subordinate and superior ratings of supervisory performance and effects on self and subordinate job satisfaction. Journal of Occupational and Organizational Psychology. 1992;65(2):151–158. [Google Scholar]
  46. Schulz D. Growth psychology: Models of the healthy personality. New York: Van Nostrand Reinhold; 1977. [Google Scholar]
  47. Sedikides C, Gregg AP. Self-enhancement: Food for thought. Perspectives on Psychological Science. 2008;3(2):102–116. doi: 10.1111/j.1745-6916.2008.00068.x. [DOI] [PubMed] [Google Scholar]
  48. Shanock LR, Allen JA, Dunn AM, Baran BE, Scott CW, Rogelberg SG. Less acting, more doing: How surface acting relates to perceived meeting effectiveness and other employee outcomes. Journal of Occupational and Organizational Psychology. 2013;86(4):457–476. [Google Scholar]
  49. Shanock LR, Baran BE, Gentry WA, Pattison SC, Heggestad ED. Polynomial regression with response surface analysis: A powerful approach for examining moderation and overcoming limitations of difference scores. Journal of Business and Psychology. 2010;25(4):543–554. [Google Scholar]
  50. Sosik JJ, Megerian LE. Understanding leader emotional intelligence and performance: The role of self-other agreement on transformational leadership perceptions. Group and Organization Management. 1999;24(3):367–390. [Google Scholar]
  51. Swann WB, Read SJ. Self-verification processes: How we sustain our self-conceptions. Journal of Experimental Social Psychology. 1981;17(4):351–372. [Google Scholar]
  52. Taylor SE, Brown JD. Illusion and well-being: A social psychological perspective on mental health. Psychological Bulletin. 1988;103(2):193–210. [PubMed] [Google Scholar]
  53. Thornton GC. Psychometric properties of self-appraisals of job performance. Personnel Psychology. 1980;33(2):263–271. [Google Scholar]
  54. Tornow WW. Editor’s note: Introduction to special issues on 360-degree feedback. Human Resource Management. 1993;32(2–3):211–219. [Google Scholar]
  55. Tsui AS, Zhang ZX, Wang H, Xin KR, Wu JB. Unpacking the relationship between CEO leadership behavior and organizational culture. Leadership Quarterly. 2006;17:113–137. [Google Scholar]
  56. Yammarino FJ, Atwater LE. Understanding self-perception accuracy: Implications for human resource management. Human Resource Management Review. 1993;32(2–3):231–247. [Google Scholar]
  57. Zohar D, Tenne-Gazit O. Transformational leadership and group interaction as climate antecedents: A social network analysis. Journal of Applied Psychology. 2008;93:744–757. doi: 10.1037/0021-9010.93.4.744. [DOI] [PubMed] [Google Scholar]

RESOURCES