Ottawa Model of Implementation Leadership and Implementation Leadership Scale: mapping concepts for developing and evaluating theory-based leadership interventions

Abstract

Purpose

Leadership in health care is instrumental to creating a supportive organizational environment and positive staff attitudes for implementing evidence-based practices to improve patient care and outcomes. The purpose of this study is to demonstrate the alignment of the Ottawa Model of Implementation Leadership (O-MILe), a theoretical model for developing implementation leadership, with the Implementation Leadership Scale (ILS), an empirically validated tool for measuring implementation leadership. A secondary objective is to describe the methodological process for aligning concepts of a theoretical model with an independently established measurement tool for evaluating theory-based interventions.

Methods

Modified template analysis was conducted to deductively map items of the ILS onto concepts of the O-MILe. An iterative process was used in which the model and scale developers (n=5) appraised the relevance, conceptual clarity, and fit of each ILS items with the O-MILe concepts through individual feedback and group discussions until consensus was reached.

Results

All 12 items of the ILS correspond to at least one O-MILe concept, demonstrating compatibility of the ILS as a measurement tool for the O-MILe theoretical constructs.

Conclusion

The O-MILe provides a theoretical basis for developing implementation leadership, and the ILS is a compatible tool for measuring leadership based on the O-MILe. Used together, the O-MILe and ILS provide an evidence- and theory-based approach for developing and measuring leadership for implementing evidence-based practices in health care. Template analysis offers a convenient approach for determining the compatibility of independently developed evaluation tools to test theoretical models.

Background

Leadership is one of the essential conditions for organizational success and is increasingly being recognized as central to ensure positive employee and organizational outcomes.^1,2 Playing a critical role in influencing an organization’s capacity to implement change and innovation,^3–5 leadership in health care is instrumental to creating supportive organizational environments and positive staff attitudes for implementing evidence-based practices (EBPs) to improve patient care and outcomes.^6–9 For example, Gifford et al⁶ found that when leaders prioritized goals for change, engaged with staff to communicate a vision for EBP, and developed action plans to address predetermined barriers and facilitators to implementation, statistically significant increases were observed in EBPs for the care of diabetic foot ulcers in home care nursing. In a large-scale study of EBP implementation to reduce child maltreatment across two US states, leadership predicted sustained implementation and was shown to be different in sites with full, partial, and no sustainment.¹⁰

The leadership of the managers and supervisors who are accountable for staff and work directly with patients has been highlighted as critical to the success or failure of implementing practice change.^11–13 Yet, most are unprepared for their roles in implementing change and are often unaware of effective approaches and strategies for success.¹⁴ Despite growing research in the field, the mechanisms by which leadership influences implementation are not well understood. To stimulate and support additional empirical work in this area, it is essential to identify and understand the aspects of leadership that are critical for implementation, so that capacity-building interventions can be developed and tested to facilitate successful implementation of EBP change.

One way to increase the understanding of what might work or not work in the leadership process is to examine it from a theoretical perspective. Theory can facilitate an understanding of the complexity of factors that influence how and why EBP implementation succeeds or fails in health care.^15–18 Theories of implementation are essentially theories of planned change, providing an understanding of how and why implementation works.¹⁹ They generally describe the process of implementation or explain the factors that influence implementation and predict the planned change.^17,19 Most leadership theories are grounded in the following three perspectives: leadership as relationships, leadership as characteristics, and leadership as behaviors.²⁰ In virtually all of the more dominant theories, leadership is described as a dynamic and complex process that involves influencing a group toward the realization of goals. Implementation leadership has been defined as a multidimensional process of influence that enables clinical staff to use evidence in their clinical decision making and includes the activities and behaviors of unit-level managers and supervisors that influence staff, their environment, and the organizational infrastructures.²¹

Theories of implementation and planned change emphasize the importance of leadership in both the acceptance of new innovations, such as EBPs, and their adoption into practice.¹⁰ For example, Van de Ven et al’s⁴ theory of the innovation journey suggests that leaders can increase the chance of successful implementation by developing skills in leadership, relationships, and concrete strategies for facilitating the process, despite the innovation process being complex. Schein’s²² theory of organizational culture and leadership suggests that leaders influence change through their focus and priorities and by engaging with groups, addressing discrepancies between values and behaviors, and developing plans and strategies. Recent work on the Promoting Action on Research Implementation in Health Services (PARIHS) theoretical framework identifies leadership and management support as key factors of the inner organizational context for influencing the implementation of EBP in health care.²³ Although research has identified many leadership factors that influence implementation, their incorporation into a theory has been incomplete and insufficiently tested. An analysis of 31 planned action theories did not isolate leadership as an explicit construct; however, many of the activities described were consistent with leadership behaviors identified in implementation research, such as identifying the problem, planning the implementation, and evaluating the change.¹⁹ With a growing emphasis on the use of theory in research,^17,19,24,25 the need for theoretical models and frameworks on leadership for implementation is of growing value and interest. Still, there remain a lack of theoretical perspectives that pertain specifically to leadership for implementing EBP in health care and, further still, a lack of validated measures to evaluate the leadership concepts described in leadership conceptual models.

Purpose

The purpose of this study is to demonstrate the alignment of the Ottawa Model of Implementation Leadership (O-MILe) with the Implementation Leadership Scale (ILS) for developing and evaluating leadership interventions for implementing EBP.^11,25 A secondary objective is to describe the methodological process for aligning concepts of a theoretical model with an independently established measurement tool for evaluating theory-based interventions.

O-MILe

The Ottawa Model of Implementation Leadership (O-MILe) is a theoretical model based on leadership theory and empirical research that has been developed, revised, and tested over the past 10 years.^6,25–27 It was initially called the Gifford model and was developed in a qualitative grounded theory study that compared and contrasted the activities of nursing leaders in nine health care organizations (hospitals, home care, and long-term care) that successfully or unsuccessfully implemented and sustained the use of evidence-based recommendations from clinical practice guidelines.²⁷ Based on the analysis, leadership for successful implementation involved the following: 1) facilitating staff, 2) creating a positive climate, and 3) influencing organizational structures and processes. The first-generation Gifford model was then integrated with behavioral leadership theory and planned change theories, and a second-generation model was developed that explicated three meta-categories of effective leadership.^{19,20,26,28,29}

Supported by decades of research,^30–32 the three meta-categories of effective leadership are as follows: 1) relations-oriented, 2) change-oriented, and 3) task-oriented leadership behaviors. Relations-oriented behaviors include supporting, developing skills, and recognizing others and their contributions to increase mutual trust, cooperation among members, and commitment to a unit and organization. Change-oriented behaviors are concerned with integrating a vision, demonstrating commitment, building coalitions to support change, and creating a sense of need. Task-oriented behaviors include planning, clarifying roles, monitoring operations and performance, and efficiently using resources.³² The leadership process hypothesized in the second-generation model influences individuals, the practice environment, and the organizational infrastructure to address barriers and implement EBP.

Based on the second-generation Gifford model, a leadership intervention was developed and piloted in a mixed-methods study with an embedded cluster randomized control trial.³² The purpose of the intervention was to develop a leadership process among frontline managers and clinical leaders to implement evidence-based nursing care for diabetic foot ulcers. The leadership intervention showed promise, with statistically significant increases in EBPs observed in the intervention group.⁶ In the intervention, managers and clinical leaders prioritized which practices needed to change based on evidence/practice gaps in care, and developed detailed leadership action plans with measurable short-term goals of change. Qualitative data expanded the findings and showed that leaders in the intervention group used more relations- and change-oriented leadership behaviors than leaders in the control group. Specific strategies included communicating more with staff, recognizing efforts, demonstrating commitment, and reinforcing goals for change. These findings were then integrated into the third-generation Gifford leadership model.⁶ However, results showed there was still a need for further refinement on how to develop implementation leadership.

In 2013, a group of implementation science researchers and organizational decision makers from the US, Sweden, and Canada convened in Ottawa, ON, for a 3-day planning meeting to clarify and refine the key components of a leadership intervention for implementing EBP. Using principles from the Medical Research Council Complex Interventions Framework,³³ tacit knowledge from the experiences of organizational decision makers was synthesized with findings from the broader research on leadership and implementation science to identify components of an intervention for developing implementation leadership that were salient to both researchers and organizational decision makers. Combined with relations-, change-, and task-oriented leadership behaviors from previous work,^6,26 a fourth-generation model was developed²⁵ and revised as the O-MILe (Figure 1).

Figure 1.

The O-MILe.

Abbreviation: O-MILe, Ottawa Model of Implementation Leadership.

In addition to specifying the leadership behaviors identified in our previous research,⁶ the O-MILe explicates the knowledge and skills that are necessary for leaders to conduct the behaviors and facilitate implementation. The O-MILe postulates that for successful implementation, front-line leaders require the knowledge of effective leadership practices, site-specific evidence-practice gaps, implementation strategies, and planned change processes, including barriers management. Leaders must additionally have skills to prioritize change, set goals and target outcomes, engage patients and staff, and develop an implementation plan. The knowledge and skills portrayed in the O-MILe are described as the key components of a leadership intervention for participants to develop relations-, change-, and task-oriented leadership behaviors for implementing EBP in health care settings.

Theoretical models are graphic or narrative representations of concepts that infer relationships between components, thereby depicting “what” is involved and “why” and “how” a specific phenomenon works.³⁴ Incorporating primary research with theoretical and empirical evidence from leadership and planned action theories, the O-MILe provides a theoretical foundation for developing and operationalizing unit-level leadership to implement EBP in health care settings, identifying both the knowledge and skills required to develop implementation leadership and the specific behaviors required for practice. Concepts within the O-MILe are consistent with research showing that leaders who focus on individualized consideration³⁵ are proactive and present,³⁶ and address barriers to change have a greater capacity for successfully implementing change.^28,37,38

ILS

The Implementation Leadership Scale (ILS) was developed by a US-based investigative team with support from the US National Institute of Mental Health. The development of the ILS was conducted completely independent from the O-MILe. The ILS is an empirically validated scale that measures unit-level leadership for implementing EBPs. Drawing on strategic climate and leadership theories,^22,39 an initial set of items were developed and tested using exploratory and confirmatory factor analyses with 459 mental health clinicians working in outpatient programs in the US. A 12-item scale of implementation leadership was supported with four subscales that represent proactive leadership, knowledgeable leadership, supportive leadership, and perseverant leadership. The scale demonstrated excellent internal consistency reliability as well as convergent and discriminant validity.¹¹ The ILS has also been validated in child welfare and substance abuse treatment settings.^10,40 While not explicating general leadership behaviors that support implementation, the ILS identifies facets of leadership that are specifically tied to implementation and may be critical to achieve positive implementation results.

Methods

Modified template analysis was used to conceptually map items of the ILS onto concepts of the O-MILe.⁴¹ Template analysis is a technique for structuring and analyzing qualitative data and involves creating a “template” of predefined categories that summarize important themes of the data set and deductively coding data onto the template categories. It can be used to analyze any form of textual data from most methodological and epistemological positions.⁴¹ For this study, concepts of the O-MILe (leadership knowledge, skills, and behaviors) acted as the template and items of the ILS were the textual data. Once the coding template was defined, ILS items were deductively coded or mapped onto the O-MILe template categories.

ILS items were mapped onto all O-MILe categories they corresponded to, and O-MILe categories that did not correspond to any ILS data were left blank. An iterative process was used in which the O-MILe developers (WG, IDG, and BLD) and ILS developers (GAA and MGE) appraised the relevance, conceptual clarity, and fit of each ILS items with the O-MILe concepts through individual feedback and group discussions. Particular attention was paid to “goodness of fit”, and interpretation was managed by corroboration between originators of the two models until consensus was reached.⁴¹ The relevance of this approach was to demonstrate how items of the ILS align with concepts of the O-MILe, a theoretically based conceptual model for developing and operationalizing implementation leadership.

Findings

All ILS items mapped to at least one O-MILe concept, and all but four O-MILe concepts (77%) corresponded to an ILS item (Table 1). With knowledge and skills considered an essential “ingredient” of an intervention for leadership development, all three items of the knowledgeable leadership subscale of the ILS corresponded to the O-MILe concept “develops knowledge of EBP”. The ILS item “develops a plan to facilitate implementation” maps directly to the O-MILe concept “develops an implementation plan”. However, the O-MILe is more specific about what knowledge and skills are required to develop a plan, such as knowledge of current practices in the setting where implementation is to occur and effective implementation methods. While the ILS implies some level of these knowledge and skills, it does not specifically identify them. Rather, it focuses on the behaviors performed during the implementation and not the prerequisite knowledge or skills to perform them. The ILS item “develops a plan” also maps onto task-oriented leadership behaviors of the O-MILe, specifically “clarifies roles and responsibilities” and “modifies documentation forms”, as these behaviors are considered part of an implementation plan. “Leadership theory”, which is part of the core knowledge required for developing leadership in the O-MILe, is not addressed in the ILS. However, investigators of the ILS found that general leadership knowledge was foundational to developing implementation leadership, in particular knowledge on both transformational and transactional leaderships.³⁶ All the relations-oriented leadership behaviors in the O-MILe aligned with items in the ILS – two ILS items from the supportive leadership category and one from the knowledgeable leadership category. Two of these items also mapped onto other O-MILe concepts. For example, the ILS item “is able to answer my questions about evidence-based practice” mapped to both the relations-oriented category of the O-MILe (communicates with staff about clinical practice issues and evidence) and the core knowledge category (knowledge of evidence-based practice).

Table 1.

Mapping the ILS onto the O-MILe

O-MILe category	O-MILe concepts (n=17)	ILS item (n=12)
Core knowledge and skills	• Knowledge of leadership theory
	• Knowledge of evidence-based best practice	• Is knowledgeable about EBP • Knows what he/she is talking about when it comes to EBP • Is able to answer my questions about EBPa (knowledgeable leadership)
	• Develops an implementation plan • Knowledge of current practices and outcomes • Knowledge of effective implementation methods • Engages clinicians, staff, and patients • Sets target goals for change	• Has developed a plan to facilitate implementation of EBPa (proactive leadership)
Relations-oriented leadership behaviors	• Recognizes efforts to change	• Recognizes and appreciates employee efforts toward successful implementation of EBPa (supportive leadership)
	• Supports change visibly and symbolically • Facilitates interprofessional consensus on change	• Supports employee efforts to use EBP (supportive leadership)
	• Communicates with staff about clinical practice issues and EBP	• Is able to answer my questions about EBPa (knowledgeable leadership)
Change-oriented leadership behaviors	• Demonstrates commitment to change • Reinforces vision and goals of change	• Perseveres through the ups and downs of implementing EBP • Carries on through the challenges of implementing EBP (perseverant leadership)
	• Understands and acts on difficulties with change	• Reacts to critical issues regarding the implementation of EBP by openly and effectively addressing the problem(s) (perseverant leadership) • Removes obstacles to the implementation of EBP (proactive leadership)
	• Advocates for change internally and externally
Task-oriented leadership behaviors	• Clarifies roles and responsibilities • Modifies documentation forms	• Has established clear department standards for the implementation of EBP • Has developed a plan to facilitate implementation of EBPa (proactive leadership)
	• Procures resources, education, and policies to reflect change	• Supports employee efforts to learn more about EBP (supportive leadership)
	• Monitors performances and outcomes	• Recognizes and appreciates employee efforts toward successful implementation of EBPa (supportive leadership)
	• Provides reminders
	• Conducts regular leadership meetings

Note:

^aILS item mapped onto more than one concept.

Abbreviations: EBP, evidence-based practice; ILS, Implementation Leadership Scale; O-MILe, Ottawa Model of Implementation Leadership.

Change-oriented leadership behaviors in the O-MILe are addressed with perseverant and proactive leadership items in the ILS. However, some change-oriented concepts do not explicitly map with ILS items. For example, “demonstrates commitment” and “reinforces vision and goals for change” are implicit within the perseverant items of the ILS. The O-MILe change-oriented leadership behavior “advocates for change internally and externally” did not align with any ILS item as the ILS focuses specifically on what leaders do to influence the immediate team and not people outside or external to the team.

Four of the six O-MILe concepts in the task-oriented leadership category are addressed by ILS items. The ILS item “recognizing and appreciating employee efforts” is captured indirectly in the O-MILe concept “monitors performances and outcomes” as a certain level of monitoring is assumed to be required to “recognize and appreciate employee efforts”. Two concepts from the task-oriented behavioral category of the O-MILe are not reflected in the ILS, and this is attributed to a difference in the degree of specification for each of the tools. For example, the O-MILe task behavior of “conducting regular leadership meetings” can be an antecedent to the ILS item “has developed a plan to facilitate implementation of EBP” and “providing reminders” in the O-MILe can be part of a targeted “implementation plan” in the ILS.

Discussion

Insights from mapping items from the ILS onto concepts of the O-MILe confirm consistencies in the evidence on implementation leadership and offer directions for developing and evaluating theory-based interventions. Findings extend the face validity of the O-MILe and confirm compatibility of using the ILS as a tool to measure leadership knowledge, skills, and behaviors developed from the O-MILe, a theory-based leadership intervention. All ILS items (100%) mapped onto O-MILe, with 13 of 17 (77%) O-MILe concepts corresponding to at least one ILS items, many of them directly, such as the behaviors “recognizing” and “supporting”. Other items mapped more indirectly, using different words but capturing parallel concepts, such as the change-oriented behaviors of “demonstrating commitment” and “reinforcing vision and goals for change” that mapped to “perseverant leadership” items of the ILS, albeit with greater specification and detail. A commitment to change can be considered an antecedent to perseverance, and research on sustainability confirms a relationship between leadership commitment and implementation over time.^42–45 Transformation leadership theory further confirms that the bilateral concepts of perseverance and commitment to a higher level of performance (in this case effective patient care) are interconnected, while aspects of transformational leadership mediate a leader’s ability to foster staff support for innovation and excellence.^46–48 As indicated by their alignment, leaders who are perseverant in confronting obstacles to change demonstrate commitment and reinforcement of the EBP change, both of which are reflected in the ILS and the O-MILe.

In addition to transformational leadership that involves relations- and change-oriented behaviors to inspire and guide change for a shared vision,^2,20,35 transactional leadership is important to the introduction and acceptance of new innovations in organizations.⁴⁹ Transactional leadership is largely reflected in the task-oriented behaviors of the O-MILe that illustrate a leader’s role in providing resources and support for employees. Providing resources and support are equally represented in the ILS proactive and supportive leadership items. While not assumed to be linear, the O-MILe concepts of “clarifying roles and responsibilities”, “procuring resources, education, and policies”, and “modifying documentation forms” are consistent with “establishing clear departmental standards and supports for the EBP” as identified in the ILS.

A greater degree of detail and specificity, not captured in the ILS, is reflected in some of the O-MILe leadership concepts. For example, the ILS item “develops a plan to facilitate implementation of EBP” mapped onto a number of O-MILe concepts that are not represented in the ILS because they are considered prerequisite knowledge or more specific components of the implementation plan, such as clarifying roles and responsibilities. These differences are partly due to the focus of the ILS on implementation at the microsystems team level, whereas the O-MILe includes change management strategies at both the microsystems team level in addition to the organizational level (eg, advocates for change internally and externally). The methods that have been used in the development of the O-MILe and ILS offer a plausible explanation for these differences in specificity. With the first generations of the O-MILe developed through qualitative methods and a cluster randomized control trial,^6,27 the concepts are more contextually detailed describing leadership behaviors that occurred within specific cultural and contextual settings. Whereas the ILS has undergone extensive factor analysis testing with large samples, therefore the emergence of higher ordered latent concepts were able to occur through greater merging of dimensions and concepts. As testing continues with the O-MILe, similar merging may occur resulting in analogous latent concepts and a more parsimonious O-MILe.

The use of template analysis to combine theoretical concepts with psychometrically validated measurement tools presents a novel approach for researchers to confirm that they are capturing key components of a theory-based intervention in their evaluation. This process is also useful to ensure that the evaluation of an intervention is grounded theoretically and the epistemological position is compatible with the measurement techniques being used. Mapping concepts of the O-MILe with the ILS outlines and reinforces the dynamic nature of implementation leadership and illustrates the necessity of unit-level leaders (ie, managers and supervisors) to strategically use a full range of transparent leadership behaviors to achieve EBP as a norm. Leading successful implementation of EBP is complex and multifaceted, and both the O-MILe and the ILS reflect that leaders must use behaviors and approaches that are strategic (change oriented), relational (relations oriented), and functional (task oriented).

While recognizing that the sample size was small, a strength of this study was that the original developers of both the O-MILe and the ILS were involved in the conceptual mapping process and agreed upon the meanings and alignment of concepts and items. However, this is also a limitation because of the inherent bias this approach brings to fit the tools together compatibly. Future research is required with the O-MILe and ILS to test the practical application and validate the alignment of concepts within the tools for intervention development and evaluation.

Another limitation of this study is that both the O-MILe and the ILS are directed toward leadership at the service unit level and have not been tested at a more macro health systems level where problems are more complex, inherently political, and do not typically have evidence-based solutions.⁵⁰ Our findings fit well for microsystems implementation approaches where EBPs have been shown to be effective. However, leadership at senior organizational and systems levels may require different considerations and approaches that cannot be addressed in a single model such as the O-MILe. Furthermore, while both the O-MILe and the ILS were developed and tested in North American settings, concepts may not be generic to other countries and cultures, particularly in countries that show different expressions of individuality, social desirability, and assertiveness.⁵¹ Work is currently in progress using the O-MILe and ILS together to develop and evaluate leadership interventions in Canada and China.

Conclusion

The O-MILe provides a theoretical basis for developing implementation leadership, and the ILS provides an empirically validated tool for measuring the same. The use of template analysis to combine concepts of a theoretical model with an independently developed evaluation tool presents an approach for researchers and decision makers to develop and test interventions with compatible tools. Mapping has affirmed compatibility between the O-MILe and the ILS and provides insights into the leadership behaviors and mechanisms that promote the implementation of EBPs. We suggest that used together, the O-MILe and ILS provide an evidence- and theory-based approach for developing and evaluating leadership for implementing EBPs in health care.