Leadership Perspectives on Operationalizing the Learning Health Care System in an Integrated Delivery System

Abstract

Introduction:

Healthcare leaders need operational strategies that support organizational learning for continued improvement and value generation. The learning health system (LHS) model may provide leaders with such strategies; however, little is known about leaders’ perspectives on the value and application of system-wide operationalization of the LHS model. The objective of this project was to solicit and analyze senior health system leaders’ perspectives on the LHS and learning activities in an integrated delivery system.

Methods:

A series of interviews were conducted with 41 system leaders from a broad range of clinical and administrative areas across an integrated delivery system. Leaders’ responses were categorized into themes.

Findings:

Ten major themes emerged from our conversations with leaders. While leaders generally expressed support for the concept of the LHS and enhanced system-wide learning, their concerns and suggestions for operationalization where strongly aligned with their functional area and strategic goals.

Discussion:

Our findings suggests that leaders tend to adopt a very pragmatic approach to learning. Leaders expressed a dichotomy between the operational imperative to execute operational objectives efficiently and the need for rigorous evaluation. Alignment of learning activities with system-wide strategic and operational priorities is important to gain leadership support and resources. Practical approaches to addressing opportunities and challenges identified in the themes are discussed.

Conclusion:

Continuous learning is an ongoing, multi-disciplinary function of a health care delivery system. Findings from this and other research may be used to inform and prioritize system-wide learning objectives and strategies which support reliable, high value care delivery.

Introduction

Health care systems are under increasing pressure to deliver high value care. Several conditions are contributing to this state, including a shift toward value-based payment, greater transparency of quality and outcomes of care, and advances in Health Information Technology. To navigate this changing landscape, health care leaders need operational strategies that support continuous learning and integration of clinical data and analytics that translate to improvement and value. In 2007, the Institute of Medicine (IOM) proposed the Learning Health System (LHS) model, which seeks to integrate organizational learning across clinical, operational, and research functions of health systems to support improved quality and high value care.1,2 Building from the IOM’s definition, organizations operationalizing the LHS model may provide leaders with strategies to navigate the current, changing landscape in health care. However, little is known about leaders’ perspectives on the value and application of systemwide operationalization of the LHS model.

A number of organizations have begun to use the LHS model to organize for learning although application of the model remains largely conceptual.3–8 Senior leadership support and operational-research partnerships have been identified as key components of successful LHS activity,9–15 yet little has been published to provide guidance on important operational issues and strategies for systemwide learning and operationalization of the LHS model.16–18 A recent survey of leaders and managers from organizations involved in patient-centered network programs funded by the Patient-Centered Outcomes Research Institute (PCORI) recognized that research had a beneficial role to play in care delivery, but discussed a number of challenges with integrating research into clinical practice.19 While learning is contextual, i.e., is highly influenced by the organizational and individual characteristics in which it occurs, leaders across health care systems could benefit from a greater understanding of the challenges and opportunities surrounding learning from their peers.

In 2013, a multidisciplinary group at Geisinger Health System (Geisinger) began working to understand the potential value of the LHS model to Geisinger and to identify potential mechanisms to operationalize an LHS. The Geisinger LHS (GLHS) group has since expanded its membership to reflect the scope of learning across the system. The initial goal of the group was to develop a framework for operationalizing the LHS at Geisinger. The group developed a nine-domain framework focusing on data and analytics, people and partnerships, patient and family engagement, ethics and oversight, evaluation and methodology, funding, organization, prioritization, and deliverables. Details of the LGHS group and operational framework are reported elsewhere.7 Our vision for operationalizing the LHS was at the system level, i.e., to integrate and align learning to continuously improve quality across the entire health care system.

The group recognized early in the process that since the LHS model was largely conceptual with limited practical application in system transformation, applied tools and education would be necessary to engage leadership and staff. The framework was therefore developed with the intention of linking the conceptual LHS model to its application in a delivery system. The framework also served as a tool to socialize the LHS model within our health system. Therefore when engaging leadership in implementing a LHS, the group could already point to a tangible framework that could be used to operationalize the LHS concept.

The operationalization framework formed the core of the GLHS group’s materials for engaging leaders and staff. Materials developed by the GLHS group included a white paper and supporting Microsoft PowerPoint presentation that described the IOM’s LHS model, the rationale for applying the model at Geisinger, and the operational framework. These materials were used in engaging leadership and staff and in the project described in this paper.

In this publication, we report on findings from a project that sought to gather senior health-system leaders’ perspectives on the LHS, our group’s operational framework, and operational strategies to support organizational learning. Our findings provide insight into how leaders of a large, integrated delivery system approach learning and the LHS model, and how this can be used to inform operational strategies for integrating continuous learning into care delivery within health care organizations.

Methods

This project was undertaken at Geisinger, an eight-hospital integrated delivery system with 23,500 employees. In 2015 the system provided over 70,000 inpatient and 2.6 million outpatient visits to patients across 45 counties in central, south-central, and northeast Pennsylvania.20 The system has made considerable investments in Health Information Technology over the last decade and has a robust enterprise data warehouse to support clinical operations and research.

The project was part of the GLHS group’s operational activities to understand and improve organizational learning. The GLHS collectively determined that the project was operational activity to improve the health system and was therefore not designated as a research study. This determination included discussion with members of research oversight who have a deep understanding of the LHS model and the GLHS group’s work. This project reflects the type of learning activity supported by the LHS in which operations generate learning (leadership perspectives to inform organizational learning strategy), utilizing adapted evaluation or research methodologies that are adapted to the context of the setting and the nature of the clinical or operational task.

A series of in-depth, key-informant interviews were held with senior system leaders. Interview candidates were identified and selected through consensus agreement of the multidisciplinary GLHS group members. The group’s project manager established a master list of candidates and tracked scheduling. The leaders represented a mix of functional areas including clinical, administrative, and research areas at the corporate (system level) and hospital levels. Leaders interviewed were at the executive officer (e.g., CEO), senior vice president, vice president, chair, or director levels. Senior clinical leaders represented different specialties (including medicine, surgery, and pediatrics) and clinical settings (hospital and outpatient).

Interviews were structured around background material on the LHS to facilitate an open discussion on leaders’ perspectives of the LHS model and operational framework. This material consisted of a white paper and Microsoft PowerPoint presentation prepared by the GLHS group. The white paper provided background information on the IOM’s LHS model1,2 and presented an overview of the operationalization framework developed by the GLHS group.7 The Microsoft PowerPoint presentation accompanied the white paper to visually present the LHS model and operationalization framework.

The white paper and Microsoft PowerPoint presentation were distributed to the first 14 interviewees prior to their interviews; however the team became concerned that the white paper was too detailed and limited review by time-constrained interviewees. We therefore developed a two-page summary of the white paper that was distributed to the remainder of interviewees, together with the presentation, a minimum of two days before an interview. The group decided to provide the materials in advance, so that interviewees were aware of the LHS model (background, rationale, and characteristics), the potential value to Geisinger, and how we intended to operationalize the model through the framework. By providing this information we were able to spend more time gaining their perspectives than on educating them on the LHS model and operational framework. The interviewees viewed the materials as informative and, based on the candid discussions we had, we do not feel that the materials biased responses. In fact responses were surprisingly candid in praise and criticism of the system’s learning structure. The white paper, presentation, and other supporting documents (including IOM LHS reports and meeting summaries) were forwarded to interviewees after the interview in a thank you email.

Among the core group, three team members were selected to conduct the interviews, each of which was conducted by two or more interviewers; however, the vast majority of the interviews (90 percent) were conducted by all three interviewers. All three interviewers had knowledge and experience with the LHS through their involvement in the GLHS group and learning activities within their work in the health system. Two of the members have doctoral level training and the third is an experienced senior level administrator. Interviews were 30 minutes to 1 hour in length. Due to the sensitive nature of this work (eliciting candid responses on the state of learning and perceptions of practices of different departments and functional areas related to learning), we did not record conversations. Notes were taken during the interviews by at least one interviewer.

At the beginning of each interview, respondents were informed of the purpose of the meeting. We also informed the respondent of other leaders that had been interviewed, so as to orient the leader to the level of leadership that was involved in the project interviews. Respondents were given the option of being walked through the Microsoft PowerPoint presentation or proceeding directly to discussion. Interviewees electing to go through the presentation often offered comments and generated discussion during the presentation, while others waited until completion of the presentation. Once the presentation was completed the interviewers would initiate discussion (1) by asking for the interviewee’s reaction to the presentation: “Do you have any feedback for our group or questions on the LHS and operational framework?”; and (2) by asking for the interviewee’s perception on learning in the system: “What challenges or opportunities exist around learning in the health care system?” Interviewers met to discuss the interview after its completion, and themes were collectively identified, discussed, and agreed upon.

Findings

In total, 41 senior leaders were interviewed. The majority of interviews (38) were held in person, while 3 were held over the phone. Two potential candidates declined interviews and one candidate canceled and did not reschedule.

All leaders expressed support for the general concept and goals of the IOM’s LHS model and for enhancing learning across the institution. Leaders were also receptive to the operational model. A few interviewees expressed reservation regarding the system’s ability to fully move from concept to application due to the limited application of the LHS model across other health care organizations and limitations on resources available for learning in the system. Other leaders encouraged further development of the operational framework including specific implementation structure and activities and were fully supportive of the GLHS activities.

The project team identified 10 major themes that recurred throughout the interviews with system leaders (Table 1.). Not all themes were raised by all 41 leaders, many of whom held opposing views— especially across functional areas. Although the themes are discussed in this paper in separate sections, leaders often perceived elements of the themes as being closely related or overlapping. For example, aligning incentives for learning may include a need or requirement for greater measurement of clinical outcomes. Likewise, current incentives for learning across operational areas may inadvertently contribute to cultural and operational silos.

Table 1.

Key Themes on Learning and the Learning Health System (LHS) Raised by System Leaders

THEME
1. Alignment of learning with system strategic goals;
2. Alignment of learning with incentives;
3. Integrating cultural and operational silos;
4. Balancing learning and work flow;
5. Shifting the focus of learning from process improvement to improving outcomes;
6. Addressing challenges in current health care environment that have an impact on learning;
7. Balancing the need to execute and evaluate operational activities given limitations of evaluation methodologies;
8. Supporting “make-or-buy” decisions for learning;
9. Oversight of the research–quality improvement (QI) continuum; and
10. Determining the costs and value of learning.

1. Alignment of Learning with System Strategic Goals

Executive leaders consistently stressed the need to align the learning infrastructure and LHS activities in support of the system’s strategic goals. Concerns and suggestions related to operationalization of the LHS model in the health system were strongly aligned with the strategic goals of interviewee’s functional area (operations, research, finance, clinical care, or quality and safety). For example, one senior finance leader was most concerned with costs and financial value of the LHS model, while several clinical leaders were more focused on incentivizing learning within the clinical environment and evaluating implementation and clinical outcomes. Leaders were also more willing to support the LHS model and activities when they aligned with an interviewee’s specific operational goals—such as implementing or evaluating clinical programs, or enhancing the research-QI oversight pathway.

Leaders had concerns when presented with the narrative of using the LHS for system transformation as described in the IOM LHS model. Some leaders considered the LHS model a challenge or a distraction from the current system strategy. Once we realized that this was occurring, we no longer used the IOM “system transformation” narrative and focused instead on how the LHS model might support, rather than change, the existing organizational strategy.

This lesson indicates that the LHS model has dual applications. The first application is as a high level (e.g., system level) strategy with the purpose of transforming or changing learning and care delivery in an organization. The second purpose is as a supporting strategy, which supplements an existing organizational strategy.

Practically, those considering using the LHS model within their institution should clearly define at the outset whether the LHS and learning will be integrated into the system as a primary strategy or in support of its existing strategic plan. Based on this decision, practitioners can then shape the narrative for continuous learning to communicate clearly the LHS model and the role of learning in the organization, and can tailor communications toward specific leaders and employees.

2. Alignment of Learning with Financial Incentives

The need to align learning with employee incentives was raised by clinical, research, and operational leaders. Most leaders did not perceive the current incentive structure in place as being amenable to support the type of learning environment and activities proposed in the LHS model. The incentive structure in place at the time of the interviews was based on productivity (Relative Value Units (RVUs)), QI activities, and a research component for some. The compensation package for all employees did not include a component specifically focused on organizational learning.21

The need to restructure incentives to reward learning activities was expressed in terms of both financial incentives and working conditions. It was suggested that changing financial incentives to include learning might generate greater equity between efficiency and learning. For example, providers might be compensated for establishing dedicated learning activities with their clinical teams (such as daily debriefs or teaching and learning sessions). These types of learning activities are typically compensated only when utilized in formal QI or organizational redesign projects. Another example of potential learning activities outside of clinical care that could be compensated are those that contribute to the implementation or scaling of QI, or innovation activities that contribute to the organization (system or hospital) as a whole, and again are not specifically focused on formal QI or research activities. To elevate the importance of learning in an organization, compensation for learning activities should be a separate area of compensation, apart from productivity, QI, and research activities. Finally, some leaders also indicated the need for dedicated time for learning activities in parallel with financial incentives. Such an approach would require a culture that embraced learning as part of employees’ work and restructuring work flow to incorporate learning activities.

3. Integrating Cultural and Operational Silos

Leaders consistently cited operational silos as being barriers to learning. Cultural differences across various units and departments—especially between clinical operations, QI, innovation, and research—were seen as the most important underlying cause. These included differences in how groups approached problems, consideration of time in solving problems, and the reward systems across disciplines. For example, clinical operations implementing a QI project are rewarded for improving a process or outcome, or for lowering costs, while a researcher is compensated based on producing generalizable knowledge such as a publication. Additionally power, politics, and personalities across functional areas were also identified as barriers to shared learning.

Leadership recognized that they could play an active, intermediary role in building and bridging relationships and can be purposeful in developing a multidisciplinary approach, which can span current structural and cultural barriers. As a best practice, some leaders suggested identifying differences in approaches to learning and needs across platforms (hospitals), service lines, and professional groups and sharing them with participants in developing learning strategies or activities. Finally, leaders should reassess the impact on organizational learning when key personnel or organizational changes occur and should identify improvements or additional barriers between different cultural or operational groups.

4. Balancing Learning and Work Flow

Among operational leaders, there was a strong reinforcement of limiting any additional work burden that might arise from operationalization of learning or additional learning activities. There was strong support for organizing learning to make work more efficient and effective. Operational leaders expressed a desire for learning activities that both matched the rapid-cycle time of operations and provided evaluation approaches that did not impede workflow. Operational leaders raised the need to institutionalize and budget for research within operations, especially in evaluating implementation and identifying risk within operational projects.

5. Shift the Focus of Learning from Process Improvement to Improving Outcomes

Leaders expressed a desire to structure learning activities that struck a greater balance between the evaluating and improving of care outcomes and the measuring of processes of care. While developing and evaluating innovative care processes is of high priority within the system, leaders saw value in structuring learning to focus on the outcomes of care delivery. This view was coupled with a desire for rigorous evaluation of clinical interventions and programs so that learning could be generalized to the internal and external clinical community. This theme was closely aligned with that of increasing incentives for learning.

6. Addressing Challenges in Current Health Care Environment that Have an Impact on Learning

Leaders perceived the current financial and regulatory environment as a major barrier to systemwide operationalization of the LHS, with limited incentives and resources available for systemwide learning. While learning was appreciated by all leaders, the practical value of systemwide learning was not clear to some and was perceived only through the leader’s functional lens. Leaders also identified several practical challenges to implementing the LHS including competing strategic and operational priorities, limited resources, and changes in senior leadership positions that were occurring in the system. The challenges expressed by leaders highlighted the importance of timing to LHS operationalization and learning activities and reinforced the need to align learning with strategic and operational goals (Theme 1).

7. Balancing the Need to Execute and Evaluate Operational Activities Given Limitations of Evaluation Methodologies

Interviewees, especially operational leaders, consistently expressed a tension between efficient execution of operational activities and their evaluation. While operational leaders recognized the value of rigorous evaluation of their work, several expressed frustration over limitations in current evaluation methodologies. Three main issues were expressed regarding these methods. First, the timelines for operational implementation and research were often incongruent, with operational activities requiring rapid implementation that current evaluation capacity could not match. Second, operational leaders were unfamiliar or uncomfortable with using certain evaluation methodologies, especially measurement of implementation. Third, operational activities often did not include, or underbudgeted for, evaluation—limiting managers’ ability to evaluate outcomes. Cultural and communication barriers between clinical operations and researchers (Theme 3) were closely associated with these concerns.6,7

Leaders might address these concerns by adopting supportive prioritization of evaluation in operational activities and building capacity for such evaluation for both managers and researchers. This may include providing operations with tools and training to better understand the types and application of evaluation methodologies. Researchers may support this process by building capacity to evaluate operational work, including implementation, and applying methodologies that are flexible and pragmatic to fit the needs of operations.

8. Supporting Make-Or-Buy Decisions for Learning

The “make-or-buy” (also referred to as “build-or-buy”) theme surrounding learning centered on two areas of concern for leaders. The make-or-buy decision in the context of the LHS centered on the leaders’ decision on whether to build an application or learning tool in house, or to purchase the product from a vendor. These types of decisions often created a tension between functional groups— e.g., clinical operations, innovation, Information Technology (IT) departments—who might disagree on the functionality or quality of the IT application and the system’s capacity to build the product. The second issue raised regarding the make-or-buy theme was leadership’s concern regarding the ability to hire, train, and retain talent—especially in technical areas, and evaluation of implementation and learning.

Addressing this theme, leaders expressed the need for a realistic assessment of the system’s current learning environment, including learning capacity and capabilities. This knowledge can be used to identify strengths and gaps in learning and to inform learners of whether to seek resources and learning support internally or externally. For managers, understanding their learning capabilities and need can inform budget decisions, especially for learning activities involving technology that may be capital intensive.

9. Oversight of the Research–Quality Improvement (QI) Continuum

Interviewees were supportive of learning activities that might integrate research and QI oversight and that might enhance transparency, efficiency, and clarity surrounding research and QI oversight. The level of comfort with research-QI oversight by leaders was highly related to their functional area. Research and clinical innovation leaders, who had constant interaction with oversight, were less concerned with this issue than were clinical and operational leaders. This theme was also closely aligned with the need to incentivize learning (Theme 2) and the need to break cultural and operational barriers (Theme 3).

At Geisinger closer integration of the research-QI continuum oversight is being addressed through formation of a multidisciplinary group consisting of representatives from the quality and safety, academic affairs, research, bioethics, research oversight, and clinical departments. The group seeks to develop a common understanding and risk-based approach to QI and research activities, in order to streamline the efficiency of oversight and learning across the system.

10. Determining the Costs and Value of Learning

Leaders expressed the importance of not adding additional costs to the health system through operationalizing the LHS. Since the model is relatively new, and there is limited evidence describing the financial impact of the LHS model, leaders–particularly those with finance roles—were wary of the ability of the LHS model to increase return on investment and decrease costs. This theme was closely related to that of make-or-buy decisions (Theme 8) and was especially challenging for clinical and operational leaders whose budgetary priorities did not support additional spending for learning activities.

Since the value of learning may not be reflected in return on investment, leaders need to be willing to support nonfinancial measures of value such as improved clinical outcomes and patient experience, or employee well-being. Leaders may also want to encourage and reward operational and clinical departments that seek funding for learning, especially unconventional sources of funding, including clinical operations that partner with researchers to fund interventions and activities that bring value (financial or nonfinancial) to the organization.

Discussion

The objective of this project was to solicit and analyze senior health-system leaders’ perspectives on the LHS and learning activities in an integrated delivery system that informs operational strategies to support learning. The practical application of the themes for developing operational strategies for learning are discussed below.

The themes discussed above reflect practical operational challenges that are likely experienced by other health systems and are consistent with Johnson et al.’s leadership surveys20 and other work in this area.6–8,16–18 Our findings suggest first that clinical and operational leaders tend to adopt a very pragmatic approach to learning, especially to the evaluation of “learning activities,” such that although they recognize the value of rigorous evaluation, that value is often trumped by the operational imperative to execute operational objectives efficiently (Theme 7). This is in contrast to research leaders who have different objectives and incentives. However despite the view of clinical and operations leaders toward execution, they expressed a strong desire for guidance in navigating the research-QI-innovation continuum, both in terms of methodology and oversight (Theme 9). As a result, there is a need to develop incentives that support integration of functional roles, yet are relevant to individual functional groups. The findings indicate that operationalizing the LHS requires a systems approach that considers multiple themes and issues simultaneously.

Financial support for learning infrastructure and activities was a common challenge across functional roles (Theme 10). Managers should align their operational priorities with leadership strategic goals, in order to gain support (Theme 1), and with creative partnerships. Unconventional funding sources should be encouraged and rewarded by leadership. Leaders may need to view learning from a broader perspective to see nonfinancial value from learning activities including improved patient-family engagement and quality of care.

Our discussions with senior leaders in an integrated delivery system highlight the challenges and opportunities that leaders face in developing mechanisms that support continuous learning and QI. Health care leaders need to develop deliberate strategies that uncover gaps and opportunities in their organization’s learning. As seen through the perspectives and themes regarding organizational learning elicited from senior leaders, organizations need to take an active role in designing learning infrastructure, processes, and activities that support research, QI, and innovation. Such activities may include leveraging data and analytics, QI initiatives, or research to generate supporting evidence. In addition, integrating learning activities can streamline evaluation and scaling up of initiatives, especially when organizational designs that build cooperative learning climates are employed.

Finally, the topic of who might have responsibility for implementing and managing the LHS and learning activities was often raised in the interviews. We consider this an extremely important issue to consider in developing an operationalization strategy. The extent to which leadership gives responsibility to any one functional group may exacerbate silos and limit the ability to employ an integrated approach to support an LHS model (Theme 3). At our institution, for example, our GLHS group has worked hard to stay an independent, multidisciplinary group with shared interest and responsibility for the project, so that leaders from all functional groups in the system do not feel that the project is owned by any one group. We believe this is consistent with the LHS model that promotes continuous learning as a multidisciplinary activity with shared responsibility across the system.1,2

Limitations

This project had several limitations. First, we did not interview frontline personnel, who may hold different views of the LHS and organizational learning. Frontline staff may have greater need to execute tasks and achieve specific outcomes due to operational incentives, with less emphasis on evaluation or research. Second, the opinions expressed were based on experience within a single delivery system and may not reflect perceptions of leaders in other systems despite the fact that several themes expressed are consistent with findings from other related studies.2,6,16,17,20 Third, due to the sensitive nature of the interview, it was decided to take notes during interviews but not to record them. We were therefore unable to review the interviews. To address this concern three members of the research group were present in nearly all of the interviews. Finally, since the project was an operational activity and not formal research, we recognize that the rigor of qualitative research methodology was sacrificed for the practical nature of executing the task efficiently.

Conclusion

Identifying leaders’ perceptions of the LHS and current state of learning is an important step in the operationalization of an LHS. Continuous learning and improvement is an ongoing, multidisciplinary function of a health care delivery system. To our knowledge, this project is one of the first efforts to understand health care system leaders’ perspectives on the LHS model and the operational challenges they see in advancing the model from concept to application. Leaders may consider findings from this and other research to inform and prioritize LHS objectives; to engage research, clinical, operational, and QI leaders in integrating learning and enhancing learning oversight and efficiency so as to deliver reliable, high value care.