Evaluating a Patient Safety Learning Lab to Create an Interdisciplinary Ecosystem for Healthcare Innovation

Abstract

Background

In response to the complexity, challenges, and slow pace of innovation, healthcare organizations are adopting interdisciplinary team approaches. Systems engineering, which is oriented to creating new, scalable processes that perform with higher reliability and lower costs, holds promise for driving innovation, in the face of challenges to team performance. A patient safety learning lab can be an essential aspect of fostering interdisciplinary team innovation across multiple projects and organizations by creating an ecosystem focused on deploying systems engineering methods to accomplish process redesign.

Purpose

Our study sought to identify the role and activities of a learning ecosystem that support interdisciplinary team innovation through evaluation of a patient safety learning lab.

Methods

Our study included three participating learning lab project teams. We applied a mixed-methods approach using a convergent design that combined data from qualitative interviews of team members conducted as teams neared the completion of their redesign projects, as well as evaluation questionnaires administered throughout the four-year learning lab.

Results

Our results build on learning theories by showing that successful learning ecosystems continually create alignment between interdisciplinary teams’ activities, organizational context, and innovation project objectives. The study identified four types of alignment: interpersonal/interprofessional, informational, structural, and processual, and supporting activities for alignment to occur.

Conclusion

Interdisciplinary learning ecosystems have the potential to foster healthcare improvement and innovation through alignment of team activities, project goals, and organizational contexts.

Practice Implications

This study applies to interdisciplinary teams tackling multi-level system challenges in their healthcare organization and suggests that the work of such teams benefits from the four types of alignment. Alignment on all four dimensions may yield best results.

INTRODUCTION

Innovation through care process redesign and improvement in healthcare delivery is occurring at an unacceptably slow pace and with variable impact (Rangachari, 2018). The complex nature of health care and fragmentation of care delivery has contributed substantially to the inadequacy of innovation efforts (Nembhard et al., 2009). Innovations that are especially challenging involve critical junctures, for example cross-disciplinary process redesign that spans multiple specialties and clinical sites. Health systems have adopted various approaches for integrating the knowledge of interdisciplinary team members to accelerate design, evaluation, and implementation of innovations that perform well within these contexts (Reiter-Palmon et al., 2018; Temkin-Greener et al., 2004; Wang et al., 2018).

Systems engineering, which draws on a variety of analysis modeling, design, and human factors methods to create new, scalable processes that perform with higher reliability, lower costs, and less burden under a range of input and operational variations (AHRQ, 2020), holds promise for driving meaningful healthcare innovations (Valdez et al., 2010). Our study examines the role of a recent patient safety learning lab that used systems engineering to facilitate process redesign in the context of interdisciplinary teams, specifically redesigning processes to enhance the prevention of harm to patients including the prevention of errors, learning from errors that occur, and creating a culture of safety (Erickson et al., 2003). We implemented a four-year learning lab consisting of health system personnel, systems engineers, quality improvement specialists, organizational scientists, and health services researchers to support interdisciplinary teams as they conducted process redesign innovation projects at critical junctures of care. The learning lab was implemented in four health systems, but one system exited after the second year. Thus, our study reports the results of the three health systems that completed the four-year learning lab. One health system exited the learning lab after the second year and was not included in this summative evaluation.

Our learning lab promoted process redesign by creating an interdisciplinary, inter-team ecosystem, conceptualized here as a network of individuals, teams, organizations, technology, resources, and other internal and external factors that support team performance by learning how to innovate, given a team’s objectives and context. Specifically, we asked: How could a learning ecosystem facilitate learning and progress within and across interdisciplinary redesign teams? To study the ways in which our lab created a learning ecosystem that allowed team members to learn and teams to progress and innovate, we performed qualitative analysis of key informant interviews from each of the three project teams as their projects neared completion. Additionally, we corroborated findings through learning evaluation questionnaires administered thrice-yearly to participating team members. Findings draw mainly on organizational learning theory and show that successful learning ecosystems continually create alignment between interdisciplinary teams’ activities and innovation project objectives. Our study highlights types of alignment and activities that support interdisciplinary team innovation.

THEORY

Interdisciplinary Teams: An Ecosystem Approach

In healthcare, innovations must address increasingly complex environments and organizational demands, requiring interdisciplinary teams to provide unique and diverse perspectives necessary to accomplish project objectives. These conditions necessitate approaches to innovation that account for the needs of collaborative networks of interdisciplinary experts from multiple organizations. An ecosystem approach lends itself to collaborative, interdisciplinary and multi-organizational innovation. The concept of an ecosystem comes from biology, where ecosystems are complex and dynamic environments comprised of organisms that continually interact with each other and their environment. While some organisms compete, often they collaborate to improve survival throughout the ecosystem (Darwin & Bonney, 1889). These tenets of natural ecosystems can be translated into organizational ecosystems that enable learning and foster innovation. A prominent example of an ecosystem for innovation in healthcare comes from the Veteran’s Health Administration (VHA), which created a collaborative community focused on developing and sharing new ideas and establishing a means to continually and systematically scale innovations (Vega & Kizer, 2020). The VHA Innovation Ecosystem has demonstrated success, particularly by diffusing best practices and implementing and sustaining innovations (Nevedal et al., 2020).

However, most healthcare organizations are not integrated like the VHA, so forming an ecosystem requires working not only across disciplines with diverse knowledge, skills, and backgrounds but also across organizations with distinct operating procedures and cultures. If innovation ecosystems are to be implemented more broadly, interdisciplinary teams must have systems in place to creatively tackle the complex challenges posed by healthcare redesign.

Fostering Innovation in Ecosystems Through Alignment and Learning

Organizational learning theory can facilitate understanding of how interdisciplinary teams can effectively progress in their innovations as part of an ecosystem. Organizational learning is the process by which groups and organizations achieve change, greater understanding, and improved performance (Argyris & Schön, 1978). It takes place at individual, team, and organizational levels and impacts attitudes as well as knowledge (Argote et al., 2020; Mayo, 2007). Contingency theory tells us that there is no one best way of organizing, since there are many contextual factors that can impact structure and performance and emphasizes alignment between structure and context in attaining a particular outcome (Drazin & Van de Ven, 1985; Thompson, 1967). While contingency and learning theories both view the organization as an open system, proponents of learning organizations envision learning environments that – by enabling learning – maintain alignment of individuals and teams with organizational goals (Senge, 1990; Stan & Puranam, 2017). Learning theory underscores the need for supportive learning environments, learning processes and practices, and leadership that reinforces learning (Garvin et al., 2008; Singer et al., 2012) and that time invested in learning to innovate can increase organizational capabilities for innovation (Repenning & Sterman, 2001, 2002).

Our research uses organizational learning theory to highlight how our lab fostered a learning ecosystem, which enabled innovation across a network of interdisciplinary teams and organizations, mainly by helping these teams learn to identify and manage misalignments within the organizational context, innovation objectives, and team activities. With learning organization frameworks in mind, we explore how a patient safety learning laboratory can create what have been referred to as the “building blocks” of a learning organization: (1) a supportive learning environment through an integrated program that included a variety of techniques (e.g., learning from “disruptors” with diverse industry and methodological perspectives, structured reflection assignments that emphasized psychological safety and encouraged opportunities to learn across disciplines and teams); (2) leadership that reinforces learning through presence and role modeling during team meetings; and (3) learning processes and practices by requiring education and training, experimentation, information collection, analysis, and information transfer through the lab’s curriculum.

Learning Lab Framework and Implementation

The main goal of our learning lab was to achieve breakthrough improvements in teams’ projects by creating an interdisciplinary, inter-team ecosystem that incorporated systems engineering analysis and design methods into project work. We intended the learning lab to serve as a mechanism for both facilitating process redesign across each of the interdisciplinary teams in the ecosystem and for demonstrating the outcomes of a learning ecosystem that encouraged teams to openly share problems experienced, crowd-source solutions, and experiment despite the risk of failure. Following this approach, each project team addressed a specific, important problem for its health system at some critical juncture between primary and specialty care. Some of the authors of this study designed the learning lab and others evaluated it, which created some separation between the perspectives of how the learning lab should function versus how it actually functioned.

Our learning lab shares similarities and differences with other innovation approaches, such as improvement collaboratives, six sigma, lean, and operational excellence. Whereas improvement collaboratives often focus on closing gaps between an evidence base and actual practice (Wells et al., 2018), our learning lab focused on developing and testing new approaches to problems without current good solutions (Leonard-Barton, 2003). A more fundamental distinction, however, is that our learning lab placed greater emphasis on interdisciplinary, inter-team collaboration to facilitate thinking differently about problems. In addition to process and outcome performance measures standard to any improvement approach (Wells et al., 2018), our learning lab also implicitly emphasized “learning to learn” and in this way has a commonality with “learning health systems” (Olsen et al., 2007). That is, teams from different organizations combine expertise of individuals across disciplines to overcome challenges of pursuing innovations and develop problem-solving skills that can be re-used on future projects (Leonard-Barton, 2003).

Through the learning lab, we explicitly sought to create an interdisciplinary ecosystem in which different health systems and disciplines work collectively to solve problems using familiar and unfamiliar approaches to foster process redesign efforts across teams. To foster this ecosystem, our learning lab initiated and oversaw activities over four years, including: thrice-yearly, half-day meetings called “learning sessions” in which all teams met to discuss and share progress and plan next steps; team-based progress reports generated between each learning session; project resources (e.g., engineering assistance, strategies for recruiting patient partners and engaging stakeholders); and administrative and project management support for each team. Learning sessions typically included a one-hour training in systems engineering methods to be applied to each team’s projects and/or an external speaker to convey similar material in a fresh manner to disrupt conventional thinking and provide new ideas for innovation. Examples of “disruptors” included experts in human factors and patient safety, design thinking and empathic listening, and adoption and scale of innovations.

Learning Lab Composition

Each project team consisted of five to ten members including health system staff (e.g., physicians, nurses, pharmacists, case managers), systems engineers, patient partners, and organizational scientists. Some engineers and organizational scientists worked with multiple teams. The larger learning lab included organizational experts and medical professionals (e.g., primary care physicians, quality and safety experts) involved in the design, implementation and/or evaluation of the learning lab. Site 1 focused on improving safe opioid prescribing processes for patients experiencing chronic pain by applying reliability engineering principles to reduce inconsistent evidence-based practice. Site 2 focused on improving the process of transitioning patients with high complexity from hospital to home care by using engineering design methods to ensure transition needs are identified, communicated, and addressed. Site 3 focused on improving perioperative care processes for children with medical complexity undergoing spinal fusion surgery by designing processes for longitudinally assessing patient readiness and collaborative decision-making including pediatricians, surgical teams, and patient families. All three teams actively engaged in the learning lab program and made progress toward achieving their innovation objectives and milestones, including: patient, leadership, and health system engagement; tangible output from activities; and process redesign tools and implementation activities, as described in a prior paper based on the same initiative (Atkinson & Singer, 2020). Teams applied various engineering tools including cross-functional process logic diagrams, failure modes and effects analysis, risk stratification, computer simulation, reliability and resiliency engineering, and other process analysis and design tools.

METHODS

Study Design

Our qualitative study used interview data, combined with quantitative assessment through participant evaluations of the learning ecosystem as a whole and of team progress toward implementing redesigned processes, to analyze experiences and perceptions of members participating in the learning lab from the three health systems participating at the time of our study. Specifically, applying Creswell’s typology for mixed methods research, we used a convergent design in our analytical approach in which we collected qualitative interview data separately from quantitatively-analyzed evaluation questionnaire data (Creswell, 2014). In doing so, we compared the results of both analyses to identify whether they converged on central insights responding to our research question.

Data Collection

We collected data using semi-structured interviews with key informants from each system to explore the role of the learning lab in promoting process redesign through interdisciplinary and inter-team collaboration. We conducted interviews near the end of the four-year learning lab period, which enabled us to examine the impact of the learning lab and to identify successes and shortcomings for each of the teams. The study period for Sites 1 and 3 was October 2015 to October 2019; Site 2 joined in March 2016. This study was approved by the institutional review board at Harvard T.H. Chan School of Public Health, with other sites ceding reliance.

We developed an interview guide reflecting five broad topics: (1) views on the learning lab’s aims and objectives; (2) barriers experienced throughout the period; (3) the skills team members learned throughout the learning lab’s implementation; (4) elements of the learning lab supporting future directions of the redesign projects and eventual spread; and (5) learning lab successes and shortcomings. Our goal was to understand the experiences of participants and identify characteristics of our learning ecosystem that enabled team success. All learning lab co-investigators reviewed the guide to ensure question clarity and relevance.

To evaluate aspects of the learning ecosystem and corroborate interview findings, we collected evaluation data from the study teams using two different instruments. The first instrument involved self-evaluation questionnaires administered to learning lab participants at each learning session. The content of the self-evaluations focused on team members progress toward implementing redesigned processes (see Supplemental Digital Content 1). Each item used a 12-point, behaviorally-anchored rating scale, which offers responses in words, based on behaviors that range from the least preferred to the most preferred form. Each item listed four behaviors, indicating no, low, medium, or high achievement of the behavior, indicating unprepared, somewhat prepared, prepared, and very prepared for implementation. Questions included in self-evaluations varied modestly across the 12 learning sessions, as we introduced new items when appropriate and replaced others when no longer relevant, given the teams’ progress. Nine items we most consistently asked evaluated the team’s project problem focus, process and outcome measures, patient engagement, preliminary design requirements, details and specifics of the design, tests of proposed designs, redesigned process elements, methods to evaluate utility, functionality, and sustainability of the design, and use of systems engineering methods.

The second evaluation instrument asked learning lab participants across the 3 sites to evaluate the learning lab, as a whole (see Supplemental Digital Content 2). The evaluation included 10 items, each using a 5-point Likert scale from 1=strongly disagree to 5=strongly agree, with a neutral midpoint. Items address participants’ satisfaction with their team’s progress, the learning lab, their team’s learning, and participants’ individual learning.

Study Participants

For our interviews, we used purposive sampling (Patton, 2002) to select nine key informants across the teams including physicians, project managers, and systems engineers. To reduce bias and improve the validity of our findings, our interview analysis applied a triangulation approach to developing theory (Berson et al., 2003; Denzin, 1978). Specifically, we conducted interviews with team members across disciplines—physicians, engineers, and project managers— such that we collected data across levels to capture different perspectives on the same phenomena. Interviews ranged from approximately 45 to 60 minutes and were audio-recorded and transcribed. The nine interviews were sufficient to approach theoretical saturation, since the teams had been working together for approximately four years and because the research team had been extensively studying the learning lab since its creation, garnering the intuition to extract relevant insights over the time period.

The composition of participants interviewed from each team were: two physicians and one engineer at Site 1; one physician and one engineer at Site 2; one physician, one engineer, and one project manager from Site 3; and one overall lead senior engineer who worked with all three sites. We regarded the interviewees as key informants with high levels of consistent involvement in the work of their respective teams. Given their unique role and perspective, we also asked patient partners similar questions. We reported patients’ perspectives on organizational constraints separately in a study examining how the teams moved forward in their redesign efforts using tactics ranging from empathic listening to enacting workarounds in overcoming a variety of organizational-level challenges (Atkinson & Singer, 2020); thus, interview data from patient partners informed but was not a part of this study.

For participant evaluations, at the conclusion of each learning session we invited all team members who attended to complete an evaluation. Both learning lab attendance and participation in the evaluations were voluntary, though most core team members for each team attended most learning sessions and completed evaluations as requested. Team members who completed the participant evaluations thus included interview participants as well as other team members across each team. Not all who were interviewed completed the self-evaluation questionnaires each time they were administered, and individuals completing the both evaluation instruments varied over the course of the 4-year learning lab.

Analysis of Interviews

We employed a thematic analysis combining inductive and deductive approaches to extract major themes characterizing effective learning ecosystems in the context of interdisciplinary teams innovations (Braun & Clarke, 2006). The coding team was led by the primary qualitative investigator (who was part of the evaluation team, but not involved in the design/implementation the learning lab itself), supported by the study principal investigator and a research assistant. Coded transcripts were analyzed using the NVivo 12 software.

We began with initial coding with a codebook that captured each of the five interview guide topics described above. Throughout the initial coding, however, we added new codes to these broad topics, expounding upon the codes captured at each site’s experience from the vantage point of the selected team members who had been engaged on the projects for the majority of the period. Applying an investigator triangulation approach to further improve the validity of our findings (Denzin, 1978), two members coded three transcripts to assess inter-rater consistency and share novel codes that emerged; one member coded the remaining six transcripts. Members of the coding team also met regularly throughout the coding process to ensure the integrity of the analysis and relevance to the study context (Constas, 1992). Coders identified 77 new codes, including: “encouragement to get past barriers,” “educating on the importance of organizational member buy-in,” and “many perspectives require flexibility in aims.” Upon reviewing the codes and example quotes associated with each (as described below), approximately one-third of these codes commonly surfaced across interviews and resonated with emergent themes (see Supplemental Digital Content 3 for data structure linking initial codes to overarching themes pertaining to alignment). We note that the number of new codes diminished after the initial three coded transcripts, signaling that we were approaching theoretical saturation as the identified concepts repeated in the remaining analysis and additional data collection would be redundant.

Enhancing the reliability of our qualitative findings, we engaged a process of constant comparison of the data (Silverman & Marvasti, 2008). We iteratively compared codes across each site to enact the thematization stage of coding, combining and revising our initial codes into overarching themes commonly experienced by team members, such that we reached data saturation. Guided by our conceptual framework, we continued to revise the themes to represent the broad areas of learning and ways of creating alignment highlighted by the prior literature. We applied conceptual depth criteria to help us further determine whether we had reached theoretical saturation (Nelson, 2017): 1) range – we had several examples of the concepts exemplified by our data across each of the interviewees; 2) complexity – concepts and themes are linked in the data in the ways we purport through our findings, as suggested by our analysis of coding matrices through NVivo; 3) subtlety – our comparison of examples of the same code demonstrated varied descriptions of experiences and meanings within that particular concept; 4) resonance – many of our identified themes built on prior learning and teams literature; and 5) validity – the applicability of our findings is in part demonstrated through comparison of themes across three different health systems comprising our learning lab.

Finally, we grouped codes into four types of practices necessary for the learning ecosystem to achieve broader alignment among teams, projects, and organizational context: interpersonal/interprofessional, informational, structural, and processual. We defined alignment as a fit between conditions experienced by an interdisciplinary team and the requirements of its innovation project, and ultimately how teams learned to make progress on their redesign efforts. We further grouped codes into major themes within each type of alignment.

Analysis of Participant Evaluations

For participant self-evaluations, we report “start scores” and “end scores” for each item, indicating participants’ average scores for first and last times we asked a given question over the course of the learning lab. To assess progress, we first assessed the “starting gap,” (i.e., respondents’ average start score subtracted from 12, the highest attainable measure). We then calculated the “percent of starting gap closed,” numerically calculated as the improvement between start and end points divided by the starting gap. For participant evaluations of the learning lab, we calculated simple means and standard deviations. We calculated data for both instruments at the learning lab level because (1) teams included small numbers of individuals, (2) some individuals participated in the ecosystem in ways that cut across teams, and (3) we wanted to reflect the inter-team learning that occurred in the learning lab.

RESULTS

Ecosystems inherently represent interdependency; such that in a thriving ecosystem, there must be alignment among the different organisms, as well as between the organisms and their physical environment. The most prominent themes that emerged from our data focused on ways that a successful learning ecosystem facilitates alignment among a team’s activities, innovation project, and organizational context. With greater alignment, teams experienced fewer barriers to achieving these aims, enabling teams to learn how to innovate to try to achieve their objectives. Figure 1 shows the four types of alignment we identified: interpersonal/interprofessional, informational, structural, and processual. These four types of alignment were core drivers of the success of our learning ecosystem, which in turn facilitated team progress and performance on achieving their innovation objectives and milestones (Atkinson & Singer, 2020).

FIGURE 1.

Types of Alignment Between Innovation Teams, Projects, and Organizational Contexts Addressed by Learning Ecosystem and Supporting Practices.

For each type of alignment, we identified specific practices that interviewees felt would foster alignment (see Supplemental Digital Content 4 for more examples). For some of these practices, the teams wished the learning lab could have fostered greater alignment and innovation success. We considered these shortcomings of our learning lab and important recommendations for enabling strong learning ecosystems.

Interpersonal/Interprofessional Alignment

Interpersonal/interprofessional alignment refers to the fit among team members and with organizational stakeholders (e.g., facility and unit leaders indirectly affected by the process innovation), such that perspectives, incentives, and priorities are the same, consistent, synergistic, or reinforcing. While we recognize that interpersonal and interprofessional factors are different, for the purposes of this study, we are treating interprofessional attributes as an aspect of interpersonal alignment; specifically, interprofessional similarities and differences played a role in the interpersonal dynamics of team members. Interview informants often described the interpersonal and interprofessional aspects of their team interactions interchangeably (e.g., referring to themselves as “engineers” or “clinicians” in describing interpersonal aspects that were crucial for alignments). For these reasons, we kept interpersonal/interprofessional aspects within the same form of alignment, as they were not sufficiently distinct from one another to warrant separating them.

Our learning ecosystem aimed to achieve this type of alignment through three practices: creating a safe space, integrating clinician versus engineering views, and gaining stakeholders’ support. Interpersonal/interprofessional alignment was critical for all the teams we studied, and in particular managing interprofessional and interdisciplinary differences across team members. Specifically, interpersonal/interprofessional alignment included both alignment across disciplines of different team members and with stakeholders external to the team. Additionally, creating a safe space enabled sharing of perspectives and ideas and fostered connections that facilitated progress toward team innovation goals. While the different methods, workstyles, and concepts that are used in each field (i.e., healthcare and systems engineering) created some dynamic tensions, respondents suggested that this blending also spurred new ways of thinking about how to improve processes. Ultimately, creating interprofessional alignment enabled team members to respect, hear, and integrate views of people from other disciplines and in other parts of the organization.

Creating a safe space.

Dedicated times for learning were helpful to enable team members to collaborate in a respectful environment. Team leaders listened to other team members, and team members across disciplines learned to trust each other, aligning member perceptions of trust across the team.

“I think maybe the [ Site 2 ] team would be a good example of the more successful teams, and I would say so because at some point we had established trust. You know what I mean? The health care team kind of stepped back and allowed the engineers to actually do what they wanted to do and actually... And they were willing to be involved to give the accurate information and the accurate data, so that the engineers would do their part accurately. And in the end when they looked at the results they were happy” [Senior Lead Engineer]

Integrating clinician, patient, and engineering views.

All interviewees stated that a strength of the learning lab was working on an interdisciplinary team. The lab created a context for sharing perspectives and expertise among engineers, clinicians, and patients. An engineer spoke about the importance of integrating perspectives for project design when discussing the viewpoints of two project leaders.

“…we would be totally clinician centric, if it was just [Clinician A] on the team and totally engineering centric if it was just [Engineer A and Engineer B].” [Site 1 Engineer]

Gaining stakeholder support.

Of the nine people interviewed, eight referred to organizational member buy-in as important for facilitating project accomplishments. This buy-in was created through interpersonal relationships between stakeholders and team members. The learning ecosystem educated members about the importance of stakeholder support and provided the impetus to engage stakeholders. In contrast, resistance from hospital stakeholders was seen as a great barrier to team accomplishments. Members noted that teams could overcome this barrier by creating alignment through common ground between hospital leaders and the team’s goals, although they were able to do so with varied success.

“‘Because you still have to prove this to somebody. ‘Oh, we wanna implement this program because it’s gonna be really good.’ That’s not gonna convince anybody. They need to know if it’s worth it and that means money. Saving lives, yes, but is it cost... How much is it... Like a life but how many lives? So it’s more of, [Learning Lab Leader] brought us back to that a lot, which is great.” [Site 3 Engineer]

Informational Alignment

Informational alignment refers to the fit between the team’s information requirements and access to information needed to move forward with the innovation project. Informational alignment was primarily achieved through two practices: selecting appropriate priorities and expanding team member perspectives across disciplines. Throughout the period, the learning lab helped teams gain the information they needed to accomplish project tasks and make progress on their innovation projects. Teams gained this information by selecting project priorities representing the most important objectives for the project at the time and by expanding member exposure to project topic contexts in order to create alignment among team members such that interdisciplinary backgrounds could overlap between the engineers, clinicians, and other members of the team.

Selecting appropriate priorities.

During the innovation process, the learning lab helped teams to select priorities by working with them to order objectives and allocate time to performing project tasks. This advanced team innovations by matching goals and priorities with the objectives the team was focusing on at the time. Often, learning lab leaders used their experiences to provide teams with advice on how to sort through project tasks to most productively move forward with their projects.

“If we were working on three things, you [learning lab leaders] would always say, ‘It is this one thing that you’ve gotta get right,’ kinda a sort through from your vast experience about how this happens, and the many, many times you’ve been through it and the studies that you have done. I think that gives you an immense perspective on what we are doing, what we’re trying to do, and if this study is to go right, you need to be doing this, or you need to have this on the table, or this again, I think that the most help was this overview, this big view, but it was also what to do at this small moment.” [Site 2 Physician]

Additionally, the learning lab was structured to routinely remind teams of their aims. “Zooming out” to the aims level then “zooming in” to the current project objectives narrowed the scope of what members should be focused on, which facilitated movement towards project accomplishments.

“I think also with learning collaboratives, it’s like, sometimes it just feels a little bit idealistic in some ways. We come together for the learning sessions, and we’re like, ‘This is so great,’ and, ‘Everything is awesome.’ [chuckle] But are we really making sure we’re keeping on task and meeting the goals? And I think [the learning lab] did a really good job of that, of trying to get us all back to, ‘What are our aims? And let’s accomplish these things that we set out to do.’” [Site 3 Physician]

Expanding team member perspectives across disciplines.

Team members described being able to innovate by experimenting with what they know about their own disciplines as well as by broadening their horizons with the knowledge gained from exposure to new disciplines. For example, during team meetings, clinician members routinely commented on how the system engineers helped them see a bigger picture by bringing together disparate pieces of information and organizing this information through such visualizations as swimlane diagrams to map the many dimensions of a process or dashboards to graphically display performance across providers. By compiling information and organizing it into a meaningful visual format, engineers assisted teams in expanding team member perspectives as they tried to make sense of the problem at hand and possible solutions. More than just enabling individuals to see the “big picture,” such expansion activities enabled team members to see things differently and often more clearly. Other interviewees reflected on how the learning lab provided the opportunity, by connecting with subject matter experts, to apply course concepts to real life health systems.

“…I’m learning some programming languages to make this data analytic simulation and predictive model. So that was really good. And for example, I learned most of them in my courses, in my lectures, but I haven’t had a chance... Okay, I haven’t had a chance to apply them in real life, so I think it’s a really good chance for me to apply, implement these theoretical things to the real life.” [Site 2 Engineer]

Members also spoke about how being exposed to new disciplines during the learning lab experience allowed the team to be more creative and effective in their innovation efforts. The engineering tools were not overly-complex and were learnable by novice team members in other disciplines. Despite the simplicity of these tools, one physician on the team at Site 3 commented that “going through the process of thinking and using [the engineering tools] elevated our thinking about the process, and maybe just have a different lens of thinking about a process of care, and seeing opportunities of how we could re-engineer it. […] my approach to that changed, just by simply thinking about how we were doing that.”

Structural Alignment

Structural alignment represents the fit of team activities and resources with the requirements of the innovation project. Structural alignment refers to practices that correspond to team characteristics, such as team composition and regularity of meetings, which members needed to productively function and accomplish project tasks. Teams accomplished structural alignment through two practices: managing team membership and allocating resources to teams.

Managing team membership.

Members commonly cited team turnover as a significant shortcoming of the learning lab experience (eight out of nine interviews referenced this as a barrier). Members explained that turnover led to a lack of aim clarity within the team and stunted project momentum.

“I think having [Engineer A] and [Engineer B] working consistently with [the team] all the time was really beneficial because we were meeting regularly and we could actually get a lot done, but I think that’s maybe kind of trickled off a little bit once [Engineer A] left, and now we’re kinda getting back into it as the grant is ending and making sure we are reaching all of the goals that we set to accomplish.” [Site 3 Project Manager]

When asked how the learning lab could be improved, members spoke about ways to better integrate team members during on-boarding and throughout the learning lab experience to foster better understanding of all member disciplines and better project continuity. This would allow teams to better utilize their interdisciplinary skillsets and understand every aspect of the projects, creating alignment between team members and project goals. As one physician on the Site 2 team described, a benefit of onboarding more formally to familiarize team members across different disciplines with each other’s role would help to create “a reference point of possibly explaining to each other what the [each team member] was doing.”

Allocating resources to teams.

Members also found that certain resources from the learning lab would be useful to get project work done. These included having an extra team member to act as a project manager, access to hospital IT support, and access to hospital data. One team chose to hire a project manager and found that helpful. A team member from Site 3 commented that bringing an IT person directly onto the team would have been helpful beyond accessing hospital IT support since there were many challenges “just trying to work within the hospital, with Cerner and PowerChart, and pulling data from 360, and just all the [issues] retrieving data and trying to have this organized system for the patient list and chart review.” When asked how the learning lab could be improved, five of the nine members interviewed suggested ensuring the project manager remained on the team and four suggested ways to integrate hospital resources, such as IT, with the team.

Processual Alignment

Finally, processual alignment refers to the fit between dedicated learning efforts of the lab and the stage of the team’s innovation efforts. Processual alignment occurred mainly through two practices: engaging in learning activities and carving out learning time. Processual alignment refers to actions that the learning lab took to create time and spaces for learning and learning activities to occur. Reflection points and learning sessions were built into the learning lab in order to facilitate learning, and members found these to be important for project success.

Engaging in learning activities.

The learning lab incorporated specific activities to teach key systems engineering concepts, created progress update assignments during and between learning sessions, and structured presentations at each learning session. Members found these to be important reflection points, supporting the adage “nothing like a deadline,” to take note of what has or has not been accomplished and what future steps are needed. They also noted that these activities were key for project momentum in the time post-activity.

“Yeah. I also think that though in the times that we worked off the learning session that that was very powerful. So the learning session as stimulant allows a movement forward that’s a lot faster and more informed.” [Site 2 Physician]

Carving out learning time.

The learning lab created protected time for project teams to come together through weekly meetings and thrice-yearly learning sessions. Members regarded this dedicated work time as extremely helpful. Having most of the interdisciplinary team members available to work in-person on the innovation project was critical to progress.

“I think overall, it was very cool. Like the very first meeting to walk in and have this standing meeting where we have staff who have protected time to think about this kind of broader work and are happy champions of the project, like I think that’s an amazing accomplishment” [Site 1 Engineer]

Learning Outcomes

During interviews, members shared how their teams successfully achieved many objectives and progressed toward process redesigns. Site 1 members created an opioid resource team and reliable urine toxicity screening workflows to support clinicians who care for opioid users. Site 2 members used theoretical tools like predictive modeling to create a standardized process for hospital to home health primary care communication. Site 3 members addressed a need for a reliable pre-operative care plan for children entering surgery by creating a standardized plan and increased collaboration between anesthesia and primary care. The learning lab fostered these accomplishments by enabling team practices that created alignment among teams, innovation projects, and organizational contexts.

However, while each site progressed, none achieved full alignment on all four dimensions, and none fully achieved project aims. Site 1 had limited stakeholder support and learning time, and; Site 2 failed to achieve stakeholder support. Both sites have been unable to implement key elements of their process redesigns, however. Site 3 came closest to achieving alignment across all dimensions, and the greatest progress toward process redesign, including implementation, preliminary evidence of improved clinical outcomes and patient/family experience, and spread to other surgeries and sites.

We conducted a final learning lab meeting virtually in the Fall of 2020 to further assess learnings from participants and perceptions of learning ecosystem effectiveness. Team member remarks highlight benefits derived through the learning lab:

“The [learning lab] enabled us to see resources and solutions that were invisible to us, especially the informatics and clinical workflow standpoints… it was great to discover those things and be able to get going on our innovative work.” [Site 3 Physician]

“The [learning lab’s] design– of merging people across different disciplines including clinicians, engineers, and organizational innovation experts […] and driving the process with weekly meetings and in-person sessions with the other teams three times a year was brilliant. It had a very powerful effect on us. It made us something that we were not before. It made us different thinkers, with different sensitivities. It made us feel more powerful to do what we needed to do.” [Site 2 Physician]

“One big thing with systems engineering tools is that they can require a big lift early on- you need to sell it to clinical staff unfamiliar with these processes […] By the end of the project, any systems engineering concept that we presented seemed to be taken up by the staff almost right away. […] Often different disciplines are talking about the same thing in different ways. Interdisciplinary learning can take patience but it’s stimulating and productive.” [Site 1 Engineer]

“As a patient partner in the learning lab, I am appreciative of how the [learning ecosystem] embraced and encouraged true patient engagement. The entire team’s willingness to not only listen to and hear the patient and family perspective, but to help translate those insights into meaningful and actionable transformation was incredibly appreciated and a rewarding experience overall. […] [The learning lab] has been a wonderful, collaborative experience and holds so much potential to create lasting, positive changes to systems in healthcare.” [Site 3 Patient Partner]

Participant self-evaluation of team progress toward implementation (see Supplemental Digital Content 1) suggest that, on average, participant understandings improved in all nine areas. The most substantial improvements were in detailing and specifying process designs and patient engagement. In these areas, learning lab participants reported 78.6% and 73.1% improvements respectively. These data further suggest that the learning lab fostered the development and improvement of systems engineering skills. The three teams have plans to sustain and spread efforts, ranging from handing off a pilot program to a standing committee to launching a permanent, system-wide center.

Participant evaluation of the learning lab indicate moderately high marks, ranging from 3.75 to 4.69 on a scale of 1 to 5 (see Supplemental Digital Content 2). This suggests that participants generally agree that they feel satisfied with their team’s progress, the learning lab, their team’s learning, and participants’ individual learning. Participants scored highest the question indicating their personal understanding of systems engineering tools had improved.

DISCUSSION

Many challenges confronting health systems require novel approaches to enable interdisciplinary teams to innovate and improve the quality of complex organizations and processes at critical junctures of care. Our research combined qualitative interviews with quantitative assessment, building on four years of weekly or biweekly observation of three interdisciplinary project teams to examine a four-year learning lab in which we created an ecosystem to foster innovation.

Although organizational conditions are not always ideal for teams to make progress, we show that it is possible for interdisciplinary teams engaged in process redesign projects to learn how to innovate to achieve improvement goals. Our findings also suggest that a successful interdisciplinary ecosystem can scan environmental conditions at team and organizational levels to create opportunities for better alignment. Our study identified four ways in which our learning lab facilitated alignment with organizational goals, albeit with varying degrees of success: interpersonally/interprofessionally, informationally, structurally, and processually. For each type of alignment, we showed how our interdisciplinary ecosystem enabled team activities, with the goal of creating the organizational fit necessary to support innovation and higher performance.

Our findings draw on organizational learning theory by suggesting that alignment between a team’s organizational context and its innovation project requires continuous learning and adaptation; where there is alignment, innovation teams can continue to learn to innovate and achieve innovation progress. Prior research that provides a framework for learning organizations has identified key building blocks that enable groups of individuals to learn together, including leadership that reinforces learning, learning processes and practices, and a supportive learning environment (Singer et al., 2012). Our four types of alignment overlap with many of these fundamental elements of learning organizations. For example, learning processes and practices— such as experimentation and information collection, and education and training— relate to aspects of processual and structural alignments from our model. While our analysis of inter- and intra-team learning yielded many similarities with the original learning theory frameworks, we highlight some differences too. For example, interviewees commented on the challenge of dealing with team member turnover and the importance of having tools to manage team membership over time, such as more formalized onboarding processes. Additionally, our model adds the importance of gaining and maintaining stakeholder support, not only for the implementation and spread of redesign efforts, but also for boosting team member morale and further promoting a learning experience. And last, our model provides novel insights into the alignments necessary for spanning differences in perspective, expertise, and experience of team members across disciplines. Thus, in dealing with their dynamic context, our research demonstrates that interdisciplinary teams learn to innovate and effectively conduct activities that create alignment between their own innovation objectives and organizational goals; the learning ecosystem is a conduit for achieving alignment between the organizational context and the demands of the team innovation projects, in the case of this research.

None of our learning lab teams achieved strong alignment in all four areas, typically missing at least one resulting in less than desired success. This highlights the challenge of achieving sufficient alignment to accomplish improvement objectives. While not part of our participant evaluation questionnaire, the research team noted that these observations applied to the lab’s fourth project, which was aligned with regard to stakeholder support, information availability, dedicated time and resources, but lacked a real desire to be part of a learning ecosystem, experimenting with novel innovation ideas, and investing time in inquiry and engineering methods. Rather, this site had a strongly ingrained system-wide emphasis on lean improvement approaches but lacked openness to using systems engineering, processual alignment, and interpersonal/interprofessional alignment due to project direction disagreements.

Our study has some limitations. Although we conducted in-depth interviews that focused on the entire period, our findings are limited by the perceptual reporting of nine key informants at one point in time. Each informant, however, was selected based on his or her exposure to as many facets of our learning ecosystem as possible, with each having been actively involved in their teams for approximately four years. Interviewee responses also may reflect social desirability bias in which interviewees report more on favorable than negative aspects of the learning lab. To mitigate this, we posed follow-up questions to interviewees that encouraged deeper examination of the questions we asked. Additionally, we have studied these teams in detail through the four-year period, so we have perspective over time that support interview findings; however, we did not have the data needed to show how alignments changed over time as this theme emerged only now. Also, while corroborating our qualitative findings with participant evaluations is a strength of this study, due to variation in attendance, natural team member turnover, and voluntary participation, our analysis of change in self-evaluation data may be biased due to our inability to compare individual assessments over time.

In our current study, we did not examine the external environment and engagement of external stakeholders (e.g., whether individuals adopted the redesigned process). Instead, we concentrated on the inner workings of our learning ecosystem and the mechanisms through which teams learned to make progress in their redesign efforts individually, within their team, and across ecosystem teams. However, in a prior paper describing the learning lab, we highlighted the role of the external environment and zoomed in from the ecosystem-level onto the learning lab teams, examining how they managed challenges arising from stakeholders and other factors outside the ecosystem (Atkinson & Singer, 2020).

Finally, although we did not examine team dynamics such as the effect of a particular leader’s qualities or prior experience of team members, future research could investigate these in relation to the learning ecosystem as a whole. Our study provides other fruitful opportunities for future research, including examining the learning ecosystem from the perspective of organizational stakeholders whom the redesign teams are trying to engage. This examination could help shed light on the role of the health organization or system in supporting a learning ecosystem. Last, future research could examine how the four alignments evolve over time as teams engage their redesign projects.

PRACTICE IMPLICATIONS

The complexity, challenges, and slow pace of healthcare innovation underscore the need to explore new approaches to improvement. Table 1 describes key aspects of our learning lab and can be used as a starting point for others interested in creating similar systems engineering and learning ecosystems. While our recommendations are largely consistent with practical recommendations for learning organizations (Singer et al., 2012)., our four-year learning lab shed light on important temporal factors.

TABLE 1.

Major Steps and Considerations for Creating an Interdisciplinary Ecosystem.

Timing	Major steps and considerations
Before innovation activities begin	Create a core leadership team: This team is comprised of 4–5 members, who span the disciplines involved in the learning ecosystem. Members oversee and manage elements of the learning ecosystem to ensure forward progress and momentum continues.
	Resource procurement: The core leadership team is responsible for assisting in finding the necessary financial support and expertise to sustain the learning ecosystem.
	Team construction across sites: Team members at each site are carefully chosen by the core leadership team as those with interest, knowledge, capacity, and enthusiasm to engage the learning ecosystem. Similarly, each site team has a leader selected who embodies similar qualities.
During main innovation activities	Creating alignment: As team innovation activities ensue, the learning ecosystem strives to create alignment interpersonally/interprofessionally, informationally, structurally, and processually. Teams learn to invoke various practices to make this alignment possible, strengthening the network and learning to navigate a variety of challenges.
	Constraint management Within and across teams in the learning ecosystem, there is persistent effort to overcome and work around organizational constraints, or challenges imposed by the external environment of the team. These efforts enable maintaining the conditions that create alignment, and are reinforced by the team’s ability to stay alert, proactive, and resilient in the face of challenges or potential problems.
In parallel with main innovation activities	Team processing: Teams continue to engage in activities (e.g., writing papers on prior studies, exploring possibilities for future work) that enable reflection and solidify elements of the learning ecosystem for future use.
	Spread and sustain: Innovation development and implementation keeps in mind the potential for future spread and sustaining improvement efforts. Throughout main innovation activities, teams are responsive to these aspects of creating long-term and sustainable change in their respective organizations.

Source: EHRLL core team discussions synthesizing lessons from 4 years of observation, interviews, and interaction with learning lab participants.

In this table, we describe the major steps and considerations highlighting the temporal elements that were important for the phases of development of the learning ecosystem: before innovation activities begin, during main innovation activities, and in parallel with main innovation activities. To achieve the alignments we identified as supportive of learning ecosystems, health systems must organize the capacity to ensure they are able to create and maintain a core leadership team, adequate resources, and team members with interest, knowledge, capacity, and enthusiasm for undertaking the proposed effort. As the work proceeds, teams need capacity to sustain alignment and to manage constraints that might undermine the work as they arise. Particularly with natural attrition of team members over time, realignment must occur on an ongoing basis. Throughout the innovation process, teams must make opportunities to reflect in order to internalize their learning, and they must consider plans for spreading and sustaining their innovations as an ongoing input into the innovation process itself, not as an addendum once the initial innovation is complete.

CONCLUSION

Lessons from our learning lab suggest that interdisciplinary ecosystems that apply our learning ecosystem framework have the potential to foster greater learning for improvement and innovation. Through alignment of team activities, project goals, and organizational contexts, our learning lab functioned to create an environment conducive to interdisciplinary collaboration, innovation, and adoption of systems engineering and design methods. Our findings most readily apply to interdisciplinary teams tackling multi-level system challenges at critical junctures of care in their healthcare organization. They suggest that the work of such teams benefits from interpersonal/interprofessional, informational, structural, and processual alignment; that the absence of alignment on one dimension may undermine the ability of a team to fully achieve its goals; and that alignment on all four dimensions may yield best results. Although study participants described some shortcomings of the learning lab, our findings show a successful learning ecosystem can facilitate alignments as they need to occur for teams solving complex problems and functioning across multiple organizational levels.