Learning from implementation setbacks: Identifying and responding to contextual challenges

Abstract

Introduction

We address organizational learning about implementation context during setbacks to primary care redesign in an ambulatory system. The redesign expanded care teams and added a medical assistant assigned administrative and coordination tasks. The redesign was expected to improve care efficiency, prevention, and continuity. In response to setbacks, redesign and system leaders used understanding of context to plan system‐wide changes, as well as program adjustments. Doing so enhanced the redesign's prospects and contributed to system learning.

Methods

We conducted a 33‐month, mixed‐methods study. Qualitative data included quarterly calls with the redesign leaders and 63 activity log entries. There were three site visits; 73 interviews with practice leaders, providers, and medical assistants. Data analysis used categories from an implementation research framework; these were refined and then expanded inductively using log reports, debriefings with change leaders, and documents. Quantitative analysis used system operational data on chronic care, prevention, efficiency, productivity, and patient access.

Results

Redesigned teams were not implemented as widely or rapidly as anticipated and did not deliver hoped‐for gains in operational metrics. Interviews reported that team redesign was leading to improvements in chronic care and prevention and eased provider burden. Besides making small adjustments to cope with setbacks, redesign and system leaders engaged in more thorough organizational learning. They examined contextual challenges underlying setbacks and posing risks to the delivery system as a whole. Their responses to challenges helped strengthen the redesign's prospects, improved the delivery system's position in its labor market, and helped the system prepare to meet emerging requirements for value‐based care and population health.

Conclusions

This case points to benefits for both health care researchers and change practitioners of paying closer attention to how context affects implementation of organizational change, and to opportunities and conditions for learning from setbacks during change.

1. INTRODUCTION

Organizations implementing large‐scale change need to develop their capacity to learn from their own experience, as well as that of other systems.1 We address learning from experience in implementation setbacks and failures, a process often overlooked by researchers and not fully exploited by system leaders and change practitioners. Our discussion of programs for organizational change includes initiatives that plan to change strategies, cultures, organization, and care practices (hereafter, “change programs” or “organizational changes”). Such programs are complex, involve multiple actors and levels, and often include innovative concepts and practices.2, 3 As in other industries,4, 5 change programs usually contain unpredictable developments, including setbacks and outright failures. Setbacks can involve delays, gradual loss of momentum, failure to attain planned objectives, and unintended consequences.6 Even changes that have succeeded in some settings may falter elsewhere.

Leaders of organizational change may enhance their prospects for success and learning from experience by considering potential and emerging interactions between proposed changes and influential contextual features.7, 8, 9 Then, leaders may take steps to adapt the program to its context or influence the context. Critical contextual domains include members' skills, experience, and attitudes; organizational structure, management system, and culture; informal politics; physical and technical infrastructure; and external operating environments.7, 9, 10, 11, 12, 13

As organizational change leaders weigh responses to context, they may discover ways to improve the organization as a whole. This paper documents a case of such broad learning about context during setbacks to a primary care redesign in an ambulatory care system. Leaders of the redesign and the delivery system used their understanding of context to plan system‐wide changes, as well as program modifications. They thereby enhanced the redesign program's prospects while also helping the care system adapt to its labor market and prepare for value‐based payment and reporting.

After reviewing background for research on intended organizational change and organizational learning, we describe our research methods and present four episodes that led to learning affecting the redesign program and the whole care system. We then discuss conditions fostering learning and implications for research and practice.

2. BACKGROUND

2.1. Effects of implementation context

Previous health care research has developed five valuable approaches to effects of context on organizational change. The first involves empirical studies of program implementation or outcomes and their contextual influences. A typical study used case studies in five hospitals to examine how intraorganizational factors affected care redesigns developed through Lean improvement techniques.14 An alternative empirical technique analyzes participants' reports on barriers and facilitators of a specific change, for example, the patient‐centered medical home.15

A second approach develops research frameworks based partly on important contextual factors found in empirical studies. Influential frameworks draw on quality improvement,16 implementation and behavioral science,7, 17 and organizational research.11

A third, more action‐oriented approach, creates general guidelines for choosing and designing proposed changes and for adapting implementation strategies to current and anticipated contextual influences (eg, Brach18). A more rigorous, but complex, procedure for implementing evidence‐based practices used in the Veteran's Administration system begins with formative evaluations diagnosing sites' implementation needs, barriers, and facilitators. Then, initiators of the change convene multi‐disciplinary teams including site staff, implementation experts, and clinical experts. The teams interpret the diagnostic findings and develop site‐specific intervention strategies.19

A fourth, agency‐based approach,20 supplements previous approaches and fits well with treatments of the learning health system.1 Rather than treating context as given, this approach shows that leaders can both “fashion” and “cultivate” the inner and outer context of the organization or micro‐system “to liberate, enable, or mobilize change.”21 Development of an organizational change initiative thus depends substantially on how leaders and other participants interpret and act upon program contexts before or during implementation. This view owes much to the strategic choice school of management, which showed the influence of executives' strategic decisions on their organization's operating environment, structure, and culture.22, 23, 24 For example, when health systems enter into business alliances with nearby ambulatory practices or other hospitals, they may reduce competition in the local health services market. Such strategic decisions can sometimes lead to organizational transformations.24, 25, 26 A fifth, related view, is that all members of an organization, and not just powerful leaders and stakeholders, negotiate and shape context. They do so through interpretations and interactions that “enact” their organization and its environments.27

Intuitive and experiential behavior including enactment are central to organizational learning and to management of organizational change. “Practical wisdom” is critical to the success of organizational change programs.28 “Insider, local, experience‐based knowledge” gives program leaders and members capacity to recognize what is important in a program's context, how context influences the program, and what steps might enhance its success.28

2.2. Organizational learning

The literature on organizational learning helps explain how members of an organization apply practical wisdom to identify, react to, and influence change contexts. Organizational learning involves improving actions through better knowledge and understanding29 and requires spreading individual and team learning across the organization.30 Organizational learning contributes to members' explicit knowledge and skills and to their implicit, practical wisdom; it can further lead to beneficial changes in routine operations and processes.30 Shared learning processes receive less attention than deserved in research on system change in health care and learning health systems.31 Current knowledge comes mainly from industries other than health care32, 33 and from growing health care experience in fields like quality improvement, which uses learning techniques, such as Plan, Do, Study, Act.34

Organizational learning from setbacks and failures, along with other unintended developments of change initiatives, are studied in other industries.35 But such learning receives less attention in health care beyond quality improvement34 and patient safety,36 both of which apply analytic and improvement techniques from other industries. Some analysts urge health care organizations to learn from setbacks while implementing large‐scale system change,37 but research and reports on such learning in health care are limited.

Research on organizational learning32 often distinguishes adjustments in a course of action from learning that loops back and critically examines a program's goals and underlying assumptions. Adjustments entail single‐loop learning, which usually leads to incremental improvements, as defined by current goals and standards. Critical questioning of goals, standards, and underlying beliefs fosters double‐loop learning.38 This second type of learning is more innovative, difficult, and risky but can produce more radical improvements and facilitate change beyond the initial learning focus.

Conditions promoting organizational learning in and beyond health care deserve wider study. Many current generalizations about learning health systems reflect observations of conditions in known, successful learning systems.1, 39 Such generalizations do not specify how learning conditions develop and influence learning, whether they do so individually or only in clusters, and whether influential conditions apply across diverse settings.

To help fill gaps in research on organizational learning in health care and focus attention on learning through setbacks, we analyze setbacks and organizational learning during redesign of primary care teams at Penobscot Community Health Care (PCHC), New England's second‐largest federally qualified health center.

3. METHODS

Our data come from a 33‐month, mixed‐methods study beginning in October 2013, sponsored by the Agency for Healthcare Research and Quality (AHRQ). The independent research team conducted the study with active cooperation by PCHC leaders and staff. Qualitative data were drawn from quarterly calls with two program change leaders. Throughout the study, the change leaders maintained and shared with researchers a personal activity log (totaling 63 entries). They discussed their change activities with researchers on a quarterly basis. The researchers conducted three site visits and 73 interviews (mainly in person) with PCHC leadership, practice leaders, providers, and medical assistants (MAs). One researcher conducted each interview, while another took notes. The note taker then finalized interview notes, and attending team members reviewed and commented on them. Additional documentation came from information and reports submitted to AHRQ before, during, and after the study.

The researchers met around twice a month to identify, discuss, classify, and code patterns through identification, review, and confirmation or refutation. Data coding, aided by NVivo 11 software,40 combined inductive coding with codes using categories (shown below in Table 2) from the Consolidated Framework for Implementation Research.7 Drawing on the interviews, the team then developed preliminary interpretations of the implementation process and contextual influences. Elements in this interpretation were clarified and inductively expanded through comparisons to log reports, debriefings with the Change Team, and documentary sources.

Table 2.

Interviews: Factors affecting implementation of the Delegate Model^a

CFIRa Domain	Main Factorsb
Intervention characteristics	Limited experience with mutual consultation and trust in teams; complexity (interaction with current workflows)
Process of implementation	[Staffing and training issues were not salient in interviews in clinics; they are mainly described in logs, debriefs, and documents].
Individual and team characteristics	PCPb skills (including electronic record use; willingness to delegate tasks to Care Team MAs)b; PCP trust in skills of MAs; MA learning, teamwork skills and attitudes; clinic MA shortages; knowledge and information about Delegate Model and MA training content
Inner setting	Practice culture—stress on physician autonomy; implementation climate—readiness for change (engagement of clinic leaders; relative priority of intervention, expected benefits); resources (funds, MA staffing);
Outer setting	[Movement toward value‐based payment and ACO implementation was not salient in interviews in clinics; it is mainly described in logs, debriefs, and documents].

^aFactors were coded in terms of domains and constructs in the Consolidated Framework for Implementation Research (CFIR). Construct and subconstruct names are modified here for clarity.

^bPCP = Primary Care Provider.

Care Team MA = New medical assistant (MA) role created by Delegate Model.

The researchers also analyzed data routinely gathered by PCHC for operational uses and reporting to regulators and payers. These data included metrics on provider performance related to preventive care, efficiency (encounters per hour; time from patient's arrival until departure); continuity (percentage of patients seeing their own clinician); and patient access (time to third next available appointment). Data were adjusted for provider turnover and leaves of absence.

4. REDESIGN INITIATIVE

4.1. The delegate model and its implementation

When the redesign initiative began, PCHC provided diverse ambulatory services at 15 clinics serving more than 60 000 patients, two thirds of whom were elderly or low income. Primary care providers (PCPs) included primary care physicians, doctors of osteopathy, nurse practitioners, and physician assistants. PCHC was experiencing growing demand and had substantially expanded its services.

In anticipation of a shortage of providers, PCHC sought to enhance and further expand primary care. Planned changes in care roles and workflows drew on previous initiatives to enhance the role of MAs41, 42, 43 and reflected PCHC leaders' interest in reducing PCP burden and improving teamwork. Additionally, the leaders expected the proposed team redesign to encourage “explicit tactical and strategic thinking about individual care and about work processes and systems.”44 Together, these changes would lead to greater clinic productivity and patient access. The changes would also improve chronic care, prevention, and staff satisfaction, while reducing PCP burnout.

The planned redesign, called the “Delegate Model” by PCHC, is summarized in Table 1. When it was proposed, PCHC's primary care teams consisted of dyads providing care for an assigned patient panel and composed of a PCP and a “Clinical” MA. The Delegate Model combined two dyads into an expanded team with a shared panel; the redesigned team also added a fifth team member, a full‐time “Care Team MA” (also called “Administrative MA” in contrast to the Clinical MA).44 After about 30 hours of special training, the Care Team MA was to assist other MAs, follow an expanded set of standing orders, and assume sole responsibility for a range of care and administrative tasks previously conducted by each PCP and assigned MA. The new orders covered pre‐visit planning, follow‐up after hospitalization, routine prescription renewals, schedule management, in‐box management, and routine auxiliary referrals. The redesigned teams were to meet regularly to review cases and monitor work flow. Figure 1 compares current patient flows and referrals to those planned for the Delegate Model

Table 1.

Delegate model for team redesign^a

Focal Element	Redesign
Team composition	Merges two existing dyads (PCP and assigned “clinical” MA) and adds one Care Team (“Administrative”) MA
Panel	Combines the patients previously assigned to the two dyads
PCP role	PCPs continue to serve their assigned panel and oversee assigned clinical MA. Now, they also substitute as needed for the other PCP; PCPs jointly oversee the Care Team MA.
Care Team MA role	Care Team MA supports patient care coordination, prevention management, information organization, and practice improvement; does so by assisting both PCPs in pre‐visit planning, electronic in‐box management, and scheduling, routine prescription renewals; identifies patients for routine referrals, including referrals to care managers.
Processes	PCP oversight of Care Team MA; enhanced teamwork; regular team meetings to review cases, and monitor work flows
Expected improvements	PCP productivity, care coordination; chronic care; prevention, provider/staff satisfaction; burnout.

^aPCP = primary care provider (physician [MD or DO], nurse practitioner), or physician assistant.

Care Team MA = New medical assistant (MA) role created by Delegate Model, called “Administrative MA,” in Figure 1 to distinguish it from existing role of “Clinical MA.”

Figure 1.

PCHC Primary Care teams before and after the Delegate Model redesign^a

The “Change Team” responsible for implementing the redesign included the system's recently hired Director of Quality Management. She was a Nurse Practitioner and former operations director in a care system, experienced in introducing changes in clinic practice and coaching staff. The team lead was a family physician and former medical director. He played an active role in PCHC senior leadership, becoming Chief Medical Officer and Vice President for Medical Affairs shortly after the change program's launch.

Before the two change leaders joined PCHC, they had implemented the Delegate Model in another, smaller, federally qualified health center. Shortly after joining PCHC, they convinced its senior leadership team of the value of broadly implementing the Model across PCHC and submitted a detailed spread strategy. According to this strategy, PCHC would launch the Delegate Model by redesigning a single team in each of five selected clinics. These teams would then serve as champions, spreading the redesign in their clinics and others.

The Change Team envisioned that by the end of the third year, most PCPs in PCHC's nine primary care clinics would be members of redesigned teams using the new Model. The redesign was predicted to increase care quality and clinic productivity, yielding an average of two additional patients per team per day. Resulting revenue gains from these patients would offset the salaries of the additional MAs needed for the redesign.

4.2. Setbacks

In practice, the Change Team encountered difficulties implementing and spreading the team redesign. Nearly three years after it began, the redesign had not spread as anticipated and seemed to yield mixed outcomes. During our study only three of the five originally targeted clinics developed a redesigned team. Eight such teams were launched across PCHC in six different practices, and seven remained operational. By the end of the study, less than a quarter of the system's PCPs had become members of redesigned teams.

Table 2 shows the main factors affecting implementation reported in the interviews, as coded in terms of categories in the Consolidated Framework for Implementation Research.7 The following discussion synthesizes findings from all data sources.

The main sources of the Delegate Model's failure to spread included concern about its feasibility among medical and practice directors and resistance among some PCPS. The directors expressed skepticism about the redesign's capacity to deliver the expected results because of insufficient staff to cover additional patients. Additionally, interview participants said PCPs were already “maxed out,” and some PCPs were unwilling to “let go” and delegate responsibility to MAs. These PCPs were used to responsibility for the tasks to be delegated to Care Team MAs and sometimes lacked confidence in their MA's ability to handle these tasks.

Slow staffing and training for Care Team MA positions was a second immediate cause of the disappointing rate of spread. The quality lead encountered difficulties in recruiting Care Team MAs and conducting 30‐hour training sessions, because practices were often short‐handed for MAs and needed designated Care Team MAs to replace missing colleagues.

PCHC's system data did not show short‐term improvements in operations and clinical outcomes as expected. However, our qualitative interviews provided a more optimistic impression of effects of newly expanded teams. Participating providers and practice managers in all six practices with Delegate Model teams reported that the redesigns improved pre‐visit planning and PCP workloads. The more mature the teams, the more effusive the praise. Engaged providers and practice managers also said that better previsit planning and the new standing orders were contributing to prevention, chronic disease management, and patient flow. For example, a provider reported often using the Care Team MA for documentation and hospital follow‐up. Another described how advanced availability of patient histories and medication lists reduced disruptions and made it easier to stay on schedule. Describing similar results, a clinic project director added that the participating PCP “is smiling more.”

Team work reportedly reduced burnout. For example, a site Medical Director, who also led a redesigned team, said he had “failed” a burnout quiz a year ago. Now he is “just as busy” but has “a lot more satisfaction in my job, in primary care, and a general happiness in my life.”

4.3. Redesign outcomes

Our quantitative re‐analysis of PCHC operational data allowed for some comparisons of performance of Delegate Model teams to that of conventional PCHC teams. These short‐term data reinforce the reporting of setbacks thus far. The data show only modest improvements during the study in preventive care for redesigned teams, compared with conventional care teams. The improvements included higher rates of blood pressure control, cervical cancer screening, and influenza immunization. These rates each improved at two sites with redesigned teams, and higher mammography rates were attained at one such site. No sites with redesigned teams showed improvements in efficiency, care continuity, or access. These findings were not yet available when the learning episodes reported below took place. Moreover, they suffered from methodological limitations, including small samples and limited time frames. For these reasons, we do not report them in detail.

5. LEARNING EPISODES

In responding to setbacks in the pace and effects of the redesign, the Change Team made important incremental adjustments to implementation processes and plans.45 For instance, to reduce the burden on clinics of releasing MAs for Care Team training, the quality lead extended the required hours of training over a longer time period.

Additionally, as the change leaders joined with PCHC's other senior leaders in analyzing and diagnosing effects of organizational and external contexts on setbacks, they found ways to address underlying program difficulties, while fostering system‐wide learning and improvement. Table 3 provides a summary of episodes in which such organizational learning opportunities occurred. The table is organized in terms of setbacks in the implementation and spread of the Delegate Model. Each row describes learning that occurred in response to the setbacks.

Table 3.

Organizational learning in response to implementation setbacks^a

Setback	Problem/Challenge	Context	Responses	Results
Delays in training Care Team MAs and formation of redesigned teams	MA shortages across system, reflecting turnover and difficulties in recruitment	Non‐competitive pay and few advancement opportunities	System‐wide pay raise; new position of Senior MA established for entire system	Reduced MA turnover. Larger pool of recruits for Care Team MAs, but some MAs trained for that role prefer to apply for Sr. MA position.
Difficulty implementing standing orders for Care Team MAs	Poor fit of new orders for Care Team MAs with existing workflows	Inefficiency of current clinic workflows and insufficient standardization	Redesign of relevant workflows in clinics with DM team; spread of new standing orders to other clinics	Improved DM implementation; improved orders and workflows in DM clinics; more standardized care in clinics having DM teams and those that do not
No productivity gains	PCP resistance to DM's productivity objectives	Experience of heavy workloads among PCPs; PCP resentment that they were not seen as sufficiently productive	Downplay of DM productivity objective; removal of productivity payments for all PCPs	Greater receptiveness to DM among PCPs
No current or likely revenue gain; no improvement in patient access or physician burnout	Added costs for salary and training of Care Team MAs without revenue gain	Shift among payers and regulators toward value‐based payment	Deployment of DM across system to support team‐based, preventive, and chronic care	Endorsement by senior leadership of DM and its continued spread; better positioning of PCHC for value‐based payment

^aCare Team MA = New medical assistant (MA) role created for Delegate Model redesign.

DM = Delegate Model.

PCP = Primary Care Provider.

PCHC = Penobscot Community Health Care.

As row one in Table 3 notes, there were delays in training Care Team MAs and in formation of redesigned teams. Shortly after beginning the Delegate Model initiative, the Change Team recognized that MA turnover and absenteeism reduced depth of clinic staffing and made clinic leaders reluctant to release MAs for Care Team training. Moreover, staffing difficulties led clinic leaders to assign routine tasks to already‐trained Care Team MAs thereby reducing opportunities for Care Team MAs to perform their distinctive tasks.

The Change Team and PCHC central leadership also knew that MA turnover and recruitment difficulties caused shortages of MAs across PCHC and added to care delivery problems. Now, these leaders acknowledged that training delays and staffing shortages both resulted partly from PCHC's failure to offer locally competitive pay rates.

In response, PCHC made MA pay improvements and added a Senior MA position during the second year of our study. The new Senior MAs began to help train and supervise MAs, occasionally moving among practices as needed. These changes contributed to reductions in MA turnover and helped expand the pool of MAs eligible to become Care Team MAs, according to our informants. The change leader described the Senior Medical Assistant role as “an improvement for our whole system.”

Despite its system benefits, introduction of this new MA position was reported to have an unintended result for implementation of the Delegate Model. After completing training for Delegate Model teams, some MAs chose instead to apply to become Senior MAs. This development slowed formation of newly designed teams but helped Senior MAs trained for the Delegate Model spread its new care procedures to settings without expanded care teams.

A second implementation setback, shown in Row 2 of Table 3, emerged as the quality lead encountered difficulties introducing standing orders for Care Team MAs in newly formed teams. The new orders did not fit prevailing practices and workflows in some clinics. Moreover, there were wide variations among clinics in existing procedures related to newly assigned tasks for Care Team MAs (see Table 1), and many existing routines were inefficient. When the quality lead uncovered these current inadequacies, she worked more intensively with the clinic's PCPs and with practice directors to redesign workflows and tailor them to the clinic's specific needs. These redesigned workflows contributed to standardization and efficiency of care in entire clinics, as well specifying the role of the Care Team MA. The Change Team spread some of the new MA standing orders to additional PCHC clinics that did not have Delegate Model teams.

The third learning opportunity (Table 3, row 3) came as the Change Team encountered physician resistance to adopting the Delegate Model and learned from operational data that providers in expanded teams were not seeing more patients per day, as expected. Gradually, PCHC leaders took into account broader physician concerns about overwork and resentment of the system's attempts to raise their productivity. But initially leaders did not seem to understand that resistant physicians viewed the redesign as just one more attempt to raise their productivity. One interviewee bluntly characterized this situation: “[The Change Team members], believe that if this [Delegate Model] works efficiently, the providers can take on two more patients a day, which would pay for the [Care Team] MA. The providers don't want to hear that.”

Recognition by the Change Team and other senior leaders of this source of resistance triggered a decision by senior leadership to make a major system change that had had been considered earlier for other reasons: elimination of productivity‐based pay components for all PCPs and transition to fully‐salaried pay. The redesign's leader viewed this movement from a “provider productivity incentive to a straight salary system,” as likely to “improve some willingness [of PCPs] to delegate [responsibilities to MAs].”

By the close of our data gathering, elimination of productivity incentives was indeed reported to have helped Delegate Model implementation and to contribute to a change in orientation in PCHC as a whole. The Medical Director of one clinic, and a member of the longest‐standing redesigned team, described developments this way:

We changed the incentive structure for providers—[Now] it is not about getting paid an extra $10,000; it's [about] providing quality care and work‐life balance and solving burn out—spending more time doing what we are trained to do.

The fourth setback became evident toward the end of the study when PCHC leaders and practice directors considered whether to continue implementing the Delegate Model, despite slower than expected spread and lack of well‐documented operational and clinical results. Although early operational and quality data were disappointing, senior leadership had heard reports from clinics with redesigned teams and from the Change Team that the redesign had begun to help improve preventive and chronic care. The Delegate Model also seemed to reduce physician burnout. Moreover, several clinic directors demonstrated their commitment to the Model by including funding for Care Team MA salaries in their clinic's annual budget.

The team redesign thus seemed to offer senior and clinic leaders a way to speed primary care's movement toward more comprehensive preventive and chronic care. This trend would improve PCHC's performance on population health metrics, which were becoming more common in reporting and value‐based payment. PCHC's application to become an Accountable Care Organization (ACO) and its recognition as one by the Centers for Medicare and Medicaid Services provide clear indications of the commitment of PCHC leadership to adapt their system to this changing payment and reporting environment.

Leaders viewed the Delegate Model redesign as likely to help PCHS operate effectively as an ACO. The leader of the Change Team repeatedly stressed this prospect in discussions with colleagues on the executive team. One characteristic log entry states, “As we move toward value‐based purchasing, the team model becomes more important; [it results in] having a more in‐depth knowledge of the [patient] panel.” Another entry adds “[We] need to make decisions in the future of the organization towards quality payment. This [Delegate] Model is a significant part about being ready for the change.” Leaders in some clinics using the Delegate Model also described this potential contribution of the Model.

In justifying continuation of the redesign effort, the leader of the change program and some colleagues thus distanced themselves from the Delegate Model's envisioned contribution to physician productivity. Instead, they stressed the redesign's capacity to improve quality, prevention, and chronic care. These original program objectives, which now seemed attainable, fit well with PCHC executives' growing recognition of the need to adapt quickly to the changing payment environment.

In summary, the Delegate Model did not spread as rapidly as anticipated or deliver anticipated gains in efficiency and access. Instead, it faced setbacks in the degree and timing of spread across clinics. The Change Team and other PCHC leaders identified immediate problems and challenges responsible for these setbacks and developed some incremental solutions, such as staggering MA training sessions. Additionally, the Care Team identified internal organizational and external contexts influencing implementation problems and suggested responses to these conditions. Besides supporting the redesign, these responses would likely help PCHC face challenges affecting the entire delivery system.

Since completion of our study, as reported by the redesign's leader, PCHC has retained the fruits of organizational learning during the Delegate Model's implementation, including the new system‐wide pay arrangements, the Senior Medical Assistant role, and new MA work orders. PCHC leadership remains committed to deploying the Delegate Model because of its potential contribution to teamwork, prevention, and chronic care and due to alignment of these processes with the system's mission and with value‐based payment. Senior leadership's expectations for the team redesign seem plausible and will be tested as PCHC spreads the redesign more widely and undertakes more value‐based contracting.

6. DISCUSSION

In reaction to implementation setbacks, the Change Team and PCHC executives engaged in single and double‐loop organizational learning about influential conditions in the system's organizational and environmental context. This learning affected PCHC as a whole, as well as the Delegate Model program. The resulting organizational learning supported PCHC's adaptation to changing external conditions and may have reduced threats to the system's revenues, operations, and performance.

The long‐term spread and outcomes of the redesign initiative are not yet known but seem promising. More critical for our analysis are potential effects of the system‐wide developments that were triggered in part by learning from the redesign's implementation setbacks. These system‐wide changes enhance PCHS's position in the local labor market and remove one source of PCP dissatisfaction. Hence, the changes position PCHC to recruit and retain care staff more effectively and may enhance staff satisfaction. PCHC's system‐wide changes help move the system away from fee‐for‐service medicine and toward alignment with value‐based payment and quality reporting. Further contributions in this direction come from the reduced emphasis on physician productivity and from spread of new orders for MAs.

6.1. Conditions for organizational learning

Several conditions supported this systemic learning. Three are often associated with group and organizational learning in health systems1, 32 and other settings.32, 33, 46 First, the Change Team consisted of two individuals with diverse skills and experience. This diversity likely contributed to their consideration of a wide range of explanations for setbacks and possible responses to them.

Second, there was honest and frequent communication about the redesign initiative between the Change Team, senior leaders, and clinic staff. Most of this communication flowed through the Change Team leads, who were members of PCHC's senior leadership and also maintained close contact with clinic staff. The change leaders' interactions with other system leaders helped speed the flow of information about implementation of the redesign up and across PCHC. Moreover, the Change Team's engagement with senior leadership seems to have enhanced the change leaders' awareness of organizational priorities and contextual forces impacting PCHC as a whole. This awareness apparently facilitated the Change Team's quick shift toward presentation of the Delegate Model as significant contributions to system readiness for value‐based payment. In a similar fashion, the Change Team's understanding of the progress of the redesign and its effects came largely from interchanges with clinic leaders and clinic staff and from the quality lead's direct engagement with clinic workflows.

Third, the Change Team recognized that assessing the redesign's program's implementation progress and interim effects was necessary for mid‐course learning and for adjustments to the Delegate Model and its implementation strategy. They also saw these assessments as opportunities to learn how best to maintain support for the change program among PCHC leadership and clinic staff. Fourth, PCHC's senior leaders supported experimentation in care redesign, allowed for midcourse adjustments to the Delegate Model initiative, and accepted changes in its rationale.

A fifth condition was crucial to enabling the learning that aided systemic adaptation to external developments. PCHC's executive leaders were already reassessing their system's strategies, goals, and operations in light of value‐based payment and were willing to consider implementation of the Delegate Model within this broad frame. Without such openness among top leadership, the Change Team would have had fewer opportunities to learn from implementation setbacks and less room to consider and suggest steps that would benefit all of PCHC, as well as the change program.

6.2. Research implications

This case study contains several implications for research on intended organizational change in health care, some of which may also apply to other industries. First, when researchers assess effects of programs, they need to look beyond a program's stated objectives in determining its success. The most important judges of program success are leaders of the organizations in which it operates, other members and stakeholders, and change leaders. Throughout program implementation, all parties may renegotiate the criteria for program success,47 as occurred during the Delegate Model initiative. Assessments should also consider positive and negative effects less directly related to success criteria, including contributions to organizational learning, system‐level effects, and development of capacity for future learning and improvement.48, 49

A second implication is that contextual influences appear at several organizational levels within a delivery system. Some frameworks for research on change in health care identify contextual factors at these multiple levels.7, 8, 11 Research on intended organizational change can benefit from consistent application of a multi‐level approach throughout the research process. Third, it is important for evaluations of organizational change programs to examine effects of influential external conditions, including payment and regulation. Many recent studies of change programs in health care pay limited attention to external contexts.50

Several implications apply directly to research on organizational learning in health care. First, the case illustrates the value of studying how context influences implementation and how program leaders learn about context. Second, research on learning needs careful treatment of conditions affecting organizational learning and particularly learning with system‐wide implications. To start, researchers might distinguish between necessary and merely facilitative conditions by examining developments when known facilitators are absent. For instance, analysts often cite standardized performance information as a critical precondition for organizational learning.1, 51 Yet PCHC's Change Team and system executives had to make decisions about the value and continuation of the team redesign program before they had access to robust operational or evaluation data. Lacking such data, the leaders mainly relied on informal feedback from clinic leaders. Moreover, they drew heavily on their own “practical wisdom.” This capacity came from past experience with team work and system management and from past learning about adapting to external change.

6.3. Practice implications

What implications does our study contain for the practice of system redesign and other forms of intentional organizational change in health care? First, when confronting program setbacks, change leaders and system executives can benefit the program and their organization as a whole by going beyond “short‐term fixes”52 to consider higher‐level and broader conditions affecting setbacks. These contextual conditions may affect issues that are particularly important to senior leaders and the care system. Practitioners can use multi‐level frameworks to alert them to potentially influential contextual conditions.

Additionally, change leaders can learn by applying the “5 Whys” tool to setbacks. This technique calls on analysts to repeatedly inquire why a problem arose. The questioning moves from immediate causes of a setback upward and outward to more distant contextual explanations. Identifying these more remote influences can lead to far‐reaching organizational improvements, as well better solutions of immediate problems. The example in Box 1 shows how the tool might have been applied when PCHC faced delays in identifying and training MAs to serve in expanded care teams.

Box 1. “5 Whys” Applied to Delays in MA Training

“Training for the Care Team MA is slower than expected.” “Why?”

“Suitable MAs cannot attend daily training sessions.” “Why?”

“Clinic directors are reluctant to release them from work so often.” “Why?”

“The clinics are short staffed and can't meet patient demand.” “Why?”

“It's hard for clinics to recruit and keep MAs.” “Why?”

“MAs don't think the pay is sufficient.” “Why? “.

“Other places in the area pay better.”“Then we need to adjust our pay rates to compete more effectively in local markets.”

A second practice implication is that leaders may select types of learning processes that are appropriate to their needs. If they think practical, incremental changes will be sufficient to address setbacks, they can restrict themselves to single‐loop learning processes. These include examining the fit of program features with their immediate operating context and considering ways to make implementation more efficient. Common, immediately relevant, contextual factors often include the needs, priorities, and beliefs of affected individuals; supportive infrastructure, resources, and staffing; and work routines. For example, examination of alignment of Care Team training with MAs' clinic work showed that spreading training hours would reduce burden on the clinics.

Solutions resulting from single‐loop learning will often be insufficient to address underlying problems and major challenges, such as movement to value‐based payment. In these cases and when practitioners seek radical changes in programs and their settings, they need double‐loop learning. That requires critical questioning of goals, priorities, and assumptions, and procedures within the change program and within influential domains in the parent organization and in its environment.

Third, practitioners and system leaders might gain by comparing conditions in their own organization or care system to those characterizing systems like PCHC that are reported to engage in organizational learning during implementation.1, 53 For example, does their own organization encourage and develop real‐time feedback about program implementation, as do learning systems? Does the organization assess a range of possible program consequences, or just consider attainment of a narrow set of program targets during a short trial period?

7. CONCLUSION

The PCHC case study reinforces the importance for both researchers and practitioners of examining how external and organizational contexts shape redesign of care delivery and other types of organizational change. Our analysis also illustrates the value to care systems of organizational learning in response to implementation setbacks. This report further points to the need for systematic research on conditions supporting organizational learning about change programs and their contexts.

FUNDING

The study described here was funded by the Agency for Healthcare Research and Quality (AHRQ) through Contract No. HHSA 2902010000341 to John Snow, Inc.

CONFLICT OF INTEREST

The views expressed in this paper are solely those of the authors and do not reflect those of any agency of the United States Government. Neither author has an affiliation with Penobscot Community Health Care or any conflicting or competing interests.

Harrison MI, Grantham S. Learning from implementation setbacks: Identifying and responding to contextual challenges. Learn Health Sys. 2018;2:e10068 10.1002/lrh2.10068