Using Learning Teams for Reflective Adaptation (ULTRA): Insights From a Team-Based Change Management Strategy in Primary Care

Abstract

PURPOSE The Using Learning Teams for Reflective Adaptation (ULTRA) study used facilitated reflective adaptive process (RAP) teams to enhance communication and decision making in hopes of improving adherence to multiple clinical guidelines; however, the study failed to show significant clinical improvements. The purpose of this study was to examine qualitative data from 25 intervention practices to understand how they engaged in a team-based collaborative change management strategy and the types of issues they addressed.

METHODS We analyzed field notes and interviews from a multimethod practice assessment, as well as field notes and audio-taped recordings from RAP meetings, using an iterative group process and an immersion-crystallization approach.

RESULTS Despite a history of not meeting regularly, 18 of 25 practices successfully convened improvement teams. There was evidence of improved practice-wide communication in 12 of these practices. At follow-up, 8 practices continued RAP meetings and found the process valuable in problem solving and decision making. Seven practices failed to engage in RAP primarily because of key leaders dominating the meeting agenda or staff members hesitating to speak up in meetings. Although the number of improvement targets varied considerably, most RAP teams targeted patient care-related issues or practice-level organizational improvement issues. Not a single practice focused on adherence to clinical care guidelines.

CONCLUSION Primary care practices can successfully engage in facilitated team meetings; however, leaders must be engaged in the process. Additional strategies are needed to engage practice leaders, particularly physicians, and to target issues related to guideline adherence.

INTRODUCTION

The quality of care in the United States is substandard,¹ and the early promise of improving care by translating research into practice has been disappointing.^2,3 Initial efforts to improve quality often target improving knowledge, attitudes, and behaviors of individual health professionals by using such strategies as audit and feedback, reminder systems, continuing medical education, and educational outreach.⁴ These strategies have been found to produce modest change.^2,3,5–8 Even when improvement changes are adopted, they are often not sustained over time⁷ and may deteriorate after practice members’ attention shifts elsewhere.⁸ Sustaining change appears to be an active process that requires continual attention as innovations are adapted to fit continually evolving environments.^9,10 Additionally, small, independent primary care practices often lack the resources or motivation needed to develop quality improvement strategies that can address multiple clinical issues.¹¹

The substantial, broad improvements required for optimal primary care cannot be achieved by focusing improvement efforts on specific diseases or on individual professional behavior. In fact, primary care practices are being asked to enact multiple guidelines simultaneously to achieve better care. That the difficulty in so doing does not appear to be from a lack of knowledge^12,13 suggests many of the current deficits in primary care require practice-level organizational change,^14–17 such as the creation of more effective clinical teams, better coordination of care, improved information management, office systems that enhance patient experience, and health system changes that would reward these activities.

Growing evidence indicates organizational interventions can effect change in health care delivery that improves both health care processes and patient outcomes.^16,17 Making practice-level change is complex, however, and many widely used approaches treat these organizations like a machine with broken parts that need fixing one at a time.¹⁸ Recently it has been suggested that quality improvement interventions should approach primary care practices as complex adaptive systems, acknowledging interdependencies within and between the practice and its local environment.^19–21 This theoretical perspective helps explain why making successful and sustainable change in practices requires systematic attention to salient characteristics within the practice as well as the environmental context,^22–25 while, at the same time, paying particular attention to the practice’s capacity to undertake and sustain change.^11,18,26,27

During the past 15 years, our collaborative team worked on a series of descriptive and intervention studies designed to improve quality of care in primary care practices.²⁸ One of these earlier studies, the Study to Enhance Prevention by Understanding Practice (STEP-UP) resulted in substantial and sustained increases in a global measure of preventive services delivery.^29,30 There was, however, tremendous variation among intervention practices, with some making dramatic changes and others making none at all. A secondary comparative case analysis of qualitative data led to a practice change model that highlighted the importance of organizational attributes, particularly communication, trust, and relationships among practice members.³¹

Insights from this program of research led to the Using Learning Teams for Reflective Adaptation (ULTRA) study, a group-randomized quality improvement intervention trial of 31 intervention and 31 control practices from the New Jersey Family Medicine Research Network and the Lehigh Valley Health Network (EPICNet). Five intervention and 5 control practices withdrew during the course of the study. This trial evaluated the effectiveness of facilitated practice improvement teams, called reflective adaptation process (RAP) teams, in improving adherence to guidelines for multiple chronic diseases. A major goal of RAP teams was to provide a safe forum in which diverse practice members could communicate about the practice issues they found most pressing.³² To foster collaboration, motivation, and participation, RAP teams were encouraged to select a topic they identified as most critical, rather than being asked to address a particular clinical target. We hypothesized that RAP team meetings would model effective communication strategies that would then spread to other parts of the practice and eventually lead to improved adherence to care guidelines across multiple clinical targets.

Because the ULTRA intervention did not result in significant improvements in adherence to multiple chronic diseases care guidelines,³³ we used qualitative data from 25 intervention practices to understand (1) whether and how primary care practices could implement and sustain a team-based collaborative change management strategy and (2) how practice improvement teams identified and addressed important practice change issues.

METHODS

The ULTRA intervention included a multimethod assessment process (MAP)³⁴ and facilitated quality improvement RAP teams.¹⁸ MAP involved collection of data over a 2-week period by a qualitative researcher who systematically observed different practice activities, conducted in-depth interviews with key informants including clinicians and staff, and wrote descriptive field notes. Field researchers, in collaboration with the larger research team, then synthesized MAP data and generated a MAP summary report for distribution to practice members. This report summarized each practice’s values, strategic vision, readiness to change, motivators, and relationships and was intended to inform the subsequent improvement process.

After a review of the MAP summary report with both the practice leadership and the entire practice, a 7- to 9-member RAP team was assembled in each practice. Teams typically included a physician, representatives from both clinical and administrative areas of the practice (eg, a clinical staff member, a receptionist, a billing specialist, etc), and a patient. A facilitator, who usually was also the MAP field researcher, led the team through up to 12 weekly 1-hour meetings. The study had 12 facilitators who were predominantly doctoral level professionals from diverse backgrounds, which included anthropology, sociology, nursing, psychology, and social work. They were well versed in qualitative data collection methodologies (participant observation and conducting depth interviews) but did not necessarily have advanced facilitation skills. They were all trained in basic facilitation strategies to ensure consistent implementation of the intervention. The facilitator was instructed to guide the RAP team through the application of practical quality improvement tools and methods, which included the following: (1) establish ground rules and clear objectives for each meeting; (2) identify and prioritize issues through brainstorming; (3) identify factors leading to unwanted variation; (4) plan and implement an intervention strategy; and (5) evaluate changes through additional data collection and reflection.³² During the last 3 to 4 meetings, facilitators trained an apprentice drawn from each RAP team to transition into the facilitative role to help ensure that the practice could sustain the improvement process.

The UMDNJ (University of Medicine and Dentistry, New Jersey) and Lehigh Valley Health Network institutional review boards reviewed and approved the protocol and all practices signed informed consent forms.

Qualitative Data Collection, Management, and Analysis

Qualitative data included baseline MAP field notes, audio-recorded interviews of key practice stakeholders, and audio-recorded RAP meeting discussions. The content and process of each RAP meeting was summarized in field notes. To guide practice observations and ensure comparability across practices, facilitators used an observation template that included staff involvement in office operations, office organization, and decision making, as well as the physical attributes of the practice. Facilitators attended weekly meetings of the larger study team throughout the MAP and RAP processes to share and discuss their observations and experiences and to identify areas for follow-up data collection and strategies to facilitate the change process.

Six months, 1 year, and 2 years after the completion of RAP, a study team member made 1- or 2-day follow-up visits to each practice to conduct interviews with practice members to assess the longer-term effects of the intervention. When possible, interviews were conducted with both RAP team members and nonmembers. Interviewees were asked whether the practice had continued RAP, and if so, what form the meetings had taken, and what changes, if any, had resulted. Facilitators probed for information about how RAP had affected work relationships, practice-wide communication, and whether the practice as a whole was involved with the process. Qualitative data were de-identified using pseudonyms for staff names and practice locations and names.

Qualitative analyses for this study involved a series of consecutive steps of coding and synthesizing. To create a coding template and to begin to understand potential responses to the intervention, the analysis team initially identified 2 intervention practices, one showing maximum improvement in adherence to diabetes and hypertension guidelines and another that substantially improved its office function. The team used a group editing strategy, reading the qualitative data aloud to identify and highlight informative text.³⁴ Highlighted data were reviewed in a second iteration to develop practice summaries detailing the course and outcomes of the RAP process. From these summaries a tabular coding framework was constructed that included RAP targets, changes made as a result of RAP, and evidence of changes in communication and/or relationships. Using this coding framework, 2 investigators (B.A.B. and S.M.C.) analyzed the remaining practices to fill in information about these key categories for each practice. Relevant text associated with each category was compiled in this tabular format to create a 30-page table with information on each practice.

The table was analyzed using an immersion-crystallization approach³⁴ to categorize practices according to their ability to engage in the RAP team process, identify change target(s), and make changes. The tabular framework was also used to ascertain whether team-selected RAP targets were successfully completed. In the final stage of analysis, a subsample of 6 practices was purposefully selected to explore variation across the intervention experience. Selected practices represented the range of intervention implementation, ranging from total failure to full implementation. Three investigators (B.A.B., S.M.C., and D.J.C.) read these practices independently using an immersion-crystallization approach and then met regularly to compare and discuss emerging insights and to identify case examples to support or negate our findings.

RESULTS

Qualitative descriptions of the intervention were based on 25 practices that had adequate qualitative data. Of these practices, 19 were group practices and 20 were owned by independent clinicians. The results of the qualitative analyses provide insights into implementing a team-based change management strategy and the types of issues they addressed.

Engagement in a Team-Based Collaborative Change Management Strategy

Before the ULTRA intervention most practices did not adhere to a consistent meeting schedule. One-half of the sample (13 of 25) either held no meetings at all or held them infrequently before the intervention began. Before the intervention, 12 practices indicated that they held at least 1 consistent monthly or weekly meeting of some kind; however, only 4 consistently had meetings with staff members represented from all functional areas of the practice. Despite this lack of meeting history, 18 out of 25 intervention practices were successfully able to convene RAP teams, identifying and addressing potential areas of improvement. These practices each held at least 10 RAP meetings with regular attendance by members representing different parts of the practice and collaborated in brainstorming, planning, and implementing change. For example, in practice 58, the RAP team addressed the problem of missing patient charts. To address this problem, they tracked chart pathways through their office, drawing on cross-practice representatives to understand the reasons for pulling and transporting charts. Using this information, the team piloted strategies for eliminating “hot spots” of temporary chart loss and changing how charts were handled. In practice 47, the RAP team addressed the issue of keeping physicians on schedule to reduce patient wait time. The RAP team worked together to create a dialogue between team members and physicians who were not on the team. By doing so, they were able to surface and address unacknowledged tensions that had plagued the practice for some time.

Seven practices (of 25) were unsuccessful at engaging in the RAP process. In each case, a key leader, primarily the physician member dominated the meeting agenda. In 3 practices, lead physicians or office managers refused to relinquish control of the meeting agenda by directing conversation or shutting down critical areas of reflection. For example, in practice 34, the lead physician and office manager operated as a faction, working together to shut down topics they disliked and preserve their control of the agenda. Other team members attempting to introduce discussion of practice problems eventually gave up in the face of this tag team opposition. In practice 14, the lead physician publicly supported RAP to the research team, but in private, resisted it, first attempting to withdraw and later subverting discussion. Consequently, the team could not function effectively despite the support of the office manager.

Staff hesitance to speak in presence of practice leaders out of a concern of not being taken seriously or fear of belittlement was another factor in practices’ being unsuccessful at engaging in RAP. For example, in practice 35 when staff shared an experience of “…[and] I get flustered,” the office manager responded with, “Yeah, you do. You really do.” This reinforced the notion that the practice’s problems were the fault of the staff rather than systemic issues to be addressed by the team as a whole. Team members learned quickly and stopped talking; thus, lack of safety became a barrier to staff participation in the change process. In practice 42, team members were reluctant to speak. When the facilitator questioned the team about their reticence, only a single person replied, saying that she spoke up in RAP meetings because “others are afraid to…they fear retaliation if they go against [the doctor’s] vision of how things should be or if they are critical.”

In 2 practices there was only superficial support from physician leaders. For example, practice 15 had only 4 RAP meetings at which time the office manager mentioned that they had accomplished their goals. Also, the physician leader was absent for 3 of these meetings. Thus, the practice RAP team went through the motions of implementing the intervention, but only superficially.

All 18 practices that engaged in RAP were able to identify improvement targets and make changes. Furthermore, 8 of these 18 practices continued using RAP in some form through the 2-year follow-up data collection point. Most practices that sustained the intervention were at a turning point when RAP was introduced. They used the intervention to organize themselves and, with time, found the process valuable in problem solving and decision making. For example, in practice 17, 1 physician was buying out the practice at the time that RAP was introduced. This physician leader was supportive of the group process and cognizant that her involvement should not “stifle the conversation.” Also, 2 years after the intervention ended, practice members saw value in continuing to meet, as noted in the following quotes: “This helped us learn how to communicate better,” “this was empowering,” “meetings helped teach us how to problem solve,” and “I learned to stop and understand the process.” Practices that sustained the intervention often adjusted the structure and format of RAP meetings by adapting the process to their own specific needs. For example, practice 10 introduced a process of rotating RAP team members every 4 to 5 months to ensure that all members of the practice would be represented. Practice 17 transitioned its RAP meetings into separate physician and staff meetings but continued to incorporate elements of the process to preserve practice-wide involvement in problem solving.

There was evidence of changes in practice-wide communication after the intervention. In 12 of the 18 practices that had engaged in the RAP process, 1 or more practice members reported improvements in practice-wide communication as a result of RAP. Members of practices in which lead physicians remained engaged in RAP and encouraged discussions were especially likely to report improvements in communication. For example, throughout the RAP process in practice 39, the lead physician encouraged discussion, inviting new ideas and refocusing the group whenever conversations strayed too far from the problem at hand. The team eventually expanded to include staff from a second office that had recently been purchased by the lead physician. When asked about the value of RAP meetings, the lead physician explained that,

…meeting once a week has made our practice run so much smoother. We were having problems a year ago between the offices, but they’ve almost disappeared now. We make sure that new people always come to the meetings right away. They make people better at teamwork. This fosters collaboration. We use it to get a lot accomplished.

Practice Change Issues Addressed by RAP Teams

The range of improvement targets on which RAP teams chose to work are described in the Table 1▶. Interestingly, not a single practice focused on improving adherence to clinical care guidelines. Most practices targeted patient care–related issues (eg, improving charting or access to practice) or practice-level organizational improvements (eg, easing staffing issues, leadership, or cross-practice communication). All teams were able to generate lists of their core issues and subsequently address 1 or more of them, although not all issues were resolved. For example, the RAP team at practice 17 noted that communication around the issue of prescription refills was a persistent problem. First, the team diagrammed the prescription refill process by observing it in real time and collecting data to assess where the process broke down. The team then concluded that patient telephone messages requesting prescription refills were often unclear, missing dosages or misspelling drug names. As a result staff members spent precious time trying to contact the patient or his/ her physician to obtain detailed and accurate information. To address this problem, the team pilot-tested potential solutions until they found a strategy that reduced the number of unclear prescription refill messages. They also designed a plan to continue measuring improvements every other month to find out whether the improvements were sustained with time. This practice used only 6 RAP meetings to complete their first improvement target. Subsequent RAP meetings were used to tackle other issues related to communication improvement and structural reorganization. The team continued to meet after the initial facilitated 12-week period, sustaining improvement activities over the entire 24-month observation period.

Table 1.

Improvement Areas Addressed in RAP Meetings

Types of Improvement Areas	Specific Issues Addressed in RAP Meetings
Patient care–related areas
Improve patient flow	Increase staff ability to schedule tests
	Front desk procedures
	Clinician time with patient
	Open access
Improve patient access to practice	Encourage patients to see new clinicians
	Telephone system
	Referrals
	Prescription refills
	Call backs with test results
Improve “interpersonal“ communication with patients	Patient surveys, patient satisfaction
	Patient education materials
Clinical Issues	Educating staff about clinical processes (eg, eligibility
	for influenza vaccines)
Charting	Improving chart documentation
	Chart organization and culling
	EMR implementation
Practice improvement areas
Staffing issues	Turnover
	Vacations
	Clinician style
	Clinical support staff
	Staff style
Improve interpersonal staff communication	Teamwork
	Respectful interactions (bridging work “silos”)
	Improving communication between practice and hospital
Improve workflow-related staff communication	Creating procedures manuals, communication logs
	Consistency in following protocols
	Creating job descriptions
Improve leadership (more facilitative leadership style)	Increase staff appreciation
	Better decision making
	Updating job descriptions
Financial issues	Improve reimbursement

EMR = electronic medical record; RAP = reflective adaptation process.

The number of improvement targets addressed during RAP meetings also varied considerably across practices. For instance, practice 39 identified new improvement targets at each meeting, brainstormed potential solutions, and pilot tested them during the ensuing week. Based on the outcomes of this testing, changes were implemented. This practice took on 2 to 3 new issues every week, and by the end of the 12-week process they made improvements in many areas. On the other hand, practice 17 took on 2 major practice-wide issues (communication improvement and structural reorganization) and used weekly RAP meetings to gradually plan incremental changes that were instituted throughout the practice and that led to improvements in both areas. In essence, practice 17 operated from a systems perspective, whereas practice 39 preferred to address problems as they emerged.

DISCUSSION

The ULTRA study hypothesized that facilitated RAP meetings would stimulate new conversations, which would, in turn, lead to broad changes in practice operations. RAP meetings were designed to allow practice members to self-select issues they deemed important in hopes of motivating stakeholders to work together and make substantial, lasting changes. It was further hypothesized that providing a diverse group of practice members with the time and space for reflection and communication would result in specific chronic disease care improvements. Instead, we found that when given a choice, RAP team members identified patient care and practice improvement issues related to communication, access to care, and access to information. They then brainstormed potential solutions and implemented changes that addressed these issues; however, not a single practice focused on improving adherence to specific chronic disease care guidelines.

ULTRA used a mixed methods design in which qualitative process data were collected simultaneously with quantitative trial data to test and refine key components of the Practice Change and Development Model.³¹ Analysis of the qualitative data permitted us to understand what actually happened in ULTRA practices, generating new understandings that can be applied to our theoretical model and the next generation of studies. For example, we found that meetings in which participants held very different degrees of power were at times affected by the emergence of psychological safety issues, which must be addressed in future interventions. Also, the qualitative data uncovered the nature of the dual hierarchy, which created unanticipated barriers to communication and relationship building in some practices.³⁵ Furthermore, in our enthusiasm to involve staff from all parts of the practice, we underestimated the need to have multiple clinicians engaged in the intervention process. New insights from this study, as well as several follow-up studies, are informing the creation of a more refined intervention model.²⁶

Although we were largely successful in creating RAP teams that included the diverse perspectives of a variety of practice members, teams were primarily composed of practice staff. In general, RAP teams included only a single physician, who was often distracted by patient care interruptions. Once constituted, RAP teams had to learn new ways of relating to each other, since most participants had limited experience in contributing opinions and sharing ideas in a team context. Given the opportunity to be part of a group composed primarily of staff, RAP members often selected improvement issues that represented recurring problems faced in their daily routines or areas where they believed they had the authority to intervene. Most practices had a large backlog of pressing issues related to disruption of practice operations, patient flow, and communication. These issues were almost always seen as important by both staff and physicians. It was therefore natural and relatively safe to build consensus around these areas of common ground, staying away from what might be perceived as physician’s territories, such as guidelines and clinical care.

The ULTRA project highlights important tensions in the larger effort to improve primary care. There is a gap between approaches that facilitate a learning organization and those that rely on traditional, content-focused quality improvement. Quality improvement strategies often assume it is sufficient to implement specific mechanistic changes directed at disease-specific processes, outcomes, and guideline adherence. Although sometimes successful, these strategies have not generally been shown to lead to consistent, sustained improvements, nor have they succeeded at creating a flexible practice culture that can innovate at all levels. On the other hand, a learning practice has been described as analogous to a jazz ensemble.²¹ This alternate model suggests that interventions should seek to facilitate a continuously adaptive organization, helping practice members generate organic changes that enhance the overall functioning of the system. ULTRA was designed to identify and implement improvement targets organically by incorporating multiple voices from the practice. This approach, however, did not lead to improved guideline adherence, suggesting that new hybrid models which better integrate the principles of a learning organization with traditional quality improvement strategies are needed.

There is also tension between improving efficiency and effectiveness of care within current practice paradigms on one hand and transforming to a new model of care that takes a proactive, population-based approach built on care teams on the other. Findings from the National Demonstration Project suggest that transitioning to a new paradigm of primary care is far more complex than merely implementing a series of changes.^11,26,27,36 This finding suggests that substantial improvements in both patient-centered care and guideline adherence may require a new practice paradigm. If so, neither incremental quality improvement nor learning organization facilitation alone will work.

These results need to be viewed within the context of several limitations. First, this study was conducted in a narrow geographic region in the eastern United States, primarily among small, independent primary care practices. These practices typically did not have the resources that may be available to an accountable care organization or an integrated health system.³⁷ Nevertheless, ULTRA practices were similar to typical practices where most primary health care is delivered.³⁸ Another potential limitation was our use of multiple facilitators with social science backgrounds who had little formal facilitation experience. They all received training in basic facilitation strategies, however, and were closely monitored in weekly research team meetings. Having facilitators with little formal facilitation training may have actually been an advantage in some practices, as RAP team members were able to mimic and adopt the straightforward facilitation techniques they saw in use. Expert facilitators may have faced a bigger challenge in attempting to teach their methods to the doctors, nurses, and administrative staff who eventually took their places.

In retrospect, the RAP team’s composition may have made it difficult to discuss the implementation of models of care that can truly affect chronic care management, such as those advocated in the chronic care model.^39–42 In most practices the organization is sharply divided into a dual hierarchy of clinicians and their support system,³⁵ with staff largely responsible for processing patients so they can be efficiently seen by physicians. These characteristics can make it difficult for either physicians or staff members to initiate conversations about practice members needing to change their roles for team-based, proactive, population-based chronic care management. Practices can also lack the leadership and vision to explore options that challenge widely held beliefs about roles and organizational structures, so it is not surprising that improvement targets fit within existing practice structures and roles. Thus, while facilitated RAP teams might be a useful strategy, they may need to be embedded within a larger approach that helps motivate practices to discover and implement new visions of improving quality of care.⁴³