Lessons Learned from Scaling up a Patient Engagement in Primary Care Quality Improvement Practice Transformation Intervention

Abstract

Healthcare transformation calls for patient engagement in quality improvement (PEQI), yet practice participation remains low. This pilot study of 8 primary care clinics at 7 statewide locations sought to determine the most effective strategies for disseminating a previously successful single system PEQI intervention. Qualitative data were obtained through site visits, interviews, observations and journaling. All material pertaining to barriers, recruitment/retention and implementation was extracted, compared, and categorized. 5 teams partially completed the intervention and 3 finished. These 3 teams did not ask for shorter trainings and were assigned a quality improvement (QI) coach. Multiple barriers to recruitment, implementation, and retention were noted at the organizational and clinic/team level including: turnover, shifting priorities, cross-level communication difficulties, lack of QI knowledge and confusion between patient engagement and patient activation. Our findings suggest that QI facilitation and dedicated time can help primary care teams identify and overcome barriers to PEQI.

Introduction

Health reform efforts are increasingly mandating patient engagement as a critical component of primary care practice transformation that improves care quality.^1–3 Interventions that engage patients are desirable at all levels of the health care system– from direct care to organizational design and governance.⁴ For example, the Centers for Medicare & Medicaid Services’ (CMS) requires Accountable Care Organizations to include a Medicare beneficiary in shared governance.⁵ Additionally, CMS supports ambulatory care practices participating in its Transforming Clinical Practice Initiative in increasing patient engagement through providing technical assistance from a Support and Alignment Network.⁶

Despite these increasing expectations, practices report being unprepared to engage patients in quality improvement work. Consequently, the percentage of practices that meaningfully engage patients in quality improvement (QI) efforts is low^7,8 with a major barrier identified as the lack of resources and knowledge about successful models for engaging patients. From 2010–2014, we developed, implemented, and evaluated a patient engagement intervention across 26 primary care clinics at a single Midwestern academic health system.^9,10 The six month intervention employed multiple concurrent implementation strategies, including a toolkit, in-person academic detailing from a patient engagement expert, and regular (weekly to monthly) in-person QI coaching. While this intervention successfully increased teams’ involvement of patients in QI, consistent with other researchers^11–13 we identified additional barriers including: competing demands, lack of leadership support, and limited time and resources to support this work.^9,10 As such, in order to make scaling this intervention beyond our own academic health system feasible, we recognized the need for the intervention to employ less resource-intensive strategies.

We conducted a multi-site pilot study designed to provide information about the highest impact and most cost effective combination of implementation strategies to be used in scaling up our intervention for engaging patients in primary care team based quality improvement work. This paper reports the lessons we learned from these efforts. We anticipate that our findings will be useful for those interested in patient involvement in system redesign and the implementation and evaluation of practice transformation efforts.

Methods

Context

The study was reviewed and granted an exemption by the Health Sciences IRB at the University of Wisconsin-Madison. It was conducted by an academic research team consisting of: a family physician/health service researcher, a patient engagement expert/health policy researcher, a qualitative and mixed methods expert researcher, a systems engineer specializing in implementation research, a quality improvement coach, and a project manager/study coordinator. This team met bi-monthly to discuss recruitment, implementation, data collection, and ongoing analyses.

The original six month intervention was embedded in a large scale primary care redesign at a single organization.¹⁴ Teams were trained in cohorts that learned about QI using a Microsystems approach.¹⁵ They met weekly, selected an improvement project, received training about engaging patients in these efforts and received 6 months of QI coaching. For examining scalability in the pilot study, we adapted the preceding steps to focus QI training into a 30 minute session discussing Plan-Do-Check-Act cycles, using an applied example to demonstrate a practical tool such as process mapping. The in-person patient engagement session was also 30 minutes. We also employed remote web-based technology and/or phone calls for coaching after an initial in-person session.

Study Design and Intervention

The purpose of the pilot study was to assess the feasibility and preliminary effectiveness of different combinations of implementation strategies for promoting patient engagement in team-based quality improvement work. The design called for recruitment of a total of nine primary care teams from clinics across the state of Wisconsin that varied in geographical location, practice size, and patient population.

We assigned clinics into four implementation strategies for this six month intervention in order to maximize variation and thereby increase the research team’s knowledge about the highest impact and most cost effective scale-up strategy. Clinics could receive: 1) A patient engagement toolkit and accompanying web-based video recording from a patient engagement expert; 2) A toolkit plus a live session with the patient engagement expert; 3) A toolkit, patient engagement live session, and monthly sessions of in-person (month 1) and remote (months 2–6) QI coaching; and 4) A toolkit, recorded patient engagement session, and QI coaching. Additionally, all sites were offered a single in-person didactic session, “QI 101,” in which the basics of QI theory and practice were reviewed.

Each clinic was asked to identify a team champion. This person was asked to serve as the primary contact with the research team and coach, as well as to encourage their team’s progress in patient engagement and improvement activities. Table 1 provides more detail about the intervention within the 6 month timeline.

Table 1.

A Timeline of Project steps, Trainings/Tools, Actions and Goals

Timeline	Project Step	Training or Tool	Recommended Actions	Goals
Month 1	Introduction to QI and Begin a QI Project	QI 101 presentation	Learn basic QI	Understand basic QI approaches & tools
			Decide on a QI project	Identify an issue to improve
			Assemble an internal team to work on the project	Organize a team
		QI handout	Learn about the issue that needs improvement	Clarify current knowledge and understand root causes of problems
Month 2	Pre-Patient Engagement Planning & Continue Planning QI Project	Patient Engagement Introductory Presentation	Complete the Pre-Engagement Planning Worksheet	Understand basics of patient engagement and pre-engagement planning steps
				Prepare to engage patients in QI project(s)
		Patient Engagement in QI Toolkit	Identify possible solution(s) for the issue	Select possible solution(s)
Month 3	Patient Engagement Planning & Start Engaging	Patient Engagement in QI Toolkit	Complete the Patient Engagement Worksheet	Identify specific engagement opportunities for patients
			Recruit/invite patients to engage	Begin recruitment for engagement
Months 4 & 5	Engage Patients as You Continue Your QI Project	Patient Engagement in QI Toolkit	Proceed through QI Steps Do-Check-Act	Adapt QI goals or project outcomes based upon patients’ inputs
			Include patients in appropriate QI steps	Improve systems, workflows, and care experiences
Month 6	Project Wrap-up & Next Steps	QI handout	Wrap-up or plan conclusion of QI project(s)	Complete QI project
		Patient Engagement in QI Toolkit	Identify and discuss lessons learned, celebrate successes, and thank participating patients	Document any benefits or challenges of: project, patient contributions, and trainings
				Publicly and privately appreciate patients’ contributions
			Talk with team about next steps after conclusion of formal project	Communicate next steps

Data Collection

The findings reported in this paper were derived from the qualitative component of the data collection plan as explained in detail below. Date came from site visits, interviews, structured observations by the coach, and a research team journal containing detailed notes about the recruitment process and all interactions with study sites during the intervention period.

Site Visits and Interviews:

Site visits were conducted at each participating clinic prior to the initiation of the assigned intervention. A team of investigators spent approximately 4 hours at each site. During this time they toured the clinic, met providers and staff, and conducted semi-structured interviews with primary care team members (approximately 10–15 minutes), and QI administrators/clinic managers (approximately 1 hour). The semi-structured interview was guided by the Consolidated Framework for Implementation Research.¹⁶ Questions sought specific information regarding previous QI and patient engagement experience, team communication processes and hierarchical structure, and perceived team, clinic and organizational barriers and facilitators for participating in this intervention.

At those clinics completing the intervention, follow up interviews were also conducted. These interviews focused on perceived facilitators and barriers to implementing the intervention, successes and failures when engaging patients in QI, overall opinions of the intervention, and plans to engage patients in QI in the future. Site visits were documented in summary reports. All interviews were audio-recorded and transcribed verbatim.

Structured Observations:

For sites assigned to the coaching component of the intervention, the coach recorded observations after each interaction using a standardized format that focused on clinics’ progress in developing and implementing their projects, changes in outcomes, and next steps.

Research Journal:

Investigators kept detailed notes on the recruitment process. All interactions with research sites, such as regular monthly check-ins during the intervention period, were documented. If sites withdrew, researchers conducted short exit interviews with health system administrators or team champions to ascertain and document the sites’ reasons for withdrawal. All team meeting discussions were also documented, along with ongoing lessons learned. A final wrap up team meeting consisted of a focused discussion on lessons learned from recruitment and project implementation.

Data Analysis

Data analysis began during the bi-monthly research team meetings, as investigators reviewed recent data (e.g. site visit reports) and discussed both the progress of the pilot study and what was being learned about engaging patients in quality improvement work. Many discussions focused on the barriers the team was encountering, and a decision was made to make these barriers one focus of the data analysis. Thus, for this paper, case files (inclusive of all site visit reports, interview transcripts, and other notes) were constructed for each of the participating clinics/teams. The files were reviewed and summary case studies developed. All material pertaining to barriers to recruitment/retention and implementation was extracted, explored, compared, and categorized. These categories were further refined through research team discussions.

Results

Recruitment

This pilot study encountered many challenges, the first of which was recruitment. Four recruitment strategies were employed. First, the team engaged with a statewide quality improvement collaborative focused on public reporting of quality measures to send out an email to its listserv of QI administrators and organizational leaders at 35 health systems. Nine (26%) of these systems responded and set up an informational phone call. However, only one system out of 35 agreed to participate in the study, enrolling three of its clinics. This system had previously worked with the research team on another project. Reasons stated for non-participation included: competing priorities from EHR updates, leadership reorganization and transitions, concern over already overburdened primary care clinicians, and strategic foci on other primary care initiatives. Next, an email sent out via a regional Practice-Based Research Network (PBRN) did not generate any response (0%). Third, personalized emails were sent from a non-profit quality improvement consulting organization to eight clinics. These yielded responses from three clinics (38%), all of which enrolled in the study. Lastly, an email to personal contacts at two systems in which the team had prior relationships yielded one response and enrollment (50%).

Due to funding and time constraints, enrollment was stopped with seven clinics, one of which enrolled two different teams (defined as a several clinical support staff that worked with a single provider and his or her patient panel). Table 2 depicts the characteristics of these clinics, the implementation strategy to which they were assigned, and those who participated in initial site visit interviews.

Table 2.

Clinic characteristics, implementation strategy, and interviewees

Clinic	Geographical location	Size*	Primary population	Implementation strategy	Interviewees
A	Urban	Large	Commercially insured	• Patient engagement toolkit • Patient engagement expert recording	• Clinic Manager • Primary care physicians (x2) • Physician Assistant (x2) • Certified Medical Assistant (X2) • Licensed Practical Nurse
B	Suburban	Small	Commercially insured	• Patient engagement toolkit • Live patient engagement session	• Primary care physician • Physician Assistant • Registered Nurse (x2) • Licensed Practical Nurse (x2) • X-Ray Technician
C	Suburban	Large	Commercially insured	• Patient engagement toolkit • Live patient engagement session	• Clinic Manager • Receptionist Manager • Primary care physicians (x2) • Registered Nurse • Certified Medical Assistant Clinic Coordinator • Certified Medical Assistant (x2) • Unit Clerk
D	Urban	Medium	Publicly insured	• Patient engagement toolkit • Patient engagement expert recording • QI coach	• Clinic Manager • Ambulatory QI Director • Primary care physician • Registered Nurse • Licensed Practical Nurse
E	Rural	Small	Publicly insured	• Patient engagement toolkit • Live patient engagement session	• Clinic Manager • Receptionist/Licensed Practical Nurse
F- 2 teams	Rural	Large	Publicly insured	• Patient engagement toolkit • Live patient engagement session • QI coach	• Clinic Manager • QI Administrative Assistant • Nursing Director • Primary care physician • Nurse Practitioner • Registered Nurse (x2) • Certified Medical Assistant (x2)
G	Urban	Large	Commercially insured	• Patient engagement toolkit • Live patient engagement session • QI coach	• Clinic Manager • Unit Clerk Supervisor • Primary care physician • Registered Nurse • Certified Medical Assistant • Unit Clerk

^*Size based on practice population: Large>10,000; Medium 5,000–9999; Small<5,000

Implementation and retention

As shown in Table 3, clinics had different motivations for enrollment. Several clinics requested adaptations to the didactic portions of the intervention (e.g. QI training, patient engagement training) to reduce training time so it could fit into the structure of preexisting staff meetings. Table 2 also summarizes what happened at each site over the course of intervention. In the end, only two clinics (three teams) completed the intervention. All of these teams did not shorten the time allotted for training, and were assigned a quality improvement coach. For these teams, the didactic patient engagement component occurred either via a recorded session or live.

Table 3.

Intervention participation motivations, adaptations, implementation summary, and outcomes by clinic

Clinic	Motivation for study participation	Implementation Adaptations	Case Summary	Outcome	Study Completion
Clinic A	Health system administrators selected this team to participate in the intervention because they hoped understanding patient engagement would help the team improve their low patient satisfaction scores	20 minutes allotted for QI 101 training	Administrators reported that all team members had been trained in quality improvement QI (or that QI knowledge “comes naturally” as part of scientific training), but researchers found this not to be the case. Team members reported that the expectation they do QI was something being imposed on them from above, and that they were given no actual resources to pursue QI projects. Team members were overwhelmed with their patient care work and with frequent personnel changes. Team members generally equated patient engagement with patient satisfaction and with patient complaints.	The health system made the decision to withdraw all three of its clinics from the study before the team selected a project. Administrators reported that the decision had been based on the fact that the team was too busy and was confused about the aims and process of the study. In addition, the team was disappointed to have been assigned to a condition without coaching.	No
Clinic B	The clinic manager volunteered the clinic because of its emphasis on high patient satisfaction	None	All team members emphasized the small size of the clinic and the closeness this engendered among staff members and between staff and patients. Previous QI experience was highly variable among team members. In general, team members valued QI because they saw it as a way to improve patient experience and health outcomes. When asked to talk about patient engagement, team members emphasized the high quality of their interpersonal relationships with patients.	At the time the health system withdrew from the study, the team had initiated a project focused on distance access to diabetes educators, begun analyzing the problem, and had a preliminary plan for patient engagement. The health system reported that they had considered allowing this clinic to remain in the study, but in the end decided that staffing issues would make continued participation too difficult.	No
Clinic C	Clinic C’s team was selected by a clinic manager, who told the researchers “I have lots of things I want to improve here.”	20 minutes allotted for QI 101 and 30 minutes for patient engagement training	Interviews with team members suggested there was a fair amount of tension between administrators and clinical staff and between different disciplines on the clinical teams. Most team members had little experience with QI and saw it as something that administrators wanted them to do, but that resulted in very little value because there was little focus or follow-through on results. Researchers’ questions about patient engagement prompted answers about the relationships between individual providers and patients. Team members emphasized that they had little time to do anything other than their clinical tasks.	At the time of drop out, the team had not yet chosen a project. At the exit interview, the health system administrators reported that the team was confused and frustrated about the work they were supposed to be doing and lacked the time to continue to participate in the study.	No
Clinic D	Health system administrators chose this clinic because the MD was a QI champion and open to change	A full hour was allocated to QI 101 to allow for questions and several additional health system leaders and clinic staff members were invited and attended	Clinic staff reported that their health system emphasized both quality improvement and population health. QI activities appeared to be driven by individual physicians, who were monetarily incentivized to improve their metrics. In an environment described as “hierarchical,” staff engaged in QI activities only at the direction of the physician(s) with whom they worked. Most staff did not seem to have had prior training in or experience with QI. Questions about patient engagement prompted clinical staff to note “patients in this community aren’t used to being engaged,” which they attributed to their older, rural patients.. In general, while clinic administrators were enthusiastic about patient engagement, which they defined as something like patient activation, members of the team seemed wary, equating it with providing a venue for patient complaints. With the help of the coach, the team decided to focus their QI project on improving workflows around routine patient lab work (i.e., lipid screening). They engaged patients by surveying them about their preferences for the timing of lab work vis-à-vis scheduled appointments. Based on the results of the survey, and their analysis of the issue, clinic staff began asking patients to get their labs done before scheduled appointments. The team tracked screening rates, conducted further patient surveys to assess patient satisfaction with the new system, and also informally assessed provider satisfaction.	All parties liked the new process, and overall clinic lipid screening rates increased from 56% to 64.4%, with the rates of the participating physician team increasing from 62% to 69%. At the conclusion of the six-month intervention, the team was figuring out how to scale up the new workflow to the entire clinic. They were optimistic about engaging patients in other situations where there was disagreement among providers about the best way to institute a change.	Yes
Clinic E	The practice had recently participated in a successful QI project with a consultant and expressed interest in broadening their QI experience, with an eye toward improving patient health.	QI 101 and Patient Engagement training combined into a 1:1 conversation that occurred while the team member had breaks from other clinical duties.	Clinic E was a three-person operation, comprised of a physician and two staff members who each fulfilled multiple roles in the practice. In interviews, clinic staff made it clear that the physician would not be participating in the project, and also that he would limit what they could do as part of the project (e.g., no patient surveys). The clinic staff described having close relationships with practice patients, but expressed skepticism that their patients would be willing or able to participate in QI work. The “team,” which consisted of a single staff member, decided to focus the project on increasing the proportion of clinic patients with advance directives on file. The QI plan did not include a formal patient engagement component; instead, the team member viewed patient engagement as involving patients in making decisions about their own advance directives.	The single staff member made efforts to figure out what needed to be done to encourage patients to complete advance directives, but encountered difficulties getting the information required. Ultimately, the practice decided to drop out of the study, citing recent personnel changes and a resulting lack of time to “[take] on extra projects.”	No
Clinic F	Two teams were selected by the health system administration to participate in this intervention based on these teams’ willingness to participate in QI.	None	Team members described a preventive care emphasis and a scheduling system that allowed them to have longer than average patient appointments. The clinic conducted frequent patient satisfaction surveys, but little was done with the results. Staff without administrative titles reported having little prior QI experience. At the site visit, team members described a new no-show policy that was about to be implemented at the clinic, and expressed a desire to focus their project on the new policy. However, this was viewed as too negative, and by the time the coaching part of the intervention started, the focus had shifted to increasing the number of asthmatic patients with asthma plans in place. The teams began their efforts by running small tests of change on the workflows of two providers. Their plan for patient engagement, beyond a focus on completing asthma action plans as part of office visits, was to invite patients to a lunch and learn at which the asthma plans would be explained and patient feedback solicited. Despite inviting more than 100 patients to the lunch, no one showed up.	By the end of the six-month intervention the teams had exceeded their stated benchmarks for the proportion of the providers’ patients with completed asthma plans—the team physician increased from 20% to 36% (goal for project: 30%) and the second team nurse practitioner increased from 16% to 51% (goal for project: 26%)--and were planning for how to improve numbers across the clinic. Clinic leadership expressed ongoing intent to consider patient engagement in future QI projects.	Yes
Clinic G	A pre-existing team was selected by the clinic manage that was working on issues related to the electronic health record in-basket	None	Team members described many recent organizational and personnel changes and a sense of always being overworked and short on time. A single individual on the team had significant QI training and experience, but this was not shared by the rest of the group. Questions about patient engagement evoked statements about patients’ customer service expectations, and team members had little vision for how patients might be engaged in QI work. During the site visit, team members spoke of their wish to focus their QI project on management of their electronic “in-basket” system. By the time the team had its initial meeting with the coach, however, they had shifted to a focus on vaccination rates.	Before its second scheduled meeting with the coach, the clinic dropped out of the study. A subsequent conversation with the health system administration revealed that the new regional system administrator had not been aware of the project and did not want the clinic to participate. They cited the failure of the researchers to obtain proper permissions from the appropriate chain of command, concerns about sharing confidential corporate information with the researchers (conflating the university with the competing health system affiliated with the university) and a belief that the focus of project did not reflect corporate priorities.	No

The barriers we observed may be roughly categorized at the system-level or clinic/team-level. (Participants discussed their perception that there also might be patient-level barriers, such as a lack of interest in or aptitude for engagement among their patient populations, but we did not observe these potential barriers directly.) At the system level these barriers included: 1) rapidly changing external environment with reorganizations and leadership turnover. 2) QI administrators were not decision makers for primary care participation. 3) Research projects values not aligned with values of corporate healthcare. 4) Shifting (or unclear/not well communicated) organizational priorities. 5) Research communication strategies not well aligned with hierarchical leadership structure; information about the study from organizational decision makers not passed along to the clinics. At the clinic/team level, these barriers included: 1) Staff turnover and other clinic-level personnel changes. 2) Lack of prior team based QI experience. 3) Lack of tangible incentives for QI and patient engagement activities. 4) Perception of QI as an administrative priority with little value for frontline staff. 5) Lack of time and feeling too overworked to take on a new initiative. 6) Interventions/expectations not clear to teams. 7) Confusion between patient engagement and patient activation/and or patient satisfaction.

As shown in Table 4 and illustrated by examples, multiple lessons were learned from both the recruitment and implementations stages of the project. These lessons highlight the difficulties in recruitment without prior established relationships. They also underscore the need for the research team to establish ongoing dialogues with individuals across an organization at both the leadership and front-line levels. Reducing the time allocated for education, even though responsive to time and resource limitations, was unsuccessful. Implementation occurred most successfully when small teams were engaged, facilitating communication.

Table 4.

Lessons learned and examples by implementation stage

Implementation Stage	Lesson Learned	Example
Recruitment	Recruiting practice participants across multiple organizational settings takes a long time	Recruitment for this project started in March 2016 and was not completed until October 2017. The research team aimed to recruit 9 practices yet only 7 practices were recruited.
	Recruitment was most successful when there was a preexisting relationship or via a warm handoff introduction from a known entity	All participating clinics had a prior relationship either with the research team or the QI consulting organization
	It is important to understand the relationship between practices and the recruitment partner	Successful recruitment came from the organization that had a positive relationship with the clinics and the clinics trusted their recommendation to participate in the intervention.
	There is a need to understand motivations for participation at multiple levels (system leadership, clinic leadership, and grass roots team members)	The clinic teams that participated ranged in motivation. Some clinic teams were volunteered by the health system to become more patient centered and to fix patient satisfaction scores and others teams thought it would make their day to day lives easier and would help with their workflow.
	Stakeholders at multiple levels need to clearly understand the intervention	One clinic dropped out because of misalignment with the overall health system goals. Other clinic teams dropped out due to leadership not understanding the importance of QI and other outcomes.
Implementation	Adapting the intervention to meet clinic’s time pressures was unsuccessful	Clinics that dropped out of the intervention received 20 minute of QI 101. One tried to fit education in while doing other clinical duties. These team expressed confusion about the intervention and had difficulty choosing their own projects. The clinics that received the full trainings were better equipped to take on their projects.
	Teams found it challenging to identify their own quality improvement projects	Clinic teams were used to being told what outcomes or projects to work on from the overarching organization and didn’t have the processes or skills in place to choose their own projects.
	If teams did not understand the intervention, they had to back track and choose new projects	Clinic teams did not understand the difference between patient activation and patient engagement. Projects were changed last minute to accommodate the patient engagement component.
	Academics and real world practices had different beliefs about the importance and meaning of engaging patient in quality improvement work	Patient engagement wasn’t an organizational goal and therefore teams didn’t have an expectation or motivation to include patient engagement in their QI projects.
	Academics trying to engage with systems that are potential competitors to their own health system may face suspicion	The research team was asked to sign a non-disclosure agreement since their university owned a competing medical system.
	Smaller sized project teams with a clear champion were most successful	The two successful clinics formed workgroups of 3–4 people that met on a regular basis. They would then relay their progress back to the bigger clinic team

Discussion

In trying to spread a previously successful intervention for patient engagement in quality improvement beyond 23 clinics affiliated with our own health system, we encountered numerous barriers to successful recruitment, implementation, and retention. Teams that completed the intervention required a resource intensive combination of quality improvement training, patient engagement resources, and regular contact with a quality improvement coach. Failures of understanding and communication that cut across clinic/team and system levels hindered the conduct of the research and the implementation of patient engagement efforts. We found limited understanding of the concept of patient engagement among both system executives and clinic staff. Therefore, clear explanations about the purpose and expectations of this study were not conveyed by administration to the frontline teams. In addition, shifting and unclear organizational priorities also created barriers. In several clinics, for example, teams’ initial choices for a project focus were suddenly changed in order to more closely match a different, apparently new, priority. Finally, the current chaotic environment in primary care, characterized by severe constraints on time and other resources and high rates of turnover and other personnel changes, proved to be the major barrier both to doing the research and to promoting the engagement of patients in QI work.

Beyond these expected time and resource constraints^9–13 we discovered additional barriers at the clinic/team level. Consonant with a recent call for primary care practice staff and leaders to practice “regular quality improvement hygiene”¹⁷ we found the lack of front line QI knowledge to be a barrier to engaging in practice transformation efforts while involving patients. At several of our clinics, QI work historically had been delegated to particular individual(s) with an administrative role, such as a practice or nurse manager. Specifically, the frontline staff who made up the primary care teams lacked the knowledge, skills, resources, and incentives needed to conduct QI work. Our QI 101 training was an attempt to address this lack, which we anticipated, but its utility proved to be somewhat limited, particularly without coaching to reinforce it.

Understanding in detail the ways that core components of an intervention are actually implemented in different contexts is critical for the scalability of practice transformation efforts. However, our findings underscore the numerous barriers that can exist to conducting research in “real-world” primary care settings with unfamiliar contexts. At the system level, we encountered difficulties in engaging the correct level of administrative decision makers by targeting our recruitment efforts to those involved in leading quality improvement efforts. Through the failure of these efforts, we discovered that actors at this level often lacked the authority to commit the organization to research participation and held an inflated sense of how attractive research participation would be to primary care clinic staff. The combination led directly to many of the difficulties we had first recruiting sites and, later, retaining them. Additionally, we encountered something of a “two communities”¹⁸ culture clash; we failed to anticipate the perception of higher level system executives that research participation risked revelation of corporate secrets that might impart a competitive advantage to the health system affiliated with our university. (While negotiating recent research partnerships between university-based investigators and corporate entities, we have noted, anecdotally, that requests for researchers to sign non-disclosure agreements seem to be becoming more common.)

Conclusion

Although this study presents barriers to involving patients in primary care team-based quality improvement discovered in the context of a research project, we believe that unless these barriers are addressed they are likely to hinder the widespread adoption of patient engagement as a standard element of primary care-based quality improvement efforts. Our findings support the importance of practice facilitation¹⁷ as only teams that received coaching were able to complete the intervention. In addition, education about the distinctions between patient engagement and patient activation or satisfaction and training in quality improvement methods appears to be critical in order for practices to understand the “why” and “how” of involving patients in practice transformation efforts.

Limitations

The findings presented in this paper are limited by the fact that they were derived from a single study conducted in a single state. Although our experiences seem to be consistent with those being reported in the literature, we do not know how the primary care environment in Wisconsin may differ from those in other areas of country and thus cannot be sure how generalizable our conclusions may be. In addition, our work reflects the strengths and weaknesses inherent when investigators set out to evaluate an intervention of their own design. The team members who collected data were intimately familiar with the intervention, which likely enhanced the relevance of their interview questions and observations, but their investment in the patient engagement intervention also may have shaped the data. For example, research participants may have been reluctant to express negative opinions. Finally, the work of learning from a project that did not go according to plan requires a capacity for critical reflection on one’s own weaknesses. In such a situation, it is likely that, like all human beings, the investigators have some blind spots.