Implementation and Qualitative Evaluation of a Primary Care Redesign Model with Expanded Scope of Work for Medical Assistants

Abstract

Background

Implementation of primary care models involving expanded scope of work and redesigned workflows for medical assistants (MAs) as primary care team members can be challenging. Implementation strategies and participatory evaluation informed by implementation science frameworks may inform organizational decisions about model scale-up and sustainment.

Objective

This paper reports implementation strategies and qualitative evaluation of a primary care redesign (PCR) model implementation that included an expanded scope of work for MAs.

Design

Qualitative evaluation of implementation strategies and clinician and staff experience with implementation of PCR using semi-structured key informant interviews. The evaluation was guided by the RE-AIM framework and the Consolidated Framework for Implementation Research.

Participants

Sixty-nine clinicians, staff, practice leaders, and administrators from 7 primary care practices (4 general internal medicine, 3 family medicine) implementing PCR.

Interventions

The PCR model included enhanced rooming and documentation support. The health system used multiple strategies to implement PCR, including rapid improvement events, changing clinic space configurations, developing electronic health record templates and performance dashboards, and practice coaching.

Approach

The Consolidated Framework for Implementation Research and the RE-AIM evaluation and planning framework guided development of semi-structured interview guides. A deductive, structural coding approach was used for analysis.

Key Results

PCR implementation was facilitated by clear communication about the intervention source, mechanisms for feedback about model goals, and physical environments and electronic health record (EHR) systems that supported the added staff and modified clinic workflow. Clinicians and staff benefited from the ability to see the model in action prior to go-live and opportunities for consistent provider-MA pairings.

Conclusions

The PCR model can support achieving the Quadruple Aim when fully implemented with paired MAs and clinicians who are well prepared to follow redesigned workflows and function as a team. Implementation can be effectively supported by a participatory evaluation guided by implementation science frameworks.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11606-021-07246-x.

The Triple Aim in healthcare is to achieve optimal population health at low cost and with excellent patient experience.¹ The Quadruple Aim addresses a fourth concern: the health and well-being of those who deliver healthcare.² Achieving the Quadruple Aim—high-value healthcare that protects the well-being of healthcare providers—requires broad dissemination and implementation of innovations in care delivery that go beyond expanding the scope of practice for clinicians. Primary care increasingly includes healthcare teams that share not only clinical work, but administrative work as well^3,4 Interprofessional team-based care is recommended by the National Academies of Sciences, Engineering, and Medicine (NASEM) as a foundational feature of high-quality primary care⁵.

One component of team-based primary care involves expanding the scope of work of medical assistants (MAs) in collaboration with a clinician, such as engaging MAs in population management through standing orders and serving as health coaches.^6–9 Eden recommended a variety of roles for medical assistants as a strategy for reducing documentation burden, such as managing incoming messages, managing and informing patients of lab and radiology results, coordinating needed care with care managers, and researching refill requests.¹⁰ The Primary Care 2.0 model, a team-based model that included increased MA-physician ratios, enhanced MA roles, integration of advanced practice clinicians, and an extended interprofessional team, was shown to promote team development and led to initial but not sustained improvements in “control of work” over time.¹¹

In 2015, to promote the Quadruple Aim through expansion of team-based care involving expanded scope of work for MAs, University of Colorado Health (UCHealth) launched a practice transformation effort referred to as Primary Care Redesign (PCR). PCR was based on the University of Utah’s Care by Design (CBD) model.^12,13 CBD was designed to be a complement to Utah’s patient-centered medical home (PCMH) practice transformation, and focused on enhanced access through care teams and planned care processes.¹⁴ Reflecting recommended MA roles in primary care¹⁰ and the University of Utah experience, PCR involves increased medical assistant (MA) ratios (2.5 MAs to 1 provider) and functions, including enhanced rooming activities, in-room documentation support, and inbox management.¹⁵ Early results showed PCR had a positive effect on a variety of operational and process outcomes, provider burnout, and commonly reported intermediate clinical outcome measures, such as depression screening.^16,17

Achieving the expected benefits of practice transformations like PCR depends upon successful implementation. Implementation refers to “the extent to which a program is delivered as intended.”¹⁸ For instance, poor implementation can limit achieving expected benefits of practice change efforts and increase the perceived burden and burnout among clinicians.^19–23 Evidence shows myriad implementation challenges can arise as healthcare organizations adopt models of primary care that expand the scope of work of MAs, such as PCR.^24–26 As noted by Dill et al., complex interventions involving team-based care with enhanced roles for MAs must address the need for training in new roles and workflows, consideration of regulatory requirements, and MA compensation and career ladders consistent with the enhanced scope of work.²⁶ Ensuring role clarity is particularly critical for ensuring both effective implementation and improved patient care.²⁷ Other elements of organizational context—such as implementation climate, technical assistance resources, leadership support, and care team dynamics—factor into successful implementation and should be attended to during planning, implementation, and sustainment stages of change.²⁶

As care models that redefine primary care team roles and redesign workflows continue to emerge and evolve, there is a need for a body of evidence on strategies for effective implementation across diverse clinical and operational contexts.^24,26 There have been calls for formal evaluation and replication of models of care involving expanded scope of work for MAs in diverse practice contexts.²⁸ In particular, Rodriguez and Zink recently noted the need for study of implementation of diverse teams.²⁹ The field of implementation science offers methods and frameworks to guide the selection, evaluation, and reporting of implementation across contexts.²⁵ Implementation science has also been noted as particularly useful for informing primary care practice change.³⁰ Rapid and participatory approaches to evaluation of complex team-based care interventions guided by implementation science frameworks represent a valuable feature of learning health systems.³¹ This report describes the implementation strategies and methods and results of a participatory evaluation of a multiphase implementation of the PCR model in 7 practices within the UCHealth system, guided by complementary implementation science frameworks.

METHOD

Design

This was a participatory evaluation of implementation strategies and experience, the results of which were used to inform organizational decisions about scale-up and sustainment of the PCR model. PCR organizational leaders collaborated with evaluators with expertise in implementation science to design, conduct, interpret, and respond to the evaluation. An initial evaluation of implementation in two pilot sites informed how, when, and where the organization would expand the model in subsequent waves of practices. We first report on findings from the evaluation of pilot site implementation. We then add contrasting findings and deeper insights from a similar evaluation of a subsequent wave of implementation following organizational modifications to pilot trainings, workflows, and policies. This report is presented in accordance with the Standards for Reporting Implementation Studies (StaRI) framework.³² Key components of StaRI include a description of both the implementation strategies (i.e., processes of change) and the intervention itself (i.e., the PCR model), the context in which the implementation occurred, and the methods for evaluation of the implementation.

Two well-established implementation science frameworks were used to guide the evaluation design: the Consolidated Framework for Implementation Research (CFIR) and the RE-AIM Framework.^18,33 CFIR is an organizing framework of five domains of constructs implicated as contextual factors driving the adoption and implementation of innovations in healthcare. Specifically, CFIR guides the assessment of characteristics of the intervention (including both core and adaptable components), the individuals involved, the inner and outer setting, and the process of implementation. RE-AIM is a program planning and evaluation framework that defines five key outcomes to consider in dissemination and implementation studies, including patient-level outcomes (Reach and Effectiveness) and setting-level outcomes (Adoption, Implementation, and Maintenance). This paper reports on CFIR contextual factors related to setting-level Adoption, Implementation, and Maintenance domains from RE-AIM. Qualitative descriptive methods³⁴ were used to assess and evaluate PCR Implementation (implementation strategies³⁵ and practices’ experiences with implementation) and organization-level Adoption and Maintenance (motivation and decision to adopt PCR and ultimately spread it throughout the system). CFIR constructs informed interview guide topics and served as a priori codes for deductive analysis. We used the Expert Recommendations for Implementing Change (ERIC) project’s compilation of 73 types of discrete implementation strategies (e.g., model and simulate change, provide local technical assistance, visit other sites, change physical structure and equipment, alter incentive/allowance structure) to systematically document the organization’s processes of change used to implement PCR.³⁶ The Colorado Multiple Institutional Review Board approved this program evaluation as “not human subjects research.”

Setting and Participants

The PCR pilot implementation and evaluation was conducted in two health system–affiliated primary care practices, first in two pilot sites (one family medicine residency practice, one internal medicine practice) and then in 5 additional practices (“wave 1”). The evaluation team conducted 69 semi-structured key informant interviews with clinicians and staff. Practice managers arranged evaluation interviews with 69 clinicians, staff (including medical assistants), and practice leaders and administrators (Table 1). Purposive sampling was used to select a range of clinicians and staff with positive and negative attitudes toward PCR and who represented a variety of roles in the practice. Practices were asked to identify a minimum of 4 individuals for interviews, including a lead clinician, a non-lead clinician, a lead MA, and a practice manager. Participants were not compensated.

Table 1.

Characteristics of Key Informant Interview Participants

Characteristic	Pilot practice post-implementation interview participants	Wave 1 baseline interview participants	Wave 1 post-implementation interview participants
Interview participants	n = 30	n = 22	n = 17
Practice types	1 general internal medicine 1 family medicine residency	3 general internal medicine 2 family medicine
Primary care clinicians	11 (36.7%)	6 (27.3%)	1 (5.9%%)
Primary care staff
MAs	9 (30.0%)	6 (27.3%)	6 (35.3%)
Other staff*	5 (16.7%)	-- (--%)	1 (5.9%)
Practice leaders and administrators	5 (16.7%)	10 (45.5%)	9 (52.9%)

^*Other staff include nurses, coaches, trainers, and front-desk staff

PCR Model

Prior to PCR, MA roles in participating practices primarily entailed basic rooming activities such as checking vital signs and assessing the reason for the visit. Under PCR, MA expanded roles included an enhanced rooming process in which MAs execute standing orders for medication refills and preventive care and then provide in-room documentation support during the clinical encounter. Figure 1 describes PCR roles for MAs and clinicians before, during, and after the office visit. MAs also do inbox management (e.g., responding to patient calls and emails) and patient follow-up (e.g., assisting with referrals, scheduling follow-up appointments). Expanded MA roles are supported by redesigned electronic health record (EHR) templates and clinical workflows. To accommodate PCR, practices transitioned from a MA to clinician ratio of 1:2 to 2.5:1; this necessitated additional physical space and computers for the additional 1.5 MAs per clinician in each practice.

Figure 1.

Primary care redesign clinical care team roles and tasks.

PCR Implementation Strategies

PCR implementation strategy types and corresponding activities by implementation phase are shown in Table 2. For instance, to prepare for implementation of PCR, practices held mock-patient sessions. Over the course of 2 days, part of the clinic would close to practice the PCR model. Each care team took turns practicing the model, and staff and providers rotated through the training to minimize patient care disruptions. The training included a didactic component about the model and the opportunity for providers and MAs to complete mock-patient appointments.

Table 2.

Primary Care Redesign Implementation Strategies by Phase of Rollout

Implementation strategy type	Primary care redesign activity	Pilot phase	Wave 1 phase
Rapid improvement events	Design protocols and workflows	X
Educational meetings	Train clinicians and staff	X	X
Mock-patient sessions	Practice skills in low-stakes situation	X	X
Learning collaboratives	Ongoing peer learning through regular cross-practice conference calls	X	X
Practice facilitation	Identify and overcome barriers	X	X
Clinic meetings and huddles	Communicate roles and workflows	X	X
Reconfigure clinic space	Team proximity and personal space	X	X
Create electronic health record templates	Create structure to protocols	X	X
Engaged leadership mandates change	Establish priority and buy-in for practice change	X	X
Implementation team visits to other sites	Establish vision and objectives	X	X
Develop dashboards	Provide audit and feedback		X
Alter pay and incentive structures	Increase MA pay scale and career ladder		X

Data Collection Instruments and Procedures

A semi-structured interview guide was developed to elicit open-ended responses regarding clinician and staff experience with PCR implementation. The interview guide featured questions about implementation context and climate, sufficiency of training and preparation, and practice experiences. Select CFIR domains and constructs related to implementation experience included intervention characteristics (intervention source, adaptability, complexity), inner setting (structure, culture, implementation climate, goals and feedback, readiness for implementation), and characteristics of individuals (knowledge and beliefs, self-efficacy). Two evaluators experienced in qualitative research (BK, MH) conducted all interviews on site and in person during fall 2015 (pilot sites), fall 2016 (wave 1 baseline), and late 2017/early 2018 (wave 1 follow-up). Interviews lasted up to 1 h. Recorded interviews were transcribed verbatim then imported into qualitative analysis software (Atlas.ti v7.5, Berlin) for data management and coding.

Analysis

An initial coding scheme for the pilot data consisted of 10 a priori codes reflecting CFIR constructs. One evaluator applied the a priori codes to the transcripts using a structural coding approach.³⁷ The coded transcripts were thematically analyzed by three evaluators with graduate training and/or professional experience coding and analyzing qualitative data. A team-based approach was used to identify emergent themes. Thematic saturation within each set of interviews and strong inter-rater reliability were observed throughout this process. Identified themes were corroborated with PCR leadership and staff (PS, AB, KC, CC). We followed relevant elements of COREQ guidelines for reporting qualitative research.³⁸

RESULTS

Pilot Implementation Themes

Pilot implementation themes were organized by CFIR construct. Illustrative quotes for each theme are displayed in Table 3.

Table 3.

Illustrative Quotes for Primary Care Redesign Implementation Themes

Implementation theme	Illustrative quote
Intervention source	“We’ve been wanting to implement [PCR] for years... it’s only been since we’ve been able to get the ear of administration and get some support that we’ve been able to implement it. We’ve been interested in this for a long time.” [GIM practice leader] “My understanding of the [PCR] model is that we are supposed to be able to see more patients in a day, a patient every 20 minutes, all patients being the same. The MAs are supposed to be able to facilitate that. Obviously, that translates into more dollars.” [FM Residency practice provider]
Goals and feedback	“Because of the team approach in the pod, and where nobody was really—there was no head MA in pod, there was no opportunity to learn as you go. If something didn’t go well, there was nowhere to give feedback.” [FM Residency practice provider] “We didn’t establish goals well at the beginning. That was, I think, the mistake that we made, the planning team made and it’s been recognized, I think, as a mistake that there weren’t defined goals the providers set in the beginning. There weren’t productivity goals set. We watched to see what happened but goals were not set.” [GIM practice provider]
Structure	“People sometimes define their value by their space that they’re given in their work environment… That took a while to recover from, you still have space, but you’re not here on one day of the week, and so that computer needs to be used by somebody who needs to feel it’s a neutral space that they feel comfortable working there as well.” [FM Residency practice leader] “Our biggest challenge was staffing and getting the right number of staff. We quickly learned that it’s hard to do it when you’re low staffed. It’s almost impossible. That was at that point our biggest challenge. Had we had the staff up front to do all of the training and implementation, it would’ve been much smoother.” [FM Residency MA]
Readiness for implementation	“I was a little surprised about how much I needed to step back and do a little bit more verbalization about processes and things like that when we initially started the in rooming with the MAs in there with me… like where we need to document, and how to document a physical exam, what orders I need, things like that. That had just become part of my internal process, and so I’m having to remember that I need to verbalize when we’re moving along or create a signpost for it. Now we’re gonna do the exam and stuff like that, so that took a little bit of personal change in my practice pattern.” [FM Residency practice provider]
Culture	“People are used to having new initiatives, trying new things. In general, I think people have a pretty good attitude about trying new things. I think that compared to the average primary care practice, we’re probably a little ahead of the curve.” [FM Residency practice provider]
Complexity	“I think we have been successful, largely because we are here most of the time. We’ve been able to develop, more quickly, effective working relationships with our medical assistants. We’ve got the same medical assistants all week.” [GIM practice provider]
Implementation climate	“I know some providers had a lot of concerns. Some were saying, “If this doesn’t work, can we just go back to the old way?” Some people have really trying to be good champions and saying, “No, it’s working! It’s helping!” Other people were like, “I don’t really like this. I have a lot of worries about this.” It’s been highly variable.” [FM Residency practice provider] “I didn’t feel very confident about it ‘cause, number one, I’m not crazy about change, and it just didn’t—when we were just told about it, it just didn’t seem feasible for our office. Like I said, then it got explained to us, and it got showed to us, and we started diving into it, and it started making more sense. Now I really couldn’t see life any other way.” [GIM practice MA]
Knowledge and beliefs	“[The MAs] have a pretty sharp learning curve so the first month or so, there was a lot of angst and insecurity and fear on their part, but they caught on so quick that I would bet within a month they really had the hang of it. Now we’re, what, six months, eight months into it and they’ve really gotten good.” [GIM practice provider] “Then I have on the other end providers that are just finding it too hard to adjust their practice style to having this type of support, so it’s not efficient for them. In fact, it probably takes them longer, because they’re not quite in sync with the MAs.” [FM Residency practice leader] “We did all kinds of training because this model isn’t just about can you go in and draw blood? Can you go in and document it in a computer? This is about being able to develop a relationship with a provider, being able to accept feedback, being able to communicate well, being able to be safe in the patient care.” [FM Residency practice MA]
Self-efficacy	“We weren’t sure everybody could do it. Guess what? Everybody has done it and, from what I can see, has embraced it and made it work. They’re very bright. We are now pretty happy that we can operate all at our—all at top levels.” [GIM practice provider]
Adaptability	“We have developed our own little customized ways of doing this [PCR] that is different from every other doctor in the clinic, yet the same in many ways.” [GIM practice provider]

Intervention Source

Perceptions of intervention source varied by practice role. Those in leadership roles saw themselves as contributing to decisions to adopt PCR and consequently viewed the intervention source as internal. Leadership reported the primary motivation was to improve patient care. Accordingly, leadership were committed to change and viewed themselves as advocates and champions. Conversely, non-leadership tended to view the intervention source as external and perceived the motivation to adopt the model as primarily financial. Accordingly, non-leadership reported being somewhat reluctant to adopt PCR.

Goals and Feedback

Several interviewees lacked clear understanding of the goals and objectives of PCR implementation. Clinicians and staff valued a defined mechanism for providing feedback on progress toward goals and clear communication of the timeline for rollout and increased practice volume expectations.

Structure

Two structural characteristics supported PCR implementation well: the electronic health record (EHR) and physical space. The EHR templates developed for PCR generally appeared to work well, and staff were adequately trained in their use. In terms of physical space, the most significant structural rearranging occurred with the influx of new staff, requiring people to give up individual workspaces. The staff eventually acclimated to the new model and physical space allocations. However, the inability to fully staff practices with adequate numbers of MAs limited the ability to fully implement the in-room documentation support in some practices, yielding frustration and dissatisfaction with the model.

Readiness for Implementation

Generally, providers and staff reported training prepared them well for implementation and emphasized the value of experiential learning when testing their own workflow. Interviewees described broad staff and clinician involvement in the “rapid improvement events” where initial model workflows were designed. However, there were critiques of the time these events took away from clinical care.

Both staff and providers appreciated seeing the model in action and practicing with mock patients during simulated clinic sessions. Shared training and opportunities to practice as a team facilitated trust between the providers and MAs and allowed providers to see how PCR would integrate into their existing workflows and how much support the MAs would need.

Culture

Staff and providers described practices as open to innovation, flexible, and willing to change, and expressed pride in having a reputation for these characteristics.

Complexity

The primary factor reflecting perceived complexity of implementation of PCR was the disruption of established roles and workflows, and well-learned routines and practice styles. The General Internal Medicine (GIM) practice seemed to overcome perceived complexity more readily, attributable to consistent provider-MA pairings; such consistent pairings were not feasible in the Family Medicine (FM) Residency practice.

Implementation Climate

PCR was characterized by significant changes in workflow for providers and staff, which was associated with a wide range of emotions that evolved over the course of the implementation. In the beginning, there was excitement and hope about benefits for patients, providers, and staff. Simultaneously, providers and MAs expressed fear, anxiety, concern, and hesitation, resulting in resistance and uncertainty about each individual’s ability to fulfill their role.

During implementation, there was frustration about the rate of progress and disruptions to well-established routines. With time and opportunity to explore the workflow within the model, for most people, frustration and anxiety dissipated, replaced by contentment. The MAs’ knowledge of vocabulary, medical terminology, and acronyms improved with increasing exposure to in-room documentation support activities. Beyond the learning curve, some providers never fully bought in to PCR. Providers and MAs both remarked on the need for additional team building and communication. The FM Residency pilot practice believed they benefitted from a practice coach who served as a liaison between providers, staff, and leadership, acting as a neutral third party to mediate conflicts and interpersonal issues and help reach solutions.

Self-Efficacy

Before implementation, both providers and MAs had concerns about MAs’ ability to fulfill expanded roles under PCR. MAs had concerns about the volume of information they had to learn and incorporate into their workflow, and they feared making mistakes. However, these concerns were not borne out by experience. In time, with support from providers and peers, MAs gained skills and confidence, and identified opportunities for self-study (e.g., working through medical scribing textbooks, re-reading notes edited by their provider to improve documentation). As trusted relationships developed between providers and MAs, so too did the dynamic allowing for feedback exchange, cooperative workflows, and interpersonal interaction.

Adaptability

While enhanced rooming and in-room documentation support appear to be core elements, there is some room for adaptation to clinical styles and preferences of providers regarding flow of clinical interviews and style and structure of clinical notes.

Organizational Response to Pilot Evaluation Findings

The evaluation team prepared a synthesis of evaluation findings in the form of brief reports, executive summaries, and PowerPoint presentations, which were delivered to organizational leadership at multiple points during and upon completion of the pilot evaluation. In response to the overall positive endorsement of the PCR model, the organization decided to both sustain the model in pilot sites and scale up to additional family and internal medicine practices, using implementation strategies noted as particularly valuable during the pilot including mock-patient sessions and practice coaches. The evaluation also informed several modifications to policies, workflows, and trainings to support PCR. This included a new MA pay ladder, MA Academy, changes to the EHR templates, and allowances for teams to opt out of in-room documentation on days with MA staffing shortages (a key adaptation).

Wave 1 Implementation Evaluation Themes

Wave 1 implementation evaluation themes highlighted here are those that differ from or expand upon pilot practice experiences.

MA Performance and Training

Similar to pilot evaluation findings, interviewees reported being initially concerned but ultimately satisfied with the MAs’ ability to fulfill their PCR roles. The introduction of MA Academy, a new system-led multi-day training program to prepare MAs for PCR roles, was a welcomed addition to the training process for the MAs. However, there was perceived to be a need for continuous improvement and skill building. Specific areas noted by providers and managers include spelling (especially medical terminology) and structuring clinical notes. Practices addressed these needs in various ways including didactics, online assessment and coursework, and MA-directed self-study (reviewing final notes after providers have made edits). Formal structure around improvement of note writing was desired by many wave 1 practices, with the anticipation that improvements to this skill would continue to increase the efficiency and quality of the clinical encounters.

Importance of Consistent MA-Clinician Pairings

Clinicians reported surprise at the significant amount of teaching that was required to achieve high-functioning relationships with MAs. Dissatisfaction with the model stemmed from inconsistent provider-MA pairings, where providers felt they were not reaping the benefit of time and effort spent teaching their MAs. With each new MA pairing, they felt as if they were starting over. In one case, a provider expressed being unwilling to run the model in clinic until consistent pairings were established. With consistent pairings, providers were willing to invest significant time, understanding, and patience, and they received return on investment though highly functional MAs. Interviewees estimated 1 to 6 months of operating PCR for the model to feel normalized in practice. This is faster than the 8–9-month timeline to achieve normalization reported by pilot sites.

Staffing and Resources

Staffing continued to be challenging, both within PCR practices and system wide. Prior to implementation in wave 1 sites, an MA pay ladder was introduced to compensate for the type and volume of work under PCR, and in response to concerns related to compensation expressed by providers, staff, and leadership at pilot sites. Practices reported some difficulty recruiting and retaining enough MAs to consistently fully staff PCR.

Allowing Flexibility and Customization

Practices have taken different approaches to providers resistant to change. In some cases, providers were permitted to opt out of PCR entirely or of specific elements (e.g., several in particular declined in-room documentation support). In other cases, at the insistence of practice leadership or even self-confident MAs, all providers were expected to participate in all aspects of PCR. In the latter case, it was reported that even the most resistant providers had come to enjoy practicing under PCR and that they too “grumbled” during clinic sessions where there was insufficient staff to run the full PCR model.

Nursing Role

Some nurses in PCR practices expressed dissatisfaction, given the role of nurses was not well defined under PCR at the time of its development or prior to roll out to wave 1 sites. While PCR did not specifically exclude nurses, a failure to explicitly consider their role in the model may have resulted in further alienation of a group that already felt external to the day-to-day operations of the clinics.

DISCUSSION

This participatory evaluation of the UCHealth Primary Care Redesign (PCR) model, guided by implementation science frameworks CFIR and RE-AIM, proved to be a valuable investment by the organization. Evaluation of the process and experience of implementation during the pilot phase was central to the organization’s decision to scale up the PCR model in additional practices. While the organization planned and executed a comprehensive set of implementation strategies informed by the literature and guidance from more experienced organizations—such as the need for EHR templates for history taking and comprehensive, ongoing training¹⁰—the pilot evaluation identified opportunities for refining the implementation in subsequent waves. The evaluation informed modifications needed in subsequent waves of implementation, such as an enhanced MA pay ladder and ongoing education in the form of MA Academy. This finding is consistent with literature describing the need for appropriate MA compensation and career development reflecting an enhanced scope of work.²⁶

The evaluation revealed PCR model components thought to be “core” to achieving the intended Quadruple Aim goal of improved job satisfaction and reduced burnout; for instance, documentation support was seen as critical for achieving clinician satisfaction. With enhanced rooming alone, clinicians did not report decreased charting time. This echoes findings of a prior study in which use of scribes in an academic family medicine setting improved physician satisfaction and clinic efficiency.³⁹ Staffing and scheduling should aim to ensure retention of the increased MA-clinician ratios and regular provider-MA pairings—supporting other reports that “non-assigned models” may not achieve the desired effect.⁷ Consistent with these insights, Shaw et al. recently reported that maintaining the increased MA-clinician ratio is critical for sustained improvements in burnout due to implementation of such team-based models.¹¹ Clinicians and staff benefited from opportunities to “see the model in action” at practices already using the model, and valued simulated clinic sessions with mock patients where they could try out the model and conceptualize how it would fit their workflows. This is consistent with a prior study finding diffusion dynamics to be most influenced by perceptions of an intervention’s compatibility with existing norms, values, and practices.⁴⁰

Several themes may generalize to other practice transformations. As implementation progressed, it was important to accommodate a learning curve and investment in MAs’ learning accurate, succinct documentation. The range of MA work increased with the addition of advanced rooming and in-room documentation support. Providers had to adapt to changes to well-established workflows to optimize expanded MA roles, fully utilizing in-room documentation support. Significant cognitive (i.e., time to adjust to change and learn new roles and workflows) and physical (i.e., space, technology) resources were needed for providers and MAs to enact changes of this magnitude.⁴¹ Practices implementing such a change must recognize the time and cognitive effort required to disrupt well-learned routines, a known contributor to chaos during transformation.⁴²

The PCR experience reflects findings from a study of 14 primary care transformation demonstration projects, which described the process of transformation as a long and difficult journey.⁴³ Additionally, clinicians felt they would benefit from a period of reduced patient load while learning the model and navigation of its components. Clinicians should be aware of the expectation to invest time and effort teaching MAs, as this investment yields a quick and significant payout as a cadre of high-functioning MAs. High-performing MAs can serve as mentors to junior or struggling MAs, answering questions or addressing issues in real time, and help on-board new hires. While we advocate patience and reasonable expectations to start, there remains a need for constant attention to model fidelity—this is where regular audits, planned adaptations, and enhanced training opportunities can be identified—as in the dynamic sustainability framework.⁴⁴

The practice coach for PCR is a “high touch,” dedicated approach to coaching that diverges from the classic quality improvement coach model.⁴⁵ PCR implementation benefited from attention to cultural aspects of transformation, such as team development, promoting change readiness, and focusing attention on “strong attractors” influential in complex adaptive change.^43,46–49 This could be particularly helpful in settings where providers and staff have different employers and different lines of authority.

Finally, a key finding was lack of clear organizational vision to all; those not directly involved in the planning were less convinced that motivation for PCR was broader than increasing revenue. Leaders undertaking implementation of a model like PCR are advised to continuously articulate reasons behind the change to clinicians and staff.⁵⁰ Evaluations and practice report cards should include data reflecting these goals, so practice members see effects of their efforts on primary goals. The role of primary care nurses should also be considered in the context of the scope of practice for all those contributing to primary care teams.⁵¹

Limitations

These results may not generalize to other settings. Other practices implementing PCR or other models with expanded MA roles may need to conduct their own evaluations to identify critical implementation factors and effective implementation strategies.

Conclusion

A participatory evaluation of the PCR model guided by implementation science frameworks RE-AIM and CFIR proved valuable for informing organizational decision-making about adapting, sustaining, and scaling up the model. The PCR model can be successfully implemented in both family and internal medicine practices. Clear and consistent communication from leadership on motivation to adopt the model, consistent pairing of MAs and clinicians, supportive technology, training and coaching on process and role change, and expecting and accommodating a short period of transformational change with decreased patient loads can support effective implementation.

Supplementary Information

Below is the link to the electronic supplementary material.

Funding

This evaluation was internally funded by the University of Colorado Department of Family Medicine.

Data Availability

The datasets analyzed during the current evaluation are available from the corresponding author on reasonable request and with appropriate ethical approvals.

Declarations

Conflict of Interest

The authors declare that they do not have a conflict of interest.