Attributes of High-Performing Small Practices in a Guideline Implementation: A Multiple-Case Study

Abstract

Objective

HealthyHearts NYC was a stepped wedge randomized control trial that tested the effectiveness of practice facilitation on the adoption of cardiovascular disease guidelines in small primary care practices. The objective of this study was to identify was to identify attributes of small practices that signaled they would perform well in a practice facilitation intervention implementation.

Methods

A mixed methods multiple-case study design was used. Six small practices were selected representing 3 variations in meeting the practice-level benchmark of >70% of hypertensive patients having controlled blood pressure. Inductive and deductive approaches were used to identify themes and assign case ratings. Cross-case rating comparison was used to identify attributes of high performing practices.

Results

Our first key finding is that the high-performing and improved practices in our study looked and acted similarly during the intervention implementation. The second key finding is that 3 attributes emerged in our analysis of determinants of high performance in small practices: (1) advanced use of the EHR; (2) dedicated resources and commitment to quality improvement; and (3) actively engaged lead clinician and office manager.

Conclusions

These attributes may be important determinants of high performance, indicating not only a small practice’s capability to engage in an intervention but possibly also its readiness to change. We recommend developing tools to assess readiness to change, specifically for small primary care practices, which may help external agents, like practice facilitators, better translate intervention implementations to context.

Introduction

Despite declines in number, small practices (<5 providers) still employ 35.7% of physicians in the US¹ and provide care to a majority of the population.² These practices face multiple challenges in meeting quality improvement goals, in part because improvement initiatives do not often account for the limited resources and constraints of the small practice setting.³ In 2014, the Agency for Healthcare Research and Quality set out to provide support to small practices, launching EvidenceNOW: Advancing Heart Health,⁴ a $112 million initiative to strengthen small practices’ capacity to adopt new models of care to improve cardiovascular disease (CVD) prevention and treatment.

Our study draws on data from HealthyHearts NYC, 1 of the EvidenceNOW grantees. HealthyHearts NYC was a cluster stepped-wedge randomized control trial that evaluated the use of practice facilitation (PF) to increase adoption of CVD guidelines.[citation redacted for blinded review] PF is the “deliberate and valued process of interactive problem-solving and support”⁵ through an external coach, and it is an emergent implementation strategy for supporting primary care practices’ efforts to meet quality improvement (QI) goals.^6-14 In HealthyHearts NYC, the intervention included 13 on-site facilitator visits over 1 year, in which facilitators worked with small practices to: educate them on the evidence-based CVD guidelines; redesign workflows to support CVD prevention and treatment; and optimize electronic health record (EHR) use for capturing and tracking patients with high risk for CVD. One of the key clinical outcomes of HealthyHearts NYC was practice-level blood pressure (BP) control.

The objective of this study was to identify attributes of small practices that signaled they would perform well in a practice facilitation intervention implementation. We use a mixed methods, multiple-case study approach¹⁵ to examine small practices from HealthyHearts NYC. Even though PF is an increasingly common implementation strategy for interventions in primary care settings,⁶ there are no studies examining what attributes of primary care practices may be determinants of high performance in PF interventions. This study builds on the primary care and PF literature by reporting on attributes of small primary care practices that may determine high performance during a PF intervention.

Methods

The study was approved by the NYU School of Medicine and New York City Department of Health and Mental Hygiene Institutional Review Boards. Our qualitative reporting adheres to the COREQ checklist.¹⁶

Study Setting and Study Design

HealthyHearts NYC is a partnership among the NYU School of Medicine and the Primary Care Information Project, a bureau of the New York City Department of Health and Mental Hygiene, which included 257 small practices.¹⁷ We use a mixed methods, convergence design, which is the concurrent analysis and interpretation of quantitative and qualitative data to explore the same phenomenon.¹⁸ We use the multiple-case study approach to focus on 6 small practices that participated in HealthyHearts NYC. The multiple-case study approach is often used to study complex implementations, as it allows for data triangulation of multiple sources of evidence.¹⁵ By design, multiple-case studies include a small number of distinct cases, with 3 to 4 as the recommended number of cases; small numbers allow for in-depth exploration of a phenomenon.^15,19

Case Selection

We purposively sampled cases to represent wide variation in performance during the intervention implementation. To optimize data quality, we targeted practices with a patient volume of at least 500 and that used the same EHR vendor. Performance was defined using the Centers for Disease Control and Prevention’s “Million Hearts” target, in which high performance was having at least 70% of its hypertensive patients with BP <140/90.²⁰ We create 3 BP performance categories: “high” performing practices (those that started above 70% and were now above 70%); “improved” practices (started below and were now above 70%); and “low” practices (started below and were still below 70%). About 9 months into the 1-year intervention, we assessed practices’ BP data to recruit 2 cases to correspond with each performance category, resulting in a final sample of 6 cases. Once cases were recruited, we collected qualitative data (described below).

Quantitative Data Sources and Measures

Practice-level BP data and number of patients came from the practices’ EHR. Practice-level characteristics were captured at baseline through the Practice Survey (completed by practice staff), including: number of providers, number of staff, and Patient-Centered Medical Home status. We extracted data on location in a medically underserved area from the U.S. Health and Human Services website.²¹ Clinician-level characteristics were captured post-intervention through the Clinician Survey (completed by the lead clinician), including: race, ethnicity, sex, age, whether born in the U.S., and whether graduated from a U.S. medical school.⁸

Qualitative Data Sources

Qualitative data sources included 3 semi-structured interviews with practice stakeholders (lead clinician, staff, and facilitator), facilitator visit notes, and ethnographic site observations. All interviews were 1-h, semi-structured, in-person (ie, in the practice), included only the interviewer and the participant, audio-recorded, and transcribed. The interview guides were tested internally and not shared with participants in advance. The research team did not have existing relationships with the practices and approached the sites by email and phone with the research objectives and requests for participation.

Interviews with the clinician and staff took place September 2016 to July 2017 were conducted by CB, DS, ER, and SK (all female; faculty with qualitative research experience). Interviews began with brief introductions about the interviewers’ roles and research goals and focused broadly about the implementation.²² All clinicians participated (n = 6) and 83.3% of staff members (n = 5), with 1 declining due to scheduling challenges; each was paid $50. Facilitator interviews took place July to August 2017 and were conducted by AC (female; project manager) and DS, began with brief introductions about the interviewers’ roles and research goals, and focused on experiences with the specific case study site. About 100% of facilitators agreed to participate (n = 6). Facilitators were offered a $25 honorarium but not required to participate. Transcripts were not returned to participants for comment.

Three researchers with formal ethnography training (AC, PN [female, researcher], LJ [female, project coordinator]) conducted site observations of a facilitator visit to each site in March-May 2017. Researchers observed for 2 to 3 h, took field notes by hand during the visit, typed up field notes immediately after the visit, and then debriefed with the research team. All practices agreed to participate (n = 6) and were paid $100.

Analytical Approach

We used inductive and deductive approaches in the context of an immersion/crystallization process, in which the qualitative team looked closely at the data (immersion) and then took a step back to identify patterns across the cases (crystallization).²³ First, the full qualitative team (AC, AN [female, researcher with mixed methods training], DS, ER, NVD [female, faculty with mixed methods training]) read documents for each case, blinded of the performance category, and met iteratively to develop a codebook (see Supplement A. Codebook). Two researchers (AN & AC) then used the codebook to independently code the documents using Dedoose^® software,²⁴ meeting weekly to review codes and create case memos. The full team met to discuss the memos and identified 3 categories for examination across cases: (1) use of EHR, (2) resources for QI, and (3) leadership engagement. Next, we used a cross-case rating comparison. The 2 coders used the criteria shown in Table 1²⁵ to review quotes in each document and assign ratings. Each case’s overall rating was calculated as the average of its documents’ ratings.²⁶ The full team met to discuss the ratings, identifying attributes (major themes and minor themes) that were determinants of high performance.

Table 1.

Rating Criteria for Qualitative Documents.

Rating	Criteria
−1	There is lack of evidence of the theme. This rating indicates that the theme does not have a positive influence on the practice or does not facilitate implementation efforts. The interviewee makes general statements about the theme manifesting in a negative way but without concrete examples:  The theme is mentioned only in passing or at a high level without examples or evidence of actual, concrete descriptions;  There is a mixed effect of different aspects of the theme but with a general overall negative effect;  There is sufficient information to make an indirect inference about the generally negative influence; and/or  Judged as weakly negative by the absence of the theme
0	The influence of the theme was unclear or neutral:  It appears to have neutral effect (purely descriptive) or is only mentioned generically without valence;  There is no evidence of positive or negative influence;  Interviewee contradicts his/herself;  There are positive and negative influences at different levels of the practice that balance each other out; and/or different aspects of the theme have positive influence while others have negative influence and overall, the effect is neutral
+1	There is evidence of the theme. This rating indicates that the theme has a positive influence on the practice or facilitated implementation efforts. The interviewee describes explicit examples of how the theme manifests itself in a positive way using concrete examples:  The theme is mentioned with examples or evidence of actual, concrete descriptions of how that construct manifests;  There is a general overall positive effect; and/or  There is sufficient information to make an indirect inference about the generally positive influence.
Missing	Interviewee(s) were not asked about the presence or influence of the construct; or if asked, their responses did not correspond to the intended theme and were instead coded to another construct. Interviewee(s)’ lack of knowledge about a theme does not necessarily indicate missing data and may indicate the absence of the theme.

Source: Adapted from Damschroder and Lowery.²⁵

Results

Table 2 shows the characteristics of the 6 cases, referred to by monikers such as High #1 and High #2. All were small, independent practices that completed the intervention as prescribed (at least 13 facilitator visits, indicating fidelity to the implementation plan). The low-performing cases had more staff than the high-performing and improved cases. The practice with the highest ratio of patients to providers was Improved #2, followed by Low #2. Both high-performing cases and Improved #1 were located in a medically underserved area. The lead clinicians’ characteristics were similar across all cases.

Table 2.

Profile of Cases.

	High #1	High #2	Improved #1	Improved #2	Low #1	Low #2
Practice characteristics
# of FTE providers*	1	2	1	1	2	2
# of FTE staff	2	2	3	2	8	6
# of patients	943	2044	727	2384	1303	4424
Patient-centered medical home status	Recognized	Recognized	Recognized	Recognized	Recognized	Recognized
Located in a medically underserved area	Yes	Yes	Yes	No	No	No
Lead clinician characteristics
Race	Asian	Black/African American	Black/African American	Asian	Black/African American	Black/African American
Ethnicity	Non-Hispanic	Non-Hispanic	Non-Hispanic	Non-Hispanic	Hispanic	Non-Hispanic
Sex	Male	Female	Male	Female	Male	Male
Age	55 to 64	65+	55 to 64	45 to 54	55 to 64	55 to 64
Born in U.S.	No	No	No	No	Yes	No
Graduated from medical school in U.S.	No	Yes	No	Yes	Yes	Yes

Note. FTE = Full-time equivalent; *Providers were defined as physicians, nurse practitioners, and physician assistants who provided primary care. We did not collect data on specialty type.

Our first key finding is: the high-performing and improved cases in our study looked and acted similarly during the intervention implementation. The 2 high-performing cases and the 2 improved cases have nearly identical characteristics (Table 1) and case ratings across the 3 code categories, “use of the EHR,” “resources for QI,” and “leadership engagement” (see Supplement B. Case ratings). These characteristics and ratings were distinctly different from the 2 low-performing cases.

The second key finding is that the high-performing and improved cases exhibited the following attributes (major themes) that may have been determinants of their high BP performance: (1) advanced use of the EHR; (2) dedicated resources and commitment to QI; and (3) actively engaged lead clinician and office manager. Given the similar nature of the high-performing and improved cases, below, we describe these 4 cases together, contrasting them with the 2 low-performing cases. Minor attributes (minor themes) are also described throughout.

Attribute 1: Advanced Use of EHR

The high-performing and improved cases all demonstrated advanced use of their EHR systems (major attribute) and a belief in the value of EHRs to facilitate high quality care. A facilitator described:

“This practice is a top spot on my list because they are using the HIT (health information technology). They’re strongly relying on and believe in HIT, and they’re using the HIT. That takes out a lot of my EHR teaching time.” (Improved #1: Facilitator)

These practices were able generate their own QI reports, which allowed them to engage with facilitators on more advanced EHR activities (minor attribute), such as discussing the use of reports to track intervention progress. To give an example, a staff member at High #2 and facilitator were observed reviewing a BP report together, where the staff member asked about how to compare data, rather than how to create the report: “How did we do on this measure last month? How are we in relation to other practices?” (High #2: Observation)

In comparison, the low cases reported having technical issues that they were unable to or did not have time to address themselves. During the intervention, facilitators spent many visits troubleshooting EHR issues to ensure the practice could document and assess the study’s measures accurately. A facilitator described a common experience in low-performing practices:

“Trying to connect to the EHR is sometimes challenging. If they need a ticket put into [EHR vendor], I actually put the ticket in for them with my name, so I can make sure it gets fixed.” (Low #2: Facilitator)

Attribute 2: Dedicated Resources and Commitment to QI

The high-performing and improved cases demonstrated dedicated resources and commitment to QI (major attribute), evidenced by protected time for specified staff to conduct QI activities. For example, High #2 specified a nurse as the point person who consistently met with the facilitator and monitored QI projects. The facilitator described the value having designated QI staff:

“Having a dedicated person in a QI role helps move the work along. It gives consistency. It’s hard to make progress when you don’t always meet with same person, especially if they don’t have the same level of EHR knowledge. You end up repeating what you did at the last visit.” (High #2: Facilitator)

Further, clinicians in these cases recognized the role and importance of data in QI (minor attribute). For example, High #1 described how they routinely used data to improve care quality:

“We get monthly or quarterly data from our independent practice association and from insurance companies. We look at the data on each doctor. We look at the performance. We have quality markers.” (High #1: Clinician)

The combination of having dedicated resources and a practice-level commitment to QI were indicative of a high-performing practice.

In comparison, the low-performing cases did not have sufficient nor consistent dedicated QI resources and commitment, despite having more staff. A lack of clarity in role definition for QI responsibilities as well as staff turnover may have contributed to reduced QI capacity, as exemplified:

“The practice had a glitch at one point [during the intervention] when there was a staff change. I had to speak with the admin during the time, not the office manager, because she was busy running the front desk.” (Low #1: Facilitator)

Attribute 3: Actively Engaged Lead Clinician and Office Manager

In the high-performing and improved cases, both the lead clinician and office manager were actively engaged in the intervention (major attribute). Office managers met with the facilitators every visit, with the lead clinicians also participating in most. One facilitator described the impact of high engagement: “You really need to have someone that’s really interested and engaged with you in order to be successful.” (High #1: Facilitator) The lead clinician and office manager were also able to recognize how the intervention goals were aligned with their own practice’s QI goals (minor attribute):

“I’m constantly involved in the HealthyHearts NYC project because it’s not just a HealthyHearts process. It’s the Patient-Centered Medical Home process.” (Improved #1: Clinician)

For the low-performing cases, the lead clinicians were not actively engaged with the intervention; facilitators documented meeting primarily with the office manager and/or other staff. These clinicians shared that they wanted to be more engaged with QI initiatives but did not have sufficient time, as they were “busy seeing patients” (Low #1: Clinician). Notably, Low #2 had 1 of the highest provider to patient ratios. This led to the clinicians “leaving everything to the office manager” (Low #2), and office managers subsequently feeling “overwhelmed” (Low #2: Facilitator) from QI activities on top of their usual responsibilities.

Discussion

This case study provided insight on attributes of small practices that may be associated with high performance in a practice facilitation-led guideline implementation. High-performing and improved cases demonstrated advanced use of the EHR, dedicated resources and commitment to QI, and an actively engaged lead clinician and office manager. Due these attributes, the practices that were already high-performing at baseline may have been able to quickly integrate the new CVD guidelines into their workflows, and the practices that improved performance over the intervention may have had greater readiness to leverage facilitator support to make changes. Our findings thus suggest that the identified attributes indicate the capability of a small practice to engage in the intervention.

There are no studies on attributes that determine high performance among small practices and thus we turn to the literature on large healthcare organizations to see if there are synergies. The attributes we identified in our study of small practices are surprisingly consistent with work that have identified attributes that determine high-performing large healthcare organizations. For example, in the study by D’Aunno et al. (2016), 3 factors that determined high-performing accountable care organizations in the Medicare Shared Savings program align with the attributes in our study: “sophisticated use of information systems,” “trusted, long-standing physician leaders focused on improving performance,” and “embedded care coordinators.”²⁷ In the study by Mannion et al. (2005), 2 factors that determined high-performing hospitals included: “leadership and management orientation” and “accountability and information systems.”²⁸ In Damschroder and Lowery (2013), 2 of the identified CFIR constructs that determined effective VA facilities align with ours: learning climate and leadership engagement.²⁵ Even though accountable care organizations, hospitals, and VA facilities are very different types of organizations than small primary care practices, it was not surprising that these studies all reported health IT, QI resources, and leadership engagement as core to better performance—these attributes are fundamental to measuring, monitoring, and improving care quality. Furthermore, we posit that these attributes speak to the need for an organization to have high levels of readiness to change to successfully implement a QI initiative.

While we did not measure readiness to change, the attributes identified in our study have similarities to the conceptual literature on organizational readiness to change.²⁹ For example, prior studies have recommended that facilitators be equipped with a baseline assessment tool to assess organizational readiness for engagement in QI.^30,31 However, there are currently no publicly-available measures of readiness for change for small practices.³² There is thus an opportunity to develop an organizational readiness tool specifically for small practices that could draw upon the findings of our study. Two existing tools serve as strong starting points: (1) the Quality Improvement Capacity Assessment (QICA), which is based on medical home principles and assesses capacity for QI in small practices, a component of readiness for change³³ and (2) the Organizational Readiness to Change Assessment instrument (ORCA), which has been widely used to assess implementation of evidence-based practices in large health organizations.³⁴ Neither assess EHR skills and competencies, however, which our study suggests is important to the success of QI projects. Prior work has also demonstrated that an EHR system’s limited capabilities can hinder QI in primary care practices.³⁵ We recommend research to create functional measures of EHR readiness that go beyond asking practices to self-report the presence of EHR functions or their perceived skill level. Such a tool could be utilized by implementation scientists and researchers to inform their knowledge translation strategies³⁶ and by practitioners to determine small primary care practices’ potential for change.

Limitations

Due to the New York City setting, findings may have limited generalizability to other settings. The multiple-case study approach, however, is not intended for wide generalizability, but rather, to gain insight on a specific phenomenon.¹⁹ Due to the case study design and resources, we were limited in the data we could use to assess thematic saturation; however, we believe thematic saturation was achieved by following case study best practices, which includes obtaining data from multiple sources (survey data, EHR data, interviewers with 3 stakeholders types, and site observations for every case) in a bounded time and space^15,19 and triangulating the data.^37-39 For the interviews and site observations, we minimized potential bias due to the interviewers’ knowledge of the primary care literature by utilizing a semi-structured interview/observation guide. For analyses, we used a multidisciplinary team and deliberately included co-author NVD, who had not been involved in the study up to this point to minimize groupthink, and the HPS and SS from the Department of Health and Mental Hygiene for external validation.

Conclusion

High-performing small practices exhibit attributes that may signify their ability and readiness for change during an intervention. For implementation scientists and practitioners, our study highlights the importance of assessing organizational readiness for change and developing functional measures of EHR readiness, specifically for the small practice setting. Appropriate readiness assessment may help external agents, like practice facilitators, better tailor intervention implementations to context and ultimately help small practices improve their care quality.