Improving Smoking and Blood Pressure Outcomes: The Interplay Between Operational Changes and Local Context

Abstract

PURPOSE

We undertook a study to identify conditions and operational changes linked to improvements in smoking and blood pressure (BP) outcomes in primary care.

METHODS

We purposively sampled and interviewed practice staff (eg, office managers, clinicians) from a subset of 104 practices participating in EvidenceNOW—a multisite cardiovascular disease prevention initiative. We calculated Clinical Quality Measure improvements, with targets of 10-point or greater absolute improvements in the proportion of patients with smoking screening and, if relevant, counseling and in the proportion of hypertensive patients with adequately controlled BP. We analyzed interview data to identify operational changes, transforming these into numeric data. We used Configurational Comparative Methods to assess the joint effects of multiple factors on outcomes.

RESULTS

In clinician-owned practices, implementing a workflow to routinely screen, counsel, and connect patients to smoking cessation resources, or implementing a documentation change or a referral to a resource alone led to an improvement of at least 10 points in the smoking outcome with a moderate level of facilitation support. These patterns did not manifest in health- or hospital system–owned practices or in Federally Qualified Health Centers, however. The BP outcome improved by at least 10 points among solo practices after medical assistants were trained to take an accurate BP. Among larger, clinician-owned practices, BP outcomes improved when practices implemented a second BP measurement when the first was elevated, and when staff learned where to document this information in the electronic health record. With 50 hours or more of facilitation, BP outcomes improved among larger and health- and hospital system–owned practices that implemented these operational changes.

CONCLUSIONS

There was no magic bullet for improving smoking or BP outcomes. Multiple combinations of operational changes led to improvements, but only in specific contexts of practice size and ownership, or dose of external facilitation.

INTRODUCTION

Cardiovascular disease (CVD) is the leading cause of death in the United States, with stroke or heart disease contributing to 1 out of every 3 deaths.¹ Primary risk factors for CVD include high blood pressure (BP), high cholesterol levels, and smoking. These risk factors are often preventable or treatable with low-cost, evidence-based interventions.¹ If the so-called ABCS of heart health—aspirin when indicated, blood pressure management, cholesterol management, and smoking cessation counseling and assistance—were consistently realized in primary care, CVD burden would be greatly reduced.^2,3 Yet, uptake of these clinical interventions is low.^4,5 Overall, only 53% of people with documented hypertension have their BP at target levels¹; less than one-half of those with elevated cholesterol have this condition treated⁶; and less than 25% of smokers get assistance with quitting.⁷

Although evidence of benefits of primary preventive care for CVD is abundant,^8,9 literature identifying how to implement these guidelines into practice is sparse. For example, there is strong evidence that the Ask-Advise-Connect approach increases smoking quit attempts.^10,11 The literature offers little evidence, however, of the operational changes (eg, systematic screening by a medical assistant, standing orders for medications when indicated) that are needed to ensure routine delivery of guideline-concordant smoking cessation counseling. The few studies that report on such operational changes tend to be small (single setting) or lack the details necessary to be usable by and transferable to other primary care practices.^12-15

Even when operational changes are known, implementing them in busy primary care practices can be difficult. External support, such as providing practices with a facilitator to help foster change, is effective.^16-19 Regional Extension Centers have employed fieldworkers to assist with meaningful use of electronic health records (EHRs), and the Centers for Medicare & Medicaid Services (CMS) and others have invested in a facilitator workforce, located in Quality Innovation Networks/Quality Improvement Organizations, to assist with quality improvement. Although organizations like these are adopting facilitation as an external support strategy, little is known about what types of practices respond best to facilitation and how much is needed to improve outcomes, such as smoking counseling and BP management.

EvidenceNOW, an Agency for Healthcare Research and Quality initiative, funded 7 grantees (called Cooperatives) across the United States to partner with or function as regional extensions, or both, engaging more than 200 primary care practices in their respective regions in quality improvement.²⁰ Cooperatives developed external support interventions, which involved a range of strategies (eg, education, health information technology support, audit and feedback), to assist practices in making operational changes to improve ABCS quality indicators. Facilitation was a core feature of each Cooperative’s approach.^21-24 Cooperatives trained and deployed their own facilitation workforce, with more than 158 facilitators employed across the initiative, and determined what facilitators would do (eg, content delivered, approach used) to support practices.

We conducted the national evaluation of EvidenceNOW.²⁵ We collected qualitative data from a subset of participating practices to answer the following research question: In the context of an initiative focused on improving CVD preventive care, what factors and operational changes were linked to improvements in smoking and BP outcomes?

To answer this question, we applied configurational comparative methods (CCMs). These methods offer a mathematical, case-based approach to cross-case analysis that uses set theory and Boolean algebra to identify crucial sets of difference-making combinations that distinguish one group of cases from another. CCMs operate from an analytic framework different from that of other quantitative approaches. Correlation-based and regression-based methods, for example, focus on relationships between variables and draw on an “interventionist” framework, assessing the incremental effect of a unit difference in independent variable x on the values of dependent variable y, controlling for all other variables. CCMs, by contrast, examine specific values of factors (ie, conditions) that are consistently necessary or sufficient for an outcome to appear, and rely on a “regularity” model of causality.^26-28 The regularity analytic framework fits our research question particularly well in that it allows for the evaluation of both causal complexity (ie, the joint presence of conditions) and equifinality (ie, multiple solution paths to the same outcome), and is robust with smaller sample sizes.^29,30 We linked this analytic framework with the theoretical framework of the Practice Change Model, which identifies critical elements for guiding practice change and emphasizes the importance of evolving interrelationships among elements, including stakeholder motivation, practice resources for change, external motivators, and options for change.³¹ CCMs are appearing more prominently in health care research.^29,30,32,33 A recent Annals article featured a glossary of commonly used CCM terms.³⁴

METHODS

Setting and Sample

This study was conducted within the context of EvidenceNOW²⁵ and was approved by the Oregon Health & Science University Institutional Review Board. Primary care practices were spread across 7 Cooperatives and 12 states, and were small to medium in size, having 10 or fewer clinicians. In the larger initiative, 1,270 practices submitted at least 4 quarters of data on outcome measures. From this larger set, we purposively selected a maximum variation practice sample, varying on Cooperative affiliation, ownership, size, facilitation dose, and outcome change.³⁵ We conducted separate interviews with each practice’s facilitator and a practice member (ie, office manager or clinic lead), and these individuals became the analytic sample for this study. We analyzed interviews as they accrued, building insights to inform subsequent sampling decisions and refine interview procedures. This iterative process continued until we had 104 practices in our analytic sample that represented nearly equal numbers of practices from each Cooperative and variation in the overall subsample on the above attributes.

Data Collection

Outcome Measures

Analysis focused on 2 outcome measures that were extracted from practices’ EHRs: smoking and BP. The smoking outcome was measured using a CMS clinical quality measure (CMS eCQM 138v4), which was defined as the proportion of patients aged 18 years and older who were screened for tobacco use at least once within 24 months and who received cessation counseling if identified as a tobacco user. The BP outcome (CMS eCQM 165v4) was defined as the proportion of patients aged 18 to 85 years with a diagnosis of hypertension whose BP was adequately controlled (less than 140/90 mm Hg) during the measurement period.

We analyzed the smoking and BP data sets separately. We calculated outcome improvement as the difference in performance from baseline (before start of the intervention) to end of intervention (12 months later). For our main analyses, we set outcome targets at improvements of 10 percentage points in absolute terms for both smoking and BP. These cutoffs were selected because changes of this magnitude were clinically meaningful and feasible in 12 months, the most common length of EvidenceNOW interventions. We also conducted secondary analyses based on improvements of at least 5 percentage points.

Supplemental Table 1 (available at https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) lists the quantitative measures contained in our analysis. We included practice characteristics (size, ownership, location on the urban-rural continuum, and turnover). Our selection of these factors was informed by the Practice Change Model,³¹ particularly the factors that influence practices’ resources for change. Facilitation was included because of its ability to increase practice capacity for and motivation to change, and was assessed based on duration, time, and dose (number of in-person touches), calculated from tracking logs maintained by facilitators.

Identification of Practice Operational Changes

To identify the operational changes practices implemented to improve smoking and BP outcomes, we collected qualitative data in the form of semistructured interviews with each practice’s facilitator and with a member of the practice for 58 of the 104 practices. Facilitator interviews were conducted first, which allowed the facilitator to identify another individual to interview at the practice, and provided information critical to deeper exploration of topics with practice participants. Facilitator and practice interviews followed a semistructured guide (Supplemental Appendix, available at https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/).

Interviews were conducted by experienced qualitative researchers by telephone, lasted 30 to 60 minutes, and were audio recorded. Interviewers first asked openly about the practice changes implemented to improve the smoking and BP outcomes, and probed for further details if needed. The interviews were professionally transcribed, checked for accuracy, and deidentified. Qualitative data were uploaded to ATLAS.ti (Scientific Software Development GmbH) for data management and analysis.

Analysis

A 6-person team (D.J.C., S.M.S., W.L.M., J.D.H., T.T.W., and S.O.) analyzed interview data in order to transform these qualitative data into quantitative factors for the CCMs analysis. Supplemental Table 2 (available at https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) shows the practice changes we identified, their definitions, and how we calibrated these measures. To accomplish this task, we used a multistep process. First, we started with the practices for which we had an interview with both the facilitator and a practice member. We analyzed these interviews to ensure responses were aligned, which they were. This alignment gave us confidence about including practices where we obtained only a facilitator interview. We also dropped practices from the analysis when we lacked sufficient detail and clarity about practice changes.

Next, we assigned numerical scores to our data in order to analyze the data with CCMs.³³ To do this, 3 team members (S.M.S., J.D.H., and T.T.W.) analyzed interviews independently and assigned a numeric value to the data, and we compared ratings. Analysis was complete when agreement was reached (interrater reliability = 95% for smoking and 96% for BP). To ensure 100% agreement, a third team member (D.J.C. or W.L.M.) reviewed the data or the group discussed the case until a determination was made. Some discussions led to codebook and scoring system revisions; these changes were then applied to the full subsample.

Qualitative analysis and the Practice Change Model³¹ suggested that certain conditions—specifically, practice characteristics, facilitation dose, and the types of operational changes practices implemented—occurred in combinations, which seemed to be important to explain outcome improvement. We used CCMs to assess the joint effects of multiple conditions on outcomes. Analyses focused on identifying condition combinations linked to improvements in smoking and BP outcomes. The R package “cna” was used to conduct Coincidence Analysis, which is a specific approach within the larger family of CCMs; we also used R (version 3.5.0) and R Studio (version 1.1.383) to support the analysis.^36-38

We used a multistep configurational approach consistent with the “regularity” analytic framework used in the overall CCM analysis for selecting relevant factors. This data reduction approach has been described in previous publications^39,40 and is summarized here. To select initial factors to use in model iteration, we applied the “minimally sufficient conditions” (ie, “msc”) function within the R package “cna” to look across all cases and all 17 factors at once, and identified all 1-, 2-, 3-, 4-, and 5-factor configurations that met dual consistency and coverage thresholds. As our primary analytic target was modifiable factors (practice operational changes and implementation characteristics), we initially focused on configurations that had at least 1 practice change and 1 implementation-related factor. We then used that factor-level information to guide selection of a smaller subset to include in model iteration. Supplemental Tables 1 and 2 list the factors, conditions, and their calibrations (numeric values).

To develop the models for improvements in smoking and BP outcomes, we started with the sample of 104 practices. Supplemental Figure 1 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) shows the number of practices dropped from each sample and the reasons why (eg, lack of performance data, ceiling effect).

Our analytic goal was to develop overall models with high consistency, substantial coverage, and no model ambiguity. For our analysis, this goal meant that our final models needed to explain at least two-thirds of the practices achieving at least 10-point gains (ie, coverage) and yield the outcome (gain of 10 points or more) at least 80% of the time the solution appeared anywhere in the data set (ie, consistency), and yield only 1 solution. After developing our final model for the smoking outcome, we removed 5 additional practices from the smoking data set because they had at least 1 missing value for a factor in the solution. We took the same step in BP model development; 5 practices were removed from the data set for this outcome as well (Supplemental Figure 1).

We report the results of the configurational analyses, which allow for equifinality in models (a solution where multiple paths lead to the same outcome). In these situations, individual paths are called pathways to indicate that any one pathway by itself is sufficient for the outcome.

RESULTS

Practice Characteristics

Practices included in the main analyses for both outcomes—59 for the smoking outcome and 73 for the BP outcome—varied with respect to ownership, size, geography, location, and patient panel characteristics (Table 1). These practices were purposively selected and therefore differed in most characteristics compared with the overall EvidenceNOW sample (data not shown).

Table 1.

Primary Care Practice Characteristics

Characteristic	Practices in Smoking Analysis (N = 59)	Practices in BP Analysis (N = 73)
Practice characteristics
Ownership, No. (%)
Clinicians	36 (61.0)	43 (58.9)
Hospital/health system	13 (22.0)	16 (21.9)
FQHC	7 (11.9)	9 (12.3)
RHC/IHS	3 (5.1)	5 (6.8)
Practice size, No. (%)
Solo practice	19 (32.2)	21 (28.8)
2-5 clinicians	31 (52.5)	41 (56.2)
6-10 clinicians	6 (10.2)	8 (11.0)
≥11 clinicians	2 (3.4)	2 (2.7)
Missing	1 (1.7)	1 (1.4)
Geographic region/Cooperative, No. (%)
Midwest (IN, IL, WI)	12 (20.3)	15 (20.5)
North Carolina	8 (13.6)	9 (12.3)
Northwest (OR, WA, ID)	4 (6.8)	7 (9.6)
New York City (5 NY boroughs)	15 (25.4)	14 (19.2)
Oklahoma	7 (11.9)	9 (12.3)
Southwest (CO, NM)	9 (15.3)	11 (15.1)
Virginia	4 (6.8)	8 (11.0)
Location, No. (%)
Rural area	4 (6.8)	8 (11.0)
Large town	10 (16.9)	12 (16.4)
Suburban	4 (6.8)	6 (8.2)
Urban core	41 (69.5)	47 (64.4)
Patient characteristics
White, mean (SD), %	61.4 (33.7)	60.9 (33.4)
Medicaid coverage, mean (SD), %	20.5 (17.1)	19.6 (17.2)
Performance on CQM metric at baseline, No. (%)
<50%	26 (44.1)	20 (27.4)
50%-60%	2 (3.4)	16 (21.9)
60%-70%	6 (10.2)	19 (26.0)
70%-80%	14 (23.7)	15 (20.5)
80%-90%	11 (18.6)	3 (4.1)

BP = blood pressure; CO = Colorado; CQM = clinical quality measure; FQHC = Federally Qualified Health Center; ID = Idaho; IHS = Indian Health Service; IL = Illinois; IN = Indiana, NM = New Mexico; NY = New York; OR = Oregon; RHC = rural health clinic; WA = Washington; WI = Wisconsin.

Note: Practices with >90% performance on the smoking CQM (11 practices) or BP CQM (1 practice) at baseline were excluded from analyses because they could not logically achieve a ≥10-point gain.

For both outcomes, most practices were small (fewer than 6 clinicians), clinician owned, and/or in an urban location (Table 1). More than one-half (57.6%) of practices failed to meet the Million Hearts threshold of more than 70% for the smoking performance metric and an even larger share (71.2%) failed to meet the BP performance metric of less than 140/90 mm Hg at baseline. We considered the potential for baseline performance to influence improvement outcomes; however, when we compared the mean baseline rates for practices that did and did not achieve gains of at least 10 points for smoking and for BP, differences in these rates were not statistically significant in either case (data not shown).

Characteristics of the practices included in the analyses of 5-point or greater gains in outcomes are shown in Supplemental Table 3, https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/.

Pathways Linked to Improved Smoking Outcome

Three pathways were linked to an improvement of at least 10 points in the smoking outcome (Table 2). In clinician-owned practices, process improvement, which we defined as implementing a workflow change so that either clinicians or medical assistants routinely screened and counseled patients, and connected them to smoking cessation resources, led to such improvement in smoking outcome. In addition, all practices that reported implementing any of the 3 improvements (process improvement, documentation, and referral to resources such as a quitline), coupled with a moderate level of facilitation support, improved the smoking outcome by at least 10 points. These 3 pathways together explained 22 of the 29 practices that had such improvement (76% coverage) in the smoking outcome with high consistency (92%). The third pathway is of note because it involved practices that implemented referral to resources, did not track this referral, and received 10 to 24.9 hours of facilitation.

Table 2.

Pathways Linked to a ≥10-Point Gain in Smoking Outcome

Pathway	Consistency, % (No./N)	Raw Coverage, % (No./N)	Unique Coverage, % (No./N)	Cases Included in Pathwaya
Process improvement = 1 AND Ownership = clinician	93 (13/14)	45 (13/29)	21 (6/29)	A1, B1, B2, B3, B4, C1, E1, E2, F1, F2, F12, G1, G2
Any operational practice change = 1b AND Duration of facilitation = 25-49.9 hours	91 (10/11)	34 (10/29)	21 (6/29)	A3, A4, B1, B2, B3, B5, B7, B12, G1, G3
Identify referral resources = 1 AND	100 (6/6)	20 (6/29)	10 (3/29)	A5, C2, E1, E2, F1, G4
Referral tracking = 0 AND
Duration of facilitation = 10-24.9 hours
Overall model	92 (22/24)	76 (22/29)	N/A

N/A = not applicable.

^aBold indicates cases uniquely explained by each pathway. Each letter refers to a different Cooperative and each number to a different practice in that Cooperative.

^bAny 1 of 3 operational changes (documentation, process improvement, and/or referral to resources).

Supplemental Figure 2 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/)) depicts this solution visually and shows that these patterns did not manifest in health- or hospital system–owned practices or Federally Qualified Health Centers. Table 3 provides excerpts from interventions that further demonstrate these findings. The analysis for an improvement of 5 points or more in the smoking outcome confirmed these results, yielding the same solution and similarly meeting criteria for model coverage and consistency (Supplemental Table 4, https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/).

Table 3.

Qualitative Excerpts Demonstrating Pathways Linked to a ≥10-Point Gain in Smoking Outcome

Pathway	Excerpt
Process improvement: changed practice workflows including processes to ensure clinicians provided brief counseling, changed workflow to enable MAs to provide brief counseling/referral for patients AND Clinician ownership	Basically, they had a standing order for these tobacco users ... that will occur during the visit. There were some aspects of it that the MA can do or the person rooming. They had plenty of literature about tobacco cessation, as well as the state quitlines and different resources that are available, so they would do some of that. Then there was more counseling done within the visit, by the provider. (Facilitator interview; 2-5 clinicians; clinician owned)
Any of 3 operational changes: Process improvement: changed practice workflows including processes to ensure clinicians provided brief counseling, changed workflow to enable MAs to provide brief counseling/referral for patients Documentation: reported working to change documentation behavior after someone in practice learned they were not documenting correctly Identify referral resources: gave information about quitlines and other resources to patients AND 25-49.9 facilitation hours	Their smoking [assessment and counseling] was a little bit on the low side, so I made sure I went back to it just to tell them, this is the protocol you need to follow. Make sure every patient is being screened for smoking. If there is a patient that’s smoking, make sure they give all the information that’s needed for them to quit. (Facilitator interview; solo practice; clinician owned; facilitation duration 4 [25-49.9 hours with a facilitator]) What I focused on was helping them to document properly in [their EHR] because [their EHR] has a very specific way to document smoking cessation counseling. It has to be documented in a particular place. (Facilitator interview; 2-5 clinicians; FQHC; facilitation duration 4 [25-49.9 hours with a facilitator]) We were given a lot of brochures, and I was able to get a lot of free gums and patches for patients that couldn’t get them before.… We [previously] didn’t even know about [the quitline], that we can refer [patients] for that, and that they can receive free patches and gum. (Practice interview; 2-5 clinicians; clinician owned; facilitation duration 4 [25-49.9 hours with a facilitator])
Identify referral resources: gave information about quitlines and other resources to patients WITHOUT Tracking referrals: tracked patient’s referral and followed through on a referral to a quitline or another smoking cessation resource AND 10-24.9 hours of facilitation	With the smoking, we did a lot of patient education. We pushed that 1-800-QUITNOW smoke line that [our state] has. We had tear-out pages where you post it on the wall and people would just tear off the number; something that no one really notices, so we put some in the bathroom. We made sure that they were in the [examination] rooms. (Facilitator interview; 2-5 clinicians; clinician owned; facilitation duration 3 [10-24.9 hours with a facilitator])

EHR = electronic health record; FQHC = Federally Qualified Health Center; MA = medical assistant.

Pathways Linked to Improved BP Outcome

Four pathways were linked to an improvement of at least 10 points in the BP outcome (Table 4). For solo practices, training medical assistants to take an accurate BP led to improvement of this magnitude. For clinician-owned practices, taking a second BP when the first was elevated and learning where to document this reading in the EHR also led to such improvement. For all practices, these operational changes led to a 10-point or greater improvement in BP outcome when coupled with a substantial amount of facilitation. These 4 pathways together explained 18 of the 26 practices that had a gain of 10 points or more (69% coverage) in the BP outcome with high consistency (82%).

Table 4.

Pathways Linked to a ≥10-Point Gain in BP Outcome

Pathway	Consistency, % (No./N)	Raw Coverage, % (No./N)	Unique Coverage, % (No./N)	Cases Included in Pathwaya
Take second BP = 1 AND Documentation = 1 AND Ownership = clinician	91 (10/11)	38 (10/26)	19 (5/26)	A2, A3, A6, B8, C3, C4, E2, E3, F3, G2
Take second BP = 1 AND Duration of facilitation = ≥50 hours	100 (3/3)	12 (3/26)	4 (2/26)	A1, B8, B9
Measurement training = 2 AND Duration of facilitation = 10-24.9 hours	75 (3/4)	12 (3/26)	4 (2/26)	E2, G4, G5
Measurement training = 1 (ie, without follow-up) AND Size = solo clinician	73 (8/11)	31 (8/26)	15 (4/26)	A2, A3, A6, A7, B8, F1, F4, F5
Overall model	82 (18/22)	69 (18/26)	N/A

BP = blood pressure; N/A = not applicable.

^aBold indicates cases uniquely explained by each pathway. Each letter refers to a different Cooperative and each number to a different practice in that Cooperative.

Supplemental Figure 3 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/)) depicts this solution visually, and Table 5 provides excerpts from qualitative interviews that demonstrate these findings. Supplemental Table 5 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) shows that the analysis for an improvement of 5 points or more in the BP outcome confirmed these results and identified some additional factors. Supplemental Table 6 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) additionally compares the maximum-variation sample of practices used in this analysis with the EvidenceNOW practices not included.

Table 5.

Qualitative Excerpts Demonstrating Pathways Linked to a ≥10-Point Gain in BP Outcome

Pathway	Excerpt
Measurement training: educated practice staff about how to take an accurate BP AND Solo practice	We gave different tips on making sure that [the patient is] keeping both feet on the floor, [their] back is supported, they have an empty bladder, they’re not talking. Different tips like that to try to help patients lower their blood pressure. My biggest thing for all my providers that I recommended is to not talk to the patient while you’re doing it. Because I know most nurses or MAs are still trying to collect information while they’re taking blood pressure. (Facilitator interview; solo practice; clinician owned) [I] brought his nurse in, and the 3 of us [the facilitator, nurse, and clinician] went through proper blood pressure measurement. I printed out this PowerPoint that we had. ... We went, step by step, how to take the blood pressure. (Facilitator interview; solo practice; clinician owned)
Take second BP: took second BP during visit if first was elevated AND Documentation: practice developed method for documenting second or home BP as a discrete field in EHR AND Clinician ownership	We’re on the third floor so some patients say, “Oh, great. I’ll take the stairs.” By the time they get here their blood pressure is high, or else they’re late and they’re frantic, or they’ve been fighting the ice and snow, traffic, whatever. So, we get a high blood pressure reading and the MAs didn’t know to retake it at the end of the visit. ... Also understanding which [blood pressure] to record in our EHR and which is counted. If they take 3 blood pressures during the course of the appointment, which one is the one that’s pulling or counting? We had to do some digging and ask some questions and figure all that out. (Practice interview; 2-5 clinicians; clinician owned)
Take second BP: took second BP during visit if first was elevated AND ≥50 facilitation hours	We did workflow-mapping exercises, which [the practice] really liked. What I appreciated is they really spent time thinking about the workflow, and what worked and what didn’t. Whether the BP should be done again before—for example, before the patient sees the provider. (Facilitator interview; ≥11 clinicians; FQHC; facilitation duration 5 [≥50 hours with a facilitator])
Measurement training: educated practice staff about how to take an accurate BP AND 10-24.9 facilitation hours	They weren’t aware of the 5-minute rule, waiting, making sure the patient wasn’t talking, feet on the floor. We did some training, internal training, of how to take a blood pressure properly. That went over well. All of the MAs, all the providers, medical staff, went to the training. (Facilitator interview; 2-5 clinicians; system owned; facilitation duration 3 [10-24.9 hours with a facilitator])

BP = blood pressure; EHR = electronic health record; FQHC = Federally Qualified Health Center; MA = medical assistant.

DISCUSSION

In this study, we identified specific operational changes linked with improving CMS smoking and BP outcomes among a subset of practices participating in the EvidenceNOW initiative. Overall, the amount of external facilitation support, practice size, and ownership were key factors that defined the settings within which specific operational changes led to meaningful outcome improvements. The important role of relatively immutable practice characteristics (eg, size and ownership) in our models was striking. We initially excluded these attributes from our analysis because we wanted to focus on modifiable factors that would help inform actionable, practical approaches and policies to help practices improve smoking and BP outcomes. Through the course of our analyses, it became clear that making operational changes alone—in certain clinical settings—was insufficient to achieve meaningful improvements. Our solutions met consistency and coverage thresholds only when we introduced practice characteristics and facilitation dose into our models as factors. Matching the appropriate improvement approach from the mix of options available to key practice characteristics is important for achieving meaningful quality improvement gains.

It was not surprising to us that initiating the change of taking a second BP and documenting this second reading in a discrete EHR field so that it is calculated as part of the CMS metric was linked to a 10-point or greater improvement among clinician-owned practices. This is a setting where clinicians and their teams have the agency and internal motivation to relatively rapidly make and implement these types of changes. In contrast, system- or hospital-owned practices can have extra bureaucracy and centralized infrastructure that may limit practice-level agency, particularly when the change involves the EHR, a systemwide tool.⁴¹ Additionally, system and hospital leaders may include this change as part of a larger package of required changes, which may complicate implementation.

For certain types of changes, successful implementation traveled hand-in-hand with external facilitation. Of the 7 combined pathways we identified across the 2 outcomes, those not linked to practice size and ownership all included a moderate to substantial dose of facilitation. Facilitators helped practices use data to identify quality gaps, fostered motivation and decision making, empowered leaders and staff to identify and implement changes, and then helped them evolve those changes if needed.^42-44 The role of external facilitators may be of greater importance in hospital- and health system–owned practices, where implementing such changes is more complex and may require some prioritizing.

The operational changes linked to improved outcomes were pragmatic and unsurprising, with one possible exception. Although it may at first appear unusual⁴⁵ that not tracking a referral would be associated with improved performance on the smoking outcome, on further consideration, this finding may reflect the fact that the CMS smoking outcome measure does not assess quit rates. Following up on a referral might improve patient engagement with a quitline and rates of quit attempts, but this operational change does not necessarily improve the CMS measure, and could potentially distract practices from making broader operational changes (eg, systematic screening and brief counseling) that would improve the outcome. One of the strengths of CCMs is that it can yield unexpected associations that, when explored further, could potentially result in new discoveries.

Our findings suggest that individuals leading quality improvement efforts within primary care settings can substantially improve prospects for implementation success when they consider and tailor operational expectations to the practice setting. The findings also align with the larger health services and organizational change literature: size and ownership are 2 important factors to consider when undertaking a change, as these have implications for agency, decision-making complexity, and how action and change happen.^46-48

This study had a number of limitations. First, we relied on self-reported practice changes from facilitators and practice members, as observation of practice operational changes was not feasible. Second, although we know from qualitative sources that facilitators’ skills and approaches vary, and that this variation manifested both within and across Cooperatives, we did not have data at the practice level to assess the impact of these variations on outcomes. Third, although we can conclude that in this subset of practices, there was ample evidence for the solutions—and that these solutions were consistent with logic, theory, and prior knowledge—replication, experimental work, and application of additional methods would be ultimately required to establish the direction and strength of any causal relationships and generalizability. Fourth, EvidenceNOW focused on engaging smaller practices (those with no more than 10 clinicians) and our data set reflects this; we have few practices with more than 5 clinicians in our overall data set, and fewer that attained the 10-point or greater gain. This sampling limited our ability to fully examine the connection between practice size and facilitation dose. Although one might speculate that larger practices required more facilitation to align operations across a more expansive team, further research is needed to examine this important connection.

In conclusion, there was no magic bullet for improving smoking and BP outcomes across the diverse primary care practices in our analyses. Multiple combinations of operational changes led to improvements, but only in the context of practice size and ownership, or dose of external facilitation. Given this complex interplay between specific operational changes and local context, our analyses underscore the value of methods that can identify how particular factors work together to explain improvement in clinical outcomes.