Abstract
PURPOSE
We undertook a study to identify conditions and operational changes linked to improvements in smoking and blood pressure (BP) outcomes in primary care.
METHODS
We purposively sampled and interviewed practice staff (eg, office managers, clinicians) from a subset of 104 practices participating in EvidenceNOW—a multisite cardiovascular disease prevention initiative. We calculated Clinical Quality Measure improvements, with targets of 10-point or greater absolute improvements in the proportion of patients with smoking screening and, if relevant, counseling and in the proportion of hypertensive patients with adequately controlled BP. We analyzed interview data to identify operational changes, transforming these into numeric data. We used Configurational Comparative Methods to assess the joint effects of multiple factors on outcomes.
RESULTS
In clinician-owned practices, implementing a workflow to routinely screen, counsel, and connect patients to smoking cessation resources, or implementing a documentation change or a referral to a resource alone led to an improvement of at least 10 points in the smoking outcome with a moderate level of facilitation support. These patterns did not manifest in health- or hospital system–owned practices or in Federally Qualified Health Centers, however. The BP outcome improved by at least 10 points among solo practices after medical assistants were trained to take an accurate BP. Among larger, clinician-owned practices, BP outcomes improved when practices implemented a second BP measurement when the first was elevated, and when staff learned where to document this information in the electronic health record. With 50 hours or more of facilitation, BP outcomes improved among larger and health- and hospital system–owned practices that implemented these operational changes.
CONCLUSIONS
There was no magic bullet for improving smoking or BP outcomes. Multiple combinations of operational changes led to improvements, but only in specific contexts of practice size and ownership, or dose of external facilitation.
Key words: quality improvement, configurational comparative methods, mixed methods, cardiovascular prevention, smoking cessation, blood pressure management, organizational change, primary care, practice-based research
INTRODUCTION
Cardiovascular disease (CVD) is the leading cause of death in the United States, with stroke or heart disease contributing to 1 out of every 3 deaths.1 Primary risk factors for CVD include high blood pressure (BP), high cholesterol levels, and smoking. These risk factors are often preventable or treatable with low-cost, evidence-based interventions.1 If the so-called ABCS of heart health—aspirin when indicated, blood pressure management, cholesterol management, and smoking cessation counseling and assistance—were consistently realized in primary care, CVD burden would be greatly reduced.2,3 Yet, uptake of these clinical interventions is low.4,5 Overall, only 53% of people with documented hypertension have their BP at target levels1; less than one-half of those with elevated cholesterol have this condition treated6; and less than 25% of smokers get assistance with quitting.7
Although evidence of benefits of primary preventive care for CVD is abundant,8,9 literature identifying how to implement these guidelines into practice is sparse. For example, there is strong evidence that the Ask-Advise-Connect approach increases smoking quit attempts.10,11 The literature offers little evidence, however, of the operational changes (eg, systematic screening by a medical assistant, standing orders for medications when indicated) that are needed to ensure routine delivery of guideline-concordant smoking cessation counseling. The few studies that report on such operational changes tend to be small (single setting) or lack the details necessary to be usable by and transferable to other primary care practices.12-15
Even when operational changes are known, implementing them in busy primary care practices can be difficult. External support, such as providing practices with a facilitator to help foster change, is effective.16-19 Regional Extension Centers have employed fieldworkers to assist with meaningful use of electronic health records (EHRs), and the Centers for Medicare & Medicaid Services (CMS) and others have invested in a facilitator workforce, located in Quality Innovation Networks/Quality Improvement Organizations, to assist with quality improvement. Although organizations like these are adopting facilitation as an external support strategy, little is known about what types of practices respond best to facilitation and how much is needed to improve outcomes, such as smoking counseling and BP management.
EvidenceNOW, an Agency for Healthcare Research and Quality initiative, funded 7 grantees (called Cooperatives) across the United States to partner with or function as regional extensions, or both, engaging more than 200 primary care practices in their respective regions in quality improvement.20 Cooperatives developed external support interventions, which involved a range of strategies (eg, education, health information technology support, audit and feedback), to assist practices in making operational changes to improve ABCS quality indicators. Facilitation was a core feature of each Cooperative’s approach.21-24 Cooperatives trained and deployed their own facilitation workforce, with more than 158 facilitators employed across the initiative, and determined what facilitators would do (eg, content delivered, approach used) to support practices.
We conducted the national evaluation of EvidenceNOW.25 We collected qualitative data from a subset of participating practices to answer the following research question: In the context of an initiative focused on improving CVD preventive care, what factors and operational changes were linked to improvements in smoking and BP outcomes?
To answer this question, we applied configurational comparative methods (CCMs). These methods offer a mathematical, case-based approach to cross-case analysis that uses set theory and Boolean algebra to identify crucial sets of difference-making combinations that distinguish one group of cases from another. CCMs operate from an analytic framework different from that of other quantitative approaches. Correlation-based and regression-based methods, for example, focus on relationships between variables and draw on an “interventionist” framework, assessing the incremental effect of a unit difference in independent variable x on the values of dependent variable y, controlling for all other variables. CCMs, by contrast, examine specific values of factors (ie, conditions) that are consistently necessary or sufficient for an outcome to appear, and rely on a “regularity” model of causality.26-28 The regularity analytic framework fits our research question particularly well in that it allows for the evaluation of both causal complexity (ie, the joint presence of conditions) and equifinality (ie, multiple solution paths to the same outcome), and is robust with smaller sample sizes.29,30 We linked this analytic framework with the theoretical framework of the Practice Change Model, which identifies critical elements for guiding practice change and emphasizes the importance of evolving interrelationships among elements, including stakeholder motivation, practice resources for change, external motivators, and options for change.31 CCMs are appearing more prominently in health care research.29,30,32,33 A recent Annals article featured a glossary of commonly used CCM terms.34
METHODS
Setting and Sample
This study was conducted within the context of EvidenceNOW25 and was approved by the Oregon Health & Science University Institutional Review Board. Primary care practices were spread across 7 Cooperatives and 12 states, and were small to medium in size, having 10 or fewer clinicians. In the larger initiative, 1,270 practices submitted at least 4 quarters of data on outcome measures. From this larger set, we purposively selected a maximum variation practice sample, varying on Cooperative affiliation, ownership, size, facilitation dose, and outcome change.35 We conducted separate interviews with each practice’s facilitator and a practice member (ie, office manager or clinic lead), and these individuals became the analytic sample for this study. We analyzed interviews as they accrued, building insights to inform subsequent sampling decisions and refine interview procedures. This iterative process continued until we had 104 practices in our analytic sample that represented nearly equal numbers of practices from each Cooperative and variation in the overall subsample on the above attributes.
Data Collection
Outcome Measures
Analysis focused on 2 outcome measures that were extracted from practices’ EHRs: smoking and BP. The smoking outcome was measured using a CMS clinical quality measure (CMS eCQM 138v4), which was defined as the proportion of patients aged 18 years and older who were screened for tobacco use at least once within 24 months and who received cessation counseling if identified as a tobacco user. The BP outcome (CMS eCQM 165v4) was defined as the proportion of patients aged 18 to 85 years with a diagnosis of hypertension whose BP was adequately controlled (less than 140/90 mm Hg) during the measurement period.
We analyzed the smoking and BP data sets separately. We calculated outcome improvement as the difference in performance from baseline (before start of the intervention) to end of intervention (12 months later). For our main analyses, we set outcome targets at improvements of 10 percentage points in absolute terms for both smoking and BP. These cutoffs were selected because changes of this magnitude were clinically meaningful and feasible in 12 months, the most common length of EvidenceNOW interventions. We also conducted secondary analyses based on improvements of at least 5 percentage points.
Supplemental Table 1 (available at https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) lists the quantitative measures contained in our analysis. We included practice characteristics (size, ownership, location on the urban-rural continuum, and turnover). Our selection of these factors was informed by the Practice Change Model,31 particularly the factors that influence practices’ resources for change. Facilitation was included because of its ability to increase practice capacity for and motivation to change, and was assessed based on duration, time, and dose (number of in-person touches), calculated from tracking logs maintained by facilitators.
Identification of Practice Operational Changes
To identify the operational changes practices implemented to improve smoking and BP outcomes, we collected qualitative data in the form of semistructured interviews with each practice’s facilitator and with a member of the practice for 58 of the 104 practices. Facilitator interviews were conducted first, which allowed the facilitator to identify another individual to interview at the practice, and provided information critical to deeper exploration of topics with practice participants. Facilitator and practice interviews followed a semistructured guide (Supplemental Appendix, available at https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/).
Interviews were conducted by experienced qualitative researchers by telephone, lasted 30 to 60 minutes, and were audio recorded. Interviewers first asked openly about the practice changes implemented to improve the smoking and BP outcomes, and probed for further details if needed. The interviews were professionally transcribed, checked for accuracy, and deidentified. Qualitative data were uploaded to ATLAS.ti (Scientific Software Development GmbH) for data management and analysis.
Analysis
A 6-person team (D.J.C., S.M.S., W.L.M., J.D.H., T.T.W., and S.O.) analyzed interview data in order to transform these qualitative data into quantitative factors for the CCMs analysis. Supplemental Table 2 (available at https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) shows the practice changes we identified, their definitions, and how we calibrated these measures. To accomplish this task, we used a multistep process. First, we started with the practices for which we had an interview with both the facilitator and a practice member. We analyzed these interviews to ensure responses were aligned, which they were. This alignment gave us confidence about including practices where we obtained only a facilitator interview. We also dropped practices from the analysis when we lacked sufficient detail and clarity about practice changes.
Next, we assigned numerical scores to our data in order to analyze the data with CCMs.33 To do this, 3 team members (S.M.S., J.D.H., and T.T.W.) analyzed interviews independently and assigned a numeric value to the data, and we compared ratings. Analysis was complete when agreement was reached (interrater reliability = 95% for smoking and 96% for BP). To ensure 100% agreement, a third team member (D.J.C. or W.L.M.) reviewed the data or the group discussed the case until a determination was made. Some discussions led to codebook and scoring system revisions; these changes were then applied to the full subsample.
Qualitative analysis and the Practice Change Model31 suggested that certain conditions—specifically, practice characteristics, facilitation dose, and the types of operational changes practices implemented—occurred in combinations, which seemed to be important to explain outcome improvement. We used CCMs to assess the joint effects of multiple conditions on outcomes. Analyses focused on identifying condition combinations linked to improvements in smoking and BP outcomes. The R package “cna” was used to conduct Coincidence Analysis, which is a specific approach within the larger family of CCMs; we also used R (version 3.5.0) and R Studio (version 1.1.383) to support the analysis.36-38
We used a multistep configurational approach consistent with the “regularity” analytic framework used in the overall CCM analysis for selecting relevant factors. This data reduction approach has been described in previous publications39,40 and is summarized here. To select initial factors to use in model iteration, we applied the “minimally sufficient conditions” (ie, “msc”) function within the R package “cna” to look across all cases and all 17 factors at once, and identified all 1-, 2-, 3-, 4-, and 5-factor configurations that met dual consistency and coverage thresholds. As our primary analytic target was modifiable factors (practice operational changes and implementation characteristics), we initially focused on configurations that had at least 1 practice change and 1 implementation-related factor. We then used that factor-level information to guide selection of a smaller subset to include in model iteration. Supplemental Tables 1 and 2 list the factors, conditions, and their calibrations (numeric values).
To develop the models for improvements in smoking and BP outcomes, we started with the sample of 104 practices. Supplemental Figure 1 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) shows the number of practices dropped from each sample and the reasons why (eg, lack of performance data, ceiling effect).
Our analytic goal was to develop overall models with high consistency, substantial coverage, and no model ambiguity. For our analysis, this goal meant that our final models needed to explain at least two-thirds of the practices achieving at least 10-point gains (ie, coverage) and yield the outcome (gain of 10 points or more) at least 80% of the time the solution appeared anywhere in the data set (ie, consistency), and yield only 1 solution. After developing our final model for the smoking outcome, we removed 5 additional practices from the smoking data set because they had at least 1 missing value for a factor in the solution. We took the same step in BP model development; 5 practices were removed from the data set for this outcome as well (Supplemental Figure 1).
We report the results of the configurational analyses, which allow for equifinality in models (a solution where multiple paths lead to the same outcome). In these situations, individual paths are called pathways to indicate that any one pathway by itself is sufficient for the outcome.
RESULTS
Practice Characteristics
Practices included in the main analyses for both outcomes—59 for the smoking outcome and 73 for the BP outcome—varied with respect to ownership, size, geography, location, and patient panel characteristics (Table 1). These practices were purposively selected and therefore differed in most characteristics compared with the overall EvidenceNOW sample (data not shown).
Table 1.
Primary Care Practice Characteristics
| Characteristic | Practices in Smoking Analysis (N = 59) | Practices in BP Analysis (N = 73) |
|---|---|---|
| Practice characteristics | ||
| Ownership, No. (%) | ||
| Clinicians | 36 (61.0) | 43 (58.9) |
| Hospital/health system | 13 (22.0) | 16 (21.9) |
| FQHC | 7 (11.9) | 9 (12.3) |
| RHC/IHS | 3 (5.1) | 5 (6.8) |
| Practice size, No. (%) | ||
| Solo practice | 19 (32.2) | 21 (28.8) |
| 2-5 clinicians | 31 (52.5) | 41 (56.2) |
| 6-10 clinicians | 6 (10.2) | 8 (11.0) |
| ≥11 clinicians | 2 (3.4) | 2 (2.7) |
| Missing | 1 (1.7) | 1 (1.4) |
| Geographic region/Cooperative, No. (%) | ||
| Midwest (IN, IL, WI) | 12 (20.3) | 15 (20.5) |
| North Carolina | 8 (13.6) | 9 (12.3) |
| Northwest (OR, WA, ID) | 4 (6.8) | 7 (9.6) |
| New York City (5 NY boroughs) | 15 (25.4) | 14 (19.2) |
| Oklahoma | 7 (11.9) | 9 (12.3) |
| Southwest (CO, NM) | 9 (15.3) | 11 (15.1) |
| Virginia | 4 (6.8) | 8 (11.0) |
| Location, No. (%) | ||
| Rural area | 4 (6.8) | 8 (11.0) |
| Large town | 10 (16.9) | 12 (16.4) |
| Suburban | 4 (6.8) | 6 (8.2) |
| Urban core | 41 (69.5) | 47 (64.4) |
| Patient characteristics | ||
| White, mean (SD), % | 61.4 (33.7) | 60.9 (33.4) |
| Medicaid coverage, mean (SD), % | 20.5 (17.1) | 19.6 (17.2) |
| Performance on CQM metric at baseline, No. (%) | ||
| <50% | 26 (44.1) | 20 (27.4) |
| 50%-60% | 2 (3.4) | 16 (21.9) |
| 60%-70% | 6 (10.2) | 19 (26.0) |
| 70%-80% | 14 (23.7) | 15 (20.5) |
| 80%-90% | 11 (18.6) | 3 (4.1) |
BP = blood pressure; CO = Colorado; CQM = clinical quality measure; FQHC = Federally Qualified Health Center; ID = Idaho; IHS = Indian Health Service; IL = Illinois; IN = Indiana, NM = New Mexico; NY = New York; OR = Oregon; RHC = rural health clinic; WA = Washington; WI = Wisconsin.
Note: Practices with >90% performance on the smoking CQM (11 practices) or BP CQM (1 practice) at baseline were excluded from analyses because they could not logically achieve a ≥10-point gain.
For both outcomes, most practices were small (fewer than 6 clinicians), clinician owned, and/or in an urban location (Table 1). More than one-half (57.6%) of practices failed to meet the Million Hearts threshold of more than 70% for the smoking performance metric and an even larger share (71.2%) failed to meet the BP performance metric of less than 140/90 mm Hg at baseline. We considered the potential for baseline performance to influence improvement outcomes; however, when we compared the mean baseline rates for practices that did and did not achieve gains of at least 10 points for smoking and for BP, differences in these rates were not statistically significant in either case (data not shown).
Characteristics of the practices included in the analyses of 5-point or greater gains in outcomes are shown in Supplemental Table 3, https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/.
Pathways Linked to Improved Smoking Outcome
Three pathways were linked to an improvement of at least 10 points in the smoking outcome (Table 2). In clinician-owned practices, process improvement, which we defined as implementing a workflow change so that either clinicians or medical assistants routinely screened and counseled patients, and connected them to smoking cessation resources, led to such improvement in smoking outcome. In addition, all practices that reported implementing any of the 3 improvements (process improvement, documentation, and referral to resources such as a quitline), coupled with a moderate level of facilitation support, improved the smoking outcome by at least 10 points. These 3 pathways together explained 22 of the 29 practices that had such improvement (76% coverage) in the smoking outcome with high consistency (92%). The third pathway is of note because it involved practices that implemented referral to resources, did not track this referral, and received 10 to 24.9 hours of facilitation.
Table 2.
Pathways Linked to a ≥10-Point Gain in Smoking Outcome
| Pathway | Consistency, % (No./N) | Raw Coverage, % (No./N) | Unique Coverage, % (No./N) | Cases Included in Pathwaya |
|---|---|---|---|---|
| Process improvement = 1 AND Ownership = clinician |
93 (13/14) | 45 (13/29) | 21 (6/29) | A1, B1, B2, B3, B4, C1, E1, E2, F1, F2, F12, G1, G2 |
| Any operational practice change = 1b AND Duration of facilitation = 25-49.9 hours |
91 (10/11) | 34 (10/29) | 21 (6/29) | A3, A4, B1, B2, B3, B5, B7, B12, G1, G3 |
| Identify referral resources = 1 AND |
100 (6/6) | 20 (6/29) | 10 (3/29) | A5, C2, E1, E2, F1, G4 |
| Referral tracking = 0 AND |
||||
| Duration of facilitation = 10-24.9 hours | ||||
| Overall model | 92 (22/24) | 76 (22/29) | N/A |
N/A = not applicable.
Bold indicates cases uniquely explained by each pathway. Each letter refers to a different Cooperative and each number to a different practice in that Cooperative.
Any 1 of 3 operational changes (documentation, process improvement, and/or referral to resources).
Supplemental Figure 2 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/)) depicts this solution visually and shows that these patterns did not manifest in health- or hospital system–owned practices or Federally Qualified Health Centers. Table 3 provides excerpts from interventions that further demonstrate these findings. The analysis for an improvement of 5 points or more in the smoking outcome confirmed these results, yielding the same solution and similarly meeting criteria for model coverage and consistency (Supplemental Table 4, https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/).
Table 3.
Qualitative Excerpts Demonstrating Pathways Linked to a ≥10-Point Gain in Smoking Outcome
| Pathway | Excerpt |
|---|---|
| Process improvement: changed practice workflows including processes to ensure clinicians provided brief counseling, changed workflow to enable MAs to provide brief counseling/referral for patients AND Clinician ownership |
Basically, they had a standing order for these tobacco users ... that will occur during the visit. There were some aspects of it that the MA can do or the person rooming. They had plenty of literature about tobacco cessation, as well as the state quitlines and different resources that are available, so they would do some of that. Then there was more counseling done within the visit, by the provider. (Facilitator interview; 2-5 clinicians; clinician owned) |
Any of 3 operational changes:
AND 25-49.9 facilitation hours |
Their smoking [assessment and counseling] was a little bit on the low side, so I made sure I went back to it just to tell them, this is the protocol you need to follow. Make sure every patient is being screened for smoking. If there is a patient that’s smoking, make sure they give all the information that’s needed for them to quit. (Facilitator interview; solo practice; clinician owned; facilitation duration 4 [25-49.9 hours with a facilitator]) What I focused on was helping them to document properly in [their EHR] because [their EHR] has a very specific way to document smoking cessation counseling. It has to be documented in a particular place. (Facilitator interview; 2-5 clinicians; FQHC; facilitation duration 4 [25-49.9 hours with a facilitator]) We were given a lot of brochures, and I was able to get a lot of free gums and patches for patients that couldn’t get them before.… We [previously] didn’t even know about [the quitline], that we can refer [patients] for that, and that they can receive free patches and gum. (Practice interview; 2-5 clinicians; clinician owned; facilitation duration 4 [25-49.9 hours with a facilitator]) |
| Identify referral resources: gave information about quitlines and other resources to patients WITHOUT Tracking referrals: tracked patient’s referral and followed through on a referral to a quitline or another smoking cessation resource AND 10-24.9 hours of facilitation |
With the smoking, we did a lot of patient education. We pushed that 1-800-QUITNOW smoke line that [our state] has. We had tear-out pages where you post it on the wall and people would just tear off the number; something that no one really notices, so we put some in the bathroom. We made sure that they were in the [examination] rooms. (Facilitator interview; 2-5 clinicians; clinician owned; facilitation duration 3 [10-24.9 hours with a facilitator]) |
EHR = electronic health record; FQHC = Federally Qualified Health Center; MA = medical assistant.
Pathways Linked to Improved BP Outcome
Four pathways were linked to an improvement of at least 10 points in the BP outcome (Table 4). For solo practices, training medical assistants to take an accurate BP led to improvement of this magnitude. For clinician-owned practices, taking a second BP when the first was elevated and learning where to document this reading in the EHR also led to such improvement. For all practices, these operational changes led to a 10-point or greater improvement in BP outcome when coupled with a substantial amount of facilitation. These 4 pathways together explained 18 of the 26 practices that had a gain of 10 points or more (69% coverage) in the BP outcome with high consistency (82%).
Table 4.
Pathways Linked to a ≥10-Point Gain in BP Outcome
| Pathway | Consistency, % (No./N) | Raw Coverage, % (No./N) | Unique Coverage, % (No./N) | Cases Included in Pathwaya |
|---|---|---|---|---|
| Take second BP = 1 AND Documentation = 1 AND Ownership = clinician |
91 (10/11) | 38 (10/26) | 19 (5/26) | A2, A3, A6, B8, C3, C4, E2, E3, F3, G2 |
| Take second BP = 1 AND Duration of facilitation = ≥50 hours |
100 (3/3) | 12 (3/26) | 4 (2/26) | A1, B8, B9 |
| Measurement training = 2 AND Duration of facilitation = 10-24.9 hours |
75 (3/4) | 12 (3/26) | 4 (2/26) | E2, G4, G5 |
| Measurement training = 1 (ie, without follow-up) AND Size = solo clinician |
73 (8/11) | 31 (8/26) | 15 (4/26) | A2, A3, A6, A7, B8, F1, F4, F5 |
| Overall model | 82 (18/22) | 69 (18/26) | N/A |
BP = blood pressure; N/A = not applicable.
Bold indicates cases uniquely explained by each pathway. Each letter refers to a different Cooperative and each number to a different practice in that Cooperative.
Supplemental Figure 3 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/)) depicts this solution visually, and Table 5 provides excerpts from qualitative interviews that demonstrate these findings. Supplemental Table 5 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) shows that the analysis for an improvement of 5 points or more in the BP outcome confirmed these results and identified some additional factors. Supplemental Table 6 (https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/) additionally compares the maximum-variation sample of practices used in this analysis with the EvidenceNOW practices not included.
Table 5.
Qualitative Excerpts Demonstrating Pathways Linked to a ≥10-Point Gain in BP Outcome
| Pathway | Excerpt |
|---|---|
| Measurement training: educated practice staff about how to take an accurate BP AND Solo practice |
We gave different tips on making sure that [the patient is] keeping both feet on the floor, [their] back is supported, they have an empty bladder, they’re not talking. Different tips like that to try to help patients lower their blood pressure. My biggest thing for all my providers that I recommended is to not talk to the patient while you’re doing it. Because I know most nurses or MAs are still trying to collect information while they’re taking blood pressure. (Facilitator interview; solo practice; clinician owned) [I] brought his nurse in, and the 3 of us [the facilitator, nurse, and clinician] went through proper blood pressure measurement. I printed out this PowerPoint that we had. ... We went, step by step, how to take the blood pressure. (Facilitator interview; solo practice; clinician owned) |
| Take second BP: took second BP during visit if first was elevated AND Documentation: practice developed method for documenting second or home BP as a discrete field in EHR AND Clinician ownership |
We’re on the third floor so some patients say, “Oh, great. I’ll take the stairs.” By the time they get here their blood pressure is high, or else they’re late and they’re frantic, or they’ve been fighting the ice and snow, traffic, whatever. So, we get a high blood pressure reading and the MAs didn’t know to retake it at the end of the visit. ... Also understanding which [blood pressure] to record in our EHR and which is counted. If they take 3 blood pressures during the course of the appointment, which one is the one that’s pulling or counting? We had to do some digging and ask some questions and figure all that out. (Practice interview; 2-5 clinicians; clinician owned) |
| Take second BP: took second BP during visit if first was elevated AND ≥50 facilitation hours |
We did workflow-mapping exercises, which [the practice] really liked. What I appreciated is they really spent time thinking about the workflow, and what worked and what didn’t. Whether the BP should be done again before—for example, before the patient sees the provider. (Facilitator interview; ≥11 clinicians; FQHC; facilitation duration 5 [≥50 hours with a facilitator]) |
| Measurement training: educated practice staff about how to take an accurate BP AND 10-24.9 facilitation hours |
They weren’t aware of the 5-minute rule, waiting, making sure the patient wasn’t talking, feet on the floor. We did some training, internal training, of how to take a blood pressure properly. That went over well. All of the MAs, all the providers, medical staff, went to the training. (Facilitator interview; 2-5 clinicians; system owned; facilitation duration 3 [10-24.9 hours with a facilitator]) |
BP = blood pressure; EHR = electronic health record; FQHC = Federally Qualified Health Center; MA = medical assistant.
DISCUSSION
In this study, we identified specific operational changes linked with improving CMS smoking and BP outcomes among a subset of practices participating in the EvidenceNOW initiative. Overall, the amount of external facilitation support, practice size, and ownership were key factors that defined the settings within which specific operational changes led to meaningful outcome improvements. The important role of relatively immutable practice characteristics (eg, size and ownership) in our models was striking. We initially excluded these attributes from our analysis because we wanted to focus on modifiable factors that would help inform actionable, practical approaches and policies to help practices improve smoking and BP outcomes. Through the course of our analyses, it became clear that making operational changes alone—in certain clinical settings—was insufficient to achieve meaningful improvements. Our solutions met consistency and coverage thresholds only when we introduced practice characteristics and facilitation dose into our models as factors. Matching the appropriate improvement approach from the mix of options available to key practice characteristics is important for achieving meaningful quality improvement gains.
It was not surprising to us that initiating the change of taking a second BP and documenting this second reading in a discrete EHR field so that it is calculated as part of the CMS metric was linked to a 10-point or greater improvement among clinician-owned practices. This is a setting where clinicians and their teams have the agency and internal motivation to relatively rapidly make and implement these types of changes. In contrast, system- or hospital-owned practices can have extra bureaucracy and centralized infrastructure that may limit practice-level agency, particularly when the change involves the EHR, a systemwide tool.41 Additionally, system and hospital leaders may include this change as part of a larger package of required changes, which may complicate implementation.
For certain types of changes, successful implementation traveled hand-in-hand with external facilitation. Of the 7 combined pathways we identified across the 2 outcomes, those not linked to practice size and ownership all included a moderate to substantial dose of facilitation. Facilitators helped practices use data to identify quality gaps, fostered motivation and decision making, empowered leaders and staff to identify and implement changes, and then helped them evolve those changes if needed.42-44 The role of external facilitators may be of greater importance in hospital- and health system–owned practices, where implementing such changes is more complex and may require some prioritizing.
The operational changes linked to improved outcomes were pragmatic and unsurprising, with one possible exception. Although it may at first appear unusual45 that not tracking a referral would be associated with improved performance on the smoking outcome, on further consideration, this finding may reflect the fact that the CMS smoking outcome measure does not assess quit rates. Following up on a referral might improve patient engagement with a quitline and rates of quit attempts, but this operational change does not necessarily improve the CMS measure, and could potentially distract practices from making broader operational changes (eg, systematic screening and brief counseling) that would improve the outcome. One of the strengths of CCMs is that it can yield unexpected associations that, when explored further, could potentially result in new discoveries.
Our findings suggest that individuals leading quality improvement efforts within primary care settings can substantially improve prospects for implementation success when they consider and tailor operational expectations to the practice setting. The findings also align with the larger health services and organizational change literature: size and ownership are 2 important factors to consider when undertaking a change, as these have implications for agency, decision-making complexity, and how action and change happen.46-48
This study had a number of limitations. First, we relied on self-reported practice changes from facilitators and practice members, as observation of practice operational changes was not feasible. Second, although we know from qualitative sources that facilitators’ skills and approaches vary, and that this variation manifested both within and across Cooperatives, we did not have data at the practice level to assess the impact of these variations on outcomes. Third, although we can conclude that in this subset of practices, there was ample evidence for the solutions—and that these solutions were consistent with logic, theory, and prior knowledge—replication, experimental work, and application of additional methods would be ultimately required to establish the direction and strength of any causal relationships and generalizability. Fourth, EvidenceNOW focused on engaging smaller practices (those with no more than 10 clinicians) and our data set reflects this; we have few practices with more than 5 clinicians in our overall data set, and fewer that attained the 10-point or greater gain. This sampling limited our ability to fully examine the connection between practice size and facilitation dose. Although one might speculate that larger practices required more facilitation to align operations across a more expansive team, further research is needed to examine this important connection.
In conclusion, there was no magic bullet for improving smoking and BP outcomes across the diverse primary care practices in our analyses. Multiple combinations of operational changes led to improvements, but only in the context of practice size and ownership, or dose of external facilitation. Given this complex interplay between specific operational changes and local context, our analyses underscore the value of methods that can identify how particular factors work together to explain improvement in clinical outcomes.
Supplementary Material
Acknowledgments
The authors wish to acknowledge Sarah Ono, PhD (S.O.) and Tanisha Tate Woodson, PhD, MPH (T.T.W.) for early contributions to data collection and analysis. Jennifer Hemler, PhD, and Andrea Baron, MA, assisted with data collection. Benjamin Crabtree, PhD, Leif Solberg, MD, and Kurt Stange, MD, PhD, in various ways, made contributions important to bringing this work to fruition.
Footnotes
Conflicts of interest: authors report none.
To read or post commentaries in response to this article, go to https://www.AnnFamMed.org/content/19/3/240/tab-e-letters.
Funding support: This work was supported by the Agency for Healthcare Research and Quality (grant number R01HS023940-01).
Disclaimer: The views expressed are solely those of the authors and do not necessarily represent official views of the authors’ affiliated institutions or funder.
Previous presentations: Presented at the 11th Annual Conference on the Science of Dissemination & Implementation in Health; Washington, DC; December 3-5, 2018.
Supplemental materials: Available at https://www.AnnFamMed.org/content/19/3/240/suppl/DC1/.
References
- 1.Go AS, Mozaffarian D, Roger VL, et al. ; American Heart Association Statistics Committee and Stroke Statistics Subcommittee . Executive summary: heart disease and stroke statistics—2014 update: a report from the American Heart Association. Circulation. 2014; 129(3): 399-410. [DOI] [PubMed] [Google Scholar]
- 2.Ockene JK, Edgerton EA, Teutsch SM, et al. Integrating evidence-based clinical and community strategies to improve health. Am J Prev Med. 2007; 32(3): 244-252. [DOI] [PubMed] [Google Scholar]
- 3.Jousilahti P, Laatikainen T, Peltonen M, et al. Primary prevention and risk factor reduction in coronary heart disease mortality among working aged men and women in eastern Finland over 40 years: population based observational study. BMJ. 2016; 352: i721. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Farley TA, Dalal MA, Mostashari F, Frieden TR.. Deaths preventable in the U.S. by improvements in use of clinical preventive services. Am J Prev Med. 2010; 38(6): 600-609. [DOI] [PubMed] [Google Scholar]
- 5.Frieden TR, Berwick DM.. The “Million Hearts” initiative—preventing heart attacks and strokes. N Engl J Med. 2011; 365(13): e27. [DOI] [PubMed] [Google Scholar]
- 6.Mercado C, DeSimone AK, Odom E, Gillespie C, Ayala C, Loustalot F.. Prevalence of cholesterol treatment eligibility and medication use among adults—United States, 2005–2012. MMWR Morb Mortal Wkly Rep. 2015; 64(47): 1305-1311. [DOI] [PubMed] [Google Scholar]
- 7.Jamal A, Dube SR, Malarcher AM, Shaw L, Engstrom MC; Centers for Disease Control and Prevention (CDC) . Tobacco use screening and counseling during physician office visits among adults—National Ambulatory Medical Care Survey and National Health Interview Survey, United States, 2005-2009. MMWR Suppl. 2012; 61(2): 38-45. [PubMed] [Google Scholar]
- 8.James PA, Oparil S, Carter BL, et al. 2014 evidence-based guideline for the management of high blood pressure in adults: report from the panel members appointed to the Eighth Joint National Committee (JNC 8). JAMA. 2014; 311(5): 507-520. [DOI] [PubMed] [Google Scholar]
- 9.US Preventive Services Task Force . USPSTF A and B recommendations. In: US Preventive Services Task Force; 2010. Accessed Aug 2020. https://www.uspreventiveservicestaskforce.org/Page/Name/uspstf-a-and-b-recommendations/ [Google Scholar]
- 10.Vidrine JI, Shete S, Cao Y, et al. Ask-Advise-Connect: a new approach to smoking treatment delivery in health care settings. JAMA Intern Med. 2013; 173(6): 458-464. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Papadakis S, Cole AG, Reid RD, et al. Increasing rates of tobacco treatment delivery in primary care practice: evaluation of the Ottawa Model for Smoking Cessation. Ann Fam Med. 2016; 14(3): 235-243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Egan BM, Sutherland SE, Rakotz M, et al. Improving hypertension control in primary care with the measure accurately, act rapidly, and partner with patients protocol. Hypertension. 2018; 72(6): 1320-1327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Huebsch JA, Kottke TE, McGinnis P, et al. A qualitative study of processes used to implement evidence-based care in a primary care practice. Fam Pract. 2015; 32(5): 578-583. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Harris MF, Parker SM, Litt J, et al. ; Preventive Evidence into Practice Partnership Group . Implementing guidelines to routinely prevent chronic vascular disease in primary care: the Preventive Evidence into Practice cluster randomised controlled trial. BMJ Open. 2015; 5(12): e009397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Jaffe MG, Lee GA, Young JD, Sidney S, Go AS.. Improved blood pressure control associated with a large-scale hypertension program. JAMA. 2013; 310(7): 699-705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Nutting PA, Crabtree BF, Stewart EE, et al. Effect of facilitation on practice outcomes in the National Demonstration Project model of the patient-centered medical home. Ann Fam Med. 2010; 8(Suppl 1): S33-S44, S92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ritchie MJ, Parker LE, Edlund CN, Kirchner JE.. Using implementation facilitation to foster clinical practice quality and adherence to evidence in challenged settings: a qualitative study. BMC Health Serv Res. 2017; 17(1): 294. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Baskerville NB, Liddy C, Hogg W.. Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med. 2012; 10(1): 63-74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Grumbach K, Bainbridge E, Bodenheimer T.. Facilitating improvement in primary care: the promise of practice coaching. Issue Brief (Commonw Fund). 2012; 15(1): 1-14. [PubMed] [Google Scholar]
- 20.Ono SS, Crabtree BF, Hemler JR, et al. Taking innovation to scale in primary care practices: the functions of health care extension. Health Aff (Millwood). 2018; 37(2): 222-230. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Weiner BJ, Pignone MP, DuBard CA, et al. Advancing heart health in North Carolina primary care: the Heart Health NOW study protocol. Implement Sci. 2015; 10(1): 160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Shelley DR, Ogedegbe G, Anane S, et al. Testing the use of practice facilitation in a cluster randomized stepped-wedge design trial to improve adherence to cardiovascular disease prevention guidelines: HealthyHearts NYC. Implement Sci. 2016; 11(1): 88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Parchman ML, Fagnan LJ, Dorr DA, et al. Study protocol for “Healthy Hearts Northwest”: a 2 × 2 randomized factorial trial to build quality improvement capacity in primary care. Implement Sci. 2016; 11(1): 138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Perry CK, Damschroder LJ, Hemler JR, Woodson TT, Ono SS, Cohen DJ.. Specifying and comparing implementation strategies across seven large implementation interventions: a practical application of theory. Implement Sci. 2019; 14(1): 32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Cohen DJ, Balasubramanian BA, Gordon L, et al. A national evaluation of a dissemination and implementation initiative to enhance primary care practice capacity and improve cardiovascular disease care: the ESCALATES study protocol. Implement Sci. 2016; 11(1): 86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Baumgartner M, Falk C.. Boolean difference-making: a modern regularity theory of causation. Br J Philos Sci. 2019. Accessed Sep 2020. 10.1093/bjps/axz047 [DOI]
- 27.Whitaker RG, Sperber N, Baumgartner M, et al. Coincidence analysis: a new method for causal inference in implementation science. Implement Sci. 2020; 15(1): 108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Rohlfing I, Zuber CI.. Check your truth conditions! Clarifying the relationship between theories of causation and social science methods for causal inference. Sociol Methods Res. 2019. Accessed Sep 2020. https://journals.sagepub.com/doi/abs/10.1177/0049124119826156
- 29.Cragun D. Configurational comparative methods. In: Nilsen P, Birken SA, eds. Handbook on Implementation Science. Edward Elgar Publishing; 2020: 497-504. [Google Scholar]
- 30.Petrik AF, Green B, Schneider J, et al. Factors influencing implementation of a colorectal cancer screening improvement program in community health centers: an applied use of Configurational Comparative Methods. J Gen Intern Med. 2020; 35(Suppl 2): 815-822. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Cohen D, McDaniel RRJ Jr, Crabtree BF, et al. A practice change model for quality improvement in primary care practice. J Healthc Manag. 2004; 49(3): 155-168, discussion 169-170. [PubMed] [Google Scholar]
- 32.Coury J, Miech EJ, Styer P, et al. What’s the “secret sauce”? How implementation variation affects the success of colorectal cancer screening outreach. Implement Sci Commun. 2021; 11; 2(1): 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Palinkas LA, Mendon SJ, Hamilton AB.. Innovations in mixed methods evaluations. Annu Rev Public Health. 2019; 40: 423-442. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Hudon C, Chouinard M-C, Pluye P, et al. Characteristics of case management in primary care associated with positive outcomes for frequent users of health care: a systematic review. Ann Fam Med. 2019; 17(5): 448-458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Cohen D, Crabtree B.. Maximum variation sampling. Qualitative Research Guidelines Project. Published 2006. Accessed Jan 26, 2021. https://www.qualres.org/HomeMaxi-3803.html
- 36.Ambuehl M, Baumgartner M.. CNA: Causal modeling with coincidence analysis. R package version 2.0.0. 2017. Accessed Sep 2020. https://rdrr.io/cran/cna/
- 37.RStudio team . RStudio: integrated development for R. Published 2015. Accessed Sep 2020. https://www.rstudio.com
- 38.R Core Team . R: A language and environment for statistical computing. Published February 10, 2015. Accessed Sep 2020. https://www.gbif.org/tool/81287/r-a-language-and-environment-for-statistical-computing
- 39.Yakovchenko V, Miech EJ, Chinman MJ, et al. Strategy configurations directly linked to higher hepatitis C virus treatment starts: an applied use of Configurational Comparative Methods. Med Care. 2020; 58(5): e31-e38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Hickman SE, Miech EJ, Stump TE, Fowler NR, Unroe KT.. Identifying the implementation conditions associated with positive outcomes in a successful nursing facility demonstration project. Gerontologist. 2020; 60(8): 1566-1574. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Bishop TF, Shortell SM, Ramsay PP, Copeland KR, Casalino LP.. Trends in hospital ownership of physician practices and the effect on processes to improve quality. Am J Manag Care. 2016; 22(3): 172-176. [PMC free article] [PubMed] [Google Scholar]
- 42.Taylor EF, Genevro J, Peikes D, Geonnotti K, Wang W, Meyers D.. Building quality improvement capacity in primary care: supports and resources. Agency for Healthcare Research and Quality. Published 2013. Accessed Dec 2018. https://www.ahrq.gov/sites/default/files/publications/files/pcmhqi2.pdf [Google Scholar]
- 43.Taylor EF, Machta RM, Meyers DS, Genevro J, Peikes DN.. Enhancing the primary care team to provide redesigned care: the roles of practice facilitators and care managers. Ann Fam Med. 2013; 11(1): 80-83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Hemler JR, Hall JD, Cholan RA, et al. Practice facilitator strategies for addressing electronic health record data challenges for quality improvement: EvidenceNOW. J Am Board Fam Med. 2018; 31(3): 398-409. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Joseph AM, Fu SS, Lindgren B, et al. Chronic disease management for tobacco dependence: a randomized, controlled trial. Arch Intern Med. 2011; 171(21): 1894-1900. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Liaw WR, Jetty A, Petterson SM, Peterson LE, Bazemore AW.. Solo and small practices: a vital, diverse part of primary care. Ann Fam Med. 2016; 14(1): 8-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Shortell SM, Schmittdiel J, Wang MC, et al. An empirical assessment of high-performing medical groups: results from a national study. Med Care Res Rev. 2005; 62(4): 407-434. [DOI] [PubMed] [Google Scholar]
- 48.Hurley VB, Rodriguez HP, Shortell SM.. The role of accountable care organization affiliation and ownership in promoting physician practice participation in quality improvement collaboratives. Health Care Manage Rev. 2019; 44(2): 174-182. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.