Readiness and Implementation of Quality Improvement Strategies Among Small- and Medium-Sized Primary Care Practices: an Observational Study

Abstract

Background

Little is known about what determines strategy implementation around quality improvement (QI) in small- and medium-sized practices. Key questions are whether QI strategies are associated with practice readiness and practice characteristics.

Objective

Grounded in organizational readiness theory, we examined how readiness and practice characteristics affect QI strategy implementation. The study was a component of a larger practice-level intervention, Heart of Virginia Healthcare, which sought to transform primary care while improving cardiovascular care.

Design

This observational study analyzed practice correlates of QI strategy implementation in primary care at 3 and 12 months. Data were derived from surveys completed by clinicians and staff and from assessments by practice coaches.

Participants

A total of 175 small- and medium-sized primary care practices were included.

Main Measures

Outcome was QI strategy implementation in three domains: (1) aspirin, blood pressure, cholesterol, and smoking cessation (ABCS); (2) care coordination; and (3) organizational-level improvement. Coaches assessed implementation at 3 and 12 months. Readiness was measured by baseline member surveys, 1831 responses from 175 practices, a response rate of 73%. Practice survey assessed practice characteristics, a response rate of 93%. We used multivariate regression.

Key Results

QI strategy implementation increased from 3 to 12 months: the mean for ABCS from 1.20 to 1.59, care coordination from 2.15 to 2.75, organizational improvement from 1.37 to 1.78 (95% CI). There was no statistically significant association between readiness and QI strategy implementation across domains. Independent practice implementation was statistically significantly higher than hospital-owned practices at 3 months for ABCS (95% CI, P = 0.01) and care coordination (95% CI, P = 0.03), and at 12 months for care coordination (95% CI, P = 0.04).

Conclusion

QI strategy implementation varies by practice ownership. Independent practices focus on patient care–related activities. FQHCs may need additional time to adopt and implement QI activities. Practice readiness may require more structural and organizational changes before starting a QI effort.

INTRODUCTION

Quality improvement (QI) strategies play an essential role in transforming primary care practices to improve population health, enhance patient experiences and outcomes, reduce costs, and improve provider experience.^{1, 2} The main focus of QI strategies is to implement evidence-based clinical care ³ and to transform practices to improve organizational efficiency ^{4, 5} and patient experience. ⁶ Among the strategies implemented by primary care practices to improve quality are the patient-centered medical home model, ^7–9 evidence-based standards of care, ¹⁰ patient portals, electronic reminder systems, ¹¹ team-based care models, ¹² and integrated behavioral health care. ¹³ Our study drew upon organizational readiness theory to explore factors associated with specific primary care strategic activities around QI.

Organizational readiness refers to a construct by which an organization’s members feel committed to a change and a shared belief in their ability to implement that change. ^{14, 15} When organizational readiness is high, team members are very likely to initiate change. ¹⁴ Contextual factors such as culture, organizational resources and structures, and policies and procedures also affect readiness for change. ¹⁵ Previous research has established a positive connection between an organization’s readiness and its change outcomes. ^16–18 Rubenstein et al.¹⁷ found a significant association between practice readiness and practices’ commitment to QI capability in predicting leaders’ commitment to change. Another study suggested that primary care practices that were ready to implement evidence-based care could do so if they were able to remove key barriers. ¹⁸ Further, providers perceive organizational readiness to change as challenging when there is a lack of time and resources, staff resistance to change, and concern around project sustainability. ¹⁹ Although readiness theory provides a compelling conceptual framework, few studies have measured readiness empirically in order to explain variation in QI activity across primary care practices.

In this study, we hypothesized that perceived organizational readiness by practice members, both clinicians and non-clinical staff, would predict QI strategy implementation among primary care practices. ^{14, 15, 19} Based on the previous research findings, we further hypothesized that practice characteristics such as ownership, size, location, and specialty type, ^{20, 21} as well as whether the practice was recognized as a patient-centered medical home (PCMH) or part of an accountable care organization (ACO),^{20, 21} would play a key role in predicting QI strategy implementation. Our study is the first to explore the relationship between QI strategy implementation, practice readiness, and practice characteristics in primary care. A better understanding of factors related to QI strategy implementation can help policymakers, large healthcare systems, and primary care practices target future efforts around QI and address potential preconditions for such efforts.

METHODS

Setting

This study was part of a national project, EvidenceNOW, which was supported by the Agency for Healthcare Research and Quality (AHRQ). ²² The objective of EvidenceNOW was to disseminate and implement evidence-based guidelines in cardiovascular care in primary care settings. The Heart of Virginia Healthcare (HVH) project was one of seven national collaboratives aimed at transforming small- and medium-sized primary care practices to improve population heart health. ²² Based on previous evidence related to the success of practice coaching, ²³ practices that enrolled in the HVH received personalized coaching and consultation support from experts who had experienced working with primary care for improvement. ²⁴ The intensive intervention phase was 3 months, followed by a maintenance phase of 9 additional months. Practice coaches worked with primary care practices to identify meaningful changes in cardiovascular disease care to improve patient outcomes. ²⁵

Recruitment

The HVH intervention recruited 203 small- to medium-sized primary care practices as described elsewhere. ²⁰ Participating small- to medium-sized practices had up to 10 providers, including physicians, nurse practitioners, and physician assistants. ²¹ Practices were categorized as independent (not owned by a hospital system), hospital-owned, or federally qualified health centers (FQHC).²⁰ Our study was approved by George Mason University’s Institutional Review Board (IRB) in 2016.

Data Source

Data were collected as part of the HVH study from 2016 to 2018 and arose from three sources. First, QI strategy implementation was assessed by coaches at the end of the 3- and 12-month intervention periods using a standardized instrument developed for this purpose. Second, self-assessed organizational readiness was assessed by member surveys at baseline, near the start of the HVH intervention, and was completed by practice members, including physicians, practice managers, nurse practitioners, physician assistants, registered nurses, and non-clinical staff. We used responses on total practice size to calculate the individual-level survey response rate. A total of 2714 members received the survey. We included 1831 complete responses on all readiness items from 175 practices, yielding a response rate of 73%. Each participant received $150 compensation upon the completion of the survey. Third, a second practice-level survey was completed by a lead clinician or practice manager at baseline to measure practice characteristics. The response rate for the practice survey was 93 %. Participating practices received $500 compensation upon completion of the survey. The practice member survey and practice survey were both administered online with paper follow-up.

Measures

Our outcome variable was the level of practice QI strategy implementation in three domains. We asked coaches to assess each practice’s QI strategy implementation across 15 different strategy items. We recoded the responses as whether the practice 1 = “has implemented the QI strategy” or 0 = “has not implemented the QI strategy” and summed the strategic activities by three domains: (1) aspirin, blood pressure, cholesterol, smoking cessation (ABCS) preparation; (2) care coordination; and (3) organizational improvement. Before aggregating, we measured the internal consistency of each domain with Cronbach’s alpha which was 0.91 for ABCS preparations, 0.94 for care coordination, and 0.92 for organizational improvement. Pearson product-moment correlations also showed strong positive correlations between each domain (ranging from r = .49 to r = .87). The differences were statistically significant when comparing the average of ABCS with that of care coordination and of organizational improvement (P = 0.001). QI strategy implementation was missing for 10 practices at 12 months.

Our key independent variable was readiness. The practice member survey assessed readiness using an instrument adapted from Weiner et al.²⁶ The 11-item instrument measured self-reported practice readiness to adopt the HVH intervention with a 5-point Likert scale (5 = “strongly agree” to 1 = “strongly disagree”). We first calculated the readiness score by taking the average of the individual responses at the practice level. Then, we calculated the overall composite score and included it in our model as a continuous variable.

Our control variables consisted of practice-level characteristics, including practice ownership (independent practice, hospital-owned practice, or federally qualified health center (FQHC)); a categorical measure of the practice size (2–5 clinicians, 6–10 clinicians, and 11 or more clinicians); practice specialty-mix (single specialty or multispecialty); and whether the practice was located in a medically underserved area. ^{20, 21, 27} We also included characteristics of the patient population in the practice including the percentage of patients with Medicaid coverage or who were uninsured ²⁰ and the percentage of patients who were age 60 and above. Further, we included indicators of whether the practice was a designated PCMH; ^{20, 21, 27} whether the practice was part of an ACO; ²¹ and whether it participated in the Million Hearts project. ²⁸ We included the Million Hearts project in our study because its main objective of achieving population cardiovascular health was aligned with the HVH intervention. ²⁸ We chose these control variables in our analysis based on prior research. ^{20, 21, 27}

Statistical Analysis

Data on QI strategy implementation, readiness, and practice characteristics were measured at the practice level and all analyses were conducted using STATA software. ²⁹ We calculated descriptive statistics of key measures (Table 1) and used multivariate linear regression analysis to assess the relationship between the level of QI strategy implementation and readiness and practice characteristics (Table 2). Each outcome variable was treated as a continuous variable in our regression and we ran separate regression models for each of the three QI strategy domains. We conducted additional analysis to examine the influence of potentially competing programs such as the Million Hearts initiative, PCMH, and ACO by excluding these variables from the regression models and testing for mediation.

Table 1.

Summary Statistics and Practice Sample Characteristics (N = 175)

	Conclusion of the HVH intervention period
QI strategies	Mean (SD)
Implemented, 3 months, mean % (95% CI)
ABCS preparations	1.20 (1.3)
Care coordination	2.15 (2.4)
Organizational improvement	1.37 (1.7)
Implemented*, 12 months, mean % (95% CI)
ABCS preparations	1.59 (1.3)
Care coordination	2.75 (2.5)
Organizational improvement	1.78 (1.9)
Readiness, mean % (95% CI)	3.83 (0.3)
	%
Practice ownership, mean % (95% CI)
Independent	31
Hospital-owned	56
Federally qualified health center	13
Practice size, number of clinicians, mean % (95% CI)
2–5 clinicians	70
6–10 clinicians	16
11 and above clinicians	14
Practice specialty mix, mean % (95% CI)
Single specialty practice	70
Multispecialty practice	30
Medically underserved area, mean % (95% CI)	64
Recognized as a patient-centered medical home, mean % (95% CI)	52
Part of an accountable care organization, mean % (95% CI)	67
Participation in the Million Hearts Initiative, mean % (95% CI)	10
Percentage of patients with Medicaid/uninsured, mean % (95% CI)	19
Percentage of patients aged 60 and above, mean % (95% CI)	31

Summary statistics display the number of observations for “QI Strategy Implementation” (95% CI) at 3-month and 12-month intervention. QI strategy implementation was measured by a recoded scale from 1 to 0: 1 = “practice has implemented the QI strategy,” 0 = “practice has not implemented the QI strategy”

QI strategies included three domains: (1) ABCS (aspirin, blood pressure, cholesterol, smoking cessation) preparations (identifying patients, define measures, adopt guidelines); (2) care coordination (engaging care team, encourage access to care, develop care plans, optimize visits, manage medication, self-management, and coordinate care); (3) organizational improvement (optimizing teamwork, optimizing workflow, optimizing clinical data, optimizing financial picture, and organizational learning)

Readiness and practice characteristics were assessed at baseline (N = 175)

*QI strategy implementation was missing for 10 practices at 12 months (N = 165)

Table 2.

Association Between QI Strategy Implementation and Readiness and Practice Characteristics

QI strategy implementation\
	Conclusion of the 3-month intervention (N = 175)			Conclusion of the 12-month intervention (N = 165)\\
	ABCS preparations†	Care coordination‡	Organizational improvement§	ABCS preparations	Care coordination	Organizational improvement
Readiness	0.27 (0.81)	0.24 (0.40)	0.66 (1.48)	0.13 (0.39)	0.26 (0.41)	0.02 (0.04)
Practice ownership
Independent	0.62 (2.59)*	0.96 (2.10)*	0.60 (1.81)	− 0.07 (− 0.26)	0.96 (2.00)*	0.62 (1.63)
Hospital	Reference	Reference	Reference	Reference	Reference	Reference
FQHC	− 0.21 (− 0.51)	− 0.64 (− 1.01)	− 0.27 (− 0.60)	0.53 (1.65)	0.44 (0.78)	0.33 (0.70)
Practice size (number of clinicians)	0.04 (0.28)	− 0.06 (− 0.22)	0.48 (2.87)**	− 0.03 (− 0.26)	0.06 (0.29)	0.10 (0.62)
Practice specialty mix (single or multispecialty)	0.28 (1.14)	0.58 (1.27)	0.14 (0.42)	0.24 (1.00)	− 0.03 (− 0.07)	− 0.20 (− 0.54)
Medically underserved area	− 0.03 (− 0.15)	0.05 (0.13)	− 0.14 (− 0.48)	0.32 (1.39)	0.59 (1.35)	0.39 (1.18)
Patient-centered medical home designation	0.81 (2.15)	0.10 (0.27)*	0.36 (1.28)	0.08 (0.33)	0.22 (0.49)	0.11 (0.32)
Part of an accountable care organization	− 0.96 (− 2.21)	0.40 (1.02)*	− 0.23 (− 0.67)	− 0.05 (− 0.22)	0.18 (0.42)	0.40 (1.20)
Participation in the Million Hearts Initiative	0.05 (0.06)	− 1.07 (− 1.92)	0.06 (0.11)	− 0.41 (− 1.09)	− 0.34 (− 0.46)	− 0.00 (− 0.01)
Percentage of patients with Medicaid/uninsured	0.00 (0.35)	− 0.01 (− 0.42)	− 0.00 (− 0.30)	− 0.01 (− 0.95)	− 0.00 (− 0.93)	− 0.00 (− 0.49)
Percentage of patients aged 60 and above	− 0.01 (− 1.15)	0.01 (1.25)	0.00 (0.02)	− 0.01 (− 0.29)	0.00 (0.47)	− 0.00 (− 0.67)

Each column under the specified QI strategy domain indicates coefficients and t statistics (in parenthesis)

^\QI strategy: 1 = “practice implemented the QI strategy,” 0 = “practice has not implemented the QI strategy”

^†ABCS preparation includes identifying patients, defining measures, adopting guidelines

^‡Care coordination includes engaging care team, encouraging access to care, developing care plans, optimizing visits, managing medication, self-management, and coordinating care

^§Organizational improvement includes optimizing teamwork, optimizing workflow, optimizing clinical data, optimizing financial picture, and organizational learning

^\\QI strategy implementation was missing 10 practices at 12 months

FQHC Federally Qualified Health Center

*P < 0.05; **P < 0.01

RESULTS

Study Population

Among the 203 small- and medium-sized primary care practices participating in the HVH, 175 practices that responded to the readiness questionnaires were included in our analysis.

Descriptive Statistics

Table 1 displays the summary statistics and characteristics of our sample. QI strategy implementation increased across the three domains from 3 to 12 months. The mean score for the ABCS strategy implementation increased by 33%, from 1.20 to 1.59 (95% CI); for care coordination by 28%, from 2.15 to 2.75 (95% CI); and for organizational improvement by 30%, from 1.37 to 1.78 (95% CI). The mean for readiness was 3.83 (95% CI).

Of the 175 practices, 54 were independent practices (31%), 98 were hospital-owned practices (56%), and 23 were FQHCs (13%). Seventy percent of the practices had 5 or fewer clinicians. Seventy percent of the practices were single specialty, and 30% were multispecialty practice. Sixty-four percent of the participating practices were located in medically underserved areas. About half of the practices (52%) had PCMH recognition and 67% of the practices were affiliated with an ACO. Ten percent of practices participated in the Million Hearts project. The average practice had 19% of its patient population with either Medicaid coverage or uninsured and 31% of patients were 60 years or older.

Figure 1 presents the level of practice QI strategy implementation at the end of the 3- and 12-month intervention periods. There was an increase in QI strategy implementation across the domains, and the ABCS-related activities showed the greatest gains from 3 to 12 months.

Fig. 1.

Heart of Virginia Healthcare (HVH) QI strategy implementation. The graph provides information on QI strategy implementation by percentages at 3-month and 12-month intervention (N = 165). QI strategies included 15 QI strategy items by three domains (ABCS preparations, care coordination, and organizational improvement). Each strategy item was assessed by practice coaches on a scale from 1 to 0: 1 = “practice has implemented the QI strategy,” and 0 = “practice has not implemented the QI strategy.” The differences between the strategy domains were statistically significant when comparing the average of ABCS with that of care coordination and of organizational improvement (P = 0.001).

Figure 2 displays practice readiness by item. The mean overall readiness composite score was 3.83.

Fig. 2.

Practice readiness (N = 175). Adapted from Weiner et al.²⁶ Self-reported 5-point Likert scale (5 = strongly agree, and 1 = strongly disagree). The graph displays an aggregated mean readiness score for each readiness item across 11 questions. The overall aggregated mean readiness score was 3.83 (SD = 0.3). The survey was conducted at baseline. The response rate was 73%.

Among the readiness components, the items endorsed with the highest scores included the following: (1) “We can support providers as they adjust to the changes” (mean score = 3.97); (2) “We want to implement these changes” (mean score = 3.91); and (3) “We are committed to making these changes” (mean score = 3.87). The lowest three mean scores included: “We can manage the politics of these changes” (mean score = 3.72); “We can keep track of progress in implementing these changes” (mean score = 3.78); and “We will do whatever it takes to implement these changes” (mean score = 3.79).

Association Between QI Strategy Implementation, Readiness, and Practice Characteristics

We conducted multivariate linear regression analysis to examine the association between QI strategy implementation and readiness. There was no statistically significant association between QI strategy implementation and readiness across the three domains (Table 2). In contrast, ownership was associated with QI implementation. Independent status, compared with hospital-owned status, was a strong predictor of higher strategy implementation, and the association was statistically significant for ABCS (95% CI, P = 0.01) and for care coordination (95% CI, P = 0.03) at 3 months. At 12 months, independent ownership continued to be statistically significantly associated with care coordination only (95% CI, P = 0.04). At 3 months, FQHC status was associated with lower implementation relative to hospital-owned practices across all three outcomes, and none was significant: ABCS implementation (95% CI, P = 0.61), care coordination (95% CI, P = 0.31), and organizational improvement (95% CI, P = 0.55). There was no effect of the FQHC status at 12 months.

Practice size was a significant predictor of a higher QI strategy implementation for organizational improvement strategies (95% CI, P = 0.005) at 3 months. Practice recognition as a PCMH was statistically significantly associated with higher strategy implementation for care coordination (95% CI, P = 0.03) at 3 months, but not at 12 months. Practice affiliation with an ACO was also statistically significantly associated with higher strategy implementation for care coordination (95% CI, P = 0.03) at 3 months. Finally, we ran an additional analysis for a possible mediating effect of PCMH and ACO affiliation and the results were unchanged.

DISCUSSIONS

Our findings provide insight into drivers of quality improvement among small- and medium-sized primary care practices. We found practice ownership plays a role in QI strategy implementation efforts, which was in line with previous research. ^{20, 21} Independent practices demonstrated a higher level of QI strategy implementation than those that were hospital-owned in the short run and sustained quality improvement strategies with a focus on patient care–related strategies at 1 year.

We also found FQHC practices had a lower level of QI strategy implementation in the short run. ²⁰ This finding may reflect the financial challenges that FQHCs face and the complexity of the patients they serve. Further, over a longer time period, FQHCs increased the level of QI strategy implementation indicating that they may need additional time to build up their ability to adopt and implement these strategies. Overall study findings suggest that independently owned practices, health system–owned practices, and FQHCs might be focusing on different types of quality improvement activities.

Practice members perceived organizational readiness as relatively high, although they reported a lower ability to keep track of change and manage politics associated with change. Overall, even though we found practice readiness was positively associated with QI implementation at 3 months and 12 months, its effect was not statistically significant. Our hypothesis that greater perceived motivation and commitment to change would predict QI strategy implementation was not supported. The findings indicate that practice readiness is perhaps established over time and more structural changes at organizational level might be necessary before starting a QI effort. Our findings also suggest that practice leaders should focus on building employee confidence towards change and provide tools to monitor quality improvement activities.

We found a practice recognition as a PCMH significantly predicted a higher QI strategy implementation for care coordination–related activities. The finding was in line with previous research and consistent with the main objectives and the requirements of being a PCMH model. ^6–9 This reflects that the practice was able to make the necessary changes and has an organizational structure in place for ongoing quality improvement. We also found practice affiliation with an ACO was a significant predictor of a higher implementation for care coordination–related strategies, which was consistent with previous research. ²¹ Finally, our study finding of practice size association with a higher implementation suggests that QI strategy implementation varies by practice size for organizational improvement–related activities.

Our study has several limitations. First, it might be possible that small practices already doing well may have been more likely to participate in the intervention. Second, this study analyzed self-reported readiness by practice staff, and responses to the questions may reflect desirability bias, although we note the significant variation in this measure across practices. Third, there is no standardization of coach qualifications; ²⁵ thus, variation in coaches’ knowledge, experience, and skillsets might have biased the assessment of the practice QI strategic activity. Finally, the study was based on nearly 200 primary care practices in Virginia willing to participate in the Heart of Virginia Healthcare, but results may not generalize to other geographic areas or practices.

To the best of our knowledge, this study is the first that examined the relationship between QI strategy implementation and readiness and practice characteristics. The study’s findings highlight the importance of QI implementation variation in small- to medium-sized primary care practice characteristics with respect to QI efforts. Our findings suggest that FQHC practices may need additional time and support for adapting and implementing QI strategies. There might be a need for more structural changes and resources in quality improvement and transformation efforts for maintaining practice readiness. QI authorities should consider these characteristics and structural needs before designing a quality improvement intervention for an ongoing quality enhancement.

Authors Contribution

There is no other person who contributed to the manuscript that did not meet the criteria for authorship.

Funding Information

This study was supported by a grant from the Agency for Healthcare Research and Quality, R18: HS023913, Anton J. Kuzel, principal investigator.

Compliance with Ethical Standards

Our study was approved by George Mason University’s Institutional Review Board (IRB) in 2016.

Conflict of Interest

The authors declare that they do not have a conflict of interest.