Associations Between Organizational Characteristics and Quality Improvement Activities of Clinics Participating in a Quality Improvement Collaborative

Abstract

Background

Few studies have rigorously evaluated the associations between organizational characteristics and intervention activities of health care organizations participating in quality improvement collaboratives (QICs).

Objective

To examine the relationship between clinic characteristics and intervention activities by primary care clinics that provide HIV care and that participated in a QIC.

Design

Cross-sectional study of Ryan White CARE Act (now called Ryan White HIV/AIDS Treatment Modernization Act) funded clinics that participated in a QIC over 16 months in 2000 and 2001. The QIC was originally planned to be a more typical 12 months long, but was extended to increase the likelihood of success. Data were collected using surveys of clinicians and administrators in participating clinics and monthly reports of clinic improvement activities.

Measures

Number of interventions attempted, percent of interventions repeated, percent of interventions evaluated, and organizational characteristics.

Results

Clinics varied significantly in their intervention choices. Organizations with a more open culture and a greater emphasis on quality improvement attempted more interventions (P < 0.01, P < 0.05) and interventions that were more comprehensive (P < 0.01, P < 0.10). Presence of multidisciplinary teams and measurement of progress toward quantifiable goals also were associated with comprehensiveness of interventions (P < 0.01, P < 0.05).

Conclusion

Clinic characteristics predicted intervention activities during a QIC. Further research is needed on how these organizational characteristics affect quality of care through their influence on intervention activities.

Problems with the quality of health care for chronic conditions^1–4 have stimulated interest in quality improvement (QI) methods.^5–6 One widely used QI method is the quality improvement collaborative (QIC). A fundamental premise of this approach is that most quality problems are due to short-comings in the organization and management of health care delivery systems rather than individuals.^7–11 Some QICs focused on chronic conditions have used the Chronic Care Model (CCM) to guide their activities.^12–24 The CCM describes 6 fundamental areas that need to be focused on to improve chronic care processes and outcomes: the delivery system, patient self-management, decision support, information systems, community linkages, and health system support.

Despite the popularity of QICs, the evidence regarding their impact on care quality is mixed.^14–20 Some organizations participating in a QIC achieve dramatic improvement while others achieve modest or no improvement.^25,26 The factors affecting decisions that organizations make regarding improvement activities during QICs need to be understood better.²⁷ Recent evaluations of QICs have found significant heterogeneity in the QI activities of organizations,^15,18,20,21 but have not investigated how these activities are associated with organizational characteristics. This association is strong in other QI contexts.^28–33

We used data from a national QIC focused on HIV care at Ryan White Clinics¹⁵ to assess the extent to which clinic characteristics, such as QI focus and openness of organizational culture, are associated with the number and comprehensiveness of interventions adopted by participating clinics.

METHODS

Data

The data for this study were collected by the Evaluation of Quality Improvement for HIV Care project,¹⁵ a national evaluation of a QIC involving 62 HIV clinics that received funding under Title III of the Ryan White CARE Act. The collaborative was conducted over 16 months between 2000 and 2001 by the Institute of Healthcare Improvement.

Data on clinic characteristics were collected using screening questionnaires that were administered when clinics were recruited for the QIC and responses to surveys of administrators and randomly selected clinicians administered (response rate of 89%) at the beginning of the QIC in 2000. Data on interventions attempted during the QIC were collected from monthly reports submitted by the participating clinics.

Of the 54 clinics that provided data to the Evaluation of Quality Improvement for HIV Care project, we analyzed data from 41 clinics, which provided intervention data and responses to the clinician and administrator surveys. We had survey data from 119 clinicians (an average of 2.8 per clinic) and 41 administrators.

Measures

Number of Attempted Interventions

As part of the QIC, each clinic submitted monthly reports describing the QI interventions attempted. Data corresponding to each intervention reported in each monthly report were coded on a separate data entry form. Each intervention was categorized under one or more of the 6 CCM components (eg, implementing a patient reminder system under Delivery System Redesign, implementing computerized reminders for providers under Decision Support). We used these data to calculate the number of interventions attempted as a measure of the intensity of implementation or overall activity level during the QIC.^18,20,21

Comprehensiveness of Attempted Interventions

Wagner et al have argued that chronic care improvement requires attention to all 6 components of the CCM.²³ Other studies have assessed comprehensiveness by counting the number of CCM components addressed in the interventions²¹ or by using the number and percentage of interventions attempted for each CCM component.²⁰ For this study, we developed indices that measure breadth (whether the clinics attempted interventions in a narrow or a broad subset of the CCM components) and depth (the extent to which individual interventions, on average, addressed multiple CCM components simultaneously), and comprehensiveness (breadth and depth combined) after controlling for the total number of attempted interventions (Appendix A, http://links.lww.com/MLR/A40).

Organizational Culture

Previous articles in both the organizational and health services literatures suggest that leadership commitment to QI^21,22,31,32 and a supportive organizational and interpersonal climate that encourages employees to engage in experimentation^{34– 36} are important organizational characteristics associated with successful QI efforts. To measure these aspects of clinics, we used responses to 9 questions in the clinician surveys about the leadership and staff attitudes towards implementing QI initiatives that were developed for this study. We averaged the responses of clinicians within each clinic; exploratory factor analyses on the aggregate responses identified 2 factors that we labeled “Organization QI focus” and “Openness of organization culture” (Appendix B, http://links.lww.com/MLR/A40).

Other Organizational Characteristics

We used responses from administrator and clinician surveys that were developed for this study to construct dichotomous variables for 2 organizational characteristics considered important in QI: regular use of multidisciplinary teams (including nonmedical staff)^24,33,37,38 and regular measurement of progress toward quantifiable goals,²⁵ respectively. Screening questionnaires were used to collect data on clinic type (community-based organization, community health center, university medical center, public health clinic or part of a larger multispecialty hospital), whether the clinic was a specialty clinic focusing on HIV or a general practice, number of HIV patients, and region (South, West, Northeast, and Midwest).

Analyses

We estimated a count regression model to identify significant predictors of the number of interventions attempted by each clinic. Preliminary descriptive analyses indicated overdispersion of the dependent variable, so we used a negative binomial model.³⁹ We estimated linear regression models to identify the predictors of the breadth, depth, and comprehensiveness of the interventions, defined earlier. The independent variables used in these models were QI focus, openness in organizational culture, measurement of quantifiable goals, and presence of multidisciplinary teams. In these models we controlled for the size of clinics, specialty, organization type, and region. Independent variables with a P value of less than or equal to 0.05 were deemed to be statistically significant.

RESULTS

The study sites were similar to all CARE Act sites with 2 exceptions (Table 1). The study sites were more likely to be in the South and less likely to be in the Northeast and West, and were less likely to be community health centers.¹⁵

TABLE 1.

Characteristics all CARE Act Clinics and Study Clinics

Variable	All CARE Act Clinics	Study Clinics
Region, %
Northeast	39.8	21.4
South	27.7	35.7
Midwest	15.0	16.7
West	17.5	7.1
Organization Type, %
Community Health Center	38.9	30.9
Hospital	11.1	11.9
Others	50.0	57.2
No. HIV-infected patients (SD)	623 (733)	682 (758)
Clinic size
Large (>400 patients), %	51.0	50.0
HIV specialty clinic
Yes, %	74.3	64.3

A total of 466 monthly activity reports were completed by the 41 study clinics over the 16-month study period (mean, 11.1 reports per clinic). Most (76.2%) of the sites submitted activity reports for at least 8 of the 16 months of the collaborative. On average, clinics attempted 34.7 unique interventions (median 34.5, range, 1–77) (Table 2). The mean breadth index was 0.87 (maximum = 1) suggesting that many clinics attempted interventions for multiple CCM components. The mean depth index was 0.25 (maximum = 1) indicating that few interventions across clinics spanned all the CCM components. Many interventions were special onetime events such as an HIV testing day or a seminar on chronic diseases at a community meeting; on average clinics repeated only 24.8% of the interventions (standard deviation 12.8%, range, 0%–54.2%) at least once. Clinics evaluated only 16.7% of the attempted interventions (standard deviation 12.5%, range, 0%–58.3%).

TABLE 2.

Descriptive Statistics for Intervention Choices

Variable	Mean	Std. Dev.
No. unique interventions	34.7	18.6
Percent of repeated interventions	24.8%	13%
Percent of evaluated interventions	16.7%	12.5%
Breadth index across CCM components (B)*	0.87	0.15
Depth index across CCM components (D)†	0.25	0.02
Comprehensiveness index across CCM components (C)‡	0.56	0.08

^*B = 1 indicates the extent to which the clinic focused on all CCM components equally.

^†D = 1, the extent to which the interventions spanned all the CCM components.

^‡C = 1/2 × B + 1/2 × D.

Predictors of Number of Interventions

As hypothesized, the number of interventions attempted by a clinic was significantly associated with openness in the organizational culture (P ≤ 0.01) and QI focus (P ≤ 0.05) (Table 3). Community-based organizations, community health centers, and hospitals attempted a combined average of 0.5 fewer interventions than university medical centers (P ≤ 0.05, P ≤ 0.01, P ≤ 0.05, respectively). The number of interventions was not significantly associated with the presence of multidisciplinary teams, regular measurement of quantifiable goals, or the number of active HIV patients in the clinic.

TABLE 3.

Predictors of Number of Interventions

Dependent Variable	No. Interventions*
Intercept	4.15†
Predictor variables
QI focus	0.29‡
Openness in organizational culture	0.37†
Measuring quantifiable goals	0.26
Multidisciplinary team	0.16
Control variables
Large clinics (>400 patients)	−0.03
Specialty site	−0.02
Organization type: university medical center	0.00
Organization type: community-based organization	−0.51‡
Organization type: community health center	−0.69§
Organization type: public health clinic	0.04
Organization type: hospital	−0.61‡
Region: West	0.00
Region: Northeast	−0.68
Region: South	−0.67
Region: Midwest	−0.56

^*Negative binomial model, N = 41, Log likelihood = 3960, and Dispersion = 0.18.

^†P ≤ 0.001.

^‡P ≤ 0.05.

^§P ≤ 0.01.

Predictors of Comprehensiveness of Interventions

Clinics with a more open organizational culture (P ≤ 0.01), clinics with multidisciplinary teams (P ≤ 0.01), and clinics that measured quantifiable goals more regularly (P ≤ 0.05) were more likely to implement broader, deeper, and more comprehensive interventions than other clinics (Table 4). We obtained similar results when we used different weights for the breadth and depth indices (data not shown) and in models predicting only the breadth and the depth index separately (Table 4).

TABLE 4.

Predictors of Intervention Choices

Dependent Variable	Breadth of Interventions*	Depth of Interventions†	Comprehensiveness of Interventions‡
Intercept	0.52§	1.29§	0.91§
Predictor variables
QI focus	0.07	0.04	0.05
Openness in organizational culture	0.07¶	0.07¶	0.07‖
Measuring quantifiable goals	0.23¶	0.24¶	0.23¶
Multidisciplinary team	0.21‖	0.16¶	0.19‖
Control variables
Large clinics (>400 patients)	0.06	−0.08	0.01
Specialty site	−0.02	−0.02	−0.02
Organization type: university medical center	0.00	0.00	0.00
Organization type: community-based organization	0.03	−0.06	−0.01
Organization type: community health center	−0.02	−0.15	−0.09
Organization type: public health clinic	0.08	−0.10	−0.01
Organization type: hospital	−0.10	−0.16	−0.13
Region: West	0.00	0.00	0.00
Region: Northeast	−0.16	−0.13	−0.14
Region: South	−0.10	−0.11	−0.10
Region: Midwest	−0.12	−0.11	−0.12

^*Linear regression model, N = 41, R² = 0.46, Adj. R² = 0.21, P = 0.09.

^†Linear regression model, N = 41, R² = 0.51, Adj. R² = 0.27, P = 0.04.

^‡Linear regression model, N = 41, R² = 0.52, Adj. R² = 0.29, P = 0.03.

^§P ≤ 0.001.

^¶P ≤ 0.05.

^‖P ≤ 0.01.

DISCUSSION

In this study, we analyzed the intervention choices made by clinics that participated in a QIC and assessed whether selected clinic characteristics were associated with the number and comprehensiveness of interventions. A clinic’s focus on QI and openness were related to the number and comprehensiveness of the interventions attempted and the presence of multidisciplinary teams and measuring progress toward QI goals were associated with more comprehensive interventions.

Multidisciplinary teams have been found to be effective for improving quality of chronic care,^23,33 but our results suggests that the presence of teams alone is not sufficient to increase the number of interventions attempted. The number of interventions might depend primarily on team effectiveness which in turn might depend on team composition, nature of the task, the context, team leadership and processes, and team culture, many of which were not measured in this study.^33,40 In addition, team effectiveness might also be related to the quality and suitability of each intervention separately from the total number of interventions, but we did not have independent assessments of these intervention characteristics.

Research on team learning suggests that a supportive organizational and interpersonal climate facilitate collaborative problem-solving and increases the willingness of employees to engage in trial and error experimentation.^22,34–36 Aspects of such a climate include facilitative leadership, psychologic safety,^32,34 and a culture of openness in the organization. Our finding that clinics with more open culture attempted more interventions is consistent with the research on team learning. That research also predicts that organizations with more open cultures will evaluate a higher proportion of the interventions, but our analysis (not shown) did not find that.

The literature on QI stresses the importance of clearly stating quantifiable improvement goals and then measuring progress towards them.²⁵ We found that measurement of progress towards quantifiable goals is associated with the comprehensiveness of the interventions but not with the number of interventions attempted. This could be because implementing more comprehensive interventions requires communication across multiple departments, which might be facilitated by quantification and measurement. On the other hand, merely attempting multiple interventions does not necessarily require quantifiable goals and measurement of progress towards them.

Comprehensiveness of the attempted interventions was significantly associated with the presence of multidisciplinary teams, an open organizational culture, and measurement of progress toward quantifiable goals. This could be because such an organizational culture provides the multidisciplinary teams with a place to discuss issues that span multiple areas and require inputs from different professions. This finding also suggests a likely mechanism through which multidisciplinary teams might affect quality of care: they might focus on interventions that are more comprehensive rather than just increasing the number of interventions attempted. If confirmed by other studies, this would provide important information on how to increase the likelihood that QICs have more positive effect on quality of care.

There are several limitations of our study. We measured only certain aspects of the clinics’ culture. It is likely that characteristics of the teams other than being multidisciplinary such as attitudes of its members and their influence within the clinic could have affected intervention choices.³³ Also, the modest clinic level reliability of our measures may have limited our ability to identify important associations.⁴¹ Ideally, the comprehensiveness would be calculated by measuring the number of CCM components targeted by each intervention. However, we did not have that information in our reports. There were only 41 clinics that had matching data on interventions and clinician and administrator surveys. More relationships might have been significant with a larger sample size. The data for the study were obtained only from HIV clinics. These findings might not generalize to improvement efforts focused on other conditions.

Demonstrating that organizational factors influence how clinics participate in QICs is only a first step in understanding the heterogeneity in performance of clinics in collaboratives.^15,29,31,32 For example, we do not know the extent to which the specific interventions attempted affected care quality. Recently, a study of the Health Disparities Collaborative did not find any relationship between improvement in quality and intervention choices measured by number and percentage of interventions in each CCM category.²⁰ In the future, it would be important to investigate whether the measures of intervention choices developed here along with other structural variables can explain the heterogeneity in the final outcomes.