The Influence of Context on Quality Improvement Success in Health Care: A Systematic Review of the Literature

Abstract

Context: The mixed results of success among QI initiatives may be due to differences in the context of these initiatives.

Methods: The business and health care literature was systematically reviewed to identify contextual factors that might influence QI success; to categorize, summarize, and synthesize these factors; and to understand the current stage of development of this research field.

Findings: Forty-seven articles were included in the final review. Consistent with current theories of implementation and organization change, leadership from top management, organizational culture, data infrastructure and information systems, and years involved in QI were suggested as important to QI success. Other potentially important factors identified in this review included: physician involvement in QI, microsystem motivation to change, resources for QI, and QI team leadership. Key limitations in the existing literature were the lack of a practical conceptual model, the lack of clear definitions of contextual factors, and the lack of well-specified measures.

Conclusions: Several contextual factors were shown to be important to QI success, although the current body of literature lacks adequate definitions and is characterized by considerable variability in how contextual factors are measured across studies. Future research should focus on identifying and developing measures of context tied to a conceptual model that examines context across all levels of the health care system and explores the relationships among various aspects of context.

Despite the growth of quality improvement (QI) initiatives, very little is known about how to improve care consistently across a variety of settings (Auerbach, Landefeld, and Shojania 2007; Mittman 2004). While some QI initiatives have substantially improved patient outcomes (Pronovost et al. 2006), others have made only modest improvement (Blumenthal and Kilo 1998; Mittman 2004) or none at all (Landon et al. 2004). Furthermore, even within QI initiatives, not all groups perform equally well (Mills and Weeks 2004; Ovretveit et al. 2002). These mixed results may be due partly to differences in the context in which QI initiatives take place.

Much theoretical work supports the notion that context affects organizational change, dissemination, innovation, implementation, and knowledge translation (Damschroder et al. 2009; Greenhalgh et al. 2004; Pettigrew, Ferlie, and McKee 1992). For example, Damschroder and colleagues developed their Consolidated Framework for Implementation Research that identifies five major domains in implementation theory relevant to understanding and studying effective implementation (Damschroder et al. 2009). Among these domains are three areas that can be considered aspects of context: the outer setting, the inner setting, and the characteristics of individuals. The outer setting comprises an organization's economic, political, and social context. The inner setting covers structural characteristics, networks/communications, culture, implementation climate (e.g., tension for change or the degree to which individuals believe the current situation requires change, organizational incentives/rewards, learning climate), and readiness for implementation (e.g., leadership engagement, available resources). The characteristics of individuals refer to factors such as knowledge, self-efficacy, identification with organization, and other personal attributes (e.g., motivation, values).

Included in Damschroder and colleagues’ review is the Organizational Transformation Model developed by VanDeusen Lukas and colleagues (VanDeusen Lukas et al. 2007). This model identifies five key elements of successful organizational transformation to improve patient care: impetus to transform, leadership commitment to quality, improvement initiatives that engage staff, alignment to achieve consistent goals and resource allocation, and integration across the organization's boundaries. VanDeusen Lukas and colleagues posit that these five elements drive change by affecting the mission, vision, and strategy of the organization, culture, operational functions/processes, and infrastructure.

Since QI initiatives require organizational change, implementation, and innovation, aspects of context like those identified in Damschroder's and VanDeusen Lukas's models are also sure to influence the success of QI initiatives. Using a systematic search of the business and health care literature, we aimed to (1) identify the contextual factors associated with QI success; (2) categorize, summarize, and synthesize these factors based on their common characteristics and the level of the health care system in which they operate (Batalden and Splaine 2002); and (3) understand the current stage of development of this field of research.

Methods

Definition of Terms

We used the Hastings Center's definition of QI: “systematic, data-guided activities designed to bring about immediate, positive changes in the delivery of health care” (Baily et al. 2006, S5). The types of QI activities most consistent with this definition are those targeting particular systems, outcomes, or processes (e.g., reducing clinic wait times or falls) and applying specific tools and methods (e.g., Plan-Do-Study-Act [PDSA] cycles) to the given problem. We classified these types of QI activities as specific QI initiatives. Other types of QI activities apply QI methods to entire organizations (e.g., Total Quality Management [TQM], Continuous Quality Improvement [CQI]). Although these activities are less strictly representative of the QI definition used, we included these in order to enrich our knowledge base. These types of QI activities were labeled general QI.

We defined context as anything not directly part of the technical QI process that includes the QI methods themselves and the clinical interventions. Therefore, context may include factors relating to the characteristics of the organizational setting, the individual, his or her role in the organization, and the environment (Rousseau 1978). We classified contextual variables based on the level of the system in which they operate according to the model of the health care system developed by Batalden and Splaine (2002). In this model, microsystems are composed of small groups of people who regularly work together to provide care. Examples are specific hospital units or physicians’ offices. Microsystems are frequently embedded in larger systems or organizations, referred to as macrosystems. Examples of macrosystems are hospitals and integrated delivery systems. The environment refers to the community and society in which these smaller systems operate and includes both socioeconomic and political forces. In some cases, contextual factors span multiple levels of the system and may, for example, exist across both the macro- and microsystem levels. We classified those factors that we believed could cross multiple levels of the health care system using our own subheadings (e.g., QI support and capacity, QI team).

Since we know of no established definition of QI success, we accepted for our review a broad range of outcomes as measures of QI success. To minimize heterogeneity, we placed each outcome measure into one of five predetermined categories. These categories were based on measures of (1) improvement in the target outcome, process, or system before and after the QI initiative; (2) superior performance/quality in the setting of a QI initiative (for cross-sectional studies); (3) perception of success or improvement; (4) extent of implementation of QI practices; and (5) adoption of QI philosophies or frameworks (e.g., TQM/CQI). We included the latter two categories because we believed them to be critical precursors to QI success.

When the articles used a particular measure as both a dependent and an independent variable and the variable met the definition of context used in our study, we allowed it to serve as both a QI success measure and a measure of context. For example, we used measures of the degree of QI implementation or QI implementation scope and intensity as measures of QI success (extent of implementation of QI practices) and as measures of context (reflective of the degree to which QI is embedded within the organization).

Eligibility Criteria

We included those studies that examined the association between contextual factors and success in the setting of a QI initiative. We report only those articles that examined the influence of context in health care QI initiatives. Details of studies in other industries can be obtained from the authors. We examined studies using observational designs, controlled designs, or meta-analysis formats and included only studies that measured contextual factors with objective variables, questionnaires, or other empirical methods. We excluded studies that did not formally test the association between context and improvement using statistical methods, and we also did not include commentaries, editorials, letters, review articles, newspaper articles, books, case studies, and journal articles without original data.

Information Sources and Search Strategy

Librarians from the medical and business school aided with designing strategies to search health care (MEDLINE, Cinhal) and business/social science (Business Source Complete, ABI/INFORM Complete) databases, restricting our search to English-language articles published between January 1, 1980, and October 7, 2008. We also hand-searched all titles published in Implementation Science from its inception in 2006 through December 11, 2009. Furthermore, to enrich the subset of articles examining context in the health care setting, we hand-searched the references of all included studies of health care QI initiatives and asked experts in the field to suggest relevant articles not identified in our search.

We tailored our search strategies for each electronic database to include both quality-related and context-related terms. Table S1 (see appendix) gives the details of our individual search strategy for each database. Search terms for the health care databases included a mix of MeSH subject heading terms and keywords found in the full citation, title, or abstract. Similarly, the search terms for the business databases were a mix of subject terms and keywords found in the full citation. Based on our search strategy, we settled on only those articles that contained both quality improvement and context-related terms in the subject heading, title, or abstract.

Study Selection

Before selecting the studies, all the reviewers used calibration exercises to ensure a consistent application of the review criteria. Pairs of reviewers independently reviewed titles identified by the searches, basing their decision to review the article abstract on whether the title referred to QI and at least one organizational variable or suggested an empirical study of QI. The abstract was reviewed if either reviewer felt that it was eligible. Two reviewers also independently reviewed abstracts for eligibility. Disagreements were adjudicated until an agreement was reached. If the reviewers could not determine eligibility from the abstract, they obtained the full article. At the abstract review stage, abstracts were excluded if they clearly met one of the exclusion criteria (e.g., not QI, no statistical testing, case study, etc.). Eligibility was confirmed for the full article by a pair of reviewers using a standardized form. Duplicates were excluded at every stage of the process.

Data Abstraction and Synthesis

Five persons reviewed the eligible articles and abstracted data from all eligible articles using standardized forms that they had developed, tested, and refined. A single second reviewer also checked all data abstractions.

Extracted information included details about the study setting, type of QI initiative, measures of QI success, contextual factors, and results. Reviewers grouped the measures of QI success into one of the five predetermined categories previously described and also conducted a semistructured assessment of methodological quality. The contextual factors assessed in each article were documented and, at the end of the review, were sorted and grouped based on common characteristics and the level of the health care system in which they operated.

Due to the heterogeneity of studies, we did not try to perform a quantitative meta-analysis. Associations between context and success were deemed significant if testing reported a p-value ≤0.05. If no p-value was provided, the author's report of significance was accepted. Articles that used the same study population but reported unique analyses were treated independently, and articles that used the same population and reported the same analyses were treated as a single study.

Results

Characteristics of Eligible Studies

Of the 15,631 potentially relevant citations, forty-one articles in health care settings met our eligibility criteria. Two eligible articles were identified by bibliographic review, and four more were suggested by experts in the field (see figure S1 in the appendix). Thus, the final review contained a total of forty-seven articles, and after we identified those articles published by a single set of investigators, using the same study population, there were forty-one unique studies.

Table 1 summarizes the characteristics of the forty-seven included articles. Although articles used a variety of study designs, most were cross-sectional (72%). A majority of the articles examined context in general QI initiatives, and of these, TQM/CQI was the most commonly used paradigm. Twenty (43%) examined context in specific QI initiatives, and of these, the Institute for Healthcare Improvement's Breakthrough Series Collaborative was the most commonly studied framework. As figure 1 shows, eighteen (39%) were published in the last five years of our review (2004–2008). Of the articles examining context in general QI initiatives, eleven (41%) were published before 2000, whereas only three (15%) of the articles examining context in specific QI initiatives were published before 2000.

Table 1.

Characteristics of Included Articles

Characteristic	n= 47 (%)
National Setting
United States	37 (78)
International	7 (15)
Both	3 (6)
Organizational Setting
Inpatient	27 (57)
Nursing home	10 (21)
Outpatient	4 (9)
Both (inpatient and outpatient)	3 (6)
Other	3 (6)
Type of QI
General	27 (59)
TQM/CQI	18 (38)
Self-labeled	8 (17)
Other	1 (2)
Specific	20 (43)
Self-labeled	8 (17)
Break-through series	7 (15)
Other	3 (6)
TQM/CQI	2 (4)
Journal Type
Health care	29 (62)
Both	11 (23)
Business	7 (15)
Study Design
Cross-sectional	34 (72)
Cohort	7 (15)
Controlled study	3 (6)
Before and after	2 (4)
Case control	1 (2)
QI Success Measure*
Extent of implementation of QI practices	15 (32)
Perception of success or improvement	19 (40)
Adoption of TQM	7 (15)
Superior organizational performance or outcome	5 (11)
Pre/Post process or outcome changes	9 (19)
Other	1 (2)

Note:*Articles may have more than one QI success measure.

Figure 1.

Distribution of Date of Publication of Included Articles

Contextual Factors Examined

Sixty-six contextual factors emerged during the review and categorization process. A list of the specific measures used in each article and how they were classified by the reviewers is summarized in the appendix, in supplemental tables S2 and S3.

Associations Found between Contextual Factors and QI Success

We synthesized the results of the statistical associations between the contextual factors that emerged from the categorization process and QI success. The articles frequently used multiple measures of a given contextual factor and tested the associations between these measures and multiple measures of QI success. For each contextual factor, table 2 counts the number of positive, negative, and nonsignificant associations found between an individual measure of context and an individual measure of QI success. The results are grouped by the class of QI success measure and also are summarized over the forty-seven included articles. Information about the individual associations tested can be obtained from the authors. The results for the subset of contextual factors examined most frequently are discussed next.

Table 2.

Results of Associations between Context and QI Success

	Adoption of CQI/TQM			Extent of Implementation of QI Activities			Perception of Success or Improvement			Performance/Quality			Pre/Post Measure of Quality			OVERALL SUMMARY
Articles	7			15*			19			5			9			47†
	+	−	NS	+	−	NS	+	−	NS	+	−	NS	+	−	NS	Articles	Associations‡	+	(%)	−	(%)	NS	(%)
Environment
Competition	3	1	4	2	3	26	1		3	7	2	34				10	86	13	15	6	7	67	78
Managed Care Penetration			2	7	4	16				2	2	24				7	57	9	16	6	11	42	74
Medicare/Medicaid Influence	3		2													3	5	3	60	0	0	2	40
Regulation	2		1													3	3	2	67	0	0	1	33
Union Influence			1			1										2	2	0	0	0	0	2	100
TQM Adoption by Other Hospitals	2		1													1	3	2	67	0	0	1	33
Accreditation				1	2	9				5		13				1	30	6	20	2	7	22	73
P4P				1		1										1	2	1	50	0	0	1	50
Organization
Size	4		3	9	6	13			1	1						18	37	14	38	6	16	17	46
QI Leadership: Top Management			2	11		3	5		1	1			1		1	15	25	18	72	0	0	7	28
Culture				11		1	15	1	15	2		5	2		4	11	56	30	54	1	2	25	45
Ownership	1		1	8	6	37			1	4	8	56				10	122	13	11	14	11	95	78
Teaching Status	1		1	2		9				1	1	13				9	28	4	14	1	4	23	82
QI Maturity: Years Involved in QI				11		3	1		4	4	4	6				9	33	16	48	4	12	13	39
QI Leadership: Board Leadership			3	16		10	2		4							8	35	18	51	0	0	17	49
QI Maturity: QI Scope							4	1	2	13	10	33				8	63	17	27	11	17	35	56
Affiliation	2			5	1	6					1	9				8	24	7	29	2	8	15	63
Location			1	2		7				1		5				6	16	3	19	0	0	13	81
Physician Involvement in QI	1					1	3		1	2						6	8	6	75	0	0	2	25
Barriers					6	4									2	5	12	0	0	6	50	6	50
Customer Focus				1	2		2		1						1	5	7	3	43	2	29	2	29
Financial Health	1		1	3	2	9						10				5	26	4	15	2	8	20	77
Structure: Clinical Integration				7		2	1									5	10	8	80	0	0	2	20
QI Leadership: Board Structure	1	1	4	4	5	8				1		13				5	37	6	16	6	16	25	68
Service Mix				4		15				8	1	19				4	47	12	26	1	2	34	72
Physician Arrangements				2		8						14				4	24	2	8	0	0	22	92
QI Leadership: Organizational Support					2		1			1			1		1	4	5	3	60	2	40	1	20
QI Leadership: Middle Management		1	2				1		1							4	5	1	20	1	20	3	60
Volume						4					7	4				3	15	0	0	7	47	8	53
QI Leadership: Strategic Planning						4	1		1							3	6	1	17	0	0	5	83
Implementation Approach				1		1	1									3	3	2	67	0	0	1	33
Motivation to Implement QI	2		4													2	6	2	33	0	0	4	67
QI Maturity: Intensity							2		10	6	3	11				2	32	8	25	3	9	21	66
Innovativeness				3	1	9										2	13	3	23	1	8	9	69
Structure: General							2	2				3				2	7	2	29	2	29	3	43
QI Maturity: QI Integrated with QA						1										1	1	0	0	0	0	1	100
QI Leadership: Manager Characteristics	2	1	1													1	4	2	50	1	25	1	25
Process Management									1							1	1	0	0	0	0	1	100
Supplier Relationships							1									1	1	1	100	0	0	0	0
QI Support and Capacity
Data Infrastructure: Information Systems			1	13	1	6			1						1	9	23	13	57	1	4	9	39
Resources: Funding				4		7	1		1		1	3	1			8	18	6	33	1	6	11	61
Resources: Funding				4		7	1		1		1	3	1			8	18	6	33	1	6	11	61
Infrastructure for QI	1		1	2				1	7							5	12	3	25	1	8	8	67
Resources		1								1			2		1	5	5	3	60	1	20	1	20
QI Consultants			1			2										3	3	0	0	0	0	3	100
Data Infrastructure: Data Feedback							1			3	1	2	2			3	9	6	67	1	11	2	22
Resources: Time	1		1												1	3	3	1	33	0	0	2	67
QI Workforce Focus							2		1							3	3	2	67	0	0	1	33
Data Infrastructure							2									2	2	2	100	0	0	0	0
QI Workforce Focus: Rewards/Incentives									3							1	3	0	0	0	0	3	100
Microsystem
Motivation to Change							4						1		4	6	10	6	60	0	0	4	40
Champion						2	2						3		5	5	12	5	42	0	0	7	58
Physician Leadership				1						1			3		3	3	8	5	63	0	0	3	38
Culture/Climate				1			1									2	2	2	100	0	0	0	0
Capability for Change													2			2	2	2	100	0	0	0	0
QI Team
Physician Involvement on Team						2	2			2		11			1	6	19	4	21	0	0	14	74
Group Process							5		4						2	5	12	6	50	0	0	6	50
Team Leadership							2						2			5	5	5	100	0	0	0	0
Team QI Skills							3		2						2	4	7	3	43	0	0	4	57
Group Climate							2		3				2			3	7	4	57	0	0	3	43
Support							3		1							3	4	3	75	0	0	1	25
Prior Experience with QI							1								1	2	2	1	50	0	0	1	50
Prior Experience Working Together									1				1			2	2	1	50	0	0	1	50
Miscellaneous
Strategic Importance to Organization													2			2	2	2	100	0	0	0	0
Listserv Use															1	1	1	0	0	0	0	1	100

Notes: Findings presented in descending order in each category based on the number of articles examining an association between the contextual factor and QI success.

^*Two articles reported the exact same analyses in the same study population, so counts of those associations reported only once.

^†Includes one article with QI success measure classified as “Other.”

^‡Number of associations often sum to more than number of articles, as many articles tested multiple associations for each contextual factor.

Environment

Competition was the most consistently studied environmental contextual factor, examined in ten articles (eighty-six different associations tested). Articles used a range of measures, both objective and perceived, to examine competition. Authors found significant associations in 22 percent of the associations tested (thirteen positive associations, six negative associations), although they varied based on the types of measures used. For example, Weiner and colleagues (Weiner et al. 2006b) found that hospitals in more concentrated markets had a greater participation of physicians in QI teams (extent of implementation of QI activities). However, they found that market concentration and hospital competition intensity had a weak and inconsistent association with QI success when measured as risk-adjusted patient safety indicators (superior quality/performance), showing significant associations with only one of the four patient safety indicators examined (Weiner et al. 2006a). Similarly, Zinn and colleagues found different results based on the type of measure of competition used. Perceived competition was a predictor of TQM adoption, but more objective competition indicators, such as the Herfindahl index and measures of excess capacity, were not (Zinn, Brannon, and Weech-Maldonado 1997; Zinn, Weech-Maldonado, and Brannon 1998).

Macrosystem (Organization)

Twenty-two (46%) studies examined at least one aspect of leadership. Leadership from top management (fifteen articles, twenty-five associations tested) and leadership by governing boards (eight articles, thirty-five associations tested) were two of the most frequently studied aspects of leadership.

Authors looked at multiple aspects of top management leadership. Taken as a whole, they showed consistent positive associations with QI success, with significant positive associations seen in 72 percent of the associations tested, although the results varied depending on the specific aspects of leadership examined. For example, LeBrasseur, Whissell, and Ojha found that CQI vision/strategy and a senior management that communicates new expectations were predictors of CQI outcome achievement (perception of success). But a senior management that knows or recognizes the CQI paradigm and engages in top-down planning and implementation was not a significant predictor of QI success (LeBrasseur, Whissell, and Ojha 2002). A series of articles by Alexander, Weiner, and Griffith and by Weiner and colleagues examined CEOs’ participation in TQM/CQI as a measure of top management leadership and consistently found positive associations with QI success (extent of implementation of QI activities) (Alexander, Weiner, and Griffith 2006; Weiner, Shortell, and Alexander 1997; Weiner et al. 2006a, 2006b).

Several studies examined the role of governing boards, particularly board leadership for quality in QI success, and found mixed results. Fifty-one percent of the relationships tested showed significant positive associations. For example, Weiner and colleagues found that board monitoring of QI was associated with greater QI implementation scope in three of the four measures used (extent of implementation of QI activities) (Weiner, Shortell, and Alexander 1997; Weiner et al. 2006a). In a related study, however, Alexander and colleagues found that board activity was associated with QI scope in only one of the four measures used (Alexander, Weiner, and Griffith 2006).

Factors relating to the maturity of an organization's experience with QI were analyzed in thirteen studies (28%). The factor most often examined was the duration of an organization's formal involvement in QI, which was examined in nine articles (thirty-three associations tested). Sixteen (48%) of the associations tested revealed a significant positive relationship between length of involvement in QI and success. Some studies also looked at the scope or degree to which QI was embedded in the organization (eight articles, sixty-three associations), although results were less consistent. Only twenty-eight (44%) of the associations were significant, and the significant findings were split between positive associations (27%) and negative associations (17%). For example, Weech-Maldonado, Zinn, and Hamilton found that the degree of TQM implementation was associated with the impact of TQM on financial and human resource performance (perception of success) (Weech-Maldonado, Zinn, and Hamilton 2001). In their series of articles, Alexander, Weiner, and Griffith and Weiner and colleagues tested the effect of QI scope and intensity across an organization on various measures of performance. They found different results depending on both the individual measure of scope and intensity and the outcome measure that was used (Alexander, Weiner, and Griffith 2006; Weiner et al. 2006a, 2006b).

The presence of a culture supportive of QI was frequently studied. Eleven studies (24%) evaluated the role of organizational culture in QI success, and despite the different measures used, significant positive associations were found in 54 percent (of fifty-six associations tested). The framework most commonly used in these studies to examine organizational culture was the competing values framework. A majority examined associations between group, developmental, rational, and/or hierarchical culture and QI success (Berlowitz et al. 2003; Carman et al. 1996; Parker et al. 1999; Shortell et al. 1995; Wakefield et al. 2001). Shortell and colleagues also examined the cultural balance among these four culture types and found that cultural balance was significantly associated with the number of changes made by QI teams (extent of implementation of QI activities) (Shortell et al. 2004). Similarly, Carman and colleagues added to the competing values framework by measuring the strength of the culture. They reported that the strength of the culture directly affected changes in patient satisfaction (pre/post change) and QI outputs (perception of success) (Carman et al. 1996).

Other authors looked at culture in different ways. Rad found that creativity, risk taking, stability, attention to detail, and collectivism were positively correlated with TQM success (perception of success) and that power distance and organic structure were negatively correlated with TQM success (Rad 2006). Rondeau and Wagar focused their examination of culture on learning-oriented values and found that the implementation of formal TQM/CQI required a supportive learning culture (Rondeau and Wagar 2002).

QI Support and Capacity

Among the factors relating to QI support and capacity, those pertaining to data infrastructure were the most consistently examined (fourteen articles), with the existence of data information systems being the most common factor (nine articles, twenty-three associations) studied. The presence of data systems was positively associated with QI success in 57 percent of the associations tested. For example, Li examined information analysis for QI (scope, validity, and management of data underlying overall QI) and found significant associations with service quality performance (perception of success) (Li 1997). In their examination of the effects of the Baldrige health care quality domains on organizational performance, Meyer and Collier found that information and analysis (e.g., management of information and data, performance comparisons and benchmarking, and analysis and use of organizational-level data) directly affected organizational performance (perception of success) (Meyer and Collier 2001).

Thirteen (28%) articles looked at various aspects relating to resources for QI. The most frequently examined contextual factors were funding (eight articles, eighteen associations), general resources (five articles, five associations), and time (three articles, three associations). Studies that assessed general resources for QI found positive associations in 60 percent of the associations tested. For example, using the same questionnaire, Mills and Weeks and Weeks and colleagues determined whether QI teams felt they had sufficient resources and found that low performers had lower measures of resources from the beginning to the end of the QI initiative (Mills and Weeks 2004) and that high performers were more likely to report sufficient resources (Weeks et al. 2003). Those studies that specifically looked at funding for QI found less consistent results, with 61 percent of the associations tested being nonsignificant.

Contextual factors related to workforce development were not consistently associated with QI success, even though they were examined frequently (ten articles). The factor examined most frequently was the presence of quality training programs (six articles, eight associations), although positive associations were only found twice. Although general aspects of QI workforce focus were examined infrequently (three articles, three associations), 67 percent of these associations were positive and significant (Li 1997; Molfenter et al. 2005). For example, in their predictive model of QI success, Molfenter and colleagues included a factor examining the work environment, including leader roles, organization structure, incentives, and staffing to support QI, and Li found a significant relationship between workforce development and service quality performance (perception of success).

Microsystem

Microsystem factors were examined in twelve (26%) of the included studies. For example, in their predictive model of QI success, Molfenter and colleagues used a range of measures (total of eight factors) at the microsystem level (Molfenter et al. 2005). The microsystem factors most frequently examined in the other articles were local physician leadership (three studies, eight associations), presence of project champions (five studies, twelve associations), and motivation to change (six studies, ten associations). Among the associations between project champions and QI success, 42 percent showed significant positive associations. For example, in their randomized controlled trial, Berner and colleagues found significant improvement in one of five quality indicators examined among the group exposed to the physician opinion leader plus traditional quality improvement versus quality improvement methods alone (Berner et al. 2003). Similarly, measures of microsystem physician leadership were associated with QI success in 63 percent of the associations tested.

The microsystem's capability to change was measured in two studies (two associations), and motivation to change was measured in six studies (ten associations) (Litaker et al. 2008; Mills et al. 2003; Mills and Weeks 2004; Molfenter et al. 2005; Stevenson et al. 2001; Weeks et al. 2003). Associations were more consistent for capability (100%) than motivation (60%). For example, Litaker and colleagues looked at both capacity (motivation + ability) and practice ability to change in their controlled study of an intervention to increase preventive services delivery. Practice ability to change was measured as the amount of assistance that a practice needed to implement tools and approaches designed to increase preventive services delivery, whereas motivation was captured by the amount of effort needed to motivate practice staff. They found that practices with greater capacity for change (motivation + ability) delivered preventive services to eligible patients at higher rates (pre/post change). In secondary analyses, they showed a strong association with change ability and a nonsignificant, weaker association with motivation, and they showed a significant interaction between motivation and ability (Litaker et al. 2008). Stevenson and colleagues (Stevenson et al. 2001) also looked at components of capability and motivation (e.g., personal interest in the disease focus of the QI project and attitude toward QI). They found that recognizing the need to overcome obstacles by developing systematic plans to implement change was associated with successful QI (pre/post change). Three studies (Mills et al. 2003; Mills and Weeks 2004; Weeks et al. 2003), all using the same instrument, examined the role of front-line staff support. Two (Mills et al. 2003; Weeks et al. 2003) found associations between greater front-line staff support and QI success (pre/post change).

QI Team

Thirteen (28%) articles examined the association between aspects of the QI team and QI success. The factors most consistently associated with QI success were team leadership (five articles, five associations), group climate (three articles, seven associations), group process (five articles, twelve associations), team QI skills (four articles, seven associations), and physician involvement on the QI team (six articles, nineteen associations).

Overall, team leadership was significantly associated with success in all the tested associations. For example, three studies using the same instrument showed that high-performing projects (pre/post change) had stronger team leadership (Mills et al. 2003; Mills and Weeks 2004; Weeks et al. 2003). Wilkens and London looked at the extent to which the facilitator/leader structured group process and found a positive association with success (perception of success) (Wilkens and London 2006). Similarly, Higgins and Routhieaux found a positive association between the perception of team leader performance and QI team effectiveness (perception of success) (Higgins and Routhieaux 1999).

Five studies evaluated QI team group climate and/or group process (Higgins and Routhieaux 1999; Lemieux-Charles et al. 2002; Mills et al. 2003; Mills and Weeks 2004; Wilkens and London 2006). Of the seven tests of the association between measures of group climate and measures of QI success, 57 percent were significantly positive. Of the twelve associations between group process and QI success, 50 percent were significantly positive. For example, Wilkens and London examined measures of group climate, including group learning orientation, self-disclosure, and psychological safety, as well as group process, including perceptions of feedback seeking/giving, task conflict, and relationship conflict. They found significant associations between the learning orientation measure of group climate and QI success but no associations between group process measures and the group leader's assessment of team performance (perception of success) (Wilkens and London 2006). Lemieux-Charles and colleagues looked at strong norms of the behavior on the team, whether the team had well-designed decision making practices and whether the team had good group processes. They found significant effects of norms and process strategies on team members’ perception of success, and the effects of process strategies persisted when the outcome measure used was an external rating of performance (Lemieux-Charles et al. 2002). Similarly, Mills and Weeks found that high-performing projects (pre/post change) were more likely to report that team members understood one another's strengths and weaknesses, expressed opinions freely, and had mutual respect (Mills and Weeks 2004).

Four studies assessed team QI skills. Among the seven associations examined, three (43%) were significantly associated with QI success. Tucker Nembhard, and Edmondson examined two types of activities on the QI team related to their ability to learn how to implement best practices (“learn how activities”) and their ability to identify best practices (“learn what activities”) and found an association between “learn how activities” (e.g., PDSA cycles) and QI success (perception of success) (Tucker, Nembhard, and Edmondson 2007). In contrast, Higgins and Routhieaux found no association between specific skill levels of team leaders or team members—including skills related to computer use, data collection and analysis, statistical process control, and QI tools and techniques—and QI team effectiveness (perception of success) (Higgins and Routhieaux 1999).

Miscellaneous

Whether the work was perceived as part of the organization's strategic goals emerged as a contextual factor that was examined in the literature in two related studies. Mills and Weeks and Weeks and colleagues found that teams engaged in work perceived as part of the organization's strategic goals were more likely to be successful (pre/post change) (Mills and Weeks 2004; Weeks et al. 2003).

Observations about Study Methodological Quality

Because no standard measures or scales were available to rate the methodological quality of the evaluated studies, reviewers conducted a semistructured assessment of the quality of the included studies. Potential methodological issues that the article reviewers noted were inadequate validation of subjective measures used to assess context (n= 21) among the subset of thirty-eight articles using questionnaire-based measures of context; use of only univariable analyses (n= 13) leading to limitations in understanding independent effects of specific factors; and use of a cross-sectional design (n= 34) limiting the ability to draw strong conclusions about the direction of the relationship between context and QI success. In addition, many studies used only subjective measures of QI success (n= 25), and in those studies that examined both subjective and objective measures of success, the results were often inconsistent across these two types of measures. Furthermore, only a small number of studies examined complex associations, such as interactions (n= 4) or direct/indirect effects (n= 7) among contextual factors.

Discussion

With this systematic review, we have provided a comprehensive review of the published literature to identify the contextual factors that have been suggested to influence QI success. Although the literature review revealed that the current body of work is in an early stage of development, a number of useful hypothesis-generating themes emerged regarding aspects of context that may be relevant to QI success.

According to our initial categorization, more than sixty-six contextual factors could be related to QI success. Although most factors were examined in only a few studies, organizational characteristics (e.g., size, ownership, teaching status), leadership from top management, competition, organizational culture, years involved in QI, and data infrastructure/information systems were examined in at least nine (20%) articles. With the exception of ownership, teaching status, and competition, all these factors were generally shown to influence QI success, with more than half the associations tested being significantly associated with success. Other factors that were examined less frequently but that had fairly consistent associations with QI success were board leadership for quality; organizational structure, particularly clinical integration across departments; customer focus; physician involvement in QI; microsystem motivation to change; resources; and QI team leadership.

Our review also revealed that much of the current research suffers from conceptual ambiguity and methodological weaknesses. The principal methodological weaknesses we identified in these studies were the use of poorly validated measurement instruments, the failure to use multivariable analyses or other methods (e.g., path analysis, structural equation modeling) to understand the combined and relative effects of multiple factors together, and the use of subjective measures of QI success. Accordingly, we cannot make definitive conclusions about the influence of particular contextual factors in QI success and the aspects of context that we have identified as related to QI success should be studied further.

It is encouraging that the contextual factors identified in this examination of QI initiatives are similar to those demonstrated in broader theories of implementation and organizational change to be important (Damschroder et al. 2009; Greenhalgh et al. 2004). Our review adds to these theories by explicitly classifying context based on a system's view of health care and by identifying those contextual factors that have been suggested to be important to QI success but were excluded from these broader theories.

With research spanning more than a decade across multiple disciplines, a systematic method was an important approach to begin collecting and categorizing previous work. Our review methods did, though, have some limitations. Although we took a systematic approach to identifying published articles, we may have missed relevant articles owing to the specific terms we elected to use in the search strategy, the lack of standardized keywords and subject headings, and the possibility that authors may not have used context-related terms in their article titles or abstracts. We did try to identify missed articles by hand-searching references and contacting experts. Future efforts in this field would be greatly aided by developing useful, standardized subject headings and keywords to improve the searchability of this literature.

Since we were interested in examining context in a QI setting, we purposely limited our search terms to those specifically relating to QI (e.g., PDSA, PDCA, Six Sigma, Lean Management, TQM). Consequently, we may have missed studies examining a wider range of methods for QI. There likely are similarities between the role of context in QI and other implementation strategies (e.g., implementation of evidence-based practice), and conducting a similar review of the literature focused specifically on these other methods might provide additional insight.

Although qualitative studies have contributed greatly to the understanding of context in QI initiatives, we chose to exclude them from this systematic review in order to more easily categorize and synthesize the contextual factors. We based our decision on an assumption that the contextual factors considered in quantitative studies would be grounded in relevant theory and previous knowledge developed in qualitative studies. This was not universally true, however. Therefore, the results of this systematic review could be augmented by conducting a similar review of the qualitative literature. In addition, future research would benefit from a more traditional social science (e.g., mixed methods) methodology that includes qualitative work as a basis for quantitative development.

For several reasons, the reviewers were required to make judgments when categorizing factors: the authors’ language was sometimes unclear; the theory behind the testing of a given contextual factor was often not explicitly stated; and studies often used different language to measure the same (or similar) phenomenon. This may have led to errors in classification and/or misrepresentation of the study authors’ intent. We have provided as much detail as possible to promote transparency and allow readers to make judgments on possible misclassification. Future research would benefit from clear definitions of contextual factors and well-specified measures of contextual factors linked directly to a conceptual model.

The weaknesses identified in the current body of literature highlight the need for advances in the field of QI research. This review provides a starting point to standardize definitions, inform theories, and generate hypotheses about the relationships among contextual factors and their influence on QI success that can be rigorously tested. Future research should focus on the development and testing of conceptual models that identify relevant interrelationships of contextual factors to one another and to outcomes, the use of explicit and uniform definitions of contextual variables that will allow for more cross-study comparisons (e.g., meta-analyses of context across related studies), study designs that use qualitative work as a basis for quantitative development, analytic methods that allow for simultaneous measurement of multiple key contextual factors, and efforts to gain a deeper understanding of the meaning and measurement of “QI success.”

It is encouraging to note that many of the aspects of context suggested as important can be modified. For example, through additional capital investments, an organization could easily modify such aspects of organizational infrastructure as information systems (e.g., data infrastructure) and general QI infrastructure (e.g., quality steering council or QI department). Similarly, there are many potential ways of increasing physicians’ involvement in QI, like aligning payment incentives and tying participation in QI to maintenance of board certification (Caverzagie et al. 2009). If future research continues to support the ideas that the number of years an organization is involved in QI and the depth with which QI pervades the organization are important to QI success, it will be critical for organizations to continue the long-term pursuit of QI across changes in leadership, economic climate, and policy. While other aspects of context, such as organizational culture, microsystem motivation, and leadership, may be modifiable, it still is not clear how best to change these factors. To realize the potential of QI methods to transform health care quality, future research must focus simultaneously on understanding the role of context in QI and developing interventions to change aspects of context important in QI success.

This systematic review of the literature is an important step in understanding the critical role of context in the success of QI initiatives in health care. Although the review yielded many ideas about the key contextual factors important to QI success as well as a preliminary categorization schema, it also highlighted the need to advance the field of improvement and implementation research. Generating new knowledge that will provide an improved understanding of the modifying effects of context and enable better application of QI methods may allow us to overcome current variability in the results of QI initiatives.

Appendix

Table S1.

Search Strategy Used for Each Electronic Database

		Medline	Cinhal	ABI/INFORM Complete	Business Source Complete
QI	1	Total quality management 2 Quality assurance, health care 3 (pdsa or pdca or six sigma or lean management or plan do study act or QI or TQM or CQI).mp 4 1 or 2 or 3	Quality Management, Organizational Quality Improvement “Quality Assessment” or pdsa or pdca or “six sigma” or “lean management” or “plan do study act” or QI or TQM or CQI 1 or 2 or 3	(Total Quality) OR (Total Quality and Implementation) OR (Total quality AND Organizational Change) OR (Total Quality AND Management) OR (Quality management) OR (Quality AND Improvements) OR (Six Sigma)	SU(total quality management) or “quality improvement” or “CQI” or “TQM” or “Six sigma” or “process improvement”
Context	5	Organizational culture or models, organizational or leadership or behavior or communication or cooperative behavior or organizational policy or motivation or institutional management teams or attitude of health personnel or health personnel/px or patient care team	“Organizational culture” or leadership or “cooperative behavior” or communication or “attitude of health professional” or “health personnel” or “psychosocial factors” or “multidisciplinary care team”	AND (Organizational Behavior) OR (Organization Theory) OR (Culture) OR (Corporate Culture) OR (Leadership) OR (Executives) OR (Motivation) OR (Teams) OR (Teamwork) OR (Organizational change) OR (Organizational structure)	AND SU(organizational behavior or organizational learning or corporate culture or leadership or executives or motivation or teams in the workplace or organizational change or organizational structure)
	6	4 and 5	4 and 5
Limits	7	Limit 6 to English language	Search Options (Limits) English Language	Limit to scholarly journals	Limit to academic journals
	8		Search Options (Limits) Exclude MEDLINE records

Table S2.

Characteristics of Articles Examining Context in the Setting of General Organizational Quality Improvement

Source	QI Framework	Dependent Variable (QI Success)	Independent Variable (Context)
			System Level	Contextual Factor (measures)
Alexander et al. 2006	Self-labeled QI	Extent of implementation of QI activities	Organization	Ownership (HMO ownership; PPO ownership; Public, nonfederal ownership; For-profit ownership); Size (Number of hospital beds set up and staffed for use); Affiliation (Owned, leased, or sponsored by healthcare system); Teaching Status (Member of Council of Teaching Hospitals); Financial Health (Hospital Profitability—2-year average cash flow); QI Maturity: Years Involved in QI (Years of formal involvement in QI); Structure: Clinical Integration (Clinical integration—hospital organization around clinical processes); Innovativeness (Number of innovative services; Percentage outpatient surgeries)
			Support/Capacity	Resources: Funding for QI (Total expenses per bed for QI); Data Infrastructure: Information System (Integrated data systems; Perceived clinical IS capabilities)
Alexander, Weiner, and Griffith 2006	Self-labeled QI	Superior organizational performance/quality	Environment	Competition (Market concentration; Perceived number of hospital competitors; Perceived hospital competition intensity); Managed Care Penetration (Managed care penetration; Percentage patients in private managed care)
		Extent of implementation of QI activities	Organization	Ownership (For-profit ownership; HMO ownership; PPO ownership; Indemnity ownership; Public, nonfederal ownership); Size (Number of hospital beds, hospital beds squared); Affiliation (System or network affiliated); Location (Metropolitan Statistical Area); Teaching Status; Volume (Number of inpatient surgeries); Service Mix (Total outpatient visits; Ratio of outpatient/inpatient services); Physician Arrangements (Number of physician arrangements); QI Leadership: Top Management (CEO participation in QI activities); QI Leadership: Governing Board Structure (Presence of governing board); QI Leadership: Board Leadership (Board monitoring of QI; Board activity in QI); QI Maturity: QI Scope (Diffusion of QI across hospital units; Proportion of FTEs on QI teams; Proportion of managers on QI teams; Proportion of physicians on QI teams); QI Maturity: Years Involved in QI (Years of formal involvement in QI); QI Maturity: QI Intensity (Proportion of QI teams using quality data; Number of guidelines developed; Intensity of process improvement tool use; Emphasis on quality improvement; Proportion of physicians using guidelines); Structure: Clinical Integration (Clinical integration); Barriers to Implementing QI (Perceived barriers to QI)
			Support/Capacity	Resources: Funding for QI (Total expenses on QI); Data Infrastructure: Information System (Integrated database; Perceived clinical IS capabilities)
			QI Team	Physician Involvement on Team (Percentage physicians on QI teams)
Bartlett, Lloyd, and Berry 1997	TQM/CQI	Extent of implementation of QI activities	Organization	Ownership (Class—public vs. private); Size (Medium vs. large); Location (Urban/rural); QI Leadership: Strategic Planning (Hospital has strategic plan; Strategic plan addresses quality; Plan addresses quality through broad statements; Plan addresses quality through goals and targets); QI Maturity: Years Involved in QI (TQM practices adopted pre/post 1991); Customer Focus (Hospital measures quality performance by consulting customer representatives); Barriers to Implementing QI (Internal barriers to TQM implementation); Other (Hospital reports its quality activities externally; TQM practices communicated internally; Quality costs measured; TQM practices implemented by senior management, middle management, nurses, medical officers, allied health staff, administrative staff, hotel staff; TQM philosophy implemented as adjunct to management practices or is fully integrated)
			Support/Capacity	QI Workforce Focus: Quality Training (Hospital has quality training program; Quality training program is offered to senior management, middle management, nurses, medical officers, allied health/administrative/hotel staff); Data Infrastructure: Information System (Hospital measures quality performance by patient survey, staff survey, meeting predetermined standards, consulting customer representatives, monitoring customer complaints, benchmarking); QI Consultants (TQM consultants)
Carman et al. 1996	TQM/CQI	Pre/Post outcome changes	Environment	Competition (Perceived competition; Growth rate of capitated, reimbursement contracts); Other (Environment/Resources: Market share, gross revenue, percentage operating margin)
		Perception of success or improvement	Organization	Culture (Group/developmental/rational/hierarchical; Strength of culture; Role conflict, supervisor support, and feelings of distress in the workplace); Implementation Approach (Prospector/analyzer/defender/reactor; Project vs. strategy dominant vs. beginner); QI Maturity: QI Scope (Depth of Implementation—total volume of QI activities, use of outcomes and patient satisfaction studies); Physician Involvement in QI (Physician involvement in CQI); Financial Health (Hospital Efficiency—LOS, cost per admission, labor productivity)
Chan and Ho 1997	TQM/CQI	Adoption of TQM Perception of Success or improvement	Organization	Size (Number of hospital beds; Number of employees); Location (Country of Origin—U.S. vs. Canada); Teaching Status; QI Maturity (Rank of organization in CQI—beginner, novice, intermediate, advanced); QI Maturity: QI Scope (Number of CQI projects undertaken); QI Maturity: Years Involved in QI (Time involved in TQM/CQI)
Douglas and Judge Jr. 2001	TQM/CQI	Perception of success or improvement	Environment	Competition (Herfindahl index); Other (Market growth rate)
		Superior organizational performance/quality	Organization	Ownership (For-profit ownership); Size (Number of hospital beds); Structure: General (Structural control; Structural exploration)
Hibbard, Stockard, and Tusler 2003	Self-labeled QI	Extent of implementation of QI activities	Environment	Public Reporting (Public reporting for quality measures)
LeBrasseur, Whissell, and Ojha 2002	TQM/CQI	Perception of success or improvement	Organization	QI Leadership: Strategic Planning (Vision, strategy, and CQI Linkages); QI Leadership: Top Management (Senior management communicates new expectations; Top-down planning and implementation; Senior management know/recognize the CQI paradigm); QI Leadership: Middle Management (Middle management does not take the lead); QI Maturity: Years Involved in QI (Duration of QI program; Sustained momentum); QI Maturity: QI Scope (Careful departmental implementation); Customer Focus (Perception of patient service expectation); Physician Involvement in QI (Involvement of medical professionals/physicians); Supplier Relationships (Structured relationships with suppliers); Other (Team skills and momentum; Teamwork and employee participation)
			Support/Capacity	QI Workforce Focus: Quality Training (Resources for implementation of training and education)
Lee et al. 2002	TQM/CQI	Extent of implementation of QI activities	Organization	Size (Number of hospital beds); Culture (Group/developmental culture score; Employee empowerment); Implementation Approach (Prospector/analyzer/defender); Other (Degree of system-focus; Use of scientific and systematic problem solving)
			Support/Capacity	Resources: Funding for QI (Independent budget for CQI); Data Infrastructure: Information System (Sophistication of information system); QI Workforce Focus: Quality Training (Offers CQI training program); Infrastructure for QI (Independent CQI department; Full-time CQI staff)
Lemieux-Charles et al. 2002	Self-labeled QI	Perception of success or improvement	QI Team	Support (Level and type of support from sponsor and organization); Group Climate (Strong norms of behavior on team); Group Process (Team follows good group processes; Team has well-designed decision-making practices)
Li 1997	Other: Quality Management	Perception of success or improvement	Organization	QI Leadership: Top Management (Top management leadership); Structure: Clinical Integration (Organizational cooperation across departments); Other (Technology leadership)
			Support/Capacity	Data Infrastructure (Information analysis for CI); QI Workforce Focus (Workforce development)
Lucas et al. 2005	TQM/CQI	Adoption of TQM	Environment	Competition (Perceived intensity of market competition, number of facilities directly competing for residents in local market); Managed Care Penetration (Percentage resident's care paid on capitated or negotiated rate); Medicare/Medicaid Influence (Proportion Medicare beds to total beds); Regulation (External CQI objectives: conformance with standards and requirements)
			Organization	Size (Number of beds); QI Leadership: Board Leadership (Leadership commitment to CQI—board requires reports, results, and initiated activities); QI Leadership: Top Management (Senior management facilitators—governance support, less administrative turnover, more administrative time, resources, and physician support); QI Leadership: Middle Management (Implementation commitment of middle management); Motivation for Implementing QI (Improve financial position; Improve quality of life/care; Improve employee relations)
			Support/Capacity	Resources: Time (Time—extent fewer demands, prioritization of CQI projects and time for staff training help CQI or TQM activities); QI Workforce Focus: Quality Training (Management training in CQI principles and methods); Infrastructure for QI (Organizational structural QI facilitators)
Meyer and Collier 2001	Self-labeled QI	Perception of success or improvement	Organization	QI Leadership: Strategic Planning (Strategic planning); QI Leadership: Top Management (Leadership); Process Management
			Support/Capacity	Data Infrastructure (Information and analysis); QI Workforce Focus (HR development and management)
Parker et al. 1999	Self-labeled QI	Extent of implementation of QI activities	Environment	Union Involvement (Union resistance to QI practices)
			Organization	Size (Hospital size); Location; Teaching Status (Medical school affiliation); Service Mix (Service orientation—acute vs. long-term); QI Leadership: Top Management (Top management commitment); QI Maturity: QI Integrated with QA (QI practices integrated with existing QA programs); Culture (Group/developmental/rational/hierarchical); Physician Involvement in QI (Medical staff involvement in QI activities); Innovativeness (Innovation experience)
			Support/Capacity	QI Consultants (Role of external consultants in QI activities)
Rad 2006	TQM/CQI	Perception of success or improvement	Organization	Culture (Entrepreneurship/creativity; Risk taking; Uncertainty avoidance/Stability; Power distance; Attention to details; Individualism-collectivism; Masculinity-femininity; Mechanistic-organic structure); Structure: General (Mechanistic-organic structure)
Rondeau and Wagar 2002	TQM/CQI	Perception of success or improvement	Organization	Culture (Learning-oriented values); QI Maturity: QI Intensity (TQM/CQI diffusion strength)
Shortell et al. 1995	TQM/CQI	Extent of implementation of QI activities	Organization	Size (Number of hospital beds); Culture (Group/developmental/rational/hierarchical); Implementation Approach (Prospector/analyzer/defender/reactor); Other (Formal involvement in TQM/CQI)
Wakefield et al. 2001	TQM/CQI	Extent of implementation of QI activities Perception of success or improvement	Organization	Culture (Group/developmental/rational/hierarchical)
Weech-Maldonado, Zinn, and Brannon 1999	TQM/CQI	Perception of success or improvement Adoption of TQM	Environment Organization	Competition (Perceived market competition) Size (Number of beds); QI Leadership: Board Leadership (Board involvement in QI—quality monitoring; Board involvement in QI—board activity)
Weech-Maldonado, Zinn, and Hamilton 2001	TQM/CQI	Perception of success or improvement	Organization	Size (Number of beds); QI Leadership: Board Leadership (Board initiates QI activities); QI Maturity: QI Scope (Degree of TQM implementation); QI Maturity: Years Involved in QI (Number of months since TQM adoption); Other (Output control systems—benchmarking, project teams required to report QI results, board involvement in QI)
			Support/Capacity	QI Workforce Focus: Rewards/Incentives (TQM criteria incorporated into reward and performance appraisal system); Infrastructure for QI (Individual with TQM role; Quality steering council)
Weiner, Alexander, and Shortell 1996	TQM/CQI	Adoption of TQM	Environment Organization	Competition (Market concentration—Herfindahl index); Regulation (Regulatory climate; Certificate of need stringency scale; Rate review stringency scale) Ownership (Private ownership), Size (Number of beds set up and staffed to use); Affiliation (Owned, leased, or sponsored by multihospital system); Teaching Status (Member of Council of Teaching Hospitals); Financial Health (Hospital cash flow; Adjusted admissions); QI Leadership: Top Management (CEO involvement in CQI/TQM—number of CQI/TQM activities in which CEO personally involved);*QI Leadership: Governing Board Structure (Board size; Management involvement in governance—formal CEO role on the board, CEO tenure, number of directors who are managers; Physician involvement in governance—active staff vs. physicians at large); QI Leadership: Board Leadership (Board quality monitoring—number of quality-related reports received by board; Board activity in quality improvement—number of QI actions taken by board);*QI Maturity: Years Involved in QI (Number of years involved in CQI/TQM)*
Weiner, Shortell, and Alexander 1997	TQM/CQI	Extent of implementation of QI practices	Environment Organization	Competition (Herfindahl index); Managed Care Penetration (Managed care/HMO penetration) Ownership (Private, nonprofit vs. investor owned); Size (Number of hospital beds); Affiliation (Multihospital system membership); Teaching Status (Member of Council of Teaching Hospitals); Financial Health (Hospital performance—hospital cash flow; adjusted admissions); QI Leadership: Top Management (CEO participation in CQI/TQM); QI Leadership: Governing Board Structure (Physician involvement in governance—active staff vs. at-large; QI Leadership: Board Leadership (Board quality monitoring; board activity in QI); QI Maturity: Years Involved in QI (Time involved in TQM/CQI)
Weiner et al. 2006a	Self-labeled QI	Superior organizational performance/quality	Environment	Competition (Market concentration; Number of hospital competitors; Hospital competition intensity); Managed Care Penetration (Percentage patients in managed care; Managed care penetration)
		Extent of implementation of QI activities	Organization	Ownership (HMO ownership; PPO ownership; Indemnity ownership; Public, nonfederal ownership; For-profit ownership); Size (Number of hospital beds, Number of hospital beds squared); Affiliation (System or network affiliated); Teaching Status (Teaching hospital); Service Mix (Total outpatient visits per bed; Ratio of outpatient/inpatient services); Volume (Number of inpatient surgeries); Financial Health (Hospital profitability—2-year average cash flow); Physician Arrangements (Number of physician arrangements); QI Leadership: Top Management (CEO participation in QI activities); QI Leadership: Governing Board Structure (Presence of governing board); QI Leadership: Board Leadership (Board monitoring of QI; Board activity in QI); QI Maturity: QI Scope (Diffusion of QI across hospital units; Total number of FTEs on QI teams; Number of managers on QI teams; Number of physicians on QI teams); QI Maturity: Years Involved in QI (Years of formal involvement in QI); Structure: Clinical Integration (Clinical integration); Barriers to Implementing QI (Perceived barriers to QI)
			Support/Capacity	Resources: Funding for QI (Total expenses per bed for QI); Data Infrastructure: Information System (Integrated data systems; Perceived clinical IS capabilities)
			QI Team	Physician Involvement on Team (Percentage physicians on QI teams)
Weiner et al. 2006b	Self-labeled QI	Superior organizational performance/quality	Environment	Accreditation (Perceived influence of FAACT, JCAHO, NCQA); Competition (Market concentration; Number of hospital competitors; Hospital competition intensity); Managed Care Penetration (Managed care penetration; Percentage patients in managed care)
		Extent of implementation of QI activities	Organization	Ownership (HMO ownership; PPO ownership; Indemnity ownership; Public, nonfederal ownership; For-profit ownership); Size (Number of hospital beds, hospital beds squared); Affiliation (System or network affiliated); Teaching Status (Teaching hospital status); Service Mix (Total outpatient visits; Ratio of outpatient/inpatient services); Volume (Inpatient surgeries); Financial Health (Hospital Profitability—2-year average cash flow); Physician Arrangements (Number of physician arrangements); QI Leadership: Governing Board Structure (Presence of governing board); QI Leadership: Board Leadership (Board activity in QI; Board monitoring of QI); QI Leadership: Top Management (CEO participation in QI);
				QI Maturity: QI Scope (Involvement of hospital units in QI; Percentage FTEs on QI teams; Percentage managers on QI teams; Percentage physicians on QI teams)*; QI Maturity: Years Involved in QI (Years of formal involvement in QI); Barriers to Implementing QI (Perceived barriers to QI); Structure: Clinical Integration (Clinical integration); Other (RNs per inpatient day)
			Support/Capacity	Data Infrastructure: Information System (Integrated database; Perceived clinical IS capabilities); Resources: Funding for QI (Total expenses on QI)
			QI Team	Physician Involvement on Team (Percent of physicians on QI teams)
Young, Charns, and Shortell 2001	TQM/CQI	Adoption of TQM	Environment	Union Involvement (Union opposition); TQM adoption by other Hospitals (TQM adoption by hospitals with similar service complexity; TQM adoption by hospital in patient referral network; TQM adoption at local private-sector hospitals)
			Organization	Size (Number of employees); QI Leadership: Manager Characteristics (Manager—age, tenure, education, prior TQM exposure)
Zinn, Brannon, and Weech-Maldonado 1997	TQM/CQI	Adoption of TQM	Environment	Competition (Number of facilities in direct competition; Perceived competition); Managed Care Penetration (Percentage revenue generated by managed care); Medicare/Medicaid Influence (Percentage Medicaid in total census; Percent Medicare in total census); Regulation (Meet JCAHO requirements)
	“Other QI”		Organization	Ownership (For-profit ownership); Affiliation (Part of multi-facility organization); Size (Average number of beds); QI Leadership: Top Management (Senior management facilitators—governance support, less administrative turnover, more administrative time, resources, and physician support); QI Leadership: Board leadership (Government support); QI Leadership: Middle Management (Middle management support and knowledge of QI); Motivation for Implementing QI (Improve financial position; Improve quality of life/care; Improve employee relations; Meet JCAHO requirements); Physician Involvement in QI (More physician support)
			Support/Capacity	Resources (More QI activity resources); Resources: Time (Time— fewer demands, prioritization of CQI projects, and time for staff training and help CQI or TQM activities); QI Workforce Focus: Quality Training (Employee training); QI consultants (Consulting support); Data Infrastructure: Information Systems (Information systems); Infrastructure for QI (Organizational structural QI facilitators)
Zinn, Weech-Maldonado, and Brannon 1998	TQM/CQI	Adoption of TQM	Environment	Competition (Herfindahl index; Perceived competition; Excess capacity; Availability of hospital-based substitutes); Managed Care Penetration (HMO membership); Medicare/Medicaid Influence (Medicare market penetration; Proportion Medicare); Other (Log per capita income in county)
			Organization	Size (Number of beds)

Notes: Articles may have measured a given contextual factor but may not have tested its association with QI success.

^*Authors used these factors as outcomes (dependent variables) only.

Abbreviations: CEO: Chief Executive Officer, CQI: Continuous Quality Improvement, CI: Continuous Improvement, FAACT: Foundation for Accountability, FTE: Full Time Equivalent, HMO: Health Maintenance Organization, HR: Human Resources, IS: Information Systems, JCAHO: Joint Commission on the Accreditation of Healthcare Organizations, LOS: Length of Stay, NCQA: National Committee of Quality Assurance, PPO: Preferred Provider Organization, QA: Quality Assurance, QI: Quality Improvement, RN: Registered Nurse, TQI: Total Quality Improvement, TQM: Total Quality Management.

Table S3.

Characteristics of Articles Examining Context in Specific Quality Improvement Initiatives

Source	QI Framework	Dependent Variable (QI Success)	Independent Variable (Context)
			System Level	Contextual Factor (measures)
Berlowitz et al. 2003	Self-labeled QI	Extent of implementation of QI practices	Organization	Size (Number of beds); Location (Urban/rural; Census region—East, South, Midwest, West); Teaching Status (Membership on Council of Teaching Hospitals); Culture (Group/developmental/rational/hierarchical)
Berner et al. 2003	TQM/CQI Self-labeled QI	Pre/Post measure of system and/or process changes	Microsystem	Champion (Opinion leader)
Bradley et al. 2005	Self-labeled QI	Superior organizational performance/quality	Organization	Ownership (Government vs. nonprofit vs. for-profit); Location (Urban vs. nonurban; Census region); Teaching Status; Volume (Clinical volume of condition [e.g., AMI]); QI Leadership: Top Management (Support from administration); Culture (Overall index of organizational culture; Decision making is participatory, not top down; Ease of coordination across departments; Change takes place too slowly; Hospital tried new activities only after others have found them successful; Hospital is likely to be first to try new activity; Hospital tends to assign blame); Physician Involvement in QI (Support from physicians; Physician participation in QI); QI Leadership:
				Organizational Support (Overall index of organizational support; Support from nurses; Support from physicians; Support from administration; Resource availability; Physician participation in QI)
			Support/Capacity	Resources (Resource availability); Data Infrastructure: Data Feedback (Has data on beta-blocker use; generates data reports at least quarterly; Discusses data feedback in staff meetings; Displays data feedback publicly for staff; Data reports reflect recent cases; Data feedback is physician specific)
			Microsystem	Physician Leadership (Physician leadership for AMI)
Glickman et al. 2007	Self-labeled QI	Pre/Post outcome change	Organization	QI Leadership: Organizational Support (Administrative commitment to quality); Culture (Employee satisfaction; Barrier to QI-cultural and physician resistance to change); Barriers to Implementing QI (Barriers to QI—nondiagnostic; Barrier to QI—operational ineffectiveness)
			Support/Capacity	Resources: Funding for QI (Financial commitment to quality); Resources (Barrier to QI—resource availability); Data Infrastructure: Data Feedback (Forum of data feedback; Extent of data feedback)
			Microsystem	Champion (Advocate clinical commitment to quality)
			Miscellaneous	CRUSADE QI tools
Higgins and Routhieaux 1999	Self-labeled QI	Perception of success or improvement	QI Team	Physician Involvement on Team (Team member position within organization—team membership included physicians and clinical staff); Group Process (Communication of team plans; Specific skill levels of team leaders or team members—negotiation, conflict resolution; Team dynamics); Team Leadership (Perception of team leader performance); Prior Experience Together (Prior experience of team leaders/members with team activities); Team QI Skills (Specific skill levels of team leaders or team members—computer use, data collection and analysis, SPC, QI techniques and tools); Other (Meeting frequency; Hours per week on team; Willingness to serve on team; Selection method for team—volunteer vs. manager request; Perception of facilitator performance)
Lammers et al. 1996	Self-labeled QI	Perception of success or improvement	Organization	Size (Number of FTEs); QI Leadership: Organizational Support (Commitment to TQI principles throughout organization); QI Leadership: Top Management (Management commitment to TQI principles); Physician Involvement in QI (Physician leadership commitment to TQI principles; Physician commitment to TQI principles); QI Maturity: QI Scope (Number of teams formed); Other (Employee commitment to TQI principles)
			Support/Capacity	Resources: Funding for QI (Quality council TQI budget); QI Workforce Focus: Quality Training (Quality council involvement in training); Infrastructure for QI (Quality council presence; Quality council meeting frequency; Age of quality council; Quality council planning involvement)
			QI Team	Support (Team involvement scale—quality council); Other (Teams progressed to point of data collection)
Litaker et al. 2008	Self-labeled QI	Pre/Post outcome changes	Microsystem	Capability for Change (Practice innate ability to change); Motivation to Change (Practice internal motivation to change); Other (Capacity for change (motivation + ability))
Mehrotra et al. 2007	Self-labeled QI	Extent of implementation of QI practices	Environment	Managed Care Penetration (>25% of group commercial plan revenue is capitated); Pay for Performance (P4P incentive tied to HEDIS measure; P4P tied to any utilization measure)
			Organization	Size (Physician group size >39); Affiliation (Affiliation with physician network); Physician Arrangements (Group employs a majority of physicians); Other (Majority of physicians use EMR; Majority of physicians are specialists)
Mills et al. 2003*	BTS Collaborative	Other: Combination of pre-post measure of outcome changes and active change participation	QI Team Microsystem	Team Leadership (Strong team leadership); Group Process (Teams can solve conflicts); Other (Use information from other teams; Early reports turned in) Motivation to Change (Front-line staff support)
Mills and Weeks 2004	BTS Collaborative	Pre/Post outcome changes	Organization	QI Leadership: Top Management (Mandate from senior leaders); Customer Focus (Gather data from patients); Other (Nonpunitive methods for investigating errors)
			Support/Capacity	Resources (Sufficient resources to achieve aims); Resources: Time (Sufficient resources to achieve aims). Data Infrastructure: Information System (Useful information systems)
			QI Team	Physician Involvement on Team (Active physician participant); Prior Experience with QI (Prior work on improvement projects); Prior Experience Working Together (Team members worked as a team before); Team QI Skills (Familiar with measurement; First test of change completed on time;
				Learned new methods during the project); Team Leadership (Strong team leadership with clout); Group Climate (Team members understand one another's strengths and weaknesses; Team members can express opinions; Mutual respect among team members); Group Process (Shared vision of team goals; Teams can solve conflicts); Other (Systemic perspective on problems)
			Microsystem	Motivation to Change (Front-line staff support)
			Miscellaneous	Strategic Importance to Organization (Work perceived as part of organization's strategic goals); Other (Project added value; Shared information with other teams; Used information from other teams; Learned new ideas; Specific plan to disseminate information
Molfenter et al. 2005	BTS Collaborative	Perception of success or improvement	Environment Organization	Other (Source of ideas from outside the organization; External influence) QI Leadership: Top Management (Mandate from senior leaders; Leader goals, involvement, and support); QI Leadership: Middle Management (Middle manager goals, involvement, and support); Customer Focus (Exploration of problem and customer needs)
			Support/Capacity	Resources: Funding for QI (Funding); Data Infrastructure: Data Feedback (Monitoring and feedback); QI Workforce Focus (Work environment—role, structure, incentives, staffing, support change)
			Microsystem	Champion (Clinical champion, agent prestige, commitment, and customer focus); Motivation to Change (Tension for change; Advantages to staff and customers; Staff changes required; Staff needs assessment, involvement, and support)
Neily et al. 2004	BTS Collaborative	Pre/Post outcome changes	Miscellaneous	Listserv Use
Neily et al. 2005	BTS Collaborative	Perception of success or improvement	Support/Capacity	Data Infrastructure: Information System (Useful information systems)
			QI Team	Prior Experience with QI (Prior experience with QI and teamwork); Team QI Skills (Teamwork skills; Skills gained in the project); Support (Leadership support)
Nembhard and Edmondson 2006	Other: Non-BTS Collaborative- QI teams, PDSA	Extent of implementation of QI practices	Microsystem	Culture/Climate (Unit psychological safety); Physician Leadership (Leader inclusiveness); Other (Years working in current NICU; Years working in any NICU; Years as an employee of the hospital; Hours per week in the NICU)
Palmer et al. 1996	Self-labeled QI	Pre/Post outcome change	Microsystem	Physician Leadership (Physician leadership—leader's own improvement; Physician leadership—leader commitment index)
Shortell et al. 2004	BTS Collaborative	Extent of implementation of QI activities	Organization	QI Leadership: Organizational Support (Organization QI commitment—focus on patient satisfaction); Culture (Group/developmental/rational/hierarchical; Cultural balance); Customer Focus (Patient satisfaction focus)
		Perception of success or improvement	QI Team	Physician Involvement on Team (Team composition—percentage physicians on team); Team QI Skills (Perceived team skill and autonomy)†; Support (Perceived organizational support)†; Group Process (Participative norms and goal agreement)†; Other (Team size; Team size squared; Overall perceived team effectiveness)†;
			Microsystem	Champion (Team champion—nurse or physician)
Stevenson et al. 2001	Other: multi-practice collaborative (QI activities)	Pre/Post outcome change	Microsystem	Champion (Degree of personal involvement in audit—lead GP and lead nurse); Capability for Change (Recognizing need to overcome obstacles—development of systematic plans to implement change); Motivation to Change (Attitude to audit; Personal interest in audited disease); Other (Willingness to audit again; Degree of teamwork in audit)
Tucker, Nembhard, and Edmondson 2007	Other: Non-BTS Collaborative- QI teams, PDSA	Perception of success or improvement	QI Team Microsystem	Team QI Skills (Learn how activities; Learn what activities) Culture/Climate (Unit psychological safety); Other (Average years worked in unit; Average hours worked per week)
Weeks et al. 2003*	BTS Collaborative	Pre/Post outcome change	Organization	QI Leadership: Top Management (Mandate from senior leaders)
			Support/Capacity	Resources (Sufficient resources)
			QI Team	Team Leadership (Strong team leadership)
			Microsystem	Motivation to Change (Front-line staff support)
			Miscellaneous	Strategic Importance to Organization (Work perceived as part of organization's strategic goals)
Wilkens and London 2006	TQM/CQI	Perception of success or improvement	QI Team	Team Leadership (Extent to which facilitator structured group process); Group Climate (Self-disclosure; Group learning orientation; Psychological safety); Group Process (Perceptions of feedback seeking/giving; Task and relationship conflict; Creativity of outcomes); Other (Group performance)

Notes: Articles may have measured a given contextual factor but may not have tested its association with QI success.

^*Listed only those factors clearly examined for the relationship with QI success; others may have been examined but report is unclear.

^†Authors used these factors as outcomes (dependent variables) only.

Abbreviations: AMI: Acute Myocardial Infarction, BTS: Break Through Series, CQI: Continuous Quality Improvement, CRUSADE: Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes with Early Implementation of the American College of Cardiology/American Heart Association Guidelines, EMR: Electronic Medical Record, FTE: Full Time Equivalent, GP: General Practitioner, HEDIS: Health Employer Data and Information Set, NICU: Neonatal Intensive Care Unit, P4P: Pay for Performance, QI: Quality Improvement, SPC: Statistical Process Control, TQI: Total Quality Improvement, TQM: Total Quality Management.

Figure S1.

Literature Search and Study Selection Process

Note:*Duplicates excluded at any stage. †Editorials, magazines, book reviews, commentaries, interviews, newspaper articles.