Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 May 6.
Published in final edited form as: Med Care Res Rev. 2014 May 29;71(5):450–471. doi: 10.1177/1077558714536619

Deliberate Learning in Health Care: The Effect of Importing Best Practices and Creative Problem Solving on Hospital Performance Improvement

Ingrid M Nembhard 1,2, Praseetha Cherian 1, Elizabeth H Bradley 1
PMCID: PMC4422484  NIHMSID: NIHMS684241  PMID: 24876100

Abstract

This article examines the effect on quality improvement of two common but distinct approaches to organizational learning: importing best practices (an externally oriented approach rooted in learning by imitating others’ best practices) and internal creative problem solving (an internally oriented approach rooted in learning by experimenting with self-generated solutions). We propose that independent and interaction effects of these approaches depend on where organizations are in their improvement journey – initial push or later phase. We examine this contingency in hospitals focused on improving treatment time for patients with heart attacks. Our results show that importing best practices helps hospitals achieve initial phase but not later phase improvement. Once hospitals enter the later phase of their efforts, however, significant improvement requires creative problem solving as well. Together, our results suggest that importing best practices delivers greater short-term improvement, but continued improvement depends on creative problem solving.

Keywords: organizational learning, best practice transfer, problem solving, quality improvement, hospitals

Deliberate organizational learning is a necessity for an effective health care system (Institute of Medicine, 2001). Deliberate learning refers to the process of actively improving organizational actions by integrating new knowledge and insights; it is a function of organizations’ concerted efforts to create, acquire, or transfer knowledge (Garvin, 2000; Zollo & Winter, 2002). The imperative for such learning in health care is rooted in at least three realities. First, health care reform, which has brought new organizational forms such as accountable care organizations, means that organizations must learn new ways of operating. Second, continuous advances in science and technology means that there is always new knowledge to be absorbed. Third, widespread problems, resulting in millions of preventable patient injuries and deaths (Institute of Medicine, 2010), mean that organizations are faced with the challenge of learning to close performance gaps. These circumstances and others have led to much interest in deliberate learning. Scholars now regard it as a key evaluation metric (Shortell et al., 2005) and develop surveys to assess the extent to which health care organizations are learning organizations (e.g., Singer, Moore, Meterko, & Williams, 2012).

To date, research on deliberate learning in health care, mirroring general management research, has largely focused on four topics: (a) the characteristics of organizations that enable or limit learning (Tucker & Edmondson, 2003); (b) which activities organizations use most (Cohen et al., 2008); (c) how performance is affected by the type of knowledge produced by deliberate efforts for example, learn-what or learn-how (Tucker, Nembhard, & Edmondson, 2007); and (d) activities that support learning such as collaboratives (Nembhard, 2009) and training (McAlearney, Robbins, Kowalczyk, Chisolm, & Song, 2012). This literature has helped us realize that different approaches to deliberate learning may promote performance improvement in diverse ways through different mechanisms.

Given that different approaches are possible, the question naturally arises as to when each approach is most beneficial. Improvement efforts generally have two phases: an initial push when organizations are launching (or renewing) their improvement effort with the intent of achieving significant improvement relative to current performance, and a later phase when organizations are nearing the performance goal and/or improving past it. The unresolved question is: are some approaches more helpful during the initial push, while others more helpful during the later phase, and are there tradeoffs or synergies between approaches depending on where organizations are in their improvement effort? These are central questions because many organizations face resource constraints and the need to optimize learning investments to achieve both immediate and later goals. The benefit of learning approaches and combinations thereof likely varies across phases because improvement needs change over time. However, research has not assessed the effectiveness of learning approaches at different stages of improvement. Without examination of the potential for different effects and for synergies or antagonisms among approaches, it remains unknown how best to use learning activities to facilitate organizations’ ability to achieve and sustain high performance.

New Contribution

The aim of this article is to advance our understanding of the effects of different approaches to deliberate learning in health care by investigating two common yet distinct approaches: importing best practices (an externally oriented approach rooted in learning by imitating others’ efficacious practices) and internal creative problem solving (an internally oriented approach rooted in learning by investigating one's own performance gaps and experimenting with self-generated solutions). Both approaches are presented as improvement options in the organizational literature (e.g., Adler et al., 2003) but have not been studied simultaneously. We introduce theoretically derived hypotheses about their main and interaction effects. Furthermore, we consider their effects on performance at different stages of improvement. Historically, the quality improvement literature has treated improvement as a single event, neglecting that it is a phased process for most and thus effects are likely to vary. We empirically examine the phase-related effects in nearly 500 U.S. hospitals pursuing improvement.

Our work not only addresses theoretical ambiguity about the individual and combined effects of different learning approaches on performance at different points in the improvement journey, but also provides empirical insight on organizational learning at a time when this information is most needed. Currently, millions of patients suffer preventable injury or death as a result of quality problems (Jha et al., 2013). Efforts to improve care have tended to focus on programs that teach methods for problem solving or enrollment in initiatives aimed at importing best practices. Previous studies have not assessed the comparative benefits of these approaches or their interaction, leaving the industry in debate about the most effective way to drive improvement (Davidoff, 2012). Thus, an additional contribution of this study is addressing a central issue in health care management.

Conceptual Framework: How Do Organizations Learn and Improve Performance?

The literature on deliberate learning suggests two prominent pathways by which organizations might learn and improve their performance—by importing best practices or creating their own.

The idea that organizations learn and pursue performance improvement by importing best practices has been discussed in the organizational learning literature on knowledge transfer, best practice transfer, and external learning. These literatures define organizations that import best practices as those that deliberately implement practices (i.e., discrete routines) that have been shown in a body of evidence to yield desired outcomes in other organizations (O'Dell & Grayson, 1998; Szulanski, 1996). Best practices can vary from guidelines for individual workers (e.g., hand washing by health professionals to reduce health care–associated infections) to policies for entire organizational systems (e.g., data reporting practices to improve safety; Berta & Baker, 2004). Regardless of the nature of the practice, the process of importing best practices involves learning from external sources through the “intentional transfer and replication of [their] knowledge” (Berta & Baker, 2004). Thus, it can be described as learning by imitation (Haunschild & Miner, 1997), the success of which is believed to hinge on organizations’ absorptive capacity (Szulanski, 1996), that is, the ability to identify, assimilate, and apply the practice (Cohen & Levinthal, 1990).

In many respects, creative problem solving contrasts with importing best practices. Table 1 presents the multiple dimensions on which the two differ. In creative problem solving, like all problem-solving processes, organizations progress from problem identification to root cause analysis to solution development and implementation (March & Simon, 1958). However, the goal of this specific process is the internal generation of novel, effective solutions via staff's in-depth investigation of the problem and trials of possible solutions (Mumford, Whetzel, & Reiter-Palmon, 1997). Thus, creative problem solving is a form of learning by internal investigation and experimentation. This learning benefits from absorptive capacity similar to most forms of learning, but benefits more from creative capacity, that is, the ability to generate novel and useful ideas (Carmeli, Gelbard, & Reiter-Palmon, 2013). Related to this observation, research shows that the success of problem-solving efforts is linked to the organization's ability to draw on the knowledge of diverse individuals (Terwiesch & Xu, 2008). Additionally, systematic problem-oriented investigation is thought to yield sizeable gains because it draws on the richness of learning from failure, which can be more informative than learning from success (Madsen & Desai, 2010).

Table 1.

Comparison of Two Deliberate Learning Approaches.

Importing best practices Internal creative problem solving
Learning strategy Learning by imitation Learning by investigation and experimentation
Description of process Process involving implementation of an evidence-based practice into a new setting Process involving problem identification, root cause analysis, solution development, and implementation
Knowledge source or origin External: Exogenous learning from information and benefits acquired from external sources Internal: Endogenous learning from employee learning within the organization
Core task Knowledge application Knowledge creation or generation
Core capacity required Absorptive capacity: ability to identify, assimilate, and apply the practice Creative capacity: ability to generate novel, useful solutions
Mechanisms of influence on performance Practices' effectiveness should transfer Organizational commitment motivates staff
Efficacy of practices motivates staff Structured method facilitates learning
Efficiency in gaining knowledge Greater depth of learning from failure
Open in a new tab

Hypotheses About the Impact of Importing Versus Creating During the Improvement Journey

We hypothesize that importing best practices is associated with greater performance improvement during the initial push than during the later phase. Great gains should follow the importation of these practices relatively quickly because these practices are the “best,” an evidence-based “working cure” (Kim & Miner, 2007). However, after the initial improvement, further gains likely require additional innovations. For instance, establishing a single call system (an evidence-based “best practice”) to facilitate timely arrival of on-call staff to begin a procedure likely results in a one-time increase in timeliness of care; added reductions in delays would require additional shifts in practices. Furthermore, the value of best practices can depreciate over time due to advances in knowledge. Annually more than 700,000 references are added to Medline, the bibliographic database for the medical industry (NIH, 2012). In such “high velocity” knowledge settings, knowledge, including best practices, declines in applicability as time passes (Eisenhardt & Martin, 2000). Best practices’ ability to facilitate improvement in later phases may decline because the context in which they operate and/or interdependent practices evolve in ways that diminish their value. Case in point, hospital regional systems have emerged to coordinate cardiac care among hospitals in ways not envisioned when best practices for treating patients with heart attacks were established in 2006. Thus:

Hypothesis 1

Importing best practices is positively associated with performance improvement during the initial phase (but not with improvement during the later phase).

We propose that the situation differs for internal creative problem solving, with it being significantly associated with later phase improvement but not with initial phase improvement because it is an exploratory process. Conducting root causes analysis and iterating through created solutions systematically until the optimal one is found can take time. Research shows that creative endeavors rarely result in an immediately valuable product or service (Amabile, Hadley, & Kramer, 2002); instead, they often take a “jagged course” prior to yielding useful output (Feinstein, 2006). Gains from creative processes may be particularly delayed due to “superstitious learning,” in which organizations derive lessons from early stages of learning that are seemingly compelling (e.g., a series of failures) but are arbitrary and not due to organizational action (Levitt & March, 1988). Reacting to superstitious learning, organizations often pursue unhelpful practices or abandon helpful practices prematurely—both of which can undermine improvement initially (Repenning & Sterman, 2002). However, with time's passing and trials of possible solutions, organizations are likely to learn what information is relevant for generating fruitful solutions, and thus develop progressively better solutions, leading to greater improvement in later phases. Thus:

Hypothesis 2

Creative problem solving is more positively associated with performance improvement during the later phase (but not with improvement during the initial phase).

In the scholarly literature, importing best practices and creative problem solving to create novel practices are often presented as an either–or choice for organizations. Adler et al. (2003), for example, in their article on performance improvement in hospitals, argued that “the innovations [that are the focus of improvement projects] can be generated within the organization or adopted from other sources” (p. 13). Similarly, Nickerson and Zenger (2004) noted that “the state of a firm's knowledge can be advanced by either absorbing existing knowledge external to the firm or by developing new knowledge by first identifying a problem and then discovering a valuable new solution” (p. 618). Although the literature suggests that organizations choose between approaches, a third option is for organizations to pursue both. We propose that pursuing both has different effects depending on whether the organizations are in the initial or later phase of the improvement journey.

In the initial phase, we expect that a negative interaction arises—that is, using both approaches together results in less performance improvement than would be predicted by the two approaches independently—because of tension between contrasting approaches. Scholars in other settings have found that organizations often struggle to manage different strategies and to integrate the knowledge acquired from different approaches. For example, Wong (2004), in a cross-sectional study of project teams in four firms, found that team efficiency suffered when teams combined learning from outside the team with learning within the team. In health care, introducing learning activities has also been associated with worse patient outcomes at first (Nembhard & Tucker, 2011). Scholars (e.g., Keating, Oliva, Repenning, Rockart, & Sterman, 1999) theorize that combining very different activities has a negative effect because integration is difficult and a tradeoff exists when resources are constrained as they are in health care. More learning activities means sacrificing resources for routine activities, and potentially undermining performance until organizations become skilled at managing both sets. Additionally, investing in one learning activity means less capacity for the other, limiting the gains from the other until resources can be shifted. This tradeoff likely exists for our focal approaches. Given resource constraints, initially improvement is likely to be less than the combined individual effects of both due to poor application of one or both. Furthermore, per Hypothesis 2, creative problem solving offers limited gains in the initial phase relative to the later phase. Thus, its potential to multiply the gains from importing practices is proposed to be limited in the shorter term. Initial failures that are part of creative problem solving are also likely to detract from the gains of importing. In sum, because of the difficulty managing different approaches at once, resource constraints, and lower effectiveness of creative problem solving initially, we hypothesize:

Hypothesis 3

Importing best practices and creative problem solving interact negatively with respect to performance improvement in the initial phase of the improvement journey.

In the longer term, however, organizations may positively benefit from using both approaches such that improvement is more than would be predicted by the two approaches independently, once organizations have the opportunity to become skilled at managing and integrating knowledge from both. The argument for eventual synergy rests on the theory of absorptive capacity. According to this theory, organizations’ ability to learn and apply knowledge is enabled by possession of related knowledge (Cohen & Levinthal, 1990). Building from this theory, creative problem solving is likely to help realize greater gains from importing best practices. It is well established in the literature that importing requires adapting best practices to fit the new context (Szulanski & Jensen, 2006; Tucker et al., 2007). It should ultimately be easier for organizations to adapt practices if they are also engaged in creative problem solving because problem solving provides related knowledge to leverage in the adaptation process. However, the greater gains from leveraging this knowledge are most likely to be observed in the later phase of improvement because it takes time to derive gains from creative problem solving (per Hypothesis 2).

Creative problem solving may not only increase the impact of importing best practices but also importing best practices might amplify the impact of creative problem solving on performance. Schwab (2007) proposed that, although most literature on absorptive capacity emphasizes the benefits of internal knowledge for absorbing external knowledge (e.g., best practices), external knowledge may facilitate internal knowledge processes. Applying this idea, learning from the iterative trials that are part of internal creative problem solving may benefit from the knowledge and skill developed through importing best practices (an external knowledge source). As noted above, successful importing requires organizations to adapt practices to their context. Having adaptive skill and contextual understanding is likely to help organizations as they engage creative problem solving. Thus, our last hypothesis is the following:

Hypothesis 4

Importing best practices and creative problem solving have a positive interaction with respect to performance improvement in the later phase of improvement.

Method

Research Setting and Design

We tested our hypotheses in a longitudinal study of U.S. hospitals’ performance improvements with respect to an established measure of quality of care: door-to-balloon time for patients with ST-segment elevation myocardial infarction (STEMI), a common and often lethal type of heart attack. STEMI affects between 400,000 and 500,000 people in the United States annually (Antman et al., 2004) and is treated via a procedure called percutaneous coronary intervention during which clinicians clear blockage in the coronary arteries that feed the heart via inflation of a small balloon or other intervention. Timeliness of care is a critical element of the treatment. National guidelines stipulate that door-to-balloon time—the interval between patient arrival at the hospital and balloon inflation—should not exceed 90 minutes, as research shows that mortality increases by 6.3 deaths per 1,000 patients treated every 15 minutes thereafter (Nallamothu, Bradley, & Krumholz, 2007). Despite awareness that treatment times should be within 90 minutes, only about 40% of hospitals had a median door-to-balloon time of 90 minutes or less in 2005 (Bradley et al., 2006).

Troubled by the poor performance with respect to door-to-balloon time, in November 2006, the American College of Cardiology and partner organizations launched the D2B Alliance for Quality, a national campaign that promoted the importation of five evidence-based (“best”) practices for improving door-to-balloon time for patients with STEMI to meet the national guideline. The campaign offered tools (e.g., webinars) to help hospitals learn about the recommended practices. More than 1,000 of the 1,400 U.S. hospitals with percutaneous coronary intervention capability enrolled in the D2B Alliance. We examined our hypotheses using data from hospitals that enrolled in the D2B Alliance. This setting was ideal for our study because the initial push (marked by the campaign launch) versus later phase of improvement (after the campaign conclusion) were relatively well defined in this context. Also, the context allowed us to study learning effects relative to an objective goal (for treatment time).

Data Collection and Sample

Data from hospitals enrolled in the D2B Alliance were collected via two web-based surveys. The first survey, the D2B Alliance Baseline Survey, was sent to the 915 hospitals that enrolled in the D2B Alliance during its first 6 months (November 2006-May 2007). Of these hospitals, 797 (87%) completed the baseline survey, which asked about their use of Alliance-recommended practices. The surveys were completed at the time of enrollment by the hospitals’ primary contact for the D2B Alliance, most often the director of quality improvement or his/her designee. The second survey, the D2B Alliance Follow-up Survey, was administered in waves corresponding to the baseline survey, such that hospitals received the survey approximately 10 to 12 months after completing the baseline survey (between February 2008 and June 2008). The follow-up survey assessed hospitals’ use of D2B Alliance-recommended practices nearly 1 year later and their use of creative problem solving. A total of 715 of the 797 (89.7%) hospitals that completed the baseline survey completed the follow-up survey. Ninety-three percent of these hospitals completed the follow-up survey within 10 to 15 months of completing their baseline survey (median follow-up time = 12 months).

To obtain the door-to-balloon time performance of the 715 hospitals that completed the two surveys, we consulted the Hospital Quality Alliance (HQA). HQA publicly reports the percentage of hospitals’ door-to-balloon times that were within 90 minutes in year-long increments on the Hospital Compare website. These data are nationally standardized and verified through quarterly audits. From the database, we obtained the number of patients with STEMI that each hospital treated and hospitals’ door-to-balloon performance during three periods relevant to this study: (a) the 12-month period before the D2B Alliance began (January to December 2006), which served as our baseline assessment of performance; (b) the year following the conclusion of the D2B Alliance (July 2008 to June 2009), which served as our initial push follow-up period; and (c) the year in which most hospitals nationally had achieved greater than 90% reliable performance (January to December 2010), which served as our later phase follow-up period. For simplicity, we refer to the first time period as 2006, the second period as 2008, and the third as 2010.

We selected 2008 as our initial follow-up period because (a) prior research has shown that more than a year is required to detect the effects of interventions on core measures in hospitals (Nembhard & Tucker, 2011); (b) the D2B Alliance (which began in earnest in 2007) ended in 2008 making 2008 a natural period for first evaluation following the initial push (organizers expected sizable improvement by then); and (c) the use of earlier time periods would have resulted in the loss of 400 hospitals (43%) from our survey sample due to missing performance data from HQA. We used 2010 as the later phase follow-up period because we desired a meaningful endpoint for the improvement phase. By 2010, the U.S. hospital average percentage of patients treated within guideline was 92%, up from 44% in 2006 (Krumholz et al., 2011), suggesting that 2010 was an appropriate final year. After 2010, virtually all U.S. hospitals had shifted from improving to sustaining their gains.

After review of the HQA data that we obtained, we found that 190 of the 715 (26.6%) hospitals in our sample had to be excluded from our dataset for one of two reasons. The first reason was that the hospital did not satisfy our minimum volume criterion of treating 12 or more patients with STEMI per year in both 2006 and 2008 (N = 174). The second reason for exclusion was that the number of patients with STEMI treated at the hospital was missing from the HQA dataset (N = 16). We established the minimum volume criterion, as researchers using other metrics have done (e.g., Bradley et al., 2006) to capture the relevant sample, that is, hospitals with sufficient volume to be motivated to learn and improve.

Of the 525 hospitals that satisfied the requirements for inclusion in the dataset, we obtained complete information on key hospital characteristics for 504 of them from the American Hospital Association Annual Survey and Profiles of U.S. Hospitals. The former provided information about hospitals’ teaching status, size (number of beds), location, and ownership type (e.g., nonprofit), while the latter provided information on financial performance (i.e., net income). We used data for this sample of hospitals (N = 504) to test our hypotheses related to initial improvement. Table 2 shows the characteristics of the hospitals in our sample. In 2010, 480 of these 504 hospitals satisfied the criteria for inclusion in the HQA database and our minimum volume criterion of 12 patients. We tested our hypotheses about later phase, continued improvement with this sample. Six hospitals were ultimately dropped from the sample because they did not provide information for all measures.

Table 2.

Characteristics of Hospitals in Sample Versus Other Hospitals in the D2B Alliance.

Sample hospitals (N = 504) Other hospitals (N = 338) All hospitals (N = 842) Sample vs. other hospitals Sample vs. all hospitals
Characteristic n/Mean (%) n/Mean (%) n/Mean (%) p Value for comparison
Teaching hospital .017 .117
    Yes 248 (49%) 138 (41%) 386 (46%)
    No 256 (51%) 200 (59%) 456 (54%)
Hospital size (beds) .012 .148
    0-299 189 (38%) 154 (46%) 343 (41%)
    300-499 240 (48%) 126 (37%) 366 (44%)
    500+ 75 (15%) 58 (15%) 133 (16%)
Geographic location <.001 .014
    Region 1 74 (15%) 64 (19%) 138 (16%)
    Region 2 201 (40%) 81 (24%) 282 (34%)
    Region 3 137 27% 107 (32%) 244 (29%)
    Region 4 92 (18%) 86 (25%) 178 (21%)
Ownership type .010 .117
    Nonprofit 404 (80%) 244 (72%) 648 (77%)
    Government 41 (8%) 48 (14%) 89 (11%)
    For profit 59 (12%) 46 (14%) 105 (13%)
Net income (millions) 26.25 (44.75) 15.66 (29.45) 22.00 (39.65) <.001 .008
Patients with STEMI
    in 2008 50.09 (25.24) 44.67 (20.50) 47.91 (23.59) .001 .027
    in 2010 51.09 (24.80) 44.39 (21.92) 48.43 (24.24) <.001 .05
Baseline performance (2006) 0.62 (0.21) 0.57 (0.21) 0.60 (0.21) <.001 .09
Initial phase (2006-2008) improvement 0.25 (0.20) 0.27 (0.21) 0.25 (0.20) .20 1.00
Late phase (2008-2010) improvement 0.05 (0.10) 0.07 (0.12) 0.06 (0.11) .006 .07
Open in a new tab

Note. p Values for continuous variables are based on t test, while p values for categorical variables are based on chi-square tests. Samples included only hospitals that treated 12 or more patients in both the baseline and follow-up time periods. For longer-term improvement (4-year), the sample size of D2B hospitals is 480.

Measures

Importing Best Practices

In both of the D2B Alliance Surveys (baseline and follow-up), hospitals were asked whether they used each of five evidence-based practices that the D2B Alliance recommended for reducing door-to-balloon time: (a) activation of the catheterization laboratory by emergency medicine physicians without consulting cardiology; (b) activation of the catheterization laboratory with a single call (vs. multiple, individual pages); (c) requiring the entire catheterization team to arrive at the laboratory within 30 minutes of being paged; (d) prompt (within a week) feedback about door-to-balloon times to emergency department and catheterization laboratory staff; and (e) activation of the catheterization laboratory while the patient was en route to the hospital, based on a pre-hospital electrocardiogram. For each hospital, we calculated the sum of these practices that the hospitals reported using at baseline and follow-up. We then subtracted the baseline from the follow-up sum, and used this number as our measure of importing best practices.

Internal Creative Problem Solving

We assessed whether hospitals used creative problem solving to develop new practices internally by their responses to four questions in the D2B Alliance Follow-up Survey. The questions asked whether hospitals engaged in each step of the process during their project and emphasized the use of the process for internally generated solutions—as opposed to adapting imported practices. Specifically, the survey asked whether the hospital (a) used root cause analysis or some similar approach to investigate delays in door-to-balloon times when they occur (problem identification and root cause analysis); (b) came up with new ideas about how to reduce door-to-balloon time that were not part of the D2B Alliance recommendations (solution development); (c) experimented with ways to reduce door-to-balloon time that were not suggested by the D2B Alliance or its member hospitals (solution evaluation); and (d) worked on implementing strategies for reducing door-to-balloon time that were not recommended by the D2B Alliance (solution implementation). We summed positive responses across questions to create a score ranging from 0 (completed no steps in the process) to 4 (completed all steps). Because we and prior research conceptualize the process as requiring all four steps, we created an indicator variable to differentiate hospitals that fully adopted creative problem solving (1 = hospital completed all steps; 0 = otherwise). We used this measure in our analyses and used the continuous measure (0-4) in sensitivity analysis.

Initial Phase Improvement

As noted above, the HQA database reports the percentage of each hospital's patients whose door-to-balloon time satisfied the national guideline (i.e., within 90 minutes). For our measure of initial improvement, we used hospitals’ changes in door-to-balloon time performance (i.e., change in percentage of patients that received treatment within the national guideline of 90 minutes between hospital arrival and opening of the coronary artery) between our baseline and initial follow-up periods (2006 and 2008).

Later Phase Improvement

To compute this measure, we again calculated hospitals’ change in door-to-balloon time performance but defined the focal period as 2008 to 2010.

Covariates

We adjusted for hospital characteristics that health services research has identified as potential correlates of hospital performance: teaching status, hospital size, geographic location, ownership type, and financial health. Teaching hospitals were defined as those with a graduate medical training program according to the Accreditation Commission for Graduate Medical Education; we gave these hospitals a value of 1 for teaching status and 0 otherwise. We also used a series of dummy variables to indicate hospital size (<300 beds, 300-499 beds, or ≥500 beds), geographic location based on the U.S. Census Bureau's state classification system (Northeast, Midwest, South, or West), and hospital ownership (public/government, for-profit, or not-for-profit). Financial health was measured by net income at baseline, which we normalized by taking the natural log of values once positively transformed; the original distribution suffered from excess peakedness (kurtosis = 28).

Because statistical research on performance changes indicates that change correlates with initial status (Willett, 1988) and because we wished to assess added effects of our focal approaches, we adjusted for hospitals’ baseline (2006 for 2008 analysis and 2008 for 2010 analysis) door-to-balloon performance. We also adjusted for hospitals’ 2006 use of best practices and the number of patients with STEMI that the hospital treated in the follow-up periods because research shows that higher patient volume is associated with better performance (Hannan et al., 2005). Last, we included performance review by management as a covariate because managers’ vigilance can motivate workers to achieve performance goals (Wooldridge, Schmid, & Floyd, 2008). Hospitals reported the frequency with which management reviewed door-to-balloon times in the Follow-up Survey using the following response scale: monthly, quarterly, annually, not at all, and other. We coded “not at all” responses as 0 and other responses, which indicated that review occurred at some regular interval, as 1. Means, standard deviations, and correlations for the variables in our study are shown in Table 3. Notably, the correlation between importing practices and creative problem solving was not significant (p = .78 for binary measure of problem solving and p = .87 for continuous measure), minimizing a possible concern that our measures captured the same process or were unable to distinguish processes (adapting imported practices vs. creating new practices).

Table 3.

Descriptive Statistics and Intercorrelations for Study Variables.

Variable Mean SD 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1. Initial phase improvement 0.25 0.20
2. Later phase improvement 0.05 0.10 –.16
3. Import best practices 0.53 1.36 .22 .06
4. Creative problem solving (binary measure) 0.31 0.46 –.05 –.04 –.01
5. Creative problem solving (continuous measure) 2.57 1.23 –.04 –.02 .007 .77
6. Teaching hospital 0.49 0.50 .01 .01 .08 –.02 .05
7. Hospital size: 300-499 beds 0.48 0.50 <.01 .01 .10 .05 .07 .16
8. Hospital size: beds >500 0.15 0.36 –.04 .01 –.10 .01 .04 .34 –.40
9. Geographic location: Midwest 0.40 0.49 –.01 –.03 .06 –.06 –.09 <.01 –.01 .06
10. Geographic location: South 0.27 0.45 .03 .09 –.04 .05 .08 –.03 –.03 –.02 –.50
11. Geographic location: West 0.18 0.39 –.02 –.07 –.09 –.04 –.03 –.16 <.01 –.10 –.39 –.29
12. Owner: Public/govt, hospital 0.08 0.27 .01 .01 <.01 –.04 –.05 .07 –.01 .10* –.02 .11 –.03
13. Owner: Not-for-profit hospitals 0.80 0.32 –.08 .01 –.11 <–.0l <–.0l –.25 –.10 –.12 –.03 .14 .02 –.11
14. Net income (in millions) 16.59 1.25 –.08 –.03 <–.0l –.01 .03 .23 –.08 .41 –.03 .01 –.02 .03 –.05
15. Baseline no. of best practices 2.51 1.36 –.24 –.11 –.65 .10 .15 .07 –.07 .19 –.02 –.02 .07 <–.0l .04 .15
16. 2006 door-to-balloon performance 0.62 0.21 –.87 –.20 –.24 .11 .11 –.03 <–.0l .04 .02 –.04 .06 –.02 –.02 .08 .33
17. Patients with STEMI in 2008 50.09 25.24 –.02 –.05 –.05 <.01 .05 .01 .14 .17 .08 –.12 .01 –.07 <.01 .14 .16 .10
18. Patients with STEMI in 2010 51.09 21.80 –.03 –.08 –.09 .03 .05 –.07 .19 .05 .11 –.19 .05 –.06 .01 <.01 .15 .12 .85
19. Managers'performance feedback 0.44 0.50 –.02 .08 –.01 –.13 –.15 <–.0l .01 –.06 .01 .01 –.02 –.01 .03 .01 –.03 –.04 –.05 –.06
Open in a new tab

Note. |Correlation| ≥ .10 is significant at p < .05. Net income reported here differs from Table 2 because values have been normalized for hypothesis testing.

Analytic Approach

We tested our hypotheses using linear regression models estimated with robust standard errors. In our first model, for each dependent variable (i.e., initial phase improvement and later phase improvement) we included our measure for importing best practices and our measure of creative problem solving to assess the main effects of these two approaches. In the second model, for each dependent variable we entered the interaction term, which we computed as the product of our two focal variables. Following Aiken and West's (1991) recommendation, we first mean-centered our measure for importing best practices to reduce the potential for multicollinearity between the interaction term and its components and to facilitate interpretation of results (Cohen, Cohen, West, & Aiken, 2003). We examined the significance and direction of the coefficient for the interaction term to determine whether our hypotheses were supported. When the interaction term coefficient was significant, we examined the effect of importing best practices in the subgroup of hospitals that used creative problem solving and the subgroup that did not using separate models. To assess the robustness of our results, we examined whether results remained the same if we used the continuous measure of creative problem solving. All models included the covariates.

Results

The results of our primary analyses are presented in Table 4. Models 1 and 2 show that, after accounting for covariates, importing best practices was positively associated with improvement in the initial phase (p < .01) but not in the later phase (p > .05). These results support Hypothesis 1 (importing best practice is more positively associated with initial than later phase improvement). In contrast, Hypothesis 2 (creative problem solving is more positively associated with later phase than initial phase improvement) was not supported. The effect of creative problem solving increased between phases, but was not independently associated with improvement in either phase (p > .10).

Table 4.

Results for Analysis of Performance Improvement.

Model 1 Model 2 Model 3 Model 4
Dependent Variable = Improvement in Phase
Initial, 2006-2008 (N = 498) Later, 2008-2010 (N = 474) Initial, 2006-2008 (N = 498) Later, 2008-2010 (N = 474) Model 5
Independent variables Standardized coefficients Overall, 2006-2010
Import best practices .084** .072 .055 .017 .024
Creative problem solving .032 .043 .031 .039 .028^
Import best practices × creative problem solving .048 .095* .055**
Covariates
    Teaching hospital –.003 –.023 –.005 –.026 –.012
Hospital size: 300-499 beds (reference = beds < 300) –.012 .041 –.010 .046 .021
Hospital size: beds > 500 (reference = beds < 300) –.025 .039 –.025 .037 .011
Geographic location: Midwest (reference = Northeast) .030 .041 .030 .041 .026
Geographic location: South (reference = Northeast) .025 .095 .024 .095 .050*
Geographic location: West (reference = Northeast) .046 .037 .046 .037 .031
Ownership: Public/Govt. (reference = for-profit) –.057* –.053 –.060* –.059 –.046*
Ownership: Not-for-profit (reference = for-profit) –.089** –.072 –.092** –.077 –.063**
Net income .016 .005 .018 .008 .008
Baseline no. of best practices .094** .075 .096** .079 .057*
Baseline performance (2006 or 2008)a –.896*** –.737 –.898*** –.743 –.947**
Patients with STEMI at follow-up .061** .045 .061** .046 .040*
Managers' performance feedback –.052* –.006 –.053* –.009 –.020
F value 92.10*** 19.93*** 85.99*** 20.20*** 220.55***
Open in a new tab

Note. F values are from ordinary linear regression models, which provided same results. Results (coefficients and significance) are the same when performance rather than improvement is used as the dependent variable.

a

Baseline performance for initial and later phase is door-to-balloon performance in 2006 and 2008, respectively.

^

p < .10.

*

p < .05.

**

p < .01.

***

p < .001.

Model 3, in which the interaction of importing best practices and creative problem solving was added to Model 1, shows that there was not a significant interaction between approaches during the initial push (p > .05). Thus, Hypothesis 3 was not supported. Hypothesis 4, in contrast, was supported. Model 4 shows that the interaction of the two approaches was significantly associated with later phase improvement, suggesting synergy between approaches once hospitals entered the final phase of improvement (average improvement in 2008-2010 was 5% compared with 25% in 2006-2008). Figure 1 depicts the interaction that was found. Among hospitals that used creative problem solving, importing best practices was positively associated with later phase improvement (β = 0.14, p = .046). In contrast, among hospitals (N = 331) that did not use creative problem solving, importing was not associated with later phase improvement (β = 0.03, p = .49). This interaction was also seen for improvement over the entire period (2006-2010; Model 5).

Figure 1.

Figure 1

Open in a new tab

The interaction of importing best practices and creative problem solving on later phase (2008-2010) improvement.

Note. When the interaction was examined the opposite way, assessing the effect of creative problem solving in the subgroup high on importing best practices (above mean) versus the low subgroup (below mean), the results were similar. Creative problem solving was associated with later phase improvement among hospitals (N = 242) with above mean importing of best practices (β = 0.13, p = .003), whereas it was not associated with later phase improvement among hospitals (N = 232) with below mean importing of practices (β = −0.04, p = .43).

Results were largely unchanged when we used the continuous measure of creative problem solving, except that the significance of the positive interaction in the later phase became marginally significant (p < .10). This was expected given that the continuous measure captured whether hospitals had used any steps in the problem-solving process, not the complete process itself. Our arguments were based on the latter. Nevertheless, we take the similarity as supporting our findings. Results were the same whether we used performance or improvement as the outcome.

The difference in overall improvement for hospitals that did not engage in problem solving versus those that were high on both approaches is notable in terms of patient outcomes. The former group increased the percentage of patients treated within guideline by 29.5% from 2006 to 2010, while the latter improved by 33.2%. Thus, 3.7% of patients did not benefit as much as they might have if all hospitals had pursued both. Given that an estimated 250,000 persons arrived at a U.S. hospital with STEMI in 2010 (most patients die en route to the hospital; Canto et al., 2012; Centers for Disease Control and Prevention, 2010), we calculate that approximately 9,250 patients in 2010 were at increased risk of mortality in hospitals that did not use creative problem solving compared with those in hospitals that used both approaches.

Discussion

At a time when health care organizations are under pressure to improve their performance in order to meet stakeholders’ evolving expectations and industry developments, managers face the task of deciding which approach or combination of approaches to use to realize their improvement goals. Common adages (e.g., “more is better”) suggest that organizations should use multiple approaches, if resources permit. However, our review of the literature on organizational learning and improvement suggested that using multiple approaches could have a positive or negative effect, depending on the phase of improvement. To provide insight on the relationship between timing and the independent and interactive effectiveness of deliberate learning approaches, we investigated two popular approaches: importing best practices and creative problem solving to generate new practices internally. In sum, our results indicate that both approaches contribute to improvement. However, their impact depends on where organizations are in their improvement journey. Importing best practices is more strongly associated with improvement in the initial phase than later phase; in contrast, the impact of creative problem solving on improvement increases between the initial and later phases, although it is not independently associated with improvement in either phase. In the later phase, however, creative problem solving interacts positively with the importing of best practice such that importing contributes more to improvement when hospitals engage in problem solving, and conversely hospitals benefit more from problem solving when they import best practices.

These results have implications for how organizations might effectively manage their deliberate learning activities during their improvement journey. The findings suggest that organizations need not pursue both approaches—importing best practices and creative problem solving—to achieve shorter-term improvement. Importing best practices is sufficient to achieve performance gains quickly. This observation is particularly important for the many organizations that have suffered from the “not invented here” syndrome (Katz & Allen, 1982), those that have resisted the importing of best practices believing that their uniqueness requires them to create new practices internally to experience improvement. Our results suggest that organizations can benefit greatly in the shorter term from developing expertise in importing others’ best practices. Over the longer term though, organizations are served well by also developing the capacity for creative problem solving. Thus, longer-term improvement is more likely achieved when organizations combine external with internal learning, embracing both industry standardization and organizational creativity.

Insights for Research and Theory

By examining the effect of deliberate learning approaches at different times, our research added a new temporal element to the study of learning in health care organizations, and consequently provided greater insight on how deliberate learning approaches affect improvement over time. Our results showed that approaches’ associations with improvement can change depending on the time period. There are at least two implications of this finding. Most clearly, it suggests the importance of careful consideration and reporting of improvement phase in studies of the effectiveness of deliberate learning. If we had only considered the initial phase, we would have missed the positive interaction between approaches in the longer term. And, if we had we not considered the initial phase, we would have missed the primary importance of importing best practices in that period.

Our results additionally contribute to the literature on deliberate learning by providing insight on the interaction of the approaches studied. Their interaction had not previously been examined, despite their popularity and calls for research on the interaction of learning strategies in the broader learning literature (Lapré & Nembhard, 2010). Our results provide evidence for a longer-term interaction, but not for shorter-term interaction. As we had hypothesized based on theory and research in other settings (Schwab, 2007; Wong, 2004) that the effect of interaction in the shorter term would be negative, our finding suggests the need to revise thinking that leads to the conclusion that qualitatively different approaches necessarily create performance-undermining tension. That said, we caution against assuming that approaches always co-exist peacefully. It may be that the practices we studied were so discrete, well-defined, and distinct so as to compete minimally with creative problem solving for resources. Hence, the theorized tension was not observed. Research is needed to determine if or under what conditions learning approaches interact negatively in health care.

Another potential step for research in this area is to assess whether there are circumstances under which importing best practices, independent of creative problem solving, provides gains later in the improvement journey. It may be that some best practices, for example, those that have a large tacit component, do not result in significant improvement gains until the later phase. In the case of tacit-heavy practices, this might be because greater learning must occur to convert the tacit to explicit knowledge usable by the organization (Nonaka, 1994). In contrast, best practices that contain more explicit knowledge (such as the practices studied here) may result in significant improvement sooner. Future research might explore such possibilities using different types of best practices.

Implications for Practice

The importance of importing best practices for realizing performance gains quickly suggests the value of organized efforts to disseminate best practices. In health care, these dissemination efforts have taken the form of national quality campaigns such as the D2B Alliance studied here as well as smaller collaboratives in which groups of organizations come together to learn about best practices and strategies for implementing them. Robust evidence on the value of these efforts is limited (Schouten, Hulscher, Everdingen, Huijsman, & Grol, 2008). Our results suggest that such efforts are valuable to the extent that they facilitate importing of best practices. Thus, sponsors (government and private) of these efforts are advised to continue honing their dissemination strategies.

Additionally, given our findings about the value of creative problem solving for multiplying later phase improvement, sponsors may wish to broaden their role to include helping organizations to develop the capacity for creative problem solving and an appreciation for starting creative problem solving early because its benefits are delayed. Only 31% of the hospitals in our sample engaged in every step of the process. Prior research proposes that capacity for this process depends on manager support, staff training, work design, norms, and rewards (Tucker & Edmondson, 2003). The need to strengthen organizations’ capacities for creative problem solving and importing best practices suggests that another key role for sponsors is helping organizations’ improvement teams to appreciate and manage seemingly conflicting approaches to improvement, such as those we studied here. Prior research suggests that ambidexterity in managing potential paradoxes depends on establishing an overarching common goal, establishing open communication with staff, and having safe forums in which conflicts can be unearthed and resolved (Lavie, Stettner, & Tushman, 2010). Helping organizations navigate the development of infrastructure and relationships to manage different approaches may be the next step in fostering performance improvement in health care.

Limitations With Suggestions for Future Research

In this study, we showed effects for hospitals participating in a national campaign. Research has found that these hospitals differ from non-enrolled hospitals by being larger, nonprofit, and teaching institutions (Bradley et al., 2009), raising the question of whether our results generalize to hospitals that do not join campaigns. However, our sample included hospitals with similar profiles to non-enrolled hospitals, we adjusted our models for covariates to account for possible differences, and past work shows no difference in importing of practices between groups (Hansen et al., 2010). Still, we cannot claim generalizability. Our study should be replicated using non-campaign hospitals.

We hope that future research will also be able to follow organizations through large environmental changes to further isolate differences in the effects of learning approaches on different time periods. Additionally, we hope that future work will capture more information about creative problem solving and the duration of its use to advance understanding of this activity that is necessary for sustaining continuous improvement. It may be that we did not observe a main effect of its use because our time frame was too short. We hope that future research will benefit from data with more time points so that our hypotheses can be tested again and using learning curve models. Such models were not possible in this study due to a limited number of hospitals with data for additional time points.

Conclusion

Research on the independent and interactive effect of qualitatively different strategies at different points in the improvement journey has been lacking, despite importance. Our study drew on the organizational learning literature to provide logic for and empirical data to demonstrate how different approaches to deliberate learning contribute to improvement. Our results show that, in the initial phase, importing best practices helps achieve early gains, while creative problem solving does not nor does it harm gains from importing practices. However, in the later phase, hospitals benefit from creative problem solving and achieve greater gains when they have pursued both approaches.

Acknowledgments

We wish to thank Christopher Morrow for research assistance, Israel Labao for assistance with data collection, and all those involved in the D2B Alliance campaign administratively as well as the hospitals that participated in this study.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Agency for Healthcare Research and Quality (Grant Number K01HS018987); the American College of Cardiology; and the National Heart, Lung, and Blood Institute (Grant Number R01HL072575). The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.

Footnotes

Authors’ Note

An earlier version of this work received the Best Abstract in Science of Quality Improvement Award at the 2012 AcademyHealth meeting.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

  1. Adler PS, Riley P, Kwon S-W, Signer J, Lee B, Satrasala R. Performance improvement capability: Keys to accelerating performance improvement in hospitals. California Management Review. 2003;45(2):12–33. [Google Scholar]
  2. Aiken LS, West SG. Multiple regression: Testing and interpreting interactions. 3rd ed. Sage; Newbury Park, CA: 1991. [Google Scholar]
  3. Amabile TM, Hadley CN, Kramer SJ. Creativity under the gun. Harvard Business Review. 2002;80(8):52–61. [PubMed] [Google Scholar]
  4. Antman EM, Anbe DT, Armstrong PW, Bates ER, Green LA, Hand M, Jacobs AK. ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction—Executive summary. Circulation. 2004;110(5):588–636. doi: 10.1161/01.CIR.0000134791.68010.FA. [DOI] [PubMed] [Google Scholar]
  5. Berta WB, Baker R. Factors that impact the transfer and retention of best practices for reducing errors in hospitals. Health Care Management Review. 2004;29(2):90–97. doi: 10.1097/00004010-200404000-00002. [DOI] [PubMed] [Google Scholar]
  6. Bradley EH, Herrin J, Wang Y, Barton BA, Webster TR, Mattera JA, Krumholz HM. Strategies for reducing the door-to-balloon time in acute myocardial infarction. New England Journal of Medicine. 2006;355(22):2308–2320. doi: 10.1056/NEJMsa063117. [DOI] [PubMed] [Google Scholar]
  7. Bradley EH, Nallamothu BK, Stern AF, Cherlin EJ, Wang Y, Byrd JR, Krumholz HM. The door-to-balloon alliance for quality: Who joins national collaborative efforts and why? Joint Commission Journal of Quality and Patient Safety. 2009;35(2):93–99. doi: 10.1016/s1553-7250(09)35012-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Canto AJ, Kiefe CI, Goldberg RJ, Rogers WJ, Peterson ED, Wenger NK, Canto JG. Differences in symptom presentation and hospital mortality according to type of acute myocardial infarction. American Heart Journal. 2012;163(4):572–579. doi: 10.1016/j.ahj.2012.01.020. [DOI] [PubMed] [Google Scholar]
  9. Carmeli A, Gelbard R, Reiter-Palmon R. Leadership, creative problem-solving capacity, and creative performance: The importance of knowledge sharing. Human Resource Management. 2013;52(1):95–121. [Google Scholar]
  10. Centers for Disease Control and Prevention Number of all-listed procedures for discharges from short-stay hospitals, by procedure category and age: United States, 2010. 2010 Retrieved from http://www.cdc.gov/nchs/nhds/nhds_tables.htm#first.
  11. Cohen AB, Restuccia JD, Shwartz M, Drake JE, Kang R, Kralovec P, Bohr D. A survey of hospital quality improvement activities. Medical Care Research and Review. 2008;65(5):571–595. doi: 10.1177/1077558708318285. [DOI] [PubMed] [Google Scholar]
  12. Cohen J, Cohen P, West SG, Aiken L. Applied multiple regression/correlation analysis for the behavioral sciences. Erlbaum; Mahwah, NJ: 2003. [Google Scholar]
  13. Cohen WM, Levinthal DA. Absorptive capacity: A new perspective on learning and innovation. Administrative Science Quarterly. 1990;35(1):128–152. [Google Scholar]
  14. Davidoff F. Is every defect really a treasure? Annals of Internal Medicine. 2012;156(9):664–665. doi: 10.7326/0003-4819-156-9-201205010-00013. [DOI] [PubMed] [Google Scholar]
  15. Eisenhardt KM, Martin JA. Dynamic capabilities: What are they? Strategic Management Journal. 2000;21(10-11):1105–1121. [Google Scholar]
  16. Feinstein JS. The nature of creative development. Stanford University Press; Stanford, CA: 2006. [Google Scholar]
  17. Garvin DA. Learning in action. Harvard Business School Press; Boston, MA: 2000. [Google Scholar]
  18. Hannan EL, Wu C, Walford G, King SB, Holmes DR, Ambrose JA, Jones RH. Volume-outcome relationships for percutaneous coronary interventions in the stent era. Circulation. 2005;112(8):1171–1179. doi: 10.1161/CIRCULATIONAHA.104.528455. [DOI] [PubMed] [Google Scholar]
  19. Hansen LO, Herrin J, Nembhard IM, Busch S, Yuan CT, Krumholz HM, Bradley EH. National quality campaigns: Who benefits? Quality & Safety in Health Care. 2010;19:275–278. doi: 10.1136/qshc.2009.036087. [DOI] [PubMed] [Google Scholar]
  20. Haunschild PR, Miner AS. Modes of interorganizational imitation: The effects of outcome salience and uncertainty. Administrative Science Quarterly. 1997;42(3):472–500. [Google Scholar]
  21. Institute of Medicine . Crossing the quality chasm: A new system for the 21st century. National Academies Press; Washington, DC: 2001. [PubMed] [Google Scholar]
  22. Institute of Medicine. The healthcare imperative: Lowering costs and improving outcomes: Workshop series summary. National Academies Press; Washington, DC: 2010. [PubMed] [Google Scholar]
  23. Jha AK, Larizgoitia I, Audera-Lopez C, Prasopa-Plaizier N, Waters H, Bates DW. The global burden of unsafe medical care: Analytic modelling of observational studies. BMJ Quality & Safety. 2013;22(10):809–815. doi: 10.1136/bmjqs-2012-001748. [DOI] [PubMed] [Google Scholar]
  24. Katz R, Allen TJ. Investigating the not invented here (NIH) syndrome: A look at the performance, tenure, and communication patterns of 50 R&D project groups. R&D Management. 1982;12(1):7–19. [Google Scholar]
  25. Keating E, Oliva R, Repenning NP, Rockart S, Sterman J. Overcoming the improvement paradox. European Management Journal. 1999;17(2):120–134. [Google Scholar]
  26. Kim J-Y, Miner AS. Vicarious learning from the failures and near-failures of others: Evidence from the U.S. Commercial banking industry. Academy of Management Journal. 2007;50(3):687–714. [Google Scholar]
  27. Krumholz HM, Herrin J, Miller LE, Drye EE, Ling SM, Han LF, Curtis JP. Improvements in door-to-balloon time in the United States, 2005 to 2010. Circulation. 2011;124:1038–1045. doi: 10.1161/CIRCULATIONAHA.111.044107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Lapré MA, Nembhard IM. Inside the organizational learning curve: Understanding the organizational learning process. Foundations and Trends in Technology, Information and Operations Management. 2010;4(1):1–106. [Google Scholar]
  29. Lavie D, Stettner U, Tushman ML. Exploration and exploitation within and across organizations. Academy of Management Annals. 2010;4:109–155. [Google Scholar]
  30. Levitt B, March JG. Organizational learning. Annual Review of Sociology. 1988;14:319–340. [Google Scholar]
  31. Madsen P, Desai V. Failing to learn? The effects of failure and success on organizational learning in the global orbital launch vehicle industry. Academy of Management Journal. 2010;53(3):451–476. [Google Scholar]
  32. March JG, Simon HA. Organizations. Wiley; New York, NY: 1958. [Google Scholar]
  33. McAlearney AS, Robbins J, Kowalczyk N, Chisolm DJ, Song PH. The role of cognitive and learning theories in supporting successful EHR system implementation training: A qualitative study. Medical Care Research and Review. 2012;69(3):294–315. doi: 10.1177/1077558711436348. [DOI] [PubMed] [Google Scholar]
  34. Mumford MD, Whetzel DL, Reiter-Palmon R. Thinking creatively at work: Organization influences on creative problem solving. Journal of Creative Behavior. 1997;31(1):7–17. [Google Scholar]
  35. Nallamothu BK, Bradley EH, Krumholz HM. Time to treatment in primary percutaneous coronary intervention. New England Journal of Medicine. 2007;357(16):1631–1638. doi: 10.1056/NEJMra065985. [DOI] [PubMed] [Google Scholar]
  36. Nembhard IM. Learning and improving in quality improvement collaboratives: Which collaborative features do participants value most? Health Services Research. 2009;44(2 Part 1):359–378. doi: 10.1111/j.1475-6773.2008.00923.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Nembhard IM, Tucker AL. Deliberate learning to improve performance in dynamic service settings: Evidence from hospital intensive care units. Organization Science. 2011;22(4):907–922. [Google Scholar]
  38. Nickerson JA, Zenger TR. A knowledge-based theory of the firm—The problem-solving perspective. Organization Science. 2004;15(6):617–632. [Google Scholar]
  39. NIH Apr 10, 2012. 2013 Citations added to Medline® by fiscal year.
  40. Nonaka I. A dynamic theory of organizational knowledge creation. Organization Science. 1994;5(1):14–37. [Google Scholar]
  41. O'Dell C, Grayson CJ. If only we knew what we know: Identification and transfer of internal best practices. California Management Review. 1998;40(3):154–174. [Google Scholar]
  42. Repenning NP, Sterman JD. Capability traps and self-confirming attribution errors in the dynamics of process improvement. Administrative Science Quarterly. 2002;47(2):265–295. [Google Scholar]
  43. Schouten LMT, Hulscher MEJL, Everdingen JJEV, Huijsman R, Grol RPTM. Evidence for the impact of quality improvement collaboratives: Systematic review. British Medical Journal. 2008;336(7659):1491–1494. doi: 10.1136/bmj.39570.749884.BE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Schwab A. Incremental organizational learning from multilevel information sources: Evidence for cross-level interactions. Organization Science. 2007;18(2):233–251. [Google Scholar]
  45. Shortell SM, Schmittdiel J, Wang MC, Li R, Gillies RR, Casalino LP, Rundall TG. An empirical assessment of high-performing medical groups: Results from a national study. Medical Care Research and Review. 2005;62:407–434. doi: 10.1177/1077558705277389. [DOI] [PubMed] [Google Scholar]
  46. Singer SJ, Moore SC, Meterko M, Williams S. Development of a short-form learning organization survey: The LOS-27. Medical Care Research and Review. 2012;69(4):432–459. doi: 10.1177/1077558712448135. [DOI] [PubMed] [Google Scholar]
  47. Szulanski G. Exploring internal stickiness: Impediments to the transfer of best practice within the firm. Strategic Management Journal. 1996;17(Special Winter Issue):27–43. [Google Scholar]
  48. Szulanski G, Jensen RJ. Presumptive adaptation and the effectiveness of knowledge transfer. Strategic Management Journal. 2006;27(10):937–957. [Google Scholar]
  49. Terwiesch C, Xu Y. Innovation contests, open innovation, and multiagent problem solving. Management Science. 2008;54(9):1529–1543. [Google Scholar]
  50. Tucker AL, Edmondson AC. Why hospitals don't learn from failures: Organizational and psychological dynamics that inhibit system change. California Management Review. 2003;45(2):55–72. [Google Scholar]
  51. Tucker AL, Nembhard IM, Edmondson AC. Implementing new practices: An empirical study of organizational learning in hospital intensive care units. Management Science. 2007;53(6):894–907. [Google Scholar]
  52. Willett JB. Questions and answers in the measurement of change. Review of Research in Education. 1988;15:345–422. [Google Scholar]
  53. Wong S-S. Distal and local group learning: Performance trade-offs and tensions. Organization Science. 2004;15(6):645–656. [Google Scholar]
  54. Wooldridge B, Schmid T, Floyd SW. The middle management perspective on strategy process: Contributions, synthesis, and future research. Journal of Management. 2008;34(6):1190–1221. [Google Scholar]
  55. Zollo M, Winter SG. Deliberate learning and the evolution of dynamic capabilities. Organization Science. 2002;13(3):339–351. [Google Scholar]

RESOURCES