Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Jan 17.
Published in final edited form as: Int J Inf Syst Change Manag. 2011 Mar 15;5(1):22–35. doi: 10.1504/IJISCM.2011.039068

THE DEVELOPMENT AND USE OF A MODEL TO PREDICT SUSTAINABILITY OF CHANGE IN HEALTH CARE SETTINGS

Todd Molfenter 1, James H Ford 1, Abhik Bhattacharya 1
PMCID: PMC3260322  NIHMSID: NIHMS329696  PMID: 22262947

Abstract

Innovations adopted through organizational change initiatives are often not sustained leading to diminished quality, productivity, and consumer satisfaction. Research explaining variance in the use of adopted innovations in health care settings is sparse, suggesting the need for a theoretical model to guide research and practice. In this article, we describe the development of a hybrid conjoint decision theoretic model designed to predict the sustainability of organizational change in health care settings. An initial test of the model’s predictive validity using expert scored hypothetic profiles resulted in an r-squared value of .77. The test of this model offers a theoretical base for future research on the sustainability of change in health care settings.

Keywords: Sustainability, Decision Theoretic Model, Organizational Change

Introduction

Rogers (1995) noted the sustained use of innovations in organizational settings represents a distinct step in the diffusion of innovations. Sustainability is important because the adoption of a new organizational innovation does not always result in sustained use of the innovation (Ham, Kipping, & McLeod, 2003; Molfenter, Gustafson, Kilo, Bhattacharya, & Olsson, 2005). In these studies, performance gains occurred while special organizational resources, such as focused leadership attention, use of outside consultants, and staff participation in the design and implementation of the innovation were present. Yet, a return to the pre-existing organizational practices can occur shortly after removal of special resources (Massati, Sweeney, Panzano, & Roth, 2008). Cutler (2002) refers to the difficulty of maintaining the change beyond the removal of special resources as the “sustainability challenge.” Abandoning an innovation can result in ineffective use of precious discretionary organizational resources, poor return on technology investments, and proves frustrating for managers who thought they had an organizational problem resolved, only to see it return.

Sustainability of innovations in health care settings is uniquely influenced by health care cultures, professional and guild issues, complex organizational processes, and financing mechanisms (Rogers, 2003). Health care culture supports autonomous clinical decision making that prevents standardization and causes variation in health care delivery (Wennberg, 1999). The clinical professionals’ autonomy in decision-making makes it easier for a clinician to ignore an innovation that opposes his or her training or beliefs. The many professional guilds or specialties that operate health care delivery systems involve multiple processes and decision-makers. This complexity creates frequent opportunities to ignore or change an adopted innovation. Financial considerations can also influence health-care decision making. For example, using Naltrexone medication for alcohol abuse disorders and the IMPACT model for geriatric depression both had positive impact on patient health and well being in several settings. Yet, these therapies but could not be sustained due to the absence of consistent reimbursement funding support (Ducharme, Knudsen, & Roman, 2006; Unutzer, Powers, Katon, & Langston, 2005).

Despite the common incidence of the lack of sustainability (Ham et al., 2003; Molfenter et al., 2005) and its potential impact on individual health and organizational performance, empirical research focused on sustainability of organizational innovations has been “very sparse” (Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). Buchanan et al. (2005) proposes that the range of factors that impact sustainability are so broad, and contexts of sustainability application so diverse, that projecting the relative significance of sustainability factors can not be determined a priori. This presents a difficult situation for organizational managers who want to know how they can increase their ability to sustain a change, and limits the possibility of conducting prospective research on this important issue.

Defining Sustainability

A fundamental question for sustainability research is, “When has sustainability been achieved?” Initial definitions were based only on observations of organizational behavioral and described how the change becomes routine or “how things are done around here” (Maher, 2004; Rogers, 1995). With the onset of sustainability research and theory development, sustainability has come to be defined as the length of time the innovation has been used within the organization. Molfenter et al. (2005) described sustainability as innovations that continue to be used two years following implementation, and Fixsen et al. (2005), four years. This research defines sustainability as “For an implemented change, the change will continue to be in place or will have been improved upon six months later.”

Organizational Change Factors

Over the past three decades, innovation research has primarily focused on the adoption of innovations (Damanpour, 1992; Granados et al., 1997; Meyers, Sivakumar, & and Nakata, 1999; Rogers, 1995; Wejnert, 2002). Innovation research has been expressed in a variety of explanatory models. Several investigators have assessed organizational readiness to adopt innovations (Anderson & Lenz, 2001; Lehman, Greener, & Simpson, 2002; Simpson, 2002). Others have noted the organizational competencies associated with positive organizational change (Escrig-Tena & Bou-Llusar, 2005). The organizational readiness and competency-based approaches are intended to be prospective models that anticipate the impact of any change made in an organizational setting that expresses a particular set of characteristics. These models, however, are not specific to the implementation of a particular innovation in a given organizational situation and lack the degree of precision (e.g., sensitivity and specificity) that models targeted to specific situations provide. For example, Gustafson et al. (2003) developed a model that demonstrated the validity of predicting the success or failure of the adoption of an innovation, giving limited attention to whether or not the innovation is sustained. Yet, the need for a model dedicated to sustainability is warranted based on one study that noted the factors used to predict adoption of an organizational change were not able to predict sustainability (Molfenter et al., 2005).

These adoption models could offer insights into factors that may affect sustainability. For instance, the organizational factors in both the Gustafson (2003) and Simpson (2002) models associated with organizational innovation adoption are leadership, staff support, and resources dedicated to the change effort. In related research on the factors of organizational change, a study by Ferlie and Shortell (2001) investigated the impact of leadership, staff support, resources, and external pressure on the degree of success of quality improvement projects. This study found that all these factors played a role in innovation adoption, but projected only external pressure had a significant effect on sustaining change. The prevalence of the factors of leadership, staff resources, external pressure, and staff support in meta-analyses of innovation research suggests these factors may also explain variance in organizational sustainability (Greenhalgh et al., 2004).

Modeling Organizational Sustainability

Several environmental conditions have limited the ability to model the sustainability of organizational changes. Recruiting an adequate number of organizational subjects can be difficult; the subjects must be engaged at the specific point in time when a change has just been implemented. Sustainability research requires longitudinal research over an extended period of time that can demand considerable resources to conduct and the turn-around time for results can be lengthy. Lastly, many factors are reported to cause variation in organizational performance. Which factors to use in modeling sustainability have been limited to conceptual models or studies focused a specific variable (Buchanan et al, 2005).

Researchers in the fields of Marketing Research and Economic Modeling have used hybrid conjoint models to assist practitioners by using expert opinion to select model factors and assign initial predictive weights to each factor (Wind, Green, Shifflet, & Scarbrough, 1989, Magat, Kip Viscusi, & Huber, 1988; Oplauch, Swallow, Weaver, Wessells, & Wichelns, 1993). Hybrid conjoint models are used to model uncertainty, or difficult to predict circumstances or events, by using probability statistics to model the decisions individuals will make when presented with a given set of environmental conditions. These models are built on the subjective opinion of experts using validated modeling techniques and require much less time to complete than a longitudinal research trial (Von Winterfeldt & Edwards, 1986). The use of subjective hybrid conjoint models has led to the creation of valid predictive models across a variety of subjects (Sheldon & Steer, 1982; Wardman, 1988, Saridakis, 2009).

The research on the development of predictive model of organizational sustainability will test two research questions:

  • Research Question 1. Will model factors attributed to leadership, resources devoted to change, external environment’s emphasis on maintaining the change, and staff motivation be included in the model and be significantly associated with sustainability of an innovation?

  • Research Question 2. To what extent does the developed Hybrid Conjoint sustainability model explain the variance in expert-derived assessments of organizational sustainability of an innovation?

Methods

Description of Model Formation

The hybrid conjoint model has a customary set of stages necessary to design a conjoint analysis study (Ryan, 1999). These stages are: (1) establish the attributes (or factors), (2) assign levels to the attributes, (3) develop hypothetical profiles, (4) measure preferences, and (5) test the model and analyze the data. In our model the outcome variable of sustainable change was defined as “For an implemented change, the change will continue to be in place or will have been improved upon six months later.”

Establish Attributes and Assigning Levels

Various methods can by used to elicit the attributes for a hybrid conjoint model: surveys, literature reviews, group discussions, or individual interviews (Wind et al., 1989; Ryan, 1999). The elicitation process is a crucial step in defining the attributes to be used in the model. Its purpose is to understand and identify key attributes associated with the selected model outcome. For this model, the list of potential attributes was generated through a total of 46 individual interviews and an extensive literature review. Individuals interviewed were either practitioners in a managerial position with greater than five years of experience, specialists employed by the organization to lead change efforts including technology implementation, or academic researchers who specialize in organizational change research. Direct questioning to elicit the model attributes used a decision theoretic methodology previously applied to develop predictive models for adherence to a medication regimen (Gustafson et al., 2001) and to determine the successful implementation of organizational change (Gustafson et al., 1994). In the interview, the predictive attributes (model factors) were solicited by asking, “Suppose you had to predict whether a change was going to be sustained. What information would you want to know before making the prediction?” For level development for the suggested attributes they were asked, “What answer to the previous question would make you optimistic? And what answer would make you pessimistic?” The optimistic answers suggested the most preferred attribute level while the pessimistic responses suggested the least preferred.

The interview results were integrated along with information gathered from literature reviews into a “straw model” that included a list of 54 attributes and their associated levels. The key pieces of literature used to complement the interviews were (Damanpour, 1991); (Frambach & Schillewaert, 2002); (Gustafson & Hundt, 1995); and (Rogers, 1995). It was projected that these 54 attributes would result in a model that would lead to simplifying effects because respondents would tend to focus on just a subset of attributes while neglecting the other ones. This would create a biased model with potentially poor predictive validity. Therefore, five researchers and four organizational change consultants, each with greater than 10 years of experience, independently reviewed all interview responses and reduced the list of factors in a focus group setting using an approach suggested by Cooksey (1996).

Develop Profiles

Hypothetical profiles containing the full set of attributes were used in the analysis. Karren and Barringer (2001) noted that profiles used in a conjoint model must be realistic for the model to be generalizable. Because the model’s attribute levels were based on expert opinion obtained through the interviews, it was projected the created profiles reflected realistic situations. Additionally, the model profiles were shared with an organizational change researcher and three organizational change agents not involved with the initial interviews. In all cases they felt the model profiles represented realistic situations.

Our full experiment involved nine factors with three levels for each factor. This would create 39 =19,683 different profiles. A concern for profile assessment was the respondent fatigue that could result in scoring this many profiles. The number of profiles that each study respondent was asked to assess was reduced by using a 1/243 fractional factorial Incomplete Block Design (IBD), which had 39–5 =91 factor combinations or profiles. In their comprehensive review of using IBD in policy-capturing research, Graham & Cable (2001) have demonstrated this method to be more suitable than full factorial methods. Previous research by Conner and Zelen (1959) provided the mechanism for developing a balanced design and determining which factor attribute level to select when developing the profiles as well as what profiles should be assigned to the different blocks. This approach was used to assign three profiles to each of the 81 respondents. The use of the fractional factorial design in creating the profiles allowed for the identification of all main effects and no interaction terms. Respondents also rated two additional profiles. One profile represented the worst level of all the attributes and the other the best level of all attributes.

For power calculations, the ratio of scenarios to factors is used in conjoint designs. Cooksey (1996) suggest a ratio of scenarios to factors of 10:1 is preferred and 5:1 is adequate. This design will have a ratio of 9:1 (e.g., 81 profiles and 9 factors).

Measure Preferences

Eighty-one health care organizational change and management practitioners were recruited to provide data for the model. The expert group testing the model included academics (n = 10; 12.3 percent), consultants (n = 17; 20.9 percent), middle managers (n = 40; 49.4 per cent) and senior leaders (n = 14; 17.4 percent). Approximately, 46 percent of the respondents represented the healthcare industry while the remaining respondents were from other industrial sectors representing manufacturing, hospitality, and software.

Three kinds of data are collected from each respondent for the purpose of model fitting. Those are:

  1. Attribute-level desirability values for the levels of each attribute separately.

  2. Attribute importance weights.

  3. Conjoint response to three full profiles, drawn from the master design.

The hybrid categorical conjoint model uses OLS regression analysis to weight the desirability values, importance weights, and conjoint responses (Green, Goldberg, & Montemayor, 1981).

A survey tool was designed to record respondents’ self-explicated attribute-level desirability values and importance data as well as respondents’ total profile scores for five profiles. The attribute-level desirability value was assessed by asking the respondent to assign a score of 0–100 for the mid-level of an attribute. For each utility assessment, the best level was assigned a 100 and the worst level a 0, and the respondents completed the mid-level utility. Two approaches were used to assess attribute importance weights: rank ordering and the distribution of 100 points across the nine factors based on perceived relative importance.

Of the five profiles, two represented the best and worst attribute levels of all factors. The remaining three profiles were assigned in a fashion such that each of the profiles was rated by three different persons. The survey tool was administered by a Web site. Each respondent was given a specific URL for accessing the Web site that presented the self-explicated assessments and the profiles they had been assigned through the confounded design. The Web was used to standardize the data collection process, enhance the completion rate, and simplify data preparation for analysis. The profiles were rated on the likelihood of success, and the best and worst profiles completed by a respondent were used to normalize their scores to a common scale across subjects.

Test the Model

The model was tested by comparing the rating for each profile with the model-derived scores to determine the amount of variation the model explains using an R-square analysis. This not only will serve as an initial test of the model, but will test the model’s internal validity by determining how well the model captured the expert’s judgments. Models developed and tested using the steps described by Ryan (1999) that demonstrated initial and internal validity have progressed to demonstrate external validity for models designed to predict health severity (Gustafson et al., 1986); implementation of an innovation in school settings (Bosworth, Gingiss, Potthoff, & Roberts-Gray, 1999); and success of quality improvement projects (Olsson, Overetveit, & Kammerlind, 2003).

Results

This section describes the components of the model including the predictive attributes with their levels and the validity of the model.

Attributes & Levels

Nine factors were identified: Adaptability, Change Reversibility, Champion Turnover, On-going Leadership, Political Environment, Resources Devoted to Change, Staff Motivation, External Pressure, and Evidence of Effectiveness. Table 1 includes the nine model attributes and their level descriptors.

Table 1.

List of Attributes and Levels of Sustainability Model

Attributes Levels
Adaptability The change can easily accept adjustments from all levels of the organization.
The change can be adjusted but not easily.
Adjustments to the change are very difficult.
Change Reversibility Due to the nature of the change, it is impossible to revert back to old system.
The change can be reversed with difficulty.
Staff can easily revert back to the old system.
Champion turnover and new staff development. Low champion and staff turnover.
High staff turnover but mentoring after initial training.
High staff turnover and no mentoring after initial training.
Ongoing Leadership Administration considers maintaining the change a top priority.
Administration considers maintaining the change desirable.
Administration is indifferent whether or not the change is maintained.
Leadership is actively seeking a return to the “old system.”
Political Environment (If cross-department cooperation is required complete this section) The level of support between departments for maintaining the change has been cooperative.
The level of support between departments for maintaining the change has been mixed.
The level of support between departments for maintaining the change has been controversial.
Staff Motivation Staff is motivated to sustain change.
Staff is reluctant to sustain change.
Staff is against sustaining the change.
Resources Devoted to Change Organization provides needed funding and personnel to sustain change.
Organization has one of the following to sustain the change: personnel or adequate funding.
Organization has neither funding nor manpower to sustain change.
Evidence of Effectiveness There is strong measurable evidence of effectiveness of the change.
There is sporadic evidence of effectiveness of the change.
There is no evidence of effectiveness or evidence it is not effective.
External Pressure External pressures from regulatory agencies, community leaders, or competitors facilitate sustaining the change.
No external pressure is occurring because of the change.
External pressures from regulatory agencies or other influential sources are against the change.
Open in a new tab

  • Research Question 1. Will model factors attributed to leadership, resources devoted to change, external environment’s emphasis on maintaining the change, and staff motivation be included in the model and be significantly associated with sustainability of an innovation?

The factors, leadership, staff motivation, external pressure and resources devoted the change where included in the model. When the nine factors in the proposed sustainability of innovation model where compared to sustainability outcomes using a simple linear regression model, three of the nine factors could significantly explain the variance in the likelihood that a change will be sustained in the organization (Table 2). Based on the relative rankings (or importance weights) assigned by the experts, on-going leadership had the highest relative ranking. Staff motivation, resources devoted to change, and evidence of effectiveness were the other factors that had rankings > 5. When the factor was compared to the sustainability outcomes, staff motivation and resources devoted to change had p-values < .001. Intriguingly, on-going leadership was significant at the p < .01 level and evidence of effectiveness was not significant. These p-values in part, support research question 1 because on-going leadership, staff motivation, and resources devoted to change had p-values < .01. The external pressure attribute, however, was not associated with sustainability outcomes.

Table 2.

Sustainability of Innovation Model Factors

Factor Average Ranking (Scale 1–9)
Ongoing Leadership 6.46*
Staff Motivation 5.88**
Resources Devoted to Change 5.58**
Evidence of Effectiveness 5.02
Adaptability 4.80
Political Environment 4.75
Champion Turnover 4.51^
External Pressure 4.31
Change Reversibility 3.69
Open in a new tab
**

P < 0.001,

*

= P < 0.01

^

P < 0.1 (in explaining variance in outcome)

  • Research Question 2. To what extent does the developed Hybrid Conjoint sustainability model explain the variance in expert-derived assessments of organizational sustainability of an innovation?

We tested the model by taking the nine-factor hybrid conjoint model developed through the 81 profile assessments and comparing that model against the average probability of success for the three individuals who scored the profile through the profile assessment. The hybrid conjoint predictive model of sustainability possessed an R-squared value of .770 when using the average probability of success. This result suggests the sustainability model was able to explain variance in expert-derived assessments of organizational sustainability in this research setting.

The model containing all nine factors better captured the practitioner’s profile ratings than a model containing only the three most predictive factors. When only the three significant factors (On-Going Leadership, Staff Motivation, and Resources Devoted to Change) were included in the model, the R-square value dropped to .46. These results indicate that all nine factors should be included in the model.

Discussion

Predictive models of organizational change are limited and a model explaining sustainability behavior is needed to foster a greater understanding of organizational behavior (Molfenter et al., 2005). Sustaining innovation has been identified as a persistent problem in organizational change efforts (Rogers, 1995). Hybrid conjoint and other decision theoretic models have primarily explained individual behavior, and were projected to be capable of explaining organizational behavior following the implementation of a change. The sustainability predictive model studied explained 77 percent of variance. This suggests the nine-factor model could lead to a valid model for predicting innovation sustainability and provides a set of factors to be considered when studying this behavior. While only three factors were significant in the regression analysis, the model should be tested as constructed, across organizational contexts before determining if specific factors can be deleted without compromising validity. Omitting factors at this point could jeopardize the model’s integrity. Furthermore, this paper represents beliefs about change sustainability, not actual practice. Testing in actual practice is a next step in the validation process.

Building on past sustainability of organizational change research, the nine-factor model supports other research findings that: individual staff commitment to change is important (Dale, Boaden, Wilcox, & McQuarter, 1999; Jacobs, 2002); organizational budgetary resources facilitate change sustainability (Reisner, 2002); and leadership plays a consistent role in organizational change as well as the sustainability of a change (Dawson, 1994; Pettigrew, 1985).

The factors that predict change implementation and change sustainability do differ. The studied model had several factors not present in Gustafson’s organizational change manager (OCM) model used to predict innovation adoption. For instance, change reversibility—the ability of the change to be reversed—is not present in change implementation models. The ability to design changes so they will not accidentally or intentionally be discontinued has relevance. As an example, the implementation of a physician order entry system that requires physicians to place pharmacy orders electronically would make the change back to often illegible hand-written orders, irreversible. Other factors from the sustainability model not in Gustafson’s OCM model are political environment (e.g., cross-department cooperation in sustaining the change), evidence of effectiveness of the change, champion/staff turnover, and staff development. Many more similarities exist between the sustainability and adoption models. For instance, the need for ongoing leadership, staff motivation, resources for the change, external pressure (for the change), and change adaptability (or flexibility) are common to both.

A couple of weaknesses existed in the model design. First, Aiman-Smith et al. (2002), suggest that using run-in profiles to acclimate respondents to their task creates more reliable results. We did not use run-in profiles due to our concerns regarding respondent fatigue. One benefit of using a small number of profiles is that we had a 100 percent response rate.

Second, the sample size limited some aspects of the research analysis. The applied block design of 81 profiles only allowed us to measure the attribute main effects, and it limited our ability to adequately assess and test for interaction effects between the attributes (Conner & Zelen, 1959). Future research should seek to adequately balance respondent burden with the need to measure both attribute main and interaction effects.

The model contributes to sustainability research by identifying potential factors to be used in this research, as well as a model for explaining variance in sustainability outcomes. The research also indicates that with some variations and modifications, hybrid conjoint analytic models show promising results in predicting organizational change.

The model’s greatest benefit could come from its application in field settings. If the sustainability model continues to prove to be reliable as well as valid, this predictive model could have several applications. In initial field applications, the model has been applied during a change project to identify pending weaknesses, and after the innovation is adopted to predict likelihood of sustainability. The tool is completed by the change team. A computer-based version of the tool generates a likelihood of success score and identifies areas of weakness that the change team or management should address in order to increase their chances of sustaining a change. For instance, if staff motivation does not score high, staff members can be queried on why they are not more supportive of the change to generate ideas on how to increase staff acceptance.

The present need is for sustainability research to inform practice and prevent the erosion of successful changes. This degradation often has a negative impact on both the organization and the customers it serves. The ultimate solution to the lack of sustainability is continued improvement. This paper does not address organizational improvement, but does address the emerging concern of discontinued use of successful changes.

In sum, the presented model, in an initial validation test, was capable of predicting sustainability in organizational settings and could advance a field of inquiry that has received limited research attention even though sustaining innovation has been identified as a persistent problem in organizational change efforts. The model could be timely in the United States because health reform is likely to impose considerable change on organizations. A model to evaluate and predict sustainability would be useful to managers and policy makers. The model addressed in this paper, along with the subject of sustainability of changes, needs further testing and attention in order to increase the understanding of organizational change and its long-term effects.

Acknowledgments

The authors would like to acknowledge the support of NIDA grant 5R01DA020832-05 in the preparation of this manuscript and helpful comments of David H. Gustafson. Professor Emeritus, College of Industrial Engineering, University of Wisconsin – Madison.

References

  1. Aiman-Smith L, Scullen SE, Barr SH. Conducting studies of decision making in organizational contexts: A tutorial for policy-capturing and other regression-based techniques. Organizational Research Methods. 2002;5(4):388–414. [Google Scholar]
  2. Anderson RD, Lenz RT. Modeling the impact of organizational change: A Bayesian network approach. Organizational Research Methods. 2001;4(2):112–130. [Google Scholar]
  3. Bosworth K, Gingiss PM, Potthoff S, Roberts-Gray C. A bayesian model to predict the success of the implementation of health and education innovations in school-centered programs. Evaluation and Program Planning. 1999;22(1):1–11. [Google Scholar]
  4. Buchanan D, Fitzgerald L, Ketley D, Gollop R, Jones JL, Lamont SS, et al. No going back: A review of the literature on sustaining organizational change. International Journal of Management Reviews. 2005;7(3):189–205. [Google Scholar]
  5. Caldwell DF, Chatman J, O'Reilly CAIII, Ormiston M, Lapiz M. Implementing strategic change in a health care system: The importance of leadership and change readiness. Health Care Management Review. 2008;33(2):124–133. doi: 10.1097/01.HMR.0000304501.82061.e0. [DOI] [PubMed] [Google Scholar]
  6. Cheng JLC, Kesner IF. Organizational slack and response to environmental shifts: The impact of resource allocation patterns. Journal of Management. 1997;23(1):1–18. [Google Scholar]
  7. Cooksey R. Judgment analysis: Theory, methods, and applications. San Diego, CA: Academic Press; 1996. [Google Scholar]
  8. Dale B, Boaden RJ, Wilcox M, McQuarter RE. Sustaining continuous improvement: What are key issues? Quality Engineering. 1999;11:369–377. [Google Scholar]
  9. Damanpour F. In: Organizational innovations effectiveness, adoption and organizational performance. West MA, Farr JL, editors. New York: Wiley; 1991. [Google Scholar]
  10. Damanpour F. Organizational size and innovation. Organization Studies. 1992;13(3):375–402. [Google Scholar]
  11. Dawson P. Organizational change: A processual approach. London: Paul Chapman; 1994. [Google Scholar]
  12. Escrig-Tena A, Bou-Llusar J. A model for evaluating organizational competencies: An application in the context of a quality management initiative. Decision Sciences. 2005;36(2):221–257. [Google Scholar]
  13. Ferlie EW, Shortell SH. Improving the quality of health care in the United Kingdom and the United States: A framework for change. The Milbank Quarterly. 2001;79(2):281–315. doi: 10.1111/1468-0009.00206. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Fixsen DL, Naoom SF, Blasé KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. National Implementation Research Network; 2005. [Google Scholar]
  15. Frambach R, Schillewaert N. Organizational innovation adoption: A multilevel framework of determinants and opportunities for future research. Journal of Business Research. 2002;55:163–176. [Google Scholar]
  16. Graham ME, Cable DM. Consideration of the incomplete block design for policy-capturing research. Organizational Research Methods. 2001;4(1):26–45. [Google Scholar]
  17. Granados A, Jonsson E, Banta HD, Bero L, Bonair A, Cochet C, et al. EUR-ASSESS project subgroup report on dissemination and impact. 1997. [DOI] [PubMed] [Google Scholar]
  18. Green PE, Goldberg SM, Montemayor M. A hybrid utility estimation model for conjoint analysis. The Journal of Marketing. 1981;45(1):33–41. [Google Scholar]
  19. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Gustafson DH, Fryback DG, Rose JH, Yick V, Prokop CT, Detmer DE, et al. A decision theoretic methodology for severity index development. Medical Decision Making. 1986;6(1):27–35. doi: 10.1177/0272989X8600600106. [DOI] [PubMed] [Google Scholar]
  21. Gustafson DH, Johnson PR, Molfenter TD, Patton T, Shaw BR, Owens BH. Development and test of a model to predict adherence to a medical regimen. Journal of Pharmacy Technology. 2001;17(5):198–208. [Google Scholar]
  22. Gustafson DH, Sainfort F, Eichler M, Adams L, Bisognano M, Steudel H. Developing and testing a model to predict outcomes of organizational change. Health Services Research. 2003;38(2):751–776. doi: 10.1111/1475-6773.00143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Gustafson DH, Bosworth K, Treece C, Moberg P, Palmer C, Wu YC. Predicting adolescent problem use of marijuana: Development and testing of a Bayesian model. International Journal of Addictions. 1994;29(7):861–886. doi: 10.3109/10826089409047915. [DOI] [PubMed] [Google Scholar]
  24. Gustafson DH, Hundt AS. Findings of innovation research applied to quality management principles for health care. Health Care Management Review. 1995;20(2):16–33. [PubMed] [Google Scholar]
  25. Ham C, Kipping R, McLeod H. Redesigning work processes in health care: Lessons from the National Health Service. The Milbank Quarterly. 2003;81(3):415–439. doi: 10.1111/1468-0009.t01-3-00062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Jacobs RL. Institutionalizing organizational change through cascade training. Journal of European Industrial Training. 2002;26:177–182. [Google Scholar]
  27. Lehman WEK, Greener JM, Simpson DD. Assessing organizational readiness for change. Journal of Substance Abuse Treatment. 2002;22(4):197–209. doi: 10.1016/s0740-5472(02)00233-7. [DOI] [PubMed] [Google Scholar]
  28. Magat WA, Kip Viscusi W, Huber J. Paired comparison and contingent valuation approaches to morbidity risk valuation. Journal of Environmental Economics and Management. 1988;15(4):395–411. [Google Scholar]
  29. Massati RR, Sweeney HA, Panzano PC, Roth D. The de-adoption of innovative mental health practices (IMHP): Why organizations choose not to sustain an IMHP. Administrative Policy in Mental Health. 2008;35:50–65. doi: 10.1007/s10488-007-0141-z. [DOI] [PubMed] [Google Scholar]
  30. Meyers PW, Sivakumar K, Nakata C. Implementation of industrial process innovations: Factors, effects, and marketing implications. Journal of Product Innovation Management. 1999;16(3):295–311. [Google Scholar]
  31. Molfenter T, Gustafson D, Kilo C, Bhattacharya A, Olsson J. Prospective evaluation of a Bayesian model to predict organizational change. Health Care Management Review. 2005;30(3) doi: 10.1097/00004010-200507000-00011. [DOI] [PubMed] [Google Scholar]
  32. Olsson J, Overetveit J, Kammerlind P. Developing and testing a model to predict outcomes of organizational change. Quality Management in Health Care. 2003;12(4):240–249. doi: 10.1097/00019514-200310000-00009. [DOI] [PubMed] [Google Scholar]
  33. Oplauch J, Swallow S, Weaver T, Wessells C, Wichelns D. Evaluating impacts from noxious facilities: Including public preferences in current siting mechanisms. Journal of Environmental Economics and Management. 1993;24:41–59. [Google Scholar]
  34. Pettigrew AM. The awakening giant: Continuity and change in ICI. Oxford: Basil Blackwell; 1985. [Google Scholar]
  35. Reisner RAF. When a turnaround STALLS. Harvard Business Review. 2002;80(2):45–52. [Google Scholar]
  36. Rogers EM. Diffusion of innovation. 4. New York: Free Press; 1995. [Google Scholar]
  37. Ryan M. A role for conjoint analysis in technology assessment in health care? International Journal of Technology Assessment in Health Care. 1999;15(03):443. [PubMed] [Google Scholar]
  38. Saridakis C. A hybrid conjoint model for the identification of the UK supermarket choice determinants: An exploratory study. The International Review of Retail, Distribution and Consumer Research. 2009;19(2):103–133. [Google Scholar]
  39. Sheldon RJ, Steer JK. The use of conjoint analysis in transport research. Planning and Transport Research and Computation. 1982:145–158. [Google Scholar]
  40. Simpson DD. A conceptual framework for transferring research to practice. Journal of Substance Abuse Treatment. 2002;22(4):171–182. doi: 10.1016/s0740-5472(02)00231-3. [DOI] [PubMed] [Google Scholar]
  41. Venkatesh V. Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information Systems Research. 2000;11(4):342. [Google Scholar]
  42. Von Winterfeldt D, Edwards W. Decision analysis and behavioral research. New York, NY: Cambridge University Press; 1986. [Google Scholar]
  43. Wardman M. A comparison of revealed preference and stated preference models. Journal of Transport Economics and Policy. 1988;22:71–91. [Google Scholar]
  44. Wejnert B. Integrating models of diffusion of innovations: A conceptual framework. Annual Review of Sociology. 2002;28(1):297–326. [Google Scholar]
  45. Wind J, Green PE, Shifflet D, Scarbrough M. Courtyard by Marriott: Designing a hotel facility with consumer-based marketing models. Interfaces. 1989;19(1):25–47. [Google Scholar]

RESOURCES