Crossing the Implementation Chasm: A Proposal for Bold Action

Abstract

As health care organizations dramatically increase investment in information technology (IT) and the scope of their IT projects, implementation failures become critical events. Implementation failures cause stress on clinical units, increase risk to patients, and result in massive costs that are often not recoverable. At an estimated 28% success rate, the current level of investment defies management logic. This paper asserts that there are “chasms” in IT implementations that represent risky stages in the process. Contributors to the chasms are classified into four categories: design, management, organization, and assessment. The American College of Medical Informatics symposium participants recommend bold action to better understand problems and challenges in implementation and to improve the ability of organizations to bridge these implementation chasms. The bold action includes the creation of a Team Science for Implementation strategy that allows for participation from multiple institutions to address the long standing and costly implementation issues. The outcomes of this endeavor will include a new focus on interdisciplinary research and an inter-organizational knowledge base of strategies and methods to optimize implementations and subsequent achievement of organizational objectives.

Introduction

Numerous biomedical system implementation failures have been recorded in the literature in recent years. However, the exact number of information system failures is unknown as organizations and individuals are reluctant to publish about these problems. With major computerized provider order entry (CPOE) failures costing upwards of $30 million ¹ and possibly creating threats to patient safety, ² health care information technology (IT) projects will increasingly be viewed in terms of the opportunity costs and risks associated with implementation. While large-scale failures of health care IT systems pose significant problems, smaller scale failures resulting from incomplete delivery on expectations also are troubling.

Survey results regarding information technology implementations across a wide range of industries from the Standish Group in the UK suggest that 18% of IT implementations are outright failures, while an additional 53% of IT implementations are challenged during implementation. ³ Additional problems are seen in cost overruns and delays in project completion. A variety of reports have suggested causes for implementation failures, including lack of user involvement, poor communication, lack of attention to people and organizational issues, and poor project planning. ^4,5,6

The 2007 American College of Medical Informatics (ACMI) Winter Symposium focused on “Research Issues in the Implementation of Information Systems.” Through presentations and discussion, the group explored many approaches to better understanding and solving implementation problems with clinical and research systems. One presentation was “Crossing the Implementation Chasm.” This paper is an expansion of the original presentation and includes the discussion from a sub-group of ACMI participants and further work by this paper's authors.

Big Problems Require Big Solutions

Information technology systems test resilience in the health care environment in ways that are not well understood. ⁷ Development and dissemination of useful strategies and insights for safe, efficient, and productive implementation of health care IT is an urgent national requirement requiring large-scale concerted action.

Collins and Porras found that the one factor that distinguished successful efforts from unsuccessful ones was the use of ambitious, even outrageous, goals to motivate people and focus them toward concrete accomplishments. A BHAG (pronounced “bee-hag”) is a Big Hairy Audacious Goal.

“A true BHAG is clear and compelling and serves as a unifying focal point of effort and acts as a catalyst for team spirit. For example, the 1960s moon mission did not need a committee to spend endless hours wordsmithing a ‘mission statement’. The goal itself was so easy to grasp, so compelling in its own right that it could be said one hundred different ways, yet be easily understood by everyone.” ⁸

The American College of Medical Informatics symposium participants created the BHAG depicted in ▶.

Figure 1.

BHAG for implementation research.

Additionally, the group articulated several objectives and statements related to context that elaborates the overall goal:

• • To gain an understanding of how to implement information technology systems in multiple types of settings to enable

  • • More effective implementations (at the 90–95% success rate)
  • • A decrease in the time required to implement (to approximately 20% of the current time)

• • To identify global knowledge required for successful local implementations

• • To create the needed tools from research knowledge for successful implementations

• • To determine what is the optimal mental “mind-set” of the implementers for the implementation of a new IT system

The participants further recommended that this research needs to take place within the context of multiple settings, e.g. community hospitals as well as academic settings and physician offices; multiple types of areas/disciplines within the settings, e.g., critical care areas, emergency departments, outpatient environments; include multiple workflows in the context of the setting; and at the same time address why “x” worked and “y” did not work in the various settings.

We do not presume that these objectives arise in a vacuum. Implementation research has a rich history and continues to be the focus of numerous productive individuals and small groups of researchers worldwide. Approaches from many disciplines have been used to understand the processes by which IT adoption occurs, identify barriers to implementation success, and develop strategies to avoid or resolve these problems. However, the reality of practice does not reflect a ‘problem solved’ but rather emphasizes the gaps between what we know and what we do and reveals areas where there still are gaps in understanding. Models of technology adoption (such as TAM and UTAUT) ⁹ are useful for research but still limited in applicability for those engaged in the daily practice of system implementation.

The following sections within this article examine another well-regarded model of adoption and identify its limited ability to represent when in the implementation process the most significant barriers to adoption occur. In our re-thinking of the model and the complex factors that contribute to implementation success and failure, we present the way forward for changes on a scale that would profoundly benefit future health IT implementations and allow them to realize the great promise that they hold. We liken this BHAG to the goals of the NIH Roadmap and propose adopting a similar team science approach for implementation research that will lead to a better understanding of how, why, and when challenges occur and create evidence-based strategies for success.

Making Sense of Implementation Challenges

The complexity of health care often makes it impossible to implement information systems simultaneously throughout an organization. Therefore, most information systems are implemented according to a specific strategy. There are multiple theories to describe technology adoption, but one that has been successfully followed is Everett Rogers' Diffusion of Innovations (DOI) theory. ¹⁰ The most important aspect of DOI is that adoption is not a momentary, irrational act, but an ongoing process that can be studied, facilitated and supported. ¹¹

Rogers classified adopters on the basis of innovativeness. According to his theory, members of a population vary greatly in their willingness to adopt a particular innovation. People adopt in a time sequence, and they may be classified into adopter categories on the basis of when they first begin using a new idea. The distribution of innovativeness within a population resembles a normal curve beginning with “Innovators” who lead in adopting an innovation and make up about 2.5% of a population. “Early Adopters” make up approximately 13.5% of a population and this group contains the majority of the opinion leaders.

Most people will fall into either the “Early Majority” (34%) or the “Late Majority” (34%) categories. They can be persuaded of the utility of new ideas, but the pressure of peers is necessary to motivate adoption. “Laggards,” who will resist adopting an innovation as long as possible, comprise about 16% of a population.

Our Approach

While applying Rogers' original process-stage model of innovation to technology adoption can be useful, a more nuanced understanding of each group and the organizational context of implementation is needed. Based on personal experience, observations and discussions with information system implementers across the world about their experiences, we believe there are two major chasms in the Rogers' Diffusion of Innovation Model. In researching the concept of chasms within the Rogers model the only reference to chasms was located in the sales and marketing literature. ¹²

The first chasm is between the early adopter group and the early majority group. The second chasm is present at the end of the late majority and before the laggard group. ▶ represents our depiction of these two chasms.

Figure 2.

A Modified view of Diffusion of Innovation depicting major implementation chasms.

The first chasm in our model represents the challenge of getting beyond the initial, enthusiastic groups of adopters. Implementers often start with these groups because they promise early “wins,” essential for convincing skeptics down the road that the technology is worth using. In clinical settings, these may be groups that have been on the vanguard of technology adoption in the past and may have characteristics that facilitate the infusion of technology into practice, such as minimal (relative) complexity of practice. The implementation team is perhaps deceived into thinking that subsequent clinical units will go just as smoothly and may be surprised by the challenges of automating highly complex clinical units (such as chemotherapy units or ICUs) and the accumulation of technical problems, which might be manageable with only a few users but get out of hand with a few hundred.

The second chasm comprises the challenges in bringing the final clinical groups on board with a new system. Some groups have such intractable issues with task-technology fit that the system may never meet their needs. Others do not adopt because of complex ecological ¹³ or political concerns. An example of this situation is presented by a chronic disease ambulatory care clinic. ¹⁴ While the majority of the ambulatory clinics at the institution utilized an electronic medical record system, the clinic chose to maintain paper charts. Because of institutional mandates to use the EMR, several documentation activities were duplicated between the paper and electronic records. The initial reasons for continuing to use paper charts included the need for a graphical display used in documenting patient status as well as concerns by the physicians that the new technology would be less efficient and less effective than paper charts. The continued effort involved in duplicating the documentation enforced initial perceptions that the system was cumbersome. The perceptions and experience of this group were not mirrored elsewhere in the institution, where the EMR implementation was viewed as being largely successful. Three years after widespread adoption of the EMR, this clinic continued using paper charts and showed no desire to fully transition to the electronic system. Whatever the reason, it is clear that the diffusion of information technology does not happen smoothly along a curve, but with bumps in adoption, diversity in infusion ¹⁵ and at least two major chasms.

Reports on IT failures are a valuable contribution to the collective wisdom related to implementation. To move forward and improve the percent of implementation successes, it is critical that we develop strategies and tactics to correct known problems. The next section of this paper explores the factors that contribute to the two implementation chasms. The contributors are classified into four categories: Design, Management, Organization, and Assessment. These categories and potential solutions will be discussed. However, we maintain that addressing the issues within each of these areas is necessary, but not sufficient to create lasting improvement in the implementation of clinical IT. Ultimately we must focus on the interactions among design, management, organization and assessment domains. This interdisciplinary challenge will be reviewed, as well as a specific proposal for moving forward that originated in the ACMI 2007 symposium.

What Creates the Chasms?

This section reviews contributors to the implementation chasms in four categories: Design, Management, Organization and Assessment. There are other ways to classify implementation issues; ¹⁶ however, we felt that this organization best captured some of the important distinctions that result in “silos” of activity and knowledge in informatics implementations. Research has produced new insights and methods in all of the areas, 17,18,19,20 but the results have been slow to translate into practice. The consequence has been that these four areas, all sources of potentially exciting innovations in efficiency, safety, care-giving, and organizational learning, are instead frequently the sources of delay, hazard, and the perpetuation of wasteful, ineffective practice patterns. In other words, they prop open the chasms instead of bridging them. ▶ depicts the impact of the major implementation problems—represented as blocks that maintain the chasm. For an example of how these obstacles manifest themselves in the real world, see the scenario in ▶. The question is what led to this scenario? Was it the system design, the project management process, or the organization's goal to implement as soon as possible? The following section discusses problems in each domain.

Figure 3.

A depiction of the major problem blocks creating the chasms.

Figure 4.

Implementation scenario.

Design

Physical properties and information models of IT products

The design of technology can directly impact adoption of information systems. An important part of system design involves the design of the user interface. Fields such as human-computer interaction, interaction design, cognitive engineering, and human factors engineering provide guidance on usability-oriented design. ²¹ Although the design of the computer interface is important, the appropriateness of the technology to the environment also contributes to the success of a design. Incorporation of new technology into an existing work environment is a two way street. As Berg has observed, there must be a “convergence of tool and practice,” ²² whereby the IT system must be customized to accommodate the activities of workers, but the workers must also change their practices to accommodate the use of the system. Goals of healthcare informatics such as improving patient safety, increasing efficiency, decreasing costs, and providing effective patient care can be compromised when systems are not designed for usability and with an awareness of workflow-related issues.

Management

Management of the Planning and Rollout of a Specific Information System

Clinical IT projects are often vast, with a need for broad (many users) and deep (extensive use of functionality) adoption across the enterprise in order to achieve the promise of the software. Workflow often is changed dramatically with the introduction of new information and artifacts that must be integrated into a new form of safe practice. The implementation of an IT system is a form of organizational change and thus requires a detailed plan that is driven by both capacity for change and context of change. Capacity represents the ability of the organization to invest in high-quality training, extensive support at go-live, and managers who can respond flexibly to changes in the environment so that patient safety is maintained as the highest priority. Context is the environment to which the implementation plan must adapt. A rollout schedule, for example, must take into account the many interdependencies that exist among clinical units as well as organizational changes that are occurring during the implementation. With clinical IT projects often taking a year or more to complete, it is challenging to predict the intersections of the system with other changes, such as the opening of a new clinical unit or the deployment of other significant technology. Most hospitals rely heavily on the IT vendor to prescribe the implementation plan for the software. However, the vendor's knowledge of both capacity and context is limited, and the organization must take responsibility for these issues.

Organization

Issues Related to Organizational Structure, Leadership, and Processes

Another “block” holding the chasm open comprises operational-organizational issues. As information systems continue to become more embedded and more essential within organizations, they also become the source of more operational frustrations. Examples of the frustration include: downtime (expected but too long or unexpected), workflow changes, getting help when needed (traditional help desk functions and on-site technical support), the perception of the IT department not listening to or being disconnected from the operational needs, etc. There are also mega-organizational issues. Some of these include organizational priorities—who establishes the priorities, timetables—the length of the process from perceived need to implementation and adoption, wanting and needing more IT products than the IT department can efficiently and effectively deliver, potential increased funding for the IT department while the operational areas need to reduce expenses and not seeing the full value of the IT products, etc.

These frustrations cause friction and diminish trust, an important social lubricant, between the operational areas and the IT department. The operational people may feel that the IT people do not really understand their needs and that the IT area is disconnected or “out-of-synch” from the operational areas. In turn the IT staff feels that they have “answers” to the needs of the operational areas, but that the operational areas either will not listen or they do not understand how difficult it is to complete the projects. This distrust can lead to indifferent or antagonistic responses to new and upgraded systems.

Assessment

Evaluation of Systems: Use, Outcomes, Impact on Work, and Structure of Care

This paper considers assessment in the broadest sense, including strategies for evaluating organizational readiness (is the organization ready to commit?), assessing the immediate context for implementation (is this unit ready to go live?), and evaluating the implementation in formative and summative manners. There are opportunities in both research and education to improve the “infrastructure” of assessment, including:

• • skills of the developers and implementers in understanding the sociotechnical environment;

• • volume and utility of implementation research;

• • diversity in approaches to evaluation;²³

• • accessibility of implementation research for implementers and designers.

There are limited opportunities for formal training in sociotechnical systems analysis. Informatics trainees who are exposed to disciplines such as human factors, industrial engineering and cognitive science, and methods such as ethnography and workflow analysis will be better prepared for participation in both real-world implementation and research that improves it.

The Real Payout: Interaction among Currently Disparate Domains of Knowledge and Activity

Although daunting issues arise within each of these categories, the impact is compounded by the lack of interaction among software designers, researchers, operational personnel, management and end users. The scenario discussed in ▶ is extended in ▶ to reveal some of the problems that can arise when integration across domains is not considered.

Figure 5.

Implementation scenario continued.

Many of the battery-related problems discussed in ▶ could have been resolved by better integration among the different groups involved in the system design, implementation, and use. The system designers were unaware of the hectic nature of the ICU and failed to build in enough warnings and data safeguards. The management of the organization failed to communicate important details of how the system was being used in the acute care units, details that they considered trivial. The end users were not actively involved in the design and implementation process and were limited to reporting concerns to their managers, rather than directly to the designers. Well-known principles of human factors engineering research, such as the importance of adding affordances ²⁴ to help the end users “do the right thing,” were not incorporated into the design. Nurses were unable to attend training courses before implementation on the ICU because of the rapid deployment schedule required to meet business goals. Even if all of the nurses had been trained, the brief mention of battery management in the training material would likely not have been noticed. On their own, any one of these single issues might not have resulted in implementation failure. However, the persistent lack of integration among the many groups involved in design and implementation resulted in a loss of trust by the end users, significant amounts of wasted time dealing with system problems and documenting everything twice, and the eventual complete implementation failure on the ICU.

▶ depicts the interaction of the four domains. We maintain that focused research and development in each domain is necessary, but not sufficient to achieve significant gains in the utility of health IT. Investment in the interaction among these domains will return not only new knowledge about implementation, but better products.

Figure 6.

Focusing on interactions among domains can result in synergistic benefits.

As we address the interaction among the four domains, we must also consider the trends in information technology that will be implemented in the near future. While workflow and implementation of informatics systems have room for improvement in current health care institutions, the types of challenges will evolve as the future directions of information systems transform to patient-centered care. The not so distant future is predicted to integrate personal health records and personalized (genomic) medicine into the health care environment. If informatics research only focuses on bridging the current gaps in knowledge and practice, we will forever be playing catch up as the landscape continues to change. Essential work will include preparing for the challenges on the horizon by understanding what existing knowledge and experience will translate to the new environments, and what types of new and unique research must be conducted to prepare for the workflow issues in patient-centered care.

Team Science for Implementation

The challenges we have described represent only the tip of the iceberg. The multi-domain interactions suggest the need for a more holistic and collaborative approach to address the implementation and adoption of information technology in health care organizations. Research and academic programs in these directions will be essential for addressing these challenges and reaching the goals of the American College of Medical Informatics BHAG.

The participants at the American College of Medical Informatics symposium recommended that the path forward towards achieving the BHAG involves the development of a new approach, Team Science for Implementation. Team Science has already been explored in the basic sciences to bring biologists, chemists, physicists, engineers, and others together to tackle complex and difficult research questions. A similar multi-disciplinary team approach is needed to find solutions to the implementation chasms.

The creation of a multi-institutional virtual team for research revolving around informatics-based technology implementation will allow for the comparative studies across multiple health-care settings. This team science approach across institutions will facilitate the connection between research and practice by bringing together system developers, project managers, executives, evaluators, and both qualitative and quantitative researchers. Collaboration with departments outside of informatics and faculty who are experts in these areas will be essential to the team science approach.

Team Science for Implementation would collect process and outcome data from a significant number of information system implementations, including those considered a success and those considered failures. From this data researchers can create benchmark strategies for success that will enable us to build bridges across the chasms or to actually fill the chasms.

In addition, institutions with informatics training programs can develop curricula tracks for implementation research that includes training in disciplines such as social psychology, business/management, human factors engineering and qualitative research methodologies such as ethnography and action research.

Team Science for Implementation will focus on a set of overarching research questions and aims, such as:

• • What are the generalizable lessons that can be gleaned from highly successful implementations?

• • Can information from implementation challenges be systematically incorporated into models that allow prediction of implementation issues?

• • How do individuals and groups integrate technology into workflow?

• • Are there different strategies to increase adoption depending on the users' professional discipline?

• • What are the differences between technology designers' perceptions of needs/priorities and practitioners' perceptions?

• • How can the implementation of EMRs and PHRs be coordinated to optimize the patients' health and wellness information along with their clinical care?

• • How might we create an “organizational development” mindset around the implementation of health IT, focusing on the active production of “fit”²⁵ between technology and work?

Summary

Implementing information systems within health care continues to challenge people daily. There are a number of major issues that lead to chasms in the implementation process. These issues start with the lack of understanding of what the users need, move to the creation or purchase of systems whose design will not support the user needs, then to the overall management of the process of implementation. These major blocks holding the chasm open are widened by issues with the organization and operational areas and the lack of attention to evidence that already exists regarding implementation. The financial and human costs of ineffective implementation are incalculable.

The American College of Medical Informatics proposes a BHAG (Big Hairy Audacious Goal) of creating a Team Science for Implementation that will produce new knowledge on how to deploy information technology in ways that help healthcare organizations meet their performance objectives.

Achieving this BHAG will not only save health care billions of dollars, we would save other industries billions as well. Equally important, we would save the human “pain” of dealing with information systems that are perceived as detriments to the patient and to the work environment.