Contextual Implementation Model: A Framework for Assisting Clinical Information System Implementations

Abstract

Objective

This paper presents a multiple perspectives model of clinical information system implementation, the Contextual Implementation Model (CIM). Although other implementation models have been developed, few are grounded in data and others fail to take adequate account of the clinical environment and users’ requirements.

Design

The CIM arose from qualitative data collected from four clinical units in two large Australian teaching hospitals. The aim of the study was to explore physicians’ test management work practices associated with the compulsory use of a hospital-wide, mandatory computerized provider order entry (CPOE) system.¹ The dataset consisted of non-participatory observations of physicians using CPOE (n=55 sessions) and interviews with health professionals (n=28) about test management work practices. Data were analyzed by two researchers independently using an iterative grounded approach.

Results

A core underlying theme of ‘contextual differences’ emerged which explained physicians’ use of the CPOE system in the sites. The CIM focuses attention on diversity at three contextual levels: the organizational level; the clinical or departmental level, and the individual level. Within each of these levels there are dimensions for consideration (for example, organizational culture, leadership and diverse ways of working) which affect physicians’ attitudes to, and use of, CPOE.

Conclusion

The CIM provides a contextual differences perspective which can be used to facilitate the implementation of clinical information systems. Developing a clinical information system implementation model serves as a framework to guide future implementations to ensure their safe and efficient use and also improve the likelihood of uptake by physicians.

Introduction

Despite reported benefits of electronic order management systems ^2–4 implementations have been slow with very few hospitals using hospital-wide computerized provider order entry (CPOE) systems. ⁵ To realize the potential of clinical information systems it is important to explore existing successful and unsuccessful implementations in-depth. A number of researchers ^6–12 promote the use of qualitative research methods to evaluate health information systems and explore their uptake. These in-depth analyses of how clinicians use and adapt clinical information systems into their existing work practices can then provide frameworks to assist future implementations.

Studies have reported CPOE implementation experiences and highlighted barriers to successful implementation. ^13–21 Prominent in the health informatics literature are Ash and her colleagues, who have undertaken qualitative work exploring in-depth the implementations of CPOE across US hospitals. ¹¹ This group has continued to refine a set of principles which can be used to guide successful implementations of CPOE. Although not presented as a model these principles form a framework based on the Multiple Perspectives Approach originally proposed by Linestone, ²² which reflects the complexity of implementing clinical information systems. ²³ They used this framework to derive twelve principles for successful implementations of CPOE systems. ²⁴ These data consisted of discussions from a panel of experts combined with observations and interviews from the field which were analyzed using grounded theory and were grouped under four categories: computer technology; personal principles; organizational principles; and environmental issues. ²⁴ The twelve principles reflect the need to consider multiple issues when implementing computerized test management systems and they highlight the interrelationships between the technology, clinical information, people, and organizational issues.

Models for clinical information system implementation have been reported in the health informatics literature. Two clinical information system evaluation models which have drawn on behavioral theory are the technology acceptance model (TAM) ^25,26 and the information technology adoption model (ITAM). ²⁷ The TAM focuses on users’ acceptance of technology and the impact of technology on this acceptance. Davis ²⁵ developed a validated measure which can be used to predict perceived usefulness and perceived ease of use of the technology which he theorized were fundamental determinants of whether the system would be used. He also proposed that the design of the information system directly influenced perceived usefulness and ease of use. ²⁶ The ITAM ²⁷ builds on the work of Davis and provides a framework for implementations and evaluations with a focus on individual users to predict the adoption of voluntary information technology. This model focuses on end-user fit, user perceptions of the usefulness and ease of use of the innovation, and adoption and utilization. Both TAM and ITAM fail to include the organizational and clinical environments which have been shown to influence clinical information system implementations. 10,13–15,28

A behavioral science perspective was also used by Kukafka and colleagues in their development of a multi-level integrated framework for information technology (IT) implementation based on an approach used for planning health promotion programs, where the goal is to change behavior. ²⁹ Their proposed framework is a multiple-factor approach which works through five phases: assessing the needs of the organization; identifying which needs are amenable to IT system solutions; identifying behaviors (individual and collective) associated with system use; identifying factors associated with those behaviors linked to IT use (for example, perceived usefulness, IT skills, rewards), and system use-inducing strategies. Their model rests on two propositions: firstly “IT use is complex, multi-dimensional, and influenced by a variety of factors at individual and organizational levels; and success in achieving change is enhanced by the active participation of members from the target user groups.” ²⁹ This model is derived from an assessment of literature and its value lies in its utilization of several perspectives of human behavior to explain IT use.

An important theory of human behavior, the Social Cognitive Theory, ³⁰ has also been used in information science research to explain individuals’ reactions to computer technology. ^31–33 Under this theory computer use (the behavior) is influenced by beliefs which might affect an individuals’ behavior, independent of the perceived outcomes. Social Cognitive Theory provides a framework for understanding, predicting and changing behavior and it identifies all human behavior as a dynamic interaction between personal factors, behavior and environmental influences. The concept of triadic reciprocality between the three determinants of behavior allows behavioral change efforts to be directed at personal, environmental, or behavioral factors.

A number of implementation models focus on the fit between the task, the clinical environment, and the technology. ^34–36 Aarts et al. described a model where there needs to be an effective fit between clinical work, the organization of medicine and the healthcare system, and the information technology, to obtain the best outcomes for clinical care delivery. ³⁵ Goodhue and Thompson ³⁴ proposed a technology-to-performance chain model where the fit between the task and the technology is one component, combined with utilization of the system. The FITT model (Fit between Individuals, Task, and Technology) arose from literature and is based on the interaction or fit of the individual (the users), the technology (usability, functionality) and the clinical tasks and processes. ³⁷ The organizational influence on IT integration is included in the individual aspect of the FITT theory as individuals work in various roles and various groups in organizations.

Scholars have discussed the link between the implementation of new technology in health and organizational change. ^28,38–41 Much of this work incorporates information on why people resist change and strategies for overcoming this resistance to facilitate the implementation of new IT. A key change theory, based on extensive empirical work, which is applicable to the implementation of IT is the diffusion of innovation theory (DOI). ⁴² This theory proposes an explanation of how individuals and groups adopt new ideas or practices or objects. Roger’s theory provides a framework for explaining how an innovation, which results in large-scale change, permeates throughout an organization, industry, or social environment. The DOI theory has application to the uptake of clinical information technology in large health care organization s such as hospitals and has been harnessed as the key theoretical paradigm in a number of studies. ^14,43–45

As well as theoretically sophisticated change management theories several practical strategies are promoted in the literature as assisting the commitment to large-scale change in health care organizations. Factors such as ensuring organizational readiness for change, promoting broad participation in the change process by all staff, defining the vision for change, clearly articulating the dissatisfaction with the current state, and maintaining clear and consistent communication during the process are some of the more frequently recurring suggestions. 19,20,28,41,46 Further, it is suggested that healthcare managers who are change leaders need to acknowledge the individual characteristics of the organization and change efforts need to be tailored to suit the specific organizational context. ⁴⁷

There are numerous models of user acceptance of technology in the information science literature. ^48,49 We have focused our discussion on models which have been derived from or applied to the health sector. Most existing models and frameworks are complementary and it is recognized that no one approach has emerged as being successful in all situations. ²⁹ However, most models to date have downplayed the importance of diversity and differentiation factors. A gap exists in developing an implementation model which is grounded in data from sites where physicians use an existing mandatory hospital-wide CPOE system and which also acknowledges the complexity of the clinical environment (hospital and clinical unit) and the requirements of users. The Contextual Implementation Model (CIM) described in this paper strives to fill that gap.

Methods

The aim of this paper is to present a clinical information system implementation model whereas a previous study using the same data ¹ described changes to physicians work practices in relation to their use of a hospital-wide, mandatory CPOE system. Therefore, the methodology for this study has previously been reported, ¹ however a summary is provided below with particular emphasis on the analysis of the data which gave rise to the model developed.

Design, Research Settings, and Data Collection

A cross-sectional qualitative design was employed with data collected by non-participatory observations (55 sessions totaling 89 hours) and field interviews (n=28) from four clinical units (two Emergency Departments and two Haematology wards) in two large Australian public metropolitan teaching hospitals Both hospitals used the same hospital-wide, mandatory CPOE system, however, one site had only implemented the test viewing function with ordering still completed manually. The data collected related to physicians’ test management work practices. There was prolonged engagement in the study sites with observation and interview data collected by one researcher between May 2002 and November 2004. ¹ Rigor of the data was achieved through investigator, data source and data analysis triangulation, prolonged engagement in the field and member checking. ¹

Analysis

Data were analyzed using a thematic grounded theory approach ⁵⁰ to develop themes which would account for physicians’ work practices associated with CPOE use. ¹ Further analysis of the three themes which originally arose from the data was undertaken to develop a higher level model which would account for what was central to CPOE usage. The analysis was inductive to allow the themes and relationships to emerge from the data rather than being imposed prior to data collection.

Analysis continued to occur during observations. Using the constant comparative method of grounded theory analysis the coding of data was conducted line by line (open coding) where each line of text was examined and then actions or events within each line were defined. Codes or categories were generated from this line by line coding and were included in a memoing document. The memoing document consisted of notes and reflections on the data. Selective or focused coding followed where codes which accounted for most of the data observed were categorized more precisely. This gave a fit between the emerging theoretical framework and the data. During the selective coding phase we constructed diagrams which attempted to make sense of what was happening and show linkages between the categories. These diagrams were constantly reviewed and refined. They assisted with analysis as they sharpened our reflections and explanations of the data. The value of visual data displays have been suggested by others. ^51,52

Theoretical sampling occurred when we returned to the hospitals to follow-up specific questions related to work processes. When gaps were found in the data we revisited the hospitals to conduct selective interviews to gain new information which would shed light on the emerging theoretical framework. Some argue that theoretical sampling should occur early in the data collection phase and others propose that it should occur after data collection so that “relevant data and analytic directions emerge without being forced.” ⁵⁰ The theoretical sampling we employed occurred after all the initial observations were made. This allowed us to explore explanations that arose from the data without these assumptions being imposed initially. Therefore, coding entailed the development of descriptive categories to identify themes and patterns and then more advanced or inferential coding occurred where the connections and concepts arose. Finally, a framework was developed to provide an explanation and understanding of work process issues surrounding test ordering and viewing and how clinicians integrate manual and electronic systems and manual and electronic information.

Results

Three themes were derived from the observations and interviews in relation to the behavior, attitudes, and talk of doctors and other health professionals centered on their use of point of care clinical information systems: the effect of the hospital and clinical environment; changes to work practices, and physicians’ management of clinical information. ¹ These data were further analyzed, as described in the Methods section, to derive a core, underlying theme which reflected clinicians’ use of the computerized test management system in the Emergency Departments and Haematology wards of the two hospitals. The core theme which arose from this analysis was ‘contextual differences.’ This core theme was then used to develop a model which could be used to facilitate the implementation of clinical information systems at the point of care. The model, known as the Contextual Implementation Model (CIM), focused attention on diversity at three levels or contexts: the organization al level; the clinical or departmental level, and the individual level (▶). Within each of these three contextual levels are dimensions for consideration which affect physicians’ attitudes to, and use of, CPOE. The dimensions for consideration relating to these levels such as ‘culture’, ‘clinical profile’, and ‘ways of working’, and the levels themselves are interrelated and collectively impact on clinicians’ use and acceptance of computerized order management systems.

Figure 1.

Diagrammatic representation of the Contextual Implementation Model.

The three levels of the model and dimensions for consideration are described below.

• • Organizational context leads to diverse attitudes to clinical information systems between hospitals which impact on their acceptance and uptake by clinicians. The dimensions for consideration at this level are: differences in culture between organizations; differences in experiences within the organization regarding previous implementations of information systems, particularly clinical information systems; different stages in the life-cycle of implementation of information systems; different attitudes and levels of support from top management; different resources in terms of money, equipment and staff; diverse sizes and locations of healthcare organization s (for example, rural versus metropolitan); diverse relationships between, and roles of, doctors and nurses which impact on the use of point of care clinical information systems; different organizational structures and mission statements and strategic plans, and the presence of particular ‘special’ people to champion clinical information systems. Hospital level differences also relate to technology infrastructure and the physical layout and age of the hospital and departments.

• • Clinical unit or departmental context relates to the distinct needs, demands and work practices of each environment, which are often associated with the clinical profile of the patients. These variations affect factors such as how, what, why and when tests are ordered and viewed electronically. Cultural diversity is a dimension for consideration at the unit level as subcultures differ in characteristics from other subcultures, and subcultures differ in various ways from the dominant culture. These can impact on clinical information system implementation processes. Diversity between clinical departments can also relate to different: experiences with information technology between departments, for example differences in the previous use of technology between an Intensive Care Unit compared to a general ward; different levels of support and diverse attitudes between departmental heads, and different resources available for information technology between clinical units.

• • Individual context relates to the dissimilar ways doctors and other health professionals use computers at the point of care. This is associated with the diverse ways in which relatively autonomous clinicians think and work. These individual differences between health professionals impact on the test ordering process in terms of: when, where, why, and how to place an order; when, where, and how to view a test result; when and how to communicate test results to other health professionals and the patient; and how to manage manual and electronic data and information sources. These individual level differences will also impact on the collaborative activities which are inherent in the test management process. There is also variation in terms of which health professionals have the authority to order tests (for example, only doctors or doctors and nurses) and amongst health professionals, which levels of doctor or nurse can order. There were also dissimilarities between individuals in terms of their computer literacy and keyboard skills which are related to attitudes to, and use of, computers at the point of care.

These three levels of contextual differences and their dimensions for consideration are situated within the complex external environment of the organization, which includes: influences from clinical professional bodies; government policy; the economic environment; the health system’s features, and the information technology industry. For example, the attitudes and support of professional associations regarding the use of IT at the point of care and government regulations regarding who can place test orders can affect the success of the implementation. These external influences impact to varying degrees on different healthcare facilities depending upon a number of variables, including the services, type, size and structure of the organization. The country where the organization is located affects the use of test management systems as there are diverse work practices and regulations between countries in relation to test ordering. ^20,40 Cross-country differences can create difficulties when integrating a test management system which has been designed outside the local clinical environment. It is important to emphasize the interdependence of the three levels and the dimensions for consideration within them. Some may be more dominant than others depending on the individuals, the clinical department and the organizational context. The three differences levels, and the dimensions within them, are not mutually exclusive, and relate in a dynamic way.

The value of this model is that it views clinical information system implementations from a sociotechnical perspective, acknowledging the complexity and diversity of clinical and organizational environments, and differences between users which need to be taken into account if implementations are to be successful and benefits to patients realized. The sociotechnical approach to patient care information systems emphasizes a user-oriented, work practice perspective. ⁵³ There is an emphasis on the need to acknowledge the healthcare workplace as a complex system where technologies, people, and organizational routines and cultures interact ^9,53,54 and that clinical information should always be considered within its context. ^55,56 It is claimed that the reason clinicians have difficulties in accepting computerized order entry systems is because they are based on a model of work which is linear, rational, and objective and in reality clinical work is “interpretative, interruptive, multitasking, collaborative, distributed, opportunistic, and reactive”. ⁵⁴

The CIM has a practical application by assisting policy makers, hospital managers, clinicians, IT professionals, and educators to appreciate and account for differences and to view implementation from the three differences perspectives. The model explains where difficulties in implementation lie so they can be alleviated. It tells us that barriers manifest at three levels (individual, unit, and organization) and are underpinned by the concept of contextual differences. If these differences are identified and addressed then clinical information system implementations might be facilitated. It will be important for implementation staff to gauge what the differences in each of the three contextual categories are and how these differences would impact on the use of the clinical information system. This assessment should occur pre-implementation, during implementation and post-implementation. If one acknowledges that information system implementations are ongoing and cyclical, and are rarely completed 19,35,57–59 strategies to acknowledge the three dimensions of differences can be continually implemented and refined.

Comparisons of the Contextual Implementation Model with Other Information System Implementation Models

Clinical information systems alter how clinicians work and communicate and hence their implementation needs to be managed as a major change process. Change management models provide strategies to assist implementers manage the change process. They attempt to explain why people resist change and how this can be overcome in order to facilitate the change. These models usually focus more on altering the behavior and work practices of the user to fit with the technology rather than acknowledging the interplay between the technology, the user, and the organization.

Roger’s DOI theory ⁴² proposes that there are two key factors to consider in assessing how quickly an innovation will be adopted: organization al factors associated with readiness to adopt, and the technology’s readiness for adoption. Roger’s organizational variables of leader characteristics, internal organizational structural characteristics (such as complexity, formalization and communication), and openness to the external environment in terms of ideas and knowledge are factors which have also been supported in the literature as important for the implementation of clinical information systems. 10,11,14,18,24,60,61 These organizational factors provide support for the organization al context in the CIM and the dimensions within this level. Rogers classification of individuals into categories based on their response to change (for example, early adopters or laggards) provides further support for the CIM in terms of individual level differences which impact on uptake of innovations.

The key difference between the DOI model and the CIM lies in their distinct foci and underlying constructs. The DOI identifies factors which affect the speed of uptake of an innovation in terms of organizational and technological readiness. Its focus therefore is on the speed of diffusion of the innovation. In terms of the constructs of organization al readiness the DOI theory proposes that certain features of organizations (low centralization, high complexity, low formalization, high interconnectedness, organizational slack and large size) influence uptake. ⁴² These organizational readiness features manifest differently in organizations. If the DOI model is proposing an ‘ideal’ organizational structure for uptake of an innovation then this fails to acknowledge the complexity and diversity of health facilities. Organizations in the healthcare industry, such as hospitals, have differences in their structural features and hence Roger’s model would be difficult to apply consistently. The technological variables proposed in Roger’s theory such as the relative advantage of the innovation and its complexity impact on the rate of adoption. However, this may not necessarily mean that the innovation has value. The compatibility of the innovation with existing values and practices is a difficult area given the ability of clinical information systems to change the relationships between health professionals which can lead to power struggles. The CIM on the other hand focuses on the contextual readiness of the organization, the unit, and the individual. The underlying construct is ‘context’ which acknowledges the diversity of organization s (particularly hospitals), clinical units and health professionals who are users of the clinical information system. The CIM provides a contextual differences perspective which, if identified and taken into account, might more readily facilitate the implementation of clinical information systems.

The individual context level in the CIM is supported by a body of literature which acknowledges that users react differently to the same information system. ^62–64 The dimension of the CIM which acknowledges differences between individual users of the technology also directs us to consider models which focus on this phenomenon. The TAM ^25,26 is a framework for explaining users’ acceptance of technology. It proposes that the overall attitude of the user to an information system is determined by two beliefs: perceived usefulness of the technology; and perceived ease of use. The design of the information system influences these two beliefs and hence has an indirect effect on attitudes towards using the technology and actual usage. ²⁶ Davis’s model can only be applied to voluntary use of an information system and therefore is not applicable to the mandatory use of the computerized test management system as in my study. The ITAM takes the technology acceptance model further by proposing the need for a fit between the technology and the user. ²⁷ This ‘IT-user fit’ enables the user to decide whether to adopt an intent to change and therefore adopt (or not) the technology. Although Dixon acknowledges the importance of organizational issues these are not included in the model. These two models focus on the users without acknowledging the organizational context. The FITT theory (Fit between Individuals, Task and Technology), based on the analysis of literature, builds on these models and does incorporate the organization al context of implementation, although it is not a primary focus. ³⁷ Aarts et al. ³⁵ also focus on ‘fit’ with further emphasis on levels of clinical activity and the context of the healthcare system. The FITT theory is relatively new and has been applied to a case study on nursing documentation but seems to require further refinement and testing in the field. The FITT theory was based on analysis of literature in comparison to the CIM which arose from field observations and interviews. The CIM also provides greater emphasis on the importance of cultural differences (at organizational and team or unit level) to the uptake of new information systems. The importance of organization al culture in the implementation of clinical information systems has been reported in other studies, 10,13,20,64 and hence provides further support for the model.

The TAM and the ITAM focus on explaining the use of information technology in relation to the users’ beliefs, ease of use and the outcome or benefits of using it. In contrast to this, Social Cognitive Theory includes other beliefs which might influence the use of technology, unrelated to the perceived benefits. The Social Cognitive Theory focuses on the concept of self-efficacy (belief or confidence about performing a specific behavior—in this case using information technology) which means individuals acknowledge that the outcome of their behavior will be futile if there is doubt regarding their ability to successfully accomplish the behavior in the first place. This strong link between self-efficacy and individuals’ reactions to information technology has been shown in a number of studies. ^33,65 The Social Cognitive Theory applied to information technology use provides further support for the CIM with its emphasis on self-efficacy (individual context in the CIM), the environment (clinical unit and organizational context in the CIM) and triadic reciprocality (interrelationships between the three contextual levels in the CIM).

Similarly, the CIM builds on Ash’s principles ²⁴ and might be seen as complementary to these, providing another vantage point from which to view clinical information system implementations. The focus of the CIM is not on the technology but the context of the implementation. This context is described on three levels with differences within each level and between levels. Essentially the model is asserting that the technology is meant to be flexible, to be an enabler and to adjust to the situation (organizational, unit and user) rather than the other way around. The technology needs to adjust to the diversity of the environment where it is being implemented and this adjustment needs to occur at three contextual levels.

These more recent models emphasize the need to acknowledge the sociotechnical aspects of clinical information system implementations. There are others who have focused on the sociotechnical nature of clinical information system implementations and the need to acknowledge the interactions between technology, people, their organizational routines and cultural distinctions. ^9,53,56 The literature on the implementation of clinical information systems therefore, highlights important variables to consider. These include the nature of clinical work as messy and unpredictable, and the need to acknowledge the interplay between people, organizations and culture. Studies of this kind provide support for the CIM. They paint a complex picture of interactions and communications between health professionals and how they carry out their clinical work in large complex health care organizations. The CIM provides a framework which explicitly accounts for the contextual complexity which implementation teams encounter.

Limitations

While this model has value in terms of its multiple perspectives orientation, it has limitations. The CIM arose systematically from the data, however, it has not been independently tested. This needs to be done in diverse clinical units, different types and sizes of hospitals, and in different countries. It also requires testing with other applications to ascertain its relevance to other clinical information systems. The model, like others, is a simplification. The key variables included in each of the three dimensions are grounded in the data, however, more variables could be included if a more detailed, fine-grained model was produced. Models are theoretical representations of the way systems or processes work and they tend to reduce complexities to provide explanations. In this respect, the strength of this model is its capacity to highlight important features of differences but its corresponding paradoxical weakness is that it simplifies the real world of implementation. Finally, this model does not take into account the phases of implementation, from the preliminary planning period, through to the staged, iterative, clinical unit by unit, ‘go live’ implementation which occurs in large settings, through to post-implementation. Nor does it provide a framework for how the levels of differences could be tested and what strategies need to be employed to account for the differences. Further work needs to be undertaken to ascertain at which stages, and how, the levels of differences in the CIM should be assessed and considered. This could be achieved by undertaking longitudinal studies, testing the model through the various phases of clinical information system implementation.

Conclusion

The CIM is a way of examining how we should approach the implementation of point of care clinical information systems. Given that implementations are ongoing and iterative this model proposes that there are three contextual levels of difference which must be acknowledged to facilitate planning for, and implementation of, clinical information systems. Prior to implementation of the clinical information system the implementation team should undertake a thorough analysis of the context, at organizational, departmental, and individual levels, which might impact on clinicians’ use of the system. This would entail an analysis of: work practices at the individual and departmental level; an evaluation of computer literacy and keyboard skills of clinicians; an evaluation of work requirements at interdepartmental levels, particularly in relation to laboratories for CPOE; an evaluation of work requirements and policies at an organizational level as they relate to the use of the computerized clinical information system; and evaluations of organizational and team cultures. Differences in these areas can then be planned for and included in the implementation project plan. During implementation, the implementation team would focus on contextual differences at individual and departmental levels and how these differences could be accommodated within the use of the information system. This would require detailed analysis of the functionality of the system and how different clinicians’ needs could be accommodated. Computer literacy and keyboard skills would require targeted training programs. Cultures which are constructive and supportive of innovations would be the goal for the implementation team. Analysis of organizational and team cultures prior to implementation would assist in devising strategies to modify the cultures (both team and organizational) to ones which are receptive to change and supportive of the change process. Interventions could then be devised to improve information sharing amongst team members, encourage support for innovation, provide a clear vision and objectives for staff and thus promote a constructive culture which is receptive to innovations.

Using the CIM will focus the attention of policy makers, information system designers, hospital management, clinicians, and the implementation team on context and diversity and the necessity of adapting to the needs of the users of clinical information systems within their clinical unit and organizational contexts. We hypothesize that if this framework is used when planning for, and implementing, point of care clinical information systems, then the implementation is likely to be facilitated and the benefits of clinical information systems to patients, clinicians, and governments are more likely to be realized.