Abstract
Introduction
Western health care is under pressure from growing demands on quality and efficiency. The development and implementation of information technology, IT is a key mean of health care authorities to improve on health care infrastructure.
Theory and methods
Against a background of theories on human-computer interaction and IT-mediated communication, different empirical studies of IT implementation in health care are analyzed. The outcome is an analytical discernment between different relations of communication and levels of interaction with IT in health care infrastructure. These relations and levels are synthesized into a framework for identifying tensions and potential problems in the mediation of health care with the IT system. These problems are also known as unexpected adverse consequences, UACs, from IT implementation into clinical health care practices.
Results
This paper develops a conceptual framework for addressing transformations of communication and workflow in health care as a result of implementing IT.
Conclusion and discussion
The purpose of the conceptual framework is to support the attention to and continuous screening for errors and unintended consequences of IT implementation into health care practices and outcomes.
Keywords: human-computer interaction, socio-technical, infrastructure, communication, information technology
Introduction
Like many other Western countries, Danish society is facing new challenges in organizing its health care. An ageing population, rising demands and possibilities in treatment and care, and a lack of clinical staff are, among other issues, pressing new national policies on health care practice, management and organization. In responding to such pressures, information technology (IT) is given a central role as a change agent of health care infrastructure to support more quality, safety and continuity in and across healthcare settings. This paper, however, questions the nature of an IT infrastructure as some ‘thing’ that can be deliberately planned for by authorities and vendors [1–8], since the unfulfilled hopes in IT strategies on health care so far [5], and the contingent nature of health and disease, leaves much open for further learning, negotiation and design.
IT systems in health care have often failed to fulfill the desired outcomes for which they were designed, so leaving health care organizations with systems that nobody uses at great financial expense. For example, health care institutions have often developed capital projects to support an ongoing process of IT implementation and customization. Such projects not only have to train the many users in clinical settings in the functionalities of a specific IT system, but also must offer continuous support (on both system and user demands) on a 24 hours a day, 7 days a week, 365 days a year basis. The effectiveness of the ‘working relationship’ [9] between IT users and the IT system depends highly on the degree of stability, usability and utility of the system in the specific care domain and practice. What this means is that the evolution of an IT system from an infrastructure to an effectively implemented product is an unpredictable process requiring risk management [10] that puts emphasis on screening, evaluation and learning from the unintended consequences of implementing IT systems into care practices. The organizers of any IT project in health care thus face additional implementation challenges beyond the purely technical. Being aware of new kinds of possible ‘errors’ arising from putting IT systems into clinical or practical use are important considerations [11]. Or as phrased in a viewpoint paper on computerized order entry systems, CPOE: “Attention to detail is critical to provide appropriate support to clinicians' work flows [. . .]. During such complex implementations, institutions must remain cognizant of, and carefully monitor, errors that these systems can cause as well as the errors that they prevent” [12 p.27].
This paper poses health care managers, IT implementers and developers to draw attention1 to inadequacies of communication and interaction from IT implementation in order to secure changes of health care infrastructures in a direction that, for the benefit of the patients, support the optimal use of technical and human resources. Discontinuities in communication and workflow counteract ‘integrated care’—both in terms of patient safety, patient centeredness and coherence of care paths—as higher demands are put on clinicians/human resources, compensating for insufficient technology tools.
Methods and materials
The conceptual framework presented in the following paper aims to facilitate how organizations that face the implementation of an IT project can identify, articulate and negotiate across any undesirable consequences of new health care infrastructures [13].
The conceptual framework is developed against the background of theoretical studies on infrastructure and communication, and empirical evaluation studies of human-computer interactions in health care. Evaluation research is, in ‘Methods of Information in Medicine’, considered “a key in identifying how people, organizational, and social issues—all crucial to system design, development and implementation, and use—interplay with informatics projects” [14 p.215]. Further, “the study of failures, partial successes, and changes in project definition or outcome” are recommended for future research [ibid]. Ammerwerth et al. confirm maturation in evaluation research in medical informatics as studies of more technical aspects of implementation have declined and shifted to an increase in studies on the quality of care processes and patient outcomes [15 p.55]. Leroy and Beuscart advocate the importance of usability evaluation by including final users in real settings in order to detect severe usability problems [16]. Classen et al. confirm that evaluation of IT implementation in individual health care organizations is more pressing than ever [17 p.51]. “All organizations will need to perform ongoing evaluation of their CPOE applications and their EHR [Electronic Health Record] if the potential benefits of these technologies are to be actually realized” [17 p.53]. Gross et al. oblige leading clinicians to participate by taking proactive responsibility and by making iterative improvements [12]. “Those implementing systems must monitor them for adverse effects on both patient care and on caregivers' workflows. Similar monitoring must occur to ensure that desired goals are achieved. Continuous refinement based on ongoing end-user feedback must guide progress” [12 p.26]. In other words, the current research literature calls for more detailed knowledge of end-users' experiences, more responsibilities of the organizations and proactive methods for dealing with the consequences of IT systems on the health care processes and outcomes.
This paper's object of study is to evaluate IT in health care infrastructure, and how to study and grasp transformation of infrastructure. Qualitative methods are used in order to collect and reflect highly contextual data. Research in telemedicine [18 p.529] shows that this area, at the end of the 1990s, was plagued by the difficulties of generalization since research was characterized by confusing definitions, the use of small sample sizes, and primarily new and evolving IT innovations. Nonetheless, opportunities for studies employing qualitative methodologies enable a close examination of the phenomenon and help researchers better understand any emerging environment [19–21]. Qualitative methods are therefore central to evaluation studies as well as the prevalence of in-depth analysis of local experiences and settings [22], in order to grasp and fixate the contextual dimensions of health care services, as processes enacted and located in concrete temporal and spatial settings [23].
This paper attempts to illustrate organizational and interpersonal processes of health care by undertaking an analysis across three recently completed qualitative studies of IT implementation. Firstly, the theoretical assumptions are outlined on how to understand ‘infrastructure’ and the role of information technology herein. Secondly, the evaluation studies refer to the results of two major, international qualitative studies on UAC and to three minor, Danish qualitative studies on the use of IT for communication in three different health care organizational settings. The analysis of the Danish studies are reflected and discussed in relation to the results of the international evaluation studies on unintended consequences. Thirdly, the methods of the three Danish case studies are presented and the cases are subsequently analyzed according to the questions on infrastructural transformations and problems of UAC from IT implementation. Fourthly, the determined relations of communication and levels of interacting with IT are synthesized into a conceptual framework that seeks to capture transformations of health care infrastructures from human-computer interaction.
What is an infrastructure?
The traditional concept of an infrastructure is something that is just there, ready to use, completely transparent and not open to question (for example, water or electricity supplies, the railway, the road system, telecommunication, the mail service, or the Internet). Under this concept there is a tendency to perceive infrastructure as ‘hardware’ – something that is built and maintained and then sinks into the invisible background, and which is noticed only when it breaks down. But as Edwards [24] points out, infrastructure is socio-technical in nature, meaning that infrastructure is not only based on the hardware of an organization, but also ‘software’ such as socially-communicated background knowledge, general acceptance and reliance, and near ubiquitous accessibility.
According to the online Open Content dictionary Wiktionary (http://en.wiktionary.org), infrastructure means “an underlying base or foundation especially for an organization or system” and “the basic facilities, services and installations needed for the functioning of a community or society”. The definition provided here points to the fact that equally important for the understanding of infrastructure is the development or evolution of ways to communicate or interface with the underlying base. For example, telephony is possible not only because signals over a distance can be transmitted using electromagnetic waves via electronic transmitters, but also because of the invention of an appropriate appliance – the telephone – which can be used for the purpose. Importantly, the system is not successful solely because the technology works, but because enough people want to use, own and pay for a phone with which to communicate with others. It works because the whole service is highly organized, making sure that it is possible to make calls to the people you want to talk to. Furthermore, it is difficult to separate the development of the ‘base’ infrastructure from the development of services and regulations that support its functionality. It is difficult, for example, to separate the development of the transmission of signals (base) from the development of phones and the other services and installation needed to make the phone system function as an infrastructure. As Latour demonstratively put it: “Boeing-747s do not fly, airlines fly” [25 p.46]. Infrastructures, therefore, include technologies that are socially co-defined by their use and are always under the process of development or change; they grow through their use, and it is their use that defines whether or not something becomes an infrastructure.
Star & Ruhleder [9] and Hanseth [10] (among others) discuss the infrastructural aspects of IT systems. They both suggest different dimensions to characterize an infrastructure. While focusing on use and use practice, Star and Ruhleder mention the following characteristics:
embeddedness (integrated in social structures and practices),
transparency (can be used without removing focus from the task),
reach or scope (goes beyond individual tasks or processes),
learned as part of membership (an inherent part of an organization),
links with conventions of practice (shapes and is shaped by practice),
embodiment of standards (builds on standards and conventions),
built on an installed base (must relate to existing technologies), and
visible upon breakdown (loses transparency and is drawn into focus when it breaks down).
Very much in line with this, but with slightly more emphasis on the necessary technical prerequisites for an infrastructure to function, Hanseth [10] suggests that an infrastructure is an evolving (evolves continuously), shared (must function as a shared resource or foundation for a community), open (lack of borders in how many elements it may include, how many users may be using it and also in the sense that there are no limits to who might contribute to its design and deployment and that the development time has no beginning and no ending), heterogeneous (including sub-infrastructures based on different versions of the same standard or different standards covering the same functionality) installed base (backward compatibility which also means that the existing heavily influences how the new can be designed and that infrastructures are considered as existing already, never developed from scratch).
Studying infrastructure from IT implementation
These dimensions suggest “an infrastructure, which is without absolute boundary on a priory definition” [9] and also points to the fact that infrastructures cannot be understood independently of the work they afford. An IT system, then, becomes an infrastructure in relation to the involved technical and social elements of the organized practices within which it functions. Furthermore, the IT system evolves over time, does not have a fixed group of users or use practices, and is a dynamic ongoing process with no fixed center of control. IT implementation, therefore, also becomes a point of departure for studying infrastructure as the ‘embeddedness’, ‘transparency’, ‘embodiment of conventions’ and ‘scope’ only become ‘visible upon breakdown’, which let the technology and ‘base’ in question become a negotiable part of a ‘shared’, ‘open’ ‘heterogeneous’ and ‘evolving’ infrastructure. The following empirical studies are evaluations of the implementation of different IT systems into health care, primarily computerized order entry systems, CPOEs, in different settings that enlighten infrastructural issues.
Understanding IT infrastructure and implementation issues in health care from qualitative studies of unintended consequences
Cambell, Sittig, Ash, Guappone and Dykstra identify nine types of unintended adverse consequences related to CPOE [11]. The types are the outcome of a major qualitative study over 9 months; 390 hours of observation, 43 hours of interview, transcripts and field notes of approximately 1900 pages at five US hospitals, using either home grown or vendor built CPOE systems. The nine major categories are in order of decreasing frequency: 1) more/new work for clinicians; 2) unfavorable workflow issues; 3) never ending system demands; 4) problems related to paper persistence; 5) untoward changes in communication patterns and practices; 6) negative emotions; 7) generation of new kinds of errors; 8) unexpected changes of power structure; and 9) overdependence on the technology. In the viewpoint paper by Ash, Berg and Coiera [4], the processes of generating errors are reflected and categorized. The categorization draws from cross-Atlantic experiences with CPOE systems in health care delivery settings. The American, Australian and Dutch data are generated from qualitative studies of ethnographic observation and semi-structured interviews with professionals [4]. The overall purpose of the CPOE systems is to prevent medical errors. The studies confirm, though, that CPOE systems are not as successful in preventing errors as is generally hoped for, but that they could actually generate new errors [4 p.105]. Ash et al. categorize the errors within two ‘main categories’; firstly, the process of ‘entering and receiving messages in and from the system’ [4 p.106], and secondly, the ‘communication and coordination processes’ that the system is supposed to support [4 p.107]. These categories of errors can be reinterpreted within the research tradition of human-computer interaction as referring to the ‘usability’ and the ‘utility’ of the system [26]. The usability of the system is determined by the direct interaction between the user and the user interface of the system, i.e. in health care delivery settings the usability of the system for ‘entering and receiving of messages’. The ‘communication and coordination process’ [4 p.107] is concerned with the utility of the system in mediating the users' collaboration on clinical tasks across different tempo-spatial locations. The three Danish implementation studies presented in the following paragraphs are examples of IT implementations to support and mediate the collaboration:
across different health care institutions,
across different wards at a university hospital, and
across different user-groups in two wards of internal medicine with different professions and login rights.
Additionally, Star and Ruhleder's ‘Steps to an ecology of infrastructure’ points to a third category or ‘step’ in human-computer interaction. Drawing on cybernetics and the communication theory of Gregory Bateson, Star & Ruhleder understand infrastructural changes as the outcome of paradoxes between the organizational purpose of the system and the utility in use [9], also termed ‘trans-contextual syndrome’ or ‘double bind’ [27]. Systemic experiences of ‘double binds’, i.e. bindings between what a user is expected to use the system for and what the system can be used for, and again double bindings between what the system is designed for and what it actually does, point to ‘socio-technical problems’ in the evolving infrastructure. These problems need further attention and articulation in order to redesign the system and the work practice in question to a more deliberate and ecological ‘healthy’ fit. We shall refer to such socio-technical tensions as ‘paradoxes’ between the purposes of implementing the system and the actual effect. Within cultural psychology [28] and the cultural–historical tradition of activity theory, ‘contradictions’ between the user's culture and tools relate to ‘the motive of the activity’ [29]. A cultural and historical motive of western health care is, for example, to prevent illness and suffering [30]. These motives and goals of health care techniques are again, within an Aristotelian tradition, related to ethical values and collective understanding of living a ‘good life’ [31–36]. These categories and analytical levels of understanding the development of technology in health care are summed up in Table 1.
The three following case studies point to such misapprehensions between intentions and actual use of IT for organizing health care. The studies are qualitative studies conducted independently of each other [37–42], but analyzed according to the three analytical levels of Table 1. Subsequently, a synthesis of the studies is made in order to support the attention to turning points in the relationships of humans and IT [footnote 1] [43] and the possible articulation of future IT-related communication problems [13].
Three minor Danish case studies of IT implementation across institutions, hospital wards and different user-groups
The objects and methods of the three case studies can be summed up as follows. The case study on the communication across different health care institutions focused on the implementation of an information system to support the communication between the local hospital and the municipality. The implementation was monitored by interviews with four central figures; in total twelve hours of interviews were conducted during the project. Besides diary notes, meeting minutes from the implementation process, and an internal evaluation report done by the responsible organization at the end of the project also entered into the material analyzed. Further description can be found in Bygholm [37].
The two other studies concerned the implementation of a vendor built CPOE in a university hospital, and at two wards of internal medicine at another hospital in the same county. The CPOE system at this stage of development is only handling ‘the bookkeeping’ of medication. It is not offering any decision support, and is developed by a professional vendor (Systematic Software Inc, Aarhus, DK) in close cooperation with the responsible Health Management Organization of Aarhus County, Denmark, who has supplied clinicians to the development project. The county is the hospital owner and is, according to the Danish Law on Patient Safety 2004 [44], collecting and reviewing unintended adverse consequences, UACs [45], in relation to using the CPOE [40]. Within a period of 6 months, clinicians in the five hospitals of the county reported 754 UACs in total, of which 97 UACs happened at the university hospital. These 97 UACs are further categorized in the Audit Report of the County [40] as 45 UACs due to ‘organizational errors’, and 52 UACs caused by ‘technical errors’. These organizational and technical circumstances of the UACs were further elaborated on by the use of qualitative methods for richness and details. Some empirical examples of ‘organizational errors’ and ‘technical errors’ were unfolded in a lecture and discussion between researchers and students in health informatics held at Ballerup, Aalborg University in Copenhagen, 21 April 2006. These examples were contextualized even further by two hours of interview with the responsible implementer at the university hospital, retelling her experiences with concrete instances that produced UACs.
The clinical use of the CPOE system at two internal medical wards builds on 48 hours of observation, six interviews with primary users, two physicians and four nurses, and an analysis of the user interface of the CPOE and of other documents. At the time of the study, the CPOE had been in use for eight months. Based on the observations, three use-scenarios for central events in the medication process were constructed (prescription, requisition, and considerations of continuation of treatment). The use-scenarios were introduced to the interviewees for four reasons: to condense, verify and generalize the observations and to trigger the memory of the interviewees.
The following three paragraphs will present the individual cases and analyze them according to the three analytical levels of interacting with IT in health care infrastructure in Table 1.
IT communication across institutions: municipality and hospital
The Municipality of Aalborg's Department of Care for the Elderly and Disabled decided to implement electronic exchange of information between on the one hand the municipality and the hospital, and on the other hand between the municipality and the general practicing doctors. The idea behind the project was that switching from communicating through telephone and letters to electronic information exchange would mean a more correct exchange of information and also a faster and more holistic treatment of the citizen involved. A well-known standard, EDI messages by way of Vans technology, was chosen for exchange of information involving an extension of the standard, but from a technical point of view it should not be too difficult to make the involved partners' systems communicate. Still, numerous problems were encountered in the implementation and use of electronic exchange for communication and coordination of work with senior citizens between the partners.
This case focuses on the exchange of information between municipality and hospital. It illustrates the tangled nature of the problems experienced in the attempts to fit in use of a new infrastructure in the work practices of the hospital staff.
Initially, there were two issues hindering the electronic exchange of information between hospital and municipality, as seen from the perspective of the hospital staff. First, the hospital used an old-fashioned patient administration system and second, the involved staff (at seven medical wards) had no experience of using ICT in their work. Thus, training of the staff (approximately 300 persons) was needed. For each of the seven wards involved, about four persons were trained for one day. The staff participating in the training was supposed to teach the others in the wards they came from, thus functioning as super users. The problem was not that the trained staff could not use the system, but more that it was an extremely time-consuming task to send messages, the patient administration system operating with many different types of screens that should be accessed in a certain order and being difficult to navigate around. For an inexperienced user, it could take around an hour to send a message and demonstrating the process for another nurse could easily take one and a half to two hours.
Apart from training, there was also a need for agreements between the municipality and the hospital to ensure that messages sent were actually received at the municipality within an acceptable time limit. A basic guideline in the set of agreements was the division of messages into two categories: emergent and non-emergent messages. Emergent messages were messages that had to be acted upon within 24 hours and those had to be phoned to make sure they were received and understood by the right persons. An un-emergent message, which is a message not needing action within 24 hours, could be sent electronically. Compared to the expectations expressed in the project application concerning “. . .a more quick and effective way of communication. . . ” by using ICT for communication this was unexpected. As for the whole set of agreements it turned out that, due to many different interpretations of the agreements, they got increasingly more and more detailed, which in turn made them less accessible for the staff. Thus, instead of one infrastructure for exchange of information between the institutions they now had two and double communication became a result of this, i.e. in many cases they used both the phone and the system to communicate the same information just to be sure that messages were actually received.
To send more detailed information on patients between the partners, personal consent was needed for each message. In practice this was not always doable and one of the solutions that was examined and discussed with the legal authorities was the possibility of a general consent from each citizen receiving care. This was, however, considered a breach of the law, thus information could only be exchanged if consent was given for each message. There was nothing new in the law; the same conditions count when information is exchanged between two persons by telephone. Use of electronic communication, however, changes and broadens the context for information exchange; what can be said in an oral phone conversation between two people is not necessarily suitable for wider distribution in a writing-based media.
This study shows the complexity of the many issues involved in the attempt to change from one infrastructure to another. Some of the problems were due to the complicated interaction style demanded by the old system and to computer inexperience among the staff. The fact that it was such a time-demanding task to send and receive messages and the doubtfulness concerning the rules for exchange of information to some extent questioned the rationale of using the system, seen from the point of view of the hospital staff. The agreements that should solve the problems got rather complicated in order to cover all situations. The necessity of operating with two different lines of communication did not afford the work practice at all and results of problems were in many cases double communication, i.e. more work instead of less work for the staff. Adding to this, and coming from societal context, was the legal issue of patient consent by the exchange of patient information, which questioned the legality of the basic intensions and goals behind the project.
Unintended transformation of the communication between the municipality and hospital with the edifact-system
The study showed the complexity of the problems related to achieving the aims of implementation of a new infrastructure. Doubtfulness and double communication were some of the unintended consequences and the findings of the study can be summed up at the following levels of interaction:
Level of interaction: the communication between municipality and hospital were interrupted, as their existing Patient Administration System was difficult and time-consuming to use.
Level of IT-mediated interaction among users: the users needed to establish new procedures for sending and receiving messages to each other, dividing them into urgent/complicated messages and non-urgent messages where only the latter could be sent electronically, and to set rules for who was responsible for reading and acting upon the electronic information.
Infrastructural level: in contrast to the expected vision of optimizing communication the information exchange expanded, double communication including IT and telephone were necessary to secure quality of collaboration. The exchange of electronic patient data also collided with the law on data security, whereby a whole new infrastructural problem arose at a national level2.
IT communication across wards in a university hospital
The project organization of a university hospital is implementing electronic order entry systems, CPOE. The aims of the system are, among others, to improve patient safety and continuity of patient treatment and care. In relation to a new Danish Health Law [44], clinicians have been reporting on unintended consequences in clinical work. Within a period of 6 months, 97 unintended consequences from using the CPOE system in the University Hospital have been reported to the county, the hospital's owners. These are in an Audit Report from the county [40], categorized as 45 unintended consequences of a technical nature, due to an unstable system and inadequate user interface, and 52 unintended consequences of an organizational nature, expressed by a lack of correspondence between electronic documentation of medication and other registration of data. The responsible implementer of the project organization experienced though, that the technical errors and the organizational errors were related. The following two stories show, they even feed each other in practice and affect the infrastructure of health care as well.
‘Technical error’ of patient safety
An example of unintended consequences caused by technical inadequacies of an order entry system, CPOE, is that on some screen dumps the wrong patient identification number is showing. Leaders of hospital and project organization were to decide quickly on how to warn the end-users. But here the implementers face a new problem of a social origin: “But how do you communicate to 3000 users? [. . .]. Warnings on the screen are simply clicked away”. Closing the system down was an equally bad possibility: “they (the clinicians, HW) are totally depending on it”. As a solution, a telephone-chain was started in which one hospital ward after the other should pass on the message about the wrong patient identification numbers in the CPOE to the next ward. The morning after, a follow-up call was made to make sure the chain went through. The person calling the emergency ward of internal medicine recalls: “I phoned the ward to pass on the warning. They were receiving patients with heart attacks, so she just answered ‘yes, yes, yes’—and hung up. She just wanted me to get off the phone. The systems should probably have been closed down”. The chain went through wards with fewer emergency cases. A control call was made to the last ward in the telephone-chain. They never received the message.
The so-called technical error becomes a story of new communication problems in the organization related to CPOE, as well as a reminder that for clinicians in situ, life and death issues take priority over technical errors, even though in a further prospect new, unintended consequences might silently come into being, as in this case of the broken telephone-chain.
‘Organizational error’ of patient continuity across wards
‘Missing documentation’ or ‘double registrations of the same patient information’ are examples of unintended consequences, UCs, of an organizational nature. This seemingly human factor had a technical side as well. Often patients move from an internal medicine ward to surgery and from surgery to intensive care, but after implementation of the CPOE, intensive care lacked information on the patients coming from surgery. This gives a gap in the patients' electronic documentation of medication. Though, as understood and widely accepted in the organization, the surgery ward has no direct need of CPOE in their work, while the doctors, as well as the nurses, in intensive care do. They need the information to control and confirm the patient's medication in the CPOE when admitting him or her to intensive care. The responsible implementer of the CPOE reflects: “How do you make work processes to fit this problem, and who gets the job to do so? It almost turned into a war, and I was sitting in trench warfare. It is a whole new problem that we have never had before. Previously we would just have received the papers from the record passing through.”
This story is not just about inadequate collaboration between wards, but also that the CPOE creates new problems for the wards and the clinicians to secure the patient continuity. This conflict was traded off as the doctor's secretary in the surgery ward was pointed out to enter all patient data in the CPOE.
The infrastructural changes expressed in these two stories show that the visions and expectations of more patient safety and continuity from implementing CPOE into health care practice are not automatically fulfilled, but require attention to new, socio-technical problems and work on communication.
Transformation of communication with IT across wards in a university hospital
The study showed that IT implementation produces new communication problems in the health care organization, and thus the intentions of more patient safety and fewer continuity problems are not automatically a result of a CPOE system, but a new challenge for health care and project organizations that cannot be reduced to either technical or organizational errors. The two examples of a so-called ‘technically’ caused UAC and organizational grounds for UACs can be summed up as problems at the following levels of interacting with IT.
Organizational aspects of the ‘technical error’ of patient safety
Level of interaction: the technical error in the CPOE system of showing wrong patient identification numbers interrupted the patient safety throughout the organization.
Level of mediated interaction: discontinuity in communicating the technical error to all end-users through a telephone-chain is collectively making the clinical collaboration more risky for patients as well as for users.
Infrastructural level points to socio-technical problems in taking responsibility for patient safety as a clinical user, and as responsible manager of the reliability of the system.
Technical aspects of ‘organizational error’ of patient continuity across wards
Level of interacting: the ward of intensive care lacks information in the CPOE on patients received from the ward of surgery.
Level of IT mediated interaction between the wards: as the surgeons do not need the CPOE for their work discontinuity appeared in the collaboration between the wards with the replacement of paper with the CPOE.
At the infrastructural level the problem of recreating patient continuity between the wards with CPOE became a new socio-technical problem, also posing the question: “Who is responsible for things to flow?”
Communication on medication between clinicians using CPOE in two medical wards
The study [36,37] points to the central situation of enacting medication with CPOE at two internal medical wards in a middle-sized Danish hospital. The programme of action for medication consists of a minimum of seven core acts or ‘subprogrammes of action’: 1) the indication of treatment, initiated by the patient's problem/diagnosis, 2) the ‘prescription’ (recommendation of treatment and patient's consent of choice), 3) ‘drug order’, i.e. registration of order in CPOE system, 4) ‘dosage’, i.e. making the drug ready for consumption, 5) ‘administration’ (the patient is given the drug), 6) ‘assessment’ of the drug's effect on the patient, and 7) ‘considerations on how to proceed’, i.e. whether to continue or withdraw the drug. This process takes place within a different temporal rhythm, and located at different places, for example at the patient's bedside, in the hallway, in different offices and the drug storage room. Global access from other wards or institutions is possible. The CPOE interacted with four of the seven subprogrammes in the medication process, 2) the prescription, 3) the order entry, 4) dosage and 7) withdrawal or continuity of treatment. The following situations of use point to transformation of ordination with the CPOE, of the registration of clinical collaboration and possibly transformation of patient trajectories with CPOE.
Transformation of ordination with CPOE
On the doctors' rounds, when the physician should order the patient-consented prescription, a detour occurred. The physician would not consult and order drugs for one patient at a time. Instead he would memorize three or four patients at a time before he would first dictate diagnosis and order to the patient record and then order the prescription in the CPOE. This detour in the medication safety of the individual patient was not only caused by a clumsy PC table that made it too troublesome to take the CPOE lap-top along on patient rounds, but the digitalized dictaphone for dictating to the patients' medical records prior to the order entry is placed on the wall in the hallway, additionally making a long walking distance from the patient's bed.
Transformation of user rights and clinical teamwork
The work divisions inscribed in the user rights of the CPOE adhere to the certification rights of physicians and nurses. In practice, though, the clinical teamwork related to ‘drug order’ and ‘dosage’ varies from the formal certification rights and duties. The inflexible user rights and time-consuming login to the CPOE, along with too few computers and computers placed far away from drug storage, made the clinical collaboration spread out in time and space, subsequently leading to more coordination work on medication. A counteraction of the clinical team was to bypass the user rights by working under the same login of a user with physicians' rights. This meant that the virtual medication work registered under a specific user name and login does not necessarily correspond to an individual person, but might cover an empirical work team of clinicians.
Possible transformation of patient trajectories with CPOE
The continuity of the patients' medication transformed with the CPOE. This is also part of the vision of electronic, instead of paper-documented, medication. CPOE gives ‘global accesses’ to the medication data of patients, not only from all hospital wards, but also from other health care institutions that might be involved in the future health care trajectory of the patient. The global access and continuity aspects of electronic communication gave a new problem when drug orders were withdrawn and when patients were discharged from the hospital ward. The CPOE system replaced a paper medication scheme containing drug orders for seven days at a time. After seven days the nurse on nightshift would copy the drug orders to be continued on to a new scheme, and leave out those that were withdrawn in the meantime. This ‘procedure of cleaning up’ the registered drug orders were left out of the CPOE, as the system does not run out of ‘writing space’ every seventh day. Additionally, it would be extra work for the physicians (having the user rights to do so) to log on to the CPOE every time they have withdrawn an order. Leaving out the ‘cleaning procedure’, on the other hand, means that drug orders stay active in the system after the end of hospitalization. This adds new risks to the patients' safety. In a possible future when a patient is readmitted into the hospital or enters another health care institution, the information in the CPOE might mislead the users, and foster a wrong understanding of the patient's medication profile and history.
Infrastructural transformation of medical health care practice
The study points out that more ‘global access’ to patients' medication data with the CPOE system might locally give more difficult access. New physical and social distances of prescribing and coordinating medication work with the CPOE have arisen, alongside new tasks and divisions of work.
Transformation of ordination with CPOE
Level of interaction: the physical distance between the patient's bed, the CPOE and other artefacts interrupts the physician's prescription and requisition of drugs in the CPOE on doctors' rounds.
Level of IT-mediated interaction: the physician would reduce the walking by memorizing three or four patients at a time and order their prescriptions in the CPOE after dictating to the patient record in the hallway.
Level of infrastructure: a socio-technical paradox arose as the doctor–patient relationship transformed into collective e-patient, when entering ‘mass-orders’ several patients at a time. Cognitive pressure is put on the individual doctor in order not to mix up the patients, the patients' prescriptions and orders with each other. Mistakes are likely to be difficult to trace in the CPOE as they happen before the orders are prescribed.
Transformation of user rights and clinical teamwork
Level of interaction: the users are interrupted in their workflow by an inflexible system for entering and retrieving information.
Level of IT-mediated interaction: discontinuity between the rationality of medication work inscribed in the software, the number and location of PCs made the clinicians work around user rights in order to coordinate and continue workflow.
Level of infrastructure: tensions are created with the CPOE between the formal rights and duties of nurses and physicians and their actual collaboration, and a collective e-clinician came into being for teamworking under seemingly strictly individualized user rights.
Possible transformation of patient trajectories with CPOE
Level of interaction: discharging patients with CPOE interrupts the clinicians' practice, as withdrawn medication has to be actively withdrawn in the system.
Level of IT-mediated interaction with future users: a discontinuity in a patient's medical treatment might happen when the patient re-enters the health care system, unless the users envision future uses and withdraw orders from the system.
Level of infrastructure: transformation of patient continuity across different health care sites with the risk of wrong medication.
Results: synthesis of the analysis into a framework of communicative relations in health care infrastructure
The presented studies of infrastructural changes in and across different health care institutions underlines that the intended effects and visions of IT are not easily realized. Instead, the complexity of implementing IT into existing health care practices draws attention to new relations of coordinating and mediating work, including other techniques, and the material surroundings. The overall findings on the disclosures of infrastructure from IT implementation can be summed up as unintended transformations that arise when relations in clinical work processes are framed differently. These relations are communicative in the sense that practices of making and exchanging information have changed by the agency of the IT system, mediating the interactions of the involved parties differently. Changes in the communicative relations from interacting with the system put continuity of care processes at risk, distributing tasks and responsibilities differently. Work around and alternative communication means for restoring continuity of health care tasks make up for some interruptions. Nonetheless, these relational changes in communication from interacting with IT are temporal and spatial ‘turning points’ [39] in the health care process and allow for further attention, articulation and design of the ongoing implementation.
Attention to ‘turning points’ in health care processes
In order to identify unintended consequences in health care processes from IT implementation, it is necessary to draw attention to the spatial and temporal turning points within which communicative relations are framed [43]. These turning points were, in the three case studies, situations and moments of interruption, discontinuity and socio-technical tensions in the users' communication with the system. The matrix presented in Table 2 frames the contextual and processional dimensions of infrastructural changes from IT implementation.
Table 2.
Relational turning points of communication in HCI | Interruption in usability | Discontinuity in utility | Socio-technical tensions/problems |
---|---|---|---|
Direct interaction of user and interface | “Receiving and entering data”<=> User-rights, accessibility, training | Performance loss <=> ‘deskilling’ | ‘Cognitive overload’ <=> Stress |
IT-mediated interaction/collaboration | Stability of system, software design, GUI <=> ‘Entfremdung’ | Coordination problems <=> Work-around Arbitrary or double communication | Conflicts, indifference, delegation of work ‘unsafe’, risky |
IT infrastructure | IT implementation <=> Information architecture, reliability and inscribed rationality | Errors in patient treatment and care | Identity/meaning transformation of patient, clinician, motive/purpose of health care |
The framework can afford the identification of IT implementation problems in relation to usability and utility of the system:
As experienced by the individual user interacting directly with the system entering or retrieving data. Here the lack of professional performance can be experienced as deskilling, to be compensated for by more work and flexibility.
From experiencing the social collaboration in mediating clinical tasks with IT, and how these work relations and clinical tasks are cultivated by the IT system.
Again, how the purpose and identities of the involved participants are co-configured by the infrastructural transformation.
Articulation of infrastructural transformations as unintended consequences
The conceptual framework presented above draws attention to determining relations of communicating and interacting with IT in health care processes. Any disturbances and interruptions in these ongoing processes produce contingencies that will need renewed attention and possible articulation in order to be dealt with accordingly to ensure more deliberate and desirable outcomes. The presented qualitative evaluation studies of IT implementations are identifications and articulations of such transformations in health care as unintended consequences. Furthermore, putting the nine types of UAC [11] back into context by the conceptual framework of infrastructural transformation from IT implementation gives the ‘picture’ as presented in Table 3.
Table 3.
Relational turning points of communication in HCI | Interruption in usability | Discontinuity in utility | Socio-technical tensions/problems |
---|---|---|---|
Direct interaction of user and interface | ‘Negative emotions’ (6) | ‘Problems related to paper persistence’ (4) | ‘More/new work for clinicians’ (1) |
IT-mediated interaction/collaboration | ‘Never ending system demands’ (3) | ‘Unfavorable workflow issues’ (2) | Untoward changes in communication patterns and practices (5) |
Infrastructures | ‘Over dependency of the technology’ (9) | ‘Generation of new kinds of errors’ (7) | ‘Unexpected changes in power structure’ (8) |
Such unintended consequences of IT implementation on health care infrastructure open up for renewal reflection on IT strategies in health care, and add on to the many tasks of the capital projects implementing the systems.
Discussion: supporting articulation work for redesigning IT in health care processes
Finally, due to its integrated nature, infrastructure is only visible upon breakdown. Unintended consequences from IT implementations can therefore also be interpreted as disclosure of infrastructure, framing the communicative relations, spatial and temporal, differently. The conceptual framework for drawing attention to relational turning points for communication and interacting with IT in health care processes intends to support the articulation work and negotiation of health care infrastructure. The identification of user experiences of interruptions, discontinuities and socio-technical problems are of central value to the overall patient safety and professional dependability of IT systems in health care. The screening for errors and articulation of unintended consequences deliver essential feedback to the implementers for customization of the system. Furthermore, the results and research purpose of this paper are not to generalize infrastructural changes a priori to positive or negative outcomes, but to develop a language to address future infrastructural changes as situations and moments that are sensitive to adjustments and renegotiation of the IT system and the work practices in question.
Conclusion
The pervasive role of IT systems as change agents in Danish health care can therefore be seen as paradoxical. IT implementation not only implies potential for change of the existing infrastructure, but also for undesired changes in the practices of the involved participants with consequences for the understanding of health care as such. Technology infrastructure is therefore not only a point a departure for further learning [46], but also a quest for what Marc Berg has called politics of design [47]. These politics of design question and negotiate the disclosed infrastructure and its more or less intended transformation of humans as well as the techniques within. Because of this entangled nature of humans and techniques, Latour poses a ‘parliament of things’ [48]. Institutionalizing the negotiating and redesign of the evolving infrastructure from IT implementation with the clinical users would clearly add on to the capital project's tasks of customization. In other words, the western need of transforming health care infrastructures with IT can also be elaborated further from the following question: to what extent are the health care organizations and capital projects ready to democratize [49] the technological infrastructure with the clinical end-users, and what are the organizational means necessary to do so apart from project organizations' ongoing screening for errors?
Footnotes
“From the phenomenological position, attention is not primarily a focus on something, but a perspective from turning points, where the in-between of relationships can be experienced. In this sense, attention shows itself as alertness for the coming to be and the realm of transitions”. Schuback, MSC. The knowledge of attention. International Journal of Qualitative Studies on Health and Well-being 2006 Sep;1(3):133-40. Available from: http://www.tandf.no/qhw.”
October 2006: The Minister of Health and Internal Public Affairs made a suggestion for a new law on data security, which is now at hearing in the Danish Parliament to be negotiated.
Contributor Information
Helle Wentzer, Department of Communication and Psychology, Virtual Center for Health Informatics, http://www.v-chi.dk/english/index.htm, Aalborg University, Denmark, http://www.kommunikation.aau.dk/index.uk.html.
Ann Bygholm, Department of Communication and Psychology, Virtual Center for Health Informatics, Aalborg University, Denmark.
Reviewers
Claus Bossen, Associate Professor Aarhus University, The Faculty of Humanities, Institute of Information and Media Studies, Denmark
One anonymous reviewer
References
- 1.National IT-strategi for sundhedsvæsnet 2003–2007 [National IT-strategy for Health Care] Indenrigs- og Sundhedsministeriet; 2003 May. Available from: http://www.sst.dk/upload/nat_itstrategi03_07.pdf. [in Danish]
- 2.Olesen F. Kommunikation og teknologisk mediering [Communication and technological mediation] In: Elsass P, Olesen F, Henriksen S, editors. Kommunikation og forståelse. Kvalitative studier af formidling og fortolkning i sundhedssektoren. Århus (Denmark): Philosophia; 1997. pp. 143–68. [in Danish] [Google Scholar]
- 3.Berg M, Langenberg C, Berg I, Kwakkernaat J. Considerations for sociotechnical design: experiences with an electronic patient record in a clinical context. International Journal of Medical Informatics. 1998 Oct-Dec;52(1–3):243–51. doi: 10.1016/s1386-5056(98)00143-9. [DOI] [PubMed] [Google Scholar]
- 4.Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. Journal of the American Medical Informatics Association. 2004 Mar-Apr;11(2):104–12. doi: 10.1197/jamia.M1471. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Wears RL, Berg M. Computer technology and clinical work: still waiting for Godot. The Journal of the American Medical Association. 2005 Mar 9;293(10):1261–3. doi: 10.1001/jama.293.10.1261. [DOI] [PubMed] [Google Scholar]
- 6.Shortliffe EH. CPOE and the facilitation of medication errors [editorial] Journal of Biomedical Informatics. 2005 Aug;38(4):257–8. doi: 10.1016/j.jbi.2005.05.007. [DOI] [PubMed] [Google Scholar]
- 7.Horsky J, Zhang J, Patel VL. To err is not entirely human: complex technology and user cognition. Journal of Biomedical Informatics. 2005 Aug;38(4):264–6. doi: 10.1016/j.jbi.2005.05.002. [DOI] [PubMed] [Google Scholar]
- 8.Koppel R, Localio AR, Cohen A, Strom B. Neither panacea nor black box: responding to three Journal of Biomedical Informatics papers on computerized physician order entry systems. Commentary. Journal of Biomedical Informatics. 2005 Aug;38(4):267–9. doi: 10.1016/j.jbi.2005.05.006. [DOI] [PubMed] [Google Scholar]
- 9.Star SL, Ruhleder K. Steps toward an ecology of infrastructure: design and access for large information spaces. Information Systems Research. 1996 Mar;7:111–34. [Google Scholar]
- 10.Hanseth O. The economics of standards. In: Ciborra CU, Braa K, Cordella A, Dahlbom B, Failla A, Hanseth O, et al., editors. From control to drift: the dynamics of corporate information infrastructures. New York: Oxford University Press; 2000. pp. 56–70. [Google Scholar]
- 11.Cambell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. Journal of the American Medical Informatics Association. 2006 Sep-Oct;13(5):547–56. doi: 10.1197/jamia.M2042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Gross PA, Bates DW. A pragmatic approach to implementing best practices for clinical decision support systems in computerized provider order entry systems. Journal of the American Medical Informatics Association. 2007 Jan-Feb;14(1):25–8. doi: 10.1197/jamia.M2173. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Star SL, Strauss A. Layers of silence, arenas of voice: the ecology of visible and invisible work. Computer supported cooperative work (CSCW) 1999 Mar;8(1–2):9–30. [Google Scholar]
- 14.Kaplan B, Shaw NT. Future directions in evaluation research: people, organizational and social issues. Methods of Information in Medicine. 2004;43(3):215–31. [PubMed] [Google Scholar]
- 15.Ammenwerth E, de Keizer D. An inventory of evaluation studies of information technology in health care trends in evaluation research 1982–2002. Methods of Information in Medicine. 2005;44(1):44–56. [PubMed] [Google Scholar]
- 16.Leroy N, Beuscart MC. Consequence of including final users in a usability evaluation: a case study with a CPOE. In: Bertelsen P, Elkin PL, Madsen I, Nøhr C, editors. Human factors engineering in health informatics. Proceedings of the second HFE conference; 2007 Jun 7–8; Aarhus University Hospital, Skejby, Denmark. Aalborg (Denmark): Virtual Centre for Health Informatics; 2007. pp. 24–25. Available from: http://www.hfeinhi2007.org/ [Google Scholar]
- 17.Classen DC, Avery AJ, Bates DW. Evaluation and certification of computerized provider order entry systems. Journal of the American Medical Informatics Association. 2007 Jan-Feb;14(1):48–55. doi: 10.1197/jamia.M2248. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Turner JW. Telemedicine: expanding health care into virtual environments. In: Thompson T, Dorsey A, Miller K, Parrot R, editors. Handbook of health communication. New Jersey: Lawrence Erlbaum Associates; 2003. pp. 515–35. [Google Scholar]
- 19.Yin RK. Case study research: design and methods. 2nd ed. Thousand Oaks: Sage Publication Inc; 1994. [Google Scholar]
- 20.Kvale S. InterViews: an introduction to qualitative research interviewing. Thousand Oaks California: Sage Publications Inc; 1996. [Google Scholar]
- 21.Kristiansen S, Kroghstrup HK. Deltagende observation: introduktion til en samfundsvidenskabelig metode [Participant Observation] Copenhagen (Denmark): Hans Reitzel Forlag; 2005. [In Danish] [Google Scholar]
- 22.Westbrook J, Georgiou A. Multi-method evaluation of the impact of clinical information systems. In: Hejlesen O, Nøhr C, editors. SHI 2005, Proceedings of the 3rd Scandinavian Conference on Health Informatics; 2005 August 25–26; Aalborg University, Denmark. Aalborg (Denmark): Virtual Centre for Health Informatics; 2005. p. 81. [Google Scholar]
- 23.Mol A. The body multiple: ontology in medical practice. Durham (NC): Duke University Press; 2002. (Science and Cultural Theory) [Google Scholar]
- 24.Edwards PN. Infrastructure and modernity: force, time, and social organization in the history of sociotechnical systems. In: Misa TJ, Brey P, Feenberg A, editors. Modernity and technology. Cambridge, MA: MIT Press; 2003. pp. 187–225. [Google Scholar]
- 25.Latour B. On technical mediation—philosophy, sociology, genealogy. Common Knowledge. 1994;3(2):29–64. [Google Scholar]
- 26.Nielsen J. Usability engineering. Boston: Academic Press Professional; 1993. [Google Scholar]
- 27.Bateson G. Steps to an ecology of mind. Chicago, London: The University of Chicago Press; 2000. [Google Scholar]
- 28.Cole M, Engeström E. A cultural-historical approach. In: Salomon G, editor. Distributed cognitions. Cambridge (UK): Cambridge University Press; 1993. pp. 1–46. [Google Scholar]
- 29.Kuutti K. Activity theory as a potential framework for human computer interaction. In: Nardi BA, editor. Context and consciousness. Cambridge: MIT Press; 1995. pp. 17–44. Available from: http://www.dwr.bth.se/kari_kuutti%20Nardi_book.pdf. [Google Scholar]
- 30.Lloyd GER, editor. Hippographic writings. London: Penguin Books; 1983. “The Oath”; p. 67. [Google Scholar]
- 31.Arendt H. The human condition. Chigago, London: The University of Chicago Press; 1998. [Google Scholar]
- 32.Dunne J. Back to the rough ground: practical judgement and the lure of technique. Indiana: Notre Dame Press; 1993. [Google Scholar]
- 33.Nussbaum M. The fragility of goodness. New York, Cambridge: Cambridge University Press; 1986. [Google Scholar]
- 34.Polkinghorne D. Practice and the human sciences: the case for a judgment-based practice of care. New York: State University of New York Press; 2004. (SUNY series in the philosophy of the social sciences) [Google Scholar]
- 35.Toulmin S. Introduction. In: Toulmin S, Gustavsen B, editors. Beyond theory: changing organisations through participation. Amsterdam/Philadelphia: John Benjamins Publishing Company; 1996. pp. 1–4. [Google Scholar]
- 36.Wentzer H, Bygholm A. IT-implementering som læreproces [IT-implementation as learning processes] Aalborg (Denmark): Aalborg University; 2001. (V-CHI tech report 01-1). [in Danish] [Google Scholar]
- 37.Bygholm A. Communicating across sectors in health care: a case of establishing new infrastructure. In: Kanstrup AM, Nyvang T, Sørensen EM, editors. Digital governing. Aalborg (Denmark): Aalborg University Press; 2007. pp. 171–92. [Google Scholar]
- 38.Bygholm A, Boisen E. Implementering og informatisering [Implementation and Informating] In: Dirckinck-Holmfeld L, Dalum B, Ulrich J, Boisen B, editors. Det Digitale Nordjylland—IKT og omstilling til netværkssamfundet [Digital North Denmark—ICT and change to network society] Aalborg (Denmark): Aalborg Universitetsforlag; 2004. pp. 209–44. [in Danish] [Google Scholar]
- 39.Wentzer H, Bygholm A. Disclosure of infrastructure in health care. In: Hejlesen O, et al., editors. SHI2006, Proceedings of the 4th Scandinavian Conference on Health Informatics; August 24–26, 2006; Aalborg University. Aalborg (Denmark): Virtual Centre for Health Informatics; 2006. p. 74. [Google Scholar]
- 40.Indberetning af utilsigtede hændelser relateret til EPJ. Århus Amts auditrapport, [Incident reporting of unintended consequences in the electronic patient record. Audit-report of Hospital Owners in Aarhus County] Aarhus (Denmark): Aarhus Amt; 2005. Mar 21, [in Danish] [Google Scholar]
- 41.Böttger U. Medicinmodulet i medicineringsprocessen [CPOE in the Medication Process]. [thesis] Aalborg (Denmark): Aalborg University; 2005. Available from: http://projekter.aau.dk/projekter/research/medicinmodulet_og_medicineringsprocessen(6979138)/. [in Danish] [Google Scholar]
- 42.Wentzer H, Böttger U, Boye N. Ubiquity: technologies for better health in aging societies: Proceedings of MIE2006, Medical Informatics of Europe. Amsterdam: IOS Press; 2006. A socio-technical study of a ubiquitous CPOE-system in local use; pp. 326–32. (Technology and Informatics; 124) [PubMed] [Google Scholar]
- 43.Schuback MSC. The knowledge of attention. International Journal of Qualitative Studies on Health and Well-being. 2006 Sep;1(3):133–40. Available from: http://www.tandf.no/qhw. [Google Scholar]
- 44.BEK Nr. 116 af 23/02/2004, § 1, stk. 3, og § 4, stk. 2, i lov nr. 429 af 10. juni 2003 om patientsikkerhed i sundhedsvæsenet. Available from: http://www.retsinfo.dk/DELFIN/HTML/B2003/0101805.htm. [in Danish]
- 45.Årsrapport 2006—DPSD Dansk PatientSikkerhedsDatabase [Yearly report from the Danish Patient Security Database] Copenhagen (Denmark): Sundhedsstyrelsen; 2007. Mar, Available from: http://www.sst.dk/publ/Publ2007/EFT/DPSD_aarsrapport/DSPDaarsrap2006.pdf. [in Danish] [Google Scholar]
- 46.Engeström Y. Developmental work research as educational research: looking ten years back and into the zone of proximal development. Nordisk Pedagogik: Journal of Nordic Educational Research. 1996;16(3):131–43. [Google Scholar]
- 47.Bergk M. The politics of technology: on bringing social theory into technological design. Science, Technology and Human Values. 1998;23(4):456–90. [Google Scholar]
- 48.Latour B. We have never been modern. Cambridge (MA): Harvard University Press; 1993. [Google Scholar]
- 49.Ehn P. Work-oriented design of computer artifacts. Falköping, Sweden: Arbejdsliv Centrum; 1988. Hillsdale, NJ: Lawrence Erlbaum Associates. [Google Scholar]