A Survey of the Literature on Unintended Consequences Associated with Health Information Technology: 2014–2015

Summary

Objective

To summarize recent research on unintended consequences associated with implementation and use of health information technology (health IT). Included in the review are original empirical investigations published in English between 2014 and 2015 that reported unintended effects introduced by adoption of digital interventions. Our analysis focuses on the trends of this steam of research, areas in which unintended consequences have continued to be reported, and common themes that emerge from the findings of these studies.

Method

Most of the papers reviewed were retrieved by searching three literature databases: MEDLINE, Embase, and CINAHL. Two rounds of searches were performed: the first round used more restrictive search terms specific to unintended consequences; the second round lifted the restrictions to include more generic health IT evaluation studies. Each paper was independently screened by at least two authors; differences were resolved through consensus development.

Results

The literature search identified 1,538 papers that were potentially relevant; 34 were deemed meeting our inclusion criteria after screening. Studies described in these 34 papers took place in a wide variety of care areas from emergency departments to ophthalmology clinics. Some papers reflected several previously unreported unintended consequences, such as staff attrition and patients’ withholding of information due to privacy and security concerns. A majority of these studies (71%) were quantitative investigations based on analysis of objectively recorded data. Several of them employed longitudinal or time series designs to distinguish between unintended consequences that had only transient impact, versus those that had persisting impact. Most of these unintended consequences resulted in adverse outcomes, even though instances of beneficial impact were also noted. While care areas covered were heterogeneous, over half of the studies were conducted at academic medical centers or teaching hospitals.

Conclusion

Recent studies published in the past two years represent significant advancement of unintended consequences research by seeking to include more types of health IT applications and to quantify the impact using objectively recorded data and longitudinal or time series designs. However, more mixed-methods studies are needed to develop deeper insights into the observed unintended adverse outcomes, including their root causes and remedies. We also encourage future research to go beyond the paradigm of simply describing unintended consequences, and to develop and test solutions that can prevent or minimize their impact.

I Introduction

It has been extensively documented that introduction of health information technology (IT) is often associated with effects that are not intended by software designers, implementers, healthcare administrators, or clinicians; also known as “unintended consequences” or “e-iatrogenesis” [1–4]. While such effects can be beneficial, a majority of the health IT-related unintended consequences reported in the extant literature is found to cause adverse outcomes, such as new types of patient safety risks (e.g., wrong patient selection), as a result of poorly designed user interface, disrupted workflow, and communication breakdown [5, 6]; and increased workload for clinicians (e.g., additional documentation demand), as a result of new regulatory requirements enabled and subsequently enforced by the use of health IT [7, 8].

Unintended adverse consequences became a particularly pronounced issue in mid-2000’s with emerging evidence demonstrating that they directly or indirectly contributed to adverse patient safety events, near misses, and unsafe conditions [5, 6, 9–11]. In recent years, this concern has escalated as many industrialized countries began to implement policies and incentive programs to promote rapid adoption of health IT across all hospitals and clinics [2]. For example, two years after the commencement of the “Meaningful Use” program as part of the 2009 Health Information Technology for Economic and Clinical Health (HITECH) Act, the U.S. Office of the National Coordinator for Health Information Technology requested the Institute of Medicine (IOM) to convene a special committee to evaluate health IT-related safety concerns, resulting in a landmark report “Health IT and Patient Safety: Building Safer Systems for Better Care” [12]. Around the same time, the U.S. Agency for Healthcare Research and Quality contracted the RAND corporation to create a guide to help practitioners reduce unintended consequences when using electronic health records (EHR) [13].

As previous work has pointed out, “unintended” and “unanticipated” should not be treated as synonyms in this context [2]. Unintended consequences emphasize that a technological intervention exerts a temporary or enduring effect deviating from its original design intent [2]. For example, while one of the widely anticipated benefits of implementing EHRs is to improve work efficiency, their initial deployment often introduces a transitory, negative impact on productivity as users adapt to the technology. Even though this temporary loss of productivity is not intended (and unwanted), this “ramp-up” effect could and should be anticipated by any organization attempting to introduce a new technology in a new environment. However, the magnitude of this productivity loss may be observed at a level that is unacceptable and unanticipated by the implementation team, or continue on for a prolonged period of time with no clear sign of improvement. In such scenarios, unintended adverse consequences are not anticipated, warranting close attention and careful treatment. It should be noted that by this logic, unanticipated consequences are always unintended, as unanticipated outcomes are not obtainable by conscious design.

This review, conducted in early 2016, aims to summarize relevant research development in the past two years (2014–2015) to report on the trends of this steam of research, areas in which unintended consequences have continued to be reported, and common themes that emerge from the findings of these studies. Based on the conceptual framework above, we define the inclusion criteria as follows.

• First, the reported consequences must be unintentional, i.e., not planned by software designers or by the implementation team. These include unintended effects exerted on intended outcomes dimensions (e.g., a reminder system designed to improve the frequency of weighing pediatric patients was instead found to decrease the frequency). They also include effects exerted on unintended, perhaps seemingly unrelated outcomes dimensions (or “collateral effects,” e.g., an electronic alert designed to reduce drug–drug interaction errors was found to cause an unexpected increase of other types of medication errors).

• Second, the intervention under investigation must be ‘digital,’ i.e., software applications (e.g., EHR) or hardware systems integrated with software systems (e.g., bar-code medication administration). By this definition, implementation of a managerial protocol on improving team coordination, or new clinical guidelines on mammographic screening, are considered out of the scope. When such managerial protocols and clinical guidelines were indeed implemented through a digital form (e.g., as an electronic checklists or as computerized reminders), we use our best judgment to determine if the evaluation focused on the stimuli itself, or on the delivering methods (i.e., health IT).

• Third, the impact of the intervention must be assessed using valid quantitative, qualitative, or mixed-methods designs; and the unintended consequences observed, if quantifiable, must be substantial. For example, if a computerized prescriber order entry (CPOE) system implemented to improve medication safety was found to cause no or statistically insignificant change, then it is not deemed within the scope of this review. However, if the study also included a rigorous qualitative investigation that led to the discovery of unintended consequences accounting for the ineffectiveness of the intervention, then it may be considered within the scope.

• Fourth, while relevant position papers, commentaries, and policy briefs are cited throughout this paper, a study to be included in the review must be original empirical investigation of an intervention that had been deployed in the field for routine or for trial use. By this definition, this review excludes opinions and thought pieces that may have offered only anecdotal evidence, and unintended effects reported in laboratory testing of early technology prototypes that did not involve naturalistic settings and real users. However, survey studies seeking general perceptions (e.g., of the potential privacy and security risks that may be associated with health IT) are included, even if some respondents may not have direct experience with the technology studied.

This paper is organized as follows. The next section, Background, presents a brief overview of the history of relevant research on health IT-related unintended consequences, in addition to existing literature surveys and systematic reviews. Then, the Methods section describes the approach we used in searching literature databases to retrieve relevant papers, and the review process. The Results section summarizes each paper reviewed, and the key findings organized into relevant thematic groups. Finally, in the Discussion section, the authors offer their reflections on new research development pertinent to the topic in 2014 and 2015, as well as gaps and recommendation for future directions.

II Background

A History of Relevant Research

While use of computers in healthcare can be traced back much earlier, health informatics research as a scientific domain emerged around the 1960’s, producing a proliferation of medical diagnostic systems based on early breakthroughs in artificial intelligence [14, 15]. However, due to the lack of supporting infrastructures at the time, such as hospital information systems that capture and store patient data crucial to diagnostic accuracy and relevance, most of these early-generation diagnostic systems did not make their way into everyday clinical practice [14].

In late 1990’s, a second wave of health IT research and practice arrived [16]. This new wave was marked by several milestone papers published in prestigious medical journals that unequivocally demonstrated the practical value of more modern forms of health IT such as computerized order entry and decision support [17]. For example, through a randomized controlled trial, Bates et al. (1998) showed that a CPOE system deployed at the Brigham and Women’s Hospital (Boston, MA, USA) decreased the rate of non-intercepted serious medication errors by more than half [18]. In another study, also conducted at Brigham and Women’s, Teich et al. (2000) found that CPOE coupled with computerized decision support was effective in altering physicians’ prescribing behavior, resulting in improvements along multiple dimensions such as guideline adherence and overdose prevention; and these improvements persisted at one- and two-year follow-up [19]. The demonstrated early successes of these modern health IT systems inspired many other healthcare organizations to pursue similar technologies. In the U.S., reports from the Institute of Medicine raised awareness of quality gaps and the harms inflicted on patients by the healthcare system, fueling a movement to support error-prone human practitioners with computational support for tasks and decisions [20, 21]. This movement was further stimulated by an executive order signed by then Bush Administration calling for most Americans to have electronic health records by 2014 [22]; and by a 2005 RAND report that predicted significant cost-savings if health IT were adopted nationwide [23, 24]. At the global stage, the World Health Assembly passed a Resolution (WHA 58.28) in 2005, acknowledging that eHealth (i.e., information and communication technologies) is a cost-effective tool for improving patient care delivery and public health. The Resolution urged its member states to consider developing and implementing eHealth services in the various areas of the health sector [25].

Despite early warnings about cultural and behavioral barriers, lack of user friendliness, and potential errors due to automated decision support/making [26–28], many healthcare organizations rushed to acquire health IT in order to become fully ‘wired’ and ‘paperless’ [29]. During this time, vendor-supplied commercial systems became prevalent, as few provider institutions had the capacity to develop homegrown systems customized to their needs and constraints. However, implementing off-the-shelf IT products, at large scales, in very complex healthcare organizations, was confronted with fierce challenges that many did not anticipate. It was during this period that unintended adverse consequences associated with health IT began to surface.

Drawing upon a literature review and a series of qualitative studies conducted in the U.S., The Netherlands, and Australia, Ash et al. (2004) began to systematically document instances in which health IT systems might foster errors, rather than reduce their likelihood [1]. They found that health IT-induced errors are common in two processes: the process of entering and retrieving information and the process of communication and coordination. A year later, Koppel et al. (2005) studied another deployment of CPOE and found that use of the system could facilitate 22 new types of medication safety risks, ranging from information errors—caused by data fragmentation and failure to integrate the CPOE with other hospital IT systems; in addition to human–machine interface flaws—originating in machine rules that do not correspond to work organization or usual behavior [5]. In the same year, Han et al. (2005) unveiled an unexpectedly increased mortality rate in a children’s hospital that appeared to coincide with the implementation of a commercial CPOE system; speculated reasons for it included chaotic workflow post-implementation, e.g., “hard-stop” alerts that inhibited proper actions from being taken during emergency situations; and reduced opportunities of communication among clinicians, especially between physicians and nurses [6]. Despite controversies in statistical inference and causality analysis [30], this paper received an enthusiastic response from the informatics community praising the authors’ courage to report their findings and calling for deeper investigation into the underlying sociotechnical root causes of the observed unintended consequences [31].

These early research accounts spurred many follow-up studies that led to a surge of publications produced on the topic in a relatively short period of time [9–11, 32–36], putting the issue in a national spotlight. The discussions culminated in 2009 when the American Medical Informatics Association (AMIA) decided to devote its Annual Health Policy Meeting to unintended consequences of health IT. A consensus report out of the meeting was published in the subsequent year [2]. The report recognized the severity and the magnitude of the issue. It also expressed a salient concern that most knowledge known about unintended consequences at the time was from a small number of early adopters. With the accelerated uptake of health IT at the national level on the horizon because of the 2009 HITECH ACT, there was an urgent need for additional research on unintended consequences of health IT and a concerted effect to collect, refine, and disseminate research findings to practitioners.

B Relevant Literature Surveys and Systematic Reviews

While there has been a significant body of work published on unintended consequences associated with health IT, literature reviews, systematic reviews, and meta-analyses dedicated to the topic have been surprisingly scarce. Besides the 2009 IOM report and the 2010 AMIA Health Policy Meeting report that included an in-depth review of the relevant literature [2, 12], we were only able to locate a few related reviews.

Pirnejad, Bal, and Shahsavar (2010) surveyed the scholarly work published before September 2009 and identified 26 relevant papers [37]. The unintended consequences reported in these papers were classified into four general areas: workflow, communication, technical, and user-related. Carling et al. (2010) conducted another review in the same year, focusing on problematic prescriptions and adverse outcomes that may be associated with electronic medication management systems used in ambulatory care [38]. By reviewing 38 studies that met the inclusion criteria (18 of them were randomized controlled trials), the authors concluded that the scientific evidence available at the time was not adequate to substantiate fears of additional patient safety risks as a result of using IT; however, the authors also concluded it was premature to reject such a hypothesis, due to the lack of evidence. In the following year, Harrington et al. (2011) searched the literature published between 2000 and 2009 and identified 24 relevant studies [29]. The authors concluded in the review that hospitals at the time were under tremendous pressure to implement health IT because of its “demonstrated and presumed improvements to patient safety.” However, unintended adverse consequences, especially those that presented a threat to patient safety, were evident based on the authors’ analysis of the literature, and should be considered.

Three other reviews were published more recently. In Voshall et al. (2013), the authors examined nurses’ behavior working around barcode medication administration systems [39]. They found 13 studies describing this behavior, concluding that workarounds were common in nursing practice, which might negate the benefits of barcode medication administration and facilitate new types of errors. Another review by Gephart, Carrington, and Finley (2015) focused on nurses’ negative experience with use of EHRs [40]. Only five studies were identified, however. These studies reported several adverse effects affecting nursing work such as undesirable workflow, constantly changing requirements for work due to imperfect EHR design, and difficulties in accessing necessary information at the point of decision-making. The last review, by Marcilly et al. (2015), looked into how usability flows of medication alerting systems may link to usage problems and consequent unintended adverse outcomes [41]. Based on a peruse of 26 relevant publications, the authors found that usability flows (e.g., low signal-to-noise ratio of medication alerts) are responsible for a wide range of usage issues (e.g., increased workload and information involuntarily missed), which in turn results in unintended consequences that have detrimental effects on workflow, technology effectiveness, the medication management process, and patient safety.

Despite a shortage of review articles on unintended consequences, it is worth noting that since 2003, a large number of systematic reviews have become available to summarize the effort of implementing and evaluating health IT across different time periods and different types of applications (e.g., EHR, CPOE, health information exchange) [42–105]. With only a few exceptions [102–105], the results of the great majority of these reviews suggested dismissal of a once widely-held belief that use of health IT would lead to significant gains in efficiency, quality of care, patient safety, and cost containment. Even though the potential was acknowledged, most of these studies concluded that the demonstrated effectiveness of health IT systems, as how they were implemented, “is not compelling and is limited by modest study sample sizes and designs” [54]. Some studies also noted that most early successes of health IT were reported by a handful of benchmark institutions of internally developed systems, and the benefits might have been selectively reported [43, 80]; casting a serious doubt on how well the results could be replicated among average healthcare organizations.

In response to the criticism that the benefits of health IT yielded in the past decade have been modest at best, a group of RAND researchers published a paper to offer their opinions why the company’s 2005 projection fell short [106]. The authors speculated that this disappointing performance could be attributed to several factors: shortcomings in the design and implementation of health IT systems, the reluctance of clinicians to invest the time and effort required to master difficult-to-use technology, and the failure of healthcare providers and institutions to reengineer care processes to reap the full benefits of health IT [106].

III Methods and Materials

To identify relevant studies, we searched three literature databases: MEDLINE, Em-base, and CINAHL. A paper to be included in the review must be in English, and must be published, either in print or as an electronic publication ahead of print, between January 1, 2014 and December 31, 2015.

The query used a set of keywords to ensure that the intervention(s) under investigation is pertinent to health IT, e.g., “health [information technology, IT],” “electronic [health, medical] records,” “computerized [physician, provider, prescriber] order entry,” “[computerized clinical] decision support,” “e-prescribing,” as well as their anonyms and spelling variants. In the first round of search we included keywords such as “unintended consequences” and “unintended outcomes.” These keywords however proved to be too limiting, as the query failed to retrieve several relevant papers published in the target timeframe that we are aware of. A look into these papers revealed that the authors did not explicitly label their study objective, or research findings, as “unintended consequences” of health IT. Therefore, we revised our search query to use more general terms, such as “evaluation” and “implementation,” in order to capture a broader range of health IT implementation and evaluation studies.

The results of the two rounds of literature search were then consolidated for subsequent screening. Each paper was independently reviewed by at least two authors, first by title and abstract and then by full text. The most challenging part of the screening was to determine if the results reported in a study met the definition of “unintended consequences of health IT” (see the Introduction section). We used our best judgment to decide if the outcomes observed were truly unintentional; if the impact was substantial; if the intervention was ‘digital;’ and if the study was an original empirical investigation conducted based on an intervention deployed and used in the field. Equivocal cases were discussed until consensus was reached; marginally relevant papers were always kept rather than dismissed.

IV Results

The two rounds of search resulted in a total of 1,535 distinct titles (755 from 2014, and 780 from 2015).

Out of the 1,535 papers screened, 1,444 papers were deemed not meeting the inclusion criteria. Over half of the papers excluded are studies about secondary use of electronic patient care data; many on developing and evaluating new predictive models, [e.g., 107] information retrieval tools, [e.g., 108] or natural language processing algorithms [e.g., 109]. Among the remainder, many were not relevant to health IT, e.g., a study evaluating an intervention program to reduce medication preparation errors [110]; or health IT was not the main intervention or only served as a delivering platform for the intervention, e.g., a study evaluating technical assistance and financial incentives alongside EHR implementation [111]. When health IT was indeed the focus of the study, the results were either in line with the researchers’ projection (i.e., intended) or were not statistically significant [e.g., 112]; or health IT was tested in simulated laboratory environments instead of being evaluated in situ in the field [e.g., 113]. This left us 34 papers to include in this review [8, 114–146]. Figure 1 shows the PRISMA diagram exhibiting the literature search and screening processes.

Fig. 1.

Literature search and screening processes

A Overall Trends

Table 1 lists the 34 papers. Nine studies were conducted outside of the U.S. [116, 118, 121, 127, 130, 137–139, 140]. Care settings included were rather heterogeneous, but ambulatory care and emergency department (ED) were studied relatively more often. Compared to early work that predominantly focused on CPOE, more studies included in our review evaluated the impact of EHR systems (23 out of 34, or 68%). This change may not necessarily suggest a shifted focus of research, though. It may be simply because many healthcare organizations have completed the transition from standalone CPOE systems to more comprehensive EHRs, of which ordering and order management functions have become an integral part.

Table 1.

Papers reviewed (ordered chronologically based on year of publication)

Study	Country	Setting	Organization type	Main intervention	Study design	Main method	Source of data
Campos-Castillo C, Anthony DL The double-edged sword of electronic health records: implications for patient disclosure. J Am Med Inform Assoc 2015;22(e1):e130–40.	U.S.	n/a	n/a	EHR	Quantitative	Cross sectional	Questionnaire survey
Cresswell KM, Bates DW, Williams R, Morrison Z, Slee A, Coleman J, Robertson A, Sheikh A. Evaluation of medium-term consequences of implementing commercial computerized physician order entry and clinical decision support prescribing systems in two ‘early adopter’ hospitals. J Am Med Inform Assoc 2014;21(e2):e194–202.	U.K.	Hospital-wide	One teaching hospital; one non-teaching hospital	CPOE and clinical decision support	Qualitative	Case study (implemented for at least 2 years)	Artifacts; Interviews; Observations
Fleming NS, Becker ER, Culler SD, Cheng D, McCorkle R, da Graca B, Ballard DJ. The impact of electronic health records on workflow and financial measures in primary care practices. Health Serv Res 2014;49(1 Pt 2):405–20.	U.S.	Ambulatory care	Primary care practices affiliated with healthcare system	EHR	Quantitative	Interrupted time series (evaluated 1-6, 7-12, and >12 months post-implementation)	Administrative, payroll, and billing data
Howley MJ, Chou EY, Hansen N, Dalrymple PW. The long-term financial impact of electronic health record implementation. J Am Med Inform Assoc 2015;22(2):443–52.	U.S.	Ambulatory care	Academic medical center/Teaching hospital	EHR	Quantitative	Pre-post (repeated measures 1 year before and 2 years after)	Billing data
Hysong SJ, Spitzmuller C, Espadas D, Sittig DF, Singh H. Electronic alerts and clinician turnover: the influence of user acceptance. Am J Manag Care 2014;20(11 Spec No. 17):SP520–30.	U.S.	Ambulatory care	Primary care practices of U.S. Department of Veterans Affairs	Electronic alerts	Quantitative	Cross sectional	Questionnaire survey
Meeks DW, Smith MW, Taylor L, Sittig DF, Scott JM, Singh H. An analysis of electronic health record-related patient safety concerns. J Am Med Inform Assoc 2014;21(6):1053–9.	U.S.	n/a	U.S. Department of Veterans Affairs	EHR	Qualitative	Retrospective (3.5 years of incident reports)	Incident reports
Melby L, Hellesø R. Introducing electronic messaging in Norwegian healthcare: unintended consequences for interprofessional collaboration. Int J Med Inform 2014;83(5):343–53.	Norway	Homecare and ambulatory care	Public healthcare system	Electronic messaging	Qualitative	Implemented 2008-2009, interviews in 2011	Interviews
Nanji KC, Rothschild JM, Boehne JJ, Keohane CA, Ash JS, Poon EG. Unrealized potential and residual consequences of electronic prescribing on pharmacy workflow in the outpatient pharmacy. J Am Med Inform Assoc 2014;21(3):481–6.	U.S.	Pharmacy	Pharmacy	Electronic prescribing	Qualitative	Implemented in 2008; study conducted over 10 weeks in 2010	Observations; Interviews
Pell JM, Cheung D, Jones MA, Cumbler E. Don’tfuel the fire: decreasing intravenous haloperidol use in high risk patients via a customized electronic alert. J Am Med Inform Assoc 2014;21(6):1109–12.	U.S.	Inpatient	Academic medical center/Teaching hospital	Electronic alerts	Quantitative	Retrospective cohort (8 months before, 8 months after)	Chart abstraction
Redd TK, Read-Brown S, Choi D, Yackel TR, Tu DC, Chiang MF. Electronic health record impact on productivity and efficiency in an academic pediatric ophthalmology practice. J AAPOS 2014;18(6):584–9.	U.S.	Pediatric ophthalmology	Academic medical center/Teaching hospital	EHR	Quantitative	Pre-post (3 months before, 3 years after)	Practice management system (pre) and EHR enterprise reporting system (post)
Risko N, Anderson D, Golden B, Wasil E, Barrueto F, Pimentel L, Hirshon JM. The impact of electronic health record implementation on emergency physician efficiency and patient throughput. Healthc(Amst)2014;2(3):201–4.	U.S.	ED	Community hospital	EHR	Quantitative	Prospective pre-post (7 months before, 10 months after)	Timestamps from the core data system
Saddik B, Al-Mansour S. Does CPOE support nurse-physician communication in the medication order process? A nursing perspective. Stud Health Technol Inform 2014;204:149–55.	Saudi Arabia	Hospital-wide	Public healthcare system	CPOE	Quantitative	Cross sectional	Questionnaire survey
Sanders DS, Read-Brown S, Tu DC, Lambert WE, Choi D, Almario BM, Yackel TR, Brown AS, Chiang MF. Impact of an electronic health record operating room management system in ophthalmology on documentation time, surgical volume, and staffing. JAMA Ophthalmol 2014;132(5):586–92.	U.S.	Operation Room	Academic medical center/Teaching hospital	EHR and Operation Room Management System	Quantitative	Prospective cohort pre-post (3 weeks before, 12 months after); Case series	Time and motion observations; Enterprise data warehouse
Ser G, Robertson A, Sheikh A. A qualitative exploration of workarounds related to the implementation of national electronic health records in early adopter mental health hospitals. PLoS One 2014(1):e77669.	U.K.	Mental health hospitals	Public healthcare system	EHR	Qualitative	Longitudinal (data collected over a 1.5 year period)	Interviews
Ward MJ, Froehle CM, Hart KW, Collins SP, Lindsell CJ. Transient and sustained changes in operational performance, patient evaluation, and medication administration during electronic health record implementation in the emergency department. Ann Emerg Med 2014;63(3):320–8.	U.S.	ED	Academic medical center/Teaching hospital	EHR	Quantitative	Longitudinal (for 6 months post-implementation)	Administrative electronic tracking systems (pre) and EHR (post)
Woolhandler S, Himmelstein DU. Administrative work consumes one-sixth of U.S. physicians’ working hours and lowers their career satisfaction. IntJ Health Serv 2014;44(4):635–42.	U.S.	n/a	n/a	EHR	Quantitative	Cross sectional	Questionnaire survey
Benda NC, Meadors ML, Zachary Hettinger A, Ratwani RM. Emergency physician task switching increases with the introduction of a commercial electronic health record. Ann Emerg Med 2015. pii: S0196-0644(15)01104-X. [Epub ahead of print]	U.S.	ED	Academic medical center/Teaching hospital	EHR; from homegrown system to commercial system	Quantitative	Pre-post (1 month before, 1 week after, and 3-4 months after)	Field observations
Carayon P, Wetterneck TB, Alyousef B, Brown RL, Cartmill RS, McGuire K, Hoonakker PL, Slagle J, Van Roy KS, Walker JM, Weinger MB, Xie A, Wood KE. Impact of electronic health record technology on the work and workflow of physicians in the intensive care unit. Int J Med Inform 2015;84(8):578–94.	U.S.	ICU	Academic medical center/Teaching hospital	EHR	Quantitative	Prospective pre-post (3 months after)	Field observations
Carrington JM, Gephart SM, Verran JA, Finley BA. Development of an instrument to measure the unintended consequences of EHRs. West J Nurs Res 2015;37(7):842–58.	U.S.	n/a	Unknown	EHR	Qualitative	n/a	Interviews
Cifuentes M, Davis M, Fernald D, Gunn R, Dickinson P, Cohen DJ. Electronic health record challenges, workarounds, and solutions observed in practices integrating behavioral health and primary care. J Am Board Fom Med 2015;28 Suppl 1:S63–72.	U.S.	Ambulatory care	Primary care practice of varied ownership	EHR	Qualitative	Cross-case comparative study	Field observations; Online diaries; Interviews; Artifacts; Questionnaire survey
Crowson MG, Vail C, Eapen RJ. Influence of electronic medical record implementation on provider retirement at a major academic medical centre. J Eval Clin Pract 2015. [Epub ahead of print]	U.S.	n/a	Academic medical center/Teaching hospital	EHR	Quantitative	Retrospective pre-post (12 months after)	Human resources records
Georgiou A, Prgomet M, Lymer S, Hordern A, Ridley L, Westbrook J. The Impact of a Health IT changeover on medical imaging department work processes and turnaround times: a mixed method study. Appl Clin Inform.2015;6(3):443–53.	Australia	ED	Academic medical center/Teaching hospital	Picture archiving and communication system and radiology information system	Mixed methods	Retrospective pre-post (4-5 months before and 1 year after)	Interviews; Charts
Graber ML, Siegal D, Riah H, Johnston D, Kenyon K. Electronic health record-related events in medical malpractice claims. J Patient Saf 2015. [Epub ahead of print]	U.S.	n/a	n/a	EHR	Mixed methods	Retrospective cohort study	Malpractice suits and claims database
Lafata JE, Shay LA, Brown R, Street RL. Office-based tools and primary care visit communication, length, and preventive service delivery. Health Serv Res 2015. [Epub ahead of print]	U.S.	Ambulatory care	Primary care practices affiliated with healthcare system	EHR in addition toother office-based tools	Quantitative	n/a	Questionnaire survey; Field observations; Office visit audio-recordings; Charts
Magrabi F, Baker M, Sinha I, Ong MS, Harrison S, Kidd MR, Ranchman WB, Coiera E. Clinical safety of England’s national program for IT: a retrospective analysis of all reported safety events 2005 to 2011. Int J Med Inform 2015;84(3):198–206.	U.K.	n/a	Public healthcare system	Any IT	Quantitative	Retrospective analysis (6 years of incident reports)	Event database
McLean TA, Lewkowitz AK, Test E, Zlatnik MG. Does an electronic health record improve completeness of prenatal studies? Appl Clin Inform 2015;6(4):669–76.	U.S.	Ambulatory care (prenatal)	Academic medical center/Teaching hospital	EHR	Quantitative	Retrospective cohort (1 year before and 2 years after)	Charts
Overhage JM, Gandhi TK, Hope C, Seger AC, Murray MD, Orav EJ, Bates DW. Ambulatory computerized prescribing and preventable adverse drug events. J Patient Saf 2015. [Epub ahead of print]	U.S.	Ambulatory care	Primary care practices affiliated with academic medical center	CPOE	Quantitative	Pre-Post (each period lasting 6 months)	Charts
Rizzato Lede DA, Benítez SE, Mayan JC 3rd, Smith MI, Baum AJ, Luna DR, Bernaldo de Quirós FG. Patient safety at transitions of care: use of a compulsory electronic reconciliation tool in an academic hospital. Stud Health Technol Inform 2015;216:232–6.	Argentina	Inpatient	Academic medical center/Teaching hospital	Electronic reconciliation	Quantitative	Cross sectional (9-month after)	Charts
Russell RA, Triscari D, Murkowski K, Scanlon MC. Impact of computerized order entry to pharmacy interface on order-infusion pump discrepancies. J Drug Deliv 2015;2015:686598.	U.S.	PICU	Academic medical center/Teaching hospital	CPOE	Quantitative	Prospective pre-post	Field observations
Tall JM, Hurd M, Gifford T. Minimal impact of an electronic medical records system. Am J Emerg Med 2015;33(5):663–6.	U.S.	ED	Academic medical center/Teaching hospital	EHR	Quantitative	Retrospective before, during, and after (each period lasting 5.5-6 months)	Charts
Thirukumaran CP, Dolan JG, Reagan Webster P, Panzer RJ, Friedman B. The impact of electronic health record implementation and use on performance of the Surgical Care Improvement Project measures. Health Serv Res 2015;50(1):273–89.	U.S.	Inpatient	Academic medical center/Teaching hospital	EHR	Quantitative	Pre-post (3 months post)	Charts
Varpio L, Day K, Elliot-Miller P, King JW, Kuziemsky C, Parush A, Roffey T, Rashotte J. The impact of adopting EHRs: how losing connectivity affects clinical reasoning. Med Educ 2015;49(5):476–86.	Canada	PICU and inpatient	Academic medical center/Teaching hospital	EHR	Qualitative	Longitudinal before, during, and after (for 11 months before, for 18 months during and after)	Field observations; Think-aloud sessions; Think-after sessions; Interviews; Artifacts
Victores AJ, Coggins K, Takashima M. Electronic health records and resident workflow: a time-motion study of otolaryngology residents. Laryngoscope 2015;125(3):594–8.	U.S.	Otolaryngology operation room and clinic	Academic medical center/Teaching hospital	EHR	Quantitative	Prospective pre-post (1 year after)	Time and motion observations
MacMillan TE, Slessarev M, Etchells E. eWasted time: Redundant work during hospital admission and discharge. Health Informatics J 2016;22(1):60–6.	Canada	Inpatient	Academic medical center/Teaching hospital	EHR	Quantitative	n/a	Time and motion observations; Questionnaire survey

Unlike early research that was primarily qualitative, a majority of these recent studies attempted to quantify the unintended impact associated with health IT implementation or use: out of the 34 studies, 24 (71%) were quantitative investigations; 2 employed mixed methods. Electronic charts and time and motion observations were the most common sources of quantitative data; and pre-post comparison was the predominant design. No studies that we reviewed were based on randomized controlled trials.

Below, we summarize each of the papers reviewed according to the nature of the unintended consequences identified. Note that these study summaries are brief, therefore may not cover all respects of the discussions contained in the paper (e.g., outcomes that demonstrated no change before and after health IT implementation may not be included in the summary). Also note that while we organized the papers under different subsections, representing different areas of impact, some unintended consequences are interrelated and may affect multiple outcomes dimensions. For example, unintended changes in workflow or in team coordination may have significant implications on patient safety and quality of care.

B Patient Safety

Whether health IT implementation is linked with escalated patient safety risks continues to be a focal point of recent research. Eight out of the 34 papers (23%) were dedicated to this topic, especially on health IT’s adverse effects on medication safety.

Among the eight papers there were four quantitative studies. Pell et al. (2014) evaluated the effectiveness of introducing a computerized alert customized to reduce adverse drug events (ADE) among hospitalized patients with prolonged QTc [114]. While the alert was found to be effective in correcting inappropriate orders for intravenous haloperidol, it also caused occasional inappropriate discontinuation of the medication among certain patients (e.g., patients receiving end-of-life care), for whom its use was justified. Russell et al. (2015) examined potential discrepancies between intravenous fluid (IVF) orders and bedside infusion pump settings in a pediatric intensive care unit (PICU) [115]. The results show that fewer discrepancies were present after implementing a new, bidirectional interface between the CPOE system used in the PICU and the pharmacy information system. However, an unexpected increase was observed in the rate of omitted medications as well as wrong dosing. Rizzato et al. (2015) studied an electronic medication reconciliation tool deployed at an academic medical center in Argentina [116]. The authors found that while the use of the tool contributed to better quality of medication lists, it also led to accidental removal of active medications, which could impose severe threats to patient safety. In another study, Overhage et al. (2015) compared the rates of ADEs before and after implementing a computerized prescribing system across 17 ambulatory primary care practices at two study sites. The results show that the use of the system was associated with a 56% reduction in the potential ADE rate at one study site; however, at the other site, it was associated with a 104% increase of potential ADEs [117].

Two qualitative studies conducted during this period sought to investigate the nature and root causes of new patient safety risks associated with health IT. Through document analysis, interviews, and observations, Cresswell et al. (2014) studied the process of implementing CPOE and medication-related decision support at two hospitals in the U.K. [118]. Their analyses unveiled a range of IT-induced patient safety risks including unsafe workarounds; additional security measures at odds with the contingent and highly pressured work routines; confusing and inflexible user interface designs; duplicate prescribing; and reduction in team-wide discussions. In another study, Nanji et al. (2014) qualitatively assessed the impact of an electronic prescribing system implemented in an outpatient pharmacy [119]. By analyzing the collected observation notes and interview transcripts, the authors identified 26 unintended consequences that they categorized under five themes: communication, workflow disruption, cost, technology, and opportunity for new errors.

Three studies published in this period used incident reports and malpractice suits and claims as a source of research data to identify adverse events attributable to health IT vulnerability. By analyzing a large malpractice database, Graber et al. (2015) found 248 cases of malpractice suits and claims filed in 2012 and 2013 that were linked with health IT use, most of which were related to medication errors, misdiagnoses, and treatment complications [120]. The authors also reported four common sources of origin of health IT vulnerabilities: the danger inherent in hybrid systems and EHR conversions; the danger of delayed, missing, or incorrect data, services, or actions; the danger of over-reliance on the EHR; and the inherent risks using copy/paste functionality, overriding alerts, and employing workarounds. Another similar study by Magrabi et al. (2015) scrutinized the 2005–2011 incident reports received by an IT safety team in the U.K., about half of which were originally submitted to the U.K. national IT help desk [121]. The authors found that 68% of the reports described potentially hazardous circumstances. A majority of them were associated with technical rather than human factors. The third study, by Meeks et al. (2015), analyzed incidents tracked by a non-punitive, voluntary reporting system used at the U.S. Veterans Health Administration [122]. The authors found that about two-thirds of the incidents were linked with unsafe technology; the remaining due to unsafe use of technology. Most of the safety concerns described in the paper were attributable to problematic display of EHR data, software upgrades or modifications, data transmission between different components of the EHR, and ‘hidden dependencies’ within the system.

C Time Efficiency and Workflow

“More/new work for clinicians” and “unfavorable workflow issues” are two most common unintended consequences of health IT reported in early qualitative research [7]. Several studies published in 2014 and 2015 attempted to quantify their effects, using methods such as retrospective chart reviews and time and motion observations.

Four such studies were conducted in the ED setting. Two of them used a longitudinal design that involved data collection at multiple time points after introducing the intervention. Both found that EHR adoption was associated with short-term negative effects, even though these effects diminished over time. The first study, by Risko et al. (2014), examined how EHR implementation affected the rapid assessment and management of life threatening conditions in the ED at two hospitals [123]. The authors found that the median patient processing time increased immediately after the implementation, and then slowly returned to and eventually dropped below the pre-implementation levels. The second study, by Ward et al. (2014), also found a transient disruption caused by EHR implementation, reflected as increased length of stay; decreased patient satisfaction; and increased rates of medication administration, laboratory testing, and overall radiologic imaging [124]. While the length of stay and patient satisfaction measures returned to the baseline after 4 to 8 weeks, the increased rates of medication administration and the utilization of advanced testing persisted.

The third ED study, by Benda et al. (2015), conducted time and motion observations in an ED before, during, and after it transitioned from a homegrown EHR to a vendor-supplied commercial system [125]. The results show that the new system did not incur a redistribution of physicians’ time to different tasks (e.g., from direct care to indirect care), but the number of tasks that these physicians engaged in per minute increased substantially. The authors argued that higher rates of task switching could increase the cognitive burden on physicians, which might result in adverse consequences such as stress and mistakes. The fourth study conducted in the ED, by Tall, Hurd, and Gifford (2015), analyzed monthly census reports to assess how ED’s work efficiency was affected before, during, and after the implementation of an enterprise-wide EHR system at a community hospital [126]. The authors found a modest decrease in the total ED length of stay for patients who were admitted to the hospital, and an increased time in the ED for transfer patients. The percentage of patients who left ED without being seen also increased significantly.

Several other studies took place in non-ED environments. In their paper entitled “eWasted time: Redundant work during hospital admission and discharge,” MacMillan, Slessarev, and Etchells (2016) described a time and motion study conducted at an academic medical center in Canada [127]. The aim was to quantify EHR-related redundant work during admission and discharge to a general internal medicine service. As the title of the paper implies, a significant proportion of clinician time was found to be unnecessarily “wasted” on tasks such as duplicative data entry due to EHR use (note that even though this paper appeared in print in 2016, it has been available as an electronic publication ahead of print since 2014). Redd et al. (2014) studied the impact of EHR implementation in a pediatric ophthalmology clinic, and found that the implementation had a negative impact on productivity and efficiency, such as an 11% drop of overall clinical volume [128]. The authors also found that nearly half of the charts were closed outside normal business hours (30% on weekdays, 14% on weekends). Similarly, Sanders et al. (2014) evaluated the impact of an EHR and an Operation Room Management system [129], and found intraoperative nursing documentation time significantly increased especially in shorter procedures. Georgiou et al. (2015) used a mixed-methods design to assess the impact of deploying a picture archiving and communication system and a radiology information system [130]. The study, conducted in a medical imaging department in Australia, shows that turnaround time was significantly reduced, representing efficiency gains. However, assimilation of new systems with existing work processes was considered inadequate and in some instances unsafe. For example, images became instantaneously available to physicians anywhere at the hospital while the radiologist’s report was still being prepared. This raised concerns of potential misinterpretation of the images, which could lead to incorrect diagnoses.

Among the studies conducted in non-ED settings, two used a time and motion design. Carayon et al. (2015) observed residents and attending physicians’ work before and after implementing a comprehensive EHR with built-in order management and physician documentation functionalities in an ICU [131]. The results show that EHR use had significant impact on ICU physician work and workflow. After adopting the EHR, both residents and attending physicians spent more time on clinical review and documentation. EHR implementation also caused an increased rate of task switching for residents, and changed temporal flow of tasks. In the other study, Victores, Coggins, and Takashima (2015) conducted time and motion observations with otolaryngology residents to assess the impact of EHR on their workflow [132]. The authors found that the overall resident efficiency was not significantly altered. However, more time post-implementation was shifted from directly caring for patients to documenting in the EHR during clinic days (as opposed to operative days).

The last study addressing the work efficiency issue used a different approach. By surveying a nationally representative sample of 4,720 U.S. physicians, Woolhandler and Himmelstein (2014) assessed how much time physicians devote to administrative tasks, how it relates to their career satisfaction, and the role that adoption of EHR plays [8]. The results of the survey suggest that more extensive use of EHR was indeed associated with a higher level of administrative burden, which in turn led to lower career satisfaction by physicians. The authors concluded that the federal mandate of EHR adoption in the U.S., in combination with other policy changes such as a shift to employment in large practices and the increasing prevalence of financial risk sharing, would likely increase physicians’ paperwork burdens and thus contribute to their career dissatisfaction.

D Documentation Quality, Clinician Performance, and Quality of Care

Several studies focused on quality-oriented measures such as documentation quality, clinician performance, and quality of care. Through qualitative observations across eight primary care clinics and three community mental health centers, Cifuentes et al. (2015) found that current EHR systems were inadequate to support integrated behavioral health and primary care, due to their insufficient ability to document and track all relevant behavioral and physical health information; facilitate communication and coordination; and exchange information with other devices and other EHRs [133]. To accommodate these limitations, clinicians employed numerous workarounds such as double data entry, which could be linked with unintended efficiency loss and increased chances for errors. In another study, Lafata et al. (2015) quantitatively assessed the impact of EHR use on ambulatory primary care by analyzing 485 office visits with 64 primary care physicians [134]. The results show that patients seen by providers with an EHR had longer visits and received fewer guideline-recommended preventive care services. While the authors did not investigate reasons underlying this quantitative finding, they suggested that “screen-driven” information gathering (i.e., the provider’s dialog with the patient is driven by the order of the screens and information that the computer displays) [147] might be a contributing factor, as an EHR-driven dialog could lead to missed opportunities for comprehensive inquiry and risk factor assessments.

In McLean et al. (2015), the authors described a chart-review study that compared documentation rates of prenatal HIV and purified protein derivative (PPD) tests before and after an EHR system was deployed [135]. The results show that in the year following the EHR implementation, there was a significant drop in the documentation rate of PPD tests. While the situation improved in the second year, the rate did not compare more favorably to that of paper charts used at baseline. No substantive change was observed for HIV tests. Similarly, when studying the impact of EHR implementation at a tertiary-care teaching hospital, Thirukumaran et al. (2015) also found a significant temporary reduction in surgical quality immediately following the EHR implementation, even though all quality indicators returned to their baseline levels three months after the implementation [136].

Lastly, Varpio et al. (2015) conducted a qualitative study to examine how patient data presented in an inpatient EHR system might affect clinical reasoning [137]. The authors found that while the EHR was effective in collecting dispersed data, it failed to display the data back to clinicians in a cognitively efficient matter that would facilitate clinical reasoning. In particular, they found that the electronic flowsheet provided in the EHR overly emphasized individual data values, diminishing the information linking the data chronologically and with other relevant data elements. By contrast, old paper flowsheets emphasized chronology and data interconnectedness, which are critical to efficient and informed decision-making.

E Communication and Coordination

Several studies presented in the earlier sections touched upon health IT’s unintended consequences on communication and coordination, with most findings being negative [118, 119, 133]. However, Melby and Hellesø (2014) reported predominantly positive unintended effects after introducing an e-message tool at a Norwegian hospital [138]. By interviewing clinicians, clerks, and managers, the authors found that the tool encouraged clinicians and staff to become more proactive in communicating with others. They also found that after implementing the tool, nurses perceived more weight to their requests as e-messages were automatically archived as part of patient records. Similarly, Saddik and Al-Mansour (2014) reported mixed experiences by nurses from a hospital in Saudi Arabia where CPOE functions were implemented as part of a comprehensive EHR system [139]. Many nurses participating in the study agreed that the newly implemented CPOE functions supported their workflow and enhanced nurse–physician communication; however, they also needed to follow up with physicians more frequently on medication orders after the implementation. Additionally, the study found that nurses with longer years of experience, and those from the surgery department, did not perceive the benefits of using computerized ordering.

F Workarounds

Workarounds are common tactics employed by end users to circumvent limitations imposed by health IT [11]. Certain workarounds can facilitate clinical work by mitigating the influence of poorly designed software systems. However, they may also engender practices that are unsafe [11, 39]. Among the studies that we reviewed, several described clinicians’ working-around behavior in response to various hindering conditions created by adoption of technology [118, 120, 133]. Further, the study by Ser, Robertson, and Sheikh (2014) was dedicated to seeking examples and explanations of health IT-induced workarounds [140]. Through interviewing stakeholders involved in implementing or using an EHR system at two early adopter mental health hospitals in the U.K., the authors found that workarounds, such as deferred data entry and deliberately entering data in wrong places, were common. They also found that a wide range of operational, cultural, organizational, and technical factors contributed to clinicians’ decision to work around an IT application rather than using it as prescribed.

G Financial Impact

Two studies assessed the potential unintended financial impact of adopting health IT. Fleming et al. (2014) analyzed the administrative, payroll, and billing data collected from 26 primary care practices across multiple time points before and after the implementation of an ambulatory EHR [141]. The results show that staffing and practice expenses increased following the implementation, although after excluding software maintenance cost the magnitude of the increase appeared to be small. Further, while productivity, patient volume, and net income decreased initially, they all recovered to the baseline levels after 12 months. The authors concluded that the longer-time productivity and financial performance of the study practices were not affected by the EHR implementation; however, the short-term negative impact was evident. In another study, Howley et al. (2015) analyzed the productivity and reimbursement data collected longitudinally from 30 ambulatory care practices for 2 years after implementing an EHR system [142]. The authors found that, by comparing each site to their pre-EHR baseline, the study practices saw fewer patients with the EHR, but their reimbursements significantly increased.

H Staff Attrition

End user dissatisfaction (and related symptoms such as negative emotions, stress, and anxiety) was discovered very early on as one of the unintended adverse consequences that may be associated with health IT adoption [9, 10]. Previous survey studies also indicated that this dissatisfaction could lead to personnel attrition issues including staff turnover or early retirement. For example, in a 2011 survey conducted in the U.S., 12% of the pediatric urologists responding to the survey expressed that they would retire early if EHR use became a federal mandate [148]. In this review, two studies were specifically conducted to address the question whether providers and other types of healthcare workers might choose to leave the workforce due to difficulties in adapting to health IT.

Through surveying primary care providers employed by the U.S. Department of Veterans Affairs, Hysong et al. (2014) assessed providers’ perception of EHR-based alert notifications, and how this perception might be linked with their intentions to quit and turnover [143]. The authors found that monitoring/feedback, i.e. “degree to which employee’s performance using EHR-based alerting systems is monitored and to which feedback is provided,” is a strong predictor of intention to quit. Further, they found that the perceived value of EHR-based alert notifications also predicts intention to quit, indirectly through its effects on provider satisfaction; and is directly linked with voluntary turnover. In another study, Crowson, Vail, and Eapen (2015) investigated staff attrition at a large academic medical center [144]. The analysis was based on three-year monthly provider attrition data collected before and after implementing a commercial EHR that replaced an in-house developed system. The results show that a peak in provider departure occurred in the month immediately before the EHR was implemented. Median age of the providers who left during this period was higher than that of the providers who departed at other times.

I Privacy and Confidentiality

One study included in our review assessed whether EHR adoption may adversely affect patients’ willingness to disclose information due to privacy and security concerns. By surveying a nationally representative patient sample in the U.S., Campos-Castillo and Anthony (2015) found that 13% of the respondents reported having withheld information from a provider to protect against the perceived EHR privacy and security risks [145]. After accounting for the respondents’ global ratings of care, the authors confirmed a positive relationship between patients withholding information and their physician using an EHR during the patient encounter.

J Methods Development

Only one study was devoted to methodological development. Carrington et al. (2015) described their effort developing a survey questionnaire to quantify unintended consequences that may be experienced by nurses [146]. The instrument contains constructs identified from the authors’ prior empirical work (which is why we deemed this paper to be with the scope of this review). These constructs include perceived barriers (e.g., hardware issues, data entry, and irretriev-ability) and nurse-initiated solutions (e.g., documentation shortcuts and saving without signature). Psychometric testing of the instrument was not reported in the paper, but was said to be currently underway.

V Discussion

Research in the past decade has unequivocally demonstrated the existence of unintended consequences that may accompany health IT’s implementation and use [2, 12]. The studies published in the recent two years contribute more evidence. These studies report a wide range of unintended effects introduced by many different types of health IT applications across many different types of care areas and organizational settings. With a few exceptions [e.g., 138], most of the unintended consequences reported in these studies were associated with adverse outcomes.

The recent research included in this review differs from early work in several distinctive ways. First, more studies sought to quantify the impact of health IT using objectively recorded data such as patient census, billing claims, and human factors records. Quantitative results from these studies, although constrained by the quality and direct applicability of data sources, have greatly enhanced the existing knowledge base mostly contributed by early studies based on qualitative accounts. Qualitative accounts are an indispensable means for discovering unintended consequences of health IT and the mechanisms by which they can produce harm. However, they often fall short of measuring the magnitude of their impact, and can be more susceptible to prejudices (e.g., reluctance to change) and biases (e.g., cognitive heuristics and recall errors). Second, compared to early work, the scope of research had expanded considerably in the past two years. The studies that we reviewed evaluated several new types of health IT applications, such as e-messaging and electronic medication reconciliation; covered more care areas, such as ophthalmology clinics and mental health hospitals; and examined additional dimensions of unintended outcomes, such as financial impact, staff attrition, and patients’ withholding of information due to privacy and security concerns. Collectively, these studies enriched the body of research on unintended consequences and extended beyond its traditional focus on CPOE and medication safety. Third, leveraging the increasing availability of clinical environments that had been exposed to health IT for longer periods of time, several studies involved longitudinal or time series designs to collect repeated measurements at multiple time points post-implementation [124, 125, 129, 135, 136, 141, 142]. These repeated measurements enabled researchers to distinguish temporary effects during and immediately following health IT implementation, from longer-term impact after the technology had been fully assimilated into clinical practice and end users’ job routines. This knowledge would help researchers and practitioners better understand the life span of different types of unintended consequences so that more effective remedy strategies can be developed accordingly.

Our analysis of the recent literature also revealed several methodological limitations yet to be addressed. More than half of the studies were conducted at academic medical centers or teaching hospitals. These settings have very unique characteristics (e.g., resident rotations) that are not commonly found in other types of healthcare organizations. Generalizability of the results obtained from studying these settings may be therefore limited. Further, while the quantitative studies contributed richer knowledge to the existing evidence base, very few studies used a mixed-methods design to explain what was observed quantitatively. As a result, questions such as what accounted for the unintended consequences, and how to mitigate their adverse impact, could only be answered with anecdotes or authors’ speculation; or were left unanswered in some of these studies. Additional mixed-methods research is therefore needed, to not only quantify the impact of unintended consequences, but also develop a better understanding of their root causes and measures that can be used to counter their adverse effects.

To this end, we encourage future research to go beyond the paradigm of proving the existence of unintended consequences, and to start developing and testing solutions that can prevent or minimize their impact [e.g., 149]. We believe that, based on the evolution of this body of research in the past decade, a widely held consensus has been reached that unintended adverse consequences brought by health IT implementation are almost inevitable, even if their effects may be temporary and the magnitude of the impact may be small [2, 12, 150]. Therefore, any healthcare organizations preparing for new health IT uptake should always anticipate such effects and have plans in place accordingly [151]. In addition, all stakeholders need to work together to continue to improve and enforce the standardization of terminologies and information exchange protocols. Vendors should also invest more in improving the usability of their software, and reducing the barrier to sharing data within their own systems and with other systems.

Even though gaps remain [152], we are now seeing efforts initiated toward these directions. The U.S. Office of the National Coordinator for Health Information Technology, for example, has funded a series of studies examining the safety aspects of health IT and suggesting strategies to safely implement health IT [150, 153–158]. Several papers cited in this review, including the IOM report on health IT and patient safety [12], are a direct result of this investment. Other federal agencies in the U.S., such as the National Institute of Standards and Technology, the Agency for Healthcare Research and Quality, and the Food and Drug Administration, joined the effort and issued several guidelines to improve the usability of EHR systems [159, 160] and to monitor and curb their unintended adverse effects [13, 161]. Another recent report, prepared by the National Quality Forum under a contract with the U.S. Department of Health and Human Services, provides guidance on identification and prioritization of health IT patient safety measures [162]. A coalition of disparate stakeholders has also come together to produce a roadmap to a Health IT Safety Collaborative [163]. Similar initiatives are being taken in the private sector as well. These range from “Partnership for Health IT Safety” spearheaded by the ECRI Institute [164]; to a recently announced million-dollar HeroX challenge to produce a safe, secure, and 100% accurate national patient identifier created by the College of Healthcare Information Management Executives [165].

This review has several limitations. First, as mentioned earlier, identifying published work reporting unintended consequences of health IT proved to be a very difficult task. In the title or the abstract, studies may not be explicit about their findings related to unintended consequences, as it may not be the main intended goal of the study. Further, studies may report relevant findings using an alternative language, such as “adverse impact on patient safety,” which makes our keywords-based literature search difficult to conduct. Thus, despite best effort, this review may not be inclusive of all papers that have reported relevant results. Second, the scope of the review is restrained to a two-year window. This is a relatively short time period which may not be indicative of all recent development of the field. Third, to facilitate literature search and data analysis, we adopted a rather restrictive definition of unintended consequences. It is possible that our review left out some studies that do not meet the inclusion criteria yet present value to understanding the phenomenon of unintended consequences associated with implementation and use of health IT. Lastly, because this review only targeted original empirical investigations published in the scientific literature, we did not include thought pieces [e.g., 166], positions papers [e.g., 167], framework development [e.g., 168], books [e.g., 169], and policy briefs and government reports [e.g., 154–157, 160, 161]. These publications however may contain valuable information and insights. We therefore encourage readers to seek them out for a more comprehensive picture of the recent development in unintended consequences research.

VI Conclusion

We surveyed the 2014–2015 scientific literature to examine studies investigating unintended consequences associated with health IT implementation and use. We found that the research published in this period represents a significant leap forward by covering more care areas, more IT applications, and more dimensions of unintended outcomes. However, while more quantitative studies were conducted, few incorporated a qualitative component to gain deeper insights into what was quantitatively observed. More research using a mixed-methods design is therefore needed. We also encourage future research to deploy and evaluate solutions that prevent or minimize the impact of unintended consequences, rather than simply seeking more evidence to prove their existence.