Summary
Objectives
To review the history of clinical information systems over the past twenty-five years and project anticipated changes to those systems over the next twenty-five years.
Methods
Over 250 Medline references about clinical information systems, quality of patient care, and patient safety were reviewed. Books, Web resources, and the author’s personal experience with developing the HELP system were also used.
Results
There have been dramatic improvements in the use and acceptance of clinical computing systems and Electronic Health Records (EHRs), especially in the United States. Although there are still challenges with the implementation of such systems, the rate of progress has been remarkable. Over the next twenty-five years, there will remain many important opportunities and challenges. These opportunities include understanding complex clinical computing issues that must be studied, understood and optimized. Dramatic improvements in quality of care and patient safety must be anticipated as a result of the use of clinical information systems. These improvements will result from a closer involvement of clinical informaticians in the optimization of patient care processes.
Conclusions
Clinical information systems and computerized clinical decision support have made contributions to medicine in the past. Therefore, by using better medical knowledge, optimized clinical information systems, and computerized clinical decision, we will enable dramatic improvements in both the quality and safety of patient care in the next twenty-five years.
Keywords: Clinical informatics, patient safety, decision making, computer assisted, electronic health records
I Strategies for Predicting the Future
Having been asked to review the past twenty-five years of clinical information systems and forecast where the field might go in the next twenty-five years, I had to turn to the late Yogi Berra, an all-pro New York Yankees baseball player, who was famous for many brilliant “quotes”. His statement (Figure 1) about predicting the future is appropriate here to put this piece into perspective.
“It’s tough to make predictions, especially about the future.”
Yogi Berra, 1925-2015
In addition to Yogi Berra’s quote, a recent book about “super forecasting” caught my eye and provided a framework for making predictions about clinical information systems [1]. Tetlock, as a research investigator, gathered large groups of experts -- academics, pundits, and the like -- and had them make thousands of predictions about the economy, stocks, elections, wars, and other issues of the day. After some time had passed, Tetlock evaluated the accuracy of their predictions. He found that on average most so-called experts did about as well as a dart-throwing chimpanzee. However, he found that there were also superforecasters in the group – they were much better than the random predictions of a dart-throwing chimpanzee. Tetlock then explained how, over a twenty-year period, he made observations about techniques used by these superforecasters as they made predictions. One must gather data about a topic, study it, think about it, and then make a prediction. Not surprisingly, gathering data is a key part of making accurate predictions. A comment from the Tetlock Superforcasting book (page 232) changed my approach to the preparation of this manuscript: “What makes them [superforecasters] so good is less what they are than what they do – the hard work of research, the careful thought and self-criticism, the gathering and synthesizing of other perspectives, the granular judgments and relentless updating.” As I was doing the research to prepare this manuscript in the summer of 2015, I quickly realized that the breadth of my knowledge was very narrow, primarily related to implementation of clinical computing systems. I felt that my knowledge of developments in clinical information systems needed updating as well as broadening and that predicting anything 25 years in the future was a nearly impossible task. As a consequence, I broadened my research approach and sought out perspectives and literature that were completely new to me. Based on the historical progress of clinical information systems development and deployment, I have hope for the future. It takes decades to implement systems and change user behavior. Implementing successful, useful, and operational clinical care systems requires many attempts.
Over the past decade society has witnessed huge changes in Information and Communications Technology (ICT) such as the “smart phone” (iPhone first sold in 2007). In addition, we have seen governments of many nations invest multi-billion dollars into health information and communications technology. Those include Australia [2], Belgium [3], Canada [4], Denmark [5], the United Kingdom [6], and the United States of America (USA) [7]. In the 1990s, I did not anticipate that the USA government would allocate nearly $30 billion through the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009 [7] to support the implementation of computerized Electronic Health Records (EHRs) in clinicians’ offices and hospitals. Computerized EHRs require the complex integration of records from physicians, nurse practitioners, pharmacists, and a myriad of other health care professionals. The HITECH Act brought with it Meaningful Use (MU) – with financial incentives for meeting MU requirements and financial penalties for non-compliance, in effect regulating EHR implementation [8, 9].
Since many of the readers of this manuscript will not be from the US, a brief overview of MU is presented here [10]. In 2009, as part of a stimulus bill, the HITECH Act designed to modernize healthcare using EHR infrastructure - an effort led by the Centers for Medicare & Medicaid Services (CMS) and the Office of the National Coordinator (ONC) for Health Information Technology - proposed MU as a strategy to insure progress and interoperability of EHRs throughout the US. CMS granted an incentive payment to healthcare professionals and hospitals who had “certified” EHR systems and made progress based on a three-stage implementation process of MU. Stage 1 began in 2011 with requirements dealing with data capture and sharing; e.g. computerized medication order entry with drug-drug and drug-allergy checks. Stage 2 began in 2013 with requirements demonstrating care improvements using advanced clinical processes; e.g. submit electronic data for immunization. Stage 3 was set to begin in 2015 to demonstrate improved patient outcomes; e.g. measure quality of care improvement. However, at the time of preparation of this manuscript Stage 3 MU requirements have not been finalized. As the Stages of MU have progressed, requirements have become more numerous, more complex, and much more difficult to meet. As a consequence, there has been increased debate and controversy about MU expectations and value.
Further, in the USA, the Affordable Care Act (ACA) -- also known as “Obama Care” -- was passed in March 2010 and is now being implemented [11]. ACA was designed to provide broader funding for medical care in the USA, making our country’s health system more like many other economically developed countries’ systems in the world. Electronic health records were mandated to help achieve the goals of ACA. It is this author’s opinion that we really don’t have a choice: We MUST implement EHRs that are more functional and smarter than hand-written paper charts have been. Although this document focuses on what has happened in the USA, the same issues relate to multiple nations [12-17]. Several EHR system implementations and assessments conducted outside the USA have been impressive (I have not forgotten that the word “International” is in IMIA’s name).
II Information Sources Used for the Prediction of the Future
To project future developments, I collected information from Web sites, books, more than 250 PubMed bibliographic search results, and I utilized my own knowledge, acquired from living through the development of EHRs. The Shortliffe book, “Medical and Biomedical Informatics”, first published in 1990, contained in its multiple editions previous future projections [18-22]. Disappointingly, many of the early predictions did not materialize. Several projections, including one of my futuristic ideas - a Medical Information Bus (MIB) to attach devices such as bedside monitors, IV pumps, ventilators, and other devices to an EHR, went through a standardization process (ISO/IEEE 11073) but were never really implemented. The most recent Shortliffe/Cimino revision published in 2014 has a chapter entitled “The Future of Informatics in Biomedicine” which outlines five Grand Challenges [21]:
Provide comprehensive and dynamic information resources for research, education, and healthcare delivery.
Have decision support systems be fully evidence-based personalized to the individual patient.
Be able to learn from every experiment and healthcare decision.
Have a ubiquitous access to information.
Establish policies that ensure that a global computing infrastructure is accessible, secure, reliable, effective, and sustainable.
These Grand Challenges provide a broad and excellent vision for the future. However, in my opinion, those dreams are not likely to be fully fulfilled in the next twenty-five years.
In addition to predictions of the future from books, there are several future-looking recommendations and strategies for improving clinical information systems. For example, in 2003, leaders of the American Medical Informatics Association (AMIA) outlined “Ten Commandments” for implementing effective clinical decision support systems [23]. In 2011, Stafford, in a clinical journal, suggested that clinical decision support might be the missing link in providing better preventive care [24]. Other clinical informatics experts have listed “ten key considerations for successful implementation of health information systems” [25]. In IMIA’s 2015 Yearbook, de Assis Moura recommended strategies for embracing eHealth [26]. Dixon-Woods et al. suggested that there are problems and promises with innovation in healthcare [27]. These authors identified three known paradoxes of innovation in healthcare:
Some innovations infuse rapidly, yet are of unproven or limited value, or pose risks, while other innovations that could potentially deliver benefits to patients remain slow to achieve uptake;
Participatory, cooperative approaches may be the best way of achieving sustainable, positive innovation, yet relying solely on such approaches may disrupt positive innovation;
Improvement clearly depends on change, but change always generates new challenges.
They summarized by stating that quality improvement efforts need to reconsider the role of innovation. Most elements in any medical system will be discarded and replaced. Many of these paradoxes are being played-out in the HITECH implementation of EHRs in the USA today.
Friedman and colleagues are enthusiastic about having health information systems become “learning health systems” [28]. Precision Medicine is an emerging approach to disease treatment and prevention that takes into account individual genes variability, the environment, and the lifestyle. The National Institutes of Health (NIH) Directors Office recently introduced the Precision Medicine initiative [29]. An esteemed informatics colleagues, Zak Kohane, has listed the ten steps needed to achieve success with Precision Medicine [30].
Bob Greenes in the 4th edition of his book entitled “Healthcare Information Management Systems”, provided a predictive chapter, “Health Information Systems 2025 – a ten year projection” [31]. Several other authors in the same text provided informative material related to the future challenges of implementing EHR systems [32-35]. Greenes suggests that there are eleven disruptive forces which will drive Health IT transformation as noted in the Table 1 below:
Table 1.
# | New & Coming Disruptive Transformations | Science | Technology | Policy | Standards | Social & Cultural |
---|---|---|---|---|---|---|
1 | Precision Medicine | X | ||||
2 | Biosensors | X | ||||
3 | Patient Engagement | X | ||||
4 | Big Data | X | X | |||
5 | Pay for Value | X | X | X | ||
6 | Wellness & Prevention | X | X | |||
7 | Meaningful Use | X | X | |||
8 | Usability | X | X | |||
9 | Rise of an App Culture | X | X | |||
10 | Interoperability | X | X | X | ||
11 | Augmented Guidance | X | X | X |
I also surveyed other historical and futuristic literature [36-46]. This information provided a historical perspective from the 1950’s as well as several projections about the future. The materials that I surveyed showed how the development of clinical information systems occurred very collaboratively. What had already been accomplished is impressive; the future will be complex and somewhat unpredictable. As McCray et al. stated in a 2011 paper [43] “this new discipline continues to stress the basic methodology and fundamentals of organizing, representing, and analyzing data, information[,] and knowledge in biomedicine and healthcare.” In 2011, Atkins and Cullen [36], provided a USA Veterans Affairs Hospitals viewpoint, and listed eight broad trends:
Healthcare is more connected;
Patients have more control of their own records;
Patient data is increasingly available;
EHRs are promoting health behavioral changes;
More customized data presentations are available;
Health data is becoming more standardized;
A better understanding of population-based medical care exists;
The ability to query large databases using “Big Data” techniques is improving.
In 2011, Mitchell et al. reviewed fifty years of informatics research on computerized clinical decision support and asked the question: “What’s Next?” [44]. Their report projected that developing next-generation decision support systems would depend on:
Learning from history;
Establishing uniform vocabularies;
Integrating databases;
Making decisions fit the clinical context;
Developing decision support systems that would be easily installed and maintained in complex interdisciplinary settings.
The Robert Wood Johnson Foundation recently published a report “Health Information Technology in the United States, 2015: Transition to a Post-HITECH World” [45], which provides an excellent overview of what has happened over the past decade, and discusses the challenges of meeting “Meaningful Use” requirements introduced by the HITECH Act in the USA. In addition the RAND Corporation has provided an updated systematic review on how meaningful use is progressing [47].
III Historical Background – The Help System Development
Until about 1990, multiple talented academic “informatics” investigators developed homegrown, typically “stand-alone”, clinical computer applications. During this same time, other hospitals and health care providers were developing systems to optimize billing and reimbursement [41]. While the US was primarily focused on hospital-based EHRs, the remainder of the world was making great strides on EHRs for use in primary care [48]. The academic informatics community built systems that integrated data from intensive care units, clinical laboratories, and a variety of other clinical data sources. The HELP system at LDS Hospital in Salt Lake City, Utah, USA, was such a system. Since the HELP system is representative and I have knowledge of its history, I will use it as a model for what the field in general learned and pursued from 1990 thru 2015.
With limited computing capability by today’s standards, the HELP development group gathered data from analog devices (bedside monitors), from nurse charting records, from clinical laboratory data, and other data sources. The HELP system integrated these data into a structured patient record that could be used for computerized decision support. Homer Warner had pioneered the idea of computerized decision support and believed it would be helpful in medical care [49, 50]. With such computing tools available, many developers created “clinical applications”. Gil Kuperman, while an NIH-sponsored Fellow at LDS Hospital, documented the early development of the HELP system and categorized the HELP applications in a book [51]. Physicians and nurses, who used the HELP system, had positive attitudes about its capabilities [52]. The HELP system group produced many innovative clinical applications, e.g., for optimizing antibiotics use [53, 54], detecting and preventing adverse drug events [55, 56], and computerized methods to wean patients from mechanical ventilators [57, 58]. The HELP system was transferred to the highly reliable Tandem computers hardware to improve its availability and provide greater computing capabilities [59, 60]. Around 2002, Paul Clayton and colleagues updated and modernized the HELP system to a system designated HELP II [61]. LDS Hospital is part of Intermountain Healthcare that currently operates 22 hospitals and 185 clinics in Utah and Idaho, and as of the summer 2015, still uses the HELP II system. In 2005, Intermountain Healthcare’s administration decided that it was not in a position to continue supporting the development of its own EHR and instead chose a commercial vendor to install and implement further development of a new Electronic Health Record (EHR). In 2015, more than a decade later, the first vendor defaulted on the EHR installation. Intermountain Healthcare then selected and is currently installing another vendor EHR.
Most healthcare systems in the USA are in the process of converting to commercially available, vendor-supplied, EHR systems. Moving from a homegrown system to a commercial vendor developed system has been and continues to be challenging for clinical informaticians everywhere. Over the past 10 years as I have watched from afar as the HELP system was in the process of being replaced, several concerns have been expressed. A few examples will illustrate:
We will lose current clinical decision support capability so richly developed in HELP such as the antibiotic assistant and adverse drug event detection.
The new system is slower and not optimized to what our clinical users expect.
We will loose control of our ability to develop the clinical applications we want versus what some vendor chooses to develop.
Being able to customize the system to the needs of OUR clinical users will be much more difficult and take much longer.
However, such a change is almost universal and will be one that clinical informaticians must embrace and optimize. Whereas the HELP system was designed, implemented, and optimized for use by medical technologists, nurses, and physicians at Intermountain Healthcare, the systems developed by commercial vendors have been designed and implemented to supply clinical computing capability to a large and diverse population of healthcare. As a consequence, local customization will likely be less frequent and more difficult with vendor-supplied systems. Columbia University in New York City has gone through the transition from using a homegrown to a commercial vendor-supplied system. Nevertheless, it has continued to provide important and excellent clinical informatics research and evaluation [62, 63]. Commercial vendors, especially in the US, have a huge task in supplying systems to meet the ever-growing complexity of modern healthcare systems.
As the hospital EHR market in the US has matured, a once-crowded field of vendors has dramatically narrowed. As of March 2015, 179 vendors supplied ONC-certified EHR products to over 4,500 hospitals participating in the Meaningful Use incentive program. Cerner, MEDITECH, and Epic Systems comprise nearly 60% of the market share [64]. Koppel and Lehmann have explored the implications of Epic emerging as a future “monoculture EHR” system [65]. Scott Evans’ chapter in this Yearbook provides a more detailed background about EHRs [66].
IV Current State of Clinical Information Systems Performance
The following information was extracted from the latest Robert Wood Johnson Report on Health Information Technology (HIT) in the United States [45]. From 2008 to 2014, the percentage of hospitals with EHRs has increased from about 9% to 75%. Similarly, adoption rate of EHRs by office-based physicians over the same time interval went from 42% to 82%. The Center for Medicare & Medicaid Services reported that 4,379 or 75% of hospitals met Stage 1 MU criteria. In 2014, only 1,826 or 9% of hospitals successfully attested to meeting Stage 2 MU criteria (however, only about 38% of hospitals were registered for the MU incentive program). Thus, it is clear that many hospitals at Stage 1 MU are facing challenges in achieving Stage 2 MU requirements. Similar to hospitals, physicians in their clinics are having difficulties implementing the more stringent Stage 2 MU criteria with only about a 20% success rate. Being able to exchange data across multiple healthcare environments using Health Information Exchanges (HIEs) is the major problem facilities are having in meeting Stage 2 MU criteria [45].
Today, we can either take a positive look at our progress and consider the “glass half full” or be negative, and consider the “glass half empty”. It is incredible that, in the USA, we have had such a dramatic increase of EHR implementation and use in both the outpatient/clinic settings as well as in hospitals. After reviewing literature about the success of recent EHR installations, I have some concerns. In 2005, Koppel and colleagues published an article entitled “Role of computerized physician order entry systems in facilitating medication errors” [67]. Following that work, in 2009, J.S. Ash and colleagues evaluated and found there were several “unintended consequences of computerized order entry” [68]. In 2013, Jim Cimino published an important editorial entitled “Improving the electronic record – are clinicians getting what they wished for?” [69]. In an important editorial viewpoint article entitled “The Tragedy of the electronic health record”, Byyny [70] was supportive and concerned about an earlier article authored by Ober and Applegate [71] asking the question, “The electronic health record. Are we tools of our tools?” Both articles were published in The Pharos, the journal of the honorary medical society’s Alpha Omega Alpha. The editorial found that in part the “programmers”, who do not understand medicine, were responsible for the poverty of EHRs.
In 2006, both AMIA and the American Health Informatics Management Association (AHIMA) recognized the need for a larger and better-trained workforce in medical informatics. They developed a plan to educate 10,000 informaticians by 2010 [72], a program still on-going in many US academic centers despite the past deadline. In addition, AMIA worked at establishing the Board Certification of physicians in Clinical Informatics [73, 74]. So, we hopefully expect that nearly 1,000 physicians will be Board Certified in Clinical Informatics in the USA [75]. Nevertheless, that certification does not guarantee that all of the diplomats can capably develop and implement optimized clinical information systems.
A recent editorial by Tom Payne entitled “Electronic health records and patient safety: Should we be discouraged?” [76] illustrates a similar sensitivity about the value of EHRs at improving patient safety. The issues are complex and likely related to user age. For example, many older clinicians would like to keep things as they were, while younger clinicians want even more technology. However, it is clear to this author that no one wants to go back to “paper records” with all their flaws.
Still, numerous authors have questioned current Health Information Technology policy with publications such as, ”Re-examining Health IT policy: What will it take to derive value from our investment?” [77]. Practicing physicians are now speaking out on the topic of the problems with current EHRs. A popular new provocative book, “The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age” by Robert Wachter, MD, is certainly not complimentary [78]. Further, an October 2015 Editorial perspective in The New England Journal of Medicine by Lisa Rosenbaum, MD, entitled “Transitional Chaos or Enduring Harm? The EHR and the Disruption of Medicine”, is sobering and insightful [79]. Clearly, we, as clinical informaticians can and must do better at designing, building, installing, and evaluating EHRs before they are installed in the clinical setting. The vendors now doing the lion’s share of EHR development must understand how EHRs can be powerful tools for not only the clinicians providing the care, but also for the patient and the healthcare field at large.
Berwick and colleagues [80] recommended three goals to achieve healthcare reform in the USA:
Improving the quality, safety, and experience of healthcare;
Enhancing population health;
Reducing the per capita costs of healthcare.
Sheikh and colleagues [81] suggested that we, in the USA, must “leverage health information systems” to achieve the above noted three goals of healthcare reform. While others take a rather pessimistic view of the future of clinical informatics, these authors provide a more optimistic view that HITECH Act has stimulated an unprecedented interest in health information and communications technology. They claim that notwithstanding the known problems with usability and interoperability, and the persistence of fee-for-service paradigm, major policy changes will be required to achieve the needed leverage improvements. Recently, in response to the multitude of problems noted above, a committee from AMIA proposed strategies for the future direction of EHR development between now and 2020 [82]. I suspect the Committee chose 2020 because human vision 2020 is optimal. They identified problems with current EHRs and made ten important recommendations that span the five broad areas noted below (Rec # = Recommendation Number):
-
SIMPLIFY AND SPEED DOCUMENTATION
Rec #1: Decrease the data entry burden for the clinician,
Rec #2: Separate data entry from data reporting,
Rec #3: EHRs should enable systematic learning and research at the point of care.
-
REFOCUS REGULATION
Rec #4: Regulation should focus on:- Clarifying and simplifying certification procedures and Meaningful Use regulations,
- Improve data exchange and interoperability,
- Reduce the need to re-enter data,
- Prioritize patient outcomes over new functional measurements,
Rec #5: Reimbursement regulations should be changed to support novel and innovative EHR improvements.
-
INCREASE TRANSPARENCY OF AND STREAMLINE CERTIFICATION
Rec #6: Improve usability of EHRs and improve patient care quality and safety,
Rec #7: Encourage innovation from healthcare organizations, providers, and vendors.
-
FOSTER INNOVATION
Rec #8: EHR vendors should use standards-based Application Program Interfaces (APIs).
-
THE EHR IN 2020 MUST SUPPORT PERSON-CENTERED CARE DELIVERY
Rec #9: Promote integration of EHRs into the full social context of care,
Rec #10: Design interfaces so that they support and build upon how people think (i.e., cognitive-supported design).
In 2014 in the USA, the ONC issued a “Connecting Health Care for a Nation: A 10-Year Vision to Achieve an Interoperable Health IT Infrastructure” plan [83]. The three-year agenda will be to, Send, Receive, Find, and Use health information to improve healthcare. The next six-year agenda will be to use information to improve healthcare quality and lower care costs. The ten-year agenda will develop a learning health system.
V Opportunities and Challenges for the Next 25 Years
The future is bright for clinical informatics; a field that I predict will still be active for at least twenty-five more years. A few of those opportunities and my projections for the future are noted below:
V.1 Clinical Informatics Provides Tools to Help Minimize Adverse Care Events
Efforts to improve the quality of patient care and patient safety have had high priority for decades. Stakeholders have long viewed the use of clinical computing, decision support, and other computer-related tools as critical components of the overall solution to providing higher quality and safer healthcare.
Just fifteen years ago, in 2000, the Committee on the Quality of Health Care in America from the Institute of Medicine (IOM) published a bold and important report about medical errors “To Err is Human: Building a Safer Health System” [84] estimating that up to 98,000 Americans may die each year as a result of medical errors. The following year, the same committee published “Crossing the Quality Chasm: A new Health System for the 21st Century” [85] which provided strategic directions about how to improve health care. The large and alarming number of deaths suggested by the “To Err is Human” report elicited controversy, some suggested that it was exaggerated [86] while others disagreed with that opinion [87]. One might ask: “How are we doing by now?” In 2014, James [88] updated the estimates about deaths caused by medical errors and claims that “the number of premature patient deaths associated with preventable harm could be as many as 400,000 per year”, more than four times the original estimate. Shabot stated [89], “400,000 preventable US hospital deaths per year are equivalent to six fully loaded Boeing 737 aircrafts crashing every day, with a total loss of life” – a sobering statement.
A subsequent 2012 IOM report “Health IT and Patient Safety: Building Safer Systems for Better Care” provided direct recommendations and strategies for improving Health IT [90]. In December 2015, the National Academy of Medicine (NAM) [formerly Institute of Medicine (IOM)] observed the 15th Anniversary of the publication of their landmark reports: “To Err is Human: Building a Safer Health System” and “Crossing the Quality Chasm: A New Health System for the 21st Century”. They dedicated their Richard & Hinda Rosenthal Symposium to celebrate patient safety improvement accomplishments and entitled it: “Protecting Patients: Advances and Future Directions in Patient Safety”. They are now sharing the Rosenthal Symposium with informative archived videos, slides, photographs, and other resources on their Web site [91].
In 2011 the Institute for Healthcare Improvement tested a “Global Trigger Tool” to identify adverse events [92]. They found that adverse events occurred in one-third of hospital admissions and that the “Global Trigger Tool” found at least ten times as many confirmed adverse events as found by the USA Agency for Health Research and Quality (AHRQ)’s “Quality’s Patient Safety Indicators” method – a voluntary clinician problem reporting project.
Three important recent developments give reasons for encouragement in how well patient safety efforts are progressing.
The first one is a malpractice claims report by Graber et al. [93], which reviewed Health IT cases from 2012 and 2013. They found a total of only 248 cases (<1%) involving Health IT over that two-year period. Ambulatory care accounted for 146 cases (59%). Cases were most typically filed as a result of an error involving medications (31%), diagnosis (28%), or a complication of treatment (31%). More than 80% of cases involved only moderate or severe harm. Findings from this study provide important knowledge about how to reduce the risk of harm in the future. The ONC has announced patient safety goals for its planned Federal Health Information Technology Center [94].
Second, Banger and Graber evaluated in 2015 four separate but linked systematic reviews of Health Information Systems to determine if such systems can help improve health care quality and safety [95]. Between 2006 and 2014 there were four publications [47, 96-98] that used similar evaluation methodologies to examine evidence of improvement. Based on their review they determined that (i) Health Information Systems are working, (ii) the number of health care organizations perceived as leaders is growing, and (iii) an affirmative answer can be given to the question “Can health IT improve the quality and safety of health care.”
Third, René Amalberti, a French physician, considered a world expert on airline and transportation safety [99, 100], has recently taken a more intense interest in healthcare. In 2011, Amalberti et al. provided a commentary entitled “Adverse events in medicine: Easy to count, complicated to understand, and complex to prevent” [101] that provides an informative overview of the tasks at hand for improving patient safety. Very recently, Vincent and Amalberti have observed that “safety in healthcare is a constantly moving target” [102]. They made important observations in their editorial related to the 2015 Baines et al. publication [103], involving sequential observations of safety in healthcare in the Netherlands [104]. Healthcare differs from almost all other safety-critical industries. They observed that in the past ten years, healthcare has found more types of harm that are now categorized as preventable, and thus the perimeter of patient safety has expanded. As a consequence, while major improvements in patient care have been made in the past ten years, assessing major progress in prevention would seem to have failed. Providing high quality healthcare can improve patient safety. The safe practice of medicine is certainty not like the transportation industry. Tasks like improving the safety of commercial airline flights has been characterized as the “paradox of almost totally safe transportation systems” [99]. Vincent and Amalberti have recently published a new, informative and provocative book entitled “Safer Healthcare: Strategies for the Real World” [105]. They expand the base of patient safety from where it is currently being investigated, in hospitals, to primary care and home care. The needs and challenges of Vincent and Amalberti’s much broader view of patient safety will require innovation and drive major policy changes.
The methods for providing high quality healthcare and improving safety are complex for many reasons. The “patient” can have multiple complex medical problems that require diagnosis, treatment, and recovery. The recent publication by the NAM outlines the problems with patient diagnosis [106]. In addition, the treatment strategies of the medical profession are not completely interchangeable with each patient’s condition and are not performed without error [99, 100, 107-115]. So where do we go from here? We must move forward, applying what we now know, and learn new things along the way. Learning healthcare systems propose to address this topic [28]. The field must develop and implement EHRs to support best-known care while minimizing errors. As we develop knowledge about how to best treat patients, that knowledge will continuously change and must be integrated into computerized decision support systems that are optimally interfaced to clinician caregivers. WHAT and HOW such processes occur is still not well understood; nor have such processes been optimized. As a consequence, gathering the knowledge, keeping it current, interfacing it with the caregiver, and optimizing patient care for the multitude of medical diagnoses and treatments will take decades. So a twenty-five year future perspective will only provide methods for optimizing care for a small percentage of the care that must be provided – perhaps 10%.
“What practices will most improve safety?” is a longstanding concern [116]. Clearly, ICT can help [117], as it has been demonstrated at the author’s institution [53, 54, 56, 118]. Clancy and Berwick, in a 2011 editorial, noted that to enhance the science of quality and safety improvement in medical care, we need to know more about the effectiveness of HOW and WHY actions taken in improving safety in our complex healthcare system [119]. Shekelle, et al. in a 2011 RAND Corporation publication, sponsored by AHRQ, lamented that despite a decade’s worth of effort since “To err is Human” was published, patient safety has only improved slowly [120]. They provide excellent recommendations for improving the science of patient safety investigation.
With the increased pressure to improve the quality of healthcare and patient safety, there have been demands from various agencies and institutions to “measure quality” [121]. One of the recommendations is to “measure what matters”. This begs the issue of what matters and to whom? As a consequence, we need to harmonize quality and safety measures and reporting. In fact, “much of healthcare remains poorly measured or unmeasured” and the EHR should become the source of these measures over time [122]. We could conclude that we are not gathering the proper data or not using the data we have to be able to prevent and minimize errors made in medicine. To help with these issues, the NAM published in 2015 “Vital Signs: Core Metrics for Health and Health Care Progress” [123]. The document is filled with excellent recommendations about what data needs to be collected from the health care system, for what purpose, and how the data should be used.
Liyanage and the IMIA Primary Health Care Informatics Working Group were concerned about whether informatics enabled or inhibited higher quality of patient care [124]. Their work at building a consensus from a careful literature review, interviews of clinicians, and clinical informaticians was successful on the conceptual ideas, but evaporated when it came to implementation. From those comments made in 2015, I have concluded that implementation of an operational clinical information systems is still an art and will likely always be complex and difficult.
In December 2015, the National Patient Safety Foundation in the USA published a document entitled “Free from Harm: Accelerating Patient Safety Improvement Fifteen Years after To Err is Human” [125]. Their document contained eight broad and important recommendations. To my surprise and disappointment, the eighth recommendation was: “Ensure that technology is safe and optimized to improve patient care”. My disappointment followed from my clinical experience using the innovative and powerful HELP decision support system. Patient safety and care improvement were key drivers for implementation of the HELP system. Very careful processes were followed with each new software installation made. A hospital wide “Software Oversight” committee actively worked with clinicians, administrators, and informaticians to optimize the HELP system [126]. Their expert panel noted that despite the fact that Health IT, including EHRs, can potentially improve patient safety, there is also the potential for harm. Their panel suggested four tactics to minimize harm: (i) Establish mechanisms for vendors and users to be transparent about Health IT safety hazards and best practice, (ii) Identify and measure the adverse events and unintended consequences of Health It, and implement best practices for risk mitigation, (iii) Establish expectations for Health IT safety performance, such as routine testing of unsafe orders, and (iv) Design Health IT to facilitate communication and coordination with the patient and family.
We, as informatics professionals, must build computerized healthcare systems that are safer and accountable to the extent that we will no longer need to estimate using crude and disturbing statistics but would measure and understand what the problems are. As a consequence, my major prediction for the future of clinical informatics is that the quality of patient care and improved safety are key elements in our future. I will next present current ideas and concepts being studied in our field and how we think they will impact the future of the healthcare system.
V.2 Clinical Informatics Must Immerse its Innovations into Clinical Practice
Let us consider three recent examples of how clinical informaticians can immerse themselves into the clinical setting to better understand and optimize patient care. Working this way, these three groups of scientists have achieved a better understanding of the gaps in the content and quality of the data collected, even with mature EHR systems. In addition, investigators from Johns Hopkins University have provided an important observation about medical records, whether they are handwritten or EHRs: “Not everything that is measured is important, and not everything important is measured” [35].
V.2.1 Role of Cognition in Understanding Clinical Errors
Patel and colleagues have recently completed in-depth studies of the role cognition plays in the generation and mitigation of clinical errors [127, 128]. These investigators immersed themselves into the clinical setting and made important discoveries. They noted that not only the occurrence of errors but also whether recognition and recovery from the errors occurred is critical in many situations. They noted that the clinical work environment is complex and that detection of what causes errors is difficult to study. However, they noted that error recovery in the clinical situation is important and that the topic is little studied or understood. Their work suggests that “shifting the focus from error intolerance to error recognition and recovery” might help us understand and mitigate errors in medicine. Further, they suggest that the healthcare system must accept what Patel et al. called “error etiquette” – accepting the inevitability of human error, emphasize that learning must occur from errors, and that encouraging vigilance is key. The healthcare system and its members must intervene when errors occur, discuss the problems openly, and avoid excessive criticism of others when such errors do occur.
V.2.2 Prospective Clinical Surveillance
Recently, Wong and colleagues conducted a near real-time quality improvement study on a general nursing ward in a hospital in Canada [129]. Within 48 hours of each identified event, a trained observer went to the specific ward where the event had occurred, gathered information from the patient record, and debriefed the front-line staff. They found that the medical record often contained limited information about the factors that contributed to the adverse event. Further, they found that factors contributing to the adverse event were very heterogeneous and complex. Thomas, in an accompanying editorial, called the method “Prospective Clinical Surveillance” (PCS) and suggested that the technique might be “the future of measuring patient safety” [130]. He noted that EHRs with computerized decision support capabilities would be essential to facilitate event detection, and that developing a safety culture in the institution is crucial.
V.2.3 Study of Adverse Drug Events in the Perioperative Period of Anesthesia
Nandi, Bates, and colleagues have recently studied medication errors and adverse drug events that occurred during the perioperative period of anesthesia [131]. They found a surprisingly large number of errors. About one in twenty (5%) perioperative medication administrations included a medication error and/or an adverse drug event. Their approach involved direct expert observation as well as a chart review of the electronic anesthesia record. They characterized the type and severity of errors. In addition, they evaluated factors that contributed to the errors and suggested potential interventions to minimize the errors noted. They found two essential approaches to preventing errors: technology-based -- point-of-care bar code-assisted documentation, and process-based -- changing the timing of documentation methods. They also determined that counting the number of errors based only on self-reporting or chart review was inadequate. The direct observation methodology enabled them to make recommendations that could potentially prevent the future errors. Further, they noted that some adverse events were not preventable. Their data-gathering methods and intimate involvement in the process of care are models of how future work in clinical informatics and error detection and prevention should be implemented.
V.3 Healthcare Systems Must Evaluate the Effectiveness of their EHR Systems
Deming was a world expert on innovation and development of high-quality products. Clearly, any innovation in medicine must meet his challenge of providing data to evidence its effectiveness (figure 2). High-quality data and evidence are key to improving healthcare. Soumerai and colleagues have provided excellent methods to help us understand “which healthcare effectiveness research we can trust.” They noted that the US investment in Health IT was in part based on a 2005 report published by the influential RAND think tank [132] that was recanted by RAND authors Kellerman and Jones in 2013 [133].
“In God we trust. All others (must) bring data.”
W. Edwards Deming, 1900-1993 Establishing the value and cost/benefit of EHR systems, an important concern for decades, will remain so as EHR system installations and evaluations continue. Several publications provide methods and strategies for the financial evaluation of EHR systems [96, 109, 133-136]. There are multiple other supporting evaluation technologies. Investigators have shown that it takes time for clinical users to adapt to new EHR systems [137]. Still others have assessed how EHR systems can be useful in establishing ties between patients, providers, students, and research investigators [138].
A great need persists for better methods to evaluate the effectiveness of the implementation, functionality, and cost effectiveness of EHRs. Fortunately, much better methods and recommendations are available regarding how to perform those evaluations. Yusof and colleagues have established what is known as Human, Organizational, and Technological fit or HOT-fit [139, 140]. The evaluation methodology provides a framework for assessing the HOT-fit of EHR systems in the clinical setting. The HOT-fit evaluation methodology has been used with success to evaluate the “fit” of a “search engine” in an outpatient setting [141], and to assess an EHR used in an intensive care unit [142]. Adler-Milstein and colleagues have recently conducted studies on EHRs’ benefits and shown that there is a growing value for such systems [137]. Scientists from the RAND think-tank have provided recommendation on how to evaluate EHRs [135], noting that three key principles should be considered: (i) Value includes both cost and benefit, (ii) Value accrues over time, and (iii) Value depends on the stakeholder’s perspective, as illustrated by Yusof’s HOT-fit technique [139, 140].
V.4 Vendors and Healthcare Systems Must Acquire and Maintain the Clinical Knowledge Needed for Decision Support
Recently publications focused on the topic “Evidence-based health informatics: How do we know what we know?” were published [143, 144]. The article by Ammenwerth and the follow-up discussion on the topic by Al-Shorbaji and colleagues illustrate the challenges we face as we build new and more “knowledge driven” clinical information systems. Ammenwerth outlines eight challenges on the way to idealized “evidence-based health informatics”: (i) Assure that knowledge is derived from high quality evaluation studies, (ii) Eliminate publication bias, (iii) Report the quality of evaluation studies, (iv) Identify high quality published evaluation studies, (v) Assure that there are frequent systematic reviews and updates, (vi) Train expert clinical informaticians, (vii) Provide methods to translate evidence into practice, and (viii) Provide post-market surveillance of implemented systems.
In 2007, Shojania and colleagues assessed how quickly medical knowledge produced by systematic reviews becomes out of date [145]. For the 100 systematic reviews they analyzed, median duration of survival free of change was only five and one-half years. Medical knowledge is a moving and hopefully improving target. As a consequence, updating computerized decision-support systems should be a continuous process. Both acquiring the knowledge needed to enable computerized decision support and keeping it up to date will be ongoing challenges and opportunities for clinical informatics community. To enable more rapid and accurate methods for gaining knowledge and using that knowledge with EHR systems, protocols for knowledge gathering and use for clinical decision support should become a key part of medical protocol development.
V.5 Miscellanious Observations
Many observers only evaluate the work of others but fail to provide innovations.
Many observers complain about mistakes that others make, but do not get involved in the process of improving system performance.
Many institutions do not allow/encourage clinical informaticians to have access to or interact with their EHR systems. The list of reasons is long and difficult to substantiate and may include vendor-introduced obstacles. It constitutes a larger problem than most would realize or admit.
Broad central control or perhaps even government mandates are not likely to optimize clinical information systems. Actionable feedback, acted upon in a timely manner, is critical to survival and evolution of all clinical systems.
Multiple new ideas and strategies have been proposed as noted throughout this manuscript. I do not know which are best, and, as a consequence, I support development and evaluation of all good ideas.
We do not yet know the optimal data that needs to be acquired and used to provide ideal patient care.
We are not likely to return to hand written paper records. Let us improve and optimize computerized patient records.
Like most other developed countries in the world, we are not likely to get large injections of financial support, like HITECH in the USA, again. So we will have to do the work of improving clinical information systems with baseline budgets.
Meaningful Use was a great method for implementing HITECH, but may not be the best way for continuing future development.
Mandating “interoperable” records may not provide the best way to implement computerized records.
Important experiments have been run, but many issues remain to be dealt with. If Kaiser-Permanente, Mayo Clinic, or Intermountain Healthcare wished to optimize their individual clinical system for their patient care needs, that strategy seems more reasonable than trying to force all of them to use a still undefined system that would permit easy and reliable exchange of clinical data. Developing optimal patient care protocols in the future should also include work on what data elements are needed, at what frequency, and at a minimum flowcharts of how the protocols could be placed into clinical practice.
VI Managing Expectations of Clinical Information Systems
One of the strengths and also a weakness of clinical informatics is the eternal optimism of the professionals. Having been involved in the field for several decades, I can relate to this weakness. During the process of preparing this manuscript, I became acquainted with the Gartner Hype Cycle, shown as Figure 1 below [146], which illustrates the Gartner’s project for the Emerging Technologies for Computing Innovations for 2015. The five time phases are noted: 1. Innovative Trigger; 2. Peak of Inflated Expectations; 3. Trough of Disillusionment; 4. Slope of Enlightenment; and 5. Plateau of Productivity. It is clear that today Clinical Information Systems are in the “Trough of Disillusionment” phase, and that the scale of “Disillusionment” should perhaps show a negative scale because of the earlier reported problems. Although the Gartner-Hype Cycle has some weaknesses such as no real-time scale, it gives developers of clinical information systems a sense of the complexity of their work. Harris and Schneider used the diagram to make what they termed a “realistic assessment” of the application of Big Data to healthcare [147]. They noted that there is “increasing skepticism among oncology researchers that access to large sets of genetic data [to] help find a cure for cancer”.
A clinical computing example will help illustrate how time expectations exceed reality.
Major problems still exist in gathering timely and accurate data. ICU bedside monitors generate megabytes of data per day - do we need it all? Probably not, but I will leave it to others to prove or disprove related concepts. I find them unlikely to provide the answers we need. We have seen in our own situation a summarization of data to be most effective. Recently, the Mayo Clinic generated an ICU rounds report that picked the most important data [148]. If we have all the raw data will we be able to provide better care by just processing it for knowledge? I doubt it.
I watched my colleagues develop ventilator-weaning protocols more than twenty-five years ago [57, 58]. They worked and improved patient care. However, even twenty-five years later they are not in common use. Why? (i) Good data needed to drive the protocols is very hard to acquire and process, (ii) No consensus exists on what is the best way to wean patients, (iii) The infrastructure to gather, process, and present the data is still not available 25 years later.
The technology to gather data automatically from devices is still woefully lacking. We developed strategies for gathering data from bedside monitors more than forty years ago. We gathered Heart Rate (HR) and Arterial Blood Pressures (SP/DP MP). Yet today most commercial EHRs systems that gather the electronic data from monitors do not process and automatically store the raw data. Instead they put the data into a file, which care providers must then access later and then pick what they think was representative data. This process is time-consuming, causes delays in care, and is likely to cause erroneous summary data entry. How does the nurse know what was happening one hour ago and make a correct judgment as to whether a fall in a parameter was real or an artifact?
When I read expectations of gathering data from multiple devices attached to an ankle or a wrist and having that data processed into meaningful and useful information, I have serious concerns. If we were gathering blood sugar levels for a diabetic patient, I could understand. If we were gathering cardiac signals like automatic defibrillators, I would understand. Just because we can measure some physiological signal continuously does not mean that it has value and that we must collect it.
It took us ten years to have the knowledge about how to gather data from a computerized ventilator [149]. However, even though we now know how, such data collection methods are almost never used outside Intermountain Healthcare.
VII Conclusion – Optimism but Realism for Clinical Information Systems
Although I am excited by Greenes’ optimistic predictions presented in his Chapter #33 entitled “Health Information Systems 2025” in “Healthcare Information Management Systems: Cases, Strategies, and Solutions. 4th ed.” [31], I am also realistic about how slowly medical technologies become integrated in day-to-day care processes, having been involved in implementing the HELP system and having reviewed the recent National Academy of Medicine’s Rosenthal Symposium “Protecting Patients: Advances and Future Directions of Patient Safety” [91]. I therefore present my concerns about Greenes’ New and Coming Disruptive Transformations and present my concerns about whether they will be accomplished by his estimated date of 2025:
Precision Medicine – I am excited about the concepts and I have seen treatment miracles in a family member with cancer. The process is guided by genomic knowledge about a person’s condition. I am very supportive of research and development of the ideas and concepts of Precision Medicine, but I think the timeframe for routine “day-to-day” operational Precision Medicine systems are well beyond ten years and may take more than twenty-five years.
Biosensors – My undergraduate degree was in Electrical Engineering, and my first few years in informatics were as a bioengineer. So, I am enthusiastic about development of new and better biosensors. However, I have already seen a very slow adoption of monitoring and sensor technologies into EHRs. Today, almost all EHRs use a person to manually select data from monitoring devices to enter the data into the EHR. That is more than forty years after the LDS Hospital group was gathering the data automatically. The primary issue of what is “signal” and what is “noise”, is still a very difficult problem that has not been solved. More biosensors do not automatically equate to more and better quality patient information.
Patient Engagement – I frequently use my ability to review my own patient data from my EHR. I find the process very helpful and promising. However, in my broad family, I am one of the few who takes the opportunity to make such access. Further, I find it troubling that in the past two years, I have had two flu shots and a Shingles shot at three different local pharmacies for convenience. None of that data was automatically uploaded into my EHR at Intermountain Health Care – my primary health service provider. Such interconnectedness will likely not occur for at least a decade.
Big Data – This is an important and exciting area of development. However, I have seen little evidence that shows dramatic changes in diagnosis or treatment as a result of Big Data. We must learn and apply the technology before we sell it.
Pay for Value – This concept has high value and high expectations. The issues are primarily related to politics and policy, and are slow to resolve and change. Many other economically developed countries are further along with this process than the USA and provide good models to follow.
Wellness and Prevention – These conceptual ideas are very appealing and clearly important. However, in the USA, medical problems such as obesity have continued to increase. Perhaps part of the reason is that we now have no fault health insurance and so it turns out there is little motivation to fix the problem. If we can eliminate obesity in twenty-five years it will be a miracle, but we must continue to try.
Meaningful Use – The conceptual idea of Meaningful Use was well intended. However, it is my opinion that deciding what the criteria are for “optimal Meaningful Use” could be difficult and subject to misuse and fraud.
Usability – The usability of EHR systems has been widely discussed and debated. Based on material presented earlier in this manuscript, we must do better. If system developers and implementers cannot solve usability issues in the next fifteen years, they will no longer be in business.
Rise of an App Culture – Based on how my grandchildren use smart phones and tablets, this projection is a winner. These same grandchildren and their peers will eventually become patients, and some even healthcare workers. They will demand such applications and clinical systems. In addition, AMIA has recommended such developments by 2020 [82].
Interoperability – The issue of interoperability has been a major part of Meaningful Use and has been difficult to resolve. Hopefully, by 2040, we will be able to accomplish some meaningful level of interoperability. In my opinion, the ONC’s plan for the next ten years is optimistic and very broad [83]. Deciding what we should do and how will be very challenging.
Augmented Guidance – Greenes defines this as “the ability to enhance all of our actions by situation-aware knowledge and advice”. Such a conceptual strategy is exciting to consider. However, with the current difficulties of “alert fatigue” and “usability” this area will be one that is important but difficult. We will not have widely available operating systems with this capability in the next twenty-five years.
Acknowledgements
Thanks to the two anonymous reviewers for their careful edits, suggestions and recommendations. They made this manuscript much better.
References
- 1.Tetlock PE, Gardner D. Superforecasting The Art and Science of Prediction. New York, NY: Crown Publishers; 2015. [Google Scholar]
- 2.HealthConnect, Evaluation in Australia 2009 [cited 2015 November 21]. Available from: http://www.health.gov.au/internet/main/publishing.nsf/Content/pacd-ehealth-Healthconnectevaluation. [Google Scholar]
- 3.France FR. eHealth in Belgium, a new “secure” federal network: role of patients, health professions and social security services. Int J Med Inform 2011;80(2):e12-6. [DOI] [PubMed] [Google Scholar]
- 4.EHRS Blueprint: An interoperable EHR Framework - Summary Version 2 Canada 2006. [Cited 2015 November 21]. Available From: Https://Www.Infoway-Inforoute.Ca/En/Component/Edocman/Resources/Technical-Documents/390-Ehrs-Blue-Print-V2-Summary. [Google Scholar]
- 5.Protti D, Johansen I. Widespread adoption of information technology in primary care physician offices in Denmark: a case study. Issue Brief (Commonw Fund) 2010;80:1-14. [PubMed] [Google Scholar]
- 6.Robertson A, Bates DW, Sheikh A. The rise and fall of England’s National Programme for IT. J R Soc Med 2011;104(11):434-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Blumenthal D. Launching HITECH. N Engl J Med 2010;362(5):382-5. [DOI] [PubMed] [Google Scholar]
- 8.Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med 2010;363(6):501-4. [DOI] [PubMed] [Google Scholar]
- 9.Classen DC, Bates DW. Finding the meaning in meaningful use. N Engl J Med 2011;365(9):855-8. [DOI] [PubMed] [Google Scholar]
- 10.CDC. Meaningful Use - Introduction 2015 [cited 2015 December 20]. Available from: http://cdc.gov/ehrmeaningfuluse/introduction.html. [Google Scholar]
- 11.Government Printing Office (GPO). 2010. Public Law 11-148: The Patient Protection and Affordable Care Act. Washington: U.S. Governmnet Printing Office Washington, DC: U.S. Government Printing Office; 2010 [cited 2015 October 1]. Available from: http://www.gpo.gov/fdsys/pkg/PLAW-111publ148/pdf/PLAW-111publ148.pdf. [Google Scholar]
- 12.Aarts J, Koppel R. Implementation of computerized physician order entry in seven countries. Health Aff 2009;28(2):404-14. [DOI] [PubMed] [Google Scholar]
- 13.Adler-Milstein J, Ronchi E, Cohen GR, Winn LA, Jha AK. Benchmarking health IT among OECD countries: better data for better policy. J Am Med Inform Assoc 2014;21(1):111-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Adler-Milstein J, Sarma N, Woskie LR, Jha AK. A comparison of how four countries use health IT to support care for people with chronic conditions. Health Affairs 2014;33(9):1559-66. [DOI] [PubMed] [Google Scholar]
- 15.Borycki E, Cummings E, Dexheimer JW, Gong Y, Kennebeck S, Kushniruk A, et al. Patient-Centred Coordinated Care in Times of Emerging Diseases and Epidemics. Contribution of the IMIA Working Group on Patient Safety. Yearb Med Inform 2015;10(1):207-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Geissbuhler A, Kimura M, Kulikowski CA, Murray PJ, Ohno-Machado L, Park HA, et al. Confluence of disciplines in health informatics: an international perspective. Methods Inf Med 2011;50(6):545-55. [DOI] [PubMed] [Google Scholar]
- 17.Hasman A, Ammenwerth E, Dickhaus H, Knaup P, Lovis C, Mantas J, et al. Biomedical informatics-a confluence of disciplines? Methods Inf Med 2011;50(6):508-24. [DOI] [PubMed] [Google Scholar]
- 18.Fagan LM, Perreault LE. The Future of Computer Applications in Medicine. Shortliffe EH, Perreault LE, Wiederhold G, Fagan LM, editors. Medical Informatics: Computer Applications in Health Care. 1st ed. Reading, MA: Addison-Wesley Publishing Company; 1990. p. 620-44. [Google Scholar]
- 19.Fagan LM, Shorliffe EH. The Future of Computer Applications in Health Care. Shorliffe EH, Perreault LE, Wiederhold G, Fagan LM, editors. Medical Informatics: Computer Applications in Health Care and Biomedicine. Computers in Health Care. 2nd ed. New York, NY: Springer-Verlag; 2001. p. 697-712. [Google Scholar]
- 20.Fagan LM, Shorliffe EH. The Future of Computer Applications in Biomedicine. Shortliffe EH, Cimino JJ, editors. Biomedical Informatics. Health Informatics Series. 3rd ed. New York, NY: Springer-Verlag; 2006. p. 829-47. [Google Scholar]
- 21.Frisse ME, Florance V, Mandel KD, Kohane IS. The Future of Informatics in Biomedicine. Shorliffe EH, Cimino JJ, editors. Biomedical Informatics: Computer Applications in Health Care and Biomedicine. 4th ed. New York, NY: Springer-Verlag; 2014. p. 797-811. [Google Scholar]
- 22.Rudin RS, Tang PC, Bates DW. Health Information Technology Policy. Shortliffe EH, Cimino JJ, editors. Biomedical Informatics: Computer Applications in Health Care and Biomedicine. 4th ed. New York, NY: Springer-Verlag; 2014. p. 781-95. [Google Scholar]
- 23.Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003;10(6):523-30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Stafford RS, Romano MJ. Is clinical decision support the missing link in prevention? Comment on “Shared electronic vascular risk decision support in primary care”. Arch Intern Med 2011;171(19):1745-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc 2013;20(e1):e9-e13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.de Assis Moura L., Jr. Embracing Strategies for eHealth. Yearb Med Inform 2015;10(1):1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Dixon-Woods M, Amalberti R, Goodman S, Bergman B, Glasziou P. Problems and promises of innovation: why healthcare needs to rethink its love/hate relationship with the new. BMJ Qual Saf. 2011;20 Suppl 1:i47-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Friedman C, Rubin J, Brown J, Buntin M, Corn M, Etheredge L, et al. Toward a science of learning systems: a research agenda for the high-functioning Learning Health System. J Am Med Inform Assoc 2015;22(1):43-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med 2015;372(9):793-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Kohane IS. Ten things we have to do to achieve precision medicine. Science 2015;349(6243):37-8. [DOI] [PubMed] [Google Scholar]
- 31.Greenes RA. Health Information Systems 2025 Ch 33. Weaver CA, Ball MJ, Kim GR, Kiel JM, editors. Healthcare Information Management Systems: Cases, Strategies, and Solutions. 4th ed. Switzerland: Springer International Publishing AG; 2016. p. 400-44. [Google Scholar]
- 32.Edmonds M, Peddicord D, Frisse ME. The Reasons Why Interoperability is Difficult Ch 7. Weaver CA, Ball MJ, Kim GR, Kiel JM, editors. Healthcare Information Management Systems: Cases, Strategies, and Solutions. 4th ed. Switzerland: Springer International Publishing AG; 2016. [Google Scholar]
- 33.Edmonds M, Peddicord D, Frisse ME. The Evolution of Health Information Technology Policy in the United States Ch 8. Weaver CA, Ball MJ, Kidd MR, Kiel JM, editors. Healthcare Information Management Systems: Cases, Strategies, and Solutions. 4th ed. Switzerland: Springer International Publishing AG; 2016. [Google Scholar]
- 34.Koppel R. Great Promises of Healthcare Information Technology Deliver Less Ch 6. Weaver CA, Ball MJ, Kim GR, Kiel JM, editors. Healthcare Information Management Systems: Cases, Strategies, and Solutions. 4th ed. Switzerland: Springer International Publishing AG; 2016. [Google Scholar]
- 35.Rosen MA, Tran G, Carolan H, Romig M, Dwyer C, Dietz AS, et al. Data Driven Patient Safety and Clinical Information Technology Ch 18. Weaver CA, Ball MJ, Kim GR, Kiel JM, editors. Healthcare Information Management Systems: Cases, Strategies, and Solutions. 4th ed. Switzerland: Springer International Publishing AG; 2016. [Google Scholar]
- 36.Atkins D, Cullen T. The future of health information technology: implications for research. Med Care 2013;51(3 Suppl 1):S1-3. [DOI] [PubMed] [Google Scholar]
- 37.Bates DW. Health Information Technology and Care Coordination: The Next Big Opportunity for Informatics? Yearb Med Inform 2015;10(1). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Bates DW, Evans RS, Murff H, Stetson PD, Pizziferri L, Hripcsak G. Policy and the future of adverse event detection using information technology. J Am Med Inform Assoc 2003;10(2):226-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Aff 2014;33(7):1123-31. [DOI] [PubMed] [Google Scholar]
- 40.Bates DW, Zimlichman E. Finding patients before they crash: the next major opportunity to improve patient safety. BMJ Qual Saf 2015;24(1):1-3. [DOI] [PubMed] [Google Scholar]
- 41.Collen MF. A history of medical informatics in the United States, 1950 to 1990. Indianapolis, IN: American Medical Informatics Association; 1995. [Google Scholar]
- 42.Collen MF, Ball MJ. The History of Medical Informatics in the United States. 2nd ed. New York: Springer; 2015. [Google Scholar]
- 43.McCray AT, Gefeller O, Aronsky D, Leong TY, Sarkar IN, Bergemann D, et al. The birth and evolution of a discipline devoted to information in biomedicine and health care. As reflected in its longest running journal. Methods Inf Med 2011;50(6):491-507. [DOI] [PubMed] [Google Scholar]
- 44.Mitchell JA, Gerdin U, Lindberg DA, Lovis C, Martin-Sanchez FJ, Miller RA, et al. 50 years of informatics research on decision support: what’s next. Methods Inf Med 2011;50(6):525-35. [DOI] [PubMed] [Google Scholar]
- 45.RWJF. Health Information Technology in the United States, 2015: Transition to a Post-HITECH World. 2015. Sept 2015. Report No.: 2015 Annual. [Google Scholar]
- 46.Winter A. The future of medical informatics. Some perspectives of intra- and inter-institutional information systems. Methods Inf Med 2009;48(1):62-5. [PubMed] [Google Scholar]
- 47.Jones SS, Rudin RS, Perry T, Shekelle PG. Health information technology: an updated systematic review with a focus on meaningful use. Ann Intern Med 2014;160(1):48-54. [DOI] [PubMed] [Google Scholar]
- 48.Detmer DE. Information technology for quality health care: a summary of United Kingdom and United States experiences. Qual Health Care 2000;9(3):181-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Warner HR, Pryor TA, Clark JS, Morgan JD. Integration of Computer Support for Institutional Practice: The HELP System. In: Charles Weller MD, editor. Proc of SAMS Meeting 1975; Houston, TX. Miami, FL: Symposia Specialists; 1976. p. 121-33. [Google Scholar]
- 50.Warner HR, Gardner RM, Pryor TA, Day WC, Stauffer WM. A system for on-line computer analysis of data during heart catheterization. Adams FH, Swan HJC, Hall VE, editors. UCLA Forum in Medical Scinecesm Pathophysiology of Congenital Heart Disease. UCLA Forum in Medical Sciences. 10. 1970/01/01 ed. Los Angeles, CA: University of California Press; 1970. p. 409-18. [PubMed] [Google Scholar]
- 51.Kuperman GJ, Gardner RM, Pryor TA. HELP : a dynamic hospital information system. New York, NY: Springer-Verlag; 1991. xxiii, 334 p. p. [Google Scholar]
- 52.Gardner RM, Lundsgaarde HP. Evaluation of user acceptance of a clinical expert system. J Am Med Inform Assoc 1994;1(6):428-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Classen DC, Evans RS, Pestotnik SL, Horn SD, Menlove RL, Burke JP. The timing of prophylactic administration of antibiotics and the risk of surgical-wound infection. N Engl J Med 1992;326(5):281-6. [DOI] [PubMed] [Google Scholar]
- 54.Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, Jr., et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med 1998;338(4):232-8. [DOI] [PubMed] [Google Scholar]
- 55.Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA 1997;277(4):301-6. [PubMed] [Google Scholar]
- 56.Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA 1991;266(20):2847-51. [PubMed] [Google Scholar]
- 57.Henderson S, Crapo RO, Wallace CJ, East TD, Morris AH, Gardner RM. Performance of computerized protocols for the management of arterial oxygenation in an intensive care unit. Int J Clin Monit Comput 1991;8(4):271-80. [DOI] [PubMed] [Google Scholar]
- 58.Sittig DF, Pace NL, Gardner RM, Beck E, Morris AH. Implementation of a computerized patient advice system using the HELP clinical information system. Comput Biomed Res 1989;22(5):474-87. [DOI] [PubMed] [Google Scholar]
- 59.Gardner RM, Pryor TA, Warner HR. The HELP hospital information system: update 1998. Int J Med Inform 1999;54(3):169-82. [DOI] [PubMed] [Google Scholar]
- 60.Warner HR. History of Medical Informatics at Utah. Blum BI, Duncan K, editors. A History of Medical Informatics - ACM Press History Series. Reading, MA: Addison-Wesley; 1990. p. 357-69. [Google Scholar]
- 61.Clayton PD, Narus SP, Huff SM, Pryor TA, Haug PJ, Larkin T, et al. Building a comprehensive clinical information system from components. The approach at Intermountain Health Care. Methods Inf Med 2003;42(1):1-7. [PubMed] [Google Scholar]
- 62.Mamykina L, Vawdrey DK, Stetson PD, Zheng K, Hripcsak G. Clinical documentation: composition or synthesis? J Am Med Inform Assoc 2012;19(6):1025-31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Vawdrey DK, Chang N, Compton A, Tiase V, Hripcsak G. Impact of electronic medication reconciliation at hospital admission on clinician workflow. AMIA Annu Symp Proc 2010;2010:822-6. [PMC free article] [PubMed] [Google Scholar]
- 64.HealthIT.gov. Hospital EHR Vendors 2015. [cited 2016 January 27]. Available from: http://dashboard.healthit.gov/quickstats. [Google Scholar]
- 65.Koppel R, Lehmann CU. Implications of an emerging EHR monoculture for hospitals and healthcare systems. J Am Med Inform Assoc 2015;22(2):465-71. [DOI] [PubMed] [Google Scholar]
- 66.Evans RS. Electronic Health Records: Then, Now and the Future. Yearbook Med Inform 2016:XX-YY. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293(10):1197-203. [DOI] [PubMed] [Google Scholar]
- 68.Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform. 2009;78 Suppl 1:S69-76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Cimino JJ. Improving the electronic health record--are clinicians getting what they wished for? JAMA 2013;309(10):991-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Byyny RL. Editorial: The Tragedy of the electronic health record. Pharos Alpha Omega Alpha Honor Med Soc 2015;78(3):2-5. [PubMed] [Google Scholar]
- 71.Ober KP, Applegate WB. The electronic health record. Are we the tools of our tools? Pharos Alpha Omega Alpha Honor Med Soc 2015;78(1):8-14. [PubMed] [Google Scholar]
- 72.Hersh W, Williamson J. Educating 10,000 informaticians by 2010: the AMIA 10x10 program. Int J Med Inform 2007;76(5-6):377-82. [DOI] [PubMed] [Google Scholar]
- 73.Gardner RM, Overhage JM, Steen EB, Munger BS, Holmes JH, Williamson JJ, et al. Core content for the subspecialty of clinical informatics. J Am Med Inform Assoc 2009;16(2):153-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Safran C, Shabot MM, Munger BS, Holmes JH, Steen EB, Lumpkin JR, et al. Program requirements for fellowship education in the subspecialty of clinical informatics. J Am Med Inform Assoc 2009;16(2):158-66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Detmer DE, Munger BS, Lehmann CU. Clinical informatics board certification: history, current status, and predicted impact on the clinical informatics workforce. Appl Clin Inform 2010;1(1):11-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Payne TH. Electronic health records and patient safety: should we be discouraged? BMJ Qual Saf 2015;24(4):239-40. [DOI] [PubMed] [Google Scholar]
- 77.Riskin L, Koppel R, Riskin D. Re-examining health IT policy: what will it take to derive value from our investment? J Am Med Inform Assoc 2015;22(2):459-64. [DOI] [PubMed] [Google Scholar]
- 78.Wachter RM. The Digital Doctor: Hope, Hype and Harm at the Dawn of Medicine’s Computer Age. New York, NY: Mc Graw Hill Education; 2015. [Google Scholar]
- 79.Rosenbaum L. Transitional Chaos or Enduring Harm? The EHR and the Disruption of Medicine. N Engl J Med 2015;373(17):1585-8. [DOI] [PubMed] [Google Scholar]
- 80.Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff 2008;27(3):759-69. [DOI] [PubMed] [Google Scholar]
- 81.Sheikh A, Sood HS, Bates DW. Leveraging health information technology to achieve the “triple aim” of healthcare reform. J Am Med Inform Assoc 2015;22(4):849-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82.Payne TH, Corley S, Cullen TA, Gandhi TK, Harrington L, Kuperman GJ, et al. Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. J Am Med Inform Assoc 2015;22(5):1102-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.ONC. Connecting health and care for the nation: a 10-year vision to achieve an interoperable health IT infrastructure 2014. [cited 2015 October 27]. Available from: http://healthit.gov/sites/default/files/ONC10yearInteroperabilityConceptPaper.pdf. [Google Scholar]
- 84.Kohn LT, Corrigan JM, Donaldson MS. To Err is Human: Building a Safer Health System. Washington, DC: The National Academies Press; 2000. [PubMed] [Google Scholar]
- 85.Richardson WC. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: The National Academies Press; 2001. [PubMed] [Google Scholar]
- 86.McDonald CJ, Weiner M, Hui SL. Deaths due to medical errors are exaggerated in Institute of Medicine report. JAMA 2000;284(1):93-5. [DOI] [PubMed] [Google Scholar]
- 87.Leape LL. Institute of Medicine medical error figures are not exaggerated. JAMA 2000;284(1):95-7. [DOI] [PubMed] [Google Scholar]
- 88.James JT. A new, evidence-based estimate of patient harms associated with hospital care. J Patient Saf 2013;9(3):122-8. [DOI] [PubMed] [Google Scholar]
- 89.Shabot MM. New tools for high reliability healthcare. BMJ Qual Saf 2015;24(7):423-4. [DOI] [PubMed] [Google Scholar]
- 90.Warden GL. Health IT and Patient Safety: Building a Safer System for Better Care. Washington, DC: The National Academies Press; 2012 2012. [PubMed] [Google Scholar]
- 91.Medicine NAo. Protecting Patients: Advances and Future Direction of Patient Safety: Washington, DC; 2015. [cited 2016 January 18]. Available from: http://nam.edu/event/2015-rosenthal-symposium/. [Google Scholar]
- 92.Classen DC, Resar R, Griffin F, Federico F, Frankel T, Kimmel N, et al. ‘Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured. Health Affairs 2011;30(4):581-9. [DOI] [PubMed] [Google Scholar]
- 93.Graber ML, Siegal D, Riah H, Johnston D, Kenyon K. Electronic Health Record-Related Events in Medical Malpractice Claims. J Patient Saf 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 94.Sittig DF, Classen DC, Singh H. Patient safety goals for the proposed Federal Health Information Technology Safety Center. J Am Med Inform Assoc. 2015;22(2):472-8. doi: 10.1136/amiajnl-2014-002988. PubMed PMID: 25332353. [DOI] [PubMed] [Google Scholar]
- 95.Banger A, Graber ML. Recent Evidence that Health IT Improves Patient Safety: Office of National Center (ONC); 2015. [cited 2016 January 16]. Available from: https://healthit.gov/sites/default/files/brief_1_final_feb11t.pdf. [Google Scholar]
- 96.Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff 2011;30(3):464-71. [DOI] [PubMed] [Google Scholar]
- 97.Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144(10):742-52. [DOI] [PubMed] [Google Scholar]
- 98.Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and benefits of health information technology: new trends from the literature. Health Aff 2009;28(2):w282-93. [DOI] [PubMed] [Google Scholar]
- 99.Amalberti R. The paradoxes of almost totally safe transportation systems. Saf Sci. 2001;37:109-26. [Google Scholar]
- 100.Amalberti R. Navigating Safety: Necessary Compromises and Trade-Off-Theory and Practice. New York, NY: Springer; 2013. [Google Scholar]
- 101.Amalberti R, Benhamou D, Auroy Y, Degos L. Adverse events in medicine: easy to count, complicated to understand, and complex to prevent. J Biomed Inform 2011;44(3):390-4. [DOI] [PubMed] [Google Scholar]
- 102.Vincent C, Amalberti R. Safety in healthcare is a moving target. BMJ Qual Saf 2015;24(9):539-40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 103.Baines R, Langelaan M, de Bruijne M, Spreeuwenberg P, Wagner C. How effective are patient safety initiatives? A retrospective patient record review study of changes to patient safety over time. BMJ Qual Saf 2015;24(9):561-71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 104.Baines RJ, Langelaan M, de Bruijne MC, Asscheman H, Spreeuwenberg P, van de Steeg L, et al. Changes in adverse event rates in hospitals over time: a longitudinal retrospective patient record review study. BMJ Qual Saf 2013;22(4):290-8. [DOI] [PubMed] [Google Scholar]
- 105.Vincent C, Amalberti R. Safer Healthcare: Strategies for the Real World. New York, NY: Springer; 2016. [PubMed] [Google Scholar]
- 106.Balogh EP, Miller BT, Ball JR. Improving Diagnosis in Health Care. Washington, DC: The National Academy of Sciences; 2015. [PubMed] [Google Scholar]
- 107.Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011;8(1):e1000387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 108.Denham CR, Classen DC, Swenson SJ, Henderson MJ, Zeltner T, Bates DW. Safe use of electronic health records and health information technology systems: trust but verify. J Patient Saf 2013;9(4):177-89. [DOI] [PubMed] [Google Scholar]
- 109.Ferranti JM, Langman MK, Tanaka D, McCall J, Ahmad A. Bridging the gap: leveraging business intelligence tools in support of patient safety and financial effectiveness. J Am Med Inform Assoc 2010;17(2):136-43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 110.Gallego B, Magrabi F, Concha OP, Wang Y, Coiera E. Insights into temporal patterns of hospital patient safety from routinely collected electronic data. Health Inf Sci Syst 2015;3(Suppl 1 HISA Big Data in Biomedicine and Healthcare 2013 Con):S2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 111.Marcilly R, Ammenwerth E, Roehrer E, Pelayo S, Vasseur F, Beuscart-Zephir MC. Usability Flaws in Medication Alerting Systems: Impact on Usage and Work System. Yearb Med Inform 2015;10(1). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 112.Meeks DW, Smith MW, Taylor L, Sittig DF, Scott JM, Singh H. An analysis of electronic health record-related patient safety concerns. J Am Med Inform Assoc 2014;21(6):1053-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 113.Phansalkar S, Edworthy J, Hellier E, Seger DL, Schedlbauer A, Avery AJ, et al. A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc 2010;17(5):493-501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 114.Schiff GD, Amato MG, Eguale T, Boehne JJ, Wright A, Koppel R, et al. Computerised physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf 2015;24(4):264-71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115.Wang Y, Eldridge N, Metersky ML, Verzier NR, Meehan TP, Pandolfi MM, et al. National trends in patient safety for four common conditions, 2005-2011. N Engl J Med 2014;370(4):341-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 116.Leape LL, Berwick DM, Bates DW. What practices will most improve safety? Evidence-based medicine meets patient safety. JAMA 2002;288(4):501-7. [DOI] [PubMed] [Google Scholar]
- 117.Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med 2003;348(25):2526-34. [DOI] [PubMed] [Google Scholar]
- 118.Evans RS, Larsen RA, Burke JP, Gardner RM, Meier FA, Jacobson JA, et al. Computer surveillance of hospital-acquired infections and antibiotic use. JAMA 1986;256(8):1007-11. [PubMed] [Google Scholar]
- 119.Clancy CM, Berwick DM. The science of safety improvement: learning while doing. Ann Intern Med 2011;154(10):699-701. [DOI] [PubMed] [Google Scholar]
- 120.Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy SM, Foy R, et al. Advancing the science of patient safety. Ann Intern Med 2011;154(10):693-6. [DOI] [PubMed] [Google Scholar]
- 121.Panzer RJ, Gitomer RS, Greene WH, Webster PR, Landry KR, Riccobono CA. Increasing demands for quality measurement. JAMA 2013;310(18):1971-80. [DOI] [PubMed] [Google Scholar]
- 122.Thomas EJ, Classen DC. Patient safety: let’s measure what matters. Ann Intern Med 2014;160(9):642-3. [DOI] [PubMed] [Google Scholar]
- 123.Blumenthal D, Malphrus E, McGinnis JM. Vital Signs: Core Metrics for Health and Health Care Progress. Blumenthal D, Malphrus E, McGinnis JM, editors. Washington (DC): The National Academies Press; 2015. April 2015. [PubMed] [Google Scholar]
- 124.Liyanage H, Correa A, Liaw ST, Kuziemsky C, Terry AL, de Lusignan S. Does Informatics Enable or Inhibit the Delivery of Patient-centred, Coordinated, and Quality-assured Care: a Delphi Study. A Contribution of the IMIA Primary Health Care Informatics Working Group. Yearb Med Inform 2015;10(1). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 125.Foundation NPS. Free from Harm: Accelerating Patient Safety Improvement Fifteen Years after To Err is Human 2015. [cited 2016 January 16]. Available from: http://www.npsf.org/free-from-harm. [Google Scholar]
- 126.Gardner RM. University of Utah Medical Informatics Research and Training program. Yearb Med Inform 2001:103-11. [PubMed] [Google Scholar]
- 127.Patel VL, Kaufman DR, Cohen T. Cognitive Informatics in Health and Biomedicine: Case studies on Critical Care, Complexity and Errors. New York, NY: Springer; 2014. [Google Scholar]
- 128.Patel VL, Kannampallil TG, Shortliffe EH. Role of cognition in generating and mitigating clinical errors. BMJ Qual Saf 2015;24(7):468-74. [DOI] [PubMed] [Google Scholar]
- 129.Wong BM, Dyal S, Etchells EE, Knowles S, Gerard L, Diamantouros A, et al. Application of a trigger tool in near real time to inform quality improvement activities: a prospective study in a general medicine ward. BMJ Qual Saf 2015;24(4):272-81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 130.Thomas EJ. The future of measuring patient safety: prospective clinical surveillance. BMJ Qual Saf 2015;24(4):244-5. [DOI] [PubMed] [Google Scholar]
- 131.Nanji KC, Patel A, Shaikh S, Seger DL, Bates DW. Evaluation of Perioperative Medication Errors and Adverse Drug Events. Anesthesiology 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 132.Hillestad R, Bigelow J, Bower A, Girosi F, Meili R, Scoville R, et al. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff 2005;24(5):1103-17. [DOI] [PubMed] [Google Scholar]
- 133.Kellermann AL, Jones SS. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff 2013;32(1):63-8. [DOI] [PubMed] [Google Scholar]
- 134.Bassi J, Lau F. Measuring value for money: a scoping review on economic evaluation of health information systems. J Am Med Inform Assoc 2013;20(4):792-801. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 135.Rudin RS, Jones SS, Shekelle P, Hillestad RJ, Keeler EB. The value of health information technology: filling the knowledge gap. Am J Manag Care 2014;20(11 Spec No. 17):eSP1-8. [PubMed] [Google Scholar]
- 136.Seroussi B, Jaulent MC, Lehmann CU. Health Information Technology Challenges to Support Patient-Centered Care Coordination. Yearb Med Inform 2015;10(1):8-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 137.Adler-Milstein J, Everson J, Lee SD. EHR Adoption and Hospital Performance: Time-Related Effects. Health Serv Res 2015; 50(6):1751-71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 138.Purkayastha S, Price A, Biswas R, Jai Ganesh AU, Otero P. From Dyadic Ties to Information Infrastructures: Care-Coordination between Patients, Providers, Students and Researchers. Contribution of the Health Informatics Education Working Group. Yearb Med Inform 2015;10(1). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 139.Yusof MM, Papazafeiropoulou A, Paul RJ, Stergioulas LK. Investigating evaluation frameworks for health information systems. Int J Med Inform 2008;77(6):377-85. [DOI] [PubMed] [Google Scholar]
- 140.Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008;77(6):386-98. [DOI] [PubMed] [Google Scholar]
- 141.Saparova D, Belden J, Williams J, Richardson B, Schuster K. Evaluating a federated medical search engine: tailoring the methodology and reporting the evaluation outcomes. Appl Clin Inform 2014;5(3):731-45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 142.Yusof MM. A case study evaluation of a Critical Care Information System adoption using the socio-technical and fit approach. Int J Med Inform 2015;84(7):486-99. [DOI] [PubMed] [Google Scholar]
- 143.Ammenwerth E. Evidence-based Health Informatics: How Do We Know What We Know? Methods Inf Med 2015;54(4):298-307. [DOI] [PubMed] [Google Scholar]
- 144.Al-Shorbaji N, Hanmer L, Hussein R, Magrabi F, Moen A, Moura LA, et al. Discussion of “Evidence-based Health Informatics: How Do We Know What We Know?”. Methods Inf Med 2015;54(4):308-18. [DOI] [PubMed] [Google Scholar]
- 145.Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 2007;147(4):224-33. [DOI] [PubMed] [Google Scholar]
- 146.Gartner’s 2015 Hype Cycle for Emerging Technologies Identifies the Computing Innovations that Organizations Should Monitor [Internet]. Stamford, Connecticut: Gartner, Inc.; 2015; August 18, 2015 [cited 20 Nov 15]. Available from: http://www.gartner.com/newsroom/id/3114217 [Google Scholar]
- 147.Harris Y, Schneider CD. Big Data: A Realisitc Assessment of its Application to Health Care Ch 4. Health Information Technology in the United States, 2015: Transition to a Post-HITECH World 2015. p. 53-79. [Google Scholar]
- 148.Pickering BW, Dong Y, Ahmed A, Giri J, Kilickaya O, Gupta A, et al. The implementation of clinician designed, human-centered electronic medical record viewer in the intensive care unit: a pilot step-wedge cluster randomized trial. Int J Med Inform 2015;84(5):299-307. [DOI] [PubMed] [Google Scholar]
- 149.Vawdrey DK, Gardner RM, Evans RS, Orme JF, Jr., Clemmer TP, Greenway L, et al. Assessing data quality in manual entry of ventilator settings. J Am Med Inform Assoc 2007;14(3):295-303. [DOI] [PMC free article] [PubMed] [Google Scholar]