Skip to main content
Journal of the American Medical Informatics Association : JAMIA logoLink to Journal of the American Medical Informatics Association : JAMIA
. 2008 Jul-Aug;15(4):397–407. doi: 10.1197/jamia.M2735

The Informatics Opportunities at the Intersection of Patient Safety and Clinical Informatics

Peter M Kilbridge a , b ,, David C Classen b , c
PMCID: PMC2442268  PMID: 18436896

Abstract

Health care providers have a basic responsibility to protect patients from accidental harm. At the institutional level, creating safe health care organizations necessitates a systematic approach. Effective use of informatics to enhance safety requires the establishment and use of standards for concept definitions and for data exchange, development of acceptable models for knowledge representation, incentives for adoption of electronic health records, support for adverse event detection and reporting, and greater investment in research at the intersection of informatics and patient safety. Leading organizations have demonstrated that health care informatics approaches can improve safety. Nevertheless, significant obstacles today limit optimal application of health informatics to safety within most provider environments. The authors offer a series of recommendations for addressing these challenges.


This position paper focuses on next steps in using health informatics to improve patient safety. The paper does not attempt to provide a comprehensive review of patient safety-related technical accomplishments, because recent Institute of Medicine (IOM) reports have done so. This paper focuses instead on current pressing issues and opportunities for addressing them in the short-term future. The intended audience includes provider organizations responsible for the safe delivery of health care; policy makers responsible for funding and regulatory decisions that influence health care safety, and the health informatics developers community (including vendors) who build the computer systems that support patient care.

Introduction

Since the time of Hippocrates, health care providers have had a basic responsibility to protect patients from accidental harm. Yet, only in recent years has the medical profession publicly acknowledged its duty to improve patient safety during the delivery of care. 1 Only within the last century has biomedical knowledge advanced far enough to teach health care providers how to do more good than harm reliably. In 1974, while describing the findings of the Boston Collaborative Drug Surveillance Program, a seminal study of medication safety, Herschel Jick wrote:

… When extrapolated to the entire American population these data suggest that adverse drug reactions in their totality yearly afflict millions of people, causing hundreds of thousands of hospitalizations, and deaths numbering in the tens of thousands. However, the rates and severity of adverse reactions to individual drugs are remarkably low in view of their pharmacologic properties. 2

At the time of Jick's article, harm of some unknown magnitude from medications was generally considered acceptable relative to the imagined benefits of these agents. Our conception of the balance of acceptable risk versus benefit during health care provision has evolved considerably. Twenty-five years after Jick's statement, an IOM report on patient safety concluded that “… it is simply not acceptable for patients to be harmed by the same health care system that is supposed to offer healing and comfort.” 1 The IOM statement clearly refers to avoidable risk, as many treatments, such as radiation therapy and chemotherapy, have unavoidable harmful side effects even when administered ideally.

Improving patient safety requires commitment by leadership, development of a culture of safety, adoption of fundamental safety principles, and incorporation of those principles into practice in the workplace. 1 Computers and information systems can make important fundamental contributions toward creation of safe systems through improving access to information, reducing reliance on memory, increasing vigilance, and contributing to standardization of processes. Many leading organizations have conducted studies that demonstrated the potential for informatics to improve aspects of patient safety. 3–10

This position paper focuses on next steps in using health informatics to improve patient safety. The paper does not attempt to provide a comprehensive review of patient safety–related technical accomplishments and challenges, because recent IOM reports 1,11–14 have done so. The Institute's first report on patient safety, To Err is Human, reviewed the obstacles to safe care in complex environments and made a series of recommendations for technology adoption by health care providers. 1 In their 2001 report Crossing the Quality Chasm, the IOM reviewed some of the challenges to effective use of information technology (IT) to improve care quality, including underinvestment by provider organizations, securing patient health information, providing adequate technical infrastructure to ensure connectivity in diverse settings, reimbursement for technology services, and others. 12 Keeping Patients Safe, the IOM's study of the nursing environment and patient safety, reviewed the potential impact of IT upon nursing decision making and documentation. 11 In Patient Safety: Achieving a New Standard for Care, the Institute delved deeply into the challenges of building a national health infrastructure and establishing standardized reporting mechanisms. 14

Rather than recapitulating past analyses, this paper instead focuses on current pressing issues and related opportunities for addressing safety concerns through informatics initiatives. We consider the role of clinical informatics in relation to three different aspects of patient safety: the cultivation of a safe environment of care, the safe provision of care to the patient, and the measurement of patient safety. The authors recommend next steps for addressing the major obstacles to effective implementation of safety-related informatics solutions in the short-term future.

Informatics and the Safe Environment of Care

Past studies emphasize the importance of the culture of the health care delivery environment to promoting patient safety. 1 A culture of safety is essential to the design, establishment, and maintenance of safe practices at all levels. A prerequisite for creating a culture of safety is senior executive-level belief in and commitment to safety as an overarching priority of the organization.

A culture of safety comprises the following elements: acknowledgement that providing health care is a hazardous undertaking; taking responsibility for reducing the associated risks; encouragement of error reporting without blame; willingness to discuss errors openly in a nonpunitive manner, and to learn from mistakes; a commitment to communication across traditional hierarchies and boundaries; a systems (not individual) approach to analyzing errors and failures; effective multidisciplinary teamwork; and establishment of administrative structures for accountability for patient safety. 1,12,15

Communication problems are a root cause of many patient safety events in health care. One example is the “sign-out,” by which one clinician communicates patient coverage information to another clinician at change-of-shift. This error-prone process leads to preventable bad outcomes that computerized sign-out applications can reduce. 16 This approach can be used more broadly for interdisciplinary communication. 17

Computer systems per se cannot perform many essential safety-related activities. Patient safety requires adequate clinical staffing to meet the basic needs of a high-risk, high-volume care environment. In the inpatient setting, inadequate nurse staffing levels have been linked to the occurrence of patient complications. 18,19 The evidence shows that advanced clinical systems cannot be used to safely augment clinical staffing requirements. 20

A well-recognized informatics contribution to the culture of safety is the adoption of automated safety reporting systems by an increasing number of health care provider organizations. Based upon the principles followed in commercial aviation's Aviation Safety Reporting System (http://asrs.arc.nasa.gov), such programs allow the reporting of safety-related events or concerns by any staff member from any location throughout an organization, anonymously if desired. These systems facilitate analysis of the resulting reports by trained safety experts in a confidential, nonpunitive environment protected from legal discovery. Computerized reporting systems have benefits and limitations, as discussed below, but they represent a substantial advance over traditional paper-based reporting systems.

A complementary approach to creating a safety culture is to teach and practice the cognitive skills, technical skills, and teamwork required for safe care. 21 Increasingly, automation contributes an improved care delivery environment through simulations that train, assess and certify health care personnel. For example, computer-based simulation has become an approved method for training physicians to place carotid artery stents. Increasingly, procedure-oriented specialties have used simulations for training and competency assessment. 22,23 Computer-based simulations have been used in the United States Medical Licensing Examination as well as other forms of certification. 24

Basic competency in clinical informatics itself has assumed a more critical role for effective clinical performance. Clinicians are required to assimilate data and make decisions rapidly in increasingly information-rich clinical environments. Building and maintaining specialized resources in applied informatics (in addition to the information systems staff) gives an organization a significant advantage in its efforts to improve patient safety. Prior to the American Medical Informatics Association (AMIA) 10 × 10 (www.amia.org) and other similar initiatives, inadequate emphasis had been placed on training, certifying, or assessing the competency of health professionals in health care IT or informatics skills. Some academic organizations have viewed informatics as purely a library-related science. 25 Among some traditional academic medical centers, skepticism exists about the role of biomedical informatics in the education of future health care professionals. 26 The lack of emphasis on pre-professional-degree informatics education corresponds to the paucity of studies demonstrating the effects of such education on health care delivery, safety, and outcomes. Too few medical and other health professional schools have moved forward with integrated health care informatics curricula. Because in many cases, the “horse is already out of the barn,” AMIA's 10 × 10 program aims to give practicing clinicians and IT professionals who have already completed their formal education better skills to utilize health informatics tools. The program increases individuals' practical understanding of informatics and improves their basic knowledge and capabilities. Much remains to be done to infuse informatics into educational curricula, training programs, and certification and licensure programs for health professionals.

Finally, patients themselves can provide another layer of safety by becoming deeply involved in their own care management, 12,14 supported by an infrastructure that includes an informatics component. Personal health records (PHRs) provide an avenue for making personal health information available to consumers by tying their own record-keeping to decision support and other records at the office practice, clinic, or hospital levels. 27 Although PHRs represent a start, optimizing how these systems can improve the safety of care with an empowered patient remains a challenge. 28 Few if any research studies have demonstrated a significant impact of PHRs on the safety of care delivered. This is an important area for future work.

Clinical Informatics and the Safe Provision of Care

Critical elements for providing safe care to patients include assembling a comprehensive picture of individual patients and their problems, identifying and organizing the biomedical knowledge relevant to each patient's specific conditions, applying this knowledge appropriately and effectively, monitoring the effects of disease and therapies on the patient over time, and detecting and preventing errors that could harm the patient. 1 Informatics has demonstrated applicability to each of these tasks.

Assembling a Complete Picture of the Patient that Includes Tracking the Course of Diseases as Well as the Effects of Therapies over Time

Establishing an accurate and complete body of information about the health of the individual patient is the first step toward ensuring safe management of care. However, obtaining information about prior care delivery in disparate settings and geographic sites comprises a monumental challenge. Safe care provision requires, among other items of information, knowledge of essential historical elements including a patient's allergies, medication history, immunizations, and current and prior medical and surgical problems and encounters. The volume and complexity of information generated in patient care and the corresponding challenges of its management will increase steadily over time.

The primary obstacle to assembling a complete picture of the patient is the lack of uniform adoption of electronic record systems in all health care settings. 29–31 Obstacles to obtaining patient information, especially in electronic format include the fragmented nature of health care delivery and financing in the United States, with multiple public and private payer and provider structures; the mobility of the American population; lack of mature and accepted standards for the electronic description, encoding, and transmission of clinical information; fragmented connectivity; vendors' use of multiple, inconsistent software models for clinical data management; and restrictions due to privacy and security concerns. These obstacles cause inadequate communication of patient data across settings, and even within individual organizations. Researchers have invested significant amounts of energy and effort toward improvement in many of these areas, but much progress remains to be made.

Although the greatest opportunity to enhance patient safety lies in the electronic integration of clinical data from multiple settings for use at the point of care, many factors inhibit widespread electronic health record (EHR) adoption. These include lack of appropriate vendor functionality, ill-conceived approaches to system implementation, system cost, cultural issues related to clinician adoption of computerized systems, current privacy and security concerns, and uneven adoption of interoperability standards. The challenges to EHR adoption are more acute in smaller-sized health care delivery settings, such as individual physician offices and rural practices. Functionality challenges and “truth in advertising” are being addressed in part by several different EHR vendor certification efforts, discussed later. 32 The greatest barrier to EHR adoption remains the lack of affordability of these systems. This barrier must be addressed in any overall strategy to improve safety with clinical systems. 33

Integration of all that is known about a patient from all previous health care system encounters is another important aspect of patient safety. Early health care data integration efforts by a number of regional information sharing initiatives (such as the Massachusetts eHealth Collaborative, 34,35 the Indiana Health Information Exchange, the MidSouth eHealth Alliance, the Seattle Patient Safety Project, and the Santa Barbara Health Information Project 36 ) have begun to demonstrate the ability to exchange patient information accurately and quickly across different care settings within large geographic regions. The ability to do so will be essential to future safe delivery of care. Paramount to accessing patient information across care settings is the ability to uniquely and accurately identify the patient. Although numerous recommendations have been made on approaches to standardized patient identification, efforts to obtain consensus have come into conflict with such concerns as protection of privacy and confidentiality. 37

Assembling the Relevant Medical Knowledge

Relevant clinical knowledge exists in many forms (e.g., expert human consultants, books and journals, clinical databanks, knowledge bases for expert systems, national guidelines) and for many purposes (diagnosis, prognosis, selecting therapy, administering therapy, following best practices guidelines for administrative/cost issues, and guidelines/procedures for improving safety).

Great strides have been made in increasing the availability and easy accessibility via the Internet of different knowledge sources, from textbooks and current clinical summary information sources (e.g., the National Cancer Institute's PDQ, the Cochrane Library, guidelines.gov, and UpToDate®) and to current literature (e.g., PubMed, EMBASE, PubMedCentral, Ovid). Nonetheless, there exists an opportunity to enhance further the value of the plethora of clinical information that is available. Although it is comparatively easy to locate online such basic resources as PubMed, sorting through the numerous sources of, for example, clinical guidelines for a specific condition is typically more challenging.

Computer applications can be used to enhance adherence of clinical practice to the tenets of evidence-based medicine in a number of ways. These include the application of clinical protocols and guidelines during patient care 38 ; prompting adherence to protocols and guidelines via computerized order entry 39 ; use of disease management registries, which remind clinicians about guideline-based standards for intervention and follow-up in patients with chronic diseases; use of remote monitoring devices that allow monitoring of patients outside the hospital setting 40,41 ; and others. The correct care should in the long run be the safest care, and specifically from the point of view of patient safety, the principal value of these kinds of decision support lies in their ability to reduce errors of omission.

An important aspect of patient safety is the ability to distinguish between the effects of native illness and the effects of clinical therapies. Only with electronic records will it be possible to track courses of illnesses and the beneficial and adverse effects of therapy at the individual patient and general population levels. Projects such as the Duke Cardiovascular Disease Databank 42 and the American Rheumatism Association's ARAMIS 43 have demonstrated the ability of informatics approaches to provide accurate prognoses and to improve therapies. The recent National Library of Medicine strategic plan for the next decade 44 identified clinical data mining as an important area to increase our understanding of disease and improve patient outcomes. Efforts by the National Institutes of Health (NIH), AMIA, and others to create clinical trials databanks are relevant in this regard. 45–47

Several areas of opportunity present themselves in regard to informatics-assisted knowledge delivery in the context of a patient's presentation. Although the Harvard Medical Practice Study presented abundant evidence of harm to patients due to improper diagnosis—diagnostic failures represented the fifth most common type of adverse event, accounting for 8.1% of all events 48 —there has been proportionally far less recent investigation of methodologies for improving diagnostic efficiency and accuracy compared with investigation in other aspects of patient safety. See Appendix 1 (online supplement) for a more detailed discussion of opportunities and challenges related to improving clinical diagnoses through decision support programs.

The use of standards for information exchange facilitates reliable transmission of vital patient data among care settings and among disparate computer systems. Diagnostic and therapeutic coding standards are required for billing purposes, but may, with appropriate modification, also serve to enhance our ability to study disease populations and therapies. A large standards-related challenge is their adoption by software and communications vendors. Here, unfortunately, there remains much room for improvement. See Appendix 2 (online supplement) for a more detailed discussion of how adoption of standards can influence patient safety.

Applying Timely and Accurate Knowledge of Medicine to the Patient's Current Condition

As stated previously, the first and most important step in providing timely, safe, appropriate advice for patient care is assembly of a comprehensive and accurate electronic record for the patient. Once such a record is available, it becomes possible to apply electronic representations of clinical knowledge to the problems of patient care.

Researchers have invested much effort in designing applications that guide clinicians in decision making to improve adherence to evidence-based care standards, with mixed results. There is evidence that computerized reminders 49 and order sets 50 can improve adherence to preventive care guidelines. Evidence regarding adherence to disease-specific guidelines is more mixed, with several studies showing a lack of effect of computerized reminders even in highly computerized settings. 51,52 However another recent study showed a positive impact of a computerized provider order entry (CPOE)–based, template-driven therapy advisor on adherence to guidelines for acute myocardial infarction care. 53 Recently much attention has been focused on the use of specialty clinical systems such as the eICU, with reports of improved safety and cost of care, especially when these systems are implemented via telemedicine; and the ability of these systems to alert care providers about abnormal laboratory or other test results can improve the safety and reliability of care. However, the lack of reproducibility of these results and integration issues between these standalone systems and EHRs have raised concerns about their ultimate impact on patient safety. 54,55

Further study of different forms and of implementation approaches for decision support should lead us to a better understanding of how to be effective with computer guidance. Some basic ingredients for success are identified in a recent review and analysis of the decision support literature. Kawamoto et al. 56 examined different decision support strategies (paper-based as well as computerized) for determinants of effectiveness and reinforced what early clinical informatics pioneers such as Homer Warner 57 in Salt Lake City, Octo Barnett, 58 Howard Bleich and Warner Slack 59 in Boston, Clem McDonald 10 in Indianapolis, and the CPOE team at Brigham and Women's Hospital in Boston 3 had instinctively known. The factors Kawamoto et al. found correlated with improvements in clinical practice included: (1) provision of decision support as part of clinician workflow; (2) providing recommendations, not just alerts; (3) providing decision support at the point of decision making; and (4) computer-based decision support. 56

Preventing Mistakes

Although errors of omission can result in poor outcomes, errors of commission may cause harm in more noticeable and preventable ways. Researchers have devoted greater attention to use of computers and information systems to prevent, intercept, and ameliorate harm to patients due to errors of commission in relationship to patient safety.

Example: Computers and Medication Safety

Medication management, the selection, ordering, dispensing, and administration of medications, is one of the most frequently computerized processes in clinical care. Because adverse drug events represent one of best-documented causes of harm to patients, medication management is viewed as a central leverage point for improving safety. Appendix 3 (online supplement) illustrates many critical applications of informatics to medication safety improvement. These include drug information databases used to improve prescribing in real time, computerized provider order entry with integrated decision support, automated dispensing technologies including bar coding of medications dispensed, smart pumps, and automated surveillance to monitor and prevent adverse drug events.

IT, Event Reporting, and Measuring the Safety of Care

A major challenge facing organizational patient safety efforts remains the difficulty of tracking the primary outcome: harm to patients, or adverse events. Most organizations use only voluntary reporting systems to detect adverse events, which although essential and valuable for cultural and regulatory purposes, are woefully inadequate for accurate and complete event detection.

Many well-recognized barriers exist to voluntary event reporting, including embarrassment, lack of time, fear of reprisal, and fear of medico-legal repercussions. 60–64 Possibly most important is that most adverse events are not recognized as such at the time of their occurrence. 60 Automated reporting systems (mentioned above) that conceal the identities of persons reporting problems encourage reporting of events that would otherwise have gone undetected by the organization, and improve upon manual, paper-based reporting. When implemented in a blame-free environment, they help to reinforce the trust and confidence of the organization in its employees. By reducing the cycle time from event reporting to event analysis, these systems also increase efficiency. 61 Such voluntary reporting systems may detect certain kinds of events that other systems currently do not detect. Hence, they can supplement complementary mechanisms such as automated surveillance systems. Automated surveillance systems differ from manual and automated reporting systems in that they use electronically detectable criteria rather than human cues to determine when events occur. Such surveillance systems typically detect adverse events at rates four to 20 times higher than those measured by voluntary reporting. 65–67

Voluntary reporting rates and behaviors are easily influenced by organizational incentives: if leadership decrees that a safety goal for the coming year is to reduce the incidence of adverse events as measured by voluntary reporting, reporters will generally oblige, regardless of the true underlying incidence of adverse events. Thus, reliance upon voluntary reporting as a metric for adverse events frequency underreports events substantially. Voluntary reporting, when used alone in a fully identifiable or punitive manner, may actually subvert the reporting process, reducing its true value, which is to promote a culture of safe communication and discussion while providing stories about safety events from which to learn. 68

Quantifying and tracking the incidence of adverse events using automated surveillance methods, not just numbers from voluntary reports, can provide organizations with baseline metrics against which to measure the success of improvement efforts and against which leadership can be held accountable for improvements. Studies have demonstrated that automated surveillance methods may over-count events, and if purely automated, provide less insight than manual auditing methods. A combination of both approaches can provide depth and breadth in improving safety.

Automated surveillance techniques similar to those described above have been used not only to avoid harm to patients, but also to quantitatively monitor rates of harm over time as well. Quantifying and characterizing adverse drug events (ADEs) in hospitalized patients was the primary objective of some early major work in patient safety. 65 Investigators utilized a set of rules based on isolated drug administration data, drug level data, and other laboratory data, which when combined indicated a high probability that an ADE had already occurred. Follow-up manual investigation by trained clinical pharmacists for each automated event detected focused on characterizing event severity and causality. This work showed that automated surveillance systems can accurately, dependably, and reproducibly monitor rates of major categories of ADEs in the hospital setting. 66,67,69 Various investigators have extended the early inpatient investigations to the ambulatory arena, where the challenge of data accessibility is far greater. 70,71 Some investigators have begun to extend automated surveillance techniques beyond medications to detect other event categories, such as patient falls. 72

Automated quality detection and reporting is becoming important for another reason. Beyond the detection of adverse events—safety outcomes—lies the challenge of detecting or measuring quality of care outcomes, or performance. As part of the growing movement toward improved transparency in the U.S. health care system, health care organizations will in the future be required to collect and publicly report increasing quantities of data about the quality of care they provide. This information will be used to compare organizational achievements in quality of care, but it will also be used to alter reimbursement through the rapidly growing private and public pay-for-performance programs. Organizations will have little choice but to collect quality and safety-related information, the amount of which continues to increase at an exponential rate.

Currently, most organizations collect this outcome information manually, even if they have EHRs, because their EHRs do not routinely incorporate many of the required outcome measures. Because organizations cannot afford to wait until vendor products offer the functionality to automate the collection of this information, their manual efforts on such tasks consume enormous resources, especially in financially strapped organizations. One recent IOM report 14 promoted the collection of outcome information as a byproduct of documenting care within EHRs. Most available vendor systems are still far from delivering this capability, and to move toward it, vendors and pioneering academic medical centers must build upon their successes in improved detection of adverse events with IT to automate better the identification and collection of quality metrics for public reporting within EHRs.

Opportunities and Recommendations

Based on the above brief summary, the authors outline a series of critical safety-related health care informatics issues, and then provide specific recommendations to address them. Based on their experience and their review of the literature, the authors present their recommendations and viewpoints about pressing opportunities to advance patient safety at its intersection with health care informatics.

Opportunity 1: Promote the Development, Dissemination, and Monitoring of Health Care Data Standards

The IOM emphasized the importance of standards for data representation, transmission, and applications that apply clinical data in a landmark report on patient safety. 14 Despite recent efforts by the Department of Health and Human Services (DHHS) best exemplified in the American Health Information Community initiative (www.hhs.gov/healthit/standards), there remain disagreements and some competition with respect to such standards. 73 It is time for the federal government to intervene to overcome barriers and move the health care IT standards process effectively forward. It has already done so in part through its fostering of the National Library of Medicine (NLM) initiatives, and through promotion of LOINC, NDF-RT, and SNOMED-CT as national standards.

Recommendation 1

The federal government could authorize and help to fund creation of a private, nonprofit public benefit organization with sufficient authority to manage all aspects of health care data standards. 73 This entity could be modeled on the National Technology Transfer Advancement Act (the law that created the National Quality Forum) such that all standards endorsed would be adopted by the relevant federal agencies. The federal government could require that the standards endorsed by this group be included in all programmatic regulatory requirements, in the contractual requirements for federal health programs such as the Department of Defense and the Veterans Administration, and in health insurance programs such as Medicare and all private health insurance programs funded by the federal government. Such requirements would build upon a recent presidential executive order that mandates the use of interoperability standards in relevant federal health programs. 74

It is important to recognize that forcing immediate adoption of certain standards could have the effect of shutting down vital, functioning noncompliant clinical informatics systems at many sites. Thus, an adequate period for stepwise adoption of the new standards, with an appropriate transition plan, will be key to the promotion of universal standards.

The proposed new health care data standards organization would focus on standards that clearly impact patient safety (data interchange, vocabularies, and knowledge representation), as outlined in the IOM report Patient Safety; Achieving A New Standard of Care. 14 The DHHS could help to provide funding to create this organization, and could direct the Agency for Healthcare Research and Quality (AHRQ) to provide necessary funding to complete the development of high-safety-impact IT standards such as Rx Norm, LOINC, SNOMED and its successors, the HL7 EHR Standard, and the HL7 CDA/ASTM CCR. The initial work might focus on standardizing critical patient information such as medication lists, allergy information, and problem lists. Alternatively, DHHS could provide funding to the NLM to make these standards available for worldwide use and for their effective dissemination and linkage to relevant international standards.

Opportunity 2: Improve Adoption and Effective Use of EHRs in Physician Offices and Rural Health Care Settings

Effective adoption of health IT remains limited. The potential for informatics systems to improve patient safety will remain unrealized if those systems are not commonly used in patient care. It is estimated that fewer than 10% of U.S. hospitals have broadly and successfully implemented CPOE, and use of CPOE in rural hospitals is extremely rare. 75–77 Physician offices have also been slow to adopt EHRs. The current U.S. health care reimbursement system places the financial burden of purchasing these systems on the provider, yet much of the benefit of these systems goes to payers. Overcoming this significant market obstacle will require interventions at the federal level. 33,78 To be effective, such interventions will not only need to help to fund initial purchase and implementation, but also need to support ongoing maintenance and upgrades. The inclusion of informatics incentives in several private pay-for-performance programs has already begun, albeit in a limited fashion. The challenge goes beyond merely purchasing these systems and aligning incentives for their effective use to their successful implementation in a manner that both improves clinical care and patient safety. Various experts differ on whether an adequate return on investment, or alternatively, an adequate return on outcome, should be expected. Relevant implementation expertise is not widely available, especially in physician offices and rural settings of care. 79,80 Currently, there are numerous bills in congress addressing health IT, some of which have provisions focused on funding the adoption of clinical informatics applications in rural health care settings and in small physician practices.

Recommendation 2

To improve adoption and effective use of health IT in the United States, a special new program should provide seed funding for the initial purchase and adoption of (preferably Certification Commission for Health Information Technology [CCHIT]-certified) EHR products for rural health care settings (including smaller hospitals) and for all physician offices. It is recognized that CCHIT certification will not guarantee that a given system is ideal for a given clinical practice, nor will it guarantee successful implementation even when a CCHIT-approved application is the best available product. Nevertheless, CCHIT certification of a product means that it will possess at least minimal appropriate functionality. There are far fewer guarantees for naïve clinician-adopters that noncertified systems will perform as advertised.

The Center for Medicare and Medicaid Services (CMS) and the state governments might consider providing financial rewards to providers participating in Medicare or Medicaid programs that invest, deploy, implement, and use EHRs. These two large public insurance programs could be encouraged to work together to re-examine their benefit and payment programs to ensure appropriate coverage of appropriate and effective health services that are delivered electronically. Other federal agencies, such as the Health Resources and Services Administration (HRSA), could develop a strategy for transitioning community health centers, rural health clinics, critical access hospitals, and other rural providers from paper to EHRs. CMS could continue to fund the Quality Improvement Organizations as regional and local technical assistance centers, able to provide expertise in the safe and effective implementation of clinical informatics to rural hospitals and clinics and to physician office practices.

Opportunity 3: Use Clinical Decision Support Effectively in EHRs

To maximize benefit from purchase and implementation of an EHR system requires the use of clinical decision support (CDS) within the application. Such CDS “provides clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care.” 81 Adoption of EHR systems without CDS features would have far less impact in improving safety and quality of care. The current state of actual CDS use within EHRs is primarily limited to a small number of leading academic, governmental (the Veterans Administration) and large-scale private initiatives (e.g., Kaiser Permanente). Virtually all exemplars of excellent CDS are these sites with custom-developed systems. Current vendor-provided CDS capabilities are quite limited. The CDS in vendor products is often clumsy and crude, and often creates workflow nightmares due to lack of local customization. These problems lead to frequent end-user overrides of CDS advice or the wholesale inactivation of CDS functionality within implemented systems. 82 The current national recommendations to move toward electronic ordering and prescribing render it imperative to take a more rational and comprehensive approach to implementation of CDS. 83 As a start in this direction, a multidisciplinary group has been developing guidance toward a comprehensive approach via a CDS Roadmap. 84

Recommendation 3

The CDS Roadmap 84 could be reviewed, put out for discussion and comments, and after appropriate modification, endorsed and adopted by the Office of the National Coordinator (ONC), NLM, National Committee on Vital and Health Statistics (NCVHS), and CMS. The AHRQ could be allocated funding to support the agenda created within the CDS roadmap for both implementation and research to include identification of best-practice decision support applications across settings of care and methods for dissemination. The NIH could test the CDS Roadmap through facilitating its use at funded Clinical and Translational Science Award (CTSA) sites. Subsequently, DHHS might require that CDS certification be incorporated into the CCHIT vendor certification process. Potentially, AMIA might assist with this, possibly through a formal collaboration. A minimum set of CDS capabilities could be required as part of all informatics incentives within public pay-for-performance initiatives. In addition, CMS could endeavor to mandate CDS capabilities be included in their pay-for-performance programs. This certification process might focus not only on EHR vendor CDS certification, but also on third-party, standards-compliant commercial knowledge bases for decision support, such as drug interaction databases.

Opportunity 4: Certify, When Appropriate, Clinical Informatics Products and Implementation Approaches

Effective clinical informatics adoption is hindered by the large financial, cultural, and reputation risks that confront health care organizations when they undertake EHR implementations. 26 Poorly done implementation can also put patient safety at risk. 85,86 The science of safe and effective EHR implementations is evolving, and best practices are both suboptimally disseminated and conveniently ignored to the detriment of those doing so. Often, provider organizations rely on their vendors for advice on the approach to clinical system implementation. 49 Such reliance can lead to unexpected and unanticipated problems, including large cost overruns, disgruntled employees, and incomplete or failed implementations.

It is important to recognize that clinical system certification by agencies such as CCHIT are analogous to the U.S. Food and Drug Administration (FDA) approval of medications. The approval process helps to improve safety considerably, but is imperfect. In general, such federal approval processes screen out grossly harmful products. Yet, all FDA-approved drugs still carry risks of potentially harmful side effects (e.g., allergic reactions) as well as having documented potential therapeutic benefits. Clinical informatics systems are similar—in addition to promised benefits, they have the potential to cause new types of errors. Just as clinical vigilance is the best approach to monitoring patients for side effects, institutions adopting clinical systems must provide an adequate safety net to catch information-system–related errors. 86 The certification of EHR vendor products under CCHIT may provide a minimal level of safety assurance, but this should not be considered to be any more effective than is FDA approval in guaranteeing medication safety.

A separate yet complementary approach to certify providers is proceeding under the auspices of the National Quality Forum and Leapfrog. 87 However, these programs are voluntary and their impact will be limited without the stronger hand of the federal government and internal review board–like local software oversight. 33,88,89

Recommendation 4

The DHHS could direct that CCHIT vendor certification be built into all programmatic requirements for Medicare and Medicaid and could encourage that the private health insurers include such requirements in their programs as well. Both HRSA and CMS, acting through community health centers and quality improvement organizations, could supply technical expertise and educational programs directed toward inpatient and ambulatory facilities for the safe and effective implementation of health care informatics systems. AHRQ could fund specific research on safe and effective approaches for the implementation of commercial informatics applications in health care organizations, building on the work of the Leapfrog CPOE flight simulator. 87 The agency could continue to fund research in certification of HIT systems, evaluating vendor products “off the shelf” (the current CCHIT approach), and also following implementation in both inpatient and ambulatory settings of care. Software oversight committees at the local level, corresponding to FDA postmarketing surveillance for approved drugs, should be promoted in conjunction with the more centralized informatics system certification efforts. 89

Opportunity 5: Integrate Pay-for-Performance, HIT, and Patient Safety

The pay-for-performance initiative has the potential to become the “next great earthquake” in American health care reimbursement. As with the creation of Medicare in 1964 and the implementation of Diagnosis-Related Groups in 1983, pay-for-performance can potentially affect the U.S. health care system profoundly. More than 150 different private and public pay-for-performance initiatives are ongoing; currently, most target a variety of quality, patient safety, and efficiency measures. Only a few pay-for-performance programs use IT-centric measures, mostly focused on the ambulatory setting of care. 90 Unfortunately, the informatics measures used in these initiatives have not been validated, and are unrelated to the previously described EHR standards and certification efforts, or to the CDS Roadmap. 84 It is widely acknowledged that misaligned incentives represent one of the principle obstacles to progress in health care informatics investment, and the burgeoning pay-for-performance movement seems a natural place to begin to address this.

Recommendation 5

The AHRQ could fund contracts or grants to extend its current and ongoing work related to pay-for-performance to activities more focused on the use of informatics incentives in pay-for-performance programs. AHRQ could also create an enriched research agenda to develop models through which pay-for-performance programs would contain incentives for adoption of clinical informatics interventions that maximize impact on patient safety. The DHHS could mandate use of validated, established informatics outcomes measures in all public pay-for-performance programs, and encourage their inclusion in all private pay-for-performance programs as detailed in Recommendations 2, 3, and 4 above.

Opportunity 6: Utilize PHRs in Patient Safety Efforts

Although much safety-related effort has focused on the aggressive adoption of EHRs and the development of the National Health Information Network (NHIN), the role of PHRs in promoting patient safety has received less attention. This is curious, as it is now well accepted that patients comprise a “natural safety net” for their own care. Informed and involved patients, partnered with their health care providers, can provide a powerful boost to patient safety. The IOM's focus on patient-centeredness and the patient as a source of control reinforces the potential of PHRs for safety improvement. 12,28 Unfortunately, PHRs are not yet widely used, nor are there broad bases of research results available to guide their design, implementation, impact, evaluation, or sustainability. Given the potential importance of PHRs to patients and providers with regard to patient safety, a research agenda in this area is needed. It is impossible to envision a safe, clinical informatics-enabled health care system without patients being electronically connected to their care processes.

Recommendation 6

The AHRQ could fund safety-related research focusing on the use of PHRs, and their relationships to EHRs, regional health information organizations, health information exchanges, and the management of security and confidentiality. This research agenda might emphasize methods to measure the current impact of PHRs on consumer safety, on patients' use of health care services, on patients' role as a partner in their own care, and on their health outcomes. The research could investigate the challenges to and impact on outcomes of integration of PHRs with provider records. Such research could also create a road map of the potential for PHRs to improve patient safety and quality of care. In doing so, PHR-related efforts should identify the most successful approaches to consumer education about PHRs and the best strategies for successful implementation and funding of PHRs. In addition, this agenda could consider potential geographically diverse demonstration projects in this area.

Opportunity 7: Extend Interfaces between Clinical Informatics and Patient Safety

Health care lags far behind many other industries in the adoption of IT. When health care organizations first automated their financial systems in years past, neither the expense nor the risk seemed alarming, except perhaps around the Y2K debacle. Today, the financial side of health care operations resembles most other industries. Unfortunately, in the clinical arena, especially in small physician practices and in rural health care settings, providers in the United States continue to struggle to achieve effective, efficient, and safe integration of clinical informatics applications into improving care delivery. Although health care has been the focus of significant basic science research funding, comparatively little has been invested in studying the patient safety–clinical informatics interface. This must change. Significantly greater funding levels should be allocated for research in this area, as lack of safety-related knowledge about best practices can be potentially devastating. 91 The state of clinical informatics in the United States is particularly troubling in comparison with that in other countries. Many other developed economies in Europe, Australia, and Asia are moving forward much more quickly in their deployment of IT at the point of health care delivery. 14

Recommendation 7

The Congress and DHHS could direct AHRQ to pursue and to provide funding for an aggressive research agenda centered on the patient safety–clinical informatics interface. This research agenda could extend the knowledge that the AHRQ has already begun to generate in this area. Special emphases might include CDS, data standards, knowledge representation, personal health records, human factors–related design and interfaces, communications around transitions of care, improved detection of adverse events, and the use of informatics incentives in pay-for-performance programs. In addition, the research agenda could evaluate how to incorporate effective educational programs about clinical informatics into health care education at all levels. The educational initiatives should include a determination of how simulation can best be used in training and competency assessment of health care professionals. This research agenda might be directly linked to the NIH CTSA initiatives as well.

Conclusion

Although pioneering organizations have demonstrated the potential of clinical informatics approaches to enhance the safety of patient care, the implementation of these technologies across health care settings in the United States has been extremely limited. The future of health care informatics will depend upon many factors, foremost of which is likely the evolution of health care financing and delivery in the United States, and the role of the federal government in that model. The discipline of health care informatics remains relatively young and largely academically based, principally as a result of the modest market for health care informatics compared with other information-intensive industries. The authors believe that the adoption of the recommendations outlined above is essential to the realization of the potential of health care informatics to improve the safety of care across all settings.

Acknowledgments

The authors thank Don Detmer, MD, MA, and Meryl Bloomrosen, MBA, for their contributions to the conceptualization and review of this article. They also thank David Bates, MD (Brigham and Women's Hospital; Harvard Medical School; Harvard School of Public Health); Gil Kuperman, MD, PhD, (Director, Quality Informatics, New York-Presbyterian Hospital); Carol Cain, PhD, Kaiser Permanente's Care Management Institute; Jon White, MD, Agency for Health Care Research and Quality (AHRQ); and Teresa Zayas-Caban, PhD, Agency for Health Care Research and Quality (AHRQ); who served as reviewers and provided valuable insight into the topic.

Footnotes

Supported by Prime Contract No. 290-04-0016 - 6275-AMIA-01 between the American Medical Informatics Association (AMIA) under subcontract to the National Opinion Resource Corporate (NORC) and the Agency for Health Care Research and Quality (AHRQ), as part of the AHRQ National Resource Center (NRC). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and should not be construed as endorsement by AHRQ or the Department of Health and Human Services (DHHS) and do not necessarily reflect the views of the DHHS, AMIA, NRC, NORC or AHRQ. The authors of this report are responsible for its content.

References

  • 1.Kohn LT, Corrigan JM, Donaldson MS. To Err is Human: Building a Safer Health SystemWashington, DC: National City Press; 2000. [PubMed]
  • 2.Jick H. Drugs—remarkably nontoxic N Engl J Med 1974;291:824-828. [DOI] [PubMed] [Google Scholar]
  • 3.Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors JAMA 1998;280:1311-1316. [DOI] [PubMed] [Google Scholar]
  • 4.Bates DW, Gawande AA. Improving safety with information technology N Engl J Med 2003;348:2526-2534. [DOI] [PubMed] [Google Scholar]
  • 5.Bates DW. Using information technology to reduce rates of medication errors in hospitals BMJ 2000;320:788-791. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.McDonald CJ, Tierney WM, Overhage JM, Dexter PR, Takesue BY, Abernathy G. The Three Legged StoolIndianapolis, IN: Regenstrief Institute for Health Care; 1997.
  • 7.Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other antiinfective agents N Engl J Med 1998;338:232-238. [DOI] [PubMed] [Google Scholar]
  • 8.Aspden P, Corrigan JM, Wolcott J, Erickson SM. Patient Safety: Achieving a New Standard for CareWashington, DC: National Academies Press; 2004. [PubMed]
  • 9.Bates DW. Using information technology to improve surgical safety Br J Surg 2004;91:939-940. [DOI] [PubMed] [Google Scholar]
  • 10.McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man N Engl J Med 1976;295:1351-1355. [DOI] [PubMed] [Google Scholar]
  • 11. Keeping patients safe: Institute of Medicine looks at transforming nurses' work environment Qual Lett Healthc Lead 2004;16:9-11. [PubMed] [Google Scholar]
  • 12. Crossing the Quality Chasm: A New Health System for the 21st CenturyWashington, DC: National Academies Press; 2002. [PubMed]
  • 13.Hurtado M, Swift E, Corrigan JM. Envisioning the National Health Care Quality ReportWashington, DC: National Academies Press; 2001. [PubMed]
  • 14.In: Aspden P, Corrigan JM, editors. Patient Safety: Achieving a New Standard of Care. Washington, DC: National Academies Press; 2004. [PubMed]
  • 15.Bagian JP. Patient safety: What is really at issue? Front Health Serv Manage 2005;22:3-16. [PubMed] [Google Scholar]
  • 16.Petersen LA, Orav EJ, Teich JM, O'Neil AC, Brennan TA. Using a computerized sign-out program to improve continuity of inpatient care and prevent adverse events Jt Comm J Qual Improv 1998;24:77-87. [DOI] [PubMed] [Google Scholar]
  • 17.Sidlow R, Katz-Sidlow RJ. Using a computerized sign-out system to improve physician-nurse communication Jt Comm J Qual Patient Saf 2006;32:32-36. [DOI] [PubMed] [Google Scholar]
  • 18.Aiken LH, Clarke SP, Silber JH, Sloane D. Hospital nurse staffing, education, and patient mortality LDI Issue Brief 2003;9(2):1-4. [PubMed] [Google Scholar]
  • 19.Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational levels of hospital nurses and surgical patient mortality JAMA 2003;290:1617-1623. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Krohn R. How healthcare IT can address the nursing shortage J Healthc Inf Manag 2006;20(2):21-23. [PubMed] [Google Scholar]
  • 21.Rall M, Manser T, Guggenberger H, Gaba DM, Unertl K. [Patient safety and errors in medicine: development, prevention and analyses of incidents] Anasthesiol Intensivmed Notfallmed Schmerzther 2001;36:321-330. [DOI] [PubMed] [Google Scholar]
  • 22.Scalese RJ, Obeso VT, Issenberg SB. Simulation technology for skills training and competency assessment in medical education J Gen Intern Med 2008;23(Suppl 1):46-49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Brydges R, Kurahashi A, Brummer V, Satterthwaite L, Classen R, Dubrowski A. Developing criteria for proficiency-based training of surgical technical skills using simulation: Changes in performances as a function of training year J Am Coll Surg 2008;206:205-211. [DOI] [PubMed] [Google Scholar]
  • 24.Dillon GF, Boulet JR, Hawkins RE, Swanson DB. Simulations in the United States Medical Licensing Examination (USMLE) Qual Saf Health Care 2004;13(Suppl 1):i41-i45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Williams TL, Lemkau Jr. HL, Burrows S. The economics of academic health sciences libraries: cost recovery in the era of big science Bull Med Libr Assoc 1988;76:317-322. [PMC free article] [PubMed] [Google Scholar]
  • 26.Buckeridge DL, Goel V. Medical informatics in an undergraduate curriculum: a qualitative study BMC Med Inform Decis Mak 2002;2:6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Clarke JL, Meiris DC, Nash DB. Electronic personal health records come of age Am J Med Qual 2006;21(3 Suppl):5S-15S. [DOI] [PubMed] [Google Scholar]
  • 28.Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: Definitions, benefits, and strategies for overcoming barriers to adoption J Am Med Inform Assoc 2006;13:121-126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Ash JS, Bates DW. Factors and forces affecting EHR system adoption: report of a 2004 ACMI discussion J Am Med Inform Assoc 2005;12:8-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Middleton B, Hammond WE, Brennan PF, Cooper GF. Accelerating U.S. EHR adoption: How to get there from here. recommendations based on the 2004 ACMI retreat. J Am Med Inform Assoc 2005;12:13-19. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Berner ES, Detmer DE, Simborg D. Will the wave finally break?. A brief view of the adoption of electronic medical records in the United States. J Am Med Inform Assoc 2005;12:3-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Classen D, Avery J, Bates DW. Evaluation and Certification of Computerized Provider Order Entry Systems J Am Med Inform Assoc 2007;14:48-55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Middleton B. Achieving U.S. Health information technology adoption: the need for a third hand Health Aff (Millwood) 2005;24:1269-1272. [DOI] [PubMed] [Google Scholar]
  • 34.Halamka J, Aranow M, Ascenzo C, et al. Health care IT collaboration in Massachusetts: The experience of creating regional connectivity J Am Med Inform Assoc 2005;12:596-601. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Halamka J, Aranow M, Ascenzo C, et al. E-Prescribing collaboration in Massachusetts: Early experiences from regional prescribing projects J Am Med Inform Assoc 2006;13:239-244. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Classen DC, Kanhouwa M, Will D, Casper J, Lewin J, Walker J. The patient safety institute demonstration project: a model for implementing a local health information infrastructure J Healthc Inf Manag 2005;19:75-86. [PubMed] [Google Scholar]
  • 37.Frisse ME. State and community-based efforts to foster interoperability Health Aff (Millwood) 2005;24:1190-1196. [DOI] [PubMed] [Google Scholar]
  • 38.Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review JAMA 2005;293:1223-1238. [DOI] [PubMed] [Google Scholar]
  • 39.Kuperman GJ, Gibson RF. Computer physician order entry: benefits, costs, and issues Ann Intern Med 2003;139:31-39. [DOI] [PubMed] [Google Scholar]
  • 40.Cross RK, Finkelstein J. Feasibility and acceptance of a home telemanagement system in patients with inflammatory bowel disease: a 6-month pilot study Dig Dis Sci 2007;52:357-364. [DOI] [PubMed] [Google Scholar]
  • 41.Fritzsche D, Eitz T, Laczkovics A, et al. Early detection of mechanical valve dysfunction using a new home monitoring device Ann Thorac Surg 2007;83:542-548. [DOI] [PubMed] [Google Scholar]
  • 42.O'Connor CM, Velazquez EJ, Gardner LH, et al. Comparison of coronary artery bypass grafting versus medical therapy on long-term outcome in patients with ischemic cardiomyopathy (a 25-year experience from the Duke Cardiovascular Disease Databank) Am J Cardiol 2002;90:101-107. [DOI] [PubMed] [Google Scholar]
  • 43.Bruce B, Fries JF. The Arthritis, Rheumatism and Aging Medical Information System (ARAMIS): Still young at 30 years Clin Exp Rheumatol 2005;23(Suppl 39):S163-S167. [PubMed] [Google Scholar]
  • 44. Charting a Course for the 21st Century: NLM's Long Range Plan 2006-2016. Washington, DC: U.S. Department of Health and Human Services, National Institutes of Health; 2006.
  • 45.Haug C, Gotzsche PC, Schroeder TV. Registries and registration of clinical trials N Engl J Med 2005;353:2811-2812. [DOI] [PubMed] [Google Scholar]
  • 46.Zarin DA, Keselman A. Registering a clinical trial in ClinicalTrials.gov Chest 2007;131:909-912. [DOI] [PubMed] [Google Scholar]
  • 47.Steinbrook R. Public registration of clinical trials N Engl J Med 2004;351:315-317. [DOI] [PubMed] [Google Scholar]
  • 48.Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324:377-384. [DOI] [PubMed] [Google Scholar]
  • 49.Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients N Engl J Med 2001;345:965-970. [DOI] [PubMed] [Google Scholar]
  • 50.Dexter PR, Perkins SM, Maharry KS, Jones K, McDonald CJ. Inpatient computer-based standing orders vs physician reminders to increase influenza and pneumococcal vaccination rates: a randomized trial JAMA 2004;292:2366-2371. [DOI] [PubMed] [Google Scholar]
  • 51.Tierney WM, Overhage JM, Takesue BY, et al. Computerizing guidelines to improve care and patient outcomes: The example of heart failure J Am Med Inform Assoc 1995;2:316-322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Tierney WM, Overhage JM, Murray, MD, et al. Effects of computerized guidelines for managing heart disease in primary care J Gen Intern Med 2003;18:967-976. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Ozdas A, Speroff T, Waitman LR, Ozbolt J, Butler J, Miller RA. Integrating “best of care” protocols into clinicians' workflow via care provider order entry: Impact on quality-of-care indicators for acute myocardial infarction J Am Med Inform Assoc 2006;13:188-196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Celi LA, Hassan E, Marquardt C, Breslow M, Rosenfeld B. The eICU: It's not just telemedicine Crit Care Med 2001;29(Suppl):N183-N189. [DOI] [PubMed] [Google Scholar]
  • 55.Clemmer TP. Computers in the ICU: Where we started and where we are now J Crit Care 2004;19:201-207. [DOI] [PubMed] [Google Scholar]
  • 56.Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success BMJ 2005;330(7494):765. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Pryor TA, Gardner RM, Clayton PD, Warner HR. The HELP system J Med Syst 1983;7:87-102. [DOI] [PubMed] [Google Scholar]
  • 58.Barnett O. Computers in medicine JAMA 1990;263:2631-2633. [PubMed] [Google Scholar]
  • 59.Safran C, Porter D, Lightfoot J, et al. ClinQuery: A system for online searching of data in a teaching hospital Ann Intern Med 1989;111:751-756. [DOI] [PubMed] [Google Scholar]
  • 60.Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL. The incident reporting system does not detect adverse drug events: A problem for quality improvement Jt Comm J Qual Improv 1995;21:541-548. [DOI] [PubMed] [Google Scholar]
  • 61.Mekhjian HS, Bentley TD, Ahmad A, Marsh G. Development of a Web-based event reporting system in an academic environment J Am Med Inform Assoc 2004;11:11-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Oken A, Rasmussen, MD, Slagle JM, et al. A facilitated survey instrument captures significantly more anesthesia events than does traditional voluntary event reporting Anesthesiology 2007;107:909-922. [DOI] [PubMed] [Google Scholar]
  • 63.Nuckols TK, Bell DS, Liu H, Paddock SM, Hilborne LH. Rates and types of events reported to established incident reporting systems in two US hospitals Qual Saf Health Care 2007;16:164-168. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Harris CB, Krauss MJ, Coopersmith CM, et al. Patient safety event reporting in critical care: A study of three intensive care units Crit Care Med 2007;35:1068-1076. [DOI] [PubMed] [Google Scholar]
  • 65.Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients JAMA 1991;266:2847-2851. [PubMed] [Google Scholar]
  • 66.Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report J Am Med Inform Assoc 1998;5:305-314. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Kilbridge PM, Campbell UC, Cozart HB, Mojarrad MG. Automated surveillance for adverse drug events at a community hospital and an academic medical center J Am Med Inform Assoc 2006;13:372-377. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Kilbridge PM, Classen DC. Automated surveillance for adverse events in hospitalized patients: back to the future Qual Saf Health Care 2006;15:148-149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Kilbridge P. Implementation of a system for computerized adverse drug event surveillance and intervention at an academic medical center J Clin Outcomes 2006;13:94-100. [Google Scholar]
  • 70.Honigman B, Light P, Pulling RM, Bates DW. A computerized method for identifying incidents associated with adverse drug events in outpatients Int J Med Inform 2001;61:21-32. [DOI] [PubMed] [Google Scholar]
  • 71.Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting JAMA 2003;289:1107-1116. [DOI] [PubMed] [Google Scholar]
  • 72.Melton GB, Hripcsak G. Automated detection of adverse events using natural language processing of discharge summaries J Am Med Inform Assoc 2005;12:448-457. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Hammond WE. The role of standards in electronic prescribing Health Aff (Millwood) 2004. Suppl Web Exclusives:W4-325-7. [DOI] [PubMed]
  • 74.The Commission on Systemic Interoperabilityhttp://endingthedocumentgame.gov 2004. Accessed June 2008.
  • 75.Brooks RG, Menachemi N, Burke D, Clawson A. Patient safety-related information technology utilization in urban and rural hospitals J Med Syst 2005;29:103-109. [DOI] [PubMed] [Google Scholar]
  • 76.Menachemi N, Burke D, Clawson A, Brooks RG. Information technologies in Florida's rural hospitals: Does system affiliation matter? J Rural Health 2005;21:263-268. [DOI] [PubMed] [Google Scholar]
  • 77.Ohsfeldt RL, Ward MM, Schneider JE, et al. Implementation of hospital computerized physician order entry systems in a rural state: Feasibility and financial impact J Am Med Inform Assoc 2005;12:20-27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Yasnoff WA, Humphreys BL, Overhage JM, et al. A consensus action agenda for achieving the national health information infrastructure J Am Med Inform Assoc 2004;11:332-338. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B. The value of health care information exchange and interoperability Health Aff (Millwood) 2005. Suppl Web Exclusives:W5-10-W5-8. [DOI] [PubMed]
  • 80.Walker JM. Electronic medical records and health care transformation Health Aff (Millwood) 2005;24:1118-1120. [DOI] [PubMed] [Google Scholar]
  • 81.Teich JM, Osheroff JA, Pifer EA, Sittig DF, Jenders RA. Clinical decision support in electronic prescribing: recommendations and an action plan: Report of the joint clinical decision support workgroup J Am Med Inform Assoc 2005;12:365-376. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Shah NR, Seger AC, Seger DL, et al. Improving acceptance of computerized prescribing alerts in ambulatory care J Am Med Inform Assoc 2006;13:5-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83.Miller RA, Gardner RM, Johnson KB, Hripcsak G. Clinical decision support and electronic prescribing systems: A time for responsible thought and action J Am Med Inform Assoc 2005;12:403-409. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 84.Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support J Am Med Inform Assoc 2007;14:141-145. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 85.Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors JAMA 2005;293:1197-1203. [DOI] [PubMed] [Google Scholar]
  • 86.Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system Pediatrics 2005;116:1506-1512. [DOI] [PubMed] [Google Scholar]
  • 87.Kilbridge PM, Welebob EM, Classen DC. Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems Qual Saf Health Care 2006;15:81-84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.Miller RA, Gardner RM. Summary recommendations for responsible monitoring and regulation of clinical software systems. American Medical Informatics Association, The Computer-based Patient Record Institute, The Medical Library Association, The Association of Academic Health Science Libraries, The American Health Information Management Association, and The American Nurses Association. Ann Intern Med 1997;127:842-845. [DOI] [PubMed] [Google Scholar]
  • 89.Miller RA, Gardner RM. Recommendations for responsible monitoring and regulation of clinical software systems. American Medical Informatics Association, Computer-based Patient Record Institute, Medical Library Association, Association of Academic Health Science Libraries, American Health Information Management Association, American Nurses Association. J Am Med Inform Assoc 1997;4:442-457. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 90. Performance measurement: accelerating improvement. . Institute of Medicine. Washington, DC: National Academy Press; 2006.
  • 91. 2004 ASHP Leadership Conference on Pharmacy Practice Management Executive Summary: improving patient care and medication safety Am J Health Syst Pharm 2005;62:1303-1310. [DOI] [PubMed] [Google Scholar]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press

RESOURCES