Recommendations for Clinical Decision Support Deployment: Synthesis of a Roundtable of Medical Directors of Information Systems

Abstract

Background: Ample evidence exists that clinical decision support (CDS) can improve clinician performance. Nevertheless, additional evidence demonstrates that clinicians still do not perform adequately in many instances. This suggests an ongoing need for implementation of CDS, in turn prompting development of a roadmap for national action regarding CDS.

Objective: Develop practical advice to aid CDS implementation in order to improve clinician performance. Method: Structured group interview during a roundtable discussion by medical directors of information systems (N = 30), with subsequent review by participants and synthesis.

Results: Participant consensus was that CDS should be comprehensive and should involve techniques such as order sets and facilitated documentation as well as alerts; should be subject to ongoing feedback; and should flow from and be governed by an organization’s clinical goals.

Conclusion: A structured roundtable discussion of clinicians experienced in health information technology can yield practical, consensus advice for implementation of CDS.

INTRODUCTION

Need for Clinical Decision Support

Substantial evidence exists that clinicians in many instances do not perform adequately in the course of patient care. For example, considerable delay often occurs between confirmation of a clinically relevant research finding in the medical literature and the incorporation of that finding into widespread clinical practice [1]. Indeed, on average, only 54.9% of adults in the USA receive recommended care for common clinical conditions [2].

In part to address findings such as these, implementation of health information technology has been recommended generally in order to improve the quality of clinical care through the reduction of errors [3]. Indeed, a recent systematic review of the literature documented a number of benefits that flow from such implementation, including improved guideline adherence, better surveillance and decreased medication errors [4]. In particular, other work that examined the effect of implementation of clinical decision support (CDS) has found improvement in preventive care, drug dosing and rendering diagnoses [5].

Facilitating Clinical Decision Support

Despite such success, use of CDS in health care organizations in the USA is relatively limited. Computerized provider order entry (CPOE) systems offer the opportunity to implement a number of kinds of CDS, including alerts and reminders as well as evidence-based order sets. Nevertheless, a recent survey documented that CPOE systems had been implemented in fewer than ten percent of U.S. hospitals [6].

Data such as these, which demonstrate an ongoing need for CDS even as they highlight a failure to implement the CDS technology that could improve clinical performance, have led to efforts to increase the uptake of CDS systems. Toward this end, the authors have participated in an effort to develop a roadmap for national action on CDS, published in 2006. This plan emphasizes three pillars of strategy to realize the promise of CDS: making the best knowledge available when needed; promoting adoption and use of CDS; and facilitating continuous improvement of knowledge about both clinical medicine and CDS implementation [7].

Anticipating such calls to facilitate CDS and provide information about best practices to health care organizations for implementation of CDS, some workers have endeavored to provide practical advice based on a synthesis of the literature and expert opinion. Bates and colleagues in 2003 identified ten general guidelines for implementing CDS based on their synthesis of the research literature. These recommendations included the provision of real-time decision support comprised of simple interventions that fit into the clinician’s workflow [8]. Nevertheless, while providing general tips for and examples of CDS implementation, this work did not formulate detailed guidance to enable such implementation.

The authors themselves in prior work have addressed the gap between current and optimal CDS implementation. This particular synthesis yielded a six-step process that health care organizations might use to implement CDS, beginning with identifying stakeholder goals and available clinical systems and continuing through selecting CDS interventions, building them, deploying them and analyzing their effect [9]. In turn, the analysis of effect is fed back into the process in order to refine interventions.

Goals of the Analysis

In order to facilitate further the realization of the benefits of CDS as stressed by the national roadmap, we sought to elicit and synthesize the experience of clinicians responsible for the implementation of health information technology in their own organizations in the hope that others may benefit from that experience. This complements and advances related past efforts by including the experiences and practical lessons of clinicians directly responsible for CDS deployment in a number of organizations.

METHODS

Setting

The authors organized a roundtable discussion at the annual Physician-Computer Connection conference of the Association of Medical Directors of Information Systems (AMDIS). AMDIS is a professional organization consisting of chief medical information officers (CMIO) and other physicians who participate actively in the deployment of health information technology in their local organizations. Indeed, a recent profile of the CMIO role, achieved using a structured interview technique, found the CMIO to be a key physician executive that participates heavily in the management of the health care organization through the application of informatics knowledge [10]. Thus, a better understanding of the CMIO perspective on CDS may shed light on some of the most important practical drivers and barriers to CDS acceptance and effectiveness, leading to better design and implementation strategies addressing those drivers and barriers.

Data Acquisition

A structured set of questions to initiate discussion was developed, grouped around four broad themes: governance, implementation, knowledge management and evaluation. The governance theme included four questions, addressing the practical aspects of establishing an infrastructure for CDS deployment; the goals and strategy for such deployment; and establishment of an organizational culture that supports CDS. The implementation theme included ten questions. These addressed methods for determining best practices; avoiding alert fatigue and unintended adverse consequences of CDS; achieving consensus on CDS content; delineating CDS needs; identifying successful examples of CDS, especially stressing patient safety; and integrating CDS with personal health records. The knowledge management theme included three questions, addressing assembling an inventory of knowledge assets; maintaining those assets; and availability of external resources. Finally, the evaluation theme included two questions, addressing metrics for determining CDS success and the benchmarks for those metrics.

After an initial overview of the national roadmap and contemporary themes in CDS, the authors used the structured questions in succession to elicit ideas and additional discussion from participants. Part of the purpose of the discussion was to identify consensus among participants that might be distilled into lessons for facilitating CDS deployment. Ideas expressed and consensus so achieved were recorded contemporaneously and later synthesized around two key themes: CDS implementation (subsuming the additional themes of knowledge management and evaluation) and governance. This synthesis was circulated to AMDIS members through the organizational listserve for additional review and comment.

RESULTS

Participants and Their Goals

The roundtable included 30 CMIOs participating in a structured discussion over a 3-hour period. The majority of participants represented community hospitals and health systems comprised of such hospitals, while fewer participants represented academic medical centers. In addition, a few attendees represented vendors, and a few others represented government agencies.

Before the structured questions were introduced, the CMIOs were asked what they would most want to know from a resource that offered practical advice regarding CDS deployment. These included a desire to understand the following:

• how to organize cross-discipline cooperation in developing CDS;

• how to deploy CDS in ways that avoided negative consequences such as alert fatigue;

• skill sets and educational interventions that would allow creation of a CDS deployment team;

• governance techniques that would facilitate CDS implementation;

• techniques for effective knowledge management;

• the role of standards in CDS;

• use of physiologic monitoring as part of CDS.

CDS Governance

Committee structure

Approximately 50% of participants report that their organizations have a committee that focuses specifically on CDS deployment. Approximately half of these include CDS activity as part of overall quality improvement (QI) efforts. The majority of participants report distinct CDS and QI committees, although most of these committees feature overlapping memberships. All participants report that CDS efforts are grounded in clinical departments and not in the information technology department.

Even for those organizations with a distinct CDS committee, however, almost none arranges for that committee to report to a medical executive committee or senior physician governing body that might approve CDS interventions. Indeed, most organizations report that implementation of CDS interventions such as alerts and order sets requires the consultation and approval of multiple committees. Some organizations have created a project management office, with representation from nursing and other clinical departments and led by a chief information officer, to carry out CDS deployment. In still other organizations, CDS is folded into a general focus on safety, in which a chief medical quality officer uses information technology as one strategy to support safety goals.

CDS Accountability

Regardless of the CDS governance model used, the consensus of participants was that accountability was key. All participants agreed that identifying personnel who were held responsible for deploying CDS was vital in order to realize the benefits of CDS. Another consensus of participants was that evaluation and feedback data regarding the outcome of CDS initiatives are a key part of establishing such accountability. However, some organizations report difficulty in obtaining such data, in part because of a lack of integrated data warehouses, easy-to-use query tools and inadequate coding of data.

Incentive to participate

Organizations reported a variety of mechanisms for encouraging participation in the CDS governance structure. Most organizations pay clinicians to participate in a CDS committee or other governance organization. Of those organizations that do not pay for such participation, non-monetary compensation typically is provided, sometimes by explicit recognition of service as part of the promotion process or by reduction in required clinical practice volume. Some organizations, such as government agencies, report that they are unable to provide additional remuneration for CDS service because of regulatory restrictions.

CDS Implementation

In general, participants felt that the current state of CDS was inadequate to meet their organizational goals. This inadequacy included both the knowledge interventions that could be delivered by CDS systems and the tools available to customize and maintain the knowledge in those interventions. After solicitation of possible improvements to CDS, participants were asked to choose the single most important among the most commonly voiced suggestions. The most favored suggestions were the following:

• System vendors should provide better mechanisms for delivering clinical knowledge (10 participants).

• System vendors should build better tools to manage and filter knowledge bases (6 participants).

• Content vendors should provide better knowledge bases (4 participants).

Range of CDS intervention types

The consensus among participants using presently available systems and knowledge bases was that CDS is much more than just alerts that apprise clinicians of possible errors. Indeed, participants emphasized the diversity of possible decision support interventions, including order sets augmented by guides to their use and checklists for electronic documentation instead of simply displaying alerts as critiques following order composition. Some participants further stressed that an important part of any CDS scheme is feedback to physicians regarding their performance.

Alert fatigue and organizational goals

Nevertheless, a concern emphasized by most participants focused on the adverse impact of alert fatigue—overloading clinicians with so many alerts that it interferes with their workflow and potentially has adverse effects on patient care. The consensus among participants to address this concern was to introduce alerts, particularly those related to drug-drug interactions (DDI), highly selectively. In concert with the consensus under the governance theme that CDS must flow from the clinical goals of an organization, many participants suggested that the focus should be on the important clinical challenges faced by an organization, with technology such as alerts for DDI then aligned with these goals. A minority of participants pointed out that, although alert fatigue is a frequently cited concern by physicians, many of their physicians still report that they want alerting to continue, even in the face of high override rates.

In further addressing the challenge of alert fatigue, some organizations reported that physicians override alerts at a very high rate—as high as 90% for DDI. This has led them to cull DDI rules to a very small number, e.g., 40 different drug-drug combinations. Some organizations have found that, even when setting alert thresholds at a relatively high level of likelihood and potential severity, they still have a high override rate, further emphasizing the need to focus on a few, highly clinically relevant alerts. Others have addressed this challenge by activating alerts for some workers (e.g., pharmacists) and not for others (e.g., physicians). A general consensus of all participants in this regard is that both knowledge base (content) and electronic health record system vendors need to adjust their tools in order to facilitate such fine-grained manipulation of alert-based CDS.

Diagnostic decision support

A final topic of discussion raised by participants related to the use of diagnostic decision support. A majority of participants believe that this has lagged behind other forms of CDS because, in general, physicians do not identify a strong need for it and because it represents a very challenging pattern-recognition problem that requires large amounts of clinical data to solve. Another explanation cited by participants was that current diagnostic decision support was not sufficiently selective. Indeed, telling physicians what they already know is deemed a waste of their time, leading some participants to call for improved techniques for measuring the accuracy of such knowledge-based interventions.

DISCUSSION

The present analysis examines the experience of a group of physicians actively responsible for supervising quality initiatives and deploying CDS. The consensus among this convenience sample was that, while CDS is sorely needed in order to improve clinical care and patient safety, currently available technology is inadequate to satisfy this need. Roundtable participants identified several reasons for this inadequacy. These included lack of tools for incorporating and manipulating clinical knowledge in information systems, which in turn can lead to such adverse phenomena as alert fatigue. In the long run, development of standards that facilitate knowledge sharing and customization and tools that incorporate these standards may address these concerns.

Regardless of technology issues, the clear message of the roundtable participants is that CDS is not only a challenge of information technology but also one of governance, workflow and organizational processes. This is reflected in other work analyzing the skill set of CMIOs that noted the pre-eminence of management skills over technical knowledge [10]. Increasingly, CDS is seen as a set of techniques to fulfill the strategic goals of an organization and to fulfill quality improvement initiatives instead of something driven simply by the availability of information technology. Accordingly, the process of deploying CDS must be grounded in sound governance that involves all stakeholders and matches technology to an organization’s clinical goals [9].

One potential limitation of the present analysis is the convenience sampling that produced the focus group from which the data are derived. While it is possible that this group may not be entirely representative of all organizations striving to deploy CDS, the experience of these leaders as active implementers of this technology remains a source of valuable lessons for those wishing to leverage CDS to improve clinical care.

FUTURE WORK

Work to collect, synthesize and disseminate best practices with regard to CDS continues in several efforts. The authors are striving to create a CDS community facilitated through several organizations and programs, including AMIA, HIMSS, AMDIS, the Scottsdale Institute and the national Doctor’s Office Quality--Information Technology (DOQ-IT) program. The hope is that organizations and clinicians will share CDS content, strategies and best practices so that all may more easily achieve and maintain good clinical care through continuously improving decision support. A key part of such efforts will be standards to facilitate capture, sharing, maintenance and computer-based processing of executable knowledge, and work on standardization is ongoing in Health Level Seven and other organizations [11]. This must be coupled with broad efforts to implement a coordinated plan to ensure optimal use and usefulness of CDS [7].

SUMMARY

Suboptimal performance by clinicians and health care organizations suggests an unmet need for CDS, yet relatively few organizations have implemented CDS in a manner that fully leads to desired improvements in patient safety and clinician performance. Other work has distilled key lessons from expert opinion and the literature regarding deployment of CDS. The present work extends this literature by synthesizing, using a structured instrument and facilitated discussion, the experience of a group of medical directors of information systems in deploying CDS. These lessons coalesce around two key themes: governance and implementation. With regard to governance, CDS committees are a popular mechanism for organizing deployment efforts, and these almost always are grounded in the clinical enterprise and not the information technology department. With regard to implementation, the key lesson is that CDS must be grounded in the overall clinical goals of the organization and not in information technology per se. Moreover, CDS implementation should be highly selective in order to achieve these goals without leading to clinician fatigue or interference with workflow.