What makes a good clinical decision support system

Clinical decision support is the provision of “clinical knowledge and patient-related information, intelligently filtered or presented at appropriate times, to enhance patient care.”¹ Medical institutions are increasingly adopting tools that offer decision support to improve patient outcomes and reduce errors. Healthcare providers and administrators with little or no training in computer science may be asked to evaluate, select, or contribute to the development of decision support systems for their practices. Is there an easy way to determine which clinical decision support systems are good?

In this issue Kawamoto and colleagues provide some evidence based guidance in a systematic analysis of the ability of decision support systems to improve practice in both statistically significant and clinically meaningful ways (p 765).² This rigorous review includes only randomised controlled trials and excludes small studies that do not meet 50% of established criteria for methodological quality.³,⁴ It identifies four independent predictors of effective decision support: systems that enhance practice generate decision support automatically as part of the normal clinical workflow and at the time and place of decision making; they use computers to deliver support; and they offer specific recommendations rather than mere assessments. Ninety four per cent of clinical decision support systems with these characteristics improved practice compared with only 46% of systems that lack one of these features.

Similar findings were reported in a recent systematic review of controlled trials evaluating computerised decision support programs, but worrying deficiencies in the evidence base were noted.⁵ Garg and colleagues found that the performance of healthcare practitioners using decision support systems improved in 64% of studies, comparable to the improvement in 68% of trials noted by Kawamoto et al,² and they also observed that automatically generated versus user-initiated decision support resulted in better delivery of care. However, of the 100 studies analysed, few specified a primary outcome for statistical analysis, and nearly three quarters were evaluated by their software developers. Developer self-assessment was the only other factor associated with better performance. The outcomes of most studies were metrics assessing the process of healthcare delivery with and without decision support systems. Only 52 trials measured at least one patient outcome, and improvements were noted in only 13% of these studies.

Unfortunately, the implementation of effective clinical decision support is a challenging task involving interactions between technologies and organisations, and there are no easy solutions to guarantee success or to avoid failure in this complex process.⁶ Many factors influence reductions in errors or improvements in health, so measuring the effectiveness of decision support systems in improving these endpoints is difficult. Moreover, another recent eye opening observational study identified 22 different ways in which an established computerised order entry system (the benefits of which are thought to include reducing errors) could actually introduce medication errors.⁷ Although many researchers have sought to prove the advantages of clinical decision support, few have carefully studied sources of harm. Clearly defining the balance between the risks and benefits of clinical decision support is a continuing challenge.

Finally, a clinical decision support system is only as effective as its underlying knowledge base, which changes rapidly as medical science evolves. Sim and colleagues have proposed that the next generation of clinical decision support systems should be not only evidence based, but also “evidence adaptive,” with automated and continuous updating to reflect the most recent advances in clinical science and local practice knowledge.⁸ Flexibility in incorporating information from diverse sources and adaptability to varied practice settings are likely to be the quality criteria by which decision support systems are judged in the future.