Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support

Abstract

Objective

Provider acceptance and associated patient outcomes are widely discussed in the evaluation of clinical decision support systems (CDSSs), but critical design criteria for tools have generally been overlooked. The objective of this work is to inform electronic health record alert optimization and clinical practice workflow by identifying, compiling, and reporting design recommendations for CDSS to support the efficient, effective, and timely delivery of high-quality care.

Material and Methods

A narrative review was conducted from 2000 to 2016 in PubMed and The Journal of Human Factors and Ergonomics Society to identify papers that discussed/recommended design features of CDSSs that are associated with the success of these systems.

Results

Fourteen papers were included as meeting the criteria and were found to have a total of 42 unique recommendations; 11 were classified as interface features, 10 as information features, and 21 as interaction features.

Discussion

Features are defined and described, providing actionable guidance that can be applied to CDSS development and policy. To our knowledge, no reviews have been completed that discuss/recommend design features of CDSS at this scale, and thus we found that this was important for the body of literature. The recommendations identified in this narrative review will help to optimize design, organization, management, presentation, and utilization of information through presentation, content, and function. The designation of 3 categories (interface, information, and interaction) should be further evaluated to determine the critical importance of the categories. Future work will determine how to prioritize them with limited resources for designers and developers in order to maximize the clinical utility of CDSS.

Conclusion

This review will expand the field of knowledge and provide a novel organization structure to identify key recommendations for CDSS.

OBJECTIVE

The objective of this work is to inform electronic health record (EHR) alert optimization and clinical practice workflow by identifying, compiling, and reporting design recommendations for clinical decision support systems (CDSSs) to support the efficient, effective, and timely delivery of high-quality care.

BACKGROUND AND SIGNIFICANCE

Provider acceptance and associated patient outcomes are widely discussed in the evaluation of CDSSs, but critical design criteria for tools have generally been overlooked. CDSSs, at the simplest level, are tools to help clinicians and patients make better informed decisions during use of the EHR. In the best examples, successful CDSSs reduce medical errors,¹ increase health care quality and efficiency,² and guide appropriate care decisions.^3–5 These are challenging tasks, and thus successful CDSSs remain difficult to develop and implement.⁶ Indeed, a significant amount of data indicates there are several unintended consequences associated with a high alert burden.^7–9 Unsystematic alert management can lead to nonadherence, high override rates, and “alert fatigue,” in which clinicians neglect CDSS and other alerts, thereby diminishing their effectiveness and potential benefits. As a result, CDSS developers and users aspire to improve alert management to achieve better acceptance rates and improved care delivery.

A CDSS is defined as “software that is designed to be a direct aid to clinical decision-making, in which the characteristics of an individual patient are matched to a computerized clinical knowledge base and patient-specific assessments or recommendations are then presented to the clinician for a decision.”¹⁰ CDSSs have the potential to enable clinicians to better address rising information needs, providing the opportunity to pick up on subtle early indications of risk or vulnerability while sorting through an avalanche of data. The availability of evidence-based guidelines for clinical care and for CDSS implementation encourages providers to deliver the best evidence-based care available.

Having patient demographic and health information in a medium that affords sophisticated and rapid analysis to identify tailored treatment solutions, and being able to deliver this information to the provider at the point of care, can transform treatment. Unfortunately, many EHRs suffer from poor usability – providers struggle to use these systems efficiently, effectively, and safely. While underlying models and algorithms of CDSSs have been intensively studied,^11–16 there remains a lack of evidence-based guidelines in terms of functional and design requirements of the system. When implemented with critical design limitations, there is the potential for difficulty in routine use and mistrust of the recommendations, leading to reduced use and increased provider frustration.^6,17,18 These issues limit the potential utility of even well-designed predictive tools. In order to promote the use of CDSSs by providers, we need to determine the best way to design and present information via the user interface. The value of this synthesis is to compile a set of recommendations for CDSS design to better define functional characteristics that impact how the system looks and performs, with the ultimate goal of supporting clinical decision-making. Evaluating the design of these systems from a human factors engineering perspective can result in improved interactions between clinicians and the system, with the potential to eventually improve clinician acceptance and patient outcomes.

MATERIALS AND METHODS

Data sources and searches

This narrative review was completed using a predefined protocol. Our research team searched PubMed and peer-reviewed manuscripts in The Journal of the Human Factors and Ergonomics Society (January 1, 2000, to December 31, 2016) to capture articles specific to clinical design making in health care as well as articles that showcase design features of CDSS and recommendations of these features. The following Medical Subject Headings terms were used for searching publications: “clinical decision support systems” OR “decision support systems” OR “clinical alerts” OR “alert systems” AND “characteristics” OR “features” OR “human factors” OR “usability heuristics” OR “usability factors” OR “recommendation.”

We designed our inclusion/exclusion criteria to identify papers that discuss/recommend design features of CDSSs that have been found to be associated with the success of these systems. Inclusion criteria were as follows: (1) English language, (2) EHR-based, (3) provide specific characteristics in terms of CDSS design, and (4) define success factors used to evaluate CDSS performance. Success factors were defined as “effective” with respect to the practitioner’s perspective (ie, provider usability; provider satisfaction; efficacy evaluation including process of care; compliance with guidelines; knowledge, attitudes, skills; and fit of workflow and cognitive processes) or improved patient outcomes (ie, morbidity or mortality, surrogate outcomes, and indicators of resource use). Exclusion criteria ruled out papers that discuss features without assessment of (1) performance or (2) success factors. Within the included papers, the authors must describe the design of the CDSS, providing key elements. Recommendations had to represent consensus, defined as recommendations identified in a systematic review and/or a design recommendation made by more than one paper (referenced within the article) demonstrating success factors. This process required reviewing all included manuscripts to identify recommendations and considering these recommendations in aggregate.

Consensus and evaluation

Two members of the research team (DM, YN) were responsible for the initial inclusion/exclusion of articles. The entire research team was then responsible for reviewing the final selection of manuscripts. To evaluate the papers, several of the authors read them individually and convened with the group (KM, DM, YN, MC, RK, and RK) to discuss the different opinions until we achieved agreement about the recommendation and its inclusion. To achieve consensus, every identified recommendation was reviewed and the review team discussed relevance and repetition to determine recommendation for inclusion. Recommendations were then organized in a meaningful way through categorization and grouping based on function, content, and design. Similar recommendations were unified into a final set of recommendations and then grouped into 3 main “I” categories: interface, information, and interaction. Interface (presentation) features were further organized into 4 “P” categories: presentation, placement and positioning, and provision of multiple presentation layers. Information (content) features were further organized into 4 “C” categories: clean and concise, content guidance, and consistency. Interaction (function) features were further organized into 5 “F” categories: fast, fit, feedback, forgiveness, and flexible design. The below summary includes recommendations identified from the 14 included papers and references to the original published work that led to the consensus recommendation.

RESULTS

Included manuscripts

The literature search identified a total of 387 unique results, of which 281 were dismissed after abstract review and 79 were dismissed after complete manuscript review. Finally, a total of 27 papers were selected as relevant to our project. Through the consensus process, 14 papers were included as meeting the criteria (Supplementary Table S1). Regarding the types of papers found, several were obtained. We identified 9 literature reviews of recommended CDSS design elements,^19–27 3 evaluations of a single specific successful CDSS,^28–30 1 comparison of 2 CDSS interfaces,³¹ and 1 focus group working to design a CDSS.³² Although The Journal of Human Factors and Ergonomics Society was searched, no relevant articles were considered appropriate for this narrative review.

Outcomes representing success

The results and effectiveness of these CDSSs were evaluated using 2 metrics: improved significance and improved outcomes of care. Improved significance (disregarding the importance of outcomes or the magnitude of the improvement) included:

• Provider usability^20,25,29

• Provider satisfaction^{20,25,27,29–32}

• Evaluating efficacy, including process of care (eg, diagnosis, treatment, monitoring)^19,21,22,24

• Compliance with guidelines^21,31

• Knowledge, attitudes, skills²²

• Fit of workflow and cognitive processes^20,28

Improved outcomes of care included:

• Morbidity or mortality (eg, blood pressure, clinical events, quality of life)^19,21,22

• Surrogate outcomes (eg, time taken to achieve a stable therapeutic dose)^21,22,27

• Indicators of resource use (eg, duration of hospital stay)²²

Design recommendations

A total of 42 unique recommendations were identified in the selected 14 papers. From these unique recommendations, we grouped features into 3 systematized categories: 11 were classified as interface features (Box 1), 10 as information features (Box 2), and 21 as interaction features (Box 3).

Box 1.

Interface features categorized as presentation, placement, positioning, and provision of multiple presentation layers

Interface (Presentation)

Presentation

Make it simple

Use appropriate font sizes

Use meaningful colors

Ensure acceptable contrast between text and background

Keep presentation consistent

Deploy space-filling techniques

Make icons bold or bigger in size

Placement and positioning

Display information in prominent positions to ensure that it is  seen

Allow for reading left to right

Localize information

Provision of multiple presentation layers

Avoid using only text

Box 2.

Information features categorized as clean and concise, content guidance, and consistency

Information (Content)

Clean and concise

Standardize terminology

Use concise and effective language

Content guidance

Provide a recommendation, not an assessment

Justify recommendations

Suggest alternative recommendations

Provide additional resources

Make evidence-based recommendations the default

Keep recommendations up to date

Consistency

Recommendations should come from the same place

Have the same display of basic CDSS for all members of the health  care team

Box 3.

Interaction features categorized as fast, fit, feedback, forgiveness, and flexible design

Interaction (Function)

Fast

Provide timely feedback

Reduce the amount of time the user is required to interact with the  CDSS

Fit

Minimize cognitive load (reduce the number of mouse clicks and  amount of free-text typing; use selection tools, sort options)

Minimize cognitive load (request information from the provider only when necessary; reduce manual input of values)

Reduce screens to facilitate navigation and to promote efficient  interactions

Automatically pull data from the EHR/integrate into the charting  system

Navigate to appropriate locations

Initiate intervention and take advantage of interactivity (system provides corollary action)

Provide a route to get to provider-specific info

Adapt its behavior according to a subset of relevant actions taken by clinicians

Incorporate functions supporting the dialog between the CDSS and the clinician

Feedback

Provide decision support automatically as part of clinician workflow

Automate alerting

Request documentation of reasons for not following system  recommendations

Forgiveness

Allow the user to be able to modify orders

Integrate a reset button

Flexible design

Involve the patient

Utilize adaptive design and feedback

Provide an indication for all professionals of the availability of  information; the designers may choose the most appropriate  way of indicating the information in the interface

Incorporate functions to support team awareness about alert  management and its evolution over time (eg, visible access to  how the alert was handled and the reasons for alert override  or rule deactivation if any has been documented)

Give access upon request to extended information (eg, justification  for the rule, attached scientific documentation) that should be  structured depending on the user profile

Interface features

Presentation

Presentation describes aesthetic appeal and integrity of design. One recommendation is to make it simple.^{17,20,25,27,31} Words like “easy” and “simple” are recommended frequently with^20,31 and without^25,27 specific guidance. Simplicity includes only the elements that are most important for communication. Techniques used to promote simplicity include consistent terminology, concise and unambiguous language, effective visualization, improved readability, and reduced density of information.^31,33–36 Specific strategies include using “at-a-glance” interface design^31,37 and using reduced and concise words.^20,31 Readability is achieved when the display is easy to identify and interpret through an appealing and attractive presentation. To improve readability, it is suggested to consider the use of appropriate font sizes,^31,33use meaningful colors,^31,33,34ensure acceptable contrast between text and background,^31,33 and make icons bold or bigger in size.²⁹Space-filling techniques help maximize the amount of information that can displayed in the available display space.^31,38,39 Visibility factors take into account human factors and cognitive informatics.

Placement and positioning

Placement and positioning refers to the optimal location of data elements within the design of the user interface. CDSS developers should display information in prominent positions to ensure that it is seen.^23,31,35,40 To meet cultural standards and address natural mapping, the system should allow for reading left to right.^41–43 With regard to positioning, localizing information³¹ groups pieces of information together to facilitate on-screen searches.^34,35,40 Techniques to strategize positioning include “at a glance,” “at hand,” and “all in one.”³⁰

Provision of multiple presentation layers

The presentation layer represents the interface between users and the rest of the application. If users cannot interact with the application in a way that lets them perform their work in an efficient and effective manner, then the overall success of the application will be severely impaired. It is recommended to avoid using only text^20,29,31 and use tables,⁴⁴ graphs,⁴⁴ buttons,³³ scroll bars,³³ and iconic language^45–47 to ensure that the density of information is appropriate. Reducing text density makes the CDSS easier to interpret during busy clinician encounters and keeps the attention of the provider. Additionally, information-oriented, systematic graphic design helps providers understand complex information.

Information features

Clean and concise

All components should be designed so their meaning is not ambiguous. Information should demonstrate clarity by being presented as clean and concise. It is recommended to standardize terminology. Designers should use concise and effective language based on a consistent terminology to promote simplicity, consistency, efficient interactions, and effective use of language.^{31,34–36,40} The most important elements should be easily perceived. Noncritical elements should be minimized so as not to hide critical information.

Content guidance

A CDSS encompasses a variety of tools, including clinical guidelines, diagnostic support, computerized alerts and reminders, and contextually relevant reference information. An effective CDSS must be relevant to those who can act on the information, and the design of recommendations within a CDSS plays an essential role. A CDSS should provide a recommendation, not solely an assessment,^{19,21,25,32,48,49} and explain and justify the recommendations and their source^28,31,34,50 by providing reasons^10,25,51 and research evidence,^10,25,41 particularly for systems that require a reason for overriding advice.^17,19,21,23 It should allow for the provision of advice, suggestions, and alternative recommendations to increase compliance and to respect the autonomy of the physician.^{17,21,23,31,34} It should provide additional resources or access to additional knowledge when needed,²⁹ including justification of recommendations and rules, and scientific documentation structured depending on the user.²⁸ It should make evidence-based recommendations the default^17,21,23 and keep recommendations up to date.²¹

Consistency

Consistency limits the number of ways recommendations are presented, reducing the cognitive workload by eliminating confusion and following standards and conventions. A CDSS should support situational awareness for an entire team of health care professionals. Therefore, recommendations should come from the same place and the system should have the same display of basic CDSS information on the case at hand for all professionals.²⁸

Interaction features

Fast

A CDSS must be fast with regard to the delivery of recommendations and use.¹⁷ It should provide timely feedback.²⁷ A CDSS should be timely upon submission of results^23,32 and at the time and location of decision-making.^{17,19,23,28,48,52–57} It should reduce the amount of time the user is required to interact with the system.^20,25,29 The assessment form (data entry) should not be too long or take too long to complete.²⁵ This can be accomplished by reducing the manual input of values,²⁰ but also the number of mouse clicks and amount of free-text typing (eg, provide a standardized selection of options).²⁹ This recommendation is critical for CDSSs that address routine medical problems, but is optional for more complicated assessments.²⁰

Fit

For CDSSs to reach their full potential, complex data must be rapidly accessible and easily understood within a provider’s workflow. Total cognitive load refers to the amount of mental processing power needed to use the CDSS and affects how easily providers enter data and retrieve results. The CDSS should minimize cognitive load.^20,21,31 The CDSS should request information from the provider only when necessary to reduce the manual input of values.^17,21,23 It should use selection tools (eg, dropdown boxes, field types) and sort options to facilitate ease of use and reduce cognitive load and potentially user error. It should have reduced screens to facilitate navigation and to promote efficient interactions.^31,34,35,40 To complement provider workflow and reduce cognitive load, the CDSS should automatically pull data from the EHR/integrate into the charting system,^{20–22,28,41,53,54,58–62} meaning the CDSS is an integrated component of charting or the order entry system.²⁶ Integration with the EHR allows the CDSS to navigate to appropriate locations and initiate intervention and take advantage of interactivity by initiating interventions, for example, by executing recommendations by noting agreement.^20,25 Lastly, the system should be the clinician’s partner.²⁸ It should provide a route to get to provider-specific information.²⁸ The CDSS should be adapted according to a subset of relevant actions taken by clinicians²⁸ and adapt its function to the evaluation of the outcome at risk over time (eg, take into account the evolution of target values to adapt for severity). It should incorporate functions supporting the dialog between the CDSS and the clinician.²⁸

Feedback

Feedback refers to the ability of a system to send information back to the user about what action has been done and what result was accomplished. The system should provide decision support automatically as part of clinician workflow,^{28,41,53,54,58–62} meaning the CDSS is provided at the point of care or order.²⁶ It should have automated alerting^21,30 and develop automatic prompting of users by the decision aid,^22,63 as opposed to systems that rely on manual processes where clinicians are required to seek out the advice of the decision support system.²⁶ It should request documentation of reasons for not following CDSS recommendations^48,52 by prompting clinicians to record a reason when they do not follow the advised course of action rather than allowing the system advice to be bypassed without recording a reason.²⁶

Forgiveness

Forgiveness refers to flexibility of use, providing immediate and reversible actions. The system should allow the user to be able to modify orders³² and should integrate a reset button.^26,29 Errors in data entry and selection occur, and users require the ability to make changes or begin using the tool again.

Flexible design

CDSSs must be flexible and adaptable, able to explore multiple assumptions and incorporate new information as circumstances change. CDSSs offer a process for enhancing health-related decisions and actions with pertinent, organized clinical knowledge and patient information. Information recipients can include patients, clinicians, and others involved in patient care delivery; therefore, it is recommended to involve the patient if possible, in both design and providing advice,¹⁹ which would empower patients to become actively involved in their own care (and provide actionable advice outside of a clinical encounter). The system should utilize adaptive design and feedback. Adaptive design maximizes the space and layout of a tool based on available space for the CDSS interface, but can also be applied in a more general sense in adapting to the provider using the CDSS. It should provide assistance without user control over output.²² It should provide an indication for all professionals of the availability of any information, allowing for the integration of new clinical information.²⁸ The system should identify latent needs and inform the end user,^17,27 improving the clinical utility of the tool. It should incorporate functions to support team awareness about the alert management and its evolution over time (eg, visible access to how the alert was handled and the reasons for alert override or rule deactivation if any has been documented)²⁸ and give access upon request to extended information (justification of the rule, attached scientific documentation, etc.) that should be structured depending on the user profile.²⁸

DISCUSSION

This narrative review article contributes to the body of literature on CDSS, as we are unaware of previously published articles at this scale and level of detail. This review will expand the field of knowledge and provide a novel organizational structure to identify key recommendations for CDSS.⁶⁴ There are serious shortcomings in the existing system of alerting providers in a meaningful way and providing actionable, evidence-based clinical decision support.^17,65 Lack of knowledge regarding alert presentation to providers has impeded alert optimization. Specifically, little guidance is provided for creating the most effective ways to differentiate alerts, highlighting important pieces of information without adding noise, or creating a universal standard.⁶⁶

It remains a challenge to implement CDSS effectively without inducing “alert fatigue,”⁶⁷ defined as a process “wherein clinicians may inadvertently ignore clinically useful alerts, thus diminishing the systems’ effectiveness and possibly leading to serious adverse consequences for patients.”⁶⁸ Researchers following all of the recommendations found in this narrative review may still not prevent alert fatigue, which may arguably be the most important barrier to achieving an effective CDSS. Research exists regarding the number of alerts providers receive through the EHR system,^69,70 but to date, no “best practices” in the design of these alerts have been identified. The recommendations identified in this narrative review will help optimize design, organization, management, presentation, and utilization of information through presentation, content, and function. The designation of 3 categories (interface, information, and interaction) should be further evaluated to determine the critical importance of the categories. Future work will determine how to prioritize them, given limited resources for designers and developers in order to maximize the clinical utility of CDSS.

Human factors engineering principles suggest that the format and presentation of CDSS may not be readily applied in the busy clinical setting and may fail to instill confidence in clinical staff. Effective presentation of an alert, including what is displayed and how it is displayed, may offer better cognitive support during busy patient encounters and may help providers extract information quickly. Following good human factors principles, alerts should signal an important matter and inform and guide the provider.⁷¹ Our review identified best practices based on effective CDSSs, including provider preference and improved patient outcomes. These recommendations are specific and actionable, meaning they can easily be applied by developers and CDSS designers for future implementations.

Limitations of this work include the subjective basis that defines the study design within a narrative review based on our opinion, which leads to a selection bias. As this review was qualitative in nature and focused on recommendations provided by the authors, our review did not focus on any quantitative methods the papers may have also provided. Another important limitation of this narrative review was that some of the recommendations we found are not very actionable, such as “use appropriate fonts,” “use meaningful colors,” and “identify latent needs.” Additional guidance is needed to determine what is actionable, as well as what recommendations may be more important to follow than others.

There is mounting proof that health information technology interventions improve patient outcomes through early diagnosis and initiation of evidence-based protocols for critically ill patients.⁷² The rapid acceleration of technology and the convergence of predictive analytics and human factors address the Centers for Medicare and Medicaid Services stage 3 meaningful use requirements.⁷³ The process of integrating real-time analytics into clinical workflow will be significantly affected by changes the Centers for Medicare and Medicaid Services has planned for EHRs. A shift toward more agile and collaborative infrastructure building is expected to be a key feature of health information technology strategies, such as CDSS design, in the future. As interoperability and big data analytics capabilities become increasingly central to crafting the health care information systems of the future, there is a critical need to optimize the flow of health information and communication to support clinical decision-making.

From a policy perspective, this review highlights the need to establish basic design standards, with a foundation in human factors, to guide the design, development, implementation, and customization of CDSSs. Policymakers must recognize the importance of usability design standards and should develop guidelines for EHR developers and adopters to follow. It is critical that these standards be adhered to by EHR vendors, health care providers, and other stakeholders. By prioritizing usability and adopting a common set of design standards to ensure consistency across EHR platforms, we can begin to improve usability, enhance functionality, and develop technology standards that allow full integration of the EHR into the care system to improve clinical decision-making. Federal agencies, like the Office of the National Coordinator for Health Information Technology, should consider policy mechanisms to encourage creation of, adoption of, and adherence to design standards to ensure the safe and effective use of CDSSs. These policies must be designed and implemented in a way that promotes safety, while also being careful not to stifle innovation.

CONCLUSION

Human factors, usability, and human-computer interaction principles are fundamental to successful CDSSs. We found that by compiling a set of recommendations for CDSS design, we were able to synthesize the design and functional characteristics that impact the performance of the system. Evaluating the recommendations derived from the papers included in this qualitative narrative review, we classified 3 “I” categories – interface, information, and interaction – further organized by 5 “P” categories (presentation, placement and positioning, and provision of multiple presentation layers; 4 “C” categories (clean and concise, content guidance, and consistency; and 5 “F” categories (fast, fit, feedback, forgiveness, and flexible design). We compiled this list of recommendations as general guidance for the design of successful CDSSs, and we believe that testing and validating such a unified set of recommendations could eventually enrich the literature and the field by providing domain-specific design heuristics.

DISCLAIMER

The authors declare no conflicts of interest.

FUNDING

This work is supported by an Institutional Development Award from the National Institute of General Medical Sciences of the National Institutes of Health under grant number U54-GM104941 (PI: Binder-Macleod), and by the National Library of Medicine of the National Institutes of Health under grant number 1R01LM012300-01A1 (PI: KM).

CONTRIBUTORS

KM was responsible for study design and overall management, and drafted the manuscript. DM conducted literature searches, retrieved publications, provided overall management of the project, participated in consensus evaluation, and revised the manuscript. MC contributed to study design, participated in consensus evaluation, and revised the manuscript. RR revised the manuscript. YM contributed to study design and literature searches, retrieved publications, and participated in consensus evaluation. RK and RK participated in consensus evaluation and revised the manuscript. SS and WW contributed to the study design and revised the manuscript. RA contributed to the study design and revised the manuscript. All authors approved the version of the manuscript to be published.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.