Using clinical decision support systems to bring predictive models to the glaucoma clinic

Abstract

Advances in the field of predictive modeling using artificial intelligence and machine learning have the potential to improve clinical care and outcomes, but only if the results of these models are appropriately presented to clinicians at the time they make decisions for individual patients. Clinical decision support (CDS) systems could be used to accomplish this. Modern CDS systems are computer-based tools designed to improve clinician decision making for individual patients. However, not all CDS systems are effective. Four principles that have been shown in other medical fields to be important for successful CDS system implementation are (1) integration into clinician workflow, (2) user-centered interface design, (3) evaluation of CDS systems and rules, and (4) standards-based development so the tools can be deployed across health systems.

Introduction

Clinicians today are faced with the challenging task of incorporating large amounts of data accurately and efficiently to make critical decisions that impact patient health.¹ Clinicians who care for patients with glaucoma are faced daily with this challenge; caring for glaucoma requires synthesis of information from many data sources (tonometry, pachymetry, perimetry, optical coherence tomography, disc photographs, ocular exam, patient history, etc.) from many visits over long time periods.² These data are often complex and have high test-retest variability.³ The data must be interpreted in the context of patient-specific circumstances.² Adding to the challenge, the data need to be evaluated and integrated quickly to make a decision in the midst of a busy glaucoma clinic. Recent advances in the field of predictive modeling may help address some of these challenges.⁴

Predictive modeling in health care involves the analysis of retrospective healthcare data to estimate the future likelihood of an event for a specific patient.⁵ Predictive modeling for glaucoma has been conducted using traditional statistical methods (for example, linear regression,^6–8 logistic regression,^6,8,9 and Cox proportional hazards models^10,11) and using more sophisticated artificial intelligence (AI) methods, including machine learning methods such as neural networks and deep learning using deep neural networks.^12–25 However, these predictive models have not yet substantially influenced glaucoma clinical practice in part because calculating a prediction is not, in itself, sufficient to influence clinician behavior.⁴ For the results of glaucoma predictive models to meaningfully improve clinical practice, clinically useful information from the models must be presented to the decision maker in an effective format at the optimal time in the clinical workflow to facilitate sound decisions.²⁶ Clinical decision support (CDS) systems can help address these challenges.^26,27

Modern CDS systems are computer-based tools designed to improve clinician decision making for individual patients.^27,28 These systems have been successfully employed for many non-ocular conditions, including diabetes,²⁹ cancer,³⁰ sepsis,³¹ acute respiratory distress syndrome,³² hyperglycemia,³³ and neonatal hyperbilirubinemia.³⁴ CDS systems can improve diagnostic test use³⁵ and treatment decisions.^36–38 An illustrative example of a successful CDS system is an electronic health record (EHR) add-on app for neonatal bilirubin management.³⁴ Similar to glaucoma care, clinicians managing neonatal bilirubin levels must retrieve data that is scattered across the medical record, synthesize the data, and apply guideline algorithms to develop patient-specific treatment plans. The CDS tool gathered the data into one display and provided guideline-based individualized treatment recommendations.

While there has been a considerable amount of research regarding CDS in other medical fields, relatively little work has been done in ophthalmology and glaucoma. This may be because ophthalmology adopted EHRs later than many other medical fields.³⁹ The use of EHRs in ophthalmology has increased dramatically.⁴⁰ This, coupled with advances in predictive modeling for glaucoma, provides an opportunity for us to develop effective CDS for glaucoma. As we do this, we can learn from CDS successes and failures from other medical fields.⁴¹ A considerable body of literature has evaluated characteristics that are important for the success of CDS systems.^42–44 Four principles that have been shown in other medical fields to be important for successful CDS system implementation are (1) integration into clinician workflow, (2) user-centered interface design, (3) evaluation of CDS systems and rules, and (4) standards-based development so the tools can be deployed across health systems.^42–44 The purpose of this paper is to describe these important CDS system principles and to discuss how they could be applied to glaucoma to allow us to develop CDS systems that leverage advances in glaucoma predictive modeling to improve clinical care.

Integration into Clinician Workflow

CDS systems need to be designed to facilitate implementation and integration into clinical workflow.^45,46 Integration into clinical workflow means that the decision support is provided at the time the decision is being made to the decision maker in an effective and seamless format.²⁶ Automatic provision of CDS as part of clinician workflow is one of the strongest predictors of whether or not a CDS tool will improve clinical practice.⁴⁷ Clinicians spend a considerable amount of time using EHRs.⁴⁸ CDS systems that require significant time and effort add to this burden and are less likely to be used.⁴⁹ Instead, CDS systems should be designed to fit into clinicians’ routine use of the EHR.⁵⁰

Studying the clinical context and workflow of decision making and incorporating the results of these studies in the design of CDS tools facilitates implementation and use of CDS.^45,51,52 For example, an in-depth analysis of the clinical workflow allowed Weir et al. to successfully implement CDS tools targeting improved geriatric care.⁵² In this study, researchers engaged users and IT departments to understand their workflow and interaction with the EHR and the CDS system. The CDS interventions were designed and adapted to fit into this workflow.

In the context of glaucoma management, integration into clinician workflow necessitates providing the CDS to the clinician seamlessly when a specific decision is being made. For example, a CDS tool designed to help identify glaucomatous progression would need to be presented to the clinician at the moment in clinical workflow that the clinician is deciding if there has been progression. In the case of glaucoma, additional research is needed to understand the glaucoma clinical workflow and decision-making process. Integration of future CDS systems for glaucoma into the established clinical workflow will make these systems easier to access and use, which will make them more likely to improve glaucoma outcomes.

User-Centered Interface Design

User-centered design focuses on the needs of users to make information systems more usable and involves identifying and understanding the system users, tasks, and environments in which the users perform the tasks.⁵³ The user-centered design process is iterative and involves users throughout the design process.⁵⁴ Established scientific methods for user-centered design include ethnographic observations, interview analysis, think-aloud studies, cognitive work analysis, and observation of real-time use in the clinical application site.⁵⁴ Though many methods can be used for user-centered design, some basic principles are (1) understanding the users, tasks, and environments, (2) understanding the usability requirements for the system being developed, (3) designing the system to meet these requirements, and (4) evaluating the design with users.⁵³

User-centered interface design of CDS systems allows effective communication of CDS system recommendations and results.^54–56 Involving clinicians in the design of CDS tools can increase usability and satisfaction.⁵⁷ An example of user-centered design for a CDS system is a CDS tool developed to present results of published randomized controlled trials to clinicians at the point of care.⁵⁷ In this study, Del Fiol et al. used rapid iterative cycles incorporating feedback from physician users of the CDS prototypes. This user-centered design process allowed for the development of a useful and usable CDS tool.

As CDS systems for glaucoma care are developed, it is important that user-centered design principles are followed. User-centered interface design for glaucoma CDS would involve clinicians who care for patients with glaucoma in the design and testing of the CDS interface to ensure that user needs are met and information is communicated effectively. Clinicians who care for patients with glaucoma should be actively involved in the planning, development, and testing of CDS systems designed to improve glaucoma care.

Evaluation of CDS Systems and Rules

CDS systems and rules should be rigorously evaluated.⁵⁸ CDS rules are the underling predictive models or algorithms that CDS systems use to provide decision support. The most rigorous study design that is feasible should be used to evaluate CDS systems.⁵⁰ Cluster randomized controlled trials are the preferred method, but if randomization is not feasible an interrupted time series study design may be appropriate.⁵⁰

It is important that the results of CDS systems are also evaluated as the systems are applied in new populations. One key challenge of CDS is the rules developed in one context may not necessarily apply in another. For example, glaucoma CDS rules developed for an inner-city population at a large academic center may not be appropriate when applied in a private-practice, rural clinic. One way to address this limitation is to retrospectively run the CDS rules on a large set of patients and examine the CDS results against each patient’s EHR data before implementing the CDS system.⁵⁰

Standards-based Development

CDS tool interoperability means the tool to be used at different sites (with different EHRs).⁵⁸ Interoperability is one of the key challenges to widespread scaling of CDS.⁵⁸ Substitutable Medical Applications, Reusable Technologies (SMART) on Fast Healthcare Interoperability Resources (FHIR) (pronounced “smart on fire”) is an interoperable, standards-based platform that can be used for CDS. SMART and FHIR are health informatics standards frameworks developed by the Health Level Seven International (HL7) standards development organization.⁵⁹ The FHIR standard provides a systematic, interoperable way to define and represent data in EHRs and allows for the exchange of healthcare information electronically. SMART on FHIR is a platform that uses the FHIR standard to enable medical applications to be written once and run unmodified across different healthcare IT systems.⁶⁰

The use of SMART on FHIR is promoted by the NIH as an important interoperable health informatics tool approach (NOT-OD-19-122).⁶¹ The SMART on FHIR platform provides a standard way for CDS systems and other health informatics applications to be integrated with the EHR and has been used for clinical decision support applications.^34,62,63 The EHR add-on app for neonatal bilirubin management discussed above is an example of how the SMART on FHIR technology can link to the EHR and provide a scalable, usable approach to integrating CDS in health care.³⁴ Other complementary standards include the HL7 CDS Hooks standard⁶⁴ which enables alerts and reminders to be integrated with EHRs, as well as the HL7 Clinical Quality Language⁶⁵ which provides a standard language for expressing rules for CDS and electronic clinical quality measurement.

FHIR already supports a number of data elements that would be useful for glaucoma CDS, such as age, gender, race, medical and ocular diagnoses, medications in use, past medications, past procedures, and visual acuity. For other data elements needed for eye care, the ophthalmology community could work together to advance the inclusion of those data elements in the U.S. Core Data for Interoperability⁶⁶ which defines the FHIR data elements that must be supported by EHR vendors.⁶⁷ Developing CDS tools for glaucoma management using a standards-based approach such as SMART on FHIR could enable these tools to provide value to glaucoma specialists across a variety of different practice types using a variety of different EHR systems.

Conclusion

For advances in predictive modeling to meaningfully improve glaucoma care, the results of these models need to be appropriately presented to clinicians at the time they make decisions for individual patients. CDS systems that are coupled with EHRs can accomplish this if these CDS systems are well-integrated into clinician workflow, have usable and understandable interfaces, and are standards-based to enable interoperability. If these CDS systems are developed and implemented appropriately with a focus on user needs, they have the potential to augment clinical decision making, enhance workflow, and improve patient outcomes.

Clinical decision support systems are computer-based tools designed to improve clinician decision making for individual patients. These tools could be used to present the results of glaucoma predictive models to clinicians as they make decisions.