Strengths and weaknesses of computerized clinical decision support systems: insights from a digital control center (C3 COVID-19) for early and personalized treatment for COVID-19

Abstract

Clinical Decision Support Systems (CDSS) are computer-based tools that leverage the analysis of large volumes of health data to assist healthcare professionals in making clinical decisions, whether preventive, diagnostic, or therapeutic. This review examines the impact of CDSS on clinical practice, highlighting both their potential benefits and their limitations and challenges. We detail the experience of clinical medical professionals in the development of a virtual control center for COVID-19 patients (C3 COVID-19) in Spain during the SARSCoV-2 pandemic. This tool enabled real-time monitoring of clinical data for hospitalized COVID-19 patients, optimizing personalized and informed medical decision-making. CDSS can offer significant advantages, such as improving the quality of inpatient care, promoting evidence-based clinical and therapeutic decision-making, facilitating treatment personalization, and enhancing healthcare system efficiency and productivity. However, the implementation of CDSS presents challenges, including the need for physicians to become familiar with the systems and software, and the necessity for ongoing updates and technical support of the systems.

INTRODUCTION

In this review, we aim to acquaint clinicians with Clinical Decision Support Systems (CDSS) by exploring their benefits and limitations. We focus on insights gained from clinical practitioners who were involved in developing a patient Control Center for COVID-19 (C3 COVID-19) during the SARS-CoV-2 pandemic.

DEFINITION AND TYPES OF DECISION SUPPORT SYSTEMS

CDSS are computer-based tools designed to assist health-care professionals in clinical decision-making [1]. These systems analyze large volumes of healthcare data providing customized assessments and recommendations to assist the clinician in making informed decisions [2]. In particular, the integration of the ability of CDSS to process and analyze large volumes of data, identify patterns, and generate evidence-based recommendations and the clinician’s expertise and clinical experience may enhance diagnostic accuracy, personalize treatment plans, and optimize patient outcomes.

In the following sections, we provide examples of these systems, categorized by their distinct approaches:

1. Rule-Based Systems: knowledge-based systems that utilize data extracted from other sources to develop and apply contextual knowledge through predefined rules [3]. In healthcare domain, rule-based CDSS utilize a set of predefined rules created by physicians and/or programmers to display patient information in an organized and categorized manner, facilitating decision-making. The results that come from the rules can be used as recommendations or alerts. This approach was used by our group in the creation of the patient Control Center for COVID-19 (C3 COVID-19).

2. Case-Based Systems: artificial intelligence tools that use past cases in solving similar problems [4]. In health-care domain, case-based CDSS compare the current patient’s data (such as symptoms, comorbid conditions, treatments, time gaps) with comparable items stored in a database to offer recommendations taken from similar cases. The most important aspect of this approach is that these systems need to be validated in a real-world settings. Case-based systems have been successfully used in medicine for improving diagnosis of rare diseases by comparing with previous cases, achieving accurate diagnosis and classification for liver diseases, indexing images according to their radiologic content [5–7].

3. Statistical and Machine Learning Model-Based Systems: employ machine learning techniques and statistical analysis to identify patterns in data and predict outcomes. These results can be integrated into a computational tool that provides alerts to physicians or is available in a format such as an application or website for consultation by healthcare professionals.. These systems have important applications in various healthcare domains including diagnosis, treatment planning, prognostics, medications management, and patient monitoring [8].

4. Systems Integrated into Electronic Health Records (EHR): these systems are integrated directly into EHR systems to provide real-time recommendations during the clinical workflow [9]. An example of these systems could be alerts for drug interactions at the time of prescription, suggestions for cheaper medications alternatives, alerts for reducing test duplication [10].

GENERAL BENEFITS OF DSS IN MEDICINE AND THEIR EVOLUTION OVER THE DECADES

CDSS in medicine hold significant potential as powerful tools for enhancing the delivery of healthcare. The general benefits of implementing these systems include: 1) Improving the quality of patient care through objective data utilization and mechanisms to minimize medical errors; 2) Increasing efficiency and productivity by automating routine tasks and swiftly providing relevant information to reduce healthcare costs; 3) Enabling personalized treatment through the analysis of large datasets and predictions based on artificial intelligence algorithms; and 4) Promoting evidence-based decision-making by integrating clinical research and guidelines.

The concept of CDSS first emerged in the 1950s, representing the pioneering efforts to incorporate informatics into clinical decision-making processes [11]. One of the pioneering systems was MYCIN, an antibiotic selection support system, developed in the 1970s at Stanford University [12]. MYCIN was an early expert system in artificial intelligence, meticulously developed over a span of 5-6 years to support the management of infectious diseases through the application of 500 predefined rules. The primary objective of MYCIN was to identify bacterial pathogens responsible for infections and to recommend suitable antibiotics and dosages adjusted to the patient’s weight. Additionally, the system was capable of alerting clinicians to critical conditions such as meningitis or bacteremia. MYCIN functioned by requiring users to answer a series of binary (yes/no) questions. Despite achieving a diagnostic accuracy of approximately 70%, MYCIN’s adoption was impeded by several factors, including concerns about the high cost of implementation, legal liabilities associated with incorrect diagnoses, and general skepticism about the efficacy of CDSS in clinical settings [13–15]. Table 1 provides an overview of significant advancements in DSS technology from the 1970s to the present.

Table 1.

Significant advancements in DSS technology from the 1970s to the present.

Decade	Technology	Examples of DSS	Strengths and weaknesses
1970s	Rule-Based SystemsIntegration with Hospital Systems DSS for Specific Therapies	Tools for differential diagnosis in internal medicine: Internist-1 QMR (Quick Medical Reference) Tool for alerts and reminders: HELP (Health Evaluation through Logical Processing) Tool to assists in chemotherapy treatment planning: ONCOCIN	Strengths: advanced diagnostic capabilities; integration of extensive medical knowledge; valuable educational tool; significant research impact. Weaknesses: limited to internal medicine; complex interface; dependence on rule-based logic; data integration challenges; Resistance to adoption
1980s	Advances in Artificial Intelligence and Machine LearningDSS for Chronic Disease ManagementUse of Relational Databases	Tools for differential diagnosis in internal medicine: DXplain Tool for management of chronic diseases as diabetes: System CASS (Computer Aided System for Staging)	Strengths: extensive differential diagnosis capabilities; effective integration of patient symptoms and data; useful for educational purposes; assists in long-term care planning; useful for monitoring disease progression. Weaknesses: limited by the quality and scope of input data; complexity in system integration; significant user training; limited generalizability beyond chronic diseases
1990s 2000s	Integrated EHR and DSSExpansion of Predictive AlgorithmsEvidence-Based DSS	Tools to incorporate evidence-based guidelines and clinical practice recommendations in CDSS: - HL7 u CDA	Strengths: to allow exchange of clinical documents between those involved in the care of a patient; re-use of clinical data for public health reporting, quality monitoring, patient safety Weaknesses: complex structure; limited Interoperability; overhead and performance Issues; difficulty in parsing and analyzing data; adoption and implementation costs
2010s	Big Data and Predictive Analytics Advanced AI and Deep Learning DSS for Personalized Medicine	Tools for treatment recommendations: Watson for Oncology: cancer treatment recommendations. Tools for Precision Medicine:IBM Watson Health: personalization treatment based on genomic analysis Google’s DeepMind Health	Strengths: to allow exchange of clinical documents between those involved in the care of a patient; re-use of clinical data for public health reporting, quality monitoring, patient safety Weaknesses: dependence on data quality; limited understanding of nuances; adaptation to local practices; clinical acceptance and trust; high costs and resource requirements; transparency and explainability
2020s to present	Telemedicine and DSSIntegration with Wearable Devices Expansion of Explainable AI	Tools for remote consultations and rapid decision support: Telestroke Networks Tools to assist in interpreting medical images: Explainable AI in Radiology Wearable devices: Apple Watch and AFib Detection	Strengths: continuous monitoring; patient engagement; data integration; improved adoption; transparency Weaknesses: data overload; privacy concerns; potential for over- reliance; incomplete explanations

HEALTH DATA: THE FOUNDATION OF DECISION SUPPORT SYSTEMS IN MEDICINE

Health data serves as the cornerstone of clinical decision support systems (CDSS) in medicine. In recent years, there has been a profound transformation in health data management, transitioning from the digitization of medical records within healthcare facilities to the widespread adoption of wearable technologies, such as smartwatches and health sensors [16–18]. These devices have become essential components of both patient lifestyles and healthcare systems [19,20].

A significant milestone in the evolution of digital health is the advent of EHRs. In the 1990s, several regions in Spain initiated efforts to digitize medical records, although these efforts were fragmented and lacked standardization. At the turn of the 21st century, the Spanish government launched the ‘Plan Avanza’ (2005), with the goal of modernizing healthcare services through the implementation of EHRs and the integration of information technologies [21]. Regions such as Catalonia, Basque Country, Madrid, and Andalusia emerged as pioneers in developing their own EHR systems [22]. Notably, Catalonia introduced the Shared Clinical History of Catalonia (HCCC), which facilitates comprehensive access to patient information across various levels of care [23,24]. Efforts have since been underway to consolidate regional EHRs into a unified platform, though significant challenges in achieving system integration persist [25].

Access to EHR data offers a rich resource for medical and epidemiological research and for the development of DSS in medicine. However, managing this data involves several challenges [26]. Ensuring the security and privacy of patient data is paramount, particularly in light of increasing cyber threats and data breaches [27]. Additionally, maintaining data quality is crucial [28]. Regulations concerning the validation of data quality in big data vary across jurisdictions and sectors, with no universally established guidelines [29,30]. Table 2 delineates the key steps involved in processing data from EHR extraction to potential utilization. These steps highlight various scenarios requiring meticulous management to ensure high-quality outcomes. The efficacy of any computerized tool is contingent upon the quality of the data it utilizes. Furthermore, identifying the most relevant data for specific fields can guide initial efforts in developing artificial intelligence programs with highly pertinent data and in selecting optimal data extraction strategies from EHRs. A recent study conducted by our group revealed that structured variables are pivotal for research in infectious diseases, while unstructured [31]. In the field of infectious diseases, the most critical unstructured data pertains to clinical manifestations. These data could be more effectively organized through the use of semi-structured medical records that target specific symptoms.

Table 2.

Key points where data quality can be compromised

Key Points
Data Extraction Data Reorganization Data Cleaning Data Integration Data Loading into the Final Tool Creation and Correction of Definitions Dataset Utilization

THE COVID-19 CONTROL CENTER

The COVID-19 pandemic presented a profound challenge to numerous countries and their healthcare systems. A substantial number of patients, many with severe conditions, required hospitalization and/or admission to intensive care units within a brief period, especially during the early stages of the pandemic [32]. The initial years were particularly challenging for physicians, who had to rapidly acquire knowledge about the behavior of a novel infectious disease while simultaneously managing numerous critically ill patients. Physicians from diverse specialties were frequently tasked with assuming new roles, serving as the primary caregivers for these patients. This situation posed a significant challenge, as the complexity of coordinating care led to variability in treatments, making it difficult to uniformly apply the latest medical knowledge on disease management across all involved professionals. In response to these challenges, we developed an informatics tool that streamlined the management of patient clinical information. A medical team from our hospital’s Infectious Diseases Department collaborated with the IT Department to access real-time data from EHRs. We developed a virtual control center for all patients hospitalized with COVID-19, known as the COVID-19 Control Center (C3). This system received crucial demographic, clinical, laboratory, microbiological, treatment, and prognostic data from EHRs. An IT consultant was engaged to design and implement both the web and mobile applications for C3. This tool categorized COVID-19 patients using real-time laboratory parameters and vital signs extracted from the hospital’s software. It classified patients into distinct subgroups—such as clinical stability, viral pattern, inflammatory pattern, co-infection pattern, or thrombotic pattern—based on clinical patterns developed by our physicians and previously published [33–35]. This tool enabled infectious disease specialists to review data for all hospitalized patients and offer guidance to the attending physicians. The C3 system facilitated the identification of several scenarios that could be optimized under expert supervision: 1) ensuring the appropriateness of patient treatments, 2) pinpointing the most critically ill patients for specialized oversight, 3) analyzing diverse patient patterns and the corresponding therapeutic strategies, 4) identifying clinically stable patients to recommend discharge, and 5) gathering data for the advancement of various research initiatives. Figures 1 and 2 illustrate the display of these tools on our computer screens and mobile devices, respectively. The C3 system offered numerous advantages. It allowed for effective real-time monitoring of patient progression, enabling the administration of personalized therapies. Furthermore, the C3 system generated valuable data that supported the execution of several research studies. With the support of a substantial European grant, it was demonstrated that the tool could be integrated into the IT systems of other hospitals with minimal adjustments, allowing for seamless data transfer from these centers. The project was recognized with the EIT Health Innovation Award in 2020. However, the development of this clinical decision support tool highlighted several limitations that should be carefully considered by those planning to undertake similar projects. The primary consideration is that C3 was a decision support system developed by our hospital’s physicians, grounded in their acquired knowledge and the routine application of the hospital’s protocols. Consequently, physicians at our center, who were well-acquainted with the system’s patterns, found it highly familiar and usable. However, when the system was deployed to other hospitals both nationally and internationally, we encountered a range of responses. Some centers welcomed the tool with enthusiasm, while others were more indifferent. In the latter centers, the tool’s effectiveness was diminished by the physicians’ lack of familiarity with the identified clinical patterns. This underscores a crucial consideration: CDDS should be developed in collaboration with the physicians who will use them or should adhere to well-established standards. Tools that do not align with clinicians’ practices are unlikely to be adopted. Another significant limitation is that once a computerized decision support system is developed and integrated into hospital systems, it requires continuous, 24/7 maintenance. Real-time data evaluation systems are susceptible to occasional down-times that need prompt resolution. Consequently, hospital administration must approve the integration of such systems, and the hospital’s IT departments must be equipped with the necessary resources for ongoing maintenance. Lastly, during the challenging period of the COVID-19 pandemic, hospitals experienced numerous modifications, including the continuous establishment of new areas to accommodate patients. These new spaces were not always seamlessly integrated with existing hospital IT systems. Consequently, in some centers where the C3 system was already operational, certain data groups, such as clinical vital signs, could not be incorporated into the tool. This omission led to gaps in the programming algorithms and affected the tool’s overall efficiency. In summary, CDSS are crucial tools designed to aid healthcare professionals in making more objective and efficient clinical decisions through the analysis of extensive health data. Our experience developing the COVID-19 Control Center (C3) during the pandemic demonstrated its effectiveness in providing real-time oversight of clinical information and supporting informed, personalized medical decision-making. Nonetheless, several challenges emerged, including the necessity for physicians to become acquainted with the tool and the ongoing requirement for meticulous maintenance of the informatics systems.

Figure 1.

Screenshot of the COVID-19 Control Center (C3) on computer monitors at our facility.

Figure 2.

Screenshot of the COVID-19 Control Center (C3) on corporate mobile phone screens.

FUNDING

This study has been co-funded by the European Regional Development Fund (EDRD). CG-V [FIS PI21/01640] has received research grants from the Ministerio de Sanidad y Consumo, Instituto de Salud Carlos III. Project PI21/01640 has been funded by Instituto de Salud Carlos III (ISCIII) and co-funded by the European Union. This work was co-funded by a research grant (SGR 01324 Q5856414G) from the AGAUR (Agencia de Gestión de Ayudas Universitarias y de Investigación) of Catalunya. TF-A has received a pre-doctoral grant supported by the Ministerio de Sanidad y Consumo, Instituto de Salud Carlos III [RH RH042953]. The present study is co-financed by the Instituto de Salud Carlos III, with file code CM23/00277, under the Resolution of the Directorate of the Instituto de Salud Carlos III, O.A., M.P. of December 13, 2023, granting the Rio Hortega Contracts, and co-financed by the European Union and by the European Social Fund Plus (ESF+).

CONFLICT OF INTEREST

CG-V has received honoraria for talks on behalf of Gilead Science, MSD, Pfizer, Jannsen, Novartis, Basilea, GSK, Shionogi, AbbVie, and Advanz Pharma, and a grant support from Gilead Science, Pfizer, GSK, MSD, and Pharmamar. AS has received honoraria for talks on behalf of Merck Sharp and Dohme, Pfizer, Novartis, Angelini, Menarini, and Gilead Science as well as grant support from Pfizer and Gilead Science. JM received honoraria for talks on behalf of Merck Sharp and Dohme, Pfizer, Novartis, and Angelini. OP has received honoraria for talks on behalf of BMS and Qiagen and expertise for Sanofi. TF-A has received funding from Gilead Science, Pfizer, Pharmamar to support registration and travel expenses for attending scientific conferences.