Artificial intelligence in clinical decision support systems for oncology

Abstract

Artificial intelligence (AI) has been widely used in various medical fields, such as image diagnosis, pathological classification, selection of treatment schemes, and prognosis analysis. Especially in the image-aided diagnosis of tumors, the cooperation of human-computer interactions has become mature. However, the ethics of the application of AI as an emerging technology in clinical decision-making have not been fully supported, so the clinical decision support system (CDSS) based on AI technology has not fully realized human-computer interactions in clinical practice as the image-aided diagnosis system. The CDSS was currently used and promoted worldwide including Watson for Oncology, Chinese society of clinical oncology-artificial intelligence (CSCO AI) and so on. This paper summarized the applications and clarified the principle of AI in CDSS, analyzed the difficulties of AI in oncology decisions, and provided a reference scheme for the application of AI in oncology decisions in the future.

Introduction

In the precision medicine era, the explosive growth of new drugs and approved indications has brought great challenges to the formulation of the best treatment scheme for different molecular phenotypes of different tumors ^1-3. With the technical support of big data and machine learning, clinical decision support systems (CDSS) based on artificial intelligence (AI) came into being ^4-6. By integrating different medical records, literature and clinical research data, the CDSS evaluates the drug efficacy, product accessibility, adverse reactions, the financial status of the patients and medical insurance types, and then provides individualized suggestions to help clinicians optimize treatment plans ^{7, 8}. The applications of AI have expanded from solving daily life problems to medical professional fields, such as image diagnosis, pathological diagnosis, clinical treatment decision-making, prognosis analysis, and new drug screening ^{9, 10}.

The Watson system of IBM has been applied to the fields of fashion, finance, medical treatment, tourism, law, education, transportation and so on. Especially in the field of cancer, Watson for Oncology (WFO, IBM Corporation, United States), as the earliest widely used CDSS ^{11, 12}, has been gradually popularized all over the world in the fields of lung cancer, colon cancer, rectal cancer, breast cancer, gastric cancer and gynecological cancer ¹³. Based on the WFO system, the medical staff only need to input the structured data of the case. Within one minute, the system will output the most standard treatment method for the specific case and provide highly consistent evidence ¹¹. The WFO has provided services to more than 70 municipal medical institutions and more than ten thousand patients ¹³. A CDSS called the Chinese society of clinical oncology-artificial intelligence (CSCO AI) system based on Chinese society of clinical oncology-breast cancer (CSCO BC) big data and CSCO BC guidelines had run into operation for three years nationwide ⁷. As of January 2022, excluding combined treatment regimens, there are currently 82 FDA drugs approved for breast cancer treatment ¹⁴. Breast cancer therapy is expected to accelerate the pace of treatment and shorten the cycle of treatment guideline changes. The Oncologists face a huge and rapidly expanding knowledge base challenge ^{15, 16}. To mitigate this challenge, CSCO AI has now been applied to breast cancer treatment in the face of rapidly changing scientific evidence ^{17, 18}, helping patients achieve personalized treatment based on drug approval and treatment guidelines. Other CDSSs, such as Archimedes IndiGO, Auminence, DiagnosisOne, and DXplain, have also gradually been developed and are beginning to be applied in the clinic ^{19, 20}.

Overview of the CDSS

The CDSS based on AI technology is essentially a computer program ^{8, 21}. In the early stage, the knowledge base was constructed through the key and prognostic information in structured medical records or literatures. Then, a search engine was used to filter the best results from the knowledge base through key information and then feedback to users ²², machine learning algorithms include logistic regression, support vector machine, artificial neural network, deep learning and so on. Due to the particularity of clinical application and the development of CDSS technology, unsupervised models are mainly used to operate. AI technology allows computers to simulate human reasoning thinking, collect and express the learned knowledge content by generating suggestions, reducing the dependence on memory, the decision error rate and the response time. It also helps make reasonable and safe decisions in clinical practice to ensure clinical safety, quality and improve the treatment efficiency ^{7, 11}.

Composition and treatment judgment of the CDSS

As one of the earliest CDSS in the world, WFO has indexed and stored test cases, documents, protocols and medical records from experts at the Memorial Sloan Kettering Cancer Center (MSKCC) and applied them to specific cases after being processed by a computational reasoning method ^{23, 24}. In addition, unlike the CSCO AI, WFO updates the latest diagnosis and treatment information every one to two months and for understudied medical records, WFO also describes unsupported case information. When enter case information that WFO does not support, it would not process the case and return any suggestions ²⁵. WFO's treatment recommendations are divided into three groups and have corresponding labels: green represents “recommended treatment”, which has a strong evidence base, orange represents “consideration treatment”, and according to their clinical judgment, doctors can consider it as a suitable substitute, while red represents non-recommended treatment, which means due to specific contraindications or strong evidence against their use. If WFO is not aware of it at the time of the analysis, the treatment recommended by the oncology committee is classified as “unavailable” ²⁶.

Different from WFO, CSCO AI system was established under the CSCO platform by using the CSCO database, guidelines. The CSCO AI system mainly built different knowledge maps based on the schemes in the CSCO guidelines. When doctors search for relevant information, it locates the knowledge map and output the results according to the key information. Similarly, it is also updated in real time with the guidelines to ensure the timeliness of the system, as shown in Figure 1. In addition to incorporating guidelines, medical records and literature data, CSCO AI also integrates medical insurance, clinical research and other data to make it more in line with the actual situation in China and to ensure that users can obtain more information with one click ⁷. For a self-assessment of the decision-making scheme, CSCO AI adopts a quantitative scoring method, if the decision-making scheme conforms to the national comprehensive cancer network (NCCN) guideline class 1 evidence recommended by the high-level guidelines or the CSCO BC guideline class I recommendation and conforms to clinical practice, it is fully compliant, and 3 points can be given; if it meets the high-level recommendation but does not meet the clinical practice, or meets the class 2A evidence of NCCN guidelines or the class II recommendation of CSCO BC guidelines and meets the clinical practice, it is in high-level compliance, and 2 points can be given. If it does not conform to the guidelines but conforms to clinical practice or only conforms to class 2B evidence of the NCCN guidelines or class III recommendations of the CSCO BC guidelines, 1 point can be given. If it does not comply with the guidelines and does not comply with clinical practice, 0 points will be given ⁷.

Figure 1.

Flow chart of the data operation in the CDSS. The data in the flow chart of the CDSS were operated according to the knowledge base, knowledge graph, interactive interface and clinical decision output sites.

Application characteristics of the CDSS

Based on the working principle, CDSS has several characteristics. First, the development knowledge base of the CDSS mainly comes from the big data of clinical guidelines or case composition included in high-level clinical research ^{27, 28}. Second, for the CDSS developed based on a guideline, convenience is improved at the cost of reducing its reliability ^{7, 29-31}. Third, the updating speed and the abundance of information sources of the knowledge base can affect the heterogeneity of conclusions between the CDSS and multi-disciplinary team (MDT) ³². Fourth, the CDSS can guide inexperienced doctors and interns ³³. However, the CDSS can only be regarded as a member of the MDT team as shown in Figure 2. After more than 20 years of development, the CDSS has played an important role in disease management, radiation dose calculation, image analysis, blood bank systems, nursing and other fields ^34-36. At present, most clinical studies on the reliability of CDSS has compared the schemes of MDT and MDT plus CDSS to explore the help provided by the AI system to doctors at different levels and for different diseases, which is closely related to the ethics of AI technology in clinical diagnosis and treatment decision-making ³⁷.

Figure 2.

The applications of CDSS in clinical work. The knowledge base was constructed through the key and prognostic information in structured medical records, literature and guidelines. The search engine was used to filter the best results from the knowledge base through key information and feedback to the doctors. The doctor could make a clinical decision based on the CDSS suggestion and their clinical experience to treat patients, and the prognostic information of patients should be studied by the knowledge base that forms a positive cycle.

For the CSCO AI system, the establishment of a standardized and unified standard dataset, reliable data source and knowledge base, the efficiency of automatically outputting the scheme in line with clinical decision-making and a friendly communication interface are important directions to improve the reliability of the CDSS ⁷. The knowledge base is the cornerstone of the CSCO AI system. System learning, testing and verification should be based on the knowledge base. The richness and representativeness of the knowledge base content directly affect the reliability of the system. The knowledge base of the CSCO AI mainly comes from clinical data and guideline evidence. The former is mostly semi-structured or unstructured, and the latter is structured data ³⁸. For the individual characteristics of patients in the literature (such as gender, age, PET, CT, MRI, blood tests, body fluid tests, stool tests, different tumor node metastasis (TNM) stages, molecular typing, past history, adverse reactions, etc.), mapping abstract data into graphic elements, supplemented by human-computer interactions, play the very important role in the application of knowledge retrieval, question and answer, knowledge recommendation, knowledge visualization and so on ^{39, 40}. It also provides an ideal technical method to solve the problem of "data islands", which is helpful to realize the integration of knowledge resources and improve the ability of knowledge services. For text-based data, it is necessary to conduct text analysis under the specifications of medical standard terms, such as semantic annotation and association analysis, to obtain all conceptual datasets about tumor diagnosis and treatment ⁷.

AI in the medical field usually focuses on acquiring knowledge from unstructured data, such as text (using natural language processing) or large structured datasets (using machine learning methods) ^41-43. After storing, indexing and delineating this knowledge, it uses computational reasoning methods to apply it to specific situations, generate and evaluate hypotheses, and then provide an evaluation for doctors ^{44, 45}. There are several diagnostic criteria for cancer, such as CancerLinq ⁴⁶ and a system called OncoDoc ⁴⁷. Through “reading” the literature, protocols and medical records, and learning from MSKCC test cases and experts, WFO obtains a large amount of knowledge as a preliminary knowledge base ^{23, 24}. At the same time, WFO also provides evidence to support recommended treatments, as well as specific case clinical trials, prescription information, potential adverse reactions related to treatment ¹¹. At present, WFO can provide rapid and accurate treatment suggestions for most cancer patients, compared the treatment regimens recommended for breast cancer cases between the Manipal multidisciplinary tumor board (MMDT) and WFO, the overall treatment concordance is up to 93% ¹¹. It can also play an important role in reducing doctors' workload and training young doctors. In addition, WFO would regulate the treatment of cancers nationwide and enhance the trust between doctors and patients, especially in rural hospitals ^{13, 48}.

Moreover, a CDSS should provide detailed evidence to support its recommendations, which is based on the credibility of the relevant literature ¹¹. Doctors can review the relevant evidence and judge whether it is applicable to the current case. When they choose to treat according to the recommended scheme, they also provide other information, such as the survival rate, incidence of adverse reactions, evidence supporting the recommended treatment, possible specific cases and clinical trials, prescription information, potential adverse reactions related to treatment to help doctors evaluate the efficacy and risk of the whole scheme. The CDSS can significantly shorten the time for junior doctors to consult the relevant literature and improve their ability to make accurate diagnosis and treatment suggestions in a short time. At the same time, it could eliminate the time cost by patients visiting top hospitals and help patients obtain the best treatment as soon as possible. Finally, the CDSS may solve the problem of doctor-patient trust. In contemporary China, patients' distrust of doctors is increasing due to many reasons, such as a shortage of funds, excessive market-oriented operation, limitations of medical insurance reimbursement and a large number of nonneutral reports of health events in the media ^49-51. In addition, patients often suspect doctors of engaging in overtreatment ⁵². The CDSS has no personal preferences. Therefore, it can be regarded as an objective and fair decision-making system and win the trust of patients. Cancer patients do not have to visit multiple experts to find a treatment they think is fair ^{53, 54}.

The difficulties of application and development of the CDSS

With the increasing popularity of WFO all over the world, in practice, many local doctors and medical institutions question the extent to whether WFO is suitable for cancer patients in their country. The problem is mainly emphasized in two aspects. On the one hand, a percentage of cancer cases is not supported by WFO. If the proportion of unsupported cases is very large (e.g., more than 50%), the total application value of WFO will be affected. On the other hand, in the supported cases, are the recommendations of WFO consistent with those of MDT? In addition, how can we make the decision-making opinions put forward by WFO be more suitable for the local people ³⁷?

In a study on lung cancer, among the medical records supported by the study on patients with isolated metastasis and patients with mutations in the driving gene of cancer progression, WFO did not support 18.1% of cases, among which 42% progressed after targeted therapy. The reason may be that the EGFR gene mutation phenotype of lung cancer in China is very different from that in the western countries. The mutation rate of the EGFR gene in patients with lung cancer is approximately 15% in Europe and the United States, while the probability of this mutation in China is 50% or higher ^55-57. Additionally, the drugs circulating in the market vary from country to country, such as immune checkpoint inhibitors involving PD-1 and PD-L1 antibodies. Patients' preferences, finance and medical insurance also need to be taken into account, which would eventually affect the consistency. Inconsistencies between WFO and MDT occurred in 7% of cases. 23% of these cases was due to differences in regulatory approval processes between countries, which can be remedied by incorporating locally approved therapies into the knowledge base of the expert system. The inconsistency may also be due to differences in treatment methods in patient subgroups affected by demographic characteristics such as the comorbidity burden, patient preference and social support system ^58-61.

There are several difficulties in the current clinical research on CDSS. The sample size of the existing research is very small, and most studies do not mention unsupported cases, which would affect the efficiency of clinical use ^{62, 63}. Although the doctors who input cases into the CDSS are familiar with the system and the key elements to be extracted during the chart review process, the quality and repeatability of these tasks have not been formally tested, which may affect the recommendations of the CDSS. Since the CDSS provides evidence for its decision-making, oncologists could examine the evidence and consider the basis of the CDSS recommendations. An unexpected recommendation by the CDSS may prompt clinicians to examine their evidence and reconsider it according to the evaluation of the MDT. Although the use of the CDSS may also help to ensure that treatment is more standardized at the appropriate time, when personal preferences or well-known cognitive biases dominate personal decision-making, there may be treatment options that are not objectively standardized ^{64, 65}. Finally, lack of consistency does not mean the recommendations of MDT or CDSS are “uncorrected”. There are many effective explanations for differences, such as the differences in treatment methods between comorbid or aging patients ^66-68.

Development direction of the CDSS

There is still much room for the development and improvement of the CDSS in the future. First, the update rate of the knowledge base needs to be further improved. Second, the abundance of input information and structured knowledge base information during training will reduce unsupported cases. Third, the CDSS feedback port should be more open, when there is no support case according to the treatment decision put forward by the CDSS, it should be discussed by the MDT and fed back to the CDSS for learning. After the treatment plan is proposed according to the decision, the dynamic changes in treatment also need to be fed back to the CDSS for learning. Fourth, the recommendation level of the CDSS for treatment decision-making should be quantified and evaluated according to the abundance of evidence. Fifth, when different countries introduce the CDSS, diagnosis and treatment equipment and drugs should be localized to improve the accessibility of the CDSS.

The storage capacity of an AI-based CDSS is much stronger than that of the human brain. It can quickly collect and sort stored information to draw accurate conclusions faster than humans, such as diagnostic radiology and pathological imaging systems ^69-71. However, to adapt to a real native medical environment, the CDSS must be significantly improved. After obtaining ethical approval, the medical data of patients should be standardized and shared nationwide, and the follow-up system should be improved to obtain the complete information of patients. In different countries, the unique medical data warehouse should be established for the CDSS research. These data should be combined with international guidelines and medical systems in different countries to enable the CDSS to give full potential to serve patients in different countries ^{11, 72, 73}.

It should be emphasized that the CDSS cannot replace oncologists at present ^{74, 75}. It should be positioned as a good assistant or teacher for young doctors ¹³. And there is another angle that is Collaborative intelligence, human and AI working synergistically. With the rapid development of CDSS, the CDSS for cervical cancer prevention automatically provides recommendations, accuracy of which is improved to 93% vs. the expert clinician as per the American Society of Colposcopy and Cervical Pathology (ASCCP) guidelines ⁷⁶. some doctors blindly follow the suggestions of the CDSS and think it will replace them in the future. However, just 60% of treatment pairs for breast, lung, colon, and rectal cancers are identical or equally acceptable, with 70% of WFO therapeutic options identical to, or acceptable alternatives to, Bumrungrad International Hospital therapy. What's more, colorectal cancers exhibited the highest proportion of identical or equally acceptable treatments; however, stage IV cancers demonstrate the lowest. Therefore, collaborative intelligence already exists and very likely to dominate in the future, the CDSS that could support, rather than replace the clinicians provides therapeutic options which are generally consistent with recommendations ⁷⁴. Also, medicine is not only a science but it also involves other aspects, such as social and psychological factors ^{77, 78}. Doctors must consider individualized measures for different patients, even those with the same diagnosis. When using the CDSS, oncologists are required to confirm whether patients can tolerate surgery or radiotherapy, or whether there is any emergency. The treatment plan could only continue if this information is confirmed. After it provides treatment suggestions, the most appropriate treatment scheme according to the patient's physical and mental status, financial status, complications and willingness to receive treatment.

In conclusion, the coincidence rate between the treatment decision made by the CDSS and the treatment decision of the MDT would be higher as AI technology continues to mature although the availability of drug treatment, professional treatment guidelines and the judgment of expert training will affect consistency. The AI-based CDSS may have a wide range of value in providing tumor treatment suggestions, especially at the hospitals where expert resources are not easy to obtain ¹¹. Before the CDSS acts as a doctor's assistant, scientists should continue to explore and innovate to further promote medical progress ¹⁴.

Author Contributions

Yinan Sun and Lu Wang: Writing - original draft. Lu Wang and Xinyi Chen: Writing - review & editing. Yongbiao Huang, Lu Zhang, Long Li and Xianglin Yuan: Writing - review & editing, Supervision.

Abbreviations

AI

Artificial intelligence

CDSS

clinical decision support system

CSCO AI

Chinese society of clinical oncology-artificial intelligence

WFO

Watson for Oncology

CSCO BC

Chinese society of clinical oncology-breast cancer

MSKCC

Memorial Sloan Kettering Cancer Center

NCCN

national comprehensive cancer network

MDT

multi-disciplinary team

TNM

tumor node metastasis

MMDT

Manipal multidisciplinary tumor board

ASCCP

American Society of Colposcopy and Cervical Pathology (ASCCP)