Explainable Case-based Diagnosis and Risk Prediction in Oncology

Abstract

In medical domains, precision medicine highlights the importance of diagnostic, prognostic, and therapeutic choices tailored to each patient, based on the multimodal data available. In particular, so-called confounding factors such as demographic and socio-economic factors affect the outcome, so that their inclusion in a predictive model is of paramount importance. In precision oncology, genetic data play a key role, along with confounding factors, so that large amounts of such data is available to researchers to help decipher the association between multifactorial variables and diagnosis, prognosis, and therapy. This project demonstrates that a case-based approach can be advantageous in the handling of confounding factors and in explainability, not to mention model efficiency. The effectiveness of the case-based approach is demonstrated on both diagnostic and prognostic tasks in comparison with state-of-the-art statistical and machine learning models while affording precise user explanations at the case level.

1. Introduction

To improve cancer diagnosis, risk stratification, and therapeutics, this article presents a case-based reasoning (CBR), diagnostic and prognostic ML model in oncology, O-CBR, proposing user explanations at the case level. This research affords:

1. Multitask modeling: the CBR system provides decision support recommendations in several tasks, including diagnosis and risk stratification, through classification and survival analysis models.

2. Multidimensional data modeling: the CBR system analyzes data from different types and dimensions, including demographic and socioeconomic data, and several types of genetic data (transcriptomic and epigenetic).

3. Effective and efficient modeling: the CBR system is at least as effective as state-of-the-art ML models used in this domain, including statistical models.

4. Explainability: the CBR system provides case-level explanations to users by design, through the discovery of genetic biomarkers, motifs and biological pathways.

2. Background

O-CBR combines the rich experience of CBR systems in medicine, enhanced with survival analysis, multitask capabilities, and explainability by design [1].

Case-based reasoning is a machine learning methodology related to analogical reasoning and cognitive modeling, with many applications in medical decision-making [2,3]. CBR proceeds by first selecting the case(s) that is/are the most similar to the new problem (retrieval step) and by adapting it/them, taking into account the differences between the retrieved case(s) and the new problem (reuse / adaptation step).

Survival Analysis.

Survival analysis is a well-known task aiming at predicting how likely an event is to happen over time. Major developments that contributed to this field are the Kaplan-Meier survival curve [4] and the Cox Proportional Hazard model [5].

Explainable AI.

In medical domains, human-competitive interpretability has been shown to be crucial for the adoption of clinical decision support systems [6, 7]. Instead of after the fact explanation methods [8], XAI researchers stress the importance of ML models that are interpretable by design [9]. Recent user studies have demonstrated that showing a retrieved case to users is the most important factor affecting users’ confidence in observed levels of goodness, convincingness, and trust (“Cases are king” [10]) and encourage the exploration of example-based explanations for XAI [11].

3. Methods

Fig. 1 show the system architecture Due to the high-dimensionality of the cases, up to 485,000 features for DNA methylation, feature selection was applied in each task.

Figure 1.

O-CBR architecture.

3.1. Diagnosis

Feature selection.

The feature selection process can be described as a multiscale and biologically significant reduction. After batch effects removal, differentially methylated regions (DMR) were selected, and features further reduced [12, 13].

Retrieval.

The retrieval process can be described as a two-stage retrieval tailored to each sample as an optimal number of similar cases is retrieved for each test case, using a confidence metric. During the two-stage process, memorized cases are retrieved by first finding cases with similar confounding factors. Then, the pool of retrieved cases is further refined using a Euclidean distance matrix from the other features.

3.2. Risk Prediction

If diagnosis is the first step in caring for a cancer patient, personalized therapy takes closely the severity risk under advisement.

Feature selection.

The feature selection process starts in the same manner as described in 3.1, as a multiscale and biologically significant reduction. To locate a subset of features highly specific to the overall survival of the cancer samples, multivariate Cox regression was applied to determine a prognostic score for each feature.

Retrieval.

To establish prognostic groups for survival, a multivariate Cox regression was constructed using each DNA methylation feature as a covariate in order to test for each probe’s contribution to survival. To find risk groups, the beta coefficient BetaCoeff for each feature j, among n, was multiplied by the DNA methylation β value MethylBetaValue and these values were then summed together to find a single prognostic score for each sample (see Equation 1). Samples were ranked by their prognostic scores and divided into equal-sized low, medium and high risk groups.

$$PrognosticScore={\sum }_{j=1}^n(BetaCoeff\ast MethylBetaValue)$$ (1)

A case-based retrieval framework was designed to automatically determine the appropriate number of retrieved cases for each test case, through a confidence metric. To better adjust for both the contribution of the prototypes and the potential power of each memorized case c at forming a solution for an unsolved case, we derived Equation 2.

$$Confidence(\mathrm{c})=1∕2dist(c,P)+\frac{d(c,m1)+\cdots +d(c,mk)}{k}$$ (2)

The confidence metric is one half of the distance d from case c to its prototype P, added to the average distance of case c to each memorized case m. The predicted survival time for a test case would then be the mean survival time of each retrieved case.

Adaptation.

Once cases were retrieved and their average survival duration was assigned to the test case, a case adaptation phase was attempted. Case adaptation is the ability to morph training cases to reflect a test case more closely. The distance between a retrieved and a test case was added or subtracted to the retrieved case survival.

Explanations.

In a CBR system, explanations may take several forms. The first form is to show the retrieved cases, with commonalities and differences with the test case. We also map genetic regions into pathways through gene set enrichment to display the main pathways involved in the commonalities and differences with the test case.

4. Results

The system’s effectiveness was experimentally tested on methylation data for the classification task and on multimodal methylation and transcriptomic data for the risk prediction task. The preferred performance metric for classification is the F1 score, particularly valued in binary classification where one class, the presence of disease, is more important. The preferred metric for survival analysis is the concordance index. For both metrics, higher values represent better performance, 1 being perfect performance.

4.1. Datasets

Classification Task.

Breast cancer data was downloaded from The Cancer Genome Atlas (TCGA). After data preprocessing, 782 cancer samples, 96 normal samples and 364,464 DNA methylation probes were used, then reduced to 8,772 features.

Survival Analysis Task.

The performance of the proposed method of survival analysis was studied on four cancer DNA methylation and gene expression datasets. We selected breast cancer (BRCA), Glioma cohort cancer (GBMLGG) and Pan-kidney cohort cancer (KIPAN) from TCGA (see Table 1).

Table 1:

Gene and clinical characteristics in three cancer types.

Characteristics	BRCA	BRCA	GBMLGG	KIPAN
Genetic data type	Gene Expression	DNA Methylation	DNA Methylation	DNA Methylation
Patient no.	761	763	650	654
Gene no.	1,151	8,725	20,116	20,533
Selected features	23	70	586	749
Survival status
Living	512	514	434	500
Deceased	249	249	216	154
Follow up (days)	1-1234	1-1234	1-481	3-5925

4.2. Experimental Results

Classification Task.

O-CBR performance was determined with and without feature selection using 10-fold cross validation and compared with k NN using 1/d as distance weight and an optimal k of 10; Random Forest with 500 iterations; Support Vector Machine; and Naive Bayes. All results indicate that integrating clinical covariates improves performance. The two-stage O-CBR outperformed all algorithms on the F1-score using all 8,772 features with high balanced accuracy over 95%. The results were close, but show the utility of clinical covariates and CBR for cancer classification.

Survival Analysis Task.

We compared O-CBR with state-of-the art survival prediction models: LASSO [14], and Random Survival Forests (RSF) [15]. For the sake of fairness, the same selected feature set was used in all 10-fold cross-validation tests.

Table 2 shows the O-CBR performed significantly better, at p=.05 level, than the other two methods on BRCA, GBMLGG and KIPAN datasets. On KIPAN dataset, RSF performed slightly better, but not significantly so.

Table 2:

Survival performance by concordance index.

Methods	BRCA Gene expression	BRCA Methylation	GBMLGG Methylation	KIPAN Methylation
LASSO	0.5	0.5	0.6215333	0.6574272
RSF	0.4656382	0.4660381	0.5629102	0.7084548
O-CBR	0.7537651	0.6795273	0.6708844	0.7039824

5. Conclusion

In this paper we presented and evaluated a multitask CBR system for oncology providing decision support for diagnosis and risk severity prediction from multidimensional data that is both effective and explainable. Our future research directions will focus on extending the system to additional tasks, datasets, dimensions, and to comparison with additional machine learning models such as Gradient Boosting and deep learning [16], based on LSTM or transformer models [17]. Once fully optimized, field test studies are planned to evaluate the system acceptance and usefulness in clinical settings.