Integrating Etiological Insights With Machine Learning for Precision Diagnosis of Obstructive Jaundice: Findings From a High-Volume Center

Abstract

INTRODUCTION:

Large-scale cohort studies exploring the etiology of obstructive jaundice (OJ) are scarce, with current serum-based diagnostic markers offering suboptimal performance. This study leverages the largest retrospective cohort of patients with OJ to date to investigate its disease spectrum and to develop a novel diagnostic system.

METHODS:

This study involves 2 retrospective observational cohorts. The biliary surgery cohort (BS cohort, n = 349) served for initial data exploration and external validation of machine learning (ML) models. The large general cohort (LG cohort, n = 5,726) enabled an in-depth analysis of etiologies and the determination of relevant diagnostic indicators, in addition to supporting ML model development. Interpretable ML techniques were used to derive insights from the models.

RESULTS:

The LG cohort highlighted a diverse disease spectrum of OJ, including cholangiocarcinoma (10.39% distal, 10.01% perihilar, and 5.59% intrahepatic), pancreatic adenocarcinoma (19.11%), and common bile duct stones (18.27%) as leading causes. Traditional serum markers such as carbohydrate antigen 19-9 and carcinoembryonic antigen lacked stand-alone diagnostic accuracy. Two ML-based models (collectively termed the ML of OJ based on common laboratory tests model) were developed: a classifier to differentiate benign from malignant causes (AUROC = 0.862) and a multiclass model to further stratify malignant and benign diseases (ACC = 0.777). Interpretable ML tools provided clarity on critical features, offering actionable insights and enhancing transparency in the decision-making process.

DISCUSSION:

This study elucidates the etiological spectrum of OJ, meanwhile providing a practical and interpretable ML-based diagnostic tool. By leveraging large-scale clinical data, our model provides a rapid and reliable primary assessment for patients with OJ, enabling clinicians to identify potential etiologies and guide further diagnostic workup.

INTRODUCTION

Obstructive jaundice (OJ) is a common problem associated with various hepatopancreatobiliary (HPB) diseases (1). Benign causes of OJ encompass a wide range of diseases, such as common bile duct stones (CBDSs), benign biliary strictures, and Mirizzi syndrome. Malignant causes typically involve tumors originating from the pancreas, bile ducts, or ampulla of Vater, including pancreatic adenocarcinoma, cholangiocarcinoma, ampullary carcinoma, and more (2–4). In addition, intrahepatic mass involving the hepatic hilus and metastatic tumors from other sites may also lead to this condition (5,6). These diverse etiologies require careful evaluation and management to determine the appropriate course of treatment.

Over the past few decades, significant advancements have been made in the diagnostic approaches for patients presenting with OJ. Serum-based diagnostics, including liver function tests and tumor markers, such as carbohydrate antigen 19-9 (CA 19-9) and carcinoembryonic antigen (CEA), have greatly facilitated the initial assessment of OJ. Advancements in imaging technology have further enhanced diagnostic accuracy of OJ-associated HPB diseases (7–9). Despite these advancements, challenges persist in these diagnostic approaches. Serum-based markers may lack specificity and sensitivity, limiting their utility as stand-alone diagnostic tools (10). Meanwhile, imaging modalities may encounter limitations in differentiating benign from malignant etiologies or accurately characterizing the extent of disease involvement, especially when the lesions are small (11). Moreover, access to advanced imaging techniques may be limited under certain healthcare settings, hampering timely diagnosis and management. Core needle biopsy, while useful for tissue sampling, may be limited by potential sampling error, complications, or difficulty in accessing anatomically challenging lesions (12,13). Recently, some liquid biopsy techniques have shown promising potential in detecting biomarkers associated with OJ, but these methods remain largely experimental and have yet to transition fully into clinical practice (14–16). Another major limitation is the lack of large-scale cohort studies, which hampers efforts to fully understand the disease spectrum underlying this condition.

Hence, harnessing the power of state-of-the-art machine learning (ML) methods, renowned for their ability to extract intricate patterns from complex data sets, we aimed to develop a robust diagnostic tool for patients presenting with OJ (17,18). This tool, enabling the integration of diverse clinical laboratory tests to enhance accuracy and efficiency of diagnosis, can be easily implemented because of its straightforward and user-friendly nature. Our objective was for it to not only differentiate between benign and malignant OJ but also to classify more specific etiologies based on this foundation. Its efficacy was validated in real-world settings using an external surgical cohort, with interpretable ML techniques used to offer transparency and insights into the decision-making process.

METHODS

Study setting and population

This study was conducted at West China Hospital of Sichuan University, a tertiary referral center and one of the largest medical institutions in Southwest China, serving a diverse population that includes both urban and rural patients. As a high-volume center, it receives referrals from across the region, providing a broad and representative sample of patients with OJ.

The study protocol was approved by the Ethics Committee for Biomedical Research at West China Hospital of Sichuan University and was preregistered in the Open Science Framework (registration DOI: https://doi.org/10.17605/OSF.IO/DC4B8). The study workflow is visualized in the graphical abstract.

Study design and cohorts

This study involved 2 retrospective observational cohorts from a single center:

1. Biliary surgery cohort (BS cohort): This cohort served as the data set for initial data exploration and external validation of ML models. It included patients presenting with OJ who were admitted to the Department of Biliary Surgery between February 2022 and September 2023. After screening with predefined criteria (see Supplementary Figure 1, http://links.lww.com/CTG/B319), 349 patients were included.

2. Large general cohort (LG cohort): This cohort was used for comprehensive data analysis and ML model construction. Data were reviewed from all hospitalized patients presenting with OJ in the general hospital between January 2008 and January 2022. After screening according to predefined inclusion/exclusion criteria, 5,726 patients were included.

Inclusion and exclusion criteria

The inclusion and exclusion criteria were designed to ensure a clear and consistent patient population for analysis. Inclusion criteria were as follows:

1. documented clinical manifestation of OJ (regardless of whether it was listed as a primary or secondary diagnosis);

2. reconfirmation of the diagnosis based on elevated cholestatic parameters (bilirubin, alkaline phosphatase, and γ-glutamyltransferase);

3. etiology pathologically confirmed by endoscopic retrograde cholangiopancreatography, percutaneous transhepatic cholangiography drainage, core needle biopsy, or surgical intervention; and

4. age older than 18 years.

Patients with OJ secondary to HPB surgery were excluded to rule out iatrogenic factors. These criteria delineated the spectrum of diseases covered by this study (Figure 1).

Figure 1.

Flowchart illustrating the patient selection process from an initial pool of 20,545 patients to establish the LG cohort. LG cohort, large general cohort.

Rationale for single-center design

While the single-center design may limit generalizability, West China Hospital's status as a high-volume referral center ensures a large and diverse patient population, including both primary and tertiary care cases. This setting provides a robust foundation for exploring the etiological spectrum of OJ and developing diagnostic models. However, we acknowledge that geographic and healthcare system differences may influence the applicability of our findings, and we have addressed this limitation in the Discussion section.

Collection of clinical data

The clinical data of each patient was retrieved from the medical record archive of our institution and underwent deidentification. The following information underwent further investigation: age, sex, clinical diagnosis, pathological report, and clinical laboratory test results. Given that the study was conducted at a single center in Southwest China, most of the patients were of Han Chinese ethnicity, reflecting the regional population demographics. More detailed information on data processing is available in the Supplementary Digital Content (see Supplementary Materials, http://links.lww.com/CTG/B319).

Development and validation of ML models

Based on these clinical data, we explored a diverse array of prediction models in 2 different tasks. The binary classification task focused on distinguishing between benign and malignant diseases, and the multiclass classification task intended to further categorize diseases into 5 detailed categories. Various ML algorithms were used to achieve best predictive performance, while internal validation and/or external validation were conducted for model evaluation. These models were meticulously interpreted. More detailed information on the ML technique is available in the Supplementary Digital Content (see Supplementary Materials, http://links.lww.com/CTG/B319).

Statistical analysis

Statistical analyses were conducted using R software. Shapiro-Wilk test and QQ plot were used to assess the normality of data distribution. For comparisons between groups, independent samples t-tests and Mann-Whitney U tests were performed for continuous variables with and without normal distribution, respectively, while χ² tests were conducted for categorical variables. DeLong test was used for the assessment of model performance. A significance level of P < 0.05 was considered statistically significant.

RESULTS

Patient profiles and disease spectrums of the study cohorts

The biliary surgery cohort (BS cohort) included 349 consecutive surgical inpatients with OJ (see Supplementary Figure S1, http://links.lww.com/CTG/B319). Demographically, there were 216 (62%) male and 133 (38%) female patients, with 60% of them older than 60 years. In terms of disease spectrum, there were 204 cases (58.5%) with malignant OJ and 145 cases (41.5%) with benign etiologies (see Supplementary Figure S2A, http://links.lww.com/CTG/B319). Because the BS cohort exclusively included patients from a biliary surgery center, these patients were predominantly associated with biliary malignancies, accounting for 49.0% of all types of diseases and 86.8% of all malignancies (see Supplementary Figure S2C, http://links.lww.com/CTG/B319). Calculous diseases, including CBDS and hepatolithiasis, accounted for 67.6% of all benign causes.

Undoubtedly, these preliminary results may not accurately reflect the true distribution of diseases in the general population, given that patients were selectively admitted to the surgical ward. Therefore, a LG cohort comprising 5,726 patients from a comprehensive medical center over a span of 14 years was analyzed (Figure 1). Similarities were observed between the LG and BS cohorts in terms of sex, age, and the relative proportion of benign and malignant diseases, underscoring the representativeness of our previous observations (see Supplementary Figure S2B, http://links.lww.com/CTG/B319). Still, statistical analysis revealed significant differences in the disease spectrum and other baseline characteristics between the 2 cohorts (see Supplementary Table 1, http://links.lww.com/CTG/B319). Based on a larger sample size, the LG cohort was able to unveil some previously undisclosed insights into OJ. To sum up, biliary malignancies (1,657 cases, 28.94%), pancreatic malignancies (1,106 cases, 19.32%), ampullary malignancies (252 cases, 4.40%), hepatic malignancies (190 cases, 3.32%), metastatic cancers (360 cases, 6.29%), and other rare malignancies (155 cases, 2.71%) built up the malignant side of the disease spectrum, while calculous diseases (1,257 cases, 21.95%), inflammatory diseases (566 case, 9.88%), and other benign causes (171 cases, 2.99%) composed the benign counterpart (Figure 2). The top 5 leading causes of OJ were revealed as pancreatic adenocarcinoma (1,094 cases, 19.11%), CBDS (1,046 cases, 18.27%), distal cholangiocarcinoma (595 cases, 10.39%), perihilar cholangiocarcinoma (573 cases, 10.01%), and acute pancreatitis (noncalculous) (328 cases, 5.73%).

Figure 2.

Summary chart depicting the disease spectrum of OJ observed in the LG cohort, highlighting the prevalence of various benign and malignant etiologies. LG cohort, large general cohort; OJ, obstructive jaundice.

Requirement for ML-based diagnostics in OJ

Our next objective was to ascertain whether a more effective diagnostic approach is warranted in distinguishing benign and malignant obstructions. A comparative analysis of the baseline characteristics between benign and malignant groups was conducted in the BS cohort and revealed a significant difference in multiple clinical indicators (see Supplementary Table 2 and Supplementary Figure 3A, http://links.lww.com/CTG/B319). However, the diagnostic performance of single laboratory markers to distinguish benign from malignant conditions was suboptimal. The top 5 diagnostic markers were CA 19-9 (AUROC = 0.768), direct bilirubin (DBIL; AUROC = 0.736), total bilirubin (TBIL; AUROC = 0.730), CEA (AUROC = 0.697), and DBIL/TBIL ratio (AUROC = 0.696) (see Supplementary Figure 3B, http://links.lww.com/CTG/B319).

The LG cohort was analyzed to validate these results based on a larger sample size. Comparative analysis unveiled statistical variances in the baseline characteristics between benign and malignant groups (Table 1).

Table 1.

Comparative analysis of baseline characteristics between benign and malignant groups in the LG cohort

Variables	LG cohort (borderline disease excluded, n = 5,714)
	Benign obstruction (n = 1994)	Malignant obstruction (n = 3,720)	P
Sex			0.36
Male	1,151 (57.7%)	2,099 (56.4%)
Female	843 (42.3%)	1,621 (43.6%)
Age (yrs)	53.9 (16.1)	59.9 (11.7)	<0.01
CA 19-9 (U/mL)	31.43 [13.63–84.61]	132.64 [34.76–306.48]	<0.01
CEA (ng/mL)	2.12 [1.35–3.35]	3.41 [2.07–6.39]	<0.01
CA 125 (U/mL)	19.33 [11.61–41.29]	24.35 [14.15–51.65]	<0.01
AFP (ng/mL)	2.64 [1.90–3.75]	2.92 [2.14–4.23]	<0.01
TBIL (μmol/L)	53.0 [42.4–81.7]	54.8 [44.3–119.4]	<0.01
DBIL (μmol/L)	21.3 [13.3–51.1]	24.7 [16.5–88.1]	<0.01
IBIL (μmol/L)	9.2 [6.6–13.2]	8.0 [5.6–11.9]	<0.01
ALT (IU/L)	42 [24–73]	39 [26–63]	0.20
AST (IU/L)	41 [27–75]	33 [24–57]	0.02
ALP (IU/L)	139 [87–226]	131 [82–229]	0.42
γ-GT (IU/L)	119 [37–288]	81 [27–195]	<0.01
ALB (g/L)	38.5 (5.3)	37.5 (5.8)	<0.01
Glo (g/L)	28.5 (6.0)	27.2 (5.2)	<0.01
A/G	1.37 [1.17–1.56]	1.41 [1.23–1.61]	0.07
GLU (mmol/L)	5.74 [5.17–6.68]	5.83 [5.23–6.86]	0.36
UREA (mmol/L)	4.8 [3.9–5.8]	4.7 [3.9–5.6]	0.59
CREA (μmol/L)	55.48 [52.00–63.98]	53.27 [48.11–61.00]	0.04
CysC (mg/L)	0.95 [0.84–1.08]	0.94 [0.83–1.06]	0.24
UA (μmol/L)	228 (79)	207 (89)	0.03
TG (mmol/L)	1.33 [1.02–1.72]	1.37 [1.03–1.80]	0.38
CHOL (mmol/L)	4.20 [3.45–5.01]	4.26 [3.42–5.30]	0.25
HDL (mmol/L)	1.00 [0.61–1.32]	0.87 [0.41–1.29]	<0.01
LDL (mmol/L)	2.16 (0.87)	2.06 (0.87)	0.06
CK (IU/L)	65 [44–104]	71 [47–101]	0.31
LDH (IU/L)	177 [155–203]	179 [157–201]	0.79
HBDH (IU/L)	137 [124–159]	138 [123–156]	0.78
RBC (×10^12/L)	3.84 (0.79)	3.58 (0.65)	<0.01
HGB (g/L)	115 (22)	109 (19)	<0.01
HCT (L/L)	0.35 (0.07)	0.33 (0.06)	<0.01
MCV (fL)	93.0 [89.5–96.3]	93.7 [90.2–97.0]	<0.01
MCHC (g/L)	328 (11)	327 (11)	0.26
MCH (pg)	30.6 [29.3–31.7]	30.8 [29.6–32.0]	<0.01
RDW-CV (%)	14.6 [13.5–16.0]	15.2 [14.2–16.6]	<0.01
RDW-SD (fL)	49.1 [45.1–53.5]	51.6 [47.9–55.9]	<0.01
PLT (×109/L)	187 [138–241]	194 [148–245]	<0.01
MPV (fL)	11.9 (1.2)	11.8 (1.2)	0.35
PDW (%)	15.3 (3.2)	15.1 (3.1)	0.08
P-LCR% (%)	39.9 (9.8)	39.5 (9.9)	0.27
WBC (×109/L)	6.97 [5.54–9.26]	7.36 [5.74–9.52]	<0.01
NEUT% (%)	72.0 [65.0–78.9]	73.6 [67.3–79.3]	<0.01
LYM% (%)	17.3 [11.6–23.9]	15.0 [10.7–20.9]	<0.01
MONO% (%)	7.0 [5.9–8.3]	7.2 [6.2–8.5]	<0.01
EO% (%)	1.4 [0.8–2.3]	1.5 [0.9–2.4]	0.04
BASO% (%)	0.4 [0.3–0.5]	0.4 [0.3–0.5]	0.58
NEUT# (×109/L)	4.81 [3.58–6.95]	5.27 [3.87–7.44]	<0.01
LYM# (×109/L)	1.17 [0.85–1.53]	1.09 [0.83–1.41]	<0.01
MONO# (×109/L)	0.48 [0.38–0.62]	0.52 [0.41–0.66]	<0.01
EO# (×109/L)	0.09 [0.05–0.16]	0.10 [0.06–0.16]	<0.01
BASO# (×109/L)	0.03 [0.02–0.04]	0.03 [0.02–0.04]	0.26
PT (s)	11.7 [10.9–12.8]	11.6 [11.0–12.6]	0.38
INR	1.04 [0.97–1.14]	1.03 [0.97–1.12]	0.06
APTT (s)	28.1 [26.1–30.8]	28.0 [26.1–30.8]	0.85
Fbg (g/L)	3.45 (1.24)	3.66 (1.08)	<0.01
TT (s)	18.4 [17.3–19.5]	18.2 [17.1–19.4]	0.03
CRP (mg/L)	14.8 [4.7–42.4]	19.8 [6.8–72.5]	<0.01

Continuous variables with normal distribution are presented as mean value (SD) while others are presented as median [interquartile range (IQR)]. Categorical variables are presented as frequency (percentage).

AFP, α-fetoprotein; A/G, albumin/globulin ratio; ALB, albumin; ALP, alkaline phosphatase; ALT, alanine aminotransferase; AST, aspartate aminotransferase; BASO#, basophil count; BASO%, basophil percentage; CA 125, carbohydrate antigen 125; CA 19-9, carbohydrate antigen 19-9; CEA, carcinoembryonic antigen; CHOL, cholesterol; CK, creatine kinase; CREA, creatinine; CRP, C-reactive protein; CysC, cystatin C; DBIL, direct bilirubin; EO#, eosinophil count; EO%, eosinophil percentage; Fbg, fibrinogen; Glo, globulin; GLU, glucose; HBDH, hydroxybutyrate dehydrogenase; HCT, hematocrit; HDL, high-density lipoprotein cholesterol; HGB, hemoglobin; IBIL, indirect bilirubin; INR, international normalized ratio; LDH, lactate dehydrogenase; LDL, low-density lipoprotein cholesterol; LG, large general cohort; LYM#, lymphocyte count; LYM%, lymphocyte percentage; MCH, mean corpuscular hemoglobin; MCHC, mean corpuscular hemoglobin concentration; MCV, mean corpuscular volume; MONO#, monocyte count; MONO%, monocyte percentage; MPV, mean platelet volume; NEUT#, neutrophil count; NEUT%, neutrophil percentage; PDW, platelet distribution width; P-LCR%, large platelet ratio; PLT, platelet count; PT, prothrombin time; RBC, red blood cell count; RDW-CV, red cell distribution width-coefficient of variation; RDW-SD, red cell distribution width-standard deviation; TBIL, total bilirubin; TG, triglycerides; TT, thrombin time; UA, uric acid; UREA, urea; WBC, white blood cell count; γ-GT, γ-glutamyl transferase.

A greater number of laboratory indicators were identified to show statistically significant distinctions in the LG cohort as opposed to the BS cohort. By contrast, the diagnostic efficacy of tumor markers in the LG cohort was generally less satisfactory (Figure 3a). Among the top 5 indicators, only CA 19-9 had an AUROC exceeding 0.7 (AUROC = 0.712), followed by CEA (AUROC = 0.685), age (AUROC = 0.617), RDW-SD (AUROC = 0.616), and DBIL (AUROC = 0.613) (Figure 3b). The DeLong test indicated a significant decrease in diagnostic efficacy of indicators including CA 19-9 and DBIL in the LG cohort (Figure 3c). These results suggested that in a larger, more comprehensive patient cohort with a more complex diagnostic environment, individual serum markers are not robust diagnostic tools for patients presenting with OJ. Hence, there is a necessity to build a more efficient diagnostic tool.

Figure 3.

Necessity and feasibility of constructing a ML model for the clinical diagnosis of OJ. Traditional serum makers, including (a) tumor makers and (b) top 5 biomarkers ranked by AUROC, were found to exhibit suboptimal diagnostic efficacy in the LG cohort. (c) In the LG cohort, the diagnostic scenario becomes more intricate, causing markers such as CA 19-9 and DBIL to exhibit diminished efficacy compared with the BS cohort, necessitating the development of combined ML models. (d) The number of features included in the ML model significantly influenced its diagnostic efficacy, as the ML model with 57 features outperformed the others (DeLong test p < 0.01). (e) The included features encompass a wide spectrum of clinical characteristics including demographic features, tumor markers, complete blood count, comprehensive metabolic panel, clotting screen and inflammatory markers. AFP, α-fetoprotein; A/G, albumin/globulin ratio; ALB, albumin; ALP, alkaline phosphatase; ALT, alanine aminotransferase; APTT, activated partial thromboplastin time; AST, aspartate aminotransferase; BASO#, basophil count; BASO%, basophil percentage; CA 125, carbohydrate antigen 125; CA 19-9, carbohydrate antigen 19-9; CEA, carcinoembryonic antigen; CHOL, cholesterol; CK, creatine kinase; CREA, creatinine; CRP, C-reactive protein; CysC, cystatin C; DBIL, direct bilirubin; dCCA, distal cholangiocarcinoma; EO#, eosinophil count; EO%, eosinophil percentage; Fbg, fibrinogen; Glo, globulin; GLU, glucose; HBDH, hydroxybutyrate dehydrogenase; HCT, hematocrit; HDL, high-density lipoprotein cholesterol; HGB, hemoglobin; IBIL, indirect bilirubin; INR, international normalized ratio; LDH, lactate dehydrogenase; LDL, low-density lipoprotein cholesterol; LYM#, lymphocyte count; LYM%, lymphocyte percentage; MCH, mean corpuscular hemoglobin; MCHC, mean corpuscular hemoglobin concentration; MCV, mean corpuscular volume; MONO#, monocyte count; MONO%, monocyte percentage; MPV, mean platelet volume; NEUT#, neutrophil count; NEUT%, neutrophil percentage; PDW, platelet distribution width; P-LCR%, large platelet ratio; PLT, platelet count; PT, prothrombin time; RBC, red blood cell count; RDW-CV, red cell distribution width-coefficient of variation; RDW-SD, red cell distribution width-standard deviation; TBIL, total bilirubin; TG, triglycerides; TT, thrombin time; UA, uric acid; UREA, urea; WBC, white blood cell count; ML, machine learning; OJ, obstructive jaundice; γ-GT, γ-glutamyl transferase.

We first attempted a traditional linear regression model to integrate the diagnostic power of multiple test indicators. The diagnostic model construction process, using stepwise logistic regression, incorporated a selection procedure of 57 variables (see Supplementary Figure 4A, http://links.lww.com/CTG/B319). The diagnostic efficacy of this model was moderate, achieving AUROC values of 0.784 and 0.791 in the internal and external validation sets, respectively. Notably, when compared with the subsequently established ML model, it presented significantly lower AUROC values as well as inadequate sensitivity and specificity (see Supplementary Figures 4B and 4C, http://links.lww.com/CTG/B319). Similarly, in subsequently established ML models, the number of features included in the model significantly influenced its diagnostic efficacy (Figure 3d). These results highlighted the inherent advantage of ML techniques in leveraging the diagnostic power of multiple parameters or features. Therefore, the subsequently established ML models included all 57 features delineating common dimensions of disease characteristics (Figure 3e).

Establishment, validation, and interpretation of binary ML of OJ based on common laboratory tests model to distinguish benign and malignant obstructions

After confirming the feasibility of constructing a ML diagnostic model based on 57 common clinical features, we proceeded to optimize its diagnostic performance. Several mainstream ML methods were used to construct this model, with their performance compared with that of the optimal one. The DeLong test between ROC curves was conducted to finalize our choice (see Supplementary Figure 5, http://links.lww.com/CTG/B319). The RF model ultimately stood out for its remarkable performance both in the internal and external validation sets (Figure 4a–c and Table 2).

Figure 4.

Establishment, validation, and interpretation of the binary MOLT model. (a) In the internal validation set, the lightGBM model showcased best performance measured by AUROC, while (b) the RF model showcased best performance in the external validation set. (c) The RF model was subsequently designated as the MOLT model, which exhibited better performance compared with traditional CA 19-9 in the external validation set. Interpretation of the MOLT model revealed (d) features with the top ranked feature importance score, (e) features with top ranked SHapley Additive exPlanations (SHAP) values, and (f) features with the top ranked interaction score. (g) Partial dependence plot (PDP) and individual conditional expectation (ICE) plots were created for the top 3 features (age, CA 19-9, and CEA) identified by SHAP values, elucidating their impact on the model's predictions. (h) The SHAP-interpreted ML model clarified individualized decision-making processes, offering understanding into prediction rationale for each case. AFP, α-fetoprotein; ALB, albumin; AST, aspartate aminotransferase; CA 19-9, carbohydrate antigen 19-9; CBDS, common bile duct stones; CEA, carcinoembryonic antigen; CHOL, cholesterol; CK, creatine kinase; DBIL, direct bilirubin; ERCP, endoscopic retrograde cholangiopancreatography; Fbg, fibrinogen; LDH, lactate dehydrogenase; LDL, low-density lipoprotein cholesterol; ML, machine learning; MOLT, ML of OJ based on common laboratory tests; MONO#, monocyte count; PLT, platelet count; RDW-CV, red cell distribution width-coefficient of variation; RDW-SD, red cell distribution width-standard deviation; TBIL, total bilirubin; UREA, urea.

Table 2.

Comparison of performance among various ML methods used for construction of the binary MOLT model

Cohort	Model	Index
		AUROC (95% CI)	ACC	AUPR	F1 score	Sensitivity	Specificity
LG cohort (training)	Random forest	1.000 (1.000–1.000)	0.996	1.000	0.997	0.987	1.000
	LightGBM	0.999 (0.999–1.000)	0.983	0.999	0.988	0.956	0.996
	XGBoost	0.891 (0.878–0.904)	0.836	0.842	0.888	0.573	0.961
	Decision tree	0.836 (0.820–0.853)	0.825	0.744	0.874	0.681	0.894
	SVM	0.920 (0.907–0.932)	0.872	0.892	0.911	0.680	0.963
	Logistic regression	0.811 (0.793–0.829)	0.787	0.707	0.855	0.501	0.923
	KNN	0.992 (0.990–0.995)	0.907	0.982	0.935	0.731	0.990
LG cohort (internal validation)	Random forest	0.875 (0.855–0.896)	0.824	0.816	0.878	0.566	0.955
	LightGBM	0.907 (0.891–0.923)	0.842	0.851	0.887	0.650	0.938
	XGBoost	0.852 (0.832–0.873)	0.798	0.774	0.862	0.504	0.947
	Decision tree	0.826 (0.803–0.849)	0.791	0.748	0.848	0.613	0.881
	SVM	0.817 (0.793–0.842)	0.772	0.719	0.841	0.500	0.910
	Logistic regression	0.781 (0.754–0.809)	0.757	0.672	0.835	0.431	0.923
	KNN	0.713 (0.684–0.743)	0.705	0.582	0.799	0.354	0.883
BS cohort (external validation)	Random forest	0.862 (0.819–0.904)	0.828	0.865	0.864	0.676	0.936
	LightGBM	0.822 (0.774–0.870)	0.802	0.827	0.841	0.669	0.897
	XGBoost	0.725 (0.667–0.782)	0.699	0.709	0.768	0.483	0.853
	Decision tree	0.678 (0.615–0.741)	0.731	0.689	0.783	0.586	0.833
	SVM	0.856 (0.812–0.900)	0.808	0.844	0.828	0.834	0.789
	Logistic regression	0.800 (0.754–0.846)	0.722	0.734	0.771	0.614	0.799
	KNN	0.816 (0.770–0.862)	0.751	0.781	0.806	0.559	0.887

Cells set in bold/italic represent the first/second best performance in each cohort.

ACC, accuracy; AUPR, area under the precision-recall curve; CI, confidence interval; KNN, k-nearest neighbors; ML, machine learning; MOLT, machine learning of obstructive jaundice based on common laboratory tests; SVM, support vector machine.

Subsequently, the RF model was selected and designated as the binary ML of OJ based on common laboratory tests (MOLT) model. To present the decision-making process of the binary MOLT model in a more transparent manner, we used several methods for model interpretation. Feature importance scores highlighted the top ranking features distinguishing benign from malignant etiologies. The top 10 features were identified as age, CA 19-9, CEA, cholesterol (CHOL), albumin (ALB), DBIL, albumin/globulin ratio (A/G), RDW-SD, fibrinogen, and aspartate aminotransferase (Figure 4d and see Supplementary Figure 6A, http://links.lww.com/CTG/B319). While feature importance scores provide a global view of feature importance across the data set, SHAP values offer a more nuanced understanding of how each feature influences individual predictions, taking interactions and dependencies between features into account (19,20). Top 10 features ranked by SHAP value were CA 19-9, CEA, age, ALB, platelet count, CHOL, red cell distribution width-coefficient of variation, RDW-SD, fibrinogen, and DBIL (Figure 4e and see Supplementary Figure 6B, http://links.lww.com/CTG/B319). The overall interaction strength provided insights into the complexity of relationships between predictor variables in the binary MOLT model (Figure 4f). Partial dependence plots and individual conditional expectation plots were generated for the top 3 features identified by SHAP values (age, CA 19-9, and CEA), illustrating how individual feature values affect the model's predictions (Figure 4g). With the SHAP-interpreted ML model, individualized decision-making processes were elucidated, allowing for a comprehensive understanding of how the model arrives at predictions for each specific case (Figure 4h). Decision boundaries pertaining to key features were also visualized (see Supplementary Figure 7, http://links.lww.com/CTG/B319).

Establishment, validation, and interpretation of multiclass MOLT model for further classification

Building upon the initial model, we converted the binary classification target into a multiclassification target to construct a more complex model, namely the multiclass MOLT model, which differentiates between calculous benign lesions, noncalculous benign lesions, metastatic malignancies, pancreatobiliary malignancies, and other types of malignancies (Figure 5a). Similar to the original binary MOLT model, a series of ML methods were used, with the best performing one selected to optimize model performance (Figure 5b). Notably, external validation was unable to be performed for the multiclass MOLT model because there were no metastatic patient in the BS cohort. Outperforming the others, the XGBoost model showcased impressive diagnostic efficiency, boasting an accuracy (ACC) of 0.777 and area under the NPV-utility curve (AUNU) of 0.882, a notable achievement given the complexity of the task encompassing 5 classes.

Figure 5.

Establishment, validation, and interpretation of the multiclass MOLT model. (a) Extending the binary MOLT model, a 5-class ML task was formulated. (b) The performance of diverse multiclass models was gauged using metrics such as ACC, AUNU, macro F1 score, precision, and recall scores. Similarly, the decision-making process of the multiclass MOLT model was elucidated with (c) feature importance score and (d) SHAP value. LG cohort, large general cohort; MOLT, machine learning of obstructive jaundice based on common laboratory tests.

ML interpretability tools were also used to explain the multiclass MOLT model. Feature importance scores highlighted the top ranking features contributing to this model, namely CA 19-9, age, CEA, ALB, CHOL, α-fetoprotein (AFP), uric acid, A/G, red blood cell count (RBC), and CA125 (Figure 5c). SHAP values were used to assess how individual features influenced the model's decisions within each disease category (Figure 5d). ALB, A/G, CA 19-9, CEA, and aspartate aminotransferase were the top 5 features contributing to the diagnosis of benign calculous disease; CA19-9, platelet distribution width, ALB, CEA, and large platelet ratio were the top 5 features contributing to the diagnosis of benign noncalculous disease; CEA, age, TBIL, CA 19-9, and RBC were the top 5 features contributing to the diagnosis of metastatic malignancies; ALB, CA 19-9, CEA, A/G, and RBC were the top 5 features contributing to the diagnosis of pancreatobiliary malignancies; while AFP, platelet count, ALB, A/G, and high-density lipoprotein cholesterol were the top 5 features contributing to the diagnosis of other malignancies.

DISCUSSION

Until now, large-scale cohorts regarding OJ remain scarce, which has led to a limited understanding of its disease spectrum and efficacy of existing diagnostic approaches. Our study included the largest retrospective cohort of OJ to date, providing valuable insights into its disease spectrum and laboratory test data through comprehensive analysis. Consequently, we developed ML models based on common clinical laboratory tests, which not only distinguishes between benign and malignant obstructions but also further differentiates between calculous benign lesions, noncalculous benign lesions, metastatic malignancies, pancreatobiliary malignancies, and other types of malignancies. To ensure transparency in the decision-making process, interpretable ML tools were used to decipher these models.

At first, it is important to address the extent to which this diagnostic model can aid in clinical decision making and ultimately improve patient outcomes. In our view, the MOLT model is not intended to replace imaging or interventional procedures, such as MRCP or endoscopic retrograde cholangiopancreatography, which remain essential for definitive diagnosis and treatment. Instead, the primary value of our model lies in its ability to provide a rapid, noninvasive, and interpretable primary assessment or quick assessment for patients with OJ. By leveraging commonly available laboratory indices, the MOLT model can help clinicians prioritize potential etiologies early in the diagnostic process. This can guide the selection of subsequent diagnostic tests (e.g., diverse imaging modalities) and streamline the diagnostic pathway, potentially reducing delays in diagnosis and treatment initiation. For instance, in resource-limited settings or for patients who cannot immediately undergo advanced imaging, our model could serve as a valuable triage tool. This approach aligns with strategies previously proposed for colorectal cancer diagnosis and other public health initiatives (21,22).

Before establishing the model, the disease spectrum of OJ needs to be carefully examined because it directly affects diagnosis, treatment, and healthcare resource allocation. Undoubtedly, the disease spectrum of OJ may vary across different countries and regions. However, the dearth of knowledge in this field has restricted our understanding in this field. Our study emerged as a significant contribution by offering a comprehensive analysis of over 5,000 patients with OJ in a single Chinese center over a period of 14 years. Smaller retrospective cohort studies conducted across Europe, Australia, Central Asia, and South Asia provided valuable insights into OJ (23–29). Notably, the retrospective analysis by Garcea et al of over 1,000 cases in the United Kingdom found similar disease patterns to ours, with CBD stones and pancreatic ductal adenocarcinoma as primary benign and malignant etiologies, respectively (26). Similarly, the analysis by Björnsson et al (23) of 241 patients in Sweden revealed a higher incidence of malignant obstruction (63.9%) compared with benign cases, with cholangiocarcinoma accounting for one-third of malignant obstructions, mirroring our findings. These findings suggest that the disease spectrum of OJ may be more consistent across different regions than previously believed. Moreover, our study supplemented these insights by revealing additional dimensions. First, we observed that noncalculous benign etiologies might have been underreported because most of the previous studies identified CBDS as the predominant benign cause. Our findings indicated that CBDS only accounted for approximately half of the benign cases, while around one-third were associated with diverse noncalculous factors. In addition, there may have been an underestimation of metastatic causes of OJ, considering the lower likelihood of obtaining a pathological diagnosis in these patients. Furthermore, intrahepatic lesions involving the hepatic hilus, mainly intrahepatic cholangiocarcinoma and hepatocellular carcinoma, constitute a significant proportion (approximately 10%) of all cases.

Our study also yielded valuable insights into diagnostics. We not only presented a practical diagnostic tool for distinguishing between different causes of OJ but also enhanced comprehension by providing transparency into the decision-making process. According to previous studies, calculous diseases in benign OJ can be accurately distinguished through various imaging modalities (30–32). However, distinguishing noncalculous benign diseases from malignant diseases can be considerably challenging (33–36). Because our study revealed that over one-third of the benign cases were associated with noncalculous etiologies, there is a need to place greater emphasis on addressing this issue. Meanwhile, in the realm of malignancy, clinicians seek to stratify cancers according to their level of aggressiveness. To this end, we used ML techniques to develop 2 models: Onene effectively distinguishes between benign and malignant causes, while the other offers nuanced insights by further classifying malignancies into 3 tiers and benign diseases into 2. Subsequently, these models may facilitate the application of appropriate diagnostic and therapeutic interventions.

Although we do not claim that our model can replace current diagnostic standards, we believe it complements existing methods by offering a cost-effective, accessible, and efficient preliminary assessment. This could be particularly beneficial in scenarios where rapid triage or resource optimization is critical. It is noteworthy that the concept of integrating diverse laboratory test results into a unified model did not arise arbitrarily. There has long been ample evidence pointing toward this direction. For instance, multiple studies have observed that patients with malignant obstruction tends to be older than the benign group (23,27). Similarly, benign OJ is observed to be associated with lower bilirubin levels because biliary obstructions caused by calculous disease tend to be intermittent (1,26). Furthermore, there is documented evidence suggesting an association between OJ and renal injury, with the severity of renal dysfunction potentially reflecting the nature of the disease (37–39). In this study, by leveraging interpretable ML models, we gained further insight into how traditional and novel biomarkers contribute to the diagnostic model. Although established markers such as CA 19-9, CEA, age, and bilirubin levels remained significant, factors like albumin levels, cholesterol levels, and red cell distribution width (RDW-SD) emerged as noteworthy contributors, offering new dimensions to the diagnostic evaluation of OJ.

CA 19-9, with a stand-alone AUROC of 0.712, demonstrated the highest predictive value among the top 5 indicators. This aligns with its well-established role as a tumor marker for pancreatobiliary malignancies, particularly pancreatic ductal adenocarcinoma and cholangiocarcinoma (40,41). However, its elevation in benign conditions, such as cholangitis or biliary obstruction, underscores the need for careful interpretation in clinical practice (10,42).

CEA, another well-established tumor marker, showed a stand-alone AUROC of 0.685. While its sensitivity and specificity for OJ are generally lower than those of CA 19-9, it remains a valuable adjunct in differentiating malignant from benign causes (43,44).

Age (stand-alone AUROC = 0.617) emerged as a significant predictor, consistent with the known association between advancing age and an increased risk of malignant biliary diseases. Older patients are more likely to present with malignancies such as pancreatic cancer or cholangiocarcinoma, whereas benign causes such as choledocholithiasis are more prevalent in younger populations (45).

DBIL, with a stand-alone AUROC of 0.613, reflects the severity of biliary obstruction. Our findings suggest that DBIL levels are typically higher in malignant OJ, likely because of the more complete and persistent nature of obstruction in malignancies (46). By contrast, benign OJ, often caused by conditions such as CBDS, may present with intermittent or partial obstruction, leading to relatively lower DBIL levels (47).

Finally, RDW-SD (stand-alone AUROC = 0.616), although less conventional, represents an intriguing biomarker in the context of OJ. Traditionally associated with hematologic disorders, RDW-SD has been linked to systemic inflammation and liver dysfunction, particularly in decompensated liver cirrhosis (48). Its inclusion in our model highlights the complex interplay between hepatic dysfunction and systemic inflammatory responses, warranting further investigation into its diagnostic and prognostic utility.

Several limitations of this study should be noted. First, because this study exclusively enrolled patients with a confirmed pathological diagnosis, the findings regarding the proportions of specific types of diseases contributing to OJ may be subject to bias because of variations in the likelihood of different diseases to be biopsied. Second, this is a single-center study, primarily involving patients from China. As a result, the potential impact of geographic variations among patients and differences in detection methods across various clinical laboratories on the study outcomes was not addressed, although we observed similarities in our findings with previous research studies from other regions. Furthermore, our MOLT model exhibited exceptional specificity; however, its sensitivity fell short of expectations. This underscores the need for future studies to improve both the specificity and sensitivity of diagnostics. Despite these limitations, our study stands as the largest cohort study in the field of OJ to date, with robust diagnostic tools developed through the utilization of state-of-the-art techniques.

To conclude, our study developed the MOLT model for the diagnosis of patients presenting with OJ, which may facilitate personalized and user-friendly clinical decision making of this condition. By providing a rapid, interpretable, and noninvasive primary assessment, our model complements existing diagnostic methods and has the potential to streamline clinical workflows, particularly in resource-limited settings or for patients requiring immediate triage.

CONFLICTS OF INTEREST

Guarantor of the article: Nansheng Cheng, PhD.

Specific author contributions: N.W., Y.W., B.L., J.L., G.L. and N.C.: study concept and design. N.W., Y.W., Y.T., B.L., J.L. and N.C.: data acquisition. N.W., Y.W., G.L., J.X. and D.Z.: data analysis and interpretation. N.W., Y.W., X.P. and G.L.: implementation of machine learning. N.W., S.W., G.L., B.L., J.L.: and N.C.: drafting of the manuscript. X.X., B.L., J.L. and N.C.: funding. All authors have read and critically revised the manuscript and agreed to the published version.

Financial support: This work was supported by Sichuan Provincial Commission of Health Science Project (20PJ059); Sichuan Science and Technology Program (Grant No. 2022YSF0060, Grant No. 2022YSF0114, Grant No. 2022NSFSC0680, Grant No. 2023YFS0094); 1 3 5 project for disciplines of excellence–Clinical Research Incubation Project, West China Hospital, Sichuan University (20HXFH021); 1 3·5 project for disciplines of excellence, West China Hospital, Sichuan University (ZYJC21049); The Key Research and Development Program sponsored by the Ministry of Science and Technology of Chengdu (Grant No. 2021-YF05- 00065-SN).

Potential competing interests: All authors have fulfilled the ICMJE uniform disclosure requirement, accessible at www.icmje.org/coi_disclosure.pdf. They affirm the absence of support from any organization for the submitted work, lack of financial associations with any organizations potentially interested in the submitted work within the past 3 years, and absence of any other affiliations or engagements that could potentially influence the submitted work.

Data sharing: We have made the source code and datasets used in this study publicly available on GitHub. The project, titled “Machine Learning of Obstructive Jaundice based on Common Laboratory Tests (the MOLT model),” can be accessed at https://github.com/re5yho/Machine-learning-of-Obstructive-jaundice-based-on-common-Laboratory-Tests-the-MOLT-model-.git. Researchers interested in replicating or extending our work are encouraged to explore the repository.

Transparency statement: The primary author (N.C., the manuscript's guarantor) confirms that the manuscript provides a truthful, precise, and transparent portrayal of the study being presented. No crucial elements of the study have been excluded, and any deviations from the original study plan (and, if applicable, registration) have been elucidated.

Patient and public involvement statement: Our statement of intent for patient and public involvement outlines our commitment to collaborating with individuals affected by obstructive jaundice to shape and guide our research projects aimed at advancing our understanding of this condition. We will also seek input from members of the public, especially in areas such as obstructive jaundice epidemiology, prevention, early detection and diagnosis.

Registration number: This is a retrospective cohort study, preregistered in Open Science Framework (registration DOI: https://doi.org/10.17605/OSF.IO/DC4B8).

Study Highlights.

WHAT IS KNOWN

• ✓ Currently, there is a deficit in large-scale cohort studies and practical diagnostic models for identifying the etiology of obstructive jaundice (OJ).

WHAT IS NEW HERE

• ✓ Our study emerged as the largest cohort study regarding OJ to date, delineating the spectrum of diseases associated with this condition. Interpretable machine learning models based on common clinical laboratory tests were developed, collectively termed the machine learning of OJ based on common laboratory tests model, which not only distinguishes between benign and malignant obstructions but also further differentiates between calculous benign lesions, noncalculous benign lesions, metastatic malignancies, pancreatobiliary malignancies, and other types of malignancies. These findings can support the identification of the underlying etiology of OJ in primary clinical settings, helping clinicians make well-informed decisions.

Supplementary Material

ct9-16-e00849-s001.docx ^{(2.3MB, docx)}

ABBREVIATIONS:

A/G

albumin/globulin ratio

AFP

α-fetoprotein

ALB

albumin

CA 19-9

carbohydrate antigen 19-9

CBDS

common bile duct stones

CEA

carcinoembryonic antigen

CHOL

cholesterol

DBIL

direct bilirubin

HPB

hepato-pancreato-biliary

LG cohort

large general cohort

ML

machine learning

MOLT

ML of OJ based on common laboratory tests

OJ

obstructive jaundice

P-LCR%

large platelet ratio

RDW-SD

red cell distribution width-standard deviation

TBIL

total bilirubin