Prediction model for ocular metastasis of breast cancer: machine learning model development and interpretation study

Abstract

Background

Breast cancer (BC) is caused by the uncontrolled proliferation of breast epithelial cells followed by malignant transformation, and it has the highest incidence among female malignant tumors. The metastasis of BC occurs through direct and lymphatic spread. Although ocular metastasis is relatively rare, it is a good indicator of a worse prognosis. We used machine learning (ML) to establish a model to analyze the risk factors of BC eye metastasis.

Methods

The clinical data of 2225 patients with BC from 2003 to 2019 were collected and randomly classified into the training and test sets using a ratio of 7:3. Based on the presence or absence of eye metastasis, the patients with BC were classified into the ocular metastasis (OM) and non-ocular metastasis (NOM) groups. Univariate and multivariate logistic regression analyses and least absolute shrinkage and selection operator (LASSO) were conducted. We used six ML algorithms to establish a predictive BC model and used 10-fold cross-validation for internal verification. The area under the receiver operating characteristic (ROC) curve was used to evaluate the predictive ability of the model. In addition, we established a web hazard calculator depending on the best-performing model to facilitate its clinical application. Shapley additive interpretation (SHAP) was used to determine the risk factors and the interpretability of the black box model.

Results

Univariate logistic regression analysis showed that histopathology (other types), axillary lymph node metastasis (ALNM) (> 4), Ca²⁺, total cholesterol (TC), low-density lipoprotein (LDL), apolipoprotein A (ApoA), carcinoembryonic antigen (CEA), carbohydrate antigen (CA) 125, CA153, CA199, alkaline phosphatase (ALP), and hemoglobin (Hb) were risk factors for BC eye metastasis. Multivariate logistic regression analysis showed that CA153, ApoA, and LDL were hazardous components for BC eye metastasis. LASSO showed that ALNM, LDL, CA125, Hb, ALP, and CA199 were the first six key variables that were useful for the diagnosis of ocular metastasis in breast cancer. Bootstrapped aggregation (BAG) demonstrated the discriminative ability (area under ROC curve [AUC] = 0.992, accuracy = 0.953, sensitivity = 0.987). Based on this, we applied the BAG machine learning model to build an online web computing system to help clinicians assist in determining the risk of BC eye metastasis. In addition, two typical cases are analyzed to determine the interpretability of the model.

Conclusion

We used ML to establish a risk prediction model for BC ocular metastasis, and BAG showed the greatest performance. The model can predict the risk of OM in patients with BC, facilitate early and timely diagnosis and treatment, and reduce the burden on society.

Introduction

BC is a major cause of cancer-related death in women, with an annual increase of approximately 0.3% [1, 2]. Recently, the survival ratio of cancer has been significantly improved through advanced therapies such as surgery and chemoradiotherapy [3]. However, it has been confirmed that up to half of patients with BC eventually develop fatal metastasis, even decades after the removal of the primary tumor. Previous studies have reported that metastasis in patients with BC is related to age, tumor pathological type, and endocrine treatment [4]. BC cells can escape from local tumors, move to distant organs, and become chemically resistant even before they are detected [5]. This makes the prognosis of patients with metastatic BC generally poor, with an average 5-year survival rate of approximately 25%; tumor metastasis accounts for approximately 90% of cancer-related deaths [6, 7]. OM is a rare clinical condition. BC accounts for 28.5–58.8% of all orbital metastases, which makes it the most dominant [8]. With the improvement of diagnostic methods and the prolongation of survival of BC patients, the incidence of OM has increased [9]. For most cases, OM occurs with the systemic progression of previously diagnosed BC, but 25% of confirmed OMs are detected in patients with newly diagnosed breast cancer [10]. The main ocular manifestations of OM in BC are eyelid tumidness, eye pain, and even blindness [11]. To reduce the eye pain of patients, early detection for patients with BC and associated eye disease is of great significance.

ML is a computer technology that is designed to study patterns in data to accomplish various tasks [12]. In applied healthcare research, ML is often applied to characterize automated, extremely flexible methods to identify patterns in complex data structures, and ML methods become better at pattern prediction as the amount of data available increases [13]. ML models have been used to detect early BC non-invasively and prevent it using hormone replacement therapy and chemotherapy [14]. Recently, ML has been more widely used for BC assessment and to develop an effective method for preoperative magnetic resonance imaging evaluation for axillary lymph node (ALN) status, which can effectively predict ALNM in patients with BC [15]. Moreover, mixed ML models can be used for more detailed classification of breast cancer subtypes, which helps doctors develop better treatment options and improve disease prognosis [16].

Although previous studies have demonstrated the advantages of ML for BC prevention, diagnosis, and classification, there is still no model to determine the risk of OM in patients with BC. In this study, we aimed to identify the potential biomarkers of BC eye metastasis in a large sample of patients with BC and use general demographic data and serological indicators, among others, to construct several predictive models to quantify the risk of recurrence. The performances of different machine learning models were compared, and the optimal machine learning model was selected. A web calculator was developed to determine the personal risks of BC eye metastasis and, in turn, improve the quality of life of patients with BC.

Methods

Participants

This retrospective cohort study included 2225 patients diagnosed with BC from June 2003 to May 2019. This study was approved by the Medical Research Ethics Committee. They were classified into a training set and an internal test set at a proportion of 7:3 at random. We classified breast cancer patients into the OM group if ocular metastasis was confirmed by ocular CT and MRI results, along with histological and cytological evidence. If no ocular metastasis was present, the patients were classified into the NOM group. We used the Synthetic Minority Oversampling Technique (SMOTE) to generalize the sample size of the OM group [17] (Fig. 1). All participants obtained messages about the research before consenting to join in the trial and subscribing the informed consent.

Fig. 1.

Summary of patients inclusion. Abbreviations Hb, Hemoglobin; TG, triglyceride; TC, total cholesterol; HDL, high-density lipoprotein; LDL, low-density lipoprotein; ApoA, apolipoprotein A; ApoB, apolipoprotein B; CA125, carbohydrate antigen-125; CA199, carbohydrate antigen-199; NOM, no ocular metastasis; OM, ocular metastasis

Data collection

Clinical indicators were assembled from patient document, including gender, age, histological grade, Hb, estrogen (ER), progestational hormone (PR), Ki 67, human epidermal growth factor receptor 2 (Her 2), ALNM, calcium, ALP, CEA, CA125, CA153, CA199, TC, triglyceride (TG), high density lipoprotein (HDL), LDL, ApoA, apolipoprotein B (ApoB), lipoprotein (a) (Lp (a)).

Statistical analysis

All statistical analyses were performed using Python (Version 3.8, Python Software Foundation) and R software (Version 4.0.2). Using Python, the data from the First Affiliated Hospital of Nanchang University, China, were randomly allocated to training and internal test sets at a ratio of 7:3. The training set was used to build the model, and the internal test set was used for model verification and evaluation. Two independent samples T-test was applied to compare numerical data with a normal distribution. The Mann–Whitney U test was used to compare the continuous data that were not normally distributed. The chi-squared test was used to compare categorical count data. Univariate and multivariate logistic regression were used to determine the risk factors for OM in patients with BC and the variables screened by LASSO regression were included in the construction of ML models. LASSO regression is a shrinkage tool that can proactively sort from a set of potentially multicollinear variables in the regression, generating a more related and explainable set of predictors [18]. Python programming was used to develop and test the ML models and web calculators. For model explanation, the Python SHAP toolbox was used [19]. P < 0.05 denoted statistical significance.

Model establishment

All algorithm models were based on scikit-learn (Version 0.24.2). In this study, we used six ML models: multilayer perceptron (MLP) [20]; AdaBoost (AB) model [21]; bootstrapped aggregation (BAG) [22] model; logistic regression (LR) [23]; gradient boosting machine (GBM) [24]; and random forest (LR) [25] model. MLP is a deep learning model based on feed-forward neural networks, consisting of multiple neuron layers, which are trained by a back-propagation algorithm. MLP can perform tasks such as classification, regression, and clustering, with strong expressive power that capable of handling non-linear problems and high-dimensional data [20]. ‌ AB is an iterative algorithm whose core idea is to assemble the same training set to train different weak classifiers that constitute a stronger final classifier. It can adjust the focus on training data instances after each iteration, especially those samples that were previously incorrectly predicted, and update the weights of the weak learners [21]. BAG, an integrated learning method, is based on the principle of generating multiple subsets of basic models that can be trained by performing putative random sampling from a training dataset, and then averaging the predictions of these basic models by are averaged. This approach reduces the variance of the model and improves the generalization ability of the model [22]. LR is a machine learning algorithm for solving binary classification problems by converting the continuous output of a linear regression to a probability value between 0 and 1 using a logistic function. It is simple to implement, and the output is easy to understand [23]. GBM is a popular integrated learning algorithm that combines multiple decision trees and gradient descent to improve the performance of the model and has shown excellent performance in solving complex prediction problems [24]. RF refers to a classifier that uses multiple trees to train and predict samples. It can suppress overfitting and handle the problem of missing values better [25]. The ML algorithm was trained and adjusted to predict the risk of OM in patients with BC. We conducted cross-validation within the train set to create a validation set for hyperparameter tuning. In each cross-validation loop, parameters were adjusted to find those that resulted in the best performance on the validation set. Additionally, we performed multicollinearity tests on all feature variables. The predictions of the ML model were verified by internal 10-fold cross-validation of the training and internal test sets, and the AUC, accuracy, sensitivity, and specificity scores were assessed. Finally, we used the best-performing model to establish a web calculator.

Results

Demographic baseline data

After screening, 2225 patients with BC were included. Among them, 29 had OM, and 2196 did not. There were significant differences in histopathology, Ki 67, ALNM, CEA, CA125, CA153, Hb, ALP, Ca²⁺, TC, LDL, and ApoA between the OM and NOM groups (P < 0.05). The demographic and clinicopathological characteristics of the above patients are detailed in Table 1. The hierarchical clustering results show that there is no significant multicollinearity between the feature variables in the training set (Fig. 2).

Table 1.

Demographic and clinicopathological characteristics of patients

Variables	Total (n = 2225)	NOM(n = 2196)	OM (n = 29)	P value
Gender, n (%)				1
Female	2219 (100)	2190 (100)	29 (100)
Male	6 (0)	6 (0)	0 (0)
Histopathology, n (%)				0.035*
Invasive ductal carcinoma	1151 (52)	1129 (51)	22 (76)
Other types	985 (44)	978 (45)	7 (24)
Unknown	89 (4)	89 (4)	0 (0)
ER, n (%)				0.495
Positive	1218 (55)	1201 (55)	17 (59)
Negative	726 (33)	719 (33)	7 (24)
Unknown	281 (13)	276 (13)	5 (17)
PR, n (%)				0.071
Positive	1088 (49)	1073 (49)	15 (52)
Negative	861 (39)	847 (39)	14 (48)
Unknown	276 (12)	276 (13)	0 (0)
Ki_67, n (%)				< 0.001*
Positive	990 (44)	973 (44)	17 (59)
Negative	342 (15)	330 (15)	12 (41)
Unknown	893 (40)	893 (41)	0 (0)
Her_2, n (%)				0.056
Positive	884 (40)	871 (40)	13 (45)
Negative	1036 (47)	1020 (46)	16 (55)
Unknown	305 (14)	305 (14)	0 (0)
Axillary_lymph_node_metastasis, n (%)				< 0.001*
Non	846 (38)	843 (38)	3 (10)
≤ 4	660 (30)	653 (30)	7 (24)
>4	464 (21)	449 (20)	15 (52)
Unknown	255 (11)	251 (11)	4 (14)
Age, Median (Q1,Q3)	47 (41, 54)	47 (41, 54)	48 (43, 52)	0.861
CEA, Median (Q1,Q3)	1.98 (1.14, 3.3)	1.97 (1.14, 3.28)	3.54 (1.96, 12.98)	< 0.001*
CA125, Median (Q1,Q3)	12.68 (9.06, 18.53)	12.58 (9.03, 18.16)	23 (19.32, 41.62)	< 0.001*
CA153, Median (Q1,Q3)	12.27 (8.24, 20.07)	12.18 (8.17, 19.42)	123 (93.97, 167)	< 0.001*
CA199, Median (Q1,Q3)	11.44 (7.6, 17.9)	11.46 (7.61, 18.08)	9.87 (6.57, 13.61)	0.323
Hb, Median (Q1,Q3)	122 (114, 130)	122 (115, 130)	100 (86, 112)	< 0.001*
ALP, Median (Q1,Q3)	63 (50, 81)	63 (50, 80)	84 (70, 189)	< 0.001*
Ca2+, Median (Q1,Q3)	2.35 (2.24, 2.48)	2.35 (2.24, 2.48)	2.21 (2.11, 2.42)	0.006*
TC, Median (Q1,Q3)	5.02 (4.24, 6.13)	5.03 (4.24, 6.14)	4.28 (3.8, 5.16)	< 0.001*
TG, Median (Q1,Q3)	1.66 (1.02, 2.79)	1.67 (1.01, 2.8)	1.62 (1.27, 1.89)	0.367
HDL, Median (Q1,Q3)	1.57 (1.22, 2.48)	1.57 (1.22, 2.47)	1.32 (1.11, 2.67)	0.267
LDL, Median (Q1,Q3)	3.1 (2.4, 4.09)	3.1 (2.41, 4.1)	2.14 (1.32, 2.43)	< 0.001*
ApoA, Median (Q1,Q3)	1.58 (1.34, 2.03)	1.59 (1.35, 2.09)	0.92 (0.78, 1.12)	< 0.001*
ApoB, Median (Q1,Q3)	1 (0.78, 1.9)	1 (0.78, 1.92)	1.31 (0.77, 1.63)	0.502
Lipoprotein_a, Median (Q1,Q3)	130 (64, 238)	128 (64, 240.25)	164 (133, 179)	0.15

(*P < 0.05)

Abbreviations OM, ocular metastasis; NOM, non ocular metastasis; ER, estrogen; PR, progestational hormone; Her 2, Human epidermal growth factor receptor 2; CEA, carcinoembryonic antigen; CA125, carbohydrate antigen-125; CA153, carbohydrate antigen-153; CA199, carbohydrate antigen-199; Hb, Hemoglobin; ALP, alkaline phosphatase; Ca²⁺, calcium; TC, total cholesterol; TG, triglyceride; HDL, high-density lipoprotein; LDL, low-density lipoprotein; ApoA, apolipoprotein A; ApoB, apolipoprotein B

Fig. 2.

Bottom-up hierarchical clustering of feature variables based on the training set

Notes: Using a threshold of 0.75, there are no significant multicollinearities between the feature variables in the training set

Univariate analysis, multivariate logistic regression and LASSO regression

A univariate logistic regression model was established. Variations with P-values of < 0.05 in the univariate analysis were selected for multivariate logistic regression analysis to choose the risk factors of OM in patients with BC. For the univariate logistic regression, histopathology (other types), ALNM (> 4), CEA, CA125, CA153, CA199, Hb, ALP, Ca²⁺, TC, LDL, and ApoA were risk factors for OM in BC. The multivariate logistic regression results showed that CA153, LDL, and ApoA were independent risk factors for BC eye metastasis (P < 0.05) (Table 2). Additionally, univariate (Fig. 3A) and multivariate (Fig. 3B) logistic regression were used for forest map visualization. LASSO regression was then used to screen out the ALNM, LDL, CA125, Hb, ALP CA199, and these were included in the six ML model characteristic variables (Fig. 4). It should be noted that histopathology (other types) includes non-invasive breast cancer as well as invasive lobular carcinoma. Since invasive ductal carcinoma accounts for the majority of the breast cancer, we deliberately included it as a histological type, with other types classified as other types. In univariate regression, the odds ratio of histopathology (other types) was 0.367 (0.156–0.864), which was significant, but there was no significance in multivariate regression. We discovered that non-invasive breast cancer and invasive lobular carcinoma had relatively low rates of distant metastasis and were therefore classified as protective factors in univariate regression [26]. However, histopathology (other types) was no longer a valuable predictor after confounding variables were removed in multivariate regression.

Table 2.

Single factor analysis and multifactor logistic regression

Characteristics	Category	Univariate analysis		Multivarite analysis
		OR (95% CI)	P value	OR (95% CI)	P value
Gender	Female	Ref	Ref	Ref	Ref
	Male	0 (0-Inf)	0.99
Histopathology	Invasive ductal carcinoma	Ref	Ref	Ref	Ref
	Other types	0.367 (0.156–0.864)	0.022*	0.32 (0.072–1.412)	0.132
	Unknown	0 (0-Inf)	0.983	0 (0-Inf)	0.989
ER	Positive	Ref	Ref	Ref	Ref
	Negative	0.688 (0.284–1.667)	0.407	\	\
	Unknown	1.28 (0.468–3.499)	0.631	\	\
PR	Positive	Ref	Ref	Ref	Ref
	Negative	1.182 (0.568–2.463)	0.655	\	\
	Unknown	0 (0-Inf)	0.988	\	\
Ki_67	Positive	Ref	Ref	Ref	Ref
	Negative	2.081 (0.984–4.404)	0.055	\	\
	Unknown	0 (0-Inf)	0.986	\	\
Her_2	Positive	Ref	Ref	Ref	Ref
	Negative	1.051 (0.503–2.197)	0.895	\	\
	Unknown	0 (0-Inf)	0.987	\	\
Axillary_lymph_node_metastasis	Non	Ref	Ref	Ref	Ref
	≤ 4	3.012 (0.776–11.694)	0.111	2.288 (0.286–18.304)	0.435
	>4	9.388 (2.703–32.598)	< 0.001*	3.808 (0.514–28.22)	0.191
	Unknown	4.478 (0.996–20.141)	0.051	2.037 (0.128–32.476)	0.615
CEA	\	1.024 (1.012–1.036)	< 0.001*	0.992 (0.933–1.055)	0.8
CA125	\	1.008 (1.004–1.012)	< 0.001*	1.003 (0.99–1.016)	0.659
CA153	\	1.021 (1.016–1.025)	< 0.001*	1.026 (1.017–1.036)	< 0.001*
CA199	\	1.013 (1-1.025)	0.049*	1.014 (0.987–1.042)	0.314
Hb	\	0.947 (0.931–0.964)	< 0.001*	0.99 (0.951–1.031)	0.626
ALP	\	1.011 (1.007–1.015)	< 0.001*	1.001 (0.99–1.011)	0.917
Ca2+	\	0.129 (0.043–0.388)	< 0.001*	0.362 (0.029–4.462)	0.427
TC	\	0.584 (0.423–0.805)	0.001*	1.411 (0.776–2.566)	0.259
LDL	\	0.32 (0.213–0.48)	< 0.001*	0.363 (0.152–0.866)	0.022*
ApoA	\	0 (0-0.002)	< 0.001*	0.002 (0-0.024)	< 0.001*
Age	\	0.987 (0.951–1.023)	0.473	\	\
TG	\	0.719 (0.512–1.008)	0.056	\	\
HDL	\	1.082 (0.906–1.293)	0.383	\	\
ApoB	\	0.669 (0.423–1.056)	0.084	\	\
Lipoprotein_a	\	0.999 (0.997–1.001)	0.533	\	\

(*P < 0.05)

Abbreviations ER, estrogen; PR, progestational hormone; Her 2, Human epidermal growth factor receptor 2; CEA, carcinoembryonic antigen; CA125, carbohydrate antigen-125; CA153, carbohydrate antigen-153; CA199, carbohydrate antigen-199; Hb, Hemoglobin; ALP, alkaline phosphatase; Ca²⁺, calcium; TC, total cholesterol; TG, triglyceride; HDL, high-density lipoprotein; LDL, low-density lipoprotein; ApoA, apolipoprotein A; ApoB, apolipoprotein B

Fig. 3.

Univariate and multivariate Logistic regression analysis. Notes(A) The univariate Logistic regression analysis showed that ALNM (> 4), CEA, CA125, CA153, CA199, Hb, ALP, Ca²⁺, TC, LDL, ApoA were risk factors for ocular metastasis. (B) The multivariate Logistic regression analysis showed that CA153, ApoA and LDL were risk factors for ocular metastasis. Abbreviations CEA, carcinoembryonic antigen; CA125, carbohydrate antigen-125; CA153, carbohydrate antigen-153; CA199, carbohydrate antigen-199; Hb, Hemoglobin; ALP, alkaline phosphatase; Ca²⁺, calcium; TC, total cholesterol; LDL, low-density lipoprotein; ApoA, apolipoprotein A; ApoB, apolipoprotein B

Fig. 4.

(A) Plot for LASSO regression coefficients. (B) Cross validation plot. Abbreviations LASSO, least absolute shrinkage and selection operator

Model performance

We evaluated the risk predictions of OM in the patients with BC and the related accuracy for six different ML models: MLP, AdaBoost, BAG, logistic regression, gradient boosting machine, and random forest. The results of the 10-fold cross-validation showed that the AUC of BAG was 0.994, and the standard error was 0.004, which was better than those of the other machine learning models (Fig. 5). The AUC for BAG was 0.992, which was better than those for the other machine learning models (Fig. 6). The BAG model performed best based on the training results, with an accuracy of 0.953, a sensitivity of 0.987, and a specificity of 0.919 (Table 3). We constructed a 5-fold cross-validation to evaluate the stability of the results based on the optimal BAG model (Fig. 7), where the AUC was 0.980 and the standard error was 0.019. For the above six ML models, we drew the maximum values of the five indicators for the radar chart evaluation. Among them, BAG had the most optimal values for the evaluation of each of the following indicators: sensitivity, F1 score, AUC, accuracy, and specificity (Fig. 8). Based on this, the confusion matrix (Fig. 9) for the model results was drawn. For the BAG algorithm, the number of accurate predictions of OM was 2176, and that of non-eye transfer samples was 2071.

Fig. 5.

AUC values of 10-fold cross-validation. Notes The AUC was used as an indicator of performance, the BAG model achieved the best predictive performance, and the AB model had the lowest predictive performance. Abbreviations AB, adaptive boosting; LR, logistic regression; RF, random forest; BAG, bootstrapped aggregating; MLP, multilayer perceptron; GBM, gradient boosting machine. AUC, area under the ROC curve; STD, Standard Deviation

Fig. 6.

Validation of machine learning algorithms. Notes The AUC was used as an indicator of performance, the BAG model achieved the best predictive performance, and the AB model had the lowest predictive performance. Abbreviations AB, adaptive boosting; LR, logistic regression; RF, random forest; BAG, bootstrapped aggregating; MLP, multilayer perceptron; GBM, gradient boosting machine. ROC, receiver operating characteristic; AUC, area under the ROC curve

Table 3.

Performance comparison of six ML models

Model	F1	AUC	Accuracy	Sensitivity	Specificity
AB	0.809	0.809	0.809	0.8	0.818
LR	0.861	0.929	0.862	0.842	0.881
RF	0.927	0.975	0.927	0.941	0.913
BAG	0.953	0.992	0.953	0.987	0.919
MLP	0.945	0.978	0.945	0.972	0.919
GBM	0.95	0.984	0.95	0.967	0.933

Abbreviations ML, machine learning; AUC, area under the curve; AB, adaptive boosting; LR, logistic regression; RF, random forest; BAG, bootstrapped aggregating; MLP, multilayer perceptron; GBM, gradient boosting machine

Fig. 7.

Receiver operating characteristic curve in the BAG model. Abbreviations ROC, receiver operating characteristic; AUC, area under the ROC curve

Fig. 8.

Radar plot of six machine learning methods. Notes Among the six machine learning models, the BAG showed the best performance in F1 score, AUC, accuracy, sensitivity and specificity

Fig. 9.

Confusion matrix. Notes The correct classification (accuracy) of recurrence for the BAG model was 0.991. Abbreviations AB, adaptive boosting; LR, logistic regression; RF, random forest; BAG, bootstrapped aggregating; MLP, multilayer perceptron; GBM, gradient boosting machine; NOM, non ocular metastasis; OM, ocular metastasis

Importance of characteristic variables

Based on BAG, we used the SHAP library to establish a risk factor model for OM in patients with BC (Fig. 10A, B). For the BAG model, the important variables were ALNM, LDL, CA125, Hb, ALP, and CA199. Using the SHAP library, we selected two participants, including OM and NOM group members. According to our model, the base value of the low-risk group was − 2.14, and the predicted value was − 2.98 (Fig. 10C), where ALNM, CA199, and ALP were variables that increased the predicted value, while LDL, CA125, and Hb decreased the predicted value. The basic value of OM in the high-risk group was − 2.14, and the predicted value was 1.07 (Fig. 10D). In this sample, CA125 reduced the predicted value, while ALNM, LDL, Hb, ALP, and CA199 increased it.

Fig. 10.

SHAP summary plot and SHAP model explanation of two typical predictions. Notes The features are ranked according to the sum of the SHAP values for all patients, and the SHAP values are used to show the distribution of the effect of each feature on the BAG model outputs. Each dot represents a case in the dataset. The color of a dot indicates the value of the feature, with blue indicating the lowest range and red the highest range. The horizontal axis shows the corresponding SHAP value of the feature. A positive SHAP value contributes to the prediction of rupture and vice versa. Figure 7D showed the low-risk SHAP interpretation model for patients with ocular metastasis of BC. Figure 7D showed the interpretation model of high-risk SHAP in patients with ocular metastasis of BC. Abbreviations SHAP, Shapley additive explanations; Hb, Hemoglobin; ALP, alkaline phosphatase; CA125, carbohydrate antigen-125; CA199, carbohydrate antigen-199; LDL, low-density lipoprotein

Web page calculator

Based on the BAG model, an ML algorithm with optimum predictive performance, we developed the above web predictor to predict the risk of OM in patients with BC. A risk prediction for OM of BC was made using variable settings in the sidebar of an online tool (https://ml-breast-cancer-eye-meta-jbegji.streamlit.app/) (Fig. 11).

Fig. 11.

Web calculator for predicting ocular metastasis of breast cancer. Notes URL was https://ml-breast-cancer-eye-meta-jbegji.streamlit.app/

Discussion

In this study, various ML algorithms were used for the first time to predict the OM of BC patients, and a BAG model for clinical prediction was obtained and explained. The BAG model is a commonly used ML algorithm. It uses ensemble learning, which can combine multiple weak classifiers into a strong classifier. It has good accuracy and classification speed [27].

BC is the most common neoplastic disorder in women, and it is listed as the main reason for death in women and second only to lung cancer [28]. Despite advances in diagnosis and treatment, up to one-third of patients with BC will develop metastatic disease [29]. BC eye metastasis is the most common ocular malignant disease [30]. The uvea is one of the most common metastatic sites of BC [31]. The survival ratio of patients with BC and OM depends on the severity of organ dysfunction caused by tumor spread [32]. Because the majority of OM cases result from hematogenous dissemination, the uvea, especially the choroid, is the main ocular target of BC metastasis [33]. Up to 10% of patients with metastatic breast cancer have uveal involvement, which is the commonest metastatic site of BC among all body tissues [34]. The vascular and microenvironment factors of the choroid are considered possible causes of increased choroidal diffusion [35]. The most common clinical symptoms of patients with OM diseases are blurred vision, eye pain, visual field defects, visual deformation, and floaters [36]. Moreover, disease metastases in the lung or bone are usually detected before the diagnosis of OM [37]. An important hazardous component for the progression of malignant ocular lesions in cancer patients is the spread of disease in the lungs and brain [38]. Therefore, it is important to make timely predictions of OM for BC and formulate relevant targeted preventive measures.

ALN status is an important prognostic factor and has effects on surgery and treatment of patients with BC [39]. It is one of the most critical factors affecting local recurrence and overall survival of BC [40]. The 5-year survival ratio of patients with BC without ALNM is as high as 99%, but the survival ratio of patients with ALNM is 85.8% [41]. Therefore, correct assessment of the malignant or benign tendency of ALN is significant in determining the recurrence of patients with BC. The maximal diameter of lymph nodes in patients with BC without ALNM was smaller than that in those with ALNM, indicating that patients with larger lymph nodes have a higher risk of metastasis [42]. Gong et al. [43] pointed out that smaller lymph nodes are associated with lower risks of ALNM, which may be due to pulmonary invasion and increased lymph nodes in patients with larger tumors. This shows that the risk of further metastasis to the eye in patients with BC increases with ALNM and enlargement, which is consistent with our conclusion that ALNM > 4 is a risk factor for BC eye metastasis. A case of a patient diagnosed with lung adenocarcinoma with ALNM who had cancer eye metastasis has been reported [44]. LDL is a key lipoprotein and cholesterol carrier that mediates cholesterol transfer from the liver to peripheral tissues [45]. At present, hyperlipidemia has been portrayed to increase the risk of cancer. Cancer cells often upregulate cholesterol biosynthesis or accumulate a large amount of cholesterol by increasing cholesterol uptake [46]. Increased intracellular cholesterol concentrations have been observed in cancer tissues [47]. Regarding tumor metastasis, LDL can improve tumor metastasis by limiting the anti-tumor therapeutic effect of human γδ-T cells in vivo [48]. LDL plays a role in BC cells through various mechanisms. Clinical and pathological studies have shown that the accumulation of cholesterol esters increases with an increase in LDL, which can increase the severity of BC [49]. LDL can promote the progression and metastasis of breast cancer through angiogenesis; increase the concentrations of mesenchymal markers Slug, vimentin, and β-catenin; and reduce the expression of adhesion molecules, which promotes the invasion of BC cells [50]. In vivo studies have shown that increased LDL receptor expression in tumors accelerates LDL-mediated breast cancer growth in hyperlipidemic mice. In contrast, LDL receptor silencing and low circulating LDL concentrations delay tumor growth in HER 2-positive and triple-negative BC mouse models [51]. Our results showed that LDL is an important risk factor for BC eye metastasis, which is consistent with the above research.

CA125 is one of the specific tumor markers, which is common in adult pleura, endometrium, and cervical endometrium [52]. BC has a higher positive rate in patients with endometrial and ovarian cancers. The expression of CA125 is associated with the tumor load of patients and increases with disease progression. Wider metastases and more lesions are associated with higher expressions. Therefore, CA125 is a sensitive indicator of tumor metastasis [53]. One study have shown that CA125 has a specificity of 87.4% in judging BC distant metastasis within 3 years [54]. CA125 has been confirmed to be involved in the incidence of BC bone metastasis and has good sensitivity and specificity for the diagnosis of BC bone metastasis, which may become a new biomarker for the diagnosis of breast cancer bone metastasis [55]. Our results also suggested that CA125 may be an essential risk indicator for BC eye metastasis. As a marker of bone development, ALP is broadly utilized to evaluate brain metastasis in breast cancer. Ritzke et al. portrayed that ALP combined with CA-153 had the best sensitivity for the prediction of brain metastasis [56]. He et al. showed that serum ALP concentrations higher than 100.5 u/L were risk factors for brain metastasis [57]. Moreover, a study of Indian women showed that elevated serum ALP can help predict the prognosis of BC, which may provide a useful diagnostic tool for monitoring disease progression. The progressive increase of ALP activity in BC patients is suggestive of metastasis [58]. In a study of risk indicators for bone metastasis of BC, ALP was one of the independent prognostic factors [59]. Our study found that the average ALP concentration of the OM group did not reach 100.5 u/L (84 u/L); it was still markedly higher than that of the NOM (63 u/L, P < 0.01), indicating that ALP may be important for BC eye metastasis.

ML methods have been broadly applied to develop various predictive models that can make effective predictions. Similarly, the use of ML can greatly improve the decision-making process and assist doctors in the early and accurate detection of cancer [60]. Researchers have leveraged the diagnostic accuracy of the polynomial ML model to predict the occurrence of BC more accurately [61]. There have also been examples of using deep learning radiology to detect ALN status early in patients with BC [62]. In ophthalmic diseases, the ML method has been applied to the diagnosis of dry eye disease because of its automatic classification of images [63]. Compared with traditional statistical models, ML models are powerful. However, due to its inherent black-box characteristics, ML is usually more complex and difficult to understand, which limits its further clinical application. Therefore, we introduce SHAP to explain the BAG model. SHAP is a method for interpreting the results of machine ML predictions that helps to understand how the model made a particular prediction by calculating the contribution of each feature to the model’s prediction [19]. The SHAP method provides both global and local interpretability, making model predictions much more transparent. SHAP’s global interpretability provides general knowledge about individual features in the model by calculating the overall importance of each feature across the entire dataset. For each specific prediction, SHAP can also provide local interpretability, it offers an explanation of how the model made its prediction based on a specific data point. In addition, SHAP values can tell us how much the feature contributes to the prediction, with SHAP values closer to zero indicating that the feature contributes less to the prediction, and SHAP values further away from zero indicating that the feature contributes more to the prediction. SHAP can help clinicians and patients to understand the predictive logic of complex machine learning models, and greatly improve the transparency and interpretability of decisions. The advantages of consistency, local accuracy, and feature importance ranking of this method make it an indispensable tool in healthcare, especially in clinical applications that require a high degree of interpretability [64]. In the present study, the SHAP values visualise several factors that contribute most to OM in BC patients.

Imaging examinations, such as contrast agent imaging, can be used to monitor BC distant metastasis. However, due to the radiation, cost, and time of these techniques, none of them are ideal for predicting distant metastasis of BC. In contrast, biomarkers are convenient, fast, non-invasive, accurate, and reproducible and can be used for the detection and monitoring of malignant tumors [65]. We compared six different ML models to forecast the risk of OM in patients with BC using the F1 score, sensitivity, specificity, AUC, accuracy, and other indicators. The BAG machine learning model showed the best performance, and ALNM, LDL, CA125, Hb, ALP, and CA199 were found to be important risk factors for predicting OM. Also, we noted that MLP was relatively close to BAG in numerical terms. In big data processing, even small enhancements in the process of large samples can likewise reduce the amount of missed diagnoses in subjects with sufficient sample size. And the disadvantage of MLP is that it is too simple, once the data set is not linearly separated, its application will be greatly limited, and it is easy to overfit. By using multiple models to forecast, ‌and then combining these prediction results, ‌BAG can reduce the overdependence of a single model on the training data. It can thus avoid the problem of overfitting and the generalization ability of the model can be improved [66]. Based on this, we created a web calculator based on the optimal BAG model. Clinicians only need to input the patient indicators on the web page to obtain the prediction probability of OM, which can assist them to carry out targeted preventive measures.

Our research had limitations. In the first place, this study involved a single center and the OM group has a relatively small sample size, the manifestation of ML may differ depending on the characteristics of patients in different regions. Also, the study was retrospective, and the findings need to be validated through prospective studies. Moreover, the database used only recorded the original diagnosis of the patient, and we could not obtain follow-up information for further analysis. In subsequent studies we will further incorporate patient data from more study centers as external validation to make our findings more convincing, as well as generalizable. We will also follow up the patients included in the study in a long and orderly manner, so that we can get more recent conclusions.

Summary

Based on a number of common and readily available clinicopathological factors, we used the ML method to establish a risk predictive model for OM in BC, and proved the BAG model performed best. The establishment of the prediction model is a procedure towards an automated diagnosis system, which can help physicians make individualised diagnostic and preventive measures for patients with BC patients with OM using clinically accessible indicators.

Author contributions

Yi Shao and Yan Lou contributed to the study inception and design. Ru-Yi Rong, Yan-Kun Shen and Shi-Nan Wu equally contributed to the literature search, analysis and writing of the manuscript. Other authors contributed to the study design and study supervision. All authors approved the final version of the manuscript.

Funding

There was no funding.

Data availability

The datasets used and/or analyzed during the present study are available from the corresponding author on reasonable request.

Declarations

Ethical approval and consent to participate

The study methods and protocols were approved by the Medical Ethics Committee of the First Affiliated Hospital of Nanchang University (Nanchang, China) and followed the principles of the Declaration of Helsinki. All subjects were notified of the objectives and content of the study and latent risks, and then provided written informed consent to participate.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.