Abstract
This study aimed to construct a robust machine learning (ML) model for predicting the disease-free survival (DFS) and risk stratification of breast cancer (BC) patients with non-pathological complete response (non-PCR) after neoadjuvant chemotherapy (NAC). The model will facilitate the initiation of early interventions for high-risk patients. This retrospective multicenter cohort study included BC patients from two hospitals in China who received NAC but did not achieve PCR. Four ML algorithms were utilized to construct models based on patients’ clinicopathological data, followed by a performance evaluation of these models. To improve the interpretability of the model, the shapley additive explanation (SHAP) method was employed to analyze the contribution of each feature to the predictive outcomes. A total of 463 non-PCR patients were included in the study. Of these, 385 patients were from Ruijin Hospital, affiliated with Shanghai Jiao Tong University, and were randomly split into a training cohort and an internal validation cohort in a 3:1 ratio for model development and preliminary performance evaluation. In addition, 78 patients enrolled from Jiaxing Women and Children’s Hospital were assigned to the external validation cohort to evaluate the model’s generalizability. Univariate and multivariate Cox regression analyses demonstrated that age, residual tumor size, Ki67 change, molecular subtype, and axillary lymph node metastasis were independent factors influencing DFS. Among the four ML models, the random survival forest (RSF) model showed the best performance, with a concordance index of 0.820 in the training cohort, 0.642 in the internal validation cohort, and 0.689 in the external validation cohort. Further analysis revealed that the RSF model had excellent discriminative ability with a high area under curve value, while its low Brier score indicated excellent calibration. Decision curve analysis indicated that the RSF model offered a higher clinical net benefit at various time points and effectively stratified risk, successfully identifying high-risk patients. SHAP analysis underscored residual tumor size as the most influential predictive feature. The RSF model can effectively predict DFS and risk of BC patients with non-PCR following NAC, offering a critical reference for developing individualized treatment strategies.
Keywords: Breast cancer, neoadjuvant chemotherapy, non-pathologic complete response, disease free survival, machine learning, shapley additive explanations
Introduction
Breast cancer (BC), the most common malignant tumor among females whose incidence has shown a yearly upward trend, poses a severe threat to women’s health [1]. In China, advances in early screening and treatment techniques have greatly improved overall survival (OS) rates among BC patients. However, some patients experience delays in diagnosis from factors such as economic constraints, limited health awareness, and unequal access to medical resources, all of which may exacerbate disease progression. Such patients often require comprehensive treatment methods such as neoadjuvant chemotherapy (NAC). This therapeutic approach is widely employed in patients with large primary tumors or axillary lymph node metastasis (ALNM), as it could facilitate “downstaging”, thus allowing for more effective surgical interventions [2,3]. Research has indicated that patients who achieve pathological complete response (PCR) post-NAC will have significantly better prognostic outcomes [4]. Nonetheless, the PCR rate post-NAC often ranges from only 20-40%, indicating that many patients fail to achieve PCR. Compared to PCR patients, their non-PCR (non-PCR) counterparts face a higher risk of local recurrence and distant metastasis, especially those with triple-negative breast cancer (TNBC) [5,6]. Non-PCR patients also suffer quite often from great psychological pressure due to the fear of cancer recurrence and metastasis, potentially accelerating disease progression and inducing a vicious circle. Therefore, accurately predicting the risk of recurrence and metastasis in non-PCR patients is crucial for reducing patient anxiety, enhancing their quality of life, and informing subsequent treatment decisions. The CREATE-X trial demonstrated that the adjuvant use of capecitabine in human epidermal growth factor receptor 2 (HER2)-negative BC patients who were non-PCR post-NAC could significantly increase OS and disease-free survival (DFS), with TNBC patients benefiting the most [7]. Similarly, the KATHERINE study revealed that, for HER2+ BC patients who were non-PCR post-NAC, administering T-DM1, as opposed to traditional trastuzumab, significantly increased the 3-year DFS rate [8]. Although these strategies offer additional treatment options for non-PCR patients, they have also been associated with high risks of adverse effects, which could lead to some patients discontinuing the treatment or adjusting the dosage. Additionally, the clinical heterogeneity of non-PCR patients could result in varied treatment responses and increased recurrence risks. Therefore, identifying high-risk non-PCR patients and conducting risk stratification is crucial for precise risk prediction, facilitating the development of individualized treatment plans.
The recent extensive application of artificial intelligence (AI) in clinical cancer research has greatly enhanced the accuracy of cancer predictions. Particularly, machine learning (ML) techniques, which extract key information from huge amounts of medical data, have improved the ability to make precise clinical decisions, greatly improving disease prognosis [9]. As a result, ML-based cancer prediction research has recently gained significant traction, especially in BC patients. Furthermore, changes in the tumor’s biological characteristics post-NAC may influence the prognosis of non-PCR patients [10-12]. These biological changes provide crucial insights for predicting patient survival rates (SRs) and recurrence and metastasis risks, guiding subsequent individualized treatments.
Herein, we utilized clinicopathological data from BC patients with non-PCR post-NAC to construct and validate four ML prediction models. The performance of each model was compared, and the best model was interpreted using shapley additive explanations (SHAP) to identify high-risk patients. The model not only helps clinicians to predict the risk of cancer recurrence and metastasis, but also facilitates early intervention for high-risk patients.
Materials and methods
Data source and patient selection
This retrospective multicenter cohort study enrolled BC patients with non-PCR following NAC and radical surgery from two hospitals in China. Between January 2015 and December 2021, data on 385 eligible non-PCR patients were extracted from the BC Database of Ruijin Hospital, which is affiliated with Shanghai Jiao Tong University. Additionally, data on 78 non-PCR patients who were treated at Jiaxing Women and Children’s Hospital between January 2014 and December 2021, were included. The 385 patients from Ruijin Hospital were randomly divided into two cohorts using the “rsample” package in a 3:1 ratio: Training (N=283) and internal validation (N=102). This categorization ensured scientific validity and reliability for model development and validation. On the other hand, the 78 patients from Jiaxing Women and Children’s Hospital served as an external validation cohort for independently assessing the model’s external applicability, thus enhancing the robustness and generalizability of the study’s results. Figure 1 shows the study flowchart.
Figure 1.
Study flowchart. BC, breast cancer; NAC, neoadjuvant chemotherapy; PCR, pathological complete response; RSF, random survival forest; CoxPH, cox proportional hazards; GBM, gradient boosting machine; XGBoost, extreme gradient boosting; SHAP, shapley additive explanations.
Inclusion and exclusion criteria
The inclusion criteria were: (1) Female patients with clinical stage II-III invasive ductal carcinoma confirmed via needle biopsy; (2) Patients who had not received any anti-tumor therapy before the diagnosis and had completed ≥ 4 cycles of NAC; (3) Patients who underwent radical surgery post-NAC; (4) Patients with non-PCR postoperative pathological results; and (5) Patients with complete clinicopathological and follow-up data. On the other hand, the exclusion criteria were: (1) Patients in other BC stages; (2) Patients who did not complete NAC; (3) Patients who had distant metastases before treatment; and (4) Patients complicated with other malignant tumors.
Treatment regimens
The chemotherapy regimens were primarily based on taxanes and/or anthracyclines, with platinum added in some cases. Some HER2+ patients received trastuzumab and/or pertuzumab treatments. After chemotherapy, patients underwent radical surgery and Axillary Lymph Node Dissection (ALND). Non-pCR was defined as residual invasive carcinoma (RIC) in the primary tumor and/or axillary lymph nodes, based on the postoperative pathology results.
Study variables
The clinicopathological characteristics extracted from the medical record databases of the two hospitals included patient age, residual tumor size, Ki67 change, menstrual status, chemotherapy regimen, targeted therapy, molecular subtype, ALNM, Lymphovascular Invasion (LVI), and histological grade. Residual tumor size was defined as the remaining tumor’s maximum diameter. The Ki67 change was the difference in Ki67 levels before and after NAC. Histological grade was determined using the Nottingham Combined Histologic Grading Scale (NHS Grade). The expressions of estrogen receptor (ER), progesterone receptor (PR), and HER2 were assessed through immunohistochemistry (IHC) staining, and ER/PR positivity was defined as a tumor cell nuclei-positive staining of ≥ 1% [13]. Patients with ER/PR expression of ≥ 1% were classified as hormone receptor (HR)-positive. On the other hand, HER2 status was determined using the HercepTest scoring system. Specifically, HER2 (+++) was considered positive, HER2 (+)/0 was considered negative, and HER2 (++) required further testing using fluorescence in situ hybridization (FISH) [14]. Based on the IHC results, patients were further categorized into three molecular subtypes: (1) Luminal (HR-positive and HER2-negative); (2) HER2-positive (HER2-positive regardless of HR status); and (3) TNBC (Negative for both HR and HER2).
Follow-up
This study’s endpoint was DFS, defined as the time from the surgery date to the first diagnosis of local recurrence or distant metastasis, or the date of death or last follow-up for patients without recurrence or metastasis. During the follow-up, the data were date of last follow-up, location and timing of local recurrence and/or distant metastasis, and date of death. Local recurrence was defined as cancer recurrence at the ipsilateral breast, chest wall, or regional lymph nodes, whereas distant metastasis involved the spread of cancer cells to distant organs or lymph nodes.
Establishment and evaluation of the ML prediction models
In this study, we compared four ML prediction models: Random Survival Forest (RSF), Cox Proportional Hazards (CoxPH), Gradient Boosting Machine (GBM), and eXtreme Gradient Boosting (XGBoost). Notably, RSF is a survival analysis algorithm derived from the traditional random forest (RF) model, designed to handle high-dimensional data and capture complex nonlinear relationships between predictor variables and survival time. Research has shown that RSF excels in survival analysis, making it particularly well-suited for predicting survival periods and treatment outcomes in clinical settings [15]. Conversely, CoxPH, a classical survival analysis method, estimates the influence of multiple factors on survival time. The CoxPH model assumes constant proportional hazards, meaning the effect of a factor on event risk remains steady over time [16]. While it performs well for interpreting results and handling low-dimensional data, it is less effective with high dimensional or nonlinear data compared to RSF. On the other hand, the GBM model iteratively trains multiple decision trees, improving its predictive performance, and thereby allowing it to capture the complex relationships between predictive features and survival time [17]. The XGBoost model, an improved version of GBM with faster training speed and higher accuracy, utilizes second-order Taylor expansion and regularization terms to prevent overfitting and further enhances its generalizability through feature subsampling techniques [18]. Each model has unique advantages in different contexts. RSF is well-suited for high-dimensional and nonlinear data, CoxPH is easily interpretable, making it ideal for lower-dimensional datasets. Meanwhile, GBM and XGBoost stand out for their strong predictive accuracy and computational efficiency. The hyperparameters of each model were refined using a grid search in the “mlr3tuning” package combined with 5-fold cross-validation to comprehensively evaluate model performance. The hyperparameter search space was created using the “paradox” package. Table S1 shows the specific hyperparameter search space and tuning results. Each model’s predictive power in the training and internal validation cohorts was evaluated using the concordance index (C-index). Furthermore, a time-dependent receiver operating characteristic (ROC) curve, area under the curve (AUC), brier score, and decision curve analysis (DCA) were employed for a more comprehensive evaluation. The selected model’s applicability was validated using an external cohort. Patients were stratified into low- and high-risk groups based on the model’s median risk score. Kaplan-meier (KM) survival curves, generated using the “survminer” package, were then employed to assess DFS probabilities across these risk groups, thereby confirming the model’s clinical effectiveness and reliability.
To improve model interpretability, we employed the SHAP method based on game theory to explain the final model [19,20]. This approach calculates SHAP values to precisely quantify the contribution of each characteristic to the prediction results and rank their importance. Beyond highlighting the overall significance of the characteristics, it also assesses the sensitivity of each characteristic within individual samples, providing a detailed analysis of how each characteristic influences the model’s output. Finally, the prediction results are decomposed into the contributions of each characteristic, achieving the accurate evaluation of characteristic significance, and comprehensively demonstrating the role and influence of different characteristics in the model.
Statistical analysis
All statistical analyses were performed using R software (version 4.4.0) (https://www.r-project.org/). Categorical data were expressed as frequencies (%), and the chi-square test was used to compare each characteristic between subgroups. For the training cohort, univariate and multivariate Cox regression analyses were performed to identify independent factors influencing DFS, and those with P < 0.05 in the multivariate Cox regression analysis were used to construct a prediction model. The final ML model was developed using the “mlr3” package [21]. All tests were two-sided, and results with P < 0.05 were considered statistically significant.
Results
Clinicopathological characteristics
A total of 463 BC patients with non-PCR post-NAC (age range =21-89 years) were enrolled in this study. Notably, > 30% of histological grade data in the external validation cohort were missing; hence, this variable was excluded from the final analysis.
Baseline characteristic analysis (Table 1) indicated a relatively balanced distribution of clinicopathological characteristics between the training and internal validation cohorts (P ≥ 0.05). However, a significant age difference was observed between the training and external validation cohorts (P=0.005). Compared to the training cohort (84.1%), the external validation cohort had a higher percentage of patients aged > 35 years (96.2%). These results indicate that the external validation cohort tended to have an older patient population, likely reflecting the demographic profile of the region they were drawn from. Moreover, the chemotherapy regimens were significantly different between the two cohorts (P < 0.001). The proportions of patients who received anthracyclines + taxanes in the external validation and training cohorts were 93.6% and 61.5%, respectively, indicating differences in treatment preference in Jiaxing Women and Children’s Hospital. Furthermore, a significant difference in LVI was observed between the two cohorts (P=0.011), with a higher incidence in the external validation cohort (38.5%) compared to the training cohort (24.0%). Although certain characteristics showed statistically significant differences, the cohorts remained comparable in terms of key characteristics, ensuring a solid foundation for model construction and validation. The differences observed between the training and external validation cohorts might be caused by variations in clinical practices and patient demographics across medical institutions or regions.
Table 1.
Baseline characteristics of the included BC cohorts
| Characteristics | Total population | Training cohort (1) | Internal validation cohort (2) | External validation cohort (3) | p. (1) vs (2) | p. (1) vs (3) |
|---|---|---|---|---|---|---|
| N=463 | N=283 | N=102 | N=78 | |||
| Age, year | 0.775 | 0.005 | ||||
| > 35 | 400 (86.4%) | 238 (84.1%) | 87 (85.3%) | 75 (96.2%) | ||
| ≤ 35 | 63 (13.6%) | 45 (15.9%) | 15 (14.7%) | 3 (3.8%) | ||
| Residual tumor size, cm | 0.581 | 0.176 | ||||
| ≤ 1 | 110 (23.8%) | 63 (22.3%) | 22 (21.6%) | 25 (32.1%) | ||
| 1 >-≤ 3 | 240 (51.8%) | 154 (54.4%) | 51 (50.0%) | 35 (44.9%) | ||
| > 3 | 113 (24.4%) | 66 (23.3%) | 29 (28.4%) | 18 (23.1%) | ||
| Ki67 change | 0.795 | 0.338 | ||||
| -10-10% | 118 (25.5%) | 69 (24.4%) | 28 (27.5%) | 21 (26.9%) | ||
| ≥ 10% | 257 (55.5%) | 157 (55.5%) | 53 (52.0%) | 47 (60.3%) | ||
| ≤ -10% | 88 (19.0%) | 57 (20.1%) | 21 (20.6%) | 10 (12.8%) | ||
| Menstrual status | 0.768 | 0.427 | ||||
| Premenopause | 235 (50.8%) | 145 (51.2%) | 54 (52.9%) | 36 (46.2%) | ||
| Postmenopause | 228 (49.2%) | 138 (48.8%) | 48 (47.1%) | 42 (53.8%) | ||
| Chemotherapy regimen | 0.577 | < 0.001 | ||||
| Others | 87 (18.8%) | 62 (21.9%) | 21 (20.6%) | 4 (5.1%) | ||
| Anthracycline combined with taxane | 315 (68.0%) | 174 (61.5%) | 68 (66.7%) | 73 (93.6%) | ||
| Platinum | 61 (13.2%) | 47 (16.6%) | 13 (12.7%) | 1 (1.3%) | ||
| Targeted therapy | 0.305 | 0.208 | ||||
| No | 355 (76.7%) | 217 (76.7%) | 73 (71.6%) | 65 (83.3%) | ||
| Yes | 108 (23.3%) | 66 (23.3%) | 29 (28.4%) | 13 (16.7%) | ||
| Molecular subtype | 0.738 | 0.966 | ||||
| Luminal | 217 (46.9%) | 131 (46.3%) | 50 (49.0%) | 36 (46.2%) | ||
| HER2 positive | 165 (35.6%) | 101 (35.7%) | 37 (36.3%) | 27 (34.6%) | ||
| Triple negative | 81 (17.5%) | 51 (18.0%) | 15 (14.7%) | 15 (19.2%) | ||
| Axillary lymph node metastasis | 0.198 | 0.053 | ||||
| 0 | 105 (22.7%) | 72 (25.4%) | 19 (18.6%) | 13 (16.7%) | ||
| 1-3 | 158 (34.1%) | 80 (28.4%) | 39 (38.2%) | 33 (42.3%) | ||
| 4-9 | 130 (28.1%) | 89 (31.4%) | 27 (26.5%) | 18 (23.1%) | ||
| ≥ 10 | 70 (15.1%) | 42 (14.8%) | 17 (16.7%) | 14 (17.9%) | ||
| Lymphovascular invasion | 0.919 | 0.011 | ||||
| No | 341 (73.7%) | 215 (76.0%) | 78 (76.5%) | 48 (61.5%) | ||
| Yes | 122 (26.3%) | 68 (24.0%) | 24 (23.5%) | 30 (38.5%) | ||
| Histological grade | 0.905 | |||||
| No | 39 (10.1%) | 29 (10.2%) | 10 (9.80%) | 0 | ||
| I/II | 189 (49.1%) | 137 (48.4%) | 52 (51.0%) | 0 | ||
| III | 157 (40.8%) | 117 (41.3%) | 40 (39.2%) | 0 | ||
| Disease free survival event | 0.398 | 0.092 | ||||
| No | 357 (77.1%) | 225 (79.5%) | 77 (75.5%) | 55 (70.5%) | ||
| Yes | 106 (22.9%) | 58 (20.5%) | 25 (24.5%) | 23 (29.5%) |
Abbreviations: BC, breast cancer; HER2, human epidermal growth factor receptor type 2. P(1) refers to P-value in the comparison between training cohort and internal validation cohort. P(2) refers to P-value in the comparison between training cohort and external validation cohort.
Univariable and multivariable COX regression analyses
Univariate and multivariate Cox regression analyses revealed that age, residual tumor size, Ki67 change, molecular subtype, and ALNM were independent prognostic factors for DFS in non-PCR patients (P < 0.05) (Table 2). Consequently, these variables were included in the ML prediction model, which accurately revealed their impact on DFS.
Table 2.
Univariate and multivariate Cox regression analyses of key characteristics for predicting DFS in BC patients
| Characteristics | N=283 | Univariable | Multivariable | ||
|---|---|---|---|---|---|
|
|
|
||||
| HR (95% CI) | P Value | HR (95% CI) | P Value | ||
| Age, year | |||||
| > 35 | 238 (84.1%) | Reference | Reference | ||
| ≤ 35 | 45 (15.9%) | 2.14 (1.19-3.85) | 0.011 | 3.56 (1.87-6.78) | < 0.001 |
| Residual tumor size, cm | |||||
| ≤ 1 | 63(22.3%) | Reference | Reference | ||
| 1 >-≤ 3 | 154 (54.4%) | 1.36 (0.59-3.17) | 0.470 | 1.64 (0.65-4.16) | 0.294 |
| > 3 | 66 (23.3%) | 4.54 (1.98-10.44) | 0.001 | 3.30 (1.31-8.33) | 0.007 |
| Ki67 change | |||||
| -10-10% | 69 (24.4%) | Reference | Reference | ||
| ≥ 10% | 157 (55.5%) | 0.70 (0.36-1.36) | 0.293 | 0.88 (0.42-1.81) | 0.721 |
| ≤ -10% | 57 (20.1%) | 2.33 (1.19-4.55) | 0.014 | 2.16 (1.08-4.33) | 0.030 |
| Menstrual status | |||||
| Premenopause | 145 (51.2%) | Reference | |||
| Postmenopause | 138 (48.8%) | 0.66 (0.39-1.12) | 0.125 | ||
| Chemotherapy regimen | |||||
| Others | 62 (21.9%) | Reference | |||
| Anthracycline combined with taxane | 174 (61.5%) | 1.48 (0.72-3.05) | 0.287 | ||
| Platinum | 47 (16.6%) | 1.38 (0.53-3.59) | 0.506 | ||
| Targeted therapy | |||||
| No | 217 (76.7%) | Reference | |||
| Yes | 66 (23.3%) | 0.87 (0.47-1.61) | 0.654 | ||
| Molecular subtype | |||||
| Luminal | 131 (46.3%) | Reference | Reference | ||
| HER2 positive | 101 (35.7%) | 1.23 (0.68-2.22) | 0.498 | 1.55 (0.25-1.87) | 0.452 |
| Triple negative | 51 (18.0%) | 2.00 (1.03-3.89) | 0.041 | 3.04 (1.47-6.30) | 0.003 |
| Axillary lymph node metastasis | |||||
| 0 | 72 (25.4%) | Reference | Reference | ||
| 1-3 | 80 (28.3%) | 0.66 (0.25-1.78) | 0.418 | 0.68 (0.25-1.87) | 0.452 |
| 4-9 | 89 (31.4%) | 2.27 (1.06-4.89) | 0.036 | 2.35 (1.04-5.33) | 0.041 |
| ≥ 10 | 42 (14.8%) | 3.88 (1.74-8.64) | < 0.001 | 3.66 (1.46-9.17) | 0.006 |
| Lymphovascular invasion | |||||
| No | 215 (76.0%) | Reference | Reference | ||
| Yes | 68 (24.0%) | 1.82 (1.06-3.14) | 0.030 | 1.06 (0.59-1.92) | 0.841 |
| Histological grade | |||||
| No | 29 (10.2%) | Reference | |||
| I/II | 137 (48.4%) | 0.49 (0.19-1.25) | 0.136 | ||
| III | 117 (41.3%) | 1.66 (0.70-3.93) | 0.250 | ||
Abbreviations: BC, breast cancer; HER2, human epidermal growth factor receptor type 2; DFS, disease free survival; HR, hazard ratio; CI, confidence interval.
Evaluation of ML prediction models
In this study, we explored the predictive performance of four ML prediction models (Table 3). The results indicated that the RSF model had the best performance in the training and internal validation cohorts. In the training cohort, the RSF model had the highest C-Index 0.820 (95% CI: 0.778-0.870), indicating its excellent prediction accuracy. Furthermore, the AUC values of the RSF model at 1, 3, and 5 years were 0.899 (95% CI: 0.853-0.945), 0.865 (95% CI: 0.816-0.915), and 0.831 (95% CI: 0.765-0.898), respectively (Figure 2A-C), further confirming its good time-dependent prediction capabilities. Furthermore, the Brier scores of the RSF model at 1, 3, and 5 years were 0.065 (95% CI: 0.045-0.086), 0.113 (95% CI: 0.092-0.134), and 0.129 (95% CI: 0.104-0.156), respectively (Figure 3A), showing its good calibration performance and minimal error between predicted and actual risks. Moreover, the DCA results showed that the RSF model had high net benefits in the 1-, 3-, and 5-year forecasts (Figure 4A-C). Similarly, the RSF model demonstrated good performance in the internal validation cohort, with a slight decrease in each metric. Its C-Index was 0.642 (95% CI: 0.520-0.766), which remains commendable when compared to the other models. Furthermore, the AUC values of RSF in the cohort at 1, 3, and 5 years were 0.778 (95% CI: 0.620-0.937), 0.613 (95% CI: 0.476-0.751), and 0.645 (95% CI: 0.503-0.788), respectively (Figure 2D-F), further highlighting the model’s strong predictive power. Additionally, the model’s Brier scores at 1, 3, and 5 years were 0.080 (95% CI: 0.039-0.120), 0.180 (95% CI: 0.129-0.230), and 0.186 (95% CI: 0.135-0.237), respectively (Figure 3B), indicating that its calibration performance was still good in the internal validation cohort. Moreover, the DCA results confirmed that the RSF model consistently had high net benefits in the forecasts across all time points (Figure 4D-F). On the other hand, the CoxPH model performed quite stably. Specifically, it was inferior to the RSF model but superior to the GBM and XGBoost models. It is also noteworthy that the XGBoost model was similar to the CoxPH model in some indicators, although its overall prediction performance was not as good as that of the RSF and CoxPH models. Among the four models, the GBM model showed the weakest performance across all indicators, especially in calibration performance. After a comprehensive comparison, RSF was selected as the final prediction model, and was evaluated in the external validation cohort to further verify its applicability. Although the RSF model continued to perform well in the external validation cohort, it showed a slight difference in its performance metrics relative to the training and internal validation cohorts. It had a C-index of 0.689 (95% CI: 0.577-0.803), which is higher than that in the internal validation cohort and lower than that in the training cohort, but still shows its strong predictive accuracy on external data. The AUC values for the model’s 1-, 3-, and 5-year predictions in the external validation cohort were 0.720 (95% CI: 0.556-0.883), 0.780 (95% CI: 0.653-0.907), and 0.679 (95% CI: 0.524-0.833), respectively (Figure 5A), suggesting robust short- and mid-term predictive accuracies and optimal performance in the 3-year prediction. Although the model’s Brier scores were slightly higher in the external validation cohort [0.125 (95% CI: 0.063-0.187), 0.147 (95% CI: 0.094-0.199), and 0.226 (95% CI: 0.149-0.303) at 1, 3, and 5 years, respectively] (Figure 5B), its calibration performance in this cohort was still reasonable. According to the DCA analysis results, the RSF model continued to show high clinical net benefits at 1-, 3-, and 5-year predictions in the external validation cohort (Figure 5C-E), fully reflecting its practicability and reliability. Finally, patients were categorized into high- and low-risk groups based on the median risk score of the RSF model, and survival analysis was performed using KM curves and log-rank tests. According to the results, the high- and low-risk groups differed significantly in the DFS rate across different cohorts (P < 0.05), further proving the accuracy and practicability of the RSF model (Figure 6).
Table 3.
Prediction performance of the ML models
| Subgroup | Model | C-Index | AUC (95% CI) | Brier score (95% CI) | ||||
|---|---|---|---|---|---|---|---|---|
|
|
|
|||||||
| 1-Year | 3-Year | 5-Year | 1-Year | 3-Year | 5-Year | |||
| Training cohort | ||||||||
| RSF | 0.820 (0.778, 0.870) | 0.899 (0.853, 0.945) | 0.865 (0.816, 0.915) | 0.831 (0.765, 0.898) | 0.065 (0.045, 0.086) | 0.113 (0.092, 0.134) | 0.129 (0.104, 0.156) | |
| CoxPH | 0.800 (0.739, 0.851) | 0.876 (0.817, 0.936) | 0.830 (0.766, 0.894) | 0.791 (0.713, 0.870) | 0.062 (0.042, 0.083) | 0.115 (0.089, 0.142) | 0.135 (0.102, 0.168) | |
| GBM | 0.808 (0.760, 0.858) | 0.867 (0.806, 0.928) | 0.848 (0.790, 0.906) | 0.809 (0.733, 0.886) | 0.086 (0.054, 0.119) | 0.184 (0.138, 0.231) | 0.208 (0.158, 0.258) | |
| XGBoost | 0.807 (0.749, 0.855) | 0.883 (0.829, 0.937) | 0.840 (0.779, 0.900) | 0.799 (0.722, 0.876) | 0.063 (0.042, 0.085) | 0.116 (0.090, 0.142) | 0.134 (0.102, 0.166) | |
| Internal validation cohort | ||||||||
| RSF | 0.642 (0.520, 0.766) | 0.778 (0.620, 0.937) | 0.613 (0.476, 0.751) | 0.645 (0.503, 0.788) | 0.080 (0.039, 0.120) | 0.180 (0.129, 0.230) | 0.186 (0.135, 0.237) | |
| CoxPH | 0.649 (0.527, 0.765) | 0.788 (0.653, 0.924) | 0.593 (0.454, 0.733) | 0.623 (0.479, 0.766) | 0.087 (0.045, 0.129) | 0.202 (0.141, 0.264) | 0.204 (0.141, 0.266) | |
| GBM | 0.639 (0.526, 0.754) | 0.764 (0.622, 0.906) | 0.591 (0.454, 0.728) | 0.638 (0.495, 0.781) | 0.097 (0.039, 0.154) | 0.221 (0.139, 0.304) | 0.257 (0.169, 0.346) | |
| XGBoost | 0.647 (0.526, 0.761) | 0.776 (0.636, 0.916) | 0.591 (0.453, 0.729) | 0.624 (0.481, 0.766) | 0.087 (0.045, 0.129) | 0.201 (0.141, 0.261) | 0.204 (0.142, 0.265) | |
| External validation cohort | ||||||||
| RSF | 0.689 (0.577, 0.803) | 0.720 (0.556, 0.883) | 0.780 (0.653, 0.907) | 0.679 (0.524, 0.833) | 0.125 (0.063, 0.187) | 0.147 (0.094, 0.199) | 0.226 (0.149, 0.303) | |
Abbreviations: ML, machine learning; C-index, concordance index; AUC, area under the curve; RSF, random survival forest; CoxPH, cox proportional hazards; GBM, gradient boosting machine; XGBoost, extreme gradient boosting.
Figure 2.
Comparison of the Time dependent ROC curves of the ML models’ performance for the 1-, 3-, and 5-year follow-up time in the training cohort (A-C) and the internal validation cohort (D-F). ML, machine learning; ROC, receiver operating characteristic; RSF, random survival forest; CoxPH, cox proportional hazards; GBM, gradient boosting machine; XGBoost, extreme gradient boosting.
Figure 3.
Brier score for the prediction performance of the ML models for the 1-, 3-, and 5-year outcomes in the training cohort (A) and the internal validation cohort (B). ML, machine learning; RSF, random survival forest; CoxPH, cox proportional hazards; GBM, gradient boosting machine; XGBoost, extreme gradient boosting.
Figure 4.
DCA curves showing the net benefit of the ML models in predicting the 1-, 3-, and 5-year outcomes in the training cohort (A-C) and the internal validation cohort (D-F). ML, machine learning; DCA, decision curve analysis; RSF, random survival forest; CoxPH, cox proportional hazards; GBM, gradient boosting machine; XGBoost, extreme gradient boosting.
Figure 5.
Time dependent ROC curves compared the performance of the RSF model at 1-, 3-, and 5-year follow-up time in the external validation cohort (A); Comparison of Brier scores for the RSF models in predicting the 1-, 3-, and 5-year outcome in the external validation cohort (B); DCA curves comparing the net benefit of the RSF model in predicting at the 1-, 3-, and 5-year outcomes in the external validation cohort (C-E). ROC, receiver operating characteristic; RSF, random survival forest; DCA, decision curve analysis.
Figure 6.
KM curves estimated DFS probabilities in the training cohort (A), internal validation cohort (B), and external validation cohort (C). DFS, disease free survival; KM, Kaplan-Meier.
Model interpretability
An in-depth analysis of the importance of the characteristics of the RSF model was performed using the SHAP method. The characteristics importance plot (Figure 7A) showed that the significance of residual tumor size, ALNM, Ki67 change, molecular subtype, and age decreased in turn. Moreover, the boxplot and partial dependence plot of categorical characteristics facilitated the visualization of the impact of each characteristic on the model’s predictions, highlighting their significance and the extent of their influence in the decision-making process. The boxplot of categorical characteristics (Figure 7B) identified the contribution size of different characteristics in the prediction, illustrating the role of each characteristic. On the other hand, the partial dependence plot for categorical characteristics (Figure 7C) demonstrated the nonlinear effect of characteristic value variations on prediction outcomes, further confirming the intricate relationship between predictive factors and model performance. Moreover, the individual waterfall plots (Figure 7D and 7E) vividly illustrated the varying contributions of each characteristic to DFS predictions, which improved the transparency and interpretability of the model’s decision-making process.
Figure 7.
Characteristics importance plot (A); Boxplot of categorical characteristics (B); Partial dependence plot of categorical characteristics (C); Individual waterfall plots vividly illustrated the varying contributions of each characteristic to DFS predictions, such as a Ki67 change of ≤ -10% and the presence of ≥ 10 ALNMs positively affected the model’s prediction outcomes, whereas ALNM absence and a residual tumor size ≤ 1 cm negatively affected the predictions (D and E). DFS, disease free survival; ALNM, axillary lymph node metastases; CHF, cumulative hazard function.
Discussion
NAC can significantly reduce tumor volume and downstage the clinical stages in BC, as well as facilitate the evaluation of the primary tumor’s response to chemotherapy. Moreover, BC patients who achieve a PCR following NAC generally experience a more favorable prognosis [22]. However, categorizing patients solely into PCR and non-PCR groups is overly simplistic, especially given the fact that the prognosis of non-PCR patients varies significantly, ranging from outcomes comparable to those of PCR patients to complete drug resistance. Consequently, more accurate stratification models for non-PCR patients need to be developed. Several evaluation systems such as the Miller-Payne (MP) and Residual Cancer Burden (RCB) systems have been demonstrated to more accurately assess the prognosis of patients post-NAC [23,24]. The MP system evaluates treatment efficacy primarily based on the degree of tumor cell reduction. However, it does not adequately account for the impact of ALNM, potentially overestimating the prognosis of some patients with ALNM. In contrast, the RCB system assesses RCB through multiple integrated parameters, offering a more comprehensive evaluation. However, it cannot effectively differentiate survival outcomes among various BC subtypes. On this basis, more accurate prediction models based on individual clinicopathological characteristics are needed to facilitate the identification of high-risk groups among non-PCR patients, thereby improve the implementation of early interventions. Nomograms are often used to predict the prognosis of BC patients. Lan et al. developed a nomogram model for predicting DFS in non-PCR patients based on biopsy and surgical specimens. The model had C-index values of 0.693 and 0.701 before and after NAC, respectively, demonstrating its good predictive performance [10]. Furthermore, Yu et al. constructed a nomogram model for predicting OS based on clinicopathological characteristics of non-PCR patients. The AUC values of the model in predicting the 3- and 5-year survival outcomes were 0.950 and 0.790, respectively [12]. In recent years, ML models are increasingly applied in clinical practice to develop prognosis prediction models. Takada et al. employed the Adtree algorithm for DFS prediction in HER2-positive BC patients treated with NAC combined with trastuzumab. The model had an AUC value of 0.785, indicating its high predictive accuracy [11]. However, few ML-based prediction models for non-PCR patients have been developed, and most lack external validation. Furthermore, the algorithms’ limitations often constrain the performance of the models.
Our univariate and multivariate Cox regression analyses revealed that age, residual tumor size, Ki67 change, molecular subtype, and ALNM were independent prognostic factors for predicting DFS in non-PCR patients. Age is one of the key factors influencing BC prognosis. According to research, young BC patients often show more aggressive biological behavior and poor prognosis [25-27]. Besides being an independent factor influencing the long-term prognosis of BC patients, age can be used to predict the recurrence and metastasis of non-PCR patients [22]. Herein, non-PCR patients aged ≤ 35 years had a significantly higher risk of reduced DFS compared to those aged > 35 years (HR: 3.56; 95% CI: 1.87-6.78). Previous research has shown that residual tumor size is closely correlated with DFS, with larger residual tumors being linked to a higher risk of recurrence and metastasis [28]. Furthermore, patients with a residual tumor size of ≤ 1 cm showed a prognosis comparable to that of PCR patients [29]. Using 1 cm and 3 cm as the comparison thresholds, the analysis revealed that non-PCR patients with a residual tumor size > 3 cm had a significantly increased risk of a reduced DFS (HR: 3.30; 95% CI: 1.31-8.33). On the other hand, Ki67 is a cell proliferation marker and its high expression often reflects active tumor proliferation and a high degree of malignancy, which correlate with a poor prognosis [30]. Most studies have primarily investigated the association between baseline Ki67 expression and PCR, with some studies demonstrating that Ki67 expression is upregulated in residual tumors following chemotherapy, which correlates with poor prognosis [31-34]. Herein, the results indicated that non-PCR patients with a Ki67 change of ≤ -10% post-NAC tended to have reduced DFS (HR=2.16, 95% CI: 1.08-4.33), implying that changes in Ki67 post-NAC may help to predict DFS in non-PCR patients. Molecular subtype have been shown to significantly influence prognosis. Research indicates that HR+ BC patients typically benefit from extended endocrine therapy, which is linked to a delayed onset of recurrence and metastasis [35]. For HER2+ BC patients, targeted combination therapies such as trastuzumab + pertuzumab could significantly increase PCR rates and improve prognosis [36]. In contrast, TNBC patients have a higher rate of recurrence and metastasis, mainly within the first 3 to 5 years after diagnosis. These patients often have a poorer long-term prognosis and are more prone to visceral metastasis [37]. Similarly, we found that TNBC correlated with a significantly high risk of DFS reduction in non-PCR patients (HR: 3.04; 95% CI: 1.47-6.30). Finally, the other critical prognostic indicator in BC patients is ALNM. Research indicates that the number of metastatic lymph nodes significantly impacts patient survival. In particular, survival rates decrease notably as the number of metastatic lymph nodes increases [38]. In a detailed analysis of the number of postoperative metastatic lymph nodes in non-PCR patients, Colleoni et al. found that the 5-year DFS rates of patients with 0, 1-3, and ≥ 4 metastatic lymph nodes were 71%, 67%, and 49%, respectively [39]. Our study reached similar conclusions, revealing that the DFS worsened with an increasing number of metastatic lymph nodes (ALNM 4-9: HR=2.35, 95% CI=1.04-5.33; ALNM ≥ 10: HR=3.66, 95% CI=1.46-9.17).
Using these five key characteristics, we constructed four ML prediction models for predicting DFS in non-PCR patients. According to the results, the RSF model consistently outperformed the other three models in the prediction of 1-, 3-, and 5-year DFS, across the training, internal validation, and external validation cohorts, exhibiting superior C-Index and AUC values, which demonstrated its high predictive accuracy and stability. Furthermore, the calibration evaluation results showed that the Brier scores of the RSF model remained low across different time points, reflecting its good calibration ability. Compared to the other models, the RSF model also showed higher net benefits in DCA across multiple time points, indicating its greater utility in clinical applications. Moreover, the impact of clinicopathological characteristics on the model’s prediction results was thoroughly examined using SHAP analysis, which ensured its interpretability and improved its application in clinical decision-making.
Although this study yields promising results, there are several limitations that need to be discussed. Firstly, more than 30% of histological grade data were missing in the external validation cohort, and significant differences were observed in various parameters such as age and chemotherapy regimens between the external validation cohort and the training cohort, which may limit the model’s generalizability. Although multiple datasets were included in the validation tests, the overall sample size was relatively small, especially in the external validation cohort, which may have decreased the robustness of the results. Secondly, the RSF model’s long-term predictive performance declined in the external validation cohort, suggesting the need for further validation of its stability in a broader population. Moreover, this study relied solely on common clinicopathological characteristics and did not incorporate potentially significant biomarkers such as gene mutations and the tumor microenvironment, which may influence the model’s prediction accuracy. Finally, since the external validation cohort was sourced from a single region, the model’s applicability to other geographic areas and populations may be limited. Future research should aim to validate the performance of the model using data from various regions and institutions to enhance its applicability and robustness.
Conclusions
In this study, we comprehensively constructed and compared four ML prediction models, and found the RSF model as the best prediction tool. This model could predict DFS in BC patients with non-PCR post-NAC. The results of this study are expected to guide the development of the individualized treatment strategies. As BC data continues to accumulate and ML technology advances, personalized cancer treatments will become more precise, which is crucial for improving patient outcomes. Moreover, the integration of AI-assisted decision systems is expected to enhance clinical cancer management, optimize treatment strategies, and provide stronger prognostic support for patients.
Acknowledgements
The authors would like to thank all the reviewers who participated in the review and MJEditor (www.mjeditor.com) for its linguistic assistance during the preparation of this manuscript.
Disclosure of conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supporting Information
References
- 1.Bray F, Laversanne M, Sung H, Ferlay J, Siegel RL, Soerjomataram I, Jemal A. Global cancer statistics 2022: globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2024;74:229–263. doi: 10.3322/caac.21834. [DOI] [PubMed] [Google Scholar]
- 2.Holmes D, Colfry A, Czerniecki B, Dickson-Witmer D, Francisco Espinel C, Feldman E, Gallagher K, Greenup R, Herrmann V, Kuerer H, Malik M, Manahan E, O’Neill J, Patel M, Sebastian M, Wheeler A, Kass R. Performance and practice guideline for the use of neoadjuvant systemic therapy in the management of breast cancer. Ann Surg Oncol. 2015;22:3184–3190. doi: 10.1245/s10434-015-4753-3. [DOI] [PubMed] [Google Scholar]
- 3.Shien T, Iwata H. Adjuvant and neoadjuvant therapy for breast cancer. Jpn J Clin Oncol. 2020;50:225–229. doi: 10.1093/jjco/hyz213. [DOI] [PubMed] [Google Scholar]
- 4.I-SPY2 Trial Consortium. Yee D, DeMichele AM, Yau C, Isaacs C, Symmans WF, Albain KS, Chen YY, Krings G, Wei S, Harada S, Datnow B, Fadare O, Klein M, Pambuccian S, Chen B, Adamson K, Sams S, Mhawech-Fauceglia P, Magliocco A, Feldman M, Rendi M, Sattar H, Zeck J, Ocal IT, Tawfik O, LeBeau LG, Sahoo S, Vinh T, Chien AJ, Forero-Torres A, Stringer-Reasor E, Wallace AM, Pusztai L, Boughey JC, Ellis ED, Elias AD, Lu J, Lang JE, Han HS, Clark AS, Nanda R, Northfelt DW, Khan QJ, Viscusi RK, Euhus DM, Edmiston KK, Chui SY, Kemmer K, Park JW, Liu MC, Olopade O, Leyland-Jones B, Tripathy D, Moulder SL, Rugo HS, Schwab R, Lo S, Helsten T, Beckwith H, Haugen P, Hylton NM, Van’t Veer LJ, Perlmutter J, Melisko ME, Wilson A, Peterson G, Asare AL, Buxton MB, Paoloni M, Clennell JL, Hirst GL, Singhrao R, Steeg K, Matthews JB, Asare SM, Sanil A, Berry SM, Esserman LJ, Berry DA. Association of event-free and distant recurrence-free survival with individual-level pathologic complete response in neoadjuvant treatment of stages 2 and 3 breast cancer: three-year follow-up analysis for the i-spy2 adaptively randomized clinical trial. JAMA Oncol. 2020;6:1355–1362. doi: 10.1001/jamaoncol.2020.2535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Gabani P, Merfeld E, Srivastava AJ, Weiner AA, Ochoa LL, Mullen D, Thomas MA, Margenthaler JA, Cyr AE, Peterson LL, Naughton MJ, Ma C, Zoberi I. Predictors of locoregional recurrence after failure to achieve pathologic complete response to neoadjuvant chemotherapy in triple-negative breast cancer. J Natl Compr Canc Netw. 2019;17:348–356. doi: 10.6004/jnccn.2018.7103. [DOI] [PubMed] [Google Scholar]
- 6.Kennedy WR, Tricarico C, Gabani P, Weiner AA, Altman MB, Ochoa LL, Thomas MA, Margenthaler JA, Sanati S, Peterson LL, Ma CX, Ademuyiwa FO, Zoberi I. Predictors of distant metastases in triple-negative breast cancer without pathologic complete response after neoadjuvant chemotherapy. J Natl Compr Canc Netw. 2020;18:288–296. doi: 10.6004/jnccn.2019.7366. [DOI] [PubMed] [Google Scholar]
- 7.Masuda N, Lee SJ, Ohtani S, Im YH, Lee ES, Yokota I, Kuroi K, Im SA, Park BW, Kim SB, Yanagita Y, Ohno S, Takao S, Aogi K, Iwata H, Jeong J, Kim A, Park KH, Sasano H, Ohashi Y, Toi M. Adjuvant capecitabine for breast cancer after preoperative chemotherapy. N Engl J Med. 2017;376:2147–2159. doi: 10.1056/NEJMoa1612645. [DOI] [PubMed] [Google Scholar]
- 8.von Minckwitz G, Huang CS, Mano MS, Loibl S, Mamounas EP, Untch M, Wolmark N, Rastogi P, Schneeweiss A, Redondo A, Fischer HH, Jacot W, Conlin AK, Arce-Salinas C, Wapnir IL, Jackisch C, DiGiovanna MP, Fasching PA, Crown JP, Wülfing P, Shao Z, Rota Caremoli E, Wu H, Lam LH, Tesarowski D, Smitt M, Douthwaite H, Singel SM, Geyer CE Jr KATHERINE Investigators. Trastuzumab emtansine for residual invasive HER2-positive breast cancer. N Engl J Med. 2019;380:617–628. doi: 10.1056/NEJMoa1814017. [DOI] [PubMed] [Google Scholar]
- 9.Swanson K, Wu E, Zhang A, Alizadeh AA, Zou J. From patterns to patients: advances in clinical machine learning for cancer diagnosis, prognosis, and treatment. Cell. 2023;186:1772–1791. doi: 10.1016/j.cell.2023.01.035. [DOI] [PubMed] [Google Scholar]
- 10.Lan A, Li H, Chen J, Shen M, Jin Y, Dai Y, Jiang L, Dai X, Peng Y, Liu S. Nomograms for predicting disease-free survival based on core needle biopsy and surgical specimens in female breast cancer patients with non-pathological complete response to neoadjuvant chemotherapy. J Pers Med. 2023;13:249. doi: 10.3390/jpm13020249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Takada M, Sugimoto M, Masuda N, Iwata H, Kuroi K, Yamashiro H, Ohno S, Ishiguro H, Inamoto T, Toi M. Prediction of postoperative disease-free survival and brain metastasis for HER2-positive breast cancer patients treated with neoadjuvant chemotherapy plus trastuzumab using a machine learning algorithm. Breast Cancer Res Treat. 2018;172:611–618. doi: 10.1007/s10549-018-4958-9. [DOI] [PubMed] [Google Scholar]
- 12.Yu Y, Wu S, Xing H, Han M, Li J, Liu Y. Development and validation of a novel model for predicting prognosis of non-PCR patients after neoadjuvant therapy for breast cancer. Front Oncol. 2021;11:675533. doi: 10.3389/fonc.2021.675533. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Allison KH, Hammond MEH, Dowsett M, McKernin SE, Carey LA, Fitzgibbons PL, Hayes DF, Lakhani SR, Chavez-MacGregor M, Perlmutter J, Perou CM, Regan MM, Rimm DL, Symmans WF, Torlakovic EE, Varella L, Viale G, Weisberg TF, McShane LM, Wolff AC. Estrogen and progesterone receptor testing in breast cancer: Asco/cap guideline update. J. Clin. Oncol. 2020;38:1346–1366. doi: 10.1200/JCO.19.02309. [DOI] [PubMed] [Google Scholar]
- 14.Wolff AC, Hammond MEH, Allison KH, Harvey BE, McShane LM, Dowsett M. HER2 testing in breast cancer: American society of clinical oncology/college of american pathologists clinical practice guideline focused update summary. J Oncol Pract. 2018;14:437–441. doi: 10.1200/JOP.18.00206. [DOI] [PubMed] [Google Scholar]
- 15.Zhou L, Wang H, Xu Q. Random rotation survival forest for high dimensional censored data. Springerplus. 2016;5:1425. doi: 10.1186/s40064-016-3113-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Zhang K, Ye B, Wu L, Ni S, Li Y, Wang Q, Zhang P, Wang D. Machine learning-based prediction of survival prognosis in esophageal squamous cell carcinoma. Sci Rep. 2023;13:13532. doi: 10.1038/s41598-023-40780-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Friedman JH. Stochastic gradient boosting. Computational Statistics & Data Analysis. 2002;38:367–378. [Google Scholar]
- 18.Li C, Liu M, Zhang Y, Wang Y, Li J, Sun S, Liu X, Wu H, Feng C, Yao P, Jia Y, Zhang Y, Wei X, Wu F, Du C, Zhao X, Zhang S, Qu J. Novel models by machine learning to predict prognosis of breast cancer brain metastases. J Transl Med. 2023;21:404. doi: 10.1186/s12967-023-04277-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Chowdhury SU, Sayeed S, Rashid I, Alam MGR, Masum AKM, Dewan MAA. Shapley-additive-explanations-based factor analysis for dengue severity prediction using machine learning. J Imaging. 2022;8:229. doi: 10.3390/jimaging8090229. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lundberg SM, Lee SI. A unified approach to interpreting model predictions. arXiv [Preprint] 2017 arXiv: 1705.07874. [Google Scholar]
- 21.Lang M, Binder M, Richter J, Schratz P, Pfisterer F, Coors S, Au Q, Casalicchio G, Kotthoff L, Bischl B. Mlr3: a modern object-oriented machine learning framework in R. J Open Source Softw. 2019;4:1903. [Google Scholar]
- 22.Haque W, Verma V, Hatch S, Suzanne Klimberg V, Brian Butler E, Teh BS. Response rates and pathologic complete response by breast cancer molecular subtype following neoadjuvant chemotherapy. Breast Cancer Res Treat. 2018;170:559–567. doi: 10.1007/s10549-018-4801-3. [DOI] [PubMed] [Google Scholar]
- 23.Ogston KN, Miller ID, Payne S, Hutcheon AW, Sarkar TK, Smith I, Schofield A, Heys SD. A new histological grading system to assess response of breast cancers to primary chemotherapy: prognostic significance and survival. Breast. 2003;12:320–327. doi: 10.1016/s0960-9776(03)00106-1. [DOI] [PubMed] [Google Scholar]
- 24.Symmans WF, Peintinger F, Hatzis C, Rajan R, Kuerer H, Valero V, Assad L, Poniecka A, Hennessy B, Green M, Buzdar AU, Singletary SE, Hortobagyi GN, Pusztai L. Measurement of residual breast cancer burden to predict survival after neoadjuvant chemotherapy. J. Clin. Oncol. 2007;25:4414–4422. doi: 10.1200/JCO.2007.10.6823. [DOI] [PubMed] [Google Scholar]
- 25.Kataoka A, Iwamoto T, Tokunaga E, Tomotaki A, Kumamaru H, Miyata H, Niikura N, Kawai M, Anan K, Hayashi N, Masuda S, Tsugawa K, Aogi K, Ishida T, Masuoka H, Iijima K, Kinoshita T, Nakamura S, Tokuda Y. Young adult breast cancer patients have a poor prognosis independent of prognostic clinicopathological factors: a study from the japanese breast cancer registry. Breast Cancer Res Treat. 2016;160:163–172. doi: 10.1007/s10549-016-3984-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Kim J, Hong S, Lee JJ, Won YJ, Lee ES, Kang HS, Lee S, Han JH, Lee EG, Jo H, Kim HH, Jung SY. Analysis of the tumor characteristics in young age breast cancer patients using collaborative stage data of the korea central cancer registry. Breast Cancer Res Treat. 2021;187:785–792. doi: 10.1007/s10549-021-06107-9. [DOI] [PubMed] [Google Scholar]
- 27.Tzikas AK, Nemes S, Linderholm BK. A comparison between young and old patients with triple-negative breast cancer: biology, survival and metastatic patterns. Breast Cancer Res Treat. 2020;182:643–654. doi: 10.1007/s10549-020-05727-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Pestana CV, Livasy CA, Donahue EE, Neelands B, Tan AR, Sarantou T, Hadzikadic-Gusic L, White RL Jr. Does residual cancer burden predict local recurrence after neoadjuvant chemotherapy? Ann Surg Oncol. 2022;29:7716–7724. doi: 10.1245/s10434-022-12038-6. [DOI] [PubMed] [Google Scholar]
- 29.Romond EH, Perez EA, Bryant J, Suman VJ, Geyer CE Jr, Davidson NE, Tan-Chiu E, Martino S, Paik S, Kaufman PA, Swain SM, Pisansky TM, Fehrenbacher L, Kutteh LA, Vogel VG, Visscher DW, Yothers G, Jenkins RB, Brown AM, Dakhil SR, Mamounas EP, Lingle WL, Klein PM, Ingle JN, Wolmark N. Trastuzumab plus adjuvant chemotherapy for operable HER2-positive breast cancer. N Engl J Med. 2005;353:1673–1684. doi: 10.1056/NEJMoa052122. [DOI] [PubMed] [Google Scholar]
- 30.Chen X, He C, Han D, Zhou M, Wang Q, Tian J, Li L, Xu F, Zhou E, Yang K. The predictive value of ki-67 before neoadjuvant chemotherapy for breast cancer: a systematic review and meta-analysis. Future Oncol. 2017;13:843–857. doi: 10.2217/fon-2016-0420. [DOI] [PubMed] [Google Scholar]
- 31.Ingolf JB, Russalina M, Simona M, Julia R, Gilda S, Bohle RM, Andrea H, Erich S, Daniel H. Can ki-67 play a role in prediction of breast cancer patients’ response to neoadjuvant chemotherapy? Biomed Res Int. 2014;2014:628217. doi: 10.1155/2014/628217. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Montagna E, Bagnardi V, Viale G, Rotmensz N, Sporchia A, Cancello G, Balduzzi A, Galimberti V, Veronesi P, Luini A, Mastropasqua MG, Casadio C, Sangalli C, Goldhirsch A, Colleoni M. Changes in pgr and ki-67 in residual tumour and outcome of breast cancer patients treated with neoadjuvant chemotherapy. Ann Oncol. 2015;26:307–313. doi: 10.1093/annonc/mdu528. [DOI] [PubMed] [Google Scholar]
- 33.Yamazaki N, Wada N, Yamauchi C, Yoneyama K. High expression of post-treatment ki-67 status is a risk factor for locoregional recurrence following breast-conserving surgery after neoadjuvant chemotherapy. Eur J Surg Oncol. 2015;41:617–624. doi: 10.1016/j.ejso.2015.01.036. [DOI] [PubMed] [Google Scholar]
- 34.Yoshioka T, Hosoda M, Yamamoto M, Taguchi K, Hatanaka KC, Takakuwa E, Hatanaka Y, Matsuno Y, Yamashita H. Prognostic significance of pathologic complete response and ki67 expression after neoadjuvant chemotherapy in breast cancer. Breast Cancer. 2015;22:185–191. doi: 10.1007/s12282-013-0474-2. [DOI] [PubMed] [Google Scholar]
- 35.Davies C, Pan H, Godwin J, Gray R, Arriagada R, Raina V, Abraham M, Medeiros Alencar VH, Badran A, Bonfill X, Bradbury J, Clarke M, Collins R, Davis SR, Delmestri A, Forbes JF, Haddad P, Hou MF, Inbar M, Khaled H, Kielanowska J, Kwan WH, Mathew BS, Mittra I, Müller B, Nicolucci A, Peralta O, Pernas F, Petruzelka L, Pienkowski T, Radhika R, Rajan B, Rubach MT, Tort S, Urrútia G, Valentini M, Wang Y, Peto R Adjuvant Tamoxifen: Longer Against Shorter (ATLAS) Collaborative Group. Long-term effects of continuing adjuvant tamoxifen to 10 years versus stopping at 5 years after diagnosis of oestrogen receptor-positive breast cancer: atlas, a randomised trial. Lancet. 2013;381:805–816. doi: 10.1016/S0140-6736(12)61963-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Gianni L, Pienkowski T, Im YH, Tseng LM, Liu MC, Lluch A, Starosławska E, de la Haba-Rodriguez J, Im SA, Pedrini JL, Poirier B, Morandi P, Semiglazov V, Srimuninnimit V, Bianchi GV, Magazzù D, McNally V, Douthwaite H, Ross G, Valagussa P. 5-year analysis of neoadjuvant pertuzumab and trastuzumab in patients with locally advanced, inflammatory, or early-stage HER2-positive breast cancer (neosphere): a multicentre, open-label, phase 2 randomised trial. Lancet Oncol. 2016;17:791–800. doi: 10.1016/S1470-2045(16)00163-7. [DOI] [PubMed] [Google Scholar]
- 37.Liedtke C, Mazouni C, Hess KR, André F, Tordai A, Mejia JA, Symmans WF, Gonzalez-Angulo AM, Hennessy B, Green M, Cristofanilli M, Hortobagyi GN, Pusztai L. Response to neoadjuvant therapy and long-term survival in patients with triple-negative breast cancer. J. Clin. Oncol. 2008;26:1275–81. doi: 10.1200/JCO.2007.14.4147. [DOI] [PubMed] [Google Scholar]
- 38.Wu SG, Li Q, Zhou J, Sun JY, Li FY, Lin Q, Lin HX, Gaun XX, He ZY. Using the lymph node ratio to evaluate the prognosis of stage ii/iii breast cancer patients who received neoadjuvant chemotherapy and mastectomy. Cancer Res Treat. 2015;47:757–764. doi: 10.4143/crt.2014.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Colleoni M, Bagnardi V, Rotmensz N, Dellapasqua S, Viale G, Pruneri G, Veronesi P, Torrisi R, Luini A, Intra M, Galimberti V, Montagna E, Goldhirsch A. A risk score to predict disease-free survival in patients not achieving a pathological complete remission after preoperative chemotherapy for breast cancer. Ann Oncol. 2009;20:1178–1184. doi: 10.1093/annonc/mdn747. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.