Machine learning approaches in the prediction of positive axillary lymph nodes post neoadjuvant chemotherapy using MRI, CT, or ultrasound: A systematic review

Abstract

Background and objective

Neoadjuvant chemotherapy is a standard treatment approach for locally advanced breast cancer. Conventional imaging modalities, such as magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound, have been used for axillary lymph node evaluation which is crucial for treatment planning and prognostication. This systematic review aims to comprehensively examine the current research on applying machine learning algorithms for predicting positive axillary lymph nodes following neoadjuvant chemotherapy utilizing imaging modalities, including MRI, CT, and ultrasound.

Methods

A systematic search was conducted across databases, including PubMed, Scopus, and Web of Science, to identify relevant studies published up to December 2023. Articles employing machine learning algorithms to predict positive axillary lymph nodes using MRI, CT, or ultrasound data after neoadjuvant chemotherapy were included. The review follows the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines, encompassing data extraction and quality assessment.

Results

Seven studies were included, comprising 1502 patients. Four studies used MRI, two used CT, and one applied ultrasound. Two studies developed deep-learning models, while five used classic machine-learning models mainly based on multiple regression. Across the studies, the models showed high predictive accuracy, with the best-performing models combining radiomics and clinical data.

Conclusion

This systematic review demonstrated the potential of utilizing advanced data analysis techniques, such as deep learning radiomics, in improving the prediction of positive axillary lymph nodes in breast cancer patients following neoadjuvant chemotherapy.

1. Introduction

With the greatest incidence rate among women, breast cancer is the most frequent malignant tumor worldwide [1]. For patients with clinically node-positive breast cancer, the usual course of treatment is neoadjuvant chemotherapy (NAC) prior to surgery. It can lessen the amount of tumor burden and downgrade the axilla, which increases the likelihood of breast conservation and limits the scope of axillary surgery [2], [3], [4].

Using neoadjuvant chemotherapy has several benefits. It gives a unique possibility for evaluating remedial response with the entire pathologic response appearing as a surrogate marker of survival and for an extra rapid evaluation of the efficacy of recent healing agents and early cessation of useless remedies. Patients can avoid toxicity and side effects by adjusting the dose and switching to another drug in case of resistance to medication. Moreover, neoadjuvant chemotherapy provides an opportunity for individualized therapy and enables the collection of tumor samples before, during, and after treatment for translational studies. This evaluation of tumor behavior in situ throughout neoadjuvant chemotherapy and its correlation with scientific final results is a top-notch model to determine the predictive role of tumor characteristics [5]. Additionally, the prognosis will be improved for patients who experienced axillary pathologic complete response (pCR) following NAC [6].

According to earlier research, between 21.9 % and 55.1 % of patients with clinically positive nodes attained axillary pCR following NAC [7]. Axillary lymph node dissection (ALND) is not required for these patients. It is crucial to correctly identify patients with axillary pCR following NAC [8]. The axillary lymph node status prognostic value is a valuable tool for precisely assessing the treatment response, which is crucial for the management of breast cancer. Confirming pCR without operation has never been simple. Currently, ultrasonography is used to evaluate axillary LN.

Nevertheless, the axillary ultrasound examination is insufficiently good. Breast MRI performed less well than ultrasound in terms of predicting axillary pCR following NAC [9], [10]. Contrast-enhanced chest CT can display the morphology of LNs and their relationship to surrounding structures, such as axillary vessels. Additionally, contrast enhancement could be used to assess LNs' blood supply. Currently, however, axillary LN response to NAC is rarely assessed by CT [11]. Furthermore, the results of the SENTINA and ACOSOG 1071 studies indicated that sentinel lymph node biopsy (SLNB) after NAC had a false negative rate of greater than 10 % for patients with unselected breast cancer [9], [12], [13], [14], [15].

To predict axillary pCR, axillary treatment response evaluation needs to be upgraded. Machine learning developments have made it easier to answer challenging clinical problems. This systematic review aimed to comprehensively examine the current research on applying machine learning algorithms for predicting positive axillary lymph nodes following NAC using imaging modalities, including MRI, CT, or ultrasound.

2. Methods

2.1. Search strategy

This review followed the preferred reporting items for systematic review and meta-analysis (PRISMA) guidelines. The protocol of this systematic review is registered on PROSPERO (CRD42023422375). We searched databases, including PubMed, Scopus, web of Science, EMBASE, and Google Scholar. Only publicly available and reported data until 2023 were eligible for inclusion. We used a combination of synonyms for NAC, MRI, CT, US, breast cancer, and machine learning as a search string. The search was limited to English language documents.

2.2. Inclusion and exclusion criteria

Data from the included studies were extracted and stored using Excel 2020 spreadsheets. Reviewers applied selection criteria after screening the possibly included studies. In the following, the full text of the selected article was reviewed by two authors. Duplicated documentation was removed using end note X9 software or manually.

2.3. Data extraction

Two reviewers independently extracted the data from the included studies. The following details are presented in this review: first author name, date, country, sample size of people, imaging modality, contrast agent, true positive, true negative, false positive, false negative, and adjusted significant confounders. Discussing with the third reviewer resolved any discrepancy between the two reviewers. For examination with more than one report, information was assembled from the foremost later finding.

2.4. Quality assessment

All included studies were evaluated through the revised quality assessment tool for diagnostic accuracy studies (QUADAS-2). This tools evaluates the risk of bias and applicability concerns through assessment of patients selection, index test, reference standard, and flow and timing. Two reviewers assessed the high-quality content of the included articles one by one and resolved disagreements primarily based on consensus.

3. Results

3.1. Literature search

Fig. 1 presents an overview of the study selection process. Applying predetermined search criteria, we identified 409 records from PubMed, 147 from Web of Science, and 187 from Scopus. Initially, we eliminated 50 duplicate records and excluded 656 studies based on the inclusion and exclusion criteria. The remaining 37 articles underwent thorough full-text examination, resulting in the selection of seven articles for inclusion in our systematic review.

Fig. 1.

PRISMA flowchart of the medical database search strategy.

3.2. Characteristics of the included studies and the predictive models

Table 1 represents the characteristics of the seven studies included in the systematic review. The studies comprised 1502 patients, with 1094 individuals in the training cohorts, 214 in the validation cohorts, and 194 in the test cohorts. Only two studies employed a prospective design, while the remaining studies utilized retrospective designs. Additionally, the studies used radiomics techniques based on various imaging modalities, including MRI (N = 4), ultrasound (N = 2), and CT (N = 1). Among the four MRI studies, three utilized dynamic contrast MRI, while one employed multiparametric MRI.

Table 1.

Characteristics of the included studies.F.

Author, Year	Country	Number of cases	Age	Postmenopausal (%)	Molecular subtype					Imaging modality	Number of cases in each cohort
					HER2+ (%)	n.HR+ (%)	PR+ (%)	ER+ (%)	Triple negative (%)		training cohort patient number	validation cohort patient number	Testing cohort patient number
Chen et al., 2022 [16]	China	158	49.6, (9.9), (26–79) (Mean, (SD), (Range))	47/5	39/2	80/4	N/S	N/S	N/S	Multiparametric MR	158	N/S	N/S
Liu et al., 2023 [17]	China	120	50.9, (10.1), (31–73) (Mean, (SD), (Range))	51/6	42/5	N/S	57/5	51/7	N/S	DCE-MRI	84	36	N/S
Gan et al., 2021 [18]	China	248	N/S	46	46/0	56/9	N/S	N/S	N/S	DCE-MRI	125	53	70
Ha et al., 2018 [19]	USA	127	50, (23–82) (Median, (Range))	N/S	27/6	N/S	N/S	63/0	26	DCE-MRI	101	26	N/S
Gu et al., 2022 [20]	China	484	47.4, (14.3) (Mean, (SD))	N/S	38/8	42/4	N/S	59/5	18/8	Ultrasound	297	99	88
Li et al., 2023 [22]	China	138	49, (9.3) (Mean, (SD))	N/S	35/5	N/S	70/3	73/2	N/S	CECT before and after NAC	102	N/S	36
Kim et al., 2019 [21]	South Korea	227	48, 49, (27–69) (Mean, Median, (Range))	34/8	29/5	71/4	N/S	N/S	N/S	Ultrasound	227	N/S	N/S

SD: Standard Deviation, HER2: Human Epidermal Growth Factor Receptor 2, HR: Hormone Receptor, ER: Estrogen receptor, PR: Progesterone Receptor, DCE-MRI: Dynamic Contrast-Enhanced Magnetic Resonance Imaging.

Table 2 represents the characteristics of the predictive models employed in the seven included studies. Five studies included clinical data in the predictive model. The predominant predictive models utilized in the included studies were multiple logistic regression models, often augmented with additional steps such as feature selection, dimension reduction, normalization, and classification. Furthermore, two of the studies incorporated deep learning methods in developing their predictive models.

Table 2.

Characteristics of the predictive models employed in the seven included studies.

Author, Year	Imaging modality	Sequence	Slice thickness (mm)	MRI field intensity (T)	Gold standard test	Predictive model type	AUC (%)	Sensitivity (%)	Specificity (%)	Are clinical data included in the predictive model?
Chen et al., 2022 [16]	Multiparametric MR	T2 - DWI - DCE T1 - Fat suppressed T1	1	1/5	LND	Multiple regression model based on MRI morphological parameters, signal intensity curve, ADC, and clincial characteristics.	Training: 69	Training: 69	Training: 59	Yes
Liu et al., 2023 [17]	DCE MRI	DCE T1	5	3	LND	Multiple regression model of pre and post NAC radiomics and delta-radiomics and clincial characteristics with feature selection using LASSO.	Training: 73.4 Validation: 68.3	Training: 70.2 Validation: 95	Training: 70.3 Validation: 50	Yes
Gan et al., 2021 [18]	DCE MRI	DCE T1	2/4	3	LND	Several regression models with several types of normalization, dimension reduction, feature selection, and classification, along with clinical data. The best model included 6 parameters derived using principal component analysis (PCA).	Training: 78.6 Validation: 67	-	-	Yes
Ha et al., 2018 [19]	DCE MRI	DCE T1	N/A	N/A	LND	CNN and deep learning model based on radiomics before NAC.	-	-	-	No
Gu et al., 2022 [20]	Ultrasound	US	N/A	N/A	LND	Deep learning radiomics nomogram model of US radiomics and clinical characteristics.	Training: 91.6 Validation: 85.3 Test: 86.3	Training: 84 Validation: 79.6 Test: 74.5	Training: 86.6 Validation: 82 Test: 81.8	Yes
Li et al., 2023 [22]	CECT before and after NAC	CECT	5	N/A	LND	ML-based pairwise auto-encoder model of CT radiomics before and after NAC and their alterations with data normalization, dimensionality reduction, and features screening	Training: 94.4 Test: 1	Validation: 98.5	-	No
Kim et al., 2019 [21]	Ultrasound	US	N/A	N/A	LND	Multivariate logistic regression of ultrasound features and clinical characteristics	-	-	-	Yes

DCE-MRI: Dynamic Contrast-Enhanced Magnetic Resonance Imaging, DWI: Diffusion Weighted imaging, ADC: Apparent Diffusion Coefficient, CECT: Contrast-Enhanced Computed Tomography US: Ultrasound, LND: Lymph Node Dissection, N/A: Not Applicable.

3.3. Quality assessment

We assessed the methodological quality of the included studies through QUADAS-2. Fig. 2 shows the results of quality assessment.

Fig. 2.

Methodological quality analysis of the included studies through QUADAS-2. Green indicates low, yellow unclear, and red high risk of bias.

3.4. MRI studies

Chen et al. [16] assessed the LN- pCR in patients with clinically node-positive breast cancer following NAC by incorporating multiparametric MRI and various clinicopathologic factors. In their retrospective study, they reviewed 158 patients who had NAC followed by surgery and correlated 224 axillary LNs identified on MRI with their pathological outcomes. Key MRI features post-NST, such as cortical thickness and fatty hilum, as well as time-intensity curve (TIC) patterns, in combination with hormone receptor and HER2 status, were identified as significant indicators of LN-pCR. The researchers developed and tested a multiple regression predictive model with these factors, which showed strong predictive ability with an AUC of 0.85, enabling the effective and non-invasive prediction of LN-pCR pre-surgery.

Liu et al. [17] studied 120 women diagnosed with axillary LN-positive breast cancer who underwent NAC and subsequent surgery were evaluated to establish a delta-radiomic model. The model aimed to predict the LN-pCR using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), using multiple regression models that incorporated both pre- and post-NAC radiomic features, as well as delta-radiomic changes. The findings revealed that the delta-radiomic model outperformed other models, achieving an AUC of 0.851 in the training set and 0.822 in the validation set. The predictive accuracy was further enhanced when the delta-radiomic model was combined with clinical features, such as ER and HER2 status, leading to AUCs of 0.932 and 0.859 in the training and validation cohorts, respectively. The study underscored the potential of delta-radiomic features of the axillary LN as non-invasive biomarkers for early prediction of LN-pCR.

Gan et al. [18] created a clinical-radiomics model, employing a variety of regression models that integrated several types of normalization, dimension reduction, feature selection, and classification as well as clinical data, to predict the LN-pCR in patients with breast cancer with axillary LN metastases. Their study included 248 patients with invasive breast cancer who underwent NAC followed by axillary LN dissection. The best-performing model in their study was refined using principal component analysis (PCA), a statistical procedure that transforms a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. This process of dimensionality reduction helped in identifying six key parameters that, when combined with the clinical data, provided the most reliable prediction of apCR. Their study revealed that the clinical-radiomics model outperformed both the clinical and radiomics models with AUC values of 0.924, 0.851, and 0.878 across the training, validation, and testing cohorts, respectively, in predicting LN-pCR.

Ha et al. [19] developed a CNN algorithm to predict LN-pCR after NAC in breast cancer patients. The study, conducted between January 2009 and June 2016, included 127 breast cancer patients who underwent breast MRI prior to NAC and had surgical pathology data available following surgery. The CNN model used in this study, composed of 10 convolutional and 4 max-pooling layers, and techniques such as dropout, augmentation, and L2 regularization were employed to prevent the overfitting of the data. Also, the model underwent independent training utilizing k-fold cross-validation. For every individual breast tumor, the highest SoftMax score determined by the model was utilized to forecast the pathologic response of the axilla. Then, the trained model was utilized to predict classes on the earlier withheld testing dataset, which contains 20 % of the whole dataset. Using 2811 volumetric slices from 127 tumors extracted from pre-NAC MRI scans, the CNN model demonstrated an 83 % accuracy rate in predicting LN-pCR, with a sensitivity of 93 %, a specificity of 77 %, and an AUC of 0.93. The study concluded that utilizing a CNN and deep learning model based on radiomics prior to NAC is a feasible method to predict post-NAC LN-pCR.

3.5. CT and ultrasound studies

Gu et al. [20] conducted a study on 628 breast cancer patients from January 2017 to July 2021. They selected 484 female patients with breast tumors for further analysis and evaluated various clinical characteristics, including estrogen receptor (ER), progesterone receptor (PR), Ki67, and HER2 statuses. Based on clinical and ultrasound imaging data, the study aimed to develop two deep learning radiomics (DLR) models aimed at individually predicting tumor pCR to NAC (DLR-pCR) and the status of LN metastasis (LNM) after NAC (DLR-LNM). These models were developed using pre-NAC and post-NAC ultrasonography images. Additionally, they introduced two Deep Learning Radiomics Networks (DLRNs), DLRN-pCR and DLRN-LNM, designed for distinct tasks based on clinical characteristics and DLR scores generated from both DLR-pCR and DLR-LNM. Their DLRN-PCR model achieved high predictive performance with AUCs of 0.970, 0.903, and 0.896 in training, validation, and test cohorts, respectively. The DLRN-LNM model also showed promising results with AUCs of 0.896, 0.842, and 0.845 in the same cohorts. The authors emphasized the importance of independent models for tumor and axillary lymph node evaluation after NAC to avoid unnecessary treatments.

Kim et al. [21] focused on predicting axillary pCR after NAC in breast cancer patients with axillary lymph node metastasis. Their study included 227 patients, of which 106 achieved pCR and 121 had non-pCR. They utilized ultrasound and CT imaging to evaluate various characteristics of axillary lymph nodes. The study identified histologic grade, hormonal receptor status, residual tumor size, fatty hilum loss, and eccentric cortical thickening as independent predictors of axillary pCR. They developed a combined model that outperformed the clinicopathologic and imaging models by integrating clinical and imaging characteristics. The combined model achieved higher sensitivity (67.9 %) and specificity (73.6 %) in predicting axillary pCR, offering the potential for better axillary management strategies.

Li et al. [22] explored the application of radiomics-based Contrast-enhanced CT in evaluating axillary lymph node response to NAC. They analyzed 138 breast cancer patients with axillary LN metastasis, of which 57 achieved LN-pCR and 81 had non-LN-pCR. A program called pyradiomics, an open-source tool, was used to retrieve radiomics features, including first-order features, shape-based features, and texture features. Then, some CT parameters were selected for comparison. These parameters included LN long-axis diameter, LN short-axis diameter, LN L/S ratio, tumor long-axis diameter, tumor short-axis diameter, area, CT enhancement, and cortical thickness. By comparing CT parameters before and after NAC, they found significant variations between LN-pCR and non-LN-pCR patients. They then used a pairwise autoencoder to develop a radiomics model, demonstrating substantial diagnostic effectiveness with AUCs of 0.981, 0.971, and 1.000 in the training, validation, and test cohorts. This research provided a novel approach for evaluating the response of metastatic lymph nodes using radiomics-based CT analysis.

4. Discussion

In patients with breast cancer presenting initially with node-positive disease, assessing the status of axillary LNs following NAC is critical for tailoring axillary treatments, potentially obviating the need for ALND if negative pathology is confirmed. To predict the status of axillary LNs post-NAC, various predictive models, such as multiple regression models and CNN-based deep learning models, have been developed. In this study, we systematically reviewed the evidence from multiple studies that employed a variety of predictive models based on radiomics and machine learning techniques, utilizing MRI, CT, and ultrasound to predict the LN status post-NAC. These studies highlighted the efficacy of machine learning approaches that used radiomics data from different imaging modalities and revealed that integrating radiological imaging characteristics with clinical patient data enhances the predictive accuracy regarding the status of LN after NAC.

4.1. The application of radiomics in breast cancer

The application of radiomics in the diagnosis or prognosis of breast cancer has been a novel topic in recent years. And a systematic review of individual studies can be advantageous to use the results in clinical practice. A study done by Gong et al. [23] performed a meta-analysis to assess the pooled diagnostic value of different radiomics modalities to predict axillary and sentinel lymph node metastasis in patients with breast cancer. Based on the subgroup analysis in the mentioned study, they reported that ultrasound had the highest diagnostic performance. However, the number of studies that used ultrasound was not big enough, and most of them combined radiomic features with clinical features or deep learning algorithms.

Also, the application of machine learning and deep learning to diagnose is emerging. Some studies reported that using machine learning and deep learning models has improved the diagnostic accuracy of medical imaging, and not only they can be used in the diagnosis of breast cancer but also in the prognosis and prediction of the tumor progression [24], [25]. For example, Zheng and her colleagues [26] reported in their study that deep learning models can be applied for the prediction of axillary lymph node status in early-stage breast cancer.

4.2. The comparison of different imaging modalities

Among the included studies predicting the pCR after neoadjuvant chemotherapy in our systematic review, three studies [27], [28], [29] used MRI, two studies [20], [21] used ultrasonography, and one study [22] used CT. Among studies that reported the diagnostic accuracy of MRI for the prediction of pCR after neoadjuvant chemotherapy, the study done by Ha et al. had the highest AUC. Ha et al. [29] developed a convolutional neural network (CNN) algorithm to predict pCR in their studies. However, other studies used radiomics features alone and in addition to clinical features to predict pCR. The AUC of both radiomic and radiomics-clinical features in these studies was lower than the study that used a deep learning model.

Also, our result showed that there is a big difference in the diagnostic performance of studies using ultrasound. This difference can be due to the models that were used in these two studies. The study done by Gu et al. [20] applied a deep learning model. However, Kim et al. [21] used a model that combined only imaging features or imaging and clinicopathologic features to predict pCR. So, the higher performance of the study done by Gu et al. [20] can be the higher capability of deep learning models to be used for the prediction of pCR.

Moreover, a study by Li et al. [22] used CT radiomics to predict the pCR of axillary lymph nodes after neoadjuvant chemotherapy in breast cancer patients with axillary lymph node metastasis. This study reported that the AUC of single features ranged from 0.598 to 0.711. Then, they combined the features, and the results showed that the AUC was 0.962 and 1.000 for the validation set and test set, respectively.

Then, the comparison of different modalities revealed that a study that used ultrasound without deep learning algorithms reported the lowest diagnostic performance. And the highest diagnostic performance was reached by a study using CT radiomics. However, only one study used CT radiomics to predict pCR. In addition, the application of deep learning algorithms in MRI and ultrasound studies has increased the accuracy of these modalities compared to the application of these modalities without deep learning models.

4.3. Limitations and suggestions

This systematic review is subject to certain limitations. Notably, the studies under review employed a diverse array of imaging techniques and model construction methods, which introduces a significant degree of heterogeneity. For example, some included studies used MRI, some used CT-scan, and others used ultrasonography. Moreover, some included studies used machine learning models. However, some other studies used deep learning models. Also, due to a lack of appropriate study the subgroup analysis based on the imaging techniques and machine learning or deep learning approaches was impossible. This diversity may disrupt the generalizability and comparability of the methodologies across different settings. In addition, the lack of an external validation cohort in the majority of the included studies raises concerns regarding the reproducibility of the results.

So, it is suggested to conduct more studies on the application of machine learning and deep learning models in different modalities to predict pCR after neoadjuvant chemotherapy in breast cancer patients with axillary lymph node metastasis. Also, more studies can be performed to find useful features of CT-scan and its overall diagnostic performance to have a better insight into the performance of this modality compared to others.

5. Conclusion

In conclusion, according to our results, the application of radiomics can be beneficial in predicting pCR after neoadjuvant chemotherapy in breast cancer patients with axillary lymph node metastasis. Also, applying deep learning models can improve the diagnostic performance of each modality. Moreover, MRI and CT's overall performance was higher than ultrasound's. So, the result of this study can be applied in clinical practice to decide whether or not neoadjuvant chemotherapy can be helpful in a breast cancer patient.

Funding statement

Authors received non funding for this research.

Ethical statement

None.

CRediT authorship contribution statement

Shirin Yaghoobpoor: Writing – review & editing, Supervision, Conceptualization. Payam Jannatdoust: Writing – original draft, Methodology. Parya Valizadeh: Writing – original draft. Hamed Ghorani: Supervision. Mobina Fathi: Writing – review & editing, Writing – original draft, Investigation. Arvin Arian: Supervision, Project administration, Conceptualization. Melika Zarei: Writing – original draft, Methodology. Arian Tavasol: Writing – original draft.

Declaration of generative AI and AI-assisted technologies in the writing process

During the preparation of this work the author(s) used OpenAI for paraphrasing and improving language and readability. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.

Declaration of Competing Interest

Authors declare no conflict of interests.