Classification of asymmetry in mammography via the DenseNet convolutional neural network

Abstract

Purpose

To investigate the effectiveness of a deep learning system based on the DenseNet convolutional neural network in diagnosing benign and malignant asymmetric lesions in mammography.

Methods

Clinical and image data from 460 women aged 23–82 years (47.57 ± 8.73 years) with asymmetric lesions who underwent mammography at Shenzhen People's Hospital, Shenzhen Luohu District People's Hospital, and Shenzhen Hospital of Peking University from December 2019 to December 2020 were retrospectively analyzed. Two senior radiologists, two junior radiologists, and the DL system read the mammographic images of 460 patients, respectively, and finally recorded the BI-RADS classification of asymmetric lesions. We then used the area under the curve (AUC) of the receiver operating characteristic (ROC) to evaluate the diagnostic efficacy and the difference between AUCs by the Delong method.

Results

Specificity (0.909 vs. 0.835, 0.790, ${\chi }^2$=8.21 and 17.22, p＜0.05) and precision (0.872 vs. 0.763, 0.726, ${\chi }^2$=9.23 and 5.22, p＜0.05) of the DL system in the diagnosis of benign and malignant asymmetric lesions were higher than those of junior radiologist A and B, and there was a statistically significant difference between AUCs (0.778 vs. 0.579, 0.564, Z = 4.033 and 4.460, p＜0.05). Furthermore, the AUC (0.778 vs. 0.904, 0.862, Z = 3.191, and 2.167, p＜0.05) of benign and malignant asymmetric lesions diagnosed by the DL system was lower than that of senior radiologist A and senior radiologist B.

Conclusions

The DL system based on the DenseNet convolution neural network has high diagnostic efficiency, which can help junior radiologists evaluate benign and malignant asymmetric lesions more accurately. It can also improve diagnostic accuracy and reduce missed diagnoses caused by inexperienced junior radiologists.

1. Introduction

The current expansion in deep learning medical image examination, particularly the convolutional neural network (CNN), can be used to ensure the output of Computer Aided Diagnosis (CAD) platforms. Since breast cancer is the foremost cause of female cancer deaths, annual breast cancer screening is essential for early disease diagnosis and for reducing death rates [1]. According to data released by the National Cancer Center in 2022, the incidence rate of breast cancer among Chinese females is increasing; it ranks first among Chinese women with a significant increase in mortality [2].

Due to its availability, efficiency, and efficacy, mammography has become one of the most favorable diagnostic modalities [3]. Over the last few decades, mammography has shown its potential significance in automatic breast lesion detection, segmentation, and recognition applications. Previously, automated medical image analysis assisted radiologists in identifying masses in mammography images and reduced the number of false negative outcomes in a machine-learning fashion. For example, Oliver et al. [4] reviewed the automated mass detection and segmentation algorithms, and He et al. [5] extensively reviewed automated mammographic tissue segmentation approaches. This study also showed the performance of risk assessment and density classification using mammography image segmentation. However, these conventional machine learning-based methods adopt handcrafted features, which consist of a combination of mathematical or heuristic descriptors. These extracted features are fed into various classifiers to be categorized into the correct types. The performance of conventional machine learning methods is based on the quality of the features extracted from the dataset. Unfortunately, the handcrafted features usually make the machine learning-based applications cumbersome and lack robustness.

Deep learning (DL) models, the most significant breakthrough technologies in the last 20 years, have improved outcomes in various areas, as they can implement adaptive learning from raw data. Many DL algorithms have been designed for supervised (labeled data set is available) and unsupervised applications (labeled dataset is unavailable). DL has become the de facto choice for supervised methods. Furthermore, the CNN has become one of the most common DL models for automated medical image analysis and processing [6]. Janowczyk and Madabhushi [7] compared automated feature extraction-based DL and hand-crafted feature-based machine learning methods for medical image classification. Jiao et al. [8] proposed a CNN model for breast mass recognition, in which deep features can be automatically generated. As a type of CNN, DenseNet combined the advantages of the ResNet and Highway models and was proposed in the works of [9], [10], [11] from 2017. Notably, DL can produce more distinguishing features from massive data without the handcrafted feature used in traditional machine learning methods [12].

In the fifth edition (2013) of the American College of Radiology (ACR) Breast Imaging Reporting and Data System (BI-RADS), asymmetries are widely accepted as one of the four image signs of mammography [13]. Asymmetry refers to the increased density of unilateral fibrous gland tissue, which cannot determine the three-dimensional space occupying characteristics and therefore is insufficient to interpret the imaging manifestation of a high-density mass. Asymmetric mass and calcification lack quantitative evaluation indicators, that differ from the mass with an apparent space-occupying effect. Objective and accurate evaluation is of great significance for early diagnosis and reducing overtreatment, and it is also a difficult point in current clinical work. In clinical work, asymmetric diagnosis is highly dependent on physician experience, leading to a high mammography recall rate and a high missed diagnosis of breast cancer. Meanwhile, the stacking of the breast glands will make it difficult for radiologists to find and diagnose asymmetric breast lesions, especially in dense breasts.

Additionally, the application of DL in mammography has significantly improved the efficiency and stability of reading images for radiologists while reducing the workload of radiologists [14]. Currently, Mammography research based on DL system primarily aims to detect breast masses, differentiate benign and malignant masses [15] and classify benign and malignant calcifications [16]. However, there are few studies of DL methods on asymmetry and architectual distortion in mammography images [17]. To bridge this gap, this study aims to explore the diagnostic ability of the DL model based on the DenseNet model in diagnosing benign and malignant breast asymmetry. Consequently, we compared the results of the diagnosis of two senior radiologists, two junior radiologists, and the DL system. Experimental results demonstrated that the proposed DL model could improve diagnostic efficacy in diagnosing breast asymmetry and provide an imaging basis for the correct diagnosis of breast asymmetry and clinical treatment decisions.

2. Materials and methods

2.1. Subjects

The mammographic data, clinical data, and pathological results of 460 diagnosed with asymmetry in the mammographic reports of Shenzhen People's Hospital, Peking University Shenzhen Hospital, and Shenzhen Luohu District People's Hospital from December 2019 to December 2020 were retrospectively analyzed. Among them, were 380 cases from the A Hospital, 40 from the B Hospital, and 40 from C Hospital. In total, 460 patients were enrolled in this study. The age ranged from 23–82 years, with an average age of 47.57 ± 8.73 years. Data from this group were reviewed by a radiologist specializing in breast diagnosis and confirmed that the breast had asymmetry. Inclusion criteria: (1) image quality, imaging conditions, and imaging position meet the diagnostic criteria; (2) Body positions taken include conventional images with complete cranio-caudal (CC) and medial-lateraloblique (MLO) positions on both sides; (3) It is necessary to meet one of the following conditions to evaluate the normal breast: a) No abnormality is found in the results of additional ultrasound or magnetic resonance examination; b) If there is no additional ultrasound or magnetic resonance examination, the suspected lesions will disappear during at least two years of follow-up. One of the following conditions must be met for the evaluation of benign lesions:

• •-Pathological confirmation of benign lesions.
• •-The lesion is stable after at least two years of follow-up.
• •-Further ultrasound or magnetic resonance examination confirmed an evident benign lesion.

Only lesions diagnosed as malignant by pathological results were evaluated as malignant. Asymmetric lesions are evaluated as normal breast tissue overlap, benign or malignant lesions based on follow-up or further examination results. Exclusion criteria: (1) patients after breast surgery; Patients after implantation of the prosthesis, breast augmentation by injection, and neoadjuvant chemotherapy; (2) It has malignant signs of calcification. This study was approved by the Ethics Committee of A Hospital (LL-KY-2021624).

2.2. Image collection

This data group comes from four mammography machines: Siemens Mamma Inspiration in Germany, GIOTTO IMAGE MD in Italy, Hologic Selenia Dimensinsa in the United States, and Senographe Pristine full-field digital mammography machine in the United States. Acquisition of mammography images is consistent with the technical standards formulated by the consensus of experts in breast imaging examination of the Chinese Medical Association in 2016 [18].

2.3. Image analysis

The categories of breast density of 460 patients in the PACS system of three hospitals (divided into four categories: a, b, c, d; category a and b breast were classified as non-dense breasts; category c and d breast were classified as dense breast) Whether there are clinical symptoms (patients with any clinical manifestation of the breast are classified as a symptomatic group, including palpable breast lumps, local thickening of the gland tissue, local pain, discharge from the nipple, trauma, breast redness, skin changes, nipple retraction and nipple eczema-like changes; those without the above symptoms are classified as an asymptomatic group); 460 patients were read by two general radiologists (with 5 and 6 years of work experience, namely, junior radiologist A and junior radiologist B) and two professional breast diagnostic radiologist (with 20 years of work experience and professional breast imaging training, namely, senior radiologist A and senior radiologist B), respectively.

BI-RADS of breast lesions was evaluated and recorded (according to the fifth edition of the BI-RADS classification standard, BI-RADS 1, 2, and 3 were classified as benign, and BI-RADS 4 (4 A, 4B, 4 C) and 5 were classified as malignant). In the fifth edition of the BI-RADS classification standard, it is recommended that BI-RADS category 4 lesions should undergo histological examination, and BI-RADS categories 1, 2, and 3 lesions should be followed up regularly [19]. Therefore, this study, defined a BI-RADS category 3 breast lesion as negative and a BI-RADS category 4 breast lesion as positive.

2.4. DL diagnosis system

The DL system used in this study is the Mammo AI discrimination system for mammography jointly developed by our hospital and the PingAn Technology Company. The DL non-calcification lesions detection model [20] in the system includes three modules: ipsilateral dual-view network (IDVN), bilateral dual-view network (BDVN), and integrated fusion network (IFN), which can receive mamographic images of the same patient in multiple projection directions. Two different high-resolution detection and segmentation DL networks are designed for the ipsilateral and contralateral images. Through the nipple detection algorithm, combined with the target detection algorithm, the lesions are jointly detected, and the asymmetric lesions detected are studied together by DenseNet-121. The model is a deep multi-task learning model.

During training, many noncalcified images, the corresponding benign and malignant labels, and their BI-RADS levels are simultaneously fed into DenseNet-121 with a depth of 121 layers to consider the relationship between benign and malignant labels and BI-RADS levels. The lesion’s characteristic is extracted, and the correlation between the BI-RADS level and the benign and malignant labels can be considered. For the classification of benign and malignant tumors, the model will output a score of 0–100 (0 is benign, 100 is malignant). Combined with a certain threshold, it can determine whether the suspicious tumor is benign or malignant. For the BI-RADS classification, the model will output a level from BI-RADS 2–5. The final result is the asymmetric lesions' benign and malignant output (as shown in Fig. 1).

Fig. 1.

Diagram of the DenseNet-based AI platform for the detection and recognition of breast lesions.

2.5. Methods

According to the results, the age, breast density classification, clinical symptoms, further examination, and follow-up results of 460 patients were recorded. The follow-up pathological results were malignant and recorded as malignant. Other cases were recorded as non-lesions or benign by further examination and follow-up for two years. Two senior radiologists, two junior radiologists, and the DL system read the images, recorded the location of the lesions, and classified the asymmetric lesions according to BI-RADS. Then, the DL system independently read the images and evaluated the benign and malignant lesions on the mammographic images of 460 patients. Finally, the pathological results were taken as the reference standard. Patients without pathological results were followed for at least two years without changes in the lesions, or benign lesions were further identified by ultrasound or magnetic resonance.

2.6. Statistical analysis

SPSS 26.0 statistical software was used for statistical analysis. We used the Kappa test to evaluate the consistency of benign and malignant asymmetry between two senior and two junior radiologists. The sensitivity, specificity, and accuracy of two senior radiologists, two junior radiologists, and the DL system in the diagnosis of benign and malignant asymmetry were calculated, respectively. The receiver operator characteristic (ROC) curve and the area under the curve (AUC) were used to evaluate the diagnostic efficacy of two senior radiologists, two junior radiologists, and the DL system for benign and malignant asymmetry. The Delong method was used to test whether the difference between the ROC curves was statistically significant. p< 0.05 is statistically significant.

3. Results

In this study, 460 cases of mammography examination were included, aged 23–82 years, with an average age of 47.57 ± 8.73 years. There were 293 cases (63.7 %) with pathology, of which 68 (23.2 %) were malignant. 167 patients (43.3 %) were without pathology, of which 21 (12.6 %) were followed-up by four kinds of imaging examination, including ultrasound, mammography, digital breast tomography (DBT), and breast magnetic resonance imaging; 41 (24.6 %) with three types of imaging examination, 67 (40.1 %) by two kinds of imaging examination, 118 (70.7 %) by ultrasound, 74 (44.3 %) by mammography, 47 (10.2 %) by DBT, and 49 (29.3 %) by breast MRI. Finally, 126 cases (75.4 %) were benign, and 41 (24.6 %) were malignant. Of the 460 patients, 316 (68.7 %) had dense breasts, and 144 (31.3 %) had non-dense breasts. The asymptomatic group represented about 210 cases (45.65 %). 250 patients (54.35 %) had clinical symptoms.

3.1. Comparison of diagnostic efficacy between physicians and the DL system for benign and malignant asymmetric signs

The Kappa value between the two senior radiologists was 0.677, which was statistically significant (p< 0.05). The Kappa value between the two junior radiologists was 0.610, which was statistically significant (Specificity (0.909 vs. 0.835, 0.790, χ² =8.207 and 17.216, p＜0.05) and accuracy (0.872 vs.0.763, 0.726, χ² =9.227 and 5.218, p＜0.05) of the DL system in the diagnosis of benign and malignant asymmetry were higher than those of junior radiologist A and junior radiologist B, and there was a statistically significant difference between AUCs (0.778 vs.0.579, 0.564, Z = 4.460, p＜0.05). The AUC (0.778 vs. 0.040, 0.862, Z = 3.191 and 2.167, p＜0.05) of benign and malignant lesions diagnosed by the DL system was lower than that of senior radiologist A and senior radiologist B (as shown in Table 1 and Table 2). The ROC curves of the subjects of senior radiologist A, senior radiologist B, DL system, junior radiologist A, and junior radiologist B are shown in Fig. 2.

Table 1.

Delong test results between 4 radiolgists and the DL system.

		Senior A vs DL	Senior B vs DL	Junior A vs DL	Junior B vs DL
Total	Z	2.931	1.984	3.108	3.572
	p	0.003	0.047	0.002	0.000
Non-dense	Z	1.412	1.188	2.384	2.543
	p	0.158	0.235	0.017	0.011
Dense	Z	2.563	1.632	2.095	2.517
	p	0.010	0.103	0.036	0.012
Asymptomatic	Z	2.521	3.301	5.144	5.248
	p	0.012	0.001	0.000	0.000
Symptomatic	Z	4.949	4.151	0.758	1.340
	p	0.000	0.000	0.448	0.180

Table 2.

Comparison of the diagnostic efficacy of the DL system, two senior and two junior radiolgists in the diagnosis of benign and malignant asymmetry.

	Sensitivity	Specificity	Accuracy	AUC	95 % CI
					Lower limit	Upper limit
Senior A	0.862	0.947	0.935	0.904	0.860	0.948
Senior B	0.831	0.894	0.885	0.862	0.814	0.911
Junior A	0.323	0.835	0.763	0.579	0.519	0.639
Junior B	0.338	0.790	0.726	0.564	0.503	0.626
DL system	0.646	0.909	0.872	0.778	0.717	0.838

Fig. 2.

ROCs for the two senior radiologists, two junior radiologists, and the DL system.

3.2. Comparison of the diagnostic efficacy of different breast density classifications

In the non-dense breast, the specificity (0.928 vs. 0.840, 0.768, χ² = 22.809 and 7.823, p＜0.05) of the DL system in the diagnosis of benign and malignant asymmetry was higher than those of the junior radiologist A and the junior radiologist B, and there was a statistical difference between the AUCs (0.859 vs. 0.525, 0.516, Z = 3.729 and 3.786, p＜0.05). In dense breasts, the AUC (0.743 vs 0.601, 0.585, Z = 2.462 and 2.857, p＜0.05) of the DL system in diagnosing benign and malignant asymmetry was higher than that of junior radiologist A and junior radiologist B. In non-dense breasts, there was no significant difference between the AUC of asymmetry diagnosed by the DL system and that diagnosed by senior radiologist A and senior radiologist B (p > 0.05) (as shown in Table 1, Table 3).

Table 3.

The influence of breast density on the diagnosis of benign and malignant asymmetric lesions by radiologists and the DL system.

		Sensitivity	Specificity	Accuracy	AUC	95 % CI
						Lower limit	Upper limit
Non-dense	Senior A	0.842	0.936	0.924	0.889	0.802	0.976
	Senior B	0.842	0.880	0.875	0.861	0.772	0.950
	Junior A	0.211	0.840	0.757	0.525	0.426	0.625
	Junior B	0.263	0.768	0.701	0.516	0.407	0.624
	DL system	0.789	0.928	0.910	0.859	0.762	0.956
Dense	Senior A	0.870	0.952	0.940	0.911	0.860	0.962
	Senior B	0.826	0.900	0.889	0.863	0.805	0.921
	Junior A	0.370	0.833	0.766	0.601	0.528	0.675
	Junior B	0.370	0.800	0.737	0.585	0.510	0.659
	DL system	0.587	0.900	0.854	0.743	0.669	0.818

3.3. The effect of no clinical symptoms on four radiologists and the DL system to diagnose benign and malignant of asymmetry

There was a statistically significant difference between the AUC of asymmetric lesions diagnosed by the DL system and those diagnosed by junior radiologists A and B (1.000 vs. 0.551, 0.541, Z = 5.427 and 5.539, p < 0.05). Among symptomatic patients, the AUC of asymmetric lesions diagnosed by the DL system differed significantly from the AUC of asymmetric lesions diagnosed by junior radiologists A and B (0.705 vs. 0.576, 0.548, Z = 2.331, 2.898, p < 0.05) (as shown in Table 1, Table 4).

Table 4.

The influence of clinical symptoms on the diagnosis of benign and malignant asymmetric lesions by four radiologists and the DL system.

		Sensitivity	Specificity	Accuracy	AUC	95 % CI
						Lower limit	Upper limit
Asymptomatic	Senior A	0.875	0.970	0.967	0.923	0.800	0.984
	Senior B	0.750	0.931	0.924	0.840	0.679	0.969
	Junior A	0.250	0.149	0.829	0.551	0.388	0.713
	Junior B	0.250	0.832	0.810	0.541	0.378	0.703
	DL	1.000	1.000	1.000	1.000	1.000	1.000
Symptomatic	Senior A	0.860	0.922	0.908	0.891	0.842	0.940
	Senior B	0.750	0.931	0.924	0.849	0.795	0.902
	Junior A	0.333	0.819	0.708	0.576	0.509	0.643
	Junior B	0.351	0.746	0.656	0.548	0.479	0.618
	DL	0.486	0.872	0.764	0.705	0.635	0.775

4. Discussion

Compared to the fourth edition of ACR BI-RADS in 2003, the fifth edition of ACR BI-RADS in 2013 has significantly changed the description of asymmetric lesions relative to masses, calcifications, and architectual distortion. The fifth edition of ACR BI-RADS subdivides the asymmetry in the first two editions into four types: structural asymmetry, overall asymmetry, focal asymmetry, and progressive asymmetry. In the current images, progressive asymmetry is characterized by overall asymmetry or focal asymmetry, defined as progressive asymmetry only when compared to the previous images. Progressive asymmetry is often mistaken for overlapping normal glands of the breast. Most missed diagnoses of breast cancer are due to asymmetric missed diagnoses. This part of progressive asymmetry causes a high missed diagnosis rate and a recall rate of breast cancer. The overall malignant asymmetry rate in the screening and diagnosis population is 15.4 %. [21] Although ACR BI-RADS in 2013 developed various terms, the final diagnostic classification is greatly affected by the clinical experience of radiologists. It is primarily subjective, especially in diagnosing asymmetric lesions, which often lacks scientific precision.

The mammography image does not quantify the asymmetry. However, in the DBT part supplemented by ACR BI-RADS in the fifth edition, the original image asymmetry is modified into two categories: actual disease and gland overlap [13]. Some mammograms showed asymmetric lesions confirmed as masses by DBT examination. Masses in dense glands also often showed asymmetry in mammograms and were finally confirmed as masses by surgery and pathology. According to the literature [22], lesions with asymmetric or asymmetric with calcification lesions on mammography were found in 88.9 % of patients after DBT examination. Therefore, the asymmetric model part of this study follows the mass algorithm of the DL system in the previous study and is improved based on the mass detection model to form an asymmetric model. In the previous study, our research team used the DL system to detect breast masses. The results show the DL system's sensitivity to breast mass detection is 86.25 %. The sensitivity of the two junior radiologists to the detection of breast masses is 83.98 % and 77.99 %, indicating that the mass detection model built by the DL system has a high sensitivity to the detection of breast masses and has good diagnostic performance. Wu et al. [23] used logistic regression to analyze clinical data and the characteristics of mammographic images to analyze and screen predictive factors. They built a predictive model to predict the malignant probability of asymmetric lesions of BI-RADS 4 or 5 on mammography. The predictive model has good diagnostic efficacy with an AUC value of 0.85 in the training group and 0.84 in the validation group. In this study, the AUC value of 460 cases of benign and malignant diagnosis of asymmetric lesions using the asymmetric focus detection and benign and malignant diagnosis model built by the DL system was 0.778, higher than that of two junior radiologist (0.579, 0.564). It shows that the DL system is also effective in diagnosing benign and malignant asymmetric lesions.

Lauritzen et al. [24]used the Transpara artificial intelligence system to screen women for breast cancer. The results showed that the sensitivity of AI-based screening (69.7 %) was close to that of radiologists (70.8 %), and the specificity (98.6 %) was higher than that of radiologists (98.1 %). The workload of radiologists was reduced by 62.6 %. This study is similar to the results of this study. The AUC (1.000) of the DL system in asymptomatic screening patients is higher than that of symptomatic patients (0.705). In asymptomatic screening patients, the radiologist and the DL system only consider the characteristics of the image and do not consider any other factors influencing them. At the same time, the extraction and analysis ability of the DL system in the original image is better than that of the human. Therefore, using the DL system in asymptomatic screening patients can reduce the missed diagnosis rate of malignant asymmetry and recall rates.

The AUC (0.743, 0.859) of the DL system for diagnosing benign and malignant asymmetric lesions in dense and non-dense breasts is higher than the two junior radiologists. Therefore, our DL system is also suitable for Chinese women with dense breasts and can be similar to DBT in reducing the impact of the typical overlap of the breast gland, and the diagnostic efficiency is stable. It can help inexperienced junior radiologists to reduce missed diagnoses caused by dense breast gland and tissue occlusion.

4.1. Limitations of this study

First, this study is retrospective, and the efficacy of asymmetric benign and malignant diagnosis requires validation through prospective studies. Second, the number of asymmetric cases is small, so it is necessary to continue to increase the sample size to improve the model’s accuracy, effectiveness, and stability. Third, there should be more data on the influence of radiologists on the diagnostic efficiency of asymmetric benign and malignant masses assisted by the DL system. The follow-up research should further count on the diagnostic results of radiologists combined with the DL system to clarify the diagnostic value of the DL system assisted by radiologists (Table 3, Table 4).

Fig. 3.

Case 1, Female, 46-year-old, uneven dense breast (ACR Type C), which was evaluated as BI-RADS 4A after the DL detected the asymmetry in the upper quadrant of the right breast (blue box); During the operation, there was a mass near the sternum at 2 points in the upper quadrant of the right breast with a size of 3.5 cm × 5.6 cm, clear boundary, hard texture, and no coating. Pathological results: invasive non-special type carcinoma.

Fig. 4.

Case 2, female, 60 years old, uneven and dense (ACR Type B). DL detected the asymmetry of the right upper breast (blue box), which was evaluated as BI-RADS 4B; DBT examination confirmed that the right upper breast had a small mass with equal density (yellow box); Pathological results: invasive non-special type carcinoma.

5. Conclusions

The proposed DL system can help junior radiologists evaluate benign and malignant asymmetric lesions with more precision than themselves. Furthermore, it could improve diagnostic efficiency, effectively reducing missed diagnoses caused by radiologists’ lack of experience. Meanwhile, the DL system is less affected by breast density with substantial stability and has clinical auxiliary value in asymptomatic breast screening while reducing the workload of junior radiologists.

Funding statement

This work was supported by the Shenzhen Science and Technology Research and Development Capital, China (GJHZ20210705142208024); National High Performance Medical Device Innovation Center Open Fund, China (NMED2021MS-01-001).

Declaration of Competing Interest

We declare that we have no financial and personal relationships with otherpeople or organizations that can inappropriately influence our work, there is noprofessional or other personal interest of any nature or kind in any product, serviceand/or company that could be construed as influencing the position presented in.or the review of the manuscript entitled.