A Hybrid CNN-Transformer Deep Learning Model for Differentiating Benign and Malignant Breast Tumors Using Multi-View Ultrasound Images

Abstract

Introduction

Conventional single-image analysis often fails to capture the comprehensive features of breast lesions, limiting the diagnostic accuracy of ultrasound. This study introduces a computer-aided diagnosis (CAD) system that emulates the clinical multi-view diagnostic process to improve reliability.

Methods

A novel hybrid deep learning model was proposed for breast cancer diagnosis task using multi-view ultrasound images, which integrates a Convolutional Neural Network (CNN) module with a Transformer-based module. The pre-trained EfficientNetV2 was used to extract spatial features, and the Transformer-based encoder was applied to fuse information from multiple, unordered ultrasound images of a single lesion. This retrospective study utilized a primary internal dataset from 840 patients and an independent external dataset from 133 patients. A strict lesion-level data partitioning strategy was implemented to prevent data leakage. The performance of the proposed model was evaluated against a baseline single-image model, advanced architectures, and human experts, followed by validation on a prospective dataset of 188 lesions.

Results

On the internal test set, the hybrid CNN-Transformer achieved an accuracy with 0.960, a sensitivity with 0.967, and an Area Under the Curve (AUC) with 0.9788. On the external dataset, it demonstrated robust generalization with 0.940 accuracy and 0.9730 AUC. Prospective validation confirmed these findings with an accuracy of 0.952 and an AUC of 0.9801. The proposed model outperforms the single-image baseline with 0.881 internal accuracy and 0.849 accuracy of a senior ultrasound physician.

Conclusion

The system combines CNNs with Transformers to analyze multi-view ultrasound images, achieving high diagnostic accuracy and robustness. By fusing multi-view information, this model aligns with clinical logic and outperforms both single-image methods and human experts.

1. Introduction

Breast cancer stands as the most frequently diagnosed malignancy among women worldwide, posing a significant threat to their lives and health. ¹ Clinical evidence indicates that early detection, diagnosis, and treatment are paramount for improving patient prognosis and achieving higher five-year survival rates. ² Among the various imaging modalities, breast ultrasound has emerged as a pivotal first-line tool for screening, diagnosis, and follow-up of breast diseases, owing to its unique advantages of being non-invasive, radiation-free, highly repeatable, real-time, and cost-effective. ³

However, the traditional ultrasound diagnostic process has notable limitations. The accuracy of a diagnosis is heavily reliant on the sonographer’s expertise, technical skill, and subjective judgment, which can lead to inter-observer variability and diagnostic discrepancies. ⁴ In clinical practice, an experienced physician does not rely on a single, static image to assess a suspicious lesion. Instead, they perform a scan from multiple angles and planes to comprehensively observe multi-dimensional features such as the lesion’s margins, shape, internal echo pattern, and posterior acoustic characteristics. ⁵ This process allows them to build a holistic, three-dimensional impression of the lesion before arriving at a final diagnosis. This integrative analysis is central to diagnostic accuracy, and a key challenge in intelligent medical imaging is to develop artificial intelligence (AI) that can effectively simulate and enhance this complex cognitive process.

In recent years, AI technologies, particularly deep learning (DL), have made transformative breakthroughs in medical image analysis.^6,7 CNNs with their powerful ability to automatically learn hierarchical feature representations, have become the gold standard for processing grid-like data such as images.^8-10 In the context of breast cancer diagnosis, various classic CNN architectures like VGG and ResNet have been successfully applied to classification tasks using mammography, histopathology, and ultrasound images, demonstrating performance comparable to, or even superior to, that of human experts on specific datasets.^11-15 These models can autonomously learn subtle textural, morphological, and boundary features associated with malignancy, reducing the reliance on manually engineered features and paving the way for objective, efficient, and automated diagnosis.

Despite the remarkable success of CNN-based models, the majority of existing studies treat each 2D ultrasound image as an independent sample.^16,17 This methodology tends to diverge from standard clinical workflows, where diagnosis relies on the cumulative evidence derived from a sequence of images rather than isolated frames. Unlike single-image models that analyze specific views in isolation, clinical reasoning involves determining how collective findings contribute to a final assessment. Addressing this disparity—by evolving from static, single-view analysis to holistic, patient-level integration—represents a significant avenue for advancing intelligent medical imaging. This study is motivated by the need to bridge this gap, positing that the intelligent analysis of breast ultrasound should treat the complete set of multi-view images from a single lesion as a holistic, information-rich sequence.

To model image sequences, some researchers have explored hybrid architectures, such as combining a CNN with a Recurrent Neural Network (RNN) or its variant, Long Short-Term Memory (LSTM). ¹⁸ While valuable for data with a clear temporal or spatial order, these models are ill-suited for the unordered sets of images typical in a breast ultrasound examination. The arbitrary acquisition order of these images is less compatible with the inherent sequential dependency of RNN. To overcome this, we introduce the Transformer model, originally developed for natural language processing.^19,20 The Transformer’s core self-attention mechanism computes attention scores between all tokens simultaneously. While self-attention itself is position-agnostic, standard Transformers explicitly inject positional encodings to account for order without being constrained by sequential recurrence. In our specific clinical context, however, the multi-view scanning process of a breast lesion such as transverse and sagittal planes generally lacks a strict chronological or spatial sequence. Therefore, a key architectural adaptation in our proposed model is the deliberate omission of positional encodings. This design choice leverages the permutation-invariant nature of self-attention, allowing the architecture to model the multi-view medical images as an unordered set rather than a sequential timeline. From a technical perspective, this enables the Multi-Head Self-Attention mechanism to function as an adaptive global filter. It calculates similarity scores across all views simultaneously, dynamically assigning higher attention weights to features with informative pathological markers such as irregular margins or microcalcifications while reducing the influence of less informative or artifact-heavy views. By mitigating the noise often present in individual ultrasound frames, this approach facilitates a more comprehensive and holistic representation of the lesion, which contributes to the improved diagnostic accuracy. ²¹ Based on this analysis, this study proposes a new intelligent diagnostic paradigm that more closely mirrors clinical logic. The primary contributions are.

• We developed an end-to-end hybrid model combining EfficientNetV2 and a Transformer for breast ultrasound classification.

• We innovatively treat multiple images originating from a single tumor as a sequence, leveraging the Transformer’s global relationship modeling capabilities for comprehensive analysis.

• Rigorous, tumor-level experiments demonstrated the superior performance (an AUC of 0.9730 on an external dataset) of the model, which surpassed traditional single-image models.

• The solution offers a computationally effective approach to the core clinical challenge of multi-view information integration, and it indicates potential for future high-precision, reliable Computer-Aided Diagnosis (CAD) systems.

2. Related Works

Deep learning (DL) has significantly advanced the accuracy and efficiency of medical image analysis, rendering it a popular approach for breast cancer detection. A variety of architectures and methodologies have been proposed to address the complex challenges associated with diagnosing breast cancer across different imaging modalities. To appropriately contextualize our proposed approach, this section reviews recent landmark studies, comparing them in terms of dataset size, architecture design, evaluation protocols, and reported metrics.

Wang et al. ²² Developed an intelligent hybrid deep learning architecture combining CNN and Gated Recurrent Unit (GRU) layers to detect invasive ductal carcinoma. They utilized the PCam Kaggle dataset consisting of 277,524 whole slide image patches extracted from 162 patients. Employing an 80% training and 20% testing evaluation protocol, their CNN-GRU model achieved an accuracy of 86.21%, a sensitivity of 85.60%, and an AUC of 0.89.

Zakareya et al. ²³ Proposed a novel CNN-based architecture inspired by GoogLeNet and residual blocks for breast cancer classification. Their methodology integrated granular computing, shortcut connections, learnable activation functions, and an attention mechanism to capture detailed and fine-grained information. They evaluated their model on an ultrasound dataset of 780 images from 600 patients, using an 80% training and 20% testing split with data augmentation techniques. The model achieved a diagnostic accuracy of 93% on the ultrasound images.

Vigil et al. ²⁴ Introduced a dual-intended deep learning framework featuring a convolutional deep autoencoder for simultaneous tumor segmentation and radiomic feature extraction in ultrasound imaging. Their dataset also comprised 780 ultrasound images from 600 women. Using a Random Forest classifier evaluated via a leave-one-out cross-validation protocol, they reported a binary classification accuracy of 85.3% when combining conventional and deep radiomics.

Uysal and Köse ²⁵ investigated the multi-class classification of breast cancer ultrasound images using customized convolutional neural network models. They evaluated three specific deep learning architectures: VGG16, ResNet50, and ResNeXt50. The study utilized an open-source dataset comprising 780 ultrasound images categorized into benign, malignant, and normal classes. Employing a 70% training and 30% validation data split alongside data preprocessing and augmentation techniques, their modified ResNeXt50 model yielded the best performance, achieving an overall accuracy of 85.83%.

In summary, while prior studies demonstrate the potential of deep learning in breast image classification, most treat 2D images independently, diverging from clinical multi-view workflows. To address this, we introduce a hybrid CNN-Transformer architecture designed to integrate multi-view, unordered ultrasound images for a holistic diagnosis. Evaluated using strict lesion-level partitioning, the proposed model demonstrates robust generalization and competitive performance compared to conventional single-image methods (Supplemental Table 1).

3. Materials and Methods

This section provides a detailed description of the data sources, preprocessing pipeline, model architecture, experimental configuration, and performance evaluation standards to ensure the study’s reproducibility. The overall workflow, illustrated in Figure 1, comprises three main stages: data preparation, model construction and training, and performance evaluation.

Figure 1.

The overall workflow and architecture of the proposed hybrid CNN-Transformer model. (A) Stage 1 Data Preprocessing: Multi-view ultrasound images of a single tumor are cropped, normalized, and collected to form the input set. (B) Stage 2 Hybrid CNN-Transformer Model: A shared-weight EfficientNetV2 backbone extracts 1D feature vectors (1280D) from the parallel views. The sequence is prepended with a [CLS] token, zero-padded, and masked. Crucially, positional encodings are deliberately omitted to enforce permutation invariance, allowing the Transformer to process the multi-view input as an unordered set. Global self attention dynamically fuses these features, and the final [CLS] state predicts the malignancy probability. (C) Stage 3 Model Evaluation: The CAD system’s clinical performance is comprehensively assessed against baselines and human experts using standard diagnostic metrics, Grad-CAM visualizations, ROC curves, and prediction distributions

3.1. Study Population and Ethical Approval

This retrospective study was conducted in accordance with the Helsinki Declaration of 1975, as revised in 2024, and approved by the Institutional Ethics Committee of the Second Affiliated Hospital of Guangzhou University of Chinese Medicine (Approval No. ZE2023-427, 05/12/2023), and being a retrospective study, the requirement for written informed consent was waived by the ethics committee. The reporting of this study conforms to the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guidelines for cohort studies. We have followed relevant Equator guidelines to ensure methodological transparency and reproducibility. ²⁶

The case inclusion process is illustrated in Figure 2. We assessed medical records from two independent sources. The primary dataset was derived from June 2023 and April 2025. Inclusion criteria were: (1) patients with complete clinical and final pathological results who underwent preoperative ultrasound; (2) preoperative ultrasound with transverse or sagittal views and high-quality images; (3) patients who had not received neoadjuvant chemotherapy or radiotherapy. Exclusion criteria were: (1) poor ultrasound image quality; (2) an interval between ultrasound and pathology greater than two weeks; (3) prior surgery, biopsy, or chemotherapy. Out of 980 initially screened patients, 140 were excluded. The final primary dataset comprised 840 patients(age range: 11–88 years; mean age: 51.28 ± 13.55 years), consisting of 430 benign patients (1566 images) and 410 malignant patients (2220 images).

Figure 2.

The case inclusion process

The external dataset was collected from October 2024 to April 2025. Following the same exclusion criteria, 47 out of 180 screened patients were excluded. The final external dataset included 133 patients, comprising 70 benign patients (223 images) and 63 malignant patients (440 images).

For model development, the datasets were categorized into training, validation, test, and external sets. Detailed clinical characteristics, including patient age distribution and clinical tumor size (T1≤2.0cm and T2 2.1–5.0cm), are summarized in Table 1.

Table 1.

The Clinical Characteristics of all Patients

Characteristic	Training set	Validation set	Test set	External dataset
No. of patients(images)	588	126	126	133
Age
<40y	126(21)	32(25)	37(30)	43(32)
40-49y	141(24)	36(29)	32(25)	34(26)
50-59y	155(26)	36(29)	29(23)	35(26)
60-69y	111(19)	16(12)	22(17)	11(8)
≥70y	55(10)	6(5)	6(5)	10(8)
Clinical tumor size (No. of tumors)
T1≤2.0cm	288	56	49	61
T2(2.1-5.0cm)	300	70	77	72

Note. Unless otherwise specified, data in parentheses are percentages.

3.2. Data Acquisition and Preprocessing

Ultrasound examinations were performed by experienced physicians using GE LOGIQ E9 and Philips EPIQ 7C systems with high-frequency linear probes (7.5–13.0 MHz). Standard images of each lesion were acquired and stored in the picture archiving and communication system (PACS).

The raw DICOM images were converted to JPG format. Two physicians, each with over five years of experience in breast ultrasound diagnosis, manually cropped the images to isolate the lesion and remove extraneous information, such as patient data and measurement markers in Figure 3. This step ensures the model focuses on relevant pathological features. All images were then resized to 300×300 pixels to match the model’s input dimensions, and pixel values were normalized to a range of [0, 1].

Figure 3.

Pre- and post-processing of ultrasound images; Example of the image preparation workflow. (a) The original ultrasound image as exported from the PACS, containing patient information and measurement markers. (b) The image after being manually cropped by an expert physician to isolate the lesion of interest for model input

3.3. Dataset Partitioning Strategy

A scientifically rigorous dataset partitioning strategy is fundamental to the objective and reliable evaluation of AI models in medical imaging. A common pitfall is the random allocation of multiple images from the same patient’s tumor into the training, validation, and test sets. This practice leads to severe data leakage, as the model may learn to identify patient-specific artifacts rather than generalizable pathological features. This results in an artificially inflated performance on the test set, which significantly decreases when the model is exposed to new data in a real-world setting.

To address this issue, this study employed a strict per-lesion partitioning methodology. Specifically, all images associated with a single tumor were treated as an indivisible unit and assigned exclusively to one of the three sets: training, validation, or test. The internal dataset was partitioned using a 70%:15%:15% ratio, ensuring complete independence at the lesion level. The entire external dataset was held out as a separate, independent test set to evaluate the model’s generalization performance. The data distribution is detailed in Table 2.

Table 2.

Dataset Distribution for the Baseline and Hybrid Models

Model type	Data unit	Training set	Validation set	Test set	External dataset	Total
Baseline	Images	2657	590	539	663	4449
Hybrid	Tumors	588	126	126	133	973

3.4. Handling Multi-View Data

A critical challenge in multi-view learning is the variable number of images available per lesion (ranging from 2 to 10). We implemented a robust technical strategy to handle this variability.

3.4.1. Fixed Max Length and Padding

We defined a hyperparameter max_images_per_lesion = 10.

• Truncation: If a lesion has >10 images, only the first 10 are used.

• Zero-Padding: If a lesion has N < 10 images, we generate 10 - N dummy images consisting of all-zero tensors. This ensures that the input tensor to the model always has a fixed shape.

3.4.2. Attention Masking

To ensure the zero-padding does not influence the diagnosis, we generated a binary Attention Mask for each sample. The mask contains 1 for real images and 0 for padded images. In the Transformer’s self-attention calculation, the similarity scores corresponding to padded positions are set to negative infinity before the Softmax operation. This forces the attention weights for padded images to become effectively zero, meaning they contribute nothing to the feature aggregation.

3.4.3. Classification Token Aggregation

Instead of simple averaging, which dilutes strong signals, we prepend a learnable token to the sequence of image features. The Transformer learns to update this token with information aggregated from all valid views. The final classification is performed solely on the state of this token.

3.5. Model Architectures

This study designed and compared two distinct models: a single-image classification baseline and the proposed hybrid CNN-Transformer model.

3.5.1. Baseline Model: EfficientNetV2 Single-Image Classifier

The baseline model was implemented using a standard transfer learning paradigm to establish a performance benchmark for a conventional single-image approach. The EfficientNetV2 model, pre-trained on the ImageNet dataset, was specifically selected as the core architecture over traditional heavy CNNs such as ResNet50 or VGG16 and larger EfficientNet variants such as V2-M or V2-L. This choice was driven by the need to balance representational capacity with computational efficiency. Given the relatively constrained size of our breast ultrasound dataset compared to massive natural image datasets, deploying overly deep backbones drastically increases the risk of overfitting. The Small variant provides an optimal depth to extract complex, fine-grained pathological features (such as margin irregularities) without memorizing patient-specific noise. Furthermore, its Fused-MBConv layers ensure rapid feature extraction, which is essential for maintaining sub-second inference latency when integrated into the subsequent hybrid CAD system. ²⁷

The specific implementation details are as follows:

• Backbone: The EfficientNetV2 model, pre-trained on the ImageNet dataset, was selected as the feature extractor due to its excellent balance between performance and efficiency.

• Feature Extraction: The parameters of the convolutional layers in the backbone were frozen, allowing it to function as a fixed feature extraction tool.

• Classification Head: The original fully connected layer was replaced with a custom head consisting of a linear layer with 1280 input dimensional features and a single output neuron with a Sigmoid activation function to produce a binary probability of malignancy.

Mathematically, the feature extraction process for the baseline model is defined as:

$$\mathrm{v}=f_{CNN}\left(X\right)$$ (1)

where $X$ represents a given 2D ultrasound image, $f_{CNN}$ denotes the frozen EfficientNetV2 backbone acting as the feature extraction function, and $v\in {\mathbb{R}}^D,\left(D=1280\right)$ is the extracted high-dimensional feature vector.

The final binary classification probability is computed as:

$$\widehat{y}=\sigma \left(\mathrm{W}·\mathrm{v}+b\right)$$ (2)

where $\widehat{y}$ is the predicted probability of malignancy, $W$ and $b$ represent the learnable weight matrix and bias vector of the dense layer respectively, and $\sigma (·)$ denotes the Sigmoid activation function.

3.5.2. Proposed Hybrid Model

This model consists of two stages.

• • Local Feature Extraction: Each of the N images in a patient’s lesion is passed independently through the EfficientNetV2 backbone, resulting in a sequence of feature vectors $F=\left\{f_1,f_2,\cdots ,f_N\right\}$ , where $f_i\in {\mathrm{R}}^D$ .

• • Global Feature Fusion: The sequence was fed into a Transformer Encoder.

  • ○ Hyper-parameters: To ensure optimal performance, we conducted a systematic grid search for hyper-parameter optimization. We evaluated dropout rates of $\{0.0,0.1,0.2,0.3,0.4,0.5\}$ , attention heads of $\left\{4,8,16\right\}$ , and network depths of $\{4,6,8,12,16\}$ layers. Based on the validation set performance of AUC, the final model was configured with 8 layers, 8 attention heads, and a dropout rate of 0.1 in Figure 4.
  • ○ Positional Encoding: Crucially, we removed the positional encoding typically used in NLP Transformers. This enforces permutation invariance, ensuring the model treats the input as an unordered set rather than a sequence.
  • ○ Output: The final embedding of the token is passed to a MLP head with a Sigmoid activation to predict the probability of malignancy.

Figure 4.

Hyperparameter sensitivity analysis

For the hybrid model, let a patient’s lesion be represented by an unordered set of $N$ multi-view ultrasound images, denoted as $X=\left\{X_1,X_2,\cdots ,X_N\right\}$ . Each image is independently passed through the EfficientNetV2 backbone to extract a sequence of localized feature vectors:

$${\mathrm{f}}_i=f_{CNN}\left(X_i\right),\forall i\in \{1,2,\cdots ,N\}$$ (3)

where $X_i$ is the $i$ -th ultrasound image in the set, and ${\mathrm{f}}_i$ is its corresponding feature vector.

To aggregate these view-specific features, we prepend a learnable classification token to the sequence. The input matrix to the Transformer encoder is constructed as:

$$Z_0=Cat\left[{\mathrm{x}}_{cls};{\mathrm{f}}_1;{\mathrm{f}}_2;\cdots ;{\mathrm{f}}_N\right]$$ (4)

where $Z_0$ is the initial input sequence matrix, ${\mathrm{x}}_{cls}\in {\mathbb{R}}^{1280}$ represents the learnable classification token, and $Cat[·]$ denotes the concatenation operation.

Crucially, to preserve the permutation invariance of the unordered views, we do not add a positional embedding matrix ( $E_{pos}$ ) to $Z_0$ . The Transformer encoder consists of $L$ layers ( $L=8$ in our optimal setting). The processing at layer $\mathcal{l}$ employs Multi-head Self-Attention (MSA) and a MLP with Layer Normalization (LN):

$$Z_{\mathcal{l}}^{\prime }=MSA\left(\mathit{LN}\left(Z_{\mathcal{l}-1}\right)\right)+Z_{\mathcal{l}-1}$$ (5)

$$Z_{\mathcal{l}}=MLP\left(\mathit{LN}\left(Z_{\mathcal{l}}^{\prime }\right)\right)+Z_{\mathcal{l}}^{\prime }$$ (6)

where $\mathcal{l}\in \left\{1,\dots ,L\right\}$ indicates the current layer index, $Z_{\mathcal{l}-1}$ is the output feature from the previous layer, and $Z_{\mathcal{l}}^{\prime }$ is the intermediate state.

The final diagnosis is derived exclusively from the state of the classification token at the final layer:

$$\widehat{y}=\sigma \left(\mathrm{W}{·Z}_L^0+b\right)$$ (7)

where $Z_L^0$ denotes the state of the classification token $x_{cls}$ after passing through all $L$ layers, and $W$ and $b$ are the parameters of the classification head.

3.5.3. Ablation Study

To strictly evaluate the contribution of the Transformer architecture and verify the optimal fusion strategy for multi-view ultrasound data, we implemented two comparative baseline models using the same EfficientNetV2 backbone.

• • CNN-Only (Global Average Pooling Baseline): This baseline represents a bag-of-features approach that aggregates multi-view information without modeling inter-view relationships.

  • ○ Batch Processing: The input tensor of shape $\left(B,V,C,H,W\right)$ is reshaped to $\left(B\times V,C,H,W\right)$ to treat all images across the batch as independent samples.
  • ○ Feature Extraction: The shared CNN backbone extracts high-dimensional features from these images.
  • ○ Reshaping & Masking: The features are reshaped back to $\left[B,N,D\right]$ , where $D=1280$ . A binary mask is applied to zero out features corresponding to padded dummy images, ensuring they do not affect the calculation.
  • ○ Aggregation: We apply Global Average Pooling along the view dimension for valid images only. This produces a single, averaged feature vector for the patient, which is fed into the classification head. This model assumes all views contribute equally and independently to the diagnosis.
• • CNN-RNN (Sequential Modeling Baseline): This model tests the hypothesis that sequential dependencies exist within the multi-view acquisition process.

  • ○ Sequential Modeling: Unlike the parallel processing of the Transformer or the simple averaging of the CNN-Only model, this architecture utilizes a LSTM network.
  • ○ Temporal Dependencies: The sequence of image features extracted by the CNN is fed into the LSTM, which updates its hidden state step-by-step, attempting to capture temporal correlations between successive ultrasound frames.
  • ○ Final Representation: Instead of using all states, we extract the hidden state of the last valid time step (determined by the sequence length of non-padded images) to serve as the comprehensive representation of the lesion. This vector is then passed to the classifier to output the final prediction.

3.5.4. State-of-the-Art Benchmarks

To validate the competitiveness of our proposed method against established solutions in medical image analysis, we benchmarked our model against three advanced architectures representing different fusion paradigms, all comparative models were retrained on our training set using their official implementations and optimal hyperparameters recommended in their original papers.

• VGG-ViT: A classic hybrid architecture that combines a VGG convolutional backbone with a Vision Transformer (ViT). This model serves as a benchmark for early CNN-Transformer integration strategies.

• TransMed: Originally designed for multi-modal medical image fusion, this model was adapted to process multi-view single-modality data. It serves to compare our method against complex, general-purpose medical transformers.

• MedFormer: A specialized Transformer architecture optimized for medical imaging tasks. Comparing against MedFormer allows us to assess whether our lightweight EfficientNetV2-Transformer design can compete with or outperform purpose-built medical transformers in specific breast ultrasound applications.

3.6. Experimental Settings and Training Parameters

All experiments were conducted on a computational platform equipped with an NVIDIA GeForce RTX 4060Ti GPU. The deep learning framework was implemented in Python 3.10 using PyTorch 2.7.1, alongside auxiliary libraries including Torchvision, Scikit-learn, Pandas, and Numpy. To ensure the reproducibility of our results, a detailed summary of the specific hyperparameters used during training is provided in Table 3.

Table 3.

Hyperparameter configuration for Model Training

Parameter	Baseline model	Hybrid model
Optimizer	AdamW	AdamW
Initial Learning Rate	1×10−4	5×10−5
Learning Rate Scheduler	Cosine Annealing	Cosine Annealing
Batch Size	32 (images)	4 (lesions)
Epochs	100 (with Early Stopping)	100 (with Early Stopping)
Early Stopping Patience	5	5
Monitored Metric	Validation AUC	Validation AUC
Loss Function	Binary Cross-Entropy	Binary Cross-Entropy

3.7. Evaluation Metrics

Model performance was assessed using five standard binary classification metrics.

• Accuracy: Measures the proportion of all predictions that are correct.

$$\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}$$ (8)

• Sensitivity: Measures the model’s ability to correctly identify malignant samples.

$$\text{Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{FN}}$$ (9)

• Specificity: Measures the model’s ability to correctly identify benign samples.

$$\text{Specificity}=\frac{\text{TN}}{\text{TN}+\text{FP}}$$ (10)

where TP, TN, FP, and FN represent correctly classified malignant lesions, correctly classified benign lesions, benign lesions misclassified as malignant, and malignant lesions misclassified as benign, respectively.

• AUC: Measures the model’s overall discriminative ability across all classification thresholds. An AUC closer to 1 indicates better performance.

3.8. Evaluation Metrics and Statistical Analysis

Performance was assessed using Accuracy, Sensitivity, Specificity, and AUC.

Statistical Rigor.

• Confidence Intervals (CI): We calculated 95% CIs for all metrics using bootstrapping ( $n=1000$ resamples) to demonstrate the stability of our results.

• DeLong Test: Used to statistically compare the AUC of different models. A p-value $<0.05$ was considered significant.

3.9. Quantitative Evaluation of Model Interpretability

To evaluate the interpretability of the proposed model and assess the clinical relevance of the Grad-CAM heatmaps quantitatively, a structured, blinded qualitative evaluation was conducted. A total of 126 heatmaps from the internal test set were independently evaluated by two ultrasound experts blinded to the model’s predictions and the pathological ground truth. The clinical relevance of the high-attention regions was graded using a 4-point scale: 4 (Highly Relevant: overlap with core lesions or key pathological features), 3 (Moderately Relevant: covers the lesion but includes some surrounding normal tissue), 2 (Weakly Relevant: focus deviates from the main lesion), and 1 (Irrelevant: focus on background or artifacts). The inter-rater reliability between the two experts was quantified using Overall Agreement and Cohen’s Kappa statistic.

4. Results

4.1. Overall Model Performance Comparison

The proposed hybrid CNN-Transformer demonstrated a significant improvement in diagnostic performance over the conventional single-image baseline model across all evaluation metrics. This advantage was consistent on both the internal test set and, crucially, on the independent external dataset, underscoring the model’s robustness and generalization capabilities.

On the internal test set, which was partitioned at the tumor level to prevent data leakage, the hybrid model achieved an accuracy of 0.960 and an AUC of 0.9788. When challenged with the external dataset, comprising data acquired with different equipment, the model’s performance remained remarkably stable, yielding an accuracy of 0.940 and an AUC of 0.9730. In contrast, the baseline EfficientNetV2 model, which classifies each image independently, achieved lower metrics on both test sets. These results, detailed in Table 4, provide strong evidence that the fusion of multi-view information through the Transformer architecture leads to a statistically significant improvement in diagnostic power.

Table 4.

Performance Comparison of the Two Models on the Test Sets

Model	Test set	Accuracy	Sensitivity	Specificity	AUC
Baseline Model (EfficientNetV2)	Internal	0.881	0.863	0.914	0.9521
	External	0.894	0.909	0.818	0.9437
CNN-Transformer	Internal	0.960	0.967	0.954	0.9788
	External	0.940	0.952	0.929	0.9730

4.2. Analysis of Generalization Ability

To assess the model’s learning dynamics and potential for overfitting, the AUC was tracked for both the training and validation sets throughout the training process in Figure 5. The training and validation AUC curves rose in tandem and remained closely aligned, ultimately converging at a high level of performance (AUC>0.97). This indicates that the model did not overfit to the training data; instead, it learned generalizable features applicable to unseen data.

Figure 5.

Model training and validation curves (AUC vs. Epoch)

The ultimate test of generalization is performance on entirely new data. The Receiver Operating Characteristic (ROC) curves for the validation, internal test, and external dataset are presented in Figure 6. The curves are nearly superimposed, with high AUC values of 0.9708, 0.9788, and 0.9730, respectively. This significant consistency across three independent datasets strongly suggests that the model has learned robust, broadly applicable diagnostic features, validating both the architectural design and the rigorous data partitioning strategy.

Figure 6.

Roc curves on validation, internal test, and external dataset

To address the concern that deep learning models may overfit to specific device artifacts and to ensure clinical generalization, we evaluated the model’s performance stratified by ultrasound machine manufacturer. The internal dataset was acquired using two distinct high-end systems: GE LOGIQ E9 and Philips EPIQ 7C, which employ different beamforming and post-processing algorithms. The model achieved an AUC of 0.9875 on the GE system and 0.9722 on the Philips system in Figure 7. The curves are nearly identical, and the high AUC across both vendors indicate that the model extracts robust, biologically relevant features rather than relying on equipment-specific noise or post-processing signatures. This minimal performance discrepancy ( $\mathrm{\Delta }\text{AUC}<0.02$ ) demonstrates that the proposed architecture successfully mitigates the influence of machine heterogeneity, validating its potential for widespread deployment in multi-vendor clinical environments.

Figure 7.

ROC curves for both devices on Internal Test

4.3. Detailed Classification Performance

A detailed breakdown of the model’s classification performance is presented via confusion matrices for the internal and external datasets (Figure 8) and a comprehensive classification report (Table 5).

Figure 8.

Confusion matrices on the (a) internal and (b) external dataset

Table 5.

Detailed Classification Report on the Internal and External Datasets

Test set	Class	Precision	Recall	F1-score	Support
Internal	Benign	0.969	0.954	0.961	65
	Malignant	0.952	0.967	0.959	61
	Macro Avg	0.960	0.961	0.960	126
External	Benign	0.956	0.929	0.942	70
	Malignant	0.923	0.952	0.938	63
	Macro Avg	0.939	0.940	0.940	133

On the internal test set, the confusion matrix shows the model correctly identified 62 of 65 benign lesions and 59 of 61 malignant lesions in Figure 8A. As detailed in Table 5, this performance corresponds to a recall of 0.954 (specificity) and a precision of 0.969 for benign cases. For the crucial task of identifying malignancy, the model achieved a Sensitivity of 0.967 and a precision of 0.952. The excellent F1-scores of 0.960 for both classes highlight a strong and desirable balance between precision and recall.

The model’s robustness was further validated on the external dataset in Figure 8B. It achieved a recall (specificity) of 0.929 and a precision of 0.956 for benign lesions. For malignant lesions, it demonstrated an even higher recall (sensitivity) of 0.952, with a corresponding precision of 0.923. This indicates a very low rate of missed malignant cases. The resulting F1-scores of 0.942 for benign and 0.938 for malignant classes confirm the model’s strong diagnostic capability on unseen data.

In summary, the consistently high performance across both datasets, highlighted by macro-averaged F1-scores of 0.960 (internal) and 0.940 (external), affirms the model’s reliability and strong generalization potential.

4.4. Analysis of Prediction Probability and Confidence

Beyond standard classification metrics, the confidence of a model’s predictions is a critical dimension of its clinical reliability. Figure 9 illustrates the frequency distribution of predicted malignancy probabilities for all benign and malignant samples in the test sets. The histograms reveal a distinct separation: probabilities for benign samples (blue) are highly concentrated near 0.0, while those for malignant samples (red) are predominantly clustered near 1.0. This distinct lack of overlap between the two distributions visually confirms the model’s strong discriminative ability. It demonstrates that the model does not merely achieve high accuracy but renders diagnostic decisions with high confidence.

Figure 9.

Histogram of predicted probability distributions on the (a) internal and (b) external dataset

4.5. Confidence Intervals Analysis

The proposed hybrid CNN-Transformer demonstrated robust performance on both internal and external datasets. The results, including 95% Confidence Intervals, are presented in Table 6.

Table 6.

Confidence Interval Analysis Results

Test set	Accuracy	AUC	Sensitivity	Specificity
Internal	0.9603	0.9788(95%CI: 0.9508 - 0.9990)	0.967(95%CI:0.917 - 1.0000)	0.954(95%CI: 0.900 - 1.000)
External	0.9398	0.9730(95%CI: 0.9414 - 0.9937)	0.952(95%CI:0.894 - 1.000)	0.929(95%CI: 0.864 - 0.985)

The model achieved an AUC of 0.9788 on the internal set and maintained an AUC of 0.9730 on the external dataset. The overlap in the 95% CIs between internal and external performance indicates excellent generalization; the model did not simply memorize the training data distribution or specific hospital artifacts. The high Sensitivity (>0.95 in both sets) is clinically pivotal, ensuring that malignant cases are rarely missed.

4.6. Ablation Study Results

To determine the optimal mechanism for multi-view information integration, a comprehensive component-wise ablation study was conducted. The objective was to evaluate the hypothesis that modeling ultrasound views as an unordered set—via a Transformer-based architecture—yields superior performance compared to sequential modeling using RNNs or view-independent processing with CNNs alone (Table 7).

Table 7.

Ablation Study Results

Model architecture	Fusion strategy	Validation AUC	Test AUC
CNN-Only	Max Pooling (Post-hoc)	0.9502	0.9576
CNN-RNN	Bi-Directional LSTM	0.9676	0.9659
CNN-Transformer	Self-Attention (Proposed)	0.9708	0.9788

The CNN-Only model achieved an AUC of 0.9576, establishing a competitive baseline; however, it is limited to post-hoc aggregation of view-specific features without cross-view feature synthesis. The CNN-RNN model (AUC 0.9659) introduces feature-level fusion through sequential modeling, yet it is outperformed by the CNN-Transformer model (AUC 0.9788). This statistically significant improvement supports our hypothesis: the permutation invariance inherent in the Transformer architecture is more suitable for multi-view ultrasound analysis than the ordered assumption underlying RNNs. By employing self-attention, the Transformer dynamically prioritizes the most informative views irrespective of their positional order, thereby emulating a cognitive integration process free from artificial temporal constraints.

4.7. Computational Efficiency

To evaluate the clinical feasibility of the proposed hybrid CNN-Transformer architecture, we assessed its computational efficiency on an NVIDIA RTX 4060Ti GPU. The total inference time for a cohort of 126 samples was recorded at 4.2249 seconds, yielding an average processing time of 0.0335 seconds per patient. With an inference latency significantly below the 40 ms threshold, our model demonstrates high operational efficiency, confirming its suitability for seamless integration into real-time clinical workflows.

4.8. Comparison With State-of-the-Art Architectures

To strictly validate the competitiveness of our proposed method, we benchmarked it against three advanced architectures representing different fusion paradigms: VGG-ViT, TransMed, and MedFormer. As detailed in Table 8, our EfficientNetV2-Transformer hybrid model achieved the highest Test AUC of 0.9788, outperforming all comparative baselines. A deeper analysis reveals the architectural reasons behind this performance gap.

Table 8.

Comparison Results of Similar Models

Model	Validation AUC	Test AUC
VGG-ViT	0.9702	0.9541
TransMed	0.9701	0.9684
MedFormer	0.9695	0.9697
CNN-Transformer	0.9708	0.9788

4.8.1. Superior Generalization vs. VGG-ViT

The VGG-ViT model exhibited a high Validation performance of AUC with 0.9702, but its performance dropped sharply on the Test set (0.9541). This suggests that the heavy parameter count of the VGG backbone, combined with a standard Vision Transformer, led to overfitting on the ultrasound dataset. In stark contrast, our model achieved superior generalization, improving from a Validation AUC of 0.9702 to a Test AUC of 0.9788. This confirms that our choice of the lightweight EfficientNetV2 backbone combined with a shallow (8-layer) Transformer strikes a better balance between model complexity and data efficiency, learning robust pathological features rather than memorizing noise.

4.8.2. Task-Specific Optimization vs. TransMed and MedFormer

TransMed (Test AUC 0.9684) and MedFormer (Test AUC 0.9697) represent complex, heavy-weight architectures originally designed for multi-modal tasks or general medical segmentation. While they performed well, they did not surpass our proposed model.

• The Bag-of-Images Advantage: TransMed assumes complex inter-modal relationships that are unnecessary for single-modality ultrasound. Our model, by removing positional encoding, enforces permutation invariance. This design is specifically optimized for the unordered nature of multi-view ultrasound, allowing the model to focus purely on aggregating complementary diagnostic evidence.

• Efficiency: Our model achieves superior accuracy without the computational burden of these heavy transformers. This demonstrates that a focused, task-specific architecture is more effective for breast ultrasound classification than adapting large-scale, general-purpose medical transformers.

In summary, the benchmarking results demonstrate that increased model complexity does not necessarily translate to superior diagnostic performance. Our model’s success stems from aligning the network architecture with the clinical reality of breast ultrasound—treating views as an unordered set and prioritizing feature robustness over model complexity.

4.9. Model Interpretability Using Gradient-Weighted Class Activation Mapping (Grad-CAM)

To investigate the model’s decision-making basis, Grad-CAM was used to visualize the regions of interest the CNN feature extractor focused on clinical ultrasound features. Although the final diagnosis results from the Transformer’s fusion of features, Grad-CAM provides insight into whether the foundational feature extraction is clinically relevant.

As shown in Figures 10 and 11, when the model correctly classifies a lesion, the high-attention areas (heatmaps) precisely overlap with clinically significant pathological features, such as irregular margins, internal microcalcifications, and angular contours. This visualization confirms that the CNN component effectively learns to identify meaningful biomarkers, providing a clinically relevant and potentially interpretable foundation for the Transformer’s subsequent global analysis. This enhances trust in the model’s decisions by demonstrating that its reasoning is aligned with established clinical diagnostic criteria.

Figure 10.

BI-RADS 4a Breast lesion with benign features on ultrasound and Grad-CAM Analysis. Ultrasound imaging reveals a solid, hypoechoic mass classified as BI-RADS 4a. The Hybrid model correctly predicted this lesion as benign. Corresponding Grad-CAM overlays visually explain the model’s decision by highlighting key benign features: (a) significant posterior acoustic enhancement, well-circumscribed margins and homogeneous internal echotexture, (b) well-circumscribed margins and homogeneous internal echotexture

Figure 11.

BI-RADS 4c Breast Lesion with Malignant Features on Ultrasound and Grad-CAM Analysis. Ultrasound imaging depicts a mass with highly suspicious characteristics, classified as BI-RADS 4c. The Hybrid model accurately predicted this lesion as malignant. The explanatory Grad-CAM overlays identify the critical malignant features driving the model’s decision: (a) angular, spiculated margins, and rich internal vascularity on color Doppler imaging, (b) a thick echogenic halo, an irregular and indistinct boundary

In addition to visual inspection, the clinical relevance of the Grad-CAM heatmaps was evaluated through a structured grading by two independent experts (Supplemental Table 2). The results indicated a favorable level of clinical interpretability. Specifically, 85.71% (Expert 1) and 84.13% (Expert 2) of the heatmaps were rated as highly or moderately relevant (Scores 4 and 3), suggesting that the CNN feature extractor frequently attends to diagnostically relevant areas rather than background noise. Furthermore, the evaluation showed substantial inter-rater reliability, with an overall agreement of 82.54% and a Cohen’s Kappa of 0.7207. These quantitative findings provide supporting evidence that the model’s focal regions are generally consistent with clinical diagnostic patterns.

4.10. Error Analysis of Misclassified Cases

While the proposed hybrid CNN-Transformer model demonstrated high overall diagnostic accuracy, a detailed error analysis of the misclassified cases reveals that specific pathological subtypes and overlapping morphological features remain challenging.

False Positives: Supplemental Figure 1a illustrates a characteristic false positive case, pathologically confirmed as mastitis. The common image characteristics in such inflammatory conditions include highly irregular margins and abundant internal blood flow signals, which heavily overlap with typical malignant indicators. The Grad-CAM heatmaps demonstrate that the model’s attention is strongly fixated on these irregular borders and hypervascular regions. Because the image-only model lacks access to the patient’s clinical history, it is easily deceived by these superficial morphological mimics, ultimately misinterpreting an acute inflammatory process as malignancy.

False Negatives: Conversely, Supplemental Figure 1b presents a false negative case involving a specific malignant subtype: mucinous carcinoma. This rare malignant tumor is notorious for presenting with benign-mimicking imaging characteristics, predominantly well-circumscribed margins and significant posterior acoustic enhancement. The Grad-CAM overlay explicitly shows the model focusing its attention on these deceptive, regular boundaries and the enhanced posterior echo. Consequently, the model relies on these classical benign rules and incorrectly predicts a low probability of malignancy.

This analysis indicates that while the CNN-Transformer effectively learns standard diagnostic rules, it is vulnerable to atypical malignant subtypes that exhibit benign features, as well as benign inflammatory conditions that mimic malignant morphology.

5. Expert Validation Test

To validate the clinical relevance of our model and address concerns regarding human-machine comparison, we conducted a blind test using the internal test set (n=126). Three ultrasound physicians participated: two junior doctors (3-5 years of experience) and one senior doctor (>15 years of experience). They independently classified the 126 lesions without access to the model’s predictions or pathology results (Table 9).

Table 9.

Comparison With Human Experts

Method	Correct cases	Accuracy	Sensitivity	Specificity
Junior Doctor 1	103	0.818	0.820	0.815
Junior Doctor 2	104	0.825	0.803	0.846
Senior Doctor	107	0.849	0.869	0.831
Average Human	104.7	0.831	0.831	0.831

As presented in Table 9, the proposed model significantly outperformed all human readers. The model achieved an accuracy of 0.960, surpassing the senior expert (0.849) by over 11 percentage points. Notably, the model’s sensitivity (0.967) was substantially higher than even the senior doctor (0.869), indicating a superior capability in detecting malignancies and reducing false negatives. This suggests that the model can serve as a highly effective suggestion to reduce missed diagnoses in clinical practice.

To understand the source of this performance gap, we analyzed discordant cases where human experts erred but the model succeeded. Figure 12 illustrates a challenging diagnostic scenario involving a hypoechoic mass with ambiguous features. The original grayscale image (Figure 12A) and Color Doppler (Figure 12B) show a hypoechoic mass with somewhat ambiguous features, possibly exhibiting minor irregularities or shadowing that mimics malignancy, leading to a potential human misdiagnosis (False Positive).

• Grad-CAM Insight: The heatmap reveals that the model’s attention was not distracted by the internal heterogeneity. Instead, it focused strongly on the upper and lateral boundaries of the lesion. By accurately identifying the continuity of the tumor and the distinct tissue interface (highlighted in red), the model confirmed the well-circumscribed nature of the tumor.

• Clinical Significance: This case exemplifies the model’s robustness in reducing unnecessary biopsies. While human eyes might be misled by suspicious-looking artifacts, the Grad-CAM confirms that the model grounds its decision on subtle but definitive markers of benignity, effectively serving as a gatekeeper to improve specificity.

Figure 12.

A Challenging case correctly reclassified by the hybrid model (Human-AI Discordance). (a) B-mode ultrasound and (b) color doppler ultrasound, where the Grad-CAM heatmap focused strongly on the upper and lateral boundaries of the lesion. It identifies the continuity of the tumor and the distinct tissue interface (highlighted in red)

6. Prospective Validation on a Newly Collected Dataset

6.1. Rationale and Data Collection and Inclusion

To rigorously address potential concerns regarding model generalization and data leakage, we performed a stringent prospective validation of the final hybrid CNN-Transformer. This test used an independent dataset collected after the development and finalization of the model, mimicking a real-world clinical deployment scenario.

Data for this prospective test were collected consecutively from September 2025 to December 2025. A total of 188 new lesions (including 122 benign and 66 malignant cases, confirmed by pathology) were collected during this period, ensuring no overlap with the initial training, validation, or internal test sets. All data collection and image acquisition protocols remained identical to the procedures outlined in Section 2.

6.2. Prospective Test Results

The hybrid CNN-Transformer was applied to the newly collected prospective test set. The results, visualized by the Classification Report, Confusion Matrix, and ROC Curve (Figure 13), demonstrate the model’s robust generalization capability.

Figure 13.

Performance evaluation on the prospective validation cohort. (a) ROC Curve: The model achieved an impressive AUC of 0.9801 on the prospective dataset. (b) Detailed Probability Distribution: The histogram shows a clear separation between predicted probabilities for benign (blue) and malignant (orange) cases, indicating high prediction confidence. (c) Classification Report: Quantitative metrics detailing the Precision, Recall, and F1-score for each class, with an overall accuracy of 95.2%. (d) Confusion Matrix: The matrix confirms 98.48% sensitivity (65/66) for malignant lesions and highlights that 8 benign cases were misclassified (114/122 correctly identified)

The model achieved an overall accuracy of 95.2% and AUC of 98.01%. Specifically, the model demonstrated a strong capability to correctly identify 65 of 66 malignant lesions and 114 of 122 benign lesions.

These results are comparable to the performance observed on the internal test set (Section 3.1), confirming that the multi-view CNN-Transformer architecture is highly effective and stable across distinct, temporally separated, real-world clinical data.

7. Discussion

The results of this study indicate that the proposed hybrid CNN-Transformer is significantly superior to conventional single-image CNN approaches for classifying breast tumors in ultrasound images. A primary factor contributing to this performance gain is an architecture that more faithfully emulates the cognitive process of clinical experts.

The effectiveness of multi-view fusion is the central finding. A single-image model processes each view in isolation, whereas the Transformer encoder in our hybrid model functions as a virtual expert panel. Its self-attention mechanism allows the model to dynamically weigh and integrate information from the entire set of images for a given lesion.^28,29 For instance, it can learn to assign greater importance to a view that clearly depicts spiculated margins while down-weighting another view obscured by acoustic artifacts. This dynamic, global evidence integration is a powerful analogy for expert diagnostic reasoning and is the key driver behind the model’s performance leap. This architecture also confers greater robustness to the variations inherent in real-world clinical data, such as differences in image quality or the number of available views, as the model can learn to prioritize high-quality, informative images.^30,31

The clinical implications of the model’s performance are substantial. The high specificity achieved on both internal (0.954) and external (0.929) dataset indicates a strong ability to correctly identify benign lesions. This could translate to a significant reduction in false-positive findings, potentially sparing many women from the anxiety, physical discomfort, and cost of unnecessary biopsies. Concurrently, the model’s high sensitivity (0.967 internal, 0.952 external) facilitates a low rate of missed malignant tumors, which is critical for facilitating early diagnosis and treatment.

Despite its promising results, this study has limitations that outline directions for future research. First, although the model demonstrated robust generalization on an independent external dataset and across two different ultrasound vendors (GE and Philips), large-scale multi-center validation encompassing a wider array of equipment manufacturers and patient demographics is necessary to fully establish the model’s reliability in diverse clinical settings. Second, while our Grad-CAM analysis confirmed that the model focuses on clinically relevant features and the expert validation study demonstrated its superiority over human readers, the current system provides only visual heatmaps and probability scores. To achieve seamless integration into routine clinical practice, future iterations must evolve to generate structured text explanations and Breast Imaging-Reporting and Data System (BI-RADS) descriptors. Unlike systems that merely provide quantitative outputs, a holistic approach integrating these results into structured clinical reports would more accurately emulate the diagnostic reasoning of human physicians. Furthermore, integrating our framework with emerging agentic AI systems could enhance methodological reproducibility and improve accessibility for non-programmers, thereby significantly enhancing trust in clinical workflows.^32,33

By using a shallow Transformer (8 layers) on top of an efficient backbone (EfficientNetV2), we achieve a sweet spot between depth and speed. This model does not require a supercomputer; it can run on a standard medical workstation equipped with a mid-range GPU.

8. Conclusion

This study successfully developed and validated a novel hybrid CNN-Transformer deep learning model for breast ultrasound diagnosis. By synergistically combining the powerful local feature extraction of a CNN with the sophisticated global information fusion capabilities of a Transformer, the model effectively simulates the multi-view diagnostic process of clinical experts. Using a rigorous, tumor-level evaluation methodology, the proposed model demonstrated outstanding classification performance on an internal test set (AUC 0.9788) and proved its excellent generalization on an independent external dataset (AUC 0.9730). The consistently high accuracy, sensitivity, and specificity across diverse datasets significantly exceed those of traditional single-image methods. This work not only validates a superior technical architecture but also underscores the methodological importance of multi-view analysis and robust external validation. The model shows potential as a reliable tool to assist clinicians in making more accurate diagnoses of breast cancer, with the ultimate aim of improving patient health and well-being.

In future work, we will focus on developing a large-scale study with more external validations. Ultimately, the goal is to integrate this technology into a user-friendly CAD system and conduct prospective clinical trials to evaluate its real-world impact on diagnostic accuracy, workflow efficiency, and patient outcomes.

Appendix.

Abbreviations

CNN

Convolutional Neural Network

AUC

Area Under the Curve

ROC

Receiver Operating Characteristic

Grad-CAM

Gradient-weighted Class Activation Mapping

BI-RADS

Breast Imaging-Reporting and Data System.

ORCID iD

Qi Zhang https://orcid.org/0009-0004-2917-1062

Ethical Considerations

The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This retrospective study was conducted in accordance with the Helsinki Declaration of 1975, as revised in 2024, and approved by the Institutional Ethics Committee of the Second Affiliated Hospital of Guangzhou University of Chinese Medicine (Approval No. ZE2023-427, 05/12/2023), and being a retrospective study, the requirement for written informed consent was waived by the ethics committee.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.*