Ultrasound-Based Deep Learning Radiomics to Predict Cervical Lymph Node Metastasis in Major Salivary Gland Carcinomas

Abstract

Introduction and Aims

Cervical lymph node metastasis (CLNM) critically impacts surgery approaches, prognosis, and recurrence in patients with major salivary gland carcinomas (MSGCs). We aimed to develop and validate an ultrasound (US)-based deep learning (DL) radiomics model for noninvasive prediction of CLNM in MSGCs.

Methods

A total of 214 patients with MSGCs from 4 medical centers were divided into training (Centers 1-2, n = 144) and validation (Centers 3-4, n = 70) cohorts. Radiomics and DL features were extracted from preoperative US images. Following feature selection, radiomics score and DL score were constructed respectively. Subsequently, the least absolute shrinkage and selection operator (LASSO) regression was used to identify optimal features, which were then employed to develop predictive models using logistic regression (LR) and 8 machine learning algorithms. Model performance was evaluated using multiple metrics, with particular focus on the area under the receiver operating characteristic curve (AUC).

Results

Radiomics and DL scores showed robust performance in predicting CLNM in MSGCs, with AUCs of 0.819 and 0.836 in the validation cohort, respectively. After LASSO regression, 6 key features (patient age, tumor edge, calcification, US reported CLN-positive, radiomics score, and DL score) were selected to construct 9 predictive models. In the validation cohort, the models' AUCs ranged from 0.770 to 0.962. The LR model achieved the best performance, with an AUC of 0.962, accuracy of 0.886, precision of 0.762, recall of 0.842, and an F1 score of 0.8.

Conclusion

The composite model integrating clinical, US, radiomics, and DL features accurately noninvasively predicts CLNM preoperatively in MSGCs.

Clinical Relevance

CLNM in MSGCs is critical for treatment planning, but noninvasive prediction is limited. This study developed an US-based DL radiomics model to enable noninvasive CLNM prediction, supporting personalized surgery and reducing unnecessary interventions.

Introduction

Major salivary gland carcinomas (MSGCs) are rare head and neck malignancies, comprising 3-11% of all head and neck cancers.¹ With the rapid advancement of medical imaging technologies in recent years, the incidence of MSGCs has significantly increased due to the detection of more occult lesions. Surgical resection remains the cornerstone of treatment for MSGCs;² however, optimizing treatment strategies, particularly the selection of neck dissection techniques, remains challenging. The lack of uniform clinical standards and expert consensus has led to substantial variability in treatment protocols across institutions, compromising patient outcomes and complicating clinical decision-making.³

Cervical lymph node metastasis (CLNM) is a critical prognostic factor for MSGCs, with reported incidence rates ranging from 15% to 38%.^4,5 Beyond its impact on survival, CLNM is associated with a higher risk of local recurrence and severe complications (e.g., facial nerve palsy), significantly deteriorating patients' quality of life.^6,7 Existing evidence indicates that, regardless of histologic type, high-grade tumor histology and advanced T stage are independent predictors of CLNM.8, 9, 10 Clinically, the 5-year survival rate drops from 75% in patients without CLNM to 50% in those with CLNM,¹¹ underscoring the urgent need for preoperatively identifying high-risk patients to tailor surgical strategies and optimize the benefit of lymph node dissection.

Despite this urgency, current clinical diagnostic approaches for CLNM in MSGCs exhibit significant limitations. Clinical indicators such as physical examination and palpation are highly operator-dependent, often failing to detect minute CLNM. Traditional pathological metrics such as TNM staging and tumor histologic grade serve as robust prognostic indicators of CLNM, informing risk stratification and treatment decision-making in clinical practice.^12,13 However, their complete and precise assessment can only be performed postoperatively, which limits their utility in preoperative planning. Although fine-needle aspiration cytology (FNAC) is often utilized as a minimally invasive and safe method for evaluating lymph node status, cytological diagnoses are not always definitive due to sampling errors and heterogeneous CLN involvement.¹⁴ In contrast, imaging examinations, while more objective, also have limitations: computed tomography (CT) involves radiation exposure, and magnetic resonance imaging (MRI) is associated with higher costs.¹⁵ Ultrasound (US), as a cost-effective, radiation-free, and widely accessible imaging modality, serves as the preferred initial tool for evaluating salivary gland lesions and CLN pathologies.^16,17 However, its interpretation is highly dependent on the experience of radiologists, leading to significant inter-observer variability in identifying subtle CLN abnormalities, and there remains a marked paucity of research on CLNM prediction in MSGCs based on US features.

These limitations underscore the need for more reliable tools, and machine learning (ML)-based decision support systems are demonstrating great potential in this regard. As a core component of artificial intelligence (AI), ML algorithms provide precise predictive insights for individual patients by identifying complex patterns in existing data, thereby guiding optimal treatment strategies.¹⁸ Currently, AI technologies, including ML, are revolutionizing the field of dentistry and the diagnosis and management of head and neck tumors through the implementation of precise risk stratification and personalized treatment plans, highlighting their significant potential to overcome the limitations of traditional diagnostic methods.18, 19, 20, 21, 22

Radiomics, an emerging imaging analysis technique, enables high-throughput extraction of quantitative texture and morphological features from medical images, transforming qualitative imaging data into high-dimensional datasets that capture tumor heterogeneity imperceptible to the human eye.²³ Currently, radiomics has demonstrated promise in the diagnosis, staging, and prognosis prediction of head and neck cancers.24, 25, 26 Deep learning (DL), leveraging neural network architectures, automates the extraction of hierarchical image features and has shown preliminary efficacy in CLNM prediction.27, 28, 29 Deep transfer learning further addresses the challenge of small clinical datasets by fine-tuning pre-trained networks for new tasks,³⁰ offering a viable solution for MSGCs research with limited data. Notably, no consensus has been reached on whether DL inherently outperforms radiomics in prediction, suggesting that integrating radiomic and DL features may synergistically enhance predictive performance.

Despite these advancements, no prior study has integrated clinical data, US features, radiomic features, and DL features to develop a comprehensive risk prediction model for CLNM in MSGCs. This study aims to develop and validate an US-based deep learning radiomics composite model for noninvasive prediction of CLNM risk in MSGCs.

Materials and methods

Patients

A comprehensive search of the electronic databases from 4 medical centers (Center 1: The First Affiliated Hospital of Xiamen University; Center 2: Fujian Medical University Union Hospital; Center 3: The Affiliated Zhongshan Hospital of Xiamen University; Center 4: Xiamen Branch of Zhongshan Hospital, Fudan University) was conducted. Medical records of patients from January 2018 to August 2024 were retrospectively reviewed, and 1,068 patients diagnosed with MSGCs based on surgical and pathological records were identified. The exclusion criteria for this study were as follows: (1) undetermined histological type or primary tumor site (to maintain cohort homogeneity, as these affect metastatic potential and could confound models); (2) prior treatment before surgical resection (preoperative interventions alter both primary lesion and lymph node characteristics, biasing US-based feature analysis); (3) CLN status unconfirmed by pathological examination via excisional biopsy or neck dissection (pathology is the gold standard, ensuring reference standard accuracy for training); (4) lymph node evaluation based solely on FNAC (insufficient for micrometastases, compromising ground truth reliability); (5) distant metastases or undocumented metastatic status (to focus on cervical involvement, avoiding heterogeneity from distant disease or unclear status); (6) recurrent lesions (exhibit distinct biological and imaging characteristics, which may confound model generalization to de novo disease); (7) unavailability or poor quality of pretreatment US images (high-quality images are essential for reliable feature extraction and robust model development).

After applying these criteria, 214 patients with MSGCs were ultimately included, consisting of 128 males (mean age 52.16 ± 17.16 years; range 18-88 years) and 86 females (mean age 48.26 ± 17.26 years; range 22-85 years). Initial clinical and pathological information, such as age, sex, location, smoking history, pain symptoms, facial nerve function, clinical staging (T1-T2/T3-T4), and histological type, was retrieved from the medical records for each patient. Facial nerve function was evaluated using the House-Brackmann grading system.³¹ Subsequently, patients from Center 1 (89 cases) and Center 2 (55 cases) were included in the training cohort (144 cases), comprising 94 pathologically lymph node-negative (pN0) cases and 50 pathologically lymph node-positive (pN+) cases; while patients from Center 3 (46 cases) and Center 4 (24 cases) were assigned to the validation cohort (70 cases), including 51 pN0 cases and 19 pN+ cases. The patient recruitment process is illustrated in Figure 1A, and the study design and workflow are depicted in Figure 1B.

Fig. 1.

Patient enrollment process and study design workflow. (A) Patient enrollment process of this study. (B) Study design and workflow. Note: MSGCs, major salivary gland carcinomas; pN0, pathologically lymph node-negative; pN+, pathologically lymph node-positive; ICC, intraclass correlation coefficient; mRMR, maximum relevance minimum redundancy; LASSO, the least absolute shrinkage and selection operator; ROC, receiver operating characteristic; DCA, decision curve analysis.

US assessment

All US examinations were performed by radiologists with 5 to 20 years of experience in superficial US evaluation, including the assessment of salivary glands, thyroid gland, and other relevant areas. Advanced diagnostic US devices were utilized, such as the Philips EPIQ 7 (Philips Ultrasound, Bothell, WA, USA), GE Logiq E9 (GE Healthcare, Wauwatosa, WI, USA), Philips IU22 (Philips Ultrasound, Bothell, WA, USA), Aixplorer ultrasound scanner (Supersonic Imagine, Aix-en-Provence, France), and Siemens Acuson Sequoia (Siemens Medical Solutions, Inc., Malvern, PA, USA), among others. These devices were equipped with high-resolution linear array probes with a frequency range of 6-18 MHz, ensuring optimal imaging clarity. Patients were positioned supine with their necks fully exposed and heads tilted to the opposite side, enabling a comprehensive scan of all sections within the parotid and submandibular gland regions. Subsequently, the clearest and most comprehensive ultrasound images were obtained and recorded. These images were stored digitally in Digital Imaging and Communications in Medicine (DICOM) format for subsequent evaluation.

The US assessment of lesions covered multiple aspects, including the number, maximum diameter, margin, edge, shape, echogenicity, heterogeneity, presence of cystic areas, calcification, internal vascularity (grade 0: no visible tumor vessels; grade 1: 1-2 isolated vessels; grade 2: 3-5 independently visible vessels; grade 3: >5 identifiable vessels),³² and vascular distributions (central, vascular signals were predominantly detected within the tumor mass; peripheral, vascular signals were primarily observed along the outer margins of the tumors; mixed, both peripheral and central vascular flows were present concurrently, indicating a combination of the 2 aforementioned patterns).³³ Additionally, the status of CLNs was evaluated. Suspicious CLNs typically appeared round, with a long-axis to short-axis ratio of less than 2, absence of a fatty hilum, increased echogenicity, calcification, ill-defined margins, and peripheral or mixed vascular patterns.³⁴ If any of these criteria were met, the US report would classify the CLN as positive. All US images were independently reviewed by Radiologist 1, who had 5 years of experience, and then verified by Radiologist 2, who had 10 years of experience, aiming to minimize potential biases. In cases where the 2 reviewers disagreed, a consensus was reached through collaborative discussions.

Interobserver agreement assessment of US features

The US features of all participants in this study were independently evaluated by 2 radiologists in a double-blind fashion. In order to precisely quantify the diagnostic consistency, an interobserver reliability analysis was conducted using Cohen's κ coefficient.³⁵ The levels of agreement are categorized as follows: a κ value ranging from 0.01 to 0.20 signifies slight agreement; a range of 0.21-0.40 indicates fair agreement; 0.41-0.60 represents moderate agreement; 0.61-0.80 implies substantial agreement; and 0.81-0.99 denotes almost perfect agreement. A κ value ≥0.60 is generally considered acceptable, indicating substantial or higher consistency between observers and ensuring the reliability of the evaluated features.

Image segmentation and feature extraction

All images were imported into ITK-SNAP software (Version 3.8; http://www.itksnap.org). Radiologist 1 manually delineated the region of interest (ROI) on the section with the maximum diameter of each lesion to define the lesion boundaries. One month later, 30 patients were randomly selected, and Radiologist 1 and Radiologist 2 redelineated the ROIs. The intraclass correlation coefficient (ICC) was calculated to assess the consistency of lesion delineation.

Radiomics features were extracted using the Pyradiomics package (https://github.com/Radiomics/pyradiomics) based on Python 3.7, following the guidelines of the Imaging Biomarker Standardization Initiative.³⁶ First, the images were resampled to a pixel size of 1mm×1mm×1mm via B-spline interpolation. Second, grayscale discretization of the histograms was performed with a fixed bin width of 25. Subsequently, original and filter-based features were extracted from each patient's images, encompassing 7 categories: logarithmic, exponential, gradient, square, square root, local binary pattern, and wavelet features. In total, 1239 quantitative radiomic features were extracted and grouped into 7 sets: first-order statistical features, shape features, gray-level co-occurrence matrix (GLCM) features, gray-level run-length matrix (GLRLM) features, gray-level size-zone matrix (GLSZM) features, gray-level dependence matrix (GLDM) features, and neighborhood gray-tone difference matrix (NGTDM) features.

DL features were extracted using the ResNet 50 network pre-trained on the ImageNet database (http://www.image-net.org), with all convolutional layers frozen to preserve pre-learned visual features. During model training, the training and validation sets were kept consistent, employing an initial learning rate of 0.01, 50 epochs, and a batch size of 32, with model parameters fine-tuned using the stochastic gradient descent optimizer. From the output of the average pooling layer of the adjusted ResNet 50 model, 2048 DL features were obtained from the ROI of each patient's US image. ResNet-50 was selected over shallower alternatives (ResNet-18/34) for its superior capacity to capture subtle pathological features in US images. Notably, its pre-training on ImageNet's 1.2 million diverse images was critical for mitigating overfitting risks—an essential consideration given our small dataset size. The batch size of 32, determined after testing smaller sizes (8, 16), balanced statistical representation (stabilizing gradients to avoid overfitting to US noise) and computational feasibility; smaller batches caused training instability with fluctuating validation metrics due to insufficient sample diversity, compromising extraction of subtle pathology-relevant features.

Fusion strategy

We adopted an early fusion strategy to integrate radiomics and DL features prior to final modeling, chosen for 3 key reasons: (1) it better suits our small dataset than end-to-end DL models (which demand large samples) by combining domain-specific radiomics features with data-driven DL features; (2) it preserves inter-feature correlations critical for CLNM detection, whereas late fusion (which merges outputs from separate models) risks losing such synergies; (3) it maintains interpretability through traceable feature contributions, a critical advantage for clinical translation.

Feature selection and construction of radiomics and DL scores

Consistent with the early fusion design, radiomics and DL features were processed separately to preserve their distinct characteristics while enabling subsequent integration. The feature selection process comprised 3 sequential steps: ICC analysis, application of the maximum relevance minimum redundancy (mRMR) algorithm, and implementation of the least absolute shrinkage and selection operator (LASSO) regression algorithm. First, features with an ICC >0.75 were kept and Z-score normalized. Then, the mRMR algorithm refined the selection on the training cohort. Next, LASSO regression with 10-fold cross-validation determined the most predictive features by identifying those with non-zero coefficients. Pearson correlation analysis was conducted to assess relationships both among radiomics features and among DL features. Finally, radiomics scores and DL scores for each patient were computed using a linear combination of the selected features, where each feature was weighted according to its corresponding coefficient obtained from the LASSO model.

Model construction and evaluation

To construct different predictive models, LASSO regression analysis was first conducted on the clinical features, US features, radiomics score, and DL score in the training cohort. This analysis aimed to identify the most relevant parameters for predicting CLNM in MSGCs. Subsequently, the effective parameters obtained from the screening process were utilized to construct 9 distinct predictive models, employing logistic regression (LR) algorithm along with 8 different ML algorithms: decision tree (DT), random forest (RF), adaptive boosting (AdaBoost), extreme gradient boosting (XGBoost), artificial neural network (ANN), support vector machine (SVM), Naive Bayes, and K-nearest neighbors (KNN). The hyperparameter settings for these various ML algorithms are detailed in Supplementary Table 1.

The performance of the models was evaluated by plotting the receiver operating characteristic (ROC) curve. The DeLong test was employed to compare the area under the curve (AUC) values of different models. In addition, metrics such as accuracy, precision, recall, F1 score, and the confusion matrix were utilized to further assess the predictive performance of the models on both the training and the validation cohorts. Through the comparison of predictive efficacy, the model with the best performance was identified. Meanwhile, the LR model was visualized as a nomogram. Finally, decision curve analysis (DCA) and clinical impact curve (CIC) were plotted to determine the clinical effectiveness of the optimal model.

Statistical analysis

Statistical analyses were conducted using IBM SPSS (Version 22.0; https://www.ibm.com/spss), R (Version 4.4.1; https://www.r-project.org), and Python (Version 3.7.0; https://www.python.org). All quantitative variables were first tested for normality using the Shapiro-Wilk test and for homogeneity of variances using Levene's test. For variables following a normal distribution, descriptive statistics were presented as mean ± standard deviation, whereas non-normally distributed variables were reported as median with range. Qualitative variables were analyzed using the Chi-square test and Fisher's exact test. In the case of quantitative variables, 1-way analysis of variance (ANOVA) was used for comparisons among 3 or more independent groups with normally distributed data and homogeneous variances, the Kruskal-Wallis H test was utilized for comparisons among 3 or more independent groups with non-normally distributed data, the independent samples t-test was used for comparisons between 2 independent groups with normally distributed data and homogeneous variances, and the Mann-Whitney U test was employed for comparisons between 2 independent groups with non-normally distributed data. A 2-sided P value < .05 was considered statistically significant. All necessary packages in the R environment could be downloaded from https://cran.r-project.org/web/packages/.

Results

Histological types of patients with MSGCs

The histologic types of MSGCs patients are shown in Supplementary Table 2. Among 214 MSGCs patients, there were 54 mucoepidermoid carcinoma, 39 adenoid cystic carcinoma, 24 lymphoepithelial carcinoma, 21 salivary duct carcinoma, 18 acinic cell carcinoma, 18 carcinoma ex pleomorphic adenoma, 16 adenocarcinoma not otherwise specified, and 24 others. A total of 69 patients (69/214, 32.24%) exhibited pN+. The highest proportion of pN+ was observed in adenocarcinoma not otherwise specified (11/16, 68.8%), followed by salivary duct carcinoma (12/21, 57.1%), lymphoepithelial carcinoma (9/24, 37.5%), and mucoepidermoid carcinoma (18/54, 33.3%). In contrast, adenoid cystic carcinoma and acinic cell carcinoma showed relatively lower pN+ proportions, at 8/39 (20.5%) and 2/18 (11.1%), respectively.

Clinical and US characteristics

Supplementary Table 3 summarizes the clinical and US characteristics of all patients in the study. Assessment of all US features demonstrated substantial interobserver agreement (κ values ranging from 0.748 to 0.867), which meets the acceptable level of consistency (κ ≥ 0.60). No significant differences were observed in any characteristics across the 4 medical centers (all P > .05). Similarly, no significant feature differences were noted between the training and validation cohorts (all P > .05; Supplementary Table 4). As shown in Table 1, significant differences existed between pN0 and pN+ groups in both the training and validation cohorts regarding patient age, tumor maximum diameter, facial nerve dysfunction, margin, edge, shape, calcification, vascularity, and US reported CLN-positive. Additionally, in the training cohort, smoking history, pain symptoms, clinical staging, tumor number, heterogeneity, and cystic areas also showed significant intergroup differences. For other characteristics, no statistical significance was detected (all P > .05). 

Table 1.

Comparison of clinical and US characteristics between the pN0 and pN+ groups in the training and validation cohorts.

Characteristics	Training cohort			Validation cohort
	pN0 (n = 94)	pN+ (n = 50)	P value	pN0 (n = 51)	pN+ (n = 19)	P value
Age, years	47.21 ± 15.54	59.52 ± 16.55	< .001*	45.45 ± 18.25	57.58 ± 13.66	.011*
sex (male/female)	50/44	34/16	.086†	30/21	14/5	.253†
Location (parotid/submandibular)	74/20	33/17	.096†	41/10	15/4	.870†
Smoking history (no/yes)	70/24	21/29	< .001†	39/12	11/8	.126†
Pain symptoms (no/yes)	82/12	26/24	< .001†	40/11	11/8	.086†
Facial nerve dysfunction (no/yes)	92/2	40/10	< .001‡	49/2	15/4	.032‡
Clinical staging (T1-T2/T3-T4)	88/6	26/24	< .001†	45/6	15/4	.546‡
Skin ulceration (no/yes)	94/0	49/1	.347‡	51/0	18/1	.083‡
Number (single/multiple)	89/5	39/11	.002†	48/3	18/1	1.000‡
Maximum diameter, cm	2.28 ± 0.89	3.81 ± 1.42	< .001†	2.46 ± 1.18	3.44 ± 1.29	.004†
Margin (clear/unclear)	72/22	15/35	< .001†	39/12	7/12	.002†
Edge (smooth/rough)	65/29	8/42	< .001†	36/15	7/12	.010†
Shape (regular/irregular)	50/44	8/42	< .001†	23/28	3/16	.025†
Echogenicity (hypoechoic/isoechoic or hyperechoic)	92/2	48/2	.906‡	50/1	19/0	1.000‡
Heterogeneity (homogeneous/heterogeneous)	26/68	5/45	.014†	9/42	2/17	.720‡
Cystic areas (no/yes)	52/42	39/11	.007†	36/15	12/7	.552†
Calcification (no/yes)	84/10	31/19	< .001†	47/4	13/6	.011‡
Vascularity (grade 0-1/grade 2-3)	49/45	10/40	< .001†	32/19	5/14	.007†
Vascular distributions (peripheral/central/mixed)	43/13/38	13/7/30	.052†	21/6/24	8/3/8	.910‡
US reported CLN-positive (no/yes)	85/9	30/20	< .001†	44/7	10/9	.003†

US, ultrasound; pN0, pathological lymph node-negative; pN+, pathological lymph node-positive; CLN, cervical lymph node.

^⁎Independent samples t-test test.

^†Chi-square test.

^‡Fisher’s exact test. P values with statistically significant differences (P < 0.05) are highlighted in bold.

Feature extraction and selection

A total of 1239 radiomics features and 2048 DL features were extracted from the US images of each patient. After excluding features with poor reproducibility (ICC < 0.75), 953 radiomics features and 1618 DL features were retained. Subsequently, the mRMR algorithm was applied to further select 50 radiomics features and 50 DL features, respectively. Finally, 11 radiomics features and 39 DL features were identified via LASSO regression analysis for constructing the radiomics score and DL score (Supplementary Figure 1). The Pearson correlation results showed that the correlation coefficients among the radiomics features and those among the DL features were each below 0.5 (P > .05; Supplementary Figure 2).

Radiomics and DL scores construction

The features selected by LASSO regression and their coefficients are presented in Figures 2A, 2B and Supplementary Tables 5 and 6. The radiomics and DL scores were calculated using the following formulas, where $coefficien{t}_i$ denotes the risk coefficient of each feature derived from the LASSO model and $Featur{e}_i$ represents the expression value of each feature (11 selected radiomics features and 39 selected DL features, respectively): (1) radiomics score: $-0.919+\sum _{i=1}^n\left(coefficien{t}_i\times Featur{e}_i\right)$; (2) DL score:$-1.792+\sum _{i=1}^n\left(coefficien{t}_i\times Featur{e}_i\right)$. Significant differences in both scores were observed between pN0 and pN+ groups in the training and validation cohorts (P < .001; Figures 2C and 2D). ROC curve analysis revealed AUCs of 0.895 and 0.819 for the radiomics score in the training and validation cohorts (Figure 3A), while the DL score achieved AUCs of 0.937 and 0.836 in the training and validation cohorts, respectively (Figure 3B).

Fig. 2.

After dimension reduction by mRMR and LASSO regression, the distribution of radiomics and deep learning features and coefficients, as well as the construction of radiomics and deep learning scores. (A, B) Distribution of radiomics (A) and deep learning (B) features and coefficients. (C, D) Distribution of radiomics scores (C) and deep learning scores (D) in the training and validation cohorts. Note: mRMR, maximum relevance minimum redundancy; LASSO, the least absolute shrinkage and selection operator. ^⁎⁎⁎Statistically significant at P < 0.001 as determined by the Mann-Whitney U test.

Fig. 3.

ROC curves for radiomics and deep learning scores. (A) ROC curves for radiomics scores in the training and validation cohorts. (B) ROC curves for deep learning scores in the training and validation cohorts. Note: ROC, receiver operating characteristic; AUC, area under the curve.

Model construction and evaluation

LASSO regression analysis of clinical and US features, radiomics score, and DL score in the training cohort identified 6 optimal predictors (Figures 4A and 4B): patient age, tumor edge, calcification, US reported CLN-positive, radiomics score, and DL score. Leveraging these features, predictive models were constructed using LR and 8 ML algorithms. In the validation cohort, the AUC values of different models ranged from 0.770 to 0.962. The LR model exhibited the optimal predictive performance, with an AUC of 0.962, accuracy of 0.886, precision of 0.762, recall of 0.842, and an F1 score of 0.8 (Figures 4C and 4D). A comparison of AUC values across models in the validation cohort is shown in Supplementary Figure 3, while predictive performance metrics for all models in both training and validation cohorts are illustrated in Figure 5. Corresponding confusion matrices for each model in the validation cohort are presented in Supplementary Figure 4.

Fig. 4.

Feature variable selection and ROC curves of different models. (A, B) 6 features with non-zero coefficients were selected through LASSO regression; (C) ROC curves for the training cohort; (D) ROC curves for the validation cohort. Note: LASSO, least absolute shrinkage and selection operator; ROC, receiver operating characteristic; AUC, area under the curve; LR, logistic regression; DT, decision tree; RF, random forest; AdaBoost, adaptive boosting; XGBoost, extreme gradient boosting; ANN, artificial neural network; SVM, support vector machine; KNN, K-nearest neighbors.

Fig. 5.

Predictive performance of different models. (A-D) Predictive performance of different models in the training cohort. (E-H) Predictive performance of different models in the validation cohort. Note: LR, logistic regression; DT, decision tree; RF, random forest; AdaBoost, adaptive boosting; XGBoost, extreme gradient boosting; ANN, artificial neural network; SVM, support vector machine; KNN, K-nearest neighbors.

The multivariate regression results of the LR model are shown in Figure 6A, and the LR model was visualized as a nomogram (Figure 6B). The nomogram assigns a score to each relevant feature according to its specific value, with higher scores indicating a greater contribution to the risk of CLNM. By summing these individual feature scores, a total score is generated, which directly corresponds to the probability of CLNM. DCA demonstrated that within most reasonable threshold probability ranges in both the training and validation cohorts, using the nomogram to predict CLNM in MSGCs provided greater clinical benefit to patients compared with strategies of “considering all patients as pN0” or “considering all patients as pN+” (Supplementary Figure 5A). The results of the CIC further confirmed that across most threshold probability intervals, the nomogram effectively stratified patients into high-risk groups (blue curve) and matched the actual risk stratification of the population (light blue curve), demonstrating high clinical utility (Supplementary Figures 5B and 5C).

Fig. 6.

Results of multivariate regression analysis and construction of nomogram. (A) The multivariate regression results of the LR model. (B) Nomogram for the LR model. Note: logistic regression.

Discussion

The presence of CLNM significantly impacts the prognosis of patients with MSGCs by influencing surgical approach selection, recurrence risk, and ultimately patient survival rates.^6,7,11 Therefore, the ability to accurately predict CLNM is critical for tailoring personalized treatment strategies and potentially improving patient outcomes. However, current routine imaging techniques struggle to reliably characterize CLNM, with diagnostic accuracy heavily dependent on radiologists' experience. This underscores the need for novel, more precise methods to identify CLNM, such as radiomics and DL-based approaches. Notably, no prior studies have explored the use of radiomics or DL features for predicting CLNM in MSGCs. This study innovatively developed and validated a composite model integrating clinical and US data, radiomics, and DL features for noninvasive prediction of CLNM in MSGCs. Our findings demonstrate that the LR model incorporating clinical, US, radiomics, and DL features exhibited superior and optimal predictive performance in the validation cohort, achieving an AUC of 0.962—outperforming other ML algorithms. Additionally, DCA and CIC further validated the clinical utility of the model, demonstrating favorable net benefits across a broad range of threshold probabilities.

Through a analysis of clinical characteristics, our study identified that CLNM in MSGCs is significantly associated with several key factors: older patient age, a history of smoking, the presence of pain symptoms, facial nerve dysfunction, and clinical T3/T4 tumor staging. These findings are consistent with the majority of previous reports,8, 9, 10^,37,38 reinforcing established clinical risk factors. Although previous studies have indicated that male patients are associated with a higher risk of CLNM,¹⁹ this correlation was not statistically significant in our cohort, potentially due to sample size limitations or distribution bias. Additionally, our pathological analysis revealed that among the 69 patients with pN+ (32.2%), mucoepidermoid carcinoma (18/69, 26.1%), salivary duct carcinoma (12/69, 17.4%), and adenocarcinoma not otherwise specified (11/69, 15.9%) were the most frequently encountered metastatic subtypes. However, assessment of metastatic propensity within specific subtypes identified adenocarcinoma not otherwise specified (11/16, 68.8%), salivary duct carcinoma (12/21, 57.1%), and lymphoepithelial carcinoma (9/24, 37.5%) as having the highest rates of CLNM, findings consistent with prior studies highlighting their inherent aggressive potential for nodal spread.^39,40 Thus, these findings suggest that selective neck dissection may be beneficial for patients with these histopathological subtypes.

In our analysis of US characteristics, MSGCs patients exhibiting multiple lesions, larger maximum tumor diameter, unclear margins, rough edges, irregular shape, heterogeneous internal echoes, absence of cystic areas, presence of calcification, grade 2-3 vascularity, and US reported CLN-positive were found to be more likely to develop CLNM. Notably, these US features demonstrated excellent interobserver consistency, collectively reflecting the tumor's aggressive biological behavior. Multiple lesions and a larger maximum tumor diameter indicate extensive tumor spread and prolonged growth duration, increasing the likelihood of lymphatic vessel invasion.⁴¹ Unclear margins, rough edges, and irregular shapes suggest an invasive growth pattern, enabling tumor cells to penetrate surrounding tissues and lymphatic vessels.⁴² Heterogeneous internal echoes and the absence of cystic areas reflect complex tumor composition and high cellular density, often indicating a higher histological grade.⁴³ Calcification may signal abnormal tumor metabolism or necrosis, while abundant vascularity (grade 2-3) reflects robust tumor angiogenesis, which not only provides nutrients for tumor growth but also pathways for metastasis.⁴⁴

Recent breakthroughs in radiomics and DL have unlocked novel pathways for radiologists, facilitating non-invasive assessment of hidden features and achieving more accurate tumor profiling.^23,27 Our study demonstrated the efficacy of these techniques in predicting CLNM in MSGCs, with the radiomics score achieving an AUC of 0.895 in the training cohort and 0.819 in the validation cohort. Similarly, the DL score reached AUC values of 0.937 and 0.836 in the training and validation cohorts, respectively.

While various individual features have been associated with CLNM in MSGCs, the current trend towards precision medicine underscores the necessity of integrating multi-dimensional parameters for comprehensive tumor evaluation. Leveraging the LASSO regression, we identified 6 optimal features, including 1 clinical characteristic (patient age), 3 US features (tumor edge, calcification, and US reported CLN-positive), the radiomics score, and the DL score. LASSO regression-based feature selection reduces model complexity, mitigates overfitting, and enhances interpretability by identifying key predictors.⁴⁵ ML is capable of automatically identifying complex patterns and relationships within datasets, thereby enabling efficient model construction and optimization.⁴⁶ In this study, we developed predictive models using LR and 8 ML algorithms. Significantly, the LR model outperformed other ML models in the validation cohort, achieving an AUC of 0.962, accuracy of 0.886, precision of 0.762, recall of 0.842, and an F1 score of 0.8. The optimal performance of the LR model might be attributed to its simplicity, which reduces the risk of overfitting in this dataset, combined with the strong linear separability of the input features.⁴⁷ Meanwhile, we visualized the LR model as a nomogram, which enhances the interpretability and clinical applicability of the prediction model for CLNM risk assessment. Subsequent clinical utility analyses further validated the LR model's practical value. DCA demonstrated that the LR model provided greater net benefits across a wide range of threshold probabilities in both the training and validation sets, outperforming strategies of "treat all" and "treat none." Additionally, CIC analysis confirmed significant net clinical benefits at realistic risk thresholds, underscoring the model's potential for clinical translation and patient management.

This study has certain limitations. First, due to the retrospective research design and the rarity of MSGCs, the sample size is small, which may lead to selection bias. Second, this retrospective study only used conventional US data because the data of multimodal imaging (such as US elastography, contrast-enhanced US, CT, MRI and PET) are incomplete. We believe that further research combining multimodal imaging techniques can show higher predictive performance. Third, defining the boundaries of the ROI may introduce the subjectivity of researchers. In the future, we plan to use DL technology to automatically identify and delineate the ROI and intend to carry out prospective, multicenter studies to further verify the model we proposed. Fourth, the validation dataset is imbalanced in the distribution of pN0 vs. pN+ cases, which could bias the model toward the majority class and compromise performance metrics such as precision and recall. No mitigation techniques (e.g., resampling, class weighting, or use of balanced accuracy) were applied, and this limitation requires further addressing in future research. Additionally, although the proposed model performs well in predicting CLNM of MSGCs, its current implementation requires a multi-step computational workflow (see the methods section for details). To better facilitate clinical translation, the integration of dedicated software is crucial for streamlining the model's accessibility in routine clinical use.

Conclusion

In conclusion, our study demonstrates that the comprehensive model integrating clinical and US data, radiomics, and DL features holds significant clinical potential in predicting CLNM of MSGCs. This AI-driven approach provides clinicians with a non-invasive preoperative diagnostic tool, enabling more accurate prediction of CLNM, thus promoting personalized surgical planning and is expected to significantly optimize treatment strategies.

Authorship contribution statement

Huan-Zhong Su and Long-Cheng Hong contributed to design of the study, acquisition, analysis, interpretation of data, drafting of the article, and review of submitted version of the manuscript and supervision. Zhi-Yong Li and Qiao-Min Fu contributed to acquisition, analysis, interpretation of data. Yu-Hui Wu and Shao-Feng Wu contributed to acquisition and interpretation of data. Zuo-Bing Zhang, Dao-Hui Yang and Xiao-Dong Zhang contributed to acquisition, analysis, interpretation of data, and review of submitted version of the manuscript.

Ethical approval

All procedures involving human participants were conducted in accordance with the ethical standards of the institutional and/or national research committee as well as the 1964 Helsinki Declaration and its subsequent amendments or comparable ethical guidelines, and this study was approved by the Ethics Committees of the First Affiliated Hospital of Xiamen University, Xiamen, Fujian, China (approval number: 2025-003).

Informed consent

The need for informed consent was waived because of the retrospective nature of this study.

Funding

This study was financially supported by the Fujian Provincial Natural Science Foundation of China (No. 2023J011617, No. 2022J011366), the Natural Science Foundation of Xiamen, China (No. 3502Z202372078), and the Xiamen Medical and Health Guidance Project (No. 3502Z20214ZD1010, No. 3502Z20244ZD1018). The funders did not influence the study design, data collection, analysis, decision to publish or preparation of the manuscript.

Conflict of interest

None disclosed.