Investigation of multimodal deep learning models for predicting ovarian tumor malignancy based on ultrasound images and clinical information – a comprehensive comparative study against readers and O-RADS

Abstract

Background

As the second deadly cancer affecting women globally, precise and timely classification of ovarian tumors plays an instrumental role in improving the rate of curing and reducing the rate of mortality. This study was set out to comprehensively investigate the effectiveness of deep learning model for classifying benign and malignant ovarian tumors, utilizing multimodal ultrasound images and clinical data, in comparison to traditional methods such as manual assessment by radiologists and those based on O-RADS.

Methods

This retrospective multicenter study recruited women diagnosed with ovarian tumors between January 2022 and June 2023, with histopathological examination results as the reference diagnoses. The dataset was divided into three subsets: training (70%), validation (10%), and test (20%). Employing the Dense Convolutional Network algorithm, we constructed and investigated two fusion models: DL_M2F, integrating multimodal features extracted ultrasound (grayscale ultrasound, color Doppler flow imaging), and DL_M3F, integrating DL_M2F with clinical data (e.g. age, CA125, CA199, HE4, SCC, ROMA index, menopausal state, and mass volume). The outcome measure was the area under the receiver operating characteristic curve (AUC). We compared the models’ performance in the test dataset against both radiologists, O-RADS and single-mode models.

Results

A total of 508 patients with ovarian tumors (mean age: 44.3 ± 15.9 years) were enrolled, including 327 benign and 181 malignant tumors. In the test set, the DL_M2F model demonstrated an AUC of 0.919, sensitivity of 0.865 and specificity of 0.879, while the DL_M3F model showed an AUC of 0.951, sensitivity of 0.865 and specificity of 0.939. Comparatively, radiologists scored AUC of 896 (Expert level III) and 0.827 (Expert level I), while O-RADS was able to achieve an AUC of 0.835. Evaluation of confusion matrices revealed that DL_M3F model exhibited almost identical accuracy as a level III expert, demonstrating its promising potential as an clinical diagnostic tool to assist junior radiologists.

Conclusion

The deep learning model integrating multimodal ultrasound images and clinical information is capable of discriminating between benign and malignant ovarian tumors, exceeding the diagnostic capabilities of both radiologists and O-RADS assessments.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12880-026-02312-4.

Background

Ovarian cancer is ranked as the second leading cause of cancer-related mortality among women globally, with a five-year survival rate below 45% [1–4]. Early diagnosis of ovarian tumors is crucial for enabling doctors to make informed decisions about optimal treatment strategies for each patient [5–7]. For benign tumors, conservative treatment is recommended to prevent unnecessary overtreatment and to preserve fertility, whereas malignant tumors necessitate accurate clinical staging followed by the design of personalized effective treatment options that may involve radical surgery.

Ultrasound (US) represents a convenient and cost-effective imaging modality for both the screening and identification of ovarian tumors, thus becoming the preferred method for clinical examination [8, 9]. Significant inter-observer variability in the assessment of tumors among radiologists has been documented [10–12]. Consequently, the most recent consensus guidelines from the American College of Radiology have introduced the Ovarian-Adnexal Reporting and Data System (O-RADS) to standardize the approach towards disease stratification and management [13, 14].

In recent years, the development of computer-aided diagnostic tools in radiology has significantly advanced due to the widespread application of deep learning (DL) technologies. Utilizing deep convolutional neural networks facilitates effective feature learning through a nonlinear network structure, which in turn reduces the number of network parameters and mitigates the risk of overfitting [15, 16]. Deep convolutional neural networks have been previously employed for diagnostic applications in thyroid and breast cancer. However, to date, studies on the utilization of DL models for evaluating ovarian tumors, especially those making use of multi-modal US images, have been limited [17, 18]. In clinical practice, ovarian cancer diagnosis utilizes various US imaging techniques, such as two-dimensional grayscale US, color Doppler flow imaging (CDFI), and power Doppler imaging. A recent study [19] proposed a DL algorithm that incorporated grayscale US and power Doppler US to distinguish between benign and malignant ovarian tumors. This algorithm exhibited a sensitivity of 96% and a specificity comparable to expert evaluations (86.7% vs. 88%; p > 0.99). However, the study focused solely on US imaging data and overlooked the clinical information which also plays a critical role in ovarian tumor diagnosis.

To this end, this study aims to develop a DL classification model integrating multiple sources of patient information, such as 2D grayscale US, CDFI, and clinical information (age, CA125, CA199, HE4, SCC, ROMA index, menopausal state, and mass volume), to improve the diagnostic accuracy and precision of ovarian tumors.

Methods

The Ethics Committee of the First Affiliated Hospital of Zhejiang Chinese Medical University (also known as Zhejiang Provincial Hospital of Chinese Medicine), overseeing the multicenter retrospective study, granted approval for this study and waived the requirement for written informed consent (2023-KLS-267-01).

Participants and datasets

Data collection was conducted at three institutions between January 2022 and June 2023: (1) The First Affiliated Hospital of Zhejiang Chinese Medical University; (2) Zhejiang Cancer Hospital; and (3) Hangzhou First People’s Hospital. Patients satisfying the following inclusion criteria were recruited: (1) at least one adnexal mass (excluding physiological cysts) detected via US; (2) surgery was completed within 30 days following the imaging; 3)a definitive diagnosis of ovarian tumor based on relevant pathological examination; and 4) exhibited no prior history of ovarian cancer. Histopathological diagnoses were established according to the routine clinical protocols of each participating institution, which reflects standard clinical practice and provides an objective reference standard for model training. Patients who were diagnosed with histopathologically confirmed uterine sarcoma or non-ovarian tumors as well as patients with indeterminate pathological findings, poor-quality US images and incomplete clinical information were excluded from this study.

US imaging data, including two-dimensional grayscale US and CDFI of the lesions, were collected. Furthermore, clinical data, encompassing age, laboratory results (CA125, CA199, HE4, SCC, ROMA index), menopausal state, volume and tissue pathological findings, were gathered from the patients’ electronic health record. In total, the data encompassed 508 ovarian tumors, comprising 327 benign tumors and 181 malignant tumors. Only one nodule per patient was included; for patients with multiple nodules, only the most complex mass was selected. All ultrasound images (including different modalities) and clinical data from the same patient were kept entirely within the same subset.

All US examinations were conducted using five US machines: GE Voluson E10, GE Voluson E8, LOGIQ E9 (GE Healthcare), Philips IU22, and Philips A70 (Baisheng Healthcare), equipped with 5.0–9.0 MHz, 4.0–8.0 MHz, and 3.0–10.0 MHz transvaginal probes, respectively, as well as 1.0–5.0 MHz transabdominal probes.

Evaluation method

Each mass was routinely assessed using standardized terminology and sonographic features defined by the International Organization for the Analysis of Ovarian Tumors [20]. The O-RADS risk stratification and management system classifies ovarian tumors into six categories based on morphological characteristics [21]. Stratification for every patient was performed by experienced gynecological ultrasound experts with certified O-RADS interpretation skills, including two Level II (≥ 2000 gynecological US examinations) and two Level III (spend the majority of their time undertaking gynecological ultrasound and/or teaching, research and development in the field) practitioners [22].

Subjective evaluation was conducted by two Level III and two Level I experts. A Level I expert is defined as one who has performed a minimum of 300 examinations under the supervision of a Level II practitioner or an experienced Level I practitioner with at least 2 years of regular practical experience. These experts combined patients’ clinical information and US expertise for a comprehensive subjective evaluation, and were blinded to the pathological results. Each case required a minimum of 10 min for image interpretation, and to prevent fatigue, no radiologist performed more than one hour of continuous review without a break. The radiologists’ subjective evaluations were performed using the same set of preprocessed and standardized static images (including key grayscale and color Doppler sections) that were used for model training and testing.

Diagnostic procedures

In this study, all masses were initially categorized based on pathological findings into two subcategories: benign and malignant. The reference standard was the histopathological diagnosis obtained from post-surgical resection, conducted in the pathology departments of the participating hospitals in accordance with World Health Organization guidelines [23]. Tumor classification adhered to the criteria recommended by the International Federation of Gynecology and Obstetrics [24]. The data were randomly segmented into a training set, validation set, and test set, based on the number of tumors, with a ratio of 7:1:2. A total of 354 cases were allocated for model training, 51 cases for model validation, and 103 cases for model testing (Fig. 1a).

Fig. 1.

Illustration showing the overall framework of this study. (a) Data collection process. (b) In the multi-modal model with multi-source feature fusion, grayscale ultrasound, color Doppler flow imaging, and clinical information served as three inputs. The grayscale ultrasound and color Doppler flow imaging were preprocessed and fed into the DenseNet121 module to obtain feature maps, which were then concatenated (shown in panel (c) of the figure). After Global Average Pooling (GAP) and before the fully connected layer (FC), these concatenated feature maps were further connected with the preprocessed clinical information to ultimately make benign (B) and malignant (M) classification predictions. (c) The figure presents detailed information on the network architecture of the DenseNet121 module

The image processing procedure is depicted in Fig. 1b. Upon acquiring the original grayscale US and CDFI, Labelme software was utilized to process the ovarian tumor US images and delineate the tumor regions. Benign tumors were labeled as 0, whereas malignant tumors were labeled as 1. The four radiologists responsible for delineating the tumor regions all had five years of experience in gynecological US, adhering to a standardized annotation protocol, and they were all unaware of the pathological outcomes.

To enhance the model’s robustness, the region of interest (ROI) containing the lesion was enlarged by a factor of 1.3. This enlargement improved the model’s ability to capture more internal image details and critical features. During model training, an imbalance in the distribution of pathological types was observed, significantly affecting the performance of DL models. Data augmentation has been identified as an effective method to address such imbalance. Utilizing data augmentation techniques, such as random cropping and contrast adjustment, enables the generation of additional synthetic samples within the training set. This enhances category representation and mitigates class distribution imbalance among samples, thereby improving the model’s generalization capabilities and classification performance. To meet the DL model’s input requirements, processed images were adjusted to a standardized resolution of 224 × 224 pixels and normalized for network training, validation, and testing.

Clinical factors, including age, ovarian lesion volume, menopausal state, and tumor markers (CA125, CA199, HE4, SCC ROMA index), were extracted from patients’ examination reports. As per clinical guidline, the ovarian mass volume was approximated by multiplying its length, width, and height. To further meet the DL model’s input requirements, imputed clinical information was encoded and normalized by converting it into numerical values ranging from 0 to 1. All continuous variables were normalized to the [0, 1] interval using min-max normalization, with the normalization parameters estimated solely from the training set and then applied unchanged to the validation and test sets. This processed data was subsequently stored for network training, validation, and testing purposes.

DL model construction

In this study, we have developed two multimodal models for feature fusion based on densely connected convolutional networks (referred to as DenseNet), incorporating different modes of feature fusion: Tri-Modal Fusion (referred to as DL_M3F) and Bi-Modal Fusion (referred to as DL_M2F). DenseNet121 was selected based on its proven effectiveness in medical image analysis, particularly its dense connectivity pattern that promotes feature reuse, mitigates vanishing gradients, and reduces parameter count—advantages that are valuable in moderate-sized medical datasets.The DL_M3F model integrated grayscale US, CDFI, and clinical information to classify benign and malignant ovarian tumors. In contrast, the DL_M2F model only utilized grayscale US and CDFI. In the proposed DL_M3F and DL_M2F models, the DenseNet121 network is utilized for feature extraction from both grayscale US and CDFI. The DenseNet121 model is pretrained on large-scale ImageNet dataset to acquire comprehensive image feature representations, and subsequently employs transfer learning for the classification of ovarian tumor US images. This approach facilitates accelerated model training and enhances classification accuracy. The grayscale US images and CDFI generate 7 × 7 feature maps through the learning process of DenseNet121. Subsequently, we concatenate these two feature maps along the channel dimension and apply global average pooling to transform them into linear features. For the DL_M3F model, clinical information was incorporated as an input node in the fully connected layer, and the clinical features were fused with the imaging features using a concatenation fusion strategy. The output probability for predicting the benign or malignant nature of each ovarian mass was then calculated based on both clinical and imaging features. Meanwhile, we evaluated the accuracy of model predictions by using a cross-entropy loss function. The proposed DL_M3F model consisted of 7.56 M parameters and processed the entire test set (103 cases) in 15.3 s, averaging 0.15 s per case.

In addition, we performed ablation experiments to assess the efficacy of feature fusion manipulations by comparing conventional US unimodal model, CDFI unimodal model, clinical information unimodal models, DL_M2F, and DL_M3F. The clinical information unimodal models include two classic machine learning classifiers based on clinical factors: a logistic regression model (Logistic) and a multilayer perceptron model (MLP). The optimization algorithm “Adam” was employed in this paper to standardize the hyperparameters of each experimental group, enabling a more effective comparison of the impacts of different networks and training mechanisms on classification. Specifically, the initial learning rate was set at 0.001, while the momentum value was fixed at 0.9. Each experiment underwent training for a total of 60 epochs, with a Batch size of 128 and a Dropout value of 0.5. The models presented in this article were implemented on Ubuntu 20.04 using the Keras2.6.0 and TensorFlow-GPU 2.6.0 frameworks, while utilizing an Intel(R) Core(TM) i9-13900k@3.0G processor and an NVIDIA GeForce RTX 3070 Ti with 8GB of GPU.

Statistical analysis

The area under the receiver operating characteristic curve (ROC) was utilized in this study to assess the DL model’s capability to discriminate between ovarian tumors, as reflected by sensitivity and specificity levels. Additionally, the positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1-score were also computed to evaluate the DL model’s performance. Furthermore, the DeLong method was employed to calculate 95% confidence intervals (CI) for each metric. Statistical analyses were conducted using Python, Matplotlib, and Scikit-learn libraries. Additionally, Landis’s standard [25] was used to calculate the kappa coefficient, evaluating the consistency between model-predicted results and the corresponding pathological assessment (i.e. gold standard). All calculated P values for the 95% CI were found to be statistically significant, with a significance level below 0.05.

Results

Characteristics of the patients

A total of 508 patients with ovarian tumors (mean age: 44.3 ± 15.9 years) were retrospectively enrolled in this study, comprising 327 benign tumors and 181 malignant tumors. To construct the predictive model, we categorized the lesions into three sets: a training set (228 benign, 126 malignant cases), a validation set (33 benign, 18 malignant cases), and a test set (66 benign, 37 malignant cases). The demographic and clinical characteristics of the tumors in the training, validation, and test data sets are detailed in Table 1.

Table 1.

The demographic and clinical characteristics of patients in training, validation, and test sets

Characteristic	Training and validation Sets	Test Set
No. of tumors	405	103
Age* (years)	43.6 ± 15.8 (13–85)	47.0 ± 16.1 (20–85)
Histologic
Benign	261(64.4%)	66(64.1%)
Malignant	144(35.6%)	37(35.9%)
Laboratory indicators positive
CA-125	171(42.2%)	35(34.0%)
CA-199	91(22.5%)	15(14.6%)
HE4	89(22.0%)	20(19.4%)
SCC	89(22.0%)	4(3.9%)
ROMA index	117(28.9%)	32(31.1%)
Mass volume* (mm3)	261613.3 ± 571043.9 (280-5543482)	266587.0 ± 560411.1 (560-2935152)
In a postmenopausal state	113(27.9%)	40(38.8%)
Color Doppler score
No flow, score 1	224(55.3%)	58(56.3%)
Minimal flow, score 2	120(29.6%)	32(31.1%)
Moderate flow, score3	47(11.6%)	10(9.7%)
Very strong flow, score4	14(3.5%)	3(2.9%)

Note: The training, validation, and test sets comprised 508 tumors. Mean data are presented as ± standard deviation (SD). Categorical variables are presented as numbers, with data in parentheses indicating percentages. CA-125 represents cancer antigen 125, CA-199 denotes cancer antigen 199, HE4 signifies Human epididymis protein 4, and ROMA refers to the ROMA Index. * Data in parentheses represent interquartile ranges (IQRs)

Performance of DL algorithms

As illustrated in Table 2; Fig. 2, it was found that among all models, the DL_M3F model utilizing grayscale US, CDFI, and clinical information showed the highest predictive power in distinguishing benign and malignant ovarian cancer, with an AUC of 0.951 (0.909, 0.993). The DL_M3F model showed superior performance than the other five models, especially in terms of accuracy, specificity and PPV. The AUC of the single-mode DenseNet121 prediction model was compared to the AUC of the overall algorithm, the DL_M2F model, which incorporated both modalities. The AUC values for the model trained solely on either the grayscale US (DL_US) image or the CDFI (DL_CD) were 0.894 (0.834, 0.953) and 0.892 (0.832, 0.952), respectively. The DL_M2F model successfully classified the lesions based on the distinctive features of two types of US images, yielding an AUC of 0.919 (0.866, 0.972), thus surpassing both the DL_US model and the DL_CD model. Additionally, both the DL_M3F and DL_M2F models demonstrated performance that surpassed expert subjective assessment and O-RADS evaluation. The AUC values of the clinical information unimodal models, Logistic and MLP, were 0.887 (0.825, 0.948) and 0.869 (0.804, 0.934), respectively.

Table 2.

Diagnostic Performance of different methods for Classifying Benign and Malignant Ovarian Tumors in the Test Set

	AUROC (95% CI)	Accuracy (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	F1	Kappa
DLM3F	0.951 (0.909, 0.993)	0.913 (0.858, 0.967)	0.865 (0.755, 0.975)	0.939 (0.882, 0.997)	0.889 (0.786, 0.992)	0.925 (0.862, 0.988)	0.877	0.809
DLM2F	0.919 (0.866, 0.972)	0.874 (0.810, 0.938)	0.865 (0.755, 0.975)	0.879 (0.800, 0.958)	0.800 (0.676, 0.924)	0.921 (0.854, 0.987)	0.831	0.731
DLUS	0.894 (0.834, 0.953)	0.845 (0.775, 0.915)	0.838 (0.719, 0.957)	0.848 (0.762, 0.935)	0.756 (0.625, 0.888)	0.903 (0.830, 0.977)	0.795	0.670
DLCD	0.892 (0.832, 0.952)	0.816 (0.741, 0.890)	0.730 (0.587, 0.873)	0.864 (0.781, 0.946)	0.750 (0.609, 0.891)	0.851 (0.765, 0.936)	0.740	0.597
Logistic	0.887 (0.825, 0.948)	0.825 (0.752, 0.899)	0.838 (0.719, 0.957)	0.818 (0.725, 0.911)	0.721 (0.587, 0.855)	0.900 (0.824, 0.976)	0.775	0.633
MLP	0.869 (0.804, 0.934)	0.806 (0.729, 0.882)	0.784 (0.651, 0.916)	0.818 (0.725, 0.911)	0.707 (0.568, 0.847)	0.871 (0.788, 0.954)	0.744	0.588
Expert level III	0.896 (0.837, 0.955)	0.913 (0.858, 0.967)	0.838 (0.719, 0.957)	0.955 (0.904, 1.005)	0.912 (0.816, 1.007)	0.913 (0.847, 0.980)	0.873	0.807
Expert level I	0.827 (0.754, 0.900)	0.854 (0.786, 0.922)	0.730 (0.587, 0.873)	0.924 (0.860, 0.988)	0.844 (0.718, 0.970)	0.859 (0.778, 0.940)	0.783	0.674
O-RADS	0.835 (0.763, 0.906)	0.864 (0.798, 0.930)	0.730 (0.587, 0.873)	0.939 (0.882, 0.997)	0.871 (0.753, 0.989)	0.861 (0.781, 0.941)	0.794	0.694

Fig. 2.

Comparison of ROC curves for all prediction methods (DL_M3F: tri-modal fusion; DL_M2F: bi-modal fusion; DL_US: DL model based on ultrasound imaging alone; DL_CD: DL model based on CDFI; Logistic: logistic regression model based on clinical data alone; MLP: multilayer perceptron model based on clinical data alone; Expert level III: expert with level 3 US experience; Expert level I: expert with level 1 US experience; O-RADS: Ovarian-Adnexal Reporting and Data System)

DL performance compared with experts and O-RADS

We compared the DL_M3F model with Expert Level III, Expert Level I, and O-RADSD diagnosis in benign and malignant ovarian cancer detection. The AUC of DL_M3F was not statistically significantly different from that of Expert level III, and was numerically higher than that of Expert level I and O-RADS diagnosis. However, no significant difference was observed between DL_M3F and Expert level III diagnosis (p = 0.111), but there were significant differences between DL_M3F and Expert Level I (p = 0.001), similarly between DL_M3F and O-RADS diagnosis (p = 0.007). The diagnostic level of the DL_M3F model has reached the Expert level III, and the diagnostic sensitivity is better. Compared to expert level I and O-RADS diagnosis, The accuracy (0.913 vs. 0.854 vs. 0.864), sensitivity (0.865 vs. 0.730 vs. 0.730), PPV (0.889 vs. 0.844 vs. 0.871) and NPV (0.925 vs. 0.859 vs. 0.861) of DL_M3F were improved (Table 2). Statistical comparisons of AUC, Accuracy, Sensitivity, PPV, and NPV among the four diagnostic methods are presented in Fig. 3. Figure 4 displays the corresponding confusion matrices, visually confirming the superior classification performance of DL_M3F over alternative methods.

Fig. 3.

Statistical comparison of the diagnostic ability of DL_M3F, experts and O-RADS (DL_M3F: tri-modal fusion; DL_M2F: bi-modal fusion; Expert level III: expert with level 3 US experience; Expert level I: expert with level 1 US experience; O-RADS: Ovarian-Adnexal Reporting and Data System)

Fig. 4.

Comparison of confusion matrices among the investigated models against expert subjective assessments and O-RADS evaluation (0: benign; 1: malignant; DL_M3F: tri-modal fusion; DL_M2F: bi-modal fusion; DL_US: DL model based on ultrasound imaging alone; DL_CD: DL model based on CDFI; Logistic: logistic regression model based on clinical data alone; MLP: multilayer perceptron model based on clinical data alone; Expert level III: expert with level 3 US experience; Expert level I: expert with level 1 US experience; O-RADS: Ovarian-Adnexal Reporting and Data System)

In this study, we also assessed the models using calibration curve and DCA. The results demonstrate that the DL_M3F model shows the best calibration performance, with its calibration curve closely aligning with the ideal diagonal line across the entire probability range and the Brier score for DL_M3F was 0.082 (Supplementary Figure S1). The DCA results (Supplementary Figure S2) show that the DL_M3F model provides the highest net benefit among all compared methods over a wide range of clinically relevant threshold probabilities (approximately 0.2–0.7). Notably, at the commonly used 50% threshold, the net benefit of DL_M3F is comparable to that of Expert Level III assessment, while significantly outperforming less experienced evaluators and O-RADS.

Visual interpretation of the DL model

The heat maps corresponding to the US images of benign and malignant ovarian tumors are given in Supplementary Figure S3. The different color distributions reflect the model’s focus on the most predictive regions of the US images. The red portion of the image provides crucial information for accurately determining the highlighted areas within the image model, thus aiding in prediction processes. The results indicate that for accurately predicted ovarian tumors, the red regions depicted in the heatmaps were predominantly located within the nodule itself and in areas of rich vasculature; thereby, the use of heatmaps enhances the interpretability of the model.

Discussion

The field of DL has gathered increasing attention in the domain of clinical diagnostic imaging of ovarian cancer. In this study, two DL models were developed, and their diagnostic performances were systematically compared against subjective assessments made by radiologists and the O-RADS assessment. The AUC values of our proposed DL method in the test set were found to be: DL_M2F 0.919 (95%CI: 0.866, 0.972), and DL_M3F 0.951 (95%CI: 0.909, 0.993). These results indicate a high level of diagnostic accuracy (0.913, 95%CI: 0.858, 0.967), which surpasses subjective expert (level I) assessment (0.854, 95%CI: 0.786, 0.922) and O-RADS risk classification (0.864, 95%CI: 0.798, 0.930).

Several DL techniques have been developed for the analysis of medical images [26–28], and convolutional neural networks have gained widespread usage in lesion detection and classification tasks [29–32] in recent years. We have chosen to utilize DenseNet for our research due to its implementation of a denser network connectivity mode, resulting in a significant reduction in parameter count and enhanced feature propagation. This enables the extraction of more comprehensive image feature information [33] and provides the radiologists with quantitative features that are imperceptible to the human eye as a tool to aid physicians. The AUC of DL_M3F outperformed that of O-RADS and subjective assessment by experts. These findings suggest that DL can be leveraged to assist radiologists, achieving comparable diagnostic performance as experts even for those with limited experience. Furthermore, given that US is the best imaging tool for screening, the DL_M2F model which integrates grayscale US and CDFI demonstrates potential as a screening tool for ovarian lesions in the general population. However, the clinical relevance of this study is primarily for patients being considered for surgery. We therefore developed the DL_M3F model, which incorporates clinical information in addition to the DL_M2F framework, serving as an auxiliary decision-making tool for ovarian lesion assessment in gynecological oncology.

Most previously developed models for assessing ovarian tumors have been trained on a single type of US image [34, 35]. A few studies conducted at single center with two US mode models have also been reported, and their effectiveness surpasses that of the single mode model [36]. Among them, neural convolutional networks have demonstrated high accuracy in the early detection of malignant ovarian tumors using CDFI. However, the present study entailed a multicenter investigation encompassing subjects with diverse histopathological types of both malignant and benign ovarian lesions. The images were acquired by different sonographers using diverse equipment. The heterogeneity in terms of case and image ensured the reproducibility of the DL model, and the data analysis encompassed a wide range of variables, thereby enhancing the generalizability and reliability of the conclusions drawn and facilitating better training and validation of our model. Moreover, in clinical practice, the diagnosis of ovarian cancer relies on utilizing various types of US images in combination with clinical features. Therefore, our research primarily focused on developing a multi-source multimodal fusion model for automatically classifying ovarian tumors. The DL_M2F model combines two modalities of US and demonstrates better performance to expert assessment. Building upon these two US modalities, the DL_M3F model incorporates additional clinical characteristics (age, CA125, CA199, HE4, SCC, ROMA index, menopausal state, and mass volume), surpassing the traditional ADNEX model in terms of included modalities [37] which could supplement the ADNEX model algorithm.The diagnostic performance of the DL_M3F model in our study was superior to that of the DL_M2F model, expert subjective evaluation, and O-RADS evaluation.

There are several limitations of this study. First, this study is retrospective in nature, which may introduce inherent limitations such as selection bias, spectrum bias, and residual confounding, thereby restricting generalizability. And retrospective designs may not reflect real-time clinical decision-making. In addition, although the study is multicenter, all participating institutions are located within a single geographic region (China), which limits ethnic, demographic, and practice-pattern diversity and may affect external validity. Future prospective studies with larger, geographically diverse, and multi-center cohorts are warranted to further validate the robustness and applicability of the proposed approach. Secondly, there is a need for expanding the utilization of US imaging modalities. The incorporation of contrast-enhanced US and three-dimensional US can enhance the image characteristics related to tumor microcirculation, thereby enabling the development of DL models to improve their clinical applicability. Third, although histopathology was used as the reference standard, inter-pathologist variability and tumor subtype heterogeneity may introduce labeling uncertainty, which could affect model performance. Fourth, the simplification of tumors into a binary classification cannot fully encompass borderline entities and clinically relevant subtypes. Fifth, Tumor segmentation relies on manual ROI delineation, which is time-consuming and subject to inter-observer variability and no quantitative assessment of inter- or intra-observer agreement for ROI annotation is provided. And radiologist assessments may be influenced by experience-related variability.

Conclusions

The deep learning model, which integrates multi-modal US imaging data, including grayscale US and CDFI, along with clinical information such as tumor markers and patient history, demonstrates superior accuracy in distinguishing benign from malignant ovarian tumors. Its performance surpasses that of both seasoned radiologists and the standardized O-RADS evaluations, offering a more precise and reliable diagnostic tool.

Supplementary Information

Below is the link to the electronic supplementary material.

Abbreviations

AUC

Area Under the Receiver Operating Characteristic Curve

DL

Deep Learning

O-RADS

Ovarian-Adnexal Reporting and Data System

CDFI

Color Doppler Flow Imaging

CA-125

Cancer Antigen 125

CA-199

Cancer Antigen 199

HE4

Human Epididymis Protein 4

ROMA

ROMA Index

DLM3F

Tri-Modal Fusion

DLM2F

Bi-Modal Fusion

PPV

Positive Predictive Value

NPV

Negative Predictive Value

CI

Confidence Intervals

GAP

Global Average Pooling

FC

Fully Connected Layer

Author contributions

Guarantors of integrity of entire study: L.L., C.C., Y.Zhou, D.X.; Study concepts/study design or data acquisition or data analysis/interpretation: All authors; Manuscript drafting or manuscript revision for important intellectual content: All authors; Approval of final version of submitted manuscript: All authors; Agrees to ensure any questions related to the work are appropriately resolved: All authors; Conceptualization: L.L.; Methodology: L.L., C.C.; Writing - original draft: L.L.;Writing - review & editing: C.C., V.Y.W.; Investigation: Q.Z, Z,Y, Jin, Q.C., M.Zhu, Y.W., C.Ma; Visualization: Y.Zhou; Funding acquisition: D.X.; Project administration: D.X.; Supervision: D.X.

Funding

None.

Data availability

The datasets generated and/or analysed during the current study are not publicly available due privacy but are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

The research was conduct in accordance with the Declaration of Helsinki and was approved by The Ethics Committee of the First Affiliated Hospital of Zhejiang Chinese Medical University (also known as Zhejiang Provincial Hospital of Chinese Medicine), overseeing the multicenter retrospective study, granted approval for this study and waived the requirement for written informed consent (2023-KLS-267-01).

Consent for publication

Due to the nature of the retrospective study, written informed consent was waived by the Institutional Review Board.

Competing interests

The authors declare no competing interests.