Nomogram based on ultrasound radiomics score and clinical variables for predicting histologic subtypes of epithelial ovarian cancer

Abstract

Objective:

Ovarian cancer is one of the most common causes of death in gynecological tumors, and its most common type is epithelial ovarian cancer (EOC). This study aimed to establish a radiomics signature based on ultrasound images to predict the histopathological types of EOC.

Methods:

Overall, 265 patients with EOC who underwent preoperative ultrasonography and surgery were eligible. They were randomly sorted into two cohorts (training cohort: test cohort = 7:3). We outlined the region of interest of the tumor on the ultrasound images of the lesion. Then, the radiomics features were extracted. Clinical, Rad-score and combined models were constructed based on the least absolute shrinkage, selection operator, and logistic regression analysis. The performance of the models was evaluated using receiver operating characteristic curves and decision curve analysis (DCA). A nomogram was formulated based on the combined prediction model.

Results:

The combined model had good performance in predicting EOC histopathological types, with an AUC of 0.83 (95% CI: 0.77–0.90) and 0.82 (95% CI: 0.71–0.93) in the training and test cohorts, respectively. The calibration curves showed that the nomogram estimation was consistent with the actual observations. DCA also verified the clinical value of the combined model.

Conclusions:

The combined model containing clinical and ultrasound radiomics features showed an excellent performance in predicting type I and type II EOC.

Advances in knowledge:

This study presents the first application of ultrasound radiomics features to distinguish EOC histopathological types. The proposed clinical-radiomics nomogram could help gynecologists non-invasively identify EOC types before surgery.

Introduction

Ovarian cancer is one of the most common causes of death in gynecological cancers, ¹ and its most common type is epithelial ovarian cancer (EOC). ² The majority of patients with EOC have poor outcomes due to difficulties obtaining an early diagnosis for this aggressive type of tumor. ³ The gold standard therapy for EOC patients is maximal cell reduction surgery, followed by chemotherapy. ^4–6 However, 80% of the patients ultimately recrudesced. ⁷ The histological classification of EOC is important for determining prognosis. ⁸ EOCs were subdivided into types I and II by Kurman and Shih based on their respective morphologic characteristics, molecular features, and clinical manifestations. ^9,10 According to the World Health Organization (WHO), type I EOC comprises low-grade serous carcinoma, clear cell carcinoma, endometrioid carcinoma, mucinous carcinoma, and malignant Brenner tumors. It has a relatively inert course with good prognosis. It is also insensitive to traditional chemotherapy. In contrast, type II EOC comprises high-grade serous carcinoma, undifferentiated carcinoma, and carcinosarcoma. It is highly invasive with a poor prognosis. It is also sensitive to standard chemotherapy. ^10–12 Therefore, preoperative prediction of the EOC histological classification is crucial for developing a precise treatment and determining prognosis.

In recent years, radiomics features have provided a non-invasive preoperative method for the objective evaluation of tumor heterogeneity. ¹³ Clinicians can use radiomics features to improve diagnostic accuracy. ¹⁴ Previous studies have shown that radiomics features based on computed tomography (CT) or magnetic resonance imaging (MRI) can differentiate the histopathological types of EOC and predict their prognosis. ^15–18 Previous studies found that combining radiomics score (Rad-score) models based on MRI could improve their ability to differentiate EOC types. ^18–20 Pelvic ultrasonography is a commonly available and inexpensive technique for diagnosing EOC in clinical practice. Ultrasound radiomics has been used to research various types of cancers, such as liver and cervical cancers. ^21–23 It was also used by Nero et al to predict the BRCA gene 1–2 status. ²⁴ Hence, we hypothesized that radiomics features on ultrasonography could show tumor heterogeneity and predict the histopathological types of EOC.

This study aimed to predict the histopathological types of EOC using a radiomics signature based on ultrasound images. Furthermore, we compared the performance of the Rad-score, clinical, and combined models in predicting the histopathological types of EOC.

Methods and materials

Patients

This retrospective study received approval from our Institutional Review Board with a waiver of informed consent. Patients were histologically confirmed to have EOC. They underwent preoperative ultrasound and surgery between April 2004 and August 2020 at the First Affiliated Hospital of Wenzhou Medical University. The inclusion criteria were as follows: a) pathologically confirmed EOC; b) transvaginal ultrasound performed within two weeks before surgery. The exclusion criteria were as follows: a) neoadjuvant radiotherapy or chemotherapy before surgery; and b) presence of other cancers. Overall, 265 patients were eligible and were randomly sorted into two cohorts (training cohort: test cohort = 7:3, Figure 1).

Figure 1.

Flowchart of patient selection.

Image acquisition, tumor segmentation, and features extraction

All the procedures were performed according to the Image Biomarker Standardization Initiative (IBSI) standards. Ultrasound images were acquired by Hitachi or Philips ultrasonic equipment with linear probes of 5.0–7.5 MHz and saved in DICOM format for further analysis. The images were acquired using different ultrasound scanners, and we normalized each image using resampling and gray-level discretization. Since the ultrasound images were two-dimensional images with a thickness of 1 mm, we resampled the ultrasound images according to the voxel 1*1*1 mm. Then, we performed an intensity standardization and discretized the gray level in the range of 0–255.

The ITK-SNAP software (http://www.itksnap.org) was used to outline the tumor region of interest (ROI). In the ROI segmentation process, a radiologist with five years of experience selected the most representative image with the largest solid components and segmented the ROIs. Then, to measure interobserver reproducibility, a total of 100 images were randomly selected to be re-segmented by a senior radiologist with 20 years of experience. An interclass correlation coefficient (ICC)>0.75 indicates a high feature stability. Discrepancies were solved through consultation. The radiologists were unaware of the specific histopathological type when delineating the ROI.

To extract the radiomics features, we imported the original images and the ROIs into the A.K. software (Artificial Intelligence Kit, version 3.0.0, GE Healthcare). Each patient’s original image and ROI were automatically matched one by one. Before features extraction, we set the fixed bin width (default = 10). A total of 1130 features were extracted, comprising seven main categories: (1) first-order statistics; (2) shape-based; (3) gray-level co-occurrence matrix (GLCM); (4) gray-level run-length matrix (GLRLM); (5) gray-level size zone matrix (GLSZM); (6) neighboring gray-tone difference matrix (NGTDM), and (7) gray-level dependence matrix (GLDM). The z-score was standardized for the extracted features. A flowchart of the study is shown in Figure 2.

Figure 2.

Study flowchart. GLCM: gray-level co-occurrence matrix, GLRLM: gray-level run-length matrix, GLSZM: gray-level size zone matrix, NGTDM: neighboring gray-tone difference matrix, GLDM: gray-level dependence matrix, LASSO: least absolute shrinkage and selection operator, ROC: receiver operating characteristic.

Establishment of the Rad-score model

The features with high stability were selected to implement the Rad-score model. Then, univariate logistic analysis and the least absolute shrinkage and selection operator (LASSO) were used to select the potential predictive factors in the training cohort. Ten-fold cross-validated models were derived to avoid overfitting the data. Finally, a total of 17 non-zero coefficient radiomics features were used to establish the Rad-score. The optimal Rad-score cut-off value was selected using Youden’s index method in the training cohort.

Establishment of the clinical and combined models

We used seven candidate clinical data parameters to build the clinical model; these parameters comprised age, carcinoembryonic antigen (CEA), alpha-fetoprotein (AFP), CA125, CA199, white blood cells (WBCs), and albumin. All clinical data were obtained from the electronic medical records of the hospital. In the training cohort, univariate and multivariate logistic regression analyses were used to compare the differences in the clinical features of types I and II EOCs. The features with a significant difference (p < 0.05) were selected to establish the clinical model.

The combined model was built using Rad-score and clinical features. The receiver operating characteristic (ROC) curve and the associated area under the curve (AUC) were used as metrics to quantify the performances of the three models. A nomogram was established based on a combined prediction model.

Statistical analysis

All analyses were performed using the SPSS software package (v. 25.0; IBM Corp., Armonk, NY, USA) and R software (v. 4.0.2; https://www.R-project.org). We used the t-test or Mann-Whitney U-test to evaluate the differences between type I and type II EOCs. Chi-square or Fisher’s exact tests were used to analyze the categorical variables. The predictive performance of the nomogram in both the training and test cohorts was evaluated using a calibration curve. The DeLong test was used to compare the AUCs of the different models. Decision curve analysis (DCA) was used to determine the clinical usefulness of each model. ²⁵ The results with p < 0.05 were considered statistically significant.

Results

Study population and baseline characteristics

Of the 265 patients (mean age 56.39 ± 11.03) enrolled in this study, 67 (25.3 %) patients had type I EOC, and 198 (74.7 %) had type II EOC. The ratio of type I EOC to type II EOC was approximately 1:3, consistent with what Kurman et al described. ⁹ The detailed patient baseline characteristics are listed in Table 1. There were no significant differences in the baseline characteristics between the training and test cohorts (p > 0.05). In the training cohort, age, CA125 levels, and CA199 levels were significantly different between types I and II EOC. The CA125 level in type I EOC patients was lower than that in type II EOC patients (p = 0.004). However, the CA199 level in type I EOC patients was higher than that in type II EOC patients (p = 0.004).

Table 1.

Comparison of patients' clinical characteristics of between the training and test cohorts

Characteristics	Training cohort (n = 186)	Test cohort (n = 79)	P
Age	56.5 (45.9–67.2)	56.0 (44.1–68.0)	0.734
CA125 (U/mL)			0.555
<35	26.0 (14.0%)	14.0 (17.7%)
>35	160.0 (86.0%)	65.0 (82.3%)
CA199 (U/mL)			1.000
<35	160.0 (86.0%)	68.0 (86.1%)
>35	26.0 (14.0%)	11.0 (13.9%)
CEA (ng/mL)			0.606
<5	169.0 (90.9%)	74.0 (93.7%)
>5	17.0 (9.1%)	5.0 (6.3%)
AFP (ng/mL)			0.794
<9	181.0 (97.3%)	78.0 (98.7%)
>9	5.0 (2.7%)	1.0 (1.3%)

AFP, alpha-fetoprotein; CEA, carcinoembryonic antigen.

Model construction and evaluation

The clinical model included the age, CA125 levels, and CA199 levels. For the Rad-score model, a total of 1130 features were extracted from the ROIs, and the median ICC was 0.987 (0.966–0.995). Good stability (ICC>0.75) was noted in 976 features. The Rad-score model comprised 17 ultrasonographic radiomics features. The details of the features are shown in Supplementary 1. In contrast, the combined model incorporated age, CA125, and CA199 with the Rad-score. The performances of the models are shown in Table 2 and Figure 3. In the training cohort, the AUCs of the clinical, Rad-score, and combined models were 0.76, 0.76, and 0.83, respectively. In the test cohort, the AUCs of the clinical, Rad-score, and combined models were 0.73, 0.74, and 0.82, respectively. The predictive efficiencies of the Rad-score and clinical models were not significantly different in the training (AUC [95% CI]: 0.76 [0.68–0.83] versus 0.76 [0.68–0.84], p = 0.953) and test cohorts (AUC [95% CI]: 0.74 [0.60–0.87] versus 0.73 [0.60–0.85], p = 0.910). Compared with the single Rad-score and clinical models, the combined model showed a higher AUC. In the training cohort, the combined model was more efficient than the single Rad-score (AUC [95% CI]: 0.83 [0.77–0.90] versus 0.76 [0.68–0.83], p = 0.031) or clinical model (AUC [95% CI]: 0.83 [0.77–0.90] versus 0.76 [0.68–0.84], p = 0.016). In contrast, in the test cohort, the AUC for the combined model was higher than that for the Rad-score or clinical model, although the differences did not reach statistical significance (p = 0.162 and 0.088, respectively).

Table 2.

Model performance of the clinical model, Rad-score model, and combined model

	AUC (95% CI)	Accuracy	Sensitivity	Specificity	PPV	NPV
Training cohort
Clinical model	0.76 (0.68–0.84)	0.67	0.65	0.76	0.89	0.42
Rad-score model	0.76 (0.68–0.83)	0.77	0.83	0.60	0.86	0.54
Combined model	0.83 (0.77–0.90)	0.77	0.78	0.77	0.91	0.54
Test cohort
Clinical model	0.73 (0.60–0.85)	0.67	0.63	0.80	0.90	0.42
Rad-score model	0.74 (0.60–0.87)	0.77	0.83	0.60	0.86	0.55
Combined model	0.82 (0.71–0.93)	0.77	0.75	0.85	0.94	0.53

AUC: area under the curve, CI: confidence interval, NPV: negative predictive value, PPV: positive predictive value, Rad-score: radiomics score.

Figure 3.

(A) Receiver operating characteristic curves of the Rad-score, clinical, and combined models in the training cohort. (B) Receiver operating characteristic curves of the Rad-score, clinical, and combined models in the test cohort. Rad-score: radiomics score.

The optimal Rad-score cut-off (−0.172) was selected using Youden’s index. The Rad-score model’s accuracy, sensitivity, specificity, positive-predictive value (PPV), and negative-predictive value (NPV) used to distinguish type I EOC from type II EOC were 0.77, 0.83, 0.60, 0.86, and 0.54, respectively, in the training cohort, and 0.77, 0.83, 0.60, 0.86, and 0.55, respectively, in the test cohort. Compared with the clinical model, the Rad-score model had better sensitivity and NPV. The combined model had similar accuracy as the Rad-score model and was better than the clinical model.

The clinical-radiomics nomogram (Figure 4A) provided a visual outcome measure of the combined model. The calibration curves (Figure 4B and C) showed that the nomogram estimation was consistent with the actual observation in the training and test cohorts. This finding confirmed the high accuracy of the nomogram. Finally, the DCA (Figure 5) showed that the combined model had a higher net benefit than the clinical or Rad-score models.

Figure 4.

(A) A clinical-radiomics nomogram was developed for predicting the probability of type II EOC with age, CA125, CA199, and Rad-score. To use this nomogram, the position of each variable on the corresponding axis is determined, and then, a line is drawn straight up to the points axis to obtain the score. Each score is added, and the total score is located on the total score axis. The total score is derived by calculating the scores of each variable and locating it on the total score axis. Then, the risk of type II EOC is determined at the lowest line of the nomogram. (B, C): Calibration curves analysis for the nomogram in predicting type II EOC in the training and test cohorts. The 45° dashed line represents a perfect prediction, the dotted line represents the actual performance by the nomogram in the training and test cohorts, and the solid lines represent bias correcting estimates employing 1,000 bootstrap sampling.

Figure 5.

Decision curve analysis for three models. The y-axis indicates the net benefit, and the x-axis indicates threshold probability. The black line represents the hypothesis that no patients are type II EOC. The gray line represents the hypothesis that all patients are type II EOC. The dark green, dark blue, and red lines represent the net benefit of the clinical, Rad-score, and combined models, respectively. Among the most range of threshold probabilities, the combined model had higher net benefit than the clinical and Rad-score models.

Discussion

This study built and validated models to predict EOC types using ultrasound image radiomics features and significant clinical variables. To the best of our knowledge, this study was the first application of ultrasound radiomics features to distinguish the histopathological types of EOC. Our analysis showed that the clinical, Rad-score and combined models could predict the histopathological types of EOC. The combined model provided the most efficient performance.

In our study, patients with type I EOC were younger than type II patients. This finding was consistent with other studies. ^19,26,27 Tumors in younger females tended to be more inert, while tumors were more aggressive in older females. ²⁸ Gershenson et al found through a retrospective study that the low-grade serous carcinoma (type I EOC) appeared at a younger age than the high-grade (type II EOC). ²⁹ Moreover, CA125, a mucinous glycoprotein present on the surface of ovarian tumor cells, was an independent predictor of the EOC histopathological types. Patients with type I EOC had lower CA125 levels than those with type II, as reported in multiple studies. ^26,30–34 Among the subtypes of EOC, high-grade serous carcinoma (type II EOC) had a higher expression of CA125. ³⁵ In contrast, CA199 was a glycolipid on the cell membrane and a mucin tumor marker. ³⁶ Some studies have shown that CA199 was elevated in patients with ovarian cancer and that CA199 levels were associated with prognosis in these patients. ^37–39 In our study, patients with type I EOC had higher CA199 levels than type II patients.

This study found that the Rad-score containing ultrasound image features was an independent factor for predicting EOC type. Various studies have focused on computer-aided identification of the type of ovarian cancer. ^18–20,40 Zhang et al studied the possible relationship between MRI radiomics features and histopathological types of tumors in a sample of 286 patients with EOC. They found that radiomics features from MRI were highly correlated with EOC classification. ¹⁸ Although some studies have reported that CT and MRI can distinguish EOC classification, no ultrasound Rad-score models have been proposed for the same purpose. Recently, ultrasound radiomics has been used in other cancers. ^41–45 Peng et al reported a potential association between ultrasound radiomics features and biological characteristics in 128 patients with intrahepatic cholangiocarcinoma. ⁴¹ Furthermore, a previous study used an ultrasound Rad-score model to predict the five-year disease-free survival of patients with EOC; the Rad-score model had good performance. ⁴⁶ In this study, a total of 17 radiomics features were used to build the Rad-score, including nine features of first-order statistics, three of GLSZM, two of GLRLM, two of GLCM, and one of NGTDM. Most of these features showed variability, non-uniformity, and heterogeneity of the lesions. For example, “LowGrayLevelRunEmphasis” measured the distribution of low gray-level values, with a higher value indicating a more uniform pixel in the image. Strength was an NGTDM, which was higher in an image with larger coarse differences in gray-level intensities. Moreover, features of GLSZM measured the texture homogeneity of lesions, and features of the GLRLM group reflected the roughness. ^47,48 All these parameters characterized the tumor heterogeneity, possibly reflecting the genomic heterogeneity. Type II EOC had a higher genetic instability and genomic heterogeneity than type I. ⁹ Therefore, using ultrasound radiomics features was a reliable way to differentiate type I and type II EOC. However, it was difficult to interpret the association between a single ultrasound radiomics feature and the underlying pathological characteristics. Thus, the Rad-score proposed by this study combines multiple ultrasound radiomics features in a multielement parameter that simplified the complexity of multicharacteristic research. ^22,49,50 In addition, the ICC was used to evaluate interobserver agreements. A high ICC value showed high stability. The features extracted from the ROIs segmented by the two radiologists (five vs 20 years of experience) were stable (ICC: 0.987 [0.966–0.995]). The Rad-score was a stable and straightforward multifactor parameter that could provide a convenient and reliable tool in clinical practice to differentiate type I and type II EOCs, as shown in this study.

This study had some limitations that warrant discussion. First, this was a retrospective study using ultrasound images acquired by different radiologists using different types of equipment. The ultrasound images' signal-to-noise ratio and resolution were different because different devices were used to save them. Therefore, the results could have been affected, although the images were normalized. Moreover, there was a potential selection bias in the study for two reasons: (1) the patients were selected from an extended period (from April 2004 to August 2020); (2) patients who did not undergo pathological diagnosis were excluded. Hence, a prospective study should be conducted to minimize these confounding variables and enhance the differentiating performance of radiomics features. Second, the AUC for the combined model was higher than that for the Rad-score or clinical model, but the differences were not significant in the test cohort. The reason for this finding might be that the study was conducted in single-center data with a relatively small size sample, and the performance of the models could be unstable. Thus, multicenter studies with a larger sample size are needed to validate the models in the future. Third, radiologists manually delineated the ROIs on a two-dimensional representative tumor image, and these images might not represent the entire lesion adequately. However, this limitation was inherent to the retrospective design of this study using ultrasound images.

In conclusion, ultrasound radiomics features could differentiate type I and II EOCs. The combined model, which comprised both clinical and ultrasound radiomics features, could offer better efficiency in distinguishing the two types of EOC in the training cohort. The clinical-radiomics nomogram might serve as a visual and personalized tool to help gynecologists non-invasively identify EOC types before surgery.