Clinical value of a radiomics model based on machine learning for the prediction of prostate cancer

Abstract

Objective

Radiomics models have demonstrated good performance for the diagnosis and evaluation of prostate cancer (PCa). However, there are currently no validated imaging models that can predict PCa or clinically significant prostate cancer (csPCa). Therefore, we aimed to identify the best such models for the prediction of PCa and csPCa.

Methods

We performed a retrospective study of 942 patients with suspected PCa before they underwent prostate biopsy. MRI data were collected to manually segment suspicious regions of the tumor layer-by-layer. We then constructed models using the extracted imaging features. Finally, the clinical value of the models was evaluated.

Results

A diffusion-weighted imaging (DWI) plus apparent diffusion coefficient (ADC) random-forest model and a T2-weighted imaging plus ADC and DWI multilayer perceptron model were the best models for the prediction of PCa and csPCa, respectively. Areas under the curve (AUCs) of 0.942 and 0.999, respectively, were obtained for a training set. Internal validation yielded AUCs of 0.894 and 0.605, and external validation yielded AUCs of 0.732 and 0.623.

Conclusion

Models based on machine learning comprising radiomic features and clinical indicators showed good predictive efficiency for PCa and csPCa. These findings demonstrate the utility of radiomic models for clinical decision-making.

Introduction

Prostate cancer (PCa) is the second-most common cancer in men. In 2020, 1,414,259 cases of PCa were newly diagnosed worldwide, an incidence that was second only to lung cancer, and there were 375,304 deaths owing to PCa. ¹ From a therapeutic perspective, the combination of abiraterone and prednisolone with androgen deprivation therapy (ADT) should now be considered the standard treatment for patients with high-risk non-metastatic PCa. However, in the presence of metastasis, it is recommended that enzalutamide and abiraterone should not be used in combination for patients initiating long-term ADT. Notably, the addition of abiraterone to ADT is associated with clinically significant improvements in survival that persist for >7 years. ² It has been agreed internationally that the early identification of PCa can improve patient survival, and urinary liquid biopsy shows promise for the detection of PCa. In addition, serum biomarkers that might be useful for precision medicine include androgen receptor variants, those related to bone metabolism and the neuroendocrine system, and metabolites. For patients with bone metastases, bone sialoprotein (BSP) and osteopontin (OPN) have prognostic value. High BSP concentrations are associated with more rapid bone metastasis in patients with PCa, and OPN may help with the assessment of the response to chemotherapy in patients with castration-resistant PCa. ³

Prostate-specific antigen (PSA) is the most widely used marker for the early detection of PCa. ⁴ Large clinical studies have shown that patients who are screened for PCa have a 44% lower mortality rate than those who are not screened over a median 14-year median period of monitoring.^5–7 However, PSA-based screening may also be associated with the overdiagnosis and overtreatment of PCa around the world. Because prostate biopsy remains the gold-standard method of diagnosing PCa, most patients with high PSA concentrations have to undergo biopsy to confirm their diagnosis. However, this is an invasive procedure that can be uncomfortable and is associated with serious potential complications, such as infection, hemorrhage, hematuria, hemospermia, urinary retention, and erectile dysfunction. ⁸ Moreover, negative results of prostate biopsy are common (58.5%–69.3%), especially in patients with high PSA concentrations but no corresponding symptoms.^9–10 Therefore, there is an urgent need to identify a model that can predict PCa more reliably than PSA, to reduce the incidence of unnecessary prostate biopsy.

Even patients with positive biopsy results do not necessarily require immediate treatment. Large randomized clinical trials have shown that for a large proportion of patients with PCa, a high PSA concentration alone has little significance for clinical treatment, ¹¹ few of their tumors show invasive growth and eventually cause death. Therefore, the concept of clinically significant PCa (csPCa) has been proposed. The definition of csPCa is based on a Gleason score (GS) ≥3 + 4 ± a tumor volume ≥0.5 cm³ ± extracapsular extension (ECE). CsPCa requires treatment, owing to the high incidence of malignancy. However, patients with clinically non-significant PCa (ciPCa) only require monitoring. ¹² Therefore, effective means of distinguishing between csPCa and ciPCa could reduce the incidences of unnecessary treatment and the adverse effects of medication in patients with PCa.

Recently, alongside rapid developments in the field of machine learning, the use of radiomics has also developed rapidly. Computer algorithms can be used to extract a large number of radiomic features of tumors from medical images and transform black-and-white medical images into quantifiable multidimensional data, which can be used to describe tumor biology. Numerous studies have demonstrated the feasibility of constructing radiomic models based on magnetic resonance imaging (MRI) and other clinical data for the prediction of bone metastasis in patients with PCa.^13–14 Fan et al. have shown that radiomic models based on multiparameter (MP)-MRI have the potential to predict biological characteristics and could serve as noninvasive methods for the risk stratification of patients with PCa. ¹⁵ Nevertheless, it is unclear whether the radiomic features extracted from prostate MR images could be used to predict PCa and csPCa before prostate biopsy.

We aimed to conduct a large two-center study to establish and evaluate the usefulness of a model for the prediction of PCa and csPCa before prostate biopsy. To this end we compared the tested modeling algorithms to identify the optimal model for these two scenarios, and developed predictive software that can be used to facilitate diagnoses for individual patients.

Materials and methods

Study design and participants

We performed a retrospective study of patients with suspected PCa at two institutions: the Fujian Medical University Union Hospital (Center 1; between January 2014 and February 2023) and the Changhai Hospital of the Second Medical Military University (Center 2; between June 2017 and November 2022). The detailed inclusion and exclusion criteria are shown in Figure 1. Of the patients recruited for the prediction of PCa, those with pathologically confirmed benign prostate lesions were excluded, and the remaining patients were studied with respect to the prediction of csPCa. A flow chart of patient selection for these groups is shown in Figure 1.

Figure 1.

Flow chart of patient selection for the two groups.

In accordance with ethical guidelines and patient confidentiality requirements, all the patient details have been thoroughly de-identified to protect their privacy. No identifiable information was used in the study. The reporting of this study conforms to the TRIPOD guidelines.^16–17 The study was conducted in accordance with the principles of the Declaration of Helsinki, and was approved by the Institutional Review Board of Fujian Union Hospital of Fujian Medical University (protocol code: 2022KY138). The necessity for patient consent was waived by the Institutional Review Board, owing to the retrospective nature of the study.

Volume of interest (VOI) segmentation

The data obtained from MR images (specific imaging parameters are shown in the Supplemental materials) was anonymized, leaving only the T2-weighted imaging (T2WI), apparent diffusion coefficient (ADC), and diffusion-weighted imaging (DWI) (b = 1500 s/mm²) to establish the VOI. Two experienced radiologists jointly evaluated the quality of the MR images and located the edges of lesions that were suspicious for PCa. Any disagreement with respect to the location of the focus and the edges was resolved by discussion with senior radiologists. After determining the locations and edges of the lesions, and under the supervision of radiologists, one researcher used 3D Slicer (4.11.20210-226) software (https://www.slicer.org/) to segment the region of interest (ROI) layer by layer on the T2WI, ADC, and DWI sequences. Subsequently, each layer of the ROI was fused and reconstructed to form the VOI, which was then used for feature extraction.

Radiomic feature extraction

The open-source Python package pyradiomics (v.3.0.1) was used to extract the radiomic features from the VOI of each sequence. Seven types of radiomic features were extracted: 14 shape features, 378 first-order features, 504 gray-level co-occurrence matrix features, 294 gray-level dependence matrix features, 336 gray-level run length matrix features, 336 gray-level size zone matrix features, and 105 neighboring gray-tone difference matrix features. Consequently, 5901 features were extracted from each patient’s prostate MR images.

Radiomic model construction and evaluation

Patients from Center 1 were divided into the training and internal validation sets at a ratio of 7:3, and those from Center 2 were used for external validation. The features derived from the T2WI, ADC, and DWI sequences were combined in seven ways after being individually analyzed (T2WI, ADC, DWI, T2WI + DWI, T2WI + ADC, ADC + DWI, and T2WI + ADC + DWI), and these combinations were further combined with clinical indices. Univariate analysis was performed for each combination to exclude parameters that were not statistically significant. Student’s t-test was used for data that was normally distributed and the Mann–Whitney U test was used for those that were not normally distributed. Subsequently, the least absolute shrinkage and selection operator (LASSO) algorithm was used to remove redundant and collinear features. For the groups used for the prediction of PCa and csPCa, owing to the imbalance in sample size between groups, the synthetic minority oversampling technique (SM-OTE) algorithm of the imbalanced-learn package (v.0.6.0) was used to oversample in the training set.

From among the machine learning algorithms in the scikit-learn package (v.1.0.2) and xgboost package (v.1.5.1), 12 algorithms were used to establish models using seven feature combinations and the training set. In this way, 84 models were established for each of the group used for PCa prediction and the subgroup used for csPCa prediction. A flow chart of the model creation and validation is provided in the Supplemental material (Supplementary Fig. 1).

Statistical analysis

Python (v.3.7.11; Fredricksburg, VA, USA) and R software (v.4.1.2; www.r-project.org) were used for data analysis and graph creation. The models generated were used to predict the outcomes in the internal and external validation sets for each group. The accuracy, area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were calculated. On the basis of the accuracy and AUC of the models in the external validation set, we constructed a stem plot of the predictive accuracy for each model. The DeLong test was used to compare the AUCs of the various models. Subsequently, decision curve analysis (DCA) was used to compare the net clinical benefit of each model. Finally, the best model for each group was selected and its performance was compared with the radiologists’ report.

Results

Patient characteristics

A total of 942 patients were recruited (432 at Center 1 and 510 at Center 2) and those from Center 1 were allocated to two groups. The first group was used for the prediction of PCa, and 287 patients were diagnosed with PCa (66.4%). The second group was used for the prediction of csPCa, and 235 patients were diagnosed with csPCa (81.9%). The remaining 510 patients, recruited at the Changhai Hospital of the Second Medical Military University, were used for the external validation of the data collected from the two groups. In Center 1, 11.57% of patients were under 60 years old, 79.86% were between 60 and 80 years old, and 8.56% were over 80 years old. In Center 2, 16.47% of patients were under 60 years old, 75.29% were between 60 and 80 years old, and 8.24% were over 80 years old. The clinical characteristics of the patients in the training, internal validation, and external validation sets are shown in the Supplemental material (Supplementary tables 1 and 2.)

Selection of the best predictive model for each group

We combined the features of the T2WI, ADC, and DWI sequences to obtain seven combinations (Supplementary Fig. 1A). These combinations were modeled using 12 algorithms (Supplementary Fig. 1C), then the accuracy, AUC, sensitivity, and specificity of each model was verified using the internal and external verification sets. Finally, 84 models were arranged in descending order of AUC based on the results obtained using the external validation set (Supplementary Fig. 2). For each group, we selected the models with the highest accuracy and AUC for this set for use in subsequent evaluations (Figure 2).

Figure 2.

Parameters evaluated in the predictive models. (a) PCa and (b) csPCa. The accuracy, AUC, sensitivity, and specificity of the selected models are shown. The models are arranged in descending order of AUC for the external validation set. AUC, area under the curve.

For the selected models, we further evaluated their AUCs using the DeLong test. No significant differences in AUC among the top three models for the prediction of PCa were identified (Supplementary Fig. 3a). Similar findings were obtained for the models for the prediction of csPCa. Therefore, we performed DCA to compare the net benefits of the top two models for use in the clinic, to select the optimal model (Figure 3). The results of the LASSO process to identify the best model for the prediction of PCa and csPCa are shown in Supplementary Fig. 4 and 5.

Figure 3.

Decision curve analysis curves for the predictive models. (a) PCa and (b) csPCa. The net benefit of each model is plotted against the threshold probability for PCa, compared with the strategy of treating all patients or none. The decision curves show that the DWI + ADC random forest model and the T2WI + ADC + DWI MLP model have the highest net benefits for the prediction of PCa and csPCa, respectively. ADC, apparent diffusion coefficient; csPCa, clinically significant prostate cancer; DWI, diffusion-weighted imaging; MLP, multilayer perceptron; PCa, prostate cancer; T2WI, T2-weighted imaging.

In the group used for the prediction of PCa, the DWI + ADC random forest model was found to be optimal and consisted of PSA and six radiomic features (Figure 4(a)). In the subgroup used for the prediction of csPCa, the T2WI + ADC + DWI multilayer perceptron (MLP) model was far superior to the other models, and consisted of 35 radiomic features (Figure 4(b)). The prediction accuracy and AUC of the two best predictive models are shown in Table 1.

Figure 4.

Features included in the optimal model for each group. (a) PCa and no PCa groups and (b) csPCa and ciPCa groups. ADC, apparent diffusion coefficient; ciPCa, clinically insignificant prostate cancer; csPCa, clinically significant prostate cancer; DWI, diffusion-weighted imaging; MLP, multilayer perceptron; PCa, prostate cancer; T2WI, T2-weighted imaging.

Table 1.

Accuracy and AUC of the optimal model for each group.

Group	Model	Training set		Internal validation set		External validation set
		Accuracy	AUC	Accuracy	AUC	Accuracy	AUC
PCa	DWI + ADC random forest model	89.31%	0.942	86.16%	0.894	73.92%	0.732
csPCa	T2WI + ADC + DWI MLP model	99.39%	0.999	78.16%	0.605	89.76%	0.623

ADC, apparent diffusion coefficient; AUC, area under the curve; csPCa, clinically significant prostate cancer; DWI, diffusion-weighted imaging; MLP, multilayer perceptron; PCa, prostate cancer; T2WI, T2-weighted imaging.

Comparison of the results generated using the models and the interpretations of the radiologists

We compared the performance of the radiologists and the optimal radiomic model for the prediction of PCa by calculating the accuracy, sensitivity, specificity, and true positive, true negative, false positive (FP), and false negative rates (Table 2). Compared with the interpretations of the radiologists, the radiomic model reduced the misclassification of PCa by 62.5% in the internal validation set.

Table 2.

Predictive performance of the radiologist and radiomic model for the internal validation set.

Parameter	Radiologist	Radiomic model	P-value
Accuracy	63.1%	86.2%
Sensitivity	0.874	0.839	0.754
Specificity	0.140	0.907	<0.0001
TP	76	73
TN	6	39
FP	37	4
FN	11	14
Reduction of misclassification	62.5%

FN, false negative; FP, false positive; PCa, prostate cancer; TN, true negative; TP, true positive.

Software design and demonstration

Finally, we designed software to extract radiomic features and predict PCa and csPCa using the optimal models. The user interface of the software and prediction result for a 60-year-old patient with a PSA concentration of 11.47 ng/mL and GS 6 are shown in Figure 5. The patient’s risk of cancer was calculated to be 82.3%, but the radiologists’ diagnosis did not take into account the possibility of malignant tumor. We also provided the MR images and clinical data, and established the optimal predictive model for six patients, so that readers would be able to reproduce the research findings. A video demonstration of this software is provided in the Supplementary material (Video S1).

Figure 5.

User interface of the software and patient demonstration. (a) Images of a 60-year-old patient with a PSA concentration of 11.47 ng/mL and GS 6. These show a lesion in the left peripheral zone of the prostate. Pathologic examination of the biopsy confirmed the presence of prostate cancer and (b) the software is used to extract radiomic features and predict disease based on the selected model. The prediction generated was consistent with the results of pathologic examination. GS, Gleason score.

Discussion

We have constructed predictive models using the radiomic and clinical features of patients to distinguish between those with PCa and non-PC, and between those with csPCa and ciPCa, before they underwent prostate biopsy. For many years, MP-MRI has been considered to be the best means of diagnosing and staging PCa, because of its high soft tissue contrast, high resolution, and synchronous imaging.^18–19 However, there is a great deal of variation in the imaging reports generated using this method, owing to subjective reader factors. ²⁰ Radiomics is a research hotspot in the field of imaging because it minimizes this interobserver variability and permits the assessment of more predictors than visual analysis. ²¹ However, many studies performed to date have been of relatively small samples and few radiomic features, which affects the credibility of their excellent results. In the present study, we aimed to overcome these limitations.

To the best of our knowledge, this is the first time that such a large dataset, comprising data collected from 942 patients with suspected PCa from two centers, prior to prostate biopsy, has been used to create radiomics models that predict PCa. The generation of precise assessments of the influence of independent features requires data relating to a sufficient number of events to be collected per feature. Specifically, multiple regression analysis requires a minimum of 10 to 15 observations per feature to be made to yield dependable estimates. ²² Hence, the advantage of the present large-scale study is that it was possible to construct predictive models using a large number of features, which has allowed us to identify more potential predictors for clinical use.

Many previous studies have shown the efficacy of radiomic models for the prediction of prostate cancer. However, radiomics has many outstanding challenges, such as a lack of a link between imaging signals and molecular biology, the management of the potential for overfitting, owing to the creation of large feature sets from limited patient data, and the prior reduction of dataset dimensionality. ²³ Qi et al. ²⁴ found that the AUCs associated with a radiomic model based on T2WI and ADC sequences for the prediction of PCa were 0.828 and 0.853, respectively, for the internal validation set. In addition, in another study of 203 patients, a radiomic model was constructed using a combination of T2WI, ADC, and DCE sequences, and the AUC for the internal validation set was 0.861. ²⁵ In the present study, the best radiomic model identified for the prediction of PCa consisted of the DWI and ADC sequences, and AUCs of 0.942 and 0.894 were obtained for the training and internal validation sets, respectively, which are much better than the previously reported values. We also found that this radiomic model could reduce the incidence of misclassification by radiologists (by 70.1% and 62.5% for the training and internal validation sets, respectively), implying that this radiomic model should help radiologists reliably identify PCa before biopsy.

Differences in the radiomic features of the various pathologic types of PCa may be the result of differences in cell differentiation, cell density, and the intercellular space between high- and low-grade PCa tissues. These differences in radiomic features can be used to distinguish csPCa from ciPCa, ²⁶ and patients diagnosed with ciPCa can choose active surveillance rather than biopsy to avoid the potential complications of this procedure. ¹⁰ A previous study of 206 patients in which a radiomic model based on the T2WI + DWI sequence was developed yielded an AUC for the identification of csPCa of 0.816. ²⁷ Furthermore, Min et al. ²⁸ established a logistic regression model based on the T2WI, ADC, and DWI sequences for the diagnosis of csPCa that yielded AUCs of 0.872 and 0.823 for the training and verification sets, respectively. In the present study, an MLP model based on the T2WI, ADC, and DWI sequences showed the best performance for the prediction of csPCa of the many models tested, with an accuracy of 99.4% and an AUC of 0.999 for the training set, which is consistent with the findings of these previous studies. However, this radiomic model can more accurately predict csPCa, and therefore patients with PCa can avoid unnecessary treatment in a clinical setting.

We have identified two useful predictive models for PCa and csPCa before prostate biopsy. In addition, we have developed software to facilitate prediction using these models that should assist clinicians with the formulation of individualized treatment strategies in clinical practice and reduce the incidences of unnecessary biopsy and treatments, thereby improving the quality of care delivered to patients.

The present study had some limitations. First, because it was a retrospective study, there may have been bias in the selection of the patients. Therefore, we will conduct a larger prospective study in the future, with the aim of improving the balance and robustness of the predictive model. Second, the ROI was manually segmented, and therefore an influence of subjective factors was inevitable. In the future, artificial intelligence could be used for automatic ROI segmentation, so as to reduce interference by subjective factors. Finally, because of the poor interpretative ability of machine algorithms during the process of selecting radiomic features and modeling, it is difficult to explain the biologic relationships between these features and the outcomes. An in-depth study of the relationships between the characteristics of cancer tissue at the cellular level and the radiomic features will be one of the directions of our future research.

Conclusions

Various radiomic models have been established for clinical purposes and these have been based on multiple types of machine learning. In the present study, we have established two optimal predictive models that are based on radiomic features and can predict PCa and csPCa before prostate biopsy. In combination with the software designed alongside this research, these should assist clinicians to design individualized treatment strategies for patients with PCa in clinical practice, thereby avoiding unnecessary biopsy and treatment. Without increasing the burdens placed on clinicians and patients, these models may therefore facilitate the creation of more appropriate treatment strategies for patients with PCa.

Supplementary material

Supplemental material for this article is available online.