Pre-Operative Prediction of Advanced Prostatic Cancer Using Clinical Decision Support Systems: Accuracy Comparison between Support Vector Machine and Artificial Neural Network

Abstract

Objective

The purpose of the current study was to develop support vector machine (SVM) and artificial neural network (ANN) models for the pre-operative prediction of advanced prostate cancer by using the parameters acquired from transrectal ultrasound (TRUS)-guided prostate biopsies, and to compare the accuracies between the two models.

Materials and Methods

Five hundred thirty-two consecutive patients who underwent prostate biopsies and prostatectomies for prostate cancer were divided into the training and test groups (n = 300 versus n = 232). From the data in the training group, two clinical decision support systems (CDSSs-[SVM and ANN]) were constructed with input (age, prostate specific antigen level, digital rectal examination, and five biopsy parameters) and output data (the probability for advanced prostate cancer [> pT3a]). From the data of the test group, the accuracy of output data was evaluated. The areas under the receiver operating characteristic (ROC) curve (AUC) were calculated to summarize the overall performances, and a comparison of the ROC curves was performed (p < 0.05).

Results

The AUC of SVM and ANN is 0.805 and 0.719, respectively (p = 0.020), in the pre-operative prediction of advanced prostate cancer.

Conclusion

The performance of SVM is superior to ANN in the pre-operative prediction of advanced prostate cancer.

INTRODUCTION

The accurate prediction of pathologic stage in cancer 594management is important in choosing an adequate treatment plan. Accurate staging is critical for prostate cancer because the majority of men diagnosed with prostate cancer are older adults with other co-morbidities (1). However, cancer staging performed by a physician may be subjective, and therefore depends heavily on experience, skill, and knowledge. Within this background, various clinical decision support systems (CDSSs) derived from computerized machine learning techniques have been introduced as unbiased and objective supporters for decision-making.

There have been several studies to improve the prostate cancer detection rate with the use of prediction models based on logistic regression analysis and an artificial neural network (ANN) (2-11). The use of a support vector machine (SVM) approach has been introduced as a potential alternative to conventional ANNs (2, 12). SVMs have attracted attention as a useful tool for image recognition and bioinformatics by defining the optimal separating hyperplane with the maximal margin.

Many predictive models to predict pathologic stage have been developed on the basis of different parameters of transrectal ultrasound (TRUS)-guided prostate biopsy (13). However, no report is available on the pre-operative prediction of advanced prostate cancer based on prostate biopsy by applying CDSSs, such as SVM and ANN models, in spite of the advancement in solving classification problems. Moreover, a comparison study between a SVM and other established CDSSs, such as an ANN, on the accurate prediction of advanced prostate cancer with parameters acquired from prostate biopsy is needed.

The purpose of this study was to develop SVM and ANN models for the pre-operative prediction of advanced prostate cancer with the parameters of TRUS-guided prostate biopsies and to compare the accuracies of each model.

MATERIALS AND METHODS

Patient Selection

Our Institutional Review Board approved this retrospective study and waived the requirement for informed consent. Between May 2003 and May 2008, 532 consecutive patients who underwent TRUS prostate biopsies and subsequent radical prostatectomies due to prostate cancer were enrolled in this study.

The indications to perform biopsies included a high prostate specific antigen (PSA) level > 4.0 ng/mL, abnormal digital rectal examination (DRE) finding and/or positive TRUS findings. The patients who underwent repeat biopsy with negative biopsy results previously were not included in the study. The mean age of the patients was 64.8 ± 6.6 years (range, 38-80 years).

The patients were divided into a training (n = 300) and test group (n = 232) using random sampling with a table of random numbers. The training group was designated for constructing CDSSs, and the test group was designated for the evaluation of accuracies in each CDSS.

Before obtaining biopsies, pre-operative DREs were performed by an urologist with 25 years of experience. Nodular palpation on DRE was considered positive. A simple, enlarged prostate or non-palpable nodular lesion in the prostate was classified as negative.

Transrectal US-Guided Prostate Biopsy

Transrectal US-guided prostate biopsies were performed by a radiologist specializing in the genitourinary tract (10 years of experience). Before performing TRUS-guided biopsies, local anesthetic was administered via a 22-gauge, 13.97-cm Chiba needle (Becton Dickinson, Franklin Lakes, NJ). An 18-gauge, 20-cm automatic cutting needle and automated biopsy gun (Pro-Mag 2.2; Manan Medical Products, Northbrook, IL) were used to obtain biopsy cores. Biopsy specimens were generally obtained from 12 separate randomized prostate regions, including three samples from the peripheral zone and three samples from the inner gland on each side. In cases in which focal lesions were detected on TRUS, biopsies encompassing the lesions were obtained. The biopsy specimen was numbered to match the focal lesion.

Five biopsy parameters from systematic needle biopsy results were obtained, as follows: number of positive cores for cancer; percentage of positive biopsy cores; total linear cancer length (absolute sum of the tumor length at each core); percentage cancer length (total cancer length divided by the total length of cores obtained × 100); and maximum cancer core length.

Histopathologic Analysis

The histopathologic analyses of biopsy specimens and extirpated prostates were performed by a pathologist specializing in genitourinary system with more than 20 years of experience. All of the extirpated prostates were obtained by radical retropubic prostatectomies. Routine sections of all surgical margins, including the prostate base and apex, urethra, bladder neck, capsule and peri-prostatic soft tissue, each seminal vesicle, lymph nodes from bilateral pelvic lymph node dissection, and the peripheral zones of the right and left prostate lobes, were examined as permanent sections.

Construction of Clinical Decision Support Systems

From the data in the training group, we constructed SVM and ANN models. The input parameters were three clinical and five TRUS parameters, as follows: age; PSA level; DRE findings; number of positive cores for cancer; percentage of positive biopsy cores; total linear cancer length (absolute sum of tumor length at each core); percentage cancer length (total cancer length divided by total length of cores obtained × 100); and maximum cancer core length. The output data were the probability for advanced prostate cancer (more than pT3a). From the data of the test group, the accuracy of output data was evaluated.

We constructed an ANN model that consisted of a three-layered perceptron architecture. The ANN model was developed by extending the multilayer perceptron (MLP) source code written by Jiang and Manry (ftp.simtel.net/pub/simtelnet/msdos/calculte/Nuclass706a.zip) (2, 14). The neural network consisted of one input layer, one hidden layer, and one output layer. The ANN that was used in our study was composed of eight input nodes in the input layer, 10 hidden nodes in the hidden layer, and 1 output node in the output layer. The number of nodes was determined by trial and error to produce the best performance. The iteration number of back propagation updating was 500, which was determined to balance the accuracy and calculation speed. During a training process, the connection weights between the neurons were adjusted by use of the back propagation-updating algorithm.

The ANN was trained using the data of the training group (n = 300) to produce a value of 1 for an advanced prostate cancer (more than pT3a) and -1 for a confined prostate cancer. As the direct output value of the ANN does not show probability, we converted the output value to the probability by applying a sigmoid function as follows: P(x) = 1/(1 + e-x), where x is the output value of the ANN. The value of P indicates the probability that the patient has advanced prostate cancer (more than pT3a). The output data were analyzed in percentage terms.

An SVM has been proposed in which the model should be used for classification and regression (15). The SVM is based on the use of powerful statistical theory. We used the LIBSVM provided by Chang and Lin on the website (16). LIBSVM is integrated software for support vector classification, regression, and distribution estimation and LIBSVM supports multi-class classification (2).

In our study, we applied the use of LIBSVM to two class problems (advanced or confined cancers). Unlike the ANN, the SVM does not require a trial and error parameter decision process. The SVM determines optimal performance conditions automatically if the kernel type is set. The decision hyperplane that separates advanced and confined prostate cancers constructed by SVM shows theoretical optimal separability concerning the training data.

The SVM received eight input data from the training group (n = 300) and one output value from each patient, and was trained to produce a value of 1 for an advanced prostate cancer and a -1 for a confined prostate cancer. Similar to the ANN, the output values of the SVM were converted to probabilities by applying a sigmoid function. The value of P indicates the probability that the patient has advanced prostate cancer (more than pT3a). The output data were analyzed in percentage terms.

Data Analysis

The accuracy of output data were evaluated with data from each test group (n = 232). The receiver operating characteristic (ROC) curve and the area under the curve (AUC) were used as performance indices. An ROC graph is a technique for visualizing the performance of classifiers and is useful to compare the performance of different classifiers in medical decision-making systems. In this study, the performances of the SVM and ANN models were evaluated using ROC analysis to predict the probabilities of the pathologic stage of prostate cancer. The ROC curves were estimated using MedCalc statistical software (version 9.6.2.0; MedCalc Software, Mariakerke, Belgium). The sensitivities, specificities, and accuracies of two CDSSs were calculated. The AUC was calculated to summarize the overall performances. Comparison of ROC curves was performed. A p value < 0.05 indicated a statistically significant difference.

RESULTS

Parameters of the Training and Test Groups

Table 1 shows the means and standard deviations (SD) of the training and test groups. In the training group, the average patient age was 65.2 ± 6.5 years (range, 38-80 years). The mean PSA value was 11.97 ± 13.26 ng/mL (range, 1.1-94.4 ng/mL). The percentage of DRE-positive cases was 27%. The mean number of positive cores for cancer was 4.5 ± 3.3. The mean percentage of positive biopsy cores was 35 ± 25.1% (range, 5-100%). The mean total linear cancer length was 2.23 ± 2.89 cm (range, 0.02-19.8 cm). The mean percentage cancer length was 27 ± 17.5% (range, 1-88%). The mean maximum cancer core length was 0.64 ± 0.44 cm (range, 0.02-1.9 cm).

Table 1.

Mean and Standard Deviations of Input Parameters in Training and Test Groups

Note.- DRE = digital rectal examination, PSA = prostate specific antigen, SD = standard deviation

In the test group, the average patient age was 64.3 ± 6.8 years (range, 42-76 years). The mean PSA value was 11.88 ± 21.23 ng/mL (range, 0.2-216.3 ng/mL). The percentage of DRE-positive cases was 23%. The number of positive cores for cancer was 3.4 ± 3.0. The mean percentage of positive biopsy cores was 27 ± 23.1% (range, 6-100%). The mean total linear cancer length was 1.45 ± 2.38 cm (range, 0.02-16.0 cm). The mean percentage cancer length was 21 ± 14.6% (range, 1-77%). The mean maximum cancer core length was 0.47 ± 0.39 cm (range, 0.02-1.9 cm).

The percentage of advanced prostate cancer (more than pT3a) in the training and test groups were 31% and 27%, respectively (Table 1).

Performance of the Decision Models

Figure 1 shows the ROC curves for the use of SVM and ANN. The sensitivity, specificity, and accuracy of SVM were 67%, 79%, and 77%, respectively. The sensitivity, specificity, and accuracy of ANN were 63%, 81%, and 78%, respectively. The AUC for use of the SVM and ANN were 0.805 and 0.719, respectively. Comparison of the ROC curves demonstrated that there was a statistical difference between use of the SVM and ANN (p = 0.020; Fig. 1). In the pre-operative prediction of advanced prostate cancer, performance of the SVM was statistically superior to ANN.

Fig. 1.

Receiver operating characteristic (ROC) curve analysis of clinical decision support systems using support vector machine (SVM) and artificial neural network (ANN) models. Area under ROC curve (AUC) value of SVM was superior to ANN.

DISCUSSION

Based on the concept that prostate cancer volume is correlated strongly with local invasiveness, metastatic potential, and a loss of histologic differentiation, various quantitative parameters from needle biopsy results have been assessed to predict pathologic stage in radical retropubic prostatectomy (17). The current study demonstrated that multiple logistic regression analysis of our quantitative parameters for predicting stage indicated that maximum cancer length is a unique independent predictor of organ-confined disease. ROC curve analysis found that "length-parameters", such as maximum cancer length, total linear cancer length, and percentage cancer length, were significantly superior to "number-parameters", including number of cores positive and percentage of cores positive. Furthermore, ROC curve analysis also revealed that percentage cancer length had a similar performance for predicting pathologic stage to total linear cancer length (13).

The combined use of the serum PSA level, the biopsy Gleason score, and biopsy parameters is advocated to improve the pre-operative predictability of final pathologic stage. Park et al. (13) reported that the maximum cancer length in combination with the PSA level and Gleason score improves pathologic stage predictability by ROC curve analysis (18-21). In our study, biopsy parameters and various clinical parameters were used as input variables to construct the SVM and ANN models.

Although many models or nomograms are available to the clinician to predict the pathologic stage and freedom from biochemical failure after treatment, few models are designed to predict the pathologic stage of prostate cancer with the outcome of TRUS-guided prostate biopsy (2). Because prostate cancer is a disease that requires more than one option for the physician to be able to determine diagnosis, staging, and treatment, it provides a good example of the need for a CDSS. We tried to construct CDSSs using the SVM and ANN models and to compare the performances of these models based on readily available parameters, including biopsy parameters.

ANNs are computer programs that simulate some of the higher-level functions of the architecture of the human brain (11). In ANNs simple processing units (nodes) which simulate neurons are linked via weighted interconnections. The interconnection weights function as multipliers that simulate the connection strengths between neurons in an analogous biological model. The nodes are commonly arranged in three or more layers (input, output, and hidden layers). The input layer accepts the values of the predictor variables presented to the neural network, while one or more output nodes represent the predicted outputs (prediction of cancer or disease stages). One or more hidden layers of nodes link the input and output layers (2, 22).

There are several reports in which ANNs were used for the early detection of prostate cancer, a prostate rebiopsy, prostate cancer staging, or for predicting biochemical failure. Zlotta et al. (23) reported that the ANN outperformed logistic regression analysis and correctly predicted pathologic stage in > 90% of the validation patients with serum PSA levels < 10 ng/mL based on clinical, biochemical, and biopsy data. Babaian et al. (24) compared the sensitivity, specificity, and negative and positive predictive values between the use of a neural network algorithm and other PSA parameters showed that the ANN was significantly better in terms of specificity when sensitivity was constantly held at 92%.

The use of SVM developed by Vapnik (25) has resulted in an advance to solve or classify pattern recognition problems. The aim of the SVM is to devise a computationally-efficient way of learning separating hyperplanes in a high-dimensional feature space. The SVM can map the input vectors into a high-dimensional feature space through some non-linear mapping. In this space, an optimal separating hyperplane is constructed. The process of training an SVM classifier is equivalent to finding this optimal hyperplane in a way that minimizes the error on the training dataset and maximizes the perpendicular distance between the decision boundary and the closest data points in classes (2, 26-29). The advantage of the SVM is that an SVM classifier depends only on the support vectors, and the classifier function is not influenced by the entire data set, as is the case for many neural network systems, including ANN. Another characteristic of the SVM is the possibility to deal efficiently with a very large number of features because of the exploitation of kernel functions (2, 30).

It has been shown that the empirical performance of SVMs is generally as good as the best ANN solutions, and it has been proposed that the good performance of SVMs results from fewer model parameters to optimize in the SVM approach, reducing the possibility of over-fitting the training data and thus increasing the actual performance. Chang et al. (31) reported that the classification ability of the SVM is nearly equal to that of the neural network model in imaging diagnosis of breast malignancy. According to their study, the training and diagnosis procedure of the SVM is faster and more stable than other neural network models.

In our study the CDSSs, such as a SVM or an ANN, could provide the probability that the patient has advanced prostate cancer (more than pT3a) with inputs of various clinical and biopsy parameters. From our results, the AUC for the SVM and the ANN were 0.805 and 0.719, respectively. There were some false-positive and -negative cases. The false-positive cases were due to an overestimate of the pre-operative stage and the false-negative cases were due to an underestimate of the pre-operative stage. The false-positive cases might occur when the selected biopsy areas had a concentration of poorly differentiated cancer cells. Conversely, the false-negative cases might occur when the selected biopsy areas had little distribution of the poorly differentiated cancer cells. In the pre-operative prediction of advanced prostate cancer, the performance of the SVM was statistically superior to ANN. Based on our results, the CDSSs, such as a SVM or an ANN derived from computerized machine learning techniques, make it possible for the clinician to predict the pathologic stage more objectively without biases and to determine adequate management planning for biopsy-proven prostate cancer patients. It is expected that the SVM aids the physician in choosing the best treatment option.

Our study had several limitations. First, each process to acquire the clinical and biopsy parameters was performed by one analyst. This might be influenced by individual experience, skill, and knowledge. Moreover, the techniques used by the same expert might undergo subtle changes with time. Second, any predictive model based on maximum cancer length should be further validated and modified because the quantity of the tissue core invaded by a tumor depends on several other factors, including tumor and prostate volume, tumor distribution, and the biopsy procedure (13, 32).

Despite the limitations to the current study, our CDSSs for pre-operative prediction of advanced prostate cancer with input data of TRUS-guided prostate biopsy and clinical parameters can offer the physician and the patient information regarding tumor staging. Based on our results, with respect to the pre-operative prediction of advanced prostate cancer, the performance of the SVM was superior to ANN.