Early prediction of clinical benefit of treating ovarian cancer using quantitative CT image feature analysis

Abstract

Background

In current clinical trials of treating ovarian cancer patients, how to accurately predict patients’ response to the chemotherapy at an early stage remains an important and unsolved challenge.

Purpose

To investigate feasibility of applying a new quantitative image analysis method for predicting early response of ovarian cancer patients to chemotherapy in clinical trials.

Material and Methods

A dataset of 30 patients was retrospectively selected in this study, among which 12 were responders with 6-month progression-free survival (PFS) and 18 were non-responders. A computer-aided detection scheme was developed to segment tumors depicted on two sets of CT images acquired pre-treatment and 4–6 weeks post treatment. The scheme computed changes of three image features related to the tumor volume, density, and density variance. We analyzed performance of using each image feature and applying a decision tree to predict patients’ 6-month PFS. The prediction accuracy of using quantitative image features was also compared with the clinical record based on the Response Evaluation Criteria in Solid Tumors (RECIST) guideline.

Results

The areas under receiver operating characteristic curve (AUC) were 0.773 ± 0.086, 0.680 ± 0.109, and 0.668 ± 0.101, when using each of three features, respectively. AUC value increased to 0.831 ± 0.078 when combining these features together. The decision-tree classifier achieved a higher predicting accuracy (76.7%) than using RECIST guideline (60.0%).

Conclusion

This study demonstrated the potential of using a quantitative image feature analysis method to improve accuracy of predicting early response of ovarian cancer patients to the chemotherapy in clinical trials.

Introduction

In gynecologic oncology, ovarian cancer is the second most prevalent cancer with the highest mortality rate (1). Given that there is no effective cancer screening method to detect early symptoms of ovarian cancer, the majority of ovarian cancer patients (>70%) are diagnosed at advanced stage with tumor metastasis in the abdominal cavity or other distant organs (lung and liver). In order to effectively control metastatic tumors, chemotherapy is necessary after surgical resection of the primary ovarian tumor (primary cytoreduction). Therefore, a large number of clinical trials have been performed to test efficacy of many new chemotherapy drugs or procedures. Due to the toxicity, high cost, and side-effects of chemotherapy drugs, it is important to predict the response of the patients who participate in clinical trials at an early stage. However, how to accurately predict patients’ response to chemotherapy at an early stage still remains a major challenge in current clinical practice. As a result, there is a strong clinical need to develop more accurate and robust prediction tools for rapidly categorizing patients into groups who are likely to benefit from participating in clinical trials or not.

In order to improve cancer detection and/or prognostic assessment, the previous studies can be divided into two classes: identifying genetic biomarkers (2–4); and performing radiographic imaging examinations (5,6). Concerning ovarian cancer, most of the metastasized tumors are driven by the mutation of tumor suppressor gene TP53, which is responsible for several different forms of cellular stress and anti-proliferative cellular functions (7). Given that this kind of mutation is genetically instable (8), no existing serum biomarker in ovarian cancer is able to accurately select treatment options, predict clinical benefit, and determine drug resistance to date (9–11). Therefore, radiographic imaging examinations is critically important in defining response in ovarian cancer treatment. Compared to other imaging modalities, perfusion X-ray computed tomography (CT) has several advantages including wide availability, high diagnostic performance, quick image acquisition, and low operating cost. Thus, perfusion CT is the most popular imaging modality used in current clinical practice for ovarian cancer diagnosis and prognostic assessment (12).

In order to assess patients’ response to the chemotherapy, radiologists track and compare tumor size changes using two sets of perfusion CT images acquired pre-treatment and 4–6 weeks post treatment. Specifically, the tumor response to the new chemotherapy drug is assessed by radiologists using the Response Evaluation Criteria in Solid Tumors (RECIST) guideline (13), which classifies the tumor response into four categories namely: (i) complete response (CR); (ii) partial response (PR); (iii) stable disease (SD); and (iv) progressive disease (PD). The response classification is accomplished by comparing the tumor size (diameter) measured from the CT image slices acquired between the pre- and post-treatment CT scans. Although RECIST is the current international standard in clinical practice, the subjectively assessed one dimension tumor size change is often unable to accurately represent the actual tumor volume change during the treatment (14). Thus, the assessment results using RECIST guideline generate low association with the clinical outcome (e.g. progression-free survival [PFS]) of the patients who participate in the clinical trials (15).

In order to better address this clinical issue or challenge, the purpose of this study was to investigate and test the feasibility of improving efficacy of clinical trials of treating ovarian cancer patients by applying a new quantitative image feature analysis concept and scheme to more accurately predict tumor response to the testing chemotherapy drugs at an early stage using CT images.

Material and Methods

Image dataset

Under an institutional review board (IRB) approved protocol, we retrospectively collected fully anonymized image and clinical data of 30 cases from the existing clinical database in our medical center. The IRB (#4168) of this study was approved by the Health Science Center of University of Oklahoma on 9 April 2014. Each case represented one ovarian cancer patient diagnosed with recurrent, high grade (serous, endometrioid, undifferentiated) ovarian/peritoneal/tubal carcinoma. The age of these patients was in the range of 50–84 years (average age, 67.9 ± 8.1 years). These 30 patients had participated in 14 different clinical trials using a variety of treatment options or drug agents including the standard cytotoxic agents (i.e. weekly paclitaxel, liposomal doxorubicin) and novel therapeutic agents (i.e. anti-angiogenic therapy using bevacizumab and/or fosbretabulin, a range of inhibitors including Reolysin, Pazopanib, and/or Cabozantinib).

For each patient, a pre-treatment and a post-treatment (4–6 week follow-up) CT examination was conducted to assess the treatment response. A pre-established CT image scanning protocol in our medical center was applied to acquire the CT images using either a 64-row detector CT machine (LightSpeed VCT, GE Healthcare, Milwaukee, WI, USA) or a 16-row detector CT machine (Discovery 600, GE Healthcare). The X-ray power was operated at 120 kVp and 100–600 mAs, depending upon the patient body size. During the CT examination, 100 cc of a contrast agent (Isovue 370, Bracco Diagnostics Inc., Monroe Township, NJ, USA) was first injected to the patient with a rate of 2–3 cc/s, using a standard power injector. The CT images were then acquired at two phases, which were performed at approximately 60 s and 5 min after the contrast injection, respectively. The exposure time was approximately 4 s for each phase of the CT scan. The CT images were acquired a pitch of 1.375 mm, slice thickness 5 mm, reconstructed to 1.25mm to make sagittal and coronal reformats at 2.5 mm. Sagittal and coronal reconstructed images were performed at 2.5 mm.

For each set of the pre- and post-treatment CT images, the radiologists marked and tracked a number of metastatic tumors spread in the different parts of the patient’s body and measured tumor size changes based on the RECIST guidelines. Since all the test cases were retrospectively collected, the clinical outcome of each patient was also available. In this study, our goal was to apply a new quantitative image feature analysis scheme to predict 6-month PFS, which is a current criterion for assessing efficacy of new cancer drugs in the clinical trials (16). In our testing dataset, 12 cases (patients) were classified in the responder group (6-month PFS = “Yes”) and 18 were in the non-responder group (6-month PFS = “No”).

Image feature analysis

For each case, we segmented the tumors that had been previously tracked and marked by the radiologists based on the RECIST guideline. We applied a computer- aided detection (CAD) scheme to segment tumor volume depicted on CT images. The tumor segmentation algorithm is a hybrid scheme including a multilayer topographic region growing algorithm (17) and a Canny edge detection operator (18). Due to the diversity of the tumors tracked and marked by the radiologists in different abdominal sections or distant organs (i.e. lung), the automated tumor segmentation failed in a small fraction (<10%) of CT image slides. Thus, the tumor segmentation results on all involved CT image slides were visually inspected and manually corrected (if needed). Fig. 1 shows the tumor segmentation results of a few examples.

Fig. 1.

Examples of four tumor region segmentation results marked by the boundary contours.

After tumor segmentation, our CAD scheme computed three image features of each segmented tumor, which include: (i) tumor volume, F₁ = NV_voxel, where N is the number of all segmented tumor pixels on the tested CT images and V_voxel is the volume of each unit voxel; (ii) average tumor CT number (density), $F_2=\frac{1}{N}{\sum }_{k=1}^N{I}_k$, where I_k is the CT number of voxel (k) inside the segmented tumor volume; and (iii) the standard deviation of tumor CT numbers,

$$F_3=\sqrt{\frac{{\sum }_{k=1}^N{(I_k-F_2)}^2}{N-1}}.$$

Then, the CAD scheme computed the difference of each feature extracted from the pre- and post- treatment CT images:

$$\mathrm{\Delta }{F}_i=F_i^{\mathit{\text{post}}}-F_i^{\mathit{\text{pre}}}$$

Based on the RECIST guidelines (13), the radiologists may mark and track up to five tumors per case. In this dataset, we found that a total number of 67 tumors were marked in the 30 cases. The CAD scheme was used to segment all tumors previously tracked by the radiologists on each case. The scheme then computed an average feature difference value, $\mathrm{\Delta }{\overline{F}}_{\mathrm{i}}=\frac{1}{\mathrm{M}}{\sum }_{\mathrm{k}=1}^{\mathrm{M}}{({\mathrm{F}}_{\mathrm{i}})}_{\mathrm{k}}$, where M varies from 1 to 5, to represent the final feature value of each case.

Based on these three quantitative image features, two different methods were applied to predict the 6-month PFS, and their performance were assessed. The first method is based on the linear combination of three features. For this algorithm, the values of these features were first linearly normalized to the range from 0 to 1 based on the mean and two standard deviations of the feature values (μ ± 2σ) (19). Next, we used a maximum likelihood based receiver operating characteristic (ROC) curve fitting program (ROCKIT, http://xray.bsd.uchicago.edu/krl/roc_soft.htm,

University of Chicago, IL, USA) to compute the area under ROC curve (AUC) for using each image feature as a prediction index. We also compared the statistical significance of the difference between the computed AUC values. In the third step, a feature fusion method was used to combine these three features to generate a new quantitative prediction score (20),

$$\mathrm{S}=\sum _{\mathrm{i}=1}^3{\mathrm{w}}_{\mathrm{i}}\times {\mathrm{F}}_{\mathrm{i}}$$

where ${\sum }_{\mathrm{i}=1}^3{\mathrm{w}}_{\mathrm{i}}=1.0$. We also used the new scores to compute AUC value of this image feature fusion based prediction model.

The second method is to develop a decision-tree based classifier (Fig. 2), which has been demonstrated as an effective method in our previous study for the similar classification purpose with a small dataset to achieve a satisfactory accuracy with low computational complexity (21). In this decision tree, three thresholds were applied to three image feature nodes. (i) If tumor volume changed, $\mathrm{\Delta }\overline{F_1}\ge 0.55$, the case was assigned to 6-month PFS = “No” (non-responsive) group. Otherwise, the case was moved to the next decision node. (ii) If tumor density changed, $\mathrm{\Delta }\overline{F_2}\le 0.48$, the case was assigned to the 6-month PFS = “Yes” (responsive) group. Otherwise, the case was moved to the third decision node. (iii) If the tumor density standard deviation changed, $\mathrm{\Delta }\overline{F_3}\le 0.7$, the case was predicted as 6-month PFS = “Yes” (responsive). Finally, the remaining cases were assigned to 6-month PFS = “No” group. Based on the prediction results, a confusion matrix was generated to assess the prediction accuracy and compute the positive and negative predictive values along with the Kappa coefficient.

Fig. 2.

The flow chart of a decision-tree based classifier.

Results

Table 1 summarizes the computed AUC values when using the change of average tumor volume, tumor CT number, and standard deviation to predict patients’ 6-month PFS. All three image features showed a discriminatory power (with a significantly higher AUC value as compared to a random guess of AUC = 0.5). Among these three image features, the tumor volume change yielded the highest AUC value. Table 2 summarizes AUC values of predicting 6-month PFS using four fusion methods to combine these three image features including: (i) selecting the minimum or maximum feature value; and (ii) using different weighting factors (i.e. w₁ = w₂ = w₃ = 1/3, and w₁ = 0.5, w₂ = w₃ = 0.25) to combine these three image features. The result shows that selecting the maximum feature value among these three normalized image features yielded the highest prediction value of AUC = 0.831 ± 0.078. At 90% and 80% specificity, the sensitivity levels of predicting the responsive cases are 50% and 69%, respectively. Fig. 3 shows the corresponding four ROC curves using three individual features and the maximum feature values.

Table 1.

The summary of the performance of using three image features to predict 6-month PFS.

	Tumor volume	Tumor CT number	CT number STD
Area under ROC curve	0.773	0.680	0.668
Standard deviation	0.086	0.109	0.101
95% confidence interval	0.57–0.90	0.45–0.86	0.46–0.84

Table 2.

The summary of the performance of using fusion method to combine three image features and predicxt 6-month PFS.

	Min	Equal weight	Un-equal wi	Max
Area under   ROC curve	0.764	0.795	0.813	0.831
Standard deviation	0.087	0.088	0.080	0.078
95% confidence   interval	0.57–0.90	0.59–0.92	0.62–0.93	0.64–0.94

Fig. 3.

Four ROC curves generated using three image features and an equal weighted fusion method of three features to predict 6-month PFS.

When applying the decision tree (Fig. 2) to predict 6-month PFS of 30 patients, we found that: (i) the first decision node (tumor volume difference) assigned 10 cases to 6-month PFS “No” group with an accuracy of 90% (9/10) and the rest of 20 cases move to node 2; (ii) the second node (tumor CT number difference) assigned eight cases to 6-month PFS “Yes” group, among which 75% was correct (6/8) and the remaining 12 cases move to node 3; and (iii) the third node (tumor CT number standard deviation difference) assigned 11 cases to 6-month PFS “No” group with an accuracy of 63.6% (7/11) and one case correctly to 6-month PFS “Yes” group. The final prediction result using this decision tree is summarized in Table 3. The overall prediction accuracy was 76.7% (23/30) with a Kappa coefficient of 0.493. The positive predictive value (PPV) was 0.78 (7/9) and the negative predictive value (NPV) was 0.76 (16/21). Finally, Table 4 shows a confusion matrix when using the original RECIST guideline based prediction result. The overall prediction accuracy was 60% (18/30). The corresponding positive and negative predictive values were: PPV = 0.5 (2/4) and NPV = 0.62 (16/26), respectively. The results demonstrated that using quantitative CT image features has potential to achieve higher prediction performance than using the conventional RECIST guideline based assessment.

Table 3.

A confusion matrix of the prediction results of 6-month PFS when using a decision-tree based classifier involving three quantitative image features.

6-month PFS
prediction	Yes	No
Yes	7	2
No	5	16

Table 4.

A confusion matrix of the prediction results of 6-month PFS when using RECIST-guided tumor diameter change measurement.

6-month PFS
assessment	Yes	No
Partial response (PR)	2	2
No response (SD or PD)	10	16

Discussion

In this study, we demonstrated that from CT images we can identify and compute a number of quantitative image features that may have higher discriminatory power to predict 6-month PFS than using the conventional RECIST guideline. Based on testing results of this limited dataset, the prediction accuracy of using a simple decision tree involving three image features increased by 16% when comparing to assessment result using RECIST guideline (from 60% to 76.7% as shown in Tables 2 and 3). The prediction performance improvement can be attributed by a number of following factors.

First, since a tumor is typically not an isotropic “ball” in the volume, the change of one-dimensional size (or diameter) assessed using RECIST guideline is not very accurate to represent the tumor volume change or response during the chemotherapy. When using a quantitative image analysis approach, our CAD scheme segments tumor volume and computes the tumor volume change to replace tumor diameter change. Our results demonstrate that detecting tumor volume change yielded a relatively higher discriminatory power (e.g. AUC = 0.773 ± 0.090 in this limited dataset). When we tested our decision-tree based classifier, the first node (tumor volume change) assigned 10 cases to 6-month PFS “No” (or non-responsive) group with an accuracy of 90%.

Second, similar to many serum biomarkers (e.g. CA 125 in ovarian cancer), using a single phenotype CT image feature (e.g. tumor size or volume change) can only achieve limited sensitivity or specificity. For example, (i) when applying a new immune-stimulatory therapy (e.g. CTLA4 agents), some tumors may experience an initial tumor size increase on CT images, as the tumor might become infiltrated with T cells before it shrinks (22,23); (ii) in a large phase II clinical trial of applying a bevacizumab based therapy to patients with recurrent ovarian cancer, 40% of the patients remained 6-month PFS, but the responsive rate was only 21% when applying RECIST based measurements (24). Hence, another advantage of applying quantitative image feature analysis is that we can extract multiple image features and develop an optimal fusion method or a statistical machine learning classifier to combine these features and achieve higher prediction performance. In this study, we also computed changes of two tumor CT number or density-related image features. We hypothesized that the first tumor density feature computed from the CT images is closely related to the tumor stiffness, which has been recognized as one important factor that determines tumor response to drug treatment (25). The second tumor density related feature computes the standard deviation of the pixel CT values, which is an indicator of the heterogeneity of tumor density. Our study results showed that both features provide useful supplementary information or discriminatory power to predict tumor response to treatment. Reducing tumor density and/or density heterogeneity is also a responsive sign to treatment.

Despite these encouraging results, this is a preliminary study with a number of limitations. First, the image testing dataset is very small. The robustness of our quantitative image analysis scheme needs to be tested in future studies using new large datasets with diverse cases. Second, for each case, we only computed an average feature difference of all the tumors tracked and marked by the radiologists, which may not be an optimal approach. It is still an open topic on how to optimally quantify the case-based image features. Third, due to the limited image dataset, we only computed and tested three image features. More quantitative image features can be explored and quantified. The effective image features will not only be limited or computed from the segmented tumors, but also from the non-tumor regions (i.e. ascites, visceral or subcutaneous fat ratios, and distribution (26)). The machine learning based classifiers will be able to predict the 6-month PFS more accurately if the image features are optimally selected and combined. Fourth, although it does not affect this proof-of-concept study, a more robust automated tumor segmentation scheme still need to be developed in future studies to further minimize the requirement of manual correction.

In conclusion, this study initially verifies that quantitative image feature analysis methods have potential to assist clinicians (i.e. radiologists and/or oncologists) more effectively predicting the clinical benefit of ovarian cancer patients participating in clinical trials. Success of this method may have significant impact on improving efficacy of clinical trials, which would help accelerate the speed of identifying active drugs for the chemotherapy of ovarian cancer patients.