AUTOMATIC DETECTION AND TRACKING OF LONGITUDINAL CHANGES OF MULTIPLE BONE METASTASES FROM DUAL ENERGY CT

Abstract

Early detection and the assessment of changes in bone metastatic cancers can enable clinicians to monitor disease progression and modify treatment to help achieve improved results for patients. However, poor contrast makes detection difficult, and multiple disease sites make tracking of their changes over time difficult. We present a method for automatically detecting and tracking the longitudinal changes in multiple sclerotic bone metastases from Dual Energy Computed Tomography (DECT) images. We employ a multi-stage approach involving (i) bone and marrow extraction, (ii) slice-wise lesion candidate detection and volumetric segmentation, and (iii) aggregation of these 3D candidates. The algorithm achieved 78% agreement with radiologist identified lesions from 10 patients. Longitudinal consistency in the lesion detection computed over 26 scans using Williams’ index was 1.02 ± 0.23 using DICE and 1.03±0.30 using Hausdorff metrics. We also present preliminary results for analyzing lesion material composition changes by using a novel representation computed from the DECT images, where clear differences between bone metastases and normal marrow can be seen.

1. INTRODUCTION

Bone is the third largest site of metastases after lungs and liver. Early diagnosis of bone metastases and ensuing early treatment can significantly impact prognosis for the patients [1]. However, bone lesions are hard to detect due to poor contrast with surrounding parenchyma. Presence of large number and confounding appearance of bony anatomy makes it difficult for radiologists to quickly analyze and monitor disease changes. Therefore, automatic detection of the metastases and analysis of their changes over time can potentially benefit clinicians to help them effectively monitor and intervene to maximize the effectiveness of treatments [2, 3]. Majority of the work in bone metastases detection has been restricted to the vertebrae [4, 5, 6] from CT.

In this paper we present a novel method to automatically detect and segment sclerotic bone metastases in the pelvis from Dual Energy CT (DECT) images. As opposed to regular CT, where targets are scanned at single energy, in DECT, targets are scanned at two different energies, thereby, producing a pair of images, typically a water and iodine density image, which are used throughout this work. The use of energy pairs provides a less ambiguous way of characterizing the material composition of the tissues.

Ours is a fully automatic method wherein all the lesions are detected with no user input, and minimal impact to clinical workflow. Our approach (Fig. 1) involves several steps, where first bone and marrow regions are extracted using [7], followed by contrast enhancement. Next, intensity clustering followed by shape filtering generates regions of interest (ROI) or lesion candidates. Each candidate generates a volumetric segmentation. These 3D segmentations are aggregated using transitive connectivity.

Fig. 1.

Workflow of the metastases detection method.

Our approach combines lesion detections from multiple scans and extracts their changes using volume and tissue composition changes. The tissue compositions are represented in ρ_efff/Z_eff space computed from DECT images [8]. The ρ_efff/Z_eff space spans the effective material density ρ_eff and effective atomic number Z_eff of the underlying tissue.

2. METHODS

Lesion segmentation following bone and marrow extraction [7] consists of three steps:

2.1. Step 1: Tumor Enhancing Bi-Histogram Equalization (TEBEQ)

The tumors were enhanced using a bi-histogram equalization similar to [9] with the following differences in our method. Instead of using the mean intensity of the image, we used the intensity at the histogram peak (mode) to prevent over-smoothing of the intensities for images with large positive or negative skew in the intensity histogram. Next, our method automatically eliminates outliers by identifying the inflection points of the histogram to remove their influence on the equalization. The inflection points were identified by fitting splines to the intensity distribution on both ends of the histogram. In this work, the inflection points were characterized as the locations where the derivate of the fitted curve was high (∇ (f) > τ), where, τ = 1.

2.2. Step 2: Slice-wise Lesions Detection and Volumetric Candidate Generation

We made the following assumptions about the detected lesions: (i) lesions are sclerotic, meaning that the lesions will appear as hyperintense regions following tumor enhancement, (ii) lesions are small, solid and rounded structures. Therefore, the lesions were detected by first identifying the high-intensity super-pixels using meanshift clustering [10]. Then the selected superpixels were further analyzed using morphological criteria to extract the lesion candidates. Slice-wise detection of the lesion candidates was employed because the bone metastases are typically small lesions (≤ 5mL) and do not often extend across multiple slices. Furthermore, the slice thickness of the scans themselves were 5mm which would result in the blurring and loss of the small lesions when using mean-shift clustering.

Four different morphological criteria were employed in order to extract small, rounded, and solid regions as opposed to elongated regions with large concavities. The criteria were: (i) eccentricity < ε, (ii) r₁ = minorAxis/majorAxis > ρ₁, (iii) solidity > σ, and (iv) skeleton/perimeter < ρ₂. Following the lesion candidate selection, seeds were extracted from inside the candidate regions by using skeletonization of the ROI, from which 3D segmentation was generated using geodesic active contours [11]. As lesion segmentations are generated from each slice-based candidates, multiple segmentations of the same lesions can result, which are aggregated as discussed in the next session.

2.3. Step 3: Candidate Volumes Aggregation Through Transitive Connectivity

Lesion volumes generated using the result from Step 2 were aggregated using transitive connectivity. Transitive connectivity between lesions was detected by computing the overlap between lesions such that two lesions that overlap either directly or through intermediate lesions were merged. The overlap between lesions was computed using the dice overlap metric such that lesions having DSC ≥ 0.2 were considered to be overlapping.

In order to compute the transitively connected groups of volumes, we built an undirected graph $\mathcal{G}(N,E)$, in which the nodes N corresponded to the different volumes and edges $E_{N_1,N_2}$ connected the nodes N₁ and N₂.

Graph $\mathcal{G}$ can be expressed by an adjacency matrix A with elements a_ij = 1 if DSC (N_i, N_j) > τ_DS and a_ij = 0 otherwise. The transitivity constraint is enforced by computing the transitive closure of A, using the Floyd-Warshall algorithm [12]. The lesion aggregation step generates spatially distinct lesions.

2.4. Measuring Longitudinal Changes of Lesions

Longitudinal changes in the lesions were measured following lesion detection by first aligning the patient scans to the baseline scan using affine and deformable B-spline image registration using Mattes mutual information cost function, and next by finding corresponding lesions using Dice metric. For simplified analysis, we only considered the lesions that matched across all the time points.

The changes in the lesions were extracted as volumes and (ρ_eff/Z_eff) changes. The ρ_eff, Z_eff decomposition was computed using a least squares approach weighted by the DECT energy spectrums [13], in the projective space via a Radon transform of the water/iodine material images, followed by filtered backprojection reconstruction.

2.5. Measuring Longitudinal Segmentation Consistency

Accurate longitudinal analysis requires that the lesions be detected consistently across multiple time points. Hence, we measured the consistency of the detection using Williams’ index, which has previously been used in [14] to test the agreement on segmentation of individual structures generated from different methods. The Williams’ index estimates the common agreement between different segmentations. Consistent segmentations would result in a Williams’ index close to “1”. Williams’ index is computed as:

$$W{I}_t=\frac{(r-2){\sum }_{t^{\prime }=1,t^{\prime }\ne t}^{r}s\left(L_t,L_{t^{\prime }}\right)}{2{\sum }_{t^{\prime }=1,t^{\prime }\ne t}^r{\sum }_{t^{″}=1,t^{″}\ne t}^{t^{\prime }-1}s\left(L_{t^{\prime }},L_{t^{″}}\right)},$$ (1)

where, r is the number of time points, L_t the set of labeled voxels at time t and s(L_t, L_t’) is the similarity measure between the labeled voxels at time point t and t’. At least three measurements are necessary to compute the Williams’ index. We employed the Hausdorff distance and the Dice score to determine the Williams’ index.

3. RESULTS

We analyzed a total of 36 different DECT volumes where the first 10 consisted of lesions validated by the radiologist. The remaining 26 consisted of scans from 6 patients who had multiple follow-up scans ranging from 2–5.

3.1. Tumor enhancement through TEBEQ

Fig. 2 shows two different examples of the lesion enhancement using the TEBEQ method. Also shown are the images before equalization for comparison. As seen, the tumors are clearly enhanced following the application of the TEBEQ method. Also shown are the radiologist identified lesions highlighted in yellow around the tumor for reference.

Fig. 2.

Two examples of TEBEQ with lesion hand-labled by radiologist in yellow.

3.2. Volumetric Lesion Detection

An experienced radiologist (> 10 years) with several years of experience in analyzing DECT images identified a random set of 64 different lesions from 10 patients completely blinded to the algorithm detection. The lesions detected by the algorithm were compared to those found by the radiologist. Our method correctly matched 50 of the 64 identified lesions (78% hit rate). Although there were more lesions present and detected (1168) from the images, the radiologist only identified a subset of prominent lesions from a single slice from each patient (the algorithm generated a volumetric segmentation). A subset of the patient lesions were visually validated by the radiologist. Examples of algorithm and radiologist identified lesions are shown in Fig. 3. The top row shows the original images with lesions hand-labeled by the radiologist, while the bottom row shows the automated lesion segmentation.

Fig. 3.

Top: examples of patient scans with label hand drawn by the radiologist overlaid in yellow. Bottom: results of our algorithm.

3.3. Longitudinal Analysis of Lesions

Fig. 4 shows results of the changes in consecutively selected lesions from three different patients using the ρ_eff/Z_eff composition space. For reference, the trajectories of normal marrow is also shown in black. As seen, whereas the trajectory of the lesions evolve quite dramatically clearly indicating a change in the tissue composition of the lesions, the normal marrow changes much less. The baseline value of the ρ_eff and Z_eff is shown by a black cross for each lesion. Only 5 lesions are shown per patient for clarity. In the figure, patient 1 shows a clear increasing trend, thereby signifying the buildup of denser tissue inside the lesion. On the other hand, in patient 3 the lesions are relatively unchanging, possibly signifying stable disease. Interestingly, patient 2 has lesions of both increasing and decreasing trends.

Fig. 4.

ρ_eff/Z_eff space for 3 patients. The evolution of the lesions is given in color. For clarity purposes, the first 5 lesions for each patient are displayed. The evolution of a patch of healthy tissue is given in black as a comparison. The initial timepoint of the lesions is marked by a black cross. It is highlighted for one example of patient 2.

Table 1 shows the mean changes in volume for all lesions ≤ 5mL as well as the ρ_eff and Z_eff values for all the 6 patients for the first three time points following the baseline scan. Whereas for some patients changes in volume and composition agree, for others they do not follow the same trend. These results suggest that measures other than volume, such as material composition can add value in the longitudinal analysis and monitoring of bone metastases.

Table 1.

Changes in volume, ρ_eff and Z_eff of small (≤ 5mL) lesions over time

	number of lesions	Mean Volume Change			Mean ρeff Change			Mean Zeff Change
		t1 − t0	t2 − t1	t3 − t2	t1 − t0	t2 − t1	t3 − t2	t1 − t0	t2 − t1	t3 − t2
P1	6	−0.15	−0.05	0.28	0.00	−0.00	0.05	−0.04	0.06	0.08
P2	5	0.19	0.05	−0.08	0.06	−0.05	0.05	−0.02	−0.09	0.08
P3	6	−0.05	−0.18	-	0.01	−0.04	-	−0.14	−0.02	-
P4	32	0.15	−0.30	0.11	0.03	0.02	0.00	−0.08	0.09	0.06
P5	3	−0.14	0.91	0.00	0.05	−0.04	0.01	0.13	−0.02	−0.19
P6	13	0.20	−0.30	−0.06	0.01	−0.01	−0.03	0.22	−0.15	−0.23

3.4. Williams’ Index Computation

The consistency of the detected lesions across the multiple time points was measured using Williams’ index. We analyzed the data of 6 patients for a total of 26 volumes. In all 96 tumors were matched across all the time points and generated 461 Williams’ indices. The mean and standard deviation for the Williams’ index computed on the Hausdorff distance was 1.03 ± 0.30 and for the Dice metric 1.02 ± 0.23, which shows the consistency of our method.

4. DISCUSSION

Quality of treatment and management of patients with bone metastases can be improved by automatically assessing the changes in bone lesions and their density. However, using routine diagnostic CT images, it is difficult even for expert radiologists to quickly identify multiple metastatic lesions and quantify their changes over time. Automated tools such as presented in this work can assist and potentially simplify radiologist effort in quantifying changes in the bone and ultimately aid clinicians to provide timely and more effective treatments. The method presented in this work is fully automated requiring no user intervention. This is important because it is difficult for a radiologist to identify each and every lesion at one or multiple time points. Our method detects lesions independent of their size. Furthermore, we have developed methods to extract material composition measures that can capture the changes in the intrinsic properties of the bone lesions. Our approach combines a number of steps including bone and marrow extraction, contrast enhancement, lesion candidate detection, and segmentation. Appropriate identification of the method’s robustness would require one to assess the performance of each step. We have assessed the efficacy of bone and marrow extraction in our prior work [7]. Typical contrast enhancement methods to our knowledge only used qualitative evaluation. We assessed the performance of lesion detection itself by comparing with the radiologist detections. However, given the very large number of detected lesions, it was difficult for the radiologist to validate each candidate, especially in a longitudinal setting. We will address lesion candidate validation by providing an easy to use GUI that will facilitate for a radiologist to follow the evolution of candidate lesions and identify appropriate lesions. As an alternative to the multi-step based lesion detection, one can envision the use of machine learning to automatically detect the lesions. However, given the very small number of lesion examples provided by the radiologist and the fact that only lesions and no normal regions were identified, we believe a generalizable learning model is not feasible. Nevertheless, our method is promising, showing that metastatic lesions can be detected with fairly good consistency and the longitudinal evolution of the lesions can be characterized automatically.

5. CONCLUSION

In this work we presented a method for fully automatic sclerotic bone metastases detection and longitudinal tracking of the lesion changes through volumetry and material composition computed from DECT images. Our method generates reasonably consistent lesion detection longitudinally and preliminary validation of the lesion detection showed reasonable agreement with radiologist detections.