Using neutrosophic graph cut segmentation algorithm for qualified rendering image selection in thyroid elastography video

Abstract

Recently, elastography has become very popular in clinical investigation for thyroid cancer detection and diagnosis. In elastogram, the stress results of the thyroid are displayed using pseudo colors. Due to variation of the rendering results in different frames, it is difficult for radiologists to manually select the qualified frame image quickly and efficiently. The purpose of this study is to find the qualified rendering result in the thyroid elastogram. This paper employs an efficient thyroid ultrasound image segmentation algorithm based on neutrosophic graph cut to find the qualified rendering images. Firstly, a thyroid ultrasound image is mapped into neutrosophic set, and an indeterminacy filter is constructed to reduce the indeterminacy of the spatial and intensity information in the image. A graph is defined on the image and the weight for each pixel is represented using the value after indeterminacy filtering. The segmentation results are obtained using a maximum-flow algorithm on the graph. Then the anatomic structure is identified in thyroid ultrasound image. Finally the rendering colors on these anatomic regions are extracted and validated to find the frames which satisfy the selection criteria. To test the performance of the proposed method, a thyroid elastogram dataset is built and totally 33 cases were collected. An experienced radiologist manually evaluates the selection results of the proposed method. Experimental results demonstrate that the proposed method finds the qualified rendering frame with 100% accuracy. The proposed scheme assists the radiologists to diagnose the thyroid diseases using the qualified rendering images.

Introduction

Elastography is a newly developed imaging technique to measure tissue’s hardness. It uses ultrasound to estimate tissue’s hardness through evaluating the degree of tissue’s deformation under an external force [1]. Its principle is when tissue is compressed, the softer parts deform more easily than the harder parts [2]. The important applications of this technique are to differentiate benign and malignant tumors [1], and also to diagnosis breast cancer [3], thyroid cancer [4], and prostate cancer [5] in clinical investigations.

The thyroid elastography video is captured using an ultrasound machine with stress elastography function. In the elastogram, the examination results were represented as color-coded images superimposing the conventional B-mode image depending on the strain magnitude [3], where red is for the softest tissues, and blue is for the hardest tissues, and green color indicates the average strain in the region of interest.

In thyroid elastogram video, the stress result of the thyroid is displayed using pseudo colors. The original ultrasound image of the thyroid is displayed next to rendering image side by side. The qualified rendering criteria are the thyroid tissue region is rendered in green dominated, the upper tegument region of the thyroid is in red dominated, and the subcutaneous tissue region is in green dominated. Due to variation of the rendering results in different frames, radiologists must select the qualified frame image manually one by one. The qualified frame selection is time-consuming and tiring. In addition, the results are subjective and depend on experience of radiologists. The purpose of this study is to automatically find the qualified rendering result in the thyroid elastogram video.

To solve the problem in qualified rendering image selection, the thyroid ultrasound image should be segmented firstly and then the anatomic regions must be identified. Then the rendering colors in these regions are extracted and valid if they satisfy the selection criteria.

Many thyroid ultrasound segmentation methods have been proposed [6–9] in literature. These methods use different approaches such as edge detection, thresholding between different gray values, region splitting and merging. These techniques are generally difficult to choose a threshold in inhomogeneous images when using thresholding method and with that produce a lot of false and discontinuous edges when using edge detection. The methods based on support vector machine and neural networks require a huge amount of data for training [10].

In this paper, an automatic approach is proposed to select the qualified rendering image in thyroid elastic video using a neutrosophic graph cut (NGC) segmentation algorithm. At first, the thyroid video is preprocessed where the elastic rendering region and gray scale region are separated individually, and the gray scale image is processed to reduce the speckle noise. Then the gray scale image is segmented using a NGC algorithm which combines neutrosophic indeterminate filtering into the graph cut for image segmentation. At the same time, the region of the upper tegument of thyroid is enhanced in gray scale image. After segmentation, a detection algorithm is used to find the thyroid tumor region, tissue region, and upper tegument region. The rendering quality of each frame is measured using the metric which is computed using the detection results and their color values. Finally, the qualified frame is selected for radiologists in the further diagnosis.

The new contribution of the proposed method includes that an automatic method to select a qualified rendering image for radiologists’ diagnosis using the thyroid elastic video, and a NGC algorithm combining neutrosophic indeterminate filtering is used for image segmentation.

The rest of the paper is structured: “Proposed method” section describes the proposed method. “Experimental results” section provides the experimental results, and conclusions are drawn in “Conclusion” secction.

Proposed method

The input of the system is the thyroid elastogram video which is composed of several frames. At first, the rendering region and gray scale region are extracted from the elastic thyroid video, and the speckle noise is reduced in the gray scale region. Then the gray scale image is segmented based on NGC algorithm. A thyroid ultrasound image is mapped into neutrosophic set, and an indeterminacy filter is constructed to reduce the indeterminacy of the spatial and intensity information in the image. A graph is defined on the image and the weight for each pixel is represented using the value after indeterminacy filtering. The segmentation results are obtained using a maximum-flow algorithm on the graph. Then the anatomic regions are identified automatically in thyroid ultrasound image. Finally the rendering colors on these regions are extracted and validated to find the frames which satisfy the selection criteria.

Elastic thyroid video preprocessing

1. Extract the elastic rending image and gray scale image

At first, each frame is extracted from the thyroid elastogram video and stored for next steps. Then, the elastic rending region and ultrasound region in each frame are extracted using the color and gray scale information. The elastic region image has same structure as the ultrasound region image and the hardness information is rendering on the ultrasound image using pseudo colors.

• 2.Speckle reduction in the gray scale region image

Due to the speckle noise in the ultrasound image, a speckle reduction algorithm [11] is then used to remove the speckle noise and increase the contrast. At the same time, the line-like structure is also enhanced by the algorithm for further processing.

Ultrasound image segmentation

In this stage, the thyroid ultrasound image is segmented using the gray scale information and different layers and tissues are identified using their anatomic structure information. Due to the indeterminate information exists in ultrasound image such as vague boundaries between layers, a novel image segmentation algorithm is proposed based on NGC, which reduces the indeterminacy on the image and define a new energy function to separate the pixels into different categories. The detailed steps are described as:

1. Map the thyroid ultrasound image into neutrosophic set domain

A pixel P(x,y) in an image is interpreted as $P_{NS}\left(\text{x},\text{y}\right)=\left\{\text{Ts}\left(x,y\right),\text{Is}\left(x,y\right),\text{Fs}\left(x,y\right)\right\}$. $\text{Ts}\left(x,y\right)$, in neutrosophic set, where $\text{Is}\left(x,y\right)$ and $\text{Fs}\left(x,y\right)$ represent the memberships belonging to foreground, indeterminate set and background, respectively.

$$\text{Ts}\left(\text{x},\text{y}\right)=\frac{g\left(x,y\right)-g_{min}}{g_{max}-g_{min}}$$ 1

$$\text{Is}\left(\text{x},\text{y}\right)=\frac{Gd\left(x,y\right)-G{d}_{min}}{G{d}_{max}-G{d}_{min}}$$ 2

where g(x,y) and Gd(x,y) are the intensity and gradient magnitude at the pixel of (x,y) on the image.

• 2.Indeterminacy filtering

A filter is defined based on the indeterminacy to remove the effect of indeterminacy information for segmentation, whose kernel function is defined using a Gaussian function:

$$G_I\left(x,\text{y}\right)=\frac{1}{2\mathit{\pi }{\mathit{\sigma }}_I^2}exp\left(-\frac{x^2+y^2}{2{\mathit{\sigma }}_I^2}\right)$$ 3

$${\mathit{\sigma }}_I\left(x,y\right)=f\left(I\left(x,y\right)\right)=aI\left(x,y\right)+b$$ 4

where ${\mathit{\sigma }}_I$ is the standard deviation of the Gaussian filter and it is defined as a function $\text{f}(·)$ associated to the indeterminacy degree. a and b the coefficient of the linear function. When the indeterminacy level is high, ${\mathit{\sigma }}_I$ is large and the filtering can make the current local neighborhood more smooth. When the indeterminacy level is low, ${\mathit{\sigma }}_I$ is small and the filtering takes a less smooth operation on the local neighborhood. The reason to use Gaussian function is that it can map the indeterminate degree to a filter weight more smooth.

An indeterminate filtering is taken on ${\text{T}}_s(\text{x},\text{y})$ to make it more homogeneous [12].

$${\text{Ts}}^{{}^{\prime }}(\text{x},\text{y})=\text{Ts}\left(\text{x},\text{y}\right)\oplus G_{Is}\left(\text{u},\text{v}\right)=\sum _{v=y-m/2}^{y+m/2}\sum _{u=x-m/2}^{x+m/2}Ts(x-u,y-v)G_{Is}\left(\text{u},\text{v}\right)$$ 5

$$G_{Is}\left(u,\text{v}\right)=\frac{1}{2\mathit{\pi }{\mathit{\sigma }}_{Is}^2}exp\left(-\frac{u^2+v^2}{2{\mathit{\sigma }}_{Is}^2}\right)$$ 6

$${\mathit{\sigma }}_{Is}\left(x,y\right)=f\left(Is\left(x,y\right)\right)=aIs\left(x,y\right)+b$$ 7

where $T_s^{{}^{\prime }}$ is the indeterminate filtering result. $\text{a}$ and $\text{b}$ are the parameters in the linear function to transform the indeterminacy level to parameter value.

• 3.Neutrosophic graph cut

A cut C = (S,T) partitions a graph G = (V,E) into two subsets: S and T. The cut set of a cut C = (S,T) is the set that have one endpoint in S and the other endpoint in T. Graph cuts can efficiently solve image segmentation problems by formulating in terms of energy minimization, which is transformed into the maximum flow problem in a graph or a minimal cut of the graph. The function based on Potts model is defined as [13]:

$$E(f)=\sum _{p\in P}{D}_p(f_p)+\sum _{\{p,q\}\in N}{V}_{\{p,q\}}(f_p,f_q)$$ 8

where p and q are pixels, and N is the neighborhood of p. $D_p$ evaluates how appropriate the segmentation is for the pixel p.

In the proposed NGC algorithm, the data function $D_p$ and smooth function $V_{\{p,q\}}$ are defined as:

$$D_{ij}(p)=|T{n}_{ij}^{{}^{\prime }}(p)-C_j|$$ 9

$$V_{\{p,q\}}(f_p,f_q)=u\mathit{\delta }(f_p\ne f_q)$$ 10

$$\mathit{\delta }(f_p\ne f_q)=\left\{\begin{array}{cc}1& if{f}_p\ne f_q\\ 0& otherwise\end{array}\right.$$ 11

where $u$ is a constant numbering [0, 1] and used for a penalty of the disagree of labeling of pixel p and q.

After the energy function is defined in neutrosophic set, the maximum flow algorithms used to optimize the energy function and classify the pixels into different categories using the minimized energy function.

Anatomic regions identification in thyroid image

After NGC algorithm, the thyroid ultrasound image is segmented into several categories according to their gray scale values. A thyroid ultrasound image includes different layers as skin, subcutaneous tissue, muscle, thyroid tissue layers from top to bottom. To measure the rendering quality, we have to separate these layers according to their characteristics. The regions of subcutaneous tissue, upper tegument of thyroid, and thyroid tissue are identified according to their intensity and morphologic features. In some cases, there are tumors in thyroid region. We have to exclude the tumor regions from the thyroid region.

Quality indicator computation

After the region detection, we identify the subcutaneous tissue region (Rst), thyroid upper tegument region (Rut), and thyroid tissue region (Rtt). Then an indicator of qualified rendering image is defined using color domination in these regions.

$$QR(t)=P{G}_{Rst}(t)\ast P{G}_{Rtt}(t)\ast P{R}_{Rut}\left(t\right)$$ 12

$$P{G}_{Rst}=\frac{\#(G(Rst))}{\#(Rst)}$$ 13

$$P{G}_{Rtt}=\frac{\#(G(Rtt))}{\#(Rtt)}$$ 14

$$P{G}_{Rut}=\frac{\#(R(Rut))}{\#(Rut)}$$ 15

where $QR(t)$ is the rendering quality value for tth frame. $P{G}_{Rst}(t)$, $P{G}_{Rtt}(t)$, and $\text{P}{R}_{Rut}(t)$ are the percentage of green color in the subcutaneous tissue region, the percentage of green color in the thyroid tissue region, and the percentage of green color in the upper tegument region, respectively.

All steps in the proposed method can be summarized as follows (Table 1).

Table 1.

Detailed steps of the qualified rendering image selection algorithm

Input: Elastic thyroid video Output: Vessel image
Step 1: Extract rendering region and gray scale region from the elastic thyroid video
Step 2: Speckle noise reduction in the gray scale region
Step 3: Gray scale image segmentation based on NGC algorithm  a. Map thyroid ultrasound image into NS  b. Apply indeterminacy filtering on the gray scale image  c. Define the node weights in a graph  d. Segment the graph using a maximum-flow algorithm
Step 4: Identify the anatomic regions in thyroid image
Step 5: Extract the rendering colors on these anatomic regions
Step 6: Find the frames which satisfy the rendering qualified criteria

Experimental results

Materials

The study was taken in accordance with the ethical guidelines of the Helsinki Declaration and was approved by the Ethics Committee of Harbin Medical University.

The cases included 33 patients diagnosed thyroid uninodular solid goiter examined by ultrasound (US) from January to December 2016. Histopathological analysis was performed in all cases and was considered the golden standard diagnosis. Conventional gray scale and strain elastography (SE) were performed prior surgery by one radiologist with more than 10 years experiences in conventional ultrasound and 5 years in elastography.

All patients received an ultrasound examination of the thyroid gland using Hitachi HI Vision Preirus or HI Vision Avius (Hitachi Medical Corporation, Tokyo, Japan) machine with 6-13 MHz linear probe. The patients were positioned in a supine position with dorsal flexion of the head. Strain elastography was performed with the same machine. The calculation of tissue elasticity distribution was performed in real-time and the examination results were represented as color-coded images over the conventional B-mode image (blue = hard, red & green = soft tissue). All the image data was recorded for offline analysis.

Results

To validate the performance of the NGC approach on image segmentation, we test it on many videos and the selection results are validated by experienced radiologists. Three cases are selected randomly to demonstrate the performance of the proposed method as shown in Figs. 1, 2, and 3.

Fig. 1.

Result of the proposed method. a Orignal ultrasound image; b segmentation result by NGC; c region identification result; d final selection of the qualified rendering image

Fig. 2.

Result of the proposed method. a Orignal ultrasound image; b segmentation result by NGC; c region identification result; d final selection of the qualified rendering image

Fig. 3.

Result of the proposed method. a Original ultrasound image; b segmentation result by NGC; c region identification result; d final selection of the qualified rendering image

All experiments are taken using the same parameters as: a = 10, b = 0.25, c = 10, d = 0.25 and u = 0.5.

In Fig. 1, a tumor appears in the thyroid region which will affect the distribution of the intensity in ultrasound image. The original ultrasound image (Fig. 1a) is segmented using NGC and result is shown in Fig. 1b, in which line-like regions and tumor region are segmented correctly. The identification results of subcutaneous tissue, upper tegument, and tumor are outlined in green, red and blue lines in Fig. 1c. The quality indicator is calculated on each frame using the detection result. The frame with biggest quality indicator value is selected as the qualified rendering image shown in Fig. 1d, where the upper tegument is rendered in red, tumor region is in blue, and normal thyroid tissue is in green. An experienced radiologist validates it as a qualified frame according to his knowledge and experience. The selection result demonstrates that in the case with thyroid tumor, the proposed method segments the ultrasound image correctly, detects the layer in thyroid image clearly, and the finding result is accurate and efficient.

Figure 2 shows the similar result as Fig. 1, where a benign tumor is in the left part of the image. The tumor is segmented accurately with small branches shown in Fig. 2b. In Fig. 2c, the subcutaneous tissue and upper tegument are identified accurately. In Fig. 2d, the qualified rendering image is found and the upper tegument region is rendered in red, and normal thyroid tissue is in green, which is very clear in the rendering and satisfies the clinic requirements.

Figure 3 is an ultrasound image without solid tumor in the thyroid region. In the detection, it is hard to identify the tumor from tissue background due to its intensity is similar to thyroid tissue. Our detection algorithm fails to find the tumor region. However, it still identifies the thyroid tissue region and use it for qualify indicator computation. The final finding result is still feasible and satisfies the clinic requirements.

In this study, we developed an efficient thyroid ultrasound image segmentation algorithm based on NGC to find the qualified rendering images. The experimental results demonstrate the proposed method is efficient to find the qualified rendering images quickly and accurately.

Conclusion

Our study presents an efficient thyroid ultrasound image segmentation algorithm based on NGC to find the qualified rendering images. In NGC, a new energy function is designed using the neutrosophic values after indeterminacy filtering. The segmentation results are obtained by maximum flow algorithm. Different anatomic regions are then identified in thyroid ultrasound image, and the rendering colors on these regions are employed to find the frames which satisfy the selection criteria. Experimental results demonstrate that the proposed method is efficient to find the qualified rendering frame with 100% accuracy. The finding results are used for further diagnosis on the thyroid cancer.