Content based medical image retrieval based on new efficient local neighborhood wavelet feature descriptor

Abstract

This paper presents a new class of local neighborhood based wavelet feature descriptor (LNWFD) for content based medical image retrieval (CBMIR). To retrieve images effectively from large medical databases is backbone of diagnosis. Existing wavelet transform based medical image retrieval methods suffer from high length feature vector with confined retrieval performance. Triplet half-band filter bank (THFB) enhanced the properties of wavelet filters using three kernels. The influence of THFB has employed in the proposed method. First, triplet half-band filter bank (THFB) is used for single level wavelet decomposition to obtain four sub-bands. Next, the relationship among wavelet coefficients is exploited at each sub-band using 3 × 3 neighborhood window to form LNWFD pattern. The novelty of the proposed descriptor lies in exploring relation between wavelet transform values of pixels rather than intensity values which gives more detail local information in wavelet sub-bands. Thus, proposed feature descriptor is robust against illumination. Manhattan distance is used to compute similarity between query feature vector and feature vector of database. The proposed method is tested for medical image retrieval using OASIS-MRI, NEMA-CT, and Emphysema-CT databases. The average retrieval precisions achieved are 71.45%, 99.51% of OASIS-MRI and NEMA-CT databases for top ten matches considered respectively and 55.51% of Emphysema-CT database for top 50 matches. The superiority in terms of performance of the proposed method is confirmed by the experimental results over the well-known existing descriptors.

Introduction

Medical image retrieval systems have gained elevated importance in the scientific community due to the advances in medical imaging technologies. The image modalities such as CT, MRI, X-ray, mammographic images are produced in large numbers every day. Content based medical image retrieval (CBMIR) systems came into existence [1, 2] for searching and indexing large medical image databases. The various researchers presented diverse CBMIR systems [3–6]. The competency of CBMIR system depends on feature extraction methods. Various image features like shape, color and texture can be extracted either in spatial domain or transform domain. Wavelets and filter banks have been used effectively in many signal and image processing applications due to its multiresolution property. Murala et al. [7] proposed directional binary wavelet pattern (DBWP) using combination of binary wavelet transform (BWT) [8] and local binary pattern (LBP) [9]. The input image is divided into eight binary bit planes and BWT is performed on each binary bit plane at three scales. The LBP features are extracted from the resultant BWT sub-bands to obtain DBWP. However, DBWP suffers from high length feature vector as BWT is performed on eight bit planes. Quellec et al. [10, 11] presented optimized wavelet transform based image retrieval. The existing wavelet basis was tuned to maximize the retrieval performance in the training data set. Dubey et al. [12] used local wavelet pattern (LWP) in order to obtain the relationship among the neighboring pixels for CT image retrieval. However, the dimension of LWP depends upon the number of local neighbors considered during feature extraction. Also, the level of wavelet decomposition varies according to the database to achieve better performance. Shinde et al. [13] proposed fast discrete curvelet transform (FDCT) based anisotropic feature extraction for biomedical image indexing and retrieval. However, obtained curvelet sub-bands are variable with database and the redundancy ratio is not fixed.

Numerous researchers presented feature descriptors based on spatial domain. Murala et al. [14–17] presented various local patterns for content based image retrieval. Local tetra patterns (LTrP) are calculated by using the first order derivatives in vertical and horizontal direction to encode the relationship between the referenced pixel and its neighbors. Local ternary co-occurrence patterns (LTCoP) used the grey values of centre pixel and its surrounding neighbors to encode the co-occurrence of ternary edges whereas peak valley edge patterns (PVEP) are formed by the first order derivatives in 0°, 45°, 90° and 135° directions. In their next work, they proposed local mesh patterns (LMeP) which encodes the relationship among the surrounding neighbours for a given referenced pixel in an image. However, the number of local neighbours is the main constraint for the dimensionality of feature vector using these descriptors. Deep et al. [18] proposed directional local ternary quantized extrema pattern (DLTerQEP) which is generated by ternary patterns from horizontal-vertical-diagonal-antidiagonal structure of directional local extrema values to encode the spatial structure of an image. However, feature vector length of this descriptor is more than existing descriptors. Shinde et al. [19] proposed local neighboring binary pattern (LNBP) based on the difference of grey values in local neighborhood. However, this method suffers from rotation variance. Dubey et al. [20, 21] investigated local diagonal extrema pattern (LDEP) and local bit plane decoded pattern (LBDP). LDEP used first order local diagonal derivatives to establish relationship among the diagonal neighbours of any centre pixel of the image. Considering only diagonal neighbours reduced the feature vector length but on account of loosing other directional information. LBDP is based on the difference of centre pixel’s intensity value and the local bit plane transformed values. However, the feature extraction time of LBDP is more than the existing descriptors.

It is observed from aforementioned literature that various feature extraction schemes are available in CBMIR systems. However, it is necessary to improve the retrieval performance with effective and efficient feature representation. Hence, this paper introduces the following:

1. Use of THFB to extract effective and compact medical image features.

2. A new local neighborhood based feature descriptor to encode decomposed wavelet coefficients named as LNWFD. The paper is organized as follows. Section 2 presents medical image retrieval framework, Experimental results and discussions are summarized in Sect. 3 followed by conclusion in Sect. 4.

Proposed medical image retrieval framework

The block diagram of proposed medical image retrieval framework is as shown in Fig. 1. The wavelet decomposition, local feature extraction, feature vector computation, and similarity measurement are main components of the retrieval system.

Fig. 1.

Proposed medical image retrieval framework

Wavelet decomposition with THFB

It is observed that medical images contain non-uniform spectral information due to various geometrical shapes involved in different body parts. Thus, wavelet based multiresolution analysis can be well suited for medical image representation. It is found that most of the existing medical image feature extraction schemes used off-the-shelf wavelets. However, there are issues related with the choice of wavelet filters. In addition, the properties of wavelet filters for medical image retrieval application have not been addressed. Ansari et al. [22] designed two channel filter bank using triplet of half band filters as three kernels known as triplet half band filter bank (THFB) in order to improve the properties of wavelet filters. This filter bank provides necessary flexibility in obtaining the desired frequency response by the design of kernels. Most of the researchers used THFB in image coding [23], iris feature extraction [24, 25], image compression [26], and image resolution [28] applications. We used a class of THFB given in [26] for medical image representation due to its desirable properties (symmetry, near orthogonality, regularity, time frequency localization and frequency selectivity). The detail literature on THFB is presented in [27]. In order to apply THFB to medical images, 2-D extension of wavelets is required. An obvious way to construct separable 2-D wavelet filters is to use the tensor product of their 1-D counterparts. A 2-D approximation and three detail functions are obtained as:

$$\begin{array}{cc}& L(z)=H_l^{1d}(z_1)\times H_l^{1d}(z_2)\hfill \\ \hfill & H(z)=H_l^{1d}(z_1)\times H_h^{1d}(z_2)\hfill \\ \hfill & V(z)=H_h^{1d}(z_1)\times H_l^{1d}(z_2)\hfill \\ \hfill & D(z)=H_h^{1d}(z_1)\times H_h^{1d}(z_2)\hfill \\ \hfill \end{array}$$ 1

where $H_l^{1d}(z_1)$ and $H_h^{1d}(z_1)$ are 1-D low pass filter and high pass filter, respectively of THFB.

The single level decomposition results in vertical (V), horizontal (H), diagonal (D) and one approximation (L) sub-bands, which corresponds to LH, HL, HH, and LL sub-bands, respectively, as shown in Fig. 2b–e on OASIS-MRI image (a). Next, relationship among resultant wavelet coefficients has been exploited by the proposed LNWFD descriptor to extract more detail information from each wavelet sub-band which will provide more accurate matching between two images.

Fig. 2.

Single level decomposition of OASIS-MRI image using THFB

Local neighborhood based wavelet feature descriptor (LNWFD)

A number of local patterns have been developed for image retrieval till now. However, the existing local patterns [14–21] are based on the comparison of grey values of centre pixel with surrounding neighbors or calculating extrema values of neighbors. In our method, we explored the relationship between wavelet transformed values of pixels rather than directly comparing intensity value of pixels which makes the proposed descriptor distinct from the existing local patterns. The proposed descriptor encodes the information contained in the local neighborhood of wavelet coefficients, hence named as local neighborhood based wavelet feature descriptor (LNWFD). This formulation proved to be more efficient for the retrieval purpose.

We considered 3 × 3 window for wavelet coefficients as shown in Fig. 3. WC_c is the central wavelet coefficient having 8 neighbors WC₁ to WC₈. In this work, we explore the information contained in the alternate neighbors of a particular wavelet coefficient for encoding it rather than considering just two of its neighbors. To encode neighboring wavelet coefficient, we calculated the difference between alternate neighboring wavelet coefficients in a clockwise direction and compared with centre wavelet coefficient. If the difference is greater than or equal to centre coefficient value, the neighboring wavelet coefficient is encoded as 1 otherwise 0. For example, encoding WC₁ in a binary value as follows:

$$D\left(W{C}_1,W{C}_3\right)=W{C}_1-W{C}_3$$ 2

$$W{C}_{1_B}=\left\{\begin{array}{cc}1,\hfill & D\left(W{C}_1,W{C}_3\right)\ge W{C}_c\\ 0,\hfill & \mathit{else}\end{array}\right)$$ 3

Fig. 3.

Wavelet neighboring coefficients of a centre coefficient in 3 × 3 window

Similarly, we generate an 8-bit pattern by performing the same method for each of 8 neighboring wavelet coefficients $\left(W{C}_i\forall i=1,2,3,\dots 8.\right)$ of the center wavelet coefficient. Further, binary pattern is multiplied by some weights and summed up to a LNWFD value of centre wavelet coefficient for that 3 × 3 window. The LNWFD value is calculated as:

$$LNWFD(W{C}_c)=\sum _{i=1}^N{2}^{(i-1)}\times f\left[D\left(W{C}_i,W{C}_{i+2}\right)\ge W{C}_c\right]$$ 4

where WC_c is the centre wavelet coefficient, N indicates number of wavelet coefficient neighbours, WC₁ is ith wavelet coefficient neighbour and, WC_i+2 is the clockwise alternate wavelet coefficient neighbour of WC_c. The Fig. 4 illustrates the calculation of LNWFD value for a given 3 × 3 window. After computing LNWFD value for each coefficient (j, k), the whole wavelet sub band, W_S is represented by building histogram [17] as:

$$H_{\mathit{LNWFD}}(W_S)=\sum _{j=1}^{N1}\sum _{k=1}^{N2}{f}_1(LNWFD(j,k),W_S);W_S\in [0,(N(N-1)+3)]$$ 5

$$f_1(x,y)=\left\{\begin{array}{cc}1,\hfill & x=y\\ 0,\hfill & \mathit{else}\end{array}\right)$$ 6

where N₁ ×  N₂ represents size of an input wavelet sub-band.

Fig. 4.

Example of obtaining LNWFD value for the 3 × 3 window

Thus, our proposed LNWFD compares influence of alternate wavelet neighboring coefficients for calculating the bit pattern and extracts more discriminative information necessary for retrieval purposes. As we are encoding the wavelet transformed values of pixels, our feature descriptor is less sensitive to gray level changes and provides robustness against illumination.

Feature vector computation

The LNWFD is applied on each wavelet sub-band and constructs histogram from (5) and treated as a feature vector of wavelet sub-band. Thus, the feature vector of wavelet sub-band is represented by,

$$FV(W_S)=H_{\mathit{LNWFD}}(W_S)$$ 7

The final feature vector of an image is calculated by concatenating feature vectors of wavelet sub-bands as follows:

$$FV(I_m)=[FV(W_{S_1}),FV(W_{S_2}),FV(W_{S_3}),FV(W_{S4})]$$ 8

The feature vector length of one wavelet sub-band is 256. So the final feature vector length of input image will be 4 × 256 = 1024.

Similarity measurement

The Manhattan distance is used as similarity measure to find the similarity between query feature and feature vector of database. The Manhattan distance M_D is defined as:

$$M_D(DB,Q)={\left(\sum _{i=1}^{d{b}_m}\left|{(DBFV(i)-QFV(i))}^{}\right|\right)}^{}$$ 9

where DB, Q, DBFV, QFV, and db_m are database image, query image, database feature vector, query feature vector and dimension of feature vector respectively. We ranked images according to the shortest Manhattan distance between images and selected n top matched images as retrieved images.

Experimental results and discussions

The performance of the proposed method is evaluated in terms of precision, average retrieval precision (ARP), recall and average retrieval rate (ARR). Precision, recall, ARP, and ARR are defined in [17]. To evaluate the performance of the proposed method, we have conducted experiments over publically available OASIS-MRI [29], NEMA-CT [30], and Emphysema-CT [31] medical databases and compared the results with LBP [9], DBWP [7], LMeP [17], LTCoP [15], DLTerQEP [18], LWP [12], LNBP [19], and FDCT [13] feature descriptors.

Experiment 1

This experiment is carried out on MRI dataset taken from open access series of imaging studies (OASIS). The OASIS is a series of neuroimaging datasets that is publically available for study and analysis. It consists of cross sectional collection of 421 subjects aged 18–96 [29]. We extracted local features using LNBP of all these 421 images. We have taken one reference image and calculated the Euclidean distance of reference image LNBP with all remaining images’ LNBP. The Euclidean distance is defined in [13]. The total images are sorted according to the smallest Euclidean distance having indexing thresholds 124, 102, 89 and 106 to form four groups. One image from each group is depicted in Fig. 5.

Fig. 5.

Sample images per group from OASIS-MRI database

Table 1 illustrates the group wise performance of different descriptors in terms of ARP on OASIS-MRI database at number of top matches n = 10. The graph shown in Fig. 6 indicates the comparison of retrieval performance of the proposed method and other existing descriptors as a function of number of top matches considered. The retrieved images for a query image of OASIS-MRI database are shown in Fig. 7 (b) using proposed method. We retrieved top 5 matching images and found that all images are from the same group of the query (i.e. 100% precision is achieved for this query example).

Table 1.

Group wise performance of different descriptors in terms of ARP on OASIS-MRI database

Descriptors	ARP (%) at n = 10
	Group 1	Group 2	Group 3	Group 4	Total
LBP	51.77	32.54	33.82	49.06	42.63
DBWP	52.74	37.74	34.38	60.00	47.05
LMeP	52.82	36.56	36.08	51.50	44.96
DLTerQEP	54.27	42.65	37.08	46.42	45.10
LNBP	61.21	38.63	40.56	66.13	51.63
FDCT	63.79	41.27	35.61	65.56	51.55
Proposed method	72.82	65.64	65.64	81.69	71.45

Fig. 6.

Comparison of proposed method with other existing descriptors as a function of number of top matches considered on OASIS-MRI database

Fig. 7.

The retrieved images for a query image of OASIS-MRI database

The performance of the proposed method is improved by 28.82%, 24.4%, 26.49%, 26.35%, 19.82%, and 19.9% as compared to the LBP, DBWP, LMeP, DLTerQEP, LNBP and FDCT descriptors respectively in terms of ARP at n = 10.

From this experiment, it is evident that the proposed method outperforms the other existing descriptors.

Experiment 2

The digital imaging and communications in medicine (DICOM) is the international standard created by National Electrical Manufacturers Association (NEMA) [30] for medical images. For this experiment, we have collected 499 CT images using the cases CT0001, CT0003, CT0020, CT0057, CT0060, CT0080, CT0082, and CT0083 of this database. These images are categorized into eight different classes (104, 46, 29, 71, 108, 39, 33, and 69 images) to form NEMA-CT database. Figure 8 shows the sample images of NEMA-CT database with one image of each category. We summarized % ARP and % ARR values using different descriptors in Table 2 over NEMA-CT database. The top 5 retrieved images for a query image of NEMA-CT database are shown in Fig. 9b with 100% precision. Figure 10 shows the graph depicting retrieval performance of the proposed method and other existing descriptors by varying the number of top matches.

Fig. 8.

NEMA-CT example images, one image from each category

Table 2.

Performance comparison of different descriptors in terms of % ARP and ARR over NEMA-CT database

Descriptors	In terms of
	ARP at n = 5	ARP at n = 10	ARR at n = 10
LBP	95.80	90.55	29.33
LTCoP	96.02	92.15	30.31
LMeP	96.5	93.09	30.62
LWP	98.12	95.32	31.33
LNBP	98.53	96.63	11.28
FDCT	98.41	94.52	18.44
Proposed method	99.71	99.51	9.37

Fig. 9.

The retrieved images for a query image of NEMA-CT database

Fig. 10.

Comparison of retrieval performance of the proposed method with other existing descriptors over NEMA-CT database as the number of top matches considered

It has been observed from Table 2, Figs. 9 and 10 that the performance of the proposed method is improved as compared to existing descriptors.

Experiment 3

Loss of lung tissue is characterized by Emphysema. Emphysema-CT database [31] consists of three classes namely normal tissue (NT) with 59 images, centrilobular Emphysema (CLE) with 50 images and paraseptal Emphysema (PSE) with 59 images composed from 39 persons. Figure 11 depicts one image for each category.

Fig. 11.

Images from Emphysema-CT database

Table 3 shows the comparison of different descriptors in terms of ARP and ARR over Emphysema-CT database. It is observed that the performance of the proposed system achieved an improvement over LBP, LTP, LMeP, and LNBP methods and slightly less than FDCT method.

Table 3.

Performance comparison of different descriptors in terms of  % ARP and ARR over Emphysema-CT database

Descriptors	In terms of
	ARP at n = 50	ARP at n = 100	ARR at n = 100
LBP	59.33	44.31	79.56
LTP	46.63	39.15	70.19
LMeP	48.27	40.99	73.28
LNBP	50.2	42.3	76.47
FDCT	60.56	48.77	87.60
Proposed method	55.51	48.18	86.53

Effect of similarity measure

The effect of different similarity measures over the performance of proposed method has demonstrated in Fig. 12 using Euclidean, Manhattan, d_1, Chebyshev, and Canberra distance similarity measures over OASIS-MRI, NEMA-CT, and Emphysema-CT databases in terms of ARP. All similarity measurement techniques have defined in [32].

Fig. 12.

Performance comparison for different similarity measures in terms of the ARP using proposed method over OASIS-MRI, NEMA-CT, and Emphysema-CT databases

From Fig. 12 it is observed that the performance using Euclidean and Manhattan distance is nearly equal over all databases. The performance with Canberra distance is worst. The Chebyshev and d1 similarity measures give lower results as compared to Manhatan. We have selected Manhattan distance as it gives best results among compared measures.

Effectiveness of local descriptor LNWFD

We have analyzed the effect of local descriptor, LNWFD over the system performance by comparing the results of LNWFD with energy features of wavelet sub-bands.

Table 4 shows the comparison of performance of LNWFD with energy features of wavelet sub-bands. The performance of proposed LNWFD is better than energy features calculated directly on wavelet sub-bands. It proves the effectiveness of proposed local descriptor, LNWFD.

Table 4.

Performance comparison of LNWFD with energy features in terms of  % ARP over OASIS-MRI, NEMA-CT and Emphysema-CT databases

Descriptors	ARP (%) over database
	OASIS-MRI	NEMA-CT	Emphysema-CT
Energy feature	42.80	96.02	46.28
Proposed LNWFD	71.45	99.51	55.51

Performance over feature vector length

Feature vector length of query image using various descriptors is mentioned in Table 5. The feature vector length of the proposed method is more as compared to the LBP, LTCoP, LMeP and LNBP local descriptors as it is calculated on each wavelet sub-band and it is less than DBWP and DLTerQEP descriptors.

Table 5.

Feature vector length of query image using various descriptors

Descriptor	Feature vector length
LBP	256
PVEP	1024
LTCoP	512
LMeP	768
DBWP	20,480
DLTerQEP	8192
LNBP	256
Proposed method	1024

Performance versus time complexity

Table 6 shows the total feature extraction time and total retrieval time in seconds of different descriptors over each database. All the experiments are conducted using a system having Intel(R) Core(TM) i7 CPU@3.60 GHz processor with MATLAB R2013a software.

Table 6.

The feature extraction time and the retrieval time in seconds of different descriptors over OASIS-MRI, NEMA-CT and Emphysema-CT databases

Descriptors	Feature extraction time			Retrieval time
	OASIS-MRI	NEMA-CT	Emphysema-CT	OASIS-MRI	NEMA-CT	Emphysema-CT
LBP	9.31	49.43	0.63	0.34	0.46	0.42
LMeP	12.6	107.4	0.81	1.42	1.52	0.58
LTCoP	22.2	241.79	1.15	0.61	0.85	1.22
LNBP	0.067	0.613	0.0043	0.065	0.625	0.002
FDCT	0.0421	0.229	0.0058	0.042	0.283	0.0037
Proposed method	0.0481	0.171	0.0088	0.0426	0.1658	0.0105

Table 6 shows that the total feature extraction and total retrieval time of proposed method are considerably less than existing descriptors. It indicates that the proposed method is time efficient.

Conclusion

We proposed a new and efficient local neighborhood based wavelet feature descriptor (LNWFD) for medical image retrieval system. Wavelet decomposition using THFB acquires four wavelet sub-bands. LNWFD explores information at each wavelet sub-band by computing relation between 8 neighbours of wavelet coefficients. Exploiting relationship among wavelet transform values of pixels rather than intensity values is totally new in image retrieval application which makes descriptor more efficient, discriminative and robust against illumination. Moreover, dimension of the proposed descriptor is not affected by any parameter and is same for all databases. The feature extraction and retrieval time of proposed descriptor is less than existing descriptors. The experimentation on three medical databases confirmed the superiority of proposed descriptor over existing descriptors.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.