Pixelation effect removal from fiber bundle probe based optical coherence tomography imaging

Abstract

A method of eliminating pixelization effect from en face optical coherence tomography (OCT) image when a fiber bundle is used as an OCT imaging probe is presented. We have demonstrated that applying a histogram equalization process before performing a weighted-averaged Gaussian smoothing filter to the original lower gray level intensity based image not only removes the structural artifact of the bundle but also enhances the image quality with minimum blurring of object’s image features. The measured contrast-to-noise ratio (CNR) for an image of the US Air Force test target was 14.7dB (4.9dB), after (before) image processing. In addition, by performing the spatial frequency analysis based on two-dimensional discrete Fourier transform (2-D DFT), we were able to observe that the periodic intensity peaks induced by the regularly arrayed structure of the fiber bundle can be efficiently suppressed by 41.0dB for the first nearby side lobe as well as to obtain the precise physical spacing information of the fiber grid. The proposed combined method can also be used as a straight forward image processing tool for any imaging system utilizing fiber bundle as a high-resolution imager.

1. Introduction

Fiber bundles or fiber arrays have been extensively used as effective imaging probes in high-resolution optical imagings such as endoscope, confocal microscopy, two-photon microscopy, fluorescence imaging, and low coherence optical imaging due to their inherent features that make it possible to achieve an effective high resolution endoscopic imaging determined by the overall structural dimension of the fiber bundle and the total number of the fiber cores/pixels at the object/sample [1–9]. The use of a fiber bundle for OCT eliminates the need of a scanning mechanism in front of the specimen. Thus, it is possible not only to miniaturize the probe design but also to accommodate imaging of the moving or live biological samples less invasively [10–16]. The lateral scanning, in sensitive environments such as inside brain or eye, can be performed at the proximal end of the fiber, outside of the organ. This requires a delicate optical coupling to each fiber pixel which can be smaller in diameter than the typical single mode fiber core. In-depth analyses over fiber bundles have been performed based on the fiber dimensions (core size, core spacing, fiber length) and the optical parameters (refractive indices, NA) by using both experimental and numerical/theoretical methods to thoroughly characterize the properties of the fiber imagers [17–21]. Especially, in the case of OCT, fiber bundle imagers have recently been suggested due to their merits for use in a variety of applications for micro-structural or biological sample imaging. However the results have not been promising due to the coupling light into the single core and the non-uniformity of the fiber array [22–26]. Thus, in order to obtain a reasonable image at reasonable speed, the fiber bundle has to be over-scanned instead of scanning core to core. This results in, for example, 1,000,000 image pixels for a 10,000 pixel fiber bundle and only 10,000 pixels out of 1,000,000 pixels represent the ideal sampling. This reduces the overall quality of the fiber bundle based OCT imager significantly.

There have been several previous works that have dealt with the image processing methods to remove the fiber bundle’s pixelation artifact (like a honeycomb) or to enhance the image quality suffered due to the inherent arrangement of grid fibers. Oh et al. [24] applied a disk filter which is a circular averaging filter for an OCT image. However the method was limited because all the image pixel data are treated with the same importance without any mask coefficient difference with the disk filter. Helmchen et al. [9] introduced a Gaussian blurring (smoothing) filter for the two photon microscopy, which was not highly effective since it caused the image blurring that reduced the image contrast and resulted in the loss of high spatial frequency components. Winter et al. [27] employed different types of automatic spectral domain filter masks for fiber optic endoscope image in which an indirect spectral domain filtering was applied to a relatively poor resolution object (~1mm). Suter et al. [28] used high pass filtering of the periodic grid frequency components in order to remove all the higher spatial frequency components while leaving the ones near the centered peak components which without it will cause loss in desired signal or deteriorate the image. The methods using selective frequency filtering are not practical when the obtained image contains complicated frequency distribution. Alaruria et al. [29] used a high-pass spatial filtering in a fluorescence image and Dickens et al. [30] applied band reject filtering to the gray scale level of the image spectrum. These results failed to evaluate the image in views of distribution of image histogram and the precise spatial frequency analysis after the Fourier transform to evaluate whether or not the image was improved in terms of the image CNR which is typically used as figure of merit [31–36].

In this work, we propose a method to improve the OCT images obtained from a common-path (CP) Fourier domain (FD) OCT with a fiber bundle probe by employing both histogram equalization method to pre-enhance the raw image incorporating limited lower gray intensity scales with a pixelation effect and post-process the image using a Gaussian spatial smoothing filter which acts as a low pass filter in a frequency domain as well as to effectively decouple the fiber pixels from the image. We have evaluated the image histogram, CNR, and spectrum for a quantitative comparison. The combined method provided a straight forward real-time image processing method for improving the images obtained using a fiber bundle.

2. Methods

2.1 OCT setup

The schematic view of the OCT imaging setup is shown in Fig. 1(a) which is a Fourier domain common-path optical coherence tomography with a fiber bundle as an imaging probe. The center wavelength and the 3dB-width of the broadband source (super luminescence diode, SLD) are 830nm and 40nm, respectively and the output power was approximately 2mW. We used a 76mm-long rigid coherent fiber bundle (Edmund Optics Inc.) having a numerical aperture of 0.53, individual core spacing of 50μm (total number of fibers: 3,000), and refractive indices for core and cladding, 1.49 and 1.58, respectively. The lateral (transversal) scanning for the imaging fiber was performed not at the object (sample) site but at the remote site where the galvanometer or other two-dimensional sweepings are possible. The data were equally sampled every 5μm both in x- and y- spatial directions. By using a well-defined structure of the US Air Force (USAF) resolution target, we were able to evaluate the quality of our image processing method more precisely than using irregular or non-uniform patterns. In Fig. 1(b), the Scanning Electron Microscope (SEM) image of the used fiber bundle imager is shown where the scratches or scarred pixels are easily observable on the bundle surface as well as the clean ones, which directly influenced the quality of resultant sample images. (The inset scale bar represents 100μm). Details of the experimental setup are reported in the previous results in [25].

Fig. 1.

OCT setup with a fiber bundle probe: (a) Fourier domain common path optical coherence tomography; (b) surface image of fiber bundle taken by scanning electron microscope (SEM). (Scale bar: 100μm)

2.2 Image histogram (intensity distribution) equalization

Histogram equalization is a simple and effective way of adjusting image contrast using the histogram of the image, especially for low contrast gray images with limited scale levels. It is based on a point transformation of gray levels in the input image so that intensities of the resulting image can be better distributed by effectively spreading out the most frequent pixel values [37]. It uses cumulative distribution function (CDF) of the gray scale values of the input image, T(k), as a transformation such that the CDF of the transformed image is linearized, i.e., k^th gray value is mapped to s_k by,

$$s_k=T(k)=\sum _{j=0}^k\frac{n_j}{n},k=0,1,2,\dots ,L-1$$

where, n is the total number of pixels, n_j is the number of pixels that have a specific intensity j, and L is the total number of gray levels in the input image. Since T(k) maps the input gray levels to the range[0,1], the output image can have a full range of possible gray levels by the following simple transformation:

$$s_k^{\prime }=s_k\left(L_{max}-L_{min}\right)+L_{min}$$

where L_max and L_min are the maximum and minimum possible gray levels, respectively.

2.3 Gaussian spatial smoothing

In order to remove irrelevant details and bridge the small gaps shown in an original OCT image, f(m, n), one can blur the image with smoothing filters, h(m, n) [38]. The resulting blurred image, g(m, n) is computed as

$$g(m,n)=h(m,n)\ast \ast f(m,n)$$

where, ** denotes two-dimensional (2-D) convolution. There are several spatial smoothing filters that could be used for eliminating the fiber bundle effect even though they would blur the edges due to the smoothing operation. Typical spatial smoothing filters such as a simple rectangular neighborhood averaging filter and other non-symmetric filters fail to accommodate the circularity of the fiber pixel dimension. With a modification to a circular averaging mask without any pixel weight, a disk filter has been utilized to significantly reduce the pixelation artifact [24]. A Gaussian smoothing filter is a form of weighted average with different coefficients multiplied to the pixels having peak at the center and tapering down as the Euclidean distance from the center pixel increases. The Gaussian filter can accommodate the circularity of the fiber structure and its corresponding symmetrical dimension by controlling the filter parameters. By introducing a weighted average in a circular spatial filter, the fiber bundle artifact has been effectively alleviated [9]. The normalized rotationally symmetric Gaussian (low-pass) filter is defined as,

$$h(m,n)=\frac{h_g(m,n)}{{\displaystyle \sum _m}{\displaystyle \sum _n}{h}_g(m,n)}$$

where, h_g(m, n)= exp(−(m²+n²)/(2σ²)), and σ determines the filter width. Fig. 2 shows an example of a Gaussian smoothing filter response with window size of 19×19 pixels and the filter width of σ = 5 pixels. Note that the filter is normalized so that the total sum of weights is equal to unity. (We have compared the effect of both parameters in the Gaussian filter later in the following chapter.)

Fig. 2.

Response of a Gaussian smoothing filter: (a) 2-D response (h) with colored representation; (b) 3-D response in a mesh form with corresponding weights.

3. Results and discussions

By B-mode (x-axis direction) and C-mode (y-axis direction) pseudo-scanning the Air Force target at the remote site, we have obtained its en face image (400×400) by slicing the volumetric image as shown in Fig. 3(a). We can clearly see the structure of the arrayed fibers as well as the element bar patterns of the sample target. This pixelization limits the resolution and the contrast of the image [25]. The corresponding 2-D magnitude of the discrete Fourier transform (DFT) is shown in Fig. 3(b) and the peak frequency spacing (~20,000 m⁻¹) matches to the spatial spacing of the periodically-spaced fiber cores (~50μm), which could be understood as a reciprocal space or a lattice in a crystalline solid [39].

Fig. 3.

Obtained original unprocessed OCT en face image of US Air Force target: (a) original image with fiber pixelation effect; (b) Magnitude of 2-D discrete Fourier transform of the image.

After analyzing the histogram or intensity distribution of the original unprocessed image in Fig. 3(a), we have observed that the most of the pixel values are distributed in the lower region of the available gray levels (L=256 or 0–255 levels) as shown in Fig. 4(a). The histogram equalization transfer function, T(k), which is the CDF of the original image is shown in the inset as well. We then fully mapped all the gray scale values of the original image to the full range of gray levels, i.e., 0–255, in our experiments. This is very useful since the relatively small signal obtained by OCT can be effectively enhanced or brightened without losing any of the original information. By applying histogram equalization to the raw image, we can highlight the image from lower gray scales by increasing the scale levels as shown in Fig. 4(b). In this case, in addition to the enhanced intensities for the image patterns, the relatively low intensity backgrounds are also enhanced. After enhancing the image contrast, we eliminated the pixelation of the fiber bundle by applying Gaussian smoothing filter with 19×19 window and σ = 5 pixels to obtain the intended final image as shown in Fig. 5(a). Due to the pre-enhancement using the histogram equalization, we were able to observe element numbers and the bars clearly without presence of any fiber pixelization effect. In addition, other structural features such as larger scratches represented by darker patterns pointed out by arrows as well as damaged fiber pixels became clearly visible. Fig. 5(b) illustrates the changes in the histogram of the processed image (red dotted line) compared to the original image (blue solid line), where it has moved from the narrow lower range to a moderate scale level with a broader width. For finite image histogram, we can further improve the image contrast by clipping histogram edges below 50 and/or above 200 so that additional contrast stretching can be achieved without image deformation.

Fig. 4.

Results of histogram equalization method: (a) histogram distribution (probability) of the raw image with gray scale level (inset: corresponding transfer function of the histogram equalization); (b) USAF chart image after histogram equalization.

Fig. 5.

Gaussian smoothing filtered result with a pre histogram equalized image: (a) USAF chart image with combined histogram equalization and Gaussian weighted filter; (b) histogram comparison between before (blue solid line) and after (red dotted line) image processing.

Meanwhile, as in Ref. [9], using Gaussian smoothing filter without applying the histogram enhancement method can also remove the fiber pixelization effect as presented in Fig. 6(a) using our original OCT image (Fig. 3(a)). In Fig. 6(b), the resultant histogram of the processed image experiences minimal change due to the weighted averaging effect. However, the image histogram is still concentrated rather in the lower gray scale range (centered at its average scale value) which is not ideal for differentiating the details when it is displayed compared to our combined method containing the histogram equalization method. Furthermore, we have lost (at least for a human viewer) some of other surface features of the fiber by applying smoothing filter which averaged or blended the image information as well as reduced the intensity level by half (the image is darker than the original one). Those features were obvious in our result in Fig. 5(a). Here, we should mention that applying additional histogram technique after this Gaussian filter deteriorates the image because this enhances the already averaged gray scale values of the backgrounds and the signal patterns which make it difficult to differentiate each other. In our work, we have successfully demonstrated that by inserting the histogram equalization as a pre-process before applying the Gaussian smoothing filter, we could obtain clear and detail features in the OCT image after Gaussian filtering. This would be preferred for real tissue imaging having less abrupt edges compared to that in the USAF chart. In this case, the edge blurring effect can be ignored if the size of fiber pixel is relatively smaller than the sample pattern. Otherwise, the image resolution is limited by fiber pixel dimension so that finer pixel size is required to appropriately image the sample/pattern. The edge blurring can also be compensated by applying other de-blurring or image segmentation algorithms if needed [40].

Fig. 6.

Image result after applying Gaussian smoothing filter only: (a) chart image after Gaussian filtering; (b) corresponding histogram of the processed image (original image: blue solid line; processed image: black dotted line). Inset figure: spatial frequency information (original image: blue solid line; processed image: green dotted line).

In terms of spatial frequencies, i.e., magnitude of the Fourier transform of the image [see Fig. 7(a)], we have observed that the histogram equalization sharpens the side lobes (black dashed line) by adjusting contrast of the image. However, we have also observed that the high frequency components created by the periodic spacing of the fibers are suppressed by Gaussian smoothing filter (red dotted line) which acts as a low pass filter, resulting in the side lobe reduction by 40.99dB (marked with a red arrow: the spectral density is squared to the FFT magnitude). In comparison, there is no significant difference taking only smoothing filter without histogram equalization. In addition, the figures of merit for images such as contrast (marked with black square) and the CNR (marked with blue circle) are shown in Fig. 7(b). Image contrast is defined as,

Fig. 7.

Comparison in characteristics of the image: (a) magnitude of Fourier transforms (blue solid line for original image; black dashed line for histogram equalized image; red dotted line for Gaussian smoothing filtered image with pre-histogram equalization); (b) image contrast and CNR (left y-axis for contrast with black square; right y-axis for CNR with blue circle).

$$C=\frac{{\mu }_o}{{\mu }_b}\text{or}C[dB]=10{log}_{10}\left(\frac{{\mu }_o}{{\mu }_b}\right)$$

where, μ_o is the mean pixel value of the object (patterns) region and μ_b is the mean pixel value of the background (no pattern) region. The corresponding CNR of the image is computed by,

$$\mathit{CNR}=\frac{{\mu }_o-{\mu }_b}{{\sigma }_n}\text{or}\mathit{CNR}[dB]=10{log}_{10}\left(\frac{{\mu }_o-{\mu }_b}{{\sigma }_n}\right)$$

where, σ_n is the standard deviation of the noise in the background region. After the histogram equalization, both the contrast and the CNR slightly decreased because not only the gray scale values of the background (μ_b) but also the background noise variation (σ_n) increased. However, the Gaussian low pass filter decreases the background variation which improved the CNR by more than 13dB and 9.8dB enhancement for the histogram equalized image and for the unprocessed raw original image, respectively, with minimum change to the image contrast.

The effect of the Gaussian filter width and window size on the image quality is compared in Fig. 8 where the image contrast varies less than 0.5dB (Fig. 8(a)) no matter what filter parameters are used whereas the CNR enhancement was dependent on those parameters in Fig. 8(b). If the filter width is less than 3 pixels, the processed image, even after Gaussian smoothing, still contains the fiber pixel effect. For larger filter dimensions and width, the images were blurred too much and lost the sharpness of the bar patterns and the small surface features of the bundle. This result matches well when one consider the data sampling rate of each image pixel (Δd: 5μm) with the fiber core spacing (~50μm). Therefore, to decouple the honey-comb effect of the fiber grid, the smoothing filter should cover the whole fiber core with significant mask coefficients at least for each core area, which is represented by filter parameters, size and width, of the Gaussian smoothing filter. Thus, the optimized conditions for the filter parameters in our case for achieving the pixel removal with minimum image deformation (distinctive features incorporated in the original image) are m=19 and σ = 5. The inset in Fig. 8(a) shows the histogram of the resultant images for σ = 5 with m=11,15,19,23,27 where there is no significant change observable in the histogram distribution.

Fig. 8.

Comparison in image qualities of Gaussian filtered image with various filter parameters (filter size, m, and filter width, σ) post the histogram equalization: (a) image contrast; (b) image CNR.

The detailed mapping image results are shown in Fig. 9 for comparison where each row represents for filter width σ = 3,4,5,6,7 and each column for filter size m = 11,15,19,23,27, respectively. Essentially those parameters of the image filters should be considered based on several conditions for optimization: size of fiber core, separation of adjacent cores, number of image pixels dedicated to a single fiber core, and the dimension of a sample object. Thus, the full width of the Gaussian filter (2σ) was set to cover the entire core spacing in the fiber bundle [9]. In our case, the filter width (σ), pixel distance, and the core spacing were 5, 5, and 50 respectively. The optimized mask size of the filter was found based on the analysis result in Fig. 8. Thus, the selected combination for filter parameters was σ = 5 and m=19 in our experiment as presented in Fig. 9(m). The filtered images, however, should be carefully compared because the figure of merits could be greater even though there still remains fiber bundle artifact. For further quantitative evaluation of the image quality with different filter parameters, one may use alternative image quality metrics such as mean square error (MSE), provided that there exists a reference image, I_R(m, n), that does not have fiber bundle artifact. For each filtered output image, I_F(m, n), the MSE can be computed as [41],

Fig. 9.

Corresponding processed image results by various filter parameters in Fig. 8: (a) σ = 3, m = 11; (b) σ = 3, m = 15; (c) σ = 3, m = 19; (d) σ = 3, m = 23; (e) σ = 3, m = 27; (f) σ = 4 m, =11; (g) σ = 4 m, =15; (h) σ = 4 m, =19; (i) σ = 4 m, =23; (j) σ = 4 m, =27; (k) σ = 5 m, =11; (l) σ = 5 m, =15; (m) σ = 5 m, =19; (n) σ = 5 m, =23; (o) σ = 5 m, =27; (p) σ = 6 m, =11; (q) σ = 6 m, =15; (r) σ = 6 m, =19; (s) σ = 6 m, =23; (t) σ = 6 m, =27; (u) σ = 7 m, =11; (v) σ = 7 m, =15; (w) σ = 7 m, =19; (x) σ = 7 m, =23; (y) σ = 7 m, =27.

$$\mathit{MSE}=\frac{{\displaystyle \sum _m}{\displaystyle \sum _n}{\left|I_F(m,n)-I_R(m,n)\right|}^2}{{\displaystyle \sum _m}{\displaystyle \sum _n}{\left|I_R(m,n)\right|}^2}.$$

Because we do not have a clean reference image that does not have fiber bundle artifact, CNR and image contrast (C) are compared before and after image processing [42–45].

Since there was no lens on the sample side of the bundle, and using a relatively high NA (0.53) fiber, the imaging depth was approximately ~500μm from the end of the fiber bundle. Therefore, the pixelation effect was different at different depths for thick samples which we did not address in this work. However, the purpose of this preliminary work was to study the effectiveness of the method in eliminating the pixelation effect in an en face OCT image based on a fiber bundle imager. By doing so, we were able to compare other previous results utilizing the fiber bundle imager and present the effectiveness of our proposed method. In this case, as an advantage, we can set a rather symmetrical image kernel that can be applied for both directions in the image plane. However, image processing for a depth-resolved 2-D image should be dealt in a different way because it lacks the symmetry between the lateral (B-mode) and axial (A-mode) directions in the acquired image. For a 3-D (including C-mode) volumetric image, the same 2-D plane mask that we used in this work can be applied at different depth layers so that all the possible en face image in different depths construct the whole 3-D volume image. For a 2-D cross-sectional (B-mode) image, the filter mask should be reduced to a 1-D line to process each depth layer line by line because there is no relationship between lateral and depth scanning directions. Our current work was rather a study targeted for a 2-D en face image incorporating only lateral dimensions at the target surface which can clearly visualize the fiber arrangement artifact as the SEM image in Fig. 1(b). This proposed image processing method can be extended for different imaging dimensions such as 3-D volumetric and 2-D cross-sectional images.

4. Conclusions

We have successfully removed the discrete honeycomb artifact of the fiber bundle structure by introducing the combination of histogram equalization and Gaussian spatial smoothing for an en face OCT image obtained using a fiber bundle imaging probe. The histogram equalization effectively transforms the gray scale to the higher range and Gaussian low pass filter removes the pixelization effect with an enhanced CNR from 4.9dB to 14.7dB as well as suppressing the higher frequency components due to the periodic array of the fibers by more than 40dB. This allowed clear observation of the detail features of the image and this combined method can be computed in real time.