A fast color image enhancement algorithm based on Max Intensity Channel

Abstract

In this paper, we extend image enhancement techniques based on the retinex theory imitating human visual perception of scenes containing high illumination variations. This extension achieves simultaneous dynamic range modification, color consistency, and lightness rendition without multi-scale Gaussian filtering which has a certain halo effect. The reflection component is analyzed based on the illumination and reflection imaging model. A new prior named Max Intensity Channel (MIC) is implemented assuming that the reflections of some points in the scene are very high in at least one color channel. Using this prior, the illumination of the scene is obtained directly by performing a gray-scale closing operation and a fast cross-bilateral filtering on the MIC of the input color image. Consequently, the reflection component of each RGB color channel can be determined from the illumination and reflection imaging model. The proposed algorithm estimates the illumination component which is relatively smooth and maintains the edge details in different regions. A satisfactory color rendition is achieved for a class of images that do not satisfy the gray-world assumption implicit to the theoretical foundation of the retinex. Experiments are carried out to compare the new method with several spatial and transform domain methods. Our results indicate that the new method is superior in enhancement applications, improves computation speed, and performs well for images with high illumination variations than other methods. Further comparisons of images from National Aeronautics and Space Administration and a wearable camera eButton have shown a high performance of the new method with better color restoration and preservation of image details.

1. Introduction

Usually there are differences between a recorded color image and the direct observation of the same scene. One of the big differences is the fact that the human visual system is able to distinguish details and vivid colors in shadows and in scenes that contain illuminant shifts. Therefore, improved fidelity of color images to human observation demands a computation that synthetically combines dynamic range compression, color consistency, and color/lightness rendition, or wide dynamic range color imaging systems. Over the last few decades, visual-image quality has been improved significantly using contrast enhancement techniques, which are increasingly necessary for the design of digital multimedia systems.

Furthermore, the state-of-art enhancement methods can be divided into histogram modification techniques (i.e. spatial domain techniques) and transform domain techniques. In spatial domain techniques, Rahman et al. [1] concluded several most popular image enhancement methods including point transforms such as automatic gain/offset, non-linear gamma correction, and non-linear intensity transforms such as the logarithmic transform, square-root transform, and global transforms including histogram equalization, manual burning, and dodging. Along these algorithms, gamma correction produces unsharp masking that damages the color rendition and blurs the details, histogram equalization may produce artifacts [2]. In transform domain techniques, homomorphic filtering has been widely used in removing shadow edges in images. Although homomorphic filtering produce a single output image with good dynamic range compression [3], the color constancy which is defined as the independence of the perceived color of the light source and tonal rendition are poor. Recently, multi-scale analysis techniques, such as enhancement in wavelet, curvelets, contourlet, and adaptive wavelet domain methods have been widely used in image enhancement processing [4]. Comparing these methods, we have found that multi-scale analysis-based enhancement methods generally out-performs other enhancement methods in noisy gray-level images, but in color images with less noise, multi-scale analysis-based enhancement methods are not remarkably better.

In the past decades, several methods have been proposed for color-image enhancement [5,6]. The retinex concept was introduced by Land [7,8] indicating that color saturation has little or no correspondence with luminance variation. Retinex is a well-known algorithm commonly used in image enhancement (e.g. medical and multispectral images). Another variation of retinex algorithm is known as Multi-Scale Retinex (MSR) [9–12], an extension of a former algorithm called Single-Scale center/surround Retinex (SSR) [11]. Meng et al. [13] proposed MSR to solve edge and color distortion problems. MSR with modified color restoration techniques of Hanumantharaju et al. [5] improved luminant effects of a set of aerial images without sacrificing image contrast.

As discussed above, though some of these techniques work well for some color images, we find that only the MSRCR presented in [10–12] performs universally well on different test set. There are two common drawbacks of retinex algorithms: one is the possible apparition of halo artifacts because it is difficult to distinguish edges and details while estimating the illumination, and the other is related to noise suppression. When attempting to amplify signals, noise is also amplified, especially in a dark image content. In addition, the retinex algorithm is computationally complex [14].

In this paper, we examine the performance of MSRCR and try to imitate the human visual system in observing scenes that contain high illumination variations. Instead of multi-scale Gaussian filtering which has a blurring effect, we extend the retinex algorithm that achieves simultaneous dynamic range compression, color consistency, and lightness rendition with the proposed Max Intensity Channel (MIC) and cross-bilateral filtering (CBF). To improve the efficiency of the image enhancement algorithm, the major factors which affect the restored images are discussed in this paper with the following main improvements:

1. Accurate estimation of the illumination veil function: In this paper, the illumination veil function describes the impact of illumination scattering on the scene. It is estimated using the MIC and CBF algorithm instead of multi-scale Gaussian filtering.

2. Restoration of the reflection component of RGB channel: Based on an illumination and reflection imaging model, we present a strategy to construct an RGB reflection component map in the spatial domain from the degraded image.

The rest of the paper is organized as follows: In Section 2, the reflection component is analyzed based on the illumination and reflection imaging model and some state-of-art image enhancement techniques are discussed in detail. In Section 3, the MIC concept is proposed, and the illumination veil is determined by performing a grayscale closing operation and fast cross-bilateral filtering (FCBF) using the MIC of input color image. In Section 4, we present a detailed flow of the proposed algorithm with color restoration. In Section 5, several comparison experiments based on subjective and quantitative evaluations are provided, and simulation results are presented. Finally, we summarize our approach and discuss its limitations in Section 6.

2. Imaging model and related work

2.1. Illumination-reflectance model

According to Lambert’s law, the illumination-reflectance model of image formation says that the intensity at any pixel, which is the amount of light reflected by a point on the object, is the product of the illumination of the scene and the reflectance of the object(s) in the scene, where light is perfectly spread into all directions, as shown in Figure 1(a). Mathematically, we have:

$$I(x,y)=L(x,y)R(x,y)$$ (1)

where I is the image, L is scene illumination defined as the illumination veil function in this paper, and R is the scene reflectance of point lines in (x, y) position. Reflectance R varies with the properties of the scene objects themselves, but illumination veil L results from the lighting conditions at the capturing time of the image. To compensate the non-uniform illumination of the image, the key problem is how to remove the illumination veil L while keep the reflectance component R only.

Figure 1.

(a) Illumination-reflectance model. (b) Homomorphic filtering, F and F⁻¹ represent the Fourier and the inverse Fourier transforms, respectively, and H(u, v) represents the homomorphic filter. (The colour version of this figure is included in the online version of the journal.)

2.2. Transform domain enhancement techniques

From the imaging model discussed above, to make the illumination of an image more even, in the transform domain, the high-frequency components should be increased and low-frequency components should be decreased, because the high-frequency components are assumed to represent mostly the reflectance in the scene (the amount of light reflected off the object in the scene), whereas the low-frequency components are assumed to represent mostly the illumination in the scene [15].

As discussed above, according to the illumination-reflectance model, homomorphic filtering first transforms the multiplicative components to additive components by moving to the log domain [3]. Then, a high-pass filter is used in the log domain to remove the low-frequency illumination component while preserving the high-frequency reflectance component as shown in Figure 1(b). The homomorphic filter usually provides an excellent dynamic range compression but a poor final color rendition [1].

The curvelet transform is a type of multi-scale analysis that decomposes input signal into high-frequency detail and low-frequency approximation components at various resolutions. Since the curvelet transform is well-adapted to represent images containing edges, it is a good candidate for edge enhancement. Curvelet coefficients can be modified in order to enhance edges in an image. To enhance features, the selected detail wavelet coefficients are multiplied by an adaptive gain value. The image is then enhanced by reconstructing the processed wavelet coefficients. The curvelet enhancement method for gray-scale images consists of the following steps [6] (the diagram for the scheme is shown in Figure 2):

Figure 2.

Diagram for the enhancement scheme by multi-scale analysis in transform domain.

1. Estimate the noise standard deviation σ in the input image I.

2. Calculate the curvelet transform of the input image. We get a set of sub-bands w_j, each band w_j contains N_j coefficients C_{j, k}(k ∈ [1, N_j]) and corresponds to a given resolution level.

3. Calculate the noise standard deviation σ_j for each band j of the curvelet transform.

4. For each band w_j, calculate the maximum value M_j of the band and multiply each curvelet coefficient C_j,k by Velde function [16] y_c(C_j,k, σ_j) which is a non-linear function enhancing the faint edges.

5. Reconstruct the enhanced image from the modified curvelet coefficients.

In the method discussed above, the Velde function must be defined which modifies the values of the curvelet coefficients [16]. It could be a function similar to the one defined for the wavelet coefficients. This function however gives rise to the drawback amplifying the noise as well as the signal of interest.

2.3. Spatial domain enhancement techniques

The retinex performs a non-linear spatial/spectral transform that synthesizes strong local contrast enhancement and color constancy. It performs well in improving the low-contrast imagery typical of poor-visibility conditions. The general mathematical formulation of the center/surround retinex is:

$$R_i(x,y)=log{I}_i(x,y)-log[F(x,y)\ast I_i(x,y)]$$ (2)

where R_i(x, y) denotes the retinex output, I_i(x, y) is the image distribution in the ith color spectral band, ‘*’ is the convolution operation, and F(x, y) = Ke^{−(x2 + y2)/c2} is the Gaussian surround function with c being a constant. A small value of c results in a narrower surrounds, and vice versa. Another constant K is selected such that ∬F(x, y)dxdy = 1. This operation is performed on each spectral band to produce Land’s triplet values specifying color and lightness.

The MSR output is then simply a weighted sum of the outputs of several different SSR outputs:

$$R_{\text{MSR}i}=\sum _{n=1}^N{\omega }_n\{log{I}_i(x,y)-log[F_n(x,y)\ast I_i(x,y)]\}$$ (3)

where N is the number of scales, R_MSRi is the ith spectral component of the MSR output, and ω_n is the weight associated with the nth scale. It is in its final exponential transform that the homomorphic filter differs the most from the MSR. MSR does not apply a final inverse transform to go back to the original domain.

MSR converts the intensity values in the image to reflectance values at three resolution scales by computing the logarithm of the ratio of the image to each of the three Gaussian blurred versions of the image (one slightly blurred, one moderately blurred, and one severely blurred). The basic processing operation can be performed in the RGB color space in order to process each channel separately or it can be performed only in the lightness channel in HSL color space. The latter is preferred as it tends to maintain the color balance in the original image.

Retinex theory addresses the problem of separating the illumination from the reflectance in a given image, and the F(x, y) * I_i(x, y) expression in Equation (2) can be explained as the illumination. Since the entire purpose of the log operation is to produce a point-by-point ratio to a large regional mean value. So, to simplify the Equation (3) and reveal the limitation of algorithms on retinex, we just take low-frequency component by a Gaussian low pass filter with c = 80 as the scene illumination L(x, y), which is given in Figure 3(c). According to the restoring method given in Equation (1), the restored image is covered with halos and block artifacts as shown in Figure 4( f ). This is because the illumination is not always constant in a patch [17,18]. In the next section, we introduce the CBF method to get the fine illumination veil.

Figure 3.

(a) Original color image. (b) Lightness channel of (a). (c) Blur (b) by Gaussian low pass filter with c = 80. Reproduced with permission from Glenn A. Woodell, NASA. (The colour version of this figure is included in the online version of the journal.)

Figure 4.

Experiment results. (a) MIC of input image. (b) Max filtering of (a) in 5 × 5 window. (c) Closing operation of (b) with r = 50. (d) CBF of (a) and (c). (e) Restored with illumination (d). ( f ) Restored with illumination Figure 2(c). Reproduced with permission from Glenn A. Woodell, NASA. (The colour version of this figure is included in the online version of the journal.)

3. Illumination veil reconstruction

In most real-world scenes, illumination typically varies slowly across the image as compared with reflectance which may change quite abruptly at object edges [19]. This difference is the key to separating the illumination component from the reflectance component. In our opinion, although enhancement based on curvelet transform or other multi-scale analysis tools is another promising direction of research [4], the transform domain techniques may not be the best choice for the contrast enhancement of a color image without noise. In addition, the output of the homomorphic filter in effect appears extremely hazy though the dynamic range compression of the two methods appears to be comparable. MSR is a good method, but it tends to increase the grayness of the images. However, the advantage of the multi-scale approach is that it is very easy to implement with respect to curvelet transform and other methods.

According to the equation of illumination-reflectance model given in Equation (1), we can imagine that if the scene illumination L(x, y) is available, then the reflectance R of the scene can be reconstructed easily by the following equation:

$$R(x,y)=I(x,y)/L(x,y)$$ (4)

Therefore, the issue lies on how to estimate the scene illumination. We cannot get the satisfied answer using techniques discussed in Sections 2.2 and 2.3. In our opinion, the illumination of the scene can change rapidly in many cases such as shadows, boundaries of different objects, and some complicated structure of objects. As a result, the illumination estimated by low-pass filtering or multi-scale Gaussian filtering is not the best choice for estimating the illumination of objects in an image. In this paper, we address the optimal method to calculate the scene illumination defined as illumination veil for partially shadowed image to overcome the drawbacks of the SSR and the MSRCR techniques.

3.1. Max Intensity Channel

Based on light absorption/reflection characteristics, specific frequency in the white light is absorbed by the colorful object and remaining light is reflected [20]. Based on the RGB color model, we can conclude that the color of an object is generated by its different reflection characteristics of the three RGB components of the light. In [21], He et al. propose a simple but effective “dark channel prior” to remove haze from a single input image. It is based on a key observation that most local patches in outdoor haze-free images contain some pixels whose intensity is very low in at least one color channel. The dark channel prior is a kind of statistics of outdoor haze-free images, in this paper, we proposed a new concept defined as a “bright channel prior” or MIC which indicates that most local patches in outdoor haze-free color images contain some pixels whose intensity is very high in at least one color channel. In other words, for a brightly colorful object, with the constant illumination, there must be at least one higher reflection coefficient closing to 1 (normalized value) in RGB channel of each point in the image.

According to illumination-reflectance model in Equation (1), we can get I_r,g,b(x, y) = L(x, y) * R_r,g,b(x, y), where R_r,g,b(x, y) is the reflection coefficient of RGB Channel in position (x, y) of the target. Based on this model, the high reflectivity of RGB channel is defined as max(R_r,g,b(x, y))→1 and Equation (1) can then be written in terms of:

$$max(I_{r,g,b}(x,y))\approx L(x,y)$$ (5)

In this paper, the largest RGB component of I_max(x, y) is defined as the preliminary scene illumination of the point, and L(x, y) is estimated by:

$$L_{max}(x,y)=\underset{c\in \{r,g,b\}}{max}{I}_c(x,y)$$ (6)

when the composition of the scene is not ideal, hypothesis max(R_r,g,b(x, y))→1 cannot be satisfied at every point. In this case, we can calculate the maximum for a small patch of the input image as given in Figure 4(b). As a result, a filtering algorithm is proposed in this paper to obtain a more accurate L(x, y).

3.2. Illumination veil estimation by FCBF

As discussed above, the basic property of scene illumination is smoothness in local area [17,22], but on the other hand, illumination has the characteristics of mutation due to the different target depths. Therefore, L(x, y) should be both relatively smooth and capable of maintaining the edge details of the target. From Equations (5) and (6), we have performed a preliminary estimation of L(x, y). In order to get an accurate distribution of the illumination veil, firstly, we do some operation on L_max(x, y) by performing morphological closing on the gray image L_max(x, y) with the structuring element. In this algorithm, the radius of the structuring element, which is typically defined as r = min[w, h]/10 (w, h is width and height of the input image), can be dynamically adjusted. If appropriate parameters are chosen, the effect of object without vivid color can be weakened or eliminated, as shown in Figure 4(c). The output of gray-scale closing operation on the image L_max(x, y) is defined as $L_{max}^{\prime }(x,y)$. The gray-scale closing operation makes the grayscale smooth but it results in blurred edges and confusion of the target level. An image is typically represented as a two-dimensional lattice and the space of the lattice is known as the spatial domain while the gray level is represented in the range domain [23], we hope to retain the edge features of L_max(x, y). The intensity changes are consistent with $L_{max}^{\prime }(x,y)$, and a cross-probability density function can check the two independent variables making the cross-bilateral filter the best choice. Therefore, in order to estimate L(x, y), we use a method named FCBF to estimate illumination veil L(x, y).

Let E = L_max(x, y), $D=L_{max}^{\prime }(x,y)$ be the input image. Let L(x, y) be the filtered image pixels in the joint spatial-range domain. Then, we assign the filtered data according to the following equation:

$$L(x,y)=\sum _{j\in \text{pw}}\frac{C}{h_s^2{h}_r}{k}_1\left(\Vert \frac{E-E_j}{h_r}\Vert \right)k_2\left(\Vert \frac{c-c_j}{h_s}\Vert \right)\ast D_j$$ (7)

where pw is the windows for calculating L(x, y), c is the spatial part, c_j is defined as the position surrounding (x, y) within pw, E_j is defined as the range of the point c_j, k₁, and k₂ is the common profile of the kernel used in both domains, h_s and h_r are the employed kernel bandwidths, and C is the corresponding normalization constant. Thus, we only set the bandwidth parameter h = (h_s, h_r) to control the size of the kernel. According to Equation (7), the assignment specifies that the filtered data at the spatial location (x, y) will have the range component of the point in $L_{max}^{\prime }(x,y)$, and then L(x, y) is illustrated in Figure 4(d).

Direct calculation of the bilateral filtering according to Equation (7) can be time consuming due to the non-linearity of the filter. Therefore, development of a fast bilateral filtering algorithm is needed to speed up computation while retaining computational precision [24,25]. On basis of the signal processing theory, Paris, etc. [26] proposed a fast approximation and analyzed its accuracy. The algorithm defined bilateral filtering as a linear shift invariant convolution in a three-dimensional product space S × R, performed low-pass filtering in the down-sampled high-dimensional space, linearly interpolated to get the grayscale of original resolution image, and obtained the final bilateral filtering results. In this paper, we use this fast approximate algorithm, which preserves the computational simplicity while still providing a near optimal accuracy.

4. Color and scene restoration

4.1. Color restoration

Hurlbert [8] studied the properties of different form of retinex and other lightness theories and found that they share a common mathematical foundation but cannot actually compute reflectance for arbitrary scenes. The color restoration method for the MSRCR is given by:

$$R_{\text{MSRCR}i}(x,y)=G[C_i(x,y)\{log{I}_i(x,y)-log[I_i(x,y)\ast F_n(x,y)]\}+b]$$ (8)

where G and b are the final gain and offset values, respectively, and they are variable parameters.

$$C_i(x,y)=\beta log[\alpha I_i^{\prime }(x,y)]$$ (9)

$$I_i^{\prime }(x,y)=I_i(x,y)/\sum _{i=1}^s{I}_i(x,y)$$ (10)

where α and β are constant parameters of the color restoration function.

Different from MSRCR, the proposed algorithm never introduces color distortion, but certain scenes violate the “gray-world” assumption that the average reflectances in the surrounding are equal in the three spectral color bands [9]. To correct the color bias of the image shown in Figure 11(a) and (d), a post-processing method for color restoration is proposed. We assume that the gain/offset correction is performed prior to the scene restoration algorithm. This correction can be performed simply by biasing the image average color towards pure white. When the gain/offset correction is correctly performed, a gain and an offset to stretch the dynamic range of an image is a linear operation for a scene with dynamic range between R_min and R_max, and a display medium with dynamic range d_max. This transform can be represented by Equation (11):

$$R_i^{\prime }(x,y)=\{\begin{array}{cc}0,& R_i(x,y)\le R_{min}\\ \frac{R_i(x,y)-R_{min}}{R_{max}-R_{min}}\times d_{max},& R_{min}<R_i(x,y)<R_{max}\\ d_{max}& R_i(x,y)\ge R_{max}\end{array}$$ (11)

where R_i(x, y) is the ith input band, and $R_i^{\prime }(x,y)$ is the ith output band. After the mean μ and variance σ of the input image are calculated, we can define R_min = μ − 2σ, R_max = μ + 2σ. This process will likely provide a good visual representation of the original scene.

Figure 11.

Comparison of different algorithms. (a) Input image ‘turtle’. (b) Result of NASA method. (c) Result of the proposed algorithm. (d) Input image ‘night’. (e) Result of NASA method. ( f ) Result of the proposed algorithm. Reproduced with permission from Glenn A. Woodell, NASA. (The colour version of this figure is included in the online version of the journal.)

4.2. Scene restoration

With the illumination veil function estimated by Equation (7), we can recover the scene radiance according to Equation (4). But the direct result R(x, y) can be infinite when the illumination veil L(x, y) close to zero [21]. Therefore, we restrict the illumination veil L(x, y) by an experience lower bound L₀ = 50 as given in Equation (12):

$$R(x,y)=I(x,y)/max(L(x,y),L_0)$$ (12)

In order to remove the impact of ambient light and recover the real appearance of the target [21], we calculate the reflection coefficient and truncate it to [01], which is of paramount importance to improve the brightness and contrast of the image.

As discussed above, the procedures of the proposed algorithm are given as follows:

1. Preliminarily estimate illumination veil function L(x, y) by Equation (6).

2. Refine L(x, y) using CBF on L_max(x, y) and $L_{max}^{\prime }(x,y)$.

3. Get R(x, y) of the target as Equation (12) and truncate it to [01].

The intermediate results of proposed algorithm are shown in Figure 5. As we can see in Figure 5(e), the halos and block artifacts are suppressed. The flow chart of the proposed algorithm is shown in Figure 6.

Figure 5.

Intermediate results of proposed algorithm. (a) Non-uniform illumination image. (b) MIC of the input image. (c) Close operation result of (b). (d) Joint bilateral filtering result of (b) and (c). (e) Final result of the proposed algorithm. (f) Final result of MSR. Reproduced with permission from Glenn A. Woodell, NASA. (The colour version of this figure is included in the online version of the journal.)

Figure 6.

Flow chart of the proposed algorithm.

5. Experiments

In our experiments, an efficient computational method of CBF procedure is used [24,26]. The computational complexity of this method is linear to the image size. It takes about 1.42 s to process a 735 × 492 image on a PC with a 3.0 GHz Intel Pentium 4 Processor and MATLAB 7.8.0. To demonstrate the effectiveness of the proposed algorithm, several sets of typical images are selected as representatives of partially shadowed images for the enhancement experiments. Some of the pictures include cases of large areas of sky. This section presents the experimental results and analysis.

5.1. Subjective evaluation

The ultimate goal of the image enhancement algorithm is to increase the visibility, while preserving the details in the image. The illumination veil function in these images is automatically estimated using the method described in Equation (7). In this paper, a comparative study is conducted among three techniques: the curvelet transform, histogram equalization, and MSR [6].

Firstly, we will compare the results of the proposed algorithm within the spatial domain and transform domain discussed in Sections 2.2 and 2.3. Simulation results have shown that the curvelet transform is superior for enhancement applications such as histogram equalization and MSR [1] which may cause loss of contrast and washed out effect shown in Figure 7. The enhancement on curvelet transform can obtain halo artifacts results, especially for images having curve edges. It can be seen that the results based on the new enhancement method is very close to the nature results, especially, in the images with little noise.

Figure 7.

Results of the proposed algorithm and the spatial domain and transform domain. (a, f ) Original image. (b, g) Enhanced by histogram equalization. (c, h) Enhanced by MSR. (d, i) Enhanced by curvelet transform. (e, j) Enhanced by the proposed algorithm. Images originally from: Starck J L, Murtagh F, Candes E J, et al. Gray and color image contrast enhancement by the curvelet transform[J]. Image Processing, IEEE Transactions on, 2003, 12(6): 706–717. Reproduced with permission from Jean-Luc Starck. (The colour version of this figure is included in the online version of the journal.)

Figures 8–10 show the results of some indoor and outdoor scenes. In Figure 8, the MSRCR with color restoration constant 125 and gain/offset 0.35 and 0.65 is composed of three SSR functions with small (c = 80), intermediate (c = 120), and large (c = 250) scale-constants. This combination allows the MSR to synthesize dynamic range compression, color consistency, and tonal rendition, except for scenes that contain violation of the gray-world assumption. We can see that scene is washed out by MSRCR and the contrast of the boy was compromised.

Figure 8.

Comparisons with MSRCR. (a) Original image ‘boy’. (b) Enhanced by MSRCR with color restoration constant 125 and gain/offset 0.35 and 0.65. (c) Enhanced by proposed algorithm. (d) Original image ‘river’. (e) Enhanced by MSRCR with color restoration constant 125 and gain/offset 0.35 and 0.65. ( f ) Enhanced by proposed algorithm. Reproduced with permission from Glenn A. Woodell, NASA. (The colour version of this figure is included in the online version of the journal.)

Figure 10.

Comparison of different algorithms. (a) Input image ‘tower’. (b) Result of NASA method. (c) Result of the proposed algorithm. (d) Input image ‘girl’. (e) Result of NASA method. ( f ) Result of the proposed algorithm. (g) Input image ‘shoes’. (h) Result of NASA method. (i) Result of the proposed algorithm. Reproduced with permission from Glenn A. Woodell, NASA. (The colour version of this figure is included in the online version of the journal.)

Finally, we give additional results based on images such as ‘door,’ ‘street,’ and ‘book’ which are captured by a wearable camera eButton [27], and the results are shown in Figure 9 with the same parameter as those in Figure 8. Obviously, the proposed algorithm produces good color rendition, providing more details even when a gray-world violation occurred.

Figure 9.

Results on eButton images and comparisons with MSRCR. (a) Original image ‘door’. (b) Enhanced by MSRCR. (c) Enhanced by proposed algorithm. (d) Original image ‘street’. (e) Enhanced by MSRCR. ( f ) Enhanced by proposed algorithm. (g) Original image ‘book’. (h) Enhanced by MSRCR. (i) Enhanced by proposed algorithm. (The colour version of this figure is included in the online version of the journal.)

In Figure 10, we utilized some images from the National Aeronautics and Space Administration (NASA). The images enhanced by the NASA method [28] described in [9–11] are shown in Figure 10. It can be seen that our approach can unveil the details and recover colors even in very dark regions. Our method recovers the structures without sacrificing the fidelity of the colors. The halo artifacts are also significantly smaller in our results.

To correct the color bias of the image as shown in Figure 11(a) and (d), we assume that the gain/offset correction is performed prior to the visibility restoration algorithm [20]; it can be performed simply by biasing the image average color towards pure white. This technique is optional in the proposed algorithm because it may result in some color distortion.

Table 1 shows a sample of the automatic visual quality assessment by classification into one of five classes bad, poor, fair, good, and excellent. The classification scheme is based upon preliminary performance of visual measures for automating visual assessment. The Mean Opinion Score (MOS) is generated by averaging the results of a set of standard, subjective tests where a number of volunteers rate the images enhanced by the methods discussed in this section. Each volunteer is required to give each image a rating using the following visual quality assessment scheme in terms of color constancy, fine detail, halo artifacts, contrast perception, and naturalness [30–32], the MOS is calculated which is the arithmetic mean of all the individual scores, with a range from 1 (worst) to 5 (best). From Table 1, we can conclude that the proposed algorithm get a highest score in MOS.

Table 1.

MOS of different enhancement algorithms.

Quality vs. algorithms	Color	Detail	Halo	Contrast	Natural	MOS	Score	Class
Histogram equalization	4	3	3	4	2	3.2	5	Excellent
MSR	2	4	3	4	2	3.0	4	Good
Curvelet transform	4	5	2	4	3	3.6	3	Fair
MSRCR	4	4	4	4	4	4.0	2	Poor
Proposed algorithm	5	4	5	4	5	4.6	1	Bad

5.2. Objective evaluation

At present, the most widely used blind assessment method in image enhancement is the visible edge gradient method proposed by Hautiere et al. [29]. This method defines a ratio e of the number of visible edges in the original image and the restored image, as well as the average gradient ratio (r̄) defined as the objective measurements of enhancement effect. At last, Σ percentage of pixels which become completely black or completely white after enhancement is computed. In this paper, we define C_gain as the ration of contrast between the enhancement image and the original image. To quantitatively assess and rate these state-of-the-art algorithms, we compute four indicators e, r̄, Σ, and C_gain and compare by the gray-level images: input image and restored image.

Implementations of the proposed algorithm are carried out in various NASA images: “girl,” “tower,” “shoe” in Figure 10. All implementation have been done in MATLAB 7.8.0 environment. In order to compare the performance of the proposed method, qualitative and quantitative performances of the proposed algorithm are compared with existing algorithms, and the results are presented in Table 2. Results of the four indicators in Table 2 show that the proposed algorithm has better performance than MSRCR algorithm. The results for the computational times are shown in Table 3. Here, MSRCR is implemented in both spatial and frequency domains, namely spatial MSRCR and frequency MSRCR, to generate the statistics. Our results show that proposed algorithm requires less computational time in comparison with MSRCR algorithms. The reason lies in that, unlike other methods requiring logarithms of the three color components, the proposed algorithm directly works on RGB components. Comparing with other algorithms, the computing speed of the proposed algorithm has been increased dramatically and the objective indicators have been improved or maintained.

Table 2.

Result comparison of different algorithms.

Image Algorithm	Girl				Tower				Shoe
	e	r̄	Σ	Cgain	e	r̄	Σ	Cgain	e	r̄	Σ	Cgain
NASA [28]	0.1599	2.6094	0.0014	1.0405	0.0008	3.3671	0.8140	1.4372	0.1314	3.3764	0.0125	2.1725
Proposed	0.0938	2.9838	0.0034	1.1074	0.0496	3.3908	0.0412	1.9831	0.2055	3.8384	0.0041	2.5506

Table 3.

Processing time of different algorithms.

Image	Algorithm	Time(s)
Tower 2000 × 1312	Spatial MSRCR	66.72
	Frequency MSRCR	20.83
	Proposed algorithm	10.66
Shoe 1312 × 2000	Spatial MSRCR	72.47
	Frequency MSRCR	16.61
	Proposed algorithm	10.57
Turtle 831 × 751	Spatial MSRCR	11.63
	Frequency MSRCR	7.83
	Proposed algorithm	2.67

6. Conclusions

We have developed a fast image enhancement algorithm based on an illumination reflectance model. The illumination veil function has been effectively estimated using bilateral filtering. The algorithm can automatically restore the scene without requirement of supplementary information. Our experimental results have verified the effectiveness of the proposed algorithm. This algorithm is suitable for a wide range of images to produce clear, natural, and realistic scenes with more details. It also increases the processing speed and improves the quality of the restored image.