Spectral Camera based on Ghost Imaging via Sparsity Constraints

Abstract

The image information acquisition ability of a conventional camera is usually much lower than the Shannon Limit since it does not make use of the correlation between pixels of image data. Applying a random phase modulator to code the spectral images and combining with compressive sensing (CS) theory, a spectral camera based on true thermal light ghost imaging via sparsity constraints (GISC spectral camera) is proposed and demonstrated experimentally. GISC spectral camera can acquire the information at a rate significantly below the Nyquist rate, and the resolution of the cells in the three-dimensional (3D) spectral images data-cube can be achieved with a two-dimensional (2D) detector in a single exposure. For the first time, GISC spectral camera opens the way of approaching the Shannon Limit determined by Information Theory in optical imaging instruments.

Conventional Camera, as one of the most important appliances to get image information, records the image of an object based on the point-to-point correspondence between the object-space and the image-space. Because the correlation between pixels of image¹ can’t be applied, the image information acquisition efficiency of such conventional point-to-point imaging mode is much lower than the Shannon Limit²,³ determined by Information Theory in optical imaging instruments⁴,⁵,⁶,⁷,⁸,⁹. Unlike the conventional direct point-to-point imaging mode, the resolution of the pixels of ghost imaging is determined by the correlation of light field fluctuations corresponding to the two pixels respectively, which can be measured on-line or pre-determined¹⁰,¹¹. Combining with compressive sensing (CS) theory¹,¹²,¹³,¹⁴,¹⁵,¹⁶, ghost imaging via sparsity constraints (GISC) has many potential applications including super-resolution imaging¹⁷,¹⁸,¹⁹, three-dimensional (3D) computational imaging with single-pixel detectors²⁰, 3D remote sensing²¹,²², imaging through scattering media²³,²⁴, object tracking²⁵, object authentication²⁶,²⁷ and X-ray Fourier transform diffraction imaging²⁸,²⁹,³⁰.

For thermal light ghost imaging, according to the illumination source, it can be classified to two categories: ghost imaging with pseudo-thermal light and true thermal light. Ghost imaging with true thermal light and sunlight have been respectively demonstrated by detecting the temporal fluctuation of thermal light and applying the intensity correlation between the intensity distributions at the reference arm and the test arm³¹,³²,³³. Comparing with ghost imaging with pseudo-thermal light, this scheme of ghost imaging with true thermal light has to face the difficulty of detecting the temporal fluctuation of true thermal light which requires the response time of detector less than the coherence time of true thermal light (λ is the wavelength, Δλ is the linewidth, c is the speed of light) which can be as short as femtosecond. In order to increase the coherence time of the illumining true thermal light, monochrome imaging is required which results in the vast majority of radiation energy from the target scene being filtered out, making the energy efficiency of ghost imaging applying the temporal fluctuation of true thermal light very low. Moreover, the fluctuating true thermal field needs to be split before the light field illuminating the object in the system and recorded in the reference path, which makes the scheme even more difficult to be applied in remote sensing.

In this paper, for the first time, we propose a spectral camera based on true thermal light ghost imaging via sparsity constraints (GISC spectral camera) without a splitter. GISC spectral camera modulates the true thermal light into a spatially fluctuating pseudo-thermal light using a spatial random phase modulator³⁴,³⁵ which, at the same time, also acts as a random grating generating the uncorrelated speckles for different wavelengths, the 3D spectral images data-cube is then modulated into a two-dimensional (2D) data plane and GISC spectral camera can achieve the whole wavelength image in a single exposure, leading to a more convenient detection process and higher energy efficiency compared to ghost imaging applying the temporal fluctuation of true thermal light. Combining with CS, GISC spectral camera can acquire the information at a rate significantly below the Nyquist rate which opens the way of approaching the Shannon Limit determined by Information Theory in optical imaging instruments³,⁴,⁵,⁷.

Schematic and Resolution

The schematic of GISC spectral camera is shown in Fig. 1. The system consists of (1) an imaging system, which projects the object image in the object plane ‘a’ onto the first image plane ‘b’, (2) a spatial random phase modulator, which disperses the image with different wavelengths as a random grating and modulates the image to generate the speckles in plane ‘c’³⁴,³⁵, (3) a microscope objective, which magnifies the speckles in plane ‘c’, and (4) a charge-coupled device (CCD) detector recording the magnified speckles.

Figure 1. Schematic of GISC spectral camera.

(a) The object plane; (b) the first image plane; (c) the speckles plane; (1) an imaging system; (2) a spatial random phase modulator; (3) a microscope objective; (4) CCD detector.

Denoting the spectral light intensity distribution in the first image plane ‘b’ by I_b(r_i, λ_l) and the intensity distribution in plane ‘c’ by I_c(r_t) respectively, we have³⁶

where h_I(r_t; r_i, λ_l) is the incoherent intensity impulse response function, r_t is the coordinate in plane ‘c’, r_i and λ_l are respectively the coordinate and wavelength of the light intensity distribution in the first image plane ‘b’. To record the pre-determined reference spatial intensity fluctuation of the pseudo-thermal light without objects, a coherent monochrome point source at pixel with wavelength in the first image plane ‘b’, denoted as , is used to illuminate the spatial random phase modulator, and the recorded light intensity in the plane ‘c’ is given by

During the imaging process, the intensity distribution in the first image plane ‘b’ is simply the image, denoted as T_i(r_i, λ_l), of the object T_s(r_s, λ_l) in the object plane ‘a’,

Combining Eqs (1,2) with (3), the intensity distribution in the speckle plane ‘c’ is

Eq. (4) shows that is the T_i(r_i, λ_l) weighted integration of the pre-determined reference spatial intensity fluctuation of pseudo-thermal light . Therefore, each pixel r_t of CCD detector is equivalent to a measurement of the bucket detector in the test arm of ghost imaging scheme. The second-order correlation function between the spatial intensity fluctuation in the pre-determined reference arm and test arm is defined as

where is the ensemble average about the coordinate of the light intensity distribution r_t. Combining Eqs (2,4) with (5), the second-order correlation function is given by

where is the second-order correlation function of the light fields at different pixels and wavelengths in the first image plane ‘b’. In order to calculate , the height autocorrelation function of the spatial random phase modulator is assumed as³⁷

where and are respectively the height of the spatial random phase modulator at r₀ and , ω and ζ are respectively the height standard deviation and transverse correlation length of the spatial random phase modulator. Assuming that the light field fluctuations in the speckles plane ‘c’ corresponding to pixel in the first image plane ‘b’ with wavelength obeys the complex circular Gaussian distribution, can be written as

where

is defined as the normalized second-order correlation function of the light fields at different pixels and wavelengths in the first image plane ‘b’. According to the Fresnel diffraction theorem, the light field in the speckles plane ‘c’ propagated from pixel in the first image plane ‘b’ with wavelength is

where is the transmission function of the spatial random phase modulator. and are respectively given by

Substituting Eqs (7,9,11) into (10) yields

where

Assuming , we have

Assuming , and the diameter σ of the illuminated region in the spatial random phase modulator by each cells of 3D data-cube in calibration satisfies πσ²/λ_lz₂ < 1, is given by

Taking Eqs (8,12,13,21) into (6), we get the correlation function of intensity fluctuations³⁸

where , , ⊗ denotes the operation of convolution. Eq. (22) specifies that can be separated from the correlation function of intensity fluctuations , and the resolution is determined by the normalized second-order correlation at different pixels and wavelengths in the first image plane ‘b’. When , according to Eq. (14), the normalized second-order correlation function of the light fields at pixel in the first image plane ‘b’ with two different wavelengths is given by

Similarly, when , according to Eq. (14), the normalized second-order correlation function of the light fields at two different pixels in the first image plane ‘b’ with wavelength is given by

Figure 2(a,b) respectively show the comparison of and between experiment and theory, and the experiment diagram is given in Fig. 1 with z₁ = 20 mm, z₂ = 0.3 mm, ω = 2.1 μm, ζ = 16.75 μm, n = 1.516 and the central wavelength .

Figure 2.

(a) The normalized second-order correlation function of the light fields at pixel in the first image plane ‘b’ with two different wavelengths; (b) The normalized second-order correlation function of the light fields at two different pixels in the first image plane ‘b’ with wavelength .

The Measurement Matrix & Reconstruction Algorithm

There are many methods to improve the imaging quality of ghost imaging³⁹,⁴⁰,⁴¹. However, ghost imaging reconstructions based on the ensemble statistics cannot provide the criterion of the necessary number of sampling for a perfect imaging, which makes it impossible to optimize the design of ghost imaging system. Combining with CS which provides the recovery condition of perfect reconstruction, the quantitative analysis for the necessary measurements data can be made. Under the framework of CS theory, the measurement matrix of GISC spectral camera is obtained as follows: each of the speckle intensity distributions generated by a point light source at pixel in the spectrum band in the first image plane ‘b’ is recorded by the randomly selected pixels of CCD detector and reshaped as a column vector of length M of the measurement matrix. Repeating the process for all the N image pixels in the first image plane ‘b’ and all the L spectral bands, one may have the pre-determined random measurement matrix A_M×K, where K = L × N. If we denote the unknown spectral object image as a K-dimensional column vector X_K×1, and reshape the modulated object intensity distribution recorded by the same M pixels of CCD detector in a similar way as a column vector Y_M×1, then we may have the discrete from Eq. (4),

Spectral object image is usually both spatially and spectrally correlated, which has already been utilized in spectral image reconstructions⁴²,⁴³,⁴⁴. The reconstruction of the spectral object image can generally be regarded as solving a minimization problem which penalizes both the l₁ norm and the nuclear norm of the data matrix:

where a matrix representation of the spectral object image whose columns represent different bands of the spectral object image, ψ the sparsifying transform, μ₁ and μ₂ the weight coefficients and μ₁, μ₂ > 0. In this work, we use a modified approach based on the method described by Eq. (26)⁴⁵:

where , is the subtraction of the largest singular value s₁ and the other s_i. The solution of Eq. (27) tends to have a simultaneous low-effective-rank and sparse structure, which much improves the reconstruction quality with low sampling rate.

Experimental Results

In the experimental setup of GISC spectral camera shown in Fig. 3, the imaging system (Tamron AF70-300 mm f/4-5.6) with focal length of f = 180 mm projects the object image onto the first image plane, a beam splitter (BS) with split ratio 50:50 splits the light field into two paths, CCD1 detector (AVT Sting F-504C with pixel size of 3.45 μm × 3.45 μm) is placed in one of the two paths at the position of the first image plane of the system to obtain the conventional image of the object for comparison, a spatial random phase modulator (SIGMA KOKI CO., LTD. DFSQ1-30C02-1000) disperses the images with different wavelengths acting as a random grating and modulates the image to generate the speckles, a microscope objective with magnification β = 10 and the numerical aperture N.A. = 0.25 magnifies the speckles which are then recorded by CCD2 detector (Andor iKon-M) with the pixel size 13 μm × 13 μm. The first image plane is divided into N_x × N_y = 140 × 140 pixels with the square of each pixel approximately equal to Δr_s determined by the Eq. (24). The number of spectrum bands for single exposure is 7, and the images in two wavelength ranges of 520 ~ 580 nm and 620 ~ 680 nm are respectively obtained in two exposures, while the theoretical spectral resolution is 20 nm in the experimental setup according to Eq. (23).

Figure 3. Experimental setup of GISC spectral camera.

In order to compare the spectral & spatial resolution of GISC spectral camera with the theoretical resolution, as shown in Fig. 4, the spectral object ‘SIOM’ with different parts passing through different wavelengths has been selected, and the illuminating source is a xenon lamp. The original spectral images of ‘SIOM’ obtained by CCD1 detector placed in the first image plane ‘b’ with corresponding narrowband filter in front of it are shown in Fig. 4 (pixel size is equal to the theoretical resolution of reconstructed images by GISC spectral camera for comparing them). The corresponding modulated object intensity distribution Y is achieved by CCD2 detector of GISC spectral camera and the reconstructed spectral images of ‘SIOM’ with 30% sampling rate of 3D date-cube are shown in Fig. 5. The comparison between the original and reconstructed spectral images shows that the resolution of GISC spectral camera is in accordance with the theoretical calculation.

Figure 4. The original spectral images of ‘SIOM’ obtained by CCD1 detector placed on the first image plane ‘b’ with corresponding narrowband filter in front of it, showing all the channels from 620 ~ 680 nm.

Figure 5. The reconstructed spectral images of ‘SIOM’ with 30% sampling rate of 3D date-cube, showing all the channels from 620 ~ 680 nm.

The images of the outdoor scene consisting of Mario & Luigi with sunlight illumination are shown in Fig. 6. Figure 6(a) is obtained by a camera, while Fig. 6(b,c) respectively show the pictures taken by CCD1 detector with narrowband filters of 550 ± 10 nm and 650 ± 10 nm in front of it (pixel size is equal to the theoretical resolution of reconstructed images by GISC spectral camera for the sake of comparison). The reconstructed spectral images of Mario & Luigi with 30% sampling rate of 3D date-cube are shown in Fig. 7. The experimental results show that the spectral imaging ability of GISC spectral camera for complex scenes is also pretty good.

Figure 6.

Mario & Luigi taken by (a) a camera; (b) CCD1 detector passing through narrowband filters of 550 ± 10 nm; (c) CCD1 detector passing through narrowband filters of 650 ± 10 nm.

Figure 7. The reconstructed spectral images of Mario & Luigi with 30% sampling rate of 3D date-cube, showing all the channels from 520 ~ 580 nm and 620 ~ 680 nm.

Discussion and Conclusion

Based on Information Theory, the transmitted information of an imaging system can be described by the entropy²,³

where p(x_i) is the probability of x_i occurrence. For the conventional direct point-of-object-space to point-of-image-space imaging mode, the conditional entropy H(X|Y) = 0, and thus the channel capacity of the conventional monochrome camera is

where I(X; Y) is the mutual information, the maximum information entropy of source X for conventional imaging instrument, which is the Shannon Limit of the imaging system. According to the principle of maximum entropy¹, the information content of an image is maximized when p(x_i) is Gaussian distribution with average power constraints, which doesn’t contain any useful information. Therefore, the entropy of the image with structured information H(X) has

Eq. (30) shows that the image information acquisition efficiency of such conventional point-to-point imaging mode is lower than the Shannon Limit determined by Information Theory in optical imaging instruments. The channel capacity of an imaging system based on Information Theory for conventional optical imaging instruments is⁴,⁵,⁶,⁷

where m is signal to noise ratio (SNR), N_DOF is degrees of freedom and has

where N_t, N_s, N_c and N_ϕ are respectively time, spatial, color and polarization degrees of freedom. Spatial degrees of freedom N_s has⁶

where S is the image area, W is the space bandwidth, α_x, α_y and N_x, N_y are respectively the image-space aperture angle and the resolved pixel number in the image-space of coordinate x and y. The color degrees of freedom N_c depend on the number of spectral channels, while polarization degrees of freedom N_ϕ is determined by the independent polarization state. According to Eqs (28,31,33), the channel capacity of the con-ventional camera in our experiment (where ) is , and the corresponding transmitted information of Fig. 6(b) is . In order to transmit the 520 ~ 580 nm wavelength ranges data, the required channel capacity of the conventional camera (where ) is C ≈ 1.63 × 10⁶, while the required channel capacity in GISC spectral camera with 30% sampling rate in our experiment is C₃ ≈ 4.90 × 10⁵. C₃ < C₂ shows that GISC spectral camera has the higher information acquisition efficiency in a single exposure compared to the conventional camera. With the development of optical imaging technology, many new imaging technologies (such as CT image⁴⁶) are not based on the point-to-point imaging mode. However, because the correlation between pixels of image data doesn’t be applied in the imaging reconstruction algorithm, the information acquisition efficiency of those new coding imaging technology also can’t approaching the Shannon Limit determined by Information Theory for conventional optical imaging instruments. However, GISC imaging solution applies a spatial random phase modulation to satisfy the restricted isometry property (RIP)¹⁷ required by applying CS that makes the improvement of information acquisition efficiency of the imaging system possible. Comparing with CS imaging technology (such as Single-Pixel Imaging via Compressive Sampling⁴⁷, coded aperture snapshot spectral imagers⁴⁸), which forces on the compressive sampling of electric signal after photoelectric conversion to improve the channel capacity utilization efficiency of the electric signal, GISC imaging solution improves the optical channel capacity utilization efficiency and achieves the compressive sampling of the image data during the imaging acquisition process, which opens the way of approaching the Shannon Limit determined by Information Theory in optical imaging instruments. As a new optical imaging technology, GISC spectral camera provides a unique solution for the spectral imaging of dynamic processes. This GISC imaging solution may also be expanded to other multi-dimensional information (such as polarization information) acquisition⁴⁹, ultra-fast measurement⁵⁰, and super-resolution imaging¹⁸,⁵¹,⁵².

Additional Information

How to cite this article: Liu, Z. et al. Spectral Camera based on Ghost Imaging via Sparsity Constraints. Sci. Rep. 6, 25718; doi: 10.1038/srep25718 (2016).