Thermal Image Enhancement through the Deconvolution Methods for Low-Cost Infrared Cameras

Abstract

An infrared (IR) thermal camera may provide a tool for real-time temperature monitoring for precise disease treatment using heat generated by light-induced photosensitisers, i.e. photothermal/ablation therapies. In this work, we quantitatively demonstrated that the spatial resolution of a low-cost low-resolution IR camera could be improved via two deconvolution methods. The camera point spread function (PSF) was modeled experimentally and used to develop the deconvolution methods: 1) Richardson-Lucy blind deconvolution (BD); and 2) total variation constrained deconvolution (TD). The experimental results showed the improved spatial resolution (at 50% modulation transfer function (MTF): from the original 1.1 cycles/mm to 2.6 cycles/mm for the BD method and to 4.8 cycles/mm for the TD method) as well as contrast-to-noise ratio. With a properly chosen parameter, the TD method can resolve 1-mm size objects with the accurate temperature reading. The thermal image from the low-resolution IR camera enhanced by the TD method is comparable to that from a high-resolution IR camera. These results show that the TD method provides an effective way to improve the thermal image quality from a low-cost IR camera to monitor temperature of an object of 1-mm size, which meets the needed precision for advanced laser scanning protocols in photothermal/ablation therapies.

1. Introduction

Photo-thermal or -ablation therapy is a minimally invasive treatment option for cancer or other diseases. It involves the injection of photosensitisers that can convert light, most in the near infrared (NIR) range for a greater depth than visible light, into heat, which can destruct the targeted cells [1–3]. Once the photosensitisers, preferably in the nano-scale for the enhanced permeability and retention effect for the tumor [4], accumulate in the target area, the NIR light can shine on it with desired parameters (such as wavelength, power and exposure time) to reach and maintain the temperature for an effective treatment. The conventional NIR light delivery is shown in Fig. 1a, where a large treatment spot covers the whole disease area (cm size). A prolonged exposure (30~60 minutes) is usually needed for photothermal therapy at lower temperature (around 40–45°C) for irreversible cell damage due to the indirectly cell killing mechanism [5]. With the advance of multi-functional photosensitisers that are capable of serving both imaging and therapeutic purposes simultaneously [3], more precise laser delivery with a focal laser spot around 1-mm size and the scanning mode through the treatment area can be used as shown in Fig 1b under the real-time image guidance. In such a scenario, much higher temperature (>50°C) can be used to achieve the ablation effect in a short period of time, i.e. coagulative necrosis, thus leading to a more precise treatment, i.e. less treatment margins in Fig 1b compared to that in Fig 1a, and possibly shorter treatment time. Although experiments can be conducted before the real treatment to determine desired parameters for NIR laser, real-time temperature monitoring is critical for a fast and effective photothermal or ablation treatment.

Fig. 1.

Methods for NIR light delivery and real-time temperature monitoring in photothermal/ablation therapies: a) Large treatment spot covering the whole tumor; b) mm-size spot scanning through the tumor; and c) the contact and non-contact temperature monitoring methods.

There are two ways to monitor real-time temperature during the photothermal/ablation treatment as shown in Fig. 1c: 1) the contact monitoring method using thermocouples attached or embedded in tissues [6, 7]; and 2) the non-contact monitoring method using an infrared (IR) camera [8, 9]. The non-contact method has no interference with the treatment area and is capable of monitoring a large surface area. Note that the surface temperature is not the same as the temperature inside the tissue. The temperature drop may not be a problem for superficial tumors that are a couple of millimeters in depth, which is the case for most photothermal therapy purposes. However, when the depth beyond a couple of millimeters, the bio-heat transfer models [6, 8, 9] have to be used to derive the temperature distribution inside the tumor and tissue, where the surface temperatures provide a necessary boundary condition.

To identify a cost-effective temperature monitoring method becomes an essential step to apply real-time temperature monitoring for precise photothermal/ablation therapies. All objects, which are above absolute zero degrees Kelvin (−273.13°C), emit IR light. The IR light emitted from the object can be detected by the IR detectors to convert to temperature readings [10]. The IR detectors generally fall into two broad categories: (1) photon detectors utilizing the photoexcited charge carriers to generate outputs; and (2) thermal detectors absorbing the incident IR radiation to change the detector material temperature to generate electrical outputs. Although the photon detectors can achieve high sensitivity and fast response, they require cryogenic cooling, which leads to bulky imaging systems and high cost. In contrast, the thermal detectors can work in room temperature and be used for compact thermal cameras with the focal panel arrays (FPA) to generate thermal images for a two-dimensional temperature distribution (“thermal image”). In addition, for photothermal treatment, the distance between the thermal camera and the target is not too far so that thermal detector based cameras are capable of real-time temperature monitoring. The commercially available thermal cameras (based on thermal detectors) range from several hundred dollars to more than ten thousand dollars, depending on focal length, sensor arrays and spatial resolution. The low-end thermal cameras are mainly limited by spatial resolution compared to the high-end cameras. For example, both FLIR C2 (MSRP $699) and FLIR A325sc (MSRP $11,950) cameras use uncooled Vanadium Oxide (VoX) microbolometer FPA for longwave IR (LWIR) (7.5 µm to 14 µm (C2) or 13 µm (A325sc)) detection (Lepton® core). FLIR A325sc is more than ten times more expensive than FLIR C2 mainly due to the greater FPA size (A325sc: 320×240 vs. C2: 80×60) and the longer range of focus (i.e. better lens). Nevertheless, the temperature measurement accuracy is the same as ±2°C or ±2% of reading for both cameras. Advanced image processing methods may significantly improve image quality of low-end thermal cameras, which makes them a cost-effective and attractive way for real-time temperature monitoring in photothermal/ablation therapies.

Similar to the conventional visible light imaging systems, a thermal imaging camera can be modeled as a transform function, which maps the object to the image space [11]. For various image degradations, e.g. the system blur and noise, advanced image enhancement methods are vigorously pursued, ranging from rudimental analytic spatial filtering [12] to advanced iterative image restoration methods [13]. If the linear shift-invariant (LSI) condition is satisfied for an imaging system, the image spatial resolution can be improved by the deconvolution methods, such as Wiener deconvolution [12] and Richardson-Lucy deconvolution [14–16], although the improvement is limited by the noise level presented in the image. In the last decade, the total variation (TV) based image denoising and its variations, first proposed by Rudin, Osher and Fatemi [17], has been widely adapted in image enhancement and reconstruction [18–22], thanks to the breakthrough of the compressed sensing theory [23, 24]. Not only can the image spatial resolution be improved, but also the noise can be effectively suppressed.

In this study, we propose to use advanced deconvolution methods to enhance thermal images from a low-end thermal camera (FLIR C2) and systematically investigate to what extent the improvement on spatial resolution can be achieved. We designed the thermal imaging experiments under the conditions pertinent to the mm-size spot scanning photothermal therapy method (Fig. 1b) with the proper temperature range (from the ambient temperature 23°C to the general photothermal therapy temperature 50°C) and spatial resolution (in mm size). These well-controlled bench-top experiments are a necessary step to move toward the future clinical investigations. Through this study, we will answer the following question for precise photothermal/ablation therapies: whether can a low-cost thermal camera be used for temperature monitoring of millimeter-size treatment spots using advanced image processing methods?

2. Methods

2.1. Image formation of IR thermal cameras

The IR light emitted from an object is related to its temperature based on the Stefan-Boltzmann’s law [25]. IR thermal cameras are made of either photon or thermal array detectors that can absorb IR radiation from the target object and convert the photon energy into electrical signals (directly for photon detectors or indirectly for thermal detectors), and subsequently temperature readings [10]. The bolometer is a widely used thermal detector for IR cameras, with a resistive element with very small thermal capacity and large temperature coefficient to produce large changes in resistance for small absorbed IR radiation. The microbolometer array is commonly used in a size of from 80 × 60 pixels to 1024 × 768 pixels for IR thermal cameras. More details about how thermal detectors convert received IR radiation into temperature readings can be found in [10].

In this work, we intend to address the problem of IR light spread in 2D image space due to the optical lens and the detector of the camera. As a simplified model (where the conversion from recorded IR radiation flux to temperature is omitted), a linear system [11] can be used to describe the temperature imaging of the IR thermal camera:

$$g=\mathit{\text{Hf}}+n,$$ (1)

where g ∈ R^MN×1 is a lexicographic vector of the observed image (2D M×N temperature distribution) from the IR camera, f ∈ R^MN×1is a vector of the unknown true temperature distribution of the target scene, n ∈ R^MN×1 is a noise vector, and H ∈ R^MN×MN is a system matrix that defines the mapping from f to g. The aim is to recover f from g by solving the inverse problem of Eq. (1) using the estimated H under the linear shift invariant (LSI) condition.

2.2. Estimation of the system matrix H

It is critical to accurately estimate the system matrix H for the success of restoration of f from g. If the thermal imaging system can be approximated by a LSI system, the point spread function (PSF) is all needed to obtain H, which denotes the convolution operation between the PSF and the image f. However, the ideal point source is hard to achieve and also suffers the photon starving problem to estimate the PSF directly using an IR camera. In this work, we use the knife edge method to get the PSF by assuming a rotational symmetry of the IR imaging system [26, 27]. First, a sharp edge can be used to estimate the edge spread function (ESF). Then, the line spread function (LSF) can be obtained by taking the derivative of the ESF. Observed that the LSF is the integral of the 2D PSF along one dimension, the Fourier transform of the LSF is the system transform function (STF) (i.e. the Fourier transform of the PSF) evaluated in one dimension in the spatial frequency domain. As the imaging system is rotationally symmetric (verified by experiments), the 2D STF can be obtained by rotating the 1D STF in the frequency domain, and then the inverse Fourier transform can be performed to get the PSF in the spatial domain. An alternative way is to use the Hankel transform to get the rotational symmetric PSF in a polar coordinate [28].

2.3. Total variation (TV) regularized deconvolution (TD) method

The inverse problem of Eq. (1) can be solved using total variation (TV) regularized deconvolution originated from the classical TV denoising problem [17]:

$$\hat{f}=\text{arg }\underset{f}{\text{min}}\frac{\mu }{2}{\Vert \mathit{\text{Hf}}-g\Vert }_2^2+{\Vert \nabla f\Vert }_1,$$ (2)

Where μ is a weighting parameter balance the first term of data fidelity (l₂ norm) and the second term of the discrete TV norm of the image f, i.e. ${\Vert \nabla f\Vert }_1={\Vert f\Vert }_{\mathit{\text{TV}}}=\sum _i|{(\nabla f)}_i|=\sum _i\sqrt{{({\nabla }_{x}f)}_i^2+{({\nabla }_{y}f)}_i^2}$, and ∇ is the gradient operator (denoting a difference operation in the discrete form). In this work, we used the alternating direction method of multipliers (ADMM) [29–31] to solve this classical TV regularized optimization problem, although other methods, such as the Nesterov method [32], the fast iterative shrinkage thresholding algorithm (FISTA) [33], and the first-order primal-dual (FOPD) method [34] etc., can be used as well. More details of ADMM [18] can be found in Appendix.

2.4 Richardson-Lucy blind deconvolution (BD) method

The Richardson-Lucy blind deconvolution method has been proposed for decovolution when the Poisson distribution is assumed for the photon detection process and the prior information of the PSF is unknown [14]. The maximum (log-)likelihood (ML) estimate of the deconvolved image f given the measurement g is

$$\hat{f}=\text{arg }\underset{f,\mathit{\text{PSF}}}{\text{max}}\sum _i{g}_i\text{ log}{(f\otimes \mathit{\text{PSF}})}_i-{(f\otimes \mathit{\text{PSF}})}_i,$$ (3)

where f ⊗ PSF = Hf denotes the convolution with the PSF and the constant term not related to f is omitted. The Richardson-Lucy deconvolution method [15, 16] can be used to solve Eq. (3) if the PSF (i.e. H) is known and fixed. Although we model the PSF for the thermal camera in this work, we will further investigate the possibility and benefit of refining the PSF using the modeled PSF as the start point of the blind deconvolution [14]. For each main blind deconvolution iteration (labeled by n), assuming f̂ⁿ⁻¹ and PSFⁿ⁻¹ are known from the last iteration, the PSF will be updated using the Richardson-Lucy method for t=0, 2, …, T−1, sub-iterations:

$${\mathit{\text{PSF}}}_{t+1}^n=\left\{\right[\frac{g}{{\mathit{\text{PSF}}}_t^n\otimes {\hat{f}}^{n-1}}]\otimes {\hat{f}}^{n-1}(-)\}{\mathit{\text{PSF}}}_t^n,$$ (4)

where ${\mathit{\text{PSF}}}_0^n={\mathit{\text{PSF}}}^{n-1}$, f̂ⁿ⁻¹ (−) is the mirror image of f̂ⁿ⁻¹ and the division and multiplication are element-wise. Once the PSF update is done (by setting ${\mathit{\text{PSF}}}^n={\mathit{\text{PSF}}}_{T-1}^n$), the image will be updated similarly as follows:

$${\hat{f}}_{t+1}^n=\left\{\right[\frac{g}{{\hat{f}}_t^n\otimes {\mathit{\text{PSF}}}^n}]\otimes {\mathit{\text{PSF}}}^n(-)\}{\hat{f}}_t^n,$$ (5)

where ${\hat{f}}_0^n={\hat{f}}^{n-1}$, PSFⁿ (−) is the mirror function of PSFⁿ and the division and multiplication are element-wise. ${\hat{f}}_{T-1}^n$ is then set as f̂ⁿ and the iteration of Eq. (4) and (5) continues until the convergence.

3. Experiments and results

We used a FLIR C2 LWIR thermal camera to conduct the experiments for PSF estimation and deconvolution-based image enhancement. The parameters of FLIR C2 are: FPA 80×60, spectral range 7.5–14 µm, thermal sensitivity <0.1°C, field of view 41°×51°, minimum thermal focus distance 0.15 m, temperature range −10°C to 150°C, accuracy ±2°C or 2% of the reading. The thermal images of steel strips, hot water, black tape, and 1-mm-diameter holes on a black acetal sheet (3.175 mm thickness) were interpolated four times to a size of 320×240. In photothermal therapy, the ideal temperature for cancer cell destruction is usually around 50°C. Thus, the range of the temperature is generally chosen between 23°C (the room temperature) and 50°C. For comparison, we also took thermal images of the phantom using the high-performance FLIR A325sc. A handheld thermometer with Type T thermocouple (Omega® HH508) was used to measure the temperature as the ground truth, unless otherwise stated.

3.1. PSF estimation using the strip edge

To estimate the PSF of FLIR C2, the thermal image of the steel strips positioned between the camera and the hot water was acquired. The strip edge produced a knife-edge in the thermal image as shown in Fig. 2a. The rectangle region (35 pixels × 110 pixels) is chosen to calculate the ESF by averaging across all rows to alleviate the noise effect (Fig. 2b). And then the LSF (blue solid line in Fig. 2c) was obtained by taking the derivative of the ESF (where the absolute values of the discrete difference were normalized to have a unit maximum). To further suppress the noise, a Gaussian function was used to fit the original LSF by least-squares fitting. The fitted LSF is shown as the red dashed line in Fig. 2c. The resulting 1D modulate transfer function (MTF, magnitude of Fourier transform of LSF) and 2D PSF are shown in Fig. 2d and 2e. Spatial solution of the camera is determined as 1.1 cycles/mm at 50% MTF and 1.8 cycles/mm at 10% MTF.

Fig. 2.

Estimation of the PSF of FLIR C2: a) the original knife edge image; b) edge spread function (ESF); c) original and fitted line spread functions (LSFs); d) modulate transfer function (MTF); and e) point spread function (PSF).

3.2. Quantitative spatial resolution enhancement

The knife edge image showed in Fig. 2 was enhanced using the TD and BD methods. Fig. 3a and 3b show the ESF and the MTF of the original image and the images processed by the TD and BD methods, respectively. The ESFs of the images processed by the TD and BD methods become steeper than the original ESF. The ESF from the TD method not only achieves the steepest edge, but also suppresses the overshooting problems suffered by the BD method. The TD method increased the spatial resolution to 4.8 cycles/mm at 50% MTF and 8.6 cycles/mm at 10% MTF, compared to the original resolution of 1.1 cycles/mm at 50% MTF and 1.8 cycles/mm at 10% MTF. The BD method improves the spatial frequency solution to 2.6 cycles/mm at 50% MTF and 4.5 cycles/mm at 10% MTF. Both deconvolution methods substantially increased the spatial resolution of the original thermal image. It is noted that the final PSF in the BD method is very similar to the modeled PSF although Eq. (4) was used to update it iteratively.

Fig. 3.

Edge spread functions (ESFs) and modulate transfer functions (MTFs) of the original and processed knife edge images.

3.3. Noise properties after enhancement

In this experiment, we investigate the temperature fluctuation in both high temperature and low temperature after TD and BD enhancement. The hot water thermal image was taken at 50°C calibrated by the thermometer. Fig. 4 shows the original thermal image and the processed images using the TD and BD methods. A square region of interest (ROI) was used to calculate the maximum, minimum, mean and standard deviation (“Std”) values of the temperatures as shown in Table 1. From the Table 1, it can be seen that the TD and BD methods maintains the low noise properties of the original thermal image in general. It is worth noting that TD reduces the standard deviation from the original 0.2993 to 0.2543, while BD raises it to 0.3229.

Fig. 4.

The original and processed thermal images of hot water at 50°C. (Square box: ROI for quantitative measures in Table 1)

Table 1.

The temperature measures for the ROI in thermal images of hot water at 50°C

	Max	Min	Mean	Std
Original	50.0	48.3	49.3	0.2993
TD	49.8	48.4	49.3	0.2543
BD	50.0	48.3	49.3	0.3229

At the room temperature of 23°C, the thermal image of a piece of black tape was acquired. Fig. 5 shows the original and processed thermal images along with the square ROI for quantitative temperature measures. The values after the deconvolution processing are similar to the original ones. A slight decrease on the standard deviation is observed for both TD (0.1067) and BD (0.1198), compared to the original (0.1226) (Table 2).

Fig. 5.

The original and processed thermal images of black tape at 23°C. (Square box: ROI for quantitative measures in Table 2)

Table 2.

The temperatures measured for the ROI in thermal images of black tape at 23°C

	Max	Min	Mean	Std
Original	23.5	22.9	23.1	0.1226
TD	23.3	22.8	23.1	0.1067
BD	23.4	22.6	23.1	0.1198

3.4. Enhancement of the stripe image

A plate with steel strips (strip width = 6.35 mm and gap between two adjacent strips = 6.35 mm) was placed between the hot water and FLIR C2 and a stripe image was taken. Fig. 6 shows the original (a) and processed thermal images ((b) for TD and (c) for BD). Fig. 6d shows the 1D temperature profiles along the 120th row the images. The original thermal image (Fig. 6a) suffers the blurred edge of the strips and lowered contrast between the high temperature (hot water IR pass through the gaps between steel strips) and the low temperature (ambient IR from the steel strips) due to the low spatial resolution of FLIR C2. The strip edges and image contrast are significantly improved using TD and BD. Better uniformity is also observed in the high and low temperature areas in the processed images. TD produces a sharper image than BD. The contrast improvement can be better appreciated from the 1D temperature profiles in Fig. 6d, where the TD profile is closest to 50°C hot water temperature and 23°C room temperature. The flat regions are observed in the TD profile, which is caused by the TV regularization. It can also be seen that TD causes some overshooting (temperature more than 50°C), which can be adjusted by the parameter μ (see Section 4.6). The Contrast-to-noise ratio (CNR) was used to quantitatively evaluate the image quality as follows:

$$\mathit{\text{CNR}}=\frac{T_{\mathit{\text{HW}}}-T_{\mathit{\text{SS}}}}{\sigma },$$ (6)

where T_HW and T_SS are the hot water temperature passing through the gaps between the strips and the temperature of the steel strips (equal to the room temperature), respectively, σ is the noise standard deviation calculated in a square uniform region at the right top corner of each image. The CNR values of all 11 strips are plotted in Fig. 7. The CNR values of images processed by TD and BD are improved significantly, compared to that of the original image. The improvement on CNR is about 2.4~4.0 times for the TD method and about 1.5 times for the BD method over the original image.

Fig. 6.

Thermal image of a steel strip plate: a) original; b) TD processed; c) BD processed; and d) 1D temperature profiles of 120^th row of thermal images, which is indicated by the black arrows.

Fig. 7.

The contrast-to-noise ratio (CNR) values of 11 strips in the stripe images.

3.5. Enhancement of 1-mm-diameter hole pattern image

A hole pattern (6×7) was drilled on a chemical-and-wear resistant black acetal plate (3.175 mm thickness) as shown in Fig. 8. The hole diameter is 1 mm (±50 µm) and the distance between the centers of two holes is 2 mm. The thermal image was taken when the plate was placed between FLIR C2 and the hot water. The original thermal image, the TD processed image and the BD processed image are shown in Fig. 9. The original image is blurred and the shape of the holes are severely distorted. The holes in the TD and BD images are more uniform and easier to differentiate from each other than that in the original image. The 1D temperature profiles across the center of each of six rows of holes for these images are shown in Fig. 10. Both TD and BD improved the contrast notably (lower valley and higher crest). The profiles of the TD image have winder and flat crests than that of the original image and the BD image.

Fig. 8.

Diagram of the hole-pattern on a chemical-and-wear resistant black acetal sheet (3.175 mm thickness).

Fig. 9.

The original thermal image of the hole pattern and images processed by the TD and BD methods.

Fig. 10.

1D temperature profiles across the centers of six rows of holes for the original and processed thermal images.

Three quantitative measures (the diameter of each hole, the distance between two adjacent holes and the CNR) are also calculated and compared as follows. The diameter of each hole is calculated as the full-width half-maximum of each peak and shown as the bars in Fig. 11 (different colors are used to differentiate different columns). The range of the diameters of the holes is improved from 0.83–1.5 mm for the original to 1.0–1.5 mm for TD and 1.0–1.3 mm for BD. The processed images provide more uniform diameter measures and closer to the true value (1 mm). The distance of two adjacent holes is calculated as the distance between two adjacent peaks and shown as the bars in Fig. 12. The range of the distances between two adjacent holes reduces to 1.7–2.3 mm for TD and 1.8–2.3 mm for BD, compared to 1.5–2.5 mm for the original. Again, the images with deconvolution yield less fluctuation and more accurate measure than the original image. Finally CNR is calculated as the difference between the peak value and the valley value normalized by the standard deviation of noise. A square area (25 pixels × 25 pixels) in the left top of the image (Fig. 9) is chosen to calculate the standard deviation of noise, which is 0.0695, 0.2309 and 0.3949 for the original image, the TD image and the BD image, respectively. The CNR values for each of seven holes in each row are plotted in Fig. 13. The TD method improves CNR in most cases. The maximum improvement is about 5 times of the original. Although the BD method boosts the signal, the maximum CNR values is only about 2 times of the original and some CNR values indeed decrease due to the elevated noise standard variation (from 0.0695 to 0.3949), likely due to the ringing effect of deconvolution. In contrast to BD, the TD method can improve the image contrast more with less ringing artifacts. These results demonstrate that the TD method can resolve the mm-size object with high CNR, which is not possible for the original FLIR C2 image.

Fig. 11.

Diameter measurements for all holes using the original and processed thermal images (ground truth diameter: 1 mm).

Fig. 12.

Distance measures between two holes in horizontal direction using the original and processed thermal images (ground truth distance: 2 mm).

Fig. 13.

CNR plots along each row of holes for the original and processed thermal images.

3.6. True temperature recovery using TD

The hot water temperature was measured as 41.4°C by the thermometer when the hole pattern thermal image was captured by FLIR C2. This true temperature was shown as the blue dashed line in Fig. 14. A ROI containing four holes at the bottom right corner were selected as the camera directly pointed to this area. The average temperature for these four holes using the original image from FLIR C2 is 31.3°C as shown the red line in Fig. 14 and the original image was also shown in the bottom left corner in the plot. For the TD method, the average temperature change with the parameter μ (from 1 to 60 with a step size of 1) in Eq. (2) is shown as the black dashed curve in Fig. 14. The images for μ=10, 20, 30, 40, and 50 are shown at the corresponding temperature points (black dots). As can be seem, the holes become clearer and clearer in the deconvolved images as the value of μ increases. When μ=20, the average temperature is the closest to the true temperature and the image quality is greatly improved compared to the original image. When μ keeps increasing beyond 20, the average temperature deviates from the true temperature, although the contrast is further increased.

Fig. 14.

Temperature (averaged over four holes) change with the parameter μ used in the TD method. The true temperature (blue dashed line) and the average temperature from the original image (red line) are also shown for the reference.

We tested this method for the various temperatures between 23°C and 50°C and found that μ around 20 is a suitable parameter to recover the true temperature and to discern the 1-mm holes under the current experimental setting. When experimental settings (such as the temperature, the object size, and the camera) change, this value of μ may not be suitable anymore. However, as long as the targeted temperature range and the object to be discerned are known, the same calibration process described in this section can be used to determine the appropriate parameter of μ for the follow-up temperature monitoring.

3.7. Comparison between FLIR C2 and FLIR A325sc

Finally, to compare the effectiveness of the TD method, we used FLIR A325sc with much higher spatial resolution (IR sensor 320 pixels × 240 pixels). The original image taken by FLIR C2, the processed image using the TD method and the image taken by FLIR A325sc are shown in Fig 15. The original image of FLIR C2 is unable to resolve each individual hole, while that of FLIR A325sc shows clearly defined holes. The TD method greatly improved the FLIR C2 image and resolved the 1-mm diameter holes. The uniformity of the whole image for the TD processed FLIR C2 image is not as good as FLIR A325sc. However, the uniformity of each hole is better for FLIR C2 image than that for FLIR A325sc.

Fig.15.

Comparison of thermal images from FLIR C2 (original and processed by the TD method) and FLIR A325sc.

4. Conclusions

In this work, we developed two deconvolution methods, Richardson-Lucy blind deconvolution (BD) and total-variation regularized deconvolution (TD), for thermal images from a low-cost thermal camera (FLIR C2) and investigated their image enhancement performance. With modeling of the PSF of the thermal camera, both BD and TD can substantially increase contrast and the spatial resolution of thermal images. The spatial resolution is improved from the original 1.1 cycles/mm at 50% MTF to 2.6 cycles/mm for BD and 4.8 cycles/mm for TD. Compared to BD, TD can effectively suppress the ringing effect, which is a common problem for deconvolution methods. With the properly chosen weighting parameter for TV regularization used in TD, the true temperature of objects in 1-mm size can be faithfully recovered by the processed thermal image, which is comparable to the image taken by a high-end thermal camera (FLIR A325sc). Future studies will focus on investigating the effectiveness of these methods with living orgasms, such as animals or human subjects, and ultimately combining thermal imaging with photothermal/ablation therapies. This study pave the way to develop a cost-effective real-time temperature monitoring tool for precise photothermal/ablation therapies.

Appendix

ADMM for TV regularized deconvolution

To solve Eq. (2), the alternating direction method of multipliers (ADMM) can be used as proposed in [18]. Briefly, an intermediate variable u (a tensor denoting the gradients of the 2D image) is introduced to transform Eq. (2) into the following equivalent problem,

$$\underset{f,u}{\text{min}}\frac{\mu }{2}{\Vert \mathit{\text{Hf}}-g\Vert }_2^2+{\Vert u\Vert }_1,\text{ subject to }u=\nabla f.$$ (A1)

The augmented Lagrangian problem of Eq. (A1) is

$$L(f,u,y)=\frac{\mu }{2}{\Vert \mathit{\text{Hf}}-g\Vert }_2^2+{\Vert u\Vert }_1-y\text{'}(u-\nabla f)+\frac{\rho }{2}{\Vert u-\nabla f\Vert }_2^2,$$ (A2)

where y is the Lagrange multiplier for the constraint u = ∇f, ρ is a regularization parameter for the quadratic penalty term ${\Vert u-\nabla f\Vert }_2^2$, and ’ denotes the vector transpose.

The saddle point of the augmented Lagrangian L(f,u,y) is the solution of the original problem (Eq. (2)). The ADMM is used to iteratively solve it using the following two minimization updates,

$$f_{k+1}=\text{arg }\underset{f}{\text{min}}\frac{\mu }{2}{\Vert \mathit{\text{Hf}}-g\Vert }_2^2-{y\text{'}}_k(u_k-\nabla f)+\frac{\rho }{2}{\Vert u_k-\nabla f\Vert }_2^2,$$ (A3)

$$u_{k+1}=\text{arg }\underset{u}{\text{min}}{\Vert u\Vert }_1-{y\text{'}}_k(u-\nabla f_{k+1})+\frac{\rho }{2}{\Vert u-\nabla f_{k+1}\Vert }_2^2,$$ (A4)

and the dual update:

$$y_{k+1}=y_k-\rho (u_{k+1}-\nabla f_{k+1}).$$ (A5)

Note that the convolution operator H is a block-circulant matrix and can be diagonalized using discrete Fourier transform matrices. Taking the first order derivative with respect to f in Eq. (A3) to be zero gives the updating formula using Fourier and inverse Fourier transforms. Eq. (A4) can be solved using a shrinkage method. More details of ADMM implementation can be found in [18].