Mitigating fringing in discrete frequency infrared imaging using time-delayed integration

Abstract

Infrared (IR) spectroscopic microscopes provide the potential for label-free quantitative molecular imaging of biological samples, which can be used to aid in histology, forensics, and pharmaceutical analysis. Most IR imaging systems use broadband illumination combined with a spectrometer to separate the signal into spectral components. This technique is currently too slow for many biomedical applications such as clinical diagnosis, primarily due to the availability of bright mid-infrared sources and sensitive MCT detectors. There has been a recent push to increase throughput using coherent light sources, such as synchrotron radiation and quantum cascade lasers. While these sources provide a significant increase in intensity, the coherence introduces fringing artifacts in the final image. We demonstrate that applying time-delayed integration in one dimension can dramatically reduce fringing artifacts with minimal alterations to the standard infrared imaging pipeline. The proposed technique also offers the potential for less expensive focal plane array detectors, since linear arrays can be more readily incorporated into the proposed framework.

1. Introduction

Mid-infrared (IR) spectroscopy is a non-destructive method for obtaining quantitative molecular information from a sample. When incorporated into imaging systems, such as microscopes, IR spectroscopy is a powerful tool for exploring the spatial distribution of molecular constituents in heterogeneous samples. There has been growing interest in integrating high-quality molecular and structural spectroscopic measurements into areas of clinical and biomedical research. For example, standard histology relies on chemical staining to differentiate tissues [1], which is non-quantitative and destroys the sample. Emerging vibrational imaging methods using mid-infrared [2] and Raman spectroscopy [3] have established label-free techniques to extract biomedical information from micrometer-thick samples. No prior knowledge of the sample composition is needed, since many molecular functional groups have resonant frequencies in the IR fingerprint region [4]. Based on this technique, many newly developed IR imaging applications are being explored for biomedical analysis and clinical diagnosis [5–7].

One of the most widely used IR imaging systems is Fourier Transform Infrared (FTIR) spectroscopy, which uses a broadband globar thermal source. Spatial information is acquired by scanning the sample with a single-point detector coupled to an aperture that limits signal to a single spatial location. An alternative approach uses focal plane array (FPA) detectors, which can provide outstanding performance with > 90% quantum efficiency, low readout noise, and low dark current [8]. Coupled with interferometry and liquid nitrogen-cooled mercury cadmium telluride (MCT) FPA detectors [9], FTIR instruments are capable of broadband spectral data acquisition [10] and are widely commercially available. MCT detectors are still relatively low resolution (≈ 128 × 128 pixels) compared to visible-range counterparts, which limits applicability in fields like clinical histology, where throughput is critical.

Several newly developed technologies have focused on increasing throughput, most notably by replacing incoherent thermal sources with coherent beams, such as synchrontron radiation [11], which provides higher flux for improved SNR. By taking advantage of coherent sources and FPA detectors, recent research has focused on higher throughput and spatial resolution [12] and opened the door to three-dimensional imaging [13]. Increased throughput can also be achieved by limiting the number of spectral measurements acquired. This can be accomplished in FTIR by reducing the scanning range of the interferogram, which is proportional to the spectral resolution. This approach provides very limited specificity, since the user has no control over the vibrational states that are probed.

More recent approaches, collectively known as discrete-frequency infrared (DFIR), provide more control over the acquired wavelengths. Multiple methods have been proposed for DFIR, including the use of narrowband filters [14, 15] and quantum cascade lasers (QCLs) [16]. Currently, QCL-based imaging systems are more common since they provide more excitation energy within an extremely narrow band. QCLs can be constructed for tunability through the fingerprint region [17], have a very narrow (< 1cm⁻¹) line width, and are operable at room temperature [18–20] using uncooled bolometer FPAs [21]. This allows precise probing of specific vibrational modes using bench-top systems. Imaging systems incorporating these techniques are commercially available [22] (Spero, Daylight Solutions), and have been used in tissue studies [23–25] and cell biology [26], with a general movement towards clinical applications [27].

However, one of the major drawbacks of these imaging systems is that the coherent illumination generates fringing patterns in the detector image (Fig. 1 ). These patterns are difficult to remove using image processing methods, since the pattern structure is dependent on wavelength and filtering spatial frequencies can distort the actual spatial content of the image. Consequently, these artifacts also cause meaningful changes in the chemical measurement that is intrinsically tied to the sample structure and substrate. Recent research has focused on mitigating coherence effects using beam diffusion or apertures combined with point detection [28]. An alternative approach relies on obtaining point measurements in reflection to rasterize the final image [29]. Confocal apertures can then be inserted to reject the scattered light that produces the fringing pattern. This approach effectively eliminates coherence effects by simulating an extended source, however throughput is limited by the scanning speed of the sample stage.

Fig. 1.

Examples of fringing artifacts in Gaussian beam coherent imaging systems. (a) Transmission image of SU-8 patterned US air forth 1951 optical resolution target on barium fluoride (BaF2) substrate imaged with single diverged QCL beam on FLIR A305 thermal camera (FLIR Systems, Nashua, NH, USA), (b) Absorbance image of breast tissue cores at 1050 cm⁻¹.

In this paper, we propose a hybrid approach that allows developers to maintain the throughput allowed with FPA detectors while significantly mitigating coherence artifacts in the final image. This is accomplished by using time-delayed integration (TDI) to simulate an extended source in one dimension. This allows us to fully utilize an FPA detectors, where one dimension provides spatial resolution and the second provides integration that reduces both noise and coherence fringing.

2. Materials and methods

2.1. Simulation

We first test the viability of the proposed technique using a Mie scattering forward model that utilizes both coherent and incoherent sources [30,31]. We approximate the absorbance image produced by a set of neighboring spheres with varying absorption values (Fig. 2 ). A traditional Gaussian beam coherent imaging system is approximated using a small condenser numerical aperture and a single point source (Fig. 2c–d). The proposed TDI imaging technique approximates a one-dimensional extended source by moving the sample relative to the source and detector. This is calculated by simulating a set of point sources that are summed incoherently at the detector (ex. Fig. 2b). The simulated intensity field and absorbance images result in a significant reduction in fringing artifacts and a more representative measurement of the heterogeneous absorption (Fig. 2e–f). For reference, a completely incoherent extended source using a high-NA objective and two-dimensional point source distribution is used to reflect a traditional FTIR image (Fig. 2g–h).

Fig. 2.

Bim-Sim simulation of spheres illuminated by Gaussian beam and extended source I₀ integrated by multiple point sources. (a) 6 spheres with the same size but different refractive indexes. (b) 4 × 8 Extended source. (c) Transmission image of spheres illuminated by Gaussian beam with NA = 0. (d) Absorbance image, Gaussian beam, NA = 0. (e) Transmission image, 4 × 16 extended source, NA = 0.2. (f) Absorbance image, 4 × 16 extended source, NA = 0.2. (g) Transmission image, 32 × 32 extended source, NA = 0.8. (h) Absorbance image, 32 × 32 extended source, NA = 0.8.

2.2. Instrumentation

We have developed a custom IR molecular imaging system (Fig. 3 ) integrated with a four-channel QCL (MIRcat-1400-PX-B, Daylight Solutions, CA, USA) with a pulsed tuning range from 5.10 μm (910 cm⁻¹) to 10.99 μm (1900 cm⁻¹), maximum peak power 500 mW and pulse repetition rate from 0.1 kHz to 1 MHz. A flat guide mirror reflect the two overlapped beams onto the sample, which is mounted on a high resolution (0.1 μm) 3D stage (MPS50SL-050, AEROTECH, PA, USA) moving vertically during imaging. The image is then collected by a 0.56 NA BD-2 glass AR coated objective (LightPath Technologies, Orlando, FL, USA) and focused onto a SBF161 128 × 128 pixel focal plane array (FPA) mercury cadmium telluride (MCT) detector (Teledyne Technologies, CA, USA). This provides a pixel size of 6.25 μm. MCT FPA detectors have much shorter response time (microseconds) compares to uncooled IR bolometers or thermal cameras, which can only be operated in millisecond range. A HeNe alignment laser (HNL202L, Thorlabs, NJ, USA) is used to aid in optical alignment.

Fig. 3.

System Diagram. The QCL-based DFIR imaging system is coupled to a 4 channel ultra-broadly tunable mid-IR external-cavity pulsed quantum cascade laser and a SBF161 128 × 128 pixel focal plane array (FPA) mercury cadmium telluride (MCT) detector.

The MCT detector has the highest noise-equivalent power (NEP) ratings among broadband IR detectors, allowing high SNR images of the raw QCL output. Additional attention must be paid to the optical path length in atmosphere in order to mitigate atmospheric absorption. In our system, all optical paths are minimized to avoid significant water vapor and carbon dioxide absorption in the finger-print region. Alternatively, the system can be nitrogen-purged.

2.3. Image acquisition

A fabricated USAF 1951 pattern was created on a BaF2 substrate using an SU8 negative photoresist. The QCL was set to scan the target from 1500 cm⁻¹ to 1650 cm⁻¹ with 2 cm⁻¹ resolution, within which we can see three major spectral features. The target image is taken using a software controlled 3D stage (Aerotech, Inc.) translated vertically with a 5 μm step size, coupled with a liquid nitrogen cooled MCT FPA detector (Teledyne Dalsa), providing a frame rate of 1612 Hz and a 0.046 ms integration time. The entire system, including QCL, stage, and detector, is synchronized by a custom C++ application using the available vendor APIs.

We first perform a fully automated tuning step to ensure that the system provides maximum illumination without saturating the detector. After the user specifies a set of discrete wavelengths, the API controls capture of a pixel-averaged image. The QCL power is set such that the detector pixels are not saturated and a background image is collected. This power value is determined using a binary search algorithm that tunes the laser power up or down to determine optimal throughput. This allows us to seamlessly compensate for atmospheric absorbance, maximize throughput, and minimize nonlinear detector behavior near saturation.

During image acquisition, the instrument collects a series of images for integration (Fig. 4 ). Since TDI MCT detectors are not currently available, integration is performed as a post-processing step. For each frame, the stage is moved upwards along Y axis for one step, while frame capture is synchronized by the software based on the stage encoder position. Image capture continues for the length of the sample. Stage can be moved along X axis if the sample being imaged is wider than the field of view (5 μm × 128 = 832 μm). This process is repeated for each additional discrete band.

Fig. 4.

(a) Flow chart of TDI (time delay integration) algorithm. (b) Intermediate image from TDI post process. The top and bottom part of the image demonstrates the SNR difference between images before TDI and after fully applied TDI (center part of the image). (c) Individual images at different position and time point (left) and corresponding integrated image (right).

For each wavenumber, the stage is moved 100 steps with a 5 μm step size. For each step, the frame grabber captures 800 frames and saves them to a buffer, where they are integrated in real-time by our software. Detector noise is subtracted from each image and the TDI algorithm is applied (Fig. 4). The final image size is calculated based on the number of frames and frame spacing (5 μm). Each frame is then aligned into the final image and integrated. The integration direction follows the the direction of stage motion. This allows us to fully utilize the 128 × 128 pixel FPA for one dimensional imaging along the integration direction, which means that the final image is integrated 128 times. In Fig. 4 (b), an intermediate image is used to demonstrate the signal intensity as additional frames are integrated into the final image. All TDI image processing is performed in Matlab (The MathWorks, Nantucket, MA, USA).

2.4. Reference image acquisition

Reference images were collected using an Agilent Cary 620 imaging system for FTIR image comparisons. Note that the FTIR imaging system is capable of collecting absorbance information outside of the 900 cm⁻¹ to 1800 cm⁻¹ fingerprint window. While this is a current limitation of QCL imaging systems, most biomedical chemometrics focus on the fingerprint region. FTIR images are collected using 32 co-additions for the foreground and 64-co-additions for the background with 0.012ms detector integration time and a commercial UDR-4 filter (Agilent Technologies). DFIR comparisons were performed using a QCL source (Daylight Solutions) spread into a Gaussian beam across the entire FPA. An image mosaic was then composed from individual tiles.

3. Results and discussion

Time delayed integration provides a significant performance improvement when compared to mosaic results using a Gaussian beam. Compared to a Gaussian beam DFIR imaging system (Fig. 5(a) (DFIR) ), fringing artifacts are significantly reduced (Fig. 5(a) (DFIR + TDI)) after applying the TDI algorithm. Both results are compared to an FTIR image collected using an incoherent globar source (Cary 620, Agilent) 5(a) (FTIR). The reduced fringing provides a benefit in increased sharpness and the reduced need for post-processing methods such as MNF. Sharper image edges in Fig. 5 (b) (DFIR + TDI) demonstrate better quality than our reference.

Fig. 5.

(a) SU-8 Air Force target on a BaF2 substrate imaged with FTIR (left), traditional DFIR (middle), and DFIR + TDI (right). Images show the same region with USAF group 1 element 5 at 1508 cm⁻¹. The black scale bar denotes 200 μm. (b) Detailed comparison of the same image.

While the pixel sizes between all three images are comparable, sharper edges in the DFIR and DFIR + TDI imaging systems are likely due to higher-NA refractive optics.Quantified sharpness for each image is given out in Fig. 6 . Number 5 region on the target is cropped out for detailed comparison. Y-profile (red dash line) is plotted in Fig. 6 (b) with Y axis stands for absorbance and X axis stands for pixel position.

Fig. 6.

(a) Number 5 region on USAF target imaged with FTIR (top), traditional DFIR (middle), and DFIR + TDI (bottom) at 1508 cm⁻¹. The black scale bar denotes 100 μm. (b) Y-profile plots for each image.

The main bottleneck to this approach is the need to implement TDI integration as a post-processing step. However, several linear arrays are used as detectors in custom and commercial systems and are far less expensive than their 2D counterparts. This research opens the door to the use of linear arrays that can provide a compromise between the throughput benefit of FPAs with the incoherent imaging of point confocal [28] or FTIR imaging systems.

In order to optimize this trade-off between imaging time and signal-to-noise ratio, a SNR analysis was done for the most efficient number of frames that should be averaged. From 10 frames to 800 frames, an 8 fold SNR increase is achieved, with results converging after 800 frames. Aside from imaging frame rate, our QCL-based DFIR system is operated at an integration time 4X longer than that of the FTIR system (which utilizes the same detector) at 0.01 μs integration time. Unlike the globar source, QCL has a maximal pulse rate of 1 MHz and maximal 5% duty cycle. A trade-off between laser power and integration time for each image must be taken into consideration in order to avoid saturation while achieving higher SNR, that is to say, laser power has to be lowered for longer integration time. Finally, a 0.046 μs integration time is chosen to ensure that there is no background saturation.

Quantitative measurement of reduced fringing artifacts is done by plotting the noise profiles as shown in Fig. 6, where X axis stands for pixels across the image and Y axis stands for absorbance (a.u.). The noise level of our system (Fig. 7 (b) ) is significantly lower than that of DFIR (Fig. 7 (a)) both in noise amplitude and variance. Specifically, in Fig. 7 (b), the intensity of the absorbance image is varing from −0.13 to 0.12, whereas in Fig. 6 (a), the image intensity is varing from −0.04 to 0.03. As a conclusion, our system is capable of doing higher-SNR IR imaging even with lower pixel size (6.5 μm) compares to DFIR system which has a higher pixel size (1.4 μm) due to the fact that the final image of our system is integrated from multiple images by Time-Delay Integration Algorithm.

Fig. 7.

Noise profile for standard DFIR imaging (a) and DFIR + TDI (b) showing the affect of fringing in the spatial domain.

For spectral comparison, target region spectra (Fig. 8 ) is given out in 1500 cm⁻¹ to 1650 cm⁻¹ range where three major SU-8 polymer features (peaks at around 1500 cm⁻¹, 1585 cm⁻¹ and 1610 cm⁻¹) can be observed. QCL itself has a power profile for each QCL channel which makes the laser emit different laser power for each wavelength. Moreover, with water vapor and carbon dioxide absorption in fingerprint region, the final image intensity varies along spectral domain which resulting with a noisier spectrum. In QCL-based DFIR system, spectrum smooth algorithm is used to make the spectrum smoother with a 2 cm⁻¹ spectral resolution. As shown in Fig. 8, mosaic images are can be collected using adjacent scans, and the proposed DFIR + TDI system provides comparable spectral quality when compared to FTIR and traditional DFIR.

Fig. 8.

USAF mosaic image and spectral comparison. A USAF mosaic was composed by taking multiple TDI passes and aligning images at their margins. Spectra are compared between the proposed QCL-based DFIR+TDI system (blue), traditional DFIR imaging system (yellow) and FTIR system (green). The spectral range (1500 cm⁻¹ to 1650 cm⁻¹) was selected because it contained the major features of the SU8 photoresist. Outside of this range, the absorbance values become flat.

To test the viability of our system for biomedical applications, we collect absorbance images of breast tissue micro-arrays (TMAs) (Fig. 9 ) for histological classification. These TMA cores (AMS802, AMSbio LLC) contain tumor biopsies (invasive ductal carcinoma) and matched adjacent normal tissue. We create a Bayesian classifier to separate tumor epithelium and the surrounding stroma. The classifier is trained on a database of FTIR breast images. In order to make efficient use of the QCL source tunability, a set of 20 bands are selected from the full FTIR spectrum using genetic algorithm (GA) optimization. [32] Once the optimal 20 bands are determined, these bands are imaged using our DFIR + TDI imaging system. The classifier is then applied per-pixel to separate tumor epithelium from the surrounding stroma. In order to validate the classification, we compare to an adjacent section stained with hematoxylin and eosin (H&E) and imaged using an Eclipse Ti inverted microscope (Nikon Corporation) (Fig. 10).

Fig. 9.

Mosaic images for breast tissue microarrays at the Amide I band (1650 cm⁻¹). Mosaics are composed by taking multiple TDI passes and aligning along the image margin.

Fig. 10.

Breast Tissue Microarrays images, (a) Brightfield image a breast biopsy stained using H&E; (b) absorbance image 1650 cm⁻¹; (c) classified image identifying epithelium (green) and stroma (magenta).

4. Conclusions

In this manuscript, we develop a QCL-based DFIR imaging system coupled with an MCT FPA detector. Time-delayed integration (TDI) is used to dramatically reduce fringing. Based on Mie-scattering simulations, we believe that this reduction in fringing is due to two factors: (a) the increased NA of the condenser, which provides access to a greater variety of scattered spatial frequencies, and (b) incoherent integration in one direction, essentially simulating an extended source. While our pixel size is comparable to a traditional FTIR imaging system, we are able to achieve greater resolution by using high-NA refractive optics, similar to current commercial QCL imaging systems.

This work demonstrates the viability of reducing coherence fringes in coherent infrared imaging systems without resorting to 2D point scanning, which significantly limits scalability. Linear MCT detectors are currently easier to manufacture than their 2D counterparts, and may provide a compromise between current spread-beam QCL sources and confocal systems.

Our current implementation averages 800 frames for noise reduction, providing an inferior acquisition time when compared to existing FTIR instrumentation. Several approaches could be taken to mitigate this problem, the most straightforward being a reduction in the number of frames (ex. 100-200), which would improve acquisition time at a relatively low cost to SNR. Our selection of 800 frames was to take a conservative approach to demonstrate the viability of TDI for eliminating coherence affects. However, more optimal approaches would require modifications to the FPA design, in particular (a) the use of anisotropic arrays (ex. 256×16), which would likely provide a better detector SNR and a wider field of view, (b) on-board support for TDI would mitigating the need for software integration and likely improve frame rate, and (c) the ability to increase the frame rate with a corresponding reduction in integration time, which is not supported by our commercial FPA (Teledyne Technologies, CA, USA). There is therefore significant room of improvement for optimizing DFIR+TDI imaging.

Funding

Cancer Prevention and Research Institute of Texas (CPRIT) #RR140013, the National Institutes of Health (NIH) / National Library of Medicine (NLM) #4 R00 LM011390-02, Agilent Technologies University Relations Grant #3938 and National Science Foundation (NSF) grant #NSF-1151154.

Disclosures

The authors declare that there are no conflicts of interest related to this article.