Multi-channel beam-scanning imaging at kHz frame rates by Lissajous trajectory microscopy

Abstract

A beam-scanning microscope based on Lissajous trajectory imaging is described for achieving streaming 2D imaging with continuous frame rates up to 1.4 kHz. The microscope utilizes two fast-scan resonant mirrors to direct the optical beam on a circuitous trajectory through the field of view. By separating the full Lissajous trajectory time-domain data into sub-trajectories (partial, undersampled trajectories) effective frame-rates much higher than the repeat time of the Lissajous trajectory are achieved with many unsampled pixels present. A model-based image reconstruction (MBIR) 3D in-painting algorithm is then used to interpolate the missing data for the unsampled pixels to recover full images. The MBIR algorithm uses a maximum a posteriori estimation with a generalized Gaussian Markov random field prior model for image interpolation. Because images are acquired using photomultiplier tubes or photodiodes, parallelization for multi-channel imaging is straightforward.

Preliminary results show that when combined with the MBIR in-painting algorithm, this technique has the ability to generate kHz frame rate images across 6 total dimensions of space, time, and polarization for SHG, TPEF, and confocal reflective birefringence data on a multimodal imaging platform for biomedical imaging. The use of a multi-channel data acquisition card allows for multimodal imaging with perfect image overlay. Image blur due to sample motion was also reduced by using higher frame rates.

1. INTRODUCTION

1.1 Background

Nonlinear optical (NLO) imaging has emerged as a robust tool for biomedical imaging of tissues samples¹ due to its inherent ability to image tissues fast, noninvasively, and label-free. Specifically two-photon excited fluorescence (TPEF) and second harmonic generation (SHG) microscopy have been implemented as they provide relatively high penetration depths,² video rate image acquisition,³ and contrast from endogenous components from within the tissues⁴. SHG provides contrast from well-ordered noncentrosymmetric materials, such as collagen, allowing for the visualization of structural details within the tissue sample,⁵ whereas TPEF provides contrast from fluorescing molecules such as keratin or elastin providing chemical information about the tissue sample.⁶ By combining TPEF and SHG on a multimodal platform, contrast can be obtained from different sources, which might not be detectable by using only TPEF or SHG alone. Presented in this manuscript is a multimodal beam-scanning microscope based on a Lissajous trajectory for simultaneous SHG, TPEF, and laser transmittance microscopy with perfect registry between the 3 imaging modes.

1.2 Lissajous trajectory

Lissajous trajectory beam-scanning is based on pairing 2 resonant mirrors together in the scan head operating in a sinusoidal trajectory. By pairing the mirrors, such that the least common multiple of the two periods is relatively high, the trajectory of the laser through the sample plane will allow for all, or most, of the pixels to be sampled. Beam-scanning based on a Lissajous trajectory has several key advantages over conventional galvo-galvo or galvo-resonant mirror pairs used for raster scanning typically employed for NLO imaging. The greatest advantage that Lissajous trajectory imaging provides is the ability to achieve frame rates much higher than video rate, as previously discussed by Sullivan et al.⁷ While increasing the frame rate allows for the visualization of highly dynamic, rapidly changing samples (i.e. visualizing the action potential as a neuron fires), arguably the greatest benefit to in vivo imaging is the reduction in image blurring as a result of sample motion due to organism movement. Several groups have employed a Lissajous trajectory beam-scanning approach with excellent success in both endoscopes and bench-top microscopes, but with frame rates limited to only a few Hz.^8–11

To achieve the high frame rates the Lissajous trajectory is simply separated into partial trajectories in time with the resulting images being sparsely sampled. To interpolate the data in the under-sampled regions, an interpolation algorithm can be used for reconstructing the entire image.

2. MATERIALS AND METHODS

2.1 Instrumentation

The Lissajous beam-scanning microscope has been detailed previously⁷ and will only be described briefly here. The Lissajous microscope utilized a MaiTai HP Ti:Sapphire laser (Spectra Physics) to produce ~100 femtosecond pulses centered around 800 nm, with an 80 MHz repetition rate, and a maximum power of ~3.0 W. The period of the pulsed laser source served as the master clock dictating all the subsequent timing of the resonant mirrors, feedback controls, and data acquisition electronics. The incident beam was coupled into the microscope through the Lissajous scanhead consisting of 2 resonant mirrors (Electro-Optical Products Corp.) scanning in the X and Y direction operating at 15.09 kHz and 13.36 kHz respectively. Using a telocentric lens-pair (Thorlabs f = 150 mm each) the incident beam was then coupled into the back of a .65 NA 50× objective to focus down onto the sample plane. The transmitted fundamental beam and the SHG signal were collected and recollimated by a condenser lens (Thorlabs, f = 25.4 mm). The SHG signal was then split from the excitation beam by reflection off a 405 nm long pass dichroic mirror (Chroma, Z405RDC) and detected by a photomultiplier tube (PMT) (Hamamatsu, H10722-10) after being passed through a filter stack containing a 400 nm interference filter (CVI, F40-400.0-4-1.00) and a KG3 (Thorlabs, FGS900) to reject the remaining incident 800 nm light. The transmitted 800 nm fundamental beam was focused by a lens (Thorlabs, f = 25.4 mm) onto a photodiode (Thorlabs, DET10A) for detection of laser transmittance. Detection of the TPEF signal was performed in the epi (backwards) direction by placing a dichroic in between the two lenses in the telocentric lens pair. The TPEF signal was then passed through a filter stack containing 450 ± 25 nm bandpass filter (Chroma) and KG3 filter (Thorlabs, FGS900), to remove the incident 800 nm, as well as the 400 nm SHG signal, and detected by a second PMT (Buhrle).

2.2 Control electronics and data acquisition

The high Q-factor of the resonant mirrors (Q > 250) resulted in amplitude stability at the sacrifice of phase stability. To stabilize the drift in phase, a custom built Lissajous timing generator (LTG) control box was designed for real-time active phase correction of the resonant mirrors. The LTG used an 8-bit Microcontroller (Silicon Laboratories, C8051f120), running at 80 MHz derived from an external 10 MHz phase-lock loop (PLL) synchronous with the 80 MHz master clock from the laser. A custom built, multitasking preemptive operating system, controlled by the microcontroller in combination with hardware and software timers, produced the drive signals for the X and Y mirrors along with the epoch pulse, which marks the start of the Lissajous trajectory. By converting the sinusoidal feedback of the resonant mirrors into a digital signal, the microcontroller can compute and compensate for X and Y phase error upon the next iteration of the epoch pulse.

Both data acquisition and timing control were performed using PCI Express digital oscilloscope cards (AlazarTech, ATS 9350), which allowed for continuous streaming of individual detection events (laser firing) on up to 4 channels simultaneously, although only 3 channels were used to perform the experiments described herein. More details about synchronous digitization with beam scanning instrumentation can be found elsewhere.¹² The digitizer cards AUX I/O port was configured to supply an output of the clock frequency divided by 8 (= 10 MHz) in order to clock the LTG through a 10 MHz PLL. As a result the digitization electronics and the mirror drivers all run synchronously with the laser as the master clock. In this manner, the position within the focal plane, of each pulse of the laser, could be determined with reasonably high confidence. The LTG epoch pulse was used to trigger the start of data acquisition with the digitizing oscilloscope cards.

The peak voltages from each detector following each individual laser pulse were digitized and the image(s) were reconstructed. Two separate analyses methods were performed on each single time-dependent data set to generate images. In the first method, each entire trajectory was binned into a single final image to yield high resolution images. Simultaneously, the second method produced images which were separated into sub-frames of the trajectory resulting in higher frame rates of more sparsely sampled images. In either case, the sine-wave trajectory of the resonant mirrors results in non-uniform pixel density across the sample, with higher density near the edges and corners of the image where the resonant mirrors were moving the slowest (turning points). Consequently, some pixels were sampled many times, some fewer times, and in the case of fast sub-frames, some pixels not sampled at all.

2.3 Interpolation algorithm

The interpolation algorithm used in this paper has been described in detail elsewhere⁷ and will only be discussed briefly here. The in-painting algorithm used is an iterative model-based image reconstruction algorithm combining a maximum a posteriori (MAP) estimation with a generalized Gaussian Markov random field (GGMRF) prior model, adopted within a 3 dimensional space-time neighborhood. The GGMRF prior model provides an estimate for the overall probability of obtaining an original image. Next, the algorithm iterates to adjust the estimate for the image, with the degree of change controlled by the regularization of 2D in space and 1D in time, which can be independently adjusted.

3. RESULTS AND DISCUSSION

3.1 High frame-rate imaging

Laser transmittance images of a United States Air Force (USAF) 1951 Resolution chart (first column of Figure 1) were acquired along with laser transmittance images of urea crystals (second column in Figure 1) using the Lissajous trajectory microscope at 25 Hz. Frame-rates in the kHz regime were acquired by rebinning the high resolution Lissajous trajectory data (top row of Figure 1), into sub frames with highly undersampled regions as seen in the middle row of Figure 1. The in-painting algorithm was used to interpolate the missing pixels in the images and the interpolated image is given in the bottom row of Figure 1. The frame rates are determined by how the data are rebinned and can be tailored to the goals of the experiment. In Figure 1 the frame-rate for the USAF Resolution Chart images was 1.25 kHz and 1.46 kHz for the urea crystals. While there is a ~275 fold increase in temporal resolution, it is important to note that this comes at a small cost in spatial resolution as seen most readily in the urea images (second column in Figure 1).

Figure 1.

Laser transmittance images of a USAF 1951 resolution chart and urea crystals. The top row contains the high resolution full Lissajous trajectory images acquired at 25 Hz. The middle row contains the subtrajectories divided out into 1.25 kHz frame rate for the USAF resolution chart and a 1.46 kHz frame rate for the urea. The last row contains the in-painted results for the subtrajectories. Figure adapted from Sullivan et al.⁷

3.2 Multimodal imaging

The multimodal Lissajous trajectory microscope was used to image a 30 μm section of a mouse tail on a glass slide. The resulting combined laser transmittance, SHG, and TPEF image is shown in Figure 2. With the ability to detect laser transmittance (red), SHG (green), and TPEF (blue) simultaneously with perfect overlay chemical and structural information can be obtained for the rat tail sample. The laser transmittance image provides structural detail similar to the information that would be obtained from a conventional bright-field microscope. The collagen in the tail gives bright SHG signal (green) whereas the TPEF signal is located in a different region of the tissue sample due to the presence of different tissue components which do not generate SHG signal and would otherwise be not observed if only SHG were used. By combining SHG and TPEF into one single image, along with the laser transmittance image, different components of the tissue can be easily observed.

Figure 2.

Single frame from a movie at 25 Hz, consisting of a composite image containing laser transmittance (red), SHG (green), and TPEF (blue) of a rat tail tissue sample. The need for multimodal imaging in a tissue is evident here as the SHG and TPEF provide contrast for different regions in the field of view, indicating

Arguably, the greatest advantage to using a Lissajous trajectory based microscope for multimodal imaging of tissue samples is the ability to tailor the frame rate to capture dynamic processes which may be occurring by simply rebinning the data to subframes with higher frame rates. Because Figure 2 was acquired using a Lissajous beam-scanning trajectory, the data can be divided into higher frame rate subframes. In this instance, however, ex vivo imaging of a rat tail is not a rapidly changing system in need of kHz frame-rates and video-rate imaging is sufficient.

In this configuration the 800 nm excitation wavelength did not give as strong of fluorescence signal as a shorter wavelength would have, which contributed to the low signal to background for this image. In the future the instrument will be optimized for a wavelength that yields maximum TPEF and SHG signal.

4. CONCLUSION

In this paper a beam-scanning microscope based on a Lissajous trajectory was presented for multimodal imaging of tissue samples. As previously reported, a beam-scanning approach based on a Lissajous trajectory can significantly reduce motion blur, a common problem in conventional in vivo microscopy. As a first step towards multimodal in vivo imaging, a Lissajous trajectory microscope was demonstrated here as a stable platform for multimodal ex vivo imaging of tissue samples. Due to the multichannel detection capabilities this microscope was able to simultaneously collect SHG, TPEF, and laser transmittance images with prefect spatial registry between the imaging modalities. Further extension to other imaging modes (i.e. CARS, SRS, birefringence, polarization analysis, etc.) is trivial as the only modifications needed for the instrumentation will be different filters and detectors.