Simple differential digital confocal aperture to improve axial response of line-scanning confocal microendoscopes

Abstract

Line-scanning confocal microendoscopy offers video-rate cellular imaging of scattering tissue with relatively simple hardware, but its axial response is inferior to that of point-scanning systems. Based on Fourier optics theory, we designed differential confocal apertures with a simple subtraction technique to improve the line-scanning sectioning performance, especially in the far defocus range. Taking advantage of digital slit apertures on a digital light projector and a CMOS rolling shutter, we demonstrate real-time optical sectioning performance comparable to point-scanning in a dual-camera microendoscope (<$6,000). We validate the background rejection capability when imaging porcine columnar epithelium stained with fluorescent contrast agents with different uptake mechanisms and staining properties.

Probe-based fluorescence microendoscopy has demonstrated great potential to improve early cancer detection due to its subcellular resolution and minimal invasiveness. When imaging thick, highly scattering biological tissue in an epi-illumination configuration, the high level of scattering usually necessitates suppressing unwanted out-of-focus signal to obtain high-resolution images with good contrast. To achieve this goal, different strategies have been employed. On the sample end of the probe, contrast agents that predominantly accumulate in specific cellular components or target specific biomarkers have been used, which result in lower levels of background fluorescence than when using nonspecific dyes such as fluorescein. Proflavine, for example, is a topical contrast agent that preferentially stains the nuclei and has been clinically validated in a wide range of applications. [1–4] On the proximal end of the probe, optical sectioning can be employed to selectively collect fluorescence from the in-focus plane. Sophisticated imaging systems with confocal scanning or structured illumination have been developed to reject scattered fluorescence. [5,6]

Line-scanning confocal imaging is an appealing technique for clinical translation since it allows for video-speed frame rate at relatively low complexity and cost. Compared with point scanning, however, the axial sectioning performance is inferior. Specifically, line scanning has a poorer full width at half maximum (FWHM) than point scanning with comparable aperture dimensions. Moreover, it suffers from slow attenuation of out-of-focus signal over a long axial range beyond a few multiples of the FWHM. [7] Subtraction techniques have been reported to improve the line-scanning axial response, but these methods either require multiple frames or suffer from undesired artifacts. [6,8] Previously, we demonstrated a line-scanning confocal high-resolution microendoscope (LSC HRME) using digital apertures on a digital light projector (DLP) and a CMOS rolling shutter. [9] In this letter, we leverage the versatility of digitally programmed apertures to further improve the line-scanning axial performance with a simple real-time subtraction technique. Using simulated and experimental data, we demonstrate differential aperture confocal (DAC) imaging in a low-cost dual-camera microendoscope that has improved axial response in both near and far defocus ranges (0 – 100 μm and >100 μm defocus in this study, respectively). We further validate the system performance by imaging highly scattering porcine columnar epithelium with proflavine staining to highlight epithelial cell nuclei. We also explore the ability of the system to image non-specific topical fluorescein staining, which has recently been shown to be useful for columnar crypt characterization but shows higher level of scattered background. [10]

The differential aperture confocal HRME (DAC HRME) is a dual-camera line-scanning fluorescence microscope, shown in Figure 1A. Blue light from a LightCrafter 4500 DLP (Texas Instruments; total power is 1.3 mW at the sample end in each confocal scanning cycle) is collimated by a lens (f = 125 mm) that replaces the built-in DLP lens to provide fluorescence excitation. Two 8-bit CMOS sensors (Mako U503, Allied Vision) were used for fluorescence descanning in rolling shutter mode. The microscope is coupled with a coherent fiber bundle (FIGH-30–850N, Myriad Fiber Imaging; 790 μm circular field of view [FOV]) as an image relay, and collected fluorescence is divided by a beam splitter. As described in further detail below, we program detection apertures of different widths on the transmission and reflection arms (CMOS 1 and CMOS 2, respectively) to enable DAC imaging.

Figure 1.

(A) DAC HRME setup, (B) the confocal aperture alignment and (C) simulations of different imaging configurations based on Fourier optics. In Figure C, the confocal apertures together with the resulting frequency spectra and axial responses are plotted for CMOS1, CMOS 2 and differential aperture confocal imaging; simulated values are normalized by the maxima in the CMOS 1 configuration, unless otherwise noted. EM: emission filter (500 nm long pass); EX: excitation filter (452/45 nm bandpass); DM: dichroic mirror (490 nm long pass); BS: beam splitter; Obj: objective.

The spatiotemporal aperture alignment for confocal imaging is illustrated in Figure 1B. Detailed scanning characteristics of the DLP and CMOS rolling shutter have been described elsewhere. [9] Briefly, the DLP binary illumination sequence (blue rectangles) is projected discretely due to the need to toggle micromirrors, and the CMOS rolling shutter (green parallelogram) scans in a synchronized manner, but at a much faster line frequency. We design spatially overlapping illumination apertures to ensure that the total effective exposure time (temporal overlap between blue rectangles and green parallelogram) on each CMOS row remains constant along the scanning direction; we also implement parallel illumination to improve the axial response by reducing the aperture width. In the dual-camera setup, CMOS 1 is used to capture the LSC images, using illumination and standard detection aperture widths of d_I and d_S. On CMOS 2, we capture a second frame within the same DLP scan but use an expanded detection aperture d_E for DAC subtraction in real time.

The axial response of confocal scanning can be modeled in the context of Fourier optics theory. [11] [12] In the frequency domain, the axial profile of a confocal system is

$$I(\mathrm{\mu })=N_0\iint F_I(m,n)F_D(m,n)C^2(m,n;\mathrm{\mu })dmdn,$$ (1)

in which N₀ is a normalizing constant, m and n are normalized spatial frequencies in the x and y directions, and μ is the defocus in normalized axial optical unit; F_I(m, n) and F_D(m, n) represent frequency spectra, or Fourier transforms of confocal illumination and detection apertures, respectively; C(m, n; μ) describes the optical transfer function. [13] Equation 1 shows that the confocal axial response is an integral of optical sectioning strength of all spatial frequencies through the imaging system, and their relative contribution is modulated by the frequency spectra of confocal apertures. It also reveals that optical sectioning at defocus μ is only exhibited in the optical transfer function (m, n; μ), which can be calculated with Stokseth’s approximation:

$$g(\mathrm{\mu },\delta )=\{\Gamma (\delta )\cdot \begin{array}{cc}\left\{2\frac{J_1\left[\mathrm{\mu }\delta \left(1-\frac{\delta }{2}\right)\right]}{\mathrm{\mu }\delta \left(1-\frac{\delta }{2}\right)}\right\},& \text{if}0<\delta <2\\ 0,& \text{if}2\le \delta \end{array},$$

$$\Gamma (\delta )=1-0.69\delta +0.0076{\delta }^2+0.0437{\delta }^3,$$ (2)

where $\delta =\sqrt{m^2+n^2}$, and J₁ denotes the first-order Bessel function of the first kind. [14] We note g(μ, 0) = 1 for all defocus, suggesting no optical sectioning for the zero frequency.

We first consider LSC imaging on CMOS 1 in Figure 1C. The confocal apertures along the x axis are

$$A_I=rect\left(\frac{x}{d_l}\right),A_D^S=rect\left(\frac{x}{d_S}\right),$$ (3)

with A_I and $A_D^S$ denoting the illumination aperture and the detection aperture of a standard width (Figure 1C, top). The corresponding frequency spectrum $F_I(m,n)*F_D(m,n)$ is a bandpass filter centered at zero (Figure 1C, middle). The LSC axial response using a standard detection aperture can be calculated as

$$I_S(\mathrm{\mu })=Np{q}_S\cdot \int sinc\left(\frac{pm}{2\pi }\right)sinc\left(\frac{q_{S}m}{2\pi }\right)g^2(\mathrm{\mu },m)dm,$$ (4)

in which N is a normalizing constant, p and q_S are aperture widths d_I and d_s measured in normalized lateral optical units. [8] As revealed by Equation 2, the zero to low frequencies at the center of the LSC spectrum are the slowly attenuating components along the axial axis. This contributes to the residual background far from the focus shown in the predicted axial response of LSC imaging on CMOS 1.

Importantly, Equations 1, 2 and 4 also indicate that the frequency spectrum can be employed to tune the confocal sectioning performance. In order to further suppress the zero to low frequencies in the LSC configuration, we program an expanded detection aperture of a physical width d_E (q_E in normalized optical unit) on CMOS 2 and engineer a virtual detection aperture with a simple subtraction:

$$A_D^{DAC}=A_D^S-\frac{ds}{d_E}{A}_D^E=rect\left(\frac{x}{d_S}\right)-\frac{ds}{d_E}rect\left(\frac{x}{d_E}\right).$$ (5)

In other words, a scaled image from CMOS 2 with an expanded detection aperture $A_D^E$ is used to selectively collect the poorly attenuating frequency components:

$$I_E(\mathrm{\mu })=Np{q}_E\cdot \int sinc\left(\frac{pm}{2\pi }\right)sinc\left(\frac{q_{E}m}{2\pi }\right)g^2(\mathrm{\mu },m)dm,$$ (6)

which can be then subtracted from the LSC image on CMOS 1. The resulting DAC frequency spectrum in Figure 1C (middle) reveals a bimodal distribution that excludes the zero frequency and suppresses low frequencies (in this example, q_E= 3q_S). The corresponding axial response for DAC imaging is:

$$\begin{array}{l}{I}_{DAC}(\mathrm{\mu })=I_S(\mathrm{\mu })-\frac{q_S}{q_E}{I}_E(\mathrm{\mu })\\ =Np{q}_s\cdot \int sinc\left(\frac{pm}{2\pi }\right)\left[sinc\left(\frac{q_{S}m}{2\pi }\right)-sinc\left(\frac{q_{E}m}{2\pi }\right)\right]g^2(\mathrm{\mu },m)dm.\end{array}$$ (7)

As expected, the simulated DAC axial response in Figure 1C (bottom) shows further attenuation of out-of-focus signal, especially in the far defocus range. We also note that the high frequencies are preserved during the subtraction.

Like other subtraction-based techniques such as structured illumination demodulation, there exists a tradeoff between background subtraction and signal loss when determining the q_E/q_S ratio. Theoretically, a wider q_E can reject the zero frequency more exclusively but is less effective in suppression of low frequencies. In addition, since both images are acquired under the same illumination conditions and the aperture width is directly proportional to the exposure time, it is not practical to use high q_E/q_S ratios beyond the camera’s dynamic range. Based on these considerations and our experimental data, we used a q_E/q_S ratio of 3; a 70:30 beam splitter was selected accordingly to effectively leverage the dynamic range of both cameras.

We experimentally characterized the axial response by imaging a mirror in reflection mode without the emission filter or the fiber bundle and compared measured values to theoretical simulations in Figure 2A. As anticipated, background signals far from the focus were observed on both cameras and they converged from about 150 μm, confirming that similar underlying zero and low frequencies were captured. After the subtraction, the FWHM was improved and background far from the focus was eliminated. We calculated the optical sectioning performance using Equations 2, 4, 6, 7 and parameters from our experimental setup (d_S and d_E on two cameras were 22.6 μm and 67.7 μm on the object plane, which corresponds to 2032 and 6070 μs exposure time on two CMOS sensors). The FWHM from the theoretical simulations was 114.2 μm in the LSC configuration and reduced to 75.1 μm in the DAC configuration. In addition, we modeled the point-scanning axial response using pinhole diameters that were the same as the LSC slit widths, showing a simulated FWHM of 61.5 μm with rapid attenuation with defocus. These results suggest that the DAC axial response was superior to the LSC and was comparable to a point-scanning setup, especially in the far defocus range.

Figure 2.

(A) Axial responses of different imaging configurations (theory and experiment), and (B) fluorescent beads imaging using the HRME.

The experimental and theoretical findings above were further validated in fluorescence mode by imaging a single-layer of 15 μm beads (F-21010, Thermo Fisher Scientific) with the fiber bundle. In Figure 2B, images in focus, at 40 μm and 80 μm defocus are shown. To facilitate visualization, images acquired with each configuration were equally adjusted. Specifically, the in focus widefield image was normalized to itself and widefield images at other defocus values were equally boosted to ensure the relative intensities of all widefield images remained the same after adjustment; the same procedure was performed for LSC and DAC HRME images. The non-scanning widefield HRME images revealed defocus blur but minimal signal loss due to the lack of optical sectioning. At 40 μm, the DAC HRME images demonstrated better signal rejection around the FWHM. At 80 μm, residual signal observed in the LSC HRME image was significantly attenuated with the DAC HRME. These results were consistent with the axial profiles in Figure 2A.

The enhanced out-of-focus background rejection when imaging scattering samples with the DAC HRME, as suggested in Figure 2A, is further evaluated by imaging excised porcine stomach tissue. The gastric mucosa is characterized by a superficial lining of columnar cells, invaginations called gastric pits, and gastric glands that form at the pit base and grow towards the surface. Given the high cellular density in columnar epithelium, optical sectioning is desired to examine this unique architecture, especially when non-specific fluorescein staining is used. During ex vivo imaging, fresh porcine stomach tissue was acquired from a local abattoir and imaged within two hours after resection. The mucosa was topically stained with either proflavine (0.01% w/v in PBS) or fluorescein (0.05% w/v in PBS) and rinsed with PBS prior to imaging.

Proflavine-stained gastric mucosa images are shown in Figure 3, highlighting nuclear staining on the surface epithelial lining and glands within the pits. The LSC image better resolved the surface mucosa and the transitional lining into the pits than the widefield image. In the DAC image, the contrast of individual nuclei (white arrows) on the superficial surface was further enhanced, mostly due to its reduced axial FWHM. Importantly, while residual background within the gastric pits was still present in the LSC image, it was significantly reduced in the DAC image. This allowed for clearer visualization of the glandular structures and could be attributed to improved rejection of scattered signal within the pits, especially from depths greater than 100 μm. These findings were confirmed in two line (braces in Figure 3A) profiles in Figures 3D and E, showing improved contrast of individual nuclei on the surface lining (black arrows) and glandular structures within the pits (asterisks within brackets).

Figure 3.

Ex vivo imaging of gastric columnar epithelium stained with proflavine. Two lines indicated by braces in (A) are profiled in (D) and (E), highlighting enhanced contrast of nuclei (arrows) on the superficial lining and glandular structures (asterisks) within the pits (brackets) in the DAC image.

Images with topical fluorescein staining are shown in Figure 4. Unlike proflavine that preferentially stains the nuclei, non-specific fluorescein filled the pits and showed a complementary pattern. Not surprisingly, the greater level of background signal from fluorescein-stained tissue made it challenging to resolve individual cells and resulted in poor contrast in widefield images. With confocal scanning, the cobblestone architecture could be readily discerned, with the contrast enhancement most striking in the DAC images. In addition, lower signal levels were seen at the center, which could be due to the elevation of glands within the pit. The differences were quantified in two line profiles (white braces in the widefield images), with the DAC images revealing sharp transition across the pits and the surface lining. Notably, due to its capability to reject long-range background, the DAC images better highlighted the glandular elevation that appeared dark (black arrows in line profiles) within the pits.

Figure 4.

Ex vivo imaging of gastric columnar epithelium stained with fluorescein. Two lines indicated by the braces in the widefield images are quantified, showing enhanced contrast of pits (brackets) and lower signal levels due to elevated glands (arrows).

In summary, we present differential confocal aperture imaging to improve the axial response of confocal line-scanning based on Fourier optics, and we implement a simple subtraction technique using digital apertures on a DLP and two CMOS sensors. Without mechanical constraints of conventional scanning, the differential aperture confocal microendoscope is versatile and compact. Our ex vivo imaging results demonstrate its potential for high-resolution imaging in clinically important yet challenging applications, including imaging highly scattering columnar epithelium with both specific and non-specific contrast agents. In addition, it can be built at a significantly lower cost (<$6,000) than commercial confocal microendoscopes, offering an opportunity for high-resolution confocal imaging in community and low-resource settings. The versatile digital aperture framework provides a convenient way to implement optical sectioning in non-confocal microscopes, and the synthetic aperture approach can also be readily adopted to improve axial performance of existing line-scanning confocal microscopes, including these with physical scanners and apertures.