Transfer function analysis in epi-illumination Fourier ptychography

Abstract

This letter explores Fourier ptychography (FP) using epi-illumination. The approach effectively modifies the FP transfer function to be coherent-like out to the incoherent limit of twice the numerical aperture over the wavelength 2NA/λ. Images reconstructed using this approach are shown to have higher contrast at finer details compared with images using incoherent illumination, indicating that the FP transfer function is superior in high spatial frequency regions.

Fourier ptychography (FP) is a recently developed coherent imaging technique that iteratively stiches together a number of variably-illuminated low-resolution (LR) images in Fourier space to recover a large field of view (FOV), high-resolution (HR) image of a complex object [1]. Similar to the concept of synthetic aperture [2–6], the high spatial frequency content is shifted into the passband of the imaging system in FP using oblique plane wave illumination. FP can also be considered a phase retrieval technique, which recovers phase information lost in the acquired irradiance measurements [7–10]. The FP technique has been demonstrated for many applications such as quantitative phase imaging [11], refocusing [12], recovering unknown pupil aberrations [13], and 3D intensity and phase imaging [14,15]. By utilizing a multiplexed illumination strategy [16, 17] or optimizing the sampling pattern of the light emitting diode (LED) source [18], the acquisition speed and data requirements can be significantly improved without sacrificing image quality. However, FP has only been demonstrated in a transmission mode, which significantly limits its applications. This imaging technique has numerous applications in reflective imaging systems with applications in surface metrology, industrial inspection, endoscopy, and reflective microscopy.

The FP recovery procedure [1,17], as shown in Fig. 1, iteratively stitches N raw intensity images, each illuminated by a unique plane wave angle, in Fourier space to achieve a higher numerical aperture (NA) equal to the sum of the NA_obj of the objective and the NA_ill of the illumination at the largest angle. It alternates between spatial and frequency domains using the Fourier transform F and its inverse F⁻¹, while iteratively imposing the LR intensity measurements in the associated HR frequency regions. The whole iteration scheme is repeated for M cycles until the reconstructed complex HR solution $\sqrt{I_h}{e}^{j{\varphi }_h}$ converges.

Fig. 1.

Iterative recovery procedure for Fourier ptychography algorithm for N intensity images and M iteration cycles.

The experimental set-up used to demonstrate Fourier ptychography in epi-illumination is shown in Fig. 2(a). The imaging system consists of an infinity-corrected, object-space telecentric microscope objective and an imaging lens. For an object-space telecentric objective, the entrance pupil is positioned at the back focal plane of the objective. To adapt FP for reflection, a beam-splitter is introduced in the imaging path and a relay lens is added in the illumination path to relay the LED to the back focal plane of the objective. After transmitting through the objective, the light is collimated, which approximates the oblique plane waves needed for FP.

Fig. 2.

(a) Experimental layout for epi-illumination FP. (b) The angle of the plane wave from the objective is determined by the position of the LED’s image in the entrance pupil of the objective.

As shown in Fig. 2(b), the position of the LED image at the entrance pupil y_pupil determines illumination angle θ of the plane wave incident on the sample, where θ is measured from the optical axis. The NA of the illuminating plane wave in air is given by

$${NA}_{\mathit{ill}}\left(\theta \right)=sin\theta =-y_{\mathit{puplic}}/\sqrt{y_{\mathit{puplic}}^2+f^2},$$ (1)

where f is the focal length of the microscope objective. The maximum angle for the plane wave is limited by the diameter of the entrance pupil, which determines the NA of the objective. Thus, the maximum shift in frequency space using epi-illumination is NA/λ. To give a greater control over the scanning path, a single LED is translated using a translation stage to simulate an LED array.

It is instructive to compare the differences in the transfer functions for coherent illumination, incoherent illumination, and Fourier ptychography. Consider an object described by complex reflection function r(x,y), where x and y denote lateral coordinates on the sample plane. With uniform, coherent plane wave illumination, the irradiance of a reflective object is given by

$$I_{\mathit{coh}}=C_I{\left|u_{\mathit{coh}}\right|}^2=C_I{\left|\left(r(x,y)e^{j2\pi \left(\frac{sin{\theta }_x}{\lambda }x+\frac{sin\theta y}{\lambda }y\right)}\right)\ast h\left(x,y\right)\right|}^2,$$ (2)

where * is the convolution operator, θ_x and θ_y are the angles of the incident plane waves with respect to the optical axis, C_I is a constant, and h is the coherent point spread function. In Fourier space, the coherent field spatial distribution is

$$U_{\mathit{coh}}(\xi ,\eta )={\mathbf{F}}_{\xi ,\eta }\left[u_{\mathit{coh}}(x,y)\right]={\mathbf{F}}_{\xi ,\eta }\left[r(x,y)\right]H\left(\xi -\frac{sin{\theta }_x}{\lambda },\eta -\frac{sin{\theta }_y}{\lambda }\right),$$ (3)

where H is the coherent transfer function (CTF) of the optical system, and ξ and η are the spatial frequencies in the x and y-directions. For a diffraction-limited system with a circular entrance pupil, H is a circular binary filter with H=1 when the radius is less than or equal to NA/λ. Thus, it transmits all spatial frequencies within a radius of NA/λ. For oblique plane wave illumination, the CTF is shifted in Fourier space by sinθ/λ. The cutoff frequency and shape of the CTF centered at the origin for normal illumination is shown in Fig. 3(a) with a red dashed-dotted circle. Figure 3(b) shows a cross-section of the CTF. Note it transmits all frequencies with no attenuation within a radius of NA/λ.

Fig. 3.

(a) The outline of the OTF (dashed line) and CTF (dashed and dotted line) are circular with cutoff frequencies of 2NA/λ and NA/λ, respectively. In FP, H is shifted in Fourier space as shown in the gray shaded regions with boundaries in black solid lines. The FPTF is unity in the shaded region enclosed by the black circles. (b) Cross-sections of the OTF, CTF, and FPTF.

Fourier ptychography takes advantage of the fact that the CTF can be shifted in Fourier space by illuminating a sample with oblique plane waves. The shift of the CTF in Fourier space allows frequencies previously extinguished by the optics to be shifted inside the CTF and captured by the optical system. The FP algorithm synthesizes all images captured at several, unique plane wave illuminations to reconstruct the Fourier spectrum of the object over an extended area in Fourier space. Thus, the field in Fourier space for Fourier ptychography is

$$U_{FP}(\xi ,\eta )={\mathbf{F}}_{\xi ,\eta }\left[r(x,y)\right]\text{FPTF}(\xi ,\eta ),$$ (4)

where FPTF is the Fourier ptychography transfer function. The FPTF is an effective extension of the CTF over a larger area in Fourier space determined by the angles of the plane wave used for illumination. It is a binary function with FPTF = 1 in all areas that H is shifted in Fourier space, which is described mathematically by

$$\text{FPTF}(\xi ,\eta )=\{\begin{array}{ll}1\hfill & \text{if}\sum _{i=1}^{N_{\mathit{shift}}}H\left(\xi -\frac{sin{\theta }_{xi}}{\lambda },\eta -\frac{sin{\theta }_{yi}}{\lambda }\right)>0\hfill \\ 0\hfill & \text{if}\sum _{i=1}^{N_{\mathit{shift}}}H\left(\xi -\frac{sin{\theta }_{xi}}{\lambda },\eta -\frac{sin{\theta }_{yi}}{\lambda }\right)<=0\hfill \end{array}$$ (5)

where the i^th shift in FP corresponds to a circular region centered at ξ_i=(sin θ_xi/λ, sin θ_yi/λ) in Fourier space. The FPTF is a summation over all angles used for plane wave illumination for the total number of shifts used, N_shift. Since the FPTF is a binary function, all values above zero in the summation are returned to unity. As shown in the experimental layout, for epi-illumination the maximum shift in Fourier space is NA/λ. Figure 3(a) shows the positions of H if it is shifted by NA/λ at eight equally spaced angles around H centered at the origin. The FPTF is unity in the shaded region enclosed by the black circles in Fig. 3(a) and zero everywhere else. A cross-section of the FPTF is shown in Fig. 3(b), which demonstrates that the FPTF can be extended to a maximum of twice the radius of the CTF using FP in epi-illumination, while transmitting all those frequencies without any attenuation.

In comparison, irradiance of the reflective objective in an incoherent system is

$$I_{\mathit{incoh}}\left(x,y\right)=C_I{\left|r\left(x,y\right)\right|}^2\ast {\left|h(x,y)\right|}^2,$$ (6)

where |h(x,y)|² is the incoherent point spread function (PSF) of the optical system. The image formed is a convolution of the reflective object’s irradiance with the PSF of the optical system. Since the image is determined by the modulus squared of the reflected field, oblique plane wave illumination will not shift the transfer function in Fourier space. In Fourier space, the irradiance spatial distribution is

$${\mathbf{F}}_{\xi ,\eta }\left[I_{\mathit{incoh}}\left(x,y\right)\right]=C_I{\mathbf{F}}_{\xi ,\eta }\left[{\left|r\left(x,y\right)\right|}^2\right]\mathit{OTF}(\xi ,\eta ),$$ (7)

where the optical transfer function (OTF) is the normalized Fourier transform of the PSF. The OTF can also be written as the normalized autocorrelation [19] of the coherent transfer function H

$$\mathit{OTF}(\xi ,\eta )=\frac{\iint H\left({\xi }^{\prime },{\eta }^{\prime }\right)H^{\ast }({\xi }^{\prime }-\xi ,{\eta }^{\prime }-\eta )d{\xi }^{\prime }d{\eta }^{\prime }}{\iint {\left|H(\xi ,\eta )\right|}^{2}d\xi d\eta }.$$ (8)

The OTF for a diffraction-limited system with a circular pupil has a cutoff frequency 2NA/λ, which is twice the cutoff frequency of the CTF, as shown in Fig. 3(a). Figure 3(b) shows the shape of the OTF is monotonically decreasing and higher spatial frequencies are severely attenuated.

While the OTF and the FPTF both have the same cutoff frequency, the shapes of their transfer functions differ drastically. The FPTF transmits all spatial frequencies up to the cutoff frequency without any attenuation, whereas the OTF attenuates higher spatial frequencies. While there is no resolution improvement in terms of cutoff frequencies using FP in reflection over incoherent imaging, the shape of the transfer functions results in sharper and higher contrast images using FP.

Experimental Results

All samples are imaged with an object-space telecentric, infinity-corrected 0.1 NA objective and a scientific CMOS camera (PCO.edge, 6.5 μm pixel pitch). For comparison, all samples are imaged using coherent illumination, FP, and incoherent illumination. For coherent illumination and FP, a 660 nm fiber-coupled LED with a 25 nm bandwidth is used as the light source. The fiber core diameter is 105 μm. The illumination is spatially coherent at the sample plane, due to the small size of the fiber core diameter, with an estimated coherence width of 151 μm. For this discussion, the image with illumination at normal incidence is used for the coherent illumination case. For the FP case, nine measurements are captured in total; one at normal incidence where the LED is placed at the entrance pupil’s center, and eight more where the LED is equally positioned around the edge of the entrance pupil, which creates a pattern in the Fourier plane similar to Fig. 3(a) during the FP reconstruction. The FP algorithm [17] is used to synthesize the nine raw images. Based on the coherence width, the FP image is reconstructed in patches of 132 pixels to maintain sufficient spatial coherence in the reconstructed patch. From the nine images, the synthesized NA is 0.19. For the incoherent illumination case, a halogen lamp that is spectrally filtered using a 650 nm bandpass filter with a 20 nm bandwidth replaces the LED. The filament of the halogen lamp is imaged at the entrance pupil of the objective.

First, a 1951 USAF target is imaged using coherent illumination, FP, and incoherent illumination shown in Figs. 4(a)–4(c), respectively. Figures 4(d)–4(f) are magnified views of group 7 of the USAF target, which is the smallest group of elements that are resolvable. Figures 4(e) and 4(f) show the obvious resolution enhancement using FP or incoherent illumination over coherent illumination in Fig. 4(d). Note that the image using FP is visually sharper and the contrast is higher than the incoherent image. Also, note the rippling in the square in Fig. 4(e) displays typical artifacts seen in coherent images, which further demonstrates that the FPTF displays coherent characteristics.

Fig. 4.

Image using (a) coherent illumination, (b) FP, and (c) incoherent illumination of 1951 USAF target. (d)–(f) Magnified view of group 7.

Cross-sections averaged through the horizontal elements in group 7 of the USAF target are shown in Figs. 5(a)–5(c). Using coherent illumination, the minimum resolvable feature is element 1 (128 cycles/mm). Using incoherent illumination, element 4 (181 cycles/mm) is just resolvable. Using FP, element 7 (228 cycles/mm) is resolved. More importantly, the contrast is greatly enhanced using FP. The contrast V is calculated for all elements from groups 4 7 for the three illumination scenarios using

Fig. 5.

Cross section of horizontal bars in group 7 of 1951 USAF target for (a) coherent illumination, (b) FP, and (c) incoherent illumination. (d) Contrast as a function of spatial frequency over the bar elements of groups 4–7.

$$V=(I_{max}-I_{min})/(I_{max}+I_{min}),$$ (9)

where I_max and I_min are the maximum and minimum pixel values of each element. A plot of the contrast as a function of spatial frequency of groups 4–7 is shown in Fig. 5(d). Since this is a contrast measurement of rectangular bar targets instead of sinusoidal signals, these plots deviate from the shape of the classic cosinusoidal transfer functions. Both coherent and FP maintain a high contrast until an associated spatial frequency, then the contrast drops sharply. In comparison, contrast of the incoherent image monotonically decreases as the spatial frequency increases. One clear advantage of FP over incoherent illumination in epi-illumination is maintaining high contrast at high spatial frequencies. Note this advantage comes at the cost of more acquired images and time required for the computational reconstruction.

During the reconstruction process, the full FOV of a raw image is broken up into smaller image segments (132 × 132 pixels), where the size of the patch is determined by the coherence width of the illumination at the sample plane. Each of these patches is independently processed by the FP algorithm to produce a high-resolution image of that patch. Breaking the full FOV into patches is beneficial since each patch can be processed in parallel and the illumination is sufficiently coherent over the small patch. Since the FP algorithm [17] solves a minimization problem for each patch, each patch may be reconstructed to a different background level. For this reason, when the reconstructed patches are stitched together in the final image, the reconstructed patches may not blend together as shown in Fig. 4(b). Alpha blending and background normalization can be used to better blend separate patches together.

Another sample used in the assessment is tissue from a chicken thigh. It is imaged using coherent illumination, FP, and incoherent illumination, and the images are shown in Figs. 6(a)–6(c), respectively. A slice of tissue was cut from the chicken thigh and was sandwiched between two glass microscope slides to create a flat surface at the sample plane. Note that only a thin slice of the surface is imaged, so it can be approximated as a thin reflective sample, which is a necessary condition for FP [1]. As shown in Fig. 6(b) and 6(c), FP and incoherent illumination are able to resolve much finer features than the image using coherent illumination in Fig. 6(a). We believe the small circular features highlighted in Figs. 6(a1),6(b1), and 6(c1) are muscle fibers. As shown in the coherent case, shown in Figs. 6(a1)–6(a4), the muscle fibers are not resolved and appear as a line. The FP case, shown in Figs. 6(b1)–6(b4), is able to resolve individual muscle fibers at high contrast, while the incoherent case, shown in Figs. 6(c1)–6(c4), is able to resolve most of the muscle fibers, but at a significantly reduced contrast. The muscle fibers are more easily identified in the image using FP. Once again, the image using FP is sharper and has a higher contrast than the image using incoherent illumination.

Fig. 6.

(a) Image of tissue from chicken thigh using coherent illumination. (a1)–(a4) Magnified views of regions of (a). (b) Image using FP. (b1)–(b4) Magnified views of regions of (b). (c) Image using incoherent illumination. (c1)–(c4) Magnified views of regions of (c).

The phase estimate of the tissue from the chicken thigh calculated by the FP algorithm is shown in Fig. 7. The phase estimate represents an additional degree of information retrieved by the FP algorithm. The additional phase estimate information can be used for phase contrast measurements or digital holography.

Fig. 7.

(a) Retrieved phase for tissue from chicken thigh. (b)–(d) Magnified views of regions of (a).

This letter demonstrates Fourier ptychography in reflection. In epi-illumination, the Fourier ptychography transfer function (FPTF) is effectively an extension of the CTF to the incoherent cutoff frequency. Since the FPTF transmits all spatial frequencies below its cutoff frequency without attenuation, contrast for images reconstructed using FP is significantly greater than for incoherent images. Experimentally, images reconstructed using FP of a 1951 USAF target and tissue from a chicken thigh are visually sharper and provide higher contrast than images using incoherent illumination. Note this does come at the cost of more acquired images and time required for reconstruction.

Replacing an incoherent light source with an LED array in an epi-illumination microscope allows the user to take advantage of the multiple benefits of Fourier ptychography. In addition to higher contrast images, FP allows for digital refocusing, wavefront correction, pupil recovery, and phase retrieval.