Virtual k-Space Modulation Optical Microscopy

Abstract

We report a novel superresolution microscopy approach for imaging fluorescence samples. The reported approach, termed virtual k-space modulation optical microscopy (VIKMOM), is able to improve the lateral resolution by a factor of 2, reduce the background level, improve the optical sectioning effect and correct for unknown optical aberrations. In the acquisition process of VIKMOM, we used a scanning confocal microscope setup with a 2D detector array to capture sample information at each scanned x–y position. In the recovery process of VIKMOM, we first modulated the captured data by virtual k-space coding and then employed a ptychography-inspired procedure to recover the sample information and correct for unknown optical aberrations. We demonstrated the performance of the reported approach by imaging fluorescent beads, fixed bovine pulmonary artery endothelial (BPAE) cells, and living human astrocytes (HA). As the VIKMOM approach is fully compatible with conventional confocal microscope setups, it may provide a turn-key solution for imaging biological samples with ~100 nm lateral resolution, in two or three dimensions, with improved optical sectioning capabilities and aberration correcting.

Fluorescence microscopy has become the workhorse tool for modern biological research and clinical diagnosis. The resolution of conventional fluorescence microscopy is determined by the diffraction limit of the employed objective lens. This diffraction limit, however, is established under the assumptions of single image acquisition and linear light-matter interaction. The structure illumination microscopy (SIM) technique is able to achieve a resolution doubling that of wide-field microscopy by sinusoidal pattern illumination, multiple image acquisition, and a numerical processing algorithm. A series of deriving techniques have been developed to further enhance resolution, including saturated SIM (SSIM) [1–3], three-dimensional (3D) SIM [4], Blind-SIM [5,6], and other Bayesian estimation approaches [7]. All the mentioned methods use illumination patterns to modulate the high-frequency information into the low-frequency passband. Imaging scanning microscopy (ISM) approach [8–12] is a good example to this end. In 1987 Bertero [13] and in 1988 Sheppard [14] described this approach for a 2D detector array imaging system to improve resolution of confocal system. Later this technique has been commercialized by Zeiss in the Airyscan system [15]. Similar to SIM, the captured ISM raw data are then processed to recover the superresolution image of the sample. To increase the imaging speed, multiple spots can be used in the ISM setup to realize signal multiplexing (MSIM) [16–19].

Recently, the link between SIM and ISM has been established by a virtual SIM method (vSIM) [20,21], which shares the same confocal setup and data acquisition method as the ISM approach. Particularly, vSIM converts the ISM data into SIM data by performing virtual k-space modulation. The superresolution images can be recovered by the conventional SIM algorithms. Because the sample modulation is implemented in a digital manner, vSIM could freely control the orientations and lateral phases of the modulation patterns. In addition, the use of point-scanning setup in vSIM reduces the background level of the recovered image. In spite of these progresses, up to now, experimental demonstration of vSIM has only been conducted for nonfluorescent samples.

In this Letter, we report a new superresolution imaging approach for fluorescent samples by integrating the virtual k-space modulation method and a ptychography-inspired imaging procedure [22,23]. The reported approach, termed virtual k-space modulation optical microscopy (VIKMOM), is able to improve the lateral resolution by a factor of 2, reduce the background level, improve the sectioning effect, and correct for unknown optical aberrations. We tested the performance of the reported approach by imaging fine structures of 2D bovine pulmonary artery endothelial (BPAE) cells and 3D living human astrocytes (HAs). As the reported approach is fully compatible with the confocal microscope setup, we can potentially extend it for deep tissue imaging via two-photon excitation and for nanoscopy imaging via stimulated emission depletion (STED).

The imaging procedures of the VIKMOM system is shown in Fig. 1, where a confocal microscope with a 2D detector array was used for image acquisition (also refer to the Supplemental Material [24], Fig. 1 for more details). For each scanning position in the x–y plane, the excitation point spot was projected on the sample and the corresponding image was recorded on the detector array. As the sample was scanned to different x–y positions, we obtained a series of intensity images I_descan(r_d;r), where r represents the scanning position in the x–y plane and r_d represents the position in the detector array. The imaging procedures can be divided into two major steps. In step 1, we convert the acquired images I_descan(r_d;r) into a set of SIM data. In step 2, we used these SIM data to recover the super-resolution images of the sample.

FIG. 1.

Schematic of virtual k-space modulation optical microscopy (VIKMOM) and its decoding procedure for superresolution image recovery. (a) The principle of our imaging system. The Airy disklike patterns (the gray rectangle region), recorded by a detector array placed on the image plane, are multiplied by digital masks to virtually modulate the sample information in k space. (b) Our decoding procedure for superresolution image recovery.

For step 1 of VIKMOM, we modulate the acquired images I_descan(r_d;r) with 144 different digital masks as followed [21]

$${I_m}^{(n)}(\mathbf{r})=\sum p^{(n)}(\mathbf{r}+{\mathbf{r}}_d)\times I_{\text{descan}}({\mathbf{r}}_d;\mathbf{r}),(n=1,2,\dots ,144),$$ (1)

where p⁽ⁿ⁾(r_d) is the nth digital mask (see Note 2, in the Supplemental Material [24]) and Σ denotes the sum of the pixel intensity values over the entire detection area. The output of this virtual modulation process in Eq. (1) is a set of SIM data I_m⁽ⁿ⁾(r). In conventional confocal microscopy with a single detector of finite size, the sample signal at the scanning position r is simply the intensity sum of the corresponding recorded intensity image I_descan(r_d;r) over the detector area. If a detector array with distinguishable elements is employed instead of a single detector, the captured images contain sample information beyond the cutoff frequency of the objective lens, and thus, facilitate the reconstruction of a superresolution image of the sample (see Note 1 in the Supplemental Material [24]). Unlike methods based on structured light, the k-space modulation method is implemented using digital masks [21]. Therefore, the parameters of the virtual modulating mask, such as its period, initial phase, and direction can be freely chosen, reducing the complexity of the imaging system.

For step 2 of VIKMOM, we use the set of SIM data I_m⁽ⁿ⁾(r) to recover the superresolution image. Inspired by the principle of Fourier ptychographic microscopy [22,23], we have developed an imaging procedure (better than the termed algorithm) that is insensitive to noise and able to correct for unknown aberrations. The imaging procedure switches between the Fourier domain and the spatial domain in an iterative manner, as shown in Fig. 1(b). We used $I_{\text{obj}}^{(0)}(\mathbf{r})$ and OTF⁽⁰⁾(k) to represent the initial estimate of the sample’s spatial information and the optical transfer function (OTF) in the excitation path, respectively. These two functions were updated through the following process. The modulated object intensity $I_{\text{op}}^{(n-1)}(\mathbf{r})$ with nonupdated object intensity was first expressed as [21]

$$I_{\text{op}}^{(n-1)}(\mathbf{r})=I_{\text{obj}}^{(n-1)}(\mathbf{r})[p^{(n)}(\mathbf{r})\otimes h_{\text{de}}(\mathbf{r})],$$ (2)

where p⁽ⁿ⁾(r) was the digital pattern used for the data update, h_de(r) was the point spread function (PSF) of the detection setup determined based on experimental parameters, and ⊗ denoted the two-dimensional (2D) convolution operation. Then, using the spatial spectrum of the nth modulated image $f_m^{(n)}(\mathbf{k})$ or the Fourier transform of I_m⁽ⁿ⁾(r) we updated the modulated object intensity in the Fourier domain through

$$f_{\text{op}}^{(n)}(\mathbf{k})=f_{\text{op}}^{(n-1)}(\mathbf{k})+[f_m^{(n)}(\mathbf{k})-{\text{OTF}}^{(n-1)}(\mathbf{k})f_{\text{op}}^{(n-1)}(\mathbf{k})]\text{Mask}(\mathbf{k}),$$ (3)

where Mask(k) was a circular low-pass filter used to block the high-frequency noise appearing in the modulated images. Its cutoff frequency was initially set to that estimated for the excitation optics and was slightly adjusted according to the reconstruction quality. For single-photon excitation it was estimated as 4πNA/λ_e, where λ_e was the excitation wavelength and NA was the objective numerical aperture. Finally, the object information was updated in the spatial domain according to

$$I_{\text{obj}}^{(n)}(\mathbf{r})=I_{\text{obj}}^{(n-1)}(\mathbf{r})+\frac{[p^{(n)}(\mathbf{r})\otimes h_{\text{de}}(\mathbf{r})][{I_{\text{op}}}^{(n)}(\mathbf{r})-{I_{\text{op}}}^{(n-1)}(\mathbf{r})]}{max{\{p^{(n)}(\mathbf{r})\otimes h_{\text{de}}(\mathbf{r})\}}^2}.$$ (4)

Concurrently, the OTF in the excitation optics was updated using

$${\text{OTF}}^{(n)}(\mathbf{k})={\text{OTF}}^{(n-1)}(\mathbf{k})+\frac{\mid {f_{\text{op}}}^{(n-1)}(\mathbf{k})\mid [{f_{\text{op}}}^{(n-1)}(\mathbf{k})]\ast [f_m^{(n)}(\mathbf{k})-{\text{OTF}}^{(n-1)}(\mathbf{k}){f_{\text{op}}}^{(n-1)}(\mathbf{k})]}{max\{\mid {f_{\text{op}}}^{(n-1)}(\mathbf{k})\mid \}[{\mid {f_{\text{op}}}^{(n-1)}(\mathbf{k})\mid }^2+\delta ]}\text{Mask}(\mathbf{k}),$$ (5)

where δ was the regularization constant required to prevent the occurrence of zero in the denominator and max{·} represented the maximal value in the 2D matrix. After all the modulated images were used to update the sample, we repeated the entire process until the solution converges.

We conducted several simulations for a theoretical prediction of the resolution enhancement of VIKMOM. It is important to determine the optimal overall size of the detector array and the size of the detector element (see details in Note 3 the Supplemental Material [24]). We first simulated the imaging results of 25 nm fluorescent beads with an emission wavelength centered at 532 nm (see the Supplemental Material [24], Fig. 2), to demonstrate our method’s ability of achieving a resolution 2 times better than that of a conventional wide-field microscopy.

FIG. 2.

Comparison or reconstructed images with white Gaussian noise and aberration. (a)–(d) Imaging results obtained using confocal microscopy with point and infinite detectors, SIM and Vikmom, respectively, for white Gaussian noise with a 10% standard deviation introduced into the detection process for every detector element in the array. (e) Pupil function phase distribution (inset) and corresponding OTF when a field curvature aberration is introduced to the excitation setup, (f) Image recovered using the SIM algorithm. (g)–(h) Recovered image and OTF obtained using the proposed algorithm in VIKMOM.

Then we investigated our approach’s imaging performance robustness to noises by simulating a thin spokelike sample placed at the objective (NA = 1.4) focal plane (Fig. 2). White Gaussian noise with a standard deviation of 10% was introduced into the detection process at every detector element. The simulated imaging results are presented, and the signal-to-noise ratio is quantified by the ratio of the noise-free image average intensity to the square root of the mean square error between the noise-free result and its noise-corrupted counterpart. It is apparent that the use of a point detector in confocal microscopy leads to an imaging result with a low signal-to-noise ratio (SNR) [Fig. 2(a)], while simply adding the signals recorded by other detector elements into the reconstructed image leads to decreased resolution [Fig. 2(b), where the resolution is equal to that of wide-field microscopy]. Experimental results obtained using different total detector sizes in the confocal microscopy further support this statement (see the Supplemental Material [24], Fig. 4). However, the use of a digital mask to virtually modulate the sample information in k space allows images with both improved resolution and SNR to be obtained. Figures 2(c) and 2(d) show the images recovered using the 2D SIM algorithm and our iterative algorithm in VIKMOM, respectively. Simulation results under white Gaussian noise with different standard deviations are presented in the Supplemental Material [24], Fig. 5. We conclude that our approach exhibits superior performance with respect to both lateral resolution and robustness to noises.

FIG. 4.

Experimental results for 2D biological samples (the mitochondria of BPAE cells). (a) Imaging results for confocal microscopy with 0.2 and 1.25 AU-sized detector, Airyscan, and VIKMOM. (b) Magnified views of areas indicated by white boxes in (a). (c) Magnified views of the areas indicated by the white boxes in (b).

Next, we designed a three-layer sample [see the Supplemental Material [24], Fig. 3(b)] to prove our approach’s ability to reject out-of-focus background noises for an improved sectioning effect. We simulated imaging using wide-field microscopy, confocal microscopy with a point detector, confocal microscopy with a 1.25 AU-sized detector, and our approach [the Supplemental Material [24], Fig. 3(c)]. Interestingly, we found the background suppression performance achieved with our approach was compatible with that of confocal microscopy with a point detector, while the lateral resolution in our approach was higher. We conclude that our approach could enhance the lateral resolution computationally, and further that the axial optical-sectioning ability is improved both computationally and physically through the finite-sized detector array.

FIG. 3.

Experimental imaging results for 100-nm fluorescent nanoparticles. (a) Confocal images with 0.2 and 1 AU detectors, and VIKMOM reconstructed image (for more views see Supplemental Material [24], Fig. 7). (b) PSFs fits to the intensity profiles along the yellow lines in (a). (c) Imaging results using confocal microscopy with 0.2 and 1 AU detectors, Airyscan, and VIKMOM, with imaging aberration caused by optical misalignment (for large field of view see Supplemental Material [24], Fig. 8). (d) Magnified views of areas indicated by white boxes in (c). (e) Intensity profiles along the yellow lines in (c). (f) Intensity profiles along the blue lines in (c). (g) Histogram of the appearance frequency histogram of the bead size in (a).

To demonstrate the aberration correction capability, we introduced a field curvature aberration into the excitation pupil function [Fig. 2(e)], and reconstructed the superresolution image by using the SIM algorithm [Fig. 2(f)] and our algorithm [Fig. 2(g)]. It is clear that SIM yielded severely decreased image quality. Our method could recover a better image and the OTF [Fig. 2(h)]. The blue circles indicate that our approach can always achieve a resolution double that of wide-field microscopy. In order to testify our approach’s applicability in more versatile situations, we introduced the spherical aberration, defocus aberration, astigmatism aberration, and combined spherical, defocus, and astigmatism aberrations, respectively, to the excitation path. In fact, the excitation OTF could all be recovered and our approach’s performance is shown to be unaffected by all these aberration types (the Supplemental Material [24], Fig. 6).

To test the practical performance of our approach, we conducted a series of experiments utilizing a confocal system (Zeiss LSM 880 with Airyscan) equipped with a detector array in the image plane [8]. First, we imaged fluorescent nanoparticles (100 nm, yellow-green FluoSpheres, 488 nm/516 nm, Molecular Probes) with a scanning resolution of 40 nm per pixel (Fig. 3, the Supplemental Material [24], Fig. 7). As mentioned above, a trade-off between the achievable resolution and the SNR must be made when choosing the detector size in confocal microscopy [Figs. 3(a), (c), (d)]. Measurement of full width at half maximum (FWHM) of one nanoparticle indicated that our approach achieved a 125 nm lateral resolution [Fig. 3(b)], which is further confirmed by the appearance frequency histogram of the experimental bead image size [Fig. 3(g)].

Our approach’s aberration correction capacity was also demonstrated. In Figs. 3(c)–3(f) and the Supplemental Material [24], Fig. 8, it is apparent that the PSF of the employed confocal configuration is oval shaped; this may be due to the system misalignment. However, our approach’s recovered PSF is isotropic, with improved resolution and SNR [Figs. 3(d), (e), (f)]. The Airyscan algorithm (developed by Zeiss) was also used to reconstruct a final image having both high resolution and high SNR. However, our approach outperforms the Airyscan algorithm in offering imaging resolution without sacrificing the SNR. This can be inferred by considering the two particles marked by the blue arrows in Fig. 3(f), which are more clearly separated than those in the other images. It should be noted here the VIKMON algorithm is actually slower than the Airyscan algorithm (see Note 4 in the Supplemental Material [24]). 3D information of the nanoparticle sample was also reconstructed (see the Supplemental Material [24], Video 1). By comparing the z slices of the nanoparticles obtained using confocal microscopy and our approach, the superior ability of our approach to reject the defocus-fluorescence-induced background is observed. This is because the reconstructed fluorescence signal of emitters attenuates more sharply as the defocus increases.

After the nanoparticle imaging verification, we applied our approach to biological samples. We imaged the mitochondria of BPAE cells using confocal microscopy, Airyscan, and VIKMOM (Fig. 4). As expected, the fine structures in the mitochondria are clearly visible in the VIKMOM image [Fig. 4(c)], proving the enhanced resolution and SNR compared with the confocal system (with a 0.2 and 1.25 AU detectors) and Airyscan. Then, we analyzed the cytoskeletons of U373 astrocyte cells to visualize their microtubule networks (see the Supplemental Material [24], Fig. 9), and also find superior resolution and SNR using VIKMOM. To check the improved details observable with VIKMOM, we also added the experiments with STED microscopy to demonstrate the authenticity (see the Supplemental Material [24], Fig. 10).

Further, we verified the improved optical sectioning ability of our approach by the analysis of 3D biological structures of the U373 HAs’ microtubules, acquiring 6-μm-thick volume information with slices separated by 200 nm (the Supplemental Material [24], Fig. 12 and video 2). The VIKMOM image contrast is greatly increased owing to the combined physical and computational removal of the out-of-focus fluorescence, thus demonstrating our approach’s improved sectioning capability [Supplemental Material [24], Figs. 12(c)–12(e)]. We also demonstrated the applicability of our approach in living cells by recording a dynamic video of living U373 HAs’ mitochondria (see Supplemental Material [24], video 3). Moreover, we experimentally investigated the feasibility of applying our approach for the analysis of multicolor biological samples by adding multiple excitation channels (Supplemental Material [24], Fig. 13).

Interestingly, the ability to recover the excitation OTF in our approach would allow us to combine this technique with nonlinear excitation mechanisms for further enhancing the resolution, while the prior knowledge of the excitation mechanisms is unnecessary here. For example, if our approach is combined with two-photon excitation, its penetration depth can be dramatically improved because of the low scattering and absorption of the sample. Compared with two-photon microscopy with an infinite detector, the lateral resolution can be enhanced by a factor of 2.6 if our detection setup and reconstruction algorithm are used (see Supplemental Material [24], Fig. 14). Further, the STED mechanism can be introduced to our approach for further resolution enhancement in scenarios with low depletion beam power. In STED the achievable resolution depends on the power of the depletion beam, denoted by the saturation factor ξ, which is the ratio of the peak intensity of the doughnut-shaped depletion spot to the saturation intensity of the used fluorophores. By implementing a detector array in the STED system and using our reconstruction algorithm to process the captured pictures, we can further improve the resolution with a given power of the depletion beam, or dramatically decrease the required depletion beam power while obtaining relatively high resolution (see simulation results in the Supplemental Material [24], Fig. 15). The latter feature has particularly important research value, because biological samples cannot endure high-power illumination.

In summary, we provide a new superresolution imaging approach for fluorescent samples by integrating the virtual k-space modulation method and a ptychography-inspired imaging procedure. The reported approach is able to improve the lateral resolution by a factor of 2, reduce the background level, improve the sectioning effect, and correct for unknown optical aberrations. We tested the performance of the reported approach by imaging fine structures of 2D BPAE cells and 3D living HAs. Besides, we can potentially extend this method for deep tissue imaging via two-photon excitation and for nanoscopy imaging via STED.