Interferometric synthetic aperture microscopy

Abstract

State-of-the-art methods in high-resolution three-dimensional optical microscopy require that the focus be scanned through the entire region of interest. However, an analysis of the physics of the light–sample interaction reveals that the Fourier-space coverage is independent of depth. Here we show that, by solving the inverse scattering problem for interference microscopy, computed reconstruction yields volumes with a resolution in all planes that is equivalent to the resolution achieved only at the focal plane for conventional high-resolution microscopy. In short, the entire illuminated volume has spatially invariant resolution, thus eliminating the compromise between resolution and depth of field. We describe and demonstrate a novel computational image-formation technique called interferometric synthetic aperture microscopy (ISAM). ISAM has the potential to broadly impact real-time three-dimensional microscopy and analysis in the fields of cell and tumour biology, as well as in clinical diagnosis where in vivo imaging is preferable to biopsy.

Methods of computed imaging have historically provided new levels of insight and utility when coupled with established instrumentation. Revolutionary advances in imaging and applications were made with the evolution of X-ray projections¹ into modern computed tomography², nuclear magnetic resonance^3,4 into magnetic resonance imaging⁵ and radar imaging into synthetic aperture radar (SAR)^6,7. The algorithms for these modalities arise from an analysis of the physics connecting the sample structure to the measured signals, that is, by solution of the appropriate inverse problem.

Existing methods in three-dimensional optical microscopy typically require that the focus be scanned through the entire region of interest. In wide-field microscopy, the focus must be scanned through each depth and the modality has poor optical sectioning ability. In confocal microscopy⁸, multiphoton microscopy⁹ and optical coherence microscopy¹⁰, volumetric data are collected over several en face planes one point at a time owing to the small focal region of high-numerical-aperture optics. Other modalities such as optical coherence domain reflectometry¹¹, coherence scanning microscopy¹², optical coherence tomography (OCT)¹³, scanning white light interferometry¹⁴, full-field optical coherence microscopy¹⁵, optical frequency-domain imaging¹⁶, spectraldomain phase microscopy¹⁷ and interferometric microscopy¹⁸ are variants of coherence imaging, each with distinct instrumentation. In each variant of OCT, a three-dimensional image may be obtained with a fixed focal depth. However, the image quality degrades away from the focal plane and is only practically representative of the sample within one confocal parameter. To resolve large volumes in OCT, low-numerical-aperture optics are used to achieve a high depth of field at the expense of degraded transverse resolution¹⁹. Images in the aforementioned modalities are formed essentially by plotting raw data. Some attempts have been made recently to develop methods of optical computed imaging. In highly scattering, thick tissues, optical diffusion tomography can provide images with resolution of about 0.5 cm (ref. 20). For semitransparent materials with slowly varying indices of refraction, projected index computed tomography²¹ provides computed reconstruction of the real refractive index. Optical projection tomography²² is based on computed tomography techniques where the gaussian beams are approximated as pencil beams and scattering is ignored, thus encountering the same resolution/distortion compromise as conventional interference microscopy. All of the above-mentioned methods fail to account for the diffraction and the coherent scattering of light by the sample, and therefore do not properly determine the object’s structure outside the focus. Narrowband holographic modalities, such as diffraction tomography^23–25, incorporate the coherent scattering of light, but require special instrumentation, where control for the illumination and detection of the field is decoupled. Extensive literature on the three-dimensional transfer function for optical microscopies, such as OCT²⁶, provides additional insight for interference microscopes. In this article, a new sampling and signal-inference strategy for coherence imaging is described and demonstrated.

We present a novel modality that achieves depth-independent resolution throughout a volume where the focus is fixed at one depth. This modality, interferometric synthetic aperture microscopy (ISAM), provides three-dimensional object structure through solution of the inverse scattering problem, accounting for scattering, diffraction and beam parameters. ISAM is a novel computed imaging method in the same class of mathematical methods, that is linear inverse problems, as computed tomography, magnetic resonance imaging and SAR. In all of these modalities, the resultant images are far simpler for the human operator to interpret than the unprocessed data. For example, SAR has long been used to produce high-resolution terrain and topography maps by combining the returned radar signals from a target that has been illuminated from many directions into a single image. Similarly, ISAM multiplexes raw acquisitions to provide quantitatively meaningful data with reliable spatially invariant resolution at all depths within a sample, greatly increasing the diagnostic value of the data.

Many design variations of broadband synthetic aperture instrumentation systems are possible, but a system that facilitates uniform coverage in the Fourier space with a relatively simple illumination and detection scheme, such as the interferometric microscope, is preferable. The ISAM system (Fig. 1) consists of a fibre-optic Michelson interferometer seeded by a source of femtosecond pulses, where the axial resolution is inversely dependent on the bandwidth. A spectral interferometer²⁷ measures the interferometric cross-correlation between a fixed-delay reference pulse and the pulses reflected back from the sample. The spectrum measured on the line camera corresponds to the Fourier transform of the cross-correlation signal, from which the amplitude and phase of the reflected field from the sample are inferred. The sample probe beam is focused by a microscope objective so that the beam focus is at a fixed plane inside the sample. The transverse resolution, which is spatially invariant for ISAM, is determined by the numerical aperture of the objective. We implement a planar scanning geometry, although ISAM solutions for other geometries, that is rotational²⁸, full-field²⁹ and so on, are possible. At each transverse position of the beam, the spectral interferogram of the back-scattered optical field is measured. The beam is laterally translated in two dimensions through the sample by moving a stage or by steering the beam with a pair of galvanometer-driven mirrors before entering the objective. An interferometric microscope may be converted to an ISAM system with modifications that would allow for phase stability, a high-numerical-aperture objective, two-dimensional scanning and computer processing. Near-infrared light for ISAM imaging can penetrate to depths in highly scattering tissue of around 2 mm. Of course, a sufficiently highly scattering or absorbing tissue may reduce the depth of penetration.

Figure 1. Interferometric synthetic aperture microscopy (ISAM) system using spectral detection.

A low-coherence light source is focused into a sample and the back-scattered field is interferometrically measured with a spectral detector.

A scalar model for light propagation has been used, where at each frequency, ω=kc/n₀, n₀ being the constant background index of the sample and c the speed of light, the field obeys the reduced wave equation ∇²U(r)+k²U(r)=−4πη(r)U(r), where r denotes the position, U is the field, η is the scattering potential within the scalar model and the index of refraction is $n^2=n_0^2(1+4\pi \eta /k^2)$. We obtain a mathematical model of ISAM by considering the propagation of the focused beam from the objective into the sample (into some volume V), scattering within the sample (in the first Born approximation), the propagation of the scattered light back into the objective (over some surface Σ) and the measurement of the cross-correlation with the reference pulse. The signal, S, resulting from the first Born scattering process is a function of the transverse position of the beam, r₀, and the wavenumber, k, and is given by the expression³⁰

$$S({\mathbf{r}}_0,k)=A(k){\int }_{\Sigma }{\mathrm{d}}^{2}r{\int }_V{\mathrm{d}}^{3}r\prime G(\mathbf{r}\prime ,\mathbf{r},k)g(\mathbf{r}\prime -{\mathbf{r}}_0,k)\times \eta (\mathbf{r}\prime )g(\mathbf{r}-{\mathbf{r}}_0,k)$$ (1)

where g describes the normalized gaussian beam profile, A²(k) is the power spectral density of the source and G is the Green function. The normalized beam profile is given by g(r, k) = W⁻²(k)e^−r2/2W2(k) /2π, where W(k) = α/k, α = π/NA and NA is the numerical aperture of the objective. The beam profile g(r, k) can be any number of aperture functions that describe the optical transfer function, but a reconstruction will only be practical if the resultant kernel in equation (1) is well conditioned and invertible. The Green function is given by G(r′, r, k) = e^ik|r−r′|/|r−r′|. After two-dimensional Fourier transformation with respect to r₀, and further manipulation, the two-dimensional Fourier transform of the signal is given by the expression

$$\mathrm{S̃}(\mathbf{Q},k)=A(k)\int {\mathrm{d}}^{2}q\int \mathbf{d}z\prime \frac{i2\pi }{k_z(\mathbf{q})}{\mathrm{e}}^{i{k}_z(\mathbf{q})(z\prime -z_0)}{\mathrm{g̃}}_0(\mathbf{q},k){\mathrm{e}}^{i{k}_z(\mathbf{Q}-\mathbf{q})(z\prime -z_0)}\times {\mathrm{g̃}}_0(\mathbf{Q}-\mathbf{q},k)\mathrm{\eta ̃}(\mathbf{Q};z\prime ),$$ (2)

where $k_z(\mathbf{q})=\sqrt{k^2-q^2},z_0$ is the position of the beam focus, η̃(Q, z′) is the two-dimensional transverse Fourier transformof the scattering potential, whose arguments are the transverse wavevector Q and the axial coordinate z′, q is a variable of integration and g̃₀(q, k) = e^−q2α2/2k2 is the two-dimensional Fourier transform for the beam profile g (r, k) in the waist plane of the beam. After the expression for g̃₀ is substituted into equation (2), and an asymptotic expansion³¹ of S̃ is made for large α², this relationship reduces to

$$\mathrm{S̃}(\mathbf{Q},k)=A(k)\left(\frac{i2{\pi }^2}{k_z(\mathbf{Q}/2)}\frac{k^2}{{\alpha }^2}{\mathrm{e}}^{-\frac{{\alpha }^2{Q}^2}{4k^2}}\right)\tilde{\tilde{\eta }}(\mathbf{Q};-2k_z(\mathbf{Q}/2)),$$ (3)

where $\tilde{\tilde{\eta }}$ is the three-dimensional Fourier transform of η, that is the one-dimensional Fourier transform of η̃(Q; z) with respect to z. This expansion is valid even when NA ≈ 1 because α² is sufficiently large for the first term of the expansion to dominate. Equation (3) relates the three-dimensional Fourier transform of the object scattering potential to the two-dimensional Fourier transform of the signal. Implicit in this formula is a diagonal linear integral operator in the three-dimensional Fourier space of the scattering potential, and so the achievable resolution is spatially invariant and does not depend on the proximity to the focus.

Because of the simple relationship between the scattering potential and the signal, ISAM can be implemented efficiently by resampling or interpolating the data in a manner analogous to the numerical implementation of the Fourier projection-slice theorem³² as used in X-ray computed tomography or SAR, but the resampling grid for ISAM is elliptical rather than polar. The resampling result can be thought of intuitively as compensation for the accumulated propagation effects in the double-pass system with the given dispersion relation. In addition, because equation (3) expresses a multiplicative (or diagonal) relationship between S̃ and $\tilde{\tilde{\eta }}$, generalization to a regularized inversion method such as Tikhonov regularization³³ is straightforward. This solution provides spatially invariant resolution, but does not correct for the intensity-dependent attenuation away from the focus. The approximate fall-off of intensity from the lens is calculated as a multiplicative factor on the scattering potential given by I_z (z) = (α²/k₀ +i(z −z₀))⁻¹, where α²/k₀ is the Rayleigh range, and (z − z₀) is the distance from the focus²⁷. Thus, at x_r Rayleigh ranges from the focus, the expected signal-to-noise ratio (SNR) is proportional to the intensity drop-off, ${(1+x_r^2)}^{-1/2}$.

Because ISAM is a multiplexed measurement method, the SNR will decrease if the measurement is not phase stable. To achieve phase stability of the signal, a microscope coverslip is placed on top of the sample and the top reflection from the air–coverslip interface acts as a fixed reference delay relative to the object. The delay fluctuations of the interferometer were removed from each crosscorrelation interferogram by locating the air–coverslip reflection in each interferogram, estimating the phase and group delay of the reflection and applying the opposite phase and group delay to the entire interferogram.

A tissue phantom consisting of a collection of titanium dioxide scatterers with a mean diameter of 1 µm and uniformly suspended in silicone was imaged using an ISAM system and a 0.05-numerical-aperture objective. Further details of the imaging system and data processing are included in the Methods section. Figure 2 shows cross-sections through an unprocessed data set (left) and ISAM reconstruction (right) of a volume 360 µm × 360 µm (transverse) ×2,000 µm (axial). It contains three pairs of en face sections for both the unprocessed data (1–3) and the ISAM reconstructions (4–6). The distances from the en face section planes to the focus, located at z = 0, are z = 1,100 µm (1,4), z = 475 µm (2,5) and z = −240 µm (3,6). These sections show that the reconstruction has resolved the scatterers throughout a range of depths over nine times the Rayleigh range from the focus, where the Rayleigh range is commonly defined as half of the depth of field, or what is considered in focus in optical imaging systems. In the unprocessed data, the interference between the signals scattered from adjacent scatterers is evident. Our method properly accounts for the diffraction of the beam, and so separates the superimposed signals from the scatterers to form well-resolved point images on all planes. Figure 3 shows the volume-rendered tissue phantom with both the unprocessed (Fig. 3a) and the reconstructed (Fig. 3b) interferometric data. A constant axial scaling of 0.25 has been applied to Fig. 3 to fully visualize the volume. Again, the unprocessed data show point scatterers outside the confocal region that are not resolved, whereas the ISAM reconstruction shows all points resolved. It should be noted that the SNR of the reconstruction decreases away from the focus. However, this is a minor effect compared with the defocus and fringing effects of the unprocessed data. Animations of these data are included as on-line supplements and demonstrate spatially invariant resolution (See Supplementary Information Movies S1–3).

Figure 2. Interferometric data from a tissue phantom consisting of titanium dioxide scatterers suspended in silicone.

Planar slices of the three-dimensional unprocessed data (left) and ISAM reconstruction (right) are shown for two en face planes above the focus and one below the focus. 1–3, Magnified unprocessed sections for three depths, z=1,100 µm (1), z=475 µm (2) and z=−240 µm (3), where z=0 µm is the focal plane. 4–6, Magnified ISAM reconstructions for the corresponding planes, respectively. The scale bar represents 20 µm.

Figure 3. Volume-rendered interferometric data.

a, Unprocessed data of the tissue phantom (Fig. 2) are volume rendered to view distortion caused by the focusing lens. b, The ISAM reconstruction of the tissue phantom (Fig. 2) is volume rendered to view the spatially invariant resolution. In both parts the longitudinal axis is scaled by 0.25.

ISAM is a novel image-formation method that uses the fundamental resolution potential of the acquired optical signal on the basis of the physics of the scattering within the detection beam. ISAM is a physics-based solution of the inverse scattering problem yielding understood quantitative results. The ISAM image-formation algorithm can be implemented with a computational complexity of O(NlogN), where N is the number of volume elements to resolve, which makes ISAM amenable to real-time imaging. Furthermore, the ISAM algorithm can be applied paraxially to planes as well as volumes, thus enhancing cross-sectional imaging.

Dispersion compensation algorithms with the same computational complexity as ISAM are currently being used in real-time cross-sectional optical imaging systems at 10 frames s⁻¹, where each frame is 1,000 axial scans^34,35. Our ISAM model incorporates dispersion compensation as a prior when computing S(r₀, k) from the acquired signal S(r₀, ω) (ref. 27). A third-order polynomial is commonly sufficient for modelling the resampling necessary for dispersion compensation. To re-index the detected digital signal from ω to a uniform k space, a re-indexing array i_n is calculated,

$$i_n=n+{\beta }_2{(\frac{n}{N}-{\omega }_{\text{ctr}})}^2+{\beta }_3{(\frac{n}{N}-{\omega }_{\text{ctr}})}^3,$$

where N is the array size, ω_ctr is the centre frequency and n is an integer between zero and N − 1. By selecting from a range of values for β₂ and β₃, we can adjust the amount of second- and third-order dispersion correction, respectively. Values of β₂ and β₃ should be selected to be less than N/2 and N/3, respectively, to prevent aliasing. To illustrate the application of dispersion compensation in ISAM imaging, Fig. 4 shows selected frames and magnified sections from a Supplementary Information Movie S4 provided on line. This movie shows a cross-sectional scan of the previously described tissue phantom for an evolution of the resampling parameters, starting from the raw data (Fig. 4a), progressing through the dispersion-compensated data (Fig. 4b) and finishing with the full two-dimensional ISAM reconstruction (Fig. 4c).

Figure 4. Cross-sectional scan of the tissue phantom.

a–c, A stepwise image sequence of image reconstruction illustrating the unprocessed temporal data (a), the dispersion-compensated data (b) and the two-dimensional ISAM reconstruction (c).

Human tumour tissue was resected and imaged ex vivo. Sections were marked with India ink after imaging and before embedding to register locations. Figure 5 includes en face planes (sections A and B) of the unprocessed data (Fig. 5c,d), where the beam diffraction effects are evident, the computed ISAM reconstructions (Fig. 5e,f) and images of corresponding registered histological sections (Fig. 5a,b). Although embedding, sectioning and staining of tissue can disrupt features to some degree, the registered histological sections provide prominent features for comparison. In the reconstruction, boundaries between adipose cells and the margin between adipose and fibrous tissue are clearly identified, with a strong correspondence to histology. Whereas the histological images were obtained by destroying the sample, ISAM could readily be applied for in vivo applications because signal collection is in the back-scattered epi-direction. All sections, regardless of the distance from the focus, were reconstructed with the resolution equivalent to the in-focus plane of the unprocessed data.

Figure 5. En face images from human breast tissue.

Images are shown for depths of z=591 µm (section A) and z=643 µm (section B), where z=0 µm is the focal plane. Histological sections (a,b) show comparable features with respect to the unprocessed interferometric data (c,d) and the ISAM reconstructions (e,f). The ISAM reconstructions resolve features in the tissue that are not decipherable from the unprocessed data. The green dashed arrow indicates the fast-scanning direction for the volumetric data acquisition. The scale bar represents 100 µm.

Using ISAM, the nearly featureless unprocessed planes in the data volume are transformed into planes with meaningful structure of potential diagnostic value. Such results may prove to be especially useful in clinical situations where in vivo imaging is preferable to a biopsy. Thus, it may be possible to carry out micrometre-scale in vivo imaging over larger volumes of tissue rather than resecting these tissue volumes. Furthermore, ISAM has the potential to achieve high-speed, high-resolution cross-sectional or volumetric imaging without the need for the time-consuming processing, sectioning and staining of a resected specimen. With the use of near-infrared light, high-resolution ISAM facilitates the noninvasive monitoring of cellular and nuclear morphology.

ISAM brings the power of computed imaging to interferometric microscopy. Specifically, research in mathematical methods and algorithms in other areas of computed imaging may be borrowed or transferred to ISAM. ISAM has the potential to improve the noninvasive in situ visualization of micrometre-scale morphological features in the biological, medical and material sciences. Nearterm future developments for ISAM include solving the vectorized problem for quantification and analysis of polarization effects. Solutions for other imaging geometries, for example rotational²⁸ (catheter-based), full-field²⁹ (array-based) solutions, are also actively being developed.

METHODS

SPECTRAL DETECTION FOR INTERFEROMETRIC MICROSCOPY

ISAM measurements are made using a femtosecond spectral interferometer (Fig. 1). A femtosecond laser (Kapteyn-Murnane Laboratories, Boulder, Colorado) delivers ultrashort pulses to provide broadband illumination for ISAM. The centre wavelength of the source is 800 nm, with a bandwidth of 100 nm. These first-order field quantities fluctuate too rapidly to be detected directly, thus an optical-fibre-based Michelson interferometer is incorporated. The illumination is divided by a 50:50 fibre-optic coupler (splitter) between a reference arm containing a delay mirror and a sample arm that contains a lens (objective) to focus the gaussian beam into the sample. The focal length of the objective is 12 mm, the spot size is 5.6 µm, the confocal parameter (depth of focus) is 239 µm and the numerical aperture is 0.05. The light returns from the sample and reference arms and is directed into a spectrometer. In the spectrometer, the light is collimated with an achromatic lens having a 100 mm focal length and dispersed off a blazed gold diffraction grating, which has 830.3 grooves mm⁻¹ and a blaze angle of 19.70° for a blaze wavelength of 828nm (53004BK02-460R, Spectra-Physics). To reduce lens aberrations, the dispersed optical spectrum is focused using a pair of achromatic lenses each with a focal length of 300 mm. The focused light is incident on a line-scan camera (L104k–2k, Basler) which contains a 2,048-element CCD (charge-coupled device) linear array with 10 × 10 µm detection elements. The camera operates at a maximum readout rate of 29 kHz, thus one axial scan can be captured during an exposure interval of about 34 µs. The sample/specimen was placed on the ISAM stage and three-dimensional image data were acquired by scanning the incident beam in the transverse en face plane using a pair of computer-controlled galvanometer-scanning mirrors and/or the stage. Raster scanning of the beam was implemented in the planar geometry with equidistant sampling. The frame rate depends on the number of axial scans acquired per image or volume. In our experiment, a series of spectrum images (1,000 × 2,048 pixels) was acquired, each in 34 ms. Subsequent volumetric scans were carried out by acquisition of a series of adjacent two-dimensional scans. The data are sent to a frame-capture card (PCI-1428, National Instruments), which receives an external trigger from a galvanometer-controller card (PCI-6711, National Instruments), which activates the frame acquisition.

METHOD FOR CALCULATING THE INVERSE SCATTERING SOLUTION

Before processing, the spectra, each representing a column of depth-dependent data, are assembled adjacently as the beam is transversely scanned over the sample. The detected digital signal is interpolated to account for the non-uniform sampling of the spectrum and to compensate for up to third-order dispersion³⁵. Specifically, the signal is interpolated by a factor of two by a band-limiting interpolator implemented using the fast Fourier transform (FFT). This prepares the signal for the cubic B-spline interpolation³⁶, which has a transfer function with an amplitude that attenuates frequencies close to the Nyquist limit. The cubic B-spline interpolator resamples the spectrum to a uniform frequency space according to a calibration procedure using a single reflector placed at the focus of the objective lens. Sample movement, inducing phase and group delay changes, is tracked using a reference microscope coverslip, and the deviations are corrected. At this point, the quantity S(r₀, k) in equation (1) has been estimated. Next, the two-dimensional FFT in the transverse directions is calculated to yield S̃(Q, k). Then, the non-uniform ISAM resampling and filtering of equation (3) using cubic B-splines is implemented to yield $\tilde{\tilde{\eta }}$. Finally, the three-dimensional inverse FFT is used to obtain the ISAM reconstruction, an estimate of η(r), and the magnitude of the result is plotted in the figures.

Although our formulation does not require longitudinal scanning of the focus to achieve uniform resolution, it may be possible to combine measurements from a variety of focal depths to improve SNR. Because our method provides a quantitative reconstruction uniform in the object space from data acquired at each fixed depth, multiple scans, even those with foci many Rayleigh ranges away from one another, may be combined, for example with a least-squares or maximum-likelihood approach.

TISSUE-PHANTOM DESIGN AND PREPARATION

Tissue phantoms were constructed using a mixture of silicone and titanium dioxide (TiO₂) particles to simulate the optical and elastic properties of tissue³⁷. A stock silicone solution was prepared consisting of 88.91% specialty polydimethylsiloxane (PDMS), 11.03% PDMS curing agent (General Electric RTV-615A and B, respectively, Circuit Specialists) and 0.06% TiO₂ (Sigma-Aldrich No. 224227, mean size 1 µm, <5 µm), resulting in a solution with a ratio of TiO₂ to silicone of 572 µg g⁻¹. This solution was sonicated for 10 min below 25 °C and mixed with 100% pure PDMS fluid (50 cSt viscosity, ClearCo) to make samples of various TiO₂ concentrations. Our imaged phantom was created with a TiO₂ concentration of 286 µg g⁻¹, and sonicated for an additional 5 min. The sample, weighing 10 g, was poured into a 35mm cell culture dish, cured overnight at 80 °C and cured for an additional 24 h at room temperature.

Human tissue was acquired under Institutional Review Board protocols approved by the University of Illinois at Urbana-Champaign and Carle Foundation Hospital. Tissue was excised from a 1.7 cm breast tumour. The specimen was placed in a sealed container with normal saline, and refrigerated. Within 10 h from resection, the specimen was imaged with ISAM. Imaging locations were marked with India ink to facilitate image registration with subsequent histological sections. Immediately after imaging, the imaged tissue was placed into buffered formalin, and underwent standard histological processing with paraffin embedding. The paraffin-embedded tissue was sectioned into 6 µm slices and placed on microscope slides. The sectioned tissue was stained with haematoxylin and eosin and viewed at ×20 magnification using a light microscope (BH-2, Olympus). Images were captured with a colour digital camera (SSC-DC54A, Sony) at the registered locations.