Real-time Jones phase microscopy for studying transparent and birefringent specimens

Abstract

Tissue birefringence is an intrinsic marker of potential value for cancer diagnosis. Traditionally, birefringence properties have been studied by using intensity-based formalisms, through the Mueller matrix algebra. On the other hand, the Jones matrix description allows for a direct assessment of the sample’s anisotropic response. However, because Jones algebra is based on complex fields, requiring measurements of both phase and amplitude, it is less commonly used. Here we propose a real-time imaging method for measuring Jones matrices by quantitative phase imaging. We combine a broadband phase imaging system with a polarization-sensitive detector to obtain Jones matrices at each point in a megapixel scale image, with near video rate capture speeds. To validate the utility of our approach, we measured standard targets, partially birefringent samples, dynamic specimens, and thinly sliced histopathological tissue.

1. Introduction

Anisotropic specimens are characterized by a refractive index tensor, which can report on the organization of tissue at the molecular scale (for waves propagation in anisotropic materials, see, e.g., the classical reference by Yariv [1]). Thus, polarization-sensitive measurements can infer birefringence, which can be used as an intrinsic contrast marker [2,3]. This marker has been widely applied to studying both material and biological samples [4–8]. The most common formalisms for describing polarized light are the intensity-based Mueller algebra and the field-based Jones algebra [9]. The Stokes-Mueller formalism fully describes partially polarized light. Various methods for measuring the Mueller matrix in tissues have been established in the past [10–18]. However, this intensity-based formalism does not capture explicitly the phase information in the sample, which plays an important role in many applications such as pathology [19–22]. On the other hand, the Jones matrix is formulated in terms of fields and requires interferometry to characterize the amplitude and phase response of the specimen.

Quantitative Phase Imaging (QPI) [23,24] reveals the phase map associated with the sample, which has proven valuable for characterizing cells and tissues [25–37]. More recently, QPI’s potential for diagnosis has been demonstrated in various types of cancer [38,39]. Previously, Jones matrix retrieval has been demonstrated in OCT and, later, QPI [40–44]. Yet, existing laser-based QPI Jones methods are limited by laser speckle, which limits the phase sensitivity of the measurements [45–48].

Here we present, for the first time to our knowledge, an LED-based Jones microscope system, capable of retrieving full Jones matrices at high throughput. This is possible by using a micropolarizer array at the camera plane and dedicated software for automation. As a result, our instrument yields Jones matrices at each pixel, with an acquisition rate of 5 frames per second, at 1.3 megapixel each. We built our real-time Jones phase microscope (rJPM) based on an off-axis, common-path QPI method, namely, diffraction phase microscopy (DPM) [49,50]. DPM can recover phase maps from only one interferogram, which in turn yields high frame rate imaging. Furthermore, the common-path interferometry geometry passively ensures the stability of our phase images. Because we use broadband LED illumination, speckles are significantly suppressed [51].

2. Results

2.1. Real-time Jones phase microscope (rJPM)

As illustrated in Fig. 1, the rJPM system is developed as an add-on to an existing inverted microscope (AxioObserver Z1, Zeiss). To reduce exposure time and achieve high throughput performance, we selected a bright LED (Thorlabs, SOLIS 623C, 623 nm) as our illumination source. The polarization state generator (Fig. 1), placed after the light source, consists of a polarizer and liquid crystal variable retarder (LCVR, Thorlabs LCC1223-A). The linearly polarized light emerging from the polarizer passes through the LCVR, with its slow axis aligned at 45° with respect to the incident polarization. The LCVR introduces a controlled phase delay between the horizontal and vertical polarizations, thus, turning linearly polarized light into any form of elliptical polarization. By precisely calibrating the voltage of the retarder to add π phase delay between the x and y optical field components, we produce two orthogonal linear polarizations at 45° and −45° with respect to the input axis. Throughout our experiments, we used 20x/0.4NA microscope objective.

Fig. 1.

Schematic of rJPM. An LED source is used for illumination. The PSG turns unpolarized light into elliptical polarization. By precisely calibrating the voltage of the LCVR, we introduce π and 2π phase delays separately, to obtain ±45° linearly polarized light. The microscope objective used throughout our experiments is 20x/0.4NA. A DPM (diffraction phase microscopy) interferometry module is built at the output of the brightfield microscope. The module contains a grating, which diffracts the light into several orders. The 0 order is spatially filtered to act as a reference for the interferogram, which is detected by a polarization camera. The Bayer-filter like mask on the camera provides 4 different images corresponding to linear polarizations oriented at 45° with respect to one another. We use the 0° and 90° analyzer images and incident light at ±45°, to measure 4 images that uniquely yield the Jones matrix.

We constructed an off-axis interferometer based on DPM [52] (Interferometry Module in Fig. 1) to measure the phase-shifts associated with the object for each incident polarization. This module starts with a grating at the image plane of the microscope and diffracts the light into several orders of decreasing intensity. The focal lengths of lenses L1 and L2 are 100 mm and 300 mm, respectively, which yields an additional magnification of 3x. A spatial filter is placed at the Fourier plane of lens L₁, which allows the +1 diffraction order to pass through, low-pass filters the 0th order with a 75 microns diameter pinhole, and blocks all the other orders. The +1 order and filtered 0 order interfere at the conjugate plane of the 4-f system. At this position, we use a polarization-sensitive camera, with a Bayer-filter like mask to create polarization-sensitive pixels (FLIR, BFS-U3-51S5P-C). In this work, it was observed that a relatively weak extinction ratio (150:1) has little effect on rJPM as the unwanted transmitted unpolarized signal is common to all pixels and canceled when multiple images are combined to form the Jones matrix map. Each group of 4 pixels detects light through polarizers oriented at 45° with respect to one another. As there are four complex numbers in a Jones matrix, at least four independent measurements of complex fields are needed. For each of the orthogonal incident polarizations, ±45°, we record an amplitude and phase image at two perpendicular analyzer directions, 0° and 90°, so that the Jones matrix can be measured in two shots.

2.2. Extracting the Jones matrix

To understand image formation in rJPM, we follow the derivation in [46], which describes a slower, non-common path technique that uses four images to obtain the Jones matrix. The two generated ±45° incident illuminations are characterized by the following Jones vectors

$$\begin{array}{c}{E}_1=\frac{1}{2}\left[\begin{array}{c}1\\ 1\end{array}\right]\\ E_2=\frac{1}{2}\left[\begin{array}{c}1\\ -1\end{array}\right].\end{array}$$ (1)

The Jones matrix characterizing the sample is generally

$$J_S=\left[\begin{array}{cc}{J}_{XX}& J_{XY}\\ J_{YX}& J_{YY}\end{array}\right].$$ (2)

Where the matrix elements are complex quantities. In our setup, the last component that the field passes through is the polarization-sensitive camera, which is illustrated in Fig. 1. The Jones matrices associated with the micropolarizers on the camera are

$$\begin{array}{c}{J}_1=\left[\begin{array}{cc}1& 0\\ 0& 0\end{array}\right]\\ J_2=\left[\begin{array}{cc}0& 0\\ 0& 1\end{array}\right].\end{array}$$ (3)

J₁ characterizes the 0° and J₂ the 90° analyzer. After the analyzers, the light is linearly polarized either along the x or y direction, according to $j=1,2$. The object field after the analyzer, $O_{jk}$, is obtained by combining Eqs. (1)–(3), namely,

$$O_{jk}=J_j\times J_S\times E_k,j,k=1,2.$$ (4)

Equation (4) yields the object field $O_{jk}$, when the sample is illuminated by vector $E_k$ and detected through analyzer $J_j,j,k=1,2$, for a total of 4 measurements.

Because the 0-order field is low-passed filtered by the pinhole, it loses the structural information about the sample, which now appears to be characterized by a unitary Jones matrix, namely,

$${\overline{J}}_S=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right].$$ (5)

Thus, we can write this reference field at the camera plane as

$$R_{jk}=J_j\times {\overline{J}}_S\times E_k,j,k=1,2.$$ (6)

The intensity distribution at the image plane is the result of the interference between the object and reference fields, specifically,

$$\begin{array}{c}{I}_{jk}=|O_{jk}+R_{jk}{|}^2\\ =|O_{jk}{|}^2+|R_{jk}{|}^2+O_{jk}{R_{jk}}^{\ast }+{O_{jk}}^{\ast }{R}_{jk},j,k=1,2.\end{array}$$ (7)

From the measurement, we obtain the interference term, $O_{jk}{R_{jk}}^{\ast }$, by performing filtering on the recorded image, in the Fourier domain. Furthermore, we can express the result explicitly in terms of Jones matrix elements by using Eqs. (4) and (6),

$$\begin{array}{c}{O}_{11}{R}_{11}^{\ast }=A_{11}(J_{XX}+J_{XY})\\ O_{12}{R}_{12}^{\ast }=A_{12}(J_{XX}-J_{XY})\\ O_{21}{R}_{21}^{\ast }=A_{21}(J_{YX}+J_{YY})\\ O_{22}{R}_{22}^{\ast }=A_{22}(-J_{YX}+J_{YY}).\end{array}$$ (8)

Which can be re-written as,

$$\left[\begin{array}{c}{O}_{11}{R}_{11}^{\ast }\\ O_{12}{R}_{12}^{\ast }\\ O_{21}{R}_{21}^{\ast }\\ O_{22}{R}_{22}^{\ast }\end{array}\right]=\left[\begin{array}{cccc}{A}_{11}& A_{11}& 0& 0\\ A_{12}& -A_{12}& 0& 0\\ 0& 0& A_{21}& A_{21}\\ 0& 0& -A_{22}& A_{22}\end{array}\right]\times \left[\begin{array}{c}{J}_{XX}\\ J_{XY}\\ J_{YX}\\ J_{YY}\end{array}\right].$$ (9)

In Eqs. (8)–(9), the A elements take into account variations in the absolute values of the illumination and detection intensities and are obtained experimentally according to Eq. (7). In order to obtain A, we measure intensity values of the background, in areas without sample. Finally, the Jones matrix elements can be extracted by using the inverse of the matrix in Eq. (9),

$$\left[\begin{array}{c}{J}_{XX}\\ J_{XY}\\ J_{YX}\\ J_{YY}\end{array}\right]={\left[\begin{array}{cccc}{A}_{11}& A_{11}& 0& 0\\ A_{12}& -A_{12}& 0& 0\\ 0& 0& A_{21}& A_{22}\\ 0& 0& -A_{21}& A_{22}\end{array}\right]}^{-1}\times \left[\begin{array}{c}{O}_{11}{R}_{11}^{\ast }\\ O_{12}{R}_{12}^{\ast }\\ O_{21}{R}_{21}^{\ast }\\ O_{22}{R}_{22}^{\ast }\end{array}\right].$$ (10)

2.3. Software for real-time operation

Much like in color imaging, as each pixel only reports a single polarization state, it is necessary to demosaic the image to obtain the remaining three polarization directions at that pixel. For this work, we only use two directions of the analyzer images, 0° and 90°, as the +/- 45° are a mix of these polarization states. We chose to use bicubic interpolation, which is known to produce particularly high quality interpolation results [53]. However, the improvement in image quality comes at a substantial computational cost, as some pixels receive information from thirty-two sample points. To address these concerns, we performed demosaicing on the GPU in conjunction with specialized texture memory, which automatically caches adjacent memory reads, while avoiding computation for index variables [54]. With this approach, large neighborhood reads are mostly performed in-cache so that rate-limiting global memory access is reduced by an order of magnitude. The resulting implementation runs in less than 4 ms on a laptop GPU (Quadro P3200). Compared to the built-in, vendor-supplied 8-bit demosaicing software, our 16-bit implementation preserves the resolution of the camera’s ADC (Analog to Digital Converter). Following the design in [55], computation is overlapped with acquisition, with the hardware portions of the acquisition process being the rate-limiting factors. Data acquired for this paper were taken at 5 frames per second, with 70 ms allowed for stabilizing the LCVR and 30 ms used for exposure. At present, the limitation on the acquisition throughput is mostly due to the polarization state generating component, and the illumination intensity.

2.4. rJPM imaging of control samples

In order to test the accuracy of the phase measurement, we imaged one-micron polystyrene microspheres, immersed in oil (Zeiss, Immersol 518F). One-micron bead looked slighter wider due to the diffraction limit, however, it can prevent potential phase wrapping. The refractive index of the beads is 1.588 at 623 nm wavelength, while that of the oil is 1.518. As a result, the expected phase shift from the bead is,

$$\begin{array}{c}\varphi =\frac{2\pi }{\lambda }\text{d}(n-n_0)\\ =0.705\end{array}$$ (11)

The measurement is shown in Fig. 2. We visualize the four complex elements of the Jones matrix by splitting them into amplitude and phase images. As expected in an isotropic sample, J_XX and J_YY look identical and can be considered the typical phase map, while there is no signal shown in J_XY and J_YX. Figure 2(b) indicates the phase values of the bead, the peak value is 0.692, which is close to the expectation. The standard deviation in J_XX phase map is 0.0296 rad.

Fig. 2.

Measured Jones matrix of a 1 micron bead (isotropic sample). For each Jones element, amplitude map is shown on the left and the phase map on the right. As an isotropic sample, the images show no signal in the off-diagonal terms, while the diagonal images look identical. The phase map in the diagonal images represent the typical quantitative phase map. The measured (0.692) and expected (0.705) phase values are comparable.

Next, we measured an anisotropic sample of known properties, namely the Thorlabs polarization resolution target (Thorlabs R2L2S1B), as shown in Fig. 3. The sample was made of a liquid crystal polymer sandwiched between two pieces of glass (N-BK7). The polymer provided a phase delay of 280 nm. The slow axis of the inscribed numbers and lines was at 45d with respect to that of the background. We measured the structure of 18 periods/ mm, rotated at an angle of approximately 20°. As one can see in Figs. 2(a) and 3, there is a significant difference in the background values associated with the phase of the off-diagonal elements, as expected. The stripes in amplitude map look slightly larger because the intensity around sample is affected by it. The global background phase is caused by the birefringent in background and varies depending on the angle of the target.

Fig. 3.

Jones matrix of a polarization resolution target. Each image is a 3 × 3 mosaic. The period of these lines is 1/18 mm. For each Jones element, amplitude map is shown on the left and the phase map on the right. All the amplitude maps and, separately, all the phase maps, share the same color bar. As an anisotropic sample, there is strong signal in the off-diagonal images.

Comparing the result of beads to the polarization test target, we note that the beads exhibit strong phase noise fluctuations in J_XY and J_YX. This noise is due to the subtraction of two images, of complex values, which results in randomly fluctuating arg values. Physically, as the off-diagonal terms vanish for isotropic samples, the phase is not defined in these Jones matrix elements. By contrast, the resolution target’s background exhibits birefringence, and, as a result, its phase map does not suffer from strong noise.

2.5. rJPM imaging of inhomogeneous and anisotropic materials

Figure 4(a) illustrates a polarization-directed-flat-lens, which is composed of a three-dimensional pattern of liquid crystals on a thin film [56]. The lens focuses one circular polarization and diverges the orthogonal circular polarization. We scanned a centimeter square area and chose four different positions to show their Jones matrices. Figure 4(b) shows a full Jones matrix at position I. Figure 4(c) shows the J_XY phase maps at positions II - IV. Our images are in good agreement with our prior knowledge, specifically, that the lens resembles a circular grating with anisotropic refractive index. The central part of the lens has a lower period, which leads to a smaller refraction angle, consistent with a Fresnel lens operation.

Fig. 4.

Imaging a polarization directed flat lens. A typical flat lens, illustrated in (a), focuses one circular polarization and diverges the orthogonal circular polarization due to the grating-like concentric circle structure. We scanned one centimeter square area. To clearly represent its property, we measured the Jones matrix at 4 different positions, as indicated. (b) Jones matrix at position I. For each parameter, amplitude map is shown on the left and phase map on the right. All the amplitudes and all the phase maps share the same color bar. (c) The phase map of J_XY at positions II - IV. Our results show the spatially-varying anisotropic grating period.

To demonstrate our real-time imaging capability, we imaged the dynamic anisotropic response from the pixels of a spatial light modulator (Haloeye, LC2002). The SLM introduces phase delays with pixel-level resolution. We generated a signal to produce periodically changing birefringence and record a 400 seconds video (see supplemental video Visualization 1 ^{(11.5MB, mp4)} ). Figure 5(a) represents the Jones matrix at t=90 s. The change with time is illustrated by choosing one-pixel traces in Fig. 5(b). We measured the mean of the amplitude versus time associated with the liquid crystal pixel shown in Fig. 5(a).

Fig. 5.

Dynamic measurement of inhomogeneous birefringence in a spatial light modulator. During the 400 seconds measurement, we recorded a Jones matrix (see supplemental video Visualization 1 ^{(11.5MB, mp4)} ). (a) The Jones matrix at t=90 s. (b) Mean values of J_XX and J_XY, as indicated, for the black rectangle area shown in (a).

2.6. rJPM imaging of histology samples

Finally, we measured the Jones matrix map associated with a dense connective tissue of a mouse ligament. The slide is prepared by typical histopathology, with a thickness of 3–4 microns [55]. The fibrous ligament tissues are known to exhibit strong birefringence [57]. It is worth noting that the strength of the signal is related to the thickness of the sample. However, our previous studies using electron microscopy revealed that the thickness across the slide is typically fairly uniform [38]. The off-diagonal elements, J_XY and J_YX, in Fig. 6 show significant anisotropy, as expected. Second-harmonic generation microscopy has revealed that collagen organization can report on cancer aggressiveness [58]. Recently, QPI has been shown to recover these findings with a faster and simpler imaging system [59]. As a QPI-based approach, we anticipate that rJPM can potentially be used for diagnosis and prognosis in cancer pathology.

Fig. 6.

Jones matrix of dense connective tissue. Each image is a 4 × 4 mosaic. For each Jones element, the left image represents the amplitude map, while the right shows the phase. This birefringent signal can potentially be used for diagnosis and prognosis in cancer pathology.

3. Summary and discussion

In this work, we proposed and validated a real-time imaging instrument that measures the Jones matrix at each pixel with near video speed. Owing to the use of GPU based image reconstruction algorithms, and task-parallel software implementation, our instrument efficiently overlaps computation with acquisition so that throughput is primarily limited by the speed of the LCVR modulator. We demonstrated this capability by imaging the dynamic fluctuations of a liquid crystal device, providing an example of an industrial application for our new-found technique. Our system’s optical design highlights the advantages of broadband illumination and common-path construction by measuring the Jones matrix of thinly sliced histopathologically resected tissue. However, because of multiple scattering, rJPM is limited to thin samples. With the polarizers and spatial filter, the light intensity is significantly reduced, so one needs to trade-off exposure time for capture speed, or increase the light source power potentially damaging living samples.

Because it is implemented as an upgrade module to an existing microscope, rJPM can be easily adopted in laboratories that already use commercial imaging systems. Due to the common-path geometry that we adopted, the system is intrinsically stable to mechanical vibrations, which lowers further the threshold for adoption. The result instrument is fully automated and represents essentially a whole slide scanner, ready to image birefringence map in thin biological specimens.

Funding

China Scholarship Council10.13039/501100004543 (201806160045); National Science Foundation10.13039/100000001 (2013AA014402, CBET0939511, NRT-UtB 173525); National Institutes of Health10.13039/100000002 (CA238191, GM129709).

Disclosures

The authors declare that there are no conflicts of interest related to this article.