Reconstruction of angle-resolved backscattering through a multimode fiber for cell nuclei and particle size determination

Abstract

We demonstrate reconstruction of angle-resolved optical backscattering after transmission through a multimode fiber. Angle-resolved backscattering is an important tool for particle sizing, and has been developed as a diagnostic modality for detecting epithelial precancer. In this work, we fully characterized the transfer function of a multimode fiber using a plane-wave illumination basis across two dimensions. Once characterized, angle-resolved scattering information which has been scrambled by multimodal propagation can be easily and accurately reconstructed. Our technique was validated using a Mie theory-based inverse light scattering analysis (ILSA) algorithm on polystyrene microsphere phantoms of known sizes. To demonstrate the clinical potential of this approach, nuclear morphology was determined from the reconstructed angular backscattering from MCF-10A human mammary epithelial cell samples and validated against quantitative image analysis (QIA) of fluorescence microscopy images.

1. Introduction

Endoscopic imaging has become a widely researched field due to its ability to visualize inaccessible structures^1–5 including the esophagus, colon, and cervix. Typically, an imaging fiber bundle is used as a flexible image conduit,^6,7 where each single-core element at the distal face maps light to a corresponding location on the proximal side of the bundle to relay intensity-based images while maintaining spatial orientation. Unfortunately, most fiber bundles perform poorly for coherent imaging applications with mismatched optical path length and coherent crosstalk between elements. Also, our recent work analyzed the open area ratio (OAR) between fiber bundles,⁷ where OAR determines the percentage of the fiber end face that is capable of transmitting light. Because fiber bundles have a sparse field of single-mode cores interspersed among a surfeit of space-occupying cladding, commercially available fiber bundles usually have relatively low OAR (poor light throughput). An alternative approach is to relay the image using a single multimode fiber, using the multiple modes supported by the fiber to encode spatial image data. Imaging through a multimode fiber has a clear advantage over transmitting images through a fiber bundle, including improved resolution, throughput, simplicity, cost, and the ability to access more spatially constrained applications such as brain imaging. However, an image transmitted through a multimode fiber is necessarily distorted into a complex speckle pattern,⁸ which must be computationally reconstructed to form the image collected at the input of the fiber. Fortunately, optical transmission through a multimode fiber constitutes a linear system, for which any input function can be reconstructed from its corresponding output function, assuming complete knowledge of the system’s transfer function.

Recent advances have shown various robust image reconstruction techniques to address multimodal image reconstruction. The most common technique is the transmission matrix (TM) method^9–13 where the multimode fiber is treated as a highly turbid medium, and the input-output response of the fiber is characterized by measuring the transmission matrix or output speckle patterns for a wide array of spatial delta functions at the fiber’s input face. This technique is highly robust to changing input functions, but can be somewhat computationally intensive. A second approach is the machine learning (ML) method,^14,15 where deep neural networks are trained to reconstruct the input images from the distorted speckle patterns at the output face. This method can be fast, but requires complex equipment such as spatial light modulators (SLMs) or digital micromirror devices (DMDs) to produce robust training sets, and is not usually effective for images outside of the training set.¹⁶ Finally, the compressive sensing (CS) method¹⁷ applies the same basic approach as the TM technique, but uses a small subset of the transmission matrix and the assumption of a sparse sample to simplify the reconstruction. While effective and fast, CS is limited to highly sparse samples.

TM, ML, and CS have proven effective for recovering spatial image data after transmission through a multimode fiber, but through a simple Fourier relationship, angular scattering information can also be obtained. Angle-resolved detection of elastic scattering has already proven to be an effective diagnostic tool in biological tissues to diagnose dysplasia,^5,18–24 an early carcinogenesis state. Pre-cancerous cells exhibit morphologic changes compared to healthy cells, including variations in size and relative refractive index of the nucleus. Because angular light scattering is highly sensitive to changes in scatterer properties, the size and relative refractive index of scattering objects such as cells, nuclei, and even smaller organelles can be monitored using angle-resolved scattered fields. In particular, techniques such as angle-resolved low-coherence interferometry (a/LCI),^4,5,24 light scattering spectroscopy (LSS)²⁵ and bright and dark field OCT (BRAD-OCT)²⁶ take advantage of angle-resolved light scattering to acquire diagnostic information from tissue, and would easily benefit from a method for transmission of angle-resolved light scattering through a multimode fiber.

In this work, we explore the capability of accurately determining the size of scattering particles using their backscattering signal, which has been scrambled via transmission through a multimode optical fiber. The transmission matrix (TM) approach is utilized with a plane-wave basis set at various angles, which more closely matches the parameters being measured than the traditional basis set of spatial delta functions. The technique is first validated by determining the size of polystyrene microspheres embedded in polydimethylsiloxane (PDMS), using an inverse light scattering analysis (ILSA) algorithm based on Mie theory. To illustrate the applicability of this approach for biological media, angle-resolved scattering from a sample of MCF-10A human mammary epithelial cells is reconstructed, and analyzed to determine the average size of the cell nuclei. The size of nuclei determined from the reconstructed angular scattering is validated against quantitative image analysis (QIA) of fluorescence microscopy images using a DAPI nuclear stain. This work will serve as a foundation for future endoscopic light scattering techniques by enabling transmission and reconstruction of the angular scattering spectrum of biological media using a simple multimode fiber.

2. Method

2.1. Instrumentation

The schematic diagram of the system used for studying multimode transmission of angular scattering is shown in Fig. 1(a). Light from a Helium-Neon laser (λ = 632.8 nm) with a power of 7.5mW is first spatially filtered and delivered into either the calibration path (shown in dark red) during measurement of the transmission matrix, or the imaging path (shown in light red) during sample measurement. To record the transmission matrix, a two dimensional MEMS mirror (Mirrorcle Technologies, Inc., Richmond, CA) is used to illuminate the multimode fiber (200 μm core, 0.39 NA, Thorlabs, Inc., NJ) at various input angles via an objective lens (OL1: 10x/0.45 NA, ZEISS). While traditional TM approaches build the matrix by scanning a point along the fiber face, we utilize a plane-wave basis by scanning a collimated beam at the fiber face at various angles, and recording the corresponding output speckle pattern for each plane wave. In total, 6561 images are recorded, covering an angular range of ±16.2° in both θ_x and θ_y, with an angular resolution of 0.20°. The complete set of output images forms a basis set which fully characterizes the relationship between the input and output of the multimode fiber. During the measurement process, the calibration path is blocked, and the sample is placed in the imaging path, where the backscattered photons from the sample are directed into the same multimode fiber. The distorted image received from the fiber which encodes the angular scattering data is relayed onto the camera (Grasshopper3, 60 FPS, 2048×2048 pixels/image, with a pixel size of 5.5 μm). Only intensity-based measurements are used in this current setup. Representative transmitted calibration images for selected incident angles are shown in Fig. 1(b).

FIG. 1.

(a) Experimental setup for measuring the transmission matrix of the multimode fiber. OL1, OL2: objective lens. The dark red path is used to measure the transmission matrix of the fiber at each angle, and is blocked except during calibration. The embedded image is a schematic showing the illumination angles (θ_x, θ_y) and an example of a transmission matrix element for a given (θ_x, θ_y) at the distal end of the fiber. In total, 6561 images were recorded, covering a range of ±16.2 degrees in both θ_x and θ_y. (b) Representative images at different illumination angles constituting the transmission matrix. Images are displayed with enhanced contrast for clarity.

2.2. Transmission Matrix Method and Mie Theory ILSA

The key challenge of the reconstruction process is to unscramble light after propagation through the multimode fiber, and accurately recompute the scattering distribution collected at the input face of the fiber. Here, we use a similar approach to the transmission matrix method, where the multimode fiber is treated as a scattering medium,²⁷ as it transforms the field at the input face into plane waves propagating through the fiber at multiple angles. The input-output response of the multimode fiber is then characterized by measuring its transmission matrix. By computing the correlation between the transmission matrix and the transmitted image, we can recover the original angular spectrum of the sample field amplitude collected at the input end of the fiber.

The reconstruction process is shown with an example in Fig. 2(a) and 2(b). The distorted image after transmission through the multimode fiber, shown in Fig. 2(a), encodes the angular scattering profile of NIST traceable microsphere phantoms with a mean size of 6.01 ± 0.04 μm. Through a projection operation of the distorted image onto each angular component of the transmission matrix, the angular spectrum of the target can be retrieved using:

$$A\left({\theta }_x,{\theta }_y\right)=\sum A_{\text{trans }}\left(x,y;{\theta }_x,{\theta }_y\right)A_{\text{I}}(x,y)$$ (1)

Where A_I(x, y) is the speckle pattern transmitted through the fiber and A_trans(x, y; θ_x, θ_y) is the complete transmission matrix. A median filter is applied to the raw reconstructed image to reduce noise. Because of the azimuthal symmetry of the backscattering profiles,²⁸ a radial integration is performed from the center of the two-dimensional reconstruction to obtain a 1D angular scattering distribution. This step helps to reduce noise arising from the inversion process.

FIG. 2.

Reconstruction and ILSA processing of angular scattering collected from a 6 μm microsphere phantom and transmitted through a multimode fiber. The distorted image is shown in (a). This image is projected onto a set of pre-recorded transmission matrix images to reconstruct a 2D angular backscattering profile, visualized in (b). The azimuthal symmetry of the scattering profile becomes evident. A radial integration is then applied from the center to obtain the 1D angular scattering distribution (red line, (c)), which is then compared to a Mie-theory-based library to find the best fit (blue line in (c)). The scatterer size is correctly determined to be 6.0 μm.

Once the angle-resolved scattering profile is reconstructed, the scatterer morphology must be determined using ILSA^29–31. A detailed description of the algorithm is discussed in the literature³². Briefly, the integrated 1D profile is low-pass filtered and detrended using a second-order polynomial to isolate the oscillatory component of interest.³³ Fortunately for our application, this also serves to suppress speckle noise arising from the inversion. The extracted oscillatory part of the light scattering distribution, shown as the red line in Fig. 2(c), is then compared to a library of simulated profiles based on Mie theory to find the best fit (shown as the blue line) using chi-squared (χ²) error as a comparative metric.³⁰ The algorithm then produces a scatterer size prediction for each collected backscattering profile.

The scattering library for microsphere phantoms library was created using MiePlot³⁴, with microsphere and medium refractive indices of 1.58 and 1.41 respectively, and including a 1% standard deviation in size distribution for the scatterers. The scatterer diameter was varied between 4–14 μm with an increment of 0.1 μm to encompass the range of scatterers measured in this work. For cell measurements, based on the typical range of cell nuclear diameter and refractive index ratio, a library was generated with nuclear and cytoplasmic refractive indices ranging from 1.42–1.47 and 1.36–1.39 respectively, with a nuclear diameter ranging between 6–18 μm in increments of 0.1 μm and a variable size standard deviation including 1%, 2.5%, 5%, 7.5% and 10% variability to account for heterogeneity of sizes in the sample. In total, the libraries contained 101 spectra for the technical phantoms, and 14,520 for cells.

The transmission matrix reconstruction and ILSA was performed using MATLAB R2019a (MathWorks, Inc., Natick, MA). The total time to acquire the transmission matrix was 2.8 minutes, which consisted of acquiring 6,561 images (2048*2048 pixels/image) with an integration time of 12.5 ms per image and resting time between images which allowed the MEMS mirror to settle. Image reconstruction took 12 mins, with an average of 120ms for constructing each transmission matrix. ILSA for a single image required 290 ms of processing time on a standard desktop computer (Intel(R) Core i7–8700 processor). Statistical analysis was performed using R 3.6.3 (RStudio, Boston, MA). The data that support the findings of this study are available from the corresponding author upon reasonable request.

2.3. Sample preparation

To validate our approach, technical phantoms were constructed using polystyrene microspheres (Thermo Fisher Scientific, Microgenics Corporation, Fremont, CA) embedded in PDMS. A detailed description of our phantom protocol can be found in the literature³⁵. Briefly, the polystyrene microspheres were centrifuged and dried to form a pellet, and mixed with PDMS elastomer base and curing agent (Sylgard 184, Dow Corning, Midland, MI) with a pre-mixed ratio of 10:1 by weight in a plastic dish. The samples were left overnight in a vacuum to remove bubbles, and left for 72 hours at room temperature to fully cure. Microspheres of 6.0, 8.0 and 10.1 μm were utilized to determine the particle sizing capacity of our system.

To determine our system’s capacity for measuring biological samples, immortalized human mammary gland epithelial cells (MCF-10A, ATCC CRL-1031) were cultured in Brugge’s media (Department of Cell Biology, Harvard Medical School) consisting of DMEM/F12 (Invitrogen) supplemented with Horse Serum (Invitrogen), EGF (Peprotech), Hydrocortisone (Sigma), Cholera Toxin (Sigma) and Insulin (Sigma). Cells were maintained at 37°C with 5% CO₂ and 100% humidity under normal passaging protocols.

In order to perform the angular scattering measurements, MCF-10A epithelial cells were removed from the culture surface using 0.25% trypsin (GIBCO), which was then deactivated by adding growth media. The sample was transferred to a conical vial and centrifuged to form a dense pellet. The supernatant was removed, and the cell pellet was sandwiched between two glass coverslips separated by an adhesive spacer. This scattering from the sample was measured using our system (N = 6 measurements for each sample, integration time = 240 ms/sample). To validate our nuclear size measurements, fluorescence microscopy was performed using a nuclear stain on a ZEISS Axiovert 200 microscope (40X). For these measurements, MCF-10A epithelial cells were plated on a dish (FluoroDish FD5040) seeded at a concentration of 6×10⁵ cells/ml. The plated cells were then stained with media containing 3μM 4’,6’-diamidino-2-phenylindole (DAPI, Thermo Fisher Scientific, Microgenics Corporation, Fremont, CA), and incubated for 10 mins at 37°C with 5% CO₂ and 100% humidity. The residual staining solution was removed by rinsing the cell culture dish with phosphate-buffered saline (PBS). Cells were covered in live cell imaging solution (Thermo Fisher Scientific) prior to imaging.

3. Results

The approach was initially validated by measuring angle-resolved backscattering from various polystyrene microspheres. Fig. 3 (a–c) shows size determination from reconstructed distributions, using the minimization of the chi-squared value, for polystyrene microspheres with a manufacturer verified size of 6.01 ± 0.04 μm, 7.98 ± 0.08 μm and 10.12 ± 0.06 μm. The computed angular scattering function and its corresponding theoretical best-fit are shown in Fig. 3(d–f) for each sample. The red, yellow and green points in Fig. 3 (a) to 3(c) indicate that the chi-squared values are minimized at values of 6.0 ± 0.3 μm, 8.0 ± 0.9 μm and 10.2 ± 0.8 μm respectively, with the uncertainty given by the range of the calculated minimum chi-squared value is doubled.²⁸ This produces a close prediction for the scatterer’s diameter, showing excellent agreement with the size specified by the manufacturer. The angular range from 0° to 16.2° collected by the multimode fiber is sufficient to accurately measure the size of the scatterer, and satisfies the required resolution for diagnostic modalities such as a/LCI.³⁶

FIG. 3.

(a-c) Minimization of chi-squared error to determine the best theoretical fit for 6, 8 and 10.1 μm polystyrene microspheres in PDMS using Mie theory-based ILSA. Mie fitting results show excellent agreement with the actual size of the polystyrene microspheres, with a measured diameter of 6.0 ± 0.3 μm, 8.0 ± 0.9 μm, and 10.2 ± 0.8 μm respectively, with the uncertainty given by finding range which doubles the minimum chi-squared error. (d-f) Reconstructed and best-fit theoretical scattering functions for representative color points in (a)–(c) for 6, 8 and 10.1 μm microspheres, shown in red, yellow and green, respectively.

To prove the reliability of the reconstruction and ILSA results, a calibration curve summarizing all microsphere phantom measurements is shown in Fig. 4. Each microsphere phantom was measured six times, with each measurement occurring at different sites to avoid degenerate measurements. The 6.01, 7.98 and 10.12 μm microspheres are accurately measured with averages and standard deviations of 5.90 ± 0.17 μm, 7.90 ± 0.21 μm and 10.09 ± 0.33 μm respectively, producing an excellent coefficient of determination (R²) of 0.964, relative to the line of perfect agreement (blue line). All measurements of scatterer size fall in the range of sub-wavelength accuracy (±λ= 0.633 μm), shown as dotted lines in the figure, and the mean error across all measurements was only 0.26 μm.

FIG. 4.

Calibration curve demonstrating scatterer sizing capability of the system. Microspheres of approximately 6, 8 and 10 μm diameters were accurately measured as 5.90 ± 0.17 μm, 7.90 ± 0.21 μm and 10.09 ± 0.33 μm, respectively, and all results fall in the range of sub-wavelength accuracy.

In order to demonstrate the capability of determining cell nuclear size in biological samples, a dense pellet of MCF-10A cells was imaged. The reconstructed 2D scattering profile of a single measurement is shown in Fig. 5(a), where a clear azimuthal pattern is visually apparent. After a radial integration, filtering and polynomial subtraction to isolate the scattering from the nuclei using a/LCI analysis³⁷, we are able to extract the average nuclear size and density from the best fitted Mie profile, shown in Fig. 5(b). The obtained nuclear size after Mie fitting was averaged across six measurements, resulting in a mean nuclear diameter of 9.63 ± 1.03 μm.

FIG. 5.

(a) Angular reconstruction from one experimental measurement of MCF-10A angular scattering. (b) Mie fitting results of the profile in (a) predicts the average nuclear diameter and relative refractive index of the sample site that to be 9.6 ± 1.0 μm and 1.04. (c) Fluorescence images of DAPI stained MCF-10A cells, scale bar = 50 μm. (d) Average predicted MCF-10A nuclear diameter from DAPI image analysis and reconstructed angular scattering.

Quantitative image analysis (QIA) of fluorescence micrographs of DAPI-stained MCF-10A nuclei were used for validation, shown in Fig. 5(c). ImageJ (U.S. National Institutes of Health) was used to segment the nuclei, and compute an effective diameter from the cross-sectional area as $d_{eff}=2\sqrt{\raisebox{1ex}{$Area$}\!\left/ \!\raisebox{-1ex}{$\pi $}\right.}$. From the core size of our fiber and the magnification of our lens, we estimate that angle-resolved detection collects an averaged scattering distribution from 20–30 cells, so QIA measurements were divided into six different clusters of cells with 25 measurements in each cluster. The mean diameters from each cluster were averaged, and the equivalent diameter of the nuclei was determined to be 10.10 ± 0.71 μm, with the uncertainty given by the standard deviation across the six clusters. As seen in Fig. 5(d), nuclear sizing based on Mie fitting is in close agreement with the fluorescence microscopy, with a difference between the mean diameter of only 0.47 μm, and no statistical difference between the predicted diameters using a Student’s t-test (p>0.05). A slight increase was observed from quantitative analysis of the cell measurements compared with Mie theory, which is consistent with results in our previous work.^38,39

4. Discussion

In our experiment, the reconstructed angular scattering information has an angular range of ±16.2 degrees, which is mainly due to the limited scanning ability of the MEMS mirror and the numerical aperture of the imaging setup, including the multimode fiber. As demonstrated in our previous work,³⁶ a minimum of 20 degrees is desired to produce high diagnostic accuracy from Mie fitting for light scattering measurements of nuclei in tissue, and the reduced angular range used here may somewhat diminish the accuracy of the computed nuclear size and density. However, the reduced angular range of the multimode fiber scheme is compensated by an improvement in angular resolution. The typical sampling period of fiber bundles used for clinical endoscopy would be ~0.5 degrees,³⁸ which is limited by the distance between two adjacent core elements in the fiber bundle. Our current setup improves upon this sampling limit, with an angular resolution of 0.20 degrees, which can be further improved by increasing the angular sampling frequency of the MEMS mirror. A further aspect to consider is the choice of multimode fiber. For this experiment, we used a 200 μm core multimode fiber with an NA of 0.39 (±22.9 degrees allowed), supporting approximately 13,000 unique guided modes which travel simultaneously through the fiber. A multimode fiber with more modes could increase the angular resolution, although the transmission matrix and computational complexity would grow larger. The fiber core also determines the field of view (FOV) that our system would be sampling. In a typical angular scattering measurement, the collected angular profile is an averaged scattering distribution formed by all the scatterers in the FOV, which would be 20–30 scatterers in our setup. Increasing the FOV brings more potential nuclei into the measurement, and this increases the optical signal. However, the nuclear sizes vary more significantly when sampled over larger length scales, which can undesirably cause the measured nuclei from the averaged distribution to be more heterogeneous if an excessively large FOV is chosen. Future work will focus on optimizing these parameters for better and faster measurements.

In our measurements of MCF-10A cells, we observed a mean difference of 0.47 μm between the light scattering measurements and the quantitative analysis based on fluorescence microscopy images. While the two measurements are quite close, the slightly smaller diameter as determined from scattering is consistent with previous work,³⁹ which suggests that assuming the cell nuclei have higher refractive index $(\text{m}=\raisebox{1ex}{$n_{nucleus}$}\!\left/ \!\raisebox{-1ex}{$n_{cytoplasm}$}\right.>1)$ leads to an underestimate of the nuclear size during nuclear morphology extraction. As recently shown, the cell nucleus has a lower refractive index than the cytoplasm for many cells.^40,41 In theory, angular scattering from lower-index nuclei will have similar scattering distributions to higher-index nuclei when m~1, where m indicates the ratio between nuclear refractive index to that of cytoplasm. Still, our assumption of the biophysical properties of the nucleus may contribute to some imprecision in our measurements.

Although our system represents an advance for reconstruction of the angular backscattering, some limitations remain that must be addressed prior to clinical implementation. Fiber bending is a major issue for computational imaging, since it severely alters the pre-collected transmission matrix. Our current setup pre-records the transmission matrix with an acquisition time of approximately 3 minutes, which may be too long to incorporate into a scheme for endoscopic surveillance. Faster acquisition, reduced sampling of the transmission matrix, and compressive sensing will each help to shorten the time required. Another limitation that hinders translation of this approach to clinical applications is the computational burden for accessing the large data set and reconstructing the angular scattering distributions from the speckle patterns. For this work, we used a high-resolution USB camera that did not support on-chip binning, and the transfer rate of the pixel data from the camera to the computer is the rate-limiting step. Our future work will optimize the setup and algorithm to design a more practical device with reduced pixel resolution and faster acquisition speed. Also, many methods have been introduced to overcome these limitations during reconstruction, including implementation of compressive sensing techniques,¹⁷ where fewer basis sets are used for reconstruction of samples with unique properties, such as sparsity. Our future work will employ symmetries and sparsity within the angular scattering distributions to create compressive sensing approaches that enable achieve faster transfer function characterization.

5. Conclusion

In this paper, we have demonstrated reconstruction of angular scattering distributions which have been transmitted through a single multimode fiber. An optomechanical system was developed to acquire nuclear morphology measurements using backscattered light. The results from this study showed accurate size determinations, demonstrating the feasibility of the transmission matrix approach and Mie-theory based ILSA to characterize the size of biological scatterers from multimode-fiber-collected angular backscattering distributions. It is our hope that improved computational imaging schemes will enable the use of multimode fiber for simple and robust medical diagnostics.