Cellular analysis using label-free parallel array microscopy with Fourier ptychography

Abstract

Quantitative phase imaging (QPI) is an ideal method to non-invasively monitor cell populations and provide label-free imaging and analysis. QPI offers enhanced sample characterization and cell counting compared to conventional label-free techniques. We demonstrate this in the current study through a comparison of cell counting data from digital phase contrast (DPC) imaging and from QPI using a system based on Fourier ptychographic microscopy (FPM). Our FPM system offers multi-well, parallel imaging and a QPI-specific cell segmentation method to establish automated and reliable cell counting. Three cell types were studied and FPM showed improvement in the ability to resolve fine details and thin cells, despite limitations of the FPM system incurred by imaging artifacts. Relative to manually counted fluorescence ground-truth, cell counting results after automated segmentation showed improved accuracy with QPI over DPC.

1. Introduction

Traditionally, the dominant imaging modalities for label-free imaging have included Zernike phase contrast, differential interference contrast, and darkfield microscopy. While qualitatively useful, these techniques can introduce various imaging artifacts and are not inherently quantitative techniques. Segmentation and cell counting remains difficult and limited due to low contrast, even with the use of modality-specific segmentation algorithms [1].

Quantitative phase imaging (QPI) is an imaging modality that can in principle provide high contrast, artifact-free images of optically thin, transparent cell samples. QPI provides a direct measurement of the phase delay which occurs as light passes through the sample. This data is ideal for downstream label-free analysis and the extraction of useful metrics [2] such as cell density, morphology, and volume.

Label-free measurements are particularly valuable in the study of biological samples as these minimize the risks of unforeseen chemical damage or staining-induced modifications. Moreover, QPI enables live-cell imaging and can eliminate the need to sacrifice samples which is inherent to running end-point analyses and is often required to inform subsequent steps in an experiment. From single cell studies [3] to entire histology slide imaging [4], QPI has helped to advance a wide range of biomedical and diagnostic applications [5].

Fourier ptychographic microscopy (FPM) [6,7] is one technique for obtaining quantitative phase and brightfield images. This technique requires an image set comprised of multiple images, each taken with a specific illumination angle and divergence. Illumination aside, the imaging path is otherwise fairly conventional and similar to that found in a traditional brightfield microscope setup. Given such an image set, it is possible to computationally reconstruct the optical field. Notably, and advantageously, the resolution of this reconstructed field is limited by the large (potentially very large) numerical aperture of combined illumination sources, and not the imaging objective as in conventional microscopy. In practice, this means that an FPM system can use a relatively low magnification objective to obtain a large field of view (FOV), while achieving a resolution over this FOV comparable to that of a higher magnification objective [8].

FPM-based microscopes and QPI offer a number of significant application-defining features. Utility in the expanding market of high-throughput screening is promising [9], with proposed label-free imaging systems expected to deliver major cost and time savings in throughput. Both automated optical aberration correction [10] and super-resolution imaging [11] are also possible with FPM. These features are particularly enabling with the availability of compact and inexpensive imaging sensors and molded plastic optics. An FPM system could be constructed as a large array with one sensor and miniaturized optics for each well of a 96, or even 384, microwell plate to provide for simultaneous imaging of all wells [12,13]. The capability to image all sample wells all at once is significant for live-cell experiments, especially for cases where fast stimulus-responses are expected and/or observed. Multiwell, parallel imaging for FPM is also compelling with its accelerated phase image reconstruction (all cameras receive identical illumination sequence conditions), minimized camera idling time, and overall faster throughput. In contrast, other approaches to quantitative phase microscopy cannot be scaled in this way. Digital holographic microscopy, for example, would be extremely challenging to implement in a massively parallel format due to the requirement for the distribution and interferometric recombination of a reference beam for each well.

In the landscape of quantitative phase microscopy techniques, there is not a strictly fundamental reason to expect FPM to exceed other techniques in terms of quantitative phase retrieval accuracy. However, and more importantly, FPM can enable implementations with unique practical capabilities. Therefore, comparisons to other imaging techniques under practical imaging conditions are of considerable interest.

2. System configurations

The SimulPhi-6 system (CBBP-FPM Inc., CA, USA) is a parallel array microscope system for imaging 6-well microplates. This system is a commercial version of an experimental system previously described by Kim et al. 2016 [13]. While conceptually similar to the original laboratory implementation, the instrument in this work represents approximately three years of research and development to the hardware and associated software. The unit provides auto-focusing, dual-band fluorescence imaging, repeatable plate loading and compatibility with live-cell environments. The software suite includes an original implementation of the FPM reconstruction algorithm, acquisition control, data management, and FPM-specific segmentation tools.

The SimulPhi-6 system includes an array of six microscope units, with each unit consisting of a pair of conventional 4 X, 0.10 NA objective lenses configured for 1:1 imaging onto a 4.3 × 5.7 mm CMOS sensor. Illumination is provided by an LED array above the samples. For each microscope unit, an individual 520 nm LED is selected to provide illumination for each frame required for the FPM reconstruction.

The target NA of the QPI images is 0.30, a factor of 3 improvement over the native imaging resolution and comparable to a high quality conventional 10 X objective (10 X objectives typically range from 0.25 NA to 0.45 NA). Although phase and brightfield images are available after reconstruction, it is the phase modality image and associated quantitative data that is of primary interest. One unit of phase shift as presented in this work represents an 18.5 mrad theoretical phase shift at 520 nm.

While conventional multi-element glass objectives are used here, this system serves as a testbed for potential systems designed with simpler optics and intended for high-density imaging arrays. Miniaturized multi-element glass objectives would likely be impractical or expensive in that application.

We have also imaged samples for comparison in this work using a commercially available phase-sensitive imaging modality, digital phase contrast (DPC), on an Opera Phenix system (PerkinElmer, MA, USA). The DPC modality works by acquiring two defocused brightfield images at positions above and below the focal plane. Reconstruction of the final image takes into account changes in light intensities introduced by changes in the refractive index of the sample [14]. This produces a final DPC image with high signal to noise useful for cell segmentation. The DPC method is specifically based on solving the transport of intensity equation (TIE) [15]. Where FPM is designed to collect and analyze the interferometric pattern of multiple beams of light, the TIE method simply relies on characterizing the free-space propagation of a single beam. Microscopes with combined QPI methods are feasible using the TIE [16] as well as more complex imaging approaches exist to boost TIE image quality [17]. Ultimately for our study, the TIE method—otherwise referred to as DPC in this work—is directly compatible with brightfield microscopy and can be effectively utilized within a commercial system like the Opera Phenix.

Acquisitions on the Opera Phenix were performed using the 10 X, 0.3 NA objective lens configuration (1.3 × 1.3 mm field of view) so as to best match the target resolution output of the SimulPhi-6 quantitative phase images. Because of the much smaller native field, it was necessary to stitch together multiple fields to obtain the same field of view as provided by a single FPM reconstruction. The Opera Phenix system was also used to acquire the fluorescence images used in this study as a ‘ground truth’ image source for cell identification. Note that while the SimulPhi-6 system is capable of acquiring fluorescence images with two integrated laser sources (445 nm and 532 nm), this feature was not used as a primary source in this study. We note that because fluorescent proteins produce incoherent, isotropic emission, which is incompatible with the FPM technique, the integrated fluorescence imaging is limited to that of the native 0.10 NA of the objectives and is thus less desirable in this application.

3. Experiment methodology

We performed cell segmentation and counting on several cell types at varying seeding densities. These include:

• 1.Jurkat T-cells (T-cells): A high contrast cell line with minimal cytoplasmic area suitable as a straightforward baseline to compare all three modalities: fluorescence, DPC, and QPI.
• 2.Osteosarcoma cells (U2OS): A lower contrast cell line with more cytoplasmic area than T-cells and representative of many common cell lines, serving as a medium-difficulty scenario for the label-free modalities.
• 3.Induced Pluripotent Stem Cell-derived Cardiomyocytes (iPSC-CMs): A cell line with significant cytoplasmic area, very flat contrast, and morphological diversity serving as a higher-difficulty scenario for the label-free modalities.

We imaged each cell type at three different seeding densities: low, medium, and high. The resulting images were used in the following analysis and discussion. FPM image examples from the U2OS dataset with the three cell seeding densities are shown in Fig. 1. Equivalent images for T-cells and iPSC-CMs are similarly provided as Figures S1 and S2 respectively. The Opera Phenix system was configured to acquire the same fields of view (FOVs) as the SimulPhi-6 system to provide for direct comparisons. Specifically, the full and single FOV dimensions provided by the SimulPhi-6 system were matched with 25 FOVs with the 10 X magnification objective on the Opera Phenix system.

Fig. 1.

Example 6-well microplate with different U2OS cell densities imaged on the SimulPhi-6 system. All images in this figure were captured after fixation. Images A1 to A6 show full field of views for wells 1 to 6, respectively. Low, medium, and high cell densities are captured in the left column (A1 and A4), the middle column (A2 and A5), and the right column (A3 and A6), respectively. Images B1 to B6 show zoomed-in regions from images A1 to A6, respectively, as indicated by the red rectangles. Green scale bars represent 1 mm. Red scale bars represent 100 µm. The large-scale overlapping ‘weave’ pattern seen in images A1 to A6 is an artifact of FPM illumination.

4. Sample preparation

4.1. Jurkat T-cells (T-cells)

Jurkat T-cells were maintained in suspension with RPMI 1640 media (Corning) containing 10% (vol/vol) heat inactivated fetal bovine serum (hiFBS) and 1% (vol/vol) Penicillin-Streptomycin. For seeding into a 6-well microplate, cells were concentrated and re-suspended in PBS. Two wells per 6-well microplate were used for seeding approximately 1.5 × 10⁵, 5 × 10⁵, and 4 × 10⁶ cells. The cells were allowed to settle and attach to the well surface for 30 minutes at 37°C. Following fixation and fluorescent nuclear staining, described in Section 4.4, the cells were imaged on both the SimulPhi-6 and Opera Phenix systems (DPC and fluorescence modes) within two days of preparation.

4.2. Osteosarcoma (U2OS) cells

U2OS cells were maintained as monolayer cultures in McCoy’s 5A media (Iwakata & Grace Mod, Corning) containing 10% (vol/vol) FBS and 1% (vol/vol) Antibiotic-Antimycotic. Cells were harvested from culture using TrypLE Select (ThermoFisher) to allow for passage and were seeded in 6-well microplates (Corning) for imaging. Two wells per plate were used for seeding approximately 5 × 10⁴, 2 × 10⁵, and 8 × 10⁵ cells. The cells were allowed to attach to the well surface overnight in culture media at 37°C. After fixation and fluorescent nuclear staining, described in Section 4.4, the cells were imaged on both the SimulPhi-6 and Opera Phenix systems (DPC and fluorescence modes) within two days of preparation.

4.3. Human-induced pluripotent stem cell-derived cardiomyocytes (iPSC-CMs)

Cardiomyocytes were differentiated from human induced pluripotent stem cells (iPSC; ThermoFisher) using modifications to the monolayer-based differentiation protocol carried out by Lian et al. and licensed from the Wisconsin Alumni Research Foundation [18]. Briefly, iPSCs were seeded on PLO-laminin (Sigma; ThermoFisher) coated tissue culture plates at 40,000 cells/cm² in mTeSR1 (StemCell Technologies) until confluency reached 85-95%. Cells were then treated with 12 µM CHIR 99021 (Tocris) in RPMI-B27 basal medium (ThermoFisher) for 22-26 hours, before CHIR was replaced by RPMI/B27 basal medium. After 48 hours in basal medium, cells were treated with 5 µM IWP2 (Tocris) and a potent and proprietary selective tankyrase inhibitor (IC₅₀<10 nM in DLD-1 epithelial cells) for 48 hours in RPMI/B27 basal medium. IWP2 and tankyrase inhibitor were then removed and cells were grown in RPMI/B27 basal medium with insulin with medium changes every other day until harvest. Cells were harvested on day 11 to day 15 post-differentiation using TrypLE (ThermoFisher) and cryopreserved in CryoStor CS10 (StemCell Technologies).

Cardiomyocytes were thawed from cryopreserved vials for this study. Briefly, 1 mL vials were thawed in a water bath at 37°C for several minutes. Cells were then gently added drop-wise to 9 mL of cardiomyocyte maintenance media (CMM: RPMI 1640 with GlutaMAX, HEPES, 1x B27, 1x Penicillin-Streptomycin) with 10% FBS (to create neutralizing media). This cell suspension was centrifuged at 300 xg for 5 minutes. The supernatant was removed, the cell pellet was re-suspended in 10 mL of neutralizing media, and a small aliquot of this was used to count cells using an automated Vi-CELL XR Cell Viability Analyzer (Beckman Coulter). A specific protocol was prepared on this instrument to more accurately detect cardiomyocytes and ensure at least 70% viability. Cells were diluted appropriately in neutralizing media with 1% RevitaCell to obtain two wells per 6-well microplate with approximately 5 × 10⁵, 1 × 10⁶, and 1.5 × 10⁶ cells. Media was exchanged 24 hours after seeding. Note that at least 4 hours prior to seeding cells, these 6-well microplates were coated with Matrigel (Corning, hESC-Qualified Matrix), incubated at 37°C, and any unbound Matrigel was aspirated. Cells were maintained for 4 days in CMM (exchanged every 48 hours) until fixed and stained as described in Section 4.4. The cells were imaged on both the SimulPhi-6 and Opera Phenix systems (DPC and fluorescence modes) the same day after staining.

4.4. Fixation and staining protocol

The procedure for fixation and staining was the same for all sample types. Cells were first washed several times with PBS to remove cell media and cell debris. Cells were fixed with 4% paraformaldehyde in PBS for 15 minutes. Following several washes with PBS, cells were stained with 5 µM of SYTO 24 in PBS for 30 minutes. SYTO 24 (Invitrogen; S7559) was selected as a green fluorescent cell-permeant nuclear stain, compatible with fluorescence imaging on the Opera Phenix system. The cells were subsequently washed several times with PBS. Each well was carefully filled with 3 mL of PBS and all samples were stored at 4°C prior to imaging.

5. Cell segmentation and counting

Due to the inherent differences in image types, particularly with respect to contrast, the approach to cell segmentation and counting differs markedly between the DPC images from the Opera Phenix and the FPM images provided by the SimulPhi-6 system.

5.1. Digital phase contrast

The Columbus high-volume image data storage and analysis system (version 2.8) was used to configure a cell segmentation and counting procedure for DPC images. This procedure is typically ad hoc, with the user being responsible for organizing a set of sequential ‘building blocks’ to create a sample-specific image processing pipeline. Each building block offers various input parameters to adapt to the specific image operation (i.e., ‘find’ or ‘select’ objects, or ‘calculate’ properties). All first-level building block inputs require a data source (channel) or a mathematical approach (method), whereas second and third level input parameters allow for fine tuning. The following analysis sequence was used to count cells in the DPC images:

• 1.Input Image. View original images from selected data.
• 2.Find Cells. ‘Method P’ selected for initial detection of all possible cells in the DPC channel. Area, splitting sensitivity, and common threshold parameters are tested and set manually based on cell type.
• 3.Select Population. Remove all segmented cells in contact with the image border.
• 4.Calculate Morphology Properties. Collect roundness and area parameters for all selected cells.
• 5.Select Population. Filter out erroneously segmented objects and cells, including cell debris and background, based on object roundness and area parameter thresholds.
• 6.Define Results. Collect final cell counts.

5.2. Quantitative phase

Quantitative phase data enables a segmentation procedure driven by fundamental properties of a cell population. The segmentation method available with the SimulPhi-6 system is specifically designed for quantitative phase data. It is possible to configure and run the segmentation for a given cell type automatically, with minimal free parameters, minimizing user bias and ensuring that cell identification is based on a clear physical definition.

The segmentation method used for quantitative phase imaging (QPI) data here shares a similar underlying principle to that described by Loewke et al. 2018 [19]. We have prepared an original python implementation which leans heavily on routines from the scikit-image library [20]. Broadly, the method uses a conventional watershed segmentation method [21], seeded by detecting corners in the euclidean distance transform [22] of the original image. Unconventionally, the threshold phase level used in watershedding is automatically adjusted to the highest level for which the total optical volume of the segmented region, v_n, exceeds a given minimum volume threshold, v_th, after hole-filling. The segmentation proceeds iteratively, dividing the image into ever smaller segments until further segmentation is not possible given the volume threshold requirement.

An additional step used in this work is the final application of a flood-fill which starts from each identified nucleus, proceeding with lower phase levels until overlap with other cells occurs. This results in an additional expanded contour with optical volume v_b, typically enclosing the cytoplasm or other associated larger-scale cell structure.

The only necessary free parameter to which the simulation is especially sensitive is the minimum volume threshold, v_th. This is considered a property of the given cell population, representing the minimum volume of material required to ‘build’ a viable cell nucleus. Given a large dataset of the representative cell type, v_th can in principle be optimized automatically. Supplement 1 ^{(3.8MB, pdf)} Section 4 describes the technique and uses the U2OS cell data as an example.

Other relevant features of the QPI segmentation software available and used on a case-by-case basis include:

• 1.A filter for reducing high spatial frequency FPM artifacts in the segmentation step (see Figure S3 for example artifacts).
• 2.Multiple methods for determining the initial background, most notably the classic Otsu’s method [23] and the more robust Yen’s method [24].
• 3.A suite of tools for calculating various metrics for each segmented cell. These can be analyzed statistically, and the final results filtered based on these values to discriminate outliers.

6. Analysis results and discussion

Images for all FOVs on the Opera Phenix, for both the SYTO 24 fluorescence and DPC channels, were acquired sequentially in each well of the 6-well microplates. Images were automatically uploaded to Columbus for subsequent analysis. FPM images of the same regions were acquired simultaneously on the SimulPhi-6 system and saved for later analysis.

6.1. Manual fluorescence counting

DPC and QPI analyses were validated using ground-truth cell counts obtained from fluorescence images of SYTO 24 nuclear staining. A manual cell counting approach was applied here with the work assigned randomly to four individuals, each of whom were experienced in cell counting and were given the same reference examples showing cells and debris. Cells overlapping with image borders were not included in the final counts. The raw images were pre-processed in order to:

• 1.Crop and align images collected on the Opera Phenix to match the acquired FOVs from the SimulPhi-6 system. Note, multiple FOVs at 10 X on the Opera Phenix were needed to match the SimulPhi-6 full FOV.
• 2.Enhance the visibility of nuclei with low signal by manually boosting the image contrast.
• 3.Add gridlines, making it easier to traverse the image to record cell numbers (especially useful in the case of high cell density).

6.2. Automated cell counting

Automated cell counting in Columbus was performed on fluorescence and DPC images as described in Section 5.1. The analysis sequence was modified at step 2 to accommodate SYTO 24 fluorescence for the specific cell type. Automated cell counting on FPM images was performed as described in Section 5.2. Results presented in all bar plot and box plot figures were prepared in MATLAB, using the bar and boxplot functions respectively. Python was used to prepare all histogram results as part of the QPI segmentation method.

6.3. T-cell counting

Figure 2 shows T-cells at a medium seeding density captured with the three different imaging modalities and their associated segmentation results. Successful segmentation of the cell boundaries was reasonably straightforward for this homogeneous and round cell type in all modalities. Given the smaller size of these cells, minor adjustments in step 2 of the Columbus segmentation parameters were made to improve detection (area thresholds lowered and minimum valid nuclei spacing decreased) in both the fluorescence and DPC images. For QPI segmentation, we used default parameters and manually applied a volume threshold of 75 units. We found the resulting cell counts to be relatively independent of an exact threshold value. Because this cell type is very uniform, we did not make use of any boundary segmentation, nor did we find it necessary to apply any metrics-based exclusion criteria.

Fig. 2.

Segmentation of Jurkat T-cells at medium seeding density in SYTO 24 fluorescence, DPC, and FPM modalities. The fluorescence and DPC images show regions with segmentation overlay from Columbus whereas the FPM image shows the overlay from the QPI segmentation. Note that nuclei are detected in the fluorescence channel, while slightly larger objects, i.e. cells, are detected in the DPC and FPM channels. All larger image scale bars represent 100 µm and smaller inset scale bars represent 20 µm.

Figure 3 shows the cell counting results for the three different modalities as deviations relative to ground truth counts derived from the fluorescence data. Overall, we observed a general deviation of around ±5% across all the modalities and cell densities. While no discernible systematic bias was detected for a given modality or seeding density, outliers are discussed below.

Fig. 3.

Cell counting results from three different T-cell seeding densities and the different imaging modalities. Each of these seeding densities includes two wells at the low (A1 and B1), medium (A2 and B2), and high (A3 and B3) cell densities defined in Section 4.1. Positive deviation indicates the segmentation method produced higher cell counts than manual counts, whereas a negative deviation indicates the segmentation method produced fewer cell counts than manual counts. The box plots on the right show the summarized absolute deviations from manual counts for each of the different imaging modalities. The notches for all three box plots overlap and thus the true medians do not differ with 95% confidence. Overall average deviations for each imaging modality: fluorescence 3.5 ± 1.9%, DPC 5.5 ± 6.1%, and FPM 3.0 ± 3.6%.

A qualitative comparison of the images showed that the FPM data can provide higher overall image quality than DPC, with FPM images having high contrast and clearly visible fine details not seen in the other modalities. We do note, however, that FPM image quality can vary spatially within a single FOV, with edge regions in particular showing poor resolution and background artifacts. Nevertheless, this did not significantly affect the counting accuracy for this cell type.

Upon inspection, the most common source of negative deviations from manual counts was poor separation of closely neighboring cells. DPC appears to be particularly problematic in this respect (see Figure S4A for a dramatic example). Segmentation of minute debris, some of which appears similar in size to the T-cells, can also cause positive deviations, and this was the cause of the outlying high FPM counts seen in well A1. The detection of faint objects that are visualized more clearly in FPM data also contributed to the additional counts.

6.4. U2OS cell counting

Figure 4 illustrates the segmentation results from the different imaging modalities applied to the same area in a high density U2OS sample (well A3). Optimization of analysis parameters in Columbus (notably, thresholded cell area > 100 µm²) enabled effective segmentation for both fluorescence and DPC images. For FPM, automatic volume threshold optimization was done prior to final segmentation. High-spatial frequency band-pass filtering was enabled for FPM artifact suppression. Boundary region segmentation was enabled, but note that boundary regions only serve to display outlines for cell cytoplasm. We did not find it necessary to employ any of the available metrics-based exclusion criteria for suppressing debris in these samples.

Fig. 4.

Segmentation of U2OS cells (at high density) across the three imaging modalities, SYTO 24 fluorescence, DPC, and FPM. Insets for the fluorescence and DPC images show zoomed-in regions with segmentation overlays from Columbus (cyan inner: nucleus; outer: cell). The inset for the FPM image shows a zoomed-in region with overlay from QPI segmentation. Dashed lines are used in the FPM output to show the boundary-region segmentation, while the Columbus output has these rendered as solid lines. All larger image scale bars represent 100 µm. All inset scale bars represent 20 µm.

Figure 5 summarizes U2OS cell counting results from the analyses performed in Columbus and the QPI segmentation method. All imaging methods (including automated fluorescence counting) had considerable trouble with the high cell density cases (wells A3 and B3), consistently underestimating cell counts. Evidently, none of the automated methods were able to discriminate closely packed U2OS cells as effectively as human counters. Examples of densely packed cells that posed some segmentation challenges for Columbus can be seen in both Fig. 4 and Supplement 1 ^{(3.8MB, pdf)} Figure S4B.

Fig. 5.

Cell counting results from three different U2OS cell seeding densities and the different imaging modalities. Each of these seeding densities includes two wells at the low (A1 and B1), medium (A2 and B2), and high (A3 and B3) cell densities defined in Section 4.2. Positive deviation indicates the segmentation method produced higher cell counts than manual counts, whereas a negative deviation indicates the segmentation method produced fewer cell counts than manual counts. The box plots on the right show the summarized absolute deviations from manual counts for each of the different imaging modalities. The notches for all three box plots overlap and thus the true medians do not differ with 95% confidence. Overall average deviations for each imaging modality: fluorescence 14.4 ± 13.8%, DPC 13.1 ± 10.0%, and FPM 12.4 ± 10.7%.

For the low and medium U2OS cell density cases, automated fluorescence counting was satisfactory and results from FPM and DPC were generally consistent with each other, though both demonstrated a tendency to over-count, by at most 20%. A qualitative comparison of FPM and DPC images did not show any systematic over-segmentation effects or debris, suggesting that a small fraction of cells with minimal fluorescence are present in the sample. Inspection did, however, reveal multiple examples where DPC failed to capture cells that were readily identified by FPM. These are illustrated in Supplement 1 ^{(3.8MB, pdf)} Figure S4B and C, and can be attributed, at least in part, to cell morphology/thickness. Though qualitatively observed, this discrepancy was not significant enough to manifest in cell count results.

6.5. iPSC-CM cell counting

Human induced pluripotent stem cell-derived cardiomyocytes (iPSC-CMs) show a large diversity of shapes and features. Segmentation results from the different imaging modalities for iPSC-CM cells seeded at medium density are shown in Fig. 6. When seeded at non-confluent densities, iPSC-CM cells can become especially flat. DPC imaging has significant difficulty in detecting the cytoplasmic regions of these cells (Figure S4D) and depending on the position or thickness of their nuclei, DPC can fail altogether to provide sufficient signal for cell counting.

Fig. 6.

Segmentation of iPSC-CMs, at medium seeding density, for the three imaging modalities, SYTO 24 fluorescence, DPC, and FPM. Insets for the fluorescence and DPC images show zoomed-in regions with segmentation overlays from Columbus (cyan inner: nucleus; outer: cell). The inset for the FPM image shows a zoomed-in region with overlay from QPI segmentation. All larger image scale bars represent 100 µm. All inset scale bars represent 20 µm.

For the fluorescence and DPC data, several of the available analysis building blocks and input parameters in Columbus were explored in order to obtain satisfactory segmentation. As with the U2OS cells, applying an area threshold (> 200 µm² in this case) enabled the removal of debris and also assisted in excluding some of the small, non-flat cells and debris (i.e., the smaller bright objects visible within the larger images shown in Fig. 6) often found ‘clinging’ to the more spread, adherent iPSC-CMs.

For the FPM data, automated v_th detection was not able to identify a clear discontinuity in the fraction of large to small cells with changing v_th (see Supplement 1 ^{(3.8MB, pdf)} Section 4), possibly due to the lack of a clear underlying distribution of a uniform cell type in this sample. Therefore, we relied on inspection of output segmentations by unbiased experts and found a threshold of 250 units to be most appropriate for this cell type.

Initial inspection of QPI segmentation results showed over-segmentation to be the primary problem. A contributor to this was the variation in background signal level across independent FPM reconstructions. Moreover, the segmentation technique is quite aggressive in identifying cell candidates which are only slightly brighter than the background. This results in a tendency to over-segment, regardless of the volume threshold used. The variation in the background level is particularly notable with the high cell density images, where background-level pixels constitute a smaller percentage of the pixels in the image.

We found it necessary to employ three of the available discrimination metrics to obtain satisfactory segmentation. Careful manual tuning of the limits based on observation of the segmentation output and metric histograms gave a qualitatively accurate segmentation. Any candidate falling within any discrimination criteria eliminated a candidate cell (an OR condition). Some candidates (such as large debris) often triggered more than one discrimination criteria. The discrimination metrics used are shown in Fig. 7 and included:

• 1.A simple filter based on the total integrated phase (phase volume) for each candidate. This eliminates rare instances of very large and very small outliers, which are generally insignificant particulate or large debris. Only a small fraction of candidates (< 0.3%), fall within this criteria, but it does serve to improve visual clarity.
• 2.An adaptive filter which removes cell candidates based on their average signal level relative to the background signal level (determined with Yen’s method [24]) for the given FPM reconstruction tile. Candidates where the average signal extended less than 20% above the local background level constituted 37% of candidate cells, and a visual analysis showed almost all of the candidates in this category to be the result of over-segmentation of background regions.
• 3.A filter which removed cell candidates in which the ratio of material identified in the nucleus to that in the bound cytoplasm region was unrealistic. This ratio is unity when no distinction between a cytoplasm and nucleus is possible, indicating debris rather than real cellular structure. Very low values of the ratio are also unrealistic, typically these are indicative of weak but detectable cytoplasm which causes the segmentation to extend the boundary region to a very large area. Candidates with either a low (< 0.1) or high (> 0.85) ratio constituted 6.7% and 6.0% of the sample, respectively.

Fig. 7.

Histograms of relevant cell metrics with exclusion brackets as applied in the QPI segmentation analysis of CM data.

Figure 8 summarizes the cell counting results from the analyses performed in Columbus and the QPI segmentation method. Overall, iPSC-CMs were a challenging case for all the modalities with relatively large counting deviations.

Fig. 8.

Cell counting results from three different iPSC-CM cell seeding densities and the different imaging modalities. Each of these seeding densities includes two wells at the low (A1 and B1), medium (A2 and B2), and high (A3 and B3) densities, as defined in Section 4.3. Positive deviation indicates the segmentation method produced higher cell counts than manual fluorescence-based counts, whereas a negative deviation indicates fewer cell counts than manual fluorescence-based counts. The box plots on the right show the summarized absolute deviations from manual counts for each of the different imaging modalities. The notches for all three box plots overlap and thus the true medians do not differ with 95% confidence. Overall average deviations for each imaging modality: fluorescence 20.8 ± 11.2%, DPC 31.7 ± 14.5%, and FPM 44.4 ± 54.5%. Excluding the outlying deviation from well B1, the average deviation for FPM improves to 23.9 ± 23.1%.

The DPC and fluorescence modalities generally performed quite similarly, especially with the low and medium density samples. At higher density, DPC analysis had a tendency to over-count by as much as 50%, where more of the small, rounded cells were present and detected in the Columbus segmentation. QPI segmentation showed a tendency to over-count in the low density cases. Detailed inspection of these cases, including the outlying well B1, revealed an abundance of cellular debris that is detected as cellular objects by FPM analysis; such debris is less prevalent with increasing density, and ultimately, over-counting is not observed in high density samples. Notably, the high density case with FPM yielded highly accurate cell counts, showing deviations at <10%. This is particularly encouraging for this cell type, which is commonly seeded at higher densities as a monolayer to promote more physiological function, i.e. propagation of electrical conduction and wave-like contractions, in cellular assays [25].

7. Conclusion

In this study, we have demonstrated label-free cell segmentation and counting in three different cell types, Jurkat T-cells, U2OS, and iPSC-CMs, using an FPM-based QPI system (SimulPhi-6) and a DPC imaging system (Opera Phenix). The same six samples for each of the three cell types were used across the imaging modalities and each cell type was seeded at three different densities. We compared each label-free imaging system along with its associated cell segmentation tools to provide automated cell counts. To accurately establish ground truth for cell counts, imaging of a fluorescent nuclear marker was also performed. The automated cell counts were compared to the gold-standard of expert manual cell counts based on the fluorescent imaging.

This study serves as a proof of concept (POC) demonstrating the utility and accuracy in applying FPM to count cells, and while only a handful of cell types were successfully counted here, the breadth of characteristics represented by these cell types suggests straightforward extensibility to a wide variety of commonly cultured cell types. Furthermore, our POC suggests that our results are not exclusive to the SimulPhi-6 system and that similar methodology could be applied broadly across other FPM implementations and to higher sample density plates (with hardware modifications) to enable higher throughput [12].

Experience with DPC image acquisition on the Opera Phenix has shown that, compared to other modalities, DPC image acquisition settings are particularly sensitive to cell thickness and overall cell density. Default settings can often be inappropriate for a cell type, leading to images as seen in Figure S4B-D, where cell interiors show too dimly to enable effective segmentation. This often requires additional setup and fine-tuning of settings during image acquisition, and thus requires re-imaging if initial settings are deemed inadequate based upon subsequent segmentation results. Furthermore, optimization of segmentation settings can help with poorly imaged cells, but this often comes at the expense of failing to capture other cells, and/or including unwanted noise or imaging artifacts.

A qualitative comparison of the FPM images and DPC images reveals many examples across all cell types in which FPM provides higher contrast and makes visible fine details and thin cells, which simply do not show up in DPC. Operationally, FPM does not require settings adjustments at the acquisition stage and the practical advantages to acquiring a single large image as opposed to stitching multiple images together are clear.

The successful application of FPM enables accurate counting of various cell types across a range of densities in a label-free manner. This obviates the need to sacrifice cells in order to accurately measure cell density (otherwise achieved using a hemocytometer or automated cell counter) as required for proliferation analysis and other downstream assays. The SimulPhi-6 system provides parallel imaging and a large FOV to enable rapid counting of a much greater fraction of the sample population relative to that sampled by a conventional hemocytometer or automated cell counter.

Quantitative comparison of our cell counts following segmentation showed comparable performance between FPM and DPC for T-cell counting. In U2OS samples, FPM provided slightly more accurate results over DPC, at 12.4% and 13.1% respectively, in terms of the absolute deviation from the reference counts. All of the imaging modalities struggled with the high density U2OS samples, including the automated segmentation of fluorescence in Columbus.

Automated cell count results for the most difficult case of iPSC-CM cells varied greatly relative to manual counts, with FPM counting an excess of cells in the low density samples. This FPM result is primarily a consequence of small cells present in this sample which did not show detectable fluorescence. With this cell type, however, FPM demonstrated examples of some of the most accurate cell counts and achieved this with the particularly challenging case of high density iPSC-CM seeding, yielding less than a 10% deviation from reference counts.

It is notable, however, that obtaining our results with FPM on the SimulPhi-6 system required enhancements to the associated QPI segmentation algorithm, specifically to mitigate the effects of known artifacts. While successful, this stands somewhat in contradiction to the purported advantages of FPM. The underlying quantitative nature of the data should make for easy segmentation with little or no post-processing steps, or subjective tuning of segmentation parameters, necessary. Automated optimization of the important volume threshold parameter used in quantitative phase segmentation can support an automated workflow without the possibility of bias introduced by the selection of segmentation parameters [19]. This was demonstrated here with the U2OS samples but was found impractical with the more complex iPSC-CM cells. Nevertheless, with the appropriate manual establishment of this parameter, high accuracy can readily be achieved.

Label-free imaging frequently presents challenges for accurate, automated, and user-friendly cell segmentation. The emergence of convolutional neural networks (CNNs) and deep learning approaches are now routinely providing adaptable solutions to this problem [26,27]. However, for the best results in applying CNNs to label-free cell segmentation, a very large, high-quality dataset is paramount [28]. Our FPM imaging provides an alternative approach, utilizing specialized hardware, to generate high-quality images amenable to traditional cell segmentation. Further, our QPI segmentation approach does not require an extensively large dataset nor does it require the time and effort spent in optimizing hyperparameters to establish a well-trained, cell-type specific CNN. By using a volume threshold instead of learning image features (textures, intensities, shapes, etc.), we are able to help ensure segmentation decisions avoid cell-type bias and thus anticipate that unique volumetric thresholds could even be applied to segment mixed cell populations.

Nevertheless, deep learning workflows could greatly complement our approach to FPM employed here to help improve image correction and analysis throughput. A SimulPhi-6 system could be devised to integrate deep learning to massively accelerate the steps and requirements for multi-well, parallel FPM imaging. Deep learning could assist in compensating for optical aberrations [29], offer a dramatic speedup in ptychographic phase image acquisition and reconstruction [30], and ultimately provide for more comprehensive cell analysis [31].

Disclosures

This work was partially conducted by employees of Clearbridge Biophotonics FPM Inc (Pasadena), the original manufacturer of the SimulPhi-6 imaging system. This company is no longer operating and the former employees listed as authors represent only themselves, as interested scientists. The ownership rights to the FPM patent estate from California Institute of Technology have been transferred to CellaVision AB.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Supplemental document

See Supplement 1 ^{(3.8MB, pdf)} for supporting content.