3DCellComposer – A versatile pipeline utilizing 2D cell segmentation methods for 3D cell segmentation

Abstract

Cell segmentation is crucial in bioimage informatics, as its accuracy directly impacts conclusions drawn from cellular analyses. While many approaches to 2D cell segmentation have been described, 3D cell segmentation has received much less attention. 3D segmentation faces significant challenges, including limited training data availability due to the difficulty of the task for human annotators, and inherent three-dimensional complexity. As a result, existing 3D cell segmentation methods often lack broad applicability across different imaging modalities. To address this, we developed a generalizable approach for using 2D cell segmentation methods to produce accurate 3D cell segmentations. We implemented this approach in 3DCellComposer, a versatile, open-source package that allows users to choose any existing 2D segmentation model appropriate for their tissue or cell type(s) without requiring any additional training. Importantly, we have enhanced our open source CellSegmentationEvaluator quality evaluation tool to support 3D images. It provides metrics that allow selection of the best approach for a given imaging source and modality, without the need for human annotations to assess performance. Using these metrics, we demonstrated that our approach produced high-quality 3D segmentations of multichannel tissue images. 3DCellComposer, when paired with well-trained 2D segmentation models, provides an important alternative to acquiring human-annotated 3D images for new sample types or imaging modalities and then training 3D segmentation models using them. It is expected to be of significant value for large scale projects such as the Human BioMolecular Atlas Program.

1. Background

The development of multiplexed imaging methods enables biomedical studies at the cellular level from numerous perspectives using various markers that target specific proteins [1–4]. The recent emergence of 3D multiplexed tissue imaging techniques further provides unprecedented potential for investigating the interaction, organization, and specialization of cells in 3D tissues [5,6]. However, realizing that potential also requires accurate 3D cell segmentation methods in order to ensure the quality of subsequent cellular analyses.

Within the challenging domain of 3D cell segmentation, tasks related to nuclear segmentation have received more attention. The main reason is that 3D nuclear segmentation is easier than 3D cell segmentation, primarily because of the well-defined round shape of nuclei and the high clarity provided by nuclear dyes. Over the years, traditional algorithms such as watershed [7], wavelet [8], gradient flow tracking [9], and a multi-model approach combining more than one traditional algorithm [10] have been employed for efficient 3D nuclei segmentation. Recently, deep learning-based methods have been employed to segment 3D nuclei more accurately with specifically tailored neural network architectures [11–14]. While 3D nuclei segmentation may be useful by itself in some applications, 3D cell segmentation is clearly needed in many others.

However, 3D cell segmentation comes with its own set of challenges. One major issue is that the shapes and boundaries of whole cells are often less well-defined than those of nuclei. This problem is particularly pronounced in tissue images, where the consistency and quality of cell membrane or cytoplasmic markers can vary significantly between different samples and imaging modalities. Various studies have explored approaches to tackle 3D cell segmentation, although not all focus on multiplexed imaging. Eschweiler et al. [15] explored the use of the spherical harmonics technique as an alternative approach to shape-constrain the star-convex segmentation prediction of 3D microscopy image data. Wolny et al. [16] introduced Plantseg, which combines a 3D U-Net with region adjacency graph partitioning to predict cell boundaries in 3D plant tissue images. Wang et al. [17] developed 3DCellSeg, which employs a lightweight U-Net-like network with a custom loss function, aiming to predict and de-clump 3D cells. The Allen Cell and Structure Segmenter (referred to below as ACSS) presented a human-in-the-loop deep learning workflow with expert correction of the segmentation results at each iteration [18,19]. One significant limitation common to these methods is their reliance on deep neural networks to handle the complexity of three-dimensional tissue data. Such approaches require large amounts of training data that might not always be available, especially when a new 3D imaging modality emerges. Even if a human-segmented dataset is created for a given 3D imaging modality with sufficient images for deep learning training, the variety and volume of such datasets would still be limited compared to the extensive libraries of 2D cell images. This limitation could reduce the applicability of the 3D cell segmentation model, meaning that it might perform well on the 3D image modality it was trained on but not on other modalities. Of course, training these models demands extensive manual labeling of 3D cells by experts [20], which inevitably suffer from intra and inter-expert variability [21,22]. Human-in-the-loop methods are designed to reduce the need of manual labeling, but still demand a considerable amount of expert input [18].

Meanwhile, 2D cell segmentation models, for both nuclear and cell segmentation, are often trained with datasets (e.g., TissueNet [20] and LIVECell [23]) containing millions of cells in 2D tissue and cell culture images across multiple imaging modalities. This allows for an alternative way to approach the 3D cell segmentation problem by using a 2D model to segment images and then assembling the 3D cells. One variation on this approach has been incorporated into Cellpose [24,25], which applies its 2D segmentation capability to individual slices along three axes and then averages the resulting spatial gradients to derive 3D cells.

To overcome these challenges, we have developed 3DCellComposer, a versatile tool capable of utilizing any 2D cell segmentation model for 3D cell segmentation tasks. Instead of training new 3D deep learning cell segmentation models using a limited number of 3D images, 3DCellComposer takes advantage of well-trained 2D models without requiring any further training. This allows users to customize their analysis pipelines by selecting a 2D segmentation model extensively trained from a similar tissue or imaging modality as their 3D data, or any other model that suits their needs. To further assist users in selecting the most effective 2D segmentation model for their 3D images, we have enhanced our earlier segmentation quality evaluation tool, CellSegmentationEvaluator [26] to support 3D segmentations. The combination of 3DCellComposer and CellSegmentationEvaluator provides a single solution for 3D cell segmentation needs, especially for multichannel images for which evaluation metrics are most informative. Fig. 1 illustrates the whole pipeline with specific steps involved in segmenting 3D cells using our software.

Fig. 1.

Overview of the 3DCellComposer pipeline for 3D cell segmentation. (A) A 3D multiplexed image serves as the input for segmentation. The image channels for the nucleus, cytoplasm, and cell membrane are shown in different colors. (B) A 2D cell segmentation method of user’s choice is applied to every slice along all three axes. Adjacent cells are colored differently for distinction. (C) 2D cell matching is conducted on each axis to generate 3D cell candidates. (D) The final 3D cell regions are obtained by performing linear indexing reconstruction and subsequent refinement.

We tested our pipeline using 6 distinct 2D cell segmentation methods, covering both traditional and deep learning-based techniques, and benchmarked their performance against previously-described 3D cell segmentation methods, including Cellpose 2D to 3D, ACSS, and 3DCellSeg (Supplementary Table S1). Using the enhanced CellSegmentationEvaluator, we computed a segmentation quality score for each method to objectively assess its performance without costly manual annotations for 3D cells. Our analyses indicate that 3DCellComposer with a well-trained 2D segmentation model can yield accurate segmentation of 3D cell boundaries across diverse tissues in 3D multichannel tissue images. In contrast, existing 3D segmentation methods often failed to segment cells or excessively merged cells from these images. Furthermore, 3DCellComposer showed performance competitive with ACSS on cell culture images used for ACSS training.

2. Methods

2.1. 3D multiplexed tissue and cell culture images

We obtained three 3D Imaging Mass Cytometry (IMC) [27,28] datasets from the Human BioMolecular Atlas Program (HuBMAP) database [29,30], corresponding to spleen, thymus, and lymph node tissues, respectively (see Supplementary Table S2 for details). These images were produced by the University of Florida Tissue Mapping Center.

For the segmentation process, we selected Iridium as the nuclear channel. The cytoplasmic channel was generated by summing the signals from three markers: Smooth Muscle Actin (SMA), Granzyme B, and Myeloperoxidase (MPO). For the cell membrane channel, we created a composite by summing the signals from five different markers: E-Cadherin/P-Cadherin, pan-Cytokeratin, CD31, CD45RA, and CD45RO (see Supplementary Table S3 for details). We chose to use multiple markers for the cytoplasmic and cell membrane channels because there are no universal markers that target these two cellular components across all cells provided in IMC data. By selecting and combining a diverse range of markers, each targeting one or multiple cell types, we aimed to cover as many cells as possible within the tissue for cell segmentation. Notably, Iridium serves as a universal marker for all nuclei in the tissue images, regardless of cell type. Therefore, even if a few cells are not captured by the combined channels used for the cytoplasm and cell membrane, they can still be segmented through the nuclear channel marked by Iridium.

From the Allen Institute’s WTC-11 hiPSC Single-Cell Image Dataset [19], we obtained “full-field” cell culture images containing multiple 3D cells, which had been used to derive all the single-cell images in this dataset. Each image has channels for cell membrane (labeled “CMDRP”) and nuclear (“H3342”) markers, respectively, as well as an additional channel for a tagged marker for one of 25 cellular structures. (They also contain channels labeled “SEG_Memb” and “SEG_DNA” that contain the output of their segmentation algorithm; it is important to note that although these are not human-provided segmentations they are of high quality and we have used them for evaluation). We selected five datasets that contain different cytoplasmic structure labels (see Supplementary Table S4 for details). From each of these, we randomly picked five images. We refer to these as the “AICS” image set.

2.2. Forming 3D cell candidates

The initial step in our pipeline is carried out (separately) for each of the three axes of the input 3D image. It consists of first applying a given 2D segmentation method to all 2D slices along a given axis, to produce 2D cell regions. The overlap between 2D cell regions in adjacent slices is then measured using the Jaccard Index (JI). Specifically, the algorithm sequentially considers each 2D cell region in the current slice (referred to below as the query 2D region) and identifies the 2D cell region in the next slice that has the highest JI (if there are any). If that JI is greater than a given threshold, that overlapping region is merged with the query 2D cell region to form a 3D cell candidate with a unique index (Algorithm 1). A merged overlapping 2D cell region cannot be selected again. This produces 3D cell candidates from each axis, each with a unique index.

It’s important to note that this JI threshold-based matching might lead to elongated stacks with identical index values along a corresponding axis. Even with large overlap, 2D segmented cells in adjacent slices might actually belong to different 3D neighboring cells. This occurs especially frequently in tissue images, where cells are densely packed across all three axes. We address this concern in the subsequent step.

Algorithm 1.

Forming 3D cell candidates

Require: $C_{2D}$: 3D array of stack of 2D slices containing 2D cell regions that are uniquely indexed within each slice
Require: JI_threshold: Jaccard Index threshold
1:	$C_{3D}$ ← empty 3D array for 3D cell candidates at Z-slice
2:	3D_candidate_index ← 1 {unique index of 3D cell candidates}
3:	{Initialization: Assign a unique 3D index to all 2D cell regions in the first slice}
4:	2D_region_indices ← unique($C_{2D}[1,:,:]$)
5:	for index in 2D_region_indices do
6:	2D_region_coords ← where($C_{2D}[1,:,:]==\mathit{\text{index}}$)
7:	$C_{3D}$[1,2D_region_coords] ← 3D_candidate_index
8:	3D_candidate_index ← 3D_candidate_index+1
9:	end for
10:	{Sequentially scan all slices to match 2D cell regions}
11:	for $i$ in 1 to length($C_{2D}$)−1 do
12:	{extract indices of 2D cell regions from current and next slice}
13:	current_slice_region_indices ← unique($C_{2D}[i,:,:]$)
14:	next slice region indices ← unique($C_{2D}[i+1,:,:]$)
15:	for $j$ in current slice region indices do
16:	{get index of best matching region in next slice if its JI is above the threshold, else return 0}
17:	$k$ ← findOverlap($j,C_{2D}[i,:,:],C_{2D}[i+1,:,:]$, JI_threshold)
18:	if $k>0$ then
19:	$C_{3D}[i+1,\text{where}(C_{2D}[i+1,:,:]==k\left)\right]\leftarrow j$ {merge in the selected 2D cell region in the next slice}
20:	next_slice_region_indices ← XOR(next_slice_region_indices, $k$)
21:	end if
22:	end for
23:	{start new 3D cell candidate for unmatched indices in slice $i+1$}
24:	for $l$ in next_slice_region_indices do
25:	$C_{3D}[i+1,\text{where}(C_{2D}[i+1,:,:]==l\left)\right]$ ← 3D_candidate_index {starting new 3D cell candidate}
26:	3D_candidate_index ← 3D_candidate_index+1
27:	end for
28:	end for

2.3. Matching 3D cell candidates from 3 axes

The next step produces a unified 3D segmentation by integrating the 3D cell candidates from all three axes. We assume that, while creating 3D cell candidates from segmentations along a single axis may be expected to yield errors, combining segmentations from all three axes will help to rectify them.

Algorithm 2.

Matching 3D cell candidates from 3 axes

Require: $X_{\mathit{\text{max}}}$: Number of voxels along the X-axis
Require: $Y_{\mathit{\text{max}}}$: Number of voxels along the Y-axis
Require: $Z_{\mathit{\text{max}}}$: Number of voxels along the Z-axis
Require: $C_{3D\_X}$: 3D array of 3D cell candidates of X-axis from Algorithm 1
Require: $C_{3D_{-}Y}$: 3D array of 3D cell candidates of Y-axis from Algorithm 1
Require: $C_{3D}$: 3D array of 3D cell candidates of Z-axis from Algorithm 1
1:	$C_{3D\_\mathit{\text{fragments}}}$ ← empty array with dimensions $\left[Z_{\mathit{\text{max}}},X_{\mathit{\text{max}}},Y_{\mathit{\text{max}}}\right]$ {Initialize 3D array for 3D cell fragments}
2:	for $z$ from 1 to $Z_{\mathit{\text{max}}}$ do
3:	for $x$ from 1 to $X_{\mathit{\text{max}}}$ do
4:	for $y$ from 1 to $Y_{\mathit{\text{max}}}$ do
5:	$C_{3D\_\mathit{\text{fragments}}}[z,x,y]\leftarrow C_{3D_{-}Y}[z,x,y]+C_{3D_{-}x}[z,x,y]\cdot Y_{\mathit{\text{max}}}+C_{3D}[z,x,y]\cdot X_{\mathit{\text{max}}}\cdot Y_{\mathit{\text{max}}}$ {Match 3D cell candidates from 3 axes to 3D cell fragment, each fragment has its unique index}
6:	end for
7:	end for
8:	end for
9:	{Exclude biologically implausible 3D cell fragments}
10:	fragment_indices ← unique($C_{3D\_\mathit{\text{fragments}}}$)
11:	for index in fragment_indices do
12:	fragment_coords ← where($C_{3D\_\mathit{\text{fragments}}}==\mathit{\text{index}}$)
13:	if length(fragment_coords[0]) < = 3 then
14:	{length along Z-axis < = 3}
15:	$C_{3D\_\mathit{\text{fragments}}}$ [fragment_coords] ←0 {Remove this fragment from $C_{3D\_\mathit{\text{fragments}}}$}
16:	end if
17:	end for

This approach combines the 3D cell candidates derived from the three axes by assigning voxels that have the same tuple of indices in all three axes to a 3D cell fragment (Algorithm 2). To be conservative, we excluded 3D cell fragments with a z-axis thickness of three slices or fewer (e.g., $\le 6\mu \text{m}$ in 3D IMC), since such fragments are unlikely to represent complete cells. This is because 3 slices would only allow for the nucleus to be one slice thick since it is not allowed to protrude through the cell membrane. (Note however that we have added a command line option to allow users to control the minimum thickness.).

2.4. Refining 3D cell fragments with 3D cell candidates from Z-axis

The 3D cell fragments from the three axes rarely agree perfectly and will almost always be smaller than the actual cell they correspond to. This is because each 3D cell fragment consists only of those voxels that match in all three segmentations, which while they typically agree for large regions of each actual cell, may disagree along edges. We noted that most imaging techniques capture 2D images along the Z axes, in which case the highest spatial resolution is in the X-Y planes. When this is the case, we have a basis for resolving differences between the 3D cell candidates from the three axes by giving preference to the segmentations resulting from Z-axis slicing.

The linear indexing approach allows for straightforward tracing of any voxel in the final 3D segmentation mask back to the index of the 3D cell candidate on each axis to which it belongs. This significantly facilitates the refinement of the final 3D segmentation by using the 2D segmentations from the Z-axis. We initiated the process (Algorithm 3) by selecting matched 3D cells in descending order of volume, as these represent regions where the three axes align best. For each, we identified its corresponding 3D cell candidate from the Z-axis. We replaced each 2D cell boundary on the Z-axis of a given matched cell with the 2D cell boundary from the same Z-slice in the corresponding 3D cell candidate. This produced final 3D cell regions.

Algorithm 3.

Refining 3D cells with 3D cell candidates from Z-axis

Require: $C_{3D\_\mathit{\text{fragments}}}$: 3D array of 3D cell fragments from Algorithm 2
Require: $C_{3D}$: 3D array of 3D cell candidates of Z-axis from Algorithm 1
1:	$C_{3D\_\mathit{\text{fragments}}}$ ← empty 3D array with the same dimensions as $C_{3D\_\mathit{\text{fragments}}}${Initialize array for final 3D cell regions}
2:	fragment_sorted_indices ⟵ sorted list of unique indices of 3D cell fragments in $C_{3D\_\mathit{\text{fragments}}}$ by descending volume
3:	refinement_flag ⟵ False 3D boolean array with the same dimensions as $C_{3D\_\mathit{\text{fragments}}}${False indicates the corresponding voxel has not been refined}
4:	for index in fragment_sorted_indices do
5:	fragment_coords ← where($C_{3D\_\mathit{\text{fragments}}}==\mathit{\text{index}}$)
6:	if all(refinement_flag[fragment_coords] = = False) then
7:	candidate_index ← $C_{3D}$ [fragment_coords][0] {get index in Z-axis segmentations for a voxel in current cell fragment}
8:	candidate_coords ← where($C_{3D}==\mathit{\text{candidate}}\_\mathit{\text{index}}$) {get 3D coordinates of corresponding 3D cell candidate from Z-axis}
9:	$z_{\mathit{\text{min}}}$ ← min(fragment_coords[0]) {first column is z}
10:	$z_{\mathit{\text{max}}}$ ← max(fragment_coords[0])
11:	filtered coords ← filterZ(candidate_coords, $z_{min}$, $z_{max}$) {Remove coordinates outside Z slices of current cell fragment}
12:	$C_{3D\_\mathit{\text{final}}}$[filtered coords] ← index {Refine $C_{3D\_\mathit{\text{final}}}$}
13:	refinement_flag[filtered coords] ⟵ True {Mark these voxels as refined}
14:	end if
15:	end for

This refinement process had the effect of using the X and Y axis segmentations primarily to decide where to separate cells along the Z axis. That is, once the choice of which Z slices should be included in a given cell, the boundaries of those slices are chosen using the segmentation that is of the highest resolution (XY). This is illustrated in Supplementary Fig. S1, which shows the intersection between the segmentations from the three axes (the 3D cell fragment) and how this determines the Z-axis extent of the final 3D cell region. By employing this approach, we effectively utilized the segmentation capabilities of each axis and exploited the accuracy of segmentation from the axis with the highest resolution. Of course if the image resolution in all directions is the same, this is not needed.

2.5. Inferring 3D segmentation of cell nuclei

For many purposes, 3D cell segmentations are most useful if they include nuclear segmentations. To achieve this, we matched every final 3D cell region from the last step with its corresponding nuclear segmentation mask from the 2D nuclear segmentation along the Z-axis to produce paired 3D cell and nuclear segmentations (Algorithm 4). Specifically, we first scanned each 2D slice of every final 3D cell region through the Z-axis and identified its overlapping 2D nuclei from the nuclear mask. During this process, we found that for certain segmentation methods, the nucleus often protruded outside the cell or lay on the cell boundaries. This typically occurred when the methods used separate models for cell and nuclear segmentation. In such case, we trimmed the nucleus to fit within the cell boundaries to maintain the biological plausibility of the cell structure. We also observed that a cell might overlap with multiple nuclei. When this happened, typically there is a dominant nucleus with minimal pixels extending beyond or on the cell boundaries. In contrast, other overlapping nuclei are actually from neighboring cells and thus have a substantial portion outside this cell. Thus, we retained the nucleus with the smallest fraction of protrusion beyond and on the cell boundaries.

For each 3D cell, the 2D nuclei from multiple Z-axis slices within that cell was assigned the same corresponding cell index, forming a final 3D nuclear region. When a 3D cell had no nucleus inside, that cell was removed from the final 3D cell segmentation.

Algorithm 4.

Inferring 3D segmentation of cell nuclei

Require: $C_{3D\_\mathit{\text{final}}}$: 3D array of final 3D cell regions with unique indices from Algorithm 3
Require: $N_{2D}$: 3D array of 2D nuclear regions from the Z-axis, each uniquely indexed at its Z-slice
1:	$N_{3D\_\mathit{\text{final}}}$ ← empty 3D array for final 3D nuclear regions
2:	final_indices ← list of unique indices of 3D cell regions in $C_{3D\_\mathit{\text{final}}}$
3:	for index in final_indices do
4:	{Iterate over each final 3D cell region}
5:	for $z$ in unique(where($C_{3D\_\mathit{\text{final}}}[z,:,:]==\mathit{\text{index}}$)) do
6:	{Iterate over the Z-slices where the current 3D cell region exists}
7:	2D_region_coords ← where($C_{3D\_\mathit{\text{final}}}[z,:,:]==\mathit{\text{index}}$)
8:	$C_{2D}$⟵empty 2D array with the same XY dimensions as $C_{3D\_\mathit{\text{final}}}${Initialize array for individual final 3D cell region at slice z }
9:	$N_{2D}$⟵ empty 2D array with the same XY dimensions as $C_{3D\_\mathit{\text{final}}}${Initialize array for individual final 3D nuclear region at slice z}
10:	$C_{2D}$ [2D_region_coords] ← index {2D array with index value at the coordinates of current final 3D cell region at slice $z$}
11:	$N_{2D}$[2D_region_coords] ← $N_{2D}$[$z$, 2D_region_coords] {2D array with indexed 2D nuclear regions overlapping with $C_{2D}$}
12:	if num(unique($N_{2D}$))> 1 then
13:	{If more than one nucleus is present}
14:	$N_{2D}$ ⟵selectNucleus($N_{2D},C_{2D}$)
15:	{Keep the nucleus with the smallest fraction beyond and on cell boundaries}
16:	end if
17:	$N_{2D}$⟵trimNucleus($N_{2D},C_{2D}$)
18:	{Remove nuclear pixels outside or on cell boundaries}
19:	$N_{3D\_\mathit{\text{final}}}[z,\text{where(}{N}_{2D}\ne 0)]$ ← index
20:	{Store the trimmed 2D nuclear region}
21:	end for
22:	end for
23:	for index in final_indices do
24:	3D_region_coords ⟵ where ($C_{3D\_\mathit{\text{final}}}==\mathit{\text{index}}$)
25:	{Get the 3D coordinates of the current final 3D cell region}
26:	num_overlap ⟵ findNucleus(3D_region_coords, $N_{3D\_\mathit{\text{final}}}$)
27:	{Find the number of overlapping 3D nuclear regions}
28:	if num overlap = = 0 then
29:	$C_{3D\_\mathit{\text{final}}}$[3D_region_coords] ← 0
30:	{Remove final 3D cell regions that do not have corresponding 3D nuclear region}
31:	end if
32:	end for

If a given cell segmentation method does not also provide nuclear segmentation, we used the 2D nuclear segmentation from DeepCell as a replacement. This avoided penalizing a potentially good cell segmentation method that does not also produce nuclear segmentations.

For simplicity, we refer to the final 3D cell regions as 3D cells in the following.

2.6. Visualizing 3D cell segmentation results

We integrated Blender software with 3DCellComposer to visualize 3D cell segmentation masks. For each 3D cell, we used the marching cubes algorithm from the Scikit-learn package of Python to generate a 2D mesh on its surface. This mesh is composed of multiple faces and vertices, which represent the shape and points of the mesh, respectively. We grouped the faces of meshes corresponding to the same 3D cell, allowing Blender to identify them as part of the same structure.

We observed that if the area of a 3D cell in the first z-slice is larger than that in the second z-slice (or if the area in the last z-slice is larger than in the second-to-last z-slice), the marching cube algorithm forms a meshed surface inside the cell rather than outside. This introduces a hole in the 3D cell visualization. To ensure a closed surface for every 3D cell, we added a 2D mesh to the first and last z-slices of any 3D cell exhibiting this issue.

We assigned colors to 3D cells with the requirement that neighboring cells not have the same color. Unlike in 2D, four colors are insufficient for this purpose in 3D. The software therefore iteratively uses increasingly larger sets of colors until the neighboring color requirement is met.

2.7. Evaluating 3D cell segmentation results

Beyond the qualitative assessment through visualization, we aimed to quantitatively evaluate the 3D cell segmentation results. The major challenge in doing this is determining the criteria to be used. Traditionally, this is done by comparing to cell segmentations created by human labeling. However, human labeling exhibits variability across different annotators and even from the same annotator over time [21,22]. Consequently, this can result in annotations that are inferior to what computational algorithms achieve, as demonstrated in our previous work [26]. In that work, we have proposed an alternative to human annotation: measuring various statistical criteria that are expected of good segmentations. Our CellSegmentationEvaluator [26] tool employs a set of innovatively designed metrics based on a series of assumptions about the desirable characteristics of cell segmentation. The most critical one, which is enabled by multichannel imaging, is that properly segmented cells of the same cell type should have similar compositions over available markers. Critically, this can avoid the need for time-consuming and error-prone human labeling.

For 3D images, the drawbacks of human labeling are expected to be even greater than for 2D, due to the greater time required and the expected lower accuracy. We therefore expanded CellSegmentationEvaluator to support evaluation of cell segmentation in 3D (see the Supplementary Information for details of metrics and Supplementary Table S5 for a summary). This tool is especially valuable for 3D datasets from emerging methods like 3D IMC, where human segmentations may not be readily available.

While one can assess various aspects of cell segmentation using the individual metrics, we desired a single overall segmentation quality score. We took an empirical approach to defining this. In addition to having results from multiple segmentation methods, we created perturbed images that would be expected to reduce the quality of segmentation. We then z-scored all metrics from all perturbed and unperturbed IMC images and constructed a Principal Component Analysis (PCA) [31–33] model for evaluating multiplexed tissue images. Similarly, we used the z-scored metrics from all hiPSC images to create a PCA model for cultured cell images (this is to take into account the different expectations about the density and arrangement of the cells in the two cases).

PCA begins by computing the covariance matrix $S=\frac{1}{n-1}{X}^{\top }X$, from the z-scored metrics matrix $\mathbf{X}$, where $n$ is the number of rows in $\mathbf{X}$, each row represents a segmentation from one method on one image, and each column a metric. Eigenvalues $\lambda$ and eigenvectors $\mathbf{V}$ of $\mathbf{S}$ were then calculated by Singular Value Decomposition (SVD), with the first two eigenvectors ${\mathbf{v}}_1$ and ${\mathbf{v}}_2$ (first two columns of $\mathbf{V}$) forming the axes of the first two principal components of the PCA. For each row $\mathbf{x}$ in $\mathbf{X}$, which is a vector of the metrics for a given segmentation method on one image, its first two principal components were computed as ${\text{PC}}_1={\mathbf{x}\mathbf{v}}_1$ and ${\text{PC}}_2={\mathbf{x}\mathbf{v}}_2$.

The final quality score [26] of this mask was then calculated as the weighted sum of ${\text{PC}}_1$ and ${\text{PC}}_2$:

$$\text{Quality}\text{score}=\text{exp}\left({\text{e}}_1\cdot {\text{PC}}_1+{\text{e}}_2\cdot {\text{PC}}_2\right)$$

where $e_1$ and $e_2$ are the proportions of variance explained by first two principal components, respectively, calculated as:

$${\text{e}}_1=\frac{{\lambda }_1}{\sum _{\text{i}=1}^{\text{p}}{\lambda }_{\text{i}}},{\text{e}}_2=\frac{{\lambda }_2}{\sum _{\text{i}=1}^{\text{p}}{\lambda }_{\text{i}}}$$

Here, ${\lambda }_1$ and ${\lambda }_2$ represent the eigenvalues associated with first two principal components and p is the number of metrics. The denominator sum represents the total variance of $\mathbf{X}$. ${\text{e}}_1$ and ${\text{e}}_2$ emphasize how each component contributes to capturing the variation in the segmentation quality metrics. By applying an exponential function to this summed value to obtain the final quality score, we ensured that the final quality score has a baseline of zero, providing a clear reference for users.

For evaluating segmentations on AICS images, we compared them to the ACSS segmentations provided with the images. We first matched each ACSS cell to the predicted cell with the largest overlap (each predicted cell was only matched once), calculated the Dice coefficient for each cell, and then calculated the average Dice coefficient for each image.

3. Results

3.1. 3DCellComposer enables cell segmentation on multiplexed 3D tissue images

We first evaluated the performance of 3DCellComposer on three Imaging Mass Cytometry (IMC) images (Supplementary Table S2) [27,28] available from the HuBMAP [30] portal (https://portal.hubmapconsortium.org/). These were of spleen, thymus, and lymph node, ensuring coverage across different types of organs to test the generalizability of our pipeline on 3D tissue images. Notably, these images have voxels that are $1\mu \text{m}$ on the X and Y axes and $2\mu \text{m}$ on the Z axis. This anisotropy, along with the densely packed cells in the 3D IMC tissue images, introduces challenges for accurate 3D cell segmentation.

We implemented 6 2D segmentation methods (including DeepCell using either a cell membrane or cytoplasmic channel as input), and used each of these methods to segment 2D slices from the X, Y, and Z axes of each 3D IMC image (the 2D methods were configured with appropriate free parameters, see Supplementary Table S6). These segmentation outputs were then processed through our 3DCellComposer pipeline to generate 3D cell segmentations. For comparison, we also used the Cellpose 2D to 3D version, and two direct 3D segmentation methods, ACSS and 3DCellSeg.

Example 3D segmentations for a region of one IMC image are shown in Fig. 2. This region contains densely packed lymph node cells. We observed that 3DCellComposer with DeepCell segmented the 3D cells in this area with high accuracy and coverage. In contrast, 3DCellComposer with CellProfiler segmented 3D cells with rough boundaries and missed many cells. Meanwhile, Cellpose 2D to 3D segmented only a few 3D cells, and 3DCellSeg tended to excessively merge neighboring cells.

Fig. 2.

Comparison of 3D cell segmentation using different methods on the Spleen IMC image. Panels A to D display the segmentation outcomes obtained from each respective method. Visualization was carried out using Blender software, utilizing mesh output from 3DCellComposer. To ensure visual distinction, adjacent 3D cells are colored differently.

Examination of the number of cells produced by each method at each step of the process (Supplementary Table S7) reveals that a major reason for the poor performance of all methods except DeepCell is the reduced number of cells detected in 2D slices, especially in the slices along the Z axis. This contributes to the low numbers of 3D cell candidates constructed, since missing cells in one slice inhibits joining of cells in adjacent slices.

From visual examination of the static projection of the results using DeepCell in Fig. 2, the segmentations in x and y appear reasonable but it is difficult to assess how well cells are being segmented in the z dimension. Supplementary Video 1 therefore allows visualization of the connectivity of each cell across different z slices. Supplementary Fig. S2 shows that the resulting cells typically span 4 slices (which is the minimum size specified in Algorithm 2) and have a reasonable cell volumes of approximately 600 microns³. As noted in section 2.3, the requirement for a minimum of 4 slices is conservative and can be controlled through a command line option.

While these assessments give a preliminary idea of the characteristics of each method, we proceeded to perform a quantitative evaluation using a 3D extension of our previously described CellSegmentationEvaluator tool [26] (see Methods).

To calibrate our cell segmentation metrics and assess the robustness of segmentation methods against image perturbation, we created three modified image sets from the original IMC images. For these sets, Gaussian noise with a mean of zero was added to each voxel’s intensity. The standard deviation of the noise varied across the sets, with values of 5, 10, and 15 voxel intensity units, respectively (a standard deviation of 15 corresponds to approximately 5 % of the maximum voxel intensity of the original IMC images). (Examples of these images are shown in Supplementary Figs. S4 and S5). In principle the perturbed images should yield poor segmentations and allow identification of how the metrics responded. To identify key modes of variation among these metrics, we trained a Principal Component Analysis (PCA) model on metrics resulting from all methods applied to all original and perturbed 3D IMC images. While the exact model would depend to some degree on the particular images used for training, the perturbation scheme ensures that the model captures variation across a range of quality. Note that this means that comparison of different methods on the same image should be reliable. Of course, a new PCA model can always be trained on new and perturbed images if desired.

The contributions of each metric to the first two principal components are shown in Supplementary Fig. S5. Consistent with our previous study, the first principal component axis reflects the overall segmentation quality, combining all of the metrics except the average cell size but primarily reflecting the measures of marker consistency (which reflect how well the segmented cell boundaries align with the actual cell boundaries). The second principal component indicates the overall coverage of segmentation, primarily reflecting the number of cells and fraction of image covered by cells. A well-performing method should score high on both PC coordinates, as it is expected to perform well across all metrics.

Fig. 3A shows the average of PC1 and PC2 for each method on the original IMC images and on the artificially perturbed images. We found that 3DCellComposer, using the DeepCell 2D segmentation model with either cell membrane or cytoplasmic input, achieved the best performance in segmenting 3D cells from IMC tissue images, yielding high values in both quality (PC1) and coverage (PC2), along with high robustness under Gaussian noise perturbation. The performance was marginally better using the cell membrane input. This is consistent with the conclusion from the previous 2D segmentation study, which found that DeepCell performs better using a cell membrane marker than a cytoplasmic marker [26].

Fig. 3.

Evaluation of 3D cell segmentation quality on 3D multiplexed tissue images without manual annotations. (A) The top two principal components (PCs) of the segmentation quality metrics for 3DCellComposer, based on different 2D segmentation methods across three organs. The percentages next to PC denote the variance that each corresponding PC accounts for. These results are compared against Cellpose 2D to 3D and two direct 3D methods ACSS, and 3DCellSeg. Each method is represented by a unique color and distinct marker, while filled markers correspond to original images. Hollow marker trajectories illustrate varying levels of Gaussian noise perturbation, arranged in ascending order from low to high. The suffixes “mem” and “cyto” after DeepCell denote the cell membrane and cytoplasmic input channels, respectively. (B) A ranking of 3D segmentation methods by overall segmentation quality score. The green bars are 2D methods through 3DCellComposer whereas blue ones are other methods.

We also found that 3DCellComposer combined with Cellpose using default settings did not produce acceptable 3D segmentations, with both PC1 and PC2 being low. This may be due to the fact that, as indicated by our previous study, the version of Cellpose we tested did not perform well on 2D IMC images [26]. Better performance could potentially be obtained by tuning Cellpose parameters to match a particular dataset.

3DCellComposer displayed varying segmentation capabilities when combined with traditional 2D cell segmentation methods. 3DCellComposer paired with CellProfiler not only demonstrated good segmentation quality (high PC1) and coverage (high PC2) but also performed well under noise perturbation. However, the ACSS classic version and CellX, while acceptable in segmentation quality (high PC1) and robustness, did not achieve good segmentation coverage (low PC2), reflecting that there are many cells they failed to segment. These series of results are also consistent with the performance of these traditional 2D methods on 2D IMC images [26].

As for two direct 3D cell segmentation methods, 3DCellSeg obtained acceptable coverage (high PC2) but low segmentation quality (low PC1). This is consistent with the example in Fig. 2 in which the large cells it produces provide high coverage but do not accurately reflect the real cell boundaries. ACSS failed to segment any cells across 3 tissue images. The performance of these two 3D cell segmentation models is consistent with our premise that models trained directly on 3D cell segmentation data might have limited generalizability.

To facilitate easy and straightforward comparisons of 3D cell segmentation quality, we calculated the final quality score (see Methods for calculation details) for various methods for 3D IMC images (Fig. 3B). The rankings are consistent with the conclusions drawn from Fig. 3A. (The numerical values for PC1, PC2 and quality score for the plots in Fig. 3 are shown in Supplementary Table S8).

3.2. 3DCellComposer optimizes segmentation criterion for different 3D images

The most important part of the 3DCellComposer algorithm is determining how to combine the cells found in the 3 different 2D views. This is controlled by a free parameter, a Jaccard Index (JI) threshold, that determines whether two 2D cells in adjacent slices along the X, Y, or Z axis are considered to belong to the same 3D cell. The principle is that if segmented cells in adjacent slices have good overlap, they are likely to be part of the same cell (see Methods for detailed description of this process). 3DCellComposer automatically chooses this threshold for a given method and image to optimize the 3D cell segmentation performance (based on the metrics). Below we describe the process and its effectiveness in detail.

For illustration, we selected a range of Jaccard Index (JI) threshold values (from 0.0 to 0.7 with 0.1 intervals) for whether to match 2D cells in adjacent slices, spanning from very loose to very strict criteria. We applied each threshold with each of the 2D segmentation models to example IMC images (overriding the automatic threshold selection).

We then trained a PCA model using our metrics from all segmentation results from 2D segmentation models across the range of JI settings, along with segmentations from Cellpose 2D to 3D and the direct 3D segmentation methods. This comprehensive model was then used to measure segmentation quality as a function of JI.

Supplementary Fig. S6 shows results (using DeepCell with cell membrane input) for three measures for three 3D IMC images: the total number of pairs of matched 2D cells in adjacent slices across all 3 axes, the total number of 3D cells produced, and the segmentation quality score. As noted in the caption, the final number of 3D cells is in rough agreement with expectations given the image dimensions and the typical cell size variation.

We first observed that the quality score increases monotonically as the JI threshold decreases until at least 0.1. In other words, a looser criterion for potentially matching 2D cells between adjacent slices generally leads to higher 3D segmentation quality. This trend may be due to the relatively large voxel size (1×1×2 microns) of these 3D IMC images. Compared to other multiplexed imaging modalities like CODEX [34], which typically have three times higher resolution, this relatively low resolution along the Z-axis leads to more considerable shifts in cell shape between 2D cells belonging to the same 3D cell in adjacent slices (and thus requires a looser criterion to match them). This is supported by the distributions of the number of pairs of matched 2D cells and of matched 3D cells, which both rise with decreasing JI criterion. The increased number of 3D cells results in higher segmentation quality scores.

It is important to note that this optimized low JI threshold setting for DeepCell/3DCellComposer on 3D IMC images need not be universally applicable to other 2D segmentation methods or other imaging modalities.

Interestingly, for two example images the JI threshold yielding the highest quality score was at 0.1. Our choice to incorporate automatic determination of JI threshold for each image and segmentation method accounts for variation from image to image and tissue to tissue.

3.3. 3DCellComposer segmentation of 3D cell culture images

Given our success with IMC images, we sought to evaluate it for other types of 3D images. Our primary goal in developing 3DCellComposer was providing segmentation capability for 3D multichannel tissue images, and it draws upon the multiple channels to evaluate and optimize results. However, the public availability of 3D tissue images is in general limited, and we were unable to find additional public 3D datasets with more than 2 or 3 channels. Thus, to get at least an idea of the generalizability of 3DCellComposer, we applied it to 3D images of cell cultures from the Allen Institute for Cell Science. As described in Methods, we used 25 3D hiPSC cell culture images, five images each for cell lines tagged with EGFP for alpha-actinin, microtubules, Golgi apparatus, lysosomes, and mitochondria. All 25 images also include a nucleus marker channel (Hoechst 33342) and a cell membrane marker channel (CMDRP).

We tested 3DCellComposer in combination with DeepCell and Cellpose 2D cell segmentation models, and compared with the results from the ACSS 3D segmentation method. DeepCell was run using the Hoechst and CMDRP markers as input and Cellpose was run using the Hoechst and EGFP channels. Given that ACSS was specifically trained on a fraction of these 3D hiPSC cell culture images with a human-in-the-loop approach, it represents a high bar for performance.

Since the ACSS images had only a few channels, and since the cultured cells were all expected to be similar, the segmentation evaluation metrics we developed above are not applicable; they rely in large part on detecting consistency of marker composition between different cell types. We therefore calculated the average Dice coefficient between the ACSS segmentations and the 3DCellComposer segmentations. The Dice coefficient ranges from 0 to 1 and is calculated as twice the area of intersection between two segmentations over the sum of the areas of the two segmentations. Fig. 4 shows the average Dice coefficient versus the difference in the number of cells between the two segmentations. As discussed above, 3DCellComposer normally uses the segmentation score to choose an optimal JI threshold for each image, which was not possible for these images. therefore the figure shows curves for a range of JI values. In contrast to 3D IMC images, 2D cells in adjacent slices in AICS images are expected to have significant overlap (due to the smaller spacing between slices). A JI threshold of 0 merges all slices that overlap, which is not unreasonable given that the cells are in a monolayer culture. However, some 2D cells (e.g., adjacent cells whose edges overlap) may be inappropriately joined. As the JI threshold increases from 0, some cells in adjacent slices are not matched, resulting in 3D cells being split and a higher total 3D cell count but also a higher average Dice coefficient. As the JI threshold continues to increase, however, fewer and fewer 2D cells are matched and some of the resulting 3D cells are removed as not having the required minimum number of slices. This results in a lower total count of final 3D cells and a lower Dice coefficient.

Fig. 4.

Comparison of cell segmentations for AICS image. For 3DcellComposer results, lines connect results for increasing Jaccard Index thresholds from 0 (filled symbol) to 0.9. Note the dramatically reduced performance of ACSS with even modest noise, compared to the reasonable performance of 3DcellComposer with DeepCell even for the highest noise level. 3DcellComposer with CellPose failed to produce a segmentation for any of the noise levels.

The Dice coefficients for moderate JI values indicate reasonable overlap between the DeepCell segmentations and the provided ACSS segmentations, especially when considering that ACSS was specifically trained on these images. However, the results clearly indicate that the 2D DeepCell segmentations are, at least in some slices, oversegmenting cells. To examine this, we directly compared the 3DCellComposer segmentations with those from ACSS for an example alpha-actinin image. As shown in Supplementary Fig. S7, the segmentations are similar for a lower slice (30) but 3DCellComposer with DeepCell produces oversegmented cells in a higher slice (18), presumably because of lower membrane signal and smaller nuclear size in the higher slice. This accounts in part for the lower performance on the Dice coefficient. Note that this issue is specific to cultured monolayers and is not expected for 3D tissue images.

Fig. 4 also shows results for 3DCellComposer paired with Cellpose using each of the EGFP markers. It produced reasonable segmentation for the alpha-actinin, but the Dice coefficients were lower than those from DeepCell. The difference with the results for DeepCell is expected since membrane markers typically provide better cell segmentation. In addition, the results highlight the fact that, depending on the input channel for segmentation, a method might work well for one marker but not for another, even if they are from the same cell type and imaging modality. This highlights the need for flexibility in 3D cell segmentation, which 3DCellComposer provides by being able to select from multiple pre-trained 2D segmentation models.

As also shown in Fig. 4, our analysis indicates that 3DCellComposer paired with DeepCell was more resilient than ACSS and Cellpose when processing perturbed images affected by Gaussian noise. This suggests that 3DCellComposer can inherit robustness from 2D cell segmentation models, such as DeepCell in this case. In contrast, the sensitivity of ACSS to noise and its failure on IMC images (Fig. 3) suggest that 3D deep learning methods that are trained with images with high signal-to-noise may be less able to generalize to other images.

4. Discussion

As advances in 3D high-content imaging techniques continue to progress, the need for effective 3D cell segmentation has become increasingly important. We have demonstrated here that 3DCellComposer can leverage any well-trained 2D cell segmentation model and perform 3D cell segmentation tasks across two different imaging modalities and voxel resolutions. Our pipeline succeeded in segmenting 3D cells in IMC multiplexed tissue images from three different organs, a task that other 3D cell segmentation methods failed to accomplish. Integration with Blender allows quick visual assessment of the quality of segmentation (Fig. 2). We also enhanced our segmentation evaluation tool, CellSegmentationEvaluator, to support 3D segmentations and integrated it into 3DCellComposer. This allows for a direct comparison of 3D segmentation results for multichannel tissue images without the need for manual annotation (Fig. 3). Our pipeline applied to 3D cultured cell images did not perform as well as a segmenter trained on those images, but it gave reasonable results that were less sensitive to added noise than the trained ACSS segmenter.

It is worth noting that because of the number and complexity of methods we have examined that we have tested them using the provided models with default parameters and have not explored different parameter settings. Thus some methods may perform better after optimizing parameters and have provided an example of how to used different parameters. The evaluation metrics we have developed should be useful for optimizing different parameters.

Our pipeline is designed to resolve the existing gap between the rise of 3D multiplexed tissue imaging and the current shortage of accurate 3D cell segmentation models. 3D multiplexed images are more challenging to prepare and costlier to produce [6,35]. This limits the number of 3D images produced for a given 3D modality. However, accurate cell segmentation methods, especially those based on deep learning, require a substantial amount of training data. We bridge this gap between availability and necessity by leveraging the generalizability of 2D cell segmentation models and making them suitable for 3D cell segmentation images. Importantly, as 3D multiplexed images often evolve from established 2D imaging modalities (e.g., 2D IMC vs. 3D IMC), a 2D cell segmentation model trained on a particular 2D imaging modality becomes a natural fit for its 3D counterpart. This applicability persists even when a 3D modality is in its developmental phase. As a result, our pipeline represents a potentially solution for imaging specialists. Not only does it remove the concerns of cell segmentation during 3D imaging technique development, but it also serves as a strategic guide, directing specialists to prioritize 3D imaging modalities that can benefit from existing, trained 2D models.

Our pipeline provides users with a clear pathway to select the best 3D cell segmentation method for their specific 3D tissue images. From a quantitative perspective, the enhanced CellSegmentationEvaluator evaluates 3D cell segmentation performance for multichannel tissue images without relying on human-provided ground truth. Our approach may liberate users from burdensome, time-consuming and potentially error-prone manual annotations for newly acquired tissue and cell images.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.ymeth.2025.07.007.

Data availability

3DCellComposer software is available at https://github.com/murphygroup/3DCellComposer. All results are in a Reproducible Research Archive, https://github.com/murphygroup/ChenMurphy3DCellCom poserRRA.