Integrated cytometry with machine learning applied to high-content imaging of human kidney tissue for in-situ cell classification and neighborhood analysis

Abstract

The human kidney is a complex organ with various cell types that are intricately organized to perform key physiological functions and maintain homeostasis. New imaging modalities such as mesoscale and highly multiplexed fluorescence microscopy are increasingly applied to human kidney tissue to create single-cell resolution datasets that are both spatially large and multi-dimensional. These single-cell resolution high-content imaging datasets have great potential to uncover the complex spatial organization and cellular make-up of the human kidney. Tissue cytometry is a novel approach used for the quantitative analysis of imaging data, but the scale and complexity of such datasets pose unique challenges for processing and analysis. We have developed the Volumetric Tissue Exploration and Analysis (VTEA) software, a unique tool that integrates image processing, segmentation, and interactive cytometry analysis into a single framework on desktop computers. Supported by an extensible and open-source framework, VTEA’s integrated pipeline now includes enhanced analytical tools, such as machine learning, data visualization, and neighborhood analyses for hyperdimensional large-scale imaging datasets. These novel capabilities enable the analysis of mesoscale two and three-dimensional multiplexed human kidney imaging datasets (such as CODEX and 3D confocal multiplexed fluorescence imaging). We demonstrate the utility of this approach in identifying cell subtypes in the kidney based on labels, spatial association, and their microenvironment or neighborhood membership. VTEA provides an integrated and intuitive approach to decipher the cellular and spatial complexity of the human kidney and complements other transcriptomics and epigenetic efforts to define the landscape of kidney cell types.

Introduction

Renal researchers increasingly appreciate the importance of characterizing the cellular niches of the kidney (cell types and subtypes, physiological state, neighborhood interactions) and how they are altered in kidney disease^1–4. Imaging of kidney tissue specimens with single-cell resolution at a large scale is an attractive approach for uncovering cellular niches in their tissue context^5–7. Such an approach has become more feasible because of the increased ability to collect mesoscale imaging datasets on accessible confocal microscopes, whole slide imagers, and light-sheet microscopes. Furthermore, combining mesoscale imaging with highly multiplexed staining or labeling approaches allows for capturing, in situ, hundreds of thousands of cells that may be classified by many markers. The scale and depth of these data create a mounting challenge for timely quantitative and interpretable analysis. This is particularly important for kidney research, where biopsy-scale multiplexed imaging datasets of the human kidney are being collected and publicly released by large collaborative consortia such as the Kidney Precision Medicine Project (KPMP) and the Human BioMolecular Atlas Program (HuBMAP)^8,9.

Tissue cytometry (TC) is a powerful approach for analyzing mesoscale fluorescence images with single-cell resolution^5,10–13. Depending upon the imaging platform, datasets may be either 2D or 3D^10,14. An important first step in TC is to survey all the cells by segmentation. This is often accomplished by using nuclei as fiduciaries for cells. Segmentation entails identifying regions of images as nuclei based on contrast provided by stains, registering each individual nucleus as an object, and identifying an associated cytoplasm spatially or with a specific marker. Features to describe these cells can be calculated on this segmentation. These features could be related to fluorescence intensities of markers within or around the nucleus (i.e., the cytoplasm) or based on the spatial coordinates or proximity relationships to these nuclei^10,13,14. Multiplexing several markers in the same experiment enhances the richness of the imaging data by providing specificity of cell types based on unique markers (or a unique combination thereof) and generating spatial information based on the distribution of stains within the tissue^5,13–15.

In a standard cytometry analysis, after cell segmentation, cells can be classified and quantitated by a supervised approach using gating based on marker intensity, in which cells are defined according to threshold levels of marker fluorescence (gating). However, as the scale and complexity of multiplex mesoscale image volumes increase, such a manual approach becomes intractable and increasingly unlikely to be successful at uncovering the complex spatial organization and cellular niches of the kidney. Multiplexed mesoscale tissue cytometry thus requires tools supporting unsupervised analysis, ideally with machine learning approaches like clustering and dimensionality reduction, to characterize the cellular makeup of a tissue accurately and completely, to identify cellular niches and to map their neighborhoods and microenvironments^{11,14,16–18}. Given the complexity of the interacting processes of segmentation, classification, quantification, and neighborhood analysis, the ideal system should incorporate all these processes into a single, integrated analysis and visualization software package.

Since the initial description of Volumetric Tissue Exploration and Analysis (VTEA) as an open-source project, several excellent tools for tissue cytometry have been developed (Table 1). At this time, VTEA (v0.5.2-v0.7) has been used in several projects involving single imaging fields to large mesoscale multi-fluorescence kidney image volumes to perform supervised cytometry analysis with gating^4,5,19–24. However, machine learning, data visualization, and analysis tools that are useful for multidimensional, big-data scale imaging data were not previously implemented in VTEA’s uniquely integrated pipeline. Here we describe VTEA 1.0, which specifically adds 1) machine learning, including clustering and dimensionality reduction, to aid in the automated classification of cell types, 2) neighborhood analysis to uncover cellular niches, and 3) new data visualization tools to support discovery. To facilitate this growth of VTEA’s integrated approach, a SciJava framework was implemented^25,26. VTEA now supports extensible image processing, segmentation, classification, visualization, and neighborhood analysis for processing on hundreds of thousands of cells and multi-gigabyte datasets with a fully integrated workflow. We demonstrate the utility of VTEA by identifying cell subtypes based on labels, spatial association, and neighborhood membership using large-scale 3D and 2D imaging data from kidney tissue.

Table 1.

Tissue Cytometry Software

Software tool	Access	Operating systems				Implementation								Extensible	Code	Image dimensions supported			Integration			Analysis
Name		Web	Linux	Windows	MacOS	C, other	Python	Java	ImageJ	R	Matlab	ImarisXT	Docker			2D	3D	NA	Image Processing	Segmentation	Analysis	Gating	Subgating	Image based gating	Unsupervised cell classification	Neighborhood Analysis	Populations batch correction
Imaris	$			x	x	x								XTensions	Closed		x			x	(4)	(5)
FCSExpress	$			x	x	x								Scripting	Closed			x				x	x	x	x	x
Halo	$	x		x		x								Scripting	Closed	x				x	x	(5)			x
CellProfiler and Cell Analyst	free		x	x	x		x							Python	Open	x	x		x	x	x	x	x		x	x	x
CODEX MAV (1)	free		x	x	x			x	x						Open	x			x	x	x	x	x		x (6)	x
XiT	free			x							x				Open			x				x	x
Chrysalis	free		x	x	x						x	x			Open			x	x	(3)	x					x
Cytokit	free		x	x	x		x						x	Python	Open	x			x	x	x	x			x (7)
CytoMap	free		x	x	x						x	x		Matlab	Open			(2)		(3)	x	x			x	x	x
QuPath	free		x	x	x			x						Scripting	Open	x			x	x		x		x
HistoCat	free		x	x	x						x			Scripting	Open	x			x	x	x	x	x		x	x	x
SAPHIR	free		x	x	x					x					Open	x			x			x	x	x
VTEA	free		x	x	x			x	x					SciJava	Open	x	x		x	x	x	x	x	x	x	x	in-progress

¹⁾Designed for CODEX platform data

²⁾Support for plotting 2D/3D scatters of cell centroids

³⁾Relies on segmentation performed elsewhere

⁴⁾Imaris’ XTensions supports several of analytical plugins

⁵⁾Gating is limited to setting minimum and maximum values on axes

⁶⁾Phenograph based cell classification

⁷⁾Relies on secondary software

Methods

Data acquisition and availability

Image data used in figures 3, 4, and 5 or figure 1 and S2 was previously and separately analyzed in Woloshuk et al., 2021, or Winfree et al., 2017 ^10,23. The analyses herein are wholly separate and independent of those analyses. The image data presented here is available upon request or, in part, at https://doi.org/10.5281/zenodo.5842279.

Figure 3. Separating masked populations of podocytes, leukocytes, and endothelium by unsupervised analysis of 3D cell image volumes in VTEA.

A. Human nephrectomy tissue was fixed in and stained with DAPI and with antibodies against CD31 (yellow), CD45 (green) and Nestin (magenta) and imaged by 3D confocal microscopy. Separated channels are given at right. All images are maximum intensity Z-projections, scale bar = 100 um. B. Nuclei segmented as fiduciaries of cell with connected components in 3D, ‘Connect 3D’, and associated intensities of CD31, CD45 and Nestin were measured. Ba. map of all segmented cells uniquely colorized. Bb-d Scatterplots of CD31, CD45, and Nestin demonstrate overlap in Nestin and CD31 signal in 2D plots. C. Unsupervised analysis of cell and marker intensity uncovers closely associated Nestin and CD31-positive cells. Ca-Cc. Mapping of cell-associated intensity (every dot is a cell) to t-SNE projection calculated from the mean intensity of DAPI, CD31, CD45 and Nestin. Cd. Ward clustering dendrogram using the same mean intensities with major bifurcation between labeled and unlabeled and at least 3 subclusters of labeled cells at k=5 (red line). Ce. Ward clusters mapped to t-SNE projection. Putative cell types are colored and labeled. Cf. Colorizing of segmented pixels for cells within CD45-positive, Nestin-positive and CD31-positive classes as given in Ce and presented as a z-projection.

Figure 4. Automated glomerular census in mesoscale 3D tissue cytometry with machine learning classification and spatial analysis in VTEA.

A. Human nephrectomy tissue was fixed in and stained with DAPI (gray) and with antibodies against CD31 (yellow), CD45 (green) and Nestin (magenta) and imaged by 3D confocal microscopy. Images are maximum intensity Z-projections, Scale bar = 500 um B. The whole 3D volume was analyzed, and 172,258 cells were segmented and processed by VTEA. Unsupervised analysis, G-means clustering and t-SNE mapping identified CD45-positive, CD31-positive, and Nestin-positive cells automatically. C. The cluster identities were confirmed by the plots generated by VTEA, including average mean fluorescence intensity of each class, (heatmap, left panel) or by the distribution of mean fluorescence intensities for each cell (violin plots, right panels). D. Unsupervised classifications were used in combination (left panel) with regions-of-interest (ROIs) drawn with ImageJ during analysis to perform a glomerular census of the endothelium (CD31), leukocytes (CD45) and podocytes (Nestin). The cell type masks are presented as a z-projection (middle panel). The census is presented as the density of cells per mm³ (right panel).

Figure 5. VTEA cell classification and neighborhood analysis identify a subclass of leukocytes associated with vascular endothelium in 3D.

Using the 3D confocal fluorescence image volume collected for Figure 4, spatially defined neighborhoods with CD45-positive cells were generated in VTEA. A. Mesoscale maximum z-projection of human nephrectomy tissue stained with DAPI (gray) and with antibodies against [CD31 (yellow), CD45 (green) and Nestin (magenta) and presented as separate channels in gray. CD45-positive leukocytes localize in clusters that associate with CD31-positive structures, including glomeruli and blood vessels (Aa-Ad). Scale bar = 500 um B. CD45-positive leukocytes and CD31+ endothelium was subgated from all cells (Figure 4B) and reclustered for k=5. Ba-Be, feature plots, and VTEA generated heatmaps of mean signal intensity for CD45, Nestin and CD31 uncover podocytes (Pod., green), Endothelium (Endo., red) and two putative populations of CD45-positive cells based on DAPI and CD45 intensity (p1 or pop1 vs. p2 or pop2, orange and blue respectively). Bf. Using cell-wise export with VTEA, 100 3D cells images from either of the two CD45-positive populations are shown after maximum z-projection. Ca-Cb. Maximum projections of mapped CD45-p1 and CD45-p2 in the image volume identifies tissue regions of mixed, and uniquely CD45-p2 cells. Cg, 3D neighborhood analysis, for every classified cell within r = 25 um and unsupervised clustering of neighborhoods based on neighborhood cell census, demonstrates CD45-p1 cells are found in neighborhoods with (N4, red) and without endothelium (Endo, N7–9, green e.g., Ca and Ce). CD45-p2 cells are also found alone without CD45-p1 nor endothelium (N10, orange, e.g., Cd and Cf).

Figure 1. Integration of VTEA into a 2D and 3D renal tissue cytometry workflow.

VTEA provides an interactive and bidirectional workspace to integrate image processing (Processing, blue, b), segmentation (Segmentation, orange, c), clustering and dimensionality reduction (green, e), mapping and back-mapping between the image and scatter plots (yellow, g and h) and generation of neighborhoods (Neighborhoods, red, i). Neighborhoods generated in VTEA can be clustered, back-mapped and additional levels of micro-environments (e-i).

Tissue was collected and processed under the Institutional Review Board at Indiana University approved protocols: 1906572234, for nephrectomy samples and 1010002261, for human biopsy samples.

Tissue acquisition

Multiplexed Immunofluorescence.

Sectioning and staining for multiplexed staining were performed as described previously²³. Briefly, 50 um sections of human nephrectomy tissue were cut from formaldehyde fixed (4.0% and stored in 0.25% in 1X PBS) with a vibratome (Leica Biosystems). Tissue sections were blocked in 10% normal donkey serum, 0.1% Triton X-100 in 1X phosphate buffered saline (PBS), and stained overnight with antibodies in block (Table 3)⁷. All secondaries were raised in donkey (ThermoFisher). In some cases, primary antibodies were directly conjugated, including CD31, CD45, and Nestin conjugated with Dylight550, Alexa488, and Alexa647 (Thermofisher). After staining with DAPI for nuclei and in some cases, with Alexa488-phalloidin for F-actin for 30 minutes, cells were washed in block, 1X PBS and mounted in Prolong Gold or Prolong Glass under #1.5 coverslips (ThermoFisher). Prolong Glass mounts were allowed to cure for 24–48 hours. As required, coverslips were sealed with clear nail polish (Electron Microscopy Supply) and stored at 4°C before imaging.

Table 3.

Antibodies

Antibody	Clone	Supplier	Catalog Number	CODEX	Confocal
CD68	PG-M1	Agilent-Dako	M0876		x
CD31	JC/70A	NovusBio	NB600–562		x
CD3	UCHT1	BD/Akoya	557706/4350008	x	x
Nestin	10C2	NovusBio	NB300–266	x	x
Uromodulin	Polyclonal	R&D	AF5144	x	x
AQP1	1/22	Santa Cruz	sc-32737-X	x	x
MPO	Polyclonal	Abcam	ab9535	x	x
CD45	HI30	Biolegend/Akoya	304002/4150003	x	x
LRP2	Polyclonal	Abcam	ab76969	x	x
Podocalyxin	Polyclonal	R&D	AF1658	x
Calbindin	CB-955	Abcam	ab82812	x
Cytokeratin8	TS1	NovusBio	NBP2–34501-0.1mg	x
α-sma	1A4	Invitrogen	14–9760-82	x
PROM1	AC133	Miltenyi Biotec	130–090-422	x
CD68	KP1	ThermoFisher	14–0688-82	x
IGFBP7	Polyclonal	Acris/Origene	AP01109PU-S	x
p- c-Jun	D47G9	Cell Signalling	3270BF	x
CD4	SK3	Akoya	4350010	x
CD11c	S-HCL-3	Akoya	4350012	x
CD31	WM59	Akoya	4250009	x
CD45RO	UCHL1	Akoya	4250023	x
HLA-DR	L243	Akoya	4250006	x
CD90	SE10	Akoya	4150021	x
E-cadherin	4A2C7	Akoya	4250021	x
⎬ -catenin	12F7	Akoya	4450036	x

Highly multiplexed immunofluorescence. Sectioning and staining for highly multiplexed immunofluorescence were performed as described previously^5,27. Briefly, 50 um human nephrectomy tissue was cryosectioned and fixed in formaldehyde (4.0% and stored in 0.25% in 1X PBS). Blocking, staining was performed with six antibodies, secondaries as needed, DAPI, and phalloidin as indicated (Table 3). Coverslips were mounted in Prolong Glass under #1.5 coverslips and cured for 24–48 hours. The eight markers were collected across 16 channels with four excitation wavelengths spanning the visible and near-infrared spectrum.

Confocal Microscopy

Multiplexed immunofluorescence specimens were imaged on either an FV1000 (Olympus) at 20× 0.75 NA oil immersion objective or a fully automated SP8 confocal with a 20× 0.75 NA Mimm objective as described previously²³. Highly multiplexed fluorescence imaging was performed on the fully automated SP8 confocal and unmixed and stitched in either LASX(Leica) or FIJI²⁶.

CODEX

Antibody Conjugation and Validation.

14 of the 23 antibodies used here were conjugated in-house using the protocol outlined by Akoya Biosciences (Table 3)^19,20. To conjugate barcodes to antibodies, antibodies were reduced using a “Reduction Master Mix” (Akoya Biosciences), to which lyophilized barcodes resuspended in molecular biology grade water and “Conjugation Solution” (Akoya Biosciences) were added and incubated for 2 hours at room temperature. Labeled antibodies were purified from a free-barcode in a 3-step wash and spin process and stored at 4°C. Successful conjugation was validated via SDS-PAGE gel electrophoresis as well as immunofluorescent staining of reference tissue followed by confocal microscopy.

Tissue Preparation.

10-micron sections of human renal tissue embedded in OCT were cut onto poly-L-lysine coated coverslips. Sections were prepared as detailed by Akoya Biosciences and as described previously^7,14. Tissue retrieval was conducted with a 3-step hydration process, followed by fixation with a PFA-containing solution. Following fixation, the coverslip-mounted tissue was incubated overnight at 4°C with an antibody cocktail of 23 of the antibodies listed in Table 3. Tissues were washed and post-fixed in 4% PFA for 15 minutes.

Imaging.

Antibodies were imaged cyclically using the CODEX system from Akoya Biosciences and a Keyence BZ-X810 slide scanning microscope fitted with a 20x air objective (0.75 NA). Images were processed using the CODEX Processor (Akoya Biosciences), and images were exported for analysis with VTEA.

Software design, development, and distribution

Volumetric Tissue Exploration and Analysis (v1.0) was developed in Java, with SQL, R-script, and Python, using the integrated development environment Netbeans (Apache) using a maven build scheme. Major application program interfaces (APIs) used in VTEA include SciJava (v.30.0.0), ImageJ (v.1.53f), h2 (v. 1.4.198), SMILE (v.1.5.3), Renjin (v3.5-beta76) and JFreeChart (v. 1.5.0). The GitHub release tag @cdfbd46 can be used to perform all the analyses presented here. VTEA v1.0, bleeding-edge and archival versions, can be downloaded and built from source code using a maven build scheme, https://github.com/icbm-iupui/volumetric-tissue-exploration-analysis . Stable releases can be installed in FIJI by using the FIJI updater and selecting the “Volumetric Tissue Exploration and Analysis” update site. General description, analysis vignettes with demonstration data, and development plans can be found at https://www.vtea.wiki and https://imagej.net/plugins/vtea.

Segmentation in VTEA

Nuclei were segmented in VTEA as fiduciaries of cells with the segmentation routine ‘Connect 2D/3D’ in 2D for CODEX data or in 3D for confocal fluorescence microscopy data. ‘Connect 2D/3D’ is a connected components approach that merges 2D segmentations, defined by Otsu thresholding and watershed splitting, into 3D volumes.

Computers used in the analysis

Image data were analyzed with VTEA on a MacBook laptop (mCorei5, 8 GB RAM, 2016), a Lenovo P51 (Xeon quad-core, 64 GB RAM), or an 8-core custom-built workstation (Xeon 8-core, 256 GB RAM).

Figure preparation

All images were generated in ImageJ/FIJI, and plots (scatter, violin, heatmaps), gated cell overlays, and segmentation maps were created by VTEA. Photoshop (Adobe) was used for the final annotation and assembly of panels. Scales were set by the microscopy platform and annotated in ImageJ/FIJI. All intensity changes are linear unless otherwise noted.

Results

VTEA, an integrated tool for image processing, segmentation, and interactive cytometry analysis

VTEA has a fully integrated analytical pipeline for image processing, segmentation, and cytometry in one software framework (Figure 1). Using the standard approach of VTEA, segmented cells can be analyzed in a 2D scatter plot, where gating can be applied based on marker intensity to identify and count specific cells of interest. Gated cells are visualized in the image volume, which allows for visual validation of the gating strategy and facilitates biological interpretation (Figure 1f). Furthermore, using ImageJ/FIJIs region-of-interest(ROI) selection tool allows quantitation and visualization of select cells in the image within the scatter plot (Figure 1g and h). This back-and-forth interaction between the image and analytical space allows for fine-tuning of the analysis and interactive exploration. VTEA also has a unique bidirectional workflow, whereby a user can make upstream adjustments in image processing and segmentation and test the effect of these adjustments on the cytometry output (Figure 1, double-headed arrows).

Building an extensible framework for an integrated tissue cytometry (TC) analysis in VTEA

The dimensionality and complexity of imaging datasets that can be collected by biologists are constantly expanding and improving with the advent of novel tissue interrogation technologies that generate imaging data²⁸. Thus, image analysis pipelines require flexibility and adaptability to new technologies and analytical approaches. To incorporate flexibility and adaptability in VTEA’s integrated pipeline, a SciJava-based plugin infrastructure was implemented. The SciJava extensible framework uses interfaces implemented by super-classes that are extended by concrete implementations, which include runtime annotations for indexing of available classes or plugins (Table 2). This extensible framework incorporated new functionality into VTEA, including functionality from SMILE, Renjin, and ImageJ/FIJI(Figure S1A)^26,29,30. To improve data handling, the SQL-based Java database h2 was also implemented (Figure S1A)³¹. Furthermore, additional functionality has been added to VTEA, including but not limited to: 1) compatibility with other segmentation and analysis tools, 2) archiving and sharing data 3) generating multiclass training datasets for use in image classification (Figure S1B)²³.

Table 2.

SciJava Object Structures and Functionality

Functionality	Interface	Superclass	Comment
Factories, processors	Processor	AbstractProcessor.class	Defines a suite of factories for processing data and generating a result.
Image Processing	ImageProcessing	AbstractImageProcessing.class	Defines image processing steps to perform on imaging data and returns a processed copy.
Segmentation	Segmentation	AbstractSegmentation.class	Defines a segmentation approach and returns segmented objects.
Measured morphologies	Morphology	AbstractMorphology.class	Defines regions associated with segmented nuclei in which to measure intensity.
Measurements to make on segmented objects	Measurment	AbstractMeasurment.class	Defines the maths or statistics to perform on the pixel/voxel values associated with segmented objects.
Measurements to make on neighborhoods	NeighborhoodMeasurement	AbstractNeighborhoodMeasurement.class	Defines the measurments that are done on neighborbood class including summary statistics of constituents.
Features derived from unsupervised analysis	Feature	AbstractFeature.class	Defines a algrorithm to perform unsupervised analysis on segmented cells based on measured features including AbstractMeasurements.class and AbstractNeighborhoodMeasurement.class.
Look up tables	LUT	AbstractLUT.class	Defines a LUT to be used in plot overlays and static plots generated by VTEA.
Static plots	Plot	AbstractPlot.class	Defines a ploting routine that returns an image file, e.g. .png.

VTEA with unsupervised machine learning improves the accuracy and resolution of cell classification in kidney image volumes

Manual gating has limitations in the accuracy and resolution of cell classification and may underperform in the setting of large multi-dimensional datasets. The use of scatterplots, gates, and sub-gating is a common strategy for analyzing cytometry data. Gating frequently requires the use of expert opinion and operator-dependent thresholding of cells that are positive or negative for a marker of interest (Figure 2A). For instance, in 3D confocal imaging data of the human renal cortex, detecting aquaporin-2 (AQP2) positive cells is expected to identify the collecting duct (CD). However, in a standard VTEA analytical 2D scatter plot, the population of AQP2-positive cells may not be accurately identified by manual gating (Figure 2A, lower right plot). To address this weakness, machine learning approaches, clustering, and dimensionality reduction, were added to VTEA by implementing algorithms in Java or by incorporating algorithms from existing libraries(Table 4)^29,32,33. Using one of VTEA’s newly implemented clustering algorithms, Ward hierarchical clustering, with the fluorescence intensities, the AQP2-positive population was readily identifiable and automatically classified as a cluster (Figure 1e, Figure 2B, lower right). Importantly, this added functionality to VTEA also identified additional AQP2-positive cells not initially included in the expert-drawn gate, but clearly a part of the collecting duct epithelium when mapped back to the imaging volume (Figure 2C).

Figure 2. Improved classification of the kidney epithelium in 3D with semi-automated machine learning.

50-micron thick human kidney tissue was stained for proximal tubule and collecting duct markers aquaporin-1 (AQP1) and aquaporin-2(AQP2) and counterstained for nuclei (DAPI) and filamentous actin (F-actin, phalloidin) and imaged by 3D confocal fluorescence microscopy. A. Supervised analysis of a volume of human kidney cortex in 3D (top panels) in VTEA with user-defined gates based on the intensity of AQP1, AQP2, and F-actin intensities where the AQP2 gate was selected to avoid cells with high F-actin intensity per a lookup-table (LUT, lower panels). Top panels are maximum z-projections, scale bar = 100 um. B. Unsupervised dimensionality reduction and clustering in VTEA of average and standard deviation of cell-associated intensities of DAPI, phalloidin, AQP1, and AQP2 partitions the cells into discrete groups. The intensity of F-actin, AQP1, and AQP2 is plotted as the LUT in t-SNE space (top panels), and Ward Hierarchical clustering (red line, k=5 based Elbow Plot) subdivides the cells into clusters that reflect the marker intensity (bottom panels). C. Changes in cell classification visualized in VTEA (a single optical section from a 3D segmentation is shown) demonstrate an increase in the number of each group of cells with the unsupervised approach in (B). Cells classified in (A) are in yellow and cells in orange were added to the classification with Ward Hierarchical clustering (cluster 4, B). Scale bars = 100 and 20 um.

Table 4.

Machine Learning Algorithms Implemented in VTEA

Type	Name	Source	URL
Dimensionality Reduction	IsoMap Embedding	SMILE	https://haifengl.github.io/manifold.html#isomap
	Laplacian of EigenMap	SMILE	https://haifengl.github.io/manifold.html#laplacian
	Principal Component Analysis	SMILE	https://haifengl.github.io/api/java/smile/projection/PCA.html
	t-SNE	lejon-tSNE	https://github.com/lejon/T-SNE-Java
Clustering	G-means	SMILE	https://haifengl.github.io/clustering.html#g-means
	X-means	SMILE	https://haifengl.github.io/clustering.html#x-means
	K-means	SMILE	https://haifengl.github.io/clustering.html#k-means
	K-means (VTEA)	in-house	https://github.com/icbm-iupui/volumetric-tissue-exploration-analysis/blob/master/src/main/java/vtea/clustering/KMeans.java
	Ward Hierachical	SMILE	https://haifengl.github.io/clustering.html#hierarchical
	Single-Link Hierachical	SMILE	https://haifengl.github.io/clustering.html#hierarchical
	Complete-Link Hierachical	SMILE	https://haifengl.github.io/clustering.html#hierarchical

To improve the resolution of classified AQP2-positive populations, non-proximal epithelial cells (AQP1-, F-actin- negative cells) in the renal cortex (clusters 2,3 and 4, Figure S2A and S2Ba) were subgated in VTEA. Using all fluorescence intensities, the segmented cells were re-clustered and mapped to t-SNE space with VTEA. Ward clustering uncovered three AQP2-positive clusters (Figure S2Bb) that mapped to the distinct tubular epithelium. The two AQP2-positive clusters with the highest intensity (clusters 1 and 2 Figure S2Bc) mapped predominantly to collecting duct cells (verified based on morphology) and putative distal epithelial cells, which could be consistent with APQ2positive progenitor cells described previously³⁴. The AQP2-positive with low intensity mapped to the distal tubular epithelium, suggesting a subpopulation of cells in the distal nephron with a low level of AQP2 (Figure S2Ad and S2Bc). The increased resolution and distribution of AQP2 expression was only apparent after using VTEA to 1) sub-gate and re-cluster putative AQP2-positive cells and 2) confirm these new populations with both mapping the gated cells to the image and selecting the tubules with an ROI in the image to map to the scatterplot (Figure S1Ae and S1Bc). Using VTEA’s unsupervised analysis, subgating, and bi-directional mapping increased the resolution of cell classification, which could not be performed using a traditional supervised approach (Figure 1A).

Separation of spatially overlapping cell populations in 3D image volumes with unsupervised machine-learning and determination of glomerular census at large scale with VTEA

Using a manual gating strategy with standard 2D scatter plots may poorly discriminate spatially congested cells (for example, within glomeruli), where staining from one cell can confound the classification of neighboring cells. As shown in Figure 3A, Gating Nestin-positive podocytes cells successfully in a standard scatter plot with VTEA is not feasible because the analysis is confounded by the close association of Nestin-positive cells with CD31-positive glomerular endothelial cells and CD45-positive immune cells. Using dimensionality reduction with t-SNE, feature plots, and unsupervised Ward hierarchical clustering as implemented in VTEA, Nestin-positive podocytes can be isolated from CD31-positive endothelial and CD45-positive immune cells (Figure 3B). Such an analytical approach can also be applied at a large scale, where unsupervised analysis can identify immune, endothelial, and podocyte cells for an entire section of kidney tissue (Figure 4A–B). Glomerular cell census in such tissue can be readily acquired by applying spatial regions of interest on the image volume (Figure 4D). Cells in the glomeruli are counted and classified as podocytes, endothelial, immune, or other (likely mesangial). The 3D volumes collected allow for volumetric cell densities to be calculated for glomeruli (Figure 4D).

Assessing infiltrating immune cell subtypes in human kidney based on spatial neighborhoods

Focusing on the CD45-positive immune cells from Figure 3 and applying VTEA’s unsupervised analysis, we uncovered 2 populations of CD45-positive immune cells, which separated based on their nuclear staining and CD45 intensity (Figure 5A and B). We asked if these two subpopulations could distribute in unique spatial niches. Using the cell-centric neighborhood analysis function and data visualization features within VTEA, we defined the spatial neighborhoods of all labeled cells, including the two types of CD45-positive immune cells defined by CD45 and DAPI intensity where CD45-pop1 cells had higher intensities of CD45 and DAPI than CD45-pop2 (Figure 5e). Most neighborhoods with immune cells had a mixture of the two CD45-positive populations (Figure 5Ca, 5Cb, and 5Cg neighborhoods 7, 8, and 9). However, a unique immune neighborhood enriched with endothelial cells (i.e., close to vessels) had a preponderance of CD45-pop1 (Figure 5Ce and 5Cg Neighborhood 4). Another neighborhood with the lowest fraction of endothelial cells (i.e., distant from vessels) had more CD45-pop2 cells (Figure 5Cf and 5Cg neighborhood 10). These findings demonstrate that unsupervised clustering with neighborhood analysis in VTEA can uncover a variety of cell niches based on spatial relationships that may have biological relevance.

Using VTEA for classification of cell types in 3D spectral confocal images of human kidney

The use of multispectral confocal imaging and linear unmixing of fluorophores enables large-scale 3D imaging with 8 multiplexed probes (Figure 6A). This approach allows the survey of tens- to hundreds of thousands of cells in situ within kidney tissue. Identifying major cell subtypes and the less abundant immune cells can be difficult using traditional gating, particularly when the emission spectra from fluorophores are overlapping. Using unsupervised clustering (Figure 6Ba–6Bc) and visualizing mean fluorescence intensity distributions of the cell subtypes in VTEA (Figure 6Bd), we can readily classify both epithelial and immune cells in an unbiased manner (Figure 6B).

Figure 6. Uncovering cell types in 3D highly multiplexed data with VTEA.

Human nephrectomy tissue was fixed and stained for 8 markers and imaged by 3D spectral fluorescence confocal microscopy. Using reference spectra, 8 channels were unmixed and analyzed by VTEA. A. Mesoscale, maximum z-projections of subsets of channels highlighting immune or tubular epithelium (Aa and Ab), with insets as given in Ab. Arrowheads indicate neutrophils (red), T-cells (blue) and eosinophils (yellow). Scale bars are 200 um and 50 um. B. Unsupervised analysis of all cells with 8 marker channels separates the major cell types expected in the renal cortex. Mixed classes of Interstitium (Int) and distal nephron including distal convoluted tubule (DCT) or collecting ducts (CD) are low for all markers (Bc). A small group of THP high cells were annotated as not defined (ND, Ba). Granulocytes, eosinophils and neutrophils, are abbreviated as Neut and Eos in Bc.

Applying VTEA’s pipeline on highly multiplexed CODEX data of the human kidney to uncover cell sub-types and biologically relevant cell neighborhoods

Human cortical biopsy underwent CODEX imaging with 23 markers (Figure 7A, Table 3) and was analyzed with VTEA to perform cytometry, cell classification, and neighborhood analysis (Figure 7B–E). Unsupervised hierarchical clustering of the 11,355 segmented cells identified the major cell types in the kidney (Figure 7B). Using subgating and subclustering, additional novel cell state phenotypes were identified, including PROM1-positive (CD133) thick ascending limb and proximal tubule cells (Figure 7B, and S3D), and CD68-positive dendritic cells, CD68-positive putative macrophages, and CD68-positive epithelial cells (Figure 7C and S3C–E). To determine if specific cellular microenvironments were present in the tissue, VTEA’s neighborhood analysis function (Figure 1i) was used to tabulate neighborhoods and revealed, after clustering and dimensionality reduction, unique neighborhoods with associations between tubular epithelial cell subtypes and immune cells, such as the association of PROM1-positive TAL and PROM1-positive PT cells with low levels of CD68-positive macrophages (Figure 7D and E, N4 and N5 respectively) and an association of CD90-positive proximal tubules with DC and T-cells (Figure 7E, N6).

Figure 7. Automated detection, classification of cell-types and assessing cellular microenvironments in 2D CODEX data with VTEA.

Multiplexed immunofluorescence image dataset of human reference kidney was processed, segmented, and analyzed with VTEA. A. Maximum projects of subsets of channels highlighting tubular epithelium or immune cells in the renal cortex. The left panel indicates three regions given in Aa, Ab and Ac and segmentation mask (inset). Scale bars are 500 um and 30 um. B. Segmented cells with associated marker intensity with clustered using hierarchical clustering and projected into t-SNE space using the average intensity of associated markers. Putative cell types as indicated. C. Marker intensity for clusters identified in B, normalized by marker. Cell types include subclasses of epithelial and leukocytes. T_MEM and T_N are putative memory and novel T-cells. epithelium identified included proximal tubule (PT) S1, S2, and S3 subsegments, loop of Henle thick ascending limb (TAL) and the distal nephron subsegments distal convoluted tubule (DCT) and collecting duct (CD). Markers are given at the bottom. A subset of clusters had overlapping proximal tubule (LRP2-positive green arrowhead) and TAL (cyan asterisk) or leukocyte signatures (green and orange asterisks). D. Using cell-types defined in C and Figure S3, neighborhoods were defined in VTEA for every cell within 50 um. The cell census for all neighborhoods was used to cluster and map the neighborhoods to t-SNE space in VTEA. E. The distribution of cell types was plotted by neighborhood as a heatmap to identify unique microenvironments in the tissue volume.

Discussion

In this work, we present an integrated tissue cytometry approach with VTEA to analyze and extract biologically relevant data from state-of-the-art and increasingly multiplexed fluorescence imaging datasets of human kidney tissue. This approach leverages innovative tools for analysis and visualization using machine learning to perform rigorous, reproducible, and informative analyses that could be used to uncover the complex spatial organization and cellular make-up of the human kidney. Using this analysis pipeline, we demonstrated how we can improve the accuracy and resolution of cell classification in kidney tissue. Furthermore, we showed unique advantages of this approach in performing advanced quantitative analysis to uncover cell populations based on spatial associations and neighborhood memberships. In addition, VTEA has the tools to perform intuitive analysis on highly multiplexed datasets and extract novel information on cell subtypes and neighborhoods that could complement findings from other omics studies at the single-cell level. VTEA is available for download through the FIJI plugin updater with the source code is available on GitHub, https://github.com/icbm-iupui/volumetric-tissue-exploration-analysis. Additional description and vignettes demonstrating the use of VTEA are available at https://vtea.wiki.

One of the advantages of incorporating unsupervised clustering and dimensionality reduction in the VTEA workflow, as compared to relying only on one label of interest to identify cells, is the ability to use information from the other label intensities and potentially additional spatial parameters. These added parameters increase the discriminative power to identify specific population of cells, such as better identification of low intensities of a specific marker (as shown for AQP2). These applications will improve the accuracy compared to a standard gating strategy and increase the confidence of identifying novel cells that may be biologically relevant, as we showed for low AQP2 expressing cells in the distal nephron.

In addition to improving the accuracy and resolution of cell classification, the workflow of VTEA with machine learning tools will facilitate the cytometry analysis of imaging data that was not previously feasible using a standard VTEA approach. We present examples of spatially overlapping cells within structures like glomeruli. Taking advantage of multiple dimensions, it is possible now to accurately quantify the various cell types within glomeruli. This process could also be semi-automated using the data analytical tools provided, thereby having important potential implications on studying human glomerular pathology, where the cell density of specific cell types such as podocyte, immune and mesangial cells may be linked to the pathogenesis of kidney disease⁵. We also demonstrated the utility of VTEA in classifying cells based on spatial parameters, such as neighborhood memberships based on association with structures such as vessels and or other cell types. In the CD45-positive immune cell example, we could classify two cell populations based on nuclear staining and association with vessels. We hypothesize that these immune cell subpopulations may reflect different stages of activity and infiltration: from margination and exiting vessels towards forming foci of inflammation within the peritubular space. These findings are only proof-of-principle of the capabilities of VTEA in using imaging-based data for the discovery of spatially based cell niches and require further validation to fully determine biological relevance and generalizability^4,19.

Next, we demonstrated VTEA’s utility in the segmentation and analysis of imaging data from kidney tissue while supporting classification, quantitation, and visualization. VTEA can process mesoscale datasets with tens to hundreds of thousands of cells both in 2D and 3D while maintaining the interactive characteristics of the analysis. In the multiplexed 3D confocal large-scale data, we used unsupervised analysis and dimensionality reduction to classify the cell types. We validated these classes of cells based on visualizing the distribution of intensity for each classified cell type. This provides a semi-automated process for large and high-content datasets that augments the rigor of other quality check measures already used, such as the validation of the identified cells by mapping them in the original image.

We also analyzed highly multiplexed large-scale data from human cortical kidney tissue imaged with CODEX. This imaging technology expands the ability to multiplex markers on the same 10 μm thick tissue sections using DNA-conjugated antibodies. With CODEX imaging, these antibodies are revealed three at a time by the reversible binding of fluorescent oligonucleotide reporters. Following imaging, the fluorescent reporters are stripped from the tissue, replaced with a second set of three probes, and imaged again. This process is repeated until all the antibodies in the tissue have been revealed. Images of DAPI-labeled nuclei are collected in each round to enable the registration of images into a single highly multiplexed image. Although CODEX imaging has been described on mouse and human kidney tissue recently⁶, the analytical output from such data has been limited. Using the integrated analytical pipeline with VTEA, we can perform cytometry and unsupervised machine learning to classify major cell types and uncover novel subtypes. For example, we demonstrate the existence of subtypes of proximal tubules (PT: CD90-positive PT, PROM1-positive PT) and thick ascending limb cells (PROM1-positive TAL)⁴. Subclassifying also identified T-cell subtypes based on multiple markers. Using neighborhood analysis, we can uncover new spatial associations that could inform the biology. For example, macrophage association with PROM1-positive TAL cells is consistent with recent single-cell transcriptomics data suggesting the transcriptional phenotype of PROM1-positive TALs, may be in niches of immune activation⁴. We expect that applying VTEA analysis on such highly multiplexed CODEX data will complement and spatially anchor single-cell transcriptomics data and may inform and confirm (at the protein level within the tissue) transcriptomics outputs such as receptor-ligand analyses.

The advantages of VTEA analysis have been outlined in this work and include the unique integrated workflow in the setting of a general framework of accessibility, flexibility, and extensibility. VTEA can work as a stand-alone tool carrying the imaging data (after collection) all the way to analysis, which offers unique advantages. For example, applying the integrated VTEA workflow to analyze human kidney multiplexed imaging data will enhance efficiency and discovery because all the steps, including advanced machine learning analysis and visualization, occur in one software space. Furthermore, VTEA has unique strengths such as: 1) fine-tuning, in real-time, image processing and segmentation parameters to optimize the analysis and 2) gating for interactive back-and-forth between image and analysis. Importantly, VTEA can also operate with custom workflows to accept inputs from other sources. For example, a better segmentation algorithm (by importing segmentation maps) or an outside set of measurements (such as from MorphoLibJ) can be easily imported. VTEA also exports csv files of measured and machine-learning features and indexed segmentation maps for integration into other tools and pipelines.

Tissue cytometry with VTEA still has limitations. Despite the advantages of an integrated workflow that has been applied on computer desktops (not requiring computer clusters or server-based), the analysis and visualization of large mesoscale datasets will require desktop computers with enhanced data processing and RAM. Although the analyses of the smaller image datasets presented here can be performed on very modest computers (e.g., 2016 Macbook). There are also persistent challenges in mobilizing large datasets from acquisition platforms to the analysis computers. Importantly, this is not unique to VTEA or even TC and a problem that pertains to other mesoscale and omics datasets.

Combining datasets into the same analytical and non-overlapping image spaces within VTEA is an ongoing area of research and development. With progress in multiscale and hierarchical image formats, this may be greatly simplified soon³⁵. Furthermore, multiple datasets can be aligned into a single analysis with VTEA using normalization and mapping strategies but testing and adequately addressing batch effects needs to be established across image datasets³⁶. A strategy for combining datasets that can be used currently is with cell-centric neighborhood analysis, which is, by default, normalized spatially. We recently used such an approach on 3D large-scale multiplex confocal image volumes of human biopsies from the Kidney Precision Medicine Project(KPMP) and successfully combined neighborhoods of ~1.2 million cells⁴.

Tissue cytometry, especially with unsupervised analyses, is poised to provide new insight into disease mechanisms. To that end, VTEA is being used to interrogate mesoscale imaging datasets collected by the KPMP to define signatures of disease and highlight mechanisms in disease which may have clinical implications^5,8. Further, tissue cytometry has translational implications that may provide unprecedented insight in digital pathology³⁷. Further, there is an untapped potential to apply our approach beyond cytometry to segment, quantify and classify renal structures (e.g., tubules and glomeruli) and subcellular or acellular structures (e.g., endosomal vesicles, protein deposits).

In conclusion, we present tissue cytometry with VTEA as a solution to analyze and interpret high-content imaging data of human kidney tissue. Using appropriate unsupervised machine learning approaches, including clustering and dimensionality reduction, we demonstrated how VTEA can classify and characterize cell populations based on a suite of cell-wise features, including intensity measurements and neighborhood cell population statistics. We anticipate that this approach will be useful in uncovering the complex spatial organization and cellular make-up of the human kidney and generalized to analyzing imaging data of tissues from other organs.