Spatial UMAP and image-cytometry for topographic immuno-oncology biomarker discovery

Abstract

Multiplex immunofluorescence (mIF) can detail spatial relationships and complex cell phenotypes in the tumor microenvironment (TME). However, the analysis and visualization of mIF data can be complex and time-consuming. Here, we used tumor specimens from 93 patients with metastatic melanoma to develop and validate a mIF data-analysis pipeline using established flow cytometry workflows (image-cytometry). Unlike flow cytometry, spatial information from the TME was conserved at single-cell resolution. A spatial Uniform Manifold Approximation and Projection (UMAP) was constructed using the image cytometry output. Spatial UMAP subtraction analysis (survivors vs. non-survivors at 5 years) was used to identify topographic and co-expression signatures with positive or negative prognostic impact. Cell densities and proportions identified by image cytometry showed strong correlations when compared to those obtained using gold-standard, digital pathology software (R2 > 0.8). The associated spatial UMAP highlighted ‘immune neighborhoods’ and associated topographic immunoactive protein expression patterns. We found that PD-L1 and PD-1 expression intensity was spatially encoded—the highest PD-L1 expression intensity was observed on CD163⁺ cells in neighborhoods with high CD8⁺ cell density and the highest PD-1 expression intensity was observed on CD8⁺ cells in neighborhoods with dense arrangements of tumor cells. Spatial UMAP subtraction analysis revealed numerous spatial clusters associated with clinical outcome. The variables represented in the key clusters from the unsupervised UMAP analysis were validated using established, supervised approaches. In conclusion, image cytometry and the spatial UMAPs presented herein are powerful tools for the visualization and interpretation of single-cell, spatially-resolved mIF data and associated topographic biomarker development.

BACKGROUND

Multiplex immunofluorescence (mIF) is an emerging technology that has great potential for quantifying and mapping immune cells and associated immunoactive protein expression (1). There is great interest in using this platform for prognostic and predictive immuno-oncology biomarker discovery. A recent meta-analysis showed that assays using mIF had higher positive and negative predictive values for response to anti-PD-1/PD-L1 when compared to PD-L1 IHC, an IFNγ gene signature, or mutational burden (2). In that meta-analysis, the typical number of markers studied was only two or three. Multispectral, multiplex IF systems are now available that can study up to 8 markers across an entire slide (>1000 high power fields).

mIF tissue-based assays allow for marker co-expression and spatial parameters to be studied with single-cell resolution. Signal intensity can also be assessed, rendering these technologies for tissue sections analogous to flow cytometry. Output parameters for mIF assays are typically assessed using image-analysis software and exported in machine-readable format on a per-cell basis, facilitating bespoke queries with software such as R. This approach poses multiple challenges and can be time consuming. Once cells are identified (segmenting), using the image-analysis software, manual training is required for the assignment of individual cell types and their expression/co-expression patterns (phenotyping). Tools to explore the complex spatial associations between markers in an unsupervised manner are also lacking. Current supervised approaches include proximity analyses using a pre-determined distance between cells or nearest neighbour analyses (3–6), both of which have the potential to oversimplify the data.

A potential alternative is to use methods from the field of flow cytometry, which has a long history of analyzing co-expression and intensity data on a cell-by-cell basis. Dimensionality reduction techniques such as t-distributed stochastic neighborhood embedding (t-SNE) and uniform manifold approximation and projection (UMAP) are available for exploring high-dimensionality flow cytometry datasets (7,8), but lack a spatial component. In this study, we set out to: 1) determine whether we could capitalize on previously defined flow cytometry workflows to analyse mIF data, while maintaining the spatial information provided by this emerging technology, i.e. ‘image cytometry’; and 2) develop a novel spatial UMAP-based, unsupervised approach to cluster and visualize the resultant data and identify spatial correlations of clinical relevance.

METHODS

Specimen collection and tissue microarray (TMA) design

This study was conducted in accordance with the Declaration of Helsinki and was performed following Johns Hopkins University IRB approval (#NA_00085595). This protocol allows for the retrieval of tissue from archives from patients who signed an informed written consent or with waiver of consent. Ninety-three specimens from unique patients with metastatic melanoma acquired from 1990–2012 were identified in the surgical pathology archives. At the 5-year follow-up timepoint, 34 of 93 patients (36%) were alive. Additional clinicopathologic characteristics are provided in Supplementary Table S1.

The diagnosis of melanoma was confirmed by a board-certified pathologist (JMT), and a representative formalin-fixed, paraffin-embedded (FFPE) block was chosen from each specimen for inclusion in TMAs. Six 1.2 mm cores were taken from each block including central and peripheral tumor areas (Supplementary Figure S1). The resultant TMAs were reviewed, and cores with <10% surface area occupied by tumor cells were excluded from analysis.

Multiplex immunofluorescence (mIF)

TMAs were stained for CD8, PD-1, PD-L1, CD163, FoxP3 and melanoma cocktail (Sox10 and S100) as previously described using the antibodies listed in Supplementary Table S2 (9) Briefly, following deparrafinization of 4 μm-thick tissue slides, antigen retrieval was performed through microwave pretreatment with AR9 followed by AR6 (Akoya Biosciences, Marlborough, MA, cat. #AR900250ML and #AR600125ML). The first primary antibody (“Position 1”) was then applied. Opal polymer HRP Ms+Rb (Akoya Biosciences, cat. #ARH1001E) or PowerVision Poly-HRP (1:1 dilution, Leica Biosystems, Buffalo Grove, IL, mouse PV cat. #PV6119 and Rb PV cat. #PV6114) was used as the secondary antibody. The slides were washed, and the tyramide signal amplification (TSA)-dye (Opal 7 color kit, Akoya Biosciences, cat. #NEL861001KT) for Position 1 was applied. Slides were then microwaved in AR6 to strip the primary and secondary antibodies, washed, and blocked again using blocking solution. The second primary antibody (“Position 2”) was applied, and the process was repeated through amplification of the sixth primary antibody (Supplementary Table S2). DAPI was applied and slides were coverslipped using ProLong Diamond Antifade Mountant (Life technologies, Cambridge, MA, cat. #MAP36961).

Stained slides were scanned using the Vectra 3.0 Quantitative Pathology Imaging System (Akoya Biosystems). InForm 2.3 Image Analysis software (Akoya Biosciences) was used for spectral unmixing, cell segmentation, and identification and quantification of cellular subsets. Sox10/S100 (tumor), CD8, CD163, and FoxP3 were considered ‘linage markers’; they were scored in a binary fashion (‘positive’ or ‘negative’) and the densities of these lineages were determined. Cells that were negative for these markers were considered in the ‘Other’ cellular subset. PD-1 and PD-L1 were considered ‘expression markers’, and their expression by the identified cellular subsets was determined.

Image cytometry

In addition to the InForm image-analysis workflow described above, a flow cytometry–like workflow was performed, Figures 1A and 1B. In this approach, the Vectra system was used only for imaging and spectral unmixing. The subsequent image analysis was performed by transferring the resultant, un-mixed multilayer-TIFF images into HALO V2.0 (Indica Labs, Corrales, NM) for cell segmentation. The resultant single-cell data (cell area and location as well as mean fluorescence intensity (g) for each marker per cell) was exported to FlowJo™ (V10, Becton, Dickinson & Company, Franklin Lakes, NJ). All cores corresponding to a single tumor sample were grouped using gates. Marker expression for each individual specimen was then gated based on mean fluorescence intensity (MFI). Analyses of cell density and distance were performed in R software. Additionally, the association between CD8 proximity and PD-1 and PD-L1 expression intensity was ascertained. For this latter analysis, only specimens containing at least 150 PD-L1⁺ or PD-1⁺ cells, respectively, were included.

Figure 1. Cell densities identified by image cytometry are comparable with those found with digital pathology software.

(A) Pipeline for the analysis of mIF-stained TMAs of metastatic melanoma using a flow cytometry–like workflow. Each blue arrow indicates data export/import. (B) Data visualization after manual gating validates our image cytometry quantification approach i.e., ‘image cytometry’. Representative image of a FFPE melanoma core stained by mIF (left), with corresponding image cytometry gates (middle), and color-coded dot plot–map of the gated populations (right). (C) Cell densities identified by our image cytometry gating strategies (n=453 TMA cores from n=93 individual tumor specimens, single experiment) showed robust correlation with those identified using the Vectra inForm ‘phenotype tool’ i.e., the current gold standard, for well-segmented lineages, i.e., CD8⁺, FoxP3⁺, and PD-1⁺ cells. In contrast, lineages prone to segmentation errors i.e., tumor cells and CD163⁺ macrophages, showed an acceptable, yet weaker correlation. Linear regression, R and slope ± 95% confidence interval are displayed.

Spatial Uniform Manifold Approximation and Projection (UMAP)

We implemented a UMAP dimensionality reduction technique to analyze the multiplex, coordinately-mapped, single-cell data. A per-cell feature vector based solely on spatial information was encoded for every cell and input into the UMAP. Specifically, the cell density of each of the 5 cell lineages (Tumor, CD163⁺, CD8⁺, FoxP3⁺, and ‘Other’) were determined within 5 concentric spatial bounds (0–25, 26–50, 51–100, 101–150, 151–200 μm) around each individual cell. This resulted in a feature vector of 25 densities [5 (cell types) × 5 (spatial bounds)] for each cell, of which we used 2,500 cells per specimen to train the UMAP embedding. This algorithm was then applied to an independent set of 2,500 cells per specimen for visualization. Information from the ‘expression’ markers (PD-1 and PD-L1) was not used to generate the UMAP. The intensity of PD-1 and PD-L1 expression was overlaid onto the UMAP landscape after it was generated to evaluate for potential spatial encoding.

Survival analyses

Clinical variables were studied, including the age of the patient, tumor stage, the size of the largest tumor deposit, and whether extracapsular extension was present if the tumor was in a lymph node. In addition, n=552 potential mIF variables were identified, including density for each cell lineage, density of cells expressing PD-1 and PD-L1, and permutations of the densities of each of these cell phenotypes within 25 μm of another cell phenotype. Representative variables included: density of PD-L1⁺CD163⁺ cells within 25μm of a CD8⁺ cell; or any PD-L1⁺ cell within 25 μm of a PD1⁺ cell.

To determine the clinical and mIF features most closely associated with 5-year overall survival, both supervised and unsupervised approaches were used. The supervised approach was performed using R/Bioconductor software (v.4.0.3), and variable selection was performed two ways. In the first, univariate Cox regression analysis was used to estimate the p-value of an association of each continuous variable with 5-year survival using the coxph function from the survival package (v.3.2–7) (10). In the second approach, least absolute shrinkage and selection operator (LASSO) was used to reduce closely-related covariates. To find the optimal threshold for variable selection, we performed 5-fold cross-validation using the glmnet package (v. 4.1) (11).

The unsupervised approach for identification of features associated with survival was performed using spatial UMAPs. The single-cell data from survivors and non-survivors at the 5-year time point were aggregated by outcome groups. Differences between the two resultant spatial UMAP average bin densities were used to identify cell population with positive and negative prognostic impact.

Statistics and data and code availability

Correlation between cell densities obtained by image cytometry and inForm platforms were evaluated with simple linear regression. Comparisons between PD-1 and PD-L1 MFI according to topographic variables, e.g., CD8⁺ proximity used Mann-Whitney (two variables) or Kruskal-Wallis (more than two variables) tests.

The data and spatial UMAP code are available at https://www.sciserver.org and https://github.com/BarasLab/SpatialUMAP, respectively. The supervised survival analysis code is available in Supplementary Computational Methods.

RESULTS

Image cytometry

We compared the densities of CD8⁺, CD163⁺, FoxP3⁺, PD-1⁺, PD-L1⁺, and tumor cells using flow cytometry software to those obtained through the current gold standard, i.e., the phenotyping software associated with the microscope platform, and showed a strong correlation for all markers (R=0.79 or above) (Figure 1C). We noted that image cytometry had the added benefit of a 5-fold reduction in run-time.

Spatial UMAP

We used the cell lineage and X,Y coordinate data generated using image cytometry to create spatial UMAPs. These allowed us to cluster and visualize the topographic data and identify cells embedded within similar tumor immune ‘neighborhoods’ (Figure 2A). Distinct neighborhoods were delineated when the densities of different cell lineages were highlighted within the UMAP (lineage-specific subset UMAPs). They included those composed predominantly of tumor cells (immune-silent or excluded areas), those enriched for CD8⁺ cells and FoxP3⁺ cells (T-cell ‘inflamed’ microenvironment), and those enriched for CD163⁺ and ‘Other’ cells (myeloid-predominant or stromal cell rich) (Figure 2B). Although tumors as a whole have previously been characterized as immune-rich or immune excluded(12), we were able to identify these motifs on a regional scale within individual tumors.

Figure 2. Spatial UMAP shows distinct tumor immune neighborhoods.

(A) The spatial UMAP was generated using a per cell feature vector based solely on spatial information, i.e., not including the per cell MFI information for PD-1 or PD-L1. Specifically, the cell density of each of the 5 cell lineages (tumor, CD163⁺, CD8⁺, FoxP3⁺, and ‘Other’) were determined within 5 concentric spatial bounds (0–25, 26–50, 51–100, 101–150, 151–200 μm) around each individual cell. This resulted in a feature vector of 25 densities [5 (cell types) x 5 (spatial bounds)] for each cell, which were then used to generate a master spatial UMAP. In total, n=93 tumor specimens were included. (B) Top panel: Lineage-specific, subset UMAPs were also generated, showing cell densities for each lineage are arranged by distinct spatial neighborhoods. Bottom panel: Photomicrograph of mIF high power field showing three CD8⁺ cells with different expression patterns which map to three different regions on the UMAP, signifying the different neighborhoods the cells reside within. The yellow box shows a CD8⁺ cell surrounded by numerous other CD8⁺ cells and thus maps to the CD8⁺ dense region of the spatial UMAP (yellow star). The red box and orange box show CD8⁺ cells that are surrounded by CD163⁺ macrophages and tumor cells, respectively, and which map to those regions on the UMAP accordingly (red star, orange star).

Next, we overlaid the expression intensity of PD-1 and PD-L1 on the lineage-specific subset UMAPs, revealing spatial encoding of PD-L1 and PD-1 expression (Figure 3A). The highest intensity PD-L1 expression was observed on CD163⁺ cells in areas with dense CD8⁺ infiltrates. PD-L1 expression was also observed on all other lineages (tumor, CD8⁺, FoxP3⁺, and ‘Other’ cells) in CD8⁺-dense neighborhoods. A cluster of PD-L1^–CD163⁺ cells in CD8-poor neighborhoods was also evident, further supporting adaptive (IFNγ-driven) PD-L1 expression (13). The highest PD-1 expression was observed on CD8⁺ cells in tumor cell–rich spatial clusters with low-level PD-L1 expression, consistent with a more exhausted T-cell phenotype following prolonged antigen exposure (14). We then corroborated the PD-L1 and PD-1 spatial cluster UMAP findings. First, we tested the MFI of PD-L1 at different distances from CD8⁺ cells, and we found that both PD-L1 expression intensity and proportion of PD-L1⁺ tumor cells, CD163⁺ macrophages, and ‘Other’ cells increased with proximity to CD8⁺ cells (Figure 3B and Supplementary Figure S2). Next, CD8⁺PD-1⁺ cells were tested for PD-1 expression intensity as a function of the density of tumor cells in the immediate neighborhood (Figure 3C). This analysis confirmed the UMAP finding that PD-1 intensity was greater when CD8⁺PD-1⁺ cells were surrounded by a higher density of tumor cells.

Figure 3. Spatial UMAP shows spatially-encoded PD-1 and PD-L1 expression patterns.

(A) The PD-L1 MFI (top row) and PD-1 MFI (bottom row) were superimposed on the lineage-specific, subset UMAPs, highlighting the distinct spatial neighborhoods where these markers were preferentially expressed. In total, n=93 tumor specimens were included. The highest PD-L1 expression intensity is observed on CD163⁺ cells in neighborhoods with high CD8⁺-cell density. The highest PD-1 expression intensity is observed on CD8⁺ cells in neighborhoods with dense arrangements of tumor cells. (B) The fraction of CD163⁺ macrophages displaying PD-L1 (median and 95% confidence interval bars) as well as the intensity of PD-L1 expression (boxplot displaying the median, interquartile range, minimum and maximum values) increased with proximity to a CD8⁺ cell, confirming the spatial UMAP. Similar findings were observed for tumor cells and ‘Other’ cells (Supplementary Figure S2). (C) An increased proportion of CD8⁺ cells displayed PD-1 (median and 95% confidence interval bars) expression when these cells were surrounded by a higher number of tumor cells i.e., <5 vs. ≥5 cells within 25μm. Further, the PD-1 expression intensity (boxplot displaying the median, interquartile range, minimum and maximum values) was also increased with increasing tumor cell density. *, p≤0.05; **, p≤0.01; ****, p≤0.0001 by Kruskal-Wallis.

Survival analysis

The spatial UMAP was apportioned into survivors and non-survivors at the 5-year time point, and the differences between their average bin densities were used to identify spatial immune clusters with positive or negative prognostic impact (Figure 4A and 4B). Spatial arrangements where tumor cells were adjacent to high CD8⁺ or PD-1⁺ cell densities and/or high densities of CD8⁺ or PD-1⁺ cells in close proximity to PD-L1⁺ cells, i.e. T-cell ‘inflamed’ tumors, were enriched in long-term survivors. In contrast, spatial neighborhoods with a high density of CD163⁺ cells lacking PD-L1 expression, especially those in close proximity to another macrophage, were enriched in non-survivors. We confirmed the prognostic impact of the spatial clusters by performing a univariate Cox linear regression analysis on the configuration for each neighborhood, e.g., density of CD8⁺cells ≤25um from any PD-L1⁺ cell (Figure 4B and Supplementary Figure S3). We next confirmed that the variables indicated by the spatial UMAP analysis were the same as those that would be identified using a conventional approach to univariate Cox regression analysis coupled with variable prioritization. Specifically, we generated more than 550 possible variables for the 6-plex mIF panel, representing different cell densities, co-expression, and supervised spatial metrics and used two different approaches to variable prioritization. Variables identified using the lowest p-values from Cox regression modelling and those selected by LASSO algorithm both overlapped with those highlighted by the spatial UMAP (Supplementary Tables S3 and S4). Clinical variables such as patient age, tumor stage, largest tumor deposit size, and presence of extracapsular extension were included in these models but did not emerge as key variables.

Figure 4. The spatial UMAP can be used to identify spatial signatures associated with prognosis.

(A) Subset UMAPs were created for survivors and non-survivors at 5 years (n=93 individual samples included, single experiment). (B) Differences between the densities were used to identify spatial immune clusters with positive or negative prognostic impact. Numerous spatial clusters associated with clinical outcome were identified by this method, and four of the most prominent ones are highlighted (Supplementary Figure S3). The variables represented in these clusters were tested separately using univariate Cox regression analysis. The variables identified by the unsupervised UMAP analysis overlapped with those identified using supervised approaches (Supplementary Tables S3 and S4).

DISCUSSION

Flow cytometry is widely used by the scientific community and is known for its ability to enable rapid and robust multi-parametric analysis of cell protein expression. Here, we present an efficient image cytometry approach for multispectral mIF image analysis that allows robust MFI-based quantification of immune populations and maintains spatial information at a single cell-level. Previous studies have reported flow cytometry–like dot plots and associated gating strategies for the quantification of specific cellular subsets in slides stained with multiplex chromogenic immunohistochemistry (15), 3-color standard IF (16), and confocal microscopy (17,18). These earlier studies did not benchmark their method against established digital pathology image analysis strategies. Moreover, multispectral mIF technology is more amenable to image cytometry analysis than some of these other technologies due to scalability as well as the potential to achieve a high signal-to-noise ratio and a large dynamic range (1). Here, we describe a method comparable to current state-of-the-art digital pathology software. This approach also has the benefit of facilitating dynamic thresholding and the potential integration of compensation tools to correct for marker bleedthrough between channels.

Spatially-annotated, multiplex datasets are relatively new to the field of immuno-oncology. So far, efforts to explore this type of data have used fairly restricted characterizations of spatial encoding in the microenvironment (arbitrary cell-to-cell distance cut-offs, so-called proximity analysis, nearest neighbor analysis) (3,4,19–21). More comprehensive approaches are needed to unravel more subtle and complex correlations. The spatial UMAP dimensionality reduction approach allows for unsupervised clustering of multifaceted spatial arrangements within the tumor microenvironment, including those associated with clinical outcome. Given that the clusters shown here are generated based on lineage markers arrangements only, the fact that PD-1 and PD-L1 expression localizes to select spatial UMAP regions indicates spatially-encoded expression. Further, the expression levels of these markers, e.g. PD-1^low vs. PD-1^high, also appears to be spatially encoded, which is of interest as such levels are known to have functional relevance (14). Future studies with spatial UMAPs may explore larger tumor surface areas, e.g. whole slides stained and imaged with multispectral mIF (9), higher resolution distance metrics (22), and comparisons between lesions from different metastatic and anatomic sites (23,24).

In summary, our work shows that the use of image cytometry provides a robust, efficient, and reproducible platform to quantify large data sets of mIF-stained tumors. Furthermore, it also supports the use of spatial UMAPs for improved interpretation of mIF data, including the discovery of relevant cell-to-cell interactions and orchestrated marker expression. Although we focused on multispectral mIF output here, our pipeline could be used with any type of high-dimensionality multiplex imaging data, e.g. imaging mass cytometry or multiplexed ion beam imaging (MIBI) (25).

SYNOPSIS.

The authors developed and validated a multiplex immunofluorescence data-analysis pipeline and spatial UMAP algorithm. These user-friendly tools enable the quantification and visualization of spatial arrangements of immune populations in the tumor microenvironment and associated immuno-oncology topographic biomarker development.