A Beginner’s Guide To Analyzing and Visualizing Mass Cytometry Data

Abigail K Kimball; Lauren M Oko; Bonnie L Bullock; Raphael A Nemenoff; Linda F van Dyk; Eric T Clambey

doi:10.4049/jimmunol.1701494

. Author manuscript; available in PMC: 2019 Jan 1.

Published in final edited form as: J Immunol. 2018 Jan 1;200(1):3–22. doi: 10.4049/jimmunol.1701494

A Beginner’s Guide To Analyzing and Visualizing Mass Cytometry Data

Abigail K Kimball ¹, Lauren M Oko ², Bonnie L Bullock ³, Raphael A Nemenoff ³, Linda F van Dyk ², Eric T Clambey ^1,^*

PMCID: PMC5765874 NIHMSID: NIHMS918275 PMID: 29255085

Abstract

Mass cytometry has revolutionized the study of cellular and phenotypic diversity, significantly expanding the number of phenotypic and functional characteristics that can be measured at the single-cell level. This high-dimensional analysis platform has necessitated the development of new data analysis approaches. Many of these algorithms circumvent traditional approaches used in flow cytometric analysis, fundamentally changing the way these data are analyzed and interpreted. For the beginner, however, the large number of algorithms that have been developed, and the lack of consensus on best practices for analyzing these data raise multiple questions: Which algorithm is the best for analyzing a dataset? How do different algorithms compare? How can one move beyond data visualization to gain new biological insights? Here, we describe our experiences as recent adopters of mass cytometry. By analyzing a single dataset using five CyTOF analysis platforms (viSNE, SPADE, X-shift, PhenoGraph and Citrus), we identify: i) important considerations and challenges that users should be aware of when using these different methods, and ii) common and unique insights that can be revealed by these different methods. By providing annotated workflow and figures, these analyses present a practical guide for investigators analyzing high-dimensional datasets. In total, these analyses emphasize the benefits of integrating multiple CyTOF analysis algorithms to gain complementary insights into these high-dimensional datasets.

INTRODUCTION

Since its inception, mass cytometry, or cytometry by time-of-flight (CyTOF), has allowed researchers to gain deep insights into cellular phenotype and function (1–3). The technology allows simultaneously quantification of >30 cellular parameters, and when integrated with high-dimensional analysis algorithms, has the potential to reveal extraordinary cellular diversity and heterogeneity (2, 4, 5). Many algorithms and software kits have been developed to facilitate analysis of CyTOF datasets, including, but not limited to: SPADE (6), viSNE (7), Wanderlust (8), FlowSOM (9), PhenoGraph (10), Citrus (11), Scaffold (12), X-shift (13), and DensVM (14). These tools are typically developed by computational biologists or by laboratories that are leaders in the field of mass cytometry, using a variety of languages (R, Matlab, Java, Python, etc), clustering methods (parametric, non-parametric, etc.), and dimensionality reduction approaches (15–17). CyTOF data visualization and quantitation continues to be a rapidly evolving field (e.g. (18, 19)).

Despite the potential of mass cytometry, there remain multiple challenges to its widespread implementation, from instrument and reagent costs to determining optimal ways to visualize and quantify these high-dimensional data. For researchers with little to no computational background, entry into these data can represent a significant challenge. Even for laboratories well-versed in multiparameter flow cytometry, analyzing mass cytometry data requires a major shift in how to approach these data, moving away from user-defined, Boolean gating strategies to automated identification of cell clusters and phenotypes. While biological knowledge remains essential to interpret these high-dimensional data, understanding which tools to use when addressing specific biological questions remains a major challenge.

Here we present a practical guide for CyTOF data analysis, a process we have defined empirically as recent adopters of these methods. First, we provide a detailed commentary on how to implement and interpret data using five established, and widely used, CyTOF analysis platforms (viSNE, SPADE, X-shift, Citrus and PhenoGraph). Second, we supply annotated examples, illustrating how to use these different algorithms to gain complementary insights into a single dataset. These analyses provide both a conceptual framework, and a resource, for investigators interested in using CyTOF to gain high-dimensional insights into biological questions.

MATERIALS AND METHODS

Experimental samples

Samples were obtained from two sources. First, 13-week old female C57BL/6J (B6, n=5) or IL-10 deficient (IL10KO, n=4; B6.129P2-Il10^tm1Cgn/J, Jackson Laboratory) mice were infected with murine gammaherpesvirus 68 (γHV68) by intranasal infection using 4×10⁵ plaque forming units (PFU) of wild-type virus containing a gene 73-beta lactamase fusion protein (γHV68.ORF73βla) (as in (20)). Mice were euthanized and lungs harvested at nine-days post-infection, followed by perfusion with phosphate buffer saline. Second, UBI.GFP (C57BL/6-Tg(UBC-GFP)30Scha/J) mice were orthotopically injected in the left lobe of the lung with 1×10⁵ cells suspended in Hank’s Buffered Salt Solution (HBSS) containing 1.35 mg/mL Matrigel (Corning #354234). Mice received either firefly luciferase expressing CMT167 cells (21) or LLC cells (LL/2-luc-M38 luciferase expressing cells, Caliper Life Sciences). Both cell lines were routinely tested for mycoplasma and were confirmed negative before orthotopic injection. All surgeries were performed under inhaled isoflurane anesthesia. A 4-5mm incision was made in the skin along the left shoulder and subcutaneous fat was removed to completely visualize the left lung, as previously described (22). Following orthotopic injection, incisions were closed using veterinary-grade skin adhesive. Mice were euthanized at 2.5 (LLC) or 3.5 weeks (CMT167) post-injection, followed by perfusion of the circulation with PBS/Heparin (20 U/mL), and collection of tumor-containing left lobe for analysis. Pools of two mice were used for LLC tumors, pools of 3-5 mice for CMT167 tumors, and both left and right lung lobes were used for uninjected controls. All procedures were performed under protocols approved by the Institutional Animal Care and Use Committee at the University of Colorado Anschutz Medical Campus.

Cell processing & antibody staining

Lungs were minced and enzymatically digested using collagenase D (from Clostridium histolyticum; Roche) for 1 h at 37°C (for virus-infected lungs) or incubated at 37 C for 30 min in HBSS containing collagenase type 4 (8480 U/mL), elastase (7.5 mg/mL), and soybean trypsin inhibitor (2 mg/mL) (all from Worthington Industries) (for tumor-containing lungs). Lungs were then mechanically disrupted to single-cell suspensions, subjected to red blood cell lysis, washed and resuspended for staining. Cell suspensions were stained with cisplatin (Fluidigm), incubated with Fc receptor blocking antibody (2.4G2, Tonbo Biosciences) for 20 minutes, following by addition of primary surface antibodies incubated for 15 minutes at 37 C and 15 minutes at 22 C. Secondary surface stains were done for 20 minutes, with intracellular antigen staining done using the eBioscience FoxP3 Fix/Perm buffer and 2 hour stain at 4 C. Following cell staining, cells were washed and resuspended in Intercalator (Cell-ID™ Intercalator-Ir, Fluidigm). Antibodies are indicated in Supplemental Table 1 and 2 and were from Fluidigm unless noted otherwise.

CyTOF run and sample normalization

Samples were collected on a Helios mass cytometer (Fluidigm), with samples resuspended with equilibration beads spiked into each sample to allow for signal normalization. Samples were normalized using NormalizerR2013b_MacOSX, downloaded from the Nolan lab github (https://github.com/nolanlab). Normalized data were subjected to traditional Boolean gating in FlowJo, identifying singlets (¹⁹¹Ir+ ¹⁹³Ir+) that were viable (¹⁹⁵Pt−). These events were then gated and exported for downstream analysis (Supplemental Fig. 1, 2).

viSNE

Algorithm settings

Manually gated singlet (¹⁹¹Ir+ ¹⁹³Ir+), viable (¹⁹⁵Pt−) events from γHV68-infected or tumor-containing mice were imported into Cytobank (www.cytobank.org), then subjected to viSNE analysis. viSNE clustering analysis was performed on 35 of 61 possible parameters, focused on antibodies (Supplemental Table 1). Equal event sampling was selected, using 9,141 events per individual (the lowest common denominator across all samples) for a total of 82,269 events across all 9 virus-infected samples.

Algorithm optimization, interrogation, and visualization

viSNE plots for each individual across all 35 parameters were downloaded from Cytobank with plots arranged in a grid format in Adobe Illustrator CC 2017. Cellular phenotypes were assigned to the viSNE plot based on distribution and expression characteristics using phenotypic markers, with viSNE overlays manually generated in Adobe Illustrator.

Investigating cellular abundance

CD4+ T cells were manually gated in viSNE plots within Cytobank, to quantify the percent of total events for all individuals.

Investigating cellular expression

CD4+ T cells were manually gated in viSNE plots, analyzed for expression across all 35 parameters, and downloaded from Cytobank. Expression profiles were visually inspected between conditions, with differences in expression between the conditions manually selected for further analysis using FlowJo.

PhenoGraph

Software downloads

The 1.0.136 version of R studio was downloaded from the official R website (http://www.r-project.org/). The 3.4 version of the Cytofkit package was downloaded from via Bioconductor (Bioconductor.org/packages/release/bioc/html/cytofkit.html) and opened in R.

Algorithm settings

Manually gated singlet (¹⁹¹Ir+ ¹⁹³Ir+), viable (¹⁹⁵Pt−) events from γHV68-infected mice were imported into Cytofkit, then subjected to PhenoGraph analysis, clustered based on 35 of 61 possible parameters (antibodies in Supplemental Table 1), with additional settings: 1) merge method: min, 2) transformation: cytofAsinh, 3) cluster method: Rphenograph, 4) visualization method: tSNE, and 5) cellular progression: NULL.

Algorithm optimization, interrogation, and visualization

All events were uploaded into PhenoGraph, which defined 29 subpopulations of cells or clusters. Clusters were displayed on tSNE plots within the R package ‘Shiny’ to visualize individuals and experimental groups. Cluster ID labels, dot size, and cluster color were customized, to indicate phenotype or to highlight specific clusters. Additional plots were colored according to the expression of cellular markers. Multiple .csv files were produced by the PhenoGraph analysis, including “cluster median data” and “cluster cell percentage”, which were used to determine cluster phenotype, distribution between conditions and statistical significance between groups.

Investigating cellular abundance

Clusters were initially visually inspected to identify potential differences in cellular abundances between conditions. Subsequently, all clusters were analyzed for statistical significance between experimental groups.

Investigating cellular expression

CD4+ T cell clusters were visually inspected for differences in expression between experimental groups.

X-shift

Software downloads

The 12/24/2016 version of VorteX was downloaded from the Nolan lab github page (https://github.com/nolanlab/vortex/releases/tag/24-Dec-2016) as well as the most recent version of Java (Version 8, Update 121, https://java.com/en/download/).

Algorithm settings

Manually gated singlet (¹⁹¹Ir+ ¹⁹³Ir+), viable (¹⁹⁵Pt−) events from virus-infected or tumor-containing lungs were uploaded into the VorteX clustering environment. The following importation settings were used: i) Minimal Euclidean length of the profile: 1.0, ii) Import max.: 10,000 events, iii) Merge all files into one dataset. These settings were either automatic or recommended by the github page. 35 parameters were selected for clustering (Supplemental Table 1). After the data were imported, before clustering, we validated the size and dimensions of the anticipated dataset. The following clustering settings were used: i) Numerical transformation: arcsinh(x/f), f=5.0, ii) noise threshold: apply noise threshold of 1.0, iii) feature rescaling: none, iv) normalization: none, v) distance measure: angular distance, vi) clustering algorithm: X-shift (gradient assignment), vii) density estimate: N nearest neighbors (fast), viii) number of neighbors for density estimate (K): from 150 to 5, with 30 steps, and ix) number of neighbors for mode finding (N): determine automatically. These clustering settings were either automatic or recommended by the github page. Following clustering, all 30 sets were selected and the K-value of the switch-point, or the elbow point, between the linear and exponential phase was calculated (here, K=20, which corresponded to 45 X-shift defined clusters within the virus-infected dataset).

Algorithm optimization, interrogation, and visualization

All 45 clusters in the K=20 cluster set were selected and a force-directed layout was created with the following settings: i) Maximum number of events sampled from each cluster: 20, ii) Sample proportionally to the power of the cluster size: unchecked, iii) Distance measurement: Angular distance, iv) Number of nearest neighbors: 10, v) Edge settings: uncheck the options to “Vary the number of node connections by node density” and “Limit connections the events within parameter range”, vi) Sample selection: select the option “Include cells from certain annotation groups” and select all nine samples. This layout was downloaded as a graphml file and opened in Gephi v 0.9.1 where color and node size was optimized. This file was then saved as a pdf and opened in Adobe Illustrator CC 2017. The clusters were also colored according to phenotype designation in Gephi.

Investigating cellular abundance

All events from each cluster were downloaded from VorteX and the frequency of events from different experimental groups (e.g. B6 vs. IL10KO) was determined in Excel, followed by manually adding these plots to the force-directed layout. These data were further interrogated to quantify the frequency of events derived from each individual mouse that contributed to the cluster.

Investigating cellular expression

We visualized cellular expression across all parameters by phenotypic barcodes within VorteX, to analyze both average expression across a cluster and expression across individual events. This feature combined with line graphs of median expression allowed definition of a core cellular phenotype for all clusters (e.g. CD4+ or CD8+ T cell), with accessory phenotypes present in only a subset of cells. Core and accessory phenotypes were depicted using manually-generated infographics.

SPADE

Algorithm settings

Manually gated singlet (¹⁹¹Ir+ ¹⁹³Ir+), viable (¹⁹⁵Pt−) events from γHV68-infected mice were imported into Cytobank, then subjected to SPADE analysis using the following settings: i) target number of nodes = 200 and ii) percent downsampling = 10% (both Cytobank default values). Single, viable cells were selected for analysis, using 35 of 61 possible parameters (antibodies in Supplemental Table 1), and no fold change calculations were made for this data set.

Algorithm optimization, interrogation, and visualization

A SPADE tree was manually rearranged online at Cytobank.org. Additional SPADE analyses were run with various alterations: i) the target number of nodes was adjusted to an X-shift informed 45 or 89 nodes, and ii) the number of clustering channels was decreased to only include 10 lineage markers. After these analyses, the SPADE tree informed by VorteX with all 35 parameters was chosen as most appropriate for this analysis. SPADE trees for each individual across all 35 parameters were downloaded from Cytobank and most relevant markers were arranged in a table via Adobe Illustrator. Basic cellular phenotypes were assigned to the SPADE trees, based on node expression and location.

Investigating cellular abundance

SPADE trees were initially visually investigated to identify nodes that were potentially different in cellular abundance between conditions. Data accompanying each SPADE tree was downloaded from Cytobank to calculate frequency of events in each node, followed by testing for statistical significance. The identification of significant nodes on the SPADE tree was determined online at Cytobank.org by hovering over the desired node.

Investigating cellular expression

Nodes of interest were colored by cellular markers that had been identified by previous algorithms to change in expression between experimental groups. A change in color was visually assessed by the user and nodes #1 and #26 were taken from their original downloaded SPADE trees. Node #1 was selected for each individual on Cytobank and the median expression value for five markers was recorded and tested for statistical significance.

Citrus

Algorithm settings

Manually gated singlet (¹⁹¹Ir+ ¹⁹³Ir+), viable (¹⁹⁵Pt−) events from γHV68-infected mice were imported into Cytobank, then subjected to Citrus analysis using the following settings: 1) Singlet, viable cells were chosen as input, 2) 35 of 61 possible parameters were selected (antibodies outlined in Supplemental Table 1), 3) the files were assigned to their appropriate experimental groups (i.e. B6 vs. IL10KO), 4) the Nearest Shrunken Centroid (PAMR) association model, 5) cluster characterization of abundance, 6) equal event sampling, 7) events sampled per file: 9141 events, 8) minimum cluster size: 5% (Cytobank default), 9) cross validation folds: 5 (Cytobank default), 10) a false discovery rate: 1% (Cytobank default).

Algorithm optimization, interrogation, and visualization

To test the sensitivity of Citrus to different variables, additional Citrus analyses were run to test the impact of: 1) random assignment of samples to different groups, 2) input cell number and 3) reducing the minimum cluster size. Analysis primarily focused on resulting “Model Error Rate” models. The Citrus run with 0% cross-validation error rate was selected as the most appropriate predictive model, with results were downloaded from Cytobank for further visual and statistical analysis. The vertical hierarchical tree was manually generated based on the radial hierarchical tree in Adobe Illustrator.

Investigating cellular abundance

Daughters identified as differing in abundance between B6 and IL10KO mice by the Min and SE statistical models were analyzed further. The direction of this change in abundance was determined via the results downloaded from Citrus, with specific p-values determined.

Investigating cellular expression

To query changes in cellular expression, another Citrus analysis was preformed, with identical settings (see above), except to examine changes in median expression rather than abundance.

Comparing Algorithms

Plots/force directed layouts/trees from each algorithm are colored according to the 8 lineage markers used to identify cellular phenotypes. The % of events in all clusters/nodes from four algorithms were identified and combined according to cellular phenotypes. The number of unique clusters/nodes within each phenotypic group was determined for four algorithms. The median expression values for particular daughters/nodes/clusters, which appear to contain similar cellular events, have been plotted as a line graph to determine phenotypic similarity and dissidence. A network was created in Adobe Illustrator where markers identified as positive within each algorithm were plotted. For this network map, positive expression for a parameter was defined as any value greater than the average expression across daughters/nodes/clusters within an algorithm.

Investigating population structure

Population structure was analyzed by the stratifying algorithms, SPADE, X-shift, and PhenoGraph. Population structure was visualized by downloading the frequencies of all nodes/clusters across experimental conditions. These data were then visualized using: 1) a vertical hierarchy plot, in which node frequency was arranged in ascending order for each experimental group, identifying nodes that significantly changed in abundance between conditions, or 2) the frequency and subset distribution of different cellular phenotypes across experimental groups. Lastly, clustering relationships were visualized in Cytofkit by dendrograms, depicting the relatedness between clusters and changing relationships between conditions. The following settings were used for dendrogram generation within the shiny app. The "Expression Heat Map’ tab was selected, Heatmap Dendrogram: row, Color Palette: spectral2, Heatmap Type: median, Scale Data: none. The dendrogram was downloaded as a PDF file and opened and edited in Adobe Illustrator.

Software used & Statistical analysis

Software for data analysis included: R studio (Version 1.0.136), downloaded from the official R website (http://www.r-project.org/); the Cytofkit package (Version 3.4), downloaded from via Bioconductor (Bioconductor.org/packages/release/bioc/html/cytofkit.html); VorteX (Version 12/24/2016), downloaded from the Nolan lab github page (https://github.com/nolanlab/vortex/releases/tag/24-Dec-2016), Java (Version 8.121, https://java.com/en/download/), Gephi v 0.9.1, Excel 15.13.14, FlowJo 10.2, GraphPad Prism 7, and Adobe Illustrator CC 2017. For Figure 7F–H, a more recent version of VorteX was used (VorteX 21-Apr-2017). This led to the discovery of 32 clusters despite using identical settings to Figure 3. Statistical significance was tested in GraphPad Prism using an unpaired t test comparing B6 and IL10KO mice, with statistical significance as identified. For situations in which we tested statistical significance for all identified nodes/clusters (Fig. 8), analysis was corrected for multiple comparisons by multiplying the individual p values for each comparison by the number of statistical tests performed.

CyTOF analysis of γHV68-infected lungs from either B6 or IL10KO individuals harvested at 9 days post-infection. (A) Direct cross-comparison of data visualization across multiple phenotypic markers (rows), comparing different algorithms (columns), including plots shown in previous figures. (B) Quantitation of the percent of events identified as different cell phenotypes, comparing viSNE/Boolean gating, SPADE, X-shift and PhenoGraph. (C) Comparison of the number of clusters/nodes identified according to each phenotype, comparing X-shift, PhenoGraph and SPADE. (D-E) Comparison of CD4 T cell clusters/nodes identified as significantly increased in IL10KO mice, depicting (D) median expression values or (E) a phenotype network identifying parameters that were positive (defined as higher than the average expression for all events). (F-H) Analysis of the impact of input cell number per sample on data visualization across algorithms, depicting (F) data visualization, (G) cluster number (in X-shift and PhenoGraph), and (H) distribution of cluster frequencies (in PhenoGraph). Data from virus-infected lungs (B6, n=5; IL10KO, n=4 mice).

Input settings (A) and PhenoGraph data visualization (B-G), focused on CyTOF analysis of γHV68-infected lungs from either B6 or IL10KO individuals at 9 days post-infection. Data show all viable single cells, subjected to PhenoGraph in Cytofkit, which calculates the optimal amount of clusters, with data plotted on a tSNE plot. (A) Input settings to run the PhenoGraph algorithm in Cytofkit. (B,C) PhenoGraph-defined cellular distribution and clustering as defined by tSNE1 and tSNE2, colored by cluster for compiled B6 or IL10KO samples (B) or for individual mice (C). (D) PhenoGraph-based visualization on a tSNE plot, colored according to expression of lineage markers demonstrates cell clustering and varied scaling. (E,F) PhenoGraph visualization with clusters colored by phenotype in either compiled B6 or IL10KO samples (E) or for individual mice (F), with cell populations defined based on basic phenotypic markers according to the key. (G) Comparison of three sequential PhenoGraph runs, in which the exact same 9,141 cells (from individual B6 #3) were subjected to PhenoGraph, visualized by CD45 (left 3 plots) or by cluster ID (right 3 plots). Numbers identify the physical location of PhenoGraph-defined clusters. Data from virus-infected lungs (B6, n=5; IL10KO, n=4 mice).

CyTOF analysis of γHV68-infected lungs from either B6 or IL10KO individuals harvested at 9 days post-infection. Data show all viable single cells, subjected to the various algorithms, and include examples demonstrating insights obtained across algorithms. (A-B) viSNE analysis showing viSNE plots for individuals B6 #1 and IL10KO #1 colored according to CD4 expression. The CD4+ T cell island was visually identified as one notable change between B6 and IL10KO mice, with this population manually gated within Cytobank, to (B) define the frequency of CD4 T cells across all individuals. (C-D) PhenoGraph analysis of B6 and IL10KO mice identified 29 clusters, colored by cluster ID and plotted according to tSNE1, 2, identified multiple clusters with apparent changes between groups. (D) Statistical analysis of all PhenoGraph defined clusters identified 2 of 29 clusters that were statistically significantly different between B6 and IL10KO mice. (E-F) X-shift analysis of B6 and IL10KO mice identified 45 clusters of cells, with each cluster comprised of different frequencies of cells from either B6 or IL10KO individuals (E). (F) The proportion of events contributed by individual mice within the top 6 most enriched clusters in either B6 or IL10KO mice identified both statistically significant clusters and clusters prominently driven by a single individual. (G-I) SPADE analysis, including a comparison of SPADE trees for B6 #1 and IL10KO#1 mice identified nodes that appear visually discrepant between these groups (G). (H) Analysis of node frequency across individuals identified 7 nodes that were statistically significantly increased in IL10KO mice, with statistically significant nodes identified on SPADE trees for B6#1 and IL10KO#1 (I). (J-L) Citrus analysis of cluster abundance identified multiple clusters that are either B6 or IL10KO biased (J), with analysis of cluster frequency across individuals (K). (L) The cellular phenotype of cluster #82243, defined by analysis of selected, Citrus-generated histogram overlays. Individual symbols on all plots identify values from individual mice. All data from optimized algorithm settings in Figs. 2–6. Statistical analysis was performed using unpaired t-test, with statistical analyses subjected to multiple testing correction and statistical significance identified as follows: p<0.05 (*), p<0.01 (**), p<0.001 (***), p<0.0001 (****).

RESULTS

Conceptual framework for CyTOF analysis

CyTOF studies can interrogate a wide range of scientific questions, from focused, deep phenotypic analysis of a specific cell subset to broad studies of cellular heterogeneity in complex cell populations (Fig. 1A). These datasets contain a wealth of data: i) cellular abundance and ii) expression profiles (akin to flow cytometry studies), iii) identification and quantification of cellular diversity, and iv) changes in population structure, defined by changes in the relative frequencies of cellular populations and phenotypic subsets across experimental conditions (Fig. 1A). While conventional flow cytometric algorithms and Boolean gating can give insights into some of these data (e.g. cellular abundance and expression), deeper data analysis quickly becomes inaccessible with >30 dimensions. Here we consider five CyTOF analysis algorithms that use computationally distinct methods, require different levels of computational skills, and significantly vary in data visualization (Fig. 1B).

Examples of the type of research questions that can be answered by CyTOF (A), and considerations in the use of different CyTOF algorithms (B).

One major distinction between the CyTOF algorithms considered here is whether cells are displayed as a continuum of phenotypes (e.g. viSNE), or stratified into subpopulations, to quantify population structure (e.g. PhenoGraph, X-shift) (Fig. 1B). These different types of data visualization and output significantly impact the data that can be retrieved. For example, changes in population structure are most readily obtained by algorithms that stratify all events into subpopulations (SPADE, PhenoGraph, X-shift) (Fig. 1B). Additional considerations of these algorithms are presented in Fig. 1B, with subsequent figures containing practical notes on how to use and to interpret each of these algorithms. These analyses primarily focus on a CyTOF dataset examining the immune response to murine gammaherpesvirus 68 (γHV68), a small animal model of herpesvirus infection (23). After sample collection, data were normalized (relative to equilibration beads) and viable, single cells were subjected to data analysis using the analysis platforms viSNE, SPADE, X-shift, PhenoGraph, and Citrus. For each method, we: 1) define basic considerations in the application and interpretation of data, and 2) provide an example of how these algorithms can provide insight.

viSNE visualization and insights

viSNE is a well-established CyTOF analysis tool that utilizes the t-distributed stochastic neighbor embedding (t-SNE) algorithm to analyze and display high-dimensional data on a two-dimensional map (7, 24). Practically, the resulting image shows a continuum of cellular phenotypes, distributed by the parameters tSNE1 and tSNE2, with cells colored according to expression of a chosen parameter (e.g. CD45). We used viSNE within Cytobank (www.Cytobank.org), a cloud-based computational platform, that requires three user-defined inputs: 1) sample selection (i.e. which samples are to be analyzed), 2) parameter selection (i.e. which phenotypic markers are to be used for the clustering analysis), and 3) event sampling (which can be done either by sampling proportionally or equally) (Fig. 2A). The resulting viSNE plot is a two-dimensional figure with the axes, tSNE1 and tSNE2, with cells plotted on a continuum of expression with phenotypically-related cells clustered together, often manifesting as phenotypic “islands”. Events can be colored according to any user-defined parameter (e.g. CD45) to identify the cellular identity of the island.

Input settings (A) and graphical representation of viSNE analysis using the Cytobank platform, with data representing CyTOF analysis of γHV68-infected lungs from either B6 or IL10KO individuals at 9 days post-infection (B-F) or B6 mice orthotopically implanted with the LLC tumor cell line (G). Data show all viable single cells, subjected to the t-distributed stochastic neighbor embedding (t-SNE) algorithm which provides each cell with a unique coordinate according to their expression of the 35 measured parameters, displayed on a two-dimensional plot (tSNE1 vs. tSNE2). (A) Input settings to run the viSNE algorithm in Cytobank. (B) Visualization grid of viSNE plots, with plots arranged according to marker expression (rows) relative to individuals (columns). (C) Identification of cellular populations identified by viSNE for individual B6 #1, with cell populations defined based on basic phenotypic markers (see Methods). (D) An additional viSNE plot produced using identical settings for individual B6 #1 and colored by CD45 demonstrating variable output of viSNE across independent runs, potentially reflecting variable viSNE calculation and events sampled. (E) Comparison of three sequential viSNE runs, in which the exact same 9,141 cells were subjected to viSNE, demonstrates variable cellular distribution (for individual B6 #3). (F) Reciprocal viSNE overlays comparing the topography of the viSNE plots from B6 and IL10KO mice. (G) viSNE analysis of cellular populations from a tumor-containing lung, with cell populations defined based on basic phenotypic markers. Data from virus-infected lungs (B6, n=5; IL10KO, n=4 mice), or from naïve (right and left lobes of lung pooled together from n=2 mice) and LLC-luc tumor-containing lung (left lobe of lung pooled from n=2 mice) per condition.

One of the first goals in interpreting a viSNE plot is to define where different cell types and phenotypes are located. This is defined empirically by the user, and heavily relies on the user’s biological knowledge. For a complex mixture of cells (e.g. virally-infected lung tissue), we find that assembling a grid of viSNE plots, organized by individual (in columns) and colored by different parameters (in rows), is particularly helpful (Fig. 2B). This grid allows: 1) rapid identification of cellular phenotypes across different islands, by visualizing where lineage markers are clustered (e.g. a B cell island, defined as CD45+ CD19+), 2) definition of inter-individual variation, with all events plotted according to a common axis of tSNE1 and tSNE2, and 3) insights into populations and cellular phenotypes that change between experimental groups (e.g. wild-type C57BL/6 (B6) versus IL-10 deficient mice (IL10KO), Fig. 2B). While viSNE plots can be annotated to identify the location of cell phenotypes, two independent viSNE runs on the same dataset will give two different plots which vary in terms of island location yet have relatively comparable island abundance and expression (Fig. 2C–D). Note that this variation between sequential viSNE runs in Cytobank is not simply a factor of variable events chosen during downsampling, but occurs even when the exact same cells are subjected to three sequential viSNE runs (Fig. 2E). Given inter-run variability, it is only possible to directly cross-compare between experimental groups when all samples are subjected to the same viSNE analysis run. This approach ensures that all events are plotted according to a common scale of tSNE1 and tSNE2, with potential differences between conditions identified by manually overlaying viSNE plots for two experimental conditions. In this case, a gap in island distribution between two conditions can indicate either an altered cell type or phenotypes (Fig. 2F). It is important to realize that there is no absolute location for any cell type in viSNE, as illustrated by a comparison of cellular distribution in a virus-infected lung versus a tumor-containing lung (Fig. 2G).

Investigating cellular abundance by viSNE

viSNE overlays comparing γHV68-infected B6 and IL10KO mice revealed multiple changes in the distribution of phenotypic islands, including changes in CD4+ T cells (Fig. 2F). Based on this visual difference, we next interrogated differences in cellular abundance of CD4 T cells between γHV68-infected B6 and IL10KO mice. First, we analyzed the frequency of CD3+ CD4+ events between B6 and IL10KO infected mice by manually gating the CD4+ T cell island on a viSNE plot (Fig. 8A). This analysis revealed that IL10KO mice had a reproducible, statistically significantly increase in the percentage of CD4 T cells relative to infected B6 mice (Fig. 8B).

Investigating cellular expression by viSNE

Another notable feature defined by viSNE was the altered spatial placement of CD4 T cells (according to tSNE1 and tSNE2) between these two groups, which suggested phenotypic differences between B6 and IL10KO CD4 T cells (Fig. 9A). By analyzing viSNE plots across multiple parameters, and looking for altered expression levels defined by varying intensities, IL10KO CD4 T cells appeared to have higher expression of multiple proteins including CD11b, CD11c, Gr-1, PD-L1 and Tbet (Fig. 9A). These findings were further corroborated by calculating median fluorescent intensities across individuals in FlowJo, a conventional flow cytometry analysis software (Fig. 9B). These data suggest heightened CD4 T cell activation in γHV68-infected IL10-deficient mice. This further exemplifies how integration of the viSNE analysis platform with conventional cytometric analyses can power new insights.

PhenoGraph visualization and insights

PhenoGraph is an algorithm that relies on graphs/networks of the recorded events and the connections between them to determine the accurate clustering of events into phenotypic categories (10). In contrast to viSNE, which portrays cells as a continuum of phenotypes, PhenoGraph stratifies all events into subpopulations (i.e. clusters), providing the user with a rapid quantification of population structure, defined as the diversity of cell types and phenotypes among all cells.

We ran the PhenoGraph algorithm in the R-based Cytofkit package, a software package that allows users to run a variety of different algorithms and data visualizations (25). We then used the Shiny application in R to examine, customize, and interrogate the results. Cytofkit requires 6 user-defined parameters, in which we selected tSNE as the visualization method, to allow cross-comparison with viSNE results (Fig. 3A). The resulting figure provides a two-dimensional figure plotted according to tSNE1 and tSNE2, like viSNE, with one notable addition: PhenoGraph stratifies all events into cell clusters, colored and numbered according to cluster identification number (Fig. 3B). For example, PhenoGraph identified 29 cell clusters that are present in varying frequencies between γHV68-infected B6 and IL10KO cohorts, plotted according to tSNE1 and tSNE2 (Fig. 3B). This visualization can examine a composite of all animals in a condition or interrogate individuals, allowing rapid cross-comparison between individuals (Fig. 3B–C). Similar to viSNE, the resulting plots can be colored according to expression levels for various markers (Fig. 3D), to assist in the identification of cell types and subtypes (Fig. 3E–F). In contrast to viSNE run in Cytobank, however, we found that sequential PhenoGraph runs on the exact same cells gave reproducible cellular distribution and cell cluster identification between runs (compare Fig. 2E and Fig. 3G).

Investigating cellular abundance by PhenoGraph

When we visually compared the frequencies of PhenoGraph-identified clusters between B6 and IL10KO individuals, clusters #4, #5, #8, #17, and #25 appeared to be dramatically different in cellular abundance (Fig. 8C). However, when we analyzed the frequencies of all clusters for statistical significance only cluster #8 (CD4 T cells) and cluster #25 (myeloid cells) were found to significantly increase in IL10KO mice (Fig. 8D). Clusters that did not reach statistical significance had high inter-individual variation particularly in B6 individuals.

Investigating cellular expression by PhenoGraph

To better understand CD4 T cell populations identified by PhenoGraph, we analyzed the frequency, distribution and phenotype of CD4+ T cell clusters between B6 and IL10KO mice. PhenoGraph defined six subsets of CD4+ T cells in virally-infected lungs (Fig. 9C). IL10KO mice had an increased frequency of CD4 T cells, with a pronounced shift in the frequencies of four different CD44^high effector T cell subsets (arbitrarily defined as effector types A-D, Fig. 9C). To determine how these CD4+ T cell effector subtypes differ phenotypically, and perhaps functionally, we visualized markers that appeared to have the greatest variation between CD4 T cell subsets (Fig. 9D). While B6 individuals had a high frequency of type A (ICOS⁺ CTLA4^+/− Lag3^+/− IRF4^mid) effector CD4 T cells, IL10KO infected mice had higher frequencies of type B (CD11b^mid CD11c^mid ICOS⁺ CTLA4⁺ Lag3⁺ IRF4^hi) and type C (CD11b^mid CD11c^mid ICOS^+/− CTLA4^+/− Lag3^negative IRF4^mid) effector CD4 T cells (Fig. 9D). These phenotypic subtypes of CD4 T cells were uniquely identifiable because of PhenoGraph-defined clustering and were not predicted based on current literature.

X-shift visualization and insights

The X-shift algorithm, run in the VorteX visual space, estimates the number of cell clusters in high-dimensional data by using weighted k-nearest-neighbor density estimation (13). To do this, X-shift defines the impact of different numbers of nearest neighbors on the number of cell clusters, with the optimal number of cell clusters defined to occur at the “switch point” between under-clustering and over-fragmenting data. Once optimized cell clustering is calculated for a dataset, these data can be visualized by a number of different methods. For purposes of this illustration, we show a force-directed layout in which cell clusters are physically clustered or separated based on relative similarity.

We ran the X-shift algorithm within VorteX. This algorithm has a high-degree of user-defined inputs, from dataset importation to clustering settings, prior to the generation of a graph that allows identification of the elbow/switch point, which is the optimized estimate of cell clusters (Fig. 4A–C). When we applied the X-shift algorithm to our virus infection data set, we calculated a K-value of 20 for the switch-point between the linear and exponential phase, which corresponded to 45 clusters (Fig. 4C–D). The force-directed layout allows further user customization that typically results in a relatively low resolution force-directed layout (Fig. 4E). This force-directed layout can be saved as a graphml file, opened in the Gephi app where the size and color of nodes and edges can be manipulated, and further edited in Adobe Illustrator. In this case, the force-directed layout was colored either by cluster identification number or by phenotype (Fig. 4F–G). Despite the high degree of computational effort required for running X-shift, it is notable how much data can be extracted from this algorithm (discussed below).

Input settings (A-E) and X-shift visualization (F-G), focused on CyTOF analysis of γHV68-infected lungs from either B6 or IL10KO individuals at 9 days post-infection. Data show all viable single cells, subjected to X-shift in the VorteX graphical environment, which calculates the optimal amount of clusters, with data plotted on a force-directed layout. Input settings to run X-shift for dataset import (A) and clustering settings (B). X-shift defined clustering is depicted as a function of the number (k) of nearest neighbors tested, which can be used to calculate elbow point (C-D). The boundaries of the linear phase, switch point, and exponential phase are indicated. (E-G) The force-directed layout curated by VorteX (E), and modified in Gephi (F-G), shows all 45 unique clusters identified at the K=20 switch point, colored by cluster ID (F) or by phenotype (G). Data from virus-infected lungs (B6, n=5; IL10KO, n=4 mice).

Investigating cellular abundance by X-shift

One notable feature of X-shift, and the VorteX graphical interface, is that it is possible to identify the individual events (i.e. cells) that contribute to each cluster. This information is incredibly useful, since different clusters are made up of variable amounts of cells derived from different individuals and experimental groups. To visualize this, we defined the percentage of each cluster that was from B6 or IL10KO mice and ranked cell clusters in virally-infected B6 and IL10KO mice, from most B6-biased, to most IL10KO-biased (Fig. 8E). This analysis identified some clusters that were equally present in both B6 and IL10KO mice (e.g. #514, #530), some B6-biased clusters (e.g. #504, #524), and some IL10KO-biased clusters (e.g. #495, #520). Notably, when we calculated the frequency of events contributed by each individual mouse for the six most-biased clusters, only two IL10KO-biased clusters were significantly different (#515, #520), with other clusters driven by a single individual (#504, #510) (Fig. 8F). These data emphasize a limitation of many of the clustering algorithms, the challenge of identifying inter-individual variation.

Despite the presence of biased clusters, there were no cell clusters that were found exclusively in B6 or IL10KO mice. One notable use of this X-shift driven approach, however, is that it can be applied to situations in which a cell type is either present or absent. To demonstrate this, we used X-shift to analyze cellular complexity in an orthotopic mouse model of lung cancer. This approach identified cell clusters that were uniquely present in the tumor-containing lung (#886, #887), suggesting that X-shift can identify tumor cells even in the absence of a unique marker for the tumor cell (Supplemental Fig. 3).

Investigating cellular expression by X-shift

The VorteX graphical environment has many unique visualization features, including the phenotypic barcode, which shows expression levels across all parameters for either a cluster or for individual cells (Fig. 9E). Analysis of expression in individual cells can be particularly informative: 1) identifying the basic cellular phenotype of the cluster, 2) defining secondary characteristics which make the cluster unique, and 3) allowing investigation of phenotypic heterogeneity/homogeneity of events within clusters (Fig. 9E). VorteX also allows analysis of expression of all parameters within a cluster by a line graph (Fig. 9F). To distill this information further, we have found that infographics can be particularly useful, for example, depicting each cluster by its core phenotypes (e.g. features present among all CD4 T cells) and accessory phenotype (e.g. features present in only a subset of CD4 T cells) (e.g. cluster #515, Fig. 9F).

SPADE visualization and insights

SPADE (Spanning-tree Progression Analysis of Density-normalized Events) was the original dimensionality reduction tool created for CyTOF and remains commonly used today (6, 26). The SPADE algorithm first down samples data in order to capture rare populations, then hierarchically clusters phenotypically similar cells into “nodes”, sorts the remaining data into these nodes, and then represents these nodes and the relationships between them in a minimum spanning tree format (i.e. a SPADE tree).

We used SPADE within Cytobank, a platform that requires four user-defined inputs: 1) target number of nodes (i.e. how many nodes will be present on the SPADE tree), 2) percent downsampling (i.e. what percent of events will be considered for the analysis), 3) population selection, and 4) parameter selection (i.e. which phenotypic markers are to be used for the clustering analysis) (Fig. 5A). The resulting SPADE tree contains a series of interconnected “nodes”, with node size indicating cell number and node color quantifying the parameter of interest (e.g. CD45) (Fig. 5B). One important distinction about the SPADE tree is that the relative distance between different nodes does not present meaningful information (e.g. in contrast to the force-directed layout method used in X-shift). As such users can manually modify the appearance of a SPADE tree within Cytobank (Fig. 5C–D). Note that due to downsampling, multiple runs with identical settings will produce slightly different SPADE trees (Fig. 5E).

Input settings (A) and SPADE visualization (B-G), focused on CyTOF analysis of γHV68-infected lungs from either B6 or IL10KO individuals at 9 days post-infection. Data show all viable single cells, subjected to the Spanning-tree Progression Analysis of Density-normalized Events (SPADE) algorithm that clusters cells with similar protein expression levels into a customizable hierarchy. Input settings to run SPADE within Cytobank (A). All of the events organized into a SPADE tree, colored by CD45 expression, using either a target of 200 nodes (default setting, B) or 45 nodes (X-shift informed, C-E). (C-D) depict a SPADE tree that has been manually modified by the user in Cytobank (C-D). (E) An independent SPADE analysis created from the same dataset using identical settings to panels C-D. (F) SPADE trees colored by CD45 generated based on clustering using either 10 lineage markers (left) or 35 markers (right), comparing SPADE trees with an X-shift defined optimal number of nodes (top row; 89 nodes for 10 lineage markers, 45 nodes for 35 marker clustering) with a 200 target node tree (bottom row; default Cytobank setting). (G) Visualization grid of SPADE trees, with data organized according to marker expression (rows) relative to individuals (columns). (H) Identification of cellular populations identified by the SPADE tree for individual B6 #1, with phenotype-based cell populations as identified. Histogram overlays depict a user-defined parameter whose expression bifurcates between parent and daughter. Data from virus-infected lungs (B6, n=5; IL10KO, n=4 mice).

One important consideration in using SPADE is that the user must decide how many target nodes should be present in a SPADE tree. For example, when we applied the SPADE algorithm to our data using a Cytobank default setting of 200 target nodes, the resulting SPADE tree contained many nodes that contained only a single event, suggesting data over-fragmentation (Fig. 5F, bottom right). While the target number of nodes can be empirically modified, it is difficult to know what is truly optimal. To minimize over fragmentation, we integrated two approaches, first applying the X-shift algorithm (discussed above) to define the number of cellular phenotypes (here, 45 clusters), which we then used to generate a SPADE tree with an informed number of nodes (Fig. 5F, top left). Another important consideration when using SPADE, and all clustering algorithms, is how many parameters are included in clustering analysis. For example, we performed clustering on all 35 markers or clustering based only on lineage markers (CD3, CD4, CD8, CD19, CD45, CD64, CD117, NKp46, Sca1, SigF), plotted onto SPADE trees using either a 200-node default or an X-Shift defined target node. While changing the number of clustering parameters altered SPADE tree structure, the most noticeable impact was that X-shift predicted 89 clusters based on 10 clustering parameters but only predicted 45 clusters based on 35 clustering parameters (Figure 5F). While each graph allows one to visualize cell populations, a major limitation of SPADE is that there are no clear quality-control metrics to identify optimal versus suboptimal analysis.

Once a SPADE tree is generated, one primary objective is to define which cells are present in which node. Here we used an optimized SPADE tree, informed by X-Shift and clustered on all 35 parameters, to generate a visualization grid comparing SPADE trees between individuals (in columns) as a function of marker expression (in rows) (Fig. 5G). Since the branches of the SPADE tree are common across samples, and nodes vary in marker intensity and size, this approach allows rapid inspection across individuals and phenotypes. Much like viSNE, this allows rapid assessment of: 1) cellular identity of various nodes and 2) inter-individual reproducibility and variability (Fig. 5H).

Investigating cellular abundance by SPADE

The SPADE tree can be a powerful visualization tool, allowing rapid clues about potential differences between groups. For example, by visual inspection it appears that nodes #1, #9, #12, and #26 had the most dramatic change in size between B6 and IL10KO mice (Fig. 8G). When we calculated the frequency of events across all nodes, however, it is clear that visual inspection alone is insufficient. For example, nodes #9 and #12 demonstrated high inter-individual variation (Fig. 8H). Conversely, multiple nodes were statistically significant between B6 and IL10KO mice (#1, #2, #18, #26, #30, #31, and #37), including some changes that would have been easily overlooked by visual inspection (e.g. nodes #30 and 37) (Fig. 8I). These data emphasize the value in integrating quantitative and visual analyses to fully interrogate a SPADE analysis.

Investigating cellular expression by SPADE

SPADE also allows rapid analysis of expression across multiple parameters. For example, we analyzed markers previously characterized by viSNE (Fig. 9A). By eye, IL10KO mice appeared to have increased expression in CD11b, CD11c, Gr-1, and PD-L1 across both node #1 and #26 (Fig. 9G). When we quantified median expression of these markers for node #1 across all individuals, however, we found that only CD11b and PD-L1 had statistically significant increases in median expression (Fig. 9H). These studies further emphasize the necessity of integrating SPADE visualization with quantitative data, to avoid misinterpretation.

Citrus visualization and insights

The Citrus algorithm was designed to identify statistically significant differences in high-dimensional datasets between different experimental groups (11), and has been able to identify immune cell signatures of patients that correlate with divergent surgical outcomes (27). Citrus accomplishes this by compiling events across multiple samples, hierarchically clustering cells by similarity, and then interrogating whether these populations differ significantly between user-identified experimental conditions. Citrus tests for statistical changes in either abundance or in median expression by using predictive or correlative linearized regression models, contingent on sufficient sample size and statistical power (n ≥ 3, with higher n associated with increased power) (11).

We investigated Citrus within Cytobank, a platform that requires a series of 10 user-defined inputs, including sample and parameter identification, statistical model to be evaluated (11), and criteria to be used for the statistical analysis (e.g. false discovery rate and minimum cluster size) (outlined in Fig. 6A). For our purposes, we tested for predictive models that had statistical power to discriminate between groups.

Input settings (A) and Citrus visualization (B-K), focused on CyTOF analysis of γHV68-infected lungs from either B6 or IL10KO individuals at 9 days post-infection. Data show all viable single cells, subjected to the Citrus (cluster identification, characterization, and regression) algorithm that hierarchically clusters cells and identifies statistically significant biological differences between two or more parameters. (A) Input settings to run Citrus within Cytobank. (B) Citrus-generated model error rate plot, which defines cross validation rate and feature false discovery rate for three different models of statistical stringency (cv.min, cv.1se, cv.fdr.constrained). Vertical dotted lines were added to better illustrate how many model features were identified by each model. (C-E) Model error rate plot (left) and radial hierarchy tree colored by CD45 expression (right), comparing different input settings (alternate setting identified by asterisk) for Citrus analysis, using (C) randomized group assignment, (D) reduced input cell number (5,000 input cells/sample), or (E) reduced minimum cluster size (1%). (F) Citrus-defined radial hierarchical plot for optimized Citrus settings (panel B), shaded according to the statistical significance of three different models. (G) Citrus-defined radial hierarchical plot colored by CD45 expression, with a magnified section of the tree to better illustrate node connections. (H) Citrus-defined radial hierarchical plot shaded by statistical significance according to three different models (identical to panel F, included for comparison). (I) A Citrus-defined vertical hierarchical tree, colored by CD45 expression, generated manually from panel G. Gray cluster, identified by asterisk, and labeled <5% indicates a “ghost daughter” whose abundance is less than 5% and is therefore is excluded from the hierarchical tree. Note that asterisk in this case denotes manual inclusion of this population, not statistical significance. (J) A Citrus-defined vertical hierarchical tree, shaded by statistical significance according to three different models. (K) Illustration of parent-daughter relationships in a Citrus tree, identifying markers whose expression bifurcates between daughters. Data from virus-infected lungs (B6, n=5; IL10KO, n=4 mice).

Once a Citrus run is completed within Cytobank there are multiple figures that are returned. Based on our experience, it is critical to first analyze the “Model Error Rate” figure which provides basic details about the validity of the statistical model. This figure plots: 1) the cross-validation rate (in red), quantifying how frequently the model correctly identifies whether a sample is in condition A or B, and 2) the feature false discovery rate (in blue) (Fig. 6B). Super-imposed on these two parameters, this graph further identifies 3 statistical models with differing stringencies: 1) the cross validation (cv).min model, that identifies the minimal number of features that predict the lowest cross validation error rate between experimental groups, 2) the cv.1se model that identifies the fewest features with a cross validation error rate one standard error greater than the minimum, and 3) the cv.fdr.constrained model that identifies the maximum number of features that can be included in the model, yet remain below the user-defined false discovery rate (https://support.cytobank.org/hc/en-us/articles/226678087-How-to-Configure-and-Run-a-CITRUS-Analysis).

Ideally, a Citrus-defined predictive model would achieve a cross-validation rate of 0 (i.e. the model can reliably predict whether a sample is in condition A or B) while having a low (<1%) false discovery rate (as in Fig. 6B). To demonstrate a Citrus analysis with no predictive power, we randomized experimental samples, inter-mingling B6 and IL10KO samples into a hypothetical group A and B; the resulting models had no predictive power, with ~50% cross validation error rate (Fig. 6C). Note that Citrus-defined models can be significantly influenced by reducing the number of input cells (e.g. from 9,141 events per sample to 5,000 events per sample, Fig. 6D) or by altering the minimum cluster size (e.g. from 5% to 1%, Fig. 6E). In both cases, these altered settings resulted in a higher cross-validation error rate, indicating a reduced predictive power of the defined features. One critical challenge when using Citrus is that even when Citrus creates models with unacceptably high rates of error (i.e. no predictive power as in Fig. 6C), the algorithm still identifies characteristics/features from the failing model. Thus, if a user does not verify the strength of the predictive model, it is easy to misinterpret features of the model. We strongly recommend reporting both Citrus settings and the “Model Error Rate” plot when reporting Citrus results, to facilitate proper Citrus interpretation.

For data visualization, Citrus generates radial hierarchy trees (“featurePlots”) with all events originating from a central node (Fig. 6F). These plots can either be visualized by parameter expression (e.g. CD45, Fig. 6G) or according to statistical model, highlighting clusters that predict differences between the two experimental conditions (Fig. 6H). Despite an apparent similarity to a SPADE tree, a Citrus radial tree portrays events in a hierarchical manner with parent clusters giving rise to two daughters. This means that each daughter contains a subset of cells that were present in the parent cluster. As a result, a single cell is not confined to a single cluster in a Citrus tree (unlike in SPADE). To clarify these relationships, we created a vertical hierarchical tree to examine phenotype and statistical significance (Fig. 6I–J). This vertical hierarchical visualization approach: 1) avoids confusion with SPADE visualization, 2) identifies branch points in which daughter populations diverge in their significance (i.e. where only one daughter is predictive of a difference whereas the other daughter is not predictive of a difference), and 3) identifies branch points with a single daughter, a context that occurs when the second, “ghost” daughter is less than the minimum cluster size and is excluded from the hierarchical tree. A “ghost daughter” is illustrated by a gray dot labeled “<5%” (Fig. 6I).

The ultimate goal of Citrus is to identify cell clusters that are significantly different between groups. For example, the cv.min model (Fig. 6B) identified 7 model features between groups, with these clusters visualized on either the radial or vertical tree (Fig. 6H, 6J). Inspecting the Citrus tree, however, it is clear that there are not 7 distinct cell types that are significantly different, since many of the clusters are linked in direct parent-daughter relationships. Detailed analysis of cell phenotypes in these parent-daughter pairs clearly shows parent clusters contain a mixed population of cells (e.g. a cluster containing both CD4 and CD8 T cells), subdivided between daughters (e.g. CD4 versus CD8 T cells) (Fig. 6K). This bifurcation of cell phenotypes can be particularly useful when the two daughter populations diverge in significance, with phenotypes associated with this bifurcation giving insight into the population of interest. Given the mixed phenotype of parent clusters, the most refined cell type designations are contained within the terminal branches of the Citrus tree.

Investigating cellular abundance

We used Citrus to identify clusters with significant differences in their abundance between WT and IL10KO infected mice, using the minimum number of cells contained across all experimental samples (9,141 events per sample), a minimum cluster size of 5%, and the nearest shrunken centroid predictive association model (PAM). Citrus identified 3 models (cv.min, cv.1se, cv.fdr.constrained) with varying numbers of features that had a high predictive value (0% false discovery rate, 0% cross validation rate) (Fig. 6H, 6J). Here, we focused on the cv.min model that identified 7 features that predict differences between groups (Fig. 8J). While Citrus identified these clusters as significantly different, it does not automatically return statistical significance values. Therefore, we downloaded individual abundance values for each cluster to calculate p-values (Fig. 8K). For each cluster, Citrus provides a series of histogram overlays depicting expression across all parameters between the cluster and background. When we analyzed the most terminal clusters identified by the cv.min model, the B6-enriched cluster #82263 had a mixed phenotype (e.g. containing both CD3+ and CD3− events) (top row, Fig. 8L). In contrast, the IL10KO-enriched cluster #82243 were uniformly CD4+ T cells, characterized by proliferation (Ki67+) and expression of multiple inhibitory receptors (CTLA4+ GITR+ PD1+ Tim3+). These studies emphasize the potential for Citrus to rapidly identify changes in cellular abundance, following careful interpretation of the results.

Investigating cellular expression

In parallel, we used Citrus to test for changes in expression between experimental groups. This analysis returned a Model Error Rate with low (~10%) cross-validation error rate, with cv.min and cv.fdr constrained models identifying multiple potential differences (Fig. 9I). Analysis of the cv.min model identified four features between groups, including increased expression of IRF4, CD11b and PD-L1 in IL10KO infected individuals (Fig. 9J). The significance of these differences in cellular expression was determined and the expression of these markers relative to background expression was visualized by Citrus-generated histogram overlays (Fig. 9K).

Comparing methods visualization and insights

Given the diverse CyTOF algorithms and visualization methods studied here, we directly compared data visualization from each of these algorithms side-by-side (Fig. 7A). This approach exemplifies the challenges of CyTOF data visualization and how a single method is insufficient to fully extract the robustness of these high-dimensional data (algorithm comparisons outlined in Supplemental Table 3). Whereas viSNE and PhenoGraph emphasize the continuum of cellular phenotypes in two-dimensional space as revealed by the tSNE algorithm, SPADE and VorteX reduce this complexity by placing multiple events into discrete bins. Citrus, on the other hand, seeks to identify significant differences between groups and may not define the full cellular complexity across all events. For example, a parameter such as CD19, which does not vary between groups, can appear to be absent from a Citrus data visualization tree although clearly present by alternate algorithms (Fig. 7A). Moving beyond visualization, we quantified the frequency of events, and number of clusters/nodes assigned to various cellular phenotypes (Fig. 7B–C). Though all algorithms identified multiple major cell populations, there were also variations in the number of cells identified by different approaches. For example, CD8 T cells were disproportionately less frequent when analyzed by viSNE, with CD11b+ CD64- myeloid cells less frequent when analyzed by SPADE and PhenoGraph (Fig. 7B). In addition, X-shift identified fewer CD4 T cell clusters, and more myeloid and non-hematopoietic clusters than its counterparts (Fig. 7C).

A major goal in applying multiple algorithms to these data is to identify reproducible changes between experimental groups. To determine if a particular cell type was identified independently across multiple methods, we selected the daughter/node/cluster that increased most significantly from B6 to IL10KO individuals (identified in Fig. 8–9): daughter #82243 (Citrus), node #1 (SPADE), cluster #8 (PhenoGraph) and cluster #515 (X-shift). We then assessed marker expression across these different methods, focusing on markers that were positive (i.e. higher than the average expression for all events) (Fig. 7D). 27 total parameters were positive among at least one of these four methods and 17 parameters were identified as positive by all of the methods, visualized by a phenotype network (Fig. 7E). This cross-comparison shows conserved features among CD4 T cells identified across algorithms as well as unanticipated variation.

How does cell number influence our interpretations? We examined the effect of varying input cell number on viSNE, X-shift, and PhenoGraph (Fig. 7F). For viSNE, low or high event counts generated suboptimal visualization that was either sparse or condensed (Fig. 7F). For X-shift and PhenoGraph, higher cell number increased both visual compression and how many cell clusters were identified (Fig. 7F, 7G). Increased cluster identification appeared to be driven by an increased identification of low abundance populations (<1%) and an overall reduction in high frequency clusters (Fig. 7H). These comparisons emphasize that input cell number can profoundly affect data output and visualization. It is also notable that these clustering algorithms are not well-equipped to handle large datasets: viSNE (a cloud-based platform) took 11.5 hours to analyze all events (1.26 million cells), while neither X-shift nor PhenoGraph were able to process all events on a standard desktop computer (Fig. 7F).

Investigating population structure

Beyond analysis of abundance and expression, CyTOF data can also be used to define alterations in population structure, characterizing changes in cellular abundance and diversity between conditions. In our experience, shifts in population structure are most readily defined by stratifying algorithms (SPADE, X-shift, or PhenoGraph) that place cells into distinct, non-overlapping cell phenotypes. Changes in population structure can be visualized by multiple approaches. For example, by ordering node abundance between experimental conditions, and then identifying SPADE-defined nodes whose abundance has significantly changed (red arrows), it is possible to see subdominant cell populations becoming dominant in IL10KO mice (Fig. 10A). Clusters can further be plotted to reveal changes in population structure, focused on both changes in cellular abundance and phenotypic subsets (Fig. 10B). This later analysis reveals specific cell types that have undergone pronounced changes in abundance and phenotype (e.g. CD4 T cells), as well as populations that are relatively constant between conditions (e.g. CD19+ B cells). Finally, PhenoGraph in Cytofkit can calculate clustering relationships by a dendrogram, to show the relatedness of cell clusters across experimental conditions. This is informative as it reveals that clustering relationships are not always constant between experimental conditions (Fig. 10C). These insights into population relatedness, abundance and phenotypic diversity are particularly accessible using stratifying CyTOF algorithms, and have not been readily accessible by previous methodologies.

DISCUSSION

The invention and application of mass cytometry transformed single-cell analysis, allowing deeper understanding of cellular heterogeneity among single cells at the molecular and phenotypic level (3). Beyond the technical aspects of performing a mass cytometry experiment (recently reviewed in (28)), there remain significant challenges in how to effectively analyze these complex data, particularly for the new user (17). The development of multiple algorithms, which utilize different data analysis methods and visualization approaches has enabled new insights, but has also made it difficult to know where to begin for CyTOF analysis.

In this manuscript, we present a practical guide for CyTOF data analysis using a common dataset to interrogate five established CyTOF analysis platforms. These algorithms varied widely in terms of ease of entry, computational skill required, data visualization and extractability. To establish a resource for those interested in CyTOF data analysis, we have provided: 1) annotated figures to identify important steps in implementing and interpreting CyTOF data analysis, 2) a direct cross-comparison of data visualization across these algorithms using a single dataset, and 3) an illustration of insights that can be gained by using complementary algorithms.

On one end of the spectrum, algorithms run through the Cytobank platform (here viSNE, SPADE and Citrus) required minimal computational background and gave complementary visualization approaches. Whereas viSNE demonstrates a continuum of phenotypes and can convey subtle variations as well as rare populations, SPADE trees provide a highly simplified overview of the cellular phenotype and structure.

One notable consideration for these algorithms is the likelihood of misinterpreting or misapplying results. We found that both SPADE and Citrus were more likely to give misleading results for an inexperienced user. For SPADE, an important and significant concern is how many nodes should be generated on the tree. The absence of clear guidelines, or the arbitrary use of a 200-node default, can easily lead to data over-fragmentation. For Citrus, there are at least two major challenges: 1) accurately interpreting statistical results for your Citrus run, and 2) understanding that a Citrus “tree” portrays parent-daughter relationships, such that a single cell can be present in multiple nodes of a Citrus tree (unlike SPADE, which places events in mutually exclusive nodes). A common limitation of each of these Cytobank-driven platforms is the lack of power to accurately predict total cluster number (i.e. approximately how many different types of cell clusters/phenotypes are present in your dataset). It is worth noting that despite the caveats noted for Citrus, Citrus remains the sole algorithm which provides a metric to quantify predictive power of the completed analysis.

We found that X-Shift and PhenoGraph are particularly useful, given their ability to quantify the number of cell clusters in a population. This metric is particularly useful for understanding the diversity of cells in a CyTOF dataset. In addition, these algorithms have the potential to extract a large amount of data, from individual event expression (in X-shift/VorteX) to the inter-relationship of cell phenotypes by dendrogram (in PhenoGraph within Cytofkit). Despite these benefits, there are important considerations in using these algorithms. Both require increased computational background for the user, including familiarity with R. These algorithms are also very sensitive to input cell number, with an increasing number of cells correlating with an increased number of clusters. Given this caveat, any comparison between experimental conditions must be standardized based on cell number. Finally, these algorithms are not equipped with automated statistical analysis (unlike Citrus). Although these issues are important to consider, the ability of X-Shift and PhenoGraph to transcend conventional, user-defined populations can provide unprecedented insights into cellular complexity and phenotypic diversity and are particularly powerful in gaining new insights from CyTOF data.

One common challenge with the approaches analyzed here is that they rely on data display and visualization that is wholly different than conventional biaxial plots and Boolean gating, the gold standard by which cell populations have traditionally been analyzed by flow cytometry. Whereas flow cytometry typically defines populations through a series of exclusions and inclusions, clustering algorithms like PhenoGraph or X-shift can sometimes identify cell clusters of indeterminate phenotype, lacking expression of a definitive lineage marker (e.g. CD45+ cells that are CD3− CD19− CD64− CD11c−). As such, assigning a unique and definitive identity to each cell cluster identified by these stratifying algorithms can be challenging. Based on this caveat, the Scaffold algorithm has been developed to allow a hybrid approach, combining Boolean-gating to define major populations, which then serves as a scaffold on which to build a force-directed layout (12).

Although the primary focus of this manuscript is on implementing and interpreting data from CyTOF algorithms, we did find populations that changed between γHV68-infected B6 and IL10KO mice. One consistent change defined across all algorithms was that γHV68-infected, IL10-deficient mice showed an increased frequency of highly activated CD4 T cells in the lungs of mice. While each algorithm identified a common core of parameters expressed by these cells, including a Ki67+ CTLA4+ GITR+ PD1+ Tim3+ phenotype, there were also discrepencies between the populations and phenotypes defined that will be the subject of future investigation. One may question whether CyTOF was required to identify an increase in effector CD4 T cells in IL10-deficient mice. It is worth noting, however, that the subsets and phenotypes of CD4 T cells identified by this approach would have been difficult to identify by conventional flow cytometry, as variations were not readily predictable based on precedent in the literature. Therefore, CyTOF afforded identification of unanticipated heterogeneity within effector CD4 T cell subsets, an area for future investigation. Whether this diversity of effector CD4 T cells is present in the context of lung cancer remains to be determined. These comparisons will require direct comparison of virus-infected and lung cancer samples, since it remains difficult to integrate and standardize data analysis across separate CyTOF runs.

In total, this study illustrates important practical considerations in the analysis of CyTOF datasets. While established algorithms can allow critical insights into these complex data, we find that different algorithms provide complementary strengths, varying significantly in terms of level of computation skill, the ability to extract additional data for further hypothesis testing, and the speed by which investigators can generate interpretable, presentation-ready figures.

Based on our experiences, we would ultimately recommend integrating at least three different algorithms/approaches to investigate CyTOF datasets, including: 1) a data visualization method, to rapidly interrogate cellular phenotypes across high-dimensional data, 2) a stratifying method (e.g. X-Shift or PhenoGraph), to define the number and diversity of cellular clusters and 3) in the case of experiments with at least 3 samples per condition, a method to define statistically significant differences in high-dimensional data (i.e. Citrus). The choice of data visualization method will ultimately depend on the point you are trying to make, whether discussing a change in the populations (e.g. conveyed by SPADE or X-Shift) or a change in the continuum of expression (e.g. conveyed by viSNE or tSNE-based algorithms). Given the changing landscape of CyTOF data analysis, we anticipate that this study will serve as an initial resource for those seeking to use CyTOF for their studies, and open the door to future discussion on best practices for analysis of these unique high-dimensional datasets.

Supplementary Material

NIHMS918275-supplement-1.pdf^{(5.9MB, pdf)}

Acknowledgments

This work was funded by National Institutes of Health grants R01CA103632 and R01CA168558 to L.F.V.D., R01 CA162226 and P50 CA058187 to R.A.N., and an American Heart Association National Scientist Development grant, the Crohn’s and Colitis Foundation of America, and a Career Enhancement Award from the University of Colorado Lung Cancer SPORE to E.T.C.. The Flow Cytometry Shared Resource receives direct funding support from the National Cancer Institute through the Cancer Center Support Grant (P30CA046934).

The authors acknowledge Amber Johnson, Jeff Kwak, and Melissa Ledezma for technical assistance, Kristina Terrell, Christine Childs, and Karen Helm for technical support for CyTOF studies, and Dr. Elena Hsieh for fruitful discussions regarding CyTOF analysis.

References

1.Bandura DR, Baranov VI, Ornatsky OI, Antonov A, Kinach R, Lou X, Pavlov S, Vorobiev S, Dick JE, Tanner SD. Mass cytometry: technique for real time single cell multitarget immunoassay based on inductively coupled plasma time-of-flight mass spectrometry. Anal Chem. 2009;81:6813–6822. doi: 10.1021/ac901049w. [DOI] [PubMed] [Google Scholar]
2.Bendall SC, Simonds EF, Qiu P, Amir el AD, Krutzik PO, Finck R, Bruggner RV, Melamed R, Trejo A, Ornatsky OI, Balderas RS, Plevritis SK, Sachs K, Pe’er D, Tanner SD, Nolan GP. Single-cell mass cytometry of differential immune and drug responses across a human hematopoietic continuum. Science. 2011;332:687–696. doi: 10.1126/science.1198704. [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Spitzer MH, Nolan GP. Mass Cytometry: Single Cells, Many Features. Cell. 2016;165:780–791. doi: 10.1016/j.cell.2016.04.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Newell EW, Sigal N, Bendall SC, Nolan GP, Davis MM. Cytometry by time-of-flight shows combinatorial cytokine expression and virus-specific cell niches within a continuum of CD8+ T cell phenotypes. Immunity. 2012;36:142–152. doi: 10.1016/j.immuni.2012.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Horowitz A, Strauss-Albee DM, Leipold M, Kubo J, Nemat-Gorgani N, Dogan OC, Dekker CL, Mackey S, Maecker H, Swan GE, Davis MM, Norman PJ, Guethlein LA, Desai M, Parham P, Blish CA. Genetic and environmental determinants of human NK cell diversity revealed by mass cytometry. Sci Transl Med. 2013;5:208ra145. doi: 10.1126/scitranslmed.3006702. [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Qiu P, Simonds EF, Bendall SC, Gibbs KD, Jr, Bruggner RV, Linderman MD, Sachs K, Nolan GP, Plevritis SK. Extracting a cellular hierarchy from high-dimensional cytometry data with SPADE. Nat Biotechnol. 2011;29:886–891. doi: 10.1038/nbt.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Amir el AD, Davis KL, Tadmor MD, Simonds EF, Levine JH, Bendall SC, Shenfeld DK, Krishnaswamy S, Nolan GP, Pe’er D. viSNE enables visualization of high dimensional single-cell data and reveals phenotypic heterogeneity of leukemia. Nat Biotechnol. 2013;31:545–552. doi: 10.1038/nbt.2594. [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Bendall SC, Davis KL, Amir el AD, Tadmor MD, Simonds EF, Chen TJ, Shenfeld DK, Nolan GP, Pe’er D. Single-cell trajectory detection uncovers progression and regulatory coordination in human B cell development. Cell. 2014;157:714–725. doi: 10.1016/j.cell.2014.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Van Gassen S, Callebaut B, Van Helden MJ, Lambrecht BN, Demeester P, Dhaene T, Saeys Y. FlowSOM: Using self-organizing maps for visualization and interpretation of cytometry data. Cytometry A. 2015;87:636–645. doi: 10.1002/cyto.a.22625. [DOI] [PubMed] [Google Scholar]
10.Levine JH, Simonds EF, Bendall SC, Davis KL, Amir el AD, Tadmor MD, Litvin O, Fienberg HG, Jager A, Zunder ER, Finck R, Gedman AL, Radtke I, Downing JR, Pe’er D, Nolan GP. Data-Driven Phenotypic Dissection of AML Reveals Progenitor-like Cells that Correlate with Prognosis. Cell. 2015;162:184–197. doi: 10.1016/j.cell.2015.05.047. [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Bruggner RV, Bodenmiller B, Dill DL, Tibshirani RJ, Nolan GP. Automated identification of stratifying signatures in cellular subpopulations. Proc Natl Acad Sci U S A. 2014;111:E2770–2777. doi: 10.1073/pnas.1408792111. [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Spitzer MH, Gherardini PF, Fragiadakis GK, Bhattacharya N, Yuan RT, Hotson AN, Finck R, Carmi Y, Zunder ER, Fantl WJ, Bendall SC, Engleman EG, Nolan GP. IMMUNOLOGY. An interactive reference framework for modeling a dynamic immune system. Science. 2015;349:1259425. doi: 10.1126/science.1259425. [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Samusik N, Good Z, Spitzer MH, Davis KL, Nolan GP. Automated mapping of phenotype space with single-cell data. Nat Methods. 2016;13:493–496. doi: 10.1038/nmeth.3863. [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Becher B, Schlitzer A, Chen J, Mair F, Sumatoh HR, Teng KW, Low D, Ruedl C, Riccardi-Castagnoli P, Poidinger M, Greter M, Ginhoux F, Newell EW. High-dimensional analysis of the murine myeloid cell system. Nat Immunol. 2014;15:1181–1189. doi: 10.1038/ni.3006. [DOI] [PubMed] [Google Scholar]
15.Chester C, Maecker HT. Algorithmic Tools for Mining High-Dimensional Cytometry Data. J Immunol. 2015;195:773–779. doi: 10.4049/jimmunol.1500633. [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Mair F, Hartmann FJ, Mrdjen D, Tosevski V, Krieg C, Becher B. The end of gating? An introduction to automated analysis of high dimensional cytometry data. Eur J Immunol. 2016;46:34–43. doi: 10.1002/eji.201545774. [DOI] [PubMed] [Google Scholar]
17.Newell EW, Cheng Y. Mass cytometry: blessed with the curse of dimensionality. Nat Immunol. 2016;17:890–895. doi: 10.1038/ni.3485. [DOI] [PubMed] [Google Scholar]
18.Diggins KE, Greenplate AR, Leelatian N, Wogsland CE, Irish JM. Characterizing cell subsets using marker enrichment modeling. Nat Methods. 2017;14:275–278. doi: 10.1038/nmeth.4149. [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Spitzer MH, Carmi Y, Reticker-Flynn NE, Kwek SS, Madhireddy D, Martins MM, Gherardini PF, Prestwood TR, Chabon J, Bendall SC, Fong L, Nolan GP, Engleman EG. Systemic Immunity Is Required for Effective Cancer Immunotherapy. Cell. 2017;168:487–502 e415. doi: 10.1016/j.cell.2016.12.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Diebel KW, Oko LM, Medina EM, Niemeyer BF, Warren CJ, Claypool DJ, Tibbetts SA, Cool CD, Clambey ET, van Dyk LF. Gammaherpesvirus small noncoding RNAs are bifunctional elements that regulate infection and contribute to virulence in vivo. MBio. 2015;6:e01670–01614. doi: 10.1128/mBio.01670-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Layton MG, Franks LM. Heterogeneity in a spontaneous mouse lung carcinoma: selection and characterisation of stable metastatic variants. Br J Cancer. 1984;49:415–421. doi: 10.1038/bjc.1984.67. [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Li HY, McSharry M, Bullock B, Nguyen TT, Kwak J, Poczobutt JM, Sippel TR, Heasley LE, Weiser-Evans MC, Clambey ET, Nemenoff RA. The Tumor Microenvironment Regulates Sensitivity of Murine Lung Tumors to PD-1/PD-L1 Antibody Blockade. Cancer Immunol Res. 2017;5:767–777. doi: 10.1158/2326-6066.CIR-16-0365. [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Barton E, Mandal P, Speck SH. Pathogenesis and host control of gammaherpesviruses: lessons from the mouse. Annual review of immunology. 2011;29:351–397. doi: 10.1146/annurev-immunol-072710-081639. [DOI] [PubMed] [Google Scholar]
24.van der Maaten L, Hinton G. Visualizing Data using t-SNE. J Mach Learn Res. 2008;9:2579–2605. [Google Scholar]
25.Chen H, Lau MC, Wong MT, Newell EW, Poidinger M, Chen J. Cytofkit: A Bioconductor Package for an Integrated Mass Cytometry Data Analysis Pipeline. PLoS Comput Biol. 2016;12:e1005112. doi: 10.1371/journal.pcbi.1005112. [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Anchang B, Hart TD, Bendall SC, Qiu P, Bjornson Z, Linderman M, Nolan GP, Plevritis SK. Visualization and cellular hierarchy inference of single-cell data using SPADE. Nat Protoc. 2016;11:1264–1279. doi: 10.1038/nprot.2016.066. [DOI] [PubMed] [Google Scholar]
27.Gaudilliere B, Fragiadakis GK, Bruggner RV, Nicolau M, Finck R, Tingle M, Silva J, Ganio EA, Yeh CG, Maloney WJ, Huddleston JI, Goodman SB, Davis MM, Bendall SC, Fantl WJ, Angst MS, Nolan GP. Clinical recovery from surgery correlates with single-cell immune signatures. Sci Transl Med. 2014;6:255ra131. doi: 10.1126/scitranslmed.3009701. [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Brodie TM, Tosevski V. High-Dimensional Single-Cell Analysis with Mass Cytometry. Curr Protoc Immunol. 2017;118:5 11 11–15 11 25. doi: 10.1002/cpim.31. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

NIHMS918275-supplement-1.pdf^{(5.9MB, pdf)}

[R1] 1.Bandura DR, Baranov VI, Ornatsky OI, Antonov A, Kinach R, Lou X, Pavlov S, Vorobiev S, Dick JE, Tanner SD. Mass cytometry: technique for real time single cell multitarget immunoassay based on inductively coupled plasma time-of-flight mass spectrometry. Anal Chem. 2009;81:6813–6822. doi: 10.1021/ac901049w. [DOI] [PubMed] [Google Scholar]

[R2] 2.Bendall SC, Simonds EF, Qiu P, Amir el AD, Krutzik PO, Finck R, Bruggner RV, Melamed R, Trejo A, Ornatsky OI, Balderas RS, Plevritis SK, Sachs K, Pe’er D, Tanner SD, Nolan GP. Single-cell mass cytometry of differential immune and drug responses across a human hematopoietic continuum. Science. 2011;332:687–696. doi: 10.1126/science.1198704. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R3] 3.Spitzer MH, Nolan GP. Mass Cytometry: Single Cells, Many Features. Cell. 2016;165:780–791. doi: 10.1016/j.cell.2016.04.019. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R4] 4.Newell EW, Sigal N, Bendall SC, Nolan GP, Davis MM. Cytometry by time-of-flight shows combinatorial cytokine expression and virus-specific cell niches within a continuum of CD8+ T cell phenotypes. Immunity. 2012;36:142–152. doi: 10.1016/j.immuni.2012.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R5] 5.Horowitz A, Strauss-Albee DM, Leipold M, Kubo J, Nemat-Gorgani N, Dogan OC, Dekker CL, Mackey S, Maecker H, Swan GE, Davis MM, Norman PJ, Guethlein LA, Desai M, Parham P, Blish CA. Genetic and environmental determinants of human NK cell diversity revealed by mass cytometry. Sci Transl Med. 2013;5:208ra145. doi: 10.1126/scitranslmed.3006702. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R6] 6.Qiu P, Simonds EF, Bendall SC, Gibbs KD, Jr, Bruggner RV, Linderman MD, Sachs K, Nolan GP, Plevritis SK. Extracting a cellular hierarchy from high-dimensional cytometry data with SPADE. Nat Biotechnol. 2011;29:886–891. doi: 10.1038/nbt.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R7] 7.Amir el AD, Davis KL, Tadmor MD, Simonds EF, Levine JH, Bendall SC, Shenfeld DK, Krishnaswamy S, Nolan GP, Pe’er D. viSNE enables visualization of high dimensional single-cell data and reveals phenotypic heterogeneity of leukemia. Nat Biotechnol. 2013;31:545–552. doi: 10.1038/nbt.2594. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R8] 8.Bendall SC, Davis KL, Amir el AD, Tadmor MD, Simonds EF, Chen TJ, Shenfeld DK, Nolan GP, Pe’er D. Single-cell trajectory detection uncovers progression and regulatory coordination in human B cell development. Cell. 2014;157:714–725. doi: 10.1016/j.cell.2014.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R9] 9.Van Gassen S, Callebaut B, Van Helden MJ, Lambrecht BN, Demeester P, Dhaene T, Saeys Y. FlowSOM: Using self-organizing maps for visualization and interpretation of cytometry data. Cytometry A. 2015;87:636–645. doi: 10.1002/cyto.a.22625. [DOI] [PubMed] [Google Scholar]

[R10] 10.Levine JH, Simonds EF, Bendall SC, Davis KL, Amir el AD, Tadmor MD, Litvin O, Fienberg HG, Jager A, Zunder ER, Finck R, Gedman AL, Radtke I, Downing JR, Pe’er D, Nolan GP. Data-Driven Phenotypic Dissection of AML Reveals Progenitor-like Cells that Correlate with Prognosis. Cell. 2015;162:184–197. doi: 10.1016/j.cell.2015.05.047. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R11] 11.Bruggner RV, Bodenmiller B, Dill DL, Tibshirani RJ, Nolan GP. Automated identification of stratifying signatures in cellular subpopulations. Proc Natl Acad Sci U S A. 2014;111:E2770–2777. doi: 10.1073/pnas.1408792111. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R12] 12.Spitzer MH, Gherardini PF, Fragiadakis GK, Bhattacharya N, Yuan RT, Hotson AN, Finck R, Carmi Y, Zunder ER, Fantl WJ, Bendall SC, Engleman EG, Nolan GP. IMMUNOLOGY. An interactive reference framework for modeling a dynamic immune system. Science. 2015;349:1259425. doi: 10.1126/science.1259425. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R13] 13.Samusik N, Good Z, Spitzer MH, Davis KL, Nolan GP. Automated mapping of phenotype space with single-cell data. Nat Methods. 2016;13:493–496. doi: 10.1038/nmeth.3863. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R14] 14.Becher B, Schlitzer A, Chen J, Mair F, Sumatoh HR, Teng KW, Low D, Ruedl C, Riccardi-Castagnoli P, Poidinger M, Greter M, Ginhoux F, Newell EW. High-dimensional analysis of the murine myeloid cell system. Nat Immunol. 2014;15:1181–1189. doi: 10.1038/ni.3006. [DOI] [PubMed] [Google Scholar]

[R15] 15.Chester C, Maecker HT. Algorithmic Tools for Mining High-Dimensional Cytometry Data. J Immunol. 2015;195:773–779. doi: 10.4049/jimmunol.1500633. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R16] 16.Mair F, Hartmann FJ, Mrdjen D, Tosevski V, Krieg C, Becher B. The end of gating? An introduction to automated analysis of high dimensional cytometry data. Eur J Immunol. 2016;46:34–43. doi: 10.1002/eji.201545774. [DOI] [PubMed] [Google Scholar]

[R17] 17.Newell EW, Cheng Y. Mass cytometry: blessed with the curse of dimensionality. Nat Immunol. 2016;17:890–895. doi: 10.1038/ni.3485. [DOI] [PubMed] [Google Scholar]

[R18] 18.Diggins KE, Greenplate AR, Leelatian N, Wogsland CE, Irish JM. Characterizing cell subsets using marker enrichment modeling. Nat Methods. 2017;14:275–278. doi: 10.1038/nmeth.4149. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R19] 19.Spitzer MH, Carmi Y, Reticker-Flynn NE, Kwek SS, Madhireddy D, Martins MM, Gherardini PF, Prestwood TR, Chabon J, Bendall SC, Fong L, Nolan GP, Engleman EG. Systemic Immunity Is Required for Effective Cancer Immunotherapy. Cell. 2017;168:487–502 e415. doi: 10.1016/j.cell.2016.12.022. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R20] 20.Diebel KW, Oko LM, Medina EM, Niemeyer BF, Warren CJ, Claypool DJ, Tibbetts SA, Cool CD, Clambey ET, van Dyk LF. Gammaherpesvirus small noncoding RNAs are bifunctional elements that regulate infection and contribute to virulence in vivo. MBio. 2015;6:e01670–01614. doi: 10.1128/mBio.01670-14. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R21] 21.Layton MG, Franks LM. Heterogeneity in a spontaneous mouse lung carcinoma: selection and characterisation of stable metastatic variants. Br J Cancer. 1984;49:415–421. doi: 10.1038/bjc.1984.67. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R22] 22.Li HY, McSharry M, Bullock B, Nguyen TT, Kwak J, Poczobutt JM, Sippel TR, Heasley LE, Weiser-Evans MC, Clambey ET, Nemenoff RA. The Tumor Microenvironment Regulates Sensitivity of Murine Lung Tumors to PD-1/PD-L1 Antibody Blockade. Cancer Immunol Res. 2017;5:767–777. doi: 10.1158/2326-6066.CIR-16-0365. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R23] 23.Barton E, Mandal P, Speck SH. Pathogenesis and host control of gammaherpesviruses: lessons from the mouse. Annual review of immunology. 2011;29:351–397. doi: 10.1146/annurev-immunol-072710-081639. [DOI] [PubMed] [Google Scholar]

[R24] 24.van der Maaten L, Hinton G. Visualizing Data using t-SNE. J Mach Learn Res. 2008;9:2579–2605. [Google Scholar]

[R25] 25.Chen H, Lau MC, Wong MT, Newell EW, Poidinger M, Chen J. Cytofkit: A Bioconductor Package for an Integrated Mass Cytometry Data Analysis Pipeline. PLoS Comput Biol. 2016;12:e1005112. doi: 10.1371/journal.pcbi.1005112. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R26] 26.Anchang B, Hart TD, Bendall SC, Qiu P, Bjornson Z, Linderman M, Nolan GP, Plevritis SK. Visualization and cellular hierarchy inference of single-cell data using SPADE. Nat Protoc. 2016;11:1264–1279. doi: 10.1038/nprot.2016.066. [DOI] [PubMed] [Google Scholar]

[R27] 27.Gaudilliere B, Fragiadakis GK, Bruggner RV, Nicolau M, Finck R, Tingle M, Silva J, Ganio EA, Yeh CG, Maloney WJ, Huddleston JI, Goodman SB, Davis MM, Bendall SC, Fantl WJ, Angst MS, Nolan GP. Clinical recovery from surgery correlates with single-cell immune signatures. Sci Transl Med. 2014;6:255ra131. doi: 10.1126/scitranslmed.3009701. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R28] 28.Brodie TM, Tosevski V. High-Dimensional Single-Cell Analysis with Mass Cytometry. Curr Protoc Immunol. 2017;118:5 11 11–15 11 25. doi: 10.1002/cpim.31. [DOI] [PubMed] [Google Scholar]

PERMALINK

A Beginner’s Guide To Analyzing and Visualizing Mass Cytometry Data

Abigail K Kimball

Lauren M Oko

Bonnie L Bullock

Raphael A Nemenoff

Linda F van Dyk

Eric T Clambey

Abstract

INTRODUCTION

MATERIALS AND METHODS

Experimental samples

Cell processing & antibody staining

CyTOF run and sample normalization

viSNE

Algorithm settings

Algorithm optimization, interrogation, and visualization

Investigating cellular abundance

Investigating cellular expression

PhenoGraph

Software downloads

Algorithm settings

Algorithm optimization, interrogation, and visualization

Investigating cellular abundance

Investigating cellular expression

X-shift

Software downloads

Algorithm settings

Algorithm optimization, interrogation, and visualization

Investigating cellular abundance

Investigating cellular expression

SPADE

Algorithm settings

Algorithm optimization, interrogation, and visualization

Investigating cellular abundance

Investigating cellular expression

Citrus

Algorithm settings

Algorithm optimization, interrogation, and visualization

Investigating cellular abundance

Investigating cellular expression

Comparing Algorithms

Investigating population structure

Software used & Statistical analysis

Figure 7. Comparison of data visualization, cellular identification and reproducibility across CyTOF algorithms.

Figure 3. Basic considerations for PhenoGraph analysis.

Figure 8. Investigating cellular abundance by viSNE, PhenoGraph, X-shift, SPADE, and Citrus algorithms.

RESULTS

Conceptual framework for CyTOF analysis

Figure 1. Important considerations in CyTOF experimental design and algorithm implementation.

viSNE visualization and insights

Figure 2. Basic considerations for viSNE analysis.

Investigating cellular abundance by viSNE

Investigating cellular expression by viSNE

Figure 9. Investigating changes in cellular expression by viSNE, PhenoGraph, X-shift, SPADE, and Citrus algorithms.

PhenoGraph visualization and insights

Investigating cellular abundance by PhenoGraph

Investigating cellular expression by PhenoGraph

X-shift visualization and insights

Figure 4. Basic considerations for X-shift analysis.

Investigating cellular abundance by X-shift

Investigating cellular expression by X-shift

SPADE visualization and insights

Figure 5. Basic considerations for SPADE analysis.

Investigating cellular abundance by SPADE

Investigating cellular expression by SPADE

Citrus visualization and insights

Figure 6. Basic considerations for Citrus analysis.

Investigating cellular abundance

Investigating cellular expression

Comparing methods visualization and insights

Investigating population structure

Figure 10. Population structure analysis using SPADE, X-shift and PhenoGraph.

DISCUSSION

Supplementary Material

Acknowledgments

References

Associated Data

Supplementary Materials

ACTIONS