Rigor and Reproducibility of Spatial Transcriptomics Performed on Clinically Sourced Human Tissues

Abstract

Spatial transcriptomic profiling enables precise quantification of gene expression with simultaneous localization of expression profiles onto tissue structures. Several implementations of these approaches have been released as commercialized platforms which will allow multiple laboratories to improve our understanding of human disease mechanisms. There is also intense interest in applying these methods in clinical trials or as laboratory developed tests to aid in diagnosis of disease. However, before these technologies can be broadly deployed in clinical research and diagnostics, it is necessary to thoroughly understand their performance in real world conditions. In this study, we vet the technical reproducibility, data normalization methods and assay sensitivity focusing predominantly on one widely used spatial transcriptomics methodology, digital spatial profiling (DSP). We also compare its performance to a single molecular imager (SMI), a newer platform with single-cell resolution. Using clinically sourced human kidney tissues and biopsies as exemplars, we find that DSP exhibits high rigor and reproducibility. We show that normalization approaches can impact the biological interpretation of spatial transcriptomics data. While there is good concordance between multi-cellular and single-cell resolution methods, there are tradeoffs in cost, execution time and sensitivity of detection, which may affect which approach is chosen. Our study lays a practical foundation for the incorporation of spatial transcriptomics methods into clinical workflows.

INTRODUCTION

Tissues have complex architecture with multiple cell types and cell states. Traditional approaches to analyze bulk gene expression and single cell gene expression destroy the complex architecture of tissues and the relationships between cells and extracellular components. Spatial biology seeks to quantify changes in biomolecules within tissues and map those changes back to regions, structures and even individual cells within the complex organization of the tissue. A variety of approaches have been developed for measuring mRNA and proteins in tissues and some of these have been developed into commercial platforms. Some commercial platforms determine the gene expression of multicellular regions overlaid on a grid of capture spots arrayed across the tissue section (10x Genomics Visium). Others, such as the NanoString GeoMx, utilize user-selected regions of interest (ROI) that can specifically target a histologic structure or lesion for expression analysis. For mRNA measurements with these platforms, whole transcriptome scale interrogation of tissues is possible, which is especially suited for discovery applications. Newer platforms such as the 10x Genomics Xenium, NanoString CosMx, and Vizgen MerScope, have pushed the envelope further and have achieved single-cell resolution in profiling tissues, and some of these are approaching full transcriptome scale.

Beyond their utility in the research arena, the rapid pace of deployment of these technologies has sparked interest in applying them to answer clinical questions using patient-sourced tissues. Formalin-fixed paraffin embedded (FFPE) tissue is the workhorse format of anatomic pathology laboratories and is likely to remain so for the foreseeable future. FFPE biospecimens comprise the largest source of patient-derived materials that can be used to understand the pathophysiology of human diseases, develop new diagnostics, and discover new therapeutic interventions. Spatial technologies that take advantage of this resource can harness the vast potential of tissue archives in pathology departments throughout the world, and have a clear runway for the development of diagnostics tests that leverage FFPE, the industry standard for processing and stabilizing patient tissue samples for clinical testing^1,2. Reassuringly, all the leading commercial spatial transcriptomics platforms are compatible with FFPE tissues.

Simultaneous and spatially registered interrogation of multiple biomolecules can provide unprecedented insight into their biological interactions within tissues and cells. While analyzing as many biomolecules as possible in a single assay (high -plex) is desirable, sensitivity of detection and spatial resolution are also important considerations. However, the trade-off between plex (number of genes) and data quality (sensitivity, specificity and resolution) is not well understood for many of the commercial spatial profiling platforms. Increasing plex also increases the overall cost of the assay, as does increasing the area of the tissue interrogated. This increased cost may be incurred in terms of the reagents themselves (probes, sequencing required for readout) or longer experiment execution times. The expense of spatial transcriptomics experiments has been an impediment to benchmarking the rigor and reproducibility of the results that they generate. However, it will be critical to understand the reproducibility of results before these technologies can be translated into the clinic. Successful clinical implementation will also need to balance the cost of these expensive technologies against the information they could provide to guide clinical decision making.

There are now numerous commercially available platforms to perform spatial transcriptomic profiling, and each system has its own advantages and disadvantages^2,3. Here, we describe our experience with one multicellular (GeoMx DSP) and one single-cell resolution platform (CosMx SMI). We test their rigor and reproducibility, compare normalization approaches, and describe our practical experience comparing results within and across platforms. These results highlight important experimental design considerations that will affect sensitivity, scope, cost, and their potential translation into a clinical test.

METHODS

Human nephrectomy tissues (ethics statement and patient consent)

Tissues were collected in deidentified fashion and with informed consent under the University of Washington’s IRB Study Protocol 1297 and in accordance with the Declaration of Helsinki. Fresh human kidney tissues were sourced from patient’s undergoing nephrectomy for removal of kidney tumors. Samples of uninvolved kidney (4 donors), and a portion of the kidney tumor (renal cell carcinoma, RCC) from one of the donors were fixed in 10% neutral buffered formalin for 24–48 hours and then transferred to 70% ethanol for another 24 hours. Samples from all 4 donors (5 pieces of tissue in total) were paraffin embedded into a single multi-tissue block prior to sectioning. These samples were placed such that all 5 tissues could be covered by a 22 × 22 mm coverslip.

Clinically sourced human kidney biopsy tissues

With approval from the University of Washington’s IRB, electronic health record searches were performed to identify 14 patients with minimal change disease. Kidney biopsies from 3 patients with normal histology that we have reported on previously⁴ were retested together with the minimal change disease biopsies using a whole transcriptome-based probe set.

Digital spatial profiling to reduce reagent utilization and to assess technical reproducibility

Four consecutive 5μm thick sections from the kidney multi-tissue block were applied to charged slides and baked at 60°C for 1 hour. After deparaffinization and rehydration, sections were subjected to heat induced antigen retrieval with Tris EDTA, pH 9 for 15 minutes followed by 1μg/ml proteinase K digestion for 15 minutes at 37°C. All 4 slides were hybridized overnight at 37°C with the Cancer Transcriptome Atlas (CTA) probe mix encompassing 1,811 genes. Per NanoString’s protocol, sections are usually incubated with 250μl of probe mix and coverslipped using a 40×22 mm RNase-free HybriSlip coverslip (ThermoFisher). However, in order to extend the use of the probes, which are the most expensive component of the digital spatial profiling workflow, we utilized 100μl of probe mix per slide and coverslipped sections using a 22×22mm HybriSlip. The following day, all 4 slides were subjected to stringency washes and counterstained with fluorescently labeled antibodies recognizing pan-cytokeratin (Cy3 channel), CD10 (Cy5 channel) as well as DNA (Syto13, FITC channel) (Fig. 1A). One slide was then loaded into the instrument while the others were stored in 2x SSC in the dark at 4°C for staggered collections. On 3 subsequent days, one of the remaining labeled slides was restained with Syto13 for 10 minutes before loading onto the instrument. After loading onto the instrument and scanning, ROI selection was performed, and UV-released barcode probes were collected in staggered fashion into a 96-well collection plate (24 ROI/day; Fig. 1B). After 4 consecutive days of ROI selection, libraries were generated using a SeqCode construction kit from NanoString. The libraries were sequenced using a NextSeq 2000 P2 flowcell (100 cycle kit). Sequenced barcodes were mapped to genes and ROIs using the NanoString GeoMx NGS Pipeline, version 2.0.21.

Figure 1. DSP experiment to assess rigor and reproducibility.

A) A fluorescence scan from the GeoMx DSP of a section of tissue microarray composed of 4 kidney tissues and 1 kidney cancer tissue from 4 donors. B) Experimental design to assess for technical reproducibility. ROI from different regions were collected into wells of a 96-well plate and gene expression from those ROI was quantified. C) Higher power views of ROIs selected for each of 4 histologically distinct kidney structures. D) Spatial cellular deconvolution of individual ROIs reveals the expected cellular composition as a function of histology. The heatmap represents the cellular abundance score (beta) output from the SpatialDecon algorithm using the KPMP detailed single-cell RNA-seq reference data set as input.

Digital spatial profiling on clinically sourced human kidney biopsies

This was performed similar to the procedure described above except for the following modifications. To maximize the number of human kidney biopsies analyzed and extend the use of the Whole Transcriptome Atlas (WTA) probe mix encompassing ~18,000 genes, 5 μm sections from 3–4 individuals were placed on to each slide used in the experiment. Slides were hybridized with the standard buffer volume of 250μl and covered with 40×22 mm HybriSlips. After glomerular and tubular ROI selection and UV-released probe collection, sequencing libraries were generated using a SeqCode construction kit. Sequencing-based readout was performed using a NextSeq 2000 P2 flowcell and the data were processed using the GeoMx NGS Pipeline.

Initial data processing

Analysis of DSP data was performed using the Bioconductor GeomxTools package, which was developed by Nanostring⁵. After reading the data into R, quality control steps were performed to identify problematic probes. For the CTA probe mix, each gene is represented by up to 5 unique probes that recognize different sections of the gene’s mRNA. Probes with a geometric mean <10% of the collapsed gene level estimate (e.g., those probes that don’t contribute much to the overall signal of that gene) were filtered out. In addition, probes with >20% of the observations flagged as outliers were removed. After filtering the probes, probe sets were collapsed to the individual gene level. Multicellular ROIs were deconvolved using the KPMP normal reference single cell RNA-seq data set’s detailed cell type clusters and the SpatialDecon algorithm^6,7. Since ROIs vary in size, cell count and mRNA content, we assessed the performance of normalization methods such as housekeeping gene normalization, 3^rd quartile (Q3) normalization (recommended by NanoString), RUV-III⁸, cyclic loess regression, and quantile normalization⁹. For gene ontology enrichment, using the PANTHER webtool, we queried a list of differentially expressed genes generated by a particular normalization approach and used the full set of 1,811 genes in the Cancer Transcriptome Atlas as background.

Sensitivity comparison: CTA vs. WTA

We compared the performance of the CTA probe mix (1,811 genes) against the Whole Transcriptome Atlas probe mix encompassing ~18,000 genes. We had previously generated CTA measurements from n=12 glomeruli of 3 individuals with histologically normal kidney biopsies⁴. Note, that the CTA probe set used in that prior experiment contained a spike-in of 39 additional genes to detect host genes relevant to the SARS-CoV-2 response. In this study, we performed WTA measurements on n=17 glomeruli from those same 3 individuals. After performing QC and quantile normalization, we identified genes in both datasets that were >LOQ in at least 2 ROIs. All 1,850 genes in the CTA experiment and 7,990 genes in the WTA experiment satisfied this criterion. We then compared the quantile normalized counts of overlapping detected genes and genes detected only in the CTA experiment (averaged across the 3 donors). Comparisons of significance were made with a 2-tailed heteroscedastic Students t-test.

Sensitivity comparison: GeoMx vs. CosMx

Second, we performed single cell transcriptional profiling on the kidney multi-tissue block that we had previously used for reproducibility assessment. We utilized the 1,000 gene universal cell characterization panel on the NanoString CosMx SMI platform. Tissue morphology was delineated using antibodies reactive for pan-cytokeratin, CD45 and cell membranes (cocktail of anti-β2 microglobulin and anti-CD298 antibodies). We tiled 205 fields of view encompassing all the ROIs selected in our GeoMx experiment that assessed reproducibility. After run completion, the data were automatically pushed to the NanoString AtoMx cloud-based storage and analysis platform, where cell segmentation, assignment of transcripts to cells and initial calculation of spatial neighborhoods was performed. Semi-supervised cell typing was performed using AtoMx’s InSituType module¹⁰. For ease of visualization, we used the KPMP normal kidney reference dataset’s coarse cell type clusters as input and permitted up to 3 unknown cell type assignments in order to account for tumor cell types not represented in KPMP⁷. Neighborhoods/spatial niches, top marker genes for cell types and cell-cell proximity analysis were also computed using AtoMx’s built-in analysis modules. UMAPs and spatial projections of cell type assignments and neighborhoods were generated from the Seurat object in R. In order to select pseudo-ROIs corresponding to the ROIs selected in the GeoMx reproducibility experiment, we utilized the Partek Flow analysis platform. Manual pseudo-ROI selection was guided by spatial neighborhood predictions and attempted to approximate the cell number of the ROIs selected in the GeoMx experiment. If the exact structure was not identifiable or not present in the tissue section, a similar structure in close proximity was chosen. 450 genes were shared between the GeoMx CTA and the 1,000 gene CosMx universal cell characterization panel. Across each pseudo-ROI in the CosMx experiment, transcript counts for these 450 genes were integrated and compared to the quantile normalized transcript counts for the corresponding ROI in the GeoMx experiment.

Data availability

The GeoMx CTA dataset to test rigor and reproducibility performed on human kidney tissue sections is available on the Gene Expression Omnibus (GEO) Database GSE277672. The GeoMx WTA dataset performed on kidney biopsies from patients with minimal change disease and normal controls is also available at GEO accession GSE277674. WTA data from the minimal change disease patients are not analyzed or presented as part of this study. Only WTA data from n=17 glomeruli from normal controls (donors C, G, J) were compared to CTA data generated on glomeruli from those same donors and published in a previous study⁴. CosMx 1000-plex single-cell resolution data from human kidney has been deposited at GEO accession GSE278766.

RESULTS

Staggered collection of spatial transcriptomic data using the GeoMx DSP platform

A map of the tissue sections used for the spatial transcriptomics experiment is shown in the GeoMx immunofluorescence overview scan (Figure 1A). Replicate sections were successfully hybridized with the reduced amount of CTA probe and the smaller HybriSlip (see Methods for detailed explanation). ROIs encompassing histologically distinct structures (glomeruli, cortex, medulla, tumor) were collected in replicate, from each tissue patch (Figures 1B, C). After sample collection, library construction, sequencing readout and mapping, all ROIs produced transcriptomic data that passed quality control thresholds. While Q3 normalization is recommended by NanoString, quantile normalization has recently been reported to have better performance in data sets containing samples with different histology⁹. Therefore, we performed quantile normalization and assessed the cellular composition of the ROIs using the SpatialDecon algorithm⁶. This is a transfer learning approach that predicts cell types in the GeoMx ROIs based on cell type profiles derived from a reference single cell RNA-seq data set. This cellular deconvolution from multicellular GeoMx ROIs revealed the expected composition of cell types within the histologically distinct ROIs (Figure 1D). For example, constituent cells of glomeruli, namely podocytes, mesangial cells and glomerular endothelial cells were the main predicted components of glomerular ROIs (Supplementary Figure 1A). Glomerular ROIs had fewer overall cells (Supplementary Figure 1B) and reduced raw transcript counts compared to the larger ROIs from cortex, medulla and tumor (Supplementary Figure 2). Raw transcript counts did not show a consistent pattern across the 4 days of acquisition. Specifically, we did not see a consistent decrease in transcript counts from day 1 to day 4 of collection. Tumor cells were not present in the KPMP normal kidney reference dataset. However, cellular deconvolution of the RCC ROIs uncovered multiple immune cell types that were largely restricted to the tumor. The pattern of cellular composition in replicate ROIs was consistent with a given histology and reproducible across the 4 days of staggered sample collection (Figure 1D). These results showed that a) reduction of CTA probe volume and hybridization area does not adversely affect data collection; b) cellular deconvolution of histologically distinct ROIs produces the expected cell type composition for a given histology, and that c) staggered collection of replicate samples over 4 days produces qualitatively similar data without drop off of signal over that time frame.

Quantification of sources of variability

To examine the different sources of variability in the data, we first performed a principal component analysis (PCA), in order to partition variability among samples. In a PCA, similar samples should cluster closely and will be separated from dissimilar samples. For example, in Fig. 2A, left panel, all ROIs of the same histology cluster together, clearly separated from ROIs with different histology. This indicates that the sample histology contributes the largest differences in gene expression with lesser contributions from donor and acquisition day (Fig. 2A middle and right panels). When we restricted PCA plots to a single histology such as Glomeruli or Cortex (Supplementary figure 3), samples clustered first by donor, and then by ROI, indicating that differences among samples were primarily due to histology, followed by donor, and then ROI. Since consecutive sections were used for data generation, it is conceivable that some of the observed variation in expression was due to variability in ROI selection. Uniform, circular ROIs were used for cortex, medulla and RCC. By contrast, even though we placed ROIs on the same glomeruli over the 4-day experiment, some glomerular profiles remained relatively invariant (Figure 2B, rows 1 and 2), while others changed noticeably (Figure 2B, rows 3 and 4). Tissues are complex structures that have changing cellular compositions at different levels in serial sections of structures. Serial or deeper level sections are often used in clinical laboratories for additional studies or for distribution to other laboratories for specialized testing. Thus, understanding the contribution of compositional, intra-histology variability to the measured gene expression variation is an important consideration for experimental design, ROI selection and clinical deployment. In order to quantify this further, we plotted the distribution of standard deviations for the log2 transformed, normalized expression of each gene grouped according to histology, donor, ROI and acquisition day (Figure 2C). At the top end of the interquartile range, histology accounted for 84.5% of the gene expression variation, with donor-to-donor variation accounting for 7.3% and ROI selection variability accounting for 6.1%. Variability attributable to stability of the specimens in storage over the 4-day span of the experiment and/or technical variation accounted for only 2.0% of the observed variation. These analyses showed that the GeoMx platform exhibited good technical reproducibility in our hands and that the majority of gene expression differences could be attributed to anticipated biological sources of variation such as histology, donor and ROI selection parameters.

Figure 2. Analysis of sources of variation in DSP workflow.

A) Plots of ROIs according to the two greatest principal components (PC1, PC2). Each principal component analysis (PCA) panel is color coded by histology, donor or acquisition day. B) For non-uniform ROIs, the ROI outline introduces another source of variation. When selecting the same glomerulus across 4 consecutive sections, some glomeruli showed relatively invariant profiles (rows 1, 2), while others changed their profiles (rows 3, 4). C) Distribution of standard deviations for measured genes as grouped by histology, acquisition day, donor and ROI.

Comparison of data normalization approaches

The goal when normalizing data is to remove as much technical variability as possible while retaining biological variability. To do so, we assume that there are one or more genes that have relatively consistent expression across a set of samples, in which case any apparent differences in expression for those genes is primarily technical, and we can therefore use those genes to remove the technical variability via normalization. Even so, there are many additional variables (e.g., age of specimen, duration of fixation, storage conditions) and experimental design considerations (e.g., number of biological replicates, sampling variation) that we could not fully assess in this study that will nevertheless impact the interpretation of any spatial transcriptomics experiment. First, we used MA-plots to visualize differences among normalization methods. In an MA-plot, the horizontal axis (A) measures the log₂ average expression level of a gene (so points to the left are lower expressing genes, and those to the right are more highly expressed), and the vertical axis (M) measures the log fold change between the same gene in the two different samples/tissues. Each MA-plot in Figure 3 compares the day 1 sample for a glomerular ROI to a different day. Since these are just different slices from the same tissue, we expect the data to be symmetrically distributed along a horizontal line at zero. This is not the case when no normalization is applied to the data (Figure 3A, Unnormalized). The normalization methods that Nanostring recommends are simple shift normalizations where either a single observation (Q3 normalization), or the geometric mean of a set of housekeeping genes is used to normalize the data. The default normalization recommended by Nanostring uses the third quartile (Q3), i.e. 50^th to 75^th percentile, from each sample to normalize the data. These shift normalizations do not adjust for differences between samples that vary as a function of gene expression (Figure 3A, Housekeeping, Q3 normalized), identified by the red line, which is more centered on M=0 but still curved. Relative log expression (RLE) plots are also helpful in visualizing sample-to-sample variation in high dimensional data and assessing if normalization procedures have successfully removed unwanted variation¹¹. RLE plots shown for selected normalization methods in Figure 3B demonstrate that housekeeping and Q3 normalization reduce differences in relative log gene expression within the different histologies, and Q3 normalization produced a greater reduction in differences across histologies. The RUV-III normalization approach assumes that we can identify genes that do not change expression (negative controls) and then uses replicate observations to identify technical differences that can then be removed⁸. In addition, RUV-III must be tuned to select the number of principal components used to extract technical variability. RUV-III normalization produced curved MA plots similar to the housekeeping and Q3 approaches. RLE plot analysis of RUV-III normalized data was qualitatively similar to housekeeping normalization (Figure 3B, RUV-III). Since it was not possible to adjust for variation as a function of gene expression using these simple normalization approaches, we next tested methods that account for distributional differences between samples. One such method is called a cyclic loess normalization, which was originally developed for normalization of microarray data¹². The red lines in Figure 3 are locally-weighted regression lines (loess lines) that identify the vertical center of the data at each point on the horizontal axis. As noted above, we assumed that there is a set of genes that do not change expression between different samples. We also assumed that up and down-regulation is relatively consistent between any two samples. If both assumptions hold, then the loess line identifies the set of unchanging genes, and if we adjust the data to linearize the loess line and center it on the horizontal line at zero, we will have removed the variability identified by the loess line, which could include both technical and biological variability. We iterated through this process three times (the cyclic part of cyclic loess) to normalize the data. This cyclic loess noticeably improved the log fold change (M) component of the MA plot (Figure 3A, Cyclic loess, red line is now nearly straight and zeroed). Lastly, we explored quantile normalization which has been reported to be a better method for normalizing GeoMx data between samples with different histology⁹. In quantile normalization, observations in each sample are rank ordered according to magnitude, the average for each rank is computed, and then the average for each rank replaces the observed values. All samples then have identical distributions. Of all the methods tested, quantile normalization produced the most symmetrical distribution along a horizontal line at zero, which is in fact a feature of quantile normalization (Figure 3A, Quantile). RLE plots also confirmed that cyclic loess and quantile normalization performed the best in removing technical variation from GeoMx data (Figure 3B, Cyclic loess and quantile).

Figure 3. Assessment of normalization approaches for DSP data.

A) MA plots for a representative glomerulus ROI showing the impact of various normalization approaches on the distribution of the data. B) Relative log expression (RLE) plots showing the impact of normalization approaches. The data for all 4 days of collection of a given ROI are grouped together (i.e. day 1, 2, 3, 4). The underlying data are the relative log expression between a given sample and a pseudo-median sample (generated by selecting the median log expression from each gene). Under the assumption that most genes do not change expression between different samples, the expectation for these plots is that they line up on a horizontal line at zero, with relatively similar box size (the boxes indicate the interquartile range).

Impact of data normalization approaches on biological interpretation

The different normalization methods make different assumptions of the underlying data structure with expectations for the numbers of differentially expressed genes. These can be visualized using a volcano plot, such as is shown for differential expression between glomerulus and cortex ROIs using 3 normalization methods (Figure 4A). Certain genes remained as significantly differentially expressed genes (DEGs) regardless of the normalization approach used (e.g., CDKN1C, VEGFA, PCK1, SPP1). Examining the overlap of significantly changing genes (FDR<0.05, regardless of magnitude of fold change), the greatest number of differentially expressed genes was identified using Q3 normalization, while cyclic loess and quantile normalization appeared to be the more stringent (Figure 4B, C). However, the distribution of DEGs was skewed in Q3 normalization and the majority of the DEGs had lower expression in glomeruli relative to cortex. When filtering for genes with >1.5x fold change in addition to adjusted p-value<0.05, the majority of apparent DEGs identified by Q3 normalization were lost.

Figure 4. Impact of normalization approach on biological interpretation of data.

A) Volcano plots showing the impact of normalization approaches on differentially expressed genes detected between cortex and glomerular ROIs (FDR<0.05, fold change >1.5x). B) Venn diagrams of overlap of significantly changing genes (FDR<0.05) in comparison of glomeruli vs. cortex. C) The number of DEGs enriched in glomeruli and cortex are shown for different normalization strategies with either a simple adjusted p-value <0.05 threshold or also with a 1.5x fold change requirement. D) Ranked list of top 10 Biological Processes enriched in gene ontology (GO) terms of differentially expressed genes in glomeruli (top) and cortex (bottom) are shown for quantile normalized data. If significant, the fold enrichment for a particular GO biological process is also shown for the other normalization strategies, using the same color scheme as in panel B.

The remaining DEGs were overrepresented in glomeruli, which matched the skew in the volcano plot for Q3 normalized data. By design, cyclic loess and quantile normalizations produced roughly similar numbers of upregulated and downregulated genes in glomeruli relative to cortex, which was also reflected in their volcano plots. Quantile normalization yielded the highest overall number of DEGs with >1.5x fold change and adjusted p-value<0.05 (Figure 4C).

The number of identified DEGs impacted downstream analyses, such as Gene Ontology (GO) enrichment. With a higher number of identified DEGs, the enrichment of particular pathways will be reduced when using the same number of genes as the background set (1,811 genes in the CTA probe set). To examine this further, we identified the top 10 GO Biologic Processes enriched for DEGs identified using quantile normalization (Figure 4D). We also show the fold enrichment if the biological process was significant in cyclic loess and Q3 normalized data. The top 10 significant biological processes in glomeruli identified by cyclic loess and quantile normalizations were identical, but Q3 only identified 5 out of these 10 as being enriched (Figure 4D, top). Notably, the GO biological process ‘Rho protein signal transduction’, which is central to podocyte biology, was not identified as significantly enriched in the Q3 normalized data. Q3 normalization only identified 5 out of 8 genes in this pathway as DEGs, whereas both cyclic loess and quantile identified 8 out of 8 DEGs. When examining pathways enriched in cortex, GO enrichment analysis of quantile and cyclic normalized DEGs identified pathways related to various metabolic processes, which are known to be important to kidney cortex function (Figure 4D, bottom). The combination of the increased variability and the skew of DEGs genes with adjusted p-value<0.05 towards cortex in the Q3 normalized data penalized detection of GO biological processes, and Q3 normalization identified none of the top 10 biological processes detected in quantile normalized data. These results suggested that the more stringent normalization methods (cyclic loess, quantile) decreased technical variability and increased the ability to discern important biological pathways between histologically distinct ROI such as glomeruli and kidney cortex.

Examining gene-level data also showed that the more stringent normalization approaches, quantile and cyclic loess, decreased variation of gene expression for ROIs within and between donors. For example, the VEGFA gene was highly expressed in glomerular podocytes and showed good intra-ROI reproducibility; however, VEGFA’s ROI-to-ROI and donor-to-donor variation was reduced more using cyclic loess and quantile normalization strategies (Supplementary Figure 4). Without orthogonal validation, which is not possible to do for every gene, it is unclear whether this represents true biological variation (to be measured) or technical variation (to be normalized). Therefore, the choice of normalization methods will affect the downstream analysis and the choice of one method over another may be dependent on the experimental system and goals. The vendor-recommended Q3 normalization may be suitable for many applications but increased variability in the data may obscure important differences in biology. Overall, quantile normalization appears to be the best at minimizing technical variation and for detection of biological differences.

Sensitivity comparison: GeoMx CTA vs. WTA

In GeoMx experiments, profiling the whole transcriptome requires extensive sequencing that adds to the cost of the assay. In many clinical scenarios, interrogation of a focused gene set is desirable to reduce costs and to ease interpretation of the resulting data. To assess the impact of a focused vs. whole transcriptome probe set on gene detection, we turned to GeoMx experiments that we had performed using the CTA and WTA on the same set of tissues. The GeoMx reproducibility analysis described above used the CTA probe set, which encompasses 1,811 genes. Each gene within the probe set is detected by up to 5 distinct probes, and the geometric mean of their signals is used to generate the observed expression value. By contrast, in the Whole Transcriptome Atlas (WTA) probeset, each of the >18,000 target genes is detected by only one oligonucleotide reporter. Only 76 of the target genes in the CTA share a probe with the WTA (Kaitlyn LaCourse, NanoString, personal communication), therefore, the majority of the WTA probes are unique to that probe set. This raised two questions: 1) does the CTA have better performance than WTA due to the increased number of probes per target gene? 2) are expression values generated using WTA comparable to CTA in terms of linearity of response? To assess this, we profiled n=17 glomeruli from n=3 patient biopsy samples using the WTA probe set (Figure 5A). We compared the results to n=12 glomeruli, sampled from those same patients, that were profiled using the CTA in our previous study⁴. After calculating the limit of quantification (LOQ) for each ROI using NanoString’s recommended method, we identified all genes where at least 2 ROIs had expression values greater than the LOQ. All 1,850 genes in the CTA experiment and 7,990 genes in the WTA experiment met this permissive threshold. Of the 1,850 measured genes in the CTA experiment, 839 were also measured above the LOQ in the WTA experiment. That meant that 1,009 genes (54.5%) were not detectable in the WTA experiment (2 genes, ACTB, HLA-DRB4 were not present in the WTA panel). When we plotted the Q3 normalized counts of the 839 detectable genes shared using the two probe sets, we observed that there was reasonable linearity of response (r=0.83) (Figure 5B). This indicated that even though different probe sequences were utilized in the CTA and WTA, they still produced comparable results. However, the measured counts using WTA were only ~13% of the CTA, across the detectable overlapping target genes in this analysis. As shown in Figure 5C, the average Q3 normalized expression value measured by CTA was much greater for the 839 overlapping genes compared to their measured values using WTA (76.8 vs. 12.3, p=2.82×10⁻⁷⁰, 2-tailed t-test). For the 1,009 genes that were not detected in the WTA experiment, their average Q3 normalized expression using the CTA (43.7) was significantly lower than that for the 839 overlapping genes (p=2.66×10⁻²¹, 2-tailed t-test). This experiment showed that reducing the number of probes/target gene in the hybridization reaction, also reduces the target’s measured expression value. Genes with lower measured expression values may fail to be detected if using a panel with only one probe/target gene, such as the WTA.

Figure 5. Comparison of GeoMx CTA vs. WTA.

A) Overview of comparison of GeoMx experiments of glomerular gene expression from n=3 normal kidney biopsies (middle biopsy had two fragments). Blue dots indicate the location of the glomeruli profiled in each experiment. B) Dot plot of quantile normalized counts of genes measured in glomeruli from kidney biopsies using CTA vs. WTA. C) Dot plots of expression of individual genes averaged across glomerular ROIs and measured using either CTA or WTA. Violin plots are overlaid to demonstrate the overall distribution. The black dot for each group represents the mean.

Sensitivity comparison: GeoMx vs. CosMx

The GeoMx uses an ROI-based strategy to generate spatially localized gene expression profiles. These ROIs are multicellular regions (usually targeting a minimum of 100 cells) and therefore, the resulting expression profiles represent an aggregate of the expression levels of all cells within the ROI. Newer spatial transcriptomics platforms with single-cell resolution (CosMx SMI, Xenium, MERScope and others) have been released to circumvent this limitation. In one implementation, the CosMx Spatial Molecular Imager hybridizes gene-specific probes to tissue localized targets followed by rounds of in situ detection using fluorescent branched DNA reporters and high-resolution imaging¹³. While whole transcriptome detection has recently been demonstrated¹⁴, more routinely, approximately 1,000- or 6,000 genes can be simultaneously profiled using current commercially available probe sets. Importantly, these single-cell platforms use imaging rather than sequencing as their readout modality. They are also more expensive on a per-sample basis than ROI-based profiling approaches. It is therefore necessary to understand the technical tradeoffs to determine if the research question requires single cell resolution and justifies the increased cost. To do so, we first performed a 1000-plex CosMx single-cell expression profiling experiment on a section of the multi-tissue block we had previously used for the reproducibility analysis. We tiled 205 fields of view (FOV) to capture gene expression at single-cell resolution across the 5 tissue patches (Supplementary Figure 5A). A total of 284,728 cells were resolved with an average of 132 transcripts detected per cell (10%-90% range = 22 – 274 transcripts/cell). Cloud-based analysis was performed on the AtoMx platform. Semi-supervised cell-type assignment and clustering was performed using AtoMx’s InSituType module¹⁰ (Supplementary Figure 5B). We used the KPMP normal kidney reference dataset as input and permitted up to 3 unknown cell type assignments in order to account for tumor cell types not represented in KPMP⁷. Two of the unknown cell types (b, c) were almost exclusively localized to a distinct neighborhood comprising the tumor specimen (Supplementary Figure 5C, D). A heatmap of expression of the top marker genes for each cell type identified the expected pattern of expression for known kidney cell types. Unknown cell type ‘b’ likely corresponded to RCC tumor epithelial cells since they expressed CLU, LDHA and CD24^15–17 (Supplementary Figure 6A). Unknown cell type ‘c’ likely represented tumor endothelial cells, since it expressed IGFBP3 and VWF, which we have recently characterized as RCC tumor endothelial cell selective marker genes in an orthogonal single cell RNA-seq study using different patient tissue samples¹⁸. From the marker gene expression heatmap in Supplementary Figure 6A, unknown cell type ‘a’ has expression intermediate between proximal and distal tubules, though it does not express these genes very strongly. When mapping the cell type into the tissue context (pink cells in Supplementary Figure 6B), this unknown cell type is best interpreted as a type of proximal tubule cell. By contrast, proximal tubule cells, which should be the most abundant cell type in kidney cortex, appear to be scant in the normal kidney, which likely represents an artifact of cell typing. These apparent cell type identification errors may be due to the reduced resolving power of the 1000-gene panel employed in the experiment, which does not necessarily include the best marker genes to distinguish kidney cell types.

A closer examination of an individual FOV of normal kidney cortex revealed satisfactory delineation of cellular outlines from the immunofluorescence preview scan (Figure 6A). Within these segments, cellular identity could be deduced from expression profiles and InSituType transfer learning from the KPMP single cell RNA-seq reference. Cell-to-cell proximity matrices (Supplementary Figure 6C) were used to construct spatial neighborhoods/niches that recapitulated biologically reasonable structures such as glomeruli and tubules. A spatially registered expression heatmap for selected genes could also be generated and produced cellular patterns of localization similar to the orthogonally produced KPMP single cell RNA-seq atlas (Figure 6B).

Figure 6. 1000-plex spatial transcriptomics of human kidney tissue using CosMx SMI.

A) Sequential annotation of CosMx data allows for granular cell and neighborhood annotation, including gene-level spatially registered expression heatmaps. B) UMAP of scRNA-seq data from the KPMP reference data set. Cell populations are indicated that are relevant to the expression of the exemplar genes IGFBP5 and APOE shown in panel A.

These initial analyses gave us confidence that the CosMx platform was capable of recapitulating key geographic and transcriptional features of kidney tissue. To determine if the sensitivity of detection was different for the CosMx platform compared to GeoMx, we defined multicellular pseudo-ROIs in the CosMx dataset to match the ROIs selected in the GeoMx experiment (Figure 7A). The number of cells in the GeoMx ROIs was comparable to those in the CosMx pseudo-ROIs (Figure 7B). We integrated counts for the 450 genes present in both datasets (GeoMx Cancer Transcriptome Atlas, CosMx 1K Discovery Panel) and found that there was good correlation (r=0.68) of gene expression levels, though a few outliers were evident (Figure 7C). Similar to the CTA, the CosMx 1K Discovery Panel uses 5 probes to detect each gene (Kaitlyn LaCourse, NanoString, personal communication). Even so, the integrated counts for the CosMx pseudo-ROIs were 71% of the GeoMx ROIs. Therefore, even though the GeoMx platform uses a sequencing-based readout and the CosMx platform uses an imaging-based readout, both produce generally comparable patterns of gene expression. Though some genes do not follow the trend, there is on average reduced signal when single-cell resolution is desired.

Figure 7. Comparison of GeoMx vs. CosMx.

A) Left, Immunofluorescence overview of a portion of kidney tissue profiles using the CosMx platform and a 1000-gene panel. Middle, the expression data was processed to infer cell type and cellular neighborhoods. Right, individual cells within histologic structures were grouped together as pseudo-ROI in order to correspond to the ROI chosen in the reproducibility experiment (compare to Figure 1C). B) Plot of cell counts in CosMx pseudo-ROIs vs. cell counts based on nuclear staining in corresponding GeoMx ROIs. C) Correlation of integrated counts in CosMx pseudo-ROIs vs. their corresponding GeoMx ROIs for genes shared in both assays.

DISCUSSION

There is increasing emphasis on improving the replicability of scientific studies, in grant applications, publications and clinical trials^19–21. Understanding how experimental procedures and technologies influence the ability to produce rigorous and reliable data forms the bedrock of replicability. Determining the technical requirements for generating rigorous and reproducible spatial transcriptomics data is also critical to aligning these expensive technologies with experimental questions. It will also form the basis for making financial decisions when planning research studies and for determining which spatial transcriptomics technologies may be amenable to clinical translation. In this study we demonstrate that altering the hybridization procedure for the GeoMx platform allows the researcher to extend the probe set to more slides, increasing flexibility in experimental design, and decreasing the per slide cost. The real-world scenario in which this may be helpful is the transfer of unstained slides between institutions for multi-center studies. Our results also demonstrate that the greatest source of variability is histology, which is the desired experimental variable. Inter-donor variation is the next greatest source of variation, followed by variation of feature (ROI) selection within and across multiple sections. Our results also demonstrate that it is better to prioritize collection of biological replicates over technical replicates. Reassuringly, after probe hybridization, samples were stable in storage and variation due to staggered acquisition over 4 days was the smallest contributor to the observed differences. Assessment of raw transcript counts also confirmed that there was not a consistent decline in signal over the 4-day course of the experiment. These findings demonstrate that the GeoMx platform allows for flexible spatial interrogation of transcriptomes with relatively little technical noise, allowing for the identification of biological differences in gene expression such as those between histologically distinct structures and among donors.

When analyzing transcriptomic data from the GeoMx, Q3 is the vendor-recommended normalization procedure. As a simple shift normalization, we found that it was less effective at reducing variation among samples, as compared to cyclic loess and quantile normalization. However, it is important to realize that different normalization methods will influence results and identification of DEGs, and selection of the normalization method may be influenced by experimental goals. For research questions and clinical test development, it may be prudent to explore the data using different normalization strategies with the understanding that some differences may be biological while others could represent false positives. Based on our results, and in agreement with a recent publication⁹, we recommend quantile normalization since it has better performance than the manufacturer recommended Q3 normalization.

Within the GeoMx platform, there are 2 off-the-shelf probe sets for use with human tissues, the CTA and the WTA. Since GeoMx probes are designed with a UV-cleavable linker and reporter oligonucleotide, expanding the target space from CTA (1,811 genes) to WTA (~18,000 genes) incurs a significant probe synthesis cost. This is somewhat mitigated by reducing the number of the probes/target gene from 5 probes/gene in CTA to 1 probe/gene in the WTA probe set. Reassuringly, when we compared DEGs for the CTA and WTA probe sets on the same set of kidney biopsies, we found that there was good correlation in expression trends. However, the WTA was only ~13% as sensitive as the CTA and genes with lower expression were more likely to not be detected using WTA. Thus, probe set choice can influence experimental results and increasing plex from CTA to WTA comes at the significant costs of decreased sensitivity, reduced ability to detect lowly expressed genes, and increased sequencing costs. This also suggests that implementing a GeoMx workflow in the clinic would benefit from a smaller number of target genes being analyzed, but with more probes/target to maximize sensitivity and reduce costs. Initial exploratory analyses using the WTA could be used to define the gene subset that is maximally informative for the clinical question at hand before synthesizing the targeted panel of probes. Recently developed bioinformatic approaches may assist in the design of such targeted gene panels for spatial transcriptomics²².

The CosMx platform permits subcellular resolution of spatial transcriptomic data. Although computational cellular deconvolution strategies can estimate cell type abundance within regions of interest, the higher resolution of CosMx allows for more precise identification of cell types, as well as the identification of novel cell types and cell states. The relationships between cells and cellular neighborhoods, can also be defined with CosMx data. Our analysis confirmed that CosMx could properly identify many of the known kidney cell types that have been identified in the KPMP, as well as to predict novel cell types that are only found in tumor tissue. Comparing the gene expression between the GeoMx CTA and the CosMx 1000-plex gene panel showed good correlation (r=0.68) for the 450 genes that were detected on both platforms in spite of their different methods of readout (sequencing vs. imaging). Therefore, the somewhat reduced sensitivity and increased cost/sample of single-cell resolution platforms such as CosMx must be balanced against the potential of increased resolution to provide biological insight into tissue physiology and disease processes. These factors will also dictate which clinical scenarios will benefit from CosMx’s single-cell resolution vs. the multi-cellular resolution afforded by GeoMx.

In summary, we demonstrate that the GeoMx and CosMx spatial transcriptomics platforms are robust tools for spatially registered gene expression analysis. The GeoMx platform lends itself to greater flexibility and potential cost savings by varying the hybridization area and probe set design and volume. Choices for data normalization for GeoMx will affect results and the detection of DEGs, though quantile normalization appears to perform the best among the methods that we tested. The potential sources of noise (donor-to-donor variation, variation due to storage time after probe hybridization, and intrasample ROI selection of biologically similar structures) are small compared to the biological variation that can be measured when selecting histologically distinct structures (e.g. glomeruli, cortex, medulla and RCC). Probe set composition (number of probes/target gene) as well as platform methodology (sequencing readout vs. imaging) can also impact sensitivity of detection. The CosMx platform provides single-cell resolution, though there is a reduction in sensitivity of detection, increased reagent costs, lower sample throughput and longer acquisition time compared to the GeoMx platform. For clinical test development, it will be critical to look at all of these steps, probe selection and design to optimize signal to noise, detection methods (sequencing vs. imaging), and normalization strategies as they pertain to the downstream analytic that is being reported. As we show here, understanding how these technical and platform considerations influence experimental results will help guide researchers through the planning and execution stage of spatial transcriptomic experiments and facilitate their adoption to answer clinical questions.