Spatially informed gene signatures for response to immunotherapy in melanoma

Abstract

Purpose:

We aim to improve the prediction of response or resistance to immunotherapies in melanoma patients. This goal is based on the hypothesis that current gene signatures predicting immunotherapy outcomes show only modest accuracy due to the lack of spatial information about cellular functions and molecular processes within tumors and their microenvironment.

Experimental Design:

we collected gene expression data spatially from three cellular compartments defined by CD68+macrophages, CD45+leukocytes and S100B+tumor cells in 55-immunotherapy-treated melanoma specimens using Digital Spatial Profiling-Whole Transcriptome Atlas (DSP-WTA). We developed a computational pipeline to discover compartment-specific gene signatures and determine if adding spatial information can improve patient stratification.

Results:

We achieved robust performance of compartment-specific signatures in predicting the outcome to ICI in the discovery cohort. Of the three signatures, S100B signature showed the best performance in the validation cohort (N=45). We also compared our compartment-specific signatures with published bulk signatures and found the S100B tumor spatial signature outperformed previous signatures. Within the 8-gene S100B signature, 5 genes (PSMB8, TAX1BP3, NOTCH3, LCP2, NQO1) with positive coefficients predict the response and 3 genes (KMT2C, OVCA2, MGRN1) with negative coefficients predict the resistance to treatment.

Conclusion:

We conclude that the spatially defined compartment signatures utilize tumor and TME-specific information, leading to more accurate prediction of treatment outcome, and thus merit prospective clinical assessment.

Translational Relevance

Our approach to predicting immunotherapy outcomes in melanoma patients through spatial gene signatures holds substantial translational potential. By focusing on the spatial aspects of gene expression within tumors, we provide oncologists with a refined method to forecast how patients will respond to immune checkpoint inhibitors (ICIs). The development of a computational pipeline that incorporates the heterogeneity of the tumor microenvironment (TME) represents a significant advancement in personalized oncology. The identified S100B signature has shown to outperform existing gene signatures, indicating a leap forward in the accuracy of patient stratification for immunotherapy. Such precision could lead to tailored treatments, minimizing unnecessary exposure to ineffective therapies and enhancing the likelihood of beneficial outcomes. The translational implications extend beyond melanoma, potentially setting a precedent for the application of spatial profiling in other cancer types, thereby optimizing immunotherapeutic approaches.

INTRODUCTION

Melanoma is highly immunogenic and is responsive to immune checkpoint inhibitors (ICI). Over the past decades the incidence of melanoma has increased (1, 2). Although ICI are the standard of care for advanced melanomas, over half the patients do not respond or develop resistant disease with time (3, 4). Moreover, these drugs are now being used in the adjuvant setting, where many patients would survive without therapy, and robust predictive biomarkers are needed to enable selective treatment of those more likely to respond (3). Previous studies have developed gene signature models derived from bulk RNA-seq data to stratify patients according to objective response (OR) to ICI (5). One important limitation of bulk RNA-seq is the combination of cells from both stroma and tumor in variable proportions. Given the quantitative nature of gene expression data, it can be difficult to deconvolve the functionally relevant cell type-specific signals from average signals derived from bulk RNA-seq. Previous signature studies have showed only modest success (6). In fact, the area under the curve (AUC) of the Tumor Inflammation Signature for the prediction of OR in advanced melanoma is 0.67 with 95% CI (0.54–0.82) (6, 7). To date, no biomarker gene signatures derived from bulk RNA-seq have shown clinical utility to predict the outcome from ICI treatment. This may be due to a limited understanding of multicellular interactions within the tumor and TME (tumor microenvironment), the inability to unmix the gene expression of stromal cells from tumor cells, and a lack of ability to include spatial information in the gene signature.

Single-cell RNA-seq can be a complementary and powerful tool to dissect intratumoral transcriptomic heterogeneity (8, 9) and generate extensive information on cell type-specific gene expression profiles (GEP). Although such information could significantly impact the clinical management of melanoma, the need for fresh patient tissues which are not archived is a major limitation for clinical study design (10). DSP-WTA (Digital Spatial Profiling-Whole Transcriptome Atlas) works optimally on formalin-fixed, paraffin-embedded (FFPE) tissues and enables in situ hybridization against 18,190 genes in areas of interest, such as tumor cells, macrophages, and leukocytes, at high throughput using a sequencing readout. Most importantly, this approach enables the capturing of gene expression profiles in the tissue with spatial information defined by different molecular compartments.

Here, using DSP-WTA, we generated the first spatial transcriptomic map from pretreatment samples of 59 melanoma patients (discovery cohort) treated with ICIs (pembrolizumab; PEMBRO, nivolumab; NIVO, or ipilimumab plus nivolumab; IPI+NIVO), who had unresectable stage III or IV melanoma at the time of treatment. Our aim was to understand the spatial gene expression enrichment within the tumor and TME and to develop spatially defined compartment-specific gene signatures that would enable us to predict immunotherapy outcomes. We built a computational pipeline to develop robust compartmentalized signature models for analyzing compartment/cell type-specific WTA data that uniquely provided differentially expressed (DE) genes in spatially defined molecular compartments compared to bulk RNA-seq observations. Compartment-specific (i.e., S100B, CD68, CD45) signatures achieved a higher AUC (area under the curve) for the prediction of OR to ICIs as compared to a pseudo-bulk signature. Moreover, cross-testing in different compartments (i.e., CD68 signature in CD45, S100B compartments, and pseudo-bulk etc.), showed poor performance indicating that signatures are compartment-specific in predicting immunotherapy outcomes. Our compartment-specific signatures further validated in a second cohort and outperformed published bulk signatures (5, 11–14).

MATERIALS AND METHODS

Study design

In this study, we used DSP-WTA to generate a spatial whole transcriptomic map (~18,000 genes) from pretreatment samples of advanced melanoma patients (discovery cohort, N = 59) who had unresectable stage III or IV at the time of ICI treatment. We built a computational framework to develop robust compartmentalized signature models from three cellular compartments-WTA data to predict the treatment outcome. To enable the efficient analysis of tissues from multiple samples, FFPE specimens were examined in a Tissue Microarray (TMA) format. The selection of Regions of Interest (ROIs) from tumor samples is a detailed process aimed at capturing the most representative and informative parts of the tumor for thorough analysis. The ROIs were derived from TMAs featuring 0.6mm diameter cores. To construct these TMAs, we first assessed the quality of the archived pretreatment FFPE tissues, focusing on its preservation and checking for the presence of any artifacts. The tumor samples then underwent a comprehensive histopathological evaluation by a board-certified pathologist. This expert marked the areas within each tumor tissue section on hematoxylin and eosin (H&E) stained slides that best represented the tumor’s pathology to identify regions of interest (ROIs) that are representative of the tumor and its microenvironment. This careful histopathological evaluation established the foundation of selection region of interest (ROIs) in our study. Response Evaluation Criteria in Solid Tumors [RECIST] version 1.1 was used to classify best overall response as complete response (CR), partial response (PR), stable disease (SD), or progressive disease (PD), and to determine the best overall response rate (BOR) (15). Response (CR) or Partial Response (PR) are classified as responders. In contrast, those with Stable Disease (SD) or Progressive Disease (PD) are considered non-responders. We then tested the performance of the developed signatures in the validation cohort (N=46). We also compared our compartment-specific signatures with published bulk signatures (5, 11–14) to identify the performance differences. Moreover, cross-testing of each signature was performed in different compartments (i.e., CD68 signature in CD45, and S100B compartments etc.) to determine the compartment specificity of the gene signatures. Finally, we assessed the association of treatment arm with the signature scores from each compartment by generating the combined score of the genes within each signature.

Study population

Two study cohorts contain retrospective collections of advanced melanoma tissue samples treated with PD-1-based immunotherapies from 2011 to 2017 (discovery cohort, N=59) and from 2017 to 2020 (validation cohort, N=46) at the Yale Cancer Center (New Haven, CT). Both cohorts include tissues collected retrospectively from patients before undergoing immunotherapy treatment, enabling a focused investigation on the impacts within PD1-naïve patient models, thus enhancing the specificity and relevance of our analysis. Four patients from the discovery cohort and one patient from the validation cohort were excluded due to insufficient tissue availability for RNA-seq processing. The data cutoff date was September 1, 2020. Clinicopathological data were collected from clinical records and pathology reports and are shown in table 1 and supplementary data S1 and S2. Patients with uveal melanoma were excluded from this study. All patients provided written informed consent or waiver of written informed consent for tissue sample collection. The study was approved by the Yale Human Investigation Committee protocol #9505008219 and conducted in accordance with the Declaration of Helsinki adopted by the world medical association.

Table 1. Baseline characteristics of the patients.

Clinicopathological features of both discovery and validation cohorts are shown.

Characteristics	Discovery cohort (2011–2017)	Validation cohort (2017–2020)
	N (%)	N (%)
Overall	59 (100)	46 (100)
Age
Median (range)	62 (16–88)	66 (31–88)
Gender
Male	33 (56)	27 (59)
Female	26 (44)	19 (41)
Stage
II	0 (0)	2 (4)
III	1 (2)	5 (11)
IV	58 (98)	39 (85)
Mutational status
BRAF	18 (31)	14 (30)
NRAS	8 (14)	10 (22)
None	32 (54)	20 (43)
Prior immune checkpoint blockade treatment
Yes	16 (27)	6 (13)
No	43 (73)	40 (87)
Treatment status
PEMBRO	23 (39)	9 (20)
NIVO	11 (19)	14 (30)
IPI + NIVO	25 (42)	23 (50)
Response
CR	10 (17)	11 (24)
PR	16 (27)	8 (17)
SD	17 (29)	8 (17)
PD	16 (27)	18 (39)

GeoMx-DSP for 18,000 genes

TMA slides were processed using the Nanostring GeoMx DSP manual slide preparation GeoMx^®-DSP protocol (MAN-10150–01). To enhance the reproducibility, we used two different TMA blocks (identified here as blocks 1 and 3) containing different tumor areas of the same tumors (block 1 contains ROI1 for each patient, and block 3 contains ROI2). Briefly, antigen retrieval of the FFPE tissue were performed for 20 minutes followed by deparaffinization and rehydration. The incubation time to retrieve the antigen varies depending on the tissue types according to the assay protocol. The slides were then exposed to proteinase K and incubated for 20 minutes according to the assay protocol prior to the application of RNA probes onto the tissues for in situ hybridization. The following day, coverslips were removed from the slides and the stringent washes were performed to remove off-target probes for 50 minutes. To further delineate areas within each ROI, morphology markers: CD68 for macrophage compartment (Cy5/666 nm, excitation 645/19 nm, emission 683/30 nm), CD45 for leukocyte compartment (Texas Red/615 nm, excitation 588/19 nm, emission 623/30 nm), S100B for tumor compartment (Cy3/568 nm, excitation 538/19 nm, emission 564/15nm) and SYTO 13 for nuclear stain, were added. The slides were then incubated for 1 hour.

Following incubation and washing, slides were loaded onto the GeoMx DSP instrument. Scanning the slides, AOI (area of interest: i.e., CD45, CD68 and S100B) selection and probe collection were performed according to the GeoMx^®-DSP user manual protocol (MAN-10088–03). Samples were collected in a 96-well probe aspirate collection plate and each AOI representing a compartment from a patient tumor core was collected in each well. In DSP, overlapping of CD45 and CD68 markers due to the co-expression in macrophages presents a challenge in signal specificity. To manage this, the collection of AOIs were prioritized based on marker intensity and compartment size - first CD68, then S100B, and lastly CD45. This strategy, while enhancing specificity for CD68 and S100B compartments, may cause the CD45 compartment to include cells from the remaining unselected tissue.

After sample collection has completed, the next step was to prepare GeoMx-NGS (next generation sequencing) readout library preparation. This process was done according to the GeoMx^®-DSP user manual (MAN-10117–01). Each GeoMx -DSP aspirate in the plate contains photocleaved DNA oligos comprised of an RNA analyte identifier, a unique molecular identifier (UMI) barcode, and primer binding sites. Prior to purification, PCR reactions were pooled into three mixtures, separating CD68, CD45 and S100B AOIs to generate three pooled libraries. The libraries were pooled in a biased manner designed to target the smallest ROI pool with 6 parts of reads for CD68, 3 parts of reads for CD45 and 1 part of reads for S100B. Illumina i5 x i7 system adapter sequences and unique dual sample indices were added when PCR is performed on the probe aspirates. Each photocleaved oligo in the GeoMx -DSP collection plate contains a readout tag sequence identifier (RTS ID) that identifies the target. It also includes a unique molecular identifier (UMI), to remove PCR duplicates when converting reads to digital counts. Read 1 (SPR1) and Read 2 (SPR2) are binding sites for Illumina sequencing primers. The GeoMx Seq code primers that hybridize to SPR1 and SPR2 contain i5 or i7 indexing sequences as well as P5 or P7 sequences for binding to Illumina flow cells. Sequencing was performed in both lanes of an Illumina Nova-Seq 6000 S2 flow cell resulting of 3 billion reads at the Yale Center for Genome Analysis (YCGA).

The sequencing data were processed with the GeoMx NGS Pipeline (DND) according to the GeoMx^®-DSP user manual (MAN-10153–01). Briefly, the pipeline was performed for trimming, merging, and aligning the reads to a list of indexing oligos to identify the source probe. The unique molecular identifier (UMI) region of each read was used to remove PCR duplicate reads. This program will convert FASTQ files to DCCs, i.e., converting reads into digital counts. Then, the zipped folder containing the DCC files were transferred back to GeoMx -DSP to obtain the read counts.

After uploading the files to GeoMx -DSP, the sequencing quality (quality control) allows for specifying the parameters for evaluating the quality of NGS data and to qualify segments by number of cells or by area. Read counts were filtered setting the threshold for the limit of detection ($\text{LoD}$) in an ROI. This was defined based on the mean and standard deviation ($\text{s}.\text{d}$.) of $log-10$ normalized target group genes. The calculation is: $\text{LoD}=\text{mean}+\left(2\times \text{s}.\text{d}.\right)$. A gene was filtered/counted as detected only if it exceeded the $\text{LoD}$ in at least 10% of all ROIs. This means that for a gene to be considered in the analysis, it must be detectable above a certain threshold in at least 10% of the ROIs. In the selection and filtering of ROIs, the following thresholds were implemented to ensure the integrity and quality of the data analyzed: 1). An ROI must exhibit a minimum of 1000 raw reads to be considered for further analysis, 2) Only ROIs with 80 or more reads aligning to the reference genome are included, 3) ROIs must have at least 80 stitched reads, indicating successful merging of read pairs, 4) ROIs should possess 80 or more reads after trimming for quality and adapter sequences, 5) The sequencing effort for an ROI is considered saturated and adequate if at least 50% of the sequencing reads are unique, 6) An ROI is expected to have a geometric mean of negative probe counts above 2, ensuring low background noise, and 7) A stringent control for contamination is applied, where ROIs must have less than 1000 counts in no-template control samples. ROIs failing to meet these criteria were excluded to maintain the high quality of data necessary for accurate further analyses. . For downstream analyses, the discovery cohort containing 55 patients and the validation cohort containing 45 patients were analyzed. To validate the findings of previous signatures generated from Bulk RNA-seq, we obtained the raw filtered read counts above $\text{LoD}$ without normalization and performed the expression vectors by using TPM normalization method. The gene expression levels in each compartment, as measured by DSP, can be inherently confounded by the number of cells within that compartment. To ensure that the predictive power of the model for treatment outcome is truly rooted in the identification of unique transcriptional programs, rather than on variations in cellular density within each compartment, quantile normalization was performed within each compartment as opposed to across compartments, was performed for all the target genes for downstream analyses.

Calculating pseudo-bulk mRNA for validation of training cohort based on external bulk signatures

We combined blocks 1 and 3 for each compartment by taking the mean raw counts for each gene across blocks per patient. We then normalized the expression vectors by using TPM. We then constructed a pseudo-bulk RNA-seq expression matrix, whose rows are the union of all patients across compartments, and whose columns are the union of all genes across compartments. For patient $i$ and gene $j$, we summed the TPM values for each compartment associated with that patient/gene combination (in patients for which a given compartment is absent, the compartments present will therefore be weighted more highly in the resulting pseudo-bulk expression matrix, hence reflecting their increased contribution). Then, for the final normalization, we recalculated TPM values across each row to create the normalized matrix $M^{TPM}$ (i.e. $M_{ij}^{TPM}=1e6*M_{ij}^{\text{'}}/{\sum }_j{M}_{ij}^{\text{'}}$, where $M_{ij}^{\text{'}}$ is the summed expression for patient $i$ and gene $j$ across compartments), and Z-scored the expression values for each gene after a log2 transformation (i.e. $Z_{ij}=\left({\text{log}}_2\left(M_{ij}^{TPM}\right)-{\mu }_j\right)/{\sigma }_j$), where ${\mu }_j=\frac{{\sum }_i{\text{log}}_2\left(M_{ij}^{TPM}\right)}{N},{\sigma }_j=\frac{{\sum }_i{({\text{log}}_2\left(M_{ij}^{TPM}\right)-{\mu }_j)}^2}{N}$ are the mean and standard deviation for gene $j$ respectively, and $N$ is the number of patients. To calculate the values of each external signature, we took the unweighted sum of the Z-scores across each gene in the signature for each patient.

Computational pipeline for developing compartment-specific signatures

We developed compartment-specific signatures for each of the 3 compartments, S100B, CD68 and CD45, as well as for pseudo-bulk. For the individual compartments, we started with a matrix $M_{ij}$ consisting of the quantile normalized expression counts for each subject and gene in that compartment, averaged across blocks 1 and 3. For each patient, we also have the response, $R_j$ which is 1 or 0 for a responder/non-responder respectively.

We then trained a set of $m=1\dots 300$ LR (logistic regression) models with a LASSO penalty to predict $R$. For each model, we divided the samples into an 80/20% training/testing split. We note that throughout this section, we use ‘training’ and ‘testing’ to refer to subsets of the discovery cohort. For each compartment, we perform 300 unique 80/20 data splits. Each of these splits results in a distinct training and validation set, which are then used to train a separate model, culminating in a total of 300 different models per compartment (i.e. 900 models across the three compartments).. The purpose of multiple random splits is to introduce variability that ensures the robustness of the resultant gene signatures by mitigating overfitting to any specific data partition. The validation cohort remains untouched throughout this procedure, so that the performance on the validation cohort is an unbiased estimate of the generalization of our signatures. For each training split, then, we selected genes which are significantly differentially expressed on the training partition at $FDR<0.2$ (to allow the LASSO to choose from a sufficient number of genes), using edgeR (16). We used this set of genes to optimize a LASSO model, using 10-fold cross-validation to find the $\lambda$ penalty, starting from 100 random seeds. The 100 random seeds are used to ensure robustness of the models for each of the 80/20 splits to initialization conditions (we note that this process is repeated for each of the 900 compartment-specific data-splits above). Hence, for a given data-split, we trained 100 sets of coefficients, such that ${\beta }_{s,j}$ is the coefficient of gene $j$ in the model using seed $s$. We then selected all genes having a non-zero coefficient for at least one of the seeds. To get the final coefficients for model $m$, we then refitted the LR model without the $\lambda$ penalty, using only those genes selected, i.e. we found the coefficients ${\beta }_{m,j}^{\mathit{\text{final}}}$ for model $m$ and gene $j$, so that the final LASSO score for patient $i$ in model $m$ is $s_{m,i}={\sum }_j{\beta }_{m,j}^{\mathit{\text{final}}}{M}_{ij}$.

In order to create a final compartment signature based on the 300 trained LASSO models, we calculated the AUC of each model $m$ on its test partition, $AU{C}_m$, and select all those models for which $AU{C}_m>0.7$. The final signature then uses all genes appearing in at least $t$ models, i.e., ${\sum }_s\left[{\beta }_{s,j}\ne 0\right]\ge t$. We set $t$ to the largest value such that the number of selected genes was at least 30. We calculated an intermediate signature with ~30 genes, by again retraining an LR model without the $\lambda$ penalty using only these selected genes, and including all patients in the original cohort, leading to the intermediate signature score $s_i^{\mathit{\text{inter}}}={\sum }_j{\beta }_j^{\mathit{\text{inter}}}{M}_{ij}$ for patient $i$. To calculate the final signature, we then used a step-down procedure to compactify the intermediate signature, so that the resulting signature has less than 10 genes. Hence, we ordered the coefficients by their absolute size (such that $\left|{\beta }_1\right|\ge \left|{\beta }_2\right|\ge \dots$), and for a compact signature of size $j\text{'}$, it has the value $s_i^{j^{\prime }}={\sum }_{j=1\dots j^{\prime }}{\beta }_j^{\mathit{\text{inter}}}{M}_{ij}$. To find the final signature, we selected the $j^{\prime }\le 10$ with the best accuracy on the discovery cohort, $j^{*}$, i.e., $s_i^{\mathit{\text{final}}}=s_i^{j^{*}}.$ Hence, evaluating the final signature score requires only summing the expression values across the reduced set of genes, weighted by their final coefficients. In summary, our final signature may be described as a logistic regression model trained on LASSO selected genes, with the lowest absolute coefficients pruned.

The method above is identical for all three of the individual compartments. For the pseudo-bulk signature, we again started with the averaged quantile normalized matrices, $M_{ij}^{S100B},M_{ij}^{CD68},M_{ij}^{CD45}$. We then constructed a new pseudo-bulk RNA-seq expression matrix $M_{ij}^{p-\mathit{\text{Bulk}}}$, whose rows are the union of all patients across compartments, and whose columns are the intersection of all genes across compartments. The values in $M_{ij}^{p-\mathit{\text{Bulk}}}$ are simply the sums of the gene expression across the individual compartments (some patients may not have all compartments present). The training for the signature is identical to above, using the matrix $M_{ij}^{p-\mathit{\text{Bulk}}}$ as input.

Testing compartment specificity of signatures

We tested the compartment specificity by testing each signature in all other compartments and compared the resulting AUC values with the AUC of each signature in its own compartment. To determine the statistical significance of the difference in AUC scores, we used the test of DeLong et al. (17).

Compartment-level deconvolution of bulk mRNA for validation of compartment-specific signatures

To deconvolve the bulk expression data into compartment-specific expression, we initially obtained the quantile normalized pseudo-bulk matrix generated by Nanostring software version 0.4. We then deconvolved this based on the 10 cell-type deconvolution matrix in CIBERSORTx (18). Hence, we used the known signature matrix $B$ and our input pseudo-bulk matrix $M^{\mathit{\text{nanoString}}}$ to find the cell fractions $F$ by solving $B\times F_{,i}\approx M_i^{\mathit{\text{nanoString}}}$. Here, $F_{ji}$ is the fraction of cells belonging to cell-type $j$ in patient $i$. To construct the pseudo-compartments, we used our own matrix $C$, where $C_{ij}$ indicates the fraction of cells of type $j$ which belong to compartment $i$. For this, we use only 6 cell-types, and associate Macrophages with CD68, NK, T-CD4, T-CD8 and B cells with CD45, and Malignant cells with S100B. Finally, to construct the pseudo-compartment matrices, we set $M_{ijk}^{p-\mathit{\text{comp}}}={\sum }_c{B}_{jc}{F}_{ji}{C}_{kc}$, where $M_{ijk}^{p-\mathit{\text{comp}}}$ is the pseudo-compartment expression of gene $j$ for patient $i$ in pseudo-compartment $k$, and the index $c$ ranges across the 6 cell-types we consider.

Validating compartment-specific signatures in an independent cohort

To validate the signatures, we used the same processing as above on our validation cohort to calculate expression matrices averaged across blocks 1 and 3 for each compartment and pseudo-Bulk. We then evaluated the final signature scores $s^{\mathit{\text{final}}}$ for each patient using this data and assessed the AUC and survival probability to test the out-of-sample accuracy of our signatures. Within each signature, genes with positive coefficients predict response, while genes with negative coefficients predict resistance. We therefore also evaluated the partial signatures for each compartment formed using only positive and negative genes respectively. The statistical significance of the AUC on the hold-out data was assessed using the method of DeLong et al. (17).

Comparing signatures according to treatment status

To assess the association of treatment arm status with the signature scores from each compartment, we generated the combined score of the genes within each signature. We then compared the signature scores of the responders and non-responders using a two-tailed $t$-test for each treatment arm (i.e., PEMBRO, NIVO and NIVO+IPI). We ran similar comparisons on both discovery and validation cohorts, using the average expression per patient across blocks, as defined above.

Comparing S100B signature with external signatures

We performed Weighted Gene Coexpression Network analysis (WGCNA, see (19)) on the S100B expression data, using a signed network with a minimum module size of 20, which generated ~30 modules. We calculated a consistency score for our S100B signature, by summing, for each gene $g$ in the signature, the number of genes in the external signatures that were in the same module as $g$, normalized by the total number of genes in the module. To assess the significance of the score, we ran a permutation test by sampling 1000 random 8-gene sets from all expressed genes, calculating a null distribution of consistency scores for each as above, and calculating the proportion of samples with scores exceeding that of our S100B signature.

Data and materials availability

Data availability:

The NanoString DSP-WTA raw RNA sequencing data of both the discovery and the validation cohorts reported in this study were made available with the accession number GSE233305. Raw, unnormalized, and unfiltered, RNA counts for all for all ~18K genes on the WTA panel are provided as online Supplementary Information. Additional data requests may be directed to the corresponding author.

Code availability:

The in-house scripts for our data analysis pipeline are available in GitHub with the link https://github.com/tznaung/Mel_SpatialSig.

RESULTS

Using DSP pseudo-Bulk data to validate training cohort against prior signatures

As the data generated in this study were derived from newly developed technology, namely the DSP-WTA 18000 genes approach developed by NanoString, to confirm our retrospective cohort data is comparable to the data generated by the previous technologies, we tested 6 pre-existing melanoma signatures developed using melanoma bulk-RNA-seq (5, 11–14). To test the bulk-RNA signatures, we generated pseudo-bulk RNA seq data from the expression matrices derived from different compartments (see Methods). The process for testing pre-existing bulk-RNA signatures is shown in Fig. 1A. The regression between gene expression data extracted from whole tissue using the Nanostring IO360 panel (~770 genes) (6) and pseudo-bulk shows significant per-gene correlation values ranging from 0.3 – 0.8 (Spearman’s Rank) (Supplementary Fig. S1A), demonstrating good agreement. Since we do not have the coefficient values of the preexisting signatures, we Z-scored the signature values and compare responders to non-responders. Responders have higher signature scores than non-responders in 4 out of 6 signatures tested in our pseudo-bulk data (Fig. 1B and C) indicating pseudo-bulk data is a good surrogate for bulk-RNA seq data generated using whole tissues and also showing that the retrospective cohort we use as a discovery set is similar to clinical trial materials used to generate the six previously published signatures (5, 11–14).

Fig. 1. Validation of published bulk-signatures in DSP-WTA pseudo-bulk data.

(A) Schematic diagram illustrating how pseudo-bulk data were generated from compartmentalized RNA expression matrices. (B) The heatmap illustrating the Z-scores of the six published signatures tested in pseudo-bulk data. (C) The scatter plot showing the signature scores of the responders and the non-responders. Significance was determined using p < 0.05* and p < 0.01**. (A, created with BioRender.com)

A pipeline for compartment-specific signatures derived for DSP compartments

Representative images of spatial gene profiling from three AOIs (area of interests) from each ROI (region of interest) before and after masking for S100B, CD68 and CD45 compartments are shown (Fig. 2A), for tumor, leukocytes, and macrophages respectively, although we note that each mask additionally captures transcriptomics from neighboring cell types. We collected two ROIs (called blocks) from each patient, in some cases exhibiting substantial tumor heterogeneity (See Supplementary Fig. S1B and C). Molecular compartmentalization defined by AOIs are identified through fluorescence-guided labeling of molecular compartments or image segmentation. Therefore, in our study, RO comprises the entire spot within the TMA, and the AOIs are designated regions within the ROI, marked by identifiers such as S100B, CD68, or CD45 (Supplementary Fig. S2A–D). To make optimal use of the DSP spatial expression data, segmented by molecular compartmentalization, we developed a novel pipeline for training compartment-specific signatures using this data (see Methods and Fig. 2B). Briefly, we split our discovery cohort into multiple training/testing 80/20 partitions (300 partitions), and for each fit a LASSO-based signature model by repeated optimization starting from 100 random seeds each. We then formed an intermediate large signature consisting of ~30 genes appearing with greatest frequency in the LASSO models achieving AUC > 0.7 from the 300 partitions. Finally, we compressed these signatures to form final signatures of ~10 genes, by using a step-down procedure where we added genes ordered by the absolute value of their coefficients, choosing the combination with the best performance in the discovery cohort. By limiting our signatures to at most 10 genes, they may be used in clinical settings where whole transcriptome sequencing is impractical, but a small number of genes may be tested using methods such as qRT-PCR or RNA-scope. We thus decreased the size of the signature for future prospective use on the basis of this step-down method. Fig. 2B shows the schematic of the pipeline and Fig. 2C shows the AUC performance of signatures formed during the step-down process to compress the intermediate signatures. As shown, each compartment achieves good performance ( > 0.7) within the 10 initial genes. The upper threshold of 10 was chosen as a practical signature size for potential clinical assay development. Note also that the pseudo-bulk signatures performed substantially worse for a given number of genes than the compartment specific signatures.

Fig. 2. Development of compartment-specific signatures.

(A) Representative figures of the tumor core before and after masking the AOIs (areas of interests) for DSP-WTA RNA probes collection. To generate the masks, S100B, CD45 and CD68 markers were used for tumor, leukocytes and macrophages respectively. (B) Schematic diagram showing the pipeline of how one spatial signature was developed. A similar workflow was used to generate all four compartment-specific signatures. (C) Compressing the large signatures to form final signatures of ~10 genes, by using a step-down coefficient procedure, where we add genes ordered by the absolute value of their coefficients, choosing the combination with the best performance in the discovery cohort. (B, created with BioRender.com)

Additionally, we have generated two further models to compare the performance of our multi-split sample approach. These include a straightforward LASSO model using DE genes across the entire discovery dataset, with lambda set to generate non-zero coefficients for an alternative 8-gene signature, and a simple forward-selection model for DE genes using the S100B compartment data, where the 8 genes with the lowest DE p-values are selected and their coefficients are fitted using logistic regression (Supplementary Fig. S3A–B and Supplementary Table S1). As shown, the performance of both models is substantially lower than that of the S100B signature trained using our multi-split approach. We observed that both these baselines only achieve AUCs below 0.5 on the validation set, indicating the robustness of our split-sample approach. When we compared the genes between the S100B intermediate large signature consisting of ~30 genes and the forward selection DEG signature and the single split gene signature models, although all the genes in the DEG and single split signatures were present in the intermediate signature, only PSMB8 remained in the final 8-gene S100B signature. This suggests that, by selecting genes that are most likely to be relevant across different data samplings and not artifacts of a single data partition, our method identifies a subset of candidates, many of which are distinct from DEG or single-split signature genes, allowing us to select a signature which is more robust than these other approaches. To further investigate the relationship of the genes selected in our S100B signature and the external bulk expression signatures, we performed Weighted Gene Coexpression Network analysis (WGCNA, see (19)) on the S100B expression data, and tested whether the genes in our signature were preferentially found in the same modules as the external signature genes. We observed this to be the case, despite minimal overlap of individual genes (p=0.023, permutation test, see Methods), suggesting broad agreement at the pathway level with respect to external signatures.

Compartment-specific signatures reflect compartment-specific expression patterns

The genes for each spatially defined signature category were illustrated in supplementary table S2. As shown, the coefficients in the signature may be both positive and negative; for positive coefficients (shaded in grey), this indicates that higher expression of these genes is beneficial (will increase the signature score), and hence is predictive of response, while negative coefficients (shaded in blue) indicate that high expression of these genes is detrimental (will decrease the score) and is predictive of resistance. We achieved best performances with the 8-gene S100B signature, 8-gene CD68 signature, 6-gene CD45 signature, and 5-gene pseudo-Bulk signature. All signature genes are unique to their compartment. The AUC of each signature for predicting OR is as follows: AUC: 0.86, for S100B (Fig. 3A), AUC: 0.94 for CD68 (Fig. 3B), AUC: 0.98 for CD45 (Fig. 3C), and AUC: 0.70 for pseudo-bulk (Fig. 3D) signatures. This shows that compartment-specific signatures outperform the pseudo-bulk signature. As a first validation of the importance of spatial specific gene expression, we evaluated the specificity of the signature performance by cross testing these signatures in all other molecular compartments. We observed a substantial drop in the performance of each signature when applied to different molecular compartments (Fig. 3E–G). The significance levels for the differences in signature performances during cross testing was determined using DeLong et al. test. S100B tested in S100B compartment significantly outperformed CD68 (p=9.6e-05), and CD45 (p=0.0001) signatures. Similarly, CD68 tested in CD68 compartment outperformed S100B (p= 1.2e-30), and CD45 (p=0.009). The observed performances were similar in CD45 compartment as well. Due to the low performance of the pseudo-bulk signature, we leave this signature out of the remaining analyses. The quantile normalized expression counts of genes from the signatures are shown in supplementary Fig. S3C–E.

Fig. 3. Performance of compartment-specific signatures in the discovery cohort.

The performances of (A) S100B, (B) CD68, (C) CD45 signatures in their own compartments, and (D) pseudo-bulk are shown. Cross-testing (each signature tested in all possible compartment) is shown for the signatures in (E) S100B, (F) CD68, (G) CD45 compartments.

Compartment-specific signatures outperform bulk signatures in an independent cohort

To validate our compartment-specific signatures, and compare their performances with bulk signatures, we collected samples from an independent cohort of melanoma patients from 2017 to 2020 (validation cohort, N=45) at Yale Cancer Center and applied the DSP WTA pipeline to collect spatial expression data (see Methods). Similar to the discovery cohort, we collected two ROIs from each patient, in some cases exhibiting substantial tumor heterogeneity. To account for the heterogeneity, we tested each spatial signature in the averaged expression data from both blocks. Our 8-gene S100B signature validated with AUC=0.79 (p=0.003), and 6-gene CD45 signature predicted AUC=0.66 (p=0.07) (Fig. 4A). Further, we performed cross-testing of the signatures in different compartments in the validation cohort. This showed poor performance of signatures when tested in different spatial compartments, confirming compartment-specificity (Supplementary Fig. S4A–D). To compare the performance of compartment-specific signatures with previously published bulk signatures (5, 11–14), we tested our signatures and a set of 4 published bulk signatures in the tumor compartment (equivalent to testing in micro-dissected tumor tissue). We observed that the S100B signature outperformed all others (Fig. 4B and C). None of the previously published bulk RNA signatures had an AUC=0.70, supporting our previous observation that compartment-specific signatures outperform bulk signatures. Additionally, we applied our S100B tumor signature to two previously published bulk-RNA sequencing data (N=90 and N=41) since many bulk-RNA seq data derived from laser micro-dissected tumor tissues, to isolate tumor regions or tumor cells. This approach aimed to provide a rigorous validation of our signature, utilizing available datasets to overcome the extensive time and resources required for new data generation, thereby enhancing the credibility and relevance of our findings. We achieved AUC: 0.74 with p < 0.001 and AUC: 0.69 with p = 0.027 indicating the reliability of our signature (Supplementary Fig. S5A–B).

Fig. 4. Performance of compartment-specific signatures in the validation cohort.

The performances of (A) S100B, CD68, and CD45 signatures in their own compartments are shown. (B) Testing of compartment-specific signatures and bulk-derived published signatures in S100B/tumor compartment is shown. (C) Test of difference in AUCs for each signature against the S100B signature in the S100B compartment of the validation cohort is shown. P-values in (A) and (B) represent tests for significance of predictive performance for each AUC individually.

Compartment-specific signatures predict overall survival

We then assessed the signature scores associated with overall survival (OS) from the start of immunotherapy treatment. We first assessed the survival probability in the discovery cohort and established the optimal cutpoint (Fig. 5A–C). We tested three different cutpoints/stratifications including median, 1^st tertile versus 2^nd +3^rd tertile and 1^st tertile versus 3^rd tertile in the discovery cohort. The hazard ratios, p-values (log-rank) and adjusted p-value (Bonferroni) are shown in supplementary table S3. The cutpoint that delivered the best hazard ratio was selected to test the association of signature score with the survival probability in each compartment of the validation cohort. We observed the median cutpoint to be the best cutpoint for CD45, and the tertile cutpoint (1^st versus 3^rd tertile) to be the best cutpoint for S100B and CD68 compartments. Using these cutpoints, we generated Kaplan-Meier survival curves in a validation cohort. We observed that patients with higher S100B scores show significantly better survival in the validation cohort (Hazard Ratio, HR = 0.2, p = 0.024) (Fig. 5D) and higher CD45 and CD68 signature scores show a trend of better OS (HR = 0.35, p = 0.28 and HR = 0.76, p = 0.64, respectively) although not significant (Fig. 5E and F).

Fig. 5. Predicting survival using compartment-specific signatures.

Upper panels show the ability of the signatures scores to determining the survival probabilities in (A) S100B compartment, (B) CD68 compartment, and (C) CD45 compartment of the discovery cohort. Lower panels show the same for (D) S100B compartment, (E) CD68 compartment, and (F) CD45 compartment of the validation cohort. Cox regression was used, and significance was determined using p < 0.05*, p < 0.01**, and p < 0.001***.

Signatures achieve higher performance in de novo compartments than pseudo-compartments using computational deconvolution of bulk data

As shown above, our results suggested that spatially informed signatures perform better than bulk signatures. We were interested in testing the performance gained by using spatial data as opposed to deconvolved bulk data. We thus developed a deconvolution approach to estimate pseudo-compartment expression data from bulk data. We used CIBERSORTx to allow us to deconvolve pseudo-bulk expression data into compartment-specific expression (i.e., pseudo-S100B compartment, pseudo-CD68 compartment and pseudo-CD45 compartment) (Supplementary Fig. S6A). The inputs for the CIBERSORTx deconvolution analysis are the constructed pseudo-bulk transcriptome and the bulk data by Gide.et.al (20). The method first associated the pseudo compartments with cell-types in a pre-existing cell-type deconvolution matrix (see methods), which were then tested with the compartment specific signatures. We observed poor performance with AUC:0.56 (S100B), AUC:0.61 (CD68) and AUC:0.59 (CD45), when testing our signatures in the CIBERSORTx-derived pseudo-compartments generated from our pseudo-bulk (Supplementary Fig. S6B). When testing the data of the pseudo compartments by Gide.et.al (20), except for the CD45 signature that predicts the outcome with AUC:0.72, p=0.0001, while we observed the poor performances of the CD68 and S100B signatures with AUC:0.55 (CD68) and AUC:0.4 (S100B), respectively (Supplementary Fig. S6C). This computational deconvolution method using pseudo-compartments thus indicated, as expected, that compartment-specific signatures perform better in de novo compartments than in computationally defined pseudo-compartments.

Responders have higher signature scores across treatments

Since this study was done on a retrospective collection rather than a clinical trial, we assessed the association between signature scores and treatment regimen for each compartment. Unpaired $t$-tests comparing signature scores between responders and non-responders for each treatment in the discovery cohort (i.e., PEMBRO, NIVO and IPI+NIVO) showed that responders have higher signature scores in general (Supplementary Fig. S7A–C). More prominent effects were observed in the PEMBRO and IPI+NIVO groups than in the NIVO group. In the validation cohort, we observed a similar trend, although significance was not detected except for the S100B compartment (Supplementary Fig. S7D–F). Since not all compartments we present in all patients, the number of patients per compartment varies in each cohort. In the S100B compartment, significantly higher scores were observed in responders receiving IPI+NIVO. Although the trend of higher signature scores was also observed for PEMBRO and NIVO, the number of patients receiving these regimens was too low to result in significance. In general, higher signature scores were observed to predict response across treatment regimens.

DISCUSSION

The spatial transcriptomic technology we used in this study, in which gene expression profiling is performed at three different spatial locations in samples from immunotherapy-treated patients with advanced melanoma, is a powerful technology for exploring the functions of complex cellular phenotypes and their interrelationships (21–23). Here, we developed a novel computational analysis pipeline to generate robust gene signature models from digital spatial sequencing readouts derived from small subregions/spatial regions of the tissue. To the best of our knowledge, our study is the first to generate spatially informed compartment-specific whole transcriptome data and learn associated gene signatures from ICI-treated advanced melanoma patients. We observed that the pseudo-bulk data we generated by combining the expression profiles from three molecularly defined compartments of the tissues (i.e., CD45, CD68 and S100B regions) were comparable to the bulk RNA-seq expression profile collected from the whole tissue region. Pseudo-bulk analysis serves as a powerful tool to merge cell-type-specific expression profiles, effectively simulating the broader data landscape captured by bulk RNA-seq yet maintaining the distinct molecular identities of individual compartments. The compartment-specific signatures from three specific compartments we developed here were validated in an independent ICI-treated melanoma cohort. We used a robust cross-validation train/test randomly shuffled and stratified dataset splits approach to derive a compartment-specific gene signature model for ICI treatment resistance/response with high biological plausibility. Although several pioneering studies have developed gene signatures derived from bulk RNA-seq to predict the outcome of ICI treatment in melanoma (5, 11–14), so far there have been no biomarker gene signatures that were incorporated in clinical practice. By integrating a step-down coefficient procedure with an advanced machine learning approach, we have developed a model that identifies approximately ten genes per compartment, demonstrating superior performance for potential clinical application. This contrasts with traditional methods, such as applying LASSO directly to the DE genes, or the identification of top DE genes as signatures, as demonstrated in our results. In selecting genes for our models, we prioritized those with novel contributions to treatment outcomes through unsupervised selection, rather than choosing genes from published pathways and gene ontologies. The CD45 signature’s AUC of 0.66, while aligning with previous studies (24), diverges from traditional immune cell activity markers. This divergence underscores our aim to explore genes beyond prior established gene sets, in the hopes of uncovering unique immunological pathways and their implications for treatment responses.

Given the “intra-tumor heterogeneity” occurring in solid tumors (25), we used two ROIs (i.e., different regions of the same primary tumor) for both discovery and validation cohorts. Out of the four prediction models we developed, three signatures CD68, CD45, and S100B signatures achieved high performance using data that averaged the ROIs for each patient. The genes in each compartment signature (i.e., CD45, CD68 and S100B) are unique to their own compartment and cross-testing of these signatures in different compartments degraded the outcome prediction significantly, confirming their spatial specificity. Since spatial transcriptomics focuses on assigning transcriptomics data to their original locations within the tissue, the spatial transcriptomic data generated in this study can be treated as an alternative RNA-seq derived from laser micro-dissected tumor tissues, which involves physically dissecting tissue sections to isolate specific regions or cells. Therefore, we assessed the performance of our signatures with four previously published bulk signatures (5, 11–14) in the tumor compartment from the validation cohort. This assessment indicated that the S100B-compartment-specific signature achieved the best performance, outperforming the bulk trained signatures (shown in Fig. 4B and C). This indicated that the compartment-specific signatures derived from transcriptomic analysis of specific cell populations within the space of complex tissues are able to differentially evaluate the tumor and the TME and thus may achieve higher accuracy in the prediction of OR to immunotherapy. Additionally, spatially informed tumor signatures (masked by S100B) may offer an advantage when compared to bulk signatures, inviting future comparison with using laser micro-dissected bulk tumor or single cell sequencing data during training.

Within the validated 8-gene S100B signature, there are five genes with positive coefficients and 3 genes with negative coefficients. The genes with positive coefficients are associated with response while the ones with negative coefficients with resistance. The genes with positive coefficients are PSMB8, TAX1BP3, NOTCH3, LCP2, and NQO1 and the genes with negative coefficients are KMT2C, OVCA2 and MGRN1. Positive prognostic values of LCP2, NQO1, NOTCH3 mutations in several solid cancer types associated with innate immune sensing, T cell differentiation and activation have been documented elsewhere (26–28). TAX1BP3 has been reported to be a negative regulator of Wnt/β-catenin signaling pathway (29) and high expression of TAX1BP3 in patients leads to decreased tumor cell proliferation. An additional element that influences ICI response is the immunoproteasome subunit PSMB8 that degrades foreign proteins into smaller peptides to be presented to the immune system by MHC molecules (30). We observed high expression of PSMB8 in responding patients, while its expression might be associated with overall better prognosis in addition to treatment. Consistent with our finding, PSMB8 has been shown to serve as a positive predictive biomarker to ICI in melanoma patients (31, 32). Genes with positive coefficients such as PSMB8, TAX1BP3, NOTCH3, LCP2, and NQO1, encompass diverse biological functions integral to tumor-immune dynamics. These genes are involved in antigen presentation, immune cell differentiation, stress responses, signal transduction, and immune evasion mechanisms, offering a comprehensive view of the tumor microenvironment. Their collective roles suggest the complex interplay between tumors and the immune system, making the S100B tumor signature a more accurate predictor of ICI outcomes than signatures focusing solely on immune cell presence. Among the genes with negative predictive coefficients in our study, KMT2C has been shown to be the seventh most commonly mutated cancer gene over 33,000 cases by the National Cancer Institute Genomic Data Commons (33). Loss of function mutation of KMT2C might contribute to the changes of cell state from epithelial to a more stem-like cell leading to metastasis (33). High expression of OVCA2 and MGRN1 were reported to be potential prognostic biomarkers. Hence, each gene within the S100B signature holds a unique value in predicting response or resistance to ICI, allowing the combined score of these genes to predict the outcome to ICIs with maximal accuracy.

There are a number of limitations in this study. One limitation is the grouping of three different ICI treatments together (i.e., PEMBRO, NIVO, IPI + NIVO). Although this is not a clinical trial, we observed that higher signature scores are associated with response and our signatures are predictive in all three treatments. Another limitation is the relatively small sample sizes of both discovery cohort (N=59) and validation cohort (N=46) and archival tissues for some patients are not sufficient for RNA sequencing. Therefore, a few patients with insufficient tissue availability needed to be excluded from the study and hence degraded the statistical power when evaluating the performance of the signatures. Additional samples may particularly increase the robustness of the CD45 and CD68 signatures, since these compartments were not observed in all individuals. Further, additional samples would allow signatures to be trained which incorporate interactions with mutational status, and cross-compartment interactions. In addition, we observed the cellular heterogeneity within two ROIs of the same patient for CD45 and CD68, resulting in the decreased performance of these two signatures. The variability in the purity or specificity of cellular compartments identified by markers such as CD45, CD68, and S100B is a result of tumor heterogeneity and the complex tumor microenvironment, where different cell types are closely intermixed. While DSP facilitates the precise quantification of marker expressions in selected ROIs, a marker’s specificity, defined by its exclusive presence in a compartment, is affected by its expression level, the DSP system’s 10μm resolution, and staining quality. Since the GeoMx instrument allows for only four morphology markers where the RNA is to be collected and sequenced, our ability to address the cellular and gene expression changes has been limited (34). The use of TMA in research presents an additional limitation, such as potential biases favoring tumor-centric factors over immune responses in predicting clinical outcomes. While TMA offers significant advantages for high-throughput analysis by allowing the simultaneous study of many tissue samples, thus enhancing statistical power, it has limitations in representing tumor heterogeneity. Even when multiple nonadjacent cores from each patient are analyzed, TMAs may not represent the full spatial heterogeneity of a tumor, overlooking critical aspects of the tumor microenvironment (35). This underscores the need for further research and the development of advanced spatial technologies to more accurately capture the complexities involved in cancer pathology and treatment outcomes. A final limitation concerns the restriction of our methodology to training separate spatial signatures for each compartment, rather than a combined signature that integrates information across compartments. Due to the small size of our discovery cohort, and the absence of compartments in multiple patients, our data is currently under-powered to train such a combined signature, hence we have left this for future work

In conclusion, our work provides a computational framework to explore spatial transcriptomic profiling of several areas of interest, using a robust statistical methodology to discover compressed compartment-specific signatures with strong out-of-sample validation accuracy for S100B. We note that our analytic framework may be straightforwardly adapted to finer-grained compartment definitions as well as cell-type specific expression observations from single-cell RNA-seq data. Translation of these gene signatures into the clinic will require prospective validation in a clinical trial setting (36).