Multiset multicover methods for discriminative marker selection

Summary

Markers are increasingly being used for several high-throughput data analysis and experimental design tasks. Examples include the use of markers for assigning cell types in scRNA-seq studies, for deconvolving bulk gene expression data, and for selecting marker proteins in single-cell spatial proteomics studies. Most marker selection methods focus on differential expression (DE) analysis. Although such methods work well for data with a few non-overlapping marker sets, they are not appropriate for large atlas-size datasets where several cell types and tissues are considered. To address this, we define the phenotype cover (PC) problem for marker selection and present algorithms that can improve the discriminative power of marker sets. Analysis of these sets on several marker-selection tasks suggests that these methods can lead to solutions that accurately distinguish different phenotypes in the data.

Graphical abstract

Highlights

• •Marker selection is critical to define cell types in large atlas studies
• •We present phenotype cover as a conceptual formulation for the marker-selection problem
• •Our algorithms for phenotype cover demonstrate high discriminatory power

Motivation

To date, marker selection is based on methods that focus on each cell type separately and do not consider the relationship between different types. Such methods can select overlapping marker sets for different cell types, making it hard to discriminate between similar cell types. To address this issue and to improve the ability to select a discriminating set of markers, we defined an optimization function for biomarker selection that takes the overlap into account.

Hasanaj et al. propose a marker-selection strategy for improving specificity and selectivity of atlas-scale cell-type assignments. They define an optimization problem for selecting a minimal set of such markers that covers all types. An analysis of the proposed approximation algorithms suggests that these marker sets have high discriminatory power.

Introduction

Several international efforts focus on characterizing gene expression in different tissues, organs, disease states, and more. Examples include HuBMAP, a large NIH effort to reconstruct a three-dimensional (3D) map of the human body at the single-cell resolution,¹ the Human Cell Atlas,^2,3 the Cancer Cell Atlas,⁴ and the Brain Atlas.⁵ One of the first steps of studies at the single-cell level is to characterize cell states or cell types. Typically, this relies on marker genes whose expression or co-expression with other such markers indicates a cell type.^6,7,8 To find such markers, researchers often perform differential expression (DE) testing, where a statistical hypothesis test is used to compare the expression of genes in one group of cells versus all other groups (one versus all). These groups are usually defined by cluster, cell type, or condition labels.

To date, marker selection has mainly focused on the most significant DE genes or proteins for each group. While this works well with a small number of distinct groups (e.g., major cell types), it may not work well when there is a much larger number of groups with overlapping DE genes. In such cases, markers are not just useful for defining a specific group or type but are also critical for discriminating between similar types. Consider these large multiorgan single-cell RNA sequencing (scRNA-seq) datasets. In such datasets, we may be interested in markers that are specific for both a cell type and a tissue (i.e., markers that are uniquely found only in cell types from this tissue). Such markers can be less significant than overall DE genes since they may only distinguish between two similar types, but they are still of major importance. An example is given by the Tabula Muris dataset,⁹ a collection of scRNA-seq profiles of over 100,000 cells from over 20 different organs and tissues in Mus musculus. When analyzing these data, the authors used traditional clustering and DE analysis without considering the issue of cell-type/tissue combination. Another example are T cells, which mature in the thymus.^10,11 While T cells later migrate and reside in tissues throughout the body, the identification of T cells that have recently left the thymus (recent thymic emigrants [RTEs]) plays a role in treatment decisions.¹² Similarly, the role of resident and infiltrating immune cell types is still an active area of research for neurodegenerative diseases. A key challenge is the current inability to distinguish the resident central nervous system (CNS) immune cells and the bone-marrow-derived immune cells.¹³ Better signatures of CNS-specific immune cells and signatures of infiltrating immune cells are needed to understand the immune responses to therapies.

In addition to cell-type/tissue markers, multivariable label partitioning is central to many other questions in functional genomics. Cell type or disease states are often simultaneously considered when identifying markers,¹⁴ and so state-specific markers for cell types are of interest. Deconvolution of cell types from bulk data is also highly dependent on the ability to select not just good markers for each individual cell type but also a set of discriminatory markers between all types.^15,16 Finally, a number of recent single-cell proteomics technologies, including CODEX¹⁷ and Cell DIVE, require the pre-selection of markers to profile. The ability to identify a subset of markers that would suffice for distinguishing all cell types in the sample is a key criterion for such a selection.¹⁸

Broadly, marker selection represents a feature-selection problem. Feature-selection methods can be largely divided into three categories: filters, wrappers, and embedded.^19,20,21 Wrapper and embedded methods interact with a specific classifier. Wrapper methods select (often in a greedy manner) a subset of the features that lead to a classifier with the highest accuracy. Examples include sequential forward and backward selection methods.^22,23 Embedded methods use the output of the classifier itself, which comes in the form of an explicit ranking of the features or implicitly via a scoring system (e.g., information gain in decision trees²⁴). Since these methods are geared toward classification, they may not be applicable to other problems, including deconvolution.

Filter methods, on the other hand, are not tied to a specific classifier. For example, scGeneFit²⁵ selects those genes that maintain a separation of the different cell types similar to that of the original space. This method supports both a flat partition or a hierarchy of labels (e.g., major cell types and subtypes). RankCorr²⁶ works in a one-versus-all fashion and selects markers for a fixed cell type by performing a rank transformation. Another algorithm, Relief,²⁷ and its extension ReliefF²⁸ penalize features that cannot distinguish a given instance from its negative (having a different label) neighbors, while assigning high scores to features that take similar values among instances from the same class. Minimum-redundancy-maximum-relevance (mRMR) selects features that are relevant to the target class but are not similar to each other.²⁹ CIBERSORT¹⁵ and a number of prior methods^16,30,31 analyzed a signature matrix of DE genes to identify submatrices with a low condition number for use in deconvolution of bulk mixtures. Thus, while these methods can successfully select discriminative features when the overlap between sets is small, the ability of such methods to select markers that discriminate all pairs of phenotypes has not been extensively studied.

In this article, we explore the problem of determining a global set of biomarkers. These represent features that collectively distinguish between higher context phenotypes. We assume we are given a phenotype × feature, binary or real score matrix M, whose (i,s) entry represents the relevance of feature s (e.g., average gene expression) for phenotype i. We formulate the task as a combinatorial optimization problem where the goal is to identify the smallest set of features such that for every phenotypic pair (i,j) there exists a set of features that can be used to “distinguish” between i and j. We term this problem phenotype cover (PC). We show that PC is equivalent to multiset multicover, which is nondeterministic polynomial-time (NP)-complete³² and propose two algorithms that can approximate it in polynomial time (STAR Methods). The first is based on the extended greedy algorithm to set cover (G-PC),³³ and the second is based on the cross-entropy method (CEM-PC).^34,35 By analyzing several marker-selection problems, we show that the greedy algorithm outperforms competitors across a variety of tasks. We also analyze some of the specific markers selected by the method and discuss their ability to distinguish between similar cell types.

Results

We developed methods to select discriminative features from a large set of (potentially overlapping) signatures. The goal of the features we select is to enable the separation of the different components in the set. This can either be for a supervised learning (for example, classification) or for other learning approaches such as deconvolution or dimensionality reduction. Our method takes as input a signature or score matrix M, which is used to estimate the importance of a feature for a phenotype of interest. Features are then selected by reformulating the problem as a multiset multicover instance where the goal is to select features such that every phenotypic pair is covered at least K times, for some positive K (Figure 1). We developed two solutions to the multiset multicover problem: the first is based on a greedy approach (G-PC), and the second based on the cross-entropy method (CEM-PC). See STAR Methods for details.

Figure 1.

Graphical illustration of (binary) phenotype cover and its reformulation as a set cover problem

Given a binary score matrix (left), each feature induces a bipartite graph between classes (center left). Edges in this graph form a set ${\epsilon }_s$. Multiset multicover is then performed on the collection of ${\epsilon }_s$ to select a small number of features that “distinguish” all phenotypic pairs (at least K times). The idea can be naturally extended to non-binary score matrices by assigning a multiplicity to each element $e_{i,j}\in {\epsilon }_s$ (STAR Methods).

We tested G-PC and CEM-PC and compared them with eight prior methods: scGeneFit,²⁵ decision trees,³⁶ top differentially expressed genes (TopDE), RankCorr,²⁶ ReliefF,²⁸ mRMR,²⁹ ANOVA F values, and mutual information.^37,38 We used three scRNA-seq datasets from lung, mouse cortex, and a human cell atlas (Table 1). We vary the coverage factor K from 1 to 20 for the idiopathic pulmonary fibrosis (IPF) dataset, from 1 to 40 for mouse cortex (MC), and from 1 to 9 for human cell atlas (HCA). For all baselines but TopDE and RankCorr, we select a number of features that matches the solution size returned by G-PC. For TopDE, we take the union of the top k differentially expressed genes for each phenotype (k varying from 1 to <10). For RankCorr, we tuned the hyperparameters until a similar number of features was returned. Finally, for CEM-PC, all features with a probability score greater than 0.98 after convergence were chosen (Methods S1, alg. 3). We compare all methods in terms of phenotype classification performance, deconvolution of bulk mixtures, and feature stability. We also validate the features selected by G-PC by performing gene set enrichment analysis and comparing them with known markers in the literature.

Table 1.

scRNA-seq datasets used in this study

Dataset	Genes	High var.	Cells	Tissues	Cell types	Reference
Idiopathic pulmonary fibrosis (IPF)	4,443	yes	96,301	1	33	Adams et al.39
Mouse cortex (MC)	20,006	no	3,005	1	7	Zeisel et al.40
Human cell atlas (HCA)	2,968	yes	84,363	15	7	He et al.41

For HCA, we consider a combination of tissues and cell types (85). For IPF, only healthy samples were kept. Endothelial-mural and astrocyte-ependymal pairs of cells were grouped for MC.

Classification

We first test the ability of a classifier to predict the correct phenotype given only a subset of the features. For each method, we select a feature set S using a subset of the data, train a logistic regression model on the same subset, and evaluate performance on left-out data. G-PC exhibits strong performance on the IPF and MC datasets across a wide range of coverage factors. For example, when 42 genes are selected on the IPF data, G-PC obtains an F1 score of 0.70, followed by scGeneFit (0.65) and CEM-PC (0.61) (Figure 2A). On the MC data (Figure S1A), G-PC again performs best when 30–140 genes are selected (F1 ≈ 0.94–0.95). mRMR also performs well on these data except when the number of genes selected is small (<30). Decision trees, on the other hand, do not improve in performance when more than 30 genes are selected (F1 ≈ 0.92).

Figure 2.

Comparison of feature selection methods for the IPF dataset

(A and B) Performance scores for (A) and (B) were averaged across five different random train and test splits. SD is shown as a shaded region.

(A) Performance of a logistic regression model trained on the selected features. G-PC achieves the highest F1 score across all coverage factors, followed by scGeneFit and CEM-PC.

(B) Jensen-Shannon divergence (lower is better) between CIBERSORT-predicted mixture proportions and the ground truth.

(C) Stability scores for all eight methods over 5 runs. Sequential methods like G-PC, decision trees, and CEM-PC suffer slightly in stability compared with other, more global methods. Nonetheless, G-PC shares about 70% of the features across runs.

(D) Selected biomarkers assigned to each cell type (rows [columns] are differentiated by color [shape]). Gene s (column) is assigned to cell type i (row) if there exists another cell type j such that M_i,s − Mj,s ≥ 1 (S. Biomarker validation). Rows and columns were ordered via hierarchical clustering.

(E) For every phenotypic pair, we compute the coverage (i.e., the score difference between the two phenotypes) provided by the selected gene set. A histogram of these coverage factors corresponding to a coverage of 10 is shown for each method. As can be seen, for G-PC and CEM-PC, which optimize for coverage, each element is covered at least 10 times. Other methods provide high coverage for some elements but miss out on others.

These two datasets are obtained from a single tissue. We thus next tested the ability of PC to differentiate between the same cell types across multiple tissues. For this, we used all tissue and cell-type combinations present in the HCA dataset. Decision trees outperform other methods on this classification task (Figure S2A). G-PC is the second-best method when more than 100 genes are selected, while scGeneFit is the second best when less than 100 genes are selected. scGeneFit, however, does not improve in performance when more than 100 genes are selected. At 235 genes, decision trees converge at 0.70, while G-PC and mutual information reach an F1 of 0.68.

We note that scGeneFit can take the hierarchy of labels into account and that the authors describe improved performance when cell subtypes are considered in the MC dataset. For a fair comparison, we ran three different variants of scGeneFit that take advantage of this hierarchical structure and evaluated performance by using a nearest centroid classifier fit on the entire data. All the hyperparameters we used were identical to those provided by the authors. While G-PC does not use cell subtype information, it still outperforms all three variants across a different number of markers (Figure S4B).

We also tested an additional classifier (k nearest neighbors) and observed very similar results to those obtained with logistic regression (Figures S3A–S3C). Finally, we tested the impact of batch effects by using two pancreas datasets^42,43 and observed that our method, G-PC, along with TopDE are the most robust to batch effects (Figure S4A).

Deconvolution

Inferring cell-type proportions from bulk transcriptomics data is an important task in understanding composition of tumors and other tissues. Many methods have been developed to perform deconvolution of bulk mixtures.^16,30,31,44 Deconvolution typically requires solving a linear equation of the form m = Sp, where m is a given mixture vector, S is a signature matrix containing cell-type-specific expression signatures (known), and p is the unknown class proportion vector. One widely used method for deconvolution is CIBERSORT,¹⁵ which uses $\nu$ -support vector regression ($\nu$ -SVR). CIBERSORT constructs the signature matrix S by considering the top k DE genes for every cell-type subset (which leads to the exact same selection as the TopDE baseline we consider in this study). Next, CIBERSORT selects the k that leads to a signature matrix S with the lowest condition number. Finally, $\nu$ -SVR is fit on the data, and the regression coefficients in the solution are used to estimate p.

To test the usefulness of the features selected by our method for deconvolution, we constructed pseudo-bulk mixtures using the IPF, MC, and HCA datasets by averaging expression levels across all single cells in the test sets. The signature matrix S was constructed with features selected from the training set and deconvolution via $\nu$ -SVR was then applied to the pseudo-bulk mixtures. As recommended by the authors, we initialize three linear $\nu$ -SVR instances with $\nu \in \{0.25,0.5,0.75\}$ and save the model that achieves the lowest root-mean-square error between the deconvolution result Sp and m. We compute the Jensen-Shannon (JS) divergence⁴⁵ between the predicted mixture p and the ground truth. G-PC performs well on the IPF data, with RankCorr doing better only when 50–80 genes are selected (Figure 2B). For example, when 163 genes are selected, G-PC achieves an average JS = 0.045, followed by RankCorr (0.056) and scGeneFit (0.062). For the MC dataset, G-PC is also the top-ranking method, though TopDE and RankCorr also accurately resolve mixture proportions (Figure S1B). All three methods obtain a JS score of less than ≈0.025 across all K. CEM-PC performs well on some instances for both datasets; however, the results are unstable and vary between runs. None of the methods clearly outperforms all others on the HCA dataset (Figure S2B). These results demonstrate the challenges of trying to distinguish cell types across tissues.

Finally, we also tested another version of deconvolution that uses linear least squares (LLS) as the target. We observed that, for LLS, G-PC performs no worse than other methods on IPF and MC (Figures S3D–S3F).

Stability

The focus of the comparison so far has been on accuracy. However, other considerations are also important, especially when selecting features that will be used across different platforms and potentially modalities. One such important issue is feature stability.⁴⁶ The stability index measures the average size of the overlap divided by the size of the union for all pairs of feature sets (STAR Methods). To test stability, we randomly sample half the data and compute the stability index for the features selected by each method over 5 runs. Stability scores are shown in Figures 2C, S1C, and S2C. G-PC is more stable than decision trees for IPF and MC. However, due to their greedy sequential nature, both G-PC and decision trees are less stable than more global methods such as ReliefF and F values. Nonetheless, G-PC uses from 60% to 70% of the same genes across all runs. Perhaps not surprisingly, due to its random sampling nature, CEM-PC is the least stable method.

Biomarker validation

To validate the set of biomarkers $\mathcal{S}$ selected by G-PC and decision trees, we performed enrichment analysis for the HCA dataset. We fix a coverage of 8, and for every phenotype i, we select from the solution $\mathcal{S}$ all those genes s for which there exists some phenotype j satisfying M_i,s ≥ M_j,s + 1. We consider each of these sets as a biomarker set for the given phenotype for both G-PC (Figures 2D and S6) and decision trees.

We next performed gene set enrichment analysis (GSEA)⁴⁷ using the HuBMAP ASCT + B marker set⁴⁸ to determine if the selected marker sets for a specific cell type are enriched for pathways associated with these cell types. We test the ability of G-PC and decision trees to identify the correct (1) tissue, (2) cell type, and (3) tissue/cell type combination. G-PC obtains lower q values for 42% (3/7) of the cell types and 54% (6/11) of the tissues (Figures 3A and 3B). No markers were found for four tissues. When tested against the correct tissue and cell-type pair, G-PC obtained lower q values for 71% (30/42) of the pairs (Figure 3C). The remaining combinations (33) either belonged to a tissue that was not present in the marker set or was not identified by either method. Some known markers assigned correctly by G-PC are shown in Figure 3D. Esophagus and trachea tissues were mapped to respiratory system in the ASCT + B set. The top two principal components of the markers that provide coverage for a given tissue or cell type show visible separation between different classes (Figure 4).

Figure 3.

GSEA q values for the HCA dataset

(A and B) We select markers that provide coverage for each cell type for both G-PC and decision trees and perform gene set enrichment analysis (GSEA) using the HuBMAP ASCT + B gene set.⁴⁸ We first record q values for the top entry, which contains the correct cell type or the correct tissue independently. When comparing the ability of each method to assign the correct class, G-PC obtains a lower q value, i.e., higher-log(q value), for 42% (3/7) of the cell types (A) and 54% (6/11) of the tissues (B). We did not find markers for four tissues in the gene set (common bile duct, muscle, rectum, stomach).

(C) When tested for the ability to identify both the correct tissue and the correct cell type, G-PC obtained lower q values in 71% (30/42) of the cases. The remaining tissue/cell-type pairs (33) either belonged to a tissue that was not present in the marker set or was not identified by either method.

(D) Connected by an edge are known markers for CD4 and myeloid cells that were assigned to the correct tissue/cell-type pair by G-PC. Some markers are assigned to multiple cell types (multiple outgoing edges), while others are pair specific.

Figure 4.

Principal-component analysis (PCA) plots of the selected markers for the different phenotypes present in the HCA dataset

(A and B) A total of 121 markers were selected via G-PC (coverage = 5). For every tissue (A) and cell type (B), the top two principal components of the markers providing coverage (≥1) for that phenotype are plotted. The exact number of markers used is shown in parentheses. There is visible separation between classes.

Due to limitations of the marker set we are using, only 20 cell types could be identified for IPF. Among these, G-PC obtains lower q values for 12 (60%) (Figure S5A). We observe good agreement between genes selected using our greedy procedures and genes known to be involved in specific cell types. For example, G-PC correctly assigns KRT19 and ADGRF5 to type I and II epithelial cells (ATI and ATII^49,50,51). CD69 is assigned to both B and T cells,^52,53 COBLL1 is assigned to B cells,⁵⁴ JCHAIN to B and B plasma cells,⁵⁵ CXCL2 to macrophages,⁵⁶ and CCL5, PRF1, and CD247 to natural killer cells.^57,58,59 See Figure 2D for a larger list of identified markers.

In addition to selecting known cell-type markers, G-PC is also able to select markers that distinguish between similar cell types. For example, it assigns CXCL2 to ATII and not to ATI⁶⁰ and CD69 and AFF3 to B cells and not plasma cells.^52,53,61 Another example is A2M and CST7, which are assigned to cytotoxic T cells,⁶² whereas NAMPT and TNFRSF18 are assigned to regulatory T cells.⁶³

Discussion

Selection and use of markers is a common step in many analysis pipelines. Most recently, this topic received increased attention due to the large number of new cell types that have been identified and characterized using scRNA-seq data.^64,65,66,67 To date, such selection was mainly based on methods that focused on each cell type separately and did not consider the relationship between markers selected for different types. Such methods can select overlapping marker sets for different cell types, making it hard to discriminate between similar cell types. This is especially important for large datasets where multiple cell types in multiple tissues are being profiled.^9,41

To address this issue and improve the ability to select a discriminating set of markers, we defined a new optimization function for biomarker selection that takes the overlap into account. Specifically, we defined the PC problem that aims to optimize the accuracy of identifying different sets when using the selected markers. We presented two heuristic filter methods since these lead to solutions that can be used in several different analysis pipelines including classification, deconvolution, experimental design, and more. The first is based on a greedy approximation algorithm (G-PC) and the second is based on the cross-entropy method (CEM-PC).

We evaluated these methods and compared them with prior methods developed for marker selection using several high-throughput scRNA-seq datasets. Our analysis indicates that G-PC assigns equal importance to all different phenotypes in the data and is affected less by class imbalance as shown by the F1 score. Other methods tend to select features that discriminate only dominating classes. Furthermore, G-PC can be used with signature matrices rather than direct expression measurements. In such cases, there is only a single score for all phenotype/gene pairs, which makes using other methods difficult. This allows G-PC to construct signature matrices for deconvolution, which leads to an accurate estimation of cell-type proportions from bulk mixtures. While G-PC is slightly less stable than some other methods, it nonetheless retains most of the features (∼70%) across runs.

Decision trees outperform G-PC with regard to the F1 score in one of the datasets we analyzed (HCA). However, even for HCA, G-PC seems to obtain a more accurate list of cell-type markers based on enrichment analysis. We note that our method is best suited for datasets that require detailed annotations, which usually means that several cell types partially overlap in their markers. In contrast, for large datasets where the focus is on coarser cell types, we see less advantage compared to standard marker selection methods. Finally, we provide a C++ implementation of G-PC with Python bindings, which makes it the fastest method we tested (Table 2). Speed is an important consideration when working with large scRNA-seq datasets.

Table 2.

Runtimes of all feature selection methods for all three datasets used in this study (s)

Data	G-PC	CEM-PC	DT	scGF	DE	RC	FVal	ReliefF	MI	mRMR
IPF	1.3	55	304	102	41	90	1.5	388	1039	980
MC	0.17	227	5	39	2.7	3	0.15	11	116	139
HCA	1.25	88	50	52	30	95	0.8	340	790	310

Method names were abbreviated. 178 features were selected for IPF, 66 for MC, and 121 for HCA. Our C++ implementation of G-PC takes less than 2 s for all three datasets, making it the fastest along with F value computation. Performance tests are conducted on a machine with a 2.3 GHz 8-Core Intel Core i9 CPU and 32 GB memory.

We observed that CEM-PC sometimes selects a smaller set of genes that achieves the same coverage as G-PC. However, due to its random sampling nature, CEM-PC is very unstable and can lead to a completely different set of features across runs.

Limitations of study

The biological analysis relied on computing the overlap with existing gene marker lists that may be incomplete. A more thorough biological analysis of the selected genes and their relation to each cell type might provide more insight into the performance of our algorithms.

While G-PC worked well for the data analyzed in this paper, it does not provide an optimal solution. It is interesting to see if other approximation algorithms that optimize for coverage will lead to better results when tested on biological data.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Deposited data
Idiopathic Pulmonary Fibrosis (IPF)	Adams et al.39	GEO: GSE136831
Mouse Cortex (MC)	Zeisel et al.40	GEO: GSE60361
Human Cell Atlas (HCA)	He et al.41	GEO: GSE159929
Software and algorithms
Multiset multicover algorithm	This paper	Zenodo: https://doi.org/10.5281/zenodo.7158750
Phenotype cover algorithms (G-PC, CEM-PC)	This paper	Zenodo: https://doi.org/10.5281/zenodo.7158780
Experiments in this paper	This paper	Zenodo: https://doi.org/10.5281/zenodo.7158788
scGeneFit algorithm	Dumitrascu et al.25	GitHub: https://github.com/solevillar/scGeneFit-python
Decision Trees, ANOVA F-values, Mutual Information, Logistic Regression	Pedregosa et al.,68 scikit-learn	Zenodo: https://doi.org/10.5281/zenodo.6968622
T-test for differentially expressed genes	Theis Lab; PI: Fabian Theis	GitHub: https://github.com/theislab/diffxpy
RankCorr algorithm	Vargo et al.26	GitHub: https://github.com/ahsv/RankCorr
mRMR algorithm	Peng et al.29	GitHub: https://github.com/smazzanti/mrmr
Approximate nearest neighbors algorithm	Johnson et al.69	GitHub: https://github.com/facebookresearch/faiss
GSEA algorithm	Subramanian et al.,47 GSEApy	Zenodo: https://doi.org/10.5281/zenodo.3748084
Python version 3.8	Python Software Foundation	https://www.python.org/

Resource availability

Lead contact

Further information and requests for resources should be directed to and will be fulfilled by the lead contact, Ziv Bar-Joseph (zivbj@andrew.cmu.edu).

Materials availability

This study did not generate new unique reagents.

Method details

Notation

Let $\mathbf{M}\in {\mathbb{R}}^{P\times F}$ represent a score matrix. We denote by P the number of phenotypes (e.g., cell types) and by F the number of features (e.g., genes). In this paper, we use scRNA-seq read count data denoted by $\mathbf{X}\in {\mathbb{R}}_{\ge 0}^{N\times G}$. Here, N denotes the number of cells and G denotes the number of genes. Given a known vector y of length N representing class labels, we derive a matrix M from X by averaging expression values of cells with the same class label. In this case, P = {number of distinct classes} and F = G. We denote by [n] the set $\{1\text{,}2\text{,}\dots \text{,n}\}$. Finally, let (x) ⁺ = max{x,0}.

Problem formulation and complexity

Phenotype cover (PC)

Given a score (signature) matrix $\mathbf{M}\in {\mathbb{R}}^{P\times F}$, find a subset $\mathcal{S}\subset \left[F\right]$ of minimal cardinality, such that for every $i,j\in \left[P\right]$ with $i\ne j$ , and some fixed positive K, the following holds

$$\sum _{s\in \mathcal{S}}{\left({\mathbf{M}}_{i,s}-{\mathbf{M}}_{j,s}\right)}^{+}\ge K$$

PC is asking for a small subset of features such that for any given ordered pair of phenotypes (i,j), one can find enough features which collectively distinguish i from j by a factor of at least K. This problem allows the selection of a gene which could cover several phenotypic pairs, e.g., multiple cell subtypes vs another major cell type, but also demands sufficient coverage between subtypes themselves. The straightforward solution of iterating over all possible feature subsets satisfying the requirements above and selecting the one with the smallest cardinality, suffers from an exponential complexity in the number of subsets considered. In fact, PC is equivalent to multiset multicover which is NP-complete.³³

To establish this equivalence, it may help to first consider a simplified version of the problem where we restrict M to be binary and K = 1; call this problem PC-B. In this case, we require a small subset of features $\mathcal{S}$, such that for any two phenotypes $i\ne j$ there exists some index $s\in \mathcal{S}$ where M_i,s−M_j,s = 1. Note that in this simplified form, every feature s induces a bipartite graph $G_s=\left(u_s,v_s,{\epsilon }_s\right)$, where

$$u_s=\left\{i|{\mathbf{M}}_{i,s}=1\right\},v_s=\left\{j|{\mathbf{M}}_{j,s}=0\right\}=\left[P\right]\setminus u_s$$

Every edge $e\in {\epsilon }_s$ corresponds to an ordered pair of phenotypes (Figure 1).

Now, given the collection of sets $\epsilon =\left\{{\epsilon }_s|s\in \left[F\right]\right\}$, set cover asks to find the smallest subset ${\epsilon }_{sol}\subset \epsilon$ such that for every element $e\in \cup {\epsilon }_s$, there exists a set in ${\epsilon }_{sol}$ which contains e. It is easy to see that the features corresponding to ${\epsilon }_{sol}$ are the solution to PC-B.

So far, we only considered a binary score matrix. However, a solution to the binary problem can be naturally extended to solve non-binary scoring matrices by assigning multiplicities to the elements of ${\epsilon }_s$. To every e = (i,j) we assign the multiplicity ${\left({\mathbf{\text{M}}}_{i,s}-{\mathbf{\text{M}}}_{j,s}\right)}^{+}$ and view ${\epsilon }_s$ as a multiset. Note that since we are working with real numbers, we need to round the multiplicities to integers. Higher precision can be easily obtained by first scaling both M and K by some scalar c and performing the rounding after. Finally, the requirement K = 1 can also be relaxed by solving for a multicover, where we require each element to be contained at least K times in $\cup {\epsilon }_{sol}$ (counting multiplicities).

Approximating a solution to phenotype cover

Given the NP-Completeness of PC, we present two greedy solutions that run in polynomial time.

Greedy phenotype cover (G-PC)

First, we consider the well-known greedy approach to solving set cover that iteratively picks the set which covers the greatest number of elements not covered yet.^70,71 The algorithm can be trivially extended to solve multiset multicover.³³ The full algorithm is presented in Methods S1, algorithms 1 and 2. Every time we select a set, we need to correct the multiplicities of all the remaining O(F) sets, each of which may contain up to O(P²) elements (all phenotypic pairs). Therefore, if we denote the solution size by k, the run-time complexity of G-PC is $O\left(k{P}^{2}F\right)$. In practice, P is small and $k\ll F$, therefore, the method is almost linear in the number of features considered. The approximation accuracy for this solution was previously analyzed and it was shown that the greedy algorithm for multiset multicover is upper bounded by a factor of H_m increase in the solution size, where $H_m=1+\frac{1}{2}+\dots +\frac{1}{m}\le \log \left(m\right)+1$ and m is the cardinality of the largest multiset.³³

Cross-entropy method phenotype cover (CEM-PC)

In addition to the greedy multiset multicover approach, we developed a new method based on cross-entropy (CEM).³⁴ CEM was originally used to estimate probabilities of rare events and it was later extended to solve combinatorial problems.⁷² Roughly, CEM consists of two steps: 1) generate a random sample based on a specific distribution, and 2) update distribution parameters such that “high-scoring” samples are more likely to be produced in the next iteration. This two-step procedure is repeated until convergence, or until a maximal number of iterations is reached. The final parameters determine the solution to the combinatorial problem (in our case, selecting features whose probability is greater than some threshold). For a more detailed analysis of CEM, the reader may refer to the excellent tutorial of De Boer et al.³⁵

We present a variant of CEM for solving set cover by introducing a scoring function that encourages high coverage but penalizes a large number of features (Methods S1, alg. 3). The run-time complexity of CEM-PC depends on the maximum number of iterations I, the number of random samples per iteration R_s, and the complexity of the scoring function (in this case, the smallest coverage attained per random sample). This leads to a total run-time complexity of $O\left(IR{}_s{P}^{2}F\right)$. In this paper, we use I = 500 and R_s = 1000. In practice, convergence is attained in fewer iterations.

Baselines

As mentioned above, several prior methods have been developed for marker and feature selection. We thus compared our method against several baselines on traditional supervised learning tasks, ability to construct signature matrices for deconvolution of bulk mixtures, and feature stability. Specifically, we compare our method to scGeneFit²⁵ and RankCorr²⁶ which were used for discriminative marker selection. We use the implementations provided by the authors of each method. For scGeneFit, we used a redundancy of 0.1 and kept the remaining parameters at defaults. We compare against an embedded method that uses decision trees with the Gini Index criterion to rank features. Note that here we use decision trees as a feature selection method and not as a classifier. The performance of decision trees as a classifier was worse than that of Logistic Regression using the same features, hence, we excluded these results from the manuscript. We also compare against several other filter methods. We consider the union of the top differentially expressed genes per phenotype as determined by Welch’s t-test⁷³ (TopDE). We compare against ReliefF²⁸ which uses nearest neighbors' information to update feature weights. Since computing exact neighbors is slow for the single cell data we are using, we developed a variant of ReliefF that uses approximate neighbors based on the faiss package.⁶⁹ We compute 30 neighbors per sample. ANOVA F-values and mutual information between gene expression and phenotype are also computed using the popular package scikit-learn.⁶⁸ Finally, we compare against minimum-redundancy-maximum-relevance (mRMR).²⁹ For mRMR, we use the open-source Python package mrmr (https://github.com/smazzanti/mrmr) which measures relevance via the F-value and measures redundancy via Pearson’s correlation. For all the baselines but TopDE and RankCorr, we take the top k scoring features, where k equals the size of the solution returned by G-PC.

Datasets and preprocessing

We use three public scRNA-seq datasets to validate our method (Table 1). For all three datasets we remove classes with less than 50 cells. This leads to 75 tissue/cell type pairs for HCA. We also filter for genes expressed in at least 10 cells, and for runtime efficiency purposes, we only consider highly variable genes for IPF and HCA for all methods. Also, scGeneFit was slow for MC, so we considered only highly variable genes for MC when running this method. Each dataset is normalized using Scanpy⁷⁴ so that the total counts for all cells are equal. The data is then log(x+1) transformed and each feature scaled to unit variance and zero mean. scGeneFit performed very poorly when the data was scaled, hence, for a fair comparison we skipped the scaling step when running scGeneFit. Log-transforming and scaling the data had a positive effect on the F1 score for all the other methods. We show these results for the MC dataset in Figure S4D. On the other hand, deconvolution via CIBERSORT works best if the data is in linear space as recommended by the authors, hence, we did not log the data during deconvolution. Feature selection, however, is applied on logged data.

We split all datasets into a train and test set of equal size in a stratified fashion. To obtain a signature matrix M for G-PC and CEM-PC, we average expression values for every phenotype. While it is true that this operation summarizes the data and leads to information loss, we note that our goal is not reconstruction or dimensionality reduction but rather marker selection. We argue that for such a task the individual cell-based expression is less important since we are looking for markers that are generally observed across most or all cells. Furthermore, commonly used DE tests such as t-test also rely on a small set of sufficient statistics.

Regarding the choice of K, in this paper we test the performance of our methods across multiple values of K. In practice, a single value for K could be obtained in a cross-validation fashion.

Quantification and statistical analysis

To compare the performance of Logistic Regression classifiers, we use the macro-average F1 score. This score equally weighs the F1 score of each class, which is desirable as we are interested in finding markers for all phenotypes, regardless of any class imbalance in the data. For a single class p, the F1 score is the harmonic mean between precision and recall

$$\text{F}{1}_p=\frac{2}{1/{\text{Precision}}_p+1/{\text{Recall}}_p}=2\frac{{\text{Precision}}_p\cdot {\text{Recall}}_p}{{\text{Precision}}_p+{\text{Recall}}_p}$$

The macro-average F1 score is simply the unweighted mean of per-class F1 scores

$$\text{F}{1}_{\text{macro}}=\frac{1}{P}\sum _{p=1}^P\text{F}{1}_p$$

To evaluate deconvolution performance, we use the Jensen-Shannon divergence⁴⁵ which is a symmetric measure between two probability distributions. Given two discrete probability distributions P and Q, the Kullback-Leibler divergence⁷⁵ is given by

$$\text{KL}\left(P\Vert Q\right)=\sum _{x\in \chi }P\left(x\right)\log \left(\frac{P\left(x\right)}{Q\left(x\right)}\right)$$

where $\chi$ is a probability space. Letting $M=\frac{1}{2}\left(P+Q\right)$, the Jensen-Shannon divergence is

$$\text{JS}\left(P\Vert Q\right)=\frac{1}{2}\text{KL}\left(P\Vert M\right)+\frac{1}{2}\text{KL}\left(Q\Vert M\right)$$

Feature stability computes the average size of the overlap divided by the size of the union for all pairs of feature sets. More precisely, given a collection of feature sets $\epsilon =\left\{S_1,\dots S_k\right\}$, stability is given by

$$s=\frac{2}{k\left(k-1\right)}\sum _{i=1}^k\sum _{j>i}^k\frac{\left|S_i\cap S_j\right|}{\left|S_i\cup S_j\right|}$$

Finally, we performed gene set enrichment analysis (GSEA) using the Python package GSEApy (https://gseapy.readthedocs.io/) and the Enrichr API.⁷⁶ We used the HuBMAP_ASCTplusB_augmented_2022 gene set.⁴⁸ All p values reported in this paper were corrected for multiple testing.

Published: November 11, 2022

Data and code availability

• •This paper analyzes existing, publicly available data. These accession numbers for the datasets are listed in the key resources table.
• •We implemented a general-purpose package for running the greedy multiset multicover algorithm in C++ and expose it to Python. The code has been deposited at https://github.com/euxhenh/multiset-multicover. The G-PC and CEM-PC algorithms for feature selection can be found at https://github.com/euxhenh/phenotype-cover. Installation instructions are available in each repository. The code for running experiments in this paper is available from https://github.com/euxhenh/phenotype-cover-experiments. DOIs are listed in the key resources table.
• •Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.