GEOMETRIC STRUCTURE GUIDED MODEL AND ALGORITHMS FOR COMPLETE DECONVOLUTION OF GENE EXPRESSION DATA

Abstract

Complete deconvolution analysis for bulk RNA-seq data is important and helpful to distinguish whether the differences of disease-associated GEPs (gene expression profiles) in tissues of patients and normal controls are due to changes in cellular composition of tissue samples, or due to GEPs changes in specific cells. One of the major techniques to perform complete deconvolution is nonnegative matrix factorization (NMF), which also has a wide-range of applications in the machine learning community. However, the NMF is a well-known strongly ill-posed problem, so a direct application of NMF to RNA-seq data will suffer severe difficulties in the interpretability of solutions. In this paper, we develop an NMF-based mathematical model and corresponding computational algorithms to improve the solution identifiability of deconvoluting bulk RNA-seq data. In our approach, we combine the biological concept of marker genes with the solvability conditions of the NMF theories, and develop a geometric structures guided optimization model. In this strategy, the geometric structure of bulk tissue data is first explored by the spectral clustering technique. Then, the identified information of marker genes is integrated as solvability constraints, while the overall correlation graph is used as manifold regularization. Both synthetic and biological data are used to validate the proposed model and algorithms, from which solution interpretability and accuracy are significantly improved.

1. Introduction

Over past decades, analysis of transcriptome or gene expression data has been an essential component to understand the biomolecular processes involved in human development and diseases [9, 30, 58, 27]. The complex nature of bulk tissue samples under investigation remains as a major obstacle[16, 56, 17]. A bulk tissue sample could include many cell types, and its heterogeneous characteristics make the interpretation of gene expression (such as RNA-seq) complicated [22, 49, 4]: for every gene, its measured gene expression profiles (GEPs) in a compound sample are actually tissue-averaged, i.e., the sum of expression of all cells in the sample. On the other side, cellular composition of bulk samples varies, and samples may have high variances between one and another in relative cell subset proportions. So the GEP of low abundant cell types could be masked by that of ones with higher proportions. Consequently, it is challenging to determine whether an experimental or clinic treatment should target one particular gene or focus on investigating possible sources of varying cell types among samples. For example, Alzheimer’s disease is marked by amyloid-beta plaques and neurofibrillary tangles, along with neuronal loss and gliosis in the affected brain regions. Transcriptome-wide GEP from brain tissue of patients and neuropathologically normal controls are different. Such differences are critical for discovering genes and biological pathways that are perturbed in and/or lead to Alzheimer’s disease [3, 40, 42, 18]. Differential expression (DE) analysis is one of the important tools to unveil these differences. It will reveal novel insights into the genes and pathways, and is potentially helpful for drug targets therapeutics. However, a fundamental knowledge gap still remains for DE, concerning whether disease-associated GEP changes in brain tissues are due to changes in cellular composition of tissue samples, or due to GEP changes in specific cells, e.g., central nervous system cells. It could be much more informative to study gene expression on specific cells, or identify cell-intrinsic differentially expressed genes. But for many complex biological mixtures, exhaustive knowledge of individual cell types in brain tissues and their specific markers is lacking. Although single-cell RNA sequencing data can be used or serve as a reference, such approaches remain costly, cumbersome and limited in sample sizes[60, 15, 35].

In contrast, computational tools can be used to leverage widely available large-scale bulk tissue RNA-seq data sets [40, 18, 2, 34, 11]. This problem, as illustrated in Figure 1, is called complete deconvolution. In this approach, expression of a gene in a sample tissue is assumed to be the linear combination of its expressions in the constituting cell types, with respect to the cell proportion, i.e.

$$g_{ij}=\sum _{l=1}^k{p}_{lj}{c}_{il},1\le i\le N,1\le j\le n,$$ (1)

where $g_{ij}$ and $c_{i,l}$ are the GEPs of gene $i$ in the $j$-th sample and $l$-th cell type, respectively, while $p_{lj}$ is the proportion of the $l$-th cell type in the $j$-th sample. For the total number of genes $N$, total number of samples $n$, and the number of cell types $k$, we usually have $N\gg n>k$. In matrix form, Eq. (1) is represented as $\mathbf{G}=\mathbf{CP}$ with all matrix entries being non-negative. Given data $\mathbf{G}$, both variables $\mathbf{C}$ and $\mathbf{P}$ are to be solved. Note that many deconvolution algorithms have been developed [51, 4, 41, 43, 46, 61, 26, 14, 1, 10, 49] for $\text{GEP}$ in bulk tissues, but their primary focuses have been only on estimating the cellular composition with prior knowledge of cell-specific markers. This type of problem to solve for $\mathbf{P}$, with known $\mathbf{C}$, is called partial convolution, and can be performed with remarkable robustness and accuracy. However, in more realistic circumstances when little or no information about the underlying cell type is available, developing reliable complete deconvolution methods is still an open problem and only a handful models have been established [31, 48, 57].

Figure 1.

Diagram of complete deconvolution of bulk tissue data

Mathematically, complete deconvolution can be solved as a nonnegative matrix factorization (NMF) problem [13, 45, 37]. Many studies have been established for various types of data in other fields, such as spectral unmixing in analytical chemistry [45], remote sensing [39], image processing [24], or topic mining in machine learning [59], etc. There is no obstacle at all if one is simply looking for a couple of solutions $\mathbf{C}$ and $\mathbf{P}$. However, the NMF is strongly ill-posed and solutions are generally not unique. Such non-uniqueness poses great challenges on solution interpretability: For RNA-seq data, different solutions represent various combinations of GEPs in each cell type and cell proportions in tissues. Meaningful explanation of these biological quantities is critical to next step DE analysis. There are a few guidelines to reduce such ill-posedness. As stated in [25], if the matrices $\mathbf{C}$ and $\mathbf{P}$ satisfy certain identifiability conditions (see Section 2 for details), it is possible for the NMF problem to have unique solution, subjective to row/column scaling and permutation ambiguities of the solution. Applying these sufficient conditions depends on the specific properties of available data in the corresponding research field. Successful methods in one field cannot be directly implanted to another because of different data characteristics. Modeling the right NMF tool for the application at hand is essential [23]. On the biology side, the GEP data $\mathbf{G}$ in bulk tissues includes expression of marker genes, or cell-type-specific genes, which are defined by their exclusive expression in only one component (cell type) in cell mixtures. These biological characteristics establish a connection to the mathematical theories of NMF. So it is possible to develop robust and accurate complete deconvolution algorithms for bulk tissue GEP data, without prior information of GEP in single cells.

The objective of the current work is to develop mathematical model and computational algorithms to perform complete deconvolution of bulk RNA-seq data with reduced solution ambiguity and high interpretability. Our approaches are based on the abovementioned inherent characteristics of bulk tissue data and the theoretical foundation of NMF problem. This goal is achieved by a structure-exploring and inheriting strategy. To explore the structure, we first define correlation distance among rows of data $\mathbf{G}$ (considered as data points in ${\text{ℝ}}^n$) and generate the corresponding graph, then spectral clustering technique is used to classify all points in $k$ (assumed number of cell types) groups. Finally, marker genes of each cell type are identified by picking the most correlated points in each cluster. In the noiseless case, rows of $\mathbf{C}$ can be understood as coefficients of data points with rows of $\mathbf{P}$ as coordinates. Thus, we expect the row space of $\mathbf{C}$ inherits the geometric structure of $\mathbf{G}$ and impose the weak identifiability (to accommodate noises in real data) condition on $\mathbf{C}$. Additionally, the manifold regularization is applied by the local invariance assumption [5, 8, 28]. Combining these approaches, we establish a structure guided non-convex optimization model, to deconvolute bulk tissue RNA-seq data without prior information about marker genes. The proposed model is numerically solved under the frame work of alternating direction method of multipliers (ADMM)[21, 6], in which each variable can be solved one at a time in a two-fold iteration. Effectiveness and accuracy of the model and algorithms are tested by both synthetic and biological data. This work is motivated by various manifold regularization NMF models, such as[7, 47], but it has the following novel features: (1) Traditionally, Euclidean distance is used in the manifold assumption of data space and it results a linear graph regularization term. But the application on biological data requires the correlation distance, from which a nonlinear graph regularizor is derived and it poses great challenges in computation; (2) More importantly, this new model is equipped with a solvability constraint, and this regularizor significantly improves solution identifiability from realistic noisy data.

The paper is organized as the follows: Section 2 briefly reviews the NMF and its separability conditions, and why this condition is related to the biological problem. Section 3 presents the geometric structure guided complete deconvolution model, including using spectral clustering analysis to identify marker genes (finding structures) and the quantitative constraints in the optimization problem (preserving the structure). For the resulting non-convex learning model, an ADMM based algorithm is introduced in Section 4. As validations, in Section 5 there display numerical results of the proposed model and algorithms for both synthetic and biological data. The paper ends with a conclusion in Section 6, where potential challenges of the work and possible future research directions are discussed. For convenience, some frequently used acronyms throughout the paper are listed in Table 1. Necessary definitions of NMF are summarized in the Appendix.

Table 1.

Some frequently used acronyms

GEP	Gene expression profile
DE	Differential expression
NMF	Nonnegative Matrix Factorization
GS-NMF	Geometric structure constrained NMF
ADMM	Alternating direction method of multipliers
NDR	Noise to data ratio

2. NMF and its identifiability conditions

In this section, we briefly review notations and some theoretical foundations in the NMF. Additional details and other necessary definition of the NMF problem are included in the Appendix.

2.1. Notations

Throughout the paper, a bold lower-case letter, such as $\mathit{x}$, represents a column vector with the appropriate dimension, and $|\mathit{x}|$ represents its $l_2$ norm. Vector $1$ is a column vector of some dimension with all entries being one. For a matrix $\mathbf{A}$, $\parallel \mathbf{A}{\parallel }_F$ represents its Frobenius norm, ${\mathbf{A}}_{(i)}$ and ${\mathbf{A}}^{(j)}$ mean its $i$-th row and $j$-th column, respectively.

Let $\mathbf{G}\in {\text{ℝ}}^{N\times n}$ with entry $g_{ij}$ being the expression of the $i$-th gene in the $j$-th sample; $\mathbf{C}\in {\text{ℝ}}^{N\times k}$ with entry $c_{ij}$ being the reference expression of the $i$-th gene in the $j$-th cell type; and $\mathbf{P}\in {\text{ℝ}}^{k\times n}$ with entry $p_{ij}$ being the proportion of the $i$-th cell type in the $j$-th sample. Dimensions $N\gg \max (n,k)$. The following linear relation is assumed:

$$\mathbf{G}=\mathbf{CP}+ϵ,$$ (2)

where $ϵ$ is noise. The problem of complete deconvolution can be summarized as: given data $\mathbf{G}\in {\text{ℝ}}^{N\times n}$, solve

$$\left({\mathbf{C}}^{*},{\mathbf{P}}^{*}\right)=\underset{\mathbf{C}\in {\text{ℝ}}_{+}^{N\times k},\mathbf{P}\in {\text{ℝ}}_{+}^{k\times n}}{\arg \min }\delta (\mathbf{CP},\mathbf{G})$$ (3)

where ${\text{ℝ}}_{+}^{N\times k}$ or ${\text{ℝ}}_{+}^{k\times n}$ represent matrices with nonnegative entries and $\delta (\cdot ,\cdot )$ is a cost function. There are many choices for the cost function, depending on prior knowledge about the probability distribution of the data noise and susceptibility to outliers. For example, when Frobenius norm is used, the minimization process is considered as a maximum likelihood estimator for additive Gaussian noise. Meanwhile, I-divergence [38] cost function is equivalent to the Expectation Maximization (EM) algorithm and maximum likelihood when noises are Poisson processes [12]. If noises follow Laplace distribution, the cost function can be taken as row-wise or column-wise $l_1$ norms of the difference matrix [41]. As summarized in [54], other choices include Earth Mover’s distance metric, $\alpha$-divergence, $\beta$-divergence, $\gamma$-divergence, $\phi$-divergence, Bregman divergence, and $\alpha$-$\beta$-divergence. The choice of cost function does not affect exploration of the geometric structures of data, so for simplicity, we consider $\delta (\mathbf{CP},\mathbf{G})=\frac{1}{2}\parallel \mathbf{G}-\mathbf{CP}{\parallel }_F^2$ and Gaussian noise in the current work.

2.2. Ill-posedness of NMF:

Solving Eq. (3) for only $\mathbf{C}$ (or $\mathbf{P}$) with the other variable known (partial deconvolution) is simply a convex regression problem. But it is well-known that solving both variables simultaneously is non-convex, NP-hard in general, and computational algorithms only converge to local minima or just stationary points [54]. Further, the NMF is ill-posed and the solution is not unique, or not identifiable: if (${\mathbf{C}}^{*},{\mathbf{P}}^{*}$) is a local minimum to (3), then for any $\mathbf{\Omega }\in {\text{ℝ}}^{k\times k}$, $\widehat{\mathbf{C}}={\mathbf{C}}^{*}\mathbf{\text{Ω}}$ and $\widehat{\mathbf{P}}={\mathbf{\text{Ω}}}^{-1}{\mathbf{P}}^{*}$ are also solutions, as long as their non-negativity is satisfied. Non-uniqueness of solution will significantly impact statistical analysis for decisions in biological implementation. Therefore, it is important to restrict searching space of variables to increase the identifiability of solutions, in order for better interpretability. The uniqueness of NMF solution is defined in the following sense [23]:

Definition 2.1 (Uniqueness of NMF solution).

The solution $\left({\mathbf{C}}^{*},{\mathbf{P}}^{*}\right)$ of NMF (3) is unique, or identifiable, if and only if for any other solution $(\overline{\mathbf{C}},\overline{\mathbf{P}})$, there exists a permutation matrix $\text{Π}\in {\{0,1\}}^{k\times k}$ and a diagonal scaling matrix $\mathbf{S}$ with positive diagonal matrix such that

$$\overline{\mathbf{C}}={\mathbf{C}}^{*}\mathbf{\text{Π}}\mathbf{S}\text{and}\overline{\mathbf{P}}={\mathbf{S}}^{-1}{\mathbf{\text{Π}}}^{\top }{\mathbf{P}}^{*}.$$ (4)

Further, it is summarized in [54, 23] that this uniqueness can be achieved under certain circumstances [29, 19, 36, 25]:

Theorem 2.2 (Strong identifiability condition).

Assuming $k=rank(\mathbf{G})$, $ϵ=0$, if problem (3) admits a solution, for which both ${\mathbf{C}}^{\top }$ and $\mathbf{P}$ are separable matrices, then the solution is unique.

Theorem 2.3 (Weak identifiability condition).

Assuming $k=rank(\mathbf{G})$, $ϵ=0$, if both ${\mathbf{C}}^{\top }$ and $\mathbf{P}$ are sufficiently scattered, then problem (3) admits a unique solution.

More details of related definitions, such as separability and sufficiently scattered properties of matrices are included in the Appendix. Specifically, Figure 14 illustrated the geometric interpretation of the NMF problem and Figure 15 intuitively explains the strong and weak identifiability conditions of NMF in order to achieve interpretable solutions. As explained, the weak identifiability condition is much relaxed than the strong one, so it is more suitable for realistic data with noises.

2.3. Relation to the gene expression data:

There are several issues when applying the above NMF theories to RNA-seq data: (1) First of all and most importantly, how do these general identifiability conditions relate to the specific biological problem, i.e. bulk tissue RNA-seq data? (2) What measure should be used to define the “sufficient scattering” of matrix columns? Euclidean distance is used to illustrate the idea in Figure 15, but it is not practical for high dimension data because column (row) normalization is needed for convex hull description. (3) Theorem 2.2 and 2.3 can be used as variable constraints in the optimization problem, but the actual questions are how many, and which columns or (rows) the constraints should be enforced? How can we obtain this information from the only available data $\mathbf{G}$?

The concept of marker genes will help to address these issues. By its name, marker genes of a certain type of cell dominantly express in that cell type while rarely express in others. Each cell type may have multiple marker genes but one marker gene is only for one cell type. Mathematically, for each cell type $r=1,2,3,\dots k$, there exists an index set ${𝓢}_r$, such that for any $i\in {𝓢}_r$: expression level $c_{ir}$ is the dominant entry (the only nonzero entry in ideal noiseless case) in the $i$-th row of matrix $\mathbf{C}$. This characteristics implies that ${\mathbf{C}}^{\top }$ is separable in ideal case and its columns are sufficiently scattered if noises present. On the other hand, no structures can be assumed for matrix $\mathbf{P}$. Although all conditions in Theorem 2.2 or 2.3 are not fully satisfied, reasonable solutions can be expected with constraints on variable $\mathbf{C}$.

Then the question is how to identify marker genes (their index ${𝓢}_r$) from the data $\mathbf{G}$. In the noiseless case, for a given $r=1,2,\dots k$ and any $i\in {𝓢}_r$, one has ${\mathbf{C}}_{(i)}={\alpha }_i{\mathbf{e}}_r^{\top }$ by the definition of marker genes, where ${\mathbf{e}}_r$ is the unit basis vector in ${\text{ℝ}}^r$. As consequences, the $i$-th row ${\mathbf{G}}_{(i)}={\alpha }_i{\mathbf{P}}_{(r)}$ and hence all rows of $\mathbf{G}$ are linearly dependent if their indices are from the same set ${𝓢}_r$. In the practical scenario where noise present, this linear dependence among vectors will become strong correlations. Many literature [33, 41, 57, 4, 50] has confirmed this phenomena that gene expressions across samples, or rows of $\mathbf{G}$, will display strong correlation, if they are from marker genes of the same cell type.

3. Mathematical model:

Bulk tissue RNA-seq data $\mathbf{G}$ has richer structures than just ${\mathbf{G}}_{(i)}\in cone\left({\mathbf{P}}^{\top }\right)\subseteq {\text{ℝ}}_{+}^n$: for marker genes of the same cell type, e.g., the $r$-th cell type, their expressions across samples are highly correlated and correlated to ${\mathbf{P}}_{(r)}$. A cone view of this property is displayed in the left-top panel of Figure 2 for $k=3$. It is easier to investigate this feature further from the convex hull view, in which each row of $\mathbf{G}$ can be represented by a dot: As shown by the left-bottom panel of Figure 2, rows of $\mathbf{G}$ for the marker genes of the $r$-th cell type tend to form a cluster around ${\mathbf{P}}_{(r)}$ due to the strong correlation. This property motivates us that it is possible to first identify marker genes from data $\mathbf{G}$ by clustering its rows and to quantitatively explore the geometric structures of its row space. Further, note that ${\mathbf{C}}_{(i)}$ is actually the coefficient vector of ${\mathbf{G}}_{(i)}$ under the basis vectors of rows of $\mathbf{P}$. Then the second step is to transfer such geometric structure of $\mathbf{G}$ to $\mathbf{C}$, hence to enforce the weak identifiability condition on variable $\mathbf{C}$. These two steps are termed as finding and preserving the structure, respectively, which will be detailed as the following:

Figure 2.

RNA-seq data structure (left) and geometric constraints (right)

3.1. Finding geometric structures by spectral clustering analysis

In this step we will classify ${\mathbf{G}}_{(i)}$, $1\le i\le N$ into $k$ groups to identify possible marker genes for the $k$ types of cells. Among many existing clustering techniques, we propose to use spectral clustering [52], which is one type of manifold learning algorithms that can explore intrinsic geometric/topological structure of high dimensional data. Thus, it has many fundamental advantages and very often outperforms traditional clustering algorithms such as $k$-means or single linkage. To perform spectral clustering, we need the similarity graph $G=(V,E)$, with vertex set $V={\left\{{\mathbf{G}}_{(i)}\right\}}_{i=1}^N\subset {\text{ℝ}}^n$. The non-negative weights ${\omega }_{ij}$ of edges $E=\left\{e_{ij}\right\}$ are calculated by a function ${\text{ℝ}}^n\times {\text{ℝ}}^n\to {\text{ℝ}}_{+}$, quantifying the correlation between two vertices. We propose to evaluate ${\omega }_{ij}$ as

$${\omega }_{ij}=\exp \left\{-\frac{d_{eisen}{\left({\mathbf{G}}_{(i)},{\mathbf{G}}_{(j)}\right)}^2}{\sigma }\right\},1\le i\le N,1\le j\le N\text{,}$$ (5)

where

$$d_{eisen}\left({\mathbf{G}}_{(i)},{\mathbf{G}}_{(j)}\right)=1-\frac{\langle {\mathbf{G}}_{(i)},{\mathbf{G}}_{(j)}\rangle }{\left|{\mathbf{G}}_{(i)}\right|\left|{\mathbf{G}}_{(j)}\right|}$$ (6)

is the Eisen cosine correlation distance and $\sigma >0$ is a parameter. The matrix $\mathbf{W}=\left({\omega }_{ij}\right)\in {\text{ℝ}}^{N\times N}$ is called the adjacency matrix. Meanwhile, define its degree matrix $\mathbf{D}=\text{diag}\left(d_1,d_2,\dots d_N\right)$, where $d_i=\sum _{j=1}^N{\omega }_{ij}$ is the degree of the vertex ${\mathbf{G}}_{(i)}$. With these matrices, different types of graph Laplacians (gL) of $G$ can be defined, such as the unnormalized gL $L=\mathbf{D}-\mathbf{W}$, symmetric normalized gL $L_{\text{sym}}=\mathbf{I}-{\mathbf{D}}^{-\frac{1}{2}}\mathbf{W}{\mathbf{D}}^{-\frac{1}{2}}$, or random walk gL $L_{\text{rw}}=\mathbf{I}-{\mathbf{D}}^{-1}\mathbf{W}$. By examining the first a few eigenvectors of gL, rows of data $\mathbf{G}$ will be clustered into $k$ groups and the set ${\left\{{𝓖}_r\right\}}_{r=1}^k$ records row indices of $\mathbf{G}$ in the corresponding clusters. Choice of different gLs depends on specific data applications [47]. For our problem, we use the normalized gL $L_{\text{sym}}$ and perform the spectral clustering package in Matlab.

3.2. Geometric structure guided model

With the clustering information, we are able to establish geometric structure guided model by applying two constraints on the row space of variable $\mathbf{C}$: the solvability constraint and manifold regularization. This work is motivated by manifold regularization which uses graph Laplacian as regularizer, but it provides new characteristics. The major novelty is to incorporate the identifiability conditions in the regularization of the NMF (primary constraint). Another new feature is to encode the geometric information of the data space (secondary constraint) based on Eisen cosine correlation distance, instead of Euclidean distance in traditional graph regularized NMF.

3.2.1. Solvability constraint:

According to Theorem 2.3, rows of $\mathbf{C}$ need to be scattered sufficiently, such that the second-order cone in ${\text{ℝ}}_{+}^r$ contained in $cone\left({\mathbf{C}}^{\top }\right)$. Further, Definition A.2 implies that this requirement is only needed for some of rows in $\mathbf{C}$, i.e., those rows corresponding to marker genes. In the previous step, all rows of $\mathbf{G}$ are clustered into $k$ groups and their row numbers are recorded in the set ${\left\{{𝓖}_r\right\}}_{r=1}^k$. In the current step, correlations are ranked within each group, and a subset, i.e., ${𝓢}_r\subset {𝓖}_r$ is determined accordingly to represent the indices of marker genes for each cell type. Note that row indices of both $\mathbf{G}$ and $\mathbf{C}$ represent gene IDs, we need rows of $\mathbf{C}$ with index ${𝓢}_r$ to scatter enough to accommodate the second-order cone. This can be done by requiring them to have strong correlations with $\left\{{\mathbf{e}}_r^{\top }\right\}$, i.e., defining the penalty function:

$${𝓕}_1(\mathbf{C})=\frac{{\lambda }_1}{2}\sum _{r=1}^k\sum _{i\in {𝓢}_r}{d}_{eisen}{\left({\mathbf{C}}_{(i)},{\mathbf{e}}_r^{\top }\right)}^2,$$ (7)

where ${\lambda }_1$ is a parameter. This idea is illustrated from the convex hull view in the right panel of Figure 2 as $k=3$. Red, blue, green dots represent rows of $\mathbf{C}$ that indexed by ${𝓖}_r$. The darker colored dots, representing selected marker genes in ${𝓢}_r$, are “required” to stay in the circular sectors (orange), such that the $conv\left\{{\mathbf{C}}^{\top }\right\}$ (dashed purple) is large enough to contain the second-order cone (dashed circle). With the Eisen cosine correlation distance, we do not need to work in $conv\left\{{\mathbf{C}}^{\top }\right\}$ (which requires normalizing coefficients), but directly in $conv\left\{{\mathbf{C}}^{\top }\right\}$.

3.2.2. Manifold constraint:

According to the local invariance assumption in manifold regularization [5, 8, 28], if two data points are close in the intrinsic geometry of the data distribution, then the representations of these two points in a new basis should also be close to each other under the same metric. Note that ${\mathbf{C}}_{(i)}$ is the representation of the data point ${\mathbf{G}}_{(i)}$ under the basis ${\mathbf{P}}^{\top }$, then by such manifold assumption, we require matrix $\mathbf{C}$ to inherit the similar geometric structure of matrix $\mathbf{G}$, i.e., rows of $\mathbf{C}$ belong to the same cluster have strong mutual correlations. To achieve this goal, we define another penalty function:

$${𝓕}_2(\mathbf{C})=\frac{{\lambda }_2}{2}\sum _{j=1}^N\sum _{i=1}^N{\omega }_{ij}{d}_{eisen}{\left({\mathbf{C}}_{(i)},{\mathbf{C}}_{(j)}\right)}^2.$$ (8)

Recall that entry ${\omega }_{ij}>0$ in the adjacency matrix $\mathbf{W}$ in (5) measures the correlations (larger value represents stronger correlation) between genes $i$ and $j$ in data $\mathbf{G}$.

Remark 1.

Equation (8) is a generalization of the traditional graph Laplacian regularization [7, 47]. Actually, if the Eisen cosine correlation distance in (8) is replaced by the Euclidean distance ($l_2$ norm), then

$$\begin{gathered} {𝓕}_2(\mathbf{C})=\frac{{\lambda }_2}{2}\sum _{j=1}^N\sum _{i=1}^N{\omega }_{ij}\parallel {\mathbf{C}}_{(i)}-{\mathbf{C}}_{(j)}{\parallel }^2 \\ ={\lambda }_2\left[\sum _{i=1}^N{d}_{ii}{\mathbf{C}}_{(i)}^{\top }{\mathbf{C}}_{(i)}-\sum _{i=1}^N{\omega }_{ij}\sum _{j=1}^N{\mathbf{C}}_{(i)}^{\top }{\mathbf{C}}_{(j)}\right] \\ ={\lambda }_2\text{Tr}\left({\mathbf{C}}^{\top }L\mathbf{C}\right), \end{gathered}$$

with $L$ being the graph Laplacian operator defined earlier.

Remark 2.

The current work is more than a generalization of traditional graph Laplacian regularized NMF by using correlation distance metric. Indeed, the manifold assumption only requires ${\mathbf{C}}_{(i)}$ and ${\mathbf{C}}_{(j)}$ to be close, if ${\mathbf{G}}_{(i)}$ and ${\mathbf{G}}_{(j)}$ are close in the distance metric, i.e., genes $i$ and $j$ are classified to belong the same cell type. However, it does not require rows of $\mathbf{C}$ to be far from each other if they are for different types of cells. This issue is addressed by Eq. (7) and this is the major novelty of the current work.

Remark 3.

In the extreme case ${\lambda }_1\to \infty$, we have ${\mathbf{C}}_{(i)}={\alpha }_i{\mathbf{e}}_r^{\top }$ if $i\in {𝓢}_r$. This case corresponds to the strong identifiability condition. However, in the realistic circumstances where noises present, this extreme requirement does not provide the optimal results, as shown in numerical simulations.

Remark 4.

Relations between constraints (7) and (8) can be explained by the right panel of Figure 2 as $k=3$. All genes are classified into $k$ groups (red, blue, and green), indexed by ${𝓖}_r$, assuming there are $k$ types of cells in those tissue samples. In addition to the constraints that darker colored dots (indexed by ${𝓢}_r$) close to ${\mathbf{e}}_r^{\top }$, the dots in the same color need to stay close, corresponding to the geometric structure of data $\mathbf{G}$ shown in the left panel.

3.3. Full model:

Combining the solvability condition (7) and manifold constraint (8), we propose the following geometric structure guided nonnegative matrix factorization (GS-NMF) model. For illustration convenience, we define the set $T:=\left\{Z\in {\text{ℝ}}_{+}^{k\times n},{\mathbf{1}}^{\top }Z={\mathbf{1}}^{\top }\right\}$, and the indicator function ${\mathrm{𝟙}}_T$ as ${\mathrm{𝟙}}_T(Z)=0$ if $Z\in T$ while ${\mathrm{𝟙}}_T(Z)=\infty$ otherwise. With these notations, solving for $\mathbf{C}$ and $\mathbf{P}$ becomes the optimization problem:

$$\underset{\mathbf{C}\ge 0,\mathbf{P}\ge 0}{\min }\frac{1}{2}\parallel \mathbf{G}-\mathbf{CP}{\parallel }_F^2+𝓕(\mathbf{C})+{\mathrm{𝟙}}_T(\mathbf{P}).$$ (9)

In the first term, the Frobenius norm is used to measure the error between deconvoluted solution and the given data. The total regularization function $𝓕(\mathbf{C})={𝓕}_1(\mathbf{C})+{𝓕}_2(\mathbf{C})$ as each component is defined in (7)-(8). The third term ${\mathrm{𝟙}}_T(\mathbf{P})$ simply means sum-to-one conditions on columns of $\mathbf{P}$, or column stochasticity, since the sum of cellular proportions in each tissue sample is supposed to be one.

4. Computational algorithms

It is well-known that the objective function in (9) is non-convex in both variables together. There are several types of numerical methods to obtain a local minimum, including Multiplicative Update Algorithm (MUA) [37], Alternating nonnegativity constrained least squares (ANLS)[32], and the alternating direction method of multipliers (ADMM), etc. The first two types of methods are quite straightforward for the basic NMF problems, or with linear constraints. While for the nonlinear constraints (7)-(8), it is convenient to adopt ADMM framework to develop numerical schemes.

To do so, we first rewrite model (9) as

$$\underset{\mathbf{C},\mathbf{P}}{\min }\frac{1}{2}\parallel \mathbf{G}-\mathbf{CP}{\parallel }_F^2+𝓕(\mathbf{C})+{\mathrm{𝟙}}_S(\mathbf{C})+{\mathrm{𝟙}}_T(\mathbf{P}),$$ (10)

where $S:={\text{ℝ}}_{+}^{N\times k}$ is the set of all nonnegative matrices of the size $N\times k$. Then we introduce two auxiliary variables $\mathbf{A}$ and $\mathbf{Q}$, and rewrite (10) into an equivalent form

$$\begin{gathered} \underset{\mathbf{C},\mathbf{P},\mathbf{A},\mathbf{Q}}{\min }\frac{1}{2}\parallel \mathbf{G}-\mathbf{CP}{\parallel }_F^2+𝓕(\mathbf{A})+{\mathrm{𝟙}}_S(\mathbf{C})+{\mathrm{𝟙}}_T(\mathbf{Q}) \\ \text{s.t.:}\mathbf{C}-\mathbf{A}=\mathbf{0},\mathbf{P}-\mathbf{Q}=\mathbf{0}, \end{gathered}$$ (11)

and the corresponding augmented Lagrange function is [55]

$$𝓛=\frac{1}{2}\parallel \mathbf{G}-\mathbf{CP}{\parallel }_F^2+𝓕(\mathbf{A})+{\mathrm{𝟙}}_S(\mathbf{C})+{\mathrm{𝟙}}_T(\mathbf{Q})+\frac{\rho }{2}\parallel \mathbf{C}-\mathbf{A}+\tilde{\mathbf{A}}{\parallel }_F^2+\frac{\gamma }{2}\parallel \mathbf{P}-\mathbf{Q}+\tilde{\mathbf{Q}}{\parallel }_F^2$$ (12)

where $\tilde{A}$ and $\tilde{Q}$ are dual variables of $\mathbf{A}$ and $\mathbf{Q}$, respectively, while $\rho >0$ and $\gamma >0$ are penalty parameters. As results, the ADMM (12) can be written as an iteration (from $i$- to $i+1$-th step) in its scaled form [6]:

$$\begin{gathered} {\mathbf{C}}^{i+1}:=\underset{\mathbf{C}}{\arg \min }\frac{1}{2}\parallel \mathbf{G}-\mathbf{C}{\mathbf{P}}^i{\parallel }_F^2+\frac{\rho }{2}\parallel \mathbf{C}-{\mathbf{A}}^i+{\tilde{\mathbf{A}}}^i{\parallel }_F^2+{\mathrm{𝟙}}_S(\mathbf{C}) \\ {\mathbf{P}}^{i+1}:=\underset{\mathbf{P}}{\arg \min }\frac{1}{2}\parallel \mathbf{G}-{\mathbf{C}}^i\mathbf{P}{\parallel }_F^2+\frac{\gamma }{2}\parallel \mathbf{P}-{\mathbf{Q}}^i+{\tilde{\mathbf{Q}}}^i{\parallel }_F^2+{\mathrm{𝟙}}_T(\mathbf{P}) \\ {\mathbf{A}}^{i+1}:=\underset{\mathbf{A}}{\arg \min }𝓕(\mathbf{A})+\frac{\rho }{2}\parallel {\mathbf{C}}^i-\mathbf{A}+{\tilde{\mathbf{A}}}^i{\parallel }_F^2 \\ {\mathbf{Q}}^{i+1}:=\underset{\mathbf{Q}}{\arg \min }\frac{\gamma }{2}\parallel {\mathbf{P}}^i-\mathbf{Q}+{\tilde{\mathbf{Q}}}^i{\parallel }_F^2 \\ {\tilde{\mathbf{A}}}^{i+1}:={\tilde{\mathbf{A}}}^i+{\mathbf{C}}^i-{\mathbf{A}}^i \\ {\tilde{\mathbf{Q}}}^{i+1}:={\tilde{\mathbf{Q}}}^i+{\mathbf{P}}^i-{\mathbf{Q}}^i \end{gathered}$$ (13)

Each variable in system (13) can be solved individually. Specifically, for the $\mathbf{C}$-subproblem, the Karush-Kuhn-Tucker (KKT) condition [44] yields a closed form for $\mathbf{C}$, i.e.,

$${\mathbf{C}}^{i+1}=\left[\mathbf{G}{\mathbf{P}}^{i^{\top }}+\rho \left({\mathbf{A}}^i-{\tilde{\mathbf{A}}}^i\right)\right]{\left({\mathbf{P}}^i{\mathbf{P}}^{i^{\top }}+\rho \mathbf{I}\right)}^{-1},$$ (14)

where ${\mathbf{P}}^i{\mathbf{P}}^{i^{\top }}+\rho \mathbf{I}$ is a small $k\times k$ matrix that can be inverted easily. The non-negativity of $\mathbf{C}$ is obtained by row-wise active set method. For the $\mathbf{P}$-subproblem, KKT condition gives

$${\mathbf{P}}^{i+1}=\text{Π}\left\{{\left({{\mathbf{C}}^i}^T{\mathbf{C}}^i+\gamma \mathbf{I}\right)}^{-1}\left[{\mathbf{C}}^{i\top }\mathbf{G}+\gamma \left({\mathbf{Q}}^i-{\tilde{\mathbf{Q}}}^i\right)\right]\right\},$$ (15)

and this is a small-scale problem, in which a $k\times k$ matrix is to be inverted with column-wise probability simplex projection $\text{Π}$ [53]. Solution of the $\mathbf{Q}$-subproblem is simply

$${\mathbf{Q}}^{i+1}=\max \left\{{\mathbf{P}}^i+{\tilde{\mathbf{Q}}}^i,\mathbf{0}\right\}.$$ (16)

The $\mathbf{A}$-subproblem involves the solvability condition (7) and manifold constraints (8), both of which are non-linear problems, so there is no closed form. To solve this subproblem we have to use the gradient descent method and make this step an inner iteration. Denote the total objective function of the $\mathbf{A}$-subproblem as

$$f(\mathbf{A})=𝓕(\mathbf{A})+\frac{\rho }{2}\parallel \mathbf{C}-\mathbf{A}+\tilde{\mathbf{A}}{\parallel }_F^2,$$ (17)

then its gradient $\nabla f(\mathbf{A})$ can be computed from (7) and (8) accordingly. Note that $\nabla f(\mathbf{A})$ is nonlinear in terms of $\mathbf{A}$. For computational efficiency, we will use the result of $\mathbf{C}$ in the current outer iteration step, so values of $\nabla f(\mathbf{A})\approx \nabla f(\mathbf{C})$ will not update in the inner loop. Algorithm 1 summarizes the entire processes of the GS-NMF. Necessary raw data processing and spectral clustering steps are not included. The stoping criteria are to set $\parallel {\mathbf{C}}^{i+1}-{\mathbf{C}}^i\parallel /\parallel {\mathbf{C}}^i{\parallel }_F$ and $\parallel {\mathbf{P}}^{i+1}-{\mathbf{P}}^i\parallel /\parallel {\mathbf{P}}^i{\parallel }_F$ smaller than some tolerance.

Algorithm 1.

Geometric structure constrained NMF (GS-NMF)

Require: Data $\mathbf{G}$, initial guesses ${\mathbf{C}}_0$, ${\mathbf{P}}_0$, structure identifier ${\mathbf{C}}_g$, graph adjacency matrix $\mathbf{W}$, tolerance $ϵ$, parameters ${\lambda }_1$, ${\lambda }_2$, $\rho$ and $\gamma$.
Ensure: Matrices $\mathbf{C}$ and $\mathbf{P}$.
1:	for $i=0,1,\dots$ until criteria is satisfied do	% outer iteration
2:	Solve $\mathbf{C}$-subproblem in Eq. (13) by (14);
3:	Solve $\mathbf{P}$-subproblem in Eq. (13) by (15);
4:	for $m=0,1,\dots$ until criteria is satisfied do	% inner iteration
5:	Solve $\mathbf{A}$-subproblem in Eq. (13) through gradient descent method;
6:	end for
7:	Solve $\mathbf{Q}$-subproblem in Eq. (13) by (16);
8:	Set ${\tilde{\mathbf{A}}}^{i+1}:={\tilde{\mathbf{A}}}^i+{\mathbf{C}}^i-{\mathbf{A}}^i$;
9:	Set ${\tilde{\mathbf{Q}}}^{i+1}:={\tilde{\mathbf{Q}}}^i+{\mathbf{P}}^i-{\mathbf{Q}}^i$.
10:	end for

5. Numerical results

In this section, we test the proposed GS-NMF algorithms on two types of data.

5.1. Simulations on synthetic data

In order to have flexibility of matrix dimensions, noise levels, and known ground truth, we first test the algorithms on synthetic data, which are generated as the following strategies: matrix $\mathbf{C}\in {\text{ℝ}}_{+}^{N\times k}$ has a structure, so first we split $N=N_1+N_2+\dots N_k+N_{k+1}$. In order to mimic the marker gene expression in the corresponding cells, we generate $N_l,l=1,2,\dots k$ rows of $\mathbf{C}$ such that they have strong correlations to ${\mathbf{e}}_l^{\top }$. The rest of $N_{k+1}$ rows are non-marker genes, so they are just generated randomly. Then all rows of $\mathbf{C}$ are assembled and a random row-permutation is performed. Matrix $\mathbf{P}$ is just a random $k\times n$ matrix with non-negative entries and columns normalized by their $l_1$ norms. Data $\mathbf{G}$ is computed simply by $\mathbf{G}=\mathbf{CP}+ϵ$, where $ϵ$ is the noise matrix following normal distribution.

Figure 3 displays geometric structures of synthetic data under different settings. In this case, we take $n=30$, and $k=3$ and various dimensions of $N$. We define the noise to data ratio (NDR) as $\parallel ϵ{\parallel }_F/\parallel \mathbf{C}P{\parallel }_F$ and consider different (low, medium, and high) noise levels in simulations. Eigenvectors of graph Laplacians of these data are computed, and they are used as coordinates to plot the $N$ points in Figure 3. Note that according to spectral clustering theory, only the second and third eigenvectors are needed (because $k=3$ and the first one is almost a constant vector).

Figure 3.

Spectral clustering of synthetic data ($k=3$) with different setup. (1), (4), (7): Only marker genes with low, medium, and high noises; (2), (5), (8): all genes with different numbers (low to high) of non-marker genes with low noises; (3), (6), (9): all genes with low, medium, and high noises.

The first column of Figures 3, or (1), (4), and (7), shows the plots for marker genes only ($N_1=N_2=N_3=300$ while $N_4=0$) with low, medium, and high levels of noises. Three different colors represent the three clustered groups. It can be concluded from (1) that after clustering, all marker genes will concentrate around the vertices of the $k-1$ simplex since all the corresponding rows of $\mathbf{C}$ are strongly correlated to some ${\mathbf{e}}_l^{\top }$. With increasing noises, data points navigate away from the vertices. Keeping the same numbers for marker genes and low noises, Figures 3 (2), (5), (8) display data with $N_4=300,600,900$, and we can see that non-marker gene data only fill in the edge of the simplex. In contrast, Figures 3 (3), (6), and (9) show data of all genes ($N_1=N_2=N_3=N_4=300$) with low, medium and high levels of noises, respectively. It can be observed that strong noises will fill in the interior of the simplex.

In order to show that the proposed constraints are important, we perform the NMF without constraints, by simply setting ${\lambda }_1={\lambda }_2=0$. Initial starting points ${\mathbf{C}}_0$ and ${\mathbf{P}}_0$ are chosen randomly, so it can be seen in Figure 4 that each initial condition will result in a different result of $\mathbf{C}$ from others. In these experiments, the stopping criteria are the same (10⁻⁵) and the relative residues $\parallel \mathbf{G}-\mathbf{CP}{\parallel }_F/\parallel \mathbf{G}{\parallel }_F$ are the same and consistent to the NDR. Hence we can claim that the approximated stationary points are achieved but none of them is even close to the ground truth. The different solutions are due to the illposedness of the original NMF model.

Figure 4.

Complete deconvolution without constraint. (i)-(iii): Comparisons of true and simulated matrices of $\mathbf{C}$ with three different initial conditions.

Figure 5 presents computational results of the GS-NMF model on these synthetic data. Two sets of randomly generated matrices ${\mathbf{C}}^{*}$ and ${\mathbf{P}}^{*}$, with different levels of noises ($\text{NDR}=0.071,0.336$) are used to obtain data $\mathbf{G}$. Comparisons of ground truth (blue) to the corresponding computational results (red) are displayed in the left and right panels in Figure 5. The first and second rows are for $\mathbf{C}$ and $\mathbf{P}$, respectively. It can be seen that the solutions from the GS-NMF model are remarkably more reasonable comparing to the ground truth.

Figure 5.

Comparison of simulations of $\mathrm{C}$ (top) and $\mathrm{P}$ (bottom) with ground truth. For $\mathrm{P}$, its three rows are displayed in the three panels, respectively, i.e., proportion of three cell types in 30 samples. Qualitative results are given in Table 5.1.

Quantitative results can be found in Table 5.1, where relative errors (comparing to ground truth) of $\mathbf{C}$, $\mathbf{P}$, and relative residues $\parallel \mathbf{G}-\mathbf{CP}{\parallel }_F/\parallel \mathbf{G}{\parallel }_F$ are displayed for different NDRs. As indicated by both Figure 5 and the table, errors in matrix $\mathbf{C}$ increase more obviously when more noises present (larger NDR). On the contrary, computation of matrix $\mathbf{P}$ seems less vulnerable to noise levels. We observe that the relative residues $\parallel \mathbf{G}-\mathbf{CP}{\parallel }_F/\parallel \mathbf{G}{\parallel }_F$ for all iterations have been already comparable to the NDR, and this result implies that pursing even smaller residues in the cost functions is not necessary.

We also compare GS-NMF with Linseed [57], a recently developed deconvolution tool, for two sets of synthetic data. Figure 6 shows comparisons in cellular proportion ($\mathbf{P}$) simulations. Both methods can solve for $\mathbf{C}$ and $\mathbf{P}$ simultaneously from data $\mathbf{G}$. For some data, as shown in Fig. 6 (a), simulation results are similar, while for other data as in Fig. 6 (b), the proposed GS-NMF has better stability, as the Linseed predicts all zero proportion for the second cell type.

Figure 6.

Simulations of cellular proportion from GS-NMF and Linseed [57] for two sets of synthetic data.

5.2. Parameter discussion

There are five major parameters in models (5) and (13): the clustering parameter $\sigma$, ADMM penalty parameters $\rho$ and $\gamma$, and the geometric constraints parameters ${\lambda }_1$ and ${\lambda }_2$. As other optimization algorithms, there is barely generic way to find “optimal” tuning parameters, and choices of parameters depend on data and specific problems. Here we would rather discuss parameter choices based on the synthetic data and provide some general insights. Parameter $\sigma$ will be discussed in Section 5.3. When choosing ADMM parameters $\rho$ and $\gamma$, it is well-known that errors in $\mathbf{C}$ and $\mathbf{P}$ decrease for larger parameters, while too large values for $\rho$ and $\gamma$ will introduce matrix singularity in the algorithm. Computational results in Table 5.1 are obtained with $\rho =1.6\times {10}^3$ and $\gamma =1.5\times {10}^4$. For the geometric constraint parameters, it is still an open problem. We follow the idea of other manifold constrained methods [47, 7, 55, 20] and perform a grid search with candidates evenly spaced over the interval. To do so, we simply take ${\lambda }_1={\lambda }_2=\lambda$ and rescale to $\tilde{\lambda }=\lambda /\rho$ as it is defined in Eq. (13). Empirically, smaller $\tilde{\lambda }$ means less constraints on the geometric structure of $\mathbf{C}$ hence could damage solution identifiability. On the other hand, the extreme case $\tilde{\lambda }\to \infty$ implies the strong identifiability, which is not realistic when noises present. Figure 7 displays relative errors in $\mathbf{C}$, when parameters $\rho$ and $\gamma$ are fixed as above but $\tilde{\lambda }$ varies. It can be concluded that for both noise levels ($\text{NDR}=0.599$ and $0.071$), the change of errors against $\tilde{\lambda }$ is not monotone. For the testing data, $\tilde{\lambda }\approx 4$ or $5$ seems the best choice for computational accuracy. How to chose reasonable parameter $\tilde{\lambda }$ according to different data sets could be a future study.

Figure 7.

Relations between prediction errors (in $\mathbf{C}$) and geometric constraint parameter $\tilde{\lambda }$ at two noise-to-data ratios (NDR).

5.3. Algorithm results on biological data

We also validate the proposed algorithms by realistic biological data from GSE19830 [50]. This data set was obtained from tissue samples of the brain, liver, and lung of a single rat using expression arrays (Affymetrix). Homogenates of these three types of tissues were mixed together at the cRNA homogenate level with a known proportion, and then the gene expression pattern of every mixed sample was measured. The GSE19830 data set mimics the common scenario of heterogeneous biological samples which vary in the relative frequency of the component subsets from one to another and has been used in some literature [31, 57] to validate computational algorithms. For this dataset, we know cell type $k=3$ and tissue sample number $n=33$. After necessary data preprocessing to exclude obvious outliers (row norm, column norm, etc), we take $N=10,000$ out of $\sim 12,000$ total genes. Note that $k$ in the NMF is also the numerical rank of $\mathbf{G}$, so it is possible to estimate the total number of cell types for a dataset if it is unknown a prior, by investigating the distribution of the singular values of the original data matrix. For the GSE19830 data, we perform the singular value decomposition and display its singular value in Figure 8. Clearly the first three singular values of $\mathbf{G}$ are dominant, which indicate three cell types.

Figure 8.

Singular value distribution of GSE19830 dataset

Figure 9 displays the structure of data $\mathbf{G}$. Mutual correlations of rows of $\mathbf{G}$, i.e. gene expression of the 10, 000 genes in those 33 samples, are computed and shown as heat maps, before (left) and after (right) clustering/permutation. Clearly, there are three clusters and gene expressions are strongly correlated within each clustered group. Based this clustering and evaluation of correlations, marker genes could be computationally identified.

Figure 9.

Correlations of gene expressions across tissue samples.

Figures 10–11 show some clustering details. Since the clustering is based on eigenvectors of the graph Laplacian of data $\mathbf{G}$, we plot the first three eigenvectors of $\mathbf{L}$, with each column as the $x$-, $y$-, and $z$-coordinates of the $N$ dots, respectively in Figure 10 (a). Different colors represent the clustered groups. The first eigenvector is almost a constant vector, so it is convenient to just display the 2D data for the rest of figures, as in Figure 10 (b). All the dots are distributed in a ($k-1$)-simplex, and each cluster is identified with its vertex.

Figure 10.

Spectral clustering of GSE19830 data.

Figure 11.

Spectral clustering of GSE19830 data with $\sigma =0.2,0.5$, 1. (a)-(c): whole set; (d) Set ${𝓢}_r$

Notice that exploring the data structure depends on the parameter $\sigma$ in Eq. (5). Figure 11 shows data distribution with values of $\sigma =1,0.5$, and $0.2$. From Eq. (5) we see that the connectivity of any two vertices in the graph increases for large value of $\sigma$. This feature is displayed in Figure 11 (a), (b) and (c): when $\sigma =1$, all data points are distributed within almost a circle and the clustering is not that significant. On the other hand, when $\sigma =0.2$, the three vertices of the triangles naturally define the three clusters. In our experiments, the graph loses majority of connectivity for even smaller value, so $\sigma =0.2$ is used for all the simulations. To determine marker genes, we pick a subset ${𝓢}_r$ from each colored group and they are chosen as the one that are closest to each vertex. With $\sigma =0.2$, each set ${𝓢}_r$, $r=1,2,3$ contains 1,000 entries, and the corresponding data points are shown in Figure 11 (d).

With such parameter choices and set $\tilde{\lambda }=0.6$, we apply the proposed algorithms to data set GSE19830. Figure 12 shows the comparison between computed (red) cellular composition (liver, brain, lung, from top to bottom) in bulk tissue samples and ground truth (blue). In the 33 samples, 11 different cellular compositions were used and each of them was replicated three times. The computational results have reproduced this pattern. Additionally, the simulated cellular proportions fit the ground truth fairly well, especially for the third cell type.

Figure 12.

Comparison of simulated and true cellular proportions. From top to bottom are for liver, brain, and lung cells.

5.4. Model discussion and future work for more complicated data

Quantitatively, correlations of the computational results (red) and the ground truth (blue) in Figure 12 are 0.9916, 0.9916 and 0.9997 for the three cell types. They show the promising algorithm performances on biological data. On the other hand, we see that the predictions systematically underestimate the proportion of liver cells, while overestimate brain cells in samples. It indicates that simulation $\tilde{\mathbf{P}}$ and ground truth $\mathbf{P}$ differ by merely a scaling factor, i.e. $\tilde{\mathbf{P}}=\text{diag}\left(s_1,s_2,s_3\right)\mathbf{P}$. This phenomenon is majorly due to the inherent characteristics of NMF problem, and the definition of its solution uniqueness in Eq. (4). Further, even for such a definition of unique solution, one needs identifiability conditions on both $\mathbf{C}$ and $\mathbf{P}$, as demonstrated in Theorems 2.2 and 2.3. But in reality, not all these assumptions on both variables are satisfied (in this application we only have it on $\mathbf{C}$). Based on this issue, one can expect that there could be other challenges in the NMF framework for more complicated data. (i) One challenge could be significant heterogeneity in cellular proportions in samples, i.e., certain cell type may outnumber others across sample tissues. Mathematically, this means large differences between magnitudes of rows of $\mathbf{P}$. Figure 13 shows such a scenario: we designed a set of synthetic data where the proportion of one cell type in tissues is significantly less than the others. It can be seen in the third panel of Figure 13 (a) that our algorithms obviously overestimate the corresponding proportion. With additional assumption that gene expressions within cells (matrix $\mathbf{C}$) are not significantly different from each other, we computed row norms of data $\mathbf{G}$ for the corresponding marker genes and used them as rescaling factors during the iteration. This modified simulation provides much more accurate results, as displayed in Figure 13(b). However, we do not know how well the assumption will hold in realistic applications. (ii) Another challenge may arise when proportions for different cell types in tissue samples are highly similar. This situation will lead to strong correlations between rows of $\mathbf{P}$, and hence destroy the geometric structure of bulk tissue RNA-seq data: two vertices of the $k$-simplex as in the left panel of Figure 2 may overlap if there are two rows of $\mathbf{P}$ are strongly correlated. In summary, the novelty of the current work is to improve solution identifiably of the NMF model by only utilizing data $\mathbf{G}$ and the concept of marker genes. To improve solution quality, a future research direction within this framework is to address the above-mentioned challenges by combing necessary biological information for various complicated data.

Figure 13.

Comparison of $\mathbf{P}$ and $\tilde{\mathbf{P}}$: Algorithm performances on data $\mathbf{G}$ generated by $\mathbf{P}$ containing a very small row. (a) Original GS-NMF algorithm; (b) Algorithm with rescaling factors.

6. Conclusions

In this paper we develop a robust mathematical model and corresponding computational algorithms for complete data deconvolution. The major technique is nonnegative matrix factorization (NMF), which has a wide-range of applications in the machine learning community. Meanwhile, the NMF is a well-known strongly ill-posed problem, so a direct application of it to RNA-seq data will suffer severe difficulties in the interpretability of solutions. To address this issue, we leverage the biological concept of marker genes, combine it with the solvability conditions of the NMF theories, and hence develop a geometric structured guided optimization model. In this approach, the geometric structure of bulk tissue data is first explored by the spectral clustering technique. In this step, correlations graph among GEPs across tissue samples is established, and more importantly, marker genes for each cell types are identified. Then, information of marker genes is integrated as solvability constraints, while the overall correlation graph is used as manifold regularization. The resulting non-convex optimization problem, termed as geometric structured nonnegative matrix factorization (GS-NMF) model is numerically solved under the framework of alternating direction method of multipliers (ADMM). Finally, synthetic and biological data are used to validate the proposed model and algorithms. With this novel method, solution interpretability is significantly improved and accuracy is satisfactory comparing to the ground truth for both types of data. It is worthwhile to note that all simulation results may still suffer a linear scaling factor comparing to the ground truth. Unfortunately this is nothing to do with marker gene selection, parameter choices, or algorithm accuracy, but due to the inherent definition of NMF solution uniqueness. In the future research, we will combine necessary biological information in realistic applications, to reduce this scaling ambiguity as much as possible.

Table 2.

Quantitative results of the GS-NMF with different noise to data ratios (NDR).

NDR	erros in $\mathbf{C}$	errors in $\mathbf{P}$	Relative residue
0.071	0.0901	0.0444	0.0693
0.162	0.1007	0.0457	0.1543
0.336	0.1372	0.0545	0.3024
0.599	0.1667	0.0569	0.4888

A. Appendix

We summarize some theoretical concepts of the NMF model as follows:

A.1. Geometric interpretation.

In order to understand the strong and weak conditions of uniqueness, we review the NMF problem from the perspective of geometric structures. For $\mathbf{A}\in {\text{ℝ}}^{m\times n}$, the notation $cone(\mathbf{A})$ denotes the convex cone generated by the columns of $\mathbf{A}$, i.e.

$$cone\left\{\mathbf{A}\right\}=\left\{\mathit{x}\in {\text{ℝ}}^m\mid x=\mathbf{A}\theta ,\text{for some}\theta \in {\text{ℝ}}^n,\theta \ge \mathbf{0}\right\}\text{,}$$ (18)

and the conex hull of $\mathbf{A}$ is defined as

$$conv\left\{\mathbf{A}\right\}=\left\{\mathit{x}\in {\text{ℝ}}^m\mid \mathit{x}=\mathbf{A}\theta ,\text{for}\theta \in {\text{ℝ}}^n,\theta \ge \mathbf{0},\text{and}{\mathbf{1}}^{\top }\theta =1\right\}.$$ (19)

Note that we will illustrate with ${\mathbf{G}}^{\top }={\mathbf{P}}^{\top }{\mathbf{C}}^{\top }$ in the geometric structure because gene expressions across sample issues, i.e., columns of ${\mathbf{G}}^{\top }$ (row of $\mathbf{G}$) are our major interested data features. By non-negativity of $\mathbf{C}$ and $\mathbf{P}$, we have

$${\mathbf{G}}_{(i)}\in cone\left({\mathbf{P}}^{\top }\right)\subseteq {\text{ℝ}}_{+}^n,1\le i\le N,$$ (20)

and equivalently $cone\left({\mathbf{G}}^{\top }\right)\subseteq cone\left({\mathbf{P}}^{\top }\right)\subseteq {\text{ℝ}}_{+}^n$. Then problem (3) can be inter-preted as finding a nested cone problem: given two nested cones, $cone\left({\mathbf{G}}^{\top }\right)$ and ${\text{ℝ}}_{+}^n$, find the nested cone $cone\left({\mathbf{P}}^{\top }\right)$ between them. This interpretation is displayed in the left panel of Figure 14 with $N=8$ and $k=n=3$. It is easier to interpret this idea as shown in the right panel, in terms of convex hull view, which is one dimension less than the cone view. This can be done easily with normalization of columns of ${\mathbf{G}}^{\top }$ and ${\mathbf{P}}^{\top }$ to their unit $l_1$ norm. From the right panel of Figure 14, we can also see that the solution of $conv\left({\mathbf{P}}^{\top }\right)$ is not unique: data $\mathbf{G}$ can be contained within two different cones (red solid and green dashed triangles) formed by different choices of matrix $\mathbf{P}$. It also motivates the idea that if rows of $\mathbf{G}$ “spread out” enough in the nonnegative orthant, $cone\left({\mathbf{P}}^{\top }\right)$ or $conv\left({\mathbf{P}}^{\top }\right)$ may be unique. This condition turns into the following theories on the identifiability of the NMF.

Figure 14.

Cone (left) and convex hull (right) views of NMF as a nested cone problem.

A.2. Strong and weak conditions on identifiability:

There are two types of conditions for unique solution of problem (3) when $ϵ=0$. The strong and weak conditions are illustrated in Section 2, in which the definitions for separable and sufficiently scattered matrices are listed as follows:

Definition A.1.

The matrix $\mathbf{A}\in {\text{ℝ}}_{+}^{m\times n}$ is separable if $cone(\mathbf{A})={\text{ℝ}}_{+}^m$.

Definition A.2.

The matrix $\mathbf{A}\in {\text{ℝ}}_{+}^{m\times n}$ is sufficiently scattered if: (i) The second-order cone in ${\text{ℝ}}_{+}^m$ is contained in $cone(\mathbf{A})$, i.e. $𝓒=\left\{\mathbf{x}\in {\text{ℝ}}_{+}^m\mid {\mathbf{e}}^{\top }\mathbf{x}\ge \sqrt{m-1}\parallel \mathbf{x}{\parallel }_2\right\}\subseteq cone(\mathbf{A})$; and (ii) There does not exist any orthogonal matrix $\mathbf{Q}$ such that $cone(\mathbf{A})\subseteq cone(\mathbf{Q})$, except for permutation matrices.

Theorem 2.2 is quite rigorous: being separable matrices means $\mathbf{P}$ and ${\mathbf{C}}^{\top }$ must contains (scaled) extreme rays of the nonnegative orthant in the corresponding space, i.e., for every $r=1,2,\dots k$ there exists a column index $l_r$, such that ${\mathbf{P}}^{\left(l_r\right)}={\alpha }_r{\mathbf{e}}_r\in {\text{ℝ}}_{+}^k$, where ${\alpha }_r$ is a scalar. This condition is illustrated in the left panel of Figure 15 as $k=3$. Note it is the convex hull view, so columns of $\mathbf{P}$ are represented as blue dots in the unit simplex (red triangle) in ${\text{ℝ}}_{+}^3$. As in the figure, some columns of $\mathbf{P}$ are required to be exactly align with unit vectors ${\mathbf{e}}_r$, $r=1,2,3$ (overlapping with red dots). Similar situation is for matrix ${\mathbf{C}}^{\top }$. Such assumptions on both variables are too strong for practical applications, especially when noises present.

On the other hand, Theorem 2.3 is much more relaxed: The right panel of Figure 15 illustrates such condition: the dashed circle represents the intersection of the second-order cone in ${\text{ℝ}}_{+}^3$ and the unit simplex. All blue objects, including dots and pentagons, are for columns of $\mathbf{P}$. In this case, none of those columns are required to overlap with ${\mathbf{e}}_r$, but some of them (pentagons) need to fall out of the circle (second-order cone).

Figure 15.

Convex hull views for strong (left) and weak (right) identifiability conditions with $k=3$.