A novel estimator of the interaction matrix in Graphical Gaussian Model of omics data using the entropy of non-equilibrium systems

Abstract

Motivation

Inferring the direct relationships between biomolecules from omics datasets is essential for the understanding of biological and disease mechanisms. Gaussian Graphical Model (GGM) provides a fairly simple and accurate representation of these interactions. However, estimation of the associated interaction matrix using data is challenging due to a high number of measured molecules and a low number of samples.

Results

In this article, we use the thermodynamic entropy of the non-equilibrium system of molecules and the data-driven constraints among their expressions to derive an analytic formula for the interaction matrix of Gaussian models. Through a data simulation, we show that our method returns an improved estimation of the interaction matrix. Also, using the developed method, we estimate the interaction matrix associated with plasma proteome and construct the corresponding GGM and show that known NAFLD-related proteins like ADIPOQ, APOC, APOE, DPP4, CAT, GC, HP, CETP, SERPINA1, COLA1, PIGR, IGHD, SAA1 and FCGBP are among the top 15% most interacting proteins of the dataset.

Availability and implementation

The supplementary materials can be found in the following URL: http://dynamic-proteome.utmb.edu/PrecisionMatrixEstimater/PrecisionMatrixEstimater.aspx.

Supplementary information

Supplementary data are available at Bioinformatics online.

1 Introduction

Direct relationships between biomolecules such as proteins, DNA and RNA, are crucial components in most of the biological processes (Bonetta, 2010; Maslov and Sneppen, 2002; Uetz et al., 2000; Wagner and Fell, 2001). By studying these biological networks, one can gain insight into important features of the involved processes and associated diseases (Barabási and Oltvai, 2004; Ideker and Krogan, 2012).

However, the success of such studies depends on how well the representing network is constructed. For this purpose, the edges of the network should be indicative of direct relationships between the biomolecules. Therefore, despite the popularity of co-expression analyses, they are not suitable for network inference because the correlations between the molecular expressions can be due to indirect as well as direct interactions while only the latter should be taken into consideration for constructing biological networks. Co-expression networks also suffer from a high number of false negatives and positives (Hansen et al., 2014; Mutwil et al., 2011). Alternative methods to derive the direct relationships from expression data are the relevance network analysis (Butte and Kohane, 1999), the Bayesian analysis (Friedman, 2004; Friedman et al., 2000), methods based on singular value decomposition (SVD) (Alter et al., 2000; Sardiu et al., 2008), methods based on the notion of entropy (Lezon et al., 2006; Margolin et al., 2006) and physics based methods (Kravchenko-Balasha et al., 2012; Wallace et al., 2019). Deriving the underlying interaction networks from expression datasets is still under active investigation (Hernaez and Gevaert, 2018).

Gaussian Graphical Model (GGM) is a graphical representation of the direct relations between the molecules in omics datasets. Despite its simplicity, GGM has strong grounds and has received much attention in the community (Friedman et al., 2008; Kishino and Waddell, 2000; Schafer and Strimmer, 2005; Toh and Horimoto, 2002; Zhang and Kim, 2014). The applicability of GGM is supported by the central limit theorem according to which the distribution of a vector of variables x, molecular expressions, with many independent driving factors, is approximately multivariate normal

$$P(x)=\frac{1}{\sqrt{{(2\pi )}^n\text{det}\Sigma }}\text{exp}\hspace{0.17em}(-\frac{1}{2}{\left(x-\mu \right)}^{\mathrm{T}}·{\Sigma }^{-1}·\left(x-\mu \right)),$$ (1)

where $x\in {ℝ}^n$, n is the number of molecular species, ${\Sigma }^{-1}$ is the n × n dimensional interaction matrix, and is the inverse of the covariance matrix Σ, and μ is the mean of x. An important property of the interaction matrix is that its components determine the conditional dependency (direct relationship) between any pair of molecules. In other words, ${\Sigma }_{ij}^{-1}=0$ if and only if molecules i and j are independent assuming all the conditions on the rest of molecules. On the other hand, ${\Sigma }_{ij}=0$ if molecules i and j are independent without taking the rest of the conditions into account. Therefore, in a GGM representation of omics data, molecules are the nodes of a graph where the edges between nodes i and j are drawn only if ${\Sigma }_{ij}^{-1}\ne 0$.

GGM was originally designed to represent datasets whose dimensions n is less than the number of samples (Whittaker, 2009), where the maximum likelihood estimation of the interaction matrix is a natural solution. However, in most of the omics experiments, the number of samples is only a few while thousands of molecules are measured each time. Therefore, it is impossible to estimate the interactions matrix of the GGM unless a sort of dimension reduction (feature selection) is employed. There has been several attempts to prescribe a systematic estimation of the precision matrix in high-dimensional underdetermined settings (e.g. Banerjee et al., 2008; Friedman et al., 2008; Ledoit and Wolf, 2004).

In this article, we develop a systematic estimation of the precision matrix in GGM using the thermodynamic entropy of the system of molecules in the data samples. The proposition that biomolecular expressions obey the physical principles of non-equilibrium systems outlined in statistical mechanics has appeared in a long series of publications under the title of ‘surprisal analysis’ (SA) (e.g. Bogaert et al., 2018; Johnston et al., 2019; Levine, 1978; Remacle et al., 2010; Vasudevan et al., 2018), and the references therein. In SA, it is assumed that the system of molecules are shifted away from their maximal entropy state, i.e. the equilibrium state, due to a few constraints whose analytic forms are known. We use the framework of SA and the assumptions therein to devise an estimator for the interaction matrix in GGM. We start with the analytic formula for the entropy of the system to construct the associated probability function and use the proposed forms of the constraints in SA to shrink the phase space of the partition function of the system. The calculations of the dimension reduction lead to the analytic form of the precision matrix in GGM.

We apply our estimator to a proteomics dataset of the human plasma proteome, where the patients are at different stages of non-alcoholic fatty liver disease (NAFLD). We show that the assumptions proposed by SA and required by our method are respected in the dataset, and derive the corresponding interaction matrix. We build the corresponding GGM representation of the dataset and show that NAFLD-related proteins are among the top 15% most interacting proteins according to our model. The constructed Gaussian distribution will be used to measure the distance of each patient sample from the equilibrium state. We show that patients with the most advanced liver disease are farthest away from the equilibrium state.

We would like to mention that another precision matrix estimator based on entropy maximization principles (MaxEnt) has been introduced in Lezon et al. (2006). The latter is based on information theory where, unlike in thermodynamics, the entropy and consequently the distribution function have no analytic formulas. The method in Lezon et al. (2006) assumes that the covariance matrix is equal to the Pearson correlation. The probability function that maximizes the information entropy under the constraint on the covariance matrix is taken as the distribution function. One of the advantages of our method is that we start with basic principles to derive, and not assume, the covariance matrix.

2 Materials and methods

Following the SA (i.e. Bogaert et al., 2018; Johnston et al., 2019; Levine, 1978; Remacle et al., 2010; Vasudevan et al., 2018), we assume that the biomolecules are governed by the rules of out of equilibrium systems whose entropies read

$$S=k\sum _i(-X_i\ln X_i+X_i+X_i{S}_i),$$ (2)

where k is the Boltzmann constant, X_i is the abundance of the ith molecule and S_i is the logarithm of the abundance X_i at the maximal entropy state.

There exists a probability of occurrence for any hypothetical sample with n given molecular expressions. To find this probability distribution, we utilize the relationship between the entropy and the number of states $\Omega (X_1\cdots X_n)$ which are microscopically different but have the same macroscopic properties in an ensemble, as prescribed by Boltzmann law,

$$S=k\ln \Omega (X_1\cdots X_n).$$ (3)

On the other hand, the probability of occurrence of a particular configuration of $(X_1\cdots X_n)$ is proportional to the number of micro-states $\Omega (X_1\cdots X_n)$, and reads

$$P\left(X_1\cdots X_n\right)=\frac{\Omega \left(X_1\cdots X_n\right)}{\tilde{\Omega }\left({\tilde{X}}_1\cdots {\tilde{X}}_n\right)}=\text{exp}\hspace{0.17em}\left(\frac{S-\tilde{S}}{k}\right),$$ (4)

where tilde refers to the maximal entropy state, i.e. a hypothetical sample with molecular abundances ${\tilde{X}}_1\cdots {\tilde{X}}_n$ that are the solutions to $\frac{\partial S}{\partial X_i}=0$ where S refers to Equation 2. We would like to emphasize that this is an unnormalized probability distribution. We have dropped the normalization factor since it cancels out in measurable quantities.

In most of the situations, the entropy does not deviate far from its maximal state. Hence, we can expand the entropy around that state and work with the first non-zero term. This near-equilibrium assumption should be verified in data as we do later in Section 3. The Taylor expansion reads

$$\begin{array}{c}S-\tilde{S}=\frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n\frac{{\partial }^{2}S}{\partial X_i\partial X_j}{|}_{\tilde{X}}\left(X_i-{\tilde{X}}_i\right)\left(X_j-{\tilde{X}}_j\right)\\ +\hspace{0.17em}\mathcal{O}\left({\left(X-\tilde{X}\right)}^3\right)\\ \simeq -\frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n\frac{k{\delta }_{ij}}{{\tilde{X}}_i}\left(X_i-{\tilde{X}}_i\right)\left(X_j-{\tilde{X}}_j\right),\end{array}$$ (5)

where we have used the definition of the maximal state $\frac{\partial S}{\partial X_i}{|}_{\tilde{X}}=0$ in the first line and Equation 2 in the second line. Therefore, the probability function takes a Gaussian form with a diagonal precision matrix

$$P\left(X\right)=\text{exp}\hspace{0.17em}\left(-\frac{1}{2}\sum _{i,j=1}^n\frac{{\delta }_{ij}}{{\tilde{X}}_i}\left(X_i-{\tilde{X}}_i\right)\left(X_j-{\tilde{X}}_j\right)\right).$$ (6)

If there were no constraint between the expression of molecules, this was the final result. In order to systematically capture the effects of constraints on the distribution function, we note that the partition function of a given statistical system is the sum over all of the possible states weighted by the associated probability. Therefore, the partition function, from which every macroscopic quantity can be calculated, reads

$$\begin{array}{c}\mathcal{Z}=\int d{X}_1\cdots d{X}_n\delta (f_1\left(X\right))\cdots \delta (f_{n_{\alpha }}\left(X\right))\times \\ \hspace{0.17em}\text{exp}\hspace{0.17em}\left(-\frac{1}{2}\sum _{i,j=1}^n\frac{{\delta }_{ij}}{{\tilde{X}}_i}\left(X_i-{\tilde{X}}_i\right)\left(X_j-{\tilde{X}}_j\right)\right),\end{array}$$ (7)

where the exponential term is the probability derived in Equation 6, and the delta functions refer to the constraints on the expressions of biomolecules, i.e. there are $n_{\alpha }$ number of relationships between the expressions of molecules in a given sample such that the expression of $n_{\alpha }$ number of molecules can be determined without directly measuring them using the expressions of the rest of molecules. According to the SA, the constraints read (Bogaert et al., 2018; Johnston et al., 2019; Levine, 1978; Remacle et al., 2010; Vasudevan et al., 2018)

$$f_{\alpha }(X)\equiv \sum _{i=1}^n{G}_{i\alpha }{X}_i-C_{\alpha }=0, \alpha =1,\dots ,n_{\alpha },$$ (8)

where $G_{i\alpha }$ is a constant for the ith molecule in the αth constraint and is independent of the samples in a dataset, and $C_{\alpha }$ are constants. Later in Section 3, we show that such constraints exist in our dataset. The existence of the constraints in other datasets can be seen in the mentioned references above. In Supplementary Text, we discuss SA prescription to derive $G_{i\alpha }$ factors from data. These are the components of a matrix equal to the first $n_{\alpha }$ columns of the left unitary matrix of the SVD of the logarithm of dataset.

When the number of molecules is high, the number of existing constraints is also expected to be high. Since omics datasets are under-determined, it is in principle not possible to learn all of the constraints. Instead, we can focus on identifying the most significant ones. This is done using SVD, as discussed above, where as many significant constraints as the number of samples are identified, i.e. $n_{\alpha }$ equals the number of samples. It should be noted that in an unlikely situation where the number of constraints is truly less than the number of samples, we will have less number of non-zero singular values after decomposing the dataset. In which case, $n_{\alpha }$ should be set equal to this number.

Using the delta functions, we can write $n_{\alpha }$ of the molecular expressions in Equation 7 in terms of the expressions of the rest of the molecules. Since the integral over any of the variables can be used to remove the delta functions, we arbitrarily choose to remove the last $n_{\alpha }$ abundances and write the partition function as

$$\mathcal{Z}=\int d{X}_1\cdots d{X}_{n_d}\hspace{0.17em}\text{exp}\hspace{0.17em}\left(-\sum _{i=1}^n\frac{{\left(X_i-{\tilde{X}}_i\right)}^2}{2{\tilde{X}}_i}\right),$$ (9)

where now $n_d\equiv n-n_{\alpha }$. Note that the last $n_{\alpha }$ terms in the sum in the exponential are not independent. The terms in the exponent can be rewritten as-

$$\sum _{i=1}^n\frac{{\left(X_i-{\tilde{X}}_i\right)}^2}{2{\tilde{X}}_i}=\sum _{i=1}^{n_d}\frac{{\left(X_i-{\tilde{X}}_i\right)}^2}{2{\tilde{X}}_i}+\sum _{\alpha =n_d+1}^n\frac{{\left(Z_{\alpha }-{\tilde{Z}}_{\alpha }\right)}^2}{2{\tilde{Z}}_{\alpha }},$$ (10)

where $Z_{\alpha }$ refer to the dependent protein abundances ($\equiv X_{\alpha }$) and need to be written in terms of the n_d independent protein abundances. We emphasize that there is no relation between $Z_{\alpha }$ and the partition function $\mathcal{Z}$. This can be done by rewriting Equation 8

$$C_{\alpha }=\sum _{i=1}^n{G}_{i\alpha }{X}_i=\sum _{i=1}^{n_d}{G}_{i\alpha }{X}_i+\sum _{\beta =1}^{n_{\alpha }}{H}_{\beta \alpha }{Z}_{\beta },$$ (11)

where $H_{\beta \alpha }$ is a $n_{\alpha }\times n_{\alpha }$ matrix equivalent to the last $n_{\alpha }$ rows of $G_{i\alpha }$ matrix. It should be noted that i runs from one to the number of molecules and α runs from one to the number of constraints $n_{\alpha }$. Hence, $G_{i\alpha }$ refers to the components of a $n\times n_{\alpha }$ matrix. The bottom $n_{\alpha }$ rows of this matrix makes a square matrix that we named H.

Therefore, the components of the dependent protein abundances are

$$Z_{\alpha }=\sum _{\beta =1}^{n_{\alpha }}{(H^{-1})}_{\alpha \beta }{C}_{\beta }-\sum _i^{n_d}\sum _{\beta =1}^{n_{\alpha }}{(H^{-1})}_{\alpha \beta }{G}_{i\beta }{X}_i.$$ (12)

Substituting this into Equation 10 and inserting the latter in Equation 9, and with a straightforward but tedious calculation, we can rewrite the partition function as

$$\begin{array}{c}\mathcal{Z}=\int d{X}_1\cdots d{X}_{n_d}\hspace{0.17em}\text{exp}\hspace{0.17em}(-\frac{1}{2}\sum _{i,j=1}^{n_d}{A}_{ij}{X}_i{X}_j+\sum _{i=1}^{n_d}{B}_i{X}_i+D),\\ A_{ij}=\frac{{\delta }_{ij}}{{\tilde{X}}_i}+\sum _{\alpha ,\beta ,\gamma =n_d+1}^n\frac{1}{{\tilde{Z}}_{\alpha }}{\left(H^{-1}\right)}_{\alpha \beta }{G}_{i\beta }{\left(H^{-1}\right)}_{\alpha \gamma }{G}_{j\gamma },\\ B_i=1+\sum _{\alpha ,\beta ,\gamma =n_d+1}^n\frac{1}{{\tilde{X}}_{\alpha }}{\left(H^{-1}\right)}_{\alpha \beta }{G}_{i\beta }{\left(H^{-1}\right)}_{\alpha \gamma }{C}_{\gamma }.\end{array}$$ (13)

where the exponential in the first line is the probability distribution function in the non-constrained true physical space, and D being the normalization factor, independent of X, with no significance.

To get an insight into the components of the partition function, we work out all of the integrals

$$\mathcal{Z}=e^D\sqrt{\frac{{(2\pi )}^{n_d}}{\text{det}(A)}}\hspace{0.17em}\text{exp}\hspace{0.17em}\left(\frac{1}{2}{B}^{\mathrm{T}}·A^{-1}·B\right),$$ (14)

where the integration details can be found in Supplementary Text. The definition of the partition function implies the following relation for the expectation values

$$\mathrm{E}\left[X_{i_1}\cdots X_{i_k}\right]=\frac{1}{\mathcal{Z}}\frac{{\partial }^k\mathcal{Z}}{\partial B_{i_1}\cdots \partial B_{i_k}}.$$ (15)

We now derive the first two moments using Equation 14

$$\begin{array}{c}\mathrm{E}[X_i]={(B^{\mathrm{T}}·A^{-1})}_i,\\ \mathrm{E}[X_i{X}_j]={(A^{-1})}_{ij}+{(B^{\mathrm{T}}·A^{-1})}_i{(B^{\mathrm{T}}·A^{-1})}_j.\end{array}$$ (16)

The covariance matrix by definition reads

$${\Sigma }_{ij}\equiv \mathrm{E}[X_i{X}_j]-\mathrm{E}[X_i]\mathrm{E}[X_j],$$ (17)

which by direct substitution is equal to the inverse of the interaction matrix in the reduced space ${(A^{-1})}_{ij}$. Therefore, ${\Sigma }^{-1}$ and μ in Equation 1 are equal to A and $(B^{\mathrm{T}}·A^{-1})$ of Equation 13, respectively. It should be noted that both ${\Sigma }^{-1}$ and μ are now given in terms of $\tilde{X}$ and $G_{i\alpha }$, which can be learned from the dataset under study using the SA prescription, which we present in Supplementary Text.

Although GGM depends on the estimation of the interaction matrix which is analytically given as matrix A in Equation 13, one can be interested in the inverse matrix $A^{-1}$. The easiest approach would be to introduce matrix A to the computer and use the available linear algebra packages to compute the inverse. We will use this method on several occasions in the rest of the article. Provided that the second term of A in Equation 13 is much smaller than the first term, an analytic formula can be derived. Intuitively, this condition should be naturally satisfied in near equilibrium systems because when the system is exactly in the equilibrium state, the second term of A is exactly zero. If we show the first term of A_ij in Equation 13 by a_ij and the second term by b_ij and assuming that $b\ll a$ such that b² is negligible, and given that a_ij is diagonal and its component-wise inverse is $a_{ij}^{-1}=\frac{{\delta }_{ij}}{a_{ii}}$, we have

$$A_{ij}^{-1}=a_{ij}^{-1}-\frac{b_{ij}}{a_{ii}{a}_{jj}}+\mathcal{O}(b^2).$$ (18)

So far, we have derived the covariance between the n_d independent molecules. The covariance between the dependent and the independent ones, by definition, is

$$\mathrm{E}[X_i{Z}_{\alpha }]-\mathrm{E}[X_i]\mathrm{E}[Z_{\alpha }]=-\sum _{j=1}^{n_d}{(A^{-1})}_{ij}{I}_{j\alpha },$$ (19)

where $I_{j\alpha }\equiv \sum _{\beta =1}^{n_{\alpha }}{G}_{j\beta }{(H^{-1})}_{\alpha \beta }$. The right hand side is derived by substituting Equation (12) directly into the left-hand side of equation above and utilizing the covariance among the independent expressions. The covariance between the dependent molecules themselves can be found by following a similar procedure

$$\mathrm{E}[Z_{\alpha }{Z}_{\beta }]-\mathrm{E}[Z_{\alpha }]\mathrm{E}[Z_{\beta }]=\sum _{i,j=1}^{n_d}{(A^{-1})}_{ij}{I}_{i\alpha }{I}_{j\beta }.$$ (20)

It should be emphasized that our analytic formula for the covariance between any pair of molecules, remains the same regardless of the way the dependent molecules labeled by $Z_{\alpha }$ are selected. The reason is that the expectation value of the polynomials of X should be calculated by placing the polynomial inside the integral in Equation 7. The result is independent of the integration procedure. However, A_ij is only the inverse of the covariance matrix of the independent molecules and is equal to the precision matrix in their space. Therefore, it is a measure of the correlation between any pair of independent molecules conditioned on the rest of the molecules labeled by $X_{\alpha }$.

Finally, we investigate the performance of our method by simulation. We arbitrarily choose the number of samples and molecules to be 48 and 266 respectively. We assume that 48 constraints of the type in Equation 8 exist among the molecular expressions in the dataset. We assume arbitrary values for the weight factors $G_{i\alpha }$, and the first $266-48=218$ molecular expressions at the maximal entropy state ${\tilde{X}}_i$. The assumed values are stored in an available Python pickle file as explained in Supplementary Text. We use the constraints to derive the values for the 48 dependent molecular expressions ${\tilde{Z}}_i$. Finally, we generate 48 samples by randomly drawing from a multivariate normal distribution with a 266-dimensional diagonal covariance matrix equal to ${\delta }_{ij}{\tilde{X}}_i$ as in Equation 7. Calculations above guarantee that the interaction matrix in the reduced 218-dimension space is given by A_ij of Equation 13 in terms of our assumed values for $G_{i\alpha }$ and ${\tilde{X}}_i$. The relevant Python code is available and discussed in Supplementary Text.

The generated dataset is then passed to our estimator, another available Python code which is discussed in Supplementary Text, as well as the Ledoit-Wolf (Ledoit and Wolf, 2004) estimator, the Oracle Approximating Shrinkage (OAS) estimator (Chen et al., 2010) both in Python’s scikit-learn package (Pedregosa et al., 2011), together with MaxEnt estimator (Lezon et al., 2006). It should be noted that the MaxEnt estimator’s covariance matrix is the same as the empirical covariance matrix defined as the observed covariance of the dataset $\frac{X·X^{\mathrm{T}}}{n_d}$. MaxEnt’s interaction matrix is the inverse of the empirical covariance matrix in the non-singular space defined by the non-zero eigenvalues of the SVD of the empirical covariance matrix. The rest of the estimators, including ours, only use the dataset to learn some parameters of the model but otherwise have different covariance matrices than the empirical one.

In Figure 1a, the logarithm of the absolute value of the estimated matricesare shown in the form of heatmaps and compared with the true values of the analytic formulas of $A_{ij}^{-1}$ and A_ij. On the other hand, in Figure 1b all $n_d^2$ estimated pair-wise interactions are compared with their true values such that the x-axis shows the difference between the logarithm of the values and the y-axis shows the number of the pair-wise interactions. In an ideal estimation, the height of the first bin of the histogram is $n_d^2$ and the rest of the bins are empty. The Frobenius norms of the errors of all the interactions in Figure 1a and b are shown in Table 1. Therefore, provided that a dataset possesses the constraints as discussed above, our method returns a better estimation of the interactions among molecules. The existence of such constraints should be confirmed in data.

Fig. 1.

(a) Comparison of the logarithm of the absolute value of the estimated covariance matrices in the top and interaction matrices in the bottom row with their corresponding true matrices. The very right column is the true matrix while the rest of the columns are our method, Ledoit-Wolf, OAS and MaxEnt methods respectively to the left. (b) The absolute value of the difference between the log of the estimated matrices and the true ones. The very right column is devoted to our mehtod while Ledoit-Wolf, OAS and MaxEnt methods are shown respectively to the left. Top row shows the histogram of errors in the estimated covariance matrices. The bottom row shows the histogram of errors in the estimated interaction matrices. The bottom row shows that our method returns a more accurate description of the true interaction matrix

Table 1.

The Frobenius norm of the difference of the logarithms of estimated and true matrices, divided by the dimension of the matrices

	$n_d^{-1}\left|\right|\left(\text{log}\hspace{0.17em}\left|\frac{{\Sigma }_{\text{est}.}}{{\Sigma }_{\text{true}}}\right|\right)|{|}_{\mathrm{F}}$	$n_d^{-1}\left|\right|\left(\text{log}\hspace{0.17em}\left|\frac{{\Sigma }_{\text{est}.}^{-1}}{{\Sigma }_{\text{true}}^{-1}}\right|\right)|{|}_{\mathrm{F}}$
Our method	2.0	2.8
Ledoit-Wolf	2.1	11.6
OAS	3.0	13.0
MaxEnt	25.8	12.4

In Supplementary Text, we present a similar simulation study to show the improved estimation of our method under other sample numbers.

3 Results

NAFLD is the most widespread type of liver disease that affects three out of ten people in developed countries. The treatment of NAFLD is challenging, and its diagnosis is only possible at advanced stages. In this section, we apply our method to a plasma proteomics dataset of humans’ with different degrees of NAFLD to estimate the interaction matrix of proteins associated with the disease and also to classify the patients for diagnostic purposes.

The dataset is made available in Niu et al. (2019) and contains the abundances of 431 proteins from 48 patients at different levels of liver disease. The patients are categorized into five major groups according to corresponding clinical inputs. These include ten obese patients, ten obese patients with NAFLD, eight patients with type-II diabetes (T2D), ten patients with T2D and NAFLD and ten patients with Cirrhosis. The reported proteins have already passed some quality selections. However, only 266 of the proteins are identified and quantified for every patient and are kept in our analysis.

To build the GGM that represent the data, we need to (i) estimate $G_{i\alpha }$ and ${\tilde{X}}_i$ using the SA prescription described in Supplementary Text, (ii) validate the near-equilibrium assumption and (iii) validate the existence of the constraints. Therefore, we construct the dataset matrix X in which rows are the 266 remaining proteins, and columns are the 48 patients. The URL to retrieve the matrix is provided in Supplementary Text. To derive the unknown parameters of our method, we decompose the natural log of X, hereafter called Y, using the singular value decomposition method and define the G matrix as the first 48 columns of the left unitary matrix of Y. We also define the λ matrix as the dot product of the diagonal matrix with the right unitary matrix multiplied by a minus sign. This, and the rest of the computations, are carried out using a Python script that is discussed in Supplementary Text. The natural log of the protein expressions at the maximum entropy state, $\tilde{Y}$, is estimated to be the dot product of the first column of G and negative of the first row of λ.

In the Taylor expansion of entropy in Equation 5, we neglected higher-order terms by assuming that the system under study never moves far from the maximal entropy state. We validate this assumption by Figure 2, which shows the measured expressions of every protein of every sample versus their corresponding maximal entropy expressions. From the figure, one can observe that $|Y-\tilde{Y}|\ll 1$, which in turn means that the higher-order terms are negligible, and according to Equation 6, the distribution function for protein expressions takes a multivariate Gaussian form, which for the 266 molecular expressions of a given sample reads

$$P\left(X\right)=\text{exp}\hspace{0.17em}\left(-\sum _{i=1}^{266}\frac{{\left(X_i-{\tilde{X}}_i\right)}^2}{2{\tilde{X}}_i}\right).$$ (21)

Fig. 2.

Measured protein expressions versus the protein expressions in the maximal entropy state, as defined below Equation 4. Each point on the figure represents one protein of one patient. The median of the differences between x and y values is negligible. Therefore, this plot confirms that all patients are close to the maximal entropy state and, therefore, the first non-zero term in the expansion of entropy, can appropriately describe the data. In Supplementary Figure S4 of Supplementary Text, we show that the maximal state expressions are approximately equal to the sample average of the protein expressions

Before constructing the graph that represents the dataset, we use the constructed distribution function to assign a probability to every patient, based on his/her measured plasma proteome, for diagnosis purposes. The maximal entropy state has the highest probability since the exponent of the expression is zero. On the other hand, if the measurements of proteins are far from the maximal state, the exponent tends to negative infinity and the probability is zero. Figure 3 shows the absolute value of the exponent of the distribution function for the 48 patients in the dataset. The figure indicates that the obese but otherwise healthy patients (the red dots) have received the highest probability, i.e. are closest to the maximal entropy state, while the patients with the most severe liver disease (the black dots) have received the lowest probability, i.e. are farthest away from the maximal entropy state. This means that the liver disease perturbs the system away from its most stable state. The degree of perturbation depends on the stage of the disease. If the color labels, that are determined clinically, are used as the truth values, we can evaluate the performance of the classification. We choose the median of all red labeled patients as the separation line between NAFLD and non-NAFLD and count the number of true and false positives and negatives. It should be noted that the dataset is nearly balanced as we have eight or ten number of samples per each category. The results are listed in Table 2, and indicate a relatively high F1 score of 84%, defined as two times the precision times the recall divided by the sum of the two.

Fig. 3.

Patient classification based on the probability assigned to his/her plasma proteome dataset. The x-axis refers to each patient (data sample), and the y-axis is minus the logarithm of the probability of the occurrence of that sample based on its given molecular expressions. The probability is given by the exponential term in the first line of Equation 13. The color labels are determined by clinicians and are used as the truth values. The red line is the median of the y-axis value of patients with red label, which are considered healthy

Table 2.

Evaluation of the patient classification using the thermodynamics-based model presented in this article

	Our method
True positive	25
False positive	4
True negative	4
False negative	5
Precision	86%
Recall	83%
F1 score	84%

To estimate the interaction matrix and construct the GGM associated with the NAFLD dataset, we need to ensure that our dataset contains the constraints in Equation 8. Therefore, we multiply the components of the G matrix by the protein expressions, and plot the sum of them in Figure 4 for every patient sample. The figure indicates that the sum is approximately constant for every patient. Therefore, the constraints exist in the dataset, and the prerequisite for our method is met.

Fig. 4.

The sum of protein abundances with constant weights is nearly the same for every patient, i.e. $C_{\alpha }$ in Equation 8 is the same for every patient. Each line represents a constraint that is labeled by $\alpha =1\cdots n_{\alpha }$

The interaction matrix of the GGM is equal to A_ij in Equation 13 after substituting the data-driven estimations of $G_{i\alpha }$ and ${\tilde{X}}_i$. In Figure 5, made with Python’s networkx library (Hagberg et al., 2008), we show a line between any pair of proteins if the corresponding component in the interaction matrix has an absolute value of more than one-fifth of the maximum absolute value among all the components of the interaction matrix. The blue dots in the figure represent the proteins with no significant interaction with any other protein. The network of proteins with significant interactions is expected to contain the proteins that are dysregulated by NAFLD. We note that among the proteins in the figure are PIGR, IGHD, SAA1 and FCGBP, which are identified as the NAFLD-related proteins by Niu et al. (2019). Moreover, the figure also contains other proteins that are mentioned in other publications as NAFLD-related proteins Piero et al. (2016). These are ADIPOQ, APOC, APOE, DPP4, CAT, GC, HP, CETP, SERPINA1 and COLA1. In Supplementary Table SI of Supplementary Text, we have summarized and compared the list of NAFLD-related proteins identified by our method as well as by references Niu et al. (2019) and Piero et al. (2016).

Fig. 5.

GGM for protein interaction network. The blue dots are proteins that are not interacting significantly with any of the other proteins. Unnamed proteins in the original dataset are labeled as nan. Only the geometry of the graph is meaningful and the length of the lines has no meaning

Before moving forward, we would like to emphasize that the protein interaction presented above is solely based on the direct relationships between molecular expressions and is different than the so-called protein–protein interaction network which represents the physical forces between proteins. To construct the latter network, experiments like yeast-2-hybrid are required and omics experiments are not relevant.

We end this section by evaluating the differences between the estimated interaction matrices of the NAFLD dataset. For this purpose, we pass the NAFLD dataset to Ledoit-Wolf, OAS and MaxEnt estimators and compare the output with our estimation of the interaction matrix. Figure 6 shows the box plot of the component-wise differences between the three methods’ and our estimation of the interaction matrix defined as $\frac{2\left({\Sigma }_{\text{our}}^{-1}-{\Sigma }_{\text{other}}^{-1}\right)}{\left({\Sigma }_{\text{our}}^{-1}+{\Sigma }_{\text{other}}^{-1}\right)}$. It should be noted that the latter definition is because unlike in simulation study, the ground truth is not known, and therefore the relative comparison in terms of the Frobenius norm is not possible. Under this condition, the latter definition provides a better baseline. The plot shows that the differences are small in general with the median of the differences all $\sim 0.2$ while MaxEnt in general has a closer performance to our method than the other two.

Fig. 6.

The box plot of the component-wise differences between the logarithm of our estimation of NAFLD interaction matrix and the logarithm of the estimations of Ledoit-Wolf, OAS and MaxEnt. Overall, the differences concerning our method’s estimation of the interaction matrix are small

4 Discussion

High-throughput omics experiments are rather prevalent in recent biomedical and biological studies. To understand the mechanisms involved in biology and also to study the diseases using omics experiments, we need an understanding of direct relationships between the measured biomolecules. GGM provides a powerful but simple representation of such interactions provided that we have a fair estimate of its interaction matrix. Since the generated datasets have too many features while only a few samples become available, estimation of the corresponding interaction matrix is challenging. Traditional maximum likelihood methods lead to underdetermined sets of equations. As a result, some sort of dimension reduction is required. The latter is the subject of many of the recent studies in the field. In this article, we contributed to the efforts for developing an accurate estimator for the GGM interaction matrix based on the thermodynamic description of out of equilibrium systems of molecules.

A novel feature of our approach is that we provide an analytic formula for the interaction matrix which increases the computation speed. Another novelty of our method is that we start with the basic principles in thermodynamics, and everything else, including the gaussianity, is derived without our intervention. For example, it is often recommended to center and scale datasets before analyzing them. In our method, these happen naturally because the probability function is determined by the difference of entropy from its maximal value, which subsequently leads to centered expressions In Equation 6. Moreover, as can be seen from the same equation, the expressions are naturally scaled by being divided by the corresponding maximal entropy state expressions, which come from the second derivative of entropy.

One of the advantages of our method is that it can represent non-Gaussian datasets as well. We have shown that the distribution of a dataset is Gaussian, i.e. the GGM representation is valid, only if no sample in the dataset is far from the maximal entropy state such that we can keep the leading term in the Taylor expansion of entropy around its maximal state. However, there can exist datasets in which some samples fall far from the maximal entropy state. In such cases, GGM is not an accurate description of the data. Nevertheless, the distribution function in Equation 4 is still valid. The difference is that we now need to use it in its exact form of

$$P\left(X_1\cdots X_n\right)=\text{exp}\hspace{0.17em}\left(\sum _i\left[(X_i-{\tilde{X}}_i)-X_i\ln \left(\frac{X_i}{{\tilde{X}}_i}\right)\right]\right),$$ (22)

where we have used $S_i=\ln {\tilde{X}}_i$ as is derived in Supplementary Text. Assuming that the constraints of Equation 8 are still valid, $n_{\alpha }$ of the molecular expressions in this distribution function are dependent and should be written in terms of the rest of the molecular expressions in order to arrive at the distribution function in the true physical space. Therefore, we need to split the sum in the exponential in the same way as in Equation 10 and substitute $Z_{\alpha }$ using Equation 12. After working out the tedious but straightforward calculations, the most general distribution function will be given in terms of $G_{i\alpha }$ and ${\tilde{X}}_i$. The prescription to estimate the latter two will stay the same.

It should be mentioned that in our method, the dimension of the reduced space is set by the number of constraints that are identified through data, but independent of how the constraints are learned. When the number of samples is low compared with the number of features, which is the case in most of the omics datasets, the number of constraints between the molecules is expected to exceed the number of samples. In this case, the singular value decomposition should be a reliable method to identify as many significant constraints as the number of samples. Alternatives to the singular decomposition method to infer the constraints are presented in the SA literature (Agmon et al., 1979).

The success of our method’s estimation of the interaction matrix is conditional on observing (i) the near-equilibrium condition as in Figures 2, and (ii) the constraints among the features of the dataset as in Figures 4. Provided that the two conditions are met, this method can estimate the interaction matrix of any dataset that reports the measurements of some features in several samples. One interesting study that is left for the future is to construct the drug-target and drug-drug GGMs using for example data portals such as DrugTargetCommons (Tang et al., 2018) and DrugComb (Zagidullin et al., 2019).

Finally, in this article, using a robust simulation where the true value is known analytically, we show that our method has a more accurate estimation of the interaction matrix than Ledoit-Wolf, OAS and MaxEnt. Moreover, our method is faster than the rest of the methods because the interaction matrix is given in terms of a known analytic formula. On the other hand, the rest of the methods lack such analytic formulation and need to search for the components of the interaction matrix through the phase-space.