Improving accuracy of protein contact prediction using balanced network deconvolution

Abstract

Residue contact map is essential for protein three-dimensional structure determination. But most of the current contact prediction methods based on residue co-evolution suffer from high false-positives as introduced by indirect and transitive contacts (i.e., residues A–B and B–C are in contact, but A–C are not). Built on the work by Feizi et al. (Nat Biotechnol 2013; 31:726–733), which demonstrated a general network model to distinguish direct dependencies by network deconvolution, this study presents a new balanced network deconvolution (BND) algorithm to identify optimized dependency matrix without limit on the eigenvalue range in the applied network systems. The algorithm was used to filter contact predictions of five widely used co-evolution methods. On the test of proteins from three benchmark datasets of the 9th critical assessment of protein structure prediction (CASP9), CASP10, and PSICOV (precise structural contact prediction using sparse inverse covariance estimation) database experiments, the BND can improve the medium- and long-range contact predictions at the L/5 cutoff by 55.59% and 47.68%, respectively, without additional central processing unit cost. The improvement is statistically significant, with a P-value < 5.93 × 10⁻³ in the Student’s t-test. A further comparison with the ab initio structure predictions in CASPs showed that the usefulness of the current co-evolution-based contact prediction to the three-dimensional structure modeling relies on the number of homologous sequences existing in the sequence databases. BND can be used as a general contact refinement method, which is freely available at: http://www.csbio.sjtu.edu.cn/bioinf/BND/.

INTRODUCTION

The three-dimensional (3D) structure of proteins is often represented by a two-dimensional residue contact map matrix, where the nodes represent the protein residues and the edges are used to represent the spatial relationship between residues. The contact map contains important constraints for determining protein structures.^1–8 Typically, when the spatial distance of two residues is close enough, for example, 8 Å, its corresponding entry in the contact map matrix is set to 1, or otherwise 0. Because wet-lab experiments are extremely time-consuming and expensive, specifically designed automated computational methods have been widely used to predict the protein residue contact map. For instance, based on the hypothesis that the contacted residues will co-mutate,^9–15 the mutual information (MI)-based approach¹⁶ and its variant, mutual information without the influence of phylogeny or entropy (MIp),¹⁷ have been widely used.

Although much progress has been obtained on developing computational algorithms for predicting residue contacts, the prediction quality is usually low. For instance, one of the observations is that the contact network from predictors is noisy and inaccurate because of the data being contaminated by the variable indirect relationship.^18,19 The transitive effects of correlations are considered as a main source of indirect contacts.¹⁸ If there are true contacts between sites AB and BC, it will result in observed false correlations between AC with a high probability. The network data quality will significantly affect the resolution of the modeled 3D structure; thus, separating the direct contact from indirect contact is an essential but tough task, especially with a shortage of prior knowledge.

Many groups have developed different approaches to find direct information flows from the initially observed contact data by filtering the transitive noise. For example, the precise structural contact prediction using sparse inverse covariance estimation (PSICOV),¹⁹ which uses an inversed partial correlation matrix, the direct-coupling analysis (DCA),²⁰ which employs a Potts model, and Gaussian DCA (gDCA),²¹ which is a variant of DCA, are three outstanding methods in separating direct dependencies from indirect dependencies in the residue contact map. Recently, the network deconvolution (ND)¹⁸ approach formulates the problem of removing transitive relationships as the inverse of network convolution. Its hypothesis is that the observed data are the sum of both direct and indirect contacts, which can be reflected by summing all direct matrix powers. Compared with other existing approaches, ND is a cutting-edge method in terms of its efficiency and general application capability.

In the ND algorithm, the noise model is represented by the sum of all direct matrix powers. The result is that the eigenvalues of the initially observed (predicted) matrix must be scaled to make the eigenvalues of the direct matrix meet the convergence requirement of the infinite series. The reason is that the items of even powers in the infinite series will make all transformed eigenvalues positive, making the eigenvalues (λ_obs) of the observed matrix G_obs in a narrow range (λ_obs > −0.5). Consequently, in order to cope with various potential λ_obs in the real observed matrix, a network-dependent parameter has to be used to enlarge the range of eigenvalues of the observed contact maps in the ND algorithm.

To solve this problem, in this article, a new balanced network deconvolution (BND) algorithm was proposed to remove transitive relationships. The BND will keep the balanced distribution of eigenvalues, endowing it with robustness without needing an optimized scaling parameter. Instead of representing the observed matrix data as the sum of direct matrix and all its powers, only the direct matrix and its odd powers are considered as the noise model. By doing this, we can infer ∀λ_obs ∈ R from λ_dir in the range of |λ_dir| < 1. In this case, there is, therefore, no further need of scaling parameters in BND.

Experimental results on the benchmark 9th critical assessment of protein structure prediction (CASP9), CASP10, and PSICOV datasets demonstrate BND as a much more accurate filter than the ND approach. The best merit of the BND for protein residue contact map prediction is that it is a parameter-free postprocessing step and independent of the prediction algorithms, which can therefore generate an additional benefit at no cost of changing the prediction algorithms.

DATASETS AND METHODS

Benchmark datasets

Three benchmark datasets were used to evaluate the proposed BND algorithm. The first one is the set of 147 proteins in the CASP9 competition, the second is the set of 121 proteins in the CASP10 competition, and the third is the set of 150 proteins used in PSICOV.¹⁹ Contacts are defined as those pairs of residues where the spatial distance of their C-βatoms (C-α in the case of glycine) is <8 Å.²²

Initial observed contact matrix

Given a query protein sequence of L residues, five sequence-based residue–residue contact predictors were used to generate the initial observed contact matrix G_obs, which is a symmetry matrix of L×L with the G_obs(i, j) entries, indicating the contact probability between the i-th and the j-th residues. The five predictors are MI,¹⁶ MIp,¹⁷ DCA,²⁰ its recent variant gDCA,²¹ and PSICOV.¹⁹ Here, we give their brief introductions separately.

The idea of MI-based residue contact prediction algorithm is to estimate the co-evolution probability though the MI statistic between two residues calculated from the multiple sequence alignment (MSA), which is generated by searching the query sequence against a large sequence database, for example, searching the query sequence using PSI-BLAST program against the NR (nonredundant) database. The MI between positions S_a and S_b along the sequence can be calculated as:

$$\mathrm{MI}({\mathrm{S}}_{\mathrm{a}},{\mathrm{S}}_{\mathrm{b}})=\mathrm{H}\left({\mathrm{S}}_{\mathrm{a}}\right)+\mathrm{H}\left({\mathrm{S}}_{\mathrm{b}}\right)-\mathrm{H}({\mathrm{S}}_{\mathrm{a}},{\mathrm{S}}_{\mathrm{b}})$$ (1)

where, H(S_a) and H(S_b) are the entropy of positions S_a and S_b derived from the MSA, respectively:

$$\{\begin{array}{cc}\mathrm{H}\left({\mathrm{S}}_{\mathrm{a}}\right)\hfill & =-{\sum }_{x}p(x,{\mathrm{S}}_{\mathrm{a}}){\log }_{10}p(x,{\mathrm{S}}_{\mathrm{a}})\hfill \\ \mathrm{H}\left({\mathrm{S}}_{\mathrm{b}}\right)\hfill & =-{\sum }_{x}p(x,{\mathrm{S}}_{\mathrm{b}}){\log }_{10}p(x,{\mathrm{S}}_{\mathrm{b}})\hfill \end{array}\phantom{\}}$$ (2)

where, p(x, S_a) and p(x, S_b) represent the frequencies of occurrence of amino acid x at columns S_a and S_b in the MSA. H(S_a, S_b) in Eq. (1) is the joint entropy of columns S_a and S_b.

An improved version of MI-based algorithm called MIp was proposed on the consideration of reduction of the influence of phylogeny or entropy.¹⁷ The MIp uses Eq. (3) to estimate the background MI shared by positions S_a and S_b:

$$\mathrm{B}({\mathrm{S}}_{\mathrm{b}},{\mathrm{S}}_{\mathrm{b}})=\frac{\mathrm{MI}({\mathrm{S}}_{\mathrm{a}},\cdot )\mathrm{MI}({\mathrm{S}}_{\mathrm{b}},\cdot )}{\overline{\mathrm{MI}}}$$ (3)

where, MI(S_a, ·) and MI(S_b, ·) represent the mean MI of columns S_a and S_b, respectively. MI(S_a, ·) is defined by $\frac{1}{M-1}{\sum }_{\begin{array}{c}\hfill i=1:M\hfill \\ \hfill i\ne a\hfill \end{array}}\mathrm{MI}({\mathrm{S}}_{\mathrm{a}},{\mathrm{S}}_i)$, where M is the number of columns in the MSA. Similarly, $\overline{\mathrm{MI}}$ represents the overall mean MI in the MSA. Then the MIp of S_a and S_b is:

$$\mathrm{MIp}({\mathrm{S}}_{\mathrm{a}},{\mathrm{S}}_{\mathrm{b}})=\mathrm{MI}\left({\mathrm{S}}_{\mathrm{a}}{\mathrm{S}}_{\mathrm{b}}\right)-\mathrm{B}\left({\mathrm{S}}_{\mathrm{a}}{\mathrm{S}}_{\mathrm{b}}\right)$$ (4)

DCA²⁰ and PSICOV¹⁹ are two methods proposed recently trying to remove the transitive contact effects. PSICOV attempts to correct indirect coupling effect by calculating partial correlation coefficient, whereas DCA tries to correct indirect coupling effect by maximizing entropy using the Potts model. For a further development on DCA, gDCA²¹ used a multivariate Gaussian modeling approach to replace discrete amino acid states with continuous Gaussian random variables.

Given a query protein sequence, the above five predictors, that is, MI, MIp, DCA, gDCA, and PSICOV, will be used to generate the initial residue contact maps G_obs, and the aim of this study was to filter the transitive noises from the G_obs to get the true contact matrix G_dir.

Network deconvolution

The ND model¹⁸ formulates the transitive closure of a network as an infinite sum of true direct network, which can be written in a closed infinite-series sum. The relationship between the true network matrix (G_dir) and the observed network matrix (G_obs) in the ND algorithm is shown in Eq. (5):

$${\mathbf{G}}_{obs}={\mathbf{G}}_{dir}+{\mathbf{G}}_{\mathrm{dir}}^2+{\mathbf{G}}_{\mathrm{dir}}^3+\cdots +{\mathbf{G}}_{\mathrm{dir}}^{\mathrm{n}}$$ (5)

where, n → ∞. The infinite series of Eq. (5) can be written as the following closed form¹⁸:

$${\mathbf{G}}_{\mathrm{obs}}={\mathbf{G}}_{\mathrm{dir}}{(\mathrm{I}-{\mathbf{G}}_{\mathrm{dir}})}^{-1}$$ (6)

where, I is the identity matrix. From Eq. (5), the G_dir can be calculated as:

$$\{\begin{array}{cc}{\mathbf{G}}_{\mathrm{dir}}\hfill & ={\mathbf{G}}_{\mathrm{obs}}{(\mathrm{I}+{\mathbf{G}}_{\mathrm{obs}})}^{-1}\hfill \\ {\lambda }_{\mathrm{obs}}\hfill & =\frac{{\lambda }_{\mathrm{dir}}}{1-{\lambda }_{\mathrm{dir}}}\hfill \\ {\lambda }_{\mathrm{dir}}\hfill & =\frac{{\lambda }_{\mathrm{obs}}}{1-{\lambda }_{\mathrm{obs}}}\hfill \end{array}\phantom{\}}$$ (7)

where, λ_obs is the eigenvalue of the observed network, and λ_dir the eigenvalue of the true network. From Eq. (7), when the observed matrixG_obs is given, the G_dir can be reconstructed. After eigenvalue decomposition, relationship between λ_dir and λ_obs is also obtained (refer to the Balanced Network Deconvolution section for details). The transformation of eigenvalues is the core of the transformation of matrices in ND.

Actually, to guarantee the infinite series of Eq. (5) converge, the λ_dir must meet the condition that |λ_dir| < 1. However, to meet this condition, all the eigenvalues of the G_obs must have λ_obs > −0.5 [Fig. 1(a)], according to ${\lambda }_{\mathrm{obs}}=\frac{{\lambda }_{\mathrm{dir}}}{1-{\lambda }_{\mathrm{dir}}}$ in Eq. (7) as illustrated in Figure 1(a), which obviously cannot be satisfied in the real-world applications, because in actuality the eigenvalue of the G_obs can be any value in the range of (−∞, +∞).

Figure 1.

(a) Plot of BND model: ${\lambda }_{\mathrm{obs}}=\frac{{\lambda }_{dir}}{1-{\left({\lambda }_{\mathrm{dir}}\right)}^2}$ and ND models: ${\lambda }_{\mathrm{obs}}=\frac{{\lambda }_{\mathrm{dir}}}{1-{\lambda }_{\mathrm{dir}}}$ and ${\lambda }_{\mathrm{obs}}=\frac{{\lambda }_{\mathrm{dir}}}{\alpha (1-{\lambda }_{\mathrm{dir}})}(\alpha =0.1)$. (b) Two solutions of Eq. (16): Solution 1: ${\lambda }_{\mathrm{dir}}=\frac{-1-\sqrt{1+4{\left({\lambda }_{\mathrm{obs}}\right)}^2}}{2{\lambda }_{\mathrm{obs}}}$, and Solution 2:${\lambda }_{\mathrm{dir}}=\frac{-1+\sqrt{1+4{\left({\lambda }_{\mathrm{obs}}\right)}^2}}{2{\lambda }_{\mathrm{obs}}}$. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.

To solve this problem, in the original ND model, a tuning parameter was introduced to linearly scale the λ_obs to cover more regions [Fig. 1(a)]:

$$\{\begin{array}{cc}{\lambda }_{\mathrm{dir}}=\frac{\alpha {\lambda }_{\mathrm{obs}}}{1+\alpha {\lambda }_{\mathrm{obs}}}\hfill & \hfill \\ \alpha \le \min (\frac{\beta }{(1-\beta ){\lambda }_{\mathrm{obs}}^{+\left(\max \right)}},\frac{-\beta }{(1+\beta ){\lambda }_{\mathrm{obs}}^{-\left(\min \right)}})\hfill & \hfill \end{array}\phantom{\}}$$ (8)

where, ${\lambda }_{\mathrm{obs}}^{+\left(\max \right)}$ and ${\lambda }_{\mathrm{obs}}^{-\left(\min \right)}$ are the biggest positive and the smallest negative eigenvalues of Gobs. The β parameter is network dependent. Different values have been suggested on different input networks in the ND approach. For instance, on the protein contact map network, a recommended β value was 0.99, whereas 0.5 was suggested on the gene regulator network and 0.95 was suggested on the social network.

Balanced network deconvolution

Why the ND approach needs an additional parameter to scale λ_obs? If we inspect the ND noise model [Eq. (5)] carefully, it is clear that it contains the even and odd powers of the direct matrix. The odd powers will maintain the plus or minus signs of the eigenvalues, whereas the even powers will make all eigenvalues positive. This phenomenon will make the eigenvalue distributions imbalanced. As we will show later, if we remove the even powers in the infiniteseries, the eigenvalues of the rebuilt noise matrix will be closer to the eigenvalues of observed matrix. In return, the solved true network will be more precise. So, the noise model of our proposed BND algorithm is as follows:

$${\mathbf{G}}_{\mathrm{obs}}={\mathbf{G}}_{\mathrm{dir}}+{\mathbf{G}}_{\mathrm{dir}}^3+{\mathbf{G}}_{\mathrm{dir}}^5+\cdots +{\mathbf{G}}_{\mathrm{dir}}^{2n-1}$$ (9)

where, n → ∞. The infinite series of Eq. (9) can be written as the following closed form:

$${\mathbf{G}}_{\mathrm{obs}}={\mathbf{G}}_{\mathrm{dir}}{(\mathrm{I}-{\mathbf{G}}_{\mathrm{dir}}^2)}^{-1}$$ (10)

By solving the quadratic equation of Eq. (10) for G_dir, we get:

$${\mathbf{G}}_{\mathrm{dir}}=(-\mathrm{I}+\sqrt{\mathrm{I}+4{\mathbf{G}}_{\mathrm{obs}}^2})\cdot {\left(2{\mathbf{G}}_{\mathrm{obs}}\right)}^{-1}$$ (11)

Let U and E_dir represent eigenvectors and a diagonal matrix of eigenvalues of G_dir, where ${\lambda }_{\mathrm{dir}}^i$ is the i-th diagonal component of the matrix E_dir. By using the eigen decomposition principle, we get:

$$\begin{array}{cc}\hfill {\mathbf{G}}_{\mathrm{obs}}& ={\mathbf{G}}_{\mathrm{dir}}+{\mathbf{G}}_{\mathrm{indir}}\hfill \\ \hfill & \stackrel{\left(\mathrm{a}\right)}{=}{\mathbf{G}}_{\mathrm{dir}}+{\mathbf{G}}_{\mathrm{dir}}^3+{\mathbf{G}}_{\mathrm{dir}}^5+\cdots +{\mathbf{G}}_{\mathrm{dir}}^{2n-1}\hfill \\ \hfill & \stackrel{\left(\mathrm{b}\right)}{=}\left({\mathrm{UE}}_{dir}{\mathrm{U}}^{-1}\right)+\left({\mathrm{UE}}_{\mathrm{dir}}^1{\mathrm{U}}^{-1}\right)+\left({\mathrm{UE}}_{\mathrm{dir}}^5{\mathrm{U}}^{-1}\right)+\cdots +\left({\mathrm{UE}}_{\mathrm{dir}}^{2n-1}{\mathrm{U}}^{-1}\right)\hfill \\ \hfill & =\mathrm{U}({\mathrm{E}}_{\mathrm{dir}}+{\mathrm{E}}_{\mathrm{dir}}^3+{\mathrm{E}}_{\mathrm{dir}}^5+\cdots +{\mathrm{E}}_{\mathrm{dir}}^{2n-1}){\mathrm{U}}^{-1}\hfill \\ \hfill & =\mathrm{U}\left(\begin{array}{ccc}\hfill \sum _{i=1}^n{\left({\lambda }_{\mathrm{dir}}^1\right)}^{2i-1}\hfill & \hfill \cdots \hfill & \hfill 0\hfill \\ \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill 0\hfill & \hfill \cdots \hfill & \hfill \sum _{i=1}^n{\left({\lambda }_{\mathrm{dir}}^{\mathrm{L}}\right)}^{2i-1}\hfill \end{array}\right)\hfill \\ \hfill & \stackrel{\left(\mathrm{a}\right)}{=}\mathrm{U}\left(\begin{array}{ccc}\hfill \frac{{\lambda }_{\mathrm{dir}}^1}{1-{\left({\lambda }_{\mathrm{dir}}^1\right)}^2}\hfill & \hfill \cdots \hfill & \hfill 0\hfill \\ \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill 0\hfill & \hfill \cdots \hfill & \hfill \frac{{\lambda }_{\mathrm{dir}}^{\mathrm{L}}}{1-{\left({\lambda }_{\mathrm{dir}}^{\mathrm{L}}\right)}^2}\hfill \end{array}\right)\hfill \end{array}$$ (12)

where, (a) is inferred from the definition of the diffusion model of Eq. (9); (b) follows from the eigen decomposition of matrix G_dir; and (c) makes use of the character of geometric series to compute the infinite summation on the assumption of |λ_dir| < 1 similar to the assumption proposed in ND.

Actually, from Eq. (11), if we use G_obs = UE_obsU⁻¹ to represent the eigenvalue decomposition of G_obs, where

$${\mathrm{E}}_{\mathrm{obs}}=\left(\begin{array}{ccc}\hfill {\lambda }_{\mathrm{obs}}^1\hfill & \hfill \cdots \hfill & \hfill 0\hfill \\ \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill 0\hfill & \hfill \cdots \hfill & \hfill {\lambda }_{\mathrm{obs}}^L\hfill \end{array}\right)$$ (13)

we get:

$${\lambda }_{\mathrm{obs}}^i=\frac{{\lambda }_{\mathrm{dir}}^i}{1-{\left({\lambda }_{\mathrm{dir}}^i\right)}^2},\forall 1\le i\le L$$ (14)

and

$${\lambda }_{\mathrm{obs}}^i\cdot {\left({\lambda }_{\mathrm{dir}}^i\right)}^2+{\lambda }_{\mathrm{dir}}^i-{\lambda }_{\mathrm{obs}}^i=0$$ (15)

When $\mid {\lambda }_{\mathrm{dir}}^i\mid$ < 1 for al i, we can derive any ${\lambda }_{\mathrm{obs}}^i\in (-\infty +\infty )$ from Eq. (14) as illustrated in Figure 1(a). This means that Eq. (14) can suit all the observed matrix G_obs in the real-world applications. So, unlike the ND model, the proposed BND model does not need an additional tuning parameter to linearly scale λ_obs for meeting the assumption of $\mid {\lambda }_{\mathrm{dir}}^i\mid$ < 1 anymore.

By solving the quadratic Eq. (15) for ${\lambda }_{\mathrm{dir}}^i$ we will get two possible solutions:

$$\{\begin{array}{cc}{\lambda }_{\mathrm{dir}}^i\hfill & =\frac{-1+\sqrt{1+4{\left({\lambda }_{\mathrm{obs}}^i\right)}^2}}{2{\lambda }_{\mathrm{obs}}^i}\hfill \\ {\lambda }_{\mathrm{dir}}^i\hfill & =\frac{-1-\sqrt{1+4{\left({\lambda }_{\mathrm{obs}}^i\right)}^2}}{2{\lambda }_{\mathrm{obs}}^i}\hfill \end{array}\phantom{\}}$$ (16)

As shown in Figure 1(b), the solution of ${\lambda }_{\mathrm{dir}}^i=\frac{-1+\sqrt{1+4{\left({\lambda }_{\mathrm{obs}}^i\right)}^2}}{2{\lambda }_{\mathrm{obs}}^i}$ satisfies the condition of $\forall {\lambda }_{\mathrm{obs}}^i\in (-\infty ,+\infty ),\mid {\lambda }_{\mathrm{dir}}^i\mid$ < 1 which is finally used in the BND model.

RESULTS

Comparison of ND and BND models

Figure 2(a) illustrates the difference between the ND and BND noise models. To quantitatively examine both the difference and similarity of the noise models between ND and the proposed BND, we did two intuitive experiments.

Figure 2.

(a) An illustration showing ND and BND noise models. (b) Eigenvalue distributions for the rebuilt noise matrices with ND and BND noise models. (c) Network topology comparison by applying ND and BND noise models on the same network. (d, e) General mathematic calculation model of BND. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

First, we constructed a 5000 × 5000 symmetric matrix containing random values obeying a standard normal distribution, which is considered as G_dir. The distribution of its eigenvalues obeys Wigner’s semicircle law.²³ Then, the eigenvalues of G_dir were used to rebuild the noise matrix G_obs with the ND and BND noise models separately [Eq. (5) and Eq. (9)]. The difference of eigenvalue distributions of the rebuilt G_obs by the two models is obvious [Fig. 2(b)]: the BND noise model has maintained the balanced eigenvalue distribution, whereas the ND noise model made almost all λ_obs eigenvalues positive.

Second, we simulated an 8 × 8 symmetric matrix containing pseudorandom weight values [see Fig. 2(c) and Supporting Information for details), which is considered as G_dir. New edges were added to the network by applying the ND and BND noise models, respectively, generating new network topologies. Two interesting results were observed: the first was that the two new network topologies derived from ND and BND are very comparable, indicating that the odd powers can cover the information in the even powers; and the second was that compared with the true topology, the BND noise network can keep the strong edge weights better, for example, edge between Nodes 5 and 7 [Fig. 2(c)].

Results of filtering noise from the predicted residue contact maps

There are three steps of conducting the experiments of removing the transitive noises contained in the residue contacted map matrix. First, given a protein sequence, we generated an MSA by the PSI-BLAST search; Second, the MSA was inputted to MI, MIp, DCA, gDCA, and PSICOV for predicting the residue–residue contact matrix (G_obs); Third, the BND algorithm was applied on the predicted G_obs for filtering the transitive noises, where for comparison, the ND approach is also tested in the third step. Note that on 150 proteins in the PSICOV dataset, the MSA provided in the original article is used for fairly comparison.¹⁹ Figure 3 gives a flowchart of the conducted experiments.

Figure 3.

The flow chart of removing transitive noise with BND algorithm. Top L/2 predictions are drew for the T0525 protein in CASP9 as an example to show how to correct the wrongly predicted contacts with BND filter. Green dots are benchmark contacts in the protein; red dots are right predictions; and blue dots are wrong predictions.

In order to measure the performance of the BND-based filtering in protein residue–residue contact prediction precisely, the accuracy of the top-ranked contacts is evaluated. Figure 4 shows the top L/5 results from MI, MIp, DCA, gDCA, PSICOV, ND, and BND in different sequence separation ranges, where L is the length of the query sequence.

Figure 4.

Accuracy of the top L/5 contact predictions by different methods on the three benchmark datasets.

Improvements are observed on all the three benchmark datasets by applying the BND for removing the transitive noises. For all the contacts with the sequence separation > 5 on the top L/5 results, BND improves the prediction accuracy at an average by 161.67%, 134.00%, and 134.58% on the contact map predicted by MI in the three datasets, respectively. Compared with the original MIp method, BND improves its prediction accuracy by 159.04% on CASP9, 127.13% on CASP10, and 134.69% on PSICOV database. For DCA, BND improves the prediction accuracy by 14.73%, 10.37,% and 7.92% on the three datasets; and for its variant gDCA, BND improves the prediction accuracy by 4.43%, 3.13%, and 2.86% on the three datasets. For PSICOV, BND improves the accuracy by 13.01% and 3.69% on the two CASP datasets, but the precision drops a little bit of 0.75% on PSICOV dataset, which was originally applied in the PSICOV article.¹⁹ However, BND outperforms ND with all the evaluations on the PSICOV dataset. For instance, the long-range contacts (residue separation distance larger than 23) prediction accuracy on top L/5 is 60.69% when applying the ND on the original PSICOV output, whereas this number is 62.42% when applying BND on PSICOV’s output (see Supporting Information for details).

Besides, in order to test the robustness of BND, we picked up 39 and 36 hard targets from CASP9 and CASP10 separately, which are more difficult to predict their contacts among all proteins evaluated because little homology sequences can be found for them. The improvement is also observed on the predicted contacts (Fig. 5). For all the contacts with the sequence separation > 5 on the top L/5 results, compared with MI, BND improves the prediction accuracy by 184.53% and 136.77% in CASP9’s hard targets and CASP10’s, respectively. For MIp, BND improves its prediction accuracy by 178.42% on CASP9 and 119.84% on CASP10. For DCA, BND improves the prediction accuracy by 11.44% and 14.86% on the two datasets; and for gDCA, BND improves the accuracy by 7.92% on CASP10, but reduces the accuracy by 4.35% on CASP9; and for PSICOV, BND improves the accuracy by 5.66% and 11.27% (see Supporting Information for details).

Figure 5.

Accuracy of the top L/5 contact predictions by different methods on the hard targets from CASP9 and CASP10 datasets, respectively.

Long-range residue–residue contacts are the most important to protein structure determinations, but they are usually more difficult for predictions as shown in the recent CASP competitions.^15,24,25 In previous studies, there are different definitions of long-range contacts, for example, residue contacts with sequence separation larger than 23, 20, 15, or 8.⁷ BND is also found to be significantly helpful for mining the long-range contacts (Table I). Taken the top L/5 results into account on the four kinds of long-range definitions, on the MI, BND helps to improve the long-range contact prediction accuracy at an average by 159.75% on CASP9, 127.86% on CASP10, and 128.57% on PSICOV dataset. Compared with MIp, BND improves the accuracy by 157.79%, 127.60%, and 128.69% on the three datasets. For DCA, BND helps to improve the accuracy by 7.45%, 4.31%, and 5.82%; for gDCA, BND helps to improve the accuracy by 4.89%, 0.17%, and 2.28%; and for PSICOV, improvements of 6.58% on CASP9 and 1.76% on CASP10, but 0.41% decrease compared with PSICOV on this dataset were observed.

Table I.

The improvements of four different long range contact definitions achieved in the top L/5 predictions by applying BND on five prediction algorithms.

Dataset	Method	|i-j|>23	|i-j|>20	|i-j|>15	|i-j|>8
CASP9	MI+BND	156.71%	160.92%	158.30%	162.63%
	MIp+BND	151.36%	162.53%	157.95%	158.99%
	DCA+BND	5.63%	5.84%	7.06%	10.95%
	PSICOV+BND	4.75%	4.96%	6.55%	9.58%
	gDCA+BND	5.06%	4.21%	5.11%	5.16%
CASP10	MI+BND	122.62%	110.59%	145.07%	133.89%
	MIp+BND	124.18%	113.50%	142.97%	130.52%
	DCA+BND	3.33%	4.44%	4.12%	5.18%
	PSICOV+BND	1.76%	0.91%	3.24%	1.15%
	gDCA+BND	−1.11%	0.04%	0.22%	1.33%
PSICOV	MI+BND	125.60%	128.86%	129.01%	130.53%
	MIp+BND	125.84%	129.13%	129.42%	130.13%
	DCA+BND	5.73%	5.76%	5.72%	6.04%
	PSICOV+BND	−0.22%	−0.69%	−0.29%	−0.42%
	gDCA+BND	2.60%	2.02%	1.67%	2.81%

Not only for the top L/5 rankings, the enhancement by BND is also found generally in top L/10, L/2, and L rankings. Tables II, III, and IV show the long-range residue–residue (residue contacts with sequence separation larger than 23) contact prediction results of different rankings on CASP9, CASP10, and PSICOV datasets, respectively. In all the tables, we can find that BND out-performs ND on all the experiments. Taking the CASP9 dataset for an example, with the same G_obs matrix generated by MI approach, where its accuracy for predicting the top L/5 contacts with the sequence separation > 5 is 10.32%, BND will improve the accuracy to 26.99%, whereas ND will only improve it to 21.95%.

Table II.

Long-range residue contact (residue separation>23) prediction accuracies on CASP 9 dataset by applying the BND to filtering transitive noises on five prediction algorithms.

Method	Top L/10	Top L/5	Top L/2	Top L
MI	10 39%	8 41%	6 12%	5 16%
MI+ND	21.60%	17.73%	12.95%	10.05%
MI+BND	26.29%	21.58%	15.49%	11.86%
MIp	10.39%	8.46%	6.16%	5.19%
MIp+ND	21.60%	18.13%	13.19%	10.25%
MIp+BND	26.29%	21.26%	15.81%	12.38%
DCA	29.88%	24.93%	17.69%	12.71%
DCA+ND	29.86%	25.59%	17.82%	13.24%
DCA+BND	31.34%	26.34%	18.87%	14.00%
PSICOV	31.94%	25.02%	17.53%	13.18%
PSICOV+ND	31.57%	24.47%	16.35%	12.21%
PSICOV+BND	33.70%	26.21%	17.38%	12.94%
gDCA	34.29%	27.66%	20.71%	15.23%
gDCA+ND	35.32%	28.57%	20.02%	14.89%
gDCA+BND	35.87%	29.06%	20.10%	14.68%

Table III.

Long-range residue contact (residue separation>23) prediction accuracies on CASP 10 dataset by applying the BND to filtering transitive noises on five prediction algorithms.

Method	Top L/10	Top L/5	Top L/2	Top L
MI	15.82%	13.82%	9.97%	7.71%
MI+ND	30.63%	25.45%	18.89%	14.15%
MI+BND	36.78%	30.77%	22.63%	16.69%
MIp	15.82%	13.82%	10.02%	7.69%
MIp+ND	30.81%	25.77%	19.24%	14.32%
MIp+BND	36.53%	30.99%	22.88%	16.98%
DCA	40.74%	34.48%	25.94%	19.13%
DCA+ND	38.98%	34.24%	25.96%	18.79%
DCA+BND	41.42%	35.62%	26.80%	19.51%
PSICOV	41.45%	35.87%	26.72%	18.90%
PSICOV+ND	39.89%	34.75%	24.81%	17.14%
PSICOV+BND	41.55%	36.50%	25.99%	17.94%
gDCA	45.22%	40.11%	30.36%	22.25%
gDCA+ND	42.78%	38.26%	27.57%	19.16%
gDCA+BND	45.13%	39.67%	29.35%	20.65%

Table IV.

Long-range residue contact (residue separation>23) prediction accuracies on PSICOV dataset by applying the BND to filter transitive noises on five prediction algorithms.

Method	Top L/10	Top L/5	Top L/2	Top L
MI	29.12%	23.15%	16.33%	12.42%
MI+ND	55.04%	46.89%	33.58%	24.18%
MI+BND	60.76%	52.23%	38.61%	27.99%
MIp	29.12%	23.15%	16.33%	12.42%
MIp+ND	55.15%	46.96%	33.86%	24.35%
MIp+BND	60.20%	52.29%	38.73%	28.11%
DCA	54.92%	50.55%	40.00%	29.18%
DCA+ND	56.06%	51.42%	40.81%	29.58%
DCA+BND	58.03%	53.44%	42.39%	30.72%
PSICOV	71.77%	62.56%	44.74%	31.48%
PSICOV+ND	70.53%	60.69%	42.10%	28.60%
PSICOV+BND	71.22%	62.42%	44.05%	29.91%
gDCA	72.86%	64.70%	50.61%	36.90%
gDCA+ND	72.86%	64.31%	47.27%	32.24%
gDCA+BND	73.62%	66.38%	50.39%	35.19%

In order to reveal the reason for the better performance of BND, we looked closely at the eigenvalue distributions on the CASP9 dataset, where Figure 6(a) shows the distribution of the real contact map G_dir. In the G_obs of the predicted contact map by MI, its λ_obs ∈ [−1.94, 117.97] [Fig. 6(b)], which violates the required range of ND (bigger than −0.5) as discussed in Eq. (7). So, ND applied a parameter β = 0.99 [α=0.256 as calculated from Eq. (8) to adjust the λ_obs, which will be rescaled to ${\lambda }_{\mathrm{obs}}^{\mathrm{ND}}$ ∈ [−0.497; 30.20]. Then, according to Eq. (7), the ND will infer the eigenvalues in the direct matrix as: ${\lambda }_{\mathrm{dir}}^{\mathrm{ND}}$ ∈ [−0.9870, 0.9679]. By contrast, BND does not need a scaling parameter but uses Eq. (16) directly to infer the eigenvalues of the direct matrix as: ${\lambda }_{\mathrm{obs}}^{\mathrm{BND}}$ ∈ [−0.7750, 0.9958] from λ_obs 2 [−1.94; 117.97]. BND keeps the absolute value of minimum transformed eigenvalues λ_dir smaller than that of maximum transformed eigenvalues λ_dir, similar to the range of the input eigenvalues λ_obs. And this is also in consistence with λ ∈ [−0.9874; 297.82] derived from the benchmark true contact matrices. The eigenvalue distribution of BND plus MI-predicted matrices has a sharper peak and concentrates on the area where eigenvalues are around minimum, which is more similar to that of the distribution shape of true contact map than that of ND plus MI and MI-predicted ones (Fig. 6). These results indicate that the reconstructed direct matrix by BND is much closer to the benchmark true contact matrix, and this is the reason that BND performs better than ND.

Figure 6.

The eigenvalue distribution of contact matrices from true contact matrix (a), MI-predicted (b), reconstructed direct matrix by BND (c), and reconstructed direct matrix by ND (d) on the CASP9 dataset.

According to our experimental results, the improvements obtained by the BND on MI, MIp, DCA, gDCA, and PSICOV are different. Why does the BND model work better for MI and MIp than DCA, gDCA, and PSI-COV? The reason is that DCA, gDCA, and PSICOV have already contained specifically designed modules for removing the transitive noise, and hence their outputted contacted matrix have filtered part of the transitive noises. Even in such cases, the BND is found helpful for enhancing the prediction power on DCA, gDCA, and PSICOV, and this probably indicates that there are still some levels of transitive contact noises remaining in the prediction, which have been further filtered out by the BND and ND methods. For example, DCA will predict the long-range contacts on the top L/5 in three benchmark datasets with an accuracy of 40.13%, but will be improved to 42.21% by applying BND as a postprocess. These results suggest that the noise model of the predicted contact map is complicated and can be a mixture of several different types, in which case, single filter is not enough.

It is also interesting to observe from the experiments that the improvement by applying BND on PSICOV algorithm on different datasets is also different. For example, the improvement on the CASP9 and CASP10 datasets is higher than that on the 150 proteins in PSI-COV dataset. The MSA quality may be the reason. The average size of MSA in PSICOV dataset is approximately 6245, which is rather larger than that in the other two datasets, that is, 991 in CASP9 and 3794 in CASP10. This feature reveals that the more sufficient co-evolution information, the higher prediction accuracy, with less amount of transitive noise. A detailed comparison by applying the BND filter on the PSICOV outputs for CASP datasets and PSICOV dataset is shown in Figure 7. As it clearly shows, BND helps to improve PSICOV better on the CASP datasets, where the MSA sizes are small.

Figure 7.

Comparison of the performance of BND on PSICOV in different benchmark datasets on top L/5 long-range contact (residue sequence separation > 23) predictions. Proteins are listed according to their predicted precision in a descending order.

Comparison of BND-enhanced methods with ab initio structure predictions

Apart from sequence-based contact predictions, residue–residue contact maps can also be derived from protein structure prediction methods. It is of interest to compare the performance on the two methods. In an early study,³ it was concluded that the relative performance of the two methods depends on the availability of homologous templates in the PDB because structure predictions with homologous templates have a much higher accuracy than the sequence-based ab initio predictions. Here, we focus our comparison on the new fold (NF) targets in the CASP9 and CASP10 experiments,^26,27 which have been verified not having homologous templates, where sequence-based contact predictions are most promising in helping the tertiary structure construction. As a control, we chose models from three successful free modeling methods in the CASP experiments, including QUARK,²⁸ Zhang-server,^29–31 and Mufold.³² These methods also represent three different approaches to the ab initio folding, where QUARK constructs models by assembling continuously distributed fragments, Zhang-Server ab initio models were built based on QUARK models followed by iterative assembly refinements, and Mufold builds models by multidimensional scaling restraints followed by molecular dynamics refinement.

Five NF targets were selected from the CASP9 (including T0534-D2, T0537-D1, T0537-D2, T0544-D1, and T0531-D1), and eight NF targets selected from CASP10 (including T0658-D1, T0684-D2, T0693-D1, T0719-D6, T0726-D3, T0735-D2, T0737-D1, and T0740-D1). One reason for selecting these targets is that there is no deletion in final PDB structure determination, which facilitates the comparisons because both sequence-based and structure-based contact predictions are based on the same full-length sequences. The predicted 3D models were downloaded from the CASP website (http://www.predictioncenter.org).

As shown in Figure 8, the relative accuracy by the two methods is contradictory in the CASP9 and CASP10 datasets. When considering the long-distance contacts, for example, the accuracy of the BND-enhanced co-evolution predictions is considerably higher than that from the ab initio structure predictions on the CASP9 proteins. But, the performance of the sequence-based contact predictions is much worse on the CASP10 dataset. Although the average accuracy from ab initio structure predictions in CASP9 (15.86%) is quite close to that in CASP10 (15.76%) (i.e., reduced only by 0.66%), the reduction of the BND-enhanced co-evolution predictions from CASP9 to CASP10 is marked, that is, reduced from 24.05% to 10.83% by 54.97%.

Figure 8.

Comparison of the BND-enhanced co-evolution methods with three state-of-the-art ab initio 3D structure prediction methods on the long-range contact predictions. Only the precisions of the contacts with a sequence separation > 23 are presented.

Why is the performance from the sequence-based contact predictions so different on the two CASP datasets? As discussed in the Results of Filtering Noise from the Predicted Residue Contact Maps section, examining the number of homologous sequences in the MSAs may help provide an answer to the question because all the co-evolution-based methods use the MSAs to derive the contact information. To get accurate prediction, a high number of homologous sequences are often needed. In fact, the average MSA size in evaluated proteins from CASP10 is 115, where in four cases the number of homologous sequences is less than or equal to 5. On the contrary, the average MSA size in evaluated proteins from CASP9 is 260, and there is only one protein whose number of homologous sequences is less than or equal to 5. Therefore, the contact predictions for the CASP9 dataset by the five co-evolution methods (MI, MIP, DCA, gDCA, and PSICOV) are more accurate than those on the CASP10 dataset, which resulted in the different performance of the BND predictions. These data suggests that an essential condition for the current co-evolution contact predictions to be helpful to the 3D structure prediction is that sufficient number of homologous sequences should exist in the sequence databases, so that a set of accurate contact predictions can be made beyond that from the traditional ab initio structure prediction approaches.

DISCUSSION

BND filtering does not work for machine-learning-based contact predictions

The BND method is designed for filtering the false-positive contacts introduced by the indirect and transitive contacts, which most often occur on the co-evolution contact predictions because the predictions of these approaches are derived based on correlation of evolutionary mutations. Apart from the co-evolution-based approaches, there are another type of approaches that derive the residue contacts based on machine learning, for example, the support vector machine-based SVMSEQ³ and SVMcon,³³ and neural network-based NNcon.³⁴ Because these approaches trained the contacts of residue pairs independently, it is expected that the transitive contact effect from these approaches should be minimum. As a test, we applied the SVMSEQ, SVMcon, and NNcon methods on the same set of test proteins from CASP9 and CASP10 experiments, followed by the BND filtering. Our results indeed showed that the filtering procedure did not help to improve the prediction accuracies for the tested machine learning-based predictors. On the contrary, the accuracy even dropped a little bit for both ND and BND cases (data not shown).

Generality of BND for gene and social network construction

As the proposed BND algorithm is a data-driven approach, which filters the transitive contact noise by analyzing the eigenvalue characteristics of the input matrix, it is expected to be a general algorithm that can also be directly applied in different domains. To examine the generality of BND, we conducted an experiment of reconstructing two real-world Escherichia coli and Saccharomyces cerevisiae gene regulatory networks, in comparison with the ND model.¹⁸ In this experiment, the inputs to ND and BND are from 10 popular prediction methods. The final results from the ND and BND were compared with the experimentally verified data, with the evaluations based on the area under the precision-recall and receiver operating characteristic curves.³⁵ Our results showed that the BND achieves 1% and 25% higher average scores on the average of the 10 methods on the E. coli and S. cerevisiae networks than the ND approach, respectively.

The other experiment we did to demonstrate the generalization ability of BND is on a social co-authorship network. The network consists of 1589 nodes,³⁶ which were constructed strictly and of high data quality. The edges weights equal or bigger than 0.5 are considered strong ties,¹⁸ where 36% of edges are strong ties in the this network. By performing ND and BND algorithms, the edges will be re-weighted. The comparison between ND and BND is made on their abilities for maintaining those strong ties because they are reliable. The correlation coefficients between original and re-weighted weights for the strong ties are 0.72 (BND) vs. 0.70 (ND). The result suggests again that BND outperforms ND on the robustness and is less likely to deteriorate the data quality when the input matrix is reliable.

CONCLUSIONS

We have extended the ND theory to a general BND model by introducing an odd-power expansion of the network matrix. One advantage of the balanced ND is that the odd-power extension does not have limit on the eigenvalue range and, therefore, results in a closer approximation to the true network.

For the first time, the BND model was used to improve the residue–residue contact predictions of protein structures by recognizing and filtering the transitive noise. The efficiency of BND was demonstrated by testing with five protein residue–residue contact prediction methods, including MI, MIp, DCA, gDCA, and PSICOV, on the CASP9, CASP10, and PSICOV datasets. Significant improvements can be achieved on both long- and medium-range contacts after the application of the BND-based filtering.

Data analysis of this study reveals that the contact maps generated by co-evolution-based and machine learning-based methods have different transitive effects and should be treated in distinct ways. BND is most efficient in improving the co-evolution-based contact predictions because the correlation matrix from MSAs can induce inherent transition noise in the contact predictions. But, for the machine learning-based contact predictions, because the pair-wise contacts were trained independently, there are much less transitive contaminations.

A further comparison was made on the sequence-based contact prediction with the contact maps derived from the ab initio folding simulations. The results showed that the relative performance by the two approaches is sensitive to the number of homologous sequences available in the sequence databases, which, as an essential factor, influences the usefulness of the sequence-based contact predictions on the ab initio 3D structure constructions.

Overall, the BND can be used as a general filtering method to improve the accuracy of the co-evolution-based contact prediction without additional central processing unit cost. The BND program is freely available at http://www.csbio.sjtu.edu.cn/bioinf/BND/.

Abbreviations

3D

three-dimensional

BND

balanced network deconvolution

CASP

critical assessment of protein structure prediction

DCA

direct-coupling analysis

gDCA

Gaussian DCA

MI

mutual information

MIp

mutual information without the influence of phylogeny or entropy

MSA

multiple sequence alignment

ND

network deconvolution

PSICOV

precise structural contact prediction using sparse inverse covariance estimation