Molecular geometric deep learning

Summary

Molecular representation learning plays an important role in molecular property prediction. Existing molecular property prediction models rely on the de facto standard of covalent-bond-based molecular graphs for representing molecular topology at the atomic level and totally ignore the non-covalent interactions within the molecule. In this study, we propose a molecular geometric deep learning model to predict the properties of molecules that aims to comprehensively consider the information of covalent and non-covalent interactions of molecules. The essential idea is to incorporate a more general molecular representation into geometric deep learning (GDL) models. We systematically test molecular GDL (Mol-GDL) on fourteen commonly used benchmark datasets. The results show that Mol-GDL can achieve a better performance than state-of-the-art (SOTA) methods. Extensive tests have demonstrated the important role of non-covalent interactions in molecular property prediction and the effectiveness of Mol-GDL models.

Graphical abstract

Highlights

• •Mol-GDL predicts molecular properties considering covalent and non-covalent information
• •Mol-GDL is benchmarked on 14 commonly used datasets
• •Mol-GDL can achieve a better performance than state-of-the-art methods
• •Mol-GDL demonstrates the role of non-covalent interactions in molecular models

Motivation

Geometric deep learning (GDL) has demonstrated huge power and enormous potential in molecular data analysis. However, a great challenge still remains for highly efficient molecular representations. Currently, covalent-bond-based molecular graphs are the de facto standard for representing molecular topology at the atomic level. Here, we demonstrate that molecular graphs constructed only from non-covalent bonds can achieve similar or even better results than covalent-bond-based models in molecular property prediction. This demonstrates the great potential of novel molecular representations beyond the de facto standard of covalent-bond-based molecular graphs. Based on the finding, we propose molecular geometric deep learning (Mol-GDL). In our Mol-GDL, molecular topology is modeled as a series of molecular graphs, each focusing on a different scale of atomic interactions. In this way, both covalent interactions and non-covalent interactions are incorporated into the molecular representation on an equal footing.

Shen et al. propose molecular geometric deep learning (Mol-GDL), which builds more general molecular representation into GDL models. In Mol-GDL, molecular topology is modeled as a series of molecular graphs reflecting different scales of atomic interactions. Effective molecular property prediction by Mol-GDL highlights the role of non-covalent interactions.

Introduction

Artificial intelligence (AI)-based models have demonstrated huge power and enormous potential for the prediction of various molecular properties. In particular, AI-based drug design has achieved great success in various steps in virtual screening and has the potential to revolutionize the drug industry.^1,2,3,4 However, even with this great progress, designing efficient molecular representations and featurization remains a great challenge. In general, all AI-based molecular models can be classified into two types, i.e., molecular-descriptor-based machine learning models and end-to-end deep learning models.

The first type uses handcrafted molecular descriptors or fingerprints as input features for machine learning models. The generation of the handcrafted descriptors is known as featurization or feature engineering. Other than physical, chemical, and biological properties, such as atomic partial charge, hydrophobicity, electronic properties, steric properties, etc., the majority of the molecular features are obtained from molecular structural properties. In fact, more than 5,000 molecular structural descriptors have been developed and can be generalized as one-dimensional (1D), two-dimensional (2D), three-dimensional (3D), and four-dimensional (4D) features.^5,6 The 1D molecular descriptors include atom counts, bond counts, molecular weight, fragment counts, functional group counts, and other summarized general properties. The 2D molecular descriptors include topological indices, graph properties, combinatorial properties, molecular profiles, autocorrelation coefficients, etc. The 3D molecular descriptors include molecular surface properties, volume properties, autocorrelation descriptors, substituent constants, quantum chemical descriptors, etc. A related higher computational cost is usually required for the generation of 3D molecular descriptors. The 4D chemical descriptors characterize configuration changes in a dynamic process. These structural descriptors are widely used in quantitative structure-activity relationships (QSARs), quantitative structure-property relationships (QSPRs), and machine learning models. Recently, deep learning models, including autoencoder, convolutional neural network (CNN), and graph neural network (GNN), have also been used in molecular fingerprint generation.^7,8,9,10,11 For all machine learning models, their performance is directly related to the efficiency of these molecular descriptors.

The second type is end-to-end geometric deep learning (GDL) models. In these GDLs, molecules are represented as molecular graphs, density functions, or molecular surfaces, and various deep learning models, such as (3D) CNNs, GNNs, recurrent neural networks (RNNs), etc., can be used to automatically learn the molecular properties.^{12,13,14,15,16} Among these different molecular representations, molecular graphs are the most popular one. In particular, covalent-bond-based molecular graphs are the de facto standard for representing molecular topology at the atomic level. Based on them, various GDL models have been proposed, including graph RNNs (GraphRNNs),¹⁷ graph convolutional networks (GCNs),¹⁸ graph autoencoders,¹⁹ graph transformers,²⁰ and others. These GDLs have been widely used in molecular data analysis, in particular drug design.^21,22,23 Recently, non-covalent-interaction-based molecular descriptors have achieved great performance in the prediction of binding affinities of protein-ligand and protein-protein,^24,25 demonstrating potential new molecular graph representations beyond the de facto standard of covalent-bond-based graphs. In fact, the incorporation of geometric information, such as bond angles, periodicity, symmetry, rotation-translation invariance, equivalence, etc., into GDL models can help to significantly improve the learning performance,^26,27,28,29 especially for machine-learned-based force-field models. Efficient representations beyond covalent-bond-based molecular graphs provide a great promise for a better representation of molecular geometric information.

Here, we show, for the first time, that molecular representations with only non-covalent interactions can achieve similar or even better results than the de facto standard of covalent-bond-based models in molecular property prediction. More specifically, we systematically compare the performance of GDL models using two types of molecular representations, i.e., covalent interaction graphs and non-covalent interaction graphs, in several of the most commonly used benchmark datasets, including BACE, ClinTox, SIDER, Tox21, HIV, ESOL, etc. It has been found that GDL models using only non-covalent interactions have comparable or even superior performance than the de facto standard models. Further, we propose molecular GDL (Mol-GDL) to incorporate a more general molecular representation into GDLs. In our Mol-GDL, molecular topology is modeled as a series of molecular graphs, each focusing on a different scale of atomic interactions. In this way, both covalent interactions and non-covalent interactions are incorporated into the molecular representation on an equal footing. We systematically test Mol-GDL on fourteen commonly used benchmark datasets. The results show that our Mol-GDL can achieve a better performance than state-of-the-art (SOTA) methods.

Results

Is covalent-bond-based molecular graph a de facto standard for GDL?

Currently, the de facto standard for molecular representation at the atomic level is covalent-bond-based molecular graphs. Here, we show that, in molecular property prediction, GDLs with molecular graphs constructed only from non-covalent interactions can achieve similar or even better results. We consider five different molecular graph representations and their GDL performance on the six most commonly used datasets, including BACE, ClinTox, SIDER, Tox21, HIV, and ESOL, as illustrated in Figure 1. Of the five different molecular graphs, one is the de facto standard of the covalent-bond-based model, and the other four are all constructed using only non-covalent interactions. Stated differently, all the edges in these four molecular graphs represent only non-covalent information, and none of them are generated from covalent bonds. Mathematically, these four non-covalent molecular graphs are constructed by defining the edges only between atoms within a certain pre-defined Euclidean distance (larger than the covalent bond distances). We specify a certain domain I for each graph in such a way that the edge in this graph exists if and only if the distance between the corresponding two atoms is in the domain I. For instance, a graph $G\left(I\right)$ with $I=[\mathrm{4,6})$ means that all the edges in graph $G\left(I\right)$ have lengths between 4 and 6 Å. Details for the molecular graph representation can be found in Mol-GDL.

Figure 1.

Illustration of different molecular graph representations and the performance of their GDLs in six commonly used datasets

The de facto standard of covalent-bond-based molecular graph representation has clear limitations and is inferior to models with only non-covalent interactions.

(A) Molecular graph representations for monobenzone molecule. The de facto standard of the covalent bond model is represented in orange, and the other four non-covalent-interaction-based graphs are in blue.

(B) The performance of GDLs with five different molecular representations on the six most commonly used datasets. The color of the bar (for each model) is the same as that of the corresponding molecular graph. For instance, the orange bars are for GDLs with covalent-bond-based molecular graphs.

It can be seen from Figure 1 that GDL with the de facto standard of covalent-bond-based molecular graphs does not have the best performance. In contrast, GDLs with non-covalent molecular graphs can not only have comparable results but can even outperform the covalent-bond-based model. In fact, for all six datasets, the non-covalent model with $I=[\mathrm{4,6})$ has superior performance over the de facto standard model. More interestingly, even for the highly non-traditional molecular graph representation with $I=[8,\infty )$, in which edges only form between atoms that have Euclidean distances larger than 8 Å, the corresponding GDL model can still have a comparably good performance and even outperform the de facto standard model in the four test datasets, including BACE, ClinTox, SIDER, and HIV. This demonstrates the great potential of novel molecular representations beyond the de facto standard of covalent-bond-based molecular graphs.

Mol-GDL

Molecular graph representation for Mol-GDL

For a molecule with N atoms and atom coordinates denoted as ${\mathbf{r}}_1,{\mathbf{r}}_2,\dots ,{\mathbf{r}}_N$, its molecular graph representation can be expressed as $G\left(I\right)=\left(V,E\left(I\right)\right)$. Here, I is a certain interaction region that we used to define our graph representation. For simplicity, we only consider region in terms of $I=[x_{\min },x_{\max })$ with $x_{\min }$ and $x_{\max }$ real values (that satisfy $0\le x_{\min }<x_{\max }$). The set of nodes (or atoms) is denoted V, and the set of edges is denoted $E\left(I\right)$. Note that each atom is modeled as a single node (or vertex) in our molecular graph representation. The adjacent matrix $A\left(I\right)={\left(a{\left(I\right)}_{ij}\right)}_{1\le i\le N;1\le j\le N}$ and its element $a{\left(I\right)}_{ij}$ are represented as

$$a{\left(I\right)}_{ij}=\{\begin{array}{cc}1,& x_{\min }\le \parallel {\mathbf{r}}_i-{\mathbf{r}}_j\parallel <x_{\max }\text{and}i\ne j\\ 0,& \text{others}.\end{array}$$ (Equation 1)

Geometrically, this means that an edge is formed between the i-th and j-th atoms if their Euclidean distance $\parallel {\mathbf{r}}_i-{\mathbf{r}}_j\parallel$ is within the domain of I in molecular graph $G\left(I\right)$. For instance, we can set $I_0=[\mathrm{0,2})$, and the corresponding molecular graph $G\left(I_0\right)$ is exactly the de facto standard molecular graph, which is constructed with only covalent bonds, as their bond lengths are all within the region of $I_0=[\mathrm{0,2})$. Mathematically, by the variation of the domain I, graph representations with dramatically different topologies can be generated. For instance, if we let $x_{\min }>2$ (thus, $x_{\max }>2$), the corresponding molecular graph will contain only non-covalent interactions.

In our Mol-GDL, instead of using only one molecular graph, a series of graphs $G\left(I_k\right)$ are systematically generated by selecting different regions $I_k$ $\left(k>0\right)$ (see STAR Method for details). Geometrically, for a certain graph $G\left(I_k\right)$, an edge is formed between two atoms only when their Euclidian distance is within the region of $I_k$. In this way, we have great flexibility to construct molecular graphs.

Geometric node features for Mol-GDL

The other important setting for our Mol-GDL is the distance-related node features. In contrast to traditional node features, our node features contain only atom types and distance information.

Mathematically, the node feature for the i-th vertex of molecular graph $G\left(I\right)$ is denoted as $f_i\left(I\right)=\left[f_i\left(I,{\alpha }_1\right),f_i\left(I,{\alpha }_2\right),\dots ,f_i\left(I,{\alpha }_m\right)\right]$. Note that here ${\alpha }_j$ (with $1\le j\le m$) means the type of atoms, such as carbon (C), nitrogen (N), oxygen (O), hydrogen (H), sulfur (S), etc. The element $f_i\left(I,{\alpha }_j\right)$ is defined as the number (or frequency) of edges in $G\left(I\right)$ that are formed between the i-th node and any other nodes that are of atom type ${\alpha }_j$. Mathematically, it is defined as follows:

$$f_i\left(I,{\alpha }_j\right)=\sum _{T_j={\alpha }_j}\chi \left(x_{\min }\le \Vert {\mathbf{r}}_i-{\mathbf{r}}_j\Vert <x_{\max }\right).$$ (Equation 2)

Here, $T_j$ is the atom type of the j-th atom, and the value of the indicator function χ is 1 if the following condition is satisfied and 0 otherwise. Geometrically, the node feature contains m descriptors, and each descriptor represents the total number of edges connecting to atoms of a specific atom type. Figure 2 illustrates the node features using a carbon atom as an example.

Figure 2.

Illustration of geometric node features for Mol-GDL

Different from all previous models, our geometric node features are solely determined by atom types and Euclidean distances between atoms.

(A) The illustration of a carbon atom (in pink) and its neighboring carbon atoms (in dark black) and hydrogen atoms (in light gray) from a molecular graph for monobenzone.

(B) In our geometric node features, the neighboring atoms are grouped based on their atom types. Here, the two carbon atoms are classified into one group, and the four hydrogen atoms are classified into the other. For each group, the Euclidian distances between all the neighboring atoms to the carbon atoms are classified into several intervals. For each interval, we count the total number (or frequency) of distances within it. Here, three equal-sized intervals are considered. For carbon atoms, their frequencies in these intervals are (2, 1, 0), and for hydrogen atoms, their numbers are (2, 0, 2). These frequency numbers are then concatenated (in a pre-defined order according to atom types) into a fixed-length long vector, i.e., our geometric node feature.

Further, a refined node feature can be generated through the subdivision of domain I. More specifically, we can divide I into L intervals $\left\{I^l=[x_{l-1},x_l);l=\mathrm{1,2},..,L\right\}$ with $x_0=x_{\min }$ and $x_L=x_{\max }$. The node feature element $f_i\left(I,{\alpha }_j\right)$ will be extended into a vector $\left[f_i\left(I^1,{\alpha }_j\right),f_i\left(I^2,{\alpha }_j\right),\dots ,f_i\left(I^L,{\alpha }_j\right)\right]$. Similar, the scale element $f_i\left(I^1,{\alpha }_j\right)$ is defined as

$$f_i\left(I^l,{\alpha }_j\right)=\sum _{T_j={\alpha }_j}\chi \left(x_{l-1}\le \Vert {\mathbf{r}}_i-{\mathbf{r}}_j\Vert <x_l\right)\text{.}$$ (Equation 3)

Note that our node features are solely determined by atom types and Euclidean distances between atoms.

The general framework for Mol-GDL

Our Mol-GDL provides a GDL architecture that can comprehensively learn the multi-scale information within a molecule. Figure 3 illustrates the flowchart of our Mol-GDL. Different from all previous GDLs, a series of molecular graphs focusing on different scales of interactions are systematically constructed. A common message-passing (MP) module is employed on each individual molecular graph. After that, two pooling operations are used. The first pooling is done at the atomic level. For each molecular graph, node features are aggregated together into a single molecular feature vector. The second pooling is done at the molecular graph level. Each molecular feature vector from the first pooling goes through a single-layer perceptron, and then they are concatenated into a single feature vector. Finally, the single feature vector, which contains information from all the molecular graphs, goes through multi-layer perceptron (MLP) to generate the final prediction.

Figure 3.

The flowchart of our Mol-GDL model

A set of molecular graphs are systematically constructed for each molecule. The geometric node features goes through the same message-passing module. The two pooling operations, one done within graphs and the other between groups, are used to aggregate individual node features into a single molecular feature vector. A multi-layer perceptron (MLP) is employed on the molecular feature vector to generate the final prediction.

Performance of Mol-GDL for molecular data analysis

In this section, we present the comparison results between our Mol-GDLs and SOTAs on different types of molecular data analysis, including classification tasks on the molecular property, regression tasks on the molecular property, and tasks on molecular interaction prediction. For a fair comparison with SOTA models, we adopt the same scaffold split method to divide task datasets into train, valid, and test parts with a ratio of 8:1:1.⁴ Our Mol-GDLs are trained individually on each task and then directly compared with SOTA models. The detailed information of datasets can be found in Table S1.

Mol-GDL for classification tasks on molecular property

In the classification task, seven commonly used datasets³⁰ are considered. These datasets can be roughly classified into two categories: one for biophysical properties (BACE, HIV, and MUV) and the other for physiological properties (BBBP, Tox21, SIDER, and ClinTox). More specifically, the BACE dataset provides quantitative binding results for inhibitors of human β-secretase 1 (BACE-1).³¹ Its average number of atoms per molecule is around 65, which is the largest of the seven datasets. The HIV dataset is for the study of the molecule’s ability to inhibit HIV replication. The MUV dataset, which is screened from PubChem BioAssay by applying a nearest-neighbor analysis,³² is devoted to the validation of virtual screening techniques. The BBBP dataset is for barrier permeability,³³ while SIDER is a database of marketed drugs and adverse drug reactions.³⁴ Both Tox21 and ClinTox datasets³⁵ are related to the toxicity of compounds, and they both contain multiply classification tasks. The measurement of area under the curve (AUC) is used for the evaluation of the results of the models. The detailed setting of our Mol-GDL parameters can be found in Table S2.

The overall performance of Mol-GDL on classification benchmarks along with SOTAs is shown in Table 1. It can be seen that our Mol-GDL has achieved the best results and consistently outperformed the SOTAs in all the tasks except only one from MUV. Our Mol-GDL on BACE is 0.863, which is better than the previous best result (AUC = 0.856). The AUC results of the Mol-GDL model on the two datasets of HIV and MUV are 0.808 and 0.675, respectively. The results of Mol-GDL on BBBP and SIDER are significantly better than SOTAs. Among them, the performance on the SIDER dataset is the most prominent, and the AUC value of the Mol-GDL model is 20.68% higher than the previous SOTA results. The performance of the Mol-GDL model on these two datasets of Tox21 and ClinTox is also much better than the comparison methods. It is worth noting that our model not only has the largest AUC value on the ClinTox dataset (AUC = 0.966) but that it also has a much lower standard deviation (SD = 0.002) than all other existing methods.

Table 1.

The comparison of Mol-GDL with SOTAs on seven commonly used datasets, which contain only classification tasks on molecular properties

Dataset	BACE	BBBP	ClinTox	SIDER	tox21	HIV	MUV
No. molecules	1,513	2,039	1,478	1,427	7,831	41,127	93,087
No. average atoms	${65}_{\left(19\right)}$	${46}_{\left(21\right)}$	${50.58}_{\left(31\right)}$	${65}_{\left(93\right)}$	${36}_{\left(23\right)}$	${46}_{\left(24\right)}$	${43}_{\left(10\right)}$
No. tasks	1	1	2	27	12	1	17
D-MPNN36	${0.809}_{\left(0.006\right)}$	${0.710}_{\left(0.003\right)}$	${0.906}_{\left(0.006\right)}$	${0.570}_{\left(0.007\right)}$	${0.759}_{\left(0.007\right)}$	${0.771}_{\left(0.005\right)}$	${0.786}_{\left(0.014\right)}$
AttentiveFP37	${0.784}_{\left(0.022\right)}$	${0.643}_{\left(0.018\right)}$	${0.847}_{\left(0.003\right)}$	${0.606}_{\left(0.032\right)}$	${0.761}_{\left(0.005\right)}$	${0.757}_{\left(0.014\right)}$	${0.766}_{\left(0.015\right)}$
${\mathrm{N}-\text{Gram}}_{RF}$38	${0.779}_{\left(0.015\right)}$	${0.697}_{\left(0.006\right)}$	${0.775}_{\left(0.040\right)}$	${0.668}_{\left(0.007\right)}$	${0.743}_{\left(0.004\right)}$	${0.772}_{\left(0.001\right)}$	${0.769}_{\left(0.007\right)}$
${\mathrm{N}-\text{Gram}}_{XGB}$38	${0.791}_{\left(0.013\right)}$	${0.691}_{\left(0.008\right)}$	0.875	${0.655}_{\left(0.007\right)}$	${0.758}_{\left(0.009\right)}$	${0.787}_{\left(0.004\right)}$	${0.748}_{\left(0.002\right)}$
PretrainGNN39	${0.845}_{\left(0.007\right)}$	${0.687}_{\left(0.013\right)}$	${0.726}_{\left(0.015\right)}$	${0.627}_{\left(0.008\right)}$	${0.781}_{\left(0.006\right)}$	${0.799}_{\left(0.007\right)}$	${0.813}_{\left(0.021\right)}$
${\text{GROVE}}_{base}$40	${0.826}_{\left(0.007\right)}$	${0.700}_{\left(0.001\right)}$	${0.812}_{\left(0.030\right)}$	${0.648}_{\left(0.006\right)}$	${0.743}_{\left(0.001\right)}$	${0.625}_{\left(0.009\right)}$	${0.673}_{\left(0.018\right)}$
${\text{GROVE}}_{large}$40	${0.810}_{\left(0.014\right)}$	${0.695}_{\left(0.001\right)}$	${0.762}_{\left(0.037\right)}$	${0.654}_{\left(0.001\right)}$	${0.735}_{\left(0.001\right)}$	${0.682}_{\left(0.011\right)}$	${0.673}_{\left(0.018\right)}$
GEM41	${0.856}_{\left(0.011\right)}$	${0.724}_{\left(0.004\right)}$	${0.901}_{\left(0.013\right)}$	${0.672}_{\left(0.004\right)}$	${0.781}_{\left(0.001\right)}$	${0.806}_{\left(0.009\right)}$	${\mathbf{0.817}}_{\left(0.005\right)}$
Mol-GDL	${\mathbf{0.863}}_{\left(0.019\right)}$	${\mathbf{0.728}}_{\left(0.019\right)}$	${\mathbf{0.966}}_{\left(0.002\right)}$	${\mathbf{0.831}}_{\left(0.002\right)}$	${\mathbf{0.794}}_{\left(0.005\right)}$	${\mathbf{0.808}}_{\left(0.007\right)}$	${0.675}_{\left(0.014\right)}$

Note that the subindex indicates standard deviation values. For instance, the element ${65}_{\left(19\right)}$ means the number of average atoms in BACE is 65, with 19 as its standard deviation. Bolding indicates best results.

Mol-GDL for regression tasks on the molecular property

The commonly used datasets for the regression task on the molecular property are mainly divided into two categories.³⁰ One is for predicting the physical and chemical properties of molecules, including ESOL, FreeSolv, and Lipo. The other is in the field of quantum chemistry, including QM7, QM8, and QM9. The detailed setting of our Mol-GDL parameters can be found in Table S3.

For a better comparison with SOTAs,^30,41 we consider two metrics, root-mean-square error (RMSE) and mean absolute error (MAE), respectively, for the two types of regression tasks. Table 2 presents the results of the Mol-GDL model and SOTAs. Generally speaking, our Mol-GDL can achieve similar or even better results than SOTA. In particular, our Mol-GDL has the smallest RMSE values on the ESOL and FreeSolv tasks. On the QM7 dataset, the MAE value of the Mol-GDL model is 62.2, second only to GEM.⁴¹ Although the performance of Mol-GDL on the three datasets (Lipo, QM8, and QM9) did not rank among the top methods, it still beats several classical prediction methods of molecular properties, such as ${N-\text{Gram}}_{RF}$³⁸, ${N-\text{Gram}}_{XGB}$³⁸, ${\text{GROVE}}_{base}$,⁴⁰ and ${\text{GROVE}}_{large}$.⁴⁰

Table 2.

The comparison of Mol-GDL with SOTAs on six commonly used datasets, which contain only region tasks on molecular properties

	RMSE			MAE
Dataset	ESOL	FreeSolv	Lipo	QM7	QM8	QM9
No. molecules	1,128	642	4,200	6,830	21,786	133,885
No. average atoms	${26}_{\left(13\right)}$	${18}_{\left(7\right)}$	${49}_{\left(15\right)}$	${16}_{\left(3\right)}$	${16}_{\left(3\right)}$	${18}_{\left(3\right)}$
No. tasks	1	1	1	1	12	12
D-MPNN36	${1.050}_{\left(0.008\right)}$	${2.082}_{\left(0.082\right)}$	${0.683}_{\left(0.016\right)}$	${103.5}_{\left(8.6\right)}$	${0.0190}_{\left(0.0001\right)}$	${0.00814}_{\left(0.00001\right)}$
AttentiveFP37	${0.877}_{\left(0.029\right)}$	${2.073}_{\left(0.183\right)}$	${0.721}_{\left(0.001\right)}$	${72.0}_{\left(2.7\right)}$	${0.0179}_{\left(0.0001\right)}$	${0.00812}_{\left(0.00001\right)}$
${\mathrm{N}-\text{Gram}}_{RF}$38	${1.074}_{\left(0.107\right)}$	${2.688}_{\left(0.085\right)}$	${0.812}_{\left(0.028\right)}$	${92.8}_{\left(4.0\right)}$	${0.0236}_{\left(0.0006\right)}$	${0.01037}_{\left(0.00016\right)}$
${\mathrm{N}-\text{Gram}}_{XGB}$38	${1.083}_{\left(0.082\right)}$	${5.061}_{\left(0.744\right)}$	${2.072}_{\left(0.030\right)}$	${81.9}_{\left(1.9\right)}$	${0.0215}_{\left(0.0005\right)}$	${0.00964}_{\left(0.00031\right)}$
PretrainGNN39	${1.100}_{\left(0.006\right)}$	${2.764}_{\left(0.002\right)}$	${0.739}_{\left(0.003\right)}$	${113.2}_{\left(0.6\right)}$	${0.0200}_{\left(0.0001\right)}$	${0.00922}_{\left(0.00004\right)}$
${\text{GROVE}}_{base}$40	${0.983}_{\left(0.090\right)}$	${2.176}_{\left(0.052\right)}$	${0.817}_{\left(0.008\right)}$	${94.5}_{\left(3.8\right)}$	${0.0218}_{\left(0.0004\right)}$	${0.00984}_{\left(0.00055\right)}$
${\text{GROVE}}_{large}$40	${0.895}_{\left(0.017\right)}$	${2.272}_{\left(0.051\right)}$	${0.823}_{\left(0.010\right)}$	${92.0}_{\left(0.9\right)}$	${0.0224}_{\left(0.0003\right)}$	${0.00986}_{\left(0.00025\right)}$
GEM41	${0.798}_{\left(0.029\right)}$	${1.877}_{\left(0.094\right)}$	${\mathbf{0.660}}_{\left(0.008\right)}$	${\mathbf{58.9}}_{\left(0.8\right)}$	${\mathbf{0.0171}}_{\left(0.0001\right)}$	${\mathbf{0.00746}}_{\left(0.00001\right)}$
Mol-GDL	${\mathbf{0.798}}_{\left(0.024\right)}$	${\mathbf{1.809}}_{\left(0.100\right)}$	${0.779}_{\left(0.007\right)}$	${62.2}_{\left(0.4\right)}$	${0.0205}_{\left(0.0001\right)}$	${0.00952}_{\left(0.00013\right)}$

Note that the subindex indicates standard deviation values. Bolding indicates best results.

Mol-GDL for molecular interaction analysis

In this task, we use the dataset from DeepDDS,⁴² which contains 36 anticancer drugs, 31 human cancer cell lines, and 12,415 unique drug pair-cell line combinations. Six different validation metrics are used to measure the performance of models, including area under the curve (AUC), area under the curve of precision and recall (AUPR), recall, precision, and F1-score. Since this test involves the drug combination prediction (between two drugs) in different cell lines, a slightly different architecture is used. The representation of synergistic drug combination can be calculated as

$$a^0=concat\left(R_{dru{g}_1},R_{dru{g}_2},MLP\left(R_{cellline}^0\right)\right)\text{,}$$ (Equation 4)

where $R_{dru{g}_1}$ and $R_{dru{g}_2}$ represent the representations of $dru{g}_1$ and $dru{g}_2$ obtained by Mol-GDL and $R_{cellline}^0$ is the expression profiling data for 954 genes, which were used as raw representations of cell lines. Here, $a^0$ is the representation vector concatenated from $dru{g}_1$, $dru{g}_2$, and cell line representations. It was then fed into an MLP to predict synergistic drug combinations

$$a^l=ReLU\left(W^l{a}^{l-1}+b^l\right),l=1,2,...,L$$ (Equation 5)

and

$$\hat{y}=softmax\left(W_{out}{a}^L+b_{out}\right)\text{,}$$ (Equation 6)

where $\hat{y}$ is the predicted value and other notations are the standard weight matrices and biases. Finally, we employ cross-entropy as the loss function to train the model.

Five deep-learning-based drug synergy prediction methods, including TranSynergy,⁴³ DeepSynergy,⁴⁴ deep tensor factorization (DTF),⁴⁵ ${\text{DeepDDS}}_{GAT}$,⁴² and ${\text{DeepDDS}}_{GCN}$⁴², and two machine learning methods, i.e., XGBoost⁴⁶ and random forest (RF),⁴⁷ are used as baselines. The detailed setting of our Mol-GDL parameters can be found in Table S4. Table 3 shows the overall performance of the Mol-GDL model in predicting synergistic drug combinations. It can be seen that our Mol-GDL model outperforms all the other methods on five validation metrics.

Table 3.

The comparison of the performance of Mol-GDL and SOTAs on synergistic drug combination dataset

Methods	Metrics
	AUC	AUPR	Recall	Precision	F1
XGBoost46	${0.92}_{\left(0.01\right)}$	${0.92}_{\left(0.01\right)}$	${0.84}_{\left(0.01\right)}$	${0.84}_{\left(0.01\right)}$	${0.84}_{\left(0.01\right)}$
RF47	${0.86}_{\left(0.01\right)}$	${0.85}_{\left(0.02\right)}$	${0.74}_{\left(0.01\right)}$	${0.78}_{\left(0.02\right)}$	${0.76}_{\left(0.01\right)}$
TranSynergy43	${0.90}_{\left(0.01\right)}$	${0.89}_{\left(0.01\right)}$	${0.80}_{\left(0.01\right)}$	${0.84}_{\left(0.01\right)}$	${0.82}_{\left(0.01\right)}$
DTF45	${0.89}_{\left(0.01\right)}$	${0.88}_{\left(0.01\right)}$	${0.77}_{\left(0.03\right)}$	${0.82}_{\left(0.01\right)}$	${0.80}_{\left(0.02\right)}$
DeepSynergy44	${0.88}_{\left(0.01\right)}$	${0.87}_{\left(0.01\right)}$	${0.75}_{\left(0.01\right)}$	${0.81}_{\left(0.01\right)}$	${0.78}_{\left(0.01\right)}$
${\text{DeepDDS}}_{GAT}$42	${0.93}_{\left(0.01\right)}$	${0.93}_{\left(0.01\right)}$	${0.84}_{\left(0.07\right)}$	${0.85}_{\left(0.07\right)}$	${0.85}_{\left(0.07\right)}$
${\text{DeepDDS}}_{GCN}$42	${0.93}_{\left(0.01\right)}$	${0.92}_{\left(0.01\right)}$	${0.84}_{\left(0.01\right)}$	${0.85}_{\left(0.01\right)}$	${0.84}_{\left(0.01\right)}$
Mol-GDL	${\mathbf{0.94}}_{\left(0.01\right)}$	${\mathbf{0.94}}_{\left(0.01\right)}$	${\mathbf{0.86}}_{\left(0.01\right)}$	${\mathbf{0.86}}_{\left(0.01\right)}$	${\mathbf{0.86}}_{\left(0.01\right)}$

Note that the subindex indicates standard deviation values. Bolding indicates best results.

Discussion

Incorporation of non-covalent bond information enhances performance

The role of non-covalent bonds in predicting molecular properties has long been underappreciated. In this work, we systematically compare the performance of GDLs from non-covalent-bond-based molecular graphs and covalent-bond-based molecular graphs in 13 datasets. Table S5 shows the performance of the Mol-GDL models with the molecular graphs constructed for different bond length ranges, as well as different MP architectures. It can be seen that the performance of our Mol-GDLs is better than that of the models that consider only covalent bond information on most datasets, indicating that non-covalent bonds provide a positive impact on improving the prediction performance of the model. Therefore, these results show the importance of non-covalent interactions in molecular property prediction.

Further, we consider some extreme cases to explore the effects of covalent and non-covalent bonds in molecular representations for GDLs. Three types of neighboring relations are used for the construction of molecular graphs, i.e., n-nearest neighbor, n-farthest neighbor, and n-random neighbor. Here, n is an integer number and can be taken from 1 to 7. The results are shown in Tables S6 and S7, and we have the following observations. First, as the number of neighbor nodes increases, the performance will get better and then tends to be stable. Second, with only a few farthest neighbors or random neighbors, i.e., n = 1 or 2, the model can still have comparably good prediction, in particular the classification tasks. Third, on some datasets, the prediction performance of molecular graphs constructed by the n-farthest neighbors is better than that of the n-nearest neighbors, in particular for BBBP, ClinTox, and QM7 datasets. These observations again demonstrate the importance of non-covalent interactions and the potential novel molecular representations beyond the de facto standard of covalent-bond-based molecular graphs.

Geometric node features enhance GDL performance

Note features are of key importance for GDLs. Here, we propose the geometric node features that are related only to atomic types and Euclidean distances. To verify the efficiency of our geometric node feature approach, we compare it with three different feature engineering methods from the learning models of AttentiveFP,³⁷ D-MPNN,^36,48 and DeepDDS.⁴² Note that the features used in these three models are derived from the structural, physical, chemical, and biological properties, such as the number of atoms, normal charge, chirality, hybridization, aromaticity, etc. For a fair comparison, the same GNN architecture from our Mol-GDL is used. That is, the same Mol-GDL architecture equipped with four types of different initial node features on 11 datasets (MUV and QM9 datasets are omitted due to memory issues) is used. The results are listed in Figure 4 and Table S8. It can be seen that our geometric node features are superior to the three feature engineering approaches in almost all the datasets. In particular, our geometric node feature approach has obvious advantages in BACE, BBBP, FreeSolv, and QM7 datasets.

Figure 4.

Comparison of different methods of calculating node features

It should be noted that our geometric node feature approach is developed based on strong relations between molecular structures and their functions. Even though our approach contains only the atomic types and Euclidean distances, various physical, chemical, and biological information has been implicitly incorporated into it. In fact, the positions of all the atoms within a molecule are the direct reflection of the overall interactions within the molecule. Stated differently, two atoms are close to each other only when the overall interactions (from all the physical, chemical, and biological effects) between them are relatively strong. Since this interaction information is implicitly embodied into its 3D structure through the structure-function relationships, an efficient molecular representation can be obtained as long as we fully explore the structural information of the molecule. From another point of view, the length of chemical bonds is used as the node feature in our Mol-GDL model. This is essentially the process of storing the information of non-covalent bonds in the initial representation. The great performance of our approach again indicates the importance of non-covalent interactions in molecular representation.

General properties of Mol-GDL

With the highly efficient molecular representation, our Mol-GDL model achieves a better performance than SOTAs on most datasets. The performance of our Mol-GDL model is highly consistent and does not depend on the size of the datasets. In Tables 1 and 2, our Mol-GDL achieves the best performance in 8 of the 13 datasets. Especially on the classification task, the average performance is improved by about 3% compared with the best model GEM in the SOTA methods, and 6 of the 7 classification datasets are optimal. Note that the number of samples varies widely in datasets for these classification tasks. There are only about 1,400 molecules in ClinTox and SIDER datasets, while more than 40,000 molecules are included in the HIV dataset. Even with the huge size difference in the datasets, the performance of our Mol-GDL model is consistently better than SOTAs.

Further, the performance of our Mol-GDL model can be related to the size of the molecules. In particular, Mol-GDL may have slightly inferior results than SOTAs for small-sized molecules. In the regression task, although the Mol-GDL model has better results than most of the models, such as ${\mathrm{N}-\text{Gram}}_{RF}$, ${\mathrm{N}-\text{Gram}}_{XGB}$, ${\text{GROVE}}_{base}$, and ${\text{GROVE}}_{large}$, it has slightly inferior results than GEM on QM7, QM8, and QM9. The average numbers of atoms per molecule for these three datasets are about 16, 16, and 18, respectively. However, it should be noted that GEM is a pre-trained self-supervised model, and it uses a much larger dataset for the pre-training process. In contrast, our Mol-GDL uses only the regular training set, which is significantly smaller in size than the dataset for pre-training. It is worth mentioning that the other three models, i.e., PretainGNN, ${\text{GROVER}}_{base}$, and ${\text{GROVER}}_{large}$, also use the pre-training process, but their performance is still inferior to our Mol-GDL for all the tasks. To explore the effects of atomic coordinates on classification and regression tasks, we downloaded the 3D coordinates of the two datasets (Lipo and BBBP) from the website https://weilab.math.msu.edu/DataLibrary/3D/, and these were density functional theory (DFT) optimized. We replace the coordinates generated by Rdkit with these coordinates, and the results of the Mol-GDL model are shown in Table S9. It can be seen from the results that different coordinates have a greater impact on Mol-GDL.

Moreover, other than molecular property analysis, our Mol-GDL model can also achieve great performance in molecular interaction prediction. As shown in Table 3, our Mol-GDL outperforms SOTA methods for predicting synergistic drug combinations on all validation metrics. This indicates that efficient molecular graph representations are important to the analysis of not only properties at the single molecular level but also interactions between two or more molecules.

Finally, our Mol-GDL models can be further generalized or extended from several different aspects. First, we can use our Mol-GDL framework to generate a series of molecular fingerprints.^7,8,9,10,11 More specifically, a series of molecular descriptors can be obtained from the aggregation of node features. Second, Mol-GDL-based multi-task learning models can be constructed to further improve the performance of Mol-GDL. Third, distance GNN models have some intrinsic limitations to distinguish some special molecular structures.⁴⁹ Many-body descriptors alone are also incomplete for molecular structure representation.⁵⁰ The generalization of Mol-GDL into higher-order topological models, such as simplicial complex, polyhedron complex, hypergraph, etc., may provide ways for a more efficient and complete molecular representation.

Limitations of the study

In this article, we address the fundamental challenge of molecular representations for GDL. In particular, we demonstrate the fundamental limitations of the de facto standard of covalent-bond-based molecular graphs and propose novel molecular representations. However, molecular fingerprints learned from a specific task usually have a lower transferability. In our future works, we will consider multi-task learning models to enhance the transferability and further improve the performance of Mol-GDL.

STARMethods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Deposited data
BACE	MoleculeNet30	https://moleculenet.org/datasets-1
BBBP	MoleculeNet30	https://moleculenet.org/datasets-1
ClinTox	MoleculeNet30	https://moleculenet.org/datasets-1
SIDER	MoleculeNet30	https://moleculenet.org/datasets-1
Tox21	MoleculeNet30	https://moleculenet.org/datasets-1
HIV	MoleculeNet30	https://moleculenet.org/datasets-1
MUV	MoleculeNet30	https://moleculenet.org/datasets-1
ESOL	MoleculeNet30	https://moleculenet.org/datasets-1
FreeSolv	MoleculeNet30	https://moleculenet.org/datasets-1
Lipophilicity	MoleculeNet30	https://moleculenet.org/datasets-1
QM7	MoleculeNet30	https://moleculenet.org/datasets-1
QM8	MoleculeNet30	https://moleculenet.org/datasets-1
QM9	MoleculeNet30	https://moleculenet.org/datasets-1
DeepDDS	Wang et al.42	https://github.com/Sinwang404/DeepDDS/tree/master
Software and algorithms
D-MPNN	Yang et al.36	https://github.com/chemprop/chemprop
AttentiveFP	Xiong et al.37	https://github.com/OpenDrugAI/AttentiveFP
${\mathrm{N}-\text{Gram}}_{RF}$	Liu et al.38	https://github.com/chao1224/n_gram_graph
${\mathrm{N}-\text{Gram}}_{XGB}$	Liu et al.38	https://github.com/chao1224/n_gram_graph
PretrainGNN	Hu et al.39	http://snap.stanford.edu/gnn-pretrain
${\text{GROVE}}_{base}$	Rong et al.40	https://github.com/tencent-ailab/grover
${\text{GROVE}}_{large}$	Rong et al.40	https://github.com/tencent-ailab/grover
GEM	Fang et al.41	https://github.com/PaddlePaddle/PaddleHelix/tree/dev/apps/pretrained_compound/ChemRL/GEM
Mol-GDL	This paper	https://github.com/CS-BIO/Mol-GDL https://zenodo.org/record/8304176

Resource availability

Lead contact

Further information and requests for resources should be directed to and will be fulfilled by the lead contact, Kelin Xia (xiakelin@ntu.edu.sg).

Materials availability

This study did not generate new unique reagents.

Method details

Molecular graph representation for Mol-GDL

Graph neural networks can encode molecular structural information into a low-dimensional representation vector from a high-dimensional space. It is a popular method to regard the atoms in the molecule as nodes and the chemical bonds as edges in the graph, and then use the method of message passing to predict the properties of the molecule.^48,51 However, ignoring the quantitative distances between atoms in traditional graphs may result in the loss of a great deal of critical physical and chemical information in molecules. Molecular graphs reconstructed from specific distance intervals can capture some different physicochemical and biophysical patterns, such as hydrogen bonding and van der Waals interactions between different atoms.^25,52

A molecule can be represented as a graph $G\left(I\right)=\left(V,E\left(I\right)\right)$, where $E\left(I\right)$ is an edge set and V is a node set. I is a certain interaction region. Physically, the covalent bonds are usually within the interaction region of $\left[\mathrm{0,2}\right]$. Non-covalent interactions are comparably weak and are within a much larger range. For instance, the distance between donor and acceptor atoms in hydrogen bonds, is roughly between $\left[\mathrm{2.0,4.0}\right]$. van der Waals forces can cover a much larger range and play an very important role in the range $\left[\mathrm{4.0,6.0}\right]$. The electrostatic forces can go further to $\left[\mathrm{6.0,8}\right]$. To fully explore the physicochemical and biophysical patterns in non-covalent interactions, we divide the interval region greater than 2Å into several non-overlapping regions. For each interaction region, a corresponding molecular graph is generated and its associated adjacency matrix is defined as Equation 1.

To balance the computational costs and model accuracy, a total of 4 segments are considered for non-covalent interactions, namely $I_1=\left[\mathrm{2,4}\right),I_2=\left[\mathrm{4,6}\right),I_3=\left[\mathrm{6,8}\right),I_4=[8,\infty )$. Note that the selection of interaction regions to build molecular graphs is not unique. For convenience, we let $I_0=[\mathrm{0,2})$ denotes the interval of covalent interactions. In this way, total 5 interaction regions are considered and a total of 5 molecular graphs, denoted as $G=\left\{G_1,G_2,G_3,G_4,G_5\right\}$, are generated for each molecular structure, in all of our three types of tasks except those from QM7, QM8, and QM9. Since the number of atoms in the three datasets QM7, QM8, and QM9 is relatively small, the number of non-covalent interactions greater than $6\AA$ is very small. Therefore, we divide the non-covalent interaction interval for these three datasets into three segments, namely $I_1=\left[\mathrm{2,4}\right),I_2=\left[\mathrm{4,6}\right),I_3=[6,\infty )$. Similarly, let $I_0=[\mathrm{0,2})$ represent the interval of covalent bonds. In this way, total 4 interaction regions are considered and a total of 4 molecular graphs, denoted as $G=\left\{G_1,G_2,G_3,G_4\right\}$, are generated for each molecular structure from QM7, QM8, and QM9.

Graph neural network architecture in Mol-GDL

For a molecular $G\left(I_k\right)$, denoted as $G_k$ for short, we use message passing to learn the feature representation of each node,

$${\mathit{h}}_i^{G_k,\left(t\right)}=\sigma \left(\sum _{j\in \mathcal{N}\left(i\right)\cup \left\{i\right\}}\frac{1}{\sqrt{d\left(i\right)}·\sqrt{d\left(j\right)}}·\left(W_{G_k}·{\mathit{h}}_j^{G_k,\left(t-1\right)}\right)\right)$$ (Equation 7)

In the t-th iteration, the i-th node feature vector $h_i^{G_k,\left(t\right)}$ of a graph $G_k$ is obtained by gathering node feature vectors of its neighbors, denoted as $\mathcal{N}\left(i\right)$, and itself, from $\left(t-1\right)$-th iteration. Here $d(·)$ represents the node degree and $W_{G_k}$ is the weight matrix (to be learned). Computationally, we usually repeat the process 1 to 3 times, and the final node feature is denoted as ${\mathit{h}}_i^{G_k}$.

After the message passing, we aggregate all node features in a molecular graph through a pooling process,

$${\mathit{h}}_{G_k}=Pooling\left({\mathit{h}}_i^{G_k}|i\in \left[\mathrm{1,2},\dots ,N\right]\right)$$ (Equation 8)

where $Pooling(·)$ is a pooling function, such as max pooling, mean pooling, etc. Note that this pooling process is within the single molecular graph.

In our Mol-GDL, each molecule will have more than one molecular graph, we aggregate all the features from its molecular graphs,

$${\mathit{h}}_{G_k^{\prime }}^{\prime }=\sigma \left(W_{G_k^{\prime }}^{\prime }{\mathit{h}}_{G_k}+b_{G_k^{\prime }}^{\prime }\right)$$ (Equation 9)

$${\mathit{h}}_G=READOUT\left({\mathit{h}}_{G_k^{\prime }}^{\prime }|k\in \left\{\mathrm{1,2},\dots ,5\right\}\right)$$ (Equation 10)

where $READOUT(·)$ denotes a pooling function, and we choose concatenation (or mean) in this study.

Finally, a multiply layer perceptron (MLP) is utilized for the final prediction,

$$\hat{y}=\delta \left(W^1\sigma \left(W^0{\mathit{h}}_G+b^0\right)+b^1\right)$$ (Equation 11)

where $\sigma (·)$ and $\delta (·)$ are $Relu(·)$ and $Sigmoid(·)$, respectively. For regression tasks, $\delta (·)$ is a linear activation function. The ${\mathcal{l}}_1$ loss and cross-entropy loss are implemented for regression and classification tasks, respectively.

Geometric node features for Mol-GDL

The geometric node features are of great importance for GDLs. In our Mol-GDL geometric node features that contain only atomic types and Euclidean distance information. Computationally, we consider a total of 12 types of atoms, including C, H, O, N, P, $Cl$, F, $Br$, S, $S_i$, I and all the rest atoms as one type. These atoms are chosen due to their high frequencies in the molecules in our datasets. For the i-th atom, atom type ${\alpha }_j$ will contribute a component $f_i\left(I,{\alpha }_j\right)$ in the geometric node feature $f_i\left(I\right)=\left[f_i\left(I,{\alpha }_1\right),f_i\left(I,{\alpha }_2\right),\dots ,f_i\left(I,{\alpha }_{12}\right)\right]$. Here $f_i\left(I,{\alpha }_j\right)$ means the number (or frequency) of all the neighboring atoms of type ${\alpha }_j$ for the i-th atom. Further, different subdivision of interaction region I is done for different types of atoms. More specifically, for C (and H), its node feature vector components are of the size 19 after subdivision. That is the component $f_i\left(I,C\right)$ has been extended to a vector $\left[f_i\left(I^1,C\right),f_i\left(I^2,C\right),\dots ,f_i\left(I^{19},C\right)\right]$. Here $f_i\left(I^1,C\right)$ is the number of all the neighboring atoms of type ${\alpha }_j$ within the interaction subregion $I^1$ for the i-th atom. For O and N, their node feature components are 4 after subdivision. All the rest types of atoms have the same node feature components of size 2 after subdivision. The detailed information for the subdivision of the interaction region (based on different atom types) is listed in Table S10.

Benchmark datasets and experimental setting

In this work, the 13 datasets used for molecular property prediction are all derived from MoleculeNet.³⁰ The detailed information of these 13 datasets can be found in literature.^30,36 The general information of these 13 datasets is shown in Table S1. It is worth noting that No. atoms represents the average number of atoms in each molecule in the corresponding dataset, with the standard deviation in parentheses. In addition, the 3D coordinate indicates the source of the atomic coordinates in each dataset. As can be seen from the table, except for the three datasets QM7, QM8 and QM9, the coordinates of other datasets are obtained through RDKIT (https://www.rdkit.org/). It is worth noting that both the node features and the original graph in Mol-GDL heavily depend on the accuracy of the atomic 3D coordinates, so accurate atomic 3D coordinates play an important role in ensuring the performance of the Mol-GDL model. For these 13 benchmarks, we adopt the scaffold split method³⁹ to create a train, valid and test split with a ratio of 8:1:1. Unlike random split, the principle of scaffold split is to reasonably distribute the substructures of molecules in the train, valid and test set. This splitting method is more realistic and more challenging.

Quantification and statistical analysis

Further, different metrics are employed for model evaluation. More specifically, two metrics, i.e., RMSE and MAE are used for regression tasks. For classification tasks, five commonly used evaluation metrics, namely AUC, AUPR, Recall, Precision and F-score, are considered. The detailed explanation of these metrics can be found below.

RMSE is the abbreviation of root mean squared error, and its definition is as follows:

$$RMSE=\sqrt{\frac{1}{n}\sum _{i=1}^n{\left(y_i-{\hat{y}}_i\right)}^2}$$ (Equation 12)

where $y_i$ and ${\hat{y}}_i$ represent the true value and predicted value of ith sample respectively.

MAE is the abbreviation of mean absolute error, and its definition is as follows:

$$MAE=\frac{1}{n}\sum _{i=1}^n\left|y_i-{\hat{y}}_i\right|$$ (Equation 13)

where $y_i$ and ${\hat{y}}_i$ represent the true value and predicted value of ith sample respectively.

For classification tasks, AUC and AUPR are the most common metrics used to evaluate the performance of classification tasks. If you want to calculate AUC, you must first calculate true positive rate (TPR) and false positive rate (FPR). The calculation process is as follows:

$$TPR=\frac{truepositive}{truepositive+falsenegative}$$ (Equation 14)

$$FPR=\frac{falsepositive}{falsepositive+truenegative}$$ (Equation 15)

where the curve composed of TPR and FPR is called ROC, and the area under the receiver operating characteristic (ROC) curve, namely area under the receiver operating characteristic convex hull (AUC-ROC), also called AUC.

AUPR is the area under the curve composed of precision and recall, where precision and recall are calculated as follows:

$$precision=\frac{truepositive}{truepositive+falsepositive}$$ (Equation 16)

$$recall=\frac{truepositive}{truepositive+falsenegative}$$ (Equation 17)

where the precision and recall can calculate F1-score, which is also an important evaluation metric for evaluating classification performance. Its calculation method is as follows,

$$F1-score=2\ast \frac{precision\ast recall}{precision+recall}$$ (Equation 18)

Published: October 23, 2023

Data and code availability

• •All data used in this study is publicly available. The accession numbers for the datasets are listed in the Key resources table.
• •Original code generated for this study is available at Github [https://github.com/CS-BIO/Mol-GDL]. An archival DOI is listed in the Key resources table.
• •Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.