Machine learning models to predict ligand binding affinity for the orexin 1 receptor

Abstract

The orexin 1 receptor (OX1R) is a G-protein coupled receptor that regulates a variety of physiological processes through interactions with the neuropeptides orexin A and B. Selective OX1R antagonists exhibit therapeutic effects in preclinical models of several behavioral disorders, including drug seeking and overeating. However, currently there are no selective OX1R antagonists approved for clinical use, fueling demand for novel compounds that act at this target. In this study, we meticulously curated a dataset comprising over 1300 OX1R ligands using a stringent filter and criteria cascade. Subsequently, we developed highly predictive quantitative structure-activity relationship (QSAR) models employing the optimized hyper-parameters for the random forest machine learning algorithm and twelve 2D molecular descriptors selected by recursive feature elimination with a 5-fold cross-validation process. The predictive capacity of the QSAR model was further assessed using an external test set and enrichment study, confirming its high predictivity. The practical applicability of our final QSAR model was demonstrated through virtual screening of the DrugBank database. This revealed two FDA-approved drugs (isavuconazole and cabozantinib) as potential OX1R ligands, confirmed by radiolabeled OX1R binding assays. To our best knowledge, this study represents the first report of highly predictive QSAR models on a large comprehensive dataset of diverse OX1R ligands, which should prove useful for the discovery and design of new compounds targeting this receptor.

1. Introduction

The neuropeptides orexins A and B (also known as hypocretins 1 and 2) are a pair of neuropeptides produced exclusively by neurons in caudal hypothalamus [1,2]. Despite their relatively small number (~4000 neurons in rat, ~70,000 in human) [3,4], orexin neurons mediate a broad range of physiological processes by acting at two receptor sub-types (orexin receptor 1 and 2; OX1R, OX2R) that are broadly distributed throughout the central nervous system [5]. Orexins have been shown to be critical regulators of arousal/wakefulness, stress reactivity and reward motivation [6–12], making this system an attractive target for pharmacological compounds designed to treat a myriad of disease states.

Early preclinical work indicated that blocking orexin signaling promoted sleep, and that better outcomes were achieved with antagonists that acted at both OX1R and OX2R compared with either receptor alone [13]. Consequently, three dual orexin receptor antagonists, suvorexant, lemborexant and daridorexant (Fig. 1A), have been approved by the FDA for clinical management of insomnia [14–17]. Since then, significant advances have been made in the understanding of OX1R vs. OX2R signaling, with substantial evidence now pointing to the potential utility of selective orexin receptor antagonists for non-sleep related indications. Most notably, excessive signaling at OX1R has been specifically implicated in various behavioral states, including substance use disorders, overeating and anxiety, hypertension, as well as several digestive cancers [18–30]. SB-334867, the first selective OX1R antagonist [31], was effective in vivo at attenuating drug and food seeking behaviors, but failed to progress to clinical use due to poor stability [32]. ACT-539313 (Fig. 1A), an orally active selective OX1R antagonist, advanced into phase 2 clinical trials for the treatment of binge eating disorder [33,34], but didn’t exhibit statistically significant efficacy in reducing binge eating episodes [35–37]. Although there are several other OX1R antagonists in the developmental pipeline (e.g. JNJ-61393215, anxiety disorder; AZD4041, opioid use disorder), at present, none have gained FDA approval for clinical use. Thus, there is interest in the development of novel OX1R antagonists [38], as well as the repurposing of existing compounds with previously unrecognized actions at OX1R.

Fig. 1.

A: Representative dual orexin receptor antagonists (suvorexant, lemborexant and daridorexant) and a selective OX1R antagonist (ACT-539313). B: Molecular alignment of 14 human OX1R crystal structures complexed with diverse ligands. C: Extracellular view of ligands binding to the OX1R. Suvorexant-bound crystal structure (6TO7.pdb) was shown in cyan ribbons. Ligands were rendered as stick models and colored by atom types. Ligands adopt an uncommon horseshoe binding conformation.

To this end, extensive efforts have been taken to solve the crystal structures of the human OX1R with diverse ligands. So far, 14 X-ray crystal structures of human OX1R have been deposited in the Protein Data Bank (PDB; www.rcsb.org) [39]). As shown in Fig. 1B, all ligands bind to the orthostatic site of OX1R, typical binding sites in GPCRs. However, structural analysis found unique interactions observed in crystal structures of orexin receptors. All OX1R ligands adopt an uncommon horseshoe binding conformation resulting from the intra-molecular pi-stacking interactions (Fig. 1C) [40–42]. Meanwhile, lipophilic hotspots and water molecules were found to play important roles in ligand-OX1R interactions. Molecular docking, the primary method for structure-based drug design, usually ignores or underestimates the energy contributions of intra-molecular interactions, lipophilic hotspots, and water-mediated hydrogen bonds. Thus, docking-based virtual screening using the OX1R crystal structures is unlikely to be effective in identifying new OX1R ligands. Consequently, we resorted to ligand-based drug design methods for our research needs. Surprisingly, PubMed searches using “orexin receptor” and “QSAR” or “machine learning” as search terms were unsuccessful in identifying any prior publications on such topics. Therefore, as a first step, we elected to develop quantitative structure-activity relationship (QSAR) models for OX1R ligands.

Here, molecular descriptor based QSAR models were developed for a large dataset of known OX1R ligands that were downloaded from the ChEMBL database [43]. KNIME workflows (https://www.knime.com/) were constructed to rigorously filter the dataset using best practices recommended by the Organization for Economic Cooperation and Development (OECD, http://www.oecd.org/). Highly predictive QSAR models were obtained employing the random forest machine learning method together with molecular operating environment (MOE) descriptors (Chemical Computing Group, Montreal, QC, Canada). The predictive capacity of the QSAR model was assessed using an external test set and an enrichment study. The practical applicability of our final QSAR model was demonstrated through virtual screening of the DrugBank database of FDA-approved drugs [44] and subsequent identification of two novel OX1R ligands.

2. Materials and methods

All calculations were performed on a MacBook with a 2.8 GHz Quad-Core Intel Core i7 processor and a memory of 16 GB RAM using Molecular Operating Environment (MOE.2020, Chemical Computing Group, Montreal, QC, Canada), KNIME (version 4.6.3) [45], Python (version 3.9.7) and Jupyter Notebook (version 6.4.5).

2.1. Data curation

We developed a workflow (Fig. 2) to collect bioactive ligands for human OX1R by using the opensource data analytics platform KNIME and its Community Extensions (available free of charge at https://www.knime.com/). The ChEMBL ID (CHEMBL5113) for human OX1R was used as the query to collect all OX1R ligands from the ChEMBL (version 30) website (https://www.ebi.ac.uk/chembl/) [43]. After all bioactive OX1R compounds were downloaded to KNIME, a series of filtering steps were adopted to eliminate those not fitting our criteria. First, only entries with exact measurements (“standard_relation” matches “=”) were kept and censored data (i.e., > or <) were removed. Second, only data measured in binding assays (“assay_type” = “B”) and reported with binding constants (“standard_type” = “Ki”) were retained. Next, compounds with missing chemical structures (“canonical_smiles” = “missing”) were removed. Compounds with identical IDs (“molecule_chembl_ id”) were then labeled as duplicates. If the difference in Ki values between duplicates was less than 10-fold, the first entry was kept for this compound. However, if the difference in Ki values was more than 10-fold, we searched the structures in ChEMBL and original publications to ensure we chose the correct Ki values. Resulting entries were further evaluated to confirm that “standard_units” were equal to “nM”. Eventually, the Ki values of all entries that passed the aforementioned filters and criteria were converted to pKi values (pKi = −log(Ki)).

Fig. 2.

KNIME workflows to download OX1R bioactive ligands from ChEMBL and to clean the dataset using various criteria.

2.2. Feature calculation and preprocessing

All compounds were then imported to MOE for further processing. First, the Database Wash operation was performed for the structures (represented as SMILES strings) to remove salts, solvents, and other minor components in their structures. After the structures were cleaned, an MOE script of “db_unique.svl” was run to remove duplicates with identical structures. Next, 216 MOE built-in 2D molecular descriptors or features (here, both terms are used interchangeably) and MACCS fingerprints were calculated for all the retained structures. To identify potential outliers, a principal component analysis (PCA) was performed for all 2D descriptors using MOE. A 3D graphical plot was built with respect to the first three principal components (PC1, PC2 and PC3; Fig. 3). A molecule was considered as an “outlier” when it was located distant from the majority of data in the 3D graphical plot. After removal of the outliers in the dataset, clustering analysis was carried out to evaluate the chemical space in the dataset using MACCS fingerprints and the Tanimoto coefficient. A resulting dataset of 1350 OX1R ligands was submitted to a KNIME workflow for the feature selection.

Fig. 3.

The PCA plot of the entire dataset of OX1R ligands. The first three principal components (PCA1, PCA2, PCA3) were displayed. Two outliers were highlighted in pink.

All compounds in the dataset were imported to a KNIME workflow (Fig. 2) to remove descriptors with constant values. The number of features was further reduced by removing those highly correlated using the KNIME nodes of “Linear Correlation” and “Correlation Filter”. Linear correlation analysis was performed to calculate Pearson coefficients for each pair of descriptors. A pair of features with a Pearson correlation coefficient ≥0.7 were identified as highly correlated and one of them was eliminated as a redundant feature.

2.3. Machine learning (ML) algorithms

Four ML methods implemented using scikit-learn (a library for traditional machine learning algorithms for the Python programming language, version 1.1.1) were used to develop QSAR models for the dataset of OX1R ligands: random forest (RF), extreme gradient boosting (XGBoost), and both linear and radial basis function (RBF) support vector machine (SVM). These methods have been widely applied to predictions of molecular properties and compound activities with mixed performance.

2.4. Dataset partition

The entire dataset of 1350 OX1R ligands was randomly partitioned into a training set and a test set with the ratio of 4:1. Preliminary QSAR models using all 76 features were first developed to compare the performance of different ML methods. To avoid the effect of randomness in data splitting, 50 independent runs with different random seeds were conducted to assess each ML method in a comprehensive way. The values of R² (squared correlation coefficient of the regression model) for the training and test sets were selected as the criteria to evaluate the statistical performance of the ML methods. R² and ${\text{R}}_{\text{pred}}^2$ are defined in Eq. (1). The method with the best collective performance for both training and test sets — RF, was selected for the current QSAR modeling study.

2.5. Feature selection and hyper-parameter optimization

Recursive Feature Elimination (RFE) with cross-validation is commonly used for feature selection. In the current study, RFE with 5-fold cross-validation from scikit-learn (the RFECV module) was run for the entire training set to select the most important features for the RF method. Then the GridSearchCV module with 5-fold cross-validation from scikit-learn was used to optimize the hyper-parameters for the ML study on the training set and the features chosen by the RFECV module. R² was used as the optimization criterion. The following parameters were used for the optimization process: n_estimators = [200,500,1000], max_features = [‘auto’, ‘sqrt’, ‘log2’], max_depth = [4,6,8,10,12].

After determining the most important features and optimal hyper-parameters, the external test set was used to validate the predictivity and generalization capability of the QSAR models. Fifty independent runs with different random seeds were conducted to avoid coincidence. A final QSAR model was built for the entire dataset of OX1R ligands with optimized molecular features and hyper-parameters (random_state = 439713).

2.6. Model performance evaluation

Three statistical indicators were adopted to evaluate the quality of the regression models, including the square correlation coefficients (R²) in Eq. (1), the root mean squared error (RMSE) in Eq. (2), and the mean absolute error (MAE) in Eq. (3).

$$R^2=1-\frac{\sum _{i=1}^N{\left(y_i-\widehat{y_i}\right)}^2}{\sum _{i=1}^N{\left(y_i-\overline{y}\right)}^2}$$ (1)

$$\mathit{\text{RMSE}}=\sqrt{\frac{\sum _{i=1}^N{\left(y_i-{\widehat{y}}_i\right)}^2}{N}}$$ (2)

$$MAE=\frac{1}{N}\sum _{i=1}^N{\left(y_i-{\widehat{y}}_i\right)}^2$$ (3)

where y_i and ŷ_i are the experimental and predicted values of the samples in the training set, respectively. ȳ_i is the mean experimental value of all the samples in the training set; N is the number of the samples in the training set. (Q², RMSE_cv, MAE_cv) and (${\text{R}}_{\text{pred}}^2$, RMSE_pred, MAE_pred) are separately used for the cross-validation dataset and test set, and defined similarly to Eqs. (1–3) shown above.

2.7. Enrichment study

To further validate our QSAR models for virtual screening purposes, an enrichment study was performed. One hundred twenty-six highly active OX1R ligands (pKi ≥ 8.0) in the dataset were selected and clustered using MACCS fingerprints and Tanimoto coefficient (0.8) with MOE, resulting in 48 clusters. Ligands with the highest pKi values in each cluster were chosen as the active OX1R ligands to generate decoys on the DUD-E server (http://dude.docking.org) [46,47]. Each active ligand collected 50 decoy molecules that possess similar physicochemical properties but distinct molecular topology. Using the default setting in the DUE-E server, 48 active molecules produced 2499 decoys, resulting in an enrichment dataset of 2548 compounds for this study. These compounds were prepared using the same protocols as molecules in the training set. All 2D descriptors were calculated using MOE. The final QSAR model developed for the entire cleaned dataset was used to predict the binding constants (pKi) for all compounds in the enrichment dataset. Enrichment factor (EF) and goodness of hit list (GH) score, defined by Eqs. (4) and (5), were used as metrics to evaluate the predictivity of the QSAR model [48,49].

$$EF=\frac{H_a/H_t}{(A/D)}$$ (4)

$$GH=\left\{\frac{H_a*\left(3A+H_t\right)}{4H_t*A}\right\}*\left[1-\frac{(H_t-H_a}{(D-A)}\right]$$ (5)

where D represents the total number of all compounds in the dataset, including true actives and decoys; and A is the total number of true active compounds. H_t and H_a stand for the number of true active and predicted active compounds (or hits) by the QSAR model in the hit list, respectively.

EF measures how well a QSAR model can identify active molecules relative to a random selection. For example, a QSAR-based screening with EF= 10 means this QSAR model can identify active compounds at the rate of 10 times that of random selection. The higher the EF, the more likely an active molecule would be selected from a given virtual screening. The GH score at ranges of 0–1 considers both true actives ratio and true inactives ratio, and therefore measures how well a model can discriminate actives from inactives. Models with GH > 0.6 are generally considered reliable, while GH= 0 means a null model and GH= 1 means a perfect model [50].

2.8. QSAR-based virtual screening

To further assess the predictivity of our QSAR models, virtual screening was performed against the DrugBank database (version 5.1.9, https://www.drugbank.ca) [44], which contains 2715 drugs approved by the US Food and Drug Administration (FDA). Peptide-like and inorganic molecules were eliminated from selection. Furthermore, drugs with molecular weights below 250 or above 700 g/mole were removed, resulting in a filtered DrugBank database of 1614 drugs for the current study.

Daridorexant (an OX1R/OX2R antagonist) was approved for clinical use by the FDA in the United States in January 2022. Therefore, when the current DrugBank database was released on Jan 4th, 2022, daridorexant was not included in the collection. Consequently, the DrugBank database was supplemented by the addition of this drug to evaluate the predictive performance of our QSAR model. All drug molecules were prepared in the same protocols as molecules in the training set. All 2D descriptors were calculated using MOE. The top-ranked hits were explored to assess the performance of the QSAR model for virtual screening.

2.9. In vitro radiolabeled binding assay

Radiolabeled binding assays of the OX1R were performed by Eurofins Panlabs Discovery Services (Taipei, Taiwan) using human recombinant CHO-S cells, according to a previously reported protocol [51]. The binding assay was conducted in duplicate on membrane preparations that had been re-suspended at pH7.4 in the buffer consisting of 25 mM Hepes/NaOH, 2.5 mM MgCl2, 2.5 mM CaCl2, 0.5 mM EDTA, and 0.025% Bacitracin. [¹²⁵I] Orexin A (0.1 nM) was used as the radioligand. Following 1-hr incubation at 25 °C, the binding assay was terminated by addition of cold buffer. The mixture was then filtered through Whatman GF/B filters and washed with cold buffer. Radioactivity was determined using the TopCount NTX liquid scintillation counter (PerkinElmer, Waltham, MA). Non-specific binding for the OX1R was measured in the presence of 1.0 μM unlabeled SB-334867. Receptor binding data were analyzed by nonlinear regression of saturation and competition curves using the GraphPad Prism 8.0 software (GraphPad Software, La Jolla, CA).

3. Results and discussion

3.1. Data curation

Best practices, recommended by OECD were followed for pooling data from disparate sources for the present QSAR study [52]. In order to achieve statistically significant and predictive models, several heuristic rules were adopted in selecting the dataset t: (1) the minimum range of bioactivities should be three log units, (2) the biological activities should be distributed evenly throughout the range, and (3) the dataset should contain diverse compounds to develop a comprehensive QSAR model [53]. Ideally, all compounds should come from a single source to minimize the inevitable variation in pharmacological data measured by different laboratories. When this is not possible, as is the case with the subject OX1R antagonists, vigorous validation should be done to evaluate pooling data from different sources, which has been shown to be a feasible strategy by our previous QSAR studies [54–56].

ChEMBL is a valuable resource for chemical and biological data [43]; however, the data are highly heterogeneous and cannot be used for QSAR modeling studies in their current state. Consequently, we developed KNIME workflows to rigorously clean and filter the bioactivity datasets downloaded from ChEMBL for the present QSAR studies (Fig. 2). The ChEMBL ID (CHEMBL5113) for human OX1R was used to search ChEMBL and found 5123 bioactivities associated with this orexin receptor, 2308 of which were reported with exact measurements in binding assays. Filtering with Ki as the binding affinity measurement yielded 1642 entries containing the chemical structure information. After removing duplicates, 1365 unique compounds were retrieved, and their binding constant (Ki) values were converted to pKi values. The experimental biological activities (pKi) in the entire dataset range from 4.18 to 10.22 with a spread of more than 6 logarithmic units: 589 weakly active compounds (pKi < 6.0), 654 moderately active compounds (6.0 ≤ pKi < 8.0), and 126 highly active compounds (pKi ≥8.0). The entire dataset of 1369 OX1R ligands were further processed using MOE to remove redundant structures and heavy compounds (molecular weights ≥1000 g/mole) since here we were only interested in small molecules.

MOE’s built-in descriptors cover a wide range of molecular properties, including physical properties, atom and bond counts, Hückel theory descriptors, subdivided surface area descriptors, shape and charge descriptors, adjacency and distance matrix descriptors, and pharmacophore feature descriptors. These descriptors have been widely used in QSAR modeling of physicochemical properties and bioactivities. In the current study, we started with all MOE 2D descriptors trying to utilize all ligand information for QSAR modeling.

A fundamental assumption of all QSAR studies is the Similar Property Principle stating that structurally similar compounds possess similar activities or properties. However, it is not uncommon that small structural changes in some compounds can translate to large changes in activities and properties, in violation of this principle. Such compounds are classified as activity cliffs or outliers, and should be avoided in any QSAR modeling study [53]. To address this issue, we conducted a Principal Component Analysis (PCA) on the 2D molecular descriptors computed using MOE for the entire dataset. A molecule is considered an outlier if it is located distant from most data in the 3D graphical plot of the first three principal components (PC1, PC2 and PC3; Fig. 3). Two activity outliers with the highest molecular weight in this dataset were identified by the PCA and, therefore, removed from the QSAR modeling (Fig. 3), resulting in 1350 unique OX1R ligands in the cleaned dataset. Clustering analysis was performed to evaluate the chemical space in the dataset using MACCS fingerprints and a Tanimoto coefficient of 0.80, a recommended value for similarity-based analysis [57]. A total of 518 clusters were identified, suggesting a highly diverse dataset.

Furthermore, 30 features (i.e., a_nP: number of phosphorus atoms) were identified at constant values by KNIME workflows (Fig. 2) and therefore were eliminated. Pair-wise linear correlation analysis between all remaining descriptors was performed to remove 110 redundant features, which greatly reduced the number of features from 186 to 76. The resulting dataset includes 1350 ligands and 76 features.

3.2. Preliminary QSAR modeling

A wide variety of machine learning (ML) algorithms have been applied to develop QSAR models; however, no single one has been proven to be superior to others. Recent studies of ML methods suggest RF, XGBoost, and SVM are among the best performers [58,59]. RF is an ensemble learning method that combines multiple decision trees to create a more robust and accurate predictive model. It works by constructing a collection of decision trees during training and then averaging their predictions during inference. XGBoost is another ensemble learning technique that focuses on boosting multiple weak learners, typically decision trees, into a strong predictive model. It employs a gradient boosting algorithm to iteratively add trees that correct the errors of the previous ones, thereby improving the overall model performance. SVM seeks to find the optimal hyperplane that best separates data points from different classes or predicts the target values in a regression context. SVM is built with different kernels, which allow them to operate effectively in higher-dimensional spaces without explicitly transforming the data. The kernels of linear and radial basis function (RBF) are commonly used for linear and complexed data analysis (SVM-linear and SVM-RBF), respectively. Meanwhile, descriptor based QSAR models have been shown to generally produce more accurate predictions and more easily interpreted models than graph neural network, although the latter has gained substantial attention and been broadly applied to various endpoints [60]. In the currently study, we based our QSAR models on molecular descriptors using 4 different ML methods: RF, XGBoost, SVM-Linear and SVM-RBF.

A random partition using the ratio of 4:1 yielded a training set of 1080 compounds and a test set of 270 compounds. Preliminary QSAR models using all 76 features were constructed for the training set and then applied to the test set for validation with four ML methods. The model statistics are shown in Fig. 4. The averaged R² values among 50 independent runs on the training set were 0.945 ± 0.002, 0.994 ± 0.002, 0.498 ± 0.012, and 0.502 ± 0.014 for RF, XGboost, SVR-RBF and SVR-Linear methods, respectively. Meanwhile, the averaged ${\text{R}}_{\text{pred}}^2$ values on the test set were 0.664 ± 0.026, 0.649 ± 0.049, 0.405 ± 0.066, and 0.384 ± 0.083 for RF, XGboost, SVR-RBF and SVR-Linear methods, respectively. RF and XGboost clearly yielded much better statistical results than SVR methods in both the training and test sets. However, XGboost produced a large performance variation for the test set in terms of ${\text{R}}_{\text{pred}}^2$, ranging from 0.506 to 0.727. On the other hand, RF yielded consistent performance for the test set with ${\text{R}}_{\text{pred}}^2$ ranging from 0.601 to 0.733, suggesting the RF model statistics are less likely generated by coincidence. Taken together, RF was selected as the ML method for the QSAR study.

Fig. 4.

Boxplots comparing the performance of 4 different ML methods. Left and right are R² values for the training and test sets, respectively. Red lines represent medians; open circles represent outliers.

3.3. Feature selection

Five-fold cross-validation is a widely used procedure to evaluate the performance of ML models and was adopted by the current QSAR study for feature selection and hyper-parameter optimization. During this process, the whole training set was randomly partitioned into five roughly equal-sized groups. The QSAR model was constructed using the data in four groups, and the prediction statistics for the remaining group were calculated. The process was regenerated five times so that every group could be used as the validation set. Recursive feature elimination (RFE) is a feature selection method that removes the least relevant features for ML models until the specified number of features is reached. The RFECV technique in scikit-learn — recursive feature elimination with a 5-fold cross-validation — was employed to identify the most relevant features for ML studies. The model performance measured by cross-validation Q² scores is shown in Fig. 5. A subset of 12 features yielded the best statistical results with ${\text{Q}}_{\text{cv}}^2=0.645$.

Fig. 5.

Cross-validation scores (Q²) and the number of features selected by RFECV. The blue line indicates the optimal 12 features selected by this process.

These features include 1 physical property descriptor (mr), 1 bond count descriptor (opr_nrot), 1 Hűckel theory descriptor (h_ema), 1 partial charge descriptor (PEOE_VSA+0), 3 subdivided surface area descriptors (SMR_VSA4, SlogP_VSA3, SlogP_VSA7), and 5 adjacency and distance matrix descriptors (balabanJ, BCUT_PEOE_0, BCUT_PEOE_3, GCUT_SLOGP_0, GCUT_PEOE_0). Brief descriptions for these descriptors are shown in Table 1. Their maximum and minimum values define the applicability domain (AD) for the QSAR models. The AD is defined as “a theoretical region in the chemical space constructed by both the model descriptors and modeled response” [61]. The concept of the AD resonates with the Similar Property Principle described above and is highly recommended by the OECD to define the model limitations related to its chemical space and biological response domain. Main methods for defining a predictive QSAR AD are based on physical-chemical properties, molecular fingerprints or fragments, and the response space [62, 63]. Different similarity metrics with respect to the model’s training set have also been employed, including range- and distance-based methods. Each AD approach has its own strengths and limitations, and depends on the purpose of the studies. In the present study, the AD is defined using the physical-chemical properties (12 selected descriptors) and range-based similarity metric and pKi values (4.18–10.22) (Table 1).

Table 1.

Descriptions of 12 selected features by the RFECV procedure.

Descriptor Name	Category	Description	Max	Min
mr	Physical Properties	Molecular refractivity	15.36	8.68
h_ema	Hückel Theory	Sum of hydrogen bond acceptor strengths	13.81	3.47
opr_nrot	Bond Counts	Number of rotatable bonds	12	2
PEOE_VSA+ 0	Partial Charge	Sum of van der Waals (vdW) surface areas of all atoms with PEOE partial charges in the range [0.00,0.05]	208.36	0
SMR_VSA4	Subdivided Surface Area Descriptors	Sum of vdW surface areas of all atoms with molar refractivity in the range [0.39,0.44]	57.39	0
SlogP_VSA3		Sum of vdW surface areas of all atoms with logP in the range [0.35,0.39]	152.49	0
SlogP_VSA7		Sum of vdW surface areas of all atoms with logP > 0.56	294.18	24.45
balabanJ	Adjacency and Distance Matrix	Balaban’s connectivity topological index	1.85	0.98
BCUT_PEOE_0	Descriptors	The smallest eigenvalues in the BCUT_PEOE calculations. The BCUT descriptors are calculated from the eigenvalues of a modified adjacency matrix	−2.26	−2.9
BCUT_PEOE_3		The largest eigenvalues in the BCUT_PEOE calculations.	2.88	2.26
GCUT_SLOGP_0		The smallest eigenvalues in the GCUT_SLOGP calculations. The GCUT descriptors are calculated from the eigenvalues of a modified graph distance adjacency matrix	−0.73	−1.37
GCUT_PEOE_0		The smallest eigenvalues in the GCUT_PEOE calculations	−0.77	−0.89

3.4. Hyper-parameter optimization

After the most relevant features were selected, the 5-fold cross-validation grid search technique (GridSearchCV) was employed to optimize three hyper-parameters for the RF models on the training set: n_estimators — the number of trees in the forest; max_features — the number of features to consider when searching for the best split; max_depth — the number of splits that each decision tree is allowed to make. The optimized hyper-parameters were: n_estimators = 500; max_features = ‘sqrt’; max_depth = 12. The resulting RF QSAR model demonstrated excellent stability with R² = 0.925 and robust cross-validation predictivity with Q² = 0.659 with low prediction errors (RMSE = 0.311 and MAE = 0.240; RMSE_cv = 0.672 and MAE_cv = 0.506).

3.5. Model evaluation

In QSAR studies, cross-validation techniques are commonly used to assess the predictive ability of the model. Although a high value of Q² is necessary and important, it is not a sufficient condition for a model to possess high predictivity. To further evaluate the predictive power of the QSAR model, an external test set of 270 compounds, which were not included for model generation, was submitted for prediction of binding affinity by the RF QSAR model based on the optimized hyper-parameters. A robust predictivity was observed for the test set with good prediction statistics (${\text{R}}_{\text{pred}}^2=0.712$, RMSE_pred = 0.694, MAE_pred = 0.450).

Since random partitions were adopted in the current study, we sought to evaluate how randomness affects model performance. Fifty independent RF models using the optimized hyper-parameters but with different random seeds were conducted for the training set and then applied to the test set. The results for the 50 QSAR models, summarized in Table 2, demonstrated excellent internal stability and predictive power with R² = 0.915 – 0.927 and ${\text{R}}_{\text{pred}}^2=0.601-0.733$, also indicating model statistics are less likely from coincidence. The prediction results for the QSAR model with random seed = 439713 are plotted in Fig. 6, showing strong correlation between experimental and predicted pKi values for the OX1R ligands. The distribution plot of the residuals (differences) between corresponding values of experimental and predicted pKi reveals that the QSAR model prediction errors are randomly distributed around the zero line (Fig. 7), denoting the absence of systematic errors that usually arise from biased measurement or calculation. Only 4 out of 1080 (0.37%) compounds in the training set were predicted with residuals > 1 log unit, demonstrating the high quality of the dataset of OX1R ligands for the present QSAR study. A final QSAR model was constructed for the cleaned dataset of 1350 OX1R ligands (R² = 0.920, RMSE = 0.322, and MAE = 0.247) and employed for the enrichment study and virtual screening.

Table 2.

QSAR model statistics for 50 independent runs with different random seeds.

	Training Set			Test Set
	R2	RMSE	MAE	${\text{R}}_{\text{pred}}^2$	RMSEpred	MAEpred
mean	0.921	0.319	0.244	0.679	0.645	0.482
sd*	0.003	0.061	0.004	0.028	0.193	0.022
min	0.915	0.305	0.235	0.601	0.579	0.444
median	0.920	0.320	0.244	0.680	0.645	0.480
max	0.927	0.330	0.252	0.733	0.738	0.559

^*standard deviation.

Fig. 6.

Scatter plots of QSAR model predicted and experimental pKi values. Left: The training set; Right: The test set. The fitted lines are shown in each plot.

Fig. 7.

Scatter plot for residual values for the training set. The red line is for the residual values = 0. Residual = Exp. pK_i − Pred. pK_i.

Thorough validation is a crucial step in the development of QSAR models. In addition to using an external test set to validate our models, we also conducted an enrichment study using a decoy dataset to further evaluate the predictivity of our QSAR model. From the curated OX1R ligand set, 126 highly active compounds (pKi ≥ 8.0) were identified. Subsequent clustering analysis with MACCS fingerprints and Tanimoto coefficient (0.8) for these ligands resulted in 48 diverse clusters. A decoy dataset for the 48 clusters of active OX1R ligands was generated on the DUD-E server (http://dude.docking.org) [46,47]. Each active ligand collected 50 decoy molecules that possess similar physicochemical properties but distinct molecular topology. Our QSAR model was employed to predict the pKi values for the entire decoy dataset including actives and decoys. Enrichment factor (EF) and goodness of hit list (GH) score, defined by Eqs. (4) and (5), were used as metrics to evaluate the predictivity of the QSAR model. Using pKi = 8.0 (Ki = 10 nM) as the cutoff to separate actives and inactives, 11 compounds predicted as actives by the QSAR model were all true actives, yielding EF= 51 and GH= 0.81. If a half-log unit is tolerated in prediction, using pKi = 7.5 (Ki = 30 nM) as the cutoff, 32 are true actives among 40 predicted actives by the QSAR model, yielding EF= 41 and GH= 0.76. For this decoy dataset, EF is equal to 2 for a random selection. These results sindicate that our QSAR model is 20–25 times more likely to identify active OX1R compounds than a random selection. Meanwhile, GH> 0.70 suggests that our model can discriminate actives from inactives during a virtual screening. Taken together, this enrichment study strongly demonstrates that our RF-based QSAR is highly predictive and discriminative at finding actives and separating them from inactives.

3.6. Virtual screening and biological testing

Validation studies using the external test set and enrichment study demonstrated the robustness and excellent predictivity of our QSAR models. As the first case study, our QSAR model was used for the virtual screening of the filtered DrugBank database, a database of 1615 drugs approved by the FDA. Three FDA-approved orexin receptor antagonists (suvorexant, lemborexant and daridorexant) ranked among the top 4 of the hit list, indicating that our QSAR model can identify true active ligands for OX1R. We visually checked the top 100 compounds predicted by the QSAR model and acquired four drugs (macitentan, zafirlukast, isavuconazole, and cabozantinib) for in vitro testing based on their commercial availability and structural diversity. Initial biological evaluation for human OX1R binding affinity was conducted at duplicate in human recombinant CHO-S cells at a concentration of 10 μM according to the protocol previously reported [51]. [¹²⁵I] Orexin A (0.1 nM) was used as the radioligand, while unlabeled SB-334867 (1.0 μM) was used to determine non-specific binding for the OX1R. Both isavuconazole and cabozantinib exhibited 30–40% binding affinity for the OX1R at tested conditions (Table 3); solubility issues with both compounds prohibited screening at higher concentrations. In contrast, macitentan and zafirlukast showed minimal (~10%) binding affinity for OX1R in this binding assay.

Table 3.

Two FDA-approved drugs exhibited binding affinity for human OX1R in radiolabeled binding assay^a.

Name	%Binding	Indication	Structure
Isavuconazole	31	Antifungal
Cabozantinib	38	Anticancer

^{a :}Radiolabeled binding assay of the OX1R was performed by Eurofins Panlabs Discovery Services in human recombinant CHO-S cells at 10 μM of drug concentration using [¹²⁵I] Orexin A as the radioligand according to the protocol previously reported [51].

To our knowledge, neither isavuconazole nor cabozantinib has been reported to exhibit any binding to OX1R. Isavuconazole has been approved as an antifungal medication for invasive and severe infection such as aspergillosis and mucormycosis [64]. It acts as an inhibitor for lanosterol 14α-demethylase, whose inhibition could disrupt the membrane structures of fungal cells [65]. On the other hand, cabozantinib is a broad-spectrum inhibitor of the tyrosine kinases such as c-Met, VEGFR2, and Axl [66], and has been approved to treat various digestive cancers, including thyroid cancer, renal cell carcinoma, and hepatocellular carcinoma [67,68]. This is interesting, as OX1Rs (but not OX2Rs) are expressed in human digestive cancers but not in corresponding healthy epithelia [69], opening the possibility that some of the therapeutic efficacy of cabozantinib might be due to its (previously unappreciated) actions at OX1R. Consistent with this, almorexant, which acts at both OX1R and OX2R, has previously been reported to have anti-tumoral properties in vitro, in vivo, and ex vivo [70]. Moreover, fatigue and reduced appetite are among the most common adverse events associated with cabozantinib treatment [71], which aligns with a role for the orexin system in arousal and feeding, respectively. Thus, although considerable additional invetigation is required, these new findings may help to further elucidate the mechanisms underlying the therapeutic effects and side effect profiles of these two drugs, and might even raise opportunities for their repurposing for other clinical indications.

We further investigated why macitentan and zafirlukast were false actives from the virtual screening. We compared the values of 12 molecular descriptors selected for QSAR modeling and found both macitentan and zafirlukast have one feature falling out of the AD defined by the minimum and maximum values of each descriptor (Table 1). On the other hand, all features for both isavuconazole and cabozantinib fall within the AD. We therefore speculate that issues with the AD may be partially responsible for the inaccurate predictions by our model. The AD plays an important role in estimating the uncertainty of predictions for test compounds based on how similar they are to the training compounds used for developing the QSAR model. Compounds flagged as AD non-compliant should be de-prioritized for further studies. In our case, future efforts should determine if new compounds fall within the ADs of all descriptors used for model development before biological testing.

4. Conclusions

Molecular descriptor-based QSAR models were developed to gain insights into the structure-activity relationships and, ultimately, to guide virtual screening and rational design, of novel OX1R ligands. Best practices recommended by the OECD were followed to develop statistically robust and predictive QSAR models. A large and diverse dataset of OX1R ligands was curated by pooling ligand information from ChEMBL. The ChEMBL ID for human OX1R was used as the query to search the ChEMBL database, which produced more than 5000 bioactivity data entries. KNIME workflows were constructed to rigorously clean and filter the heterogeneous bioactivity dataset downloaded from ChEMBL, resulting in a refined dataset of 1369 structurally diverse OX1R ligands with the bioactivity pKi. PCA was then conducted to eliminate structural or activity outliers from the dataset assembled for the QSAR studies. Preliminary QSAR studies suggested RF generated the best statistics among four ML methods (RF, XGBoost, SVM-Linear and SVM-RBF) for this OX1R ligand dataset. Feature elimination with 5-fold cross-validation was performed to select twelve out of 216 MOE 2D molecular descriptors as the most relevant for the current QSAR modeling. Hyper-parameters for the RF algorithm were further optimized by 5-fold cross-validation grid search technique. Descriptor-based QSAR models were constructed for the selected features and optimized RF hyper-parameters by randomly dividing the modeling set into a training and test set with the ratio of 4:1. In order to evaluate how random partitions affect the model performance, 50 independent RF models with different random seeds were conducted for the training set and then applied to the test set, indicating model statistics are less likely due to coincidence with excellent internal stability (R² = 0.915 – 0.927) and predictive power (${\text{R}}_{\text{pred}}^2=0.601-0.733$).

Enrichment studies were conducted to further evaluate the predictivity of our QSAR models by using a decoy set for the most potent OX1R ligands generated by the DUD-E server, confirming their excellent predictivity with results of EF > 40 and GH > 0.70. To further evaluate the predictive power of our models, QSAR model based virtual screening was carried out against the filtered DrugBank database. Three FDA-approved orexin receptor antagonists (suvorexant, lemborexant and daridorexant) ranked among the top 4 of identified ‘hits’, indicating that our QSAR model can accurately identify true active ligands for OX1R. Two FDA-approved drugs, isavuconazole and cabozantinib, were found to exhibit 30–40% binding to the OX1R via in vitro radiolabeled binding assays under the test conditions. To our knowledge, this is the first report of actions at the OX1R for either of these compounds, and thus may be useful in further understanding the mechanisms for their therapeutic uses and side effects.

It should be emphasized that these QSAR models are statistically valid only within their applicability domains with respect to their range of binding affinities (pKi = 4.18–10.22) and the chemical space represented by molecular descriptors of the OX1R ligands in the present study. Furthermore, our QSAR models can predict only binding affinities (pKi) for OX1R but not the ligands’ functionality as a receptor agonist vs. antagonist. Moreover, our models cannot predict the ligand binding selectivity for OX1R vs. OX2R. This will be important for future work to address, as some physiological processes are preferentially mediated by signaling at OX1R vs. OX2R [7,72,73]. Indeed, we intend to extend our work here to further develop QSAR models to identify compounds that act at OX2R. Considering the high degree of similarity between the two orexin receptors [1], these QSAR models are expected to share similar features and overlapping ADs. hAny unique features might be useful in determining ligand specificity for either OX1R or OX2R.

In recent years, the field of QSAR has witnessed a significant advance in chemical representation and ML methods. Beyond the conventional molecular descriptors, the inclusion of molecular graphs and sophisticated fingerprints has empowered the encoding of intricate structural information [74,75]. Furthermore, quantum chemical descriptors have added a new dimension to chemical representations, offering a more realistic portrayal of molecular interactions and properties [76,77]. Meanwhile, the emergence of deep learning has revolutionized QSAR by allowing the development of highly complex and sophisticated models. Deep neural networks (NNs), such as convolutional and recurrent NNs, have demonstrated exceptional performance in extracting features from molecular data [78,79]. While these NN-based models excel at capturing complex SARs, they often demand large datasets and can be perceived as ‘black boxes’ with limited interpretability. In contrast, traditional QSAR models are generally more interpretable and computationally efficient but may struggle to capture non-linear SARs between molecular structures and activities. The performance of NN-based QSAR models in property predictions has been widely studied, yielding mixed results [60,80–82]. Comprehensive comparisons between traditional and NN-based ML methods indicate that no single method has conclusively proven superior to the other [58,59]. The selection of chemical representations and ML methods depends on many factors, including the availability and quality of data, as well as the objectives of the study.

In our current study, we present a rigorous data curation workflow and a highly predictive QSAR model based on a traditional RF method and only 12 physicochemical properties. This model offers ease of interpretation, making it particularly appealing to medicinal chemists. Furthermore, it can be readily adapted to address various other pharmacological targets. Furthermore, this report represents the first successful attempt to develop comprehensive QSAR models for a large dataset of more than 1300 structurally diverse OX1R ligands. These efforts should prove useful for the identification of new drug leads and for prediction of their OX1R binding affinity (pKi) prior to the resource-demanding tasks of chemical synthesis and experimental biological evaluation. Moreover, these QSAR models might be useful as a filter to exclude molecules that exhibit off-target activity at the OX1R in drug discovery campaigns.

Abbreviations:

QSAR

quantitative structure-activity relationship

OX1R

orexin 1 receptor

OX2R

orexin 2 receptor

GPCR

G-protein coupled receptor

PDB

Protein Data Bank

MOE

molecular operating environment

PCA

principal component analysis

RF

random forest

XGBoost

extreme gradient boosting

SVM

support vector machine

RBF

radial basis function

RFECV

recursive Feature Elimination with cross-validation

RMSE

root mean squared error MAE, mean absolute error

EF

enrichment factor

GH

goodness of hit list

AD

applicability domain

Data Availability

The DrugBank database can be found at https://www.drugbank.com/. The ChEMBL database can be found at https://www.ebi.ac.uk/chembl/.