Exploring the Potential of Adaptive, Local Machine Learning in Comparison to the Prediction Performance of Global Models: A Case Study from Bayer’s Caco-2 Permeability Database

Abstract

Machine learning (ML) techniques are being widely implemented to fill the gap in simple molecular design guidelines for newer therapeutic modalities in the extended and beyond rule of five chemical space (eRo5, bRo5). These ML techniques predict molecular properties directly from the structure, allowing for the prioritization of promising compounds. However, the performance of models varies greatly among ML use cases. A molecular property for which achieving sufficient performance in generalizing global models still remains difficult is Caco-2 permeability. Especially within the lower permeability ranges, which are specific for larger molecules belonging to the e/bRo5 space, accurate regression predictions have proven to be challenging. The present study, therefore, identifies a suitable combination of ML algorithm and descriptors, consisting of the LightGBM algorithm and RDKit molecular property descriptors, to predict Caco-2 permeability very efficiently by a simple global model. An additionally introduced local model uses the same algorithm and descriptors but selects its training data based on Tanimoto fingerprint similarity to match the individual test compound’s structure. Evaluation of this adaptive model, by systematically varying the number of most similar structures for training, shows that, in comparison to the global model, there was only marginally improved performance with specific training data constellations. These random improvements indicate that deriving general rules for local model parametrization is not possible a priori for the chosen algorithm and descriptor combination, and preselecting training data does not seem advantageous over global ML based on all available data, while creation of more data-efficient models was generally proven to be possible.

Introduction

In the past decades, the discovery and development of orally bioavailable small-molecule drugs has focused on the virtues of Lipinski’s rule-of-five (Ro5) chemical space.¹ In recent years, however, there is an emergence of novel drug targets that require drug candidates from a chemical space that lies more and more outside of the Ro5 constraints,²⁻⁵ a trend that we have also seen in the evolution of the Bayer compound library.⁶ With the emergence and first successes of these new modalities,⁷⁻⁹ whose molecular weights are beyond what was long deemed to be reasonable, drug discovery scientists can no longer benefit from Lipinski’s Ro5¹⁰ to guide them toward compounds with high probability of good oral bioavailability. Because of the necessary move away from classical small molecules and the absence of molecular design guidelines in the extended and beyond rule of five (e/bRo5) space,¹¹ there is an increasing need to predict key properties required for high oral bioavailability in the early drug discovery phases. In line with our previous analysis, intestinal permeability is one of the key parameters that should be addressed very early on.⁶ Building on this work, where we have identified general trends in molecular properties that can be used to guide the optimization of permeability, we have now set out to explore machine learning (ML) algorithms for predicting in vitro permeability as measured in our in-house Caco-2 permeability assay to aid the design of new chemical molecules. By anticipating permeability estimates directly from the chemical structure before synthesizing a compound, medicinal chemists can avoid unnecessary synthesis of compounds with a high probability of insufficient permeability and thereby enhance the time, cost and resource efficiency of the drug discovery process.

Caco-2 permeability is known to be notoriously hard to predict very accurately because of the inherent variability of the assay’s experimental results.¹²⁻¹⁶ This well-known shortcoming is opposing the dependence of ML algorithms on reliable data to build high performance prediction models. Therefore, we have evaluated different options in this study to move beyond the conventional approach to permeability prediction that utilizes a compounds structure which gets fed into a single, global quantitative structure–property relationship (QSPR) model¹⁷⁻²⁰ that is informed by all data in the database to predict an absolute permeability value estimate. Since in the drug discovery phase of projects there is a considerable amount of standardized Caco-2 assay data, typically coming from slight variations of a chemical scaffold, we thus have assessed predictive models that are based on a comparably rather small but structurally more related chemical space as a training set to explore whether it may produce equally good or even better predictions when working in the structurally more confined and more relevant space of a specific project. We hypothesized that this approach has the potential to outperform global models, which give good predictions for random test sets but depend on large amounts of data from different places inside the available chemical space of the around 36,000 structures with well-defined Caco-2 data in the Bayer collection. We, therefore, have developed an adaptive model that is based on the principle of instance selection and only uses a similarity-based subset of the data available during every prediction.

To evaluate this alternative approach, we were first seeking a combination of ML algorithm and molecular descriptors that is suitable for permeability prediction in general. We identified the combination of the highly efficient LightGBM algorithm with molecular descriptors from the RDKit package to give the best performance in a global model while at the same time being highly suitable for the evaluation of the local, adaptive model.

Methods

In the following, tools and techniques utilized for data preprocessing, visualization, evaluation of algorithms and descriptors, establishing of structure–property relationships, and model building are described. Additionally, formulas for calculation of used metrics are provided in this chapter and distributions of the utilized data can be found in the Supporting Information Figure S1 and Figure S2. These data are internal training data from the steadily growing Bayer in-house database consisting of all available Caco-2 assay results up to 15-Jan-2024, extracted from that internal project-related testing data and 10% randomized internal testing data, as well as external training and testing data. Both internal and external randomized test sets contain molecules that have molecular weights associated with Ro5, eRo5, and bRo5 compounds, while the internal project-specific test only consists of structures belonging to the e/bRo5 space. The used integrated development environment is Visual Studio Code with Python²¹ (v3.10.8) as programming language.

Data Preparation and Visualization

To obtain a passive apparent permeability (P_app,passive) from our bidirectionally measured data (P_app,AB, apical to basolateral and P_app,BA, from basolateral to apical), eq 1(22) is used as conventional practice to calculate the arithmetic mean without the impact of efflux transporters:

1

Preparation of the internal training/testing data includes taking the arithmetic mean of Caco-2 experimental measurements (if multiple available) to get a single value per compound and exclusion of measurements with unusual test compound concentrations in the assay (≠ 2 μmol/L). Only results are included where the assay was performed with a standard BSA concentration and where no additional inhibitor was given. Additionally, measurements with recovery rates of more than 200% or less than 50% were excluded. Ultimately, 33.398 structures are available in total. Permeabilities are categorized according to internal standards by labeling compounds with less than 1 × 10^–6 cm/s as low permeable, up to 7 × 10^–6 cm/s as medium permeable and compounds which showed a permeability of more than 7 * 10^–6 cm/s as highly permeable. Resulting permeabilities are transformed via a decadic logarithm. Data is prepared and transformed by “pandas”²³ (v2.1.2) and “numpy”²⁴ (v1.26.1), while “seaborn”²⁵ (v0.13.0), “matplotlib”²⁶ (v3.8.1), and “plotly”²⁷ (v5.18.0) are used for visualization. Structures encoded as simplified molecular-input line-entry system (SMILES) notation as well as molecule objects are standardized using RDKit’s “StandardizeSmiles()” and “Cleanup()” functions, respectively. The full set of 209 RDKit descriptors is calculated by utilizing the output of RDKit’s “Descriptors.descList” function.

The final global model’s algorithm and descriptor combination performance is evaluated by applying it to a public benchmark set, which was compiled by Wang et al.²⁸ and is made accessible for benchmarking via Therapeutics Data Commons (TDC).²⁹ To access the data “PyTDC” (v0.4.1) is used with the group “Caco2_Wang”. Additionally, the global model is tested on a random 10% test set of internal data. The adaptive model uses a project-related internal test set of 189 more homogeneous compounds belonging to the e/bRo5 class for evaluation instead. Overviews of molecular property distributions for the internal and external training and testing data sets can be found in the supplements (Figure S1 and Figure S2).

Evaluation of Algorithms and Descriptors

Machine learning algorithms for regression are implemented by utilizing the packages “lightgbm”³⁰ (v4.1.0) for LightGBM and “scikit-learn”³¹ (v1.3.2) for support vector machines (SVMs) as well as multilayer perceptrons (MLPs), while descriptors are calculated by “rdkit”³² (v2023.9.1) and “padelpy”³³ (v0.1.15) from molecular structures encoded as SMILES. For details of so-called continuous and data-driven descriptor (CDDD)³⁴ generation, see the publicatio by Winter et al. Scikit-learn is also used to compute the metrics MAE, RMSE and R². During algorithm evaluation, no standard settings were changed for SVMs, except for modification of the kernel to a radial basis function or a polynomial one. MLPs were implemented with two layers containing 1024 and 600 neurons as deep MLP and only a single 600 neurons layer as shallow MLP, while for LightGBM the maximum number of leaves for a base learner was set to 35, the number of boosted trees to fit increased to 2000 and the learning rate decreased to 0.05. To evaluate the global model’s performance realistically in a project work-like scenario, time-split as implemented in scikit-learn’s TimeSeriesSplit function³⁵ is chosen for cross-validation, which means sorting the compounds by date, selecting a number of splits and an amount for samples in the unseen test set (here 1000). This decides in consequence an increasing number of compounds in the training data per split since for each split the previous testing data is added to the training data. Like that, multiple retrainings every 1000 compounds and following model evaluations by test data from future projects are simulated.

Structure–Property Relationships

The structure–property relationships are evaluated by applying the t-distributed stochastic neighbor embedding (t-SNE) technique (angle = 0.3, perplexity = 100, n_components = 3) as well as multidimensional scaling (MDS) of extended connectivity fingerprint (ECFP) descriptors (radius 3, 2048 bit) to two components with default settings as implemented in “scikit-learn” (v1.3.2).

Adaptive Local Caco-2 Regression Model

For the adaptive Caco-2 regression model, a custom instance selection is performed: (1) Tanimoto similarities between all compounds and a single test compound are computed by “rdkit” (v2023.9.1) based on ECFP6 (2048 bit) descriptors and equation (2):³⁶

2

Variables a and b are the amount of “on” (= 1) bits in molecule A and B, respectively, while c refers to the number of bits that are shared by molecule A and B. (2) After computing all similarities, the test compound is deleted from the training data pool and “lightgbm” (v4.1.0) is used to train a regression model on all remaining compounds matching a user-defined similarity threshold or a predefined minimum amount of most similar structures. (3) Finally, a value is predicted for the test compound and the algorithm starts the cycle again to check for similar compounds in the full pool to train a new model for the next test compound. This makes it effectively a realistic, project-like “leave-one out” testing scenario where single new compound structures are tested. Accordingly, for a project-specific test set of 189 compounds, 189 different models with varying amounts of training structures are created and used for predicting the permeability of the respective compounds. The concept of pool-based selection of training data has already been established as so-called “active learning” in a slightly different way than implemented here. In active learning, the algorithm selects data based on criteria from a pool, to then be labeled by the user and finally to be fed to the model, while in our case the data is already “labeled” with continuous values (the assay results) and automatically fed into our adaptive model based on a criterion (Tanimoto coefficient) without user interaction.

Metric Calculations

Model performance is evaluated on decadic logarithm-scaled data by mean absolute error (MAE, (3)), absolute average fold error (AAFE, (4)), root-mean-square error (RMSE, (5)) and coefficient of determination (R², (6)). The variable ŷ_i is thereby a predicted value, while y_i is the corresponding measured value from the Caco-2 assay and y̅ the mean over all measured values.

3

4

5

6

Results

To arrive at the adaptive model, first a global model was developed, which was then modified by excluding dissimilar compounds from the training data based on each test compound. The following sections describe the results that arose during the steps to obtain the adaptive model. This includes evaluating different machine learning algorithms and descriptors as well as their predictivity for a randomized internal test set via the global model and for a project-specific test set via the adaptive model. Additionally, results from time-split cross-validation for the global model and evaluation of structure-permeability relationships are presented, as well as for the relationship between the amount of most similar structures in the training data and performance for the adaptive model.

Algorithm Evaluation

By systematic comparison of different machine learning algorithms and molecular descriptors, the best optimal combination for efficient Caco-2 permeability regression has been identified. This combination consists of LightGBM as the regression framework, while the input descriptors are a selection of 209 molecular property descriptors calculated by the cheminformatics package RDKit. The calculated descriptors are a combination of, e.g., molecular weight, polar surface area, counts of specific atoms and functional groups, as well as connectivity indices and electrotopological state descriptors. This descriptor selection can be grouped into about 15 different groups of calculated descriptors. These groups contain electronic state (e-state) (e.g., “MaxAbsEStateIndex”) descriptors, descriptors for drug-likeness (e.g., “qed”), molecular weight (e.g., “MolWt”), partial charge (e.g., “MaxPartialCharge”), fingerprint density (e.g., “FpDensityMorgan1”), BCUT values (e.g., “BCUT2D_MWHI”), topological indices quantifying molecule complexity (e.g., “AvgIpc), connectivity indices (e.g., “Chi0”), surface area descriptors (e.g., “LabuteASA”, “TPSA”), partial equalization of orbital electronegativity, molecular refractivity and log octanol/water partition coefficients based on approximate van der Waals surface area (e.g., “PEOE_VSA1”, “SMAR_VSA1”, “SlogP_VSA1”), molecular property counts (e.g., “FractionCSP3”), lipophilicity (e.g., “MolLogP”), and functional group counts (e.g., “fr_Al_COO”).

The gradient-boosted decision tree algorithm LightGBM and a support vector machine with radial or polynomial basis function performed best (see Table 1, MAE: 0.357, R²: 0.620 and MAEs: 0.346, R²s: 0.627, respectively), followed by a two-layer (“deep”) multilayer perceptron (MAE: 0.354, R²: 0.621), a “shallow” single layer perceptron (MAE: 0.356, R²: 0.614) and finally LightGBM (MAE: 0.357, R²: 0.620). For 10-fold random split cross-validation results, see Table S1. Since the performance of LightGBM and SVM were only negligibly different, the final algorithm was decided to be LightGBM because of its extremely short training and prediction times: only ∼3 s including training on 30.058 compounds and the following prediction of 3340 test compounds. The SVMs took 4490× (polynomial basis function) and 520× (radial basis function) longer while MLPs took 79× (deep) and 66× (shallow) longer for the same task on a Laptop with Intel Core i7-11800H CPU at 2.30 GHz and 32 Gb RAM (3200 MHz).

Table 1. Comparison of Algorithm Performance with ECFP4 (1024 bit) Descriptor for a Randomized Internal 10% Test Set (n_Training = 30.058 and n_Testing = 3340)^a.

Algorithm	tTrain + tTest	MAE/AAFE	RMSE	R2	1.5-Fold	2-Fold	10-Fold
SVM (poly)	224.50 min	0.346/2.22	0.474	0.627	36.3%	56.3%	94.8%
SVM (rbf)	26.02 min	0.346/2.22	0.474	0.627	36.5%	56.3%	94.7%
MLP (deep)	3.93 min	0.354/2.26	0.478	0.621	35.4%	54.5%	95.1%
MLP (shallow)	3.32 min	0.356/2.27	0.482	0.614	35.2%	54.7%	95.0%
LightGBM	0.05 min	0.357/2.27	0.479	0.620	34.8%	53.8%	94.9%

^aEvaluated regression algorithms are LightGBM, support vector machine (SVM) with polynomial (poly) and radial basis function (rbf) and multilayer perceptron (MLP) with a single (shallow) or two (deep) hidden layers while ECFP4 (1024 bit) is used as descriptor for a randomly selected internal 10% test set. Performance metrics are mean absolute error (MAE), root-mean-square error (RMSE), coefficient of determination (R²) and fraction of predictions within 1.5-, 2-, and 10-fold error.

Descriptor Evaluation

The corresponding metrics for the evaluated descriptors show that the RDKit descriptors performed best (see Table 2, MAE: 0.318, R²: 0.683), closely followed by the PaDEL descriptors (MAE: 0.327, R²: 0.668). The worst performance was a result of using topological fingerprints with path-length of 1 at 2048 bits (MAE: 0.511, R²: 0.277) while other fingerprint configurations lead to considerably better performance. Structural MACCS fingerprints (MAE: 0.378, R²: 0.578) and CDDD (MAE: 0.355, R²: 0.625) showed a performance in between. Accordingly, the final model consists of LightGBM as the regression algorithm and all 209 molecular descriptors as calculated by RDKit. Results of recursive feature elimination (5-fold cross validation) to determine optimal feature amount resulted in 150 features giving the optimal performance (MAE: 0.324, see Figure S3 and List S1). Since the performance is only slightly increased in comparison to the full feature set (ΔMAE = −0.0055), the full set was kept for all further analyses, including the local models. For feature importance see Figure S4 and for 10-fold random split cross-validation see Table S2.

Table 2. Comparison of Descriptor Performance with LightGBM for a Randomized Internal 10% Test Set (n_Training = 30.058, n_Testing = 3340)^a.

Descriptor	Configuration	MAE/AAFE	RMSE	R2	1.5-Fold (%)	2-Fold (%)	10-Fold (%)
ECFP (circular)	2048 Bit, r = 5	0.365/2.32	0.489	0.604	33.6	52.8	95.2
	2048 Bit, r = 4	0.362/2.30	0.485	0.610	33.2	52.8	95.3
	2048 Bit, r = 3	0.357/2.28	0.479	0.620	34.1	53.8	95.6
	2048 Bit, r = 2	0.354/2.26	0.476	0.625	34.8	54.5	95.0
	2048 Bit, r = 1	0.352/2.25	0.474	0.627	34.8	55.2	95.7
	1024 Bit, r = 5	0.377/2.38	0.500	0.585	32.0	50.4	94.9
	1024 Bit, r = 4	0.373/2.36	0.497	0.591	33.0	52.3	95.2
	1024 Bit, r = 3	0.366/2.32	0.488	0.606	33.2	52.8	95.1
	1024 Bit, r = 2	0.357/2.28	0.479	0.620	34.8	53.8	94.9
	1024 Bit,r = 1	0.350/2.24	0.472	0.630	36.1	55.0	95.3
RDKit (topological)	2048 Bit, l = 6	0.352/2.25	0.480	0.617	36.3	55.7	95.2
	2048 Bit,l = 5	0.349/2.23	0.475	0.625	36.7	55.8	95.3
	2048 Bit, l = 4	0.350/2.24	0.477	0.623	36.7	55.7	95.3
	2048 Bit, l = 3	0.366/2.32	0.495	0.594	34.1	53.4	94.4
	2048 Bit, l = 2	0.413/2.59	0.549	0.499	29.3	47.3	92.9
	2048 Bit, l = 1	0.511/3.24	0.660	0.277	22.9	37.8	88.3
	1024 Bit, l = 5	0.353/2.25	0.481	0.617	35.9	55.5	95.0
	1024 Bit, l = 4	0.354/2.26	0.482	0.614	35.8	56.2	95.2
	1024 Bit, l = 3	0.365/2.32	0.494	0.596	34.6	53.3	94.7
	1024 Bit, l = 2	0.413/2.59	0.550	0.498	29.4	47.7	92.5
	1024 Bit, l = 1	0.509/3.23	0.658	0.282	23.6	38.0	88.4
MACCS (structural)	166 Bit	0.378/2.39	0.505	0.578	32.3	51.6	94.0
RDKit	209×	0.318/2.08	0.437	0.683	40.7	60.6	96.4
PaDEL	639×–1875×	0.327/2.12	0.447	0.668	38.1	58.1	96.4
CDDD	Bayer internal	0.355/2.26	0.476	0.625	34.6	54.1	95.7

^aEvaluated descriptors are extended connectivity fingerprint (ECFP) circular fingerprints with different bit lengths and radii (r), rational discovery kit (RDKit) topological fingerprints with different bit lengths and path lengths (l), as well as molecular property descriptors derived by RDKit (RDKit), molecular access system (MACCS) structural bit keys, parallel descriptor calculation (PaDEL), and continuous and data-driven descriptors (CDDD) in combination with LightGBM as algorithm for a randomly sampled internal test set. Performance metrics are mean absolute error (MAE), root-mean-square error (RMSE), coefficient of determination (R²) and fraction of predictions within 1.5-, 2-, and 10-fold error. Underlined are the best performing configurations for the respective topological/circular fingerprint. PaDEL was able to calculate 639 descriptor columns for all structures, while the full 1875 descriptors were available for only 23893 structures at a conversion timeout of 30 s.

Global Model External Evaluation (TDC Test Set)

Having identified a suitable algorithm and descriptor selection (full RDKit set), the combined performance was first evaluated by the TDC benchmark set to be able to compare the performance to other published models. The MAE, AAFE, RMSE, and R² for the benchmark set are 0.286, 1.93, 0.370, and 0.709, respectively, while 42.3% of predictions showed less than a 1.5-fold error, 59.9% of predictions are within 2-fold and none of the predicted compounds had an error bigger than 10-fold (Figure 1). With a MAE of 0.286 and corresponding AAFE of 1.93 for the benchmark set, the performance was similar, if anything only slightly worse (<4%) than the two best performing models according to TDC at the time.

Figure 1.

Comparison of total apparent Caco-2 permeabilities from the apical to basolateral side (P_app,passive) to experimentally measured values for an external TDC test set (n_Training = 728; n_Testing = 182). Predicted by the global LightGBM model with RDKit descriptors.

Global Model Internal Evaluation (Time Series Split)

The suitability of this algorithm and descriptor combination for prediction of Caco-2 permeability is also indicated by the steadily sufficient performance for data over the last 20 years, which becomes visible when applying the model to all available internal compounds that have corresponding Caco-2 results and cross-validating it by time-split. The resulting mean MAE of 0.393 (σ = 0.043) corresponds to an AAFE of 2.47. Looking at the corresponding R² range between 0.391 and 0.617 (σ = 0.070), a more distinct dependence on training and subsequently testing data becomes apparent (Figure 2).

Figure 2.

Coefficient of determination (R², right) and mean absolute error (MAE, left) evaluated by time series split for Caco-2 permeability data from the last ∼21 years (n_Testing = 1000).

Global Model Internal Evaluation (Randomized 10% Test Set)

In addition to the evaluation of the global model consisting of LightGBM and RDKit descriptors based on an external data set and internal time-split cross validation, the model was also challenged with a randomized internal test set consisting of 10% of the available data. This same internal test set that was used for algorithm and descriptor evaluation results in good performance for prediction of high permeable compounds, while medium and especially low permeable compounds tend to be overpredicted (MAE: 0.318, AAFE: 2.08, RMSE: 0.437, R²: 0.683, 1.5-Fold: 40.7%, 2-Fold: 60.6%, 10-Fold: 96.4%, Figure 3). An additional classification model trained and tested with the same internal data (see Figure S5 and Table S3) reproduces the same trend: a higher false-positive rate for classifying actually low permeable compounds as highly permeable (7.6% of predictions) than for classifying highly permeable as low permeable compounds (0.12% of predictions). This result is in line with the requirement for predictive models to have a low false-positive rate for classifying highly permeable as low permeable, to not accidentally remove actually very promising compounds from the drug discovery pipeline due to falsely predicted low permeability.

Figure 3.

Comparison of total apparent Caco-2 permeabilities (P_app,passive) to experimentally measured values for a randomized 10% internal test set (n_Training = 30058; n_Testing = 3340). Predicted by the global LightGBM model with RDKit descriptors.

The difference between AAFE performance on the benchmark set (AAFE_External,TDC = 1.93) and our internal data (mean AAFE_{Internal,Time-Split} = 2.47, AAFE_{Internal,10%-Test} = 2.08) indicates that difficulties in prediction of low permeable compounds are not a consequence of a poor choice of algorithm and descriptors, but rather that the model’s prediction bias toward highly permeable compounds originates most likely from the unbalanced composition of our training data and consequently results in deteriorated predictions.

Local Model Internal Evaluation (Project Test Set—Impact of Amount of Most Similar Training Structures)

By systematically varying the composition of training data through utilization of the Tanimoto coefficient as parameter for exclusion of dissimilar structures, the following Figure 4 was created to assess the impact of different amounts of most similar training structures on the model’s performance. Best performance according to MAE is achieved for n_Training = 10000 most similar appearing structures according to Tanimoto similarity (MAE: 0.406), while 100, 5000, and 7500 training structures lead to a slightly worse but in general alike MAE of 0.411.

Figure 4.

Mean performance of the adaptive, local model predicting permeabilities of the project-related test set (n = 189) according to root mean squared error (RMSE), mean absolute error (MAE), and coefficient of determination (R²), with respect to different amounts of included, most similar compounds as training data. The last data point (n_Training = 33397 structures) has all available structures included for training and is therefore equivalent to applying a global model to the project test set.

The R² metric on the other hand is best, but very modest in general, with 0.257 for only 100 most similar structures, followed by n_Training = 10000 (R²: 0.253) and n_Training = 7500 (R²: 0.241). Looking at the RMSE yields the same picture as R²: the best performance was achieved by only 100 most similar structures (RMSE: 0.538) followed by 10000 and 7500 structures (RMSE: 0.540 and RMSE: 0.544, respectively). The global model (33397 structures), on the other hand, only resulted in a mean MAE of 0.419 (R²: 0.239; RMSE: 0.545). This implies a relative performance increase of 3.2% according to MAE and a 7.5% increase according to R² between the best performing local model and the global model. Comparison of these results with an alternative training data selection via k-nearest neighbors (kNN)³⁷ vote (see Figure S6), shows that models trained with data selected via Tanimoto similarity outperform models where training data was selected via kNN. Especially for lower numbers of included structures for training, the difference in performance according to MAE, RMSE, and R² is substantial. This means our method of training data preselection allows the model to perform as good as a global model or even better, with only a limited number of included structures (e.g., 100), while preselection of training data via kNN fails to result in equally good models.

Discussion

In the present study, we have explored whether adaptive, local ML based on training structures that are more closely related to a project’s chemical space can overcome prediction limitations of global ML models using Bayer’s extensive Caco-2 permeability database. We have demonstrated that the LightGBM algorithm with 209 molecular property descriptors is suitable for permeability prediction and can be implemented for both a global ML approach and an adaptive ML approach.

Evaluation of Algorithms and Descriptors

By comparing different algorithms and descriptors, we found a combination for the prediction of passive apparent Caco-2 permeability has been found. This combination of LightGBM and 209 molecular property descriptors derived by RDKit allows predicting permeabilities with a lower MAE than most published models on the TDC leaderboard at the time, while at the same time being highly efficient regarding training and prediction times. Compared to Andrew Li’s BaseBoosting model³⁸ (#2), our MAE was only 0.7% worse (MAE: 0.285) while in comparison to Jim Notwell’s MapLight³⁹ (#1) model on the other hand it was 3.3% worse (AAFE: 1.89, MAE: 0.276). Other submissions to the leaderboard included models with AAFEs up to 8.09 (MAE: 0.908). An advantage over, e.g., the leading MapLight model that goes hand in hand with the higher total efficiency of our model is the reduced complexity compared to their combination of CatBoost as algorithm and four different kinds of descriptors (ECFP, Avalon counts, ErG, molecular properties). This combination results in 2863 features that need to be calculated and considered by the model, while our model only needs less than 14% (209 molecular properties) of MapLight’s total feature amount and still only performs less than 4% worse on the benchmark set.

Use of Global Model and Implication of Sampling Resolution on Adaptive Model

Even though the global regression model is less performant for low and medium permeable compounds in comparison to high permeable compounds, it can still be used to exclude certain low permeable compounds because of its low false-positive rate for classifying highly permeable compounds as low permeable. This means that structures that are predicted to be low permeable most likely will indeed turn out to be low permeable and can therefore be excluded from synthesis to save time and experimental resources. Specifically, predictions of more than 10^–4.75 to 10^–4.5 cm/s (∼200–300 nm/s) reliably indicate high permeability (defined as >100 nm/s) while values predicted as lower than 10^–6.5 cm/s (∼3 nm/s) imply with a high probability that the compound will actually be low permeable (defined as <10 nm/s). Because project related scaffolds are increasing in size lately, this might amplify difficulties with permeability and property prediction by machine learning in general. Because of their size, these compounds contain more substructures which increases possibilities for dissimilarity and therefore increase distances in fingerprints across compound classes but also regarding the structure-dependent molecular properties. Accordingly, this leads to higher uncertainties in predictions by a global model that attempts to generalize and interpolate between these structural differences. These emerging difficulties are also indicated by Figure 2, where a drop in performance below R² < 0.4 around the last third of new structures is visible. This drop correlates with the increased usage of bigger modalities in research that has recently begun.

Figure 5 illustrates the decline of available structures for adaptive model training as a result of increasing the similarity cutoff. The plot shows that for a Tanimoto similarity cutoff of 0.8 already about half of the test compounds do not have enough relating structures for training available from the pool. One might think that this means that structural differences are too big and meaningful interpolation is therefore not possible, but looking at the actual structural differences of molecules with different similarity values shows that the differences are not that big at all. For example, a similarity of 0.76 for a nitrile to amide conversion, 0.83 for methyl to fluorine substitution, and inclusion of a single carbon to convert a hydroxy to an ethylhydroxy result in 0.84 similarity. All three mentioned changes imply only a difference in a single heavy atom. Accordingly, most compounds are actually very similar, as one would expect on account of knowing that they are all a result of making changes to a target-dependent, main structural scaffold. Therefore, enriching the training data pool with compounds with higher similarity to improve predictions appears to be less helpful than originally thought and hardly possible, while implying that the “resolution” of measurements in the project-related chemical space is already quite good and major prediction improvements by inclusion of additional measurements of derivatives are not to be expected. To further assess if structural similarity and permeability are associated, significance analysis⁴⁰ could be performed.

Figure 5.

Relationship between number of structures available for training and Tanimoto similarity cutoff. Each line represents a project test set compound (n = 189), for which the similarity cutoff is steadily increased until only the test compound itself matches the similarity cutoff and the training database is accordingly reduced to 0 structures.

Structure–Property Relationship, Ro5 Bias, and Impact on Modelability

Analysis of the distribution of permeability classes shows that our internal data is highly unbalanced and contains significantly more high than medium or especially low permeable compounds (see Figure S7). Additionally, structurally similar appearing compounds show a high permeability variance (Figure 6, left), which subsequently decreases modelability of the data⁴¹ by a global model. This attenuated modelability is also indicated by displaying the biased chemical space of our training data through the approximation of structural clusters via compound-specific circular fingerprints (Figure 6, right). There it becomes visible that most of the structural clusters are rather mixed regarding their permeability classes, but at the same time do not contain every permeability class to the same extent as other clusters, since most clusters are dominated by highly permeable (green) compounds. This bias is a direct consequence of the historic use of the Ro5 for compound optimization and the subsequent enrichment of the database with highly permeable compounds.

Figure 6.

Left: Multi dimensional scaling (MDS) of circular, extended connectivity fingerprints (ECFP) (2048 bit, r = 3) descriptors to two dimensions (X/Y) of Caco-2 permeability data of compounds with MW > 500 g/Mol shows distinct permeability-cliffs (Z-axis). Right: t-distributed stochastic neighbor embedding (t-SNE) of ECFP6 (2048 bit) of full Caco-2 permeability data (n = 33398) with three dimensions (TC1, TC2, TC3) shows permeability class distributions in structural seemingly similar clusters (green: highly permeable compounds; yellow: medium permeable; red: low permeable).

Amount of Most Similar Structures/Adaptive Model Performance Relationship

The performance of the adaptive model with different numbers of the most similar training structures indicates, according to different metrics, that a selection of training compounds from a more confined chemical space can lead to similar or even slightly improved performance for the project test set in comparison to a global model. According to MAE (see Figure 5), 8 other models with reduced amounts of training structures in the training data lead to a model performance better than the global mode (MAE < 0.419). On the other hand, 21 training data-reduced models lead to worse overall performance (MAE > 0.419). Looking at the single compound’s relationship between number of similar structures and performance shows that for some compounds the reduction of training structures can decrease the prediction error, while other compounds from the same series are best predicted by models with high amounts of training structures (examples see Figure S8). Accordingly, no general conclusion can be drawn from these ostensibly random results of the experiments. This most likely means that the good performance of the model for some cut-offs/amounts of training structures is rather attributed to LightGBM pooling certain compounds together to create a leaf in the decision tree, which ultimately results in better predictions by chance, rather than the molecular similarity being a good criterion for selection of training data. Nevertheless, preselecting training data via Tanimoto similarity allowed better performance with lower numbers of structures in comparison to a selection via KNN, which shows that it is generally possible to build more efficient models with our method of training data selection. The difficulties in accurately predicting Caco-2 permeability are most likely due to “permeability/property cliffs”, analogous to activity cliffs,⁴² which are amplified by assay result variability and contained in the training and test data. These cliffs could be the result of multiple factors involved in Caco-2 permeability like transporter-specificity,⁴³ where small molecular changes can induce or reduce the ability to act as substrate for active transport. Additionally, the ability to form intramolecular hydrogen bonds can result in permeability increasements like for some macrocyclic compounds, but it is hard to predict without an extensive database of experimental descriptors which quantify this chameleonicity (e.g., ChamelogK⁴⁴ or change in EPSA⁴⁵).

Outlook: Adaptive Model Improvement/Uncertainty Quantification and Alternative Structure Optimization Strategies

While other companies already have implemented local models for specific compound series,⁴⁶ the herein presented adaptive model based on Tanimoto similarity did not show a generally better performance. The performance on the specific test set is highly dependent on the test structure and respective training structures, so a general rule for a “sweet spot” amount of most similar training structures cannot be derived. Same goes for a general rule of Tanimoto similarity cutoff. A more flexible approach than a fixed number for most similar compounds or a predefined Tanimoto similarity cutoff (see Figure S9 and Figure S10) is the utilization of active learning-like approaches for selection of most informative compounds.⁴⁷ Also, the experimental variability can be quantified to put the model’s performance into perspective,⁴⁸ since this uncertainty in the data defines the highest model performance end. Alternatively, conformal prediction⁴⁹ models could be established to estimate the aleatoric uncertainty and by that give the medicinal chemists a better idea of the reliability of an implemented model. To develop this kind of conformal prediction models, python packages like “mapie”⁵⁰ or “nonconformist”⁵¹ could be utilized.

In our experience, apart from regression or classification models, Matched Molecular Pair Analysis (MMPA) can give synthesis recommendations to guide medicinal chemistry based on a database of already existing compounds. A MMPA algorithm derives rules from that database and outputs a list of possible transformations that might lead to improved properties.⁵² This technique can also be combined with machine learning techniques to predict changes in properties from changes in a structure.⁵³ Alternatively, classification models can be built on their own or predefined structural key descriptors (e.g., 166 bit MACCS or 880 bit PubChem) and analyzed by SHAP feature importance to statistically investigate which exact bits and their corresponding structural feature are statistically most predictive for low or high permeability. Other options include, for example, atom-based attribution approaches which can be visualized as molecular heatmaps.⁵⁴ Another approach would be to derive values of a composite score like the Balanced Permeability Index⁵⁵ or Abbvie’s multiparametric score (AB-MPS)² from molecular properties and simply categorize compounds according to the score.

Conclusion

We have demonstrated that the LightGBM algorithm, in combination with molecular descriptors included in RDKit, is a simple and highly efficient choice for the in silico prediction of Caco-2 permeability. While our global regression model performed exceptionally well for the TDC test set, the high permeability bias in our internal data led to the overprediction of low and medium permeable compounds from the Bayer library. An adaptive, local model based on structural Tanimoto fingerprint similarity did not perform notably better for a project-specific test set containing compounds from a chemical series designed for a specific drug target but showed that more data-efficient models are generally possible to be built. Especially in comparison with the kNN-clustering method for training data selection, the differences in performance with lower training data numbers are major. The challenges in accurately predicting permeability are most likely due to permeability cliffs, which can result from transporter specificity, chameleonicity, and experimental variability in general. As a result, predictions might remain challenging, also because of the comparably larger distances in the e/bRo5 space, which aggravate inter- and extrapolations in general.

Glossary

Abbreviations

AAFE

average absolute fold error

bRo5

beyond rule of five

CDDD

continuous and data-driven descriptors

ECFP

extended connectivity fingerprint

eRo5

extended rule of five

LightGBM

light gradient-boosting machine

MAE

mean absolute error

ML

machine learning

MLP

multilayer perceptron

MMPA

matched molecular pair analysis

MDS

multidimensional scaling

P_app

AB apparent Caco-2 permeability from apical to basolateral side

P_app

BA apparent Caco-2 permeability from basolateral to apical side

P_app

passive, apparent passive permeability

QSPR

quantitative structure–property relationship

R²

coefficient of determination

RMSE

root-mean-square error

Ro5

rule of five

SMILES

simplified molecular-input line-entry system

SVM

support vector machine

TDC

therapeutics data commons

t-SNE

t-distributed stochastic neighbor embedding

Data Availability Statement

LightGBM framework, RDKit, the CDDD implementation, PaDEL and an open-source Caco-2 data set as external benchmark by TDC are available at the respective github URLs. Utilized packages are referenced in the methods. Proprietary structures for the internal tests cannot be disclosed, but molecular property distributions for comparisons to own data can be found in the Supporting Information. Main code chunks used for the global and local models can be found at: https://github.com/ffstghc/caco2ml.

Supporting Information Available

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acs.jcim.4c01083.

• Distributions of internal experimental and external calculated test data; Algorithm comparison with 10-fold cross-validation; Recursive feature elimination results; List of feature names leading to best performance; Feature Importance via SHAP; Descriptor comparison with 10-fold cross-validation; Confusion matrix and classification report for classification model; kNN local model results; Fixed similarity local model results; Histogram of internal permeabilities; Local model results for specific compounds; Explanations to picture abstract; Algorithm comparison for standard-scaled binary training data (PDF)

Author Contributions

The manuscript was written through contributions of F.F.S. and A.R. T.L., A.R., and F.F.S. have given approval to the final version of the manuscript. Conceptualization by F.F.S. and A.R. Data analysis by F.F.S.

There are no external funding sources and grant/award numbers relevant to this manuscript. F.F.S. and A.R. are employees of Bayer, while T.L. is employee of Saarland University.

The authors declare the following competing financial interest(s): T.L. declares no competing financial interest. F.F.S. is employee of Bayer AG. A.R. is employee of and has ownership interest in Bayer AG.