In Silico Prediction of Physicochemical Properties of Environmental Chemicals Using Molecular Fingerprints and Machine Learning

Abstract

There are little available toxicity data on the vast majority of chemicals in commerce. High-throughput screening (HTS) studies, such as being carried out by the U.S. Environmental Protection Agency (EPA) ToxCast program in partnership with the federal Tox21 research program, can generate biological data to inform models for predicting potential toxicity. However, physicochemical properties are also needed to model environmental fate and transport, as well as exposure potential. The purpose of the present study was to generate an open-source Quantitative Structure-Property Relationship (QSPR) workflow to predict a variety of physicochemical properties that would have cross-platform compatibility to integrate into existing cheminformatics workflows. In this effort, decades-old experimental property data sets available within EPA EPI Suite™ were reanalyzed using modern cheminformatics workflows to build updated QSPR models capable of supplying computationally efficient, open, and transparent HTS property predictions in support of environmental modeling efforts. Models were built using updated EPI Suite data sets for the prediction of six physicochemical properties: octanol-water partition coefficient (log P), water solubility (log S), boiling point (BP), melting point (MP), vapor pressure (log VP) and bioconcentration factor (log BCF). The coefficient of determination (R²) between the estimated values and experimental data for the six predicted properties ranged from 0.826 (MP) to 0.965 (BP), with model performance for five of the six properties exceeding those from the original EPI Suite™ models. The newly derived models can be employed for rapid estimation of physicochemical properties within an open-source HTS workflow to inform fate and toxicity prediction models of environmental chemicals.

INTRODUCTION

The U.S. Environmental Protection Agency (EPA) has identified ~32,000 chemicals with the potential for human exposure, ranging from pesticides and industrial chemicals to food additives and personal care products.^1–6 As only a fraction of these chemicals have been fully assessed and characterized,^7–9 there is a need for more rapid and inexpensive approaches to prioritize thousands of chemicals for mechanistically relevant toxicity testing. The cross-agency U.S. federal Tox21 and EPA’s ToxCast research programs have developed promising tools for chemical hazard characterization and prioritization, such as in vitro high-throughput screening (HTS) assays, computational toxicology approaches, and quantitative structure activity relationship (QSAR) models. ^10–18 What these tools lack, however, is the ability to provide insight into the fate and transport of the chemicals, which is needed for chemical risk assessment.

The behavior of chemicals in humans and the environment often depends on some key physicochemical properties, such as octanol-water partition coefficient (log P), water solubility (log S), melting point (MP), boiling point (BP), vapor pressure (VP) and bioconcentration factor (BCF).^19–23 These properties affect bioavailability, permeability, absorption, transport, and persistence of chemicals in the body and in the environment, and are used extensively in exposure, toxicological hazard, and risk assessments of organic chemicals. Certain properties, such as BCF, are required by regulations such as REACH (Registration, Evaluation, Authorization and Restriction of Chemicals) and the United Nations Globally Harmonized System of Classification and Labeling of Hazardous Chemicals.^24–26 These properties are often used in evaluating new or problematic chemicals and are valuable parameters in developing QSAR models for toxicity endpoints.^27–29 These parameters and models support initiatives being driven by the OECD (Organization of Economic Cooperation and Development) and ICATM (International Cooperation on Alternative Test Methods) to reduce or waive animal tests using alternative methods such as in silico modeling.^30,31

Whereas physicochemical properties have been experimentally determined for some chemicals, the majority lack freely available experimental data. For new chemicals, such data are often considered confidential business information and are thus not available to regulatory authorities. Obtaining needed data via experimental measurements can be expensive and time-consuming, it may be difficult to handle hazardous or reactive chemicals, and some pre-manufacturing chemicals are unavailable for testing.

Quantitative structure-property relationship (QSPR) methods are designed to identify the relationship between the physicochemical property of interest and the chemical molecular structure without testing, and are widely used to provide inputs for toxicity prediction models.^32–35 Numerous computational approaches have been proposed to construct QSPR models, and these methods can be generally categorized into three classes: models based on other experimentally determined physicochemical properties;^35–37 models based on calculated molecular descriptors;^32–34,38 and models based on group contributions.^39–43 The third approach was the foundation of initial work in QSPR development. In this approach, a molecule is divided into basic structural building blocks such as atoms or larger functional groups that constitute unique descriptors. Such methods are conceptually simple and computationally efficient for a wide range of chemicals because they only required counting occurrence of functional fragments in a molecule. Nevertheless, the model may suffer from the “missing fragment problem” when new fragments are encountered that were not available in the training set. This issue may be addressed by employing a large learning data set and considering the applicability domain (AD) of the model. The predictive performance of published models in the literature depend on the size, diversity, and composition of the data, rendering it difficult to make direct comparisons of models built using different training sets.

EPA’s Estimation Program Interface (EPI) Suite™ Data program (EPI Suite) provides QSPR models to predict a variety of key fate and transport parameters for environmental chemicals.⁴⁴ However, the decades-old datasets upon which EPI Suite models were originally built contain numerous errors and the models used in predictions are not open-source. Accordingly, we used structure-curation and data-curation workflows to develop an updated dataset^45,46 with which to build new open source models using EPI Suite data sets for log P, log S, BP, MP, log VP and log BCF. This allowed us to reanalyze EPI Suite experimental property data sets and build updated QSPR models capable of supplying computationally efficient, open, and transparent high-throughput property predictions for environmental chemicals.

The goal of the present study was to build QSPR models for in silico prediction of six physicochemical properties using diverse data sets of environmental chemicals exclusively based on analysis of their binary molecular fingerprints. We aimed to improve upon the existing EPI Suite platform and provide the community with an open-source model built in R that can be leveraged in workflows for several computational languages (e.g., Python and, Java). We applied various computational methods, ranging from simple linear regression to sophisticated machine learning approaches, and make recommendations on which models are more appropriate to predict each property. Ultimately, we sought to develop an open-source approach that provides reliable and accurate estimation of physicochemical properties for a wide range of environmental chemicals requiring only input of chemical structures in SMILES (Simplified Molecular Identification and Line Entry System) notation to generate predictions.⁴⁷ To facilitate their acceptance and application, we have adhered to the validation principles defined by the Organisation for Economic Co-operation and Development (OECD) when building and evaluating the QSPR models.⁴⁸ These models allow physicochemical property predictions to be readily generated and integrated with other types of information for regulatory and research purposes.

MATERIALS AND METHODS

Data Sets.

The experimentally measured physicochemical property values of structurally diverse sets of environmental chemicals used in this study were taken from a publicly available data source: Estimation Program Interface (EPI) Suite™ Data.⁴⁹ These organic chemicals cover diverse chemical structures and a broad variety of use classes, including industrial compounds, pharmaceuticals, pesticides, and food additives. Prior to modeling, all chemical structures were subjected to a curation step to prepare QSAR-ready structures free from structural ambiguities (e.g., unconnected fragments, mixtures, salts, inorganics, duplicates), and to reconcile inconsistencies in the EPI Suite dataset. The chemical information were processed by structure-curation and data-curation workflows developed at EPA to provide a QSAR-ready dataset.^45,46 This process found and fixed a number of errors in the EPI Suite chemical library, and discarded chemicals that were not able to be fixed (e.g., entries for which the chemical name and structure could not be reconciled).

The chemical names, CAS Registry Numbers, SMILES notations and experimental data corresponding to the six properties are given in the Supporting Information. The data sets were randomly partitioned into training sets (75% of the chemicals) and test sets (25% of the chemicals) to build the models and externally validate their predictive power, respectively. Table 1 lists the summary statistics for the training and test sets. As shown in Figure 1 and Figure S1, several property values are approximately normally distributed, where log P spans nearly 17 log units from -5.40 to 11.29 with a median of 1.99 while MP ranges from -196 to 385 °C and is centered at 79 °C. The data for log S, log VP and log BCF are skewed since there is an upper limit to log S and log VP and a lower limit to log BCF.

Table 1.

Distribution of Six Physicochemical Property Values for Training and Test Sets

Property	Data Seta	Minimum	Maximum	Mean	Median	Standard Deviation
log P	Training (11370)	−5.40	11.29	2.07	1.99	1.83
	Test (2837)	−5.08	9.36	2.06	1.96	1.82
log S	Training (1507)	−12.06	1.58	−2.56	−2.40	2.15
	Test (503)	−11.25	1.35	−2.71	−2.39	2.30
log BCF	Training (456)	−0.35	5.97	1.88	1.71	1.25
	Test (152)	−0.30	5.82	1.90	1.68	1.29
BP	Training (4074)	−88.60	548.00	188.09	188.50	85.20
	Test (1358)	−84.70	536.00	188.66	190.80	84.69
MP	Training (6485)	−196.00	385.00	79.60	79.00	98.45
	Test (2163)	−187.00	385.00	82.60	83.00	101.09
log VP	Training (2034)	−13.68	5.67	−2.01	−1.22	3.58
	Test (679)	−11.80	4.72	−2.15	−1.40	3.56

^a Numbers of chemicals in each set are indicated in parentheses.

Figure 1.

Data distribution of log P (A) and log S (B).

Molecular Fingerprints.

The chemicals were represented by fingerprints derived from their molecular structures. Fingerprints were calculated using a wide variety of publicly available SMARTS systems implemented in PaDEL:^50,51 Estate (79 bits), Extended (1024 bits), Substructure (307 bits), Klekota Roth (4860 bits), PubChem (881 bits), Atom Pairs 2D (780 bits), and MACCS (166 bits). A total of 8097 binary bits were generated, with 1 and 0 denoting the presence or absence, respectively, of a specific structural fragment. Fingerprint bits with zero variance (i.e., uniform observations across the set) were removed. To obtain reliable models, sufficient occurrences of the fingerprint bits throughout the entire data sets are necessary and, thus, bits with low occurrences (< 2%) were eliminated. Following removal of highly correlated and infrequently occurring bits, the resulting number of bits retained and employed to build the regression models were: 1681 for log P; 1061 for log S, 450 for log BCF; 1050 for BP; 1424 for MP; and 1145 for log VP. A genetic algorithm (GA)^52,53 was used to reduce the feature space by assigning an initial population of chromosomes to two times the number of variables (fingerprint bits).The crossover probability on each chromosome in a population and mutation rate on each gene in a chromosome were set to 50% and 1%, respectively. There were no improvements in the fitness score after 1000 generations.

Multiple Linear Regression.

Multiple linear regression (MLR) is widely used in the modeling of property data.^40,54 We used MLR to produce a linear model to describe the relationship between a physicochemical property and the molecular fingerprint bits:

$$\text{Property}={\displaystyle \sum _{j=1}^m{c}_j{f}_j}$$ (1)

In Equation 1, Property is one of the six physicochemical properties (log P, log S, log BCF, BP, MP or log VP); c_j is the contribution coefficient, which is determined by regression analysis; and f_j is the binary bit of the jth fingerprint, with its presence or absence represented by the numeric values 1 or 0 respectively. Any fragment that occurred in a molecule was counted only once for that molecule, no matter how many times it occurred in the molecule.

Partial Least Squares Regression.

Partial least squares regression (PLSR) is a widely used multivariate analytical technique in QSPR studies.^55,56 The advantage of PLSR over MLR lies in its ability to build a regression model based on highly correlated descriptors, extract the relevant information, and reduce data dimensions. We employed PLSR to generate linear statistical models based on the fingerprint bits and the physicochemical property being predicted. A set of orthogonal latent variables or principal components (PCs) were first generated through a linear combination of the original molecular fingerprint bits, which served as new variables for regression with the response variables (i.e., the physicochemical properties) to build QSPR models. The optimal number of PCs was determined by 10-fold cross validation (CV).

Random Forest Regression.

Random forest (RF) is a non-linear consensus method based upon an ensemble of decision trees which are grown from separate bootstrap samples of the training data.⁵⁷ Bootstrap sampling is conducted via random selection with replacement from the training chemicals during tree growth. The chemicals that are not selected in the construction of the forest are called out-of-bag (OOB) samples, which are used to evaluate the prediction accuracy as trees are added to the forest. Each tree gives a prediction for its OOB chemicals, and the average of these results over all trees provides an overall unbiased external validation. There are three possible model parameters for RF regression: ntree - the number of trees in the forest; mtry - the number of variables randomly sampled at each tree node; and nodesize - the minimum node size below which nodes are not further subdivided. In the present study, the RF model was trained based upon a parameter combination of ntree = 500, nodesize = 5 and mtry = 1/3 the number of fingerprint bits.

Support Vector Regression.

Support vector regression (SVR) models a non-linear relationship between the property and molecular descriptors by utilizing an appropriate kernel function to map the input variables from a lower dimensional space to a higher dimensional feature space and transform the non-linear relationship into a linear form.^35,58,59 An ε-insensitive loss function was used for the SVR modeling, in which the training chemical samples were represented as a tube with radius ε and a Gaussian radial basis function (RBF) was employed as a kernel function. The accuracy of SVR relies on the optimization of the model parameters. An ε-based SVR analysis needs to tune the RBF kernel parameter γ, the radius of the tube ε, and the regularization parameter C which determines the trade-off between model complexity and the training error. Thus, 10-fold CV via parallel grid search was performed in order to find the optimal combination of the three parameters.

Model Validation.

The performance of each QSPR model was evaluated by examining the correlation between the experimental and predicted values using the following parameters:^33,60 R² (coefficient of determination) and RMSE (root mean squared error) for training or test sets with n chemicals; Q² (coefficient of determination) and RMSEcv for 10-fold CV with v chemicals not included in the CV model building set. The 10-fold CV procedure was completed using only the training set.

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^n{(p_i-{\widehat{p}}_i)}^2}}{{\displaystyle \sum _{i=1}^n{(p_i-\overline{p})}^2}}$$ (2)

$$Q^2=1-\frac{{\displaystyle \sum _{i=1}^v{(p_i-{\widehat{p}}_i)}^2}}{{\displaystyle \sum _{i=1}^v{(p_i-\overline{p})}^2}}$$ (3)

$$RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{(p_i-{\widehat{p}}_i)}^2}}$$ (4)

$$RMSEcv=\sqrt{\frac{1}{v}{\displaystyle \sum _{i=1}^v{(p_i-{\widehat{p}}_i)}^2}}$$ (5)

In Equations 2 through 5, p_i and ${\widehat{p}}_i$ are the measured and predicted property values for chemical i, respectively, and $\overline{p}$ is the mean of all chemicals in the data set. In addition, standard error of prediction (SEP) was employed as a criterion to select the optimal principal components in the PLSR analysis.⁵⁵

$$SEP=\sqrt{\frac{1}{n-1}{\displaystyle \sum _{i=1}^n{(p_i-{\widehat{p}}_i-bias)}^2}}$$ (6)

$$bias=\frac{1}{n}{\displaystyle \sum _{i=1}^n(}{p}_i-{\widehat{p}}_i)$$ (7)

Applicability Domain.

Three distance-based measures (i.e., leverage, distance from centroid and k-nearest neighbors (kNN)), were applied to assess the applicability domain (AD) of each regression model. The distance of a test chemical from a defined point in the descriptor space of the training set was calculated and compared to a predefined threshold.^61,62 The test chemical is considered to be within AD if its distance is less than or equal to the threshold. Leverage is defined as the diagonal element of the covariance matrix for a given dataset, and the leverage of a test chemical is proportional to Hotellings T² statistic and its Mahalanobis distance. The threshold was set to three times the average of the leverage (3m/n, with m being the number of variables and n the number of training chemicals). For the measure of distance from centroid, the distance of a test chemical from the training set centroid is compared with a threshold, which is determined as follows: (1) calculate the distances of training chemicals from their centroid; (2) sort the vector of distances in ascending order; (3) set the distance value corresponding to 95^th percentile as the threshold. The kNN measure defines the model’s AD based on the similarity between a test chemical and the training chemicals. The average distance of the test chemical from its five nearest neighbors in the training set is compared with a threshold, which is the 95^th percentile of average distance of training chemicals from their five nearest neighbors.⁶³

Statistical Analysis.

Mathematical processing for data standardization, multivariate regression analysis, and statistical model building were performed using the R statistical computing environment for Windows (version 3.2.1).⁶⁴ Genetic algorithm, multiple linear regression, partial least squares regression, random forest regression, support vector regression and distance of k-nearest neighbors were implemented by the R packages subselect, stats, pls, randomForest, e1071 and FNN, respectively. The R code for feature selection and regression analysis is provided in the Supporting Information.

Online Resource.

The CompTox Dashboard from the EPA National Center for Computational Toxicology integrates experimental and predicted physicochemical endpoint data for over 700,000 individual chemical structures (https://comptox.epa.gov/dashboard/). It also meshes together other data including bioassay screening data, exposure models and product categories. Tens of thousands of experimental data points have been curated and are being used to develop a new suite of prediction models across the database.⁴⁶ Integrated calculation reports are available for each property prediction associated with a chemical and these provide details regarding the performance of the prediction algorithm, associated information regarding whether the chemical is contained within the local applicability domain for the algorithm and a series of nearest neighbors based on the prediction algorithm descriptors. The data generated from the predictions reported in this manuscript are available via the CompTox dashboard and are listed as NICEATM models.

RESULTS AND DISCUSSION

Prior to regression modeling, we investigated the correlation among the six properties as well as their relationship with molecular weight (MW) to potentially inform additional feature sets for model building. For example, if the BP prediction is very accurate and has high correlation to MP, one could use the BP as a predictor for MP. Table 2 gives Pearson correlation coefficients (r) for each combination of physicochemical properties, which were calculated using the following formula:⁶⁰

$$r=\frac{n\sum p_k{p}_l-\sum p_k\sum p_l}{\sqrt{n\sum p_k^2-{(\sum p_k)}^2}\sqrt{n\sum p_l^2-{(\sum p_l)}^2}}$$ (8)

Table 2.

Correlations (r) among Molecular Weight (MW) and the Six Physicochemical Properties^a

	MW	log P	log S	log BCF	BP	MP	log VP
MW	1.000	0.256	−0.648	0.367	0.475	0.460	−0.721
log P		1.000	−0.873	0.830	0.365	−0.043	−0.387
log S			1.000	−0.825	−0.444	−0.285	0.564
log BCF				1.000	0.355	0.163	0.351
BP					1.000	0.733	−0.959
MP						1.000	−0.833
log VP							1.000

^aThe number of chemicals for each pair:

1664 (log P vs log S)

3531 (log P vs MP)

975 (log S vs BP)

285 (log BCF vs BP)

3421 (BP vs MP)

482 (log P vs log BCF)

1560 (log P vs log VP)

1811 (log S vs MP)

473 (log BCF vs MP)

1775 (BP vs log VP)

1609 (log P vs BP)

330 (log S vs log BCF)

1156 (log S vs log VP)

334 (log BCF vs log VP)

2143 (MP vs log VP).

In Equation 8, p_k and p_l represent different physicochemical properties and n is the number of chemicals in each pair of properties. ⁶³As shown in Table 2, MW is moderately correlated to log VP (r = -0.721) and log S (r = -0.648), poorly correlated to BP (r = 0.475), MP (r = 0.460) and log BCF (r = 0.367), and nearly uncorrelated with log P (r = 0.256). According to the correlation results reported in Table 2, the six properties can be divided into two groups, one including log P, log S and log BCF with r of -0.873, 0.830 and -0.825 for log P vs log S, log P vs log BCF, and log S vs log BCF, respectively; another including BP, MP and log VP, where MP is significantly correlated to BP and log VP with r of 0.733 and -0.833, respectively. BP is also highly correlated to log VP with r of -0.959. In contrast, the experimental data of log P, log S and log BCF are uncorrelated with those of MP, BP and log VP, with low r for each pair.

Our dataset included large number of independent variables (molecular fingerprint bits) available but not all of them made substantial contributions to modeling the physicochemical properties. The presence of irrelevant or redundant features limits the applicability of a model and may result in overfitting. When too many variables are employed, a complicated regression model can fit the training data extremely well with very low deviations between experimental values and predicted values. However, an over-adapted or over-fitted model yields large prediction errors for test chemicals and thus loses its generalization. Therefore, it is critical to identify appropriate subsets of informative variables from the original set of molecular fingerprint bits. We applied the well-established GA dimension reduction method to our dataset for feature selection. Prediction models were built by regression against the training sets with different subsets of fingerprint bits using four contrasting approaches, and the model performance was evaluated using 10-fold cross validation and an independent external test set. The results of each predicted property and regression statistics are discussed in detail below.

Octanol-Water Partition Coefficient (log P) Model

Among the six properties, log P had the largest data set of chemicals (over 14,000) and fingerprint bits (1681). As shown in Table 3 and Figure 2A, model performance varied with respect to the number of fingerprint bits used. When models were trained using the entire set of fingerprint bits, some of the variables were unrelated to the variation of the property. When using 600 fingerprint bits selected by GA, the MLR results show a significant correlation between the estimated and measured values on the test set with an R² of 0.888 and a minimum RMSE of 0.576. These statistics are very similar to those for the training set (R² of 0.901 and RMSE of 0.546), indicating the stability of the model. The inclusion of MW marginally improved the prediction on the test set with R² increased to 0.891.

Table 3.

Regression Statistics of log P Using Subsets of Fingerprint Bits and MW

Variable	Model Statistics	Data Set	MLR	PLSR	RF	SVR
1681 FP bits	R2	Training	0.916	0.915	0.880	0.991
		Test	0.879	0.878	0.876	0.920
	RMSE	Training	0.509	0.510	0.552	0.176
		Test	0.607	0.608	0.564	0.502
600 FP bits	R2	Training	0.901	0.902	0.885	0.983
		Test	0.888	0.889	0.879	0.932
	RMSE	Training	0.546	0.546	0.548	0.264
		Test	0.576	0.575	0.569	0.457
600 FP bits + MW	R2	Training	0.904	0.905	0.891	0.987
		Test	0.891	0.892	0.886	0.935
	RMSE	Training	0.539	0.537	0.539	0.207
		Test	0.569	0.568	0.562	0.451

Figure 2.

The relationship between model complexity and prediction errors (RMSE) (A) and plot of experimental data versus estimated values by SVR using 600 fingerprint bits (B) for log P.

When PLSR was utilized to build models, the number of significant principal components (PCs) was determined using a 10-fold cross-validation (CV) procedure on the training set. The relationship of the standard error of prediction (SEP) versus the number of PCs is displayed in Figure 3. The gray lines were produced by repeating this procedure 100 times, while the black line depicts the lowest SEP value from a single 10-fold CV; the dashed vertical lines represent the optimal number of PCs and the dashed horizontal lines indicate the SEP value for the test set when the optimal PCs are applied. The variation of SEP is much larger for the all-variable (1681 fingerprint bits) model than the model of 600 fingerprint bits selected by GA, implying that the optimized feature set exhibits greater model stability. For the all-variable model, SEP initially decreases with PCs; the trend then reverses when noise emerges as the complexity of the model increases. Although more components improve the fitting quality of the model, they also lower the predictive power due to overfitting; the minimal cross-validation error was obtained using 13 PCs. For the 600-bit model, the SEP decreases monotonically and gradually approaches a stable value, and the use of 42 PCs yielded the optimal model, which gave a minimum RMSE of 0.575 corresponding to an R² of 0.889 for the test chemicals.

Figure 3.

The relationship between the number of principal components (PCs) and the standard error of prediction (SEP) for the PLSR model of log P. The black lines were produced from a single 10-fold CV while the gray lines correspond to 100 repetitions of the 10-fold CV. (A) Plot of SEP versus PCs for the all-bit model. (B) Plot of SEP versus PCs for the 600-bit model selected by GA.

Unlike other modeling approaches, RF regression was insensitive to the number of fingerprint bits. Figure 2A shows that the model statistics do not vary with the number of variables. A RF model trained on the 600 fingerprint bits produced nearly the same prediction as that trained on all bits. Furthermore, in each case the correlation coefficient of the test set was found to be similar to that of the training set, and the RMSE values were also very close to each other. Since RF is not sensitive to the number of variables, feature selection did not improve the predictive performance, and hence it was unnecessary to remove irrelevant fingerprint bits from the model. The RF algorithm encompassed a large number of simple tree models, and thus greatly mitigated overfitting. Additionally, it was not essential to perform cross or independent validation on RF models as this was inherently provided by the OOB estimate.

The optimum RBF kernel parameter γ, the radius of the tube ε, and the regularization parameter C that collectively yielded the lowest RMSE from cross validation were chosen to build the SVR model. SVR is robust against the existence of irrelevant or of mutually correlated variables, and here even using all fingerprint bits yielded satisfactory results. Feature selection did further improve the predictive performance with test set R² increasing from 0.920 to 0.932 using 600 fingerprint bits. Figure 2B shows a scatter plot of predicted versus experimental values from SVR modeling. In general, the measured and predicted values were highly correlated and the majority of the data points are concentrated around the regression line with a small deviation over a large range for both training and test sets.

Water Solubility (log S) Model

Compared to the log P data set, the log S data has a low ratio of training chemicals to bits (1507/1061). As shown in Table 4, small sets of variables achieved better predictive power than large ones. When all fingerprint bits were employed, the model yielded R² of 0.983, 0.933 and 0.995 for the training set, but only 0.604, 0.869 and 0.910 for the test set corresponding to MLR, PLSR and SVR, respectively, implying that overfitting occurred. The predictive performance of the models was improved remarkably when a more appropriate number of fingerprint bits was selected using GA (Figure 4A). The optimal model with 350 bits produced test R² of 0.931 for both MLR and PLSR, and 0.932 for SVR, which were very close to the training set values (Figure 4B). These results confirm the effectiveness of the GA feature selection in capturing the relevant information and building stable models. Using an optimal combination of fingerprint bits selected using GA, the results from MLR and PLSR are almost identical.

Table 4.

Regression Statistics of log S Using Subsets of Fingerprint Bits, MW and log P

Variable	Model Statistics	Data Set	MLR	PLSR	RF	SVR
1061 FP bits	R2	Training	0.983	0.933	0.878	0.995
		Test	0.604	0.869	0.896	0.910
	RMSE	Training	0.275	0.540	0.673	0.156
		Test	1.805	0.791	0.663	0.649
350 FP bits	R2	Training	0.953	0.952	0.873	0.959
		Test	0.931	0.931	0.892	0.932
	RMSE	Training	0.454	0.456	0.687	0.422
		Test	0.588	0.587	0.671	0.579
350 FP bits + MW	R2	Training	0.957	0.956	0.887	0.960
		Test	0.932	0.933	0.899	0.933
	RMSE	Training	0.436	0.437	0.648	0.414
		Test	0.584	0.580	0.655	0.575
350 FP bits + MW + log P	R2	Training	0.961	0.960	0.925	0.966
		Test	0.935	0.938	0.932	0.939
	RMSE	Training	0.415	0.416	0.547	0.388
		Test	0.552	0.548	0.554	0.542

Figure 4.

The relationship between model complexity and prediction errors (RMSE) (A) and plot of experimental data versus estimated values by SVR using 350 fingerprint bits (B) for log S.

Many factors can affect the solubility of a molecule in an aqueous solution, including the molecule’s size, shape, polarity, and hydrophobicity. For example, a cavity must be formed in water for a chemical to dissolve in an aqueous solution. The larger the molecular weight or the molecular size, the larger the cavity is required, so the greater the energy needed to make a bigger cavity and thus the lower the solubility. Hence, log S is negatively correlated to MW with r = −0.648. The partition coefficient measures how strongly a chemical molecule interacts with n-octanol in comparison to water. The aqueous solubility reduces with increasing hydrophobicity, and hence log S has a negative correlation with log P (r = -0.873). When MW and log P were incorporated as additional descriptors to model water solubility, both were found to play important roles and led to improved predictions compared to the pure fingerprint bit model. A significant improvement was achieved for all four regression approaches, and SVR yielded the highest test R² of 0.939 and lowest RMSE of 0.542. This improvement was most pronounced for RF with R² increasing from 0.892 to 0.932, and it performed nearly as well as the other models, suggesting that the use of numeric descriptors favors RF modeling.

Bioconcentration Factor (log BCF) Model

As the smallest data set with only 456 training chemicals, the validation statistics of log BCF rely highly on the ratio of the number of chemicals to the fingerprint bits. When all 450 bits were employed for MLR, for example, the fit to the training data was excellent with R² close to 1, but for the test chemicals R² was close to zero (Table S1). This was expected, as overly complex models have a tendency to overfit the training set and lose their generalization capability. Unlike the all-variable model from MLR, which yielded extremely low prediction accuracy for the test set, PLSR, RF and SVR achieved acceptable correlation with test R² between 0.723 and 0.725.

As with the previous results, when the most information-rich variables were retained and redundant ones were discarded via feature selection, the predictive performance was enhanced significantly. The quality of the model depended heavily on the number of selected fingerprint bits, and Figure S2A shows the initial decline of the RMSE for the test set until attaining a minimum at a medium number of bits, and then a gradual increase with the number of bits. While using too many variables resulted in overfitting and hence lowered the predictive ability of the model, too few variables cannot capture sufficient structural information, and with a smaller set of chemicals it was more challenging to derive a stable and meaningful model due to underfitting. The lowest prediction errors occurred on the models with moderate complexity around 200 bits. MLR, PLSR, and SVR yielded very similar results in terms of the predictive ability on the test set with R² of 0.863–0.879 (Figure S2B). Using MW and log P as additional descriptors to model log BCF did not improve model performance except when using RF, which yielded an increase in R2 from 0.734 to 0.816. Among the four approaches, SVR achieved the best predictive performance with R² of 0.885 and RMSE of 0.405 for the test set.

Boiling Point (BP) Model

Since the ratio of training chemicals to bits (4074/1050) was large for the BP data set, the modeling statistics were not sensitive to the bit number, and the model performance did not vary considerably with different subsets of fingerprint bits for the test set (Figure S3A). The BP model with 400 bits had the highest test R² of 0.943 and lowest RMSE of 19.72 using SVR (Table S2). Our regression analysis suggested that MW is an important contributing factor for boiling point. When incorporating MW into the model, the predictive performance was improved with test set R² increasing to 0.965 for SVR model, which outperformed the other three approaches (R² ranging from 0.935 to 0.940). The model had difficulty accurately predicting high boiling points; large errors were observed near the upper extreme of the experimental range, and chemicals with medium and low experimental values were predicted more accurately (Figure S3B). These errors may result from experimental measurement or are due to fewer data points at the high end of the range of values and resulting lack of model coverage.

Melting Point (MP) Model

When modeling MP, feature selection did not have a significant impact on the external validation results (Figure S4A) due to a large ratio of training chemicals to bits (6485/1424). Similar to the other physicochemical property modeling, SVR achieved the best results with a test R² of 0.813 (Figure S4B), followed by RF (0.802), PLSR (0.781) and MLR (0.780) (Table S3) using 500 fingerprint bits. To improve the predictive ability, we considered employing estimated BP as an additional descriptor to predict MP. This consideration was based on the fact that the BP dataset was large (5400 chemicals) and good predictive performance with R² greater than 0.960 was achieved from the fingerprint model, leading to reliable BP estimates.

A series of models were derived through correlating the experimental MP with MW and estimated BP. Table S3 compares the regression statistics obtained using various combinations of fingerprint bits, MP, and BP. It is evident that the regression models were very sensitive to BP, which could provide useful information and more accurate estimation for MP. The inclusion of both BP and MW as two descriptors into MP models enhanced the predictive performance for all four regression approaches. The test R² increased to 0.826 while RMSE decreased to 39.14 using SVR. Overall, SVR regression substantially outperformed the other approaches and RF was slightly superior to MLR and PLSR. These facts reflect the advantage of nonlinear approaches over linear approaches for modeling MP. The RMSE reported here compares well with that reported by Tetko et al. modeling on a much larger set of over 200,000 data points extracted from patents.⁶⁵

Vapor Pressure (log VP) Model

The data set of log VP was relatively small with 2034 training chemicals. Hence, feature selection by GA remarkably improved the predictive ability of the model when fewer fingerprint bits were used instead of all 1145 bits, particularly for MLR (Figure S5A). The four approaches had low prediction errors and comparatively similar statistical performance when 350 bits were employed to build the models, with test R² values ranging from 0.902 to 0.930 (Table S4). A significant correlation between log VP and BP was observed with an r of -0.959, and meanwhile MW was also correlated to log VP with an r of -0.721 (Table 2). After introducing MW as a feature, the correlation of the model was greatly improved, and the inclusion of BP further enhanced the predictive performance (test set R² values between 0.941 and 0.946). Linear methods MLR and PLSR performed nearly as well as nonlinear methods RF and SVR. MLR is a simple regression method that does not require time-consuming parameter optimization in order to achieve good performance. If this constraint were a concern, the MLR model would be particularly suitable for the prediction of log VP.

Although these models yielded accurate predictions for most chemicals, some predictions deviated considerably from the experimental values. As indicated in Figure S5B, the model performed poorly for chemicals with low vapor pressure (log VP < -5.0 log units). It is difficult to accurately measure vapor pressure for chemicals with very low volatility and experimental errors tend to be larger. The lack of high quality experimental data may be a driving factor for the failure of the models at extreme values.

Comparison to the Prediction from EPI Suite.

Several software programs are available for predicting physicochemical properties. Among them, the EPI Suite - Estimation Program Interface,⁴⁴ developed by Syracuse Research Corporation (SRC), is a standalone, reproducible, EPA-endorsed, and branded product with a long history of usage. It is widely used by governmental regulatory agencies in the United States, Canada, and Europe to predict physicochemical properties of environmental chemicals. However, it uses proprietary prediction models. To compare our models with EPI Suite predictions, the KOWWIN (version 1.67), WATERNT (version 1.01), BCFWINNT (version 3.00) and MPBPNT (version 1.43) modules were employed to estimate log P, log S, and log BCF, as well as MP, BP and log VP, respectively. For the six property sets, not all the information about the training and test chemicals from EPI Suite original data is available. We broke the entire data set into training and test sets as described in Table 1. As shown in Table 5, the entire, training and test sets have very similar correlation statistics between experimental data and estimated values for all six properties. When compared with the test sets using the coefficient of determination R², our log S model is inferior to EPI Suite’s (0.939 vs 0.955), and other our models exhibit better predictions for log P (0.935 vs 0.904), BP (0.965 vs 0.937), log VP (0.946 vs 0.900), and log BCF (0.885 vs 0.813). Our estimation for melting point is substantially superior to EPI Suite’s, with an R² of 0.826 compared to 0.638 from EPI Suite.

Table 5.

Correlation Statistics between Experimental Data for Chemicals Used in This Study and EPI Suite Predictions

Property	All Chemicals			Training Set			Test Set
	Number	R2	RMSE	Number	R2	RMSE	Number	R2	RMSE
logP	14207	0.895	0.605	11370	0.893	0.612	2837	0.904	0.576
logS	2010	0.948	0.490	1507	0.945	0.495	503	0.955	0.472
log BCF	608	0.818	0.484	456	0.820	0.485	152	0.813	0.481
BP	5432	0.937	21.73	4074	0.938	21.56	1358	0.937	21.78
MP	8648	0.635	57.25	6485	0.634	57.58	2163	0.638	57.14
logVP	2713	0.917	0.965	2034	0.923	0.928	679	0.900	1.071

Analysis of Applicability Domain.

The QSPR models were developed using training sets and, thus, their applicability to external chemicals depends on the structural similarity between the training chemicals and the external test chemicals. The models would be expected to provide more reliable predictions for chemicals that fall in the AD, as defined by the three distance measures defined earlier. In this study, a test chemical is deemed to be outside the AD only if the thresholds from all three distance measures are exceeded. If only one or two thresholds are exceeded, the chemical is considered to be potentially outside the AD. Table 6 and Table S5 summarize the number of test chemicals outside the AD for each regression model identified by individual distance measures and their combinations. These tables also show R² and RMSE for the test sets after removing the chemicals considered outside the AD. In all cases, the number of test chemicals completely outside the AD was very small (0 to 9), and their removal did not have a significant influence on the model statistics. Using kNN and distance from centroid measures, the model statistics were much larger number of improved remarkably with kNN exhibiting better performance than the distance from centroid. Taking the log P model as an example, R² increased from 0.935 to 0.940 while RMSE decreased from 0.451 to 0.434 after removing 121 test chemicals which were identified outside the domain by five nearest neighbors. kNN is a more appropriate measure of distance between training and test chemicals because the descriptors employed in this study are binary variables. Leverage measure did not considerably impact the model statistics. This is in part because the number of chemicals outside the AD identified by leverage was small. It is also important to consider that not all chemicals outside the AD are always wrongly predicted and not all chemicals inside the AD are correctly predicted, which is in agreement with the literature.⁶¹ This phenomenon can be observed in the plots of leverage versus standardized residuals (Figure 5 and Figure S6), which show many test chemicals outside the AD were correctly predicted and some of the test chemicals inside the AD were not accurately predicted.

Table 6.

Applicability Domain (AD) of log P and log S Models: Test Set Evaluation^a

Property	Measure	Chemicals outside AD	Chemicals inside AD	Experimental vs Predicted Test Chemicals inside AD
				R2	RMSE
log P	Leverage (I)	10	2827	0.935	0.448
	Distance from centroid (II)	136	2701	0.936	0.443
	Distacne by kNN (III)	121	2716	0.940	0.434
	I and II and III	3	2834	0.935	0.450
	I or II or III	247	2590	0.938	0.439
log S	Leverage (I)	7	496	0.939	0.541
	Distance from centroid (II)	28	475	0.941	0.538
	Distance by kNN (III)	22	481	0.943	0.531
	I and II and III	0	503	0.939	0.542
	I or II or III	43	460	0.941	0.534

^a The models were built by SVR using 600 FP bits + MW for log P and 350 FP bits + MW + log P for log S.

Figure 5.

Plots of leverage versus standardized residuals for log P (A) and log S (B) models’ training and test sets. The models were built by SVR using 350 and 600 fingerprint bits for log S and log P, respectively. Vertical dashed line marks AD threshold based on the leverage value. Horizon dashed lines define region where predictions were within two standardized residuals.

Adherence to OECD (Q)SAR Validation Principles.

OECD has defined five validation principles to facilitate the consideration of a (Q)SAR model for regulatory purposes. To encourage the acceptance and application of the models presented here, we have provided the following information in conjunction with the OECD validation principles. The QSAR Model Reporting Format (QMRF) is available in the Supporting Information.

• 1)A defined endpoint. We have built models for six well-defined physicochemical properties (log P, log S, log BCF, BP, MP and log VP).
• 2)An unambiguous algorithm. We have evaluated four unambiguous machine learning algorithms (MLR, PLSR, RF and SVR). To increase transparency, the R code associated with these modeling approaches is made available in the supplemental material.
• 3)A defined domain of applicability. We have evaluated the AD using three different distance measures.
• 4)Appropriate measure of goodness-of-fit, robustness, and predictivity. We have evaluated our models based on cross-validation and external validation sets, iteratively varied the feature sets to examine robustness, and compared them to currently available gold standard physicochemical property prediction software.
• 5)A mechanistic interpretation, if possible. Here it is not practical to make an interpretation linking each and every selected fingerprint bit to the modeled endpoints. However, we assume that the statistically selected fingerprint bits represent fragments that are relevant to the studied endpoints. The physicochemical properties being predicted by the QSPR models presented here are critical inputs to environmental fate and transport and toxicity prediction models.

Accurate prediction of physicochemical properties using validated QSPR models will ensure a much clearer understanding of chemical hazard and exposure potential.

CONCLUSIONS

In the present study, QSPR models were implemented to predict six physicochemical properties from binary molecular fingerprints on the basis of large and structurally diverse sets of environmental chemicals using four distinctly different modeling approaches. Satisfactory predictive performance was achieved using optimal subsets of fingerprint bits and optimized regression parameters, and the estimated values correlated very well with experimental values. Both linear and nonlinear approaches provided accurate property predictions with similar regression statistics.

Since not all fingerprint bits contain useful or unique information related to the physicochemical properties, it was crucial to select the most relevant variables to construct the model. On one hand, an insufficient number of fingerprint bits led to underfitted and statistically unstable models and potential lack of coverage in new chemistry space. On the other hand, use of excessive fingerprint bits produced overfitted models from the training data, resulting in poor prediction of the test set values. Feature selection using GA was found to substantially improve the predictive ability of MLR models, influence PLSR and SVR models only slightly, and exert no effect on RF models.

The four regression approaches exhibited different modeling characteristics. MLR is a simple linear regression method and does not require a time-consuming parameter optimization procedure. PLSR yields relatively reasonable accuracy of predictions and performs well even in the presence of noise variables. RF is robust against overfitting since the algorithm constructed an ensemble of regression trees to model the data, and we found it predicted the training and test sets equally well. Nevertheless, the RF model did not perform as well as the other models when binary fingerprints were used as descriptors.

SVR, a complex and nonlinear modeling technique, was shown to be superior to the other three approaches for modeling these properties. SVR coupled with GA in selecting the most significant descriptors achieved excellent results and predicted all the six properties accurately in terms of the coefficient of determination and RMSE for both 10-fold cross validation and the external test sets. Table 7 summarizes the best performing models and shows that the statistics between cross validation and test sets are close to each other, confirming the stability, robustness and reliability of these regression models. The correlation between experimental and predicted values for the test sets was found to vary over a range of R² for different properties. The BP model had the highest test R² of 0.965, followed by log VP (R² = 0.946), log S (R² = 0.939) and log P (R² = 0.935). MP and log BCF model predictions were less accurate, with test R² = 0.826 and 0.885, respectively.

Table 7.

Regression Statistics of Best Performing SVR Models for Each Property

Property	Variables	10-Fold Cross Validation				R2 for Test Set	RMSE for Test Set
		Q2		RMSEcv
		Mean	Interval	Mean	Interval
log P	600 bits + MW	0.932	0.926–0.935	0.478	0.428–0.558	0.935	0.451
log S	350 bits + MW + log P	0.928	0.921–0.936	0.580	0.487–0.666	0.939	0.542
log	200 bits + MW + log P	0.863	0.851–0.877	0.465	0.361–0.525	0.885	0.444
BCF
BP	400 bits + MW	0.955	0.941–0.968	17.98	14.30–20.64	0.965	15.63
MP	500 bits + MW + BP	0.824	0.812–0.831	41.41	38.37–45.29	0.826	39.14
log VP	350 bits + MW + BP	0.929	0.915–0.939	0.975	0.806–1.080	0.946	0.810

Taken together, the results of this study demonstrate that the combination of careful data curation, binary molecular fingerprints, and machine learning approaches provides a rapid and efficient way to estimate physicochemical properties of environmental chemicals. We have thus developed QSPR models that are highly stable and reliable. The models conform with most of the validation principles put forth by the OECD and therefore have broad applicability for property estimation of many classes of compounds. The models are freely available as downloadable workflows via the NICEATM website to support high-throughput calculations, and can be used by researchers and regulators to make predictions on new chemical sets, improve toxicity models, and inform hazard/risk characterization. In the near future, the models developed here will be used to predict data for the six reported properties for the CompTox dashboard website content of >720,000 chemicals and made available to the public.