Toxic Alerts of Endocrine Disruption Revealed by Explainable Artificial Intelligence

Abstract

The local interpretable model-agnostic explanation method was used to unveil substructures (toxic alerts) that cause endocrine disruption in chemical compounds using machine learning models. The random forest classifier was applied to build explainable models with the TOX21 data sets after data curation. Using these models applied to the EDC and EDKB-FDA data sets, the substructures that cause endocrine disruption in chemical compounds were unveiled, providing stable, more specific, and consistent explanations, which are essential for trust and acceptance of the findings, mainly due to the difficulty of finding relevant experimental evidence for different receptors (androgen, estrogen, aryl hydrocarbon, aromatase, and peroxisome proliferator-activated receptors). This approach is significant because of its contribution to the interpretability of explainable machine learning algorithms, particularly in the context of unveiling substructures associated with endocrine disruption in five targets (androgen receptor, estrogen receptor, aryl hydrocarbon receptors, aromatase receptors, and peroxisome proliferator-activated receptors), thereby advancing the relevant field of environmental toxicology, where a careful evaluation of the potential risks of exposure to new compounds is needed. The specific substructures thiophosphate, sulfamate, anilide, carbamate, sulfamide, and thiocyanate are presented as toxic alerts that cause endocrine disruption to better understand their potential risks and adverse effects on human health and the environment.

Introduction

Endocrine disruption is a phenomenon that has significant implications for human health, relying on the subtle interplay between chemical compounds and the endocrine system.¹ Understanding endocrine disruption is a crucial challenge in the field of chemical toxicology, where the effects of molecular interactions resonate in the environment and affect health.^1,2 The exploration of the substructures that render chemical compounds potent endocrine disruptors is essential for comprehending the molecular interactions between the endocrine disruptors and receptors.^3,4

Machine learning explainer models,⁵ finely tuned to capture the complexities of chemical structures, introduce a new perspective to the challenge of finding the correlation between substructures and endocrine disruption. By analyzing massive data sets that include diverse chemical compounds and their corresponding endocrine disruption outcomes, these models discern patterns and substructures that elude conventional approaches. The identification of molecular motifs that act as potent kernels of endocrine disruption is the key to unraveling the significance of these substructures, shedding light on the molecular fingerprints that underpin the toxicological impact of diverse chemical compounds. Therefore, the method adopted here promotes a deeper understanding of the unknown substructures that contribute to endocrine disruption.

Explainable machine learning⁵ has emerged as an invaluable tool in toxicology, offering transformative capabilities in understanding the relationships between chemical structures and biological responses.⁶⁻¹³ The interpretability of explainable machine learning models allows us to unlock it and gain profound insights into the substructures that drive endocrine disruption.^3,4,14 Each step from data curation and cross-validation to random tree model training, statistical analysis,^8,9,12,13 and local interpretable model-agnostic explanations (LIME),¹⁵ is undertaken meticulously to ensure robustness and reliability in identifying patterns, decoding relationships, and elucidating the specific substructures that correlate with endocrine disruption.¹⁴

The groundwork laid in this study aims to uncover key insights and answer an important question: Which substructures distinguish endocrine disruptors at the molecular level?¹⁶ Our findings promise to reshape our understanding of these chemical compounds, providing a nuanced and deeper perspective on the substructures that make them potent endocrine disruptors.^3,4 The purpose of this study is to identify these substructures and classify them as interacting with androgen receptors (AR), estrogen receptors (ER), aryl hydrocarbon receptors (AhR), aromatase receptors (ARO), and peroxisome proliferator-activated receptors (PPAR) using random forest (RF) classifier models. The machine learning model is trained and validated using the meticulously curated TOX21¹⁷ data set and tested with the EDC¹⁸ and EDKB-FDA¹⁹ data sets. The quality of these models is assessed using metrics such as precision, recall, F1, Matthew Correlation Coefficient (MCC), and accuracy scores. The machine learning models are interpreted using the local interpretable model-agnostic explainer LIME,¹⁵ which identifies the most relevant substructures that cause endocrine disruption. The novelty of our work lies in providing an explained, comprehensive list of more specific toxic alerts relevant for endocrine disruption that surpasses the existing knowledge in five tasks as compared with literature^4,20−23 from experimental and explainable methods so far.

Methodology

The Python packages DeepChem (version 2.7.1.), RDKit (version rdkit-2023.9.1), and LIME (version lime-0.2.0.1) were installed on a Google Colaboratory platform. Other Python auxiliary packages such as mols2grid (version mols2grid-2.0.0), Matplotlib (version 3.7.1), Scikit-learn (version 1.2.2), pandas (version 1.5.3), Numpy (version 1.23.5), and IPython (version 8.19.0) were also installed.

The TOX21,¹⁷ EDC,¹⁸ and EDKB-FDA¹⁹ data sets were carefully curated for training, validating, and testing machine learning models for endocrine disruption. The TOX21 data set includes the SMILES representation and binary labels (0 for inactive and 1 for active molecules) for the activity of 8014 compounds on AR, ER, AhR, ARO, and PPAR. We used DeepChem to implement various data splitters, transformers, featurizers, machine learning classifier models, and infrastructure to create a model for classifying compounds with endocrine disruption ability. The TOX21¹⁷ data set was featurized using 1024 extended-connectivity fingerprints (ECFPs), a type of topological fingerprint for molecular characterization. As TOX21 data set is imbalanced, we treated it with the resample method using the Scikit-learn library to create two different dataframes of majority and minority classes, and upsampled the minority class. The featurized data set was then randomly split (80/20 split) for training and validation using the 5-fold cross-validation method for the RF classifier models. It is important to note that the model is randomly generated for each run, so the results may represent the expected diversity. We splitted 20% of the TOX21 data set for internal 5-fold cross validation. Then, the external validation was performed using the EDC and EDKB-FDA data sets.

Initially, RF classifier models were used from the Scikit-learn library. The hyperparameters of these models were optimized, specifically, the number of estimators (50 and 500) and maximum depth (1 and 20) of the trees, using the randomized search CV method and the ROC-AUC metric. The best models were subsequently used in the next steps with the training and validation data sets, evaluating the area under the ROC curve (ROC-AUC metric). The results were assessed using the confusion matrix, accuracy, precision, F1, MCC, and recall scores.

LIME¹⁵ was utilized to explain the predictions made by the model using the training and validation data sets (TOX21), and test data sets (EDC and EDKB-FDA) using the statistical analysis. In the classification mode, the model explainer was employed with categorical and feature names using 1024 circular fingerprints to represent the training data set and feature names. Then, the model predicted and explained the validation data set using 100 features, resulting in a dictionary that maps the fingerprint index to the list of SMILES strings that activated that fingerprint, which contains their respective substructure weights that are important for endocrine disruption, allowing us to understand why a molecule is classified as such. Each molecule in the validation data set was evaluated to explain why the model predicted it as an endocrine disruptor or not. Disrupting molecules that were correctly predicted as endocrine disruptors in the validation data set were filtered out by applying a threshold of 0.8 to use only the most important disruptors. Therefore, the explainer identified the most sensitive features to the prediction by analyzing the elements in the fingerprint that correspond to one or more fragments. Then, these findings were stored to be mapped and visualized. LIME assigned weights to indicate the contribution of a fragment in a molecule to the prediction, ranging from 0 (no disruption) to −1 (negative impact on disruption) or +1 (positive impact on disruption). Therefore, the sum of these weights for each molecule does not necessarily equal 1, but still indicates the contribution of a fragment to the prediction. Thus, the model classified molecules as “endocrine disruptor” or “nonendocrine disruptor” by separately summing all the positive and negative contributions of its fragments. If the sum of the positive weights is greater than the sum of the negative weights, the molecule is classified as an “endocrine disruptor”. Lastly, the classification data of each active molecule for endocrine disruption was displayed with the number of occurrences and total weight of contribution of each substructure in the endocrine disruption prediction. Fragments with small or negative weights were filtered out to enhance the interpretability of the results, focusing on the substructure that strongly enables endocrine disruption. The RDKit resources were applied to highlight the substructures, and the visualization of them in the associated compounds was rendered by the mols2grid and Pandas libraries.

Results and Discussion

Data Curation

The TOX21 data set originally had 8014 chemical compounds distributed in five tasks of nuclear receptors: AR, ER, AhR, ARO, and PPAR. Then, for each receptor, to eliminate repeated instances for each compound, the Tanimoto similarity coefficients were calculated using Morgan fingerprints. We dropped compounds with coefficients equal to 1 from the data set, only keeping the first occurrence of the compound. Additionally, the repeated chemical compounds with different labels (0 for one compound and 1 for the same compound) were both dropped to avoid inconsistence. All multiple occurrences were also dropped, keeping only the first occurrence for each repeated molecule with matching labels. Consequently, the five subdata sets with each of the five tasks were reduced to 6670, 5627, 5964, 5268, and 5909 compounds for the receptors AR, ER, AhR, ARO, and PPAR, respectively. The EDC and EDKB-FDA test data sets had originally 1107 and 632 chemical compounds, respectively. They were also curated using the same procedure presented above. Thus, the remaining number of compounds in the EDC and EDKB-FDA testing data sets was 1077 and 287 after curation, respectively. Table 1 shows the numbers of initial and final compounds with activity data per assay and number of duplicates and multiple occurrences for each task.

Table 1. Numbers of Initial and Final Compounds with Activity Data Per Assay, Compounds with No Data, and Number of Duplicates and Multiple Occurrences for Each Task.

Task	# of initial compounds	Compounds with no data	Compounds with unmatched data	Duplicates/Multiple occurrences	Salts/Complexes	# of final compounds
AR	8014	575	71	506/495	219	6670
ER	8014	1698	431	326/315	0	5627
AhR	8014	1323	171	446/455	217	5964
ARO	8014	2074	312	373/387	216	5268
PPAR	8014	1431	218	418/436	217	5909
EDC	1107	0	0	36/15	0	1077
EDKB	632	0	0	350/199	0	286

Building the Best Classifier Model

Previously, we tested Deep Residual Network (DRN) implemented in the DeepChem package in previous studies in our group (not shown here).^14,24,25 From our experience, also demonstrated here, RF and ET give equivalent results using the metrics mentioned earlier, however GNB and DRN are inferior compared to RF and ET. In addition, DRN is much slower method than RF, ET, and GNB, so we decided to use RF model because it is a traditional and robust ML model. The ROC-AUC, precision, recall, F1, accuracy, MCC and Cohen′s Kappa scores for Extra Trees (ET), Random Forest (RF), and Gaussian Naïve Bayes (GNB) classifier models for the train and validation data sets of endocrine disruption found in TOX21 data set are presented in Supporting Information (Table S1).

Treating the Imbalanced TOX21 Data Set

We performed oversampling before 80/20 splitting to tune hyperparameters and optimize the model using internal 5-fold cross validation (See below). We also performed several combinations of oversampling and downsampling methods before and after splitting to find the best approach. We found that the best approach is oversampling the minority class before the splitting. First, we used internal validation to evaluate the performance on a data set that is often derived from the same population or distribution as the training data. This approach usually yields overfitting, particularly when imbalanced classes are resampled, leading to optimistic performance metrics. Then, we performed two external validations using the EDC and EDKB-FDA data sets as presented in Supporting Information (Table S2). These external validations tested the model on two entirely independent data sets that were not involved in model development. This step was crucial for assessing the generalizability of the model. This approach gives insights into how well the model fits the data used during training. And, the 5-fold cross validation ensures that the model is tested against various portions of the data, improving reliability. This ensures that the model performs well on unseen data, reflecting its ability to make accurate predictions in real-world scenarios and detects whether the model is overly influenced by artifacts or biases specific to the internal data set. Finally, this approach prevents data leakage and overlap of training and testing data, which can lead to artificially inflated performance metrics.

Hyperparameter Optimization and Statistical Analysis

k-fold cross-validation involves dividing the data set into k subsets (folds) of equal size, using 1-fold for training and the remaining one for validation, iteratively cycling through all folds. This ensures that every data point is used for both training and validation. A smaller k (e.g., k = 3) increases variance in the validation results because fewer data points are used for training, and the folds may not fully represent the data set’s diversity. A larger k (e.g., k = 10) provides more stable and reliable estimates of model performance but increases computational cost. Model robustness refers to the model’s consistency and stability across different folds and unseen data. If ROC-AUC fluctuates significantly with increasing k, the model may rely on specific data patterns rather than learning generalizable features. By analyzing ROC-AUC metrics under different k, it is possible to check for consistency across folds and evaluate generalization. If ROC-AUC remains stable across all folds and across different k values, the model is likely robust to variations in training data subsets. However, significant variability in ROC-AUC across folds may indicate sensitivity to data splits, suggesting the model is not robust. If ROC-AUC scores drop significantly when transitioning from training to validation folds, it suggests the model may be overfitting (performing well on training data but poorly on validation data). Overfitting occurs when the model captures noise or specifics of the training data that do not generalize to unseen data. To detect overfitting using k-fold validation and ROC-AUC, it is important to compare the ROC-AUC scores of the training folds to those of the validation folds. Large discrepancies (training AUC much higher than validation AUC) indicate overfitting. Smaller k values (e.g., k = 3) can exaggerate overfitting, as the training set is smaller, and the model is less likely to generalize well to the validation fold. However, larger k values (e.g., k = 10) mitigate this risk by providing a larger training set and more representative validation results, offering a better assessment of overfitting. The ROC-AUC scores across k = (3, 5, and 10) indicate robustness by a stable or slight variation as presented in Supporting Information (Tables S3, S4, and S5). Then we chose to perform most analysis using the 5-fold cross validation.

We would like to mention that ROC-AUC scores are recognized as good metrics for balanced data sets, however MCC and Kohen′s kappa are better metrics for imbalanced data sets. As our data sets are imbalanced, we preferred to use most of analysis using MCC and Kohen′s kappa metrics. Thus, we preferred to use the best metrics for imbalanced data sets.

The hyperparameter optimization of the RF classifier models obtained with the five subdata sets was investigated to find the best parameters. The optimized hyperparameters are presented in Table 2. While the number of estimators for the RF classifier models ranged between 129 and 446 for each run, the depth of the tree consistently remained 19. Using these optimum parameters, the ROC-AUC metrics for each classification model were always higher than 0.981 and 0.953 for the training and validation data sets, respectively, as shown in Table 3. This indicates an absence of substantial overfitting because 5-fold cross-validation was used.

Table 2. Optimized Parameters in the Hyperparameterization Process for Random Forest (RF) Classifier Models for the Endocrine Disruption of Compounds Using the EDC and EDKB-FDA Datasets^a.

Data set	Task	Depth	Estimators
EDC	AR	19	171
	ER	19	129
	AhR	19	280
	ARO	19	441
	PPAR	19	325
EDKB-FDA	AR	19	175
	ER	19	446
	AhR	19	315
	ARO	19	139
	PPAR	19	313

^aAR stands for androgen receptors; ER—estrogen receptors; AhR—aryl hydrocarbon receptors; ARO—aromatase receptors; PPAR—peroxisome proliferator–activated receptors.

Table 3. Metrics (Mean ROC-AUC) of the Training and Validation Datasets for Random Forest (RF) Classifier Models for the Endocrine Disruption of Compounds Using the EDC and EDKB-FDA Datasets^a.

Data set	Task	Precision of Training	Precision of validation
EDC	AR	0.999	0.998
	ER	0.981	0.953
	AhR	0.996	0.990
	ARO	0.999	0.998
	PPAR	0.999	0.997
EDKB-FDA	AR	0.999	0.997
	ER	0.982	0.959
	AhR	0.997	0.989
	ARO	0.998	0.997
	PPAR	0.999	0.999

^aAR stands for androgen receptors; ER—estrogen receptors; AhR—aryl hydrocarbon receptors; ARO—aromatase receptors; PPAR—peroxisome proliferator–activated receptors.

The evaluation metrics, including precision, recall, F1 score, MCC and accuracy, used to assess the performance of classifier models in the context of endocrine disruptors are presented in Table 4. These metrics are described in terms of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). The F1 score is the harmonic mean of precision and recall. The accuracy of the models was evaluated for both endocrine disruption and nonendocrine disruption, focusing particularly on the performance of RF classifier models. The confusion matrix, presented in Figure 1, was employed to probe the accuracy of classification for endocrine disruption. The importance of building separate models for endocrine disruption and nonendocrine disruption is underscored due to the aim of only obtaining substructures important to represent endocrine disruption. Both models perform well for endocrine disruption and nonendocrine disruption, yet our focus was solely on the models that describe endocrine disruption. Nevertheless, for this classification and for this data set, MCC is a more reliable score, because it only produces a high score if the prediction obtains good results in all four categories of the confusion matrix (TP, FN, TN, and FP), while proportionally considering the sizes of both positive and negative elements in the data set.

Table 4. Precision, Recall, F1, MCC and Accuracy Scores for Random Forest (RF) Classifier Models for the Validation Dataset of Endocrine Disruption Using the EDC and EDKB-FDA Datasets^a.

Data set	Task	Scores
		Precision	Recall	F1	Accuracy	MCC
EDC	AR	0.993	0.927	0.960	0.960	0.923
	ER	0.962	0.995	0.977	0.977	0.956
	AhR	0.917	0.987	0.948	0.948	0.899
	ARO	0.991	0.976	0.984	0.984	0.968
	PPAR	0.989	0.973	0.982	0.982	0.964
EDKB-FDA	AR	0.987	0.907	0.948	0.948	0.899
	ER	0.909	0.926	0.914	0.914	0.828
	AhR	0.914	0.990	0.948	0.948	0.899
	ARO	0.987	0.978	0.983	0.983	0.965
	PPAR	0.993	0.989	0.991	0.991	0.982

^aAR stands for androgen receptors; ER—estrogen receptors; AhR—aryl hydrocarbon receptors; ARO—aromatase receptors; PPAR—peroxisome proliferator–activated receptors.

Figure 1.

Confusion matrix for the RF classification model using the TOX-21 validation data set to build the model to be used for the EDC ((A) for AR, (C) for ER, (E) for AhR, (G) for ARO, and (I) for PPAR) and EDKB ((B) for AR, (D) for ER, (F) for AhR, (H) for ARO, and (J) for PPAR) test data sets.

The parameters of the LIME explainer instance were thoroughly investigated. The number of features was scrutinized. Although the classifier model incorporated 1024 features, less than a dozen were found to be significant for each sample. Adjusting this parameter had no discernible impact on the execution time or results. In our LIME code, it is appropriated to use 1024 fingerprints and used the 100 most important for each molecule as default.

We used a conservative setting of 100 features chosen for safety in the LIME interpretation. The impact of modifying the number of explanations on the probability of prediction for each substructure was also examined, varying from zero to 10, with higher values leading to more refined explanations, but with fewer quantity due to increased complexity. However, a reduction in the number of substructures with a weight greater than 0.1 was observed to be associated with higher values. Consequently, the number of explanations was set to 1, maximizing the quantity of meaningful substructures. It is important to note that higher values resulted in increased evaluation times, exceeding 4 h. This emphasizes the need to avoid excessively high values to ensure reasonable processing times during the evaluation process.

Model Explanation

The LIME explanations for endocrine disruption are presented in the following. We filtered the results with weights greater than 0.1 to exclude less significant substructures. Then, LIME selected the most influential substructures for endocrine disruption for each one of the five nuclear receptors using the two testing data sets EDC and EDKB-FDA. Table S6 shows the summary of the most important chemical compounds classified by the classification models for the AR, ER, AhR, ARO, and PPAR targets. The full list of references related to the endocrine disruption of each chemical compound are in Table S6, not in the text below. The application of each chemical compound is also presented in Table S6. The structural alerts generated by our models are compared with the structural alerts found in literature for AR and ER acitivities.^4,20−23 It is noteworthy to mention that most structural alerts generated here are more specific than those found in literature,^4,20−23 i.e, most alerts are a portion of the structural alert (marked as P in Table S6) found in literature. Some structural alerts are found at vicinity (indicated as V in Table S6) of the alert found in literature. In these cases, it is possible to consider them as new structural alerts (specified as N in Table S6) because they are generally formed as different groups. In some situations (steroidal compounds and Forskolin), several structural alerts encompass most of the molecule structure. In summary, more specific structural alerts than existing ones are compared with those alerts found in the online chemical modeling (OCHEM) platform and/or in literature.^4,20−23 Therefore, it is not possible to perform the statistical comparison of identified groups with set of already existing alerts. In terms of rigor and strictness, they have structural differences that prevent statistical confirmation of agreement. In fact, the structural alerts found here are mostly like those in literature.^4,20−23 Unfortunately, it seems that each new explanation generated by different explainable artificial intelligence methods yields to different structural alerts depending on the model and data set used to train the model, but most alerts are a portion of the existing structural alert. On the other hand, as presented below, each new explanation brings some new knowledge, and in some degree, they are similar and comparable to previous studies.^4,20−23

Figures 2A and 2B present the endocrine disruptors highlighted using the AR model trained with the TOX21 and tested with the EDC and EDKB-FDA data sets. For the AR nuclear receptor, the 1,2-dibromo-4-(1,2-dibromoethyl)cyclohexane compound is the only one highlighted for endocrine disruption using the EDC data set in Figure 2A. It is used primarily as an additive flame retardant in polystyrene foams and contains α- and β- diastereoisomers present in equimolar amounts. The detection of this compound in the environment suggested that isomers are androgenic, indicating its potential risk to the environment due to toxic and biochemical effects of the β-isomer in a controlled laboratory environment.²⁶ It is an agonist to human AR. In vivo, in vitro ligand-binding and receptor-activation assays demonstrated that this compound activates and binds to the human AR, indicating that it is a potential endocrine disruptor.²⁷ There is preliminary evidence of disruption in endocrine and reproductive systems of birds and fish. Molecular docking, in vitro ligand-binding and receptor-activation assays present that 1,2-dibromo-4-(1,2-dibromoethyl)cyclohexane binds to and activates the human AR.²⁸

Figure 2.

Most important substructures found by LIME for EDC (A) and EDKB-FDA (B) test data sets using the AR-EDC and AR-EDKB classification models.

Using the data set EDKB-FDA, Figure 2B shows the highlights of epitestosterone (compounds 0, 1, 2, 3, 5, 6, 7, and 8), which is an endogenous steroid and an epimer of the androgen sex hormone testosterone. It is a competitive AR antagonist and a masking agent for testosterone banned by many sporting authorities.²⁹Figure 2B also presents the highlights of progesterone (compound 4), which stimulates the growth of mammary alveolar tissue and relaxes uterine smooth muscle. It binds to the progesterone and estrogen receptors, but it also has androgenic activity.³⁰

Figures 3A and 3B show the endocrine disruptors highlighted using the ER model trained with the TOX21 and tested with the EDC and EDKB-FDA data sets. Using the EDC data set, zearalenone (compounds 0 and 4) is presented as ER endocrine disruptor in Figure 3A; however, it is a macrolide comprising a 14-membered lactone fused to the potent estrogenic metabolite, 1,3-dihydroxybenzene. It is a nonsteroidal compound with estrogenic activity. As this compound itself is not an endocrine disruptor, but its metabolite is, the highlights may indicate a misinterpretation of the real explanation for endocrine disruption. Therefore, it is important for the reader to pay attention to the LIME interpretation as it may lead to a wrong direction. However, the literature presents in vitro and in vivo evidence for the estrogen effects of the zearalenone and some of its metabolites.^31,32 Compound 1 in Figure 3A is named as quinalphos, an organic thiophosphate compound used as a pesticide, which acts via acetylcholinesterase inhibition. Its thiophosphate group, like an ant antenna, is highlighted to be the cause of endocrine disruption and its shape is like several known endocrine disruptors. The link between endocrine disruptors and acetylcholinesterase (AC) is reported in literature.³³ It decreases testosterone level considerably, and consequently results in reduced sperms in mice.³⁴ Furthermore, it presents AhR-mediated transcriptional activity using in vitro assay.³⁵ Compounds 2 (and 6) and 5 in Figure 3A are very similar. Propylparaben (compounds 2 and 6) and 2-Ethylhexyl 4-hydroxybenzoate (compound 5) are benzoate esters (parabens) that are known as endocrine disruptors. They are members of phenols and parabens, typically found in food additives, drug preservatives, and many cosmetics, such as lotions, shampoos, creams and bath products. They are also antifungal and antimicrobial agents. Their capacity to disrupt endocrine function via estrogen based on in vitro and in vivo data is presented in literature.^36,37 Instead of highlighting the phenol group, LIME highlighted the ester group. Compound 3 in Figure 3A is a bisphenol A compound called hydroxychlor, a biologically active liver metabolite of methoxychlor that is commonly used as pesticide. It acts as an estrogen receptor agonist, and an antagonist at the estrogen receptor beta and androgen receptor.^38,39 It may also disrupt the activities of aromatase.⁴⁰ Instead of highlighting the phenol groups, LIME highlighted the trichloroethane group.

Figure 3.

Most important substructures found by LIME for EDC (A) and EDKB-FDA (B) test data sets using the ER-EDC and ER-EDKB classification models.

Figure 3B shows the highlights generated for the ER model using the EDKB-FDA data set. The compounds highlighted are epitestosterone (compounds 0, 3, 4, and 7), estradiol (compound 1), ethinyl estradiol (compounds 2 and 6), dihydrotestosterone (compound 5), 5α-androstane-3α,17β-diol (compound 8), and progesterone (compound 9). In fact, the concentrations of epitestosterone in human serum are negatively correlated with that of estradiol. The possible explanation for this relation and the influence of epitestosterone on estradiol formation in vitro are due to their structural similarity. As estradiol is an estrogen, the mechanism of action of estradiol is as an ER agonist like epitestosterone.⁴¹ Ethinyl estradiol is an estrogen synthesized from estradiol that exhibits high estrogenic disruption.⁴¹ Dihydrotestosterone is a testosterone in which the 4,5 double bond has been reduced to a single bond with alpha-configuration at position 5. It has a role as an androgen⁴² but it also inhibits estrogen accumulation in granulosa cells,^43,44 and here it is highlighted by LIME as an estrogen-like molecule. 5α-androstane-3α,17β-diol is highlighted like the androgen metabolite, 5α-androstane-3β,17β-diol, that is a potent modulator of estrogen receptor.^45,46 Progesterone is a C21-steroid hormone with a pregnane skeleton that attenuates beneficial neural effects of estrogen, and reduces ER-dependent transcriptional activity and neuroprotection by decreasing expression of ERα and ERβ receptors,⁴⁷ as it is highlighted by LIME. Usually, LIME does not suggest any specific group in steroid-like molecules.

Figures 4A and 4B present the endocrine disruptors highlighted using the AhR model trained with the TOX21 data set and validated with the EDC and EDKB-FDA data sets. Using the EDC data set, the compounds profenofos (compound 0), triadimefon (compounds 1 and 5), isofenphos (compound 2), cybutryne (compound 3), methyl parathion (compounds 4 and 7), and bupirimate (compound 6) are highlighted for AhR activity. Profenofos is an organophosphate insecticide, and member of monochlorobenzenes. It causes endocrine disruption and shows drastic effects on the testicular tissues in rabbits via AR and AhR activities.³⁵ Triadimefon is a member of triazoles and monochlorobenzenes. It increases expression and enzymatic activities of a series of related cytochrome P450s (CYP), which is evidence to its effects on nuclear receptors, and causes thyroid endocrine disruption, but it is unknown to present AhR activity.³⁵ Instead of the triazole and monochlorobenzene, the ketone group was found to be the cause of endocrine disruption by LIME. Isofenphos is an organothiophosphate insecticide that has a role as an AC inhibitor, and consequently, there is an association with endocrine disruption, but it does not present any AhR activity.⁴⁸ The thiophosphate group in Profenofos and Isofenphos was found to be the cause of endocrine disruption by LIME. Cybutryne is a diamino-1,3,5-triazine that has a role as an organothiophosphate insecticide. No information regarding endocrine disruption was found for Cybutryne in literature. Instead of the triazine group, LIME found that the secondary amine group is the cause of endocrine disruption. Methyl parathion is an AC inhibitor; thus, its action mechanism of endocrine disruption is via AR activity.⁴⁸ The thiophosphate group in Methyl Parathion was found to be the explanation for its endocrine disruption. Bupirimate is an antifungal agrochemical that has a role as an androgen antagonist. It also activates the pregnane X cellular receptor, but it is unknown to show AhR activity.^35,48 The sulfamate ester group is highlighted to explain the cause of endocrine disruption in Bupirimate.

Figure 4.

Most important substructures found by LIME for EDC (A) and EDKB-FDA (B) test data sets using the AhR-EDC and AhR-EDKB classification models.

Using the EDKB-FDA data set (Figure 4B), eucalyptol (compounds 0 and 1), carbaryl (compound 2), propanil (compound 3), parathion (compound 4), melatonin (compound 5), and linuron (compound 6) are highlighted for AhR activity in Figure 4B. Eucalyptol is a monoterpenoid that controls airway mucus hypersecretion and asthma via anti-inflammatory cytokine inhibition; however it also has AR and ER activities, and it directly binds to AhR.⁴⁹ The cyclic ether group was found to be the cause of endocrine disruption by LIME. Carbaryl is a carbamate ester that has a role as an AC inhibitor.⁵⁰ It can activate AhR, and was first believed not to be an AhR ligand, but then it was proved to be a weak ligand.⁵¹ Later, it was also found to act as an AhR agonist in AhR-mediated reporter gene assays. Propanil is an anilide that has a role as herbicide with AC activity, and consequently, AR activity.³³ It also has ER activity and a potent AhR agonist activity.^35,48 Parathion is an organothiophosphate insecticide that has a role as an AC inhibitor, and consequently it has AR activity.³³ It also activates AhR and stimulates the formation of an AhR-DNA-binding complex.^35,48 Melatonin is a hormone secreted by the pineal gland in humans that has a role in the treatment of sleep disorders. Melatonin and its indolic and kynuric metabolites act as agonists to AhR, and are proved to stimulate the AhR protein translocation from the cytoplasm to the nucleus in human keratinocytes.⁵² It also has AR and ER activities.^53,54 Linuron is a N-methyl phenylurea that has a role as herbicide. It is an AC inhibitor, and has AR and AhR activity.^35,48 LIME found that the anilide group is the cause of endocrine disruption in Carbaryl, Propanil, Melatonin, and Linuron. On the other hand, the thiophosphate group in Parathion was found to be the explanation for its endocrine disruption.

Figures 5A and 5B show the endocrine disruptors highlighted using the ARO model trained with the TOX21 data set and tested with the EDC and EDKB-FDA data sets. Using the EDC data set, the compounds forskolin (compounds 0, 2, 4 and 5), chlorfenvinfos (compounds 1 and 8), methyl paraoxon (compound 3), tolylfluanid (compound 6), and enilconazole (compound 7) are highlighted for ARO activity in Figure 5A. Forskolin is a diterpenoid that has AR,⁵⁵ ER,⁵⁶ and ARO⁵⁷ activities. It also produces PPAR isotype-specific mRNA, protein effects and a consistent decrease of ER-α mRNA levels.⁵⁸ It was not found any specific group in Forskolin that causes the endocrine disruption. Chlorfenvinfos is a dichlorobenzene phosphate that has a role as an AC inhibitor.^35,48 LIME found that the side-chain phosphorus group is the cause of endocrine disruption. Methyl paraoxon is the active metabolite of the organophosphate insecticide parathion that acts as a cholinesterase inhibitor.^35,48 Paraoxon is one of the most potent acetylcholinesterase-inhibiting insecticides. The side-chain phosphorus group is found to be the cause of endocrine disruption by LIME. Tolylfluanid is a sulfamide that has a role as fungicide and wood preservative. The sulfamide group is highlighted by LIME to be the cause of endocrine disruption. To the best of our knowledge, the literature reports that it only has activity on the insulin receptor.⁵⁹ It is identified as a potent endocrine disruptor using in vitro and ex vivo studies.⁶⁰ Enilconazole is an imidazole that has role as fungicide with AR, ER, and ARO activities.⁶¹ It is a surprise that neither imidazole nor dichlorobenzene groups are the cause of endocrine disruption. Instead, the ether branched group was found by LIME to be the cause of endocrine disruption. The literature shows a need for in vivo studies to establish the extent of endocrine-disrupting effects.⁶²

Figure 5.

Most important substructures found by LIME for EDC (A) and EDKB-FDA (B) test data sets using the ARO-EDC and ARO-EDKB classification models.

Using the EDKB-FDA data set (Figure 5B), the compounds S-fulvestrant (compound 1, 3, 5, and 8), estradiol (compound 2), and tamoxifen (compound 4) are highlighted for ARO activity in Figure 5B. Fulvestrant (S enantiomer) is an ER antagonist with a novel pharmacological profile, and it does not present agonist activity. It has been shown to significantly reduce cellular levels of the AR, ER and progesterone receptor in both preclinical studies and clinical trials of postmenopausal women with primary breast cancer.^63,64 In addition, it exists information on its ARO activity that catalyzes estradiol synthesis.⁶⁵ Estradiol can act as an antiaromatase agent. However, the factors regulating aromatase activity are unknown.⁶⁶ Tamoxifen is a stilbenoid that has AR activity, is an ER antagonist and upregulates ARO expression.⁶⁷⁻⁶⁹

Figure 6 shows the endocrine disruptor from the EDC data set highlighted using the PPAR model trained with the TOX21 data set and validated with the EDC and EDKB-FDA data sets. This model was not able to find any endocrine disruptor from the EDKB-FDA data set. The compound 2-(thiocyanomethylthio)benzothiazole from the EDC data set is a wood preservative, marine biocide, fungicide, and a preservative in paint. It is an emerging contaminant with only one study on PPAR activity and no activity for any other nuclear receptor activity up to date.⁷⁰ LIME found that the side chain and thiocyanate group are the explanation for endocrine disruption in the 2-(thiocyanomethylthio)benzothiazole.

Figure 6.

Most important substructures found by LIME for EDC (A) and EDKB-FDA (B) test data sets using the PPAR-EDC and PPAR-EDKB classification models.

These structural explanations should not be regarded as strict rules. Our goal is to create structural interpretability using LIME and assess whether the most important toxic alerts for endocrine disruption align with human scientific understanding. It is important to acknowledge that LIME can make incorrect interpretations, which are not the reason for endocrine disruption. Although LIME highlights the side group in propylparaben (compounds 2 and 6 in Figure 3A), it is known as an inactive group, and the active group is the phenol group. Similarly, LIME seems to be nonspecific in some cases such as flame-retardant compounds (Figure 2A) and steroid-like compounds (Figure 2B). In both cases, LIME is unable to select one specific group and highlights the whole molecule. Another issue is evident in metabolites because the parent molecule is not the real endocrine disruptor, but LIME predicts the existing substructures in both the parent molecule and metabolite, and sometimes this substructure is not the reason for endocrine disruption as exemplified in zearalenone (compounds 0 and 4 in Figure 3A). In this case, the macrolide comprising a 14-membered lactone is not the cause of endocrine disruption, but its metabolite, 1,3-dihydroxybenzene, is. Despite these limitations, the interpretation of important substructures for endocrine disruption with a certain level of confidence can be used as a basis for analyzing counterfactuals, which suggest structural changes in molecules to reduce or eliminate their endocrine disruption.

Concluding Remarks

The skepticism about the effectiveness of LIME in aiding researchers stems from several factors. The design of new molecules with desired safety in terms of toxicology is a complex problem. LIME can explain individual model predictions, but it may not fully capture the nuances of molecular interactions and structural factors for endocrine disruption. Its explanations are limited to instance-specific, local insights and may not provide a holistic view, which is essential for molecule design. Furthermore, the interpretation of results requires expertise in both chemistry and machine learning, making it challenging for researchers lacking strong backgrounds in both fields to use effectively. Any insights from LIME would still need experimental validation, which can be costly and time-consuming. Most importantly, LIME can offer insights into machine learning model predictions in toxicology, but it may have limited utility in guiding the synthesis of new molecules due to the lack of experimental findings about structural alerts. Researchers should consider these potential drawbacks and limitations when incorporating LIME into their workflow.

The use of state-of-the-art techniques in explainable machine learning applied to toxicology may limit the understanding and interpretation of predictions from complex machine learning models. LIME offers several advantages over traditional explainable machine learning methods. It provides instance-specific and localized explanations, which are often more intuitive and easier to understand than global model level explanations. It is model agnostic, i.e., it can be applied to any machine learning model, independent of complexity or underlying architecture. It also generates typically simpler, quicker, and more transparent explanations than other methods, enhancing accessibility for nonexperts and fostering trust in model predictions. The identification of important features that influence individual predictions allows researchers to discern the features driving the model predictions. Moreover, it sheds light into complex models, such as deep neural networks, where traditional explainable machine learning methods may struggle. Then, it can improve the ability of researchers to interpret and trust machine learning model predictions in toxicology, leading to more informed decision-making in environmental risk assessment, and environmental policy and regulation.

Some machine learning models treat the prediction task as a black box model, take the input derived from a chemical structure, and produce a predicted value. However, these models do not provide an explanation for their predictions by itself, making it difficult for the user to evaluate ideas. On the other hand, it would be valuable to have random forest models that provide human-interpretable explanations, driving subsequent molecule design.

Although there has not been much progress in explainable machine learning for molecules, toxicity data sets have been successfully interpreted using LIME.¹⁴ The challenge has been to get experimental evidence to validate the explanations of unknown compounds or their action molecular mechanisms in real biological entities. The explanation depends on the data set. Models built with a data set give different results from models created by another data set. Remarkably, it is important to focus benchmarking efforts on simple, robust, relevant, diverse, and clearly defined end points for endocrine disruption with different nuclear receptors. These end points must be reproducible, relatively inexpensive, and can provide reasonably large data sets, preferentially performed with the same methodology and in the same lab. Consequently, it would be possible to make better structural explanations for each nuclear receptor. Additionally, LIME¹⁴ may sometimes ignore substructures, which are normally responsible for endocrine disruption. Also, compounds with two substructures responsible for endocrine disruption are highlighted with the explanation of the most relevant one. Finally, it is also noteworthy to comment that these structural explanations cannot be interpreted as definitive rules. Nowadays, we focus on creating reliable structural interpretability from LIME18¹⁵ and assessing whether the most important substructures to endocrine disruption are consistent with existing human scientific understanding. And, in a future perspective, we can possibly predict new trustful molecular insights on each new endocrine disruptor using the developed models with benchmarking data sets.

The analysis of endocrine disruption using LIME explanations for the EDC and EDKB-FDA data sets reveals multiple significant substructures associated with the endocrine disruption of chemical compounds. They were classified in terms of interactions with five specific nuclear receptors: AR, ER, AhR, ARO, and PPAR. Notably, LIME identified substructures such as thiophosphate, sulfamate ester, anilide, carbamate, sulfamide, and thiocyanate groups as influential in endocrine disruption. It is important to acknowledge that some compounds presented only endocrine disruption data in animals, with no evidence in humans available in the literature. However, the comprehensive evaluation revealed instances where detected toxic alerts aligned with expectations and highlighted potential features associated with endocrine disruption. The relevance of LIME in explaining endocrine disruption is consistent across most outputs. In summary, the use of LIME to explain endocrine disruption in the EDC and EDKB-FDA data sets provides valuable insights into the specific substructures that contribute to the classification of compounds as endocrine disruptors. The detailed analysis of LIME results enhances the overall understanding of the endocrine disruption of the analyzed compounds that lack specific information in literature. Identifying of more specific toxic alerts that contribute to endocrine disruption is crucial for the development of alternative methods to evaluate the risks of chemical exposures, both on environmental and human health, without relying on animal testing. The outcomes of our findings go beyond theoretical scopes, providing tangible benefits for environmental monitoring agencies, regulatory agencies, and pharmaceutical and food industries. The capacity to accurately predict endocrine disruption grants decision-makers essential tools to assess the safety of chemicals and enhance environmental risk assessments.

Data Availability Statement

Code, input data and processing scripts for this paper are available at GitHub repository (https://github.com/andresilvapimentel/endocrine-disruption-explainer). The data analysis scripts of this paper are also available in the interactive notebook Google Colab.

Supporting Information Available

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/envhealth.4c00218.

• The ROC-AUC, precision, recall, F1, accuracy, MCC and Cohen′s Kappa scores for Extra Trees (ET), Random Forest (RF), and Gaussian Naïve Bayes (GNB) classifier models for the train and validation data sets of endocrine disruption found in TOX21 data set. The external validation using the precision score for Random Forest (RF) classifier models in the EDC and EDKB-FDA data sets of endocrine disruption. The ROC-AUC, precision, recall, F1, accuracy, MCC, and Cohen′ Kappa scores for Random Forest (RF) classifier models for the train and validation data sets of endocrine disruption found in TOX21 data set using the (3, 5, and 10)-fold cross validation method. The full list of references related to the endocrine disruption of each chemical compound (PDF)

Author Contributions

L.C.S.R. : conceptualization, methodology, software, validation, formal analysis, investigation, data curation, writing - review & editing, and visualization. M.S.: validation, formal analysis, investigation, data curation, writing - review & editing, and visualization. A.S.P.: resources, writing - review & editing, visualization, supervision, project administration, and funding acquisition.

The authors declare no competing financial interest.

Special Issue

Published as part of Environment & Healthspecial issue “Artificial Intelligence and Machine Learning for Environmental Health”.