Predicting Prenatal Developmental Toxicity Based on the Combination of Chemical Structures and Biological Data

Abstract

For hazard identification and classification and labeling purposes, animal testing guidelines are required by law to evaluate developmental toxicity for new and existing chemical products. However, guideline developmental toxicity studies are costly, time-consuming, and require many laboratory animals. Computational modeling has emerged as a promising, animal-sparing, and cost-effective method for evaluating the developmental toxicity potential of chemicals, such as endocrine disruptors, without the use of animals. We aimed to develop a predictive and explainable computational model for developmental toxicants. To this end, a comprehensive dataset of 1,244 chemicals with developmental toxicity classifications was curated from public repositories and literature sources. Data from 2,140 toxicological high throughput screening (HTS) assays were extracted from PubChem and the ToxCast program for this dataset and combined with information about 834 chemical fragments to group assays based on their chemical-mechanistic relationships. This effort revealed two assay clusters containing 83 and 76 assays, respectively, with high positive predictive rates for developmental toxicants identified with animal testing guidelines (PPV = 72.4% and 77.3% during cross-validation). These two assay clusters can be used as developmental toxicity models and were applied to predict new chemicals for external validation. This study provides a new strategy for constructing alternative chemical developmental toxicity evaluations that can be replicated for other toxicity modeling studies.

Introduction

Traditional chemical toxicity evaluation methods rely on animal testing guidelines for hazard identification. These animal studies are required by law and are considered the gold standard for safety assessment testing. However, these studies are expensive, time-consuming, and require highly specialized study designs to detect developmental toxicants^1,2. Furthermore, these methods raise ethical concerns because of the many laboratory animals required. The European Registration, Evaluation, Authorization, and Restriction of Chemicals (REACH) regulations have extensive developmental and reproductive toxicity (DART) testing requirements for industrial and consumer chemicals^3,4. DART testing represented 90% of animal use and 70% of chemical toxicity testing costs associated with completing phase one of REACH³, with individual testing protocols sometimes requiring up to 3,200 animals per chemical⁴. One of the most well-known regulatory testing requirements for this purpose is the prenatal developmental toxicity study, described as the Organization for Economic Co-operation and Development (OECD) Test No. 414⁵ and the United States Environmental Protection Agency (US EPA) Office of Prevention, Pesticides, and Toxic Substances (OPPTS) test guideline 870.3700⁶. Briefly, this protocol requires the administration of a test chemical to 4 groups of animals (e.g., at least 20 pregnant rats or rabbits with litters per group) at three different concentrations from the time of implantation of the embryo in utero and continuing throughout pregnancy until one dayterm before the expected day of delivery. A comprehensive array of developmental endpoints is defined based on the testing results, fetal body weight, embryo-fetal survival, fetal morphology (external, visceral, skeletal), and endocrine endpoints, to determine No Observed Adverse Effect Levels (NOAELs) or Low Observed Adverse Effect Levels (LOAELs) values for maternal and developmental toxicity effects.

The complexity and cost of the associated animal testing guidelines for prenatal developmental toxicity have produced a critical need to develop alternative approaches based on non-animal models, such as computational models. However, most current computational toxicology models for developmental toxicity predictions were developed using Quantitative Structure-Activity Relationship (QSAR) modeling and other structure-based approaches. These approaches only incorporated structural and physicochemical information of known toxicants^7–13. Unfortunately, for complex in vivo endpoints such as developmental toxicity, relying on structural and physicochemical information alone for modeling and evaluations can be error-prone^14,15. For example, compounds with similar structures may exhibit dissimilar toxicities, a phenomenon called activity cliffs, and cause incorrect predictions for new compounds^16,17. Furthermore, international guidance regarding the chemical risk assessments requires that the toxicity predictions of new compounds need to have identified toxicity mechanisms^18–20.

Over the past twenty years, advances in high-throughput screening (HTS) protocols and combinatorial chemistry revolutionized the environmental and health science data landscape^21–23. Initiatives such as the US EPA’s Toxicity Forecaster (ToxCast) program, which screened approximately 1,800 chemicals in over 700 HTS assays, generated large amounts of biological data for mechanistic toxicity evaluations^24,25. At the same time, a collaboration among the US EPA, National Center for Advancing Translational Sciences (NCATS), and the National Toxicology Program (NTP) launched a parallel initiative called Toxicity in the 21^st Century (Tox21). The goal of Tox21 was to generate more detailed toxicity data, such as concentration-responses, for a larger chemical library using quantitative HTS protocols. The collaborative Tox21 initiative is ongoing, now including the Food and Drug Administration (FDA) and working toward a goal of testing about 10,000 chemicals in about 70 HTS assays^26–29. Public databases such as PubChem host much of this data, making it available for modeling studies^30,31. Integrating biological data into computational toxicity evaluations has shown great promise in addressing the backlog of registered chemicals that have not undergone complete safety assessments^32–40. However, these models only incorporated manually selected biological data covering a well-known and narrow scope of possible mechanisms relevant to developmental toxicity^38–41. Automatic data mining methods that can extract relevant public biological data offer a new strategy to shed light on unknown toxicity mechanisms that are not incorporated into the existing models²³.

This study aimed to address the above limitations of the existing computational approaches for evaluating developmental toxicity using a data-driven read-across approach by combining chemical information and biological data (Figure 1). To this end, a large dataset for developmental toxicity was collected from public databases and literature sources. An automated data mining web tool was then used to extract biological data from PubChem for all of the chemicals in this dataset⁴². Additional biological data generated as part of the ToxCast initiative for these chemicals were collected from the US EPA’s CompTox Chemistry Dashboard⁴³. Then, the collected PubChem and ToxCast assays were clustered based on their biological mechanisms using a recently developed method that incorporates the structural features of the chemicals in the developmental toxicity dataset³³. This process revealed chemical in vitro-in vivo relationships used to reveal suites of assays representing developmental toxicity mechanisms. The assays with extra chemical information can be used as models to screen new compounds to prioritize potential toxicants.

Figure 1:

Overview of the workflow used in this study. The data-driven workflow employed in this study consists of four main stages: generation of chemical fingerprints and retrieval of in vitro bioassay data for each compound in the training set, implementation of QSAR models to fill data gaps, clustering PubChem and ToxCast assays based on correlations between chemical fragments and bioactivity, and identification of potential developmental toxicants by mechanism-driven read-across. Orange cells in the bioprofile with a value of 1 represent active results, gray cells with a value of 0 represent inconclusive/missing results, and blue cells with a value of −1 represent inactive results.

Methods

In Vivo Prenatal Developmental Toxicity Database

Chemicals with prenatal developmental toxicity data were collected from public database resources^44–46 and individual datasets in the literature⁷. The definition of developmental toxicant was limited to adverse effects selective to the developing embryo or fetus after pregnant animal exposure. Therefore, non-toxicants in this study were limited to chemicals showing no adverse effects in the developing embryo or fetus in prenatal testing or only showing non-selective embryo-fetal effects (i.e., secondary effects from toxicity in the pregnant test animal).

The European Chemical Hazards Agency (ECHA) and Toxicity Reference Database (ToxRefDB)^44,45 datasets consisted of 529 and 156 chemicals, respectively, with experimental data generated by test protocols similar to OECD 414⁵/OPPTS 870.3700⁶. These datasets contained NOAEL values associated with prenatal developmental toxicity testing of chemicals administered to preclinical mammalian species (e.g., dogs, monkeys, rabbits, and rodents) orally or by inhalation. First, results marked as unreliable or unacceptable by ECHA or the EPA were removed from the datasets. Then, the NOAEL values were converted to developmentally toxic and nontoxic classifications. Chemicals with oral NOAEL values below the OECD 414/OPPTS 870.3700 limit dose of 1,000 mg/kg/day and inhalation NOAEL values below the OPPTS 798.4350 limit dose of 5,100 mg/m³ were classified as developmentally toxic. Because this study focuses on identifying selective prenatal developmental toxicants, chemicals were classified as developmentally toxic only if they had lower developmental NOAEL values than pregnant test animal NOAEL values derived from the same prenatal study. In some cases, chemicals had study results available in more than one species. Where the presence of developmental effects varied across species for a chemical, the chemical was conservatively considered toxic if it exhibited selective prenatal developmental toxicity in at least one species.

The Proctor & Gamble⁷ dataset consisted of 637 compounds with primarily mammalian data. Chemicals were classified into five categories: D (developmental toxicity in the absence of maternal toxicity), D(MT) (developmental toxicity only in the presence of maternal toxicity), DTer (teratogenicity in the absence of maternal toxicity), DTer(MT) (teratogenicity in the presence of maternal toxicity), and No Evidence (no developmental toxicity observed)⁷. These categories were derived using similar criteria to those used to classify the ECHA and ToxRefDB chemicals (i.e., chemicals classified as D(MT) and DTer(MT) showed developmental or teratogenic NOAEL below the pregnant test animal NOAEL). Therefore, chemicals with the D and DTer classifications were considered developmentally toxic, and those with all other categories were considered nontoxic, consistent with this study’s selective developmental toxicity focus.

Chemical structure information was collected for the chemicals from each dataset using PubChem’s identifier exchange service⁴⁷ using their CAS number, European Community (EC) number, common name, or Distributed Structure-Searchable Toxicity (DSSTox) identifiers as input⁴⁸. The CASE Ultra v1.8.0.0 DataKurator tool curated the individual datasets by standardizing their chemical structures. Salts were treated as their corresponding organic acids. Positively charged nitrogen atoms were neutralized by subtracting hydrogens, and all negatively charged atoms were neutralized by adding hydrogens. After these curation steps, the datasets were merged. In some cases, chemicals with duplicate curated structures showed conflicting designations across the component datasets, and the most conservative designation (i.e., developmentally toxic) was kept. The fully compiled in-house prenatal developmental toxicity database was used as the training set in this study and contained 1,244 unique chemicals, of which 660 were active, and 584 were inactive (Supplementary Table 1).

In Vitro Data Collection and Bioassay Clustering

A bioprofile was generated for the chemicals in the developmental toxicity database using public in vitro bioassay hit calls. PubChem bioassay hit calls were automatically collected for these chemicals using our in-house Chemical In Vitro-In Vivo Profiling tool (http://ciipro.rutgers.edu/)^37,42. ToxCast hit calls were retrieved from the EPA ToxCast/Tox21 summary files for invitroDBv3.2^43,49. Bioassays with at least five active responses among the training set chemicals were retained for the final bioprofile to reduce the chances of overfitting due to limited data³³.

These assays were then clustered into mechanistically related groups using statistically significant relationships among active results and the presence of specific chemical fragments using a recently published workflow³³. First, Saagar fingerprints were generated for all training set chemicals⁵⁰. The Saagar fingerprints consist of binary vectors denoting the presence or absence of approximately 1,000 chemical fragments for chemical toxicity studies⁵⁰. Next, each substructure was compared pairwise with each bioassay using Fisher’s exact test to determine the statistical significance of the fragment’s presence in a chemical with its activity in the bioassay. The Fisher’s exact test’s outcome is a p-value, which indicates the probability of a chemical fragment-in vitro bioassay response relationship existing by random chance. The relationship between the existence of a chemical fragment in an assay dataset and these chemicals’ active responses was considered statistically significant if its corresponding p-value was less than 0.05 (i.e., a random-chance probability of less than 5%). Compounds showing active results in mechanistically related bioassays are likely to have the same key chemical fragments. Therefore, the bioassays can be clustered based on their statistically significant relationships to chemical fragments.

Biological Read-Across to Determine Developmental Toxicity Potential

Each mechanistically related cluster of bioassays was used to perform read-across to assess the chemical developmental toxicity using a biosimilarity search (Figure 1). Biosimilarity refers to the similarity between patterns of results in a battery of bioassays, such that chemicals showing similar patterns have higher biosimilarity^42,51. A frequent challenge associated with using HTS screening data for toxicity evaluations is the bias toward inactive results, which is confounded by the relatively lower importance of inactive responses than active responses^22,23. Therefore, a biosimilarity search assigns a lower weight to inactive responses to minimize the impact of this bias (Equation 1)⁴². In this equation, A_a and B_a represent the active responses in two bioassays within the same cluster, and A_i and B_i represent their inactive responses. In addition, the parameter w, which equals the ratio of active to inactive responses present among the bioassays in each cluster, lessens the impact of the bias usually present toward inactive responses.

$$biosimilarity(A,B)=\frac{|A_a\cap B_a|+|A_i\cap B_i|\times w}{|A_a\cap B_a|+|A_i\cap B_i|\times w+|A_a\cap B_i|+|A_i\cap B_a|\mathrm{ }}$$ (1)

However, missing data are common when profiling target compounds against public data^22,23. Therefore, a high biosimilarity value does not always ensure a confident prediction. For example, the value may have been generated by comparing two chemicals with only one shared active bioassay response among many inactive and missing results. For this reason, an additional confidence parameter was calculated for each read-across prediction, representing the richness of assay data used to make the prediction and, therefore, indicates the prediction reliability (Equation 2).

$$confidence(A,B)=|A_a\cap B_a|+|A_i\cap B_i|\times w+|A_a\cap B_i|+|A_i\cap B_a|$$ (2)

Five-fold cross-validation was performed for each bioassay cluster to assess its ability to predict developmental toxicity potentials for new compounds. First, the target chemicals (i.e., the compounds in the developmental toxicity modeling set) tested in at least one bioassay within a cluster were randomly divided into five equally sized groups. Then, in each of five iterations, each chemical in one group was compared to the compounds of the other four groups, and its developmental toxicity was predicted by its most biosimilar neighbor. This procedure optimized the minimum biosimilarity between two compounds and associated confidence for each cluster using an exhaustive grid-search algorithm implemented in scikit-learn v0.24.1⁵². Briefly, read-across predictions were made using each combination of parameters to identify the best-performing conditions, which were retained and then used to predict external compounds. When multiple bioassays within the same cluster contained at least ten mutual responses across the represented chemicals and had a Pearson correlation coefficient greater than 0.9, one was selected at random and retained for read-across predictions. The others were removed to reduce inflation of biosimilarity values by bioassays that may be represented more than once in the dataset (e.g., a bioassay deposited into PubChem twice with slightly different interpretations).

The predictions resulting from the cross-validation procedure were statistically evaluated using various universal parameters, including specificity (Equation 3), sensitivity (Equation 4), Correct Classification Ratio (CCR, Equation 5), and positive predictive value (PPV, Equation 6). The number of true positives (TP) represents correctly predicted prenatal developmental toxicants in each equation. The number of false positives (FP) represents nontoxic chemicals incorrectly predicted as prenatal developmental toxicants. The number of true negatives (TN) represents correctly predicted nontoxic chemicals. The number of false negatives (FN) represents prenatal developmental toxicants incorrectly predicted as nontoxic. Because toxic predictions are more meaningful than nontoxic predictions, only the PPV was used during each cluster’s read-across parameter optimization.

$$sensitivity=\mathrm{ }\frac{TP}{TP+FN}$$ (3)

$$specificity=\frac{TN}{TN+FP}$$ (4)

$$CCR=\frac{sensitivity+specificity}{2}$$ (5)

$$PPV=\frac{TP}{TP+FP}$$ (6)

Quantitative Structure-Activity Relationship Modeling to Fill Data Gaps

Read-across predictions are only reliable if their associated confidence value is above a minimum threshold. Therefore, chemicals with substantial amounts of missing bioassay data cannot be predicted without filling these gaps. This data gap filling was accomplished using QSAR modeling, similar to previous studies^33,41. First, all the chemicals tested in a given bioassay were retrieved along with their structural information and activities. This information was collected by querying the PUG-REST web service⁴⁷ for PubChem bioassays or invitroDBv3.2^43,49 for ToxCast bioassays. After removing inconclusive results, each bioassay’s dataset was balanced by randomly removing inactive chemicals until reaching an equal number of active and inactive chemicals. For large bioassay datasets, the number of chemicals used for QSAR model development was limited to 10,000 to save computational time.

Four machine-learning algorithms were then used to develop the QSAR models for each assay in the target clusters: Bernoulli naïve Bayes (BNB), k-nearest neighbors (kNN), random forest (RF), and support vector machines (SVM). Each algorithm was implemented using the Python machine-learning library scikit-learn v0.19.0 (http://scikit-learn.org/)⁵² using the RDKit v2019.09.1.0 (http://rdkit.org/) implementation of Functional Connectivity FingerPrints (FCFPs) using a bond radius of 3⁵³. Finally, the models’ predictive performances were evaluated using the CCR from a five-fold cross-validation procedure (Equation 6).

Each of the four algorithms’ hyperparameters was previously described in detail⁵⁴, along with the procedure used to optimize them during model training^53,54. BNB models “naively” assume that the presence or absence of each FCFP descriptor is independent of all others to calculate the probability that a chemical will be active in a particular bioassay by applying Bayes’ Theorem^55,56. kNN models predict a new chemical’s activity by a majority vote of its k most similar training set chemicals⁵⁷. RF models build decision trees based on randomly selected FCFP descriptors and average their outputs to predict new chemicals’ activities⁵⁸. Finally, SVM models optimize a set of thresholds for each descriptor used in model development that best distinguishes between active and inactive training chemicals⁵⁹.

The predictions generated by each of the four algorithms for a single chemical were averaged to form a consensus prediction. Consensus predictions are robust because they leverage the strengths of various algorithms and have shown advantages for predicting new compounds in previous studies^53,60–63. A chemical similarity-based applicability domain was also implemented by only reporting predictions for chemicals with a minimum of 40% similarity using FCFPs across their three nearest neighbors among the QSAR model training set for a particular assay. However, the consensus predictions showed poor predictive performance despite these precautions in rare cases (i.e., CCR < 0.6). In these cases, missing data were not populated with QSAR predictions.

Results

Prenatal Developmental Toxicity Database Overview

The prenatal developmental toxicity database used in this study consisted of data generated by regulatory animal test guidelines (i.e., OECD 414/OPPTS 870.3700) or guideline-like protocols compiled from the literature⁷ and public databases⁴⁴. Because the sources had different data types (e.g., study-derived NOAELs versus data sorted into categories), original data were harmonized into two groups for model development. In the final database, chemicals labeled as toxic were selective embryo-fetal toxicants associated with adverse effects on developing embryos and fetuses. Chemicals labeled as nontoxic showed no adverse developmental effects or only showed adverse effects in the presence of pregnant animal toxicity. Among the three data sources, 68 chemicals existed more than once. Twenty-six (38.2%) of these chemicals had conflicting toxicity classifications across the data sources. In these circumstances, the most conservative classification was retained, such that any chemical with at least one result indicating toxicity was considered toxic in the database. The final database contained 1,244 unique chemicals, of which 660 (53.1%) were toxic, and 584 (46.9%) were nontoxic (Supplementary Table 1). Principal Component Analysis (PCA) was implemented using 206 descriptors available within the Molecular Operating Environment (MOE) software v2018.01 to visualize the chemical space covered by the final database. The top three principal components explain 54.5% of the total variance of the final database and show a sufficiently large and diverse chemical space, aside from several outliers from both classes (Supplementary Figure 1).

Profiling and Assay Clustering

The public in vitro bioassay data for the target chemicals in the training set were extracted from PubChem^30,31 and the EPA ToxCast/Tox21 summary files for invitroDBv3.2^43,49. The resulting in vitro bioprofile used for clustering contained 2,140 bioassays, of which 1,590 (74.3%) were from PubChem and 550 (25.7%) were from invitroDBv3.2 (Figure 2). In total, this bioprofile contained 41,570 active results (1.8%), 325,900 inactive results (13.9%), and 1,967,270 inconclusive results or untested chemical-bioassay pairs (84.3%) for 1,091 training chemicals (Figure 2). This profiling result was then used to cluster mechanistically related bioassays based on the presence of specific chemical fragments. To this end, the presence or absence of 834 Saagar fragments was identified for each chemical.

Figure 2:

Bioprofile of 1,091 in vivo developmental toxicity database chemicals (y-axis) used for training across 2,140 PubChem and ToxCast bioassays (x-axis). Active results are shown as red squares, inactive results are shown as blue squares, and inconclusive results and untested chemical-bioassay pairs are shown as gray squares.

Among the 2,140 bioassays and 834 fragments, 61,928 statistically significant chemical-in vitro relationships were identified (p < 0.05) (Supplementary Figure 2). Next, mechanistically related bioassays were clustered by calculating the Jaccard distance between all pairs of bioassays using their profiles of existing significant chemical fragments. Bioassay pairs with a Jaccard similarity of at least 25% were used for clustering. After applying this minimum similarity threshold, 1,091 bioassays (51.0%) were considered to have “unique” mechanisms (i.e., < 25% similarity to all other individual bioassays) for the training set compounds and therefore excluded from further modeling. Therefore, 1,049 bioassays with presumed mechanistic connections to other bioassays remained available for the clustering analysis.

A network graph was generated to show the bioassay clusters (Figure 3). Each node represented one of the remaining 1,049 bioassays. Each edge connected two bioassays and had a length inversely representing the Jaccard similarity between the two bioassays (i.e., shorter edges indicate higher similarity). This network graph was then clustered using the Louvain modularity algorithm⁶⁴ with resolution 0.35 and visualized using the Force Atlas algorithm⁶⁵ with default parameters, as implemented in Gephi v0.9.2 (https://gephi.org/) (Figure 3). This process resulted in 68 clusters (Supplementary Table 2). Among the resulting clusters, 37 contained mixtures of bioassays from the PubChem and ToxCast databases. In Figure 3, each bioassay is represented by colored circles, and edges represent potential mechanistic relationships (i.e., bioassay pairs with > 25% similarity).

Figure 3:

Cluster map of in vitro PubChem and Toxicity Forecaster (ToxCast) assays based on correlations between chemical fragments and assay activities. The Louvain modularity algorithm identified 68 clusters, each represented by a different color, as outlined in Supplementary Table 2. Each colored circle represents one bioassay such that circles of the same color belong to the same cluster. Each edge inversely correlates to the similarity between the connected assays, with longer edges representing lower similarity.

Read-Across Model Selection

Each of the 68 clusters depicted in Figure 3 was evaluated for their ability to predict prenatal developmental toxicity. First, 47 clusters containing less than five bioassays were removed to ensure the predictions could be biologically interpretable (Supplementary Table 2). Next, the remaining 21 clusters were evaluated to predict prenatal developmental toxicity using biological read-across studies. Each read-across study was evaluated using a five-fold cross-validation procedure.⁶⁶ The minimum biosimilarity and confidence required to make read-across predictions were optimized during cross-validation. Then, the optimized PPV for each cluster was recorded, resulting in PPV values ranging from 51.4% to 100.0% (Equation 7, Figure 4A). Among the 21 clusters, 9 showed high predictive performance for prenatal developmental toxicity (PPV > 70%). Then, four of these nine clusters with confidence values equal to one were removed to reduce the effect of missing data on the resulting model reliability (Figure 4B). The read-across cross-validation results showed that three of the remaining five clusters had high sensitivity (Equation 4, ranging from 83.3% to 100.0%), low specificity (Equation 5, 0% for all three clusters), and low CCR (Equation 6, ranging from 41.7% to 50.0%) (Supplementary Table 3). These results indicated that the chemicals that had enough results in these clusters’ included in vitro assays for performing read-across were disproportionally in vivo developmental toxicants, limiting the model’s ability to optimize read-across parameters to predict nontoxic results properly. After excluding these three clusters, two potentially viable bioassay cluster models remained candidates to predict prenatal developmental toxicity.

Figure 4:

Cross-validation (A) positive predictive values (PPVs) for individual clusters containing at least five bioassays and (B) optimized confidence values for individual clusters with acceptable PPVs, as shown in (A). In both panels, the numbers along the x-axis represent cluster numbers as identified within the cluster map in Figure 3. The solid lines represent the thresholds used for cluster selection (PPV = 70% and confidence = 1).

The first model was Cluster 17, which had a higher predictive performance during cross-validation (PPV = 77.3%). This cluster contained 83 bioassays, of which 82 were collected from the PubChem database, and one was collected from the ToxCast database (Supplementary Table 2). The included bioassays collected from PubChem were deposited by Tox21 (48 bioassays, 55.4%), the National Center for Advancing Translational Sciences (NCATS) (29 bioassays, 39.8%), the Broad Institute (1 bioassay, 1.2%), New Mexico Molecular Libraries Screening Center (1 bioassay, 1.2%), Southern Research Institute (1 bioassay, 1.2%), the Scripps Research Institute Molecular Screening Center (1 bioassay, 1.2%), and the Vanderbilt High-Throughput Screening Facility (1 bioassay, 1.2%). Of these bioassays, 45 (54.2%) measured cell viability or cytotoxicity as counter screens for functional bioassays or drug repurposing screens (i.e., to treat various cancers^67,68 or inhibit the growth of infectious bacteria⁶⁹ or parasites⁷⁰). Four additional bioassays were drug screens to identify chemicals that blocked cell entry of viruses, including those causing hemorrhagic fevers⁷¹ (i.e., Ebola, Lassa virus, and Marburg virus) and respiratory syndromes⁷² (i.e., the Middle East and severe acute respiratory syndrome-related coronaviruses). In addition, one bioassay identified chemicals that could inhibit antifungal efflux pumps⁷³. The remaining 30 bioassays were functional and associated with protein targets, such as nuclear receptors, cytochrome P450 enzymes, G-protein coupled receptors, and transcription factors (Table 1).

Table 1.

Cluster 17 bioassay targets

Target	PubChem AIDs	ToxCast AEIDs	References suggesting relevance to developmental toxicity
Muscarinic acetylcholine receptor M1 (CHRM1)	628, 943, 944, 588852	-	96
Cytochrome P450 2D6 (CYP2D6)	891, 1851, 1645840		90,91,97
Caspase 3, apoptosis-related cysteine peptidase (CASP3)	1346980, 1347034	-	98,99
Cytochrome P450 3A4 (CYP3A4)	1851, 1645841	-	90,91
Thyroid-stimulating hormone receptor (TSHR)	1259385, 1259395	-	100
Atrial natriuretic peptide receptor 3 precursor (NPR2)	1347050	-	84
Chromobox protein homolog 1 (CBX1)	488953	-	101
Cytochrome P450 (CYP) 1A2, 2C9, 2C19	1851	-	90,91
Cytochrome P450 19A1 (CYP19A1, aromatase)	743083	-	92
D(1A) dopamine receptor (DRD1)	488983	-	97,102
D(2) dopamine receptor isoform long (DRD2)	485344	-	102,103
Estrogen receptor beta (ERβ)	1259378	-	97,104
FAD-linked sulfhydryl oxidase ALR (GFER)	485317	-	105
Firefly luciferase	1224835	-	-
Heat shock transcription factor 1 (HSF1)	743228	-	106
Jun proto-oncogene (JUN)	1159528	-	97,107–109
Nuclear receptor subfamily 1, group I, member 2 (NR1I2)	1346977	-	97,110
One cut homeobox 1 (ONECUT1)	-	83	87,88
Peroxisome proliferator-activated receptor delta (PPARD)	743215	-	111
Potassium voltage-gated channel subfamily H member 2 isoform d (KCNH2)	588834	-	112,113
SMAD family member 1 (SMAD1)	1347032	-	80
Tyrosyl-DNA phosphodiesterase 1 (TDP1)	686979	-	-

Cluster 41, which contained 76 bioassays, also emerged as a potential model for prenatal developmental toxicity predictions, showing a PPV of 71.7% during cross-validation. These 76 bioassays were collected from both the ToxCast (55 bioassays, 72.4%) and PubChem (21 bioassays, 27.6%) databases (Supplementary Table 2). Unlike Cluster 17, only a low percentage of these bioassays measured cell viability or cytotoxicity (9 bioassays, 11.8%). Additional bioassays from the ToxCast and Tox21 programs measured mitochondrial membrane potential disruptions^74,75 (three bioassays, 3.9%), microtubule stability⁷⁴ (two bioassays, 2.6%), and cytochrome P450 metabolism (eight bioassays, 10.5%). One further bioassay was an in vivo zebrafish assay developed as an alternative to mammalian tests to identify chemicals that induce embryonic death and structural anomalies⁷⁶. The remaining 52 bioassays in Cluster 41 were functional, associated with protein targets, such as transcription factors, metalloproteinases, transmembrane receptors, and chemokines (Table 2).

Table 2.

Cluster 41 bioassay targets

Target	PubChem AIDs	ToxCast AEIDs	References suggesting relevance to developmental toxicity
Androgen receptor (AR)	588516, 743042, 1259243	1856	114,115
Nuclear receptor subfamily 1, group I, member 2 (NR1I2)	720659, 1346982, 1347033	103	97,110
Aryl hydrocarbon receptor (AHR)	743085, 743122	63	114,116,117
Nuclear receptor subfamily 1, group I, member 3 (NR1I3)	1224839, 1224892	101	118,119
Estrogen receptor alpha (ERα)	743080, 1259244	-	97,104
Major histocompatibility complex, class II, DR alpha (HLA-DRA)	-	147, 185	97,110
Progesterone receptor (PGR)	1346795, 1347031	-	104
Acetylcholinesterase	1347395	-	120
cAMP responsive element binding protein 3 (CREB3)	-	69	97,117
CCAAT/enhancer binding protein (C/EBP), beta (CEBPB)	-	67	117
CD69 molecule	-	303	97
Chemokine (C-C motif) ligand 2 (CCL2)	-	173	97,110,117
Chemokine (C-X-C motif) ligand 10 (CXCL10)	-	241	117
Chemokine (C-X-C motif) ligand 8 (CXCL8)	-	307	110
Colony stimulating factor 1 (macrophage) (CSF1)	-	243	121
Cytochrome P450 19A1 (CYP19A1, aromatase)	743139	-	92
Estrogen-related nuclear receptor alpha (ERRα)	1259401	-	122
FBJ murine osteosarcoma viral oncogene homolog (FOS)	-	64	123
Forkhead box O3 (FOXO3)	-	1426	-
Matrix metallopeptidase 1 (interstitial collagenase) (MMP1)	-	248	-
Nuclear factor erythroid 2-related factor 2 isoform 1 (NFE2L2)	651741	97	97,117,124
Nuclear factor I/A (NFIA)	-	95	125
Nuclear factor of kappa light polypeptide gene enhancer in B-cells 1 (NFKB1)	-	94	-
Nuclear respiratory factor 1 (NRF1)	-	1460	124
Peroxisome proliferator-activated receptor delta (PPARD)	-	102	126
Peroxisome proliferator-activated receptor gamma (PPARG)	-	134	127
POU class 2 homeobox 1 (POU2F1)	-	98	97
Prostaglandin E receptor 2 (subtype EP2) (PTGER2)	-	289	110
RAR-related orphan receptor B (RORB)	-	104	97
Retinoic acid receptor, alpha (RARA)	-	136	110
Retinoic acid receptor, beta (RARB)	-	71	110
Selectin E (SELE)	-	305	128
SMAD family member 1 (SMAD1)	-	66	80
Sterol regulatory element binding transcription factor 1 (SREBF1)	-	107	-
Thrombomodulin (THBD)	-	162	129
Upstream transcription factor 1 (USF1)	-	72	-
Vitamin D (1,25-dihydroxyvitamin D3) receptor (VDR)	-	113	97
X-box binding protein 1 (XBP1)	-	114	130

External Validation by Predicting Test Chemicals

External validation with new chemicals is necessary to prove the utility of the selected models. To this end, Clusters 17 and 41 were used to predict the toxicity of chemicals outside of the training set⁷⁷. Of the full in vivo developmental toxicity database compiled for this study, only 1,090 chemicals had bioassay data for the model training procedure. Therefore, the remaining 154 chemicals were used as an external validation set, with bioassay results populated by QSAR model predictions, consistent with previous computational toxicology studies^32,33,41,53. Each QSAR model was first evaluated for its predictive performance before filling data gaps using their CCR (Equation 6). The QSAR models used to fill data gaps for these 154 chemicals had an average CCR of 68.4% (Supplementary Figure 3). Although this reliance on QSAR predictions adds uncertainty to the resulting predictions, this procedure mimics the model’s future use to fast screen new and untested chemicals early in the discovery and development process.

When using the cross-validation optimized confidence value for predictions (Equation 3), the Cluster 17 (confidence = 16) and 41 (confidence = 26) models showed high predictive performance for developmentally toxic chemicals in the external validation set (PPV = 100%) (Figure 5). Both clusters 17 and 41 were identified as potentially viable models for predicting developmental toxicity due partly to their high optimized confidence values compared to other clusters, which minimized the effects of missing data on the cluster selection process. Understandably, this low tolerance for missing data combined with the stringent applicability domain used for QSAR predictions proved to limit predictions of new and untested chemicals, resulting in low coverage of the overall dataset (4.5% and 3.9% for Clusters 17 and 41, respectively) and low predictive performance for nontoxic chemicals (Figure 5). This issue can, in some cases, be resolved by testing new chemicals in the relevant bioassays rather than relying on QSAR predictions. However, especially early in the discovery and development process, this solution is not always viable (e.g., when chemicals are not yet synthesized). In this circumstance, a potential solution is to increase the tolerance for missing data by lowering the confidence value required to report a prediction. High confidence values can limit the number of nontoxic predictions made since nontoxic chemicals often have a high proportion of inactive responses in the relevant bioassays.

Figure 5.

Clusters 17 and 41 models’ predictive performances, as shown by external validations. A and B show the performance of clusters 17 and 41, respectively, using activities from QSAR for chemicals whose three nearest neighbors in an assay showed at least 40% similarity. Panels C and D show the performance of clusters 17 and 41, respectively, using activities from QSAR for chemicals whose three nearest tested neighbors in an assay showed at least 30% similarity. The solid blue line in each graph represents the percent coverage of the external validation set containing 154 chemicals. The dashed orange lines represent the positive predictive values, or the fraction of predicted active chemicals predicted correctly, at varying confidence levels. The green dotted curves represent the specificity, or the fraction of correctly predicted in vivo inactive chemicals, at varying confidence levels.

For this reason, lowering the confidence value required to report read-across predictions increased both the coverage of the new chemicals and the balance of correctly predicted toxic and nontoxic chemicals. For example, using a confidence value of 6 instead of 16 for Cluster 17 model predictions showed high predictive performance for toxic and nontoxic chemicals (PPV = 100%, specificity = 100%) (Figure 5A). Similarly, using a confidence value of 16 instead of 26 for Cluster 41 model predictions improves the balance of predictive performance between toxicants and non-toxicants by retaining acceptable predictive performance for toxic chemicals (PPV = 66.7%) and increasing the cluster’s predictive performance for nontoxic chemicals (specificity = 50%) (Figure 5B). Further, using these adjusted confidence values increases the coverage more than two-fold for each cluster.

By lowering the similarity threshold required to report individual QSAR assay results to fill missing data before read-across, the coverage of predicting new chemicals further increased. Figures 5C and 5D show the results of varying confidence values with a lower similarity threshold (i.e., 30% instead of 40% similarity to three nearest tested neighbors required to report an assay result for a specific chemical). For example, using a confidence value of 11 for Cluster 17 predictions yields a high predictive performance for toxic chemicals (PPV = 93%), improved predictive performance for nontoxic chemicals (specificity = 50%), and improved coverage of 14.9%. Similarly, a confidence value of 21 for Cluster 41 predictions yields acceptable predictive performances for both toxic and nontoxic chemicals (PPV = 71.4%, specificity = 75.0%) and improved coverage of 19.5%.

Discussion

Implementing computational approaches can reduce the need for time-consuming, cost-inefficient, and often ethically undesirable animal tests and is particularly attractive for endpoints such as prenatal developmental toxicity where highly specialized animal study designs are necessary. Although previous computational approaches to evaluating new chemicals for prenatal developmental toxicity have shown promise, they have been limited in biological interpretability by only incorporating chemical structural information^7–13,78,79 or focusing on limited and specific well-understood biological mechanisms^38–41. Here, we presented a prenatal developmental toxicity database of over 1,200 chemicals spanning various use categories that underwent mammalian prenatal tests similar to the OECD 414/OPPTS 870.3700 protocols. Then, chemical structural information and biological data were integrated into a workflow that allowed for increased biological interpretability of the resulting predictions (Figure 1). As a result, clusters of bioassays were formed based on chemical-in vitro relationships, and two bioassay clusters, 17 and 41, were identified as models for predicting prenatal developmental toxicity.

These clusters contained several bioassays that align with plausible mechanisms of prenatal developmental toxicity (Tables 1 and 2). For example, the transforming growth factor beta (TGFβ)/SMAD pathway is well-known for its importance in embryonic development⁸⁰. Bioassays identifying agonists and antagonists of this pathway included in Clusters 41 and 17, respectively, were, therefore, previously identified as relevant to predicting prenatal developmental toxicity [ToxCast Assay Endpoint Identifier (AEID) 66 and PubChem Assay Identifier (AID) 1347032]⁸¹. Similarly, endocrine disruption by drugs or environmental chemicals is a well-established mechanism of developmental toxicity⁸². Bioassays measuring endocrine disruption by binding to hormone receptors (e.g., nuclear estrogen, androgen, progesterone, and thyroid-stimulating hormone receptors) were present in both clusters.

Besides these well-known developmental toxicity mechanisms, some less established but potentially relevant targets were captured by this study. For example, one Cluster 17 bioassay measured natriuretic polypeptide receptor B (NPR-B) antagonism (PubChem AID 1347050)⁸³ (Table 1). In previous studies, loss-of-function mutations in human NPR-B’s precursor gene were associated with impaired skeletal growth, suggesting that disrupting this receptor’s function may interfere with skeletal development⁸⁴. An additional Cluster 17 bioassay was associated with hepatocyte nuclear factor 6 (HNF6)’s precursor gene ONECUT1 [ToxCast Assay Endpoint Identifier (AEID) 83]^85,86. HNF6 was identified in previous studies as a regulator of liver⁸⁷ and pancreatic⁸⁸ development. Finally, in Cluster 41, several bioassays associated with various chemokines associated with immune response and inflammation were represented, along with matrix metallopeptidase 1 (MMP1) (Table 2). These targets were recently included in a proposed embryonic vascular disruption adverse outcome pathway (AOP) with endocrine disruption targets and TGFβ⁸⁹.

Interestingly, both clusters also contained assays measuring the activity of cytochrome P450 enzymes. Although the contributions of these enzymes to developmental toxicity are not fully understood, previous studies showed that cytochrome P450 enzyme expressions varied by developmental stages and modulated fetal exposures to toxicants such as carcinogens, drugs, and alcohol^90,91. Further, aromatase (cytochrome P450 19A1), a target of assays present in both clusters, plays a key role in hormone regulation by converting androgens to estrogens⁹².

This study highlighted the benefits and opportunities of using the publicly available biological data for computational toxicity predictions to resolve common issues in classic modeling studies (e.g., QSAR) for complex toxicity endpoints. Both well-established and putative prenatal developmental toxicity mechanisms were incorporated in the developed models, including endocrine disruption and embryonic blood vessel development. By relying on relevant biological data, this workflow provides users with an increased capacity to mechanistically interpret predictions, as required by international regulatory guidelines such as those by the OECD^18,93. Further, using biological data combined with chemical structural information reduced the frequency of activity cliffs in previous studies by incorporating information about chemicals’ interactions with biomolecules into the prediction process^14,34,51.

The predictions for external validation set chemicals, especially developmentally toxic ones, could be explained by looking at their nearest biological neighbors in the training set (Figure 6). For example, one developmentally toxic external validation set chemical was 2,2’,4,4’,5-pentachlorodiphenyl ether (CAS 60123–64-0). Chlorodiphenyl ethers are generated as byproducts during the manufacture of chlorinated phenols such as fungicides and wood preservatives⁹⁴. Based on the Cluster 17 bioassays, this chemical was correctly predicted as toxic based on its most biologically similar nearest neighbor (confidence = 16), as identified by comparing their bioassay responses using the biosimilarity metric (Equation 2). This neighbor was tributyltetradecylphosphonium chloride (CAS 81741–28-8), a developmentally toxic training chemical used as a biocide in hydraulic fracking (Figure 6A)⁹⁵. The high biosimilarity between these two chemicals was influenced mostly by their shared active responses in 11 viability/cytotoxicity assays and five functional assays. The functional assays represented four targets: the jun proto-oncogene, thyroid-stimulating hormone receptor, pregnane X receptor (NR1I2), and Small Mothers Against Decapentaplegic (SMAD) family member 1 (Table 1). Similarly, one example among Cluster 41’s toxic external validation predictions was 2,4-difluoro-1-(4-nitrophenoxy)benzene (CAS 28280–37-7). In the training set, based on the Cluster 41 assays, this chemical’s biological nearest neighbor was prochloraz (CAS 67747–09-5), a developmentally toxic fungicide (Figure 6B). These two chemicals shared active responses in 40 assays. Of these assays, five were viability/cytotoxicity assays. Two additional assays measured disruption of the mitochondrial membrane potential in vitro^74,75 and embryonic death and structural anomalies in vivo using zebrafish [ToxCast assay endpoint identifier (AEID) 1507]⁷⁶. The remainder were functional assays representing protein targets, including endocrine targets, cytochrome P450 enzymes, the aryl hydrocarbon receptor, and chemokines (Table 2).

Figure 6.

Sample toxic external validation predictions from the cluster models. Predictions are shown for 2,2’,4,4’,5-pentachlorodiphenyl ether (CAS 60123–64-0) based on its biological nearest neighbor within the training set chemicals tested in Cluster 17 bioassays, tributyltetradecylphosphonium chloride (CAS 81741–28-8), and for 2,4-difluoro-1-(4-nitrophenoxy)benzene (CAS 28280–37-7) based on its biological nearest neighbor within the training set chemicals tested in Cluster 41 bioassays, prochloraz (CAS 67747–09-5). A subset of the complete profile is shown in both panels, which includes the most influential bioassays on the resulting toxic prediction (i.e., those containing at least one active response between the validation chemical and its biological nearest neighbor). The columns in each profile represent assay identifiers from PubChem (prefixed with “P”) and ToxCast (prefixed with “T”), as listed in Tables 1 and 2. Active results are shown as red rectangles, inactive results are shown as blue rectangles, and untested chemical-bioassay pairs are shown as gray rectangles. Thick bold boxes surround shared active responses.

The computational workflow described here automatically identified bioassay data relevant to prenatal developmental toxicity from public databases and created a new strategy for predicting untested chemicals early in the discovery and development procedure or existing chemicals with limited safety data. Although the read-across models developed were insufficient to encompass all possible prenatal developmental toxicity mechanisms, this study highlighted the benefits and opportunities of using the publicly available biological data for computational toxicity predictions. Both well-established and putative prenatal developmental toxicity mechanisms were incorporated in the developed models, including endocrine disruption and embryonic blood vessel development. In addition, the adaptability built into the workflow paves the way for the easy incorporation of new bioassays as they are submitted to public database resources in future studies and applied to other complex toxicity endpoints.

Synopsis:

A computational approach can quickly and cost-efficiently identify potential prenatal developmental toxicants based on biological data to reduce, refine, and eventually replace animal testing.