Identification of Nonmonotonic Concentration-Responses in Tox21 High-Throughput Screening Estrogen Receptor Assays

Abstract

Environmental endocrine-disrupting chemicals (EDCs) interfere with the metabolism and actions of endogenous hormones. It has been well documented in numerous in vivo and in vitro studies that EDCs can exhibit nonmonotonic dose response (NMDR) behaviors. Not conforming to the conventional linear or linear-no-threshold response paradigm, these NMDR relationships pose practical challenges to the risk assessment of EDCs. In the meantime, the endocrine signaling pathways and biological mechanisms underpinning NMDR remain incompletely understood. The US Tox21 program has conducted in vitro cell-based high-throughput screening assays for estrogen receptors (ER), androgen receptors, and other nuclear receptors, and screened the 10K- compound library for potential endocrine activities. Containing 15 concentrations across several orders of magnitude of concentration range and run in both agonist and antagonist modes, these Tox21 assay datasets contain valuable quantitative information that can be explored to evaluate the nonlinear effects of EDCs and may infer potential mechanisms. In this study we analyzed the concentration-response curves (CRCs) in all 8 Tox21 ERα and ERβ assays by developing clustering and classification algorithms customized to the datasets to identify various shapes of CRCs. After excluding NMDR curves likely caused by cytotoxicity, luciferase inhibition, or autofluorescence, hundreds of compounds were identified to exhibit Bell or U-shaped CRCs. Bell-shaped CRCs are about 7 times more frequent than U-shaped ones in the Tox21 ER assays. Many compounds exhibit NMDR in at least one assay, and some EDCs well-known for their NMDRs in the literature were also identified, suggesting their nonmonotonic effects may originate at cellular levels involving transcriptional ER signaling. The developed computational methods for NMDR identification in ER assays can be adapted and applied to other high-throughput bioassays.

Introduction

Endocrine-disrupting chemicals (EDCs) are a diverse group of synthetic or naturally occurring compounds that can interfere with the turnover, distributions and actions of endogenous hormones (Combarnous and Nguyen 2019, La Merrill et al. 2020). Numerous epidemiological and experimental studies have demonstrated that by disrupting the synthesis, metabolism, transportation, and signaling of the hormones, exposures to EDCs are associated with and cause a variety of adverse health outcomes in development, reproduction, immunity, metabolism, and behavior, etc. (Skakkebaek et al. 2001, Delbès et al. 2006, Hatch et al. 2006, Alonso-Magdalena et al. 2011, Boas et al. 2012, Sifakis et al. 2017, Ghassabian and Trasande 2018). The economic burden associated with EDC exposure has been rising and costs hundreds of billions of US dollars worldwide annually (Attina et al. 2016, Trasande et al. 2016).

One particular concern with the health risks of EDCs is the uncertainty in low-dose exposure. It has been amply documented by numerous in vivo and in vitro studies that EDCs can exhibit nonmonotonic dose response (NMDR) relationships, where their biological effects change directions with the exposure levels (Vandenberg et al. 2012, Lagarde et al. 2015). Epidemiological evidence has also begun to emerge suggesting the existence of nonmonotonic relationship between human exposure and health outcomes (Tuomisto et al. 2006, Lee et al. 2011). Not conforming to the conventional linear or linear no threshold paradigm, these U or inverted U (Bell)-shaped dose-response relationships pose practical challenges to the risk assessment of EDCs (Vandenberg et al. 2013, Zoeller and Vandenberg 2015, Hill et al. 2018). In such nonmonotonic scenarios, reducing exposure from current levels will no longer guarantee a reduction in the disease risk. NMDR may explain some of the divergent or even contradictory effects of EDCs observed under different biological and experimental conditions for a variety of biological and health endpoints (Vandenberg et al. 2012, Vandenberg 2014). Therefore, the existence of low-dose NMDR has significant implications in the regulation of EDCs.

Many NMDRs have been observed in in vitro assays (Vandenberg et al. 2012), indicating some nonmonotonic mechanisms may operate at the cellular level. While NMDRs are often haphazard discoveries, cell-based high-throughput screening (HTS) assays provide great opportunities to examine NMDRs in vitro for a large number of chemicals. The inter-agency Tox21 program has conducted a number of quantitative HTS (qHTS) endocrine activity assays for the 10K-compound library, for estrogen receptors (ER), androgen receptors (AR), and other nuclear receptors such as aryl hydrocarbon receptor (AHR) and constitutive androstane receptor (CAR) (Huang et al. 2011, Attene-Ramos et al. 2013, Tice et al. 2013, Filer et al. 2014, Huang et al. 2014). A prominent feature of these highly automated qHTS in vitro assays is the high-quality concentration-response curves (CRCs) produced in agonist and antagonist modes, which are in triplicate or higher number of replicates and span nearly 5 orders of magnitude in concentrations with 15 concentration points. These CRCs contain valuable quantitative information, such as the shapes and degrees of nonlinearity in the cellular response, which can be extracted to help evaluate the hazard and health risk of the tested chemicals. However, most of the studies utilizing Tox21 data are limited to using single numerical values such as AC₅₀, IC₅₀, maximal activity, and no observed adverse effect level (NOAEL), or categorical classification such as active or inactive, on which machine-learning algorithms were developed to predict biological activities based on chemical structures (Zhu et al. 2014, Capuzzi et al. 2016, Huang et al. 2016, Mayr et al. 2016, Sipes et al. 2017). Some of the Tox21 CRCs are nonmonotonic which cannot be explained away with general cytotoxicity or assay interference (Huang et al. 2014, Witt et al. 2017, Klimenko 2021). We believe that certain NMDR mechanisms, including cooperative DNA binding, receptor dimerization, coactivator recruitment and squelching (Kohn and Portier 1993, Kohn and Melnick 2002, Conolly and Lutz 2004, Li et al. 2007, Cookman and Belcher 2014, Xu et al. 2017), may operate in the cell constructs used in Tox21 assays, rendering the qHTS data an excellent resource to study NMDR behaviors at cellular levels.

Computational methods have been used to group objects into distinct classes based on similar features (Tamayo et al. 1999, Peddada et al. 2003). Some have been applied to cluster gene expression dynamics (Chu et al. 1998, Heyer et al. 1999, Tamayo et al. 1999, Peddada et al. 2003). Applying these methods directly to identify NMDR in qHTS data can be somewhat challenging. Many of the existing methods requires prior knowledge including candidate profiles (Peddada et al. 2003) and the number of clusters (Tamayo et al. 1999), while the type of CRCs in a Tox21 assay can be multiple and a priori not completely known. Moreover, many of these existing methods tend to cluster curves based on significant local variations while discounting the global trend. Similar NMDR curves, such as U shapes, may be grouped to different clusters just because of differences in concentration at the inflection point. Therefore, new methods customized to the CRCs of the 10K compounds in the Tox21 assays are needed to more reliably and accurately identify NMDRs. In the present study we developed a machine learning algorithm to identify NMDR curves in the Tox21 ER assays. We chose ER assays because (1) as detailed below (Fig. 1), they are the most comprehensive ones compared with other Tox21 nuclear receptor assays, and (2) overwhelmingly more NMDRs have been reported in the literature for estrogenic compounds, including bisphenol A (BPA) and many others (Welshons et al. 2003, Vandenberg et al. 2012, Vandenberg 2014, Lagarde et al. 2015, Xu et al. 2017, Prins et al. 2018). ER-mediated endocrine signaling plays pivotal roles in female reproduction and other systems and its perturbation can result in adverse outcomes in steroidogenesis, folliculogenesis, and a variety of ER-mediated carcinogenesis such as breast and uterine cancers. Assessing the NMDR relationships in Tox21 ER assays will provide important biological insights into the toxicity, mechanisms, and health risk assessment of estrogenic chemicals.

Figure 1. Overview of Tox21 ER assays.

(A) MCF7 VM7Luc4E2 cell reporter system expressing full-length ERα. (B) HEK 293 ER-UAS-bla cell reporter system expressing chimeric protein of ER ERα-LBD or ERβ-LBD fused to GAL4-DBD. (C) Eight ER reporter assays distributed among receptor types, cell systems, agonist/antagonist mode, and availability of accompanying viability assays.

Methods

Overview of Tox21 ER assays

The Tox21 qHTS assays for both the ERα and ERβ nuclear receptor signaling pathways tested 9667 or 10496 compounds (8,300 unique chemicals) at 15 logarithmically evenly-spaced concentrations ranging approximately between 1 nM - 100 μM. The screening was carried out in either agonist or antagonist mode in two stably transfected reporter cell lines: (1) a variant of the human breast adenocarcinoma cell line, MCF7 VM7Luc4E2 and (2) human embryonic kidney HEK 293 ER-UAS-bla. The MCF7 VM7Luc4E2 was formerly mistaken as human ovarian carcinoma cell line and labelled as BG1Luc4E2 (https://ntp.niehs.nih.gov/iccvam/methods/endocrine/bg1luc/bg1luc-vm7luc-june2016-508.pdf) (Li et al. 2014). The MCF7 VM7Luc4E2 cells express endogenous full-length ERα protein predominantly, with little, if any, ERβ expression (Rogers and Denison 2000, Li et al. 2014, Brennan et al. 2016), and the cells host a stably transfected estrogen responsive reporter (pGudLuc7ere) containing 4 estrogen responsive elements (EREs) and firefly luciferase gene (Fig. 1A) (Rogers and Denison 2000). Therefore, the screening assays with these cells were mainly intended for ERα active compounds. The HEK 293 ER-UAS-bla cells were stably transfected with plasmids expressing (1) a fusion protein comprising the truncated ligand-binding domain (LBD) of human ERα or ERβ and the DNA-binding domain (DBD) of GAL4, and (2) β-lactamase reporter gene under the control of 7 GAL4 DNA-binding sites, i.e., the upstream activating sequence (UAS, Fig. 1B) (Qureshi 2007). For each ER cell reporter system, the assays were run in both agonist and antagonist modes, multiplexed with cytotoxicity (viability) readout in all antagonist and some agonist assays. Specifically, for MCF7 VM7Luc4E2 cells, a fluorescence-based viability assay was used to measure conserved constitutive protease activities in live cells; for HEK 293 ER-UAS-bla cells, a luminescence-based viability assay was used to measure intracellular ATP levels. Totally there are 8 ER assays as summarized in Fig. 1C. The Tox21 program also conducted interference assays to identify chemicals that may inhibit luciferase to affect the MCF7 cells-based readouts and those chemicals that may be autofluorescent to affect HEK 293 cells-based readouts (Borrel et al. 2020). A detailed statistical analysis of the Tox21 ERα assays on the quality control and numbers of active agonist and antagonist chemicals was reported previously (Huang et al. 2014).

Data procurement, preparation and initial screening

The Tox21 ER datasets, and the luciferase inhibition and autofluorescence datasets were downloaded from https://tripod.nih.gov/tox. The Protocol Names corresponding to ER Assays 1–8, as numerically indicated in Fig. 1C, are (1) tox21-er-luc-bg1-4e2-agonist-p4, (2) tox21-er-lucbg1-4e2-agonist-p2, (3) tox21-er-luc-bg1-4e2-antagonist-p2, (4) tox21-er-luc-bg1-4e2-antagonist-p1, (5) tox21-er-bla-agonist-p2, (6) tox21-er-bla-antagonist-p1, (7) tox21-erb-bla-p1, and (8) tox21-erb-bla-antagonist-p1. The Protocol Names corresponding to the luciferase inhibition and autofluorescence assays are tox21-luc-biochem-p1 and tox21-spec-hek293-p1 respectively. To facilitate the identification of nonmonotonic CRCs, the ER datasets were first processed by following the three steps below: (1) average CRC, (2) remove CRCs that only exhibit small fluctuations likely due to intra-assay variability (random noise), and (3) filter out and set aside CRCs that contain outlier-activity points which would interfere with assessing the global trend of the affected curves (Fig. 2A). In step (1), the activity values at each concentration of a compound (with unique substance identifier, SID) in an assay were averaged across the replicates to obtain the averaged CRC for the compound. For each assay, all averaged data points (about 10,000 × 15 = 150,000 data points) were plotted against the random indices of the SID x concentrations matrix for visualization, along with the distribution of these data points (Fig. S1). In step (2), after examining the distributions across all 8 ER assays, a universal cut-off range, [−20, 20] inclusive (Table S1 and indicated with the red lines in Fig. S1), was applied to filter out inactive compounds in each assay, i.e., those whose maximal and minimal activity levels were bound within this range. In step (3), in the remaining chemicals, replicates-averaged CRCs which have a single upward or downward spike point that is >=30 higher or lower in activity (Table S1) than the two immediate neighboring points were also filtered out and set aside for further analysis. Examples of such CRCs with large local spikes were given in Fig. S4. As indicated by the large error bar, these outlier spikes were often due to large differences between replicates. Moreover, the rationale for the tentative exclusion is also based on the reasonable assumption, according to (Varret et al. 2018), that a true biological Bell or U CRC is unlikely to be determined solely by a single spike that spans a narrow (5-fold) range of concentrations (the spacing of two neighboring concentrations is 2.24-fold in the Tox21 datasets).

Figure 2. Flow chart of the identification process for nonmonotonic CRCs in Tox21 ER assays.

* The magnitude cut-off of 30 in C4, C5, and C6 was applied to the replicates-averaged CRCs, not the 2-MA curves.

Identification of major curve shapes with recursive, correlation-based clustering

The purpose of this unsupervised learning process here is to identify the major shapes of CRCs that likely exist in an ER assay. Correlation coefficient is widely used to measure the extent two sets of data are statistically related (Lee Rodgers and Nicewander 1988, Boslaugh 2012). If the activity levels of a compound within a defined range of concentrations are viewed as a set of data, then two compounds are considered to be similar when the correlation coefficient is high. Once a correlation coefficient is computed for a pair of compounds, an algorithm is needed to group compounds with similarly shaped CRCs. Existing clustering methods have their limitations in our application here. Traditional distance metrics such as Euclidean, Manhattan, and Cosine-based distance tend to separate similarly-shaped CRCs that have either markedly different response magnitudes or baseline offset into distinct clusters (Wang et al. 2002). So here we propose to adapt a Pearson correlation-based method (Lee Rodgers and Nicewander 1988, Boslaugh 2012), which has been used to group temporal or dose-dependent gene expression patterns (Chu et al. 1998, Heyer et al. 1999). Although such method can be sensitive to local variations, the global trend can still be captured by using the moving average of the CRCs (Fig. 2B).

Step (1): Curve smoothing.

Convert each remaining averaged CRC from the initial screening section above into 5-concentration moving average (5-MA). The purpose is to reduce the effects of local variations on assessing the global shape of a CRC. In general, for k-MA, the average MA ER activity level at a concentration is calculated over a sliding window of length k across the neighbors of the concentration. When k is an odd number, the sliding window includes the current concentration itself and (k-1)/2 concentrations on both sides. When k is an even number, the sliding window includes the current concentration itself, k/2 concentrations on the lower-concentration side, and k/2-1 concentrations on the higher-concertation side. At the beginning and end of the curve where there are not enough concentrations to fill the k-length window, the MA is taken only over the concentrations that are available in the window. The MA algorithm was implemented by using the movmean function in MatLab (The Mathworks, Natick, Massachusetts, USA). The 5-MA was selected at this step because it is considered a good tradeoff that can effectively smooth out the local bumps of most of the ER CRCs without significantly altering the global trend.

Step (2): Curve grouping.

The set of 5-MA CRCs of all compounds in an assay after initial filtering is defined as set A. (a) Randomly select a compound i from A. (b) Find a compound j in A which has the highest Pearson correlation with i on the 5-MA curves. Then compounds i and j form a cluster set S=[i, j] and are removed from A. (c) For each remaining curves in A, calculate its pair-wise correlation coefficient with each curve in S and only keep the lowest correlation coefficient. Rank these lowest coefficients and add to S the curve k in A which has the highest coefficient value. Remove curve k from A. (d) Repeat step (c) until the maximal lowest-correlation coefficients is < 0.75. (e) Recursively repeat steps (a)-(d) until all compounds in A are exhausted and assigned into a cluster.

The correlation-based clustering algorithm above can identify possible curve shapes without prior knowledge. But the algorithm is not perfect for our purpose in the following two scenarios. (1) Similarly-shaped NMDR curves with vastly different peak/nadir (inflection point) concentrations are unlikely grouped together (Wang et al. 2002, Peddada et al. 2003). (2) Globally dissimilar curves may still be grouped together due to similar, strong local trend. In addition, multi-phasic NMDRs with more than one inflection point are also possible. Therefore, the clustering results from the above steps cannot be used as is. Instead the resulting clusters were visually examined to identify major curve shapes that exist in the datasets and then a classification algorithm was applied to assign the CRCs into these major shapes.

Pattern-restricted classification of CRCs

After the identification of major curve shapes using the unsupervised approach above, a supervised, pattern-restricted algorithm was used to classify each CRC into the different shapes identified (Fig. 2C). Briefly, to determine whether a CRC is Bell or U-shaped, we first calculated the magnitude of an NMDR curve by using the 5-MA. The magnitude of a Bell curve is defined as the difference between the activity level at the peak and the greater of the activity levels at the lowest and highest concentrations (Fig. S5A). Conversely, the magnitude of a U curve is defined as the difference between the activity level at the nadir and the lesser of the activity levels at the lowest and highest concentrations (Fig. S5B). If the magnitude exceeds a certain threshold (>15, Table S1), the curve is labeled as either Bell or U depending on the sign of the magnitude. A Bell or U curve whose peak or nadir falls on the second lowest or second highest concentrations are excluded, following the principle that a trend change in NMDR requires more than one data point (Varret et al. 2018). If both Bell and U shapes are identified for a curve, it would be classified as multi-phasic NMDR (S shape). For the remaining curves that were not identified as NMDR by using the 5-MA, 2-MA was used to increase the sensitivity to extract more potential NMDRs (Fig. 2C). Using 2-MA, the CRCs were classified into five categories, including Bell (Ո), U, monotonic increase (↑), monotonic decrease (↓), and flat (−). The Bell or U shapes were identified by using the same magnitude and exclusion criteria above. If a curve is neither Bell nor U shape, the difference (magnitude) between the maximal and minimal activity levels of the averaged CRC is calculated. If the magnitude exceeds a threshold value (>30 for all assays, which is 1.5-fold of the unidirectional threshold used for inactivity filtering in the initial screening step, Table S1), then the curve is classified as either monotonic increase or monotonic decrease, depending on the sign of the magnitude. If the difference is within the predefined threshold, then the curve is classified as flat.

Exclusion of false-positive NMDRs based on cytotoxicity and interference assays

Some NMDR curves in the Tox21 ER datasets may result from cytotoxicity, chemical inhibition of the luciferase activity in the MCF7-based assays, and chemical autofluorescence in the HEK 293-based assays. Specifically, the descending phase of a Bell-shaped curve may result from cytotoxicity or luciferase inhibition, and the ascending phase of a U-shaped curve or that of a Bell-shaped curve may arise from autofluorescence of the test compound. To rule out these potential false-positive NMDR curves, the following method was used (Fig. 2D). We first identified compounds that are active in the cytotoxicity (Fig. S2), luciferase inhibition, or autofluorescence assays (Fig. S3) by using data-dependent threshold values (Table S1) to filter out those inactive compounds. The remaining, active compounds were stored in sets B, C, and D for cytotoxicity, luciferase inhibition, and autofluorescence respectively. To determine if a Bell curve is due to a decrease in cell viability or luciferase inhibition, we calculated the Pearson correlation coefficient between the descending phase (named segment a) of the replicates-averaged CRC and the corresponding segment (same-concentrations range) of the cytotoxicity or luciferase inhibition curves (segment b or c respectively) if available. If the Pearson coefficient is > 0.4 and the drop of the viability or luciferase inhibition levels of segment b or c is > 1.5-fold of the unidirectional inactivity threshold values used above (Table S1), then the CRC is eliminated as a false-positive Bell curve. Similarly, to determine if a U curve is due to a decrease in cell viability or luciferase inhibition, we calculated the Pearson correlation coefficient between the descending phase (segment a) of the replicates-averaged CRC and the corresponding segment (same-concentrations range) of the cytotoxicity or luciferase inhibition curves (segment b or c respectively) if available. If the Pearson coefficient is > 0.4 and the drop of the viability or luciferase inhibition levels in segment b or c > 1.5-fold of the inactivity threshold values (Table S1), then the CRC is eliminated as a false-positive U curve. Assays 2 and 5 do not contain cytotoxicity screening data; the cytotoxicity data of assays 4 and 6 were used instead for assays 2 and 5 respectively because they used the same cell reporter systems and identical compound library. Since the autofluorescence assay only tested for the 5 highest concentrations and the number of active compounds identified is much smaller than the cytotoxicity and luciferase inhibition assays, we visually examined the ascending phase of a Bell curve or the ascending phase of a U curve against the autofluorescence data (set D) for correlation, to exclude false-positive NMDRs likely caused by autofluorescence.

Programming language and code sharing

All coding was conducted in MatLab and is available at https://github.com/pulsatility/2022-Tox21-ER-NMDR. The provided code can be used to query with SID to display CRCs in all 8 ER assays along with associated cytotoxicity, luciferase inhibition, and autofluorescence curves if available.

Results

Initial screening

The total number of compounds (with unique SID) in each of the 8 Tox21 ER assays is either 9667 or 10496 depending on the assay. After the initial screening to filter out inactive compounds using the boundary condition specified in Methods (Table S1), between 6951–9629 compounds were removed (and labeled as flat CRC) and between 331–2897 compounds remain (Table 1). By applying the outlier-spike screening criterion, additional 1–41 compounds were filtered out for further visual inspection, and 324–2873 compounds remain.

Identification of major curve shapes with recursive, correlation-based clustering

Using the correlation-based clustering algorithm, the replicates-averaged CRCs in assays 1 through 8 were grouped into 14, 31, 21, 24, 18, 44, 18, and 35 clusters respectively. While a cluster contains similarly-shaped curves, Bell curves with different peak concentrations can end up in different clusters (an example was given in Fig. S6A–S6B), as expected with the correlation-based algorithm. Likewise, U curves with different nadir concentrations may also end up in different clusters (Fig. S6C vs. S6D). Nonetheless, after visual inspection, all clusters belong to one of 6 broad shape categories – flat, monotonic decrease, monotonic increase, Bell, U, and S curves.

Pattern-restricted classification of CRCs

After applying the classification algorithm as illustrated in Fig. 2C, a number of NMDR curves were identified in each ER assay (Table 2). The numbers of Bell curves range from 46 to 186 across the 8 assays. In comparison, the numbers of U curves are much smaller, ranging from 0 to 78. S-shaped curves are rare, only 4 in total. In contrast to NMDR curves, the majority of the CRCs are either monotonic increase, monotonic decrease, or flat.

Table 2.

Results of Pattern-Restricted Classification

Assay #	1	2	3	4	5	6	7	8
Starting compounds	372	2873	2200	2242	849	2350	324	2677
Ո	52	186	176	166	54	178	46	96
Ս	3	25	11	25	0	44	1	78
S	0	1	1	0	0	2	0	0
↑	179	1087	361	560	548	300	159	185
↓	16	953	1276	1139	15	1219	9	1831
–	122	621	375	352	232	607	109	487

Exclusion of false-positive nonmonotonic CRCs and addition of nonmonotonic CRCs with outlier spikes

After applying additional screening to the NMDR curves identified above to exclude false positives due to cytotoxicity, luciferase inhibition, and autofluorescence, a number of compounds were further filtered out. Between 22%−69% of compounds were filtered out for exhibiting false-positive Bell curves, leaving the numbers of compounds with true Bell curves to range between 28–128 across the 8 assays (Table 3). In comparison, between 0%−78% of compounds were filtered out for exhibiting false positive U curves, leaving the numbers of compounds with true U curves to range between 0–23 (Table 4), which are much smaller than the numbers of true Bell curves. The number of S curves remaining is 1. Upon visual inspection of the 1–41 CRCs with outlier spikes (Table 1), additional 1–8 Bell curves were identified, and the final Bell curves ranges between 28–129 among the 8 assays (Table 3). No additional U curves were identified from the CRCs with outlier spikes.

Table 3.

Exclusion of False-Positive Bell (Ո) Curves and Addition of Bell Curves with Outlier Spikes

	Assay #	1	2	3	4	5	6	7	8
	Starting Ո curves	52	186	176	166	54	178	46	96
Filtering	Cytotoxicity	−14	−69	−82	−87	−12	−45	−15	−44
	Interference	−16	−34	−80	−40	NA	NA	NA	NA
	Fluorescence	NA	NA	NA	NA	−0	−5	−2	−4
	Outlier rescue	0	+1	+1	+8	+1	+1	0	+2
	Final Ո curves	28	98	56	76	43	129	29	50

Note: The “−“ sign denotes numbers of compounds that were removed. NA: not applicable

Table 4.

Exclusion of False-Positive Ս Curves and Addition of U Curves with Outlier Spikes

	Assay #	1	2	3	4	5	6	7	8
	Starting Ս curves	3	25	11	25	0	44	1	78
Filtering	Cytotoxicity	0	−2	−5	−12	0	−25	0	−61
	Interference	0	0	−4	−3	NA	NA	NA	NA
	Fluorescence	NA	NA	NA	NA	0	−3	0	−1
	Outlier rescue	0	0	0	0	0	0	0	0
	Final Ս curves	3	23	3	10	0	17	1	17

Note: The “−“ sign denotes numbers of compounds that need to be removed by the filtering process. NA: not applicable

Table 1.

Tox21 ER Assay Summary and Results of Initial Screening

Assay #		1	2	3	4	5	6	7	8
Assay mode		Agonist	Agonist	Antagonist	Antagonist	Agonist	Antagonist	Agonist	Antagonist
ER type		ERα	ERα	ERα	ERα	ERα	ERα	ERα	ERα
Assay system		MCF7	MCF7	MCF7	MCF7	HEK 293	HEK 293	HEK 293	HEK 293
Total compounds		9667	10496	9667	10496	10496	10496	9667	9667
Inactivity filtering	Yes	−9294	−7599	−7457	−8232	−9629	−8105	−9336	−6951
	No	373	2897	2210	2264	867	2391	331	2716
Outlier filtering	Yes	−1	−24	−10	−22	−18	−41	−7	−39
	No	372	2873	2200	2242	849	2350	324	2677

Note: The “−“ sign denotes numbers of compounds that were removed in the corresponding filtering process.

Numbers in the last row denote the numbers of remaining compounds that were moved forward for clustering and classification.

Statistical summary of nonmonotonic CRCs and representative examples

In total, there are 436 compounds that display at least one type of NMDR in the Tox21 ER assay (compound names and SIDs are provided in Supplemental Spreadsheet). The NMDR magnitude and the inflection point concentration, i.e., the concentration at the peak activity of a Bell curve (Fig. 3) or at concentration at the nadir activity of a U curve (Fig. 4), were scatter-plotted for each assay. A higher magnitude and lower peak- or nadir-activity concentration would indicate a stronger nonmonotonic effect. With the assay concentrations ranging approximately between 1 nM – 100 μM, the majority of the peak-activity concentrations of Bell curves are distributed between 10 nM – 10 μM, and the magnitudes distributed between 20 – 100 (Fig. 3). For U curves, the majority of the nadir-activity concentrations are distributed in a similar range, but their magnitudes are generally much smaller (Fig. 4). Select Bell-shaped CRCs are presented in Fig. 5, and select U-shaped CRCs and a S-shaped CRCs are presented in Fig. 6A and 6B, respectively, along with associated cytotoxicity, luciferase inhibition, and autofluorescence data where available.

Figure 3. Scatter plots of log10 peak-activity concentration vs. magnitude of identified Bell-shaped CRCs in Tox21 ER assays.

Assay # is indicated on each panel.

Figure 4. Scatter plots of log10 nadir-activity concentration vs. magnitude of identified U-shaped CRCs in Tox21 ER assays.

Assay # is indicated on each panel. Note: no U curves were identified in assay 5.

Figure 5. Select Bell-shaped replicates-averaged CRCs in Tox21 ER assays.

Assay #, compound SID and name, line color scheme for reporter activity, cytotoxicity, luciferase inhibition, and autofluorescence are indicated. Vertical bars: standard error of mean.

Figure 6. Select U-shaped (A) and S-shaped (B) replicates-averaged CRCs in Tox21 ER assays.

Assay #, compound SID and name, line color scheme for reporter activity, cytotoxicity, luciferase inhibition, and autofluorescence are indicated. Vertical bars: standard error of mean.

The 8 ER assays differ in the cell type/reporter construct, ER type, and agonist/antagonist mode. We next examined whether the frequency of NMDRs is favored in certain assay conditions. We found that Bell curves have a probability of slightly over 0.6 to appear in antagonist assays, vs. near 0.40 in agonist assays (Fig. 7A). U curves have a similar probability split between agonist and antagonist assays, so are when both Bell and U curves are tallied together. Bell curves have a probability of 0.63 to appear in ERα assays vs. 0.37 in ERβ assays (Fig. 7B). The dominance of Bell curves in ERα assays could be due to the bias introduced by the 4 MCF7 ERα assays while there is no MCF7 ERβ assays. However, by comparing the ERα (Assays 5 and 6) and ERβ (Assays 7 and 8) assays which all used HEK 293 cells, the frequency of Bell curves is still twice higher in the ERα assays than ERβ assays (Table 3). U curves have nearly equal chances to appear in both ERα and ERβ assays, while Bell and U curves combined are more frequently encountered in ERα than ERβ assays, due to the fact that there are nearly 7 times more Bell curves identified than U curves. As far as the cell constructs of the reporter assays are concerned, there is a negligible difference in the probability of Bell curves appearing in MCF7 cells, which express the full-length ERα protein driving a 4xERE promoter, vs. in HEK 293 cells, which express the ER_LBD-GAL4_DBD fusion protein driving a 7xUAS promoter (Fig. 7C). Similarly, U curves are also equally likely to be observed in the two reporter systems.

Figure 7. Distribution probability of nonmonotonic CRCs in Tox21 ER assays.

(A-C) Probability of Bell (Ո), U, or both type of curves appearing in (A) agonist-mode vs. antagonist-mode assays, (B) in ERα vs. ERβ assays, and (C) in MCF7 construct vs. HEK 293 construct assays. The probabilities were calculated by normalizing to the total number of compounds in each assay type. (D-E) Percentage distribution of compounds exhibiting Bell (D) or U (E) curves in 1, 2, 3, 4, or 5 ER assays.

Some compounds exhibit Bell or U-shaped CRCs in multiple ER assays. Among the 367 compounds that were identified to exhibit Bell curves, about 71% exhibit Bell curves in 1 assay, 21% in 2, 6% in 3, 1.1% in 4, and 0.5% in 5 assays (Fig. 7D). In contrast, among the 69 compounds that were identified to exhibit U curves, 93% exhibit U curves in 1 assay, 7% in 1 assays, and no compounds exhibiting U curves in 3 or more assays (Fig. 7E). For example, dl-Norgestrel exhibits Bell curves in 5 assays (Fig. S7). Two digoxin compounds, each with a unique SID, exhibit Bell curves in 5 (Fig. S8) and 4 (Fig. S9) assays respectively. Additional 3 compounds also exhibit Bell curves in 4 assays, including diethylstilbestrol dipropionate (Fig. S10), 5α-dihydrotestosterone (Fig. S11), and quabain (Fig. S12). 5 compounds exhibit U curves in 2 assays, including carfilzomib, 4,4”-octahydro-1H-4,7-methanoindene-5,5-diyldiphenol, (Z)-4-hydroxytamoxifen, trihexyltetradecylphosphonium chloride, and perfluoroundecanoic acid (Figs. S13–S17). There is one single compound, bisoxatin acetate, that exhibits a U curve (assay 2) and 2 Bell curves (assay 3 and 4) (Fig. S18).

A variety of chemicals including endogenous estrogens have been reported to exhibit NMDR behaviors in in vitro and/or in vivo assays. We next examined the CRCs in the Tox21 ER assay for some EDCs well-known for their NMDR behaviors (Figs. S19–S31), including 17β-estradiol, bisphenols (BPA, BPB, and BPAF), synthetic steroids (diethylstilbestrol, ethinylestradiol, tamoxifen, and trenbolone), phytoestrogens (genistein and resveratrol), industrial chemicals (4-nonylphenol), and antimicrobial agents (triclosan and triclocarban). Nearly all of them do exhibit nonmonotonic CRCs in some assays, but not every curve passed the criteria set here. For example, although 17β-estradiol exhibits Bell-shaped CRCs in assays 1, 7, and 8 (Fig. S19), but because of the high correlations with cytotoxicity and/or luciferase inhibition activity at high concentrations, it did not make it to the final list of compounds exhibiting Bell curves in our analysis. BPA also exhibits a Bell curve in assay 2, but the descending phase of the curve is likely due to luciferase inhibition, so it is disqualified (Fig. S20). BPB (Fig. S21) and BPAF (Fig. S22) both exhibit a Bell curve in assay 2 without cytotoxicity and luciferase interference concerns. Diethylstilbestrol, ethinylestradiol, tamoxifen citrate, 17β-trenbolone, genistein, resveratrol, and nonylphenol all exhibit one or multiple qualified Bell or U curves (Figs. S23–29). Triclosan shows antagonistic activities but likely due to cytotoxicity (Fig. S30), while triclocarban exhibits a Bell curve in one assay (Fig. S31).

Discussion

By developing and utilizing a custom clustering and classification algorithm, we identified a number of chemicals in the Tox21 ER activity assays that exhibit NMDR relationships. Nearly 50% of the initially identified Bells curves likely resulted from cytotoxicity, luciferase inhibition, and/or autofluorescence. After the exclusion of these potentially false-positive curves, 367 compounds remained, representing 509 Bell-shaped CRCs. Likewise, 60% of the initially identified U curves are excluded, and 69 compounds remained, representing 74 U-shaped CRCs.

The finding that there are overwhelmingly more Bell than U curves is not unexpected. While Bell curves may emerge in both agonist and antagonist mode assays, U curves are much less likely to emerge in agonist assays. In an antagonist assay where E2 is added such that the baseline activity of the reporter is elevated, a chemical may inhibit or further activate the reporter activity at low concentrations, then somehow reverse its action at higher concentrations, thus producing either a U or Bell-shaped CRC, respectively. In contrast, in an agonist assay where no E2 is added such that the baseline reporter activity is near zero, while a Bell curve is still possible, a U curve is very unlikely to arise because it would require initial dipping below the near-zero baseline at low concentrations. For a U curve to emerge in an agonist assay, a chemical has to be very potent, essentially inducing some activity at the lowest tested concentrations first, a scenario that is very rare if not entirely impossible. This anticipated lower probability of U curves in agonist assays is consistent with our finding – among the 4 agonist assays (# 1, 2, 5, and 7), there are 27 U curves, in contrast to the 198 Bell curves in the same assays and 47 U curves in the 4 antagonist assays (# 3, 4, 6, and 8). Further inspection revealed that assay 2 alone contributed 23 of the 27 U curves identified in the 4 agonist assays. Interestingly enough, 953 monotonic-decrease curves were also identified after pattern-restricted classification, which are in contrast to the much smaller numbers of monotonic-decrease curves in the other three agonist assays – 16 in assay 1, 15 in assay 5, and 9 in assay 7 (Table 2). These results suggest that despite its agonist-mode nature, assay 2 may somehow has elevated baseline activities, which can be inhibited by some test chemicals at low concentrations.

Potential molecular mechanisms

To appreciate NMDR behaviors beyond observations and possibly apply the concept in risk assessment, it is necessary to understand the biological mechanisms behind these nontrivial effects. Within the framework of adverse outcome pathways (AOPs) of EDCs (Browne et al. 2017), a number of mechanisms have been postulated for NMDR behaviors. At the cellular level, where the Tox21 or other similar in vitro assays were performed, plausible mechanisms include opposing biological actions via two nuclear receptors, homodimerization and formation of mixed-ligand heterodimers of nuclear receptors, weak coactivator recruitment by EDC-liganded spare receptors on promotors, incoherent feedforward regulation through membrane and nuclear receptors, ligand-induced receptor desensitization or degradation, opposing effects of parent compound and its metabolite, coactivator squelching, induction of repressor, and negative feedback regulation (Kohn and Portier 1993, Kohn and Melnick 2002, Conolly and Lutz 2004, Li et al. 2007, Vandenberg et al. 2012, Cookman and Belcher 2014, Lagarde et al. 2015, Xu et al. 2017). Except few limited experimental investigations (Liang et al. 2014, Villar-Pazos et al. 2017), most of these proposed mechanisms remain to be validated. As shown in the present study, many EDCs produced NMDRs in the Tox21 ER assays, suggesting that the ER signaling pathways in the cell constructs used might be sufficient to underpin at least some of these nonmonotonic responses. With the Tox21 ER assay constructs (Fig. 1A and 1B), since the MCF7 cells used here express ERα predominantly with little, if any, ERβ expression (Rogers and Denison 2000, Li et al. 2014, Brennan et al. 2016), it is unlikely to involve the two-receptor-opposing-action mechanism, although other nuclear receptors which may bind to the EREs cannot be completely ruled out. The full-length ERα and formation of homodimers as well as possible mixed-liganded heterodimers may provide some nonlinearity amenable for the emergence of U or Bell-shaped CRCs as predicted by mathematical models (Conolly and Lutz 2004, Li et al. 2007). In comparison, although GAL4 by itself can also form homodimers to bind to the DNA response element (Hong et al. 2008), it is unclear whether the ER_LBD-GAL4_DBD fusion protein engineered in the HEK 293 cells can homodimerize. While it is not known exactly what coactivators may be involved in these cells to enable reporter gene transcription, weak recruitment of coactivator to the ER already bound on the response elements and coactivator squelching can underpin Bell-shaped CRCs (Kohn and Melnick 2002, Li et al. 2007). Bell curves are nearly twice likely to appear in ERα assays than ERβ assays, independent of the reporter cells used. It is likely that the differential involvement of AF1 and/or AF2 domains in ERα vs. ERβ required for coactivator recruitment may be responsible for the difference (McInerney et al. 1996, Cowley and Parker 1999). Lastly, given the multiple clustered response elements in the reporter constructs in both MCF7 and HEK 293 cells, positive cooperative binding may be another mechanism for generating U-shaped CRCs (Kohn and Portier 1993). All in all, with many estrogenic or non-estrogenic compounds exhibiting NMDRs in the same cells, it is worth further investigation for potential common mechanisms.

As one of the most studied EDCs, BPA was found to exhibit NMDR in many studies, as reviewed in (Vandenberg 2014) and also examined in (Lagarde et al. 2015). The most recent CLARITY-BPA studies have also identified consistent low-dose NMDR effects on a variety of biological endpoints (Prins et al. 2018). Yet, experimental design flaws and contaminations were suspected to contribute to the low-dose effects in the CLARITY-BPA studies although it remains to be further validated (Vom Saal 2018, Vandenberg et al. 2019). Among the 81 statistically significant findings in the in vivo core study, only 2 were found to be NMDR with high confidence (Badding et al. 2019). BPA produces a U-shaped response in Ca²⁺ entry through Cav2.3, a voltage-gated Ca²⁺ channel, in mouse pancreatic β cells upon plasma membrane depolarization. This nonmonotonic effect was demonstrated to be underpinned by the differential actions mediated by ERα and ERβ (Villar-Pazos et al. 2017). Low-dose BPA represses Cav2.3 gene expression via ERβ, resulting in a decrease in Ca²⁺ entry, while high-dose BPA activates ERα to enhance Ca²⁺ current in a PI3K-dependen manner, thus reversing the downtrend at low doses. In female rat ventricular myocytes, it was demonstrated that multiple endpoints, including intracellular Ca²⁺ transient, fractional shortening of myocytes, and arrhythmic activity, followed Bell-shaped dose response relationships (Liang et al. 2014). In this case, the NMDR effects are due to BPA’s stimulatory action on sarcoplasmic reticulum Ca²⁺ release/uptake at low doses and inhibitory effect on inward L-type Ca²⁺ current at high doses. In the Tox21 ER assays, BPA shows Bell-shaped CRCs in assays 2, 5, and 7 (Fig. S20), yet neither meets the criteria set here to be identified as real Bell curves (Fig. S20). In comparison, both BPB (Fig. S21) and BPAF (Fig. S22) exhibit a Bell curve in the absence of cytotoxicity and luciferase inhibition.

Comparison with other approaches, limitations, and uncertainties

Identifying and evaluating NMDR curves is a challenging task. A variety of factors have to be considered, including the numbers of dose points, number of replicates, response magnitude, random sampling errors, assay interference and assay-specific characteristics. By adapting a method developed for evaluating hormesis (Calabrese and Baldwin 1997), Lagarde et al. proposed a set of criteria to assess the plausibility of NMDR relationships for EDCs (Lagarde et al. 2015). These criteria include study quality, number of doses tested, number of doses at which the responses are significantly different from control, and magnitude of NMDR, etc. A numerical score is then calculated based on these criteria to rank the NMDR probability of a dose-response curve, in addition to mechanistic considerations. Applying to in vitro and epidemiological data (Beausoleil et al. 2016) and to in vivo data (Varret et al. 2018), a more stringent, 6-checkpoint method has been recently proposed to evaluate NMDR relationships of EDCs. In this Beausoleil-Varret approach, the minimal number of dose points is 6 including the control group. Then checkpoints 1 and 2 are applied to evaluate statistically whether an NMDR model fits the data better than the null (flat response) model and a monotonic model. Checkpoint 3 is applied to determine whether the NMDR profile relies on the existence of a single outlier-response dose, in another word, whether the dose response data excluding the outlier dose can be reasonably fitted with a monotonic model. Checkpoint 4 is to make sure that the effect sizes (magnitude) in both directions are >5%. Checkpoint 5 constrains the slope of the curve within a limit, and checkpoint 6 excludes multi-phasic curves.

In comparison with the Beausoleil-Varret approach which uses curve-fitting, we used a classification approach in the present study to first identify NMDR curves and then evaluate their validity against a number of criteria, including magnitude, peak or nadir concentration locations, and interference by cytotoxicity, luciferase inhibition, and autofluorescence. Our exclusion of NMDRs resulting from a single-point spike and of NMDRs with peak/nadir point falling on the second lowest or second highest concentrations is similar to checkpoint 3 in the Beausoleil-Varret approach. However, our approach can lead to NMDR cases where although the peak/nadir points fall on the third lowest/highest concentrations, the peak/nadir activity level is only marginally different than the activity level at the second lowest/highest concentrations. Examples are the U curves identified for (Z)-4-hydroxytamoxifen in assays 6 and 8 (Fig. S15), where their U shape profile seems to hinge on the dramatically increased activity at the highest concentration point. Such results reduce the confidence in the identified NMDR curves. In future iterations of our approach, the Beausoleil-Varret’s checkpoint 3 can be adopted to address this issue of uncertainty, which proposes to overlay a monotonic curve to check whether it falls on the 95% confidence interval of the single data point that renders the NMDR profile possible. A caveat of the Beausoleil-Varret’s approach is that implementing checkpoint 3 requires visual inspection of the data points and the manually overlaid monotonic curve. Automation of checkpoint 3 is needed in future for large HTS datasets such as Tox21 assays. Compared to the 5% magnitude cut-off used in checkpoint 4 in Beausoleil-Varret’s approach, we used 15% which is more stringent and compensates for the lack of using statistical analysis as a step to determine the confidence in NMDR. Reducing the magnitude cut-off to lower values is expected to identify additional NMDR curves. Steep, sigmoidal responses are not uncommon in biological responses (Bagowski et al. 2003, Zhang et al. 2013), especially for in vitro systems, therefore our approach does not consider slope as a limiting factor. We also identified an S-shaped curve, but it is rare in the Tox21 ER assays.

The cut-off values applied to each step in our approach are based on the datasets and were set manually at levels that tend to remove CRCs of small changes aggressively. Some more objective metrics, such as multiples of standard deviations of the dataset (Inglese et al. 2006), may be used. But we expect that it will only affect the borderline cases, not altering the general results and conclusion. Moreover, some chemicals were tested by using two or more compounds from various sources (each with a different SIDs) and these compounds often exhibit similar NMDR behaviors in the same assays. Such consistency enhances the confidence and reduces the uncertainties in the NMDRs identified for these chemicals. A compound can also exhibit NMDR, especially Bell, curves in multiple ER assays, which provides additional confidence to the trueness of the nonmonotonic effects. Regardless, since the reaction volume of the Tox21assays is only 4–6 μl in a 1536-well plate format, pipetting errors and other experimental variations can be significant with small volumes. The borderline NMDR cases and those with low confidence can be validated with large-volume assays if needed. Taken together, the NMDR identification approach we developed here can be further improved and adapted for other Tox21 qHST assays.

Significance and Implications

Tox21 assays are a rich resource for dose response information. Besides the ER assays analyzed here, NMDR curves have also been identified in other Tox21 assays. 29 chemicals were found to exhibit Bell or multi-phasic CRCs in p53 Tox21 assays (Witt et al. 2017). Klimenko adapted Beausoleil-Varret’s approach to analyze NMDRs of the Tox21 AR agonist luciferase assay and identified 107 compounds that show NMDR on at least one occasion. (Klimenko 2021). In the study chemicals flagged as cytotoxic or luciferase inhibitor were taken into consideration to eliminate false positives. In comparison, our exclusion method in this aspect is better because chemical’s cytotoxicity, luciferase inhibition, and autofluorescent activity were not simply treated as binary, i.e., active or inactive. Rather, their magnitude and correlations with the reporter activity within the same concentration range were examined to determine whether their changes coincide.

Besides the Tox21 ER assays, many other ER-related in vitro assays were also used to screen for estrogenic compounds, including those in the EPA’s ToxCast program (Browne et al. 2015). These assays cover a variety of key events in ER signaling, including ER binding, dimerization, DNA binding, transcriptional activation, and endpoint responses such as cell proliferation. Integrating across these assays to evaluate an estrogenic chemical holistically has been done to provide bioactivity signature and confidence that cannot be achieved by a single type of assay (Rotroff et al. 2014, Dreier et al. 2015, Judson et al. 2015). Key parameters governing the ER signaling pathways, including the expression levels of ERα, ERβ, coactivators, and corepressors, may vary with cell type, gender, and age. Therefore, the frequency of NMDRs involving ER signaling is likely to vary in different tissues, between genders, and at different physiological stages. In conjunction with exposure assessment (Becker et al. 2015) and better characterization of the free and cellular concentrations against the nominal concentration in in vitro assays (Fischer et al. 2017), NMDR assessment can provide additional dimensions to the risk assessment and regulatory decision making on low-dose exposure to environmental EDCs (Zoeller and Vandenberg 2015, Hill et al. 2018).

Highlights.

• A method was developed to identify concentration-response shapes in Tox21 ER assays

• Hundreds of compounds exhibit nonmonotonic concentration-response curves (CRCs)

• Bell-shaped CRCs are about 7 times more frequent than U-shaped CRCs

• Many compounds exhibit nonmonotonic CRCs in more than one ER assays

• The algorithm can be adapted and applied to other high-throughput bioassays