Cytotoxicity Burst? Differentiating Specific from Nonspecific Effects in Tox21 in Vitro Reporter Gene Assays

Abstract

Background:

High-throughput screening of chemicals with in vitro reporter gene assays in Tox21 has produced a large database on cytotoxicity and specific modes of action. However, the validity of some of the reported activities is questionable due to the “cytotoxicity burst,” which refers to the supposition that many stress responses are activated in a nonspecific way at concentrations close to cell death.

Objectives:

We propose a pragmatic method to identify whether reporter gene activation is specific or cytotoxicity-triggered by comparing the measured effects with baseline toxicity.

Methods:

Baseline toxicity, also termed narcosis, is the minimal toxicity any chemical causes. Quantitative structure–activity relationships (QSARs) developed for baseline toxicity in mammalian reporter gene cell lines served as anchors to define the chemical-specific threshold for the cytotoxicity burst and to evaluate the degree of specificity of the reporter gene activation. Measured 10% effect concentrations were related to measured or QSAR-predicted 10% cytotoxicity concentrations yielding specificity ratios (SR). We applied this approach to our own experimental data and to $\sim \mathrm{8,000}$ chemicals that were tested in six of the high-throughput Tox21 reporter gene assays.

Results:

Confirmed baseline toxicants activated reporter gene activity around cytotoxic concentrations triggered by the cytotoxicity burst. In six Tox21 assays, 37%–87% of the active hits were presumably caused by the cytotoxicity burst ($\text{SR}<1$) and only 2%–14% were specific with $\text{SR}\ge 10$ against experimental cytotoxicity but 75%–97% were specific against baseline toxicity. This difference was caused by a large fraction of chemicals showing excess cytotoxicity.

Conclusions:

The specificity analysis for measured in vitro effects identified whether a cytotoxicity burst had likely occurred. The SR-analysis not only prevented false positives, but it may also serve as measure for relative effect potency and can be used for quantitative in vitro–in vivo extrapolation and risk assessment of chemicals. https://doi.org/10.1289/EHP6664

Introduction

The increasing abundance, number, and heterogeneity of anthropogenic chemicals in our environment call for high-throughput effect screening of chemicals while complying with the strategy to reduce, refine, and replace animal testing (Burden et al. 2015; National Research Council 2007). The application of in vitro cell-based bioassays has emerged in the last decade, and their implementation in miniaturized test systems increased the throughput of such bioassays, culminating in collaborative high-throughput screening (HTS) approaches like the Tox21 program (Tice et al. 2013), in which thousands of chemicals were tested in a battery of $\sim 70$ in vitro bioassays (Attene-Ramos et al. 2013) or ToxCast™, where a smaller number of chemicals were tested in hundreds of cellular bioassays at receptor and pathway level (Hsieh et al. 2017; Judson et al. 2010) but also in cell-free assays (Sipes et al. 2013). The resulting large database is freely available on the U.S. EPA’s Chemistry Dashboard (Richard et al. 2016; U.S. EPA 2019) and has been linked to in vivo databases (Hu et al. 2015), applied for hazard assessment (Pham et al. 2016; Reif et al. 2010) and quantitative in vitro–in vivo extrapolation (QIVIVE) for risk assessment (Sipes et al. 2017; Thomas et al. 2019; Wetmore 2015).

In vitro assays that apply cells stably transfected with reporter genes coupled to the biological receptor of interest are particularly promising tools enabling the classification of chemicals according to their mode of action and/or their potential to disturb biological pathways in humans (Krewski et al. 2020). Upon interaction of a chemical with the receptor, the reporter gene is expressed, triggering the synthesis of reporter proteins and enzymes, e.g., luciferase or $\mathrm{\beta }\text{-lactamase}$, that can be detected by adding appropriate substrates to quantify the enzyme activity. ToxCast™/Tox21 includes reporter gene assays for hormone activity [e.g., estrogen receptors (Filer et al. 2014; Huang et al. 2014) or others (Kleinstreuer et al. 2015)], for activation of metabolic enzymes [e.g., aryl hydrocarbon receptor AhR (Brennan et al. 2015; Murk et al. 1996)] or adaptive stress response assays (Martin et al. 2010).

One important confounding factor of in vitro reporter gene activation is the cytotoxicity of the dosed chemicals. Cytotoxicity has been assessed in parallel to reporter gene activation in many of the assays in Tox21. Time-dependence of cytotoxicity was assessed for two cell lines (HepG2 and HEK293) using metabolic activity or loss of cell membrane integrity as a measure of cytotoxicity (Hsieh et al. 2017). Kinetics of cytotoxicity depended on the underlying pathway triggering cytotoxicity with activation of $\mathrm{\gamma }\text{-}\mathrm{H}2\text{AX}$, which is related to genotoxicity pathways being particularly fast, followed by those associated with mitochondrial disruption (Hsieh et al. 2017). However, no quantitative picture emerged. In our experience the typically used cell viability assays using fluorometric methods to detect metabolic activity or loss of cell membrane integrity are prone to artifacts and, as observed by Hsieh et al. (2017), produce inconsistent results. We are therefore routinely screening cytotoxicity during reporter gene assays measurements by assessing confluence with a cell imager directly after dosing and prior to activity measurement of the reporter protein (Escher et al. 2019). Although, for most chemicals, this imaging method provided effect concentrations similar to those of the fluorometric cell viability assay, artifacts of apparent high metabolic activity while cells have already disappeared under the microscope can be avoided.

The minimal toxicity any chemical can elicit is baseline toxicity (or narcosis) that results from disturbance of structure and functioning of the cell membrane by the presence of chemicals in the membrane (van Wezel and Opperhuizen 1995). Many functions are lost in response to baseline toxicity but mitochondria are especially affected and lose their ability for energy transduction leading to ATP depletion as membranes become permeable (Vinken and Blaauboer 2017). Lipophilic chemicals that have high sorption affinities to phospholipid membranes trigger baseline toxicity at lower dosed concentrations than hydrophilic chemicals, but the cytotoxic concentrations in the cell membranes do not differ much between different chemicals causing baseline toxicity (van Wezel and Opperhuizen 1995). We confirmed for eight reporter gene cell lines that chemicals triggered baseline toxicity when reaching a critical membrane concentration of approximately $70\mathrm{mmol}\times {\mathrm{L}}_{\mathrm{lip}}^{-1}$ (Escher et al. 2019). We developed quantitative structure–activity relationships (QSARs) for these cell lines to predict the 10% inhibitory concentrations (${\text{IC}}_{10}$) based on one chemical parameter, the liposome–water partition constant ($K_{\text{lip}/\mathrm{w}}$) (Escher et al. 2019). Known baseline toxicants (Vaes et al. 1998) were used to derive the QSARs. The QSARs were very similar across cell lines with differences caused by the assays conditions, mainly the serum content of the medium (Escher et al. 2019).

Close to cell death, the cells activate many cellular signaling pathways hence the exposure to high concentrations of chemicals may lead to a nonspecific activation of the reporter gene, a phenomenon termed “cytotoxicity burst” (Judson et al. 2016). Judson et al. (2016) developed a statistical approach to identify cytotoxicity burst in large in vitro platforms that include thousands of data points. Another measure of selectivity of chemicals was the number of assays activated by a chemical and the ratio of the effect concentration of the most sensitive assay to the 10th percentile of the distribution of all assays (Thomas et al. 2013). Fay et al. (2018) recently refined the diagnosis of the cytotoxicity burst phenomenon. They proposed a diagnostic odds ratio that differentiates assays that respond in the range of cytotoxic concentrations from those that are active at much lower concentrations and hence true responders. They also related the predicted baseline toxicity concentrations from Fischer et al. (2017) with the threshold for the cytotoxicity burst and concluded that the cytotoxicity burst phenomenon is more complex than baseline toxicity.

The goal of the present study was to quantify the degree of specificity of a chemical for each reporter gene assay directly from the experimental effect data without the need to analyze them in context with other chemicals and bioassays. We hypothesized that if the reporter gene activation occurs at concentrations close to baseline toxicity, it is likely not a specific effect but that the effect resulted from the cytotoxicity burst. To test this hypothesis, we measured the cytotoxicity and reporter gene activation of seven confirmed baseline toxicants and eight additional chemicals of environmental and toxicological relevance in eight standardized and widely used in vitro reporter gene assays. For chemicals that are more cytotoxic than baseline toxicity, the relationship between the receptor or pathway affected and the degree of cytotoxicity enhancement over baseline toxicity was investigated. Aiming to capture the big picture in an HTS in vitro platform, we applied the specificity analysis to the Tox21 in vitro reporter gene database to evaluate which role the cytotoxicity burst may play.

Materials and Methods

Experimental Study Chemicals

We selected seven confirmed baseline toxicants (2-Phenylphenol, 3-Nitroaniline, 4-Chloro-3-methylphenol, 4-Pentylphenol, 2-Allylphenol, 2-Butoxyethanol, 2,4,5-Trichloroaniline) that had been used to set up baseline toxicity QSARs for eight reporter gene cell lines (Escher et al. 2019) and seven additional chemicals that were frequently found in environmental samples (Bisphenol A, Quinoxyfen, Fluoranthene) or in foodstuff (Genistein, Coumarin, 8-Gingerol, Zingerone). Their names and physicochemical properties are listed in Table S1. None of the chemicals tested were below the volatility cutoff of a medium–air partition constant of 10,000 (Escher et al. 2019).

The experimental liposome–water partition constants ($K_{\text{lip}/\mathrm{w}}$) of the seven baseline toxicants were taken from Vaes et al. (1997). In the second set, the $K_{\text{lip}/\mathrm{w}}$ stem from Kwon et al. (2006) and van der Heijden and Jonker (2009). We included ionizable chemicals that dissociate into a neutral ($f_{\text{neutral}}$) and ionized fraction ($f_{\text{ionized}}$) in the in vitro assay medium at pH 7.4. From the acidity constants $\mathrm{p}{K}_{\mathrm{a}}$ as well as the $K_{\text{lip}/\mathrm{w}}$ of the neutral and charged species (Table S1), ionization-corrected liposome–water distribution ratios [$D_{\text{lip}/\mathrm{w}}$ (pH 7.4)] can be calculated (Equation 1) or directly measured (Henneberger et al. 2019, 2020).

$$\log D_{\text{lip}/\mathrm{w}}\left(\text{pH\hspace{0.17em}}7.4\right)=f_{\text{neutral}}\times \log K_{\text{lip}/\mathrm{w}}(\text{neutral})+(1\text{-}{f}_{\text{neutral}})\times {\log \mathit{K}}_{\text{lip}/\mathrm{w}}(\text{ion})$$ (1)

Reporter Gene Assays

Eight reporter gene assays (Table 1) were performed as described in (König et al. 2017; Neale et al. 2017). The cell confluency served as surrogate for cell viability as previously described (Escher et al. 2019).

Table 1.

Reporter gene cell lines applied in this study and baseline toxicity QSAR (Escher et al. 2019).

Reporter gene cell line	Derived from	Slope of baseline toxicity QSAR	Intercept of baseline toxicity QSAR	Corresponding assay in Tox21 for reporter gene activationa	Corresponding cytotoxicity assay in Tox21a
AREc32	MCF7	$0.56\pm 0.09$	$1.76\pm 0.28$	(not in Tox21)	(not in Tox21)
ARE-BLA	HepG2	$0.68\pm 0.08$	$1.19\pm 0.26$	TOX21_ARE_BLA_Agonist_ratio	TOX21_ARE_BLA_Agonist_viability
AhR-CALUX (H4L7.5c2)	H4IIe	$0.73\pm 0.10$	$1.28\pm 0.31$	TOX21_AhR_LUC_Agonistb	TOX21_AhR_LUC_Agonist_viabilityb
$\text{PPAR}\mathrm{\gamma }\text{-BLA}$	HEK293H	$0.64\pm 0.20$	$1.71\pm 0.69$	TOX21_PPARg_BLA_Agonist_ratio	TOX21_PPARg_BLA_Antagonist_viability
AR-BLA	HEK293T	$0.76\pm 0.14$	$1.44\pm 0.49$	TOX21_AR_BLA_Agonist_ratio	TOX21_AR_BLA_Antagonist_viability
$\text{ER}\mathrm{\alpha }\text{-BLA}$	HEK293T	$0.76\pm 0.10$	$1.54\pm 0.42$	TOX21_ERa_BLA_Agonist_ratio	TOX21_ERa_BLA_Antagonist_viability
PR-BLA	HEK293T	$0.70\pm 0.16$	$1.52\pm 0.11$	(not included)	(not included)
GR-BLA	HEK293T	$0.72\pm 0.13$	$1.67\pm 0.42$	TOX21_GR_BLA_Agonist_ratioc	TOX21_GR_BLA_Antagonist_viabilityc

Note: AhR, aryl hydrocarbon receptor; AR, androgen receptor; ARE, antioxidant response element; BLA, GeneBLAzer reporter gene cell line; CALUX, Chemical Activated LUciferase gene eXpression; ER, estrogen receptor; GR, glucocorticoid receptor; $\text{PPAR}\mathrm{\gamma }$, peroxisome proliferator activated receptor gamma; PR, progesterone receptor; QSAR, Quantitative Structure Activity Relationship.

^aAs referenced in https://comptox.epa.gov/dashboard.

^bIn Tox21 a HepG2-based AhR-CALUX cell line was used (He et al. 2011).

^cIn Tox21 a HeLa-derived cell line was used.

Briefly 2,500 to 5,000 cells in $30\mathrm{\mu }\mathrm{L}$ medium were plated in each well of a black 384-well polystyrene microtiter plate with clear bottom (AREc32 #3764; all other cell lines BioCoat #356663, Corning), leaving the last column as control without cells for the GeneBLAzer cell lines, and incubated for 24 h at 37°C, 5% ${\mathrm{CO}}_2$ to let the cells attach. All medium components were purchased from Gibco. Media were 90% $\text{DMEM\hspace{0.17em}}+\text{\hspace{0.17em}GlutaMAX}$ plus 10% FBS and $100\mathrm{U}/\mathrm{mL}$ penicillin and $100\mathrm{\mu }\mathrm{g}/\mathrm{mL}$ streptomycin for AhR-CALUX and AREc32 cells, 90% phenol red–free DMEM, 10% dialyzed FBS, $0.1\text{mM}$ NEAA, $25\text{mM}$ HEPES, $1\text{mM}$ sodium pyruvate, $100\mathrm{U}/\mathrm{mL}$ penicillin and $100\mathrm{\mu }\mathrm{g}/\mathrm{mL}$ streptomycin and $4\text{mM}$ GlutaMAX for ARE-BLA and 98% Opti-MEM supplemented with 2% charcoal-stripped FBS $100\mathrm{U}/\mathrm{mL}$ penicillin and $100\mathrm{\mu }\mathrm{g}/\mathrm{mL}$ streptomycin for all other GeneBLAzer cell lines. During the initial 24 h, the cell number did not increase visibly, but cells attached (Escher et al. 2019). Plated cell numbers were adjusted between 2,500 to 5,000 cells per well depending on the cell line that the confluency was ∼30% to 50% prior to dosing and no more than 80% after 24 h of exposure (Escher et al. 2019). Before dosing and after additional 24 h of exposure, the cell confluency was measured with an IncuCyte S3 live cell imaging system (Essen BioScience).

The detection of activation in the reporter gene assays AREc32 and AhR-CALUX encoding for luciferase and the GeneBLAzer assays encoding for $\mathrm{\beta }\text{-lactamase}$ was performed as previously described (Escher et al. 2012; König et al. 2017; Neale et al. 2017). Briefly, after 24 h of exposure and measurement of cell confluency, AREc32 and AhR-CALUX cells were washed twice with PBS. Subsequently $10\mathrm{\mu }\mathrm{L}$ of lysis buffer per well was added [$25\text{mM}$ Tris (AppliChem, A13790500), 1% Triton-X 100 (Geyer Chemsolute, 8059), $2\text{mM}$ EDTA (AppliChem, A11040500), $2\text{mM}$ DTT (Sigma-Aldrich, D0632), 10% glycerol (AppliChem, A11231000)], followed by a 15–20 min incubation at room temperature (RT) and shaking at $\mathrm{1,500}\text{rpm}$ to allow complete lysis of cells. Afterward, $40\mathrm{\mu }\mathrm{l}$ of luciferase substrate buffer [$20\text{mM}$ Tricine (Sigma-Aldrich, T0377), $2.67\text{mM}$ MgSO4 (AppliChem, 131404.1210), $33.3\text{\hspace{0.17em}mM}$ DTT (Sigma-Aldrich, D0632), $0.1\text{\hspace{0.17em}mM}$ EDTA (AppliChem, A11040500), $0.261\text{\hspace{0.17em}mM}$ coenzyme A (Sigma-Aldrich, C3144), $0.53\text{mM}$ ATP (Sigma-Aldrich, A2383), $0.235\text{mM}$ D-luciferin (AREc32), and $0.059\text{mM}$ D-luciferin (AhR) (AAT Bioquest, ABD-12506)] was added, and luminescence was read with a Tecan Infinite® M1000 plate reader. For the detection of the activity of the $\mathrm{\beta }\text{-lactamase}$ in the GeneBLAzer® assays $8\mathrm{\mu }\mathrm{l}$ of ToxBLAzer™ substrate (ThermoFisher Scientific) were added to each well of the plate, and fluorescence was measured immediately to allow elimination of autofluorescence and after 2 h incubation at RT for reporter gene activation using a Tecan Infinite® M1000 plate reader. The ToxBLAzer™ substrate allowed a ratiometric detection of the reporter gene activity by measuring the emission of blue ($460\text{\hspace{0.17em}nm}$) and green ($530\text{\hspace{0.17em}nm}$) light using the M1000 plate reader.

Dosing of Chemicals

Chemical stocks dissolved in DMSO or neat liquid chemicals were dosed into medium at 4 × concentrations on 384-well plates with a Tecan D300e Digital Dispenser (Tecan) and then $10\mathrm{\mu }\mathrm{l}$ per well of chemicals in medium were transferred with a 96-pipette head (Hamilton Microlab Star) into the cell plates that contain cells as described above in $30\mathrm{\mu }\mathrm{L}$ medium. The chemicals were dosed in final concentrations up to three times their predicted ${\text{IC}}_{10}$ for baseline toxicity (Fischer et al. 2019), with the dose range depicted in the concentration–response curves (Figures S1–S15) with different symbols for each independent experiment ($n\ge 3$).

Data Evaluation

Cytotoxicity was expressed as percent inhibition of cell viability in comparison with unexposed cells (ratio of confluency of exposed to confluency of unexposed cells). The inhibitory concentration for 10% cytotoxicity, ${\text{IC}}_{10}$ (Equation 2) was determined from the linear range of the concentration–cytotoxicity curves ($\%\text{cytotoxicity}=\text{slope}\times \text{concentration}$) as described previously (Escher et al. 2018).

$${\text{IC}}_{10}=\frac{10\%}{\text{slope}}$$ (2)

The effect concentrations ${\text{EC}}_{10}$ were calculated analogously from the linear concentration–effect curves ($\%\text{\hspace{0.17em}effect}=\text{slope}\times \text{concentration}$) with Equation 3.

$${\text{EC}}_{10}=\frac{10\%}{\text{slope}}$$ (3)

For AREc32 and ARE-BLA, the effect concentration causing an induction ratio (IR) of 1.5, ${\text{EC}}_{\text{IR}1.5}$, was derived from the linear regression of the IR against the concentration ($\text{IR}=1+\text{slope}\times \text{concentration}$) for $\text{IR}<4$ (Escher et al. 2018).

$${\text{EC}}_{\text{IR}1.5}=\frac{0.5}{\text{slope}}$$ (4)

Baseline Toxicity QSARs

Baseline toxicity QSARs of the form given in Equation 5 previously developed for the eight cell lines [(Escher et al. 2019), Table 1] were used to predict the ${\text{IC}}_{10}$ for baseline toxicity (${\text{IC}}_{10,\text{baseline}}$) of the study chemicals using the ionization-corrected log $D_{\text{lip}/\mathrm{w}}$ (pH 7.4) calculated by Equation 1.

$${\log [1/\text{IC}}_{10,\text{baseline}}\left(\mathrm{M}\right)]=\text{slope}\times \log D_{\text{lip}/\mathrm{w}}(\text{pH\hspace{0.17em}}7.4)+\text{intercept}$$ (5)

We replaced the $K_{\text{lip}/\mathrm{w}}$ of the neutral species in the original QSAR by the ionization-corrected $D_{\text{lip}/\mathrm{w}}$ (pH 7.4) in Equation 5 to include also charged chemicals. The expansion of QSARs for neutral chemicals to ionizable chemicals was previously described for bacteria (Escher et al. 2017) and the zebrafish embryo toxicity test (Klüver et al. 2019). A potential ion-trapping effect does not need to be accounted for, provided the pH does not deviate much from pH 7.4, because ion-trapping becomes relevant only if the intracellular pH and the extracellular pH differ by more than one pH unit (Escher et al. 2020).

Specificity Analysis

The toxic ratio (TR, Equation 6) is a measure of enhanced cytotoxicity, i.e., how much more potent a chemical is in comparison with its predicted baseline toxicity.

$$\text{TR}=\frac{{\text{IC}}_{10,\text{baseline}}}{{\text{IC}}_{10}}$$ (6)

For TR, it is commonly accepted that a $\text{TR}\ge 10$ is associated with specific or reactive toxicity (Maeder et al. 2004), and any chemical with $\text{TR}<10$ is considered a baseline toxicant.

We defined the specificity ratio as the ratio between cytotoxicity (${\text{IC}}_{10}$) and effect concentration (${\text{EC}}_{10}$ or ${\text{EC}}_{\text{IR}1.5}$). The specificity ratio can relate either to the experimental ${\text{IC}}_{10}$ (${\text{SR}}_{\text{cytotoxicity}}$, Equation 7) or to the predicted ${\text{IC}}_{10,\text{baseline}}$ (${\text{SR}}_{\text{baseline}}$, Equation 8), as conceptually illustrated in Figure 1.

$${\text{SR}}_{\text{cytotoxicity}}=\frac{{\text{IC}}_{10}}{{\text{EC}}_{10}}{\text{\hspace{1em}or\hspace{1em}SR}}_{\text{cytotoxicity}}=\frac{{\text{IC}}_{10}}{{\text{EC}}_{\text{IR}1.5}}$$ (7)

$$\begin{array}{ccc}{\text{SR}}_{\text{baseline}}=\frac{{\text{IC}}_{10,\text{baseline}}}{{\text{EC}}_{10}}& \text{or}& {\text{SR}}_{\text{baseline}}=\frac{{\text{IC}}_{10,\text{baseline}}}{{\text{EC}}_{\text{IR}1.5}}\end{array}$$ (8)

Figure 1.

Conceptual illustration of the proposed specificity analysis framework. The line corresponds to the Quantitative Structure Activity Relationship (QSAR) for baseline toxicity. The experimental effect concentration [log (1/EC)] is depicted by a blue square; the experimental inhibitory concentration [log ($1{/\text{IC}}_{10}$)] leading to 10% cytotoxicity is depicted by a red circle. The distance between the log [$1{/\text{IC}}_{10}$ (QSAR)] and the experimental log ($1{/\text{IC}}_{10}$) is the toxic ratio log TR. The distance between the experimental log ($1{/\text{IC}}_{10}$) and the experimental log (1/EC) is the specificity ratio log ${\text{SR}}_{\text{cytotoxicity}}$. The distance between the log [$1{/\text{IC}}_{10}$ (QSAR)] and the experimental log (1/EC) is the specificity ratio log ${\text{SR}}_{\text{baseline}}$. For better legibility, log(1/y) was omitted in the graph, but all measures are in form of negative logarithms. Note: EC, effect concentration; exp., experimental; QSAR, Quantitative Structure Activity Relationship; SR, specificity ratio; TR, toxic ratio.

In analogy with the threshold for TR, we considered $\text{SR}\le 1$ as not specific, $1\le \text{SR}<10$ as moderately specific (with high uncertainty), $10\le \text{SR}<100$ as specific, and $100\le \text{SR}$ as highly specific.

Tox21 Data Extraction and Processing

Concentration–effect data for the 8,628 chemicals from the Tox21 10K library with available chemical identifiers (CAS number and DSSTox ID) were extracted for ARE-BLA, AhR-CALUX, $\text{PPAR}\mathrm{\gamma }\text{-BLA}$, AR-BLA, $\text{ER}\mathrm{\alpha }\text{-BLA}$, and GR-BLA (see Table 1 for Tox21 assay notation). Reporter gene activation and cell viability were downloaded from the Tox21 Concentration-Response Browser as percent of the maximum effect of the positive control ($=100\%$) (https://sandbox.ntp.niehs.nih.gov/tox21-curve-visualization, last accessed 10 June 2019). No data for AREc32 were available in Tox21. The database includes effect data from the U.S. EPA, the National Toxicity Program, and the Federal Drug Administration laboratories. In Tox21, chemicals were partly tested multiple times in one or more laboratories, and provided that the same chemical ID was tested (equal vendor and purity), the concentration–effect data were merged and reevaluated using the same linear concentration–response analysis (Escher et al. 2018) as that used for the experimental data measured for this study, yielding single ${\text{IC}}_{10}$ and ${\text{EC}}_{10}$ for each chemical and assay. In the BLA-bioassays, cell viability was quantified via adenosine triphosphate (ATP) with CellTiter-Glo, and the decrease of the signal served as measure of cytotoxicity. In AhR-CALUX, the cytotoxicity was quantified with CellTiter-Fluor based on protease activity. Note that cell viability was measured by microscopy in our own experiments; thus differences in the sensitivity between the two cytotoxicity measurement methods may represent a source of error. For the Tox21 ARE-BLA, $\text{PPAR}\mathrm{\gamma }\text{-BLA}$, AR-BLA, and $\text{ER}\mathrm{\alpha }\text{-BLA}$, the reporter gene constructs, cell lines, and specific effect measurement techniques were the same as those used in our experiments. AhR agonism was measured by CALUX luciferase quantification in both Tox21 and our experiments, but Tox21 used the HG2L7.5c1 cells derived from HepG2 (He et al. 2011), whereas we applied H4L7.5c2 cells (Brennan et al. 2015). Another difference was that we used IR as an effect measure for ARE-BLA. For GR-BLA, we used the construct based on HEK293T, and Tox21 applied the HeLa-derived GR-BLA. Cytotoxicity data were available for ARE-BLA and AhR-CALUX, but for the other cell lines cytotoxicity was derived from the antagonistic assays. This difference can be justified because the potent agonists were added at very low constant concentrations, where they did not cause any cytotoxicity.

The data were fitted by linear regression as described in (Escher et al. 2018) using MATLAB R2018a with the code detailed in Text S1 and S2. ${\text{EC}}_{10}$ and ${\text{IC}}_{10}$ for all chemicals in the six assays were calculated by Equations 2 and 3, and standard errors were derived according to Escher et al. (2018). Concentrations triggering $>30\%$ effect and $>50\%$ cytotoxicity were excluded from the fit, because linearity was observed only for the lower portion of concentration–response curves in in vitro reporter gene assays (Escher et al. 2018). Chemicals with ${\text{EC}}_{10}$ and ${\text{IC}}_{10}>0$ and $<\text{their maximum test}$ concentration were classified active and/or cytotoxic. ${\text{EC}}_{10}$ and ${\text{IC}}_{10}$ with relative standard errors $>50\%$ were classified “inconclusive” and excluded from the specificity analysis. The remaining ${\text{EC}}_{10}$ and ${\text{IC}}_{10}$ were analyzed for their TR, ${\text{SR}}_{\text{cytotoxicity}}$, and ${\text{SR}}_{\text{baseline}}$ by Equations 6–8.

The ${\text{IC}}_{10,\text{baseline}}$ of all Tox21 chemicals were predicted with the baseline toxicity QSARs (Table 1). The log $K_{\text{lip}/\mathrm{w}}$ of the neutral species were predicted by a log $K_{\text{ow}}\text{-based}$ QSAR [(Endo et al. 2011), Equation 9] and the log $D_{\text{lip}/\mathrm{w}}$ (pH 7.4) with Equation 10 that was derived from Equation 1 by assuming 10 times lower affinity of the ionized species to phospholipid liposomes (Bittermann et al. 2016; Escher et al. 2020). The fraction of the neutral and ionized species was calculated with the Henderson-Hasselbalch equation from the acidity constants $\mathrm{p}{K}_{\mathrm{a}}$, which were estimated with ACD/Percepta $\mathrm{p}{K}_{\mathrm{a}}$ using the GALAS algorithm (www.acdlabs.com/software/pka/).

$$\log K_{\text{lip}/\mathrm{w}}=1.01\times \log K_{\text{ow}}+0.12$$ (9)

$$\log D_{\text{lip}/\mathrm{w}}=f_{\text{neutral}}\times \log K_{\text{lip}/\mathrm{w}}+f_{\text{ionized}}\times (\log K_{\text{lip}/\mathrm{w}}-1)$$ (10)

Only ${\text{IC}}_{10,\text{baseline}}$ for chemicals with $1<\log$ $D_{\text{lip}/\mathrm{w}}$ $\left(\text{pH\hspace{0.17em}}7.4\right)<5$ were predicted; the others were marked “outside QSAR domain.” The entire workflow of data analysis of the Tox21 data is outlined in the SI, Figure S18.

Results

Cytotoxicity and Reporter Gene Activation of Baseline Toxicants

All seven evaluated baseline toxicants triggered cytotoxicity in the eight reporter gene assays and the ${\text{IC}}_{10}$ were consistent with those previously reported (Escher et al. 2019). The TR of the chemicals in the assays were all within one order of magnitude around 1 ($0.1\le \text{TR}<10$); hence, the measured ${\text{IC}}_{10}$ were similar to the ${\text{IC}}_{10,\text{baseline}}$ predicted by the baseline toxicity QSARs (Figure 2A).

Figure 2.

(A) Toxic ratios and (B) specificity ratios of the baseline toxicants (2-Phenylphenol, 3-Nitroaniline, 4-Chloro-3-methylphenol, 4-Pentylphenol, 2-Allylphenol, 2-Butoxyethanol, 2,4,5-Trichloroaniline) in eight reporter gene assays. The solid line is a ${\text{SR}}_{\text{cytotoxicity}}$ of 1, and the dashed lines are the thresholds of 10 and 0.1. If the effects did not exceed 10%, then no ${\text{IC}}_{10}$ and/or ${\text{EC}}_{10}$ could be derived, and there is no symbol in the figure. Underlying data are in Tables S2–S9.

We previously proposed that all concentrations above the ${\text{IC}}_{10}$ must be omitted before analyzing reporter gene activation (Escher et al. 2018). Here, we have purposely violated this rule to extrapolate the ${\text{EC}}_{\text{IR}1.5}$ and ${\text{EC}}_{10}$ whenever possible, and there are a few examples, where even after the ${\text{IC}}_{10}$ cutoff, valid activation would have been detected. As we measured both cytotoxicity and reporter gene activation for the baseline toxicants, ${\text{SR}}_{\text{cytotoxicity}}$ were calculated according to Equation 7. In AREc32, all confirmed baseline toxicants activated the oxidative stress response (Figure 2B and Figure S1), albeit with very low ${\text{SR}}_{\text{cytotoxicity}}$ ranging from 0.3 to 4.3 (Table S2). Only five chemicals activated ARE-BLA (Figure S2) but did not exceed the SR threshold of 10 [${\text{SR}}_{\text{cytotoxicity}}$ 0.5 to 5.0 (Table S3)], and five chemicals activated AhR-CALUX (Figure S3) with ${\text{SR}}_{\text{cytotoxicity}}$ 0.4 to 8.1 (Table S4). In $\text{PPAR}\mathrm{\gamma }\text{-BLA}$ (Figure S4), 2,4,5-Trichloroaniline showed a specific effect that appeared not to be influenced by cytotoxicity with a ${\text{SR}}_{\text{cytotoxicity}}$ of 28 (Figure 2B and Table S5). No ${\text{SR}}_{\text{cytotoxicity}}$ could be deduced for the hormone receptors apart from 4-Chloro-3-methylphenol and 4-Pentylphenol in $\text{ERA}\mathrm{\alpha }\text{-BLA}$ (Tables S6–S9); still low activity was recorded at cytotoxic concentrations (Figures S4–S8).

Cytotoxicity and Reporter Gene Activation of Environmental Chemicals

To further explore the cytotoxicity burst, we selected seven environmentally relevant chemicals with diverse physicochemical properties, five of which (Bisphenol A, Quinoxyfen, Fluoranthene, Genistein, Coumarin) overlapped with and were reported to be active in the Tox21 reporter gene assays corresponding to the assays performed in this study (https://comptox.epa.gov/dashboard, Tables S1–S6). The concentration–response curves in all eight reporter gene assays are depicted in Figures S9–S16, and ${\text{IC}}_{10}$ and effect concentrations are listed in the corresponding Tables S10–S17. When inserting the experimental ${\text{IC}}_{10}$ into the figures of all baseline QSARs (Figure S19), visual inspection indicated that the more hydrophobic chemicals were often below the baseline, which could be an artifact due to sorptive loss processes or degradation. The QSARs were developed with a test set of chemicals, whose log $K_{\text{lip}/\mathrm{w}}$ ranged from 0.60 (2-Butoxyethanol) to 4.31 (4-Pentylphenol), but three of the environmental chemicals (Quinoxyfen, 8-Gingerol and Fluoranthene) exceeded this range (Table S1). The TR analysis of the ${\text{IC}}_{10}$ for cytotoxicity (Tables S10–S17, Figure 3A) revealed that apart from one single outlier (Genistein in PR-BLA with a TR of 15), all environmental chemicals caused baseline toxicity in the cytotoxicity end point.

Figure 3.

(A) Toxic ratios and (B) specificity ratios of the environmental chemicals (Bisphenol A, Quinoxyfen, Fluoranthene, Genistein, Coumarin, 8-Gingerol, Zingerone) in all reporter gene assays. Underlying data are in Tables S10–S17. The solid line is a ${\text{SR}}_{\text{cytotoxicity}}$ of 1, and the dashed lines are the thresholds of 10 and 0.1. If the effects did not exceed 10%, then no ${\text{IC}}_{10}$ and/or ${\text{EC}}_{10}$ could be derived, and there is no symbol in the figure. Underlying data are in Tables S2–S9.

As expected for known estrogen agonists, Genistein (${\text{SR}}_{\text{cytotoxicity}}=118$, ${\text{SR}}_{\text{baseline}}=441$) and Bisphenol A (${\text{SR}}_{\text{cytotoxicity}}=98$, ${\text{SR}}_{\text{baseline}}=282$) were highly specific in the $\text{ER}\mathrm{\alpha }\text{-BLA}$ (Figure 3B) and because the TR were $<10$, ${\text{SR}}_{\text{cytotoxicity}}$ and ${\text{SR}}_{\text{baseline}}$ were in the same range. Unexpected $\text{SR}\ge 10$ were found for Genistein in AREc32 (${\text{SR}}_{\text{cytotoxicity}}=38$) and Quinoxyfen in $\text{PPAR}\mathrm{\gamma }\text{-BLA}$ (${\text{SR}}_{\text{cytotoxicity}}=25$). All other chemicals did not show activation of the reporter gene or had ${\text{SR}}_{\text{cytotoxicity}}<10$ indicative for moderate or nonspecific effects. Bisphenol A had a ${\text{SR}}_{\text{cytotoxicity}}$ of 1–1.5 and ${\text{SR}}_{\text{baseline}}$ of 1.2–4 in AREc32, ARE-BLA, and AhR-CALUX, which indicates that these activations are of low specificity, and the estrogenic effect was the only true pathway among the tested assays.

Specificity Analysis of Tox21 Effect Data

The corresponding cell lines in Tox21 are identical with exception of AhR and GR-BLA (Table 1). Table 2 summarizes the output of the toxic ratio analysis and Tables 3 and 4 the specificity analysis for the six Tox21 reporter gene assays. Individual results are in the Tables S1–S6. The number of chemicals included in the ${\text{SR}}_{\text{cytotoxicity}}$ analysis was limited to the availability of chemicals, for which both ${\text{EC}}_{10}$ and ${\text{IC}}_{10}$ were measured. Because a relatively narrow and constant concentration range was measured for all chemicals in Tox21, a large proportion of the chemicals were not tested up to baseline cytotoxic concentrations; therefore, false negative counts are possible.

Table 2.

Toxic ratio TR analysis for the six Tox21 in vitro reporter gene assays.

Reporter gene assay	$n$ (chemicals)a	$n$ (TR)b	$n\left(\text{TR}>10\right)$c		$n\left(1\le \text{TR}\le 10\right)$c		$n\left(\text{TR}<1\right)$c
ARE-BLA	7,214	694	384	(55.3%)	275	(39.6%)	35	(5.0%)
AhR-CALUX	7,968	930	524	(56.3%)	353	(38.0%)	52	(5.6%)
$\text{PPAR}\mathrm{\gamma }\text{-BLA}$	6,968	723	326	(45.1%)	349	(48.3%)	48	(6.6%)
AR-BLA	7,959	1,172	485	(41.4%)	532	(45.4%)	155	(13.2%)
$\text{ER}\mathrm{\alpha }\text{-BLA}$	7,947	610	193	(31.6%)	266	(43.6%)	151	(24.8%)
GR-BLA	7,963	599	204	(34.1%)	277	(46.2%)	118	(19.7%)

Note: The underlying data is given in Tables S1–S6.

^aNumber ($n$) of the evaluated chemicals [$n$ (chemicals)].

^bNumber of chemicals $n$ for which a TR could be derived $n$ (TR).

^cBinning into the TR categories ($\text{TR}>10$, $1\le \text{TR}\le 10$, $\text{TR}<1$). In parentheses are the percentages of categories in each bin.

Table 3.

Specificity ratio ${\text{SR}}_{\text{cytotoxicity}}$ analysis for the six Tox21 in vitro reporter gene assays.

Reporter gene assay	$n$ (chemicals)a	$n$ (${\text{SR}}_{\text{cytotoxicity}}$)b	$n$ (${\text{SR}}_{\text{cytotoxicity}}>10$)c		$n$ ($1\le {\text{SR}}_{\text{cytotoxicity}}\le 10$)c		$n$ (${\text{SR}}_{\text{cytotoxicity}}<1$)c
ARE-BLA	7,214	332	29	(8.7%)	180	(54.2%)	123	(37.0%)
AhR-CALUX	7,968	305	29	(9.5%)	115	(37.7%)	161	(52.8%)
$\text{PPAR}\mathrm{\gamma }\text{-BLA}$	6,968	75	2	(2.7%)	12	(16.0%)	61	(81.3%)
AR-BLA	7,959	204	29	(14.2%)	42	(20.6%)	133	(65.2%)
$\text{ER}\mathrm{\alpha }\text{-BLA}$	7,947	181	13	(7.2%)	46	(25.4%)	122	(67.4%)
GR-BLA	7,963	121	2	(1.7%)	14	(11.6%)	105	(86.8%)

Note: The underlying data is given in Tables S1–S6.

^aNumber ($n$) of the evaluated chemicals [$n$ (chemicals)].

^bNumber of chemicals $n$ for which a ${\text{SR}}_{\text{cytotoxicity}}$ could be derived $n$ (${\text{SR}}_{\text{cytotoxicity}}$).

^cBinning into the ${\text{SR}}_{\text{cytotoxicity}}$ categories (${\text{SR}}_{\text{cytotoxicity}}>10$, $1\le {\text{SR}}_{\text{cytotoxicity}}\le 10$, ${\text{SR}}_{\text{cytotoxicity}}<1$). In parentheses are the percentages of categories in each bin.

Table 4.

Specificity ratio ${\text{SR}}_{\text{baseline}}$ analysis for the six Tox21 in vitro reporter gene assays.

Reporter gene assay	$n$ (chemicals)c	$n$ (${\text{SR}}_{\text{baseline}}$)b	$n$ (${\text{SR}}_{\text{baseline}}>10$)c		$n$ ($1\le {\text{SR}}_{\text{baseline}}\le 10$)c			$n$ (${\text{SR}}_{\text{baseline}}<1$)c
ARE-BLA	7,214	1,077	700	(65.0%)	341	(31.7%)	36		(3.3%)
AhR-CALUX	7,968	837	495	(59.1%)	270	(32.3%)	72		(8.6%)
$\text{PPAR}\mathrm{\gamma }\text{-BLA}$	6,968	145	86	(59.3%)	45	(31.0%)	14		(9.7%)
AR-BLA	7,959	318	186	(58.5%)	94	(29.6%)	38		(11.9%)
$\text{ER}\mathrm{\alpha }\text{-BLA}$	7,947	307	114	(37.1%)	117	(38.1%)	76		(24.8%)
GR-BLA	7,963	187	106	(56.7%)	43	(23.0%)	38		(20.3%)

Note: The underlying data is given in Tables S1–S6.

^aNumber ($n$) of the evaluated chemicals [$n$ (chemicals)].

^bNumber of chemicals ($n$) for which a ${\text{SR}}_{\text{baseline}}$ could be derived $n$ (${\text{SR}}_{\text{baseline}}$).

^cBinning into the ${\text{SR}}_{\text{baseline}}$ categories (${\text{SR}}_{\text{baseline}}>10$, $1\le {\text{SR}}_{\text{baseline}}\le 10;{\text{SR}}_{\text{baseline}}<1$). In parentheses are the percentages of categories in each bin.

The ${\text{SR}}_{\text{baseline}}$ and TR analyses included more chemicals than the ${\text{SR}}_{\text{cytotoxicity}}$ analysis because the QSAR-predicted ${\text{IC}}_{10\text{,baseline}}$ were used as an anchor for baseline-associated cytotoxicity. The ${\mathrm{IC}}_{10\text{,baseline}}$ could be derived for approximately 60% of the chemicals (59% for ARE-BLA, 57% for AhR-CALUX, 60% for $\text{PPAR}\mathrm{\gamma }\text{-BLA}$, 57% for AR-BLA, 57% for $\text{ER}\mathrm{\alpha }\text{-BLA}$, and 57% for GR-BLA), the remainder was outside the applicability domain of the baseline toxicity QSAR of $1<\log$ $D_{\text{lip}/\mathrm{w}}$ $\left(\text{pH\hspace{0.17em}}7.4\right)<5$.

Figure 4A and Table 2 report the TR values for all cytotoxic chemicals in the Tox21 reporter gene assays. Approximately 50% of the experimental ${\text{IC}}_{10}$ values were in the range of one order of magnitude around ${\text{IC}}_{10\text{,baseline},}$ hence, the $\text{TR}<10$ and measured cytotoxicity were baseline-associated. Both neutral and ionizable chemicals were included in this analysis. The baseline toxicity QSAR can in principle be extended to ionizable chemicals by replacing the log $K_{\text{lip}/\mathrm{w}}$ with the $D_{\text{lip}/\mathrm{w}}$ (pH 7.4) corrected for ionization, but this has not yet been demonstrated for mammalian cell lines. We therefore split the data set in two subsets of predominantly neutral ($f_{\text{neutral}}>98\%$) and (partially) charged ($f_{\text{neutral}}<98\%$) chemicals, and the resulting TR analysis is depicted in Figure S20 and Table S18. Further, 44% to 61% of chemicals were (partially) charged (Table S18). There was little difference between the TR ranges of neutral and charged chemicals (Figure S20), with slightly more chemicals with $\text{TR}>10$ for the charged chemicals, which can be explained by the fact that some specific modes of action, such as uncoupling, require charged chemicals. Therefore, neutral and charged chemicals were evaluated together.

Figure 4.

(A) Toxic ratio TR, (B) specificity ratio ${\text{SR}}_{\text{cytotoxicity}}$, and (C) specificity ratio ${\text{SR}}_{\text{baseline}}$ of chemicals that triggered the specific effect (${\text{EC}}_{10}$) and/or were cytotoxic (${\text{IC}}_{10}$) within the measured concentration range in the six Tox21 reporter gene assays. The underlying data are in the Tables S1–S6. The total number of chemicals included in the analysis is given in the top row [number of chemicals n for which a TR could be derived n (TR); number of chemicals n for which an ${\text{SR}}_{\text{cytotoxicity}}$ could be derived n (${\text{SR}}_{\text{cytotoxicity}}$); number of chemicals n for which an ${\text{SR}}_{\text{baseline}}$ could be derived $n$ (${\text{SR}}_{\text{baseline}}$)]. The percentages in the top refer to all data with $\text{TR}/\text{SR}>10$ (diamond symbols); the percentages in the bottom refer to all data with $\text{TR}/\text{SR}\le 10$ (circle symbols).

The other 50% of the cytotoxic chemicals triggered cytotoxicity at lower concentrations than predicted by the baseline QSARs. Interestingly, chemicals triggered cytotoxicity at lower concentrations than their predicted ${\text{IC}}_{10,\text{baseline}}$, especially in ARE-BLA and AhR-CALUX, with 55% and 56% of cytotoxicity data exceeding the $\text{TR}=10$ threshold.

In addition, 5% to 25% of the chemicals with experimental ${\text{IC}}_{10}$ and matching ${\text{IC}}_{10,\text{baseline}}$ had $\text{TR}<1$ (Table 2). These numbers reflect the uncertainty and variability of the assay results because in theory a true baseline toxicant has $\text{TR}=1$. Deviation to higher TR can be related to specific cytotoxicity, but those below 1 must reflect uncertainty as well as measurement artifacts.

${\text{SR}}_{\text{cytotoxicity}}$ were $<10$ for a large proportion of the chemicals, for which both, ${\text{IC}}_{10}$ and ${\text{EC}}_{10}$ were reported (Figure 4B) and, apart from ARE-BLA, the majority of those were even ${\text{SR}}_{\text{cytotoxicity}}<1$ (Table 3). Taking all assays together, only 9% of the active chemicals had a ${\text{SR}}_{\text{cytotoxicity}}>10$ and were classified as specifically active. The highest proportion of specifically active chemicals was found for AR-BLA (14%), with another 21% in the range of $1\le {\text{SR}}_{\text{cytotoxicity}}\le 10$, which still means that 65% of all chemicals reported as activating the androgen receptor triggered cytotoxicity at concentrations similar to that of reporter gene activation. For GR-BLA, only 2 of the 121 active chemicals exceeded the range of cytotoxicity by a factor of 10. In $\text{PPAR}\mathrm{\gamma }\text{-BLA}$, only troglitazone (${\text{SR}}_{\text{cytotoxicity}}=\mathrm{1,742}$) and in GR-BLA ciclesonide (${\text{SR}}_{\text{cytotoxicity}}=\mathrm{3,390}$) were highly specific, whereas the other chemicals were either of low specificity (16% and 12%, respectively, with $1<{\text{SR}}_{\text{cytotoxicity}}<10$) or did not exceed ${\text{SR}}_{\text{cytotoxicity}}$ of 1. Sixteen (8%) and three (1.8%) chemicals were highly specific in the AR-BLA and $\text{ER}\mathrm{\alpha }\text{-BLA}$ assays, respectively, with ${\text{SR}}_{\text{cytotoxicity}}>\mathrm{1,000}$. All were well-known agonists of these hormone receptors. Contrarily, no chemicals exceeded a ${\text{SR}}_{\text{cytotoxicity}}$ of 1,000 in the ARE-BLA and AhR-CALUX assays.

The ${\text{SR}}_{\text{baseline}}$ analysis indicated that a larger proportion of the evaluated Tox21 chemicals were specifically active in the reporter gene assays (Table 3 and Figure 4C). Taking all chemicals together, $\sim 60\%$ of the active chemicals in the assays were specific with ${\text{SR}}_{\text{baseline}}>10$ while 23% to 38% fell into the $1\le {\text{SR}}_{\text{baseline}}\le 10$ range of ${\text{IC}}_{10,\text{baseline}}$ and were classified as moderately specific, with fewer than 25% having ${\text{SR}}_{\text{baseline}}<1$. As for the ${\text{SR}}_{\text{cytotoxicity}}$ analysis, more chemicals were highly specific in the hormone receptor assays AR-BLA and $\text{ER}\mathrm{\alpha }\text{-BLA}$, in a few cases even exceeding an ${\text{SR}}_{\text{baseline}}$ of $>{10}^6$. Estriol, a metabolite of estradiol and estrone and known to be a good ER-ligand, had an ${\text{SR}}_{\text{baseline}}$ of 45,000 in $\text{ER}\mathrm{\alpha }\text{-BLA}$ and fluorometholone, a synthetic glucocorticoid, had an ${\text{SR}}_{\text{baseline}}>106$ in AR-BLA. For ARE-BLA and AhR-CALUX, the range of specific chemicals were narrow, with only very few chemicals exceeding an ${\text{SR}}_{\text{baseline}}$ of ${10}^4$.

Discussion

Specific Toxicity

We found that 32% to 56% of all chemicals in Tox21 had $\text{TR}>10$ and could therefore be classified as specifically acting. This is a higher proportion than in the experimental data set of 15 compounds, where most chemicals were classified as baseline toxicants, with 7 of them specifically selected because they are known to be baseline toxicants (Vaes et al. 1998). However, when comparing with classification of a large set of ecotoxicity data using different mode-of-action classification tools (Kienzler et al. 2017), depending on the method, 27% to 69% of chemicals were assigned as baseline toxicants. The remainder are not necessarily expected to have a $\text{TR}>10$ but also included chemicals that could not be assigned to a mode-of-action class. There exist no such estimates for cytotoxicity, but the ranges in the present analysis seem realistic in comparison with the analysis of ecotoxicity data (Kienzler et al. 2017).

Identification of the Cytotoxicity Burst

Some of the reporter gene assays could be activated by the known baseline toxicants, but, as expected from baseline toxicants, the ${\text{SR}}_{\text{cytotoxicity}}$ ranged within an order of magnitude around one (Figure 2B), with the only exception being 2,4,5-Trichloroaniline. Thus, the proposed classification ($1\le \text{SR}<10$ moderately specific, $10\le \text{SR}<100$ specific, and $100\le \text{SR}$ highly specific) appeared reasonable because the confirmed baseline toxicants fell into the range of $\text{SR}<10$. Of course, it is possible that the initial classification of baseline toxicants according to Verhaar et al. (Verhaar et al. 1992, 2000) was faulty, but with respect to cytotoxicity they were all at $\text{TR}<10$. Furthermore, these chemicals are not reactive and too small to bind to hormone receptors.

The observed reporter gene activation by baseline toxicants supports our earlier proposed data evaluation strategy (Escher et al. 2018): All chemical concentrations $>{\text{IC}}_{10}$ should be excluded from analysis of reporter gene activation to ensure that the derivation of effect concentrations is not influenced by the cytotoxicity burst ($\text{SR}<1$). Considering that the present analysis demonstrated that the cytotoxicity burst is a critical confounding factor when using in vitro reporter gene assays, we suggest an even more cautious approach when using reporter gene assay data for risk assessment: Any chemical with $1<{\text{SR}}_{\text{cytotoxicity}}<10$ should be further scrutinized. A particularly striking example is butoxyethanol, with a ${\text{SR}}_{\text{cytotoxicity}}$ of 0.31 in AREc32, which is even below SR 1 and would not have been identified as an active chemical if the data were analyzed only up to ${\text{IC}}_{10}$ however, if all activity data were considered, it would have been mistaken as specifically acting (false positive).

The cytotoxicity burst was even more pronounced for the selected environmental chemicals where an even larger proportion of chemicals triggered the cytotoxicity burst (${\text{SR}}_{\text{cytotoxicity}}<1$) or were moderately specific ($1<{\text{SR}}_{\text{cytotoxicity}}<10$) (Figure 3B).

Not all literature data are subject to such a rigorous data treatment; hence, it is very likely that some of the reported specific activity in the literature are false positives caused by the cytotoxicity burst. Cytotoxicity does not equate to baseline toxicity, but we can use baseline toxicity as a reference and ${\text{SR}}_{\text{baseline}}$ (Equation 7) can be a proxy for identification of a potential cytotoxicity burst in absence of ${\text{SR}}_{\text{cytotoxicity}}$. ${\text{SR}}_{\text{baseline}}$ may even serve better to identify specific effects in cases when a different specific mechanism than the receptor/pathway targeted by the reporter gene assay had led to cell death, i.e., for chemicals with $\text{TR}\ge 10$. We have not included p53-BLA in our analysis because it is an example of a bioassay where it is hard to derive an ${\text{EC}}_{10}$ because, in almost all cases, the activation of the p53 pathway is close to cell death, and therefore cytotoxicity masks activation, which is hard to differentiate from the cytotoxicity burst.

Comparison of Experimental EC and Tox21 Database

To make our own data comparable with the Tox21 database, we reevaluated the concentration–response curves and derived ${\text{EC}}_{10}$ and ${\text{IC}}_{10}$ values for Tox21 assays, which are typically reported as ACC (Filer et al. 2016; Judson et al. 2016). Neither ACC nor the previously used AC10 was suitable for the TR and SR analyses because the baseline toxicity QSARs are available only for ${\text{IC}}_{10}$, and the ACC is not associated to a fixed effect level but rather a measure of the lowest concentration that is statistically robust to show an effect. In this respect, ACC is rather similar to the lowest observed effect concentration LOEC. The ${\text{EC}}_{10}$ was derived from the absolute 10% effect in relation to the maximum effect triggered by a potent reference compound, and these types of raw data could be extracted from the Tox21 Concentration–Response Browser as outlined in “Materials and Methods.”

Our goal was not to evaluate the uncertainty of Tox21 data or to propose an alternative effect measure but to extract robust data for the TR and SR analyses. Therefore, we evaluated all data sets for one compound together, even if they stemmed from different laboratories. If they differed too much (relative standard errors $>50\%$), the entire data set was excluded and classified as “inconclusive,” but for most chemicals, there was high consistency among labs, and the joint evaluation of all data sets yielded robust and representative ${\text{EC}}_{10}$ values, as is demonstrated for the example of $\text{ER}\mathrm{\alpha }\text{-BLA}$ in Figure S20.

The ${\text{EC}}_{10}$ and ${\text{IC}}_{10}$ values agreed well between our measurements and the Tox21 database for ARE-BLA (Figure S22A), AhR-CALUX (Figure S22B), and $\text{PPAR}\mathrm{\gamma }\text{-BLA}$ (Figure S22C). The only exception was $\text{ER}\mathrm{\alpha }\text{-BLA}$, but here the SR were so low that the evaluated $\text{ER}\mathrm{\alpha }\text{-receptor}$ is clearly not of relevance for those chemicals that showed no agreement, such as Quinoxyfen with a ${\text{SR}}_{\text{cytotoxicity}}$ of 5. The good agreement of most data (Figure S23) indicated that, despite different assay protocols and different plate formats, the responses are fairly robust. In the present study we worked in 384-well plates with larger medium volumes ($40\mathrm{\mu }\mathrm{L}$) in comparison with the Tox21 reporter gene assays that were performed in 1,536-well plates with $4–6\mathrm{\mu }\mathrm{L}$ of medium. The application of 384-well plates and larger medium volumes typically reduced the chemical losses from the system, in comparison with 1,536-well plates, due to higher storage capacity of the medium proteins and slower uptake in well-plate plastic (Fischer et al. 2018), but the difference appeared to be rather small.

Analysis of Tox21 Database

Approximately 50% of the chemicals triggered cytotoxicity at 10 times lower concentrations than their predicted ${\text{IC}}_{10,\text{baseline}}$. This finding means that a specific effect or reactive toxicity led to premature cell death. The mechanisms leading to cytotoxicity must not necessarily be the same as the receptor/pathway associated with the reporter gene. A larger proportion of chemicals were specifically toxic ($\text{TR}\ge 10$) for ARE-BLA (55%) and AhR-CALUX (56%) than for $\text{PPAR}\mathrm{\gamma }\text{-BLA}$ (45%) and the hormone receptors (32%–41%). Well-known toxicants stood out, such as digitoxin ($\text{TR}=\mathrm{43,000}$ in ARE-BLA, no data in AhR-CALUX, 3,400 in $\text{PPAR}\mathrm{\gamma }\text{-BLA}$, 115 in AR-BLA, 99 in $\text{ER}\mathrm{\alpha }\text{-BLA}$, and 2,200 in GR-BLA), which is cardiotoxic and highly cytotoxic with the proposed mechanisms related to oxidative stress response and interferon-related pathways, which explains the much higher TR in AREc32 (Prassas et al. 2011). The next highest TR chemical was the reactive dye 1,8-Dihydroxy-4,5-dinitroanthraquinone with a TR of 4,600 in ARE-BLA and again much more moderate TRs in the other assays.

The cytotoxicity burst phenomenon may have led to a significant number of false positives in all evaluated Tox21 reporter gene assays. The ${\text{SR}}_{\text{cytotoxicity}}$ analysis clearly showed that a large proportion (86% to 97%) of the measured ${\text{EC}}_{10}$ were above cytotoxic concentrations (${\text{IC}}_{10}$) and thus impacted by the cytotoxicity burst. Only 1.7%–14.2% of the chemicals classified as specifically active with another large fraction (12%–54%) in the range of $1\le \text{SR}<10$. Given the variability of ${\text{SR}}_{\text{cytotoxicity}}$ in our own experimental data of confirmed baseline toxicants (Figure 2B), it is likely that not only chemicals of low specificity were included in the range of $1\le \text{SR}<10$ but also some with effects triggered by the cytotoxicity burst.

Due to practical reasons, all Tox21 chemicals were tested in the same concentration range (typically between $\sim 0.001$ and $\sim 100\mathrm{\mu }\mathrm{M}$), which in turn resulted in only a small proportion of the chemicals reaching ${\text{IC}}_{10}$ concentrations. Thus, the total number of chemicals with both experimental ${\text{IC}}_{10}$ and ${\text{EC}}_{10}$ was reduced. A larger proportion of chemicals were classified as specific in the ${\text{SR}}_{\text{baseline}}$ analysis than ${\text{SR}}_{\text{cytotoxicity}}$, because this analysis could be performed for more chemicals in the absence of experimental cytotoxicity data (up to three times more data points included) but also because chemicals with a cytotoxicity triggered by a different pathway had smaller ${\text{SR}}_{\text{cytotoxicity}}$. A large proportion of the active chemicals ($\sim 60\%$, exception $\text{ER}\mathrm{\alpha }\text{-BLA}$) were likely truly specific (Fig. 4C). Nevertheless, for 3% to 25% of the active chemicals with an ${\text{EC}}_{10}$, the cytotoxicity burst presumably led to a false positive ${\text{EC}}_{10}$ and a further 23%–30% was only moderately specific, some of which, given the experimental variability could also have been impacted by the cytotoxicity burst. For $\text{ER}\mathrm{\alpha }\text{-BLA}$, 25% of the ${\text{EC}}_{10}$ fell below ${\text{IC}}_{10,\text{baseline}}$ (Fig. 4C); thus, the cytotoxicity burst was particularly impeding the outcome of this assay.

Conclusion

Our findings clearly confirm previous reports of the cytotoxicity burst phenomenon and show a systematic way to identify and interpret the cytotoxicity burst. For chemicals with a high TR (i.e., those that are more toxic than baseline toxicity), one needs to be especially cautious because the specific effect may in some cases also cause enhanced cytotoxicity, e.g., in the two assays of oxidative stress response, AREc32 and ARE-BLA. For these assays, the activation of defense mechanisms might go hand-in-hand with the higher cytotoxicity caused by specific effects. In contrast, the activation of the hormone receptors $\text{ER}\mathrm{\alpha }\text{-BLA}$, AR-BLA, PR-BLA, and GR-BLA is not likely to be associated with increased cytotoxicity, so any activation of hormone receptors at cytotoxic concentration is likely to be a result of the cytotoxicity burst phenomenon.

For routine applications of cell-based bioassays, we recommend performing the evaluation of activity data only at concentrations smaller than ${\text{IC}}_{10}$ to avoid false-positive responses ($\text{SR}<1$). Likewise, measures can be taken to avoid false negatives with respect to highest tested concentration. The baseline toxicity QSARs can help to predict, at which concentrations we expect the minimal toxicity (baseline toxicity). We previously suggested to dose chemicals up to their solubility limit in medium or to their predicted ${\text{IC}}_{10,\text{baseline}}$, whichever is lower (Fischer et al. 2019). Although it is challenging for HTS experiments to adjust each concentration, it might be possible to group chemicals according to their physicochemical properties (hydrophobicity, speciation, etc.) and test them groupwise at fixed concentrations depending on their expected ${\text{IC}}_{10}\text{-range}$. Both medium solubility limit and ${\text{IC}}_{10,\text{baseline}}$ can be predicted for neutral chemicals by the log $K_{\text{ow}}$ of the chemicals as sole descriptor (Escher et al. 2019; Fischer et al. 2019). For ionizable chemicals, the speciation additionally needs to be included. By ensuring that the tested concentration range reached cytotoxicity, one can calculate both the ${\text{SR}}_{\text{cytotoxicity}}$ and the ${\text{SR}}_{\text{baseline}}$ as a diagnostic tool for evaluation of the specificity of a chemical. This approach will also help to reduce the uncertainty of HTS data. Watt and Judson (2018) identified data with high variability by an uncertainty analysis of the Tox21 data. It is conceivable that the elimination of data triggered by nonspecific effects will decrease uncertainty.

Although we have validated the classification method with a small selection of reporter gene assays only, the approach is easily transferable to other reporter gene assays because the baseline toxicity QSARs are based on constant critical membrane concentrations and can be predicted for any cell-based bioassay, provided that lipid and protein composition of cells and medium are known (Escher et al. 2019).

For application of in vitro test methods in risk assessment, it is important that mode-of-action specific QIVIVE are performed only with data that are not compromised by the cytotoxicity burst.