Decision-Making with New Approach Methodologies: Time to Replace Default Uncertainty Factors with Data

Dourson et al. (2022) provide an informative summary of the debate on “The Future of Uncertainty Factors with in Vitro Studies Using Human Cells” that was held at the 2021 Society of Toxicology Annual Meeting. We are pleased to see a robust discourse regarding the use of the data from new approach methodologies pivoting from the questions of “if?” to considerations of “how?” Indeed, collaborations and discussions between the regulators and researchers are needed to make use of in vitro bioactivity data and pivot from priority setting of chemical inventories to risk-based decisions (Dourson et al., 2022). We believe that the time is right to have substantive discussions about the utility of new approach methodologies data in the current general framework for deriving safe exposure levels. We disagree, however, with the apparent proposal to invent new default uncertainty factors for the data from new approach methodologies as a path forward. We reason that new approach methodologies offer the opportunity to derive chemical-specific adjustment factors, not to move “back to the future” of over-reliance on default assumptions. In this respect, we offer three points of criticism of Dourson et al. (2022).

First, we note that Dourson et al. (2022) may be inadvertently contributing to the long-standing conflation of uncertainty and variability. As discussed extensively in the report by the International Programme on Chemical Safety (WHO/IPCS, 2018), the usual “uncertainty” factors are blends of three elements: (1) adjustments due to characteristics of the study population or design that differ from the target human population; (2) accounting for variability due to heterogeneity in the human population; and (3) accounting for uncertainty in (1) or (2) due to incomplete data. Also, as shown in their Figures 2 and 3, Dourson et al. (2022) may be eroding the distinction between individual dose-response (in which the magnitude or severity of effect changes with dose) and population dose-response (in which the fraction of the population experiencing a particular effect changes with dose). The importance of this distinction was reiterated in the Science and Decisions report (National Research Council, 2009) and by the International Programme on Chemical Safety (WHO/IPCS, 2018). Therefore, we reason that the framework proposed by Dourson et al. (2022), which depicts all dose-response functions at the population level, is conceptually flawed. This distinction is especially important for the data from new approach methodologies based on human cells because they can measure responses at the individual level using a single cell line or at the population level using cells from multiple individuals.

Second, with respect to the issue of the size of the default uncertainty factor for human variability, we note that several published datasets on quantifying inter-individual variability using in vitro data in different human cells are already available on hundreds of chemicals (Abdo et al., 2015; Blanchette et al., 2019; Burnett et al., 2019). Other in vitro datasets for population-wide human and animal models have been published. In addition, Bayesian approaches to estimating population variability from in vitro data have also been proposed (Chiu et al., 2017). Collectively, these in vitro data are largely consistent with the more limited available human in vivo data and suggest that although 10-fold overall (or 3-fold for toxicodynamics alone) is a “reasonable” or “typical” value for this factor, for numerous specific chemicals and endpoints/phenotypes the experimental data from population-based human in vitro models suggest larger factors may be necessary to ensure the protection of 95%–99% of the population.

Finally, we found the examples of tissue chip-derived data provided in support of “achieving greater certainty in risk assessment” with “alternative or complementary approaches” to be informative. However, these examples appear to be largely unconnected to the topic of the debate—the need for, and derivation of, the uncertainty factors. Given the low throughput of current tissue chip models, it is unlikely that this new approach methodology will be a feasible approach to either characterize population variability, or to inform derivation of other uncertainty factors. Moreover, examples of in vitro-to-in vivo extrapolation using tissue chip data are still few, and thus their potential use in deriving points of departure remains to be determined. By contrast, numerous case studies using population-based human cell types (Abdo et al., 2015; Blanchette et al., 2019; Burnett et al., 2019; Chiu et al., 2017) have demonstrated a sensible workflow for deriving toxicity values. First, start with in vitro points of departure for toxicodynamically relevant endpoints in human cells, such as induced pluripotent stem cell-derived models obtained from multiple individuals. Then, perform a population-based reverse toxicokinetics in vitro-to-in vivo extrapolation, implemented within a Bayesian framework to separately account for uncertainty and variability. Moreover, advances in machine learning offer opportunities to develop new approach methodologies that make chemical-specific predictions (with uncertainty estimates) within such a workflow even when there are in vitro data gaps. A framework based on new approach methodologies, one that uses chemical-specific information from in vitro and other data streams at each step of the toxicity value derivation process, would render the use of “default” uncertainty factors unnecessary.

In conclusion, we caution against suggestions made by Dourson et al. (2022) that new default uncertainty factors are the path to acceptance of the data from new approach methodologies in regulatory decision-making. Rather, such methodologies actually offer an opportunity to develop chemical-specific extrapolation factors that can be applied both to traditional experimental animal data, and to these data themselves. Moreover, the appropriate application of new approach methodologies will require a clear conceptual understanding that acknowledges key distinctions such as uncertainty versus variability, and individual- versus population-level responses. Thus, while we strongly believe that new approach methodologies-based decisions will become widely acceptable in the near future because they can now be used to make certain decisions without supporting human or experimental animal data (Samet et al., 2020), a rigorous conceptual framework that includes balanced integration across evidence streams shall be the path forward to increasing confidence in the implementation of the Toxicity Testing in the 21st Century vision.

DECLARATION OF CONFLICTING INTERESTS

The authors served on or chaired a number of World Health Organization/International Agency for Research on Cancer Monographs working groups. W.A.C. was a lead author for the WHO/IPCS Guidance Document on Evaluating and Expressing Uncertainty in Hazard Characterization.