Systematic Update to the Mammalian Relative Potency Estimate Database and Development of Best Estimate Toxic Equivalency Factors for Dioxin-like Compounds

Abstract

The World Health Organization (WHO) assesses potential health risks of dioxin-like compounds using Toxic Equivalency Factors (TEFs). This study systematically updated the relative potency (REP) database underlying the 2005 WHO TEFs and applied advanced methods for quantitative integration of study quality and dose-response. Data obtained from fifty-one publications more than doubled the size of the previous REP database (~1300 datasets). REP quality and relevance for these data was assessed via application of a consensus-based weighting framework. Using Bayesian dose-response modeling, available data were modeled to produce standardized dose/concentration-response Hill curves. Study quality and REP data were synthesized via Bayesian meta-analysis to integrate dose/concentration-response data, author-calculated REPs and benchmark ratios. The output is a prediction of the most likely relationship between each congener and its reference as model-predicted TEF uncertainty distributions, or the ‘best estimate TEF’ (BE-TEF). The resulting weighted BE-TEFs were similar to the 2005 TEFs, though provide more information to inform selection of TEF values as well as to provide risk assessors and managers with information needed to quantitatively characterize uncertainty around TEF values. Collectively, these efforts produce an updated REP database and an objective, reproducible approach to support development of TEF values based on all available data.

Introduction

Polychlorinated dibenzo-p-dioxins (PCDDs), polychlorinated dibenzofurans (PCDFs), and dioxin-like polychlorinated biphenyls (DL-PCBs) are persistent and bioaccumulative compounds which are relatively ubiquitous in the diet and environment. A subset of these – seventeen laterally-substituted PCDD/F congeners and 12 non-/mono-ortho chlorine-substituted PCBs commonly referred to as dioxin-like compounds (DLCs) – are included in the toxic equivalency factor (TEF) methodology (Haws et al., 2006; Van den Berg et al., 2006). The TEF methodology is one of the most well-established approaches for assessing potential risk associated with a mixture, first proposed by Ahlborg et al. (1994), and utilized by authoritative entities worldwide to assess potential human health risks associated with exposure to these compounds via the environment (e.g., soil contamination) and the diet (e.g., amounts in foodstuffs).

The TEF methodology is based on the understanding of a common molecular initiating event of binding and activating the aryl hydrocarbon receptor, and subsequent toxicities due to sustained transactivation. Effects observed, primarily in experimental animal studies, are broad, ranging across apical outcomes including immunosuppression, reproductive toxicity, developmental toxicity, hepatotoxicity, and enzyme induction (European Food Safety Authority (EFSA) Panel on Contaminants in the Food Chain, 2018; Haws et al., 2006). In the TEF approach, the potency of each individual congener is expressed relative to the potency of a reference chemical, typically 2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD). The 2005 TEF values are point estimates assigned by a 2005 WHO expert panel based on scientific judgment of a database of relative potency (REP) estimates derived from a multitude of studies conducted over the past several decades (Van den Berg et al., 2006). The WHO 2005 Expert Panel acknowledged that deriving TEFs based on point estimates and expert judgement may introduce potential bias while also not considering the range of REPs observed in the literature (Van den Berg et al., 2006).

In selecting the 2005 TEF values, the distributions of underlying REP data were considered; these data were based on that described in the REP₂₀₀₄ database, which was comprised of 634 mammalian REPs derived from in vivo and in vitro studies, and included REPs curated from literature published through 2004 (Haws et al., 2006). However, the distributions were not directly used to select a TEF value (e.g., pick a percentile of the distribution, such as the 75%th) because the underlying data: (a) represented studies with varying reliability and relevance; and (b) were heterogenous with respect to how the actual REP values were derived – ranging from sophisticated dose-response methods to simple visual extrapolation from graphs. To address the challenges presented by the underlying data in the REP database, Wikoff et al., (Accepted, 2023) proposed a consensus-based weighting framework to address aspects of REP study reliability and relevance using machine learning, and Ring et al. (2023) proposed an approach to address heterogeneity in dose-response modeling using Bayesian dose-response fitting and meta-regression. Together, these techniques predict the most likely relationship between each congener and its reference and allow for derivation of model predicted TEF uncertainty distributions. The most likely relationship is referred to as the ‘best estimate TEF’ or BE-TEF.

Given the solutions to REP weighting and dose-response modeling, the objective herein was to systematically update the REP database (recognizing much literature has been published since 2004), and apply the techniques proposed by Wikoff et al. (Accepted, 2023) and Ring et al. (2023) to the updated database to produce BE-TEFs based on weighted synthesis of available REP data. Collectively, these efforts produce an updated REP database and an objective, transparent, and reproducible approach to support development of TEF values, as well as to provide risk assessors and risk managers with information needed to quantitatively characterize uncertainty around TEF values and their impact on decision making.

Methods

The methods used involve two parts: first, a systematic update of the REP database, and second, application of the Bayesian meta-analysis workflow described by Ring et al. (2023) to the updated database. As described below, these collaborative efforts also involved several levels of peer-review of both parts prior to submission of the work for publication. The peer review also included in depth review and discussion of the database and modeling by the Expert Panel at the 2022 WHO conference on TEFs in their deliberation of use of the information reported herein. A workshop report regarding the use of this data is accompanying this publication (DeVito et al., Submitted 2023).

Systematic Update to the REP Database

Problem Formulation

As a standard component of systematic review, the update began with a formal problem formulation stage that involved four steps: evaluation of the methods underlying REP₂₀₀₄, development of an a priori protocol (i.e., prior to initiating research efforts), consideration of the utility to stakeholders, and piloting exercises. These efforts provided a foundation for implementing a systematic and transparent workflow for the development of REP₂₀₂₁.

The framework for this database update was based on previous updates of the REP database made by WHO expert panels as reported Haws et al. (2006). Additionally, authors of the REP₂₀₀₄ database (Haws et al., 2006) were consulted about specific aspects of the database development such as determination of REP eligibility criteria, data sources, and post-database development processing (e.g., removing repetitive endpoints). Based on this scoping, the research protocol was drafted to document problem formulation exercises and detail the anticipated methods for each stage of the update (see Supplemental Material).

Following principles of systematic review, inclusion and exclusion criteria to be applied to the literature review were defined. These aligned with criteria reported in Haws et al. (2006) and were explicitly documented in the protocol. Publications were included, and study data extracted for REP development, based on the following eligibility criteria:

• exposure to both a reference compound (i.e., TCDD or PCB126) and at least one DLC at more than one dose or concentration; studies evaluating mixtures were excluded

• both a reference compound and at least one DLC elicited a statistically significant response compared to an untreated or vehicle control

• the study was performed in non-marine mammals, a mammalian cell line, or cells transfected with a relevant sequence (e.g., DR CALUX)

• publication reported original, peer-reviewed study data; case reports, conference abstracts, and reviews were excluded

Following the development of the protocol, project-specific literature screening and data collection forms were developed via DistillerSR (DistillerSR, Inc., Canada) (SF, DW). Piloting exercises based on the methods outlined in the protocol were implemented and eligibility criteria and form fields were adjusted as warranted based on data types, findings, and decisions made by the piloting team (SF, KF, DW, LH).

Literature Identification

A comprehensive literature search was developed by an Information Specialist (SF), with input by topic experts (DW, LH). The complete search strategy is available in the protocol. REP₂₀₀₄ was relied on for related publications published on or before December 31, 2004. To identify studies from January 1, 2005 to present, date limited searches were conducted in PubMed and Embase on May 29, 2018; a second search update was performed on November 1, 2021. Publications were restricted to those reported or available in English.

Citations were deduplicated using a reference management software and uploaded to DistillerSR for screening based on the inclusion criteria outlined in the protocol. At the title and abstract review stage, two reviewers screened each citation for relevance. If there was a conflict between reviewers, or inclusion was uncertain, a third reviewer was consulted for input. Studies meeting inclusion criteria according to the title and abstract were advanced to full-text screening. Full-text review was performed by a single reviewer based on the same eligibility criteria. If the study was still relevant based on a review of the full text, it was advanced to data extraction which occurred in tandem with full text screening to increase efficiency and reduce unnecessary repetition. Publications and REPs from the REP₂₀₀₄ database were assumed to meet the inclusion criteria and were not screened at the title/abstract or full-text levels. These data were automatically advanced to data extraction with the eligible updated literature.

Data Extraction

Data extraction was performed in DistillerSR via project-specific data collection forms. As previously stated, these forms were piloted and revised during the problem formulation phase. The following fields were collected for each eligible study: citation information, study details (e.g., objective), experimental details (e.g., compounds, purity, animal model, endpoints) and outcome data. When available, reported dose-/concentration-response data, benchmark values, and REP calculations for each endpoint with a significant response for both a congener and a reference compound was collected. In the circumstance that data was reported graphically (i.e., in figures only), reviewers utilized GraphClick (Arizona Software) to estimate data points; for consistency, this was also executed for the references included in the REP₂₀₀₄ database when graphical dose-/concentration-response data was available. Graphical data extraction was noted when these methods were applied.

Information related to the consensus-based weighting criteria was also recorded, including fields related to the following quality-related attributes: study type, experimental model, details about experimental pharmacokinetics, author-derived REP derivation method, REP derivation quality (i.e., maximum response, statistical reliability), and biological relevance. Descriptions of each of these fields and their relevance to human health risk assessment are reported in depth by Wikoff et al. (Accepted, 2023).

Data Hierarchy and Input Selection

In congruence with the methods of the development of REP₂₀₀₄, database and associated review by the 2005 WHO expert panel, extracted datasets were reviewed to remove repetitive endpoints. Specifically, responses from different assays used to measure the same biological response (e.g., total thyroxine and free thyroxine) were reviewed for removal. REPs were considered for removal based on repetitive endpoints if they both represented measures of the same biological response. Similarly, if two REPs measured both an upstream and downstream effect in the same system, the most downstream effect was retained. Table 1 displays examples of repetitive endpoints considered for removal; representative measures were carried forward from Haws et al. (2006) and confirmed by topic experts (DW, LH, MD, NW).

Table 1.

Examples of Repetitive Endpoints Removed from REP₂₀₂₁

Repetitive Endpoint	Most representative measure
Plaque-forming cells (PFCs)/spleen and PFCs/106 viable cells	PFCs/106 viable cells
Aryl hydrocarbon hydroxylase (AHH) and ethoxyresorufin-o-deethylase (EROD)	EROD
Benzo-a-pyrene (B[a]P) hydroxylase and 4-chlorobiphenyl hydroxylase	B(a)P hydroxylase
Cleft palate/fetus (%) and cleft palate/litter (%)	Cleft palate/litter (%)
Promotion index, hepatic foci, and GGT+ foci	GGT+ foci
Thyroxin glucuronidation (T4UGT) and uridine diphosphate glucuronyltransferase 1 family, polypeptide A1 (UGT1A1)	UGT1A1
Total thyroxine (TT4) and free thyroxine (Ft4)	TT4
Fixed ratio (FR) response rate and FR reinforcement rate	FR response rate
Ova/rat and ova/ovulating rat	Ova/rat
Absolute and relative measures of organ weight	Relative weight
Cytochrome P4501A1 (CYP1A1) expression/protein measures and Ethoxyresorufin O-deethylase (EROD) induction	EROD induction
Cytochrome P4501A2 (CYP1A2) expression/protein measures and Phenacetin-O-deethylase (POD) induction	POD induction
Hormone mRNA expression and hormone concentration	Hormone concentration

Following deduplication of endpoints, a data selection hierarchy was applied:

• If dose/concentration response data was available, these data were retained, or

• If these data were not available, author-calculated REPs were retained, and

• If neither of these were available, benchmark ratios were calculated (e.g., REP_EC50 = EC_{50 ref comp}/EC_{50 congener})

This curation was the final step in developing the model inputs for the application of the combined Bayesian dose-response and meta-analysis workflow.

Model Input Quality Control

Quality control (QC) of the REP₂₀₂₁ model inputs consisted of an internal QC (SF, AB, KF) and an external peer review effort (KvE, EAF). The internal QC was performed for all extracted data and hierarchy selections of all congeners; changes were recorded in DistillerSR. Next, an external peer review was performed on select congeners covering approximately 50% of the newly added datasets: 1,2,3,4,6,7,8-HpCDD; 2,3,4,7,8-PeCDF; 1,2,3,4,7,8-HxCDF; 1,2,3,6,7,8-HxCDF; PCB-126; PCB-169. This peer review focused on studies not included in REP₂₀₀₄, as REP₂₀₀₄ was peer reviewed extensively at the 2005 WHO-TEF re-evaluation meeting. The peer review herein included a QC of each REP₂₀₂₁ input for bibliographic information, experimental details, and dose/concentration response data. Finally, a review of the database was performed by an Expert Panel during the 2022 WHO Consultation on TEFs (DeVito et al., in process). This review included an assessment of the biological relevance of endpoints compiled in the database.

Derivation of REP₂₀₂₁ Best-Estimate TEFs

The database inputs described above from REP₂₀₂₁ were applied to the Best-Estimate TEF (BE-TEF) workflow, established by Ring et al. (2023) to derive BE-TEF values. In the first stage of the BE-TEF workflow, the REP datasets (both dose/concentration response and author-calculated) were assigned study quality categories based on their associated quality-related study attributes, using a machine-learning model developed by Wikoff et al. (Accepted, 2023). Briefly, this fit-for-purpose machine-learning model was trained to reproduce the quality categorizations made by the WHO 2005 expert panel using expert judgment. The response variable was categorical quality ratings of REP₂₀₀₄ studies made by the WHO 2005 expert panel; the predictor variables were the associated quality-related attributes for each REP₂₀₀₄ study (Wikoff et al., Accepted, 2023). The machine-learning model predicted the probability that the WHO 2005 expert panel would assign each REP study to each of the possible quality categories. The trained machine-learning model was then provided the reviewer-assigned quality-related attributes for each REP dataset in REP₂₀₂₁ and predicted the corresponding quality category probabilities. This workflow was run both with and without this step to produce both weighted and unweighted results.

In the second stage of the BE-TEF workflow, Bayesian dose-response modeling was performed for all available dose/concentration response data. Briefly, dose/concentration response data were harmonized on an individual dataset basis to ensure dose and response metrics were consistent within them for the reference compound (i.e., TCDD or PCB126) and the congener(s). A dose/concentration response dataset was considered incomplete if it did not report both sample size and sample variance for all observations, or if it had fewer than 3 observations for either the reference compound or the congener. When this occurred, these datasets were removed from the final database inputs and replaced with the author-calculated REP or a benchmark ratio, if available. Dose/concentration response datasets were then fit using a Hill model, whose parameters were estimated by Bayesian inference (see Ring et al. (2023) for details), producing samples from the uncertainty distribution (i.e., the Bayesian posterior distribution) of the estimated Hill model parameters for each dataset (Supplemental Table 1). The resulting estimated dose/concentration response Hill curve for each dataset was standardized relative to its corresponding reference dataset, such that response was re-scaled as a fraction of the reference maximal response, and dose/concentration was re-scaled as a fraction or multiple of the reference ED50/EC50. The Hill model parameters for these standardized dose/concentration response curves (sDR curves) were carried forward to the next stage.

In the third stage of the BE-TEF workflow, Bayesian meta-analysis was used to synthesize all available REP and study quality data to estimate a single “average” sDR curve for each congener (Ring et al., 2023). Briefly, the meta-analysis model assumes that individual REP studies for a congener each measure study-specific sDR curves that are all distributed around one central congener-specific sDR curve. The study-specific sDR curves can therefore be used to infer the location and shape of the congener-specific sDR curve. Study-specific sDR curves are measured directly by the Hill model fits to dose/concentration response data (from the previous stage), and indirectly by author-calculated REPs and benchmark ratios. The meta-analysis therefore integrates dose/concentration response data with author-calculated REPs and benchmark ratios. The meta-analysis model also integrates the consensus weighting framework for study quality: it assumes that higher-quality studies are clustered more tightly around the congener-specific sDR curve (less heterogeneity), and lower-quality studies are scattered more broadly around it (more heterogeneity). In this way, data from higher-quality studies are weighted more heavily when inferring the congener-specific sDR curve. (The exact quantitative weighting factors — i.e., the heterogeneity for each study quality category — are estimated from the available data.) The Bayesian meta-analysis was run both with and without this weighting assumption, resulting in both weighted and unweighted results. The Bayesian meta-analysis produces samples from the uncertainty distribution (i.e., the Bayesian posterior distribution) of the Hill model parameters that describe the congener-specific sDR curves for each congener.

In the fourth stage of the BE-TEF workflow, the congener-specific sDR curves for each congener are compared to the TCDD-specific sDR curve to derive the BE-TEF value. Uncertainty in the estimated congener-specific sDR curves is propagated forward to also quantify uncertainty in the BE-TEF. Figure 1 provides an illustration of the BE-TEF derivation from the congener- and TCDD-specific sDR curves, as well as the uncertainty around the BE-TEF.

Figure 1.

Example visualization of a reference-congener standardized dose-response relationship (left) and its surrounding uncertainty (shading) from which the TEF uncertainty distribution (i.e., the range of TEFs derived due to uncertainty in the congener position at the chosen response level, represented by the solid and segmented line) can be derived (right). The right panel also provides a guide for the interpretation of the BE-TEF violin plots, and visualizes an example BE-TEF (diamond) that would be derived from the TEF uncertainty distribution.

In practice, the BE-TEF and its uncertainty are derived through the generation of a TEF uncertainty distribution for each congener from their respective parameter distributions describing their sDR curves; the mean value is then chosen as the BE-TEF. Here, this mean value represents the most probable ratio of standardized dose/concentrations for the congener and for TCDD that produces a response equal to 50% of TCDD maximal response. The underlying TEF uncertainty distribution that is generated does not use any samples of the Bayesian meta-analysis posterior distribution for which the Hill parameters would indicate that the congener maximal response does not reach the 50% response level. Should these samples exist, rather than being used for the TEF uncertainty distribution, they are applied to determine an estimate of the probability that the TEF is “non-quantifiable” (Prob NQ). Specifically, Prob NQ that is the probability that the maximal response of the congener-specific sDR curve does not reach 50% of TCDD maximal response (see Ring et al (2023) for details). These metrics provide additional insight into the underlying dose/concentration response relationship between each congener and TCDD.

Best-Estimate TEF Quality Control

The Best-Estimate TEF Workflow, like the database, was also subjected to extensive internal and external QC to ensure the input data was processed correctly, the models functioned as they should, and that the results returned by the workflow are accurate, consistent, and reproducible. Initially, an external QC was conducted in which the code for each stage of the workflow was reviewed by an independent 3^rd party (MW), and recommendations for changes and additions to the code were implemented. Next, for the internal QC, as this workflow relies heavily on Bayesian methods that use random sampling, the majority of the QC efforts in this phase focused on identifying and controlling all factors that could affect the reproducibility of outcomes from the model runs. Multiple runs of the workflow were conducted in which conditions were altered in either the dose/concentration response Modeling phase or the Meta-Analysis to produce model outputs that were then compared against the outputs of the workflow when run as intended (i.e. one complete run all the way through the workflow with all data included) to identify what conditions could result in output variability.

Results

Literature Identification and Data Extraction

The search strategy was implemented in two phases, with PubMed and Embase queries occurring on May 29, 2018, and again on November 1, 2021. As the REP₂₀₂₁ database is considered as a single entity, these two phases will be described as one. Following deduplication, 3687 unique citations were identified and screened by title and abstract for inclusion. Of these, 208 were reviewed at the full-text level. Fifty-one (51) met the defined eligibility criteria for data extraction and inclusion in the update (Figure 2).

Figure 2.

Literature identification flow chart of studies added to the REP₂₀₂₁ database

Characteristics of the REP₂₀₂₁ Database

The deduplication criteria and data selection hierarchy were applied to datasets extracted from each publication. Based on these criteria, REPs collected from the same publication were excluded due to repetitive endpoints or multiple data types existing for an individual endpoint (i.e., dose/concentration response data, author-calculated REPs, and benchmark data were reported for one measure). In these scenarios, a single representative REP was retained to be representative of the dataset.

Additional decisions were made by the 2022 WHO TEF Expert Panel to remove endpoints and specific REP datasets from REP₂₀₂₁ based on expert review of data. These included REP datasets relying on endpoints related to androgen receptor inhibition, estrogen receptor activation, cell proliferation, prostatic specific antigen, and cell viability. Based on these eliminations, twenty-two additional datasets were removed including five from the REP₂₀₀₄ database. The removal of these endpoints also resulted in the exclusion of five publications identified in the update (Bredhult et al., 2007; Hamers et al., 2011; Hu et al., 2020; Shi et al., 2019; Zhang et al., 2019) and one publication from REP₂₀₀₄ (Gupta et al., 1998); the number of REPs and specific congeners included in these datasets are available in Supplemental Table 2.

As a result of these efforts, 684 REP datasets were added from 47 publications, effectively doubling the database since 2004. New data were added for all congeners, other than 1,2,3,7,8,9-HxCDF. In total, REP₂₀₂₁ hosts 1269 in vitro and in vivo REP datasets for all congeners (Table 2). The data contained in the REP₂₀₂₁ database include 570 dose/concentration response data sets, 684 author-calculated REP point estimates, and 15 benchmark ratios calculated by database authors (Table 3).

Table 2.

In vivo, in vitro, and total datasets added to the REP₂₀₂₁. Numbers in the “Added” column indicate REPs added since the REP₂₀₀₄ database publication.

	in vivo		in vitro		REP2021 Total In Database
REP Compound	REP2004	ADDED	REP2004	ADDED
1,2,3,7,8-PeCDD	36	19	9	37	101
1,2,3,4,7,8-HxCDD	15	0	6	2	23
1,2,3,6,7,8-HxCDD	0	0	5	20	25
1,2,3,7,8,9-HxCDD	1	0	5	1	7
1,2,3,4,6,7,8-HpCDD	12	0	6	20	38
OCDD	1	0	5	2	8
TCDF	20	32	12	50	114
1,2,3,7,8-PeCDF	20	5	7	4	36
2,3,4,7,8-PeCDF	80	50	17	59	206
1,2,3,4,7,8-HxCDF	6	0	6	25	37
1,2,3,6,7,8-HxCDF	11	0	6	2	19
1,2,3,7,8,9-HxCDF	0	0	2	0	2
2,3,4,6,7,8-HxCDF	3	0	6	18	27
1,2,3,4,6,7,8-HpCDF	0	0	1	20	21
1,2,3,4,7,8,9-HpCDF	0	0	2	22	24
OCDF	6	1	3	4	14
PCB77	16	5	27	21	69
PCB81	0	0	10	2	12
PCB126	79	33	29	55	196
PCB169	15	1	15	18	49
PCB105	16	1	8	14	39
PCB114	2	0	6	2	10
PCB118	15	34	6	21	76
PCB123	2	0	4	1	7
PCB156	16	15	12	23	66
PCB157	2	0	5	5	12
PCB167	0	0	5	12	17
PCB189	3	0	2	9	14
Total	377	196	227	469	1269

Table 3.

Author-calculated REP (A-C), benchmark ratio (BMR), and dose/concentration-response datasets in REP₂₀₂₁ broken down by REP Compound.

REP Compound	A-C	BMR	Dose/Concentration Response	Total
1,2,3,7,8-PeCDD	57	1	43	101
1,2,3,4,7,8-HxCDD	17	0	6	23
1,2,3,6,7,8-HxCDD	23	0	2	25
1,2,3,7,8,9-HxCDD	7	0	0	7
1,2,3,4,6,7,8-HpCDD	33	0	5	38
OCDD	5	0	3	8
TCDF	53	2	59	114
1,2,3,7,8-PeCDF	16	0	20	36
2,3,4,7,8-PeCDF	79	1	126	206
1,2,3,4,7,8-HxCDF	32	0	5	37
1,2,3,6,7,8-HxCDF	11	0	8	19
1,2,3,7,8,9-HxCDF	2	0	0	2
2,3,4,6,7,8-HxCDF	24	0	3	27
1,2,3,4,6,7,8-HpCDF	19	0	2	21
1,2,3,4,7,8,9-HpCDF	22	0	2	24
OCDF	6	0	8	14
PCB77	33	3	33	69
PCB81	12	0	0	12
PCB126	68	4	124	196
PCB169	32	0	17	49
PCB105	22	0	17	39
PCB114	9	1	0	10
PCB118	27	0	49	76
PCB123	7	0	0	7
PCB156	36	1	29	66
PCB157	8	1	3	12
PCB167	14	1	2	17
PCB189	10	0	4	14
Total	684	15	570	1269

In comparison to REP₂₀₀₄, of which most REPs were represented by in vivo data, the majority of REP datasets identified in this update were based on in vitro models (469 in vitro vs. 196 in vivo). Another important characteristic of the updated database is the addition of substantial human in vitro data. Previously, REP₂₀₀₄ hosted 28 human in vitro REPs, representing 5% of the REP database. In this update, 135 human in vitro datasets were added to REP₂₀₂₁. As a result, human in vitro data now represents 13% of the database.

This database represents 39 unique in vitro endpoints and 52 in vivo endpoints (Supplemental Tables 3 and 4), among 8 mammalian species. Ethoxyresorufin O-deethylase (EROD) induction was the most frequent measurement among in vitro study models (n=206), in vivo study models (n=105) and all study models combined (n=311); changes in histopathology (across various tissues) was the second most frequent in vivo endpoint measurement (n = 89).The full REP₂₀₂₁ database inputs described herein are available in Supplemental Tables 5 and 6. Tables provided therein include all candidate REP inputs, along with any associated dose/concentration response data.

Derivation of REP₂₀₂₁ Best-Estimate TEFs

REP₂₀₂₁ database inputs, following all curation and QC, were applied to the Best-Estimate TEF Workflow (Ring et al., 2023). Both a weighted and unweighted version of this model were run, and both successfully converged with an R-hat = 1.00 (Gelman et al., 1992). Outputs were derived from both models, but results presented herein are from the weighted meta-analysis; data from the unweighted model are provided in Supplemental Table 7.

From the Bayesian meta-analysis, congener-specific sDR curves were estimated for each congener along with their uncertainty (Figure 3). These sDR curves enable direct assessment of the degree of parallelism between each congener (teal) and referent (purple). Parallelism varies notably among congeners. For most congeners, the slope of the sDR curve is fairly similar to TCDD; notable exceptions include 1,2,3,7,8-PeCDF, 1,2,3,7,8,9-HxCDF, and OCDF, all of which exhibit markedly shallower slopes than TCDD. For about half of congeners, maximal response is fairly similar to TCDD maximal response. For some congeners, maximal response is markedly greater than TCDD, including 1,2,3,4,6,7,8-HpCDD; 1,2,3,4,6,7,8-HpCDF; 1,2,3,4,7,8-HxCDF; 1,2,3,4,7,8,9-HpCDF; 1,2,3,6,7,8-HxCDD; 1,2,3,7,8,9-HxCDD; 2,3,4,6,7,8-HxCDF; PCB123; and PCB81. For other congeners, maximal response is markedly less than TCDD, including 1,2,3,7,8-PeCDF; OCDF; PCB167; and TCDF.

Figure 3. Model Estimate of the Standardized Reference-Congener Dose-Response relationship for each congener.

Median curves are shown as solid lines and colored according to whether it is the reference or the weighted congener curve (red = congener; blue = reference). The shading surrounding the curves representing the 90% CI is also colored accordingly.

Uncertainty in congener-specific sDR curves is primarily driven by database uncertainty: uncertainty is largest for the congeners with the fewest available REP datasets, especially for those with few or no available dose/concentration response datasets. Credible intervals (CI), shown as shaded areas of Figure 3, give the uncertainty of the dose-response estimate at the 90% level. Tighter CIs generally imply more data for the given congener for a given dose. Some congeners have large CIs around the maximum response, which often indicates that there was very little data at doses that elicited the maximum response for that congener.

Resulting BE-TEF values (i.e. the mean TEF value derived from the TEF uncertainty distribution, see Figure 1), and their respective 90% CI, are provided in Table 6 and visualized as violin plots extending from each congener’s lower to upper 90% in Figure 4. Generally, congeners with the highest degree of uncertainty surrounding their BE-TEF estimations tended to also be those which were the most data poor (and typically also involved assignment to a “low” quality category by the machine-learning model).

Table 6.

Best-Estimate TEFs derived from the weighted analysis of REP₂₀₂₁ and comparison TEF values from REP₂₀₀₄ and WHO (2005)

REP Compound	REP2021 N	WHO (2005) TEF	REP2004 TEF	REP2021 BE-TEF	REP2021 Lower 90% BE-TEF	REP2021 Upper 90% BE-TEF
1,2,3,7,8-PeCDD	101	1	0.3	0.4	0.3	0.6
1,2,3,4,7,8-HxCDD	23	0.1	0.07	0.09	0.03	0.3
1,2,3,6,7,8-HxCDD	25	0.1	0.08	0.07	0.03	0.2
1,2,3,7,8,9-HxCDD	7	0.1	0.04	0.05	0.004	0.7
1,2,3,4,6,7,8-HpCDD	45	0.01	0.01	0.05	0.02	0.1
OCDD	8	0.0003	0.0009	0.001	0.0002	0.006
TCDF	114	0.1	0.05	0.07	0.04	0.1
1,2,3,7,8-PeCDF	36	0.03	0.01	0.01	0.003	0.04
2,3,4,7,8-PeCDF	206	0.3	0.1	0.1	0.09	0.2
1,2,3,4,7,8-HxCDF	37	0.1	0.2	0.3	0.1	0.8
1,2,3,6,7,8-HxCDF	23	0.1	0.06	0.09	0.03	0.3
1,2,3,7,8,9-HxCDF	2	0.1	0.2	0.2	0.004	18
2,3,4,6,7,8-HxCDF	27	0.1	0.08	0.1	0.05	0.3
1,2,3,4,6,7,8-HpCDF	23	0.01	0.03	0.02	0.007	0.05
1,2,3,4,7,8,9-HpCDF	24	0.01	0.04	0.1	0.04	0.3
OCDF	14	0.0003	0.002	0.002	0.0005	0.01
PCB77	69	0.0001	0.0002	0.0003	0.0002	0.0006
PCB81	12	0.0003	0.006	0.006	0.0008	0.07
PCB126	211	0.1	0.08	0.05	0.04	0.06
PCB169	49	0.03	0.02	0.005	0.003	0.010
PCB105*	39	0.00003	0.00003	0.00003	0.00001	0.00005
PCB114*	10	0.00003	0.0002	0.0002	0.00001	0.0001
PCB118*	76	0.00003	0.00004	0.00004	0.00003	0.007
PCB123*	7	0.00003	0.00005	0.00002	0.000002	0.0002
PCB156*	66	0.00003	0.0001	0.00009	0.00005	0.0001
PCB157*	12	0.00003	0.0002	0.0001	0.00002	0.0006
PCB167*	17	0.00003	0.00005	0.000009	0.000002	0.00004
PCB189*	14	0.00003	9E-06	0.000008	0.000003	0.00002

^*Congeners have their TEF set at 0.00003 as default. Model derived BE-TEFs are given for comparison but would be substituted with the default value in practice

Figure 4.

Best-Estimate TEF Estimates (diamonds) and their Surrounding Uncertainty Distributions (violins), truncated at their respective lower and upper 90% CI. The WHO (2005) TEF (asterisk) is also shown for comparison.

Weighting did not have a large impact on BE-TEF values (see unweighted BE-TEF values in Supplemental Table 7). Lastly, following a Prob NQ analysis, only 1,2,3,7,8-PeCDF and OCDF TEFs had > 50% probability of being non-quantifiable and the median probability across all congeners was 0.26%. This indicates that all congeners except for 1,2,3,7,8-PeCDF and OCDF have maximal effects that more than likely exceed 50% of TCDD maximal effect, i.e., the response level for which the TEFs were derived (Supplemental Table 8). This can be observed visually for these two congeners in Figure 3 in their comparatively shallow congener dose response curves.

Discussion

The underlying database (REP₂₀₀₄) considered at the time of the 2005 WHO review was developed based on early systematic methods for identification and selection of studies, and extraction (and calculation) of REP values. Despite the evolving scientific landscape and continued stakeholder interest in the characterization of TEFs, the REP database has not been updated in over a decade. This effort describes a modern, fit-for-purpose systematic review that follows explicit methods outlined in an a priori protocol in order to identify and select relevant research and minimize bias (Higgins et al., 2022). The consensus-based weighting schema developed by Wikoff et al., (Accepted, 2023) and applied using machine-learning herein provides a fit-for-purpose critical appraisal to evaluate the quality and relevance of the REPs in context of human health risk assessment.

This update has more than doubled the size of the REP database, with nearly 1300 REP values across the 29 congeners. In contrast to previous versions of the database underlying the development of TEFs, REP₂₀₂₁ includes not only point-estimate REPs, but also dose- and concentration-response data. All of these types of REP data are integrated in a Bayesian weighted meta-analysis. This recommended method of integration is novel to the development of TEFs and is anticipated to better account for the large degree of variability in REP study quality and the heterogeneity in methods used by study authors to derive REPs. Application of the Bayesian meta-analysis workflow subsequently produced BE-TEFs for all DLC congeners, accounting both for study quality and dose-response methods, thereby producing a dataset addressing the issues which previously precluded the WHO expert working groups from utilizing REP distributions (Van den Berg et al., 2006). The resulting BE-TEF values are transparently derived, are reproducible, and are entirely data-driven.

The BE-TEFs developed based on the REP₂₀₂₁ database are consistent with the 2005 TEFs developed by the WHO expert panel and are within an order of magnitude of such. The comparability between the values across the congeners, which is also seen in the application of the workflow on the REP₂₀₀₄ database (Ring et al., 2023), lends significant confidence to the previous TEF selection process which included expert judgement. It also increases the confidence in the predictions developed based on the BE-TEF workflow. For OCDD, OCDF, and PCB81, which only had 2, 5, and 2 REP datasets added to their databases, respectively, it is likely that deviations from the WHO TEFs were due to small overall database size. For 1,2,3,4,6,7,8-HpCDD, 1,2,3,4,7,8-HxCDF, 1,2,3,4,7,8,9-HpCDF it is likely these deviations were due to large increases in database size (85%, 102%, 169%, increases, respectively).

Interpretation of the BE-TEFs reported herein should also consider Prob NQ. As stated previously, 1,2,3,7,8-PeCDF and OCDF had a >50% probability of being non-quantifiable for this database. For these two congeners, their high Prob NQ indicates that the maximum response of the congener is less than the maximum response of the reference compound (i.e., there may be lack of parallelism). In this circumstance, the TEFs are not constant and as such, it is possible that different TEFs could be derived based on the response level of interest. This is an important factor to consider in determining the appropriate response level for the derivation of TEFs/REPs, and more broadly how the TEFs for these congeners should be applied in a risk assessment context. It is important to reiterate, however, that a high Prob NQ here does not invalidate the derived BE-TEFs, as the TEF uncertainty distributions for each congener are generated only from the Bayesian meta-analysis posterior samples that describe an sDR relationship in which the congener response reaches 50% of the TCDD maximal response.

When interpreting the uncertainty in the Best-Estimate TEFs (BE-TEFs), quantified by the Bayesian credible intervals (CI), practitioners should be aware of two caveats that imply the BE-TEF CI bounds cannot be used for quantitative risk assessment or risk management decision-making. That is, they cannot be interpreted as “best-case” and “worst-case” TEF values.

With all Bayesian analyses, BE-TEF CI bounds are impacted by prior assumptions used in the analysis; thus, the CI represents mathematical uncertainty and all assumptions therein, which is not the same as toxicological uncertainty. A priori BE-TEF CIs were assumed to have bounds between 1×10⁻²⁰ and 1×10⁶, and the bounds on the standardized Hill model parameters were not selected considering the toxicological plausibility of the resulting bounds on BE-TEF CI. If there is little information on the true TEF, the Bayesian posterior CIs would likely represent the prior assumptions. Additionally, if relative potency were truly as low as 1×10⁻²⁰ the compound would hardly be considered dioxin-like at all, and if relative potency were truly as high as 1×10⁶, the compound would be a million times more potent than TCDD. The central estimates of BE-TEF for these data-poor congeners are still valid, because they are in the range of the available data and their concordance with 2005 estimates; however, the BE-TEF CI bounds represent prior assumptions that are not necessarily toxicologically plausible. Further refinements of the model should include toxicological assumptions.

Additionally, the statistical model for the meta-analysis did not take into account potential correlations among different REP response endpoints or congeners measured by the same laboratory. In other words, potential laboratory effects that may arise due to differences in the methods, equipment, or procedures used by different laboratories are not included in the model. The laboratory may induce bias within a single study, however, there is no systematic reason to believe this bias is directional across all studies. Consequently, it was assumed that any potential bias is randomly distributed around the central BE-TEF estimate, which lends to the applied methodology. However, the intra-lab correlations may have implications for calculating the BE-TEF uncertainty and as such this correlation could be further considered in future updates and refinements to the BE-TEF methodology and application to relative potency for dioxins or other chemicals.

Uncertainty could be underestimated if REPs measured in the same experiment are positively correlated, i.e., all shift in the same direction. In this case, multiple REPs from the same experiment provide less total information than if the REPs were from different experiments. However, uncertainty could also be overestimated if REPs measured in the same experiment are negatively correlated. Some types of intra-laboratory correlation might have no net effect on BE-TEF uncertainty estimates if non-standardized dose-response curves for the reference compound and all congeners were shifted by the same amount in the same direction; in that case, the standardized dose-response curves would be unchanged.

For these reasons, the BE-TEF CI bounds should be interpreted only as semi-quantitive estimates of uncertainty. They are useful in the context of scientific inquiry regarding database uncertainty and study quality differences, but they are not appropriate for use in justifying risk management decisions. Despite these limitations, the Bayesian meta-analysis framework represents a step forward in evidence integration. Before the Bayesian meta-analysis framework, TEF estimates were based upon expert judgment from the literature rather than a quantitative evaluation. The Bayesian framework allows transparent analysis and interpretation of uncertainty estimates, including identifying limitations of those estimates. Future work to advance the state of the science with the application of this framework may include refinements to the Bayesian hierarchical models to better model toxicological plausibility and potential intra-laboratory correlations.

The BE-TEFs developed herein are considered conservative. In following decisions made by previous database updates and WHO expert panel reviews, inclusion in the database is dependent on the measurement of a statistically significant response. Congener data were not included if a statistically significant response was not observed; as such, not including negative data introduces a potential bias away from the null, and it is possible that true toxic equivalencies could be less than the values calculated herein.

It is also of note that like past iterations of the REP database, epidemiological data were considered outside of the scope of this database update. There are several reasons for the exclusion of epidemiological data, including but not limited to the known uncertainties in internal validity such as exposure assessment, confounding, and other potential sources of bias (Burns et al., 2014; LaKind et al., 2020; Ockleford et al., 2017). Future iterations could possibly include both negative data and epidemiological data (assuming limitations in exposure and outcome measurements can be accommodated to confidently assess relative potency) to characterize toxic equivalency more fully.

One of the greatest utilities of the BE-TEF approach is the collective and systematic consideration of heterogeneous experimental designs, outcomes, study types, and quality of the studies captured in the database. As with previous iterations of the underlying database, REP₂₀₂₁ contains effect data for a wide variety of exposure scenarios (e.g., duration, route, concentrations) and outcome measures. For example, studies included in REP₂₀₂₁ provided up to 160 REP input datasets across 17 congeners, and 3 species (Larsson et al., 2015). Further emphasizing the broad nature of this approach, different data types (e.g., dose/concentration response, author-derived REPs, or benchmark ratios) were collected based on availability and reporting. In many of the reviewed publications, these data types were reported together and, as such, were all collected during data extraction for possible inclusion based on the hierarchy of selection as presented here; e.g., Frawley et al. (2014) presents dose-response curves, ED50s, and author-calculated REPs for multiple endpoints across seven congeners, all which were collected at the data extraction phase. This approach required complex data extraction form development to account for the variety of datasets in comparison to a more traditional hazard review, such as National Toxicology Program’s monograph on perfluorooctanoic acid and perfluorooctane sulfonate (2016), where the data extraction phase is designed to capture only the effect dose and/or single related benchmark types (e.g., NOAEL/LOAEL). This elaborate workflow also highlights the importance of the QC process and justifies the need for both an extensive internal QC and external peer review as was implemented.

In addition to QC of the database and BE-TEF model input, QC of the BE-TEF workflow was also conducted to assess factors affecting reproducibility of the model results. Reproducibility is affected by random sampling inherent in the Bayesian-inference steps of the workflow. Reproducibility was ensured by first testing numerical convergence of the model (i.e., that model results are minimally sensitive to the specific sequence of random numbers generated in any individual run). However, even though the model converges, the exact results for two different runs will still have small numerical differences, because they use two slightly different random samples. Results were made exactly-reproducible by controlling the initial state of the random number generator by setting an integer “seed” for the random number generator to produce exactly-reproducible sequences of random numbers and thus exactly-reproducible results. If the model is run with a different integer seed, the results will be slightly numerically different. Furthermore, if the model is stopped and restarted without re-setting the seed, the sequence of random numbers will not begin from the same point, and the results will again be slightly numerically different.

Reproducibility is also affected by whether the same input data is used for two different runs. In particular, when the workflow is run for a subset of the REP₂₀₂₁ database, the results for that subset will not be the same — and are not expected to be the same — as when the workflow is run for the full REP₂₀₂₁ database (a condition tested during QC analysis). For example, if the workflow is run only on the subset of REP₂₀₂₁ data for PCB126 and is then run on the full REP₂₀₂₁ database including data for all congeners, the two runs will produce two different PCB126 BE-TEF estimates. This is the expected behavior of the workflow as the Bayesian dose-response and Bayesian meta-analysis models are both hierarchical, meaning that datasets and congeners are not modeled independently of other datasets or congeners. To continue the previous example, the PCB126 BE-TEF estimate does not depend only on the PCB126 data from REP₂₀₂₁; it also depends partially on all the other REP₂₀₂₁ data. See Ring et al. (2023) for details about the Bayesian hierarchical models. Results of the workflow are exactly reproducible between runs only if the same seed is set for the random number generator for both runs, and if the same input data are used for both runs. These factors should be kept in mind when attempting to reproduce the results shown herein, or if applying the workflow to subsets of the REP₂₀₂₁ database, or if applying the workflow to new data.

In conclusion, the REP₂₀₂₁ database and associated BE-TEFs provide the most up-to-date quantitative TEF estimates for each DLC using all available data, while accounting for quality, relevance, and a consistent approach for determining the TEF for each DLC as recommended by the 2005 WHO Expert Panel (Van den Berg et al., 2006). Importantly, the BE-TEF workflow described herein also allows for readily producing up-to-date TEFs as the underlying database is updated in order to produce estimates that align with current scientific research.

Supplementary Material

MMC1

• Supplemental Table 1. Hill Model Parameters and Assigned Bayesian Dose/Concentration-Response Modeling Prior distribution types and characteristics

• Supplemental Table 2. Refinement of REP candidate inputs by the WHO TEF Expert Panel based on endpoint review

• Supplemental Table 3. in vitro endpoints included in REP2021

• Supplemental Table 4. in vivo endpoints included in REP2021

• Supplemental Table 5. Dose-response data underlying the REP candidate model inputs for Bayesian Meta-Analysis

• Supplemental Table 6. Candidate model input list used in the development of BE-TEFs

• Supplemental Table 7. Best-Estimate TEFs derived from the unweighted analysis of REP2021 and comparison TEF values from the weighted REP2021 analysis and WHO (2005).

• Supplemental Table 8. The calculated Prob NQ of weighted and unweighted analyses expressed as probabilities and percent probabilities for each congener

Highlights:

• The dioxin-like congener relative potency (REP) database has been updated for the first time since 2004

• Systematic methods documented by an a priori protocol increase transparency and reliability of the database

• The size of the database available to update toxic equivalency factors (TEFs) was increased by over 100%

• Best Estimate TEFs (BE TEFs) were developed via Bayesian dose-response modeling and meta-analysis

• Modeled BE TEFs are within an order of magnitude of WHO 2005 TEF values

Frequently Used Abbreviations and Definitions

BE-TEF

Best Estimate Toxic Equivalency Factors

CI

credible intervals

DLC

dioxin-like compounds

ED50

median effective dose

LOAEL

lowest observable adverse effect level

NOAEL

no observable adverse effect level

Prob NQ

Probability the value is not quantifiable

REP

relative potency

sDR

standardized dose/concentration response curves

TEF

Toxic Equivalency Factors

QC

quality control

WHO

World Health Organization