Applying ‘omics technologies in chemicals risk assessment: Report of an ECETOC workshop

Abstract

Prevailing knowledge gaps in linking specific molecular changes to apical outcomes and methodological uncertainties in the generation, storage, processing, and interpretation of ‘omics data limit the application of ‘omics technologies in regulatory toxicology. Against this background, the European Centre for Ecotoxicology and Toxicology of Chemicals (ECETOC) convened a workshop Applying ‘omics technologies in chemicals risk assessment that is reported herein. Ahead of the workshop, multi-expert teams drafted frameworks on best practices for (i) a Good-Laboratory Practice-like context for collecting, storing and curating ‘omics data; (ii) the processing of ‘omics data; and (iii) weight-of-evidence approaches for integrating ‘omics data. The workshop participants confirmed the relevance of these Frameworks to facilitate the regulatory applicability and use of ‘omics data, and the workshop discussions provided input for their further elaboration. Additionally, the key objective (iv) to establish approaches to connect ‘omics perturbations to phenotypic alterations was addressed. Generally, it was considered promising to strive to link gene expression changes and pathway perturbations to the phenotype by mapping them to specific adverse outcome pathways. While further work is necessary before gene expression changes can be used to establish safe levels of substance exposure, the ECETOC workshop provided important incentives towards achieving this goal.

Background

‘Omics technologies hold the promise of generating comprehensive toxicologically relevant information on molecular changes in cells and tissues more quickly, more accurately and with less resources than ever before (ECETOC, 2008). They provide approaches to precisely measure substance-induced molecular perturbations that are associated with adverse outcomes in animals and humans. Thereby, these emerging technologies have the potential to improve chemical safety assessment and, at the same time, to refine and reduce animal testing in regulatory toxicology. ‘Omics technologies are poised to increase the number of substances that can be efficiently tested in a given time and to increase the number of molecular and cellular endpoints that can be detected and simultaneously evaluated. Also, ‘omics technologies have the potential to identify new and emerging hazards through enhanced coverage of biological (or biochemical) pathways (cf. Box for definition) during toxicological screening, and this information can be mined for the development of new, reliable biomarkers for the detection of adverse effects (ECETOC, 2010). However, in addition to prevailing knowledge gaps in linking specific molecular changes to apical outcomes, current methodological uncertainties in interpreting and assessing data limit the application of ‘omics technologies in regulatory toxicology. This includes issues surrounding the generation, storage, processing, and interpretation of ‘omics data, as well as a lack of experience with such kind of data across the regulatory community. Hence, best practices for generating, storing, processing, and interpreting ‘omics data are needed as one starting point so that the outcomes of ‘omics-based studies can be reliably verified and confidently integrated into regulatory hazard and risk assessment.

Against this background, the European Centre for Ecotoxicology and Toxicology of Chemicals (ECETOC) convened a workshop entitled Applying ‘omics technologies in chemicals risk assessment that took place from 10-12 October 2016 in Madrid, Spain. Thirty-six invited experts from Europe, Canada, Japan, and the United States, attended this workshop representing the European Commission; the Organisation for Economic Cooperation and Development (OECD); national authorities from EU Member States, EU associated countries and North America; academia; industry and independent consultants. An additional three experts contributed to the workshop in the form of video or dial-in presentations (cf. Appendix for the list of workshop participants).

The 2016 ECETOC workshop, reported herein, was conceived to contribute to fulfilling the recommendations from previous ECETOC workshops on the Application of ‘omic technologies in toxicology and ecotoxicology: Case studies and risk assessment (ECETOC, 2008), ‘Omics in (eco)toxicology: case studies and risk assessment (ECETOC, 2010), and ‘Omics and risk assessment science (ECETOC, 2013). The recommendations from these workshops addressed the need to define quality standards on the design and performance of ‘omics-based studies and to communicate these quality standards as best practices. Such guidance would serve to enhance the regulatory acceptance and use of ‘omics data (ECETOC, 2008, 2010, 2013).

Ahead of the 2016 workshop, ECETOC multi-expert teams drafted frameworks addressing the key objectives to establish best practices for (i) a Good-Laboratory Practice (GLP)-like context for collecting, storing and curating ‘omics data; (ii) the processing of ‘omics data, and (iii) weight-of-evidence (WoE) approaches for integrating ‘omics data. Additionally, the key objective (iv) to establish approaches to connect ‘omics perturbations to phenotypic alterations was addressed, albeit without an in-advance draft of a framework.

As Alan Poole (Secretary General of ECETOC, Belgium) explored in the opening presentation, the workshop aimed to further elucidate these four key objectives and the contents of the draft frameworks. Invited presentations served to provide background information on relevant case studies or related ongoing activities. Importantly, four breakout sessions and extensive plenary sessions provided ample opportunity for in-depth discussions. The recommendations from the workshop would be used to advance the drafted frameworks to ensure their fitness for purpose, i.e. for regulatory acceptance and use. Once finalised, the frameworks would contribute to increasing confidence in the regulatory use of ‘omics data. Importantly, the frameworks were intended to provide baselines on best practice but not to be prescriptive.

Alan Poole further explained that the goal to generate consensus of opinion on the steps needed to achieve best practices for the generation and storage, processing and interpretation of ‘omics data for the hazard and risk assessment of substances is highly relevant in a global context. Generally, the drafted frameworks were prepared to cover all purposes of regulatory use of ‘omics data, including use in supporting or predicting no (or lowest) observed adverse effect levels (NOAEL or LOAEL) as points of departure (PoD) for hazard assessment, and investigating adverse outcome pathways (AOPs) or substance-specific modes-of-action (MoAs) (Ankley et al., 2010; OECD, 2012, 2013; Meek et al., 2014); cf. Box for definitions. Further definitions of ‘omics-related terminology and details on the regulatory use of ‘omics data as well as on the need to develop frameworks on the collection and storage, processing, and interpretation of ‘omics data are provided in a background document to this report (Sauer et al., 2017) in this journal Supplement.

In accordance with its four key objectives, the workshop was divided into four work streams, preceded by an opening session. Work stream 1 covered the generation and storage of ‘omics data, work stream 2 the processing of ‘omics data, and work streams 3 and 4 the interpretation of ‘omics data (Figure 1). Below, the topics and concepts that were addressed in the presentations are summarised, and the main issues and recommendations that arose during the breakout sessions and plenary discussions (moderation: Alan Poole and Madeleine Laffont, ECETOC, Belgium) are highlighted.

Figure 1:

Steps of ‘omics studies (taking the example of transcriptomics) and their correspondence to the work streams of the ECETOC workshop

GLP: Good Laboratory Practice; WoE: Weight-of-evidence.

Opening session: Regulatory perspectives on scientific needs, challenges and opportunities of ‘omics technologies

Alan Poole chaired the opening session, in which representatives from the OECD, national authorities and the European Commission provided keynote presentations of their perspectives on scientific needs, challenges and opportunities of ‘omics-based technologies.

Presentations Opening Session:

Eeva Leinala (OECD, France) summarised OECD activities on the development of AOPs (OECD, 2013) and integrated approaches to testing and assessment (IATAs). IATAs and integrated testing strategies (ITSs) serve to combine different (non-animal) test methods reflecting the different steps of an AOP (OECD, 2015). The ongoing OECD Extended Advisory Group on Molecular Screening and Toxicogenomics (EAGMST) is dedicated to the topic of use of ‘omics-based technologies for toxicological evaluations, while also supporting AOP development and MoA assessment (OECD, 2014).

Matt Martin (National Center for Computational Toxicology (NCCT), U.S. Environmental Protection Agency (EPA), USA) spoke about high throughput and transcriptomics work being conducted at the U.S. EPA. This work contributes to the establishment and use of in vitro PoDs that might be suitable for hazard assessment, to the identification of primary and secondary molecular targets of substances via profile matching with reference substances, and to the identification of molecular initiating events (MIE; i.e. the first event in an AOP), as well as subsequent key events.

Aldert H. Piersma (Centre for Health Protection, National Institute for Public Health and the Environment (RIVM), The Netherlands) took the example of embryonic stem cell neural differentiation to highlight important aspects to consider when establishing best practices for ‘omics studies. He highlighted that a scientifically sound study design is pivotal for a successful study outcome. Further, it should be ensured that the statistical analysis of ‘omics data does not override the evaluation of their biological relevance since the magnitude and/or variance of the gene expression response does not provide useful information about the dynamic range or the dose-response relationship of the measured response. He cautioned that in mixed cell type tissues, individual genes might show limited responses, whereas their expression changes might be highly significant when related to specific cell types. Further, single gene expression changes might gain weight when pathways of related genes, and their expression changes, are analysed. At best, studies should address time courses and dose-responses of substance-induced gene expression changes and not only single time points and doses. In evaluating the outcomes of ‘omics studies, the reproducibility of gene expression changes in more than one test system improves the overall WoE, and investigators should strive to understand the biology of the test systems used.

Tewes Tralau (German Federal Institute of Risk Assessment (BfR), Germany) presented a regulator’s view on the challenges and perspectives for using ‘omics studies for regulatory toxicology. With the advent of ‘omics, high throughput testing, three-dimensional cell cultures and, more recently, micro-physiological systems, the pressure to bring such methods to regulatory use has increased as has the debate about their usability and application domains (Tralau et al., 2015). Yet, regulators continue to take a cautious stand on this matter. Currently, ‘omics data are mostly seen as supplementary information, for example for MoA analysis or to substantiate the application of read-across techniques. Main obstacles to the implementation of ‘omics-based methods are uncertainties in how to use and validate such data, as well as their predictability and performance for hazard and risk assessment when compared to established test methods. Without changes in long-established practices, it will be difficult to readily adapt ‘omics data for the purpose of quantifiable risk assessment or to transpose test results to the classification and labelling (C&L) of substances, as implemented in the United Nations Globally Harmonised System of Classification and Labelling of Chemicals (GHS; United Nations, 2015) and Regulation (EC) No 1272/2008 on the classification, labelling and packaging of substances and mixtures (CLP; EP and Council of the EU, 2008). From a current regulatory perspective, the biggest promise of ‘omics data is use in ITSs, to identify biomarkers of adverse effects and to prioritise testing. Key scientific, legal and regulatory issues thus have to address the following questions: Is it the aim to provide supplementary information, to replace traditional toxicological endpoints or to address unresolved toxicological issues? How can molecular measures be converted into quantitative (or probabilistic) measures of adversity that are suitable for risk assessment? Can relevant biomarkers be identified, i.e. specific genes that define the pathological point-of-no-return that will ultimately lead to an adverse effect? How shall ‘omics data be analysed and validated? Do validation studies include the right standards? Are the currently used test systems sufficiently complex? Also discussed was the need for blinded pilot studies (in which information on the test substances was not disclosed to the experimenter until after the completion of the study) in order to obtain a better understanding of the performance of ‘omics-based methodologies in routine practice.

Andrew Worth (European Commission, Joint Research Centre, European Reference Laboratory for Alternatives to Animal Testing (EURL ECVAM), Italy) added further perspectives on the regulatory use of ‘omics technologies. The primary mission of the EURL ECVAM is to make available validated test methods that support the 3Rs principle to replace, reduce and refine animal testing that was originally introduced by Russell and Burch (1959). Possible applications of ‘omics studies include the development of AOPs; the quality assurance of cell lines with respect to their identity, genetic stability, and contamination; the provision of supplementary data in WoE approaches; the support of grouping of substances and read-across technologies; and, finally, to set PoDs for risk assessment and to provide information relevant for C&L. Generally, ‘omics-based test methods need to be validated to demonstrate their reliability and reproducibility, their mechanistic relevance and utility, prior to routine use in these applications.

On behalf of Leming Shi (Fudan University, Shanghai, China), Weida Tong (National Center for Toxicological Research (NCTR), U.S. Food and Drug Administration (FDA), USA) provided an overview on lessons learned from the first parts of the Microarray Quality Control (MAQC) and Sequencing Quality Control (SEQC) projects in striving for reproducible ‘omics data for risk assessment. The goal of the FDA-led community-wide MAQC consortium was to assess the technical performance and application of ‘omics technologies in clinical application, safety assessment and precision medicine. The MAQC consortium was established in 2005 and, by end of 2014, had completed three projects evaluating the performance of microarrays, genome-wide association studies and RNA-sequencing (RNA-seq), with particular reference to:

1. the reproducibility of transcriptomics data, defined as the percentage of overlapping genes between lists of differentially expressed genes (DEGs),

2. between-experiment concordance, within-laboratory repeatability, and

3. cross-platform reproducibility.

In these projects, relative measures, but not absolute measurements, agreed well across laboratories. Slight changes in the methods applied for statistical analysis considerably affected the composition of the DEG lists. Further, the responsiveness of the individual test animals accounted for approximately 30% of the data variation. Using fold changes in gene expression (as compared to the expression levels observed in control groups) together with p-value thresholds to identify DEGs enhanced reproducibility, while still optimally balancing sensitivity and specificity. Thus, the work of the MAQC consortium has advanced microarray and RNA-seq analytical pipelines that can be leveraged for developing analytical frameworks and best practices. Further details on the MAQC project are provided in Sauer et al. (2017) in this journal Supplement.

Work stream 1: Establishing a GLP-like context for collecting, storing and curating ‘omics-based data

Chair: Ben van Ravenzwaay (BASF SE Germany; Chair of the ECETOC Scientific Committee)

Presentations Work Stream 1:

Hans-Martin Kauffmann (BASF SE, Germany) presented the ECETOC concept for quality assurance of new technology data considering GLP requirements. The details on this framework are provided in Kauffmann et al. (2017) in this journal Supplement. Briefly, a GLP environment comprises a standard operating procedure (SOP) system, proper pre-planning and conduct documentation as well as independent quality assurance inspections. Particular challenges to fulfil GLP requirements for ‘omics-based studies are likely to be associated with (1) the definition of raw data; (2) the reproducibility of final results based on stored raw data; (3) the transparent description of all data processing steps; and (4) the validation of procedures and software (that could be conducted as ‘black box’ validation). However, even if individual parts of ‘omics studies cannot be addressed in full GLP-compliance, an ‘as GLP-like as possible’ procedure serves to promote the regulatory acceptance of ‘omics data.

This introductory lecture was followed by presentations exploring the feasibility of collecting specific types of ‘omics data for regulatory purposes in a GLP-compliant manner.

Ben van Ravenzwaay presented a case study to exemplify the state-of-the-art collection of metabolomics data under GLP and GLP-like conditions. In this case study, all steps of the animal studies were conducted in-house, under GLP conditions. The metabolome analysis was undertaken externally in a test facility with GLP status for mass spectrometry analytics. The different steps of bioanalytical measurements and archiving of the bioanalytical data were also conducted following GLP criteria. Only the quality control (QC) and potential subsequent manual improvements of data (that might be necessary to improve separation of non-baseline separated peaks in cases where standard software is unable to handle this) are less than ideal with respect to GLP requirements. To transparently document such manual improvements, all individual adjustments must be recorded in a GLP-like manner. Finally, the interpretation of the outcomes of the studies was not amenable to GLP criteria since it required case-by-case expert judgment-based evaluations.

Amber Goetz (Syngenta Crop Protection LLC, USA) reported that quantitative real-time polymerase chain reaction (qRT-PCR) studies might or might not be not conducted according to GLP guidelines, depending on the capabilities of the contract research organisations. She noted that transcriptomics studies are generally not conducted according to GLP guidelines. At best, the spirit of GLP can be maintained by demonstrating that the in vivo studies were conducted according to GLP and then listing which parts of the study were not conducted in compliance with GLP. Generally, GLP standards serve to ensure data integrity, without this being an assurance of the scientific merit, or relevance, of the study.

David Rouquié (Bayer CropScience, Sophia Antipolis, France) presented a GLP perspective of gene expression analyses by qRT-PCR. These studies are performed both for early phase compound selection and for product support, i.e. to identify MoAs. All the steps of the wet lab workflow are described in SOPs and can readily be performed under GLP conditions. By contrast, software validation is the challenging part that may preclude the achievement of a full GLP status for the qRT-PCR analysis.

Summary of Breakout Session Work Stream 1:

The outcome of the subsequent breakout session Feedback on the GLP framework served to update the draft Framework for establishing a GLP-like context for collecting, storing and curating ‘omics data presented in Kauffmann et al. (2017) in this journal Supplement. The workshop participants agreed that the establishment of a GLP-like context for collecting, storing and curating ‘omics data would be a valuable contribution to fostering the regulatory acceptance and use of ‘omics data. Nevertheless, a key question raised in the discussions was whether GLP criteria would alone provide the guidance for conducting high-quality ‘omics studies, or whether more detailed SOPs and analysis criteria, such as the ones provided by the MAQC consortium, would be required to capture the levels of detail and flexibility required for accommodating the use of ‘omics data in regulatory toxicity studies.

Workshop participants defined raw data, in terms of ‘omics technologies, as data produced by the ‘omics instruments following the application of a data acquisition tool (e.g. the data produced from a microarray scan following the application of the image analysis software tools provided by the respective vendor). It was noted that the sheer magnitude of ‘omics datasets, and the fact that such data might exhibit a high degree of between-sample heterogeneity and variation as a result of vendors’ methods variations, may pose special challenges for quality assurance auditing. With respect to the software tools, the availability of audit trails to prevent uncontrolled manipulation was considered especially valuable.

In concluding work stream 1, Ben van Ravenzwaay highlighted that advancing technology standardisation is essential to improve best practice for ‘omics studies. However, he cautioned that GLP guidelines serve to ensure the reproducibility of studies, based on existing raw data, but not necessarily their quality. GLP compliance ensures the traceability of study results, and it prevents data manipulation. While audit trails are important (particularly for the electronic storage of raw data), pending their inclusion in all relevant software, the need for audit trails can be circumvented by using paper records of the data with every manipulation. If this is not possible for technical reasons, such as for extremely large datasets (e.g., RNA-seq), checksums can be computed on each intermediate dataset, storing both the checksums and the datasets.

Work stream 2: Towards establishing criteria and best practices for analysing ‘omics-based data

Chair: Weida Tong (NCTR, FDA, USA)

Presentations Work Stream 2:

Timothy W. Gant (Centre for Radiation, Chemical and Environmental Effects (CRCEE), Harwell Science and Innovation Campus, Public Health England (PHE), UK) provided an overview of the ECETOC framework for the analysis of transcriptomics and other big data for regulatory application. This framework is presented in detail in the corresponding article of this journal Supplement. Since the Minimum Information About Microarray Experiments guidelines (MIAME; Brazma et al., 2001) and the work of the MAQC consortium (MAQC et al., 2006; SEQC/MAQC III Consortium, 2014) successfully addressed issues to consider in generating and storing ‘omics data, the ECETOC framework focuses on the second step of ‘omics studies, i.e. data processing. Data processing encompasses (i) the selection of raw data; (ii) data normalisation; (iii) the ‘filtering’ of data to remove outliers; and (iv) statistical analysis.

Weida Tong summarised lessons learned from the MAQC and SEQC projects and presented next steps towards reproducible ‘omics for risk assessment. Generally, the reproducibility of DEG lists can be affected by (i) sample size; (ii) the endpoint under investigation (reproducibility increases with increasing treatment-related effect); and (iii) the complexity of the study design (taking into account the availability of technical and biological replicates). The MAQC criterion to include fold change and p-value thresholds to account for cross-laboratory and cross-platform variation as well as variance filters to account for signal/noise ratio has improved the reproducibility of transcriptomics data. For statistical analysis, the t-test is recommended over the Analysis of Variance (ANOVA) model, since parameters affecting variance are not necessarily known and hence cannot be fully accounted for by ANOVA. Further investigations should aim at assessing how different levels of gene expression (large versus small modulations) affect reproducibility and how different approaches to process and analyse transcriptomic data affect the resulting lists of DEGs. Further, it is unknown how the interpretation of the biological relevance of the resulting lists of DEGs is affected by the specific pathway database used for data interpretation.

Tim Ebbels (Imperial College London, UK) presented a metabolomics case study to explore aspects of data processing relevant for this type of ‘omics study. Metabolomics can be applied either in targeted or untargeted mode, i.e. with or without a prior hypothesis on the involvement of specific metabolites. Accordingly, the experimental design, and particularly the assay technique, determines the range of metabolites that can be observed. In liquid chromatography – mass spectrometry (LC-MS), which is frequently used for metabolomics studies, hundreds to thousands of observed peaks could correspond to hundreds of metabolites (one metabolite can have more than one peak). Since the metabolites are not identified, statistical analysis (e.g., between control and test samples) is typically performed at the peak level. In evaluating data, peaks will typically have to be aligned to account for differences in the chromatographic retention times in order to match different peaks from different samples. Nuclear magnetic resonance (NMR) techniques are also used for metabolite profiling and offer a reproducible analytical approach with low data variability. However, the number of analytes is limited due to a lower sensitivity compared to LC-MS. In NMR, the assignment of peaks to metabolites is encumbered by peak overlap and peak shifts caused by variations in pH and ionic strength. NMR, LC-MS, and also gas chromatography - mass spectrometry are frequently used in parallel to achieve as broad a coverage of the metabolome as possible.

Summary of Breakout Session Work Stream 2:

The outcome of the breakout session Feedback on the framework of best practices for analysing ‘omics data served to update the corresponding draft framework that is presented in this journal Supplement. Generally, workshop participants found it premature, or even unfeasible, to provide specific guidance on how to conduct the different steps of data processing. Instead, the imminent value of presenting a framework for reporting the specific steps of ‘omics data processing used in a particular study was highlighted and proposed as a generic ‘Omics Reporting Framework. Initially focusing on data processing, the ‘Omics Reporting Framework should eventually include parameters for experimental design, and, in the long term, it might be expanded to cover all types of ‘omics (e.g., transcriptomics, proteomics, etc.) and the related technologies (e.g., microarrays, RNA-seq, etc.). The ‘Omics Reporting Framework should be organised flexibly to allow for the inclusion of new evidence, and new methods or new technologies, as they become available. Adherence to the ‘Omics Reporting Framework would provide transparency on the data processing and evidence that best practice had been followed, reflecting the current maturity of the technology and further taking into consideration the complexity and diversity of ‘omics technologies. Compliance with the steps of the ‘Omics Reporting Framework would enable the generation of reproducible ‘omics data, which would facilitate their regulatory applicability. Also, a Topical Scientific Workshop on New Approach Methodologies (that include ‘omics-based studies) organised by the European Chemicals Agency (ECHA, 2016) came to the conclusion that reporting templates were required for such methodologies to encourage their use for regulatory purposes.

As such, issues on the reproducibility of ‘omics data could relate to between-study reproducibility as well as to the reproducibility of the analysis of a given set of raw data. In the future, the ‘Omics Reporting Framework could also provide the basis for the development of technology-specific analysis frameworks, in which process the vendors of the technologies should be involved. Case studies, using reference samples, and possibly also validation studies, should substantiate the appropriateness of the steps of the ‘Omics Reporting Framework.

In wrapping up work stream 2, Weida Tong and Tim Gant expressed the wish that the ‘Omics Reporting Framework would also be beneficial for the work of the OECD AOP programme to ensure that all newly developed AOPs utilising ‘omics data are founded on transparently collected, reproducible data sets. The ‘Omics Reporting Framework would be flexible and sufficiently general to accommodate future advancements and new technologies, while also being sufficiently specific to be useful.

Work stream 3: Best practices for identifying biological pathways and developing AOPs to connect ‘omics data to the phenotype

Chair: Alan Poole (ECETOC, Belgium)

Work stream 3 covered the first of two topics addressing the interpretation of ‘omics data, i.e. best practices for establishing biological pathways to connect ‘omics data to the phenotype.

Presentations Work Stream 3:

Beatriz Silva Lima (Faculty of Pharmacy, Lisbon University, Portugal) presented lessons learned from the Health and Environmental Sciences Institute (HESI) framework for the improved integrated and harmonised regulatory application of non-animal alternative methods in safety assessments. Established in 2014 as a multi-stakeholder forum encompassing more than 60 scientists, the HESI framework aims to define integration criteria to be used in assessing the fitness-for-purpose of test methods and testing approaches for regulatory decision-making. The HESI framework encompasses three modules, i.e. (i) assay performance; (ii) prediction model performance; and (iii) utilisation of the prediction model. The HESI framework is especially helpful in assessing IATA and ITS data. IATAs and ITSs are gaining increasing importance in regulatory toxicology since animal tests generally cannot be replaced one-by-one by single non-animal methods. Test methods addressing specific key events of an AOP, rather than entire toxicological endpoints, have to be assessed against the specific biological alterations rather than against apical effects. To achieve regulatory acceptance of IATAs and ITSs, a period during which both the traditional animal tests and the new testing approaches are applied in parallel and then compared, may prove beneficial. Further, it was discussed that the GHS / CLP provisions on the C&L of substances might make it difficult to apply criteria based on mechanistic information (that formed the basis for IATAs and ITSs) in regulatory toxicology, rather than apical endpoint data.

Kamin Johnson (Dow Chemical Company, USA) presented a preliminary proposal to pivot from employing ‘omics data for hazard identification to using them to generate bioactivity-based PoDs for the risk assessment of substances. This tiered approach is intended to facilitate the application of ‘omics data, not only to investigate MoAs, but also to derive PoDs for regulatory hazard and risk assessment. Since prevailing knowledge gaps generally make it difficult to link ‘omics data to a specific apical effect, the goal to identify an apical effect might be replaced by the goal to determine bioactivity, i.e. the perturbation of a biological pathway at the molecular level. If no bioactivity is observed, there should be no hazard. In a future paradigm, in which bioactivity in combination with exposure constitutes risk, PoDs could be based upon the benchmark dose that affects the most sensitive biological pathway(s) (Thomas et al., 2013a; Farmahin et al., 2015). A tiered approach for a bioactivity-based risk assessment is envisioned in which substances with potentially greater health risk (exemplified by human health assessment) are tested against increasingly complex biological systems:

• Tier 0: Determination of in vitro bioactivity PoD, e.g., using in vitro assays from the ToxCast™ program (Judson et al., 2014) or an ‘omics-based analysis of one or more appropriate in vitro models. Using predictions of human exposure and internal dosimetry, the in vitro PoD is used to calculate a margin-of-exposure (MoE; Benford et al., 2010), and substances with a Tier 0 MoE below a specific threshold value are submitted to Tier 1.

• Tier 1: Determination of in vivo PoD, using ‘omics data recorded for all relevant organs in a short-term repeated-dose toxicity study. This in vivo PoD is combined with predictions of human exposure and internal dosimetry to generate a MoE, and substances with a Tier 1 MoE below a specific threshold are submitted to Tier 2

• Tier 2: Generation of bioactivity PoD, using ‘omics data recorded for all relevant organs in an in vivo repeated-dose toxicity study encompassing all possible critical routes of exposure (e.g. an extended one generation-like study).

When applying such a bioactivity-based risk assessment approach, it will have to be considered whether the recorded bioactivity could ultimately lead to an apical effect. However, the limited data generated to date comparing ‘omics and apical PoDs for cancer endpoints suggest that these PoDs lie within an order of magnitude (Bercu et al., 2010; Thomas et al., 2013a, b; Jackson et al., 2014). Also in a comparative assessment of 104 cases, metabolomics profiling showed a similar sensitivity as the classical 28-day rat oral toxicity study conducted in accordance with OECD Test Guideline 407 (van Ravenzwaay et al., 2014). With respect to the interpretation of in vitro data, the different sensitivities of different cell lines might need to be considered. Workshop participants suggested that the biggest hurdle to using a bioactivity-based approach for risk assessment might be the hazard-based GHS / CLP provisions for the C&L of substances used to communicate toxicological effects.

Richard Meehan (MRC Human Genetics Unit, Western General Hospital, University of Edinburgh, Scotland) presented preliminary results from studies using liver tissues from rats and mice (handled in the same way as control groups in 28-day oral toxicity studies complying with OECD Test Guideline 407) to establish rodent liver epigenome ground states, i.e. ‘normality’ (European Chemical Industry Council (CEFIC) Long-range Research Initiative (LRI) project C3). Epigenetics reflect changes in gene activity in the absence of altered DNA sequences, and these are indicative of specific cell states. During toxicity testing, the vast majority of epigenetic alterations are reversible once the test substance is no longer applied. Hence, those epigenetic alterations that prevail might be toxicologically relevant and have the potential to enhance integrated pathway analysis in elucidating a substance’s MoA.

Summary of Breakout Session Work Stream 3:

The subsequent breakout session Connecting results of ‘omics data to phenotype did not serve to update a previously drafted framework document. Therefore, the following paragraphs (structured in accordance with the questions posed to the breakout groups) summarise important results from this session and the subsequent plenary discussion. This summary focuses on transcriptomics studies, where data interpretation begins with the selection of DEGs.

How can gene expression analysis move away from using arbitrary fold change thresholds to measures that take appropriate account of the dynamic expression ranges of different genes?

Frequently, DEGs are selected by assessing fold changes using thresholds that were set arbitrarily, i.e. without specific scientific justification. Taking into account the toxicological context under investigation, this approach has proven useful, since fold change thresholds, just as p-values, constitute easily applicable filters to condense complex data sets. However, from a biological point of view, applying fold change thresholds may prevent relevant biological effects from being observed. In this regard, the biological evaluation of small gene expression changes may be challenging.

The application of multiple testing adjustment factors may enable more a stringent statistical filtering of DEGs, than the use of fold change thresholds. Further alternative approaches to avoid using arbitrary fold change thresholds may be pathway- or pattern-matching approaches, applying read-across from common databases to match the generated data. Such approaches consider the rank orders and the direction of gene expression changes (ECETOC, 2008), but not their amplitude or magnitude. Pathway- or pattern-matching approaches include a probability analysis of the similarity of the gene expression profiles, which can either be based on comparative techniques (using, e.g., a pre-defined predictive gene set or a rank-based correlation analysis) or on multivariate analyses (e.g., principal component analyses or clustering techniques) (ECETOC, 2008). Further approaches to avoid fold change thresholds involve the use of generalised linear models, e.g. the benchmark dose modelling of the dose-dependence of effects (Thomas et al., 2013a; Farmahin et al., 2016). An orthogonal approach avoiding fold change thresholds uses the correlation structure among genes to reconstruct networks, to identify modules within these networks, to relate modules to variables of interest (e.g., treatment or dose), and to select candidate genes in interesting modules by their network properties (e.g., association with the module or hub-gene-like properties; i.e. genes with high whole-network connectivity (Langfelder et al., 2013)).

How can irrelevant changes in gene expression be distinguished from meaningful ones?

Generally, gene expression changes should be evaluated for their biological relevance, statistical significance, and technical reliability and reproducibility. The qualitative and quantitative distinction between changes in gene expression that do or do not result in phenotypic alterations depends on the biological function of the respective gene and its overall effect on the biological pathway or network that it is part of as well as the type of cell under investigation and the magnitude of effects. In in vivo studies, the life stage of the animals and the time point of sampling may also affect the levels of gene expression changes (ECETOC, 2010). An understanding of physiological processes and pathways allows placing gene expression changes into their biological context. Using pathway databases, recurrent changes that are not of toxicological interest can be identified and sorted out (ECETOC, 2013). Pathway experts, preferably collaborating in multi-disciplinary teams, should interpret the toxicological relevance of gene expression changes and the coherence of pathway results.

Further, the interrelation of genes, just as baseline (or background) levels of gene expression that are unrelated to the exposure to the given substance under investigation, should be taken into consideration in determining the toxicological relevance of gene expression changes (ECETOC, 2010). These parameters may be accounted for by including appropriate controls in the study and by using gene expression pattern-based analyses and benchmark dose modelling of dose-response relationships.

Are changes in the expression of a single gene meaningful? Or should all gene expression changes be considered as a part of a biological pathway?

The altered expression of a single gene may or may not be biologically relevant. If a specific gene encodes a rate-limiting factor, its altered expression can have major implications for the phenotype. In this case, this particular gene can constitute a biomarker of an entire biological pathway. Consideration of the AOP concept to identify genes and gene products that represent the MIE or key events of the AOP, or that are considered relevant proxies for one or more events along the AOP continuum, may facilitate an evidence-based approach to identify relevant gene(s) (cf. also Section below on the mapping of ‘omics changes to AOPs). However, individual genes may operate in more than one biological pathway, and, generally, substance administration is likely to affect multiple biological pathways. In interpreting the toxicological implications of changes in the expression of individual genes, further mechanistic understanding of cell plasticity, i.e. dynamic changes in gene expression, is required.

Regardless of the possible biological relevance of single gene expression changes, it was agreed that a combined evaluation of gene expression changes across different levels of biological organisation using pathway- or pattern-matching approaches was much more relevant for regulatory toxicology. It was considered very unlikely that a regulatory decision would be made on the altered expression of a single gene alone.

Can ‘omics changes be mapped to AOPs and, if so, how will this information be used?

Generally, the effects of gene expression changes on the phenotype evolve from alterations in single cell types, to changes in organs and tissues, and, ultimately, in the entire organism. Cell type-specific evaluations of gene expression changes may be of interest to identify specific biomarkers or the MIE, or other key events, for specific AOPs. Such evaluations should take into account if effects are exposure-specific or time point-specific.

As knowledge increases, gene expression changes can increasingly be mapped to AOPs (ECETOC, 2013). Examples are the key events for oxidative stress or cholestasis (cf. list of OECD-approved AOPs, available at https://AOPwiki.org/oecd_page; accessed 09 May 2017). Mechanistic information provided by ‘omics technologies can also be useful to amend already established AOPs or to identify connections between different AOPs.

AOPs may assist in identifying the relevance of those biological pathways that are affected for one or more adverse outcomes. For this purpose, multiple AOP information (AOP networks), captured in knowledge bases, such as the AOP-wiki (www.AOPwiki.org), should be utilised. While most currently available AOPs describe linear sequences of events, AOP networks can be developed to represent interlinked AOPs (Angrish et al., 2016), and such networks may also contain feedback loops. To enable a more effective use of the associated ‘omics data, interrelationships between AOPs must be more fully developed, and a comprehensive understanding of AOPs and the MoAs of substances is required. In this regard, different biological pathways may represent different levels of molecular, cellular, or biological organisation, and some biological pathways are closer to apical effects than others. Therefore, the integration of different ‘omics measurements (e.g. to determine the transcriptome, metabolome, and proteome) may be necessary to capture all aspects of an AOP.

Knowledge on AOPs can support read-across and substantiate the human health relevance of observed apical effects (as well as the relevance of specific ITSs). For AOPs to be applicable to hazard and risk assessment, quantitative measurements are required. For instance, MIEs or any other key events (further taking into consideration species- and dose level-specificity) can be useful to establish a threshold for an apical effect. Recent advances such as introduction of quantitative AOPs (Conolly et al., 2017) and comprehensive WoE evaluation of the individual steps of the AOP using modified or tailored Bradford-Hill criteria (Meek et al., 2014; Becker et al., 2015) will likely facilitate larger implementation of AOPs within regulatory approaches.

Workshop participants agreed that it was a promising approach to link gene expression changes and pathway perturbations to the phenotype by mapping them to specific AOPs. However, other approaches were also discussed, such as the mapping of gene expression changes with signatures of toxicity.

Can genes or biological pathways be identified where there is generally a good linkage to phenotypic changes and ones where the linkage is not very good?

Using phenotypic anchoring approaches, changes in annotated genes and curated pathways that carry out specific biological functions can be linked to functional alterations at the phenotypic level. While changes in RNA levels are often not accompanied by cellular or phenotypic alterations, a number of genes and biological pathways can already be linked to phenotypic changes. Examples include transcriptional changes that are mechanistically linked to phthalate ester-induced testicular dysgenesis in rats (Liu et al., 2005) and changes in the uterus during the oestrus cycle, the expression of cytochrome P450 gene inducer effects (e.g. leading to an increase in metabolic activity), or exposure to heavy metals and subsequent metallothionein induction (Fabbri et al., 2012). Again, expert knowledge is required to establish such links, in addition to distinguishing between adaptive and adverse effects. At best, such considerations should address by how many orders of magnitude adaptive and adverse effects differ, and they should take into account the dose- and time-dependence of the transition from normal variability, through adaptive response, to adverse effect (ECETOC, 2010). Aggregate measures may be required for biological pathways to be able to assess the orders of magnitude difference between adaptive and adverse effects. However, it is not always possible to establish quantitative relationships between gene expression changes and phenotypic changes. Generally, it was considered difficult to determine the human health relevance of ‘omics data (just as other toxicological data) obtained under very controlled study conditions at present.

Can expression changes be used as a basis for establishing safe levels of exposure? If so, how should the dose-response relationship be assessed? How can expression changes be used to obtain a PoD?

The workshop participants agreed that further work is necessary before gene expression changes can be used as a basis for establishing safe levels of exposure and that the present workshop provided important incentives towards achieving this goal. Research work must be conducted on a case-by-case basis to identify potentially relevant biological pathways (or genes) that are prognostic of specific apical effects. Biological pathway-specific benchmark dose modelling, considering in vivo data, shows promise for the assessment of dose-response relationships to determine PoDs. However, such approaches remain to be validated, and criteria have to be identified to conduct such validation studies. To use ‘omics to derive a PoD, it should be evident that the identified biological pathway is causally related to an adverse effect (ECETOC, 2008, 2010).

Importantly, the applicability of ‘omics-based PoDs depends on the intended use within regulatory toxicology (e.g. to prioritise substances for hazard assessment or to exclude the need for further testing). In most cases, ‘omics data are not being used on their own, but embedded in a tiered process. Thus, frameworks are needed to enable the integration of ‘omics data into lines of evidence, e.g. by applying the quantitative WoE approach presented in work stream 4.

In wrapping up work stream 3, Alan Poole emphasised the need to discriminate whether ‘omics-based studies were intended to be used to supplement toxicity studies or to supplant toxicity studies. In the short-term, it might be more likely that ‘omics data would be used to improve a mechanistic understanding of AOPs and to fill data gaps in incomplete lines of evidence when applying read-across.

Work Stream 4: Best practices for WoE approaches for integrating ‘omics-based data

Chair: Alan Poole (ECETOC, Belgium)

Work stream 4 covered the second of two topics addressing the interpretation of ‘omics data, i.e. the ECETOC framework for applying ‘omics data using quantitative WoE (qWoE) approaches that is presented in detail in the corresponding article of this journal Supplement.

Presentations Work Stream 4:

Mark Pemberton (Systox Ltd., UK; ECETOC Scientific Committee) and Jim Bridges (Surrey University, UK), in two video presentations, provided further details on the utilisation and integration of ‘omics data within qWoE approaches.

A qWoE approach is defined as the identification, objective analysis and scoring (using predefined scientifically justified criteria) of all potentially relevant studies, for both their reliability and relevance in testing a hypothesis. Accordingly, the steps of a qWoE approach comprise the formulation of the hypothesis and data collection; the assessment of the reliability and relevance of the studies; the identification of critical endpoints; the evaluation of the concordance of findings both within and between lines of evidence; comparisons with findings for related substances; and, finally, the conclusion and the characterisation of uncertainties. The draft qWoE framework is applicable to any hypothesis, e.g. both to identify MoAs and to determine PoDs for subsequent hazard assessment. By providing transparency in the way that data are evaluated and integrated, it minimises the opportunity for bias and provides a basis for dialogue between stakeholders. When applying the qWoE framework to ‘omics data, their relative weighting as compared to traditional toxicological data or within a set of ‘omics data has to be taken into consideration.

Amber Goetz (Syngenta Crop Protection LLC, USA) presented a case study to demonstrate how ‘omics data could support WoE approaches when assessing the human health relevance of experimental findings. The hypothesis tested in this study was that the hepatocarcinogenic MoA of the fungicide Sedaxane is initiated by activation of the constitutive activated receptor (CAR; Oshida et al., 2015). The study design encompassed oral 22-day toxicity studies using male CD-1 mice and in vitro studies using primary hepatocytes from rats and humans. Organ weight changes, liver enzyme activation, CAR3 transactivation and pregnane X receptor (PXR) activation were assessed. Gene expression changes were evaluated using qRT-PCR and Agilent microarray technologies. The microarray data were analysed using the GeneSpring™ 13 software and applying >1.5 fold-change and <0.05 p-value thresholds. Bioinformatics analyses were conducted using Ingenuity Pathways Analysis (IPA™) applying a 2-way ANOVA. Multiple testing correction was performed using Benjamini-Hochberg false discovery rate. The analysis showed that the expression of xenobiotic metabolising enzyme genes and other genes associated with CAR and/or PXR activation increased in a dose-dependent manner. Changes were consistent with increased liver cell proliferation observed at higher doses of Sedaxane. Based upon the genomic alterations, lipid metabolism pathways, but not inflammatory pathways, appeared to be affected. The findings from the study confirmed the hypothesis that Sedaxane can induce liver tumours in mice by CAR activation.

The case study provides a pragmatic example of the application of ‘omics tools in regulatory evaluations. The subsequent discussion highlighted some of the challenges involved in the conduct and interpretation of gene expression studies including the selection of appropriate time points for sample collection, the high degree of sample-to-sample variability, the evaluation of heterogeneous cellular populations within tissues, and the distinction between adaptive and adverse effects.

With respect to the high degree of within-group variability (i.e. ‘noise’) of ‘omics studies, it was noted that such ‘noise’ was observed for samples from test groups and control groups alike. The ‘noise’ was speculated to be partly caused by sample-to-sample variability from a heterogeneous cell population representative of the measured tissue sample. The evaluation of single cell types or even single cells might serve to reduce variability, thereby making it possible to observe significant changes in gene expression also at very low substance concentrations. While investigations on single cells might be difficult to conduct, in situ hybridisation or RNA-seq are practicable tools for such investigations.

Summary of Breakout Session Work Stream 4:

The subsequent breakout session Feedback on best practices for WoE approaches for integrating ‘omics data served to update the qWoE framework presented in this journal Supplement. The workshop participants agreed that a qWoE framework would serve to improve the transparency and reproducibility of decisions derived from WoE approaches. Therefore, it constitutes a useful decision-making tool for regulatory science when assessing the quality of different types of data and integrating them for hazard and risk assessment (Roth and Ciffroy, 2016). Similar approaches have been proposed both for the MoA and AOP frameworks using modified or tailored Bradford-Hill considerations (Meek et al., 2014; Becker et al., 2015).

Importantly, the qWoE framework should not become a ‘new check box approach’, just as the application of any scoring system should not override case-by-case expert judgment during hazard and risk assessment. For this reason, it was also suggested that it might be more appropriate to name the framework ‘semi-quantitative’ WoE approach. Case studies would serve to assess how consideration of traditional and ‘omics-based data (either in combination or each on their own) within the qWoE affected the outcome of hazard and risk assessment. Such case studies would have to distinguish between the use of ‘omics data to determine MoAs and to determine PoDs.

In wrapping up work stream 4, Alan Poole emphasised that the qWoE framework was intended to ensure transparency of the process, but not to place undue weight on any given numerical score. While irrelevant studies could always be filtered out, an unsubstantiated focus on any single study outcome would be prevented by applying the qWoE framework.

Close of the workshop

Closing the workshop, Alan Poole thanked all participants for their valuable contributions to achieve best practice for the acquisition, storage, processing, and interpretation of ‘omics data. After the workshop, the draft frameworks were amended, taking into account the recommendations from the breakout sessions and plenary discussions. The frameworks for a GLP-like context for collecting, storing and curating data; for the processing of ‘omics data; and for qWoE for integrating ‘omics data were conceived as reporting templates to enhance the standardisation, transparency and reproducibility of ‘omics studies. Overall, this work is closely linked to the activities of the OECD EAGMST to achieve the ultimate goal of regulatory acceptance and use of ‘omics studies.

Box: Definition of biological pathways and adverse outcome pathways.

Adverse outcome pathway (AOP):

A linear sequence of key events from a molecular initiating event (MIE) to an adverse (toxic) effect at the individual level (for human health) or population level (for environmental health). Key events in an AOP are scientifically defensible events that are necessary, but not always sufficient, for an adverse outcome. AOPs should be definable, measureable, and plausible, and they should provide a causal link between the events occurring from the initial interaction of a substance with its molecular target (i.e. the MIE) and the adverse outcome (Ankley et al., 2010; OECD, 2012, 2013). Unlike modes-of-action (see below), AOPs are substance-agnostic and not species-specific, and thus do not consider factors that are directly related to the exposure to a given substance, e.g., absorption, distribution, metabolism, and excretion (ECETOC, 2017).

Mode-of-action (MoA):

The biologically plausible sequence of substance-specific key events that are specific to the exposure to a given substance, starting with the definition of the substance and proceeding through the interaction of the substance or its metabolites with a cell, to functional and anatomical changes leading to an observed effect supported by robust experimental observations and mechanistic data (Sonich-Mullin et al., 2001; Boobis et al., 2009; Fenner-Crisp and Dellarco, 2016).

Biological (or biochemical) pathway:

There is no universal definition of a biological (or biochemical) pathway (ECETOC, 2013). Throughout this workshop report, the following definition is used: A series of molecular events occurring in a cell (or extracellularly) that leads to a certain product or an intra- or extracellular alteration. A pathway can, for example, trigger the assembly of new molecules, and will thus include direct substance-target interactions, cellular signalling and cellular regulatory processes (adapted from: US National Human Genome Research Institute; cf. https://www.genome.gov/27530687/biological-pathways-fact-sheet/).