PRACTICAL APPROACHES TO ADVERSE OUTCOME PATHWAY (AOP) DEVELOPMENT AND WEIGHT OF EVIDENCE EVALUATION AS ILLUSTRATED BY ECOTOXICOLOGICAL CASE STUDIES

Abstract

Adverse Outcome Pathways (AOPs) describe toxicant effects as a sequential chain of causally linked events beginning with a molecular perturbation and culminating in an adverse outcome at an individual or population level. Strategies for developing AOPs are still evolving and depend largely on the intended use or motivation for development and data availability. Four ecotoxicological AOP case studies, developed for different purposes, are described herein. In each situation, creation of the AOP began in a manner determined by the initial motivation for its creation, and expanded either to include additional components of the pathway, or to address the domains of applicability in terms of chemical initiators, susceptible species, life stages, etc. From these case studies, some general strategies can be gleaned which a developer may find useful for supporting an existing AOP or creating a new one. Several web-based tools which can aid in AOP assembly and evaluation of weight of evidence for scientific robustness of AOP components are highlighted.

INTRO

The adverse outcome pathway (AOP) framework is a systematic, transparent approach used to organize existing toxicological knowledge and translate mechanistic information to adverse effects at higher levels of organization based on causal relationships between endpoints [1]. As such, the AOP framework has been proposed as a means to relate alternative types of data (in silico, in vitro and in vivo biomarker-type data) to endpoints of concern to chemical risk assessors (e.g., survival, growth and reproduction). This application in the regulatory field is timely, given the current momentum to move toxicology from largely empirical in vivo assessments of single chemical effects to predictive approaches which make use of newer tools and alternative data [2]. The framework, likewise, can serve as a retrospective tool, in that it can be used to deduce which chemicals may be associated with adverse effects of complex mixtures [3]. Other proposed uses include: a means to improve the mechanistic understanding of chronic toxicity in ecotoxicology, the identification of data uncertainties and associated research needs, and the ability to guide appropriate extrapolation of chemical effects across species [4]. Given this broad potential utility, since its introduction in 2010, rapid progress has been made towards accepting and implementing the AOP framework [1, 5]. As noted by Groh et al. [4], this progress is evidenced, for example, by the Organization for Economic Cooperation and Development’s (OECD) AOP development program and the creation of an AOP repository in a publicly-accessible knowledgebase (AOP-wiki; https://aopwiki.org). In addition, a chemical exposure framework, the Aggregate Exposure Pathway (AEP), has recently been proposed which parallels the AOP structure with the intention of aiding the integration of exposure considerations with the hazard/effects-driven emphasis of AOPs [6].

The structure of an individual AOP is unidirectional, sequential, and modular (Figure 1). The AOP begins with a molecular initiating event (MIE), which describes the interaction of a chemical initiator with a biomolecule (molecular target). The MIE causes downstream, observable (measurable) key event (KE) perturbations. These KEs typically describe effects at increasing levels of organization (e.g., cell → tissue → organ), culminating in one or more adverse outcomes (AOs) at the individual, population, or ecosystem level. The MIE and AO are considered special types of KEs. Between adjacent or non-adjacent KEs in the sequence, a key event relationship (KER) describes or models how the severity of perturbation in the upstream event relates to the outcome at the downstream event. Because all events are modular, they may be shared by multiple AOPs, forming networks [7, 8]. Importantly, AOPs are not chemical specific, but describe how an explicit biological perturbation by any stressor may lead to an adverse effect at the individual or population level; thus, there may be numerous chemical initiators for any given AOP. The modular nature of the AOP framework is fully implemented within the AOP-wiki. Each AOP description is captured as a separate AOP page with links to chemical initiator, KE and KER pages. Each of these pages exist as independent entities within the AOP-wiki. Furthermore, each AOP page representing an organized, sequential assemblage of KE and KER pages, includes a network view of its component KEs which are shared with other AOPs.

Figure 1.

Diagram of the general AOP construct. The Molecular Initiating Event (MIE) is linked to intermediary Key Events (KE) and the Adverse Outcomes (AO) via Key Event Relationships (KER).

Guidance on how to develop an AOP description is available from several sources. For example, the OECD has published general instructions for describing AOPs [9], and Villeneuve et al. [8] introduced several development strategies with illustrative examples. However, as AOP development experience is gained, deliberations by the OECD Extended Advisory Group on Molecular Screening and Toxicogenomics (EAGMST), which oversees the AOP development program, acknowledge the need for on-going revisions to the current guidance. For example, while an individual AOP was, and remains, defined as a single, non-branching, sequence of KEs linked by KERs [8], it has recently been recognized that there are cases where representation of a simple network of AOPs on an AOP description page in the wiki may be a more pragmatic approach to capturing and evaluating the support for an overall link between the MIE (or other upstream KEs) and the AO. This is particularly the case where a MIE (or early upstream KE) can trigger multiple KEs which, in all likelihood, contribute concurrently and collectively to the same AO, and cannot be resolved from one another temporally, or isolated experimentally to determine whether one event can drive the downstream response in the other(s). In such cases, including multiple concurrent and parallel KEs that can be shown to contribute causally to the AO strengthens the support for the pathway.

One specific situation related to that described above is the need to describe mutually-enhancing, temporally indistinguishable KEs which result in positive feedback loops, or a KER appearing as a “double” arrow. For example, inflammation within a tissue (KE2) can cause degeneration of the tissue (KE3). This degeneration, in turn, can trigger or enhance inflammation, so the relationship could also be represented as degeneration of the tissue (KE3) leading to inflammation (KE2). Both representations may be biologically accurate, as the two events are occurring simultaneously and either measureable phenomenon could be viewed as cause or effect (i.e., they exacerbate one another). Two separate, linear AOPs could be described in the AOP-wiki, but a double arrow would appear between the inflammation and the tissue degeneration KEs once applied as a network view. The arrow does not imply reversibility of the pathway, but rather an inability to temporally or mechanistically distinguish the KEs. In such a case, the entry of two completely separate AOPs is not a pragmatic solution.

With this in mind, many of the AOPs discussed in the present paper are depicted in a network context rather than as a complete series of parallel, linear AOPs. This approach provides the most pragmatic level of development for these cases and the most effective road-map to guide the reader through the discussion, as opposed to showing many repeated, modular elements. Importantly, while we present and discuss the information as simple AOP networks, all the information remains structured in a manner that is consistent with the guiding principles of AOP development. All KEs and KERs are described as separate entities (pages) within the AOP-wiki and can be re-used within other AOPs or AOP networks.

The process of describing an AOP is flexible to allow for the different availability of supporting data, developer expertise, and/or the intended application(s) of the AOP. The diverse approaches that may be utilized can yield useful AOPs of varying complexity [10]. Indeed, as more toxicological information is captured in an AOP format, a broad spectrum of AOPs with different degrees of empirical support have emerged. For example, some AOPs may exist as simple box and arrow diagrams based on biological plausibility for which little or no supporting evidence has been assembled in the wiki (e.g., https://aopwiki.org/wiki/index.php/Aop:14; Aop:36; Aop:96). At the other end of the spectrum are AOPs with robust weights of evidence which may be used for predicting risk (e.g., [11]). In the latter case, there is a high degree of confidence in the AOP, as determined through application of the modified Bradford-Hill (BH) considerations described by Becker et al. [12]. The biological plausibility of the KERs, essentiality of KEs, and empirical support (dose-response, temporality, and incidence) for KERs are considered, with plausibility being weighted most heavily, followed by evidence for essentiality and finally empirical support [9, 12]. Nonetheless, despite growing interest in AOP development, there currently are few accounts describing the development process, and fewer detailing how a developer may use various resources to help derive AOPs in a manner consistent with the BH considerations.

The intent of this article is to illustrate the process of developing an AOP using four case studies. The initial motivation for each case study, how it influenced where the AOP development began (its genesis), and how the AOP was subsequently expanded are described. From these examples, general themes that we and others in the field have found useful are discussed. These themes provide a basis for more detailed guidance for efficient derivation and evaluation of AOPs. In particular, several web-based resources which can aid the process of AOP development are described.

AOP DEVELOPMENT APPROACHES: ECOTOXICOLOGICAL CASE EXAMPLES

Provided below are four examples of ecotoxicological AOPs initiated for different purposes and developed using varied approaches. Ecologically-focused AOPs can be somewhat unique from those developed for human health risk assessment in several respects. First, the emphasis in terms of an AO typically is at the population rather than the individual level. Second, there often is a desire/need to extrapolate AOPs developed in one (or a few) model species to many species. Third, available empirical data (physiological or toxicity) may be quite limited. Nonetheless, many of the concepts and tools described herein are applicable to AOP development for either human health or ecological risk assessment. The examples presented here, in addition to most AOPs under consideration by the OECD, are available in the AOP knowledgebase: https://aopwiki.org; AOP and KE numbers noted below reflect entries within this database. For simplicity, KER numbers (direct and indirect) are not displayed in the AOP figures, however, the web addresses for each AOP, its corresponding KEs and KERs are provided in the Supplementary data, Table S1.

CASE STUDY 1: ECDYSONE RECEPTOR (ECR) ACTIVATION LEADING TO MORTALITY

Motivation: Develop an AOP using a prototypical toxicant with a known mode of action (MOA) in a target species for subsequent use in chemical screening and to predict effects in non-target species

The insecticide tebufenozide (TEB, or RH-5992) is a potent non-steroidal ecdysone receptor (EcR) agonist used to control pest insects (e.g., Lepidoptera) by interfering with molting in order to cause mortality in developing juveniles [13, 14]. However, ecdysteroid (Ec) signaling via modulation of EcR activity regulates development, reproduction and survival in a wide range of arthropods. Other chemicals that impact Ec signaling also have potential to interfere with molting and survival, thereby causing unintended adverse effects in pest and non-pest species of arthropods alike. Case study 1 provides an example in which AOP development was initiated based on the well-studied MoA (i.e., EcR agonism) of Ec agonists, such as TEB, on a target species. Predictive approaches along with experimental efforts were employed to address the relevance of this AOP for non-target species and other potential EcR agonists, expanding the domains of chemical and taxonomic applicability. Further, subsequent interrogation of available data was performed to identify novel and relevant KEs occurring post-EcR activation. These KEs provide additional measurement endpoints that might be employed in a tiered testing strategy aimed at assessing potential risks associated with chemicals.

Genesis

Development of this AOP began with attaining a fundamental biological understanding of the role of Ec in the regulation of molting, primarily via review of the extensive data available for Ec agonists in arthropods. In insects and crustaceans, the ligand binding pocket of the EcR is flexible and displays binding affinity for a number of steroidal and non-steroidal ligands [14, 15]. In both insects and crustaceans, the endogenous Ec hormone (20-hydroxyecdysone; 20E) directly influences molting; substantial increases during the inter-molt period followed by a decrease shortly before the next molting cycle are required for successful molting [16, 17]. Comparative analyses also demonstrate large similarities in the toxicological responses of 20E and xenobiotics such as TEB in insects [13, 18, 19], suggesting that interaction with the EcR may be the key indicator of downstream events and an appropriate MIE for an AOP describing death in arthropods due to dysregulation of molting (Aop:4; Figure 2).

Figure 2.

Putative AOP describing ecdysone receptor (EcR) activation leading to organism mortality and population decline by EcR agonists (AOP: 4). Key event numbers reflect page numbers in the AOP Wiki (https://aopwiki.org).

The downstream signaling and metabolic processes regulated by EcR can be rather complex. An elevation in EcR signaling promotes arthropod molting preparations by up-regulating genes and enzymes involved in the processes of new cuticle formation (e.g., chitin synthases) [18, 20–22], old cuticle degradation (e.g. chitinases and chitobiase) [23, 24], and synthesis of peptide hormones such as the ecdysis triggering hormone (ETH) to regulate the body contractions of the old cuticle [16, 25]. A drop in EcR activity may be required to trigger subsequent physical escape from the old cuticle. A number of EcR agonists, including TEB, suppress ETH secretion (Event: 988) and muscle contractions (Event: 993) that are necessary for cuticle shedding [25, 26], and thus cause molting disruption in insects. The organisms exposed to these EcR agonists fail to shed their old exoskeletons during molting (i.e., incomplete ecdysis; Event: 990), and suffer from increased mortality (Event: 351) due to a combination of physiological abnormalities and reduced feeding. This molting defect-associated AO has been widely reported in insects such as the spruce budworm (Choristoneura fumiferana), cotton leafworm (Spodoptera littoralis) and tobacco hornworm (Manduca sexta) [13, 24, 27], and occasionally reported in crustaceans such as the waterflea (Daphnia magna), crab (Rhithropanopeus harrisii) and barnacle (Balanus amphitrite) [28, 29] for both endogenous (e.g., 20E) and exogenous (e.g., TEB) EcR agonists. Individual mortality may potentially affect the population size, and ultimately cause population decline.

Expansion

While our fundamental understanding of the biology and empirical results for Ec agonists was sufficient to formulate pathway KEs, a major interest was defining the broader chemical and taxonomic domains for which this AOP was likely to be applicable. First, to better define the chemical domain of applicability beyond classical Ec agonists (such as 20E and TEB), computational (in silico) and in vitro screening approaches were employed. The majority of data used for computational predictions originate from studies with insects, although studies with non-target species such as crustaceans are slowly increasing in number. Several in silico approaches identifying EcR active compounds (structure-activity relationships [SARs], quantitative structure-activity relationships [QSARs], and identification of structural alerts) were used to assist category formation and facilitate read-across. These models typically identify compounds that have structural similarities to well-characterized EcR agonists (and/or antagonists), and hence are likely to bind to the EcR. Recent work by Mellor et al [30] specifically demonstrated application of a data analytics platform (KNIME; www.knime.org) to identify EcR-active compounds in D. magna, where as many as 32 putative active structures out of 7678 unique chemicals were found to be potential ligands for the EcR. Advanced predictive approaches involving three dimensional (3D) models developed from homologous EcR protein sequences in insects and daphnids, have further enhanced our knowledge by identifying compounds with affinity for the ligand binding sites of EcR using a flexible docking approach [31]. The resulting model enabled generation of a library of chemical structures potentially capable of interacting with the EcR, and prediction of the potency of putative environmental EcR ligands. These in silico predictions are currently being evaluated on the basis of reported effect concentrations derived from EcR in vitro transactivation assays and in vivo studies focusing on molting and/or developmental effects in arthropods. A number of compounds including 20E, ponasterone A (PoA), muristerone A, turkesterone, TEB, RH-5849, methoxyfenozide (RH-2485) and halofenozide (RH-0345) bind to the EcR in a number of insect species [32, 33] and some of these compounds (e.g., 20E, PoA, TEB) were also reported to act as EcR agonists in a few crustacean species [14]. Others are being confirmed in the competitive receptor protein binding bioassay (e.g., [32]).

Complementary to the modeling approach used to predict EcR ligands, the conservation of the ligand binding domain was also evaluated across species as a means to understand the taxonomic domain of applicability for this MIE. Sequence Alignment to Predict Across Species Susceptibility (SeqAPASS; https://seqapass.epa.gov/seqapass) is a web-based tool which uses available molecular target information to evaluate conservation among taxa in terms of protein sequence similarity using three levels of analyses: 1) the primary amino acid sequence (and ortholog detection), 2) functional domain, and, where applicable, 3) key individual amino acid residues [34]. Previously, these three levels of analyses were used to compare tobacco budworm (a known target species of TEB) EcR across species to identify other potentially susceptible taxa [35]. Based on sequence/structural similarity of the EcR, these evaluations suggested that species in the Insecta (insects), Malacostraca (prawn, lobster, crab), Branchiopoda (water flea), Arachnida (spider, tick, scorpion), Merostomata (horseshoe crabs), Maxillopoda (copepod), Priapulidae (marine worm), and Chilopoda (centipedes) classes were likely susceptible to chemicals that act on this target in the tobacco budworm. Further, alignment of key amino acid residues identified as contributing to the sensitivity of tobacco budworm or insensitivity of other insects (e.g., sweet potato whitefly) to TEB (and methoxyfenozide) demonstrated inconsistent conservation when compared with crustaceans, making it less likely that the chemical interaction with EcR is identical among these organisms (Supplementary data, Table S2). Additional evidence indicates that crustaceans are, indeed, less sensitive to TEB compared to targeted lepidopteran pests (e.g., tobacco budworm); however, they may still respond moderately to TEB and more acutely to other xenobiotic EcR agonists (e.g., muristerone, ponasterone A) [36]. As anticipated, in considering the taxonomic domain of applicability for the AOP developed with D. magna, a SeqAPASS evaluation using the cladoceran as a query species also supports robust EcR conservation across invertebrates, including other crustaceans, arachnids and insects (Supplementary data; Figure S1), therefore providing evidence for a broader taxonomic relevance of the defined MIE. This in silico analysis of EcR conservation, combined with the identification of additional, putative D. magna EcR agonists, support the broader generalization of an AOP which began as a chemical and taxonomic-specific MoA.

CASE STUDY 2: 5-HYDROXYTRYPTAMINE TRANSPORTER (5-HTT; SERT) INHIBITION LEADING TO POPULATION ALTERATIONS (DECREASE OR INCREASE) IN BIVALVES

Motivation: Link known or suspected chemical exposure to possible AOs

Fluoxetine is a widely-prescribed anti-depressant frequently found in aquatic environments influenced by waste water treatment effluent, raising concerns over potential adverse effects in exposed aquatic species [37–41]. Case study 2 was developed with a specific emphasis on describing potential effects in aquatic species which may be uniquely sensitive to low concentrations of fluoxetine and related selective serotonin reuptake inhibitor (SSRI) anti-depressants also known to be present in aquatic systems. Identification of the types of AOs associated with exposure to SSRIs can be used to identify appropriate taxa and KE-related endpoints to monitor in environments receiving these pharmaceuticals.

Genesis

Given that exposure to a particular chemical of interest was the motivation for development of this set of AOPs, a ‘bottom up” strategy [8] was employed, which began with the identification of the likely target biomolecule(s) in the exposed organisms. Considering the route of exposure, aquatic organisms were of particular interest/concern. Although chronic toxicity data related to effects of pharmaceuticals in non-target aquatic organisms are limited [42], information regarding their biological activity in humans is widely available, especially the interaction(s) with specific therapeutic targets. DrugBank (www.drugbank.ca) is a publically-accessible database containing curated data on human drug-protein interactions. Molecular interactions are categorized as drug-target, drug-enzyme, or drug-transporter, along with descriptions of the type of interaction (e.g., inhibitor, substrate, etc). A query of this database revealed that, in humans, the therapeutic target of fluoxetine is the sodium-dependent serotonin transporter (5-HTT; SERT; SLC6A4). Read-across approaches suggest that, given sufficient biological conservation, chemicals designed to interact in a therapeutic manner with a given molecular target in one species may have adverse effects by interacting with the same molecule in other species[43]. While fluoxetine and related drugs may interact with additional molecular targets [44], these AOPs were developed specifically with the inhibition of 5-HTT as the presumptive MIE. To utilize this MIE in AOPs for aquatic species, an assessment of the conservation of this target across taxa was a primary step in establishing biological plausibility. Using the human 5-HTT as the query protein, a SeqAPASS analysis indicated substantial conservation across a broad spectrum of major animal classes for which there were protein sequence data (Supplementary data; Figure S2). Within the scope of potentially exposed aquatic taxa (fish, turtles, crustaceans, bivalves, etc), the high degree of conservation at the levels of the primary amino acid sequence and functional domain (solute-binding domain; cd11513) suggests exposure to drugs which interact with the human transporter would likewise inhibit transporters in these taxa.

Expansion

Once a plausible molecular target was identified and shown to be conserved in a range of aquatic organisms, the next step in the AOP development was to identify the AOs and related KEs that may be relevant to effects of a 5-HTT inhibitor in aquatic species. First, with support for read-across of the therapeutic target in humans to 5-HTT orthologs as a potential point of perturbation in other species, an expanded list of potential chemical initiators was generated. DrugBank was used to identify 40 additional drugs with similar inhibitory effects on 5-HTT. Selecting the drugs with known pharmacological activity at the target of interest, and no off-target activity (per DrugBank), the list subsequently was abridged to four: fluvoxamine, fluoxetine, sertraline, and dexfenfluramine.

Next, using this expanded list of chemical initiators specific for 5-HTT, the USEPA ECOTOXicology database (ECOTOX; http://cfpub.epa.gov/ecotox) was queried for all significant findings from studies with the four chemicals in aquatic species. This database contains manually curated literature results concerning toxicant effects on ecologically-relevant species. The query generated more than 900 hits, spanning different taxonomic categories, exposure routes and doses, and measured endpoints. The results were then filtered to focus on water-borne exposures and sorted by exposure concentration. Significant effects at the lowest concentrations (0.00003 and 0.0003 µg/L) corresponded to biochemical, enzymatic, and genetic endpoints measured in mollusks from a single study by Franzellitti et al. [45]. Within this study, the measured endpoints (decreased cAMP concentration and protein kinase A activity, with subsequent reduction in the ATP-binding cassette protein B activity) relate to 5-HT signaling. These endpoints could potentially correspond to a downstream KE, but did not provide enough context for understanding possible in vivo effects. Focusing on records of whole organism responses, which typically would be more directly related to AOs, four studies reported effects of fluoxetine or sertraline at concentrations ≤ 0.001 µg/L. These records identified effects on swimming behavior in the crustacean Echinogammarus marinus, and changes in feeding strikes, pigmentation, courtship and burrowing in two species of molluscs [46–49]. The ECOTOX output suggested that among aquatic species, molluscs and crustaceans may be uniquely sensitive to fluoxetine and related chemicals, a conclusion consistent with that of others [46, 50]. Based solely on 5-HTT sequence similarity relative to that of humans, turtles or fish might be predicted to be more sensitive than aquatic invertebrates to 5-HTT inhibition; however, sequence data alone do not provide insight into comparative functional attributes associated with pathway perturbation. As such, a lower affinity of a chemical for a molecular target in one species compared to another may not equate to diminished sensitivity if the physiology between the species is very different. Given the apparent sensitivity of molluscs to 5-HTT inhibition, this became the focus of our investigation into potential adverse outcomes among aquatic species. The life-strategies and physiology of organisms belonging to the phylum Mollusca are extremely diverse, so we further restricted our efforts to effects in bivalves. The applicability of each pathway to different species within bivalvia is addressed within the AOP-wiki pages for the AOPs and individual KEs.

Recent reviews by Fong and Ford summarize much of the literature populating the ECOTOX records regarding antidepressant effects in molluscs [44, 50]. These reviews and other toxicological studies, along with several sources on general mollusc physiology and biology (e.g., [51, 52]), were used to understand how 5-HTT inhibition might control the observed effects on bivalves and to develop a preliminary network of AOPs (Figure 3). Based on biological plausibility and available empirical data (Supplementary data; Table S3), pathways were developed in which 5-HTT inhibition potentially results in three general AOs: increased risk to predation (AOP 197) and altered reproductive success (increased, AOPs 195 and 203; decreased, AOP 204). Sumpter et al [53] have raised concerns that several of the studies reporting fluoxetine impacts on reproductive endpoints of mussels have used concentrations far exceeding environmental exposure levels, and this may be a limitation of several key events described here. However, recent work to address exposure concentration concerns supports relevant effects at ng/L concentrations [44, 54].

Figure 3.

Putative AOPs describing 5-hydroxytryptamine transporter (5-HTT) inhibition leading to population changes (AOPs 97, 195, 203, 204). Key event numbers reflect page numbers in the AOP Wiki (https://aopwiki.org).

Anti-depressants such as fluoxetine inhibit the reuptake of 5-HT by blocking the 5-HTT (Event: 619), causing an increase in serotonin levels at neural junctions (Event: 620) and increased serotonergic signaling. In bivalves, a primary effect of this signaling is the release of catch state of muscle contraction, resulting in valve (shell) opening (Event: 1142). Valve contraction is initiated by acetylcholine and a unique state of catch contraction allows it to remain tightly closed long after the excitation by acetylcholine has ended and intracellular Ca²⁺ stores return to normal. Importantly, this prolonged contraction is accomplished with minimal (or no) use of energy (for reviews, see [55, 56]). While additional phyla may also be able to undergo catch contraction, the role of serotonin in releasing the contraction state appears to be unique to mollusks [57]. In bivalves, serotonin promotes the transition from the passive catch state to active valve movement [57]; exposure to 5-HTT inhibitors has been observed to cause increased valve movement (Event: 1142) in swan mussels [58] and an inability to maintain a closed shell in wavy-rayed lamp mussels [46]. Increased valve movement can cause excessive water retention in the foot (Event: 1139), which further exacerbates valve opening (Event: 1142) [58]. Disruption in normal valve closure not only depletes the organism’s energy reserves (Event: 1143), but can directly make the organism more susceptible to predation (Event: 623). The water retention in the foot is speculated to cause the foot detachment (Event: 624) observed in mussels exposed to 5-HTT inhibitors [58, 59], although gastropods, terrestrial and aquatic, similarly experience foot detachment with exposure [60, 61]. Mussels detached from the subsurface also have greater energy requirements (Event:1143) [62] and are more susceptible to predation (Event: 623) than those attached to a substrate, where they clump together for protection [63].

Serotonin has also been identified as a primary neurotransmitter used to control both pedal foot and ciliary movement in land and aquatic mollusks (Event:625 and 621, respectively) [57, 60, 64, 65]. The specific impacts on locomotion, as well as the concentration-dependence, varies among molluscs because of differing physiology and life history strategies [50]. In bivalves, movement alterations may take the form of increased crawling or burrowing during daylight hours (Event:622) [46]. The untimely and excessive movement of molluscs due to amplified serotonergic activity could feasibly enhance their visibility and/or diminish their energy reserves, making them more susceptible to predation (Event:623). Recently, prolonged, low-dose (30–300 ng/L) exposures of mussels to fluoxetine were reported to cause decreases in filter feeding rates, energy reserves and growth [54], supporting indirect links between the MIE and energy depletion (Event:1143). It has also been suggested that disrupted coordination of the pedal muscle (Event:625) may contribute to the observed foot detachment (Event:624) [60].

Serotonin also is widely reported to control reproduction in bivalves, via induction of egg maturation [66–68], initiation of spawning [69, 70], and stimulation of parturition (release of brooded larvae) [58, 71]. The pervasive influence of serotonin in their reproductive physiology suggests bivalves could be profoundly impacted by SSRIs. Whether such exposure results in enhanced or diminished reproductive success may depend on the appropriateness of the timing of the events. Any perturbation from normal reproduction can be considered an AO. We have distinguished this case study’s AOPs related to reproductive alterations as those involving induced spawning (Figure 3; AOP 195) and induced parturition (Figure 3; AOPs 203 and 204), which relate to taxa with different life history strategies.

Most bivalves release gametes to the water column for fertilization (broadcast spawn), but some females take in released sperm via their normal respiration process for internal fertilization. Marine species and invasive freshwater species (i.e., Dreissenids) typically broadcast spawn [70]. Native freshwater species of the order Unionoida, many of which are threatened, protected and/or of commercial value [72, 73], are characterized by internal fertilization and parturition of obligate parasitic larvae (glochidia). Glochidia have hooks which allow the larvae to attach to gills, fins or sides of a host fish for a period of time before the larva detach and undergo metamorphosis to their adult form [74]. Many freshwater mussels attract a host fish by moving a part of their body as a “lure” (mantel display), while others release glochidia in conglutinates which resemble fly larva or fish eggs and break apart upon consumption by the host fish (for a review of Unionid host infection strategies, see [52]).

Several studies investigating 5-HT or 5-HTT inhibitor–induced spawning in bivalves report the release of viable gametes (Event: 1255) [58, 69, 75–77], which can reasonably lead to increased reproductive success (Event:1163) and population growth (Event:1164). This increased spawning is attributed to several factors under serotonergic control. Expulsion of gametes (sperm or egg) relies on valve movement (Event: 1142) and ciliary action (Event: 621) to increase water flow [78–81]. Serotonin effects on oocyte development (Event: 1161) also contribute to egg spawning and involve alteration of the oocyte electrical state, initiation of the germinal vesicle break down (GVBD) usually required for fertilization, and progression from prophase to metaphase [66, 69, 76, 82].

The increase in water flow due to valve opening (Event: 1142) and gill ciliary action (Event: 621) is also critical to parturition of larva of Unionid species (Event: 1257). Meechonkit et al. [71] observed the highest concentration of endogenous serotonin in the demibranches of Hyriopsis bialatus, where glochidia are brooded, suggesting serotonin also may control the detachment of glochidia. While Fong et al. [83] found SSRIs induced or potentiated parturition of viable glochidia in fingenail clams, a study by Bringolf et al. [59] found fluoxetine-induced parturition in the mussel Ellipto complanata resulted in the release of non-viable (immature) glochidia, supporting possibly negative effects on reproduction (Event: 1182). The Unionids which use mantel display to attract a host fish also may have increased susceptibility to SSRI-induced parturition. Typically, these mussels move their mantel to resemble the prey of a host fish and release their glochidia directly into the gills of the fish once it strikes at the “lure”. SSRI-exposed mussels have been observed to have increased mantel display behavior [46, 59, 84] (Event: 1256). Some have suggested excessive or ill-timed mantel display could result in inappropriate host selection for such species, with negative consequences to reproduction (Event 1182) [50, 59].

The development of these AOPs highlight possible divergent outcomes of bivalve populations exposed to SSRIs. Given the concerns over the conservation of Unionid species and the management of invasive Dresseinid species (e.g., zebra and quagga mussels) [85], the evidence of increased serotonergic signaling leading to greater reproductive success of broadcast spawners but possible diminished reproductive effects in Unionids may be important from a risk perspective. More research is needed to discern precisely how reproduction in each species of concern may be influenced by increased serotonergic signaling and at what relevant concentrations.

CASE STUDY 3: SODIUM CHANNEL (NAV1.1) INHIBITION LEADING TO POPULATION DECLINE

Motivation: Enhance the predictive utility of an alternative assay

An ongoing OECD effort (project 1.29; Supporting Information) aims to identify and catalogue AOPs “that will enhance the utility of US EPA ToxCast high-throughput screening data for hazard identification.” The ToxCast (Toxicity Forecaster; http://actor.epa.gov) program uses approximately 700 high-throughput assays to rapidly screen thousands of chemicals for their ability to interact with well-defined biological pathways [86, 87]. The majority of these are in vitro assays which measure effects at a specific (mammalian, often human) molecular target/MIE, although some more downstream responses are also examined. An ongoing question concerning the ToxCast dataset involves interpretation of the regulatory significance of the results of specific assays in the context of potential human health or ecological hazards [88]. Case study 3 is an example of an AOP developed to describe how changes at the molecular level, as measured in an in vitro, high-throughput, chemical screening assay, may relate to AOs of concern for ecotoxicological risk assessment.

Genesis

The starting point for this AOP development effort was to determine the physiological role of the MIE target represented in a ToxCast assay and whether a plausible linkage to an AO could be drawn based on its known function. The ToxCast Novascreen ion channel assay NVS_IC_rNaCh_site2 measures chemical inhibition of the voltage-gated sodium channel, Nav1.1 (scn1a), in a cell-free system. To evaluate the utility of this assay for identifying potential toxicants which may cause an AO, the physiological role of the Nav1.1 channel and consequences of its inhibition were researched in the general physiology and biology literature to understand possible physiological perturbations associated with its inhibition or malfunction.

The Nav1.1 channel is the primary sodium channel in the cell bodies of several classes of GABAergic interneurons. Upon excitation triggered by the influx of sodium ions via the Nav1.1 channels, these interneurons release gamma-aminobutyric acid (GABA) [89]. In the adult vertebrate central nervous system, GABA is the primary inhibitory neurotransmitter which, upon binding ionotropic GABA_A receptors, causes an influx of chloride ions and hyperpolarization of the cell. Reduced Nav1.1 channel function (Event: 584) leads to a decrease in neuronal inhibition (Event:669), and is recognized as a risk factor for epilepsy (Event: 613) and cardiac arrhythmias [90, 91]. Exposure to drugs which inhibit the channel’s function has been associated with increased seizure events in epileptic patients [92]. Furthermore, a heterozygous loss of function in humans causes Dravet Syndrome, a form of pediatric epilepsy wherein reduced sodium currents impair the excitability of downstream GABAergic inhibitory neurons, resulting in seizures with comorbidity of cognitive impairment and premature death.

Recently, Baraban et al. (2013) used zebrafish with a mutation in the gene corresponding to the Nav1.1 equivalent as a model for Dravet Syndrome [93]. In zebrafish, two homologs, scnLaa and scnLab, code for the Nav1.1 channel protein and are prominently expressed during the early stages of larval development in the brain regions corresponding to the forebrain, optic tectum and cerebellum [93]. Fish with a mutation in scnLab, which shares 77% identity with human SCN1A, exhibit spontaneous abnormal electrographic activity, hyperactivity and convulsive behaviors (Event: 613) [93]. While these findings hold promise for using the mutant zebrafish as a screening tool for anti-epileptic drug discovery [94], they also suggest that fish species may be adversely impacted by Nav1.1 channel inhibitors.

Based on the effects of the scnLab mutation in zebrafish and the etiology of Dravet syndrome in humans, the components for an AOP with a MIE of Nav1.1 inhibition were identified (Figure 4). Furthermore, three AOPs describing ionotropic GABA_A receptor antagonism have previously been developed, distinguished only by the antagonist action of chemical initiators with various chemical structure domains (competitive antagonism, negative allosteric modulation, or non-competitive channel blocking) [95, 96]. The same authors also describe their GABA_A receptor antagonism AOP in the AOP knowledgebase; https://aopkb.org/aopwiki/index.php/Aop:10, although the events differ slightly from those in their published manuscript (Figure 4, shaded events). The diminished release of GABA due to Nav1.1 inhibition (Event: 584) acts, ostensibly, in the same manner as a chemical competing with GABA for the binding site on the GABA_A receptor (Event: 667); thus, a separate AOP describing Nav1.1 inhibition leading to diminished GABA release can dovetail with the already-established GABA_A receptor antagonism AOP to form an AOP network (Figure 4, unshaded events).

Figure 4.

Putative AOP network illustrating Nav1.1 inhibition and GABA_A Receptor antagonism leading to seizure-related adverse outcomes (AOPs 197 and 10). Key event numbers reflect page numbers in the AOP Wiki (https://aopwiki.org).

Expansion

Once an AO and corresponding sequence of KEs was linked to the MIE examined in the assay, our next step was to explore the support for applicability of these AOPs in fish. An approach similar to the one used to describe 5-HTT inhibition (case study 2) was employed. Specifically, chemical initiators were identified and then used to query ECOTOX for empirical evidence of effects in fish. Additional searches in literature databases were performed in addition to the ECOTOX query. As part of this basic analysis, support for lifestage applicability of the derived AOP was also addressed.

In this case, since the MIE aligns with an assay used in the ToxCast program, one potential source of chemical initiator data is the ToxCast knowledgebase itself (http://actor.epa.gov/dashboard/). The ToxCast program has evaluated over 2000 substances with the NVS_IC_rNaCh_site 2 assay, identifying 118 active chemicals in the process (http://actor.epa.gov/dashboard). A ToxCast assay for GABA_A receptor inhibition (NVS_LGIC_bGABARa1) also exists and has identified 25 active chemicals. While these assay results suggest chemicals which have the putative ability to trigger an AOP with Nav1.1 channel or GABA_A receptor inhibition as its MIE, for development purposes only chemicals which are very specifically-acting should be used. Of the chemicals identified as active in either the NVS_IC_rNaCh_site2 or NVS_LGIC_bGABARa1 assays, none were selected for AOP development because they were either proprietary compounds or had at least one other primary molecular target (affected at lower concentrations, as indicated by the ToxCast data).

A more focused list of chemical initiators was compiled by querying DrugBank for specific antagonists of Nav1.1 and the GABA_A receptor. Eliminating several drugs with known off- target activity and two reported to be Nav1.1 agonists (vs antagonists) in the literature, three chemicals were identified as reasonable candidates: phenacemide, phenazopyridine, and dehydroepiandrosterone. Additional drugs used to treat anesthetic overdose or to induce seizures in epilepsy models were identified as well: bicuculline, metrazol, flumazenil, thiothixine, buproprion, thujone, and picrotoxin. These drugs were confirmed to be GABA_A receptor antagonists in the peer-reviewed literature. Finally, the pyrethroid pesticides fipronil and endosulfan are known to selectively antagonize the GABA_A receptor (in addition to the insect-specific Glycine receptor) [97, 98] and were also included as chemical initiators used to query ECOTOX..

Comparatively little data were found in ECOTOX for effects in fish from waterborne exposure to most of the target chemicals. The studies reporting significant effects at the lowest concentrations indicated that metrazol causes increased activity of adenosine deaminase in the brain of zebrafish [99]. Closer review of that study revealed that this enzymatic measurement, while not a KE in the AOP under consideration, is nonetheless considered a biomarker of seizure susceptibility. Furthermore, at higher test concentrations, seizures were observed in the fish [99]. In other work, intra-peritoneal injection of metrazol in zebrafish likewise resulted in seizures and immobility which, presumably, would lead to death [100]. Swimming distance was significantly decreased in larval fathead minnows (Pimephales promelas) exposed to fipronil and the authors note that these fish had ineffective tail flip and poor muscle coordination in response to touch [101]. Several ECOTOX records identified studies of fipronil exposure and reproductive effects in fish, suggesting there may be additional AO considerations for GABA signaling. However, a study by Bencic et al [102] specifically addressed GABA_A Receptor antagonism and fathead minnow reproduction (egg production) and found no support for an AOP with this endpoint.

The initial goal was to develop an AOP in support of the ToxCast chemical screening effort. Further investigation into the taxonomic domain of applicability of the AOP network described above used SeqAPASS to evaluate the structural conservation of the MIEs relative to the species from which the assay proteins were derived. Therefore, the Norway rat (Rattus norwegicus) and bovine (Bos Taurus) were selected as query species to evaluate Nav1.1 and GABARa1 assays, respectively. As anticipated, sequence comparisons of the functional domains of both molecular targets indicate conservation across many animal classes, including bony fish (Actinopteri; Supplementary data, Figure S3), providing a line of evidence that the MIE can be more broadly extrapolated beyond mammals.

AOP developers also must consider lifestage applicability for their AOPs. Scn1Lab transcript is detectable in zebrafish as early as 1 day post fertilization (dpf), suggesting that exposure to Nav1.1 inhibitors at any lifestage would likely cause a downstream inhibition of the GABA_A receptor [103]. However, the response of GABA_A receptor activation can be excitatory or inhibitory at different lifestages, which can also be species-dependent. In the developing vertebrate brain, GABA has an excitatory effect which is important in brain development; it controls the proliferation, migration and differentiation of neural progenitor cells [104–106]. Endosulfan exposure in rats is teratogenic [107]. In humans, the switch from excitatory to inhibitory action occurs during parturition, accelerated by oxytocin produced by the mother during childbirth [108]. However, less is known about when this switch occurs in fish embryos. In zebrafish, some evidence of this switch in GABA-responsive retinal ganglion cells suggests this happens in concert with the development of light responsiveness at 2.5 dpf [109]. If the receptor is functionally excitatory at that stage, the impact of GABA_A receptor inhibition in embryos younger than 2.5 dpf may have an altogether different outcome, such as impaired brain development. However, Raftery and Volz [110] exposed zebrafish embryos at 5–23 hour post fertilization (hpf) to fipronil or endosulfan and observed decreased spontaneous activity at the highest concentrations tested (25 and 5 µM, respectively). These effects were attributed to the paralyzing effects of neuron-overstimulation and not developmental defects. In the same study, zebrafish embryos similarly exposed to a GABA receptor agonist (abamectin) also experienced (reversible) hypoactivity without alterations to neural outgrowth. The reduction in spontaneous movement was attributed to the sedative-type effects of abamectin, either due to blocked signal transmission from interneurons to excitatory neurons or by increased concentrations of (inhibitory) GABA. Regardless, perturbation of GABA transmission at the 5–23 hpf developmental window did not appear to have neurodevelopmental, but rather motorneuron effects in zebrafish.

Sex-dependent effects are likewise not well studied in fish. However, several lines of evidence point to a differential presence and activation of GABAergic neurons in male and female developing mammalian brains, as well as the interplay of GABA with estradiol and estrogen receptor responsiveness [111]. For example, the activation of the GABA_A receptor has been suggested to be a critical event for hippocampal development in rats [112]. Specific forms of epilepsy also show predisposition in males or females [113–115]. These findings highlight that once a sequence of plausible KEs are defined, it becomes easier to systematically probe the literature to examine the taxonomic, life-stage, and sex-based conservations of those events and capture that information in the KER descriptions and/or by adding additional (e.g., life-stage, sex, or taxa-specific) branches to an overall AOP network.

CASE STUDY 4: CYCLOOXYGENASE 1 (COX1) AND ORGANIC ANION TRANSPORTER 1 (OAT1) INHIBITION LEADING TO MORTALITY IN AVIAN SPECIES

Motivation: Deduce responsible chemical(s) and mechanistic causes of observed in vivo adverse effects

An observed AO in individuals or populations can be a strong motivation for developing one or more AOPs. For example, any observation of wildlife population declines potentially associated with chemical exposure has direct relevance for risk assessment. In such a situation, KEs that immediately (or closely) precede an AO may be easily identified if sampling of affected organisms can be performed. Furthermore, the susceptible lifestage(s) and/or sex of the adversely-affected species may be evident from the demographics of the population. In many studies, the known or suspected chemical initiator(s) may be deduced from chemistry data at the field site or chemicals detected in the organism, although the MIE and early KEs may be unknown. Case study 4 is an example of AOP development initially motivated to understand potential toxicant-related causes of an observed wildlife population decline.

Genesis

Dramatic and sudden declines in Gyps vulture populations occurred throughout the Asian subcontinent beginning in the 1990s [116]. Concerned by the critically endangered listing of the Oriental white-backed vultures in India, Oaks et al. (2004) examined 259 carcasses of affected birds and found 85% had urate deposits on organ surfaces, strongly suggesting visceral gout (renal failure) as the cause of death. After eliminating many potential causes of this pathology, the authors hypothesized that the birds were dying from adverse effects due to drug treatment of cattle, a primary food source for the vultures. They detected diclofenac in the carcasses obtained from their field studies and tested the toxicity of diclofenac-treated meat on healthy, captive vultures. The birds that consumed the treated meat experienced rapid hyperuricemia, subsequent tubular necrosis, uric acid crystal (tophi) deposition in the kidneys and other tissues, and abrupt mortality [117]. The findings of the Oaks study, alone, provided the basis for a putative AOP (Figure 5), albeit without evidence of a specific MIE.

Figure 5.

Putative AOP network of cyclooxygenase 1 (COX1) inhibition (AOP 177) and organic anion transporter 1 (OAT1) inhibition (AOP 138) leading to renal failure and mortality. Key event numbers reflect page numbers in the AOP Wiki (https://aopwiki.org).

Expansion

While the empirical observations of apical toxicity help to define a putative AOP, determination of the specific molecular target(s) diclofenac perturbs to cause these effects could facilitate the identification of additional chemicals that might cause similar effects, and/or additional sensitive taxa. A search of literature that cited the Oaks study identified several publications which have investigated the mechanistic basis of diclofenac toxicity. Diclofenac is a non-steroidal anti-inflammatory drug (NSAID) designed to inhibit cyclooxygenase (COX) activity. While newer NSAIDs primarily inhibit the inducible form of cyclooxygenase (COX2), which controls the inflammatory response, diclofenac also acts on the constitutive form (COX1; Event: 1103). Among its functions, COX1 controls the production of the vasodilatory prostaglandin PGf2α (Event:1104), which regulates renal blood flow and the glomerular filtration rate. NSAID vasoconstriction causes proximal tubular ischemia (Event:1105), resulting in oxidative stress (Event: 1088) and cellular necrosis (Event:1097), although the exact mechanism through which this occurs is debated [118, 119]. As proximal tubule function is compromised, circulating levels of uric acid rise (hyperuricemia; Event:1096) and bicarbonate is lost, resulting in metabolic acidosis (lowering of blood pH). This condition manifests as central nervous system depression: lethargy, drooping neck in birds, drop in blood pressure, and failure to eat [120]. Many of the events in the pathway exacerbate one another: uric acid is an antioxidant and its absence within the renal cells may play a large role in susceptibility to oxidative stress (Event: 1088) [119], while the extensive necrosis of the proximal tubules (Event: 1097) further limits the ability of the tissue to transport the uric acid from the blood, amplifying the hyperuricemia (Event: 1096). Consequently, the AOP network representation manifests as feed-forward or self-reinforcing relationships (not pathway irreversibility). Elevated uric acid in the blood can also can lead to the precipitation of uric acid crystals (tophi) in the viscera (Event: 1102) which further intensifies the progression of necrosis, resulting in lesions in the surrounding tissue (visceral gout) [118]. Ultimately, the kidneys are no longer able to maintain control over potassium blood levels, resulting in hyperkalemia (Event:1098) [118]. As suggested by Naidoo et al., this increase in potassium is the likely cause of death, as it can lead to fatal cardiac arrhythmia (Event: 1106) [121, 122].

The findings by Naidoo [88] and Ng et al [86] demonstrate the acute toxicity of diclofenac and related drugs may involve an additional MIE (Figure 5; AOP 138). Many of the NSAID drugs also inhibit uric acid transport via the organic anion transporter 1 (OAT1) (highly similar to OAT3; previously referred to as the p-aminohippuric acid channel) [123], and may produce reactive oxygen species by an unknown mechanism [119, 124]. This pathway plausibly converges with the ischemic effects described above due to COX1 inhibition (AOP 177) to exacerbate renal tubular necrosis (Event:1097).

Follow-on studies further contributed to AOP development by extending the effects beyond individual birds to the population level (Event:361) [125, 126], providing empirical support for the essentiality of the KEs and concordance of the KERs (see discussion, below), and expanding the domain of species applicability to numerous other species, including other raptors and fowl [118, 126–130]. Reports of other COX inhibitors causing comparable pathology in birds expanded the list of chemical initiators to include flunixin, ketoprofen, indomethacin, ibuprofen, carprofen, and nimesulide [126, 130–133]. Substantial research has investigated the importance of exposure and toxicokinetic considerations in determining differential toxicity. Specifically, variability exists in the adsorption, distribution, metabolism and excretion (ADME) of each NSAID among different species [118, 128, 134, 135]. Most notably, despite its ability to inhibit COX1, the widespread exposure of susceptible species to meloxicam has not resulted in any known pathology [136]. This finding has been attributed to the rapid elimination of this compound relative to other NSAIDs, underscoring the importance of toxicokinetic considerations when determining differential toxicity and applying AOPs in risk assessment or other types of regulatory decision-making. Interestingly, the ToxCast assay NVS_ENZ_oCOX1 identified several pharmaceuticals as active COX1 inhibitors which are not NSAIDs: clofibrate, theobromine, methyldopa sesquihydrate, reserpine, ilepatril and farglitazar. Most of these drugs are classified as anti-hypertensives or PPARα agonists. Cleves and Jain [137] demonstrated recently that structural similarities between PPARα agonists and cyclooxygenase inhibitors allow for cross activity at these targets. Clofibrate, a lipid-lowering agent, was investigated in a hyperlipidemic chick model and found to produce the same pathology described in this case study: hyperuricemia, depression, and abrupt mortality [138].

GENERAL THEMES: POPULATING DATA GAPS AND ASSEMBLING WEIGHT OF EVIDENCE

The potential applications of the AOP framework are diverse, which has incentivized the capture and depiction of toxicological knowledge as AOPs as well as the dissemination of that organized information via the AOP Knowledgebase. The development of an AOP may be stimulated by expert knowledge in an area, compelling observational data in terms of extant effects, or needs associated with a specific regulatory or monitoring application. The initial motivation influences how development proceeds, as well as the degree of confidence required for certain components of an AOP. That is, detailed information for every component of an AOP may not be necessary for a given application [12]. Several strategies and tools described in the above examples can be used to support novel AOP development or to expand existing AOPs. Through the case studies and our own experiences with other AOPs, we have identified a number of publically-available databases that provide the kinds of data useful for AOP development (Table 1). Based on the knowledge already available to the developer and the type(s) of additional knowledge being sought, each source has discrete advantages and disadvantages. For example, some are ideally suited for identifying putative chemical initiators if an MIE is known (e.g., DrugBank, ToxCast, ChEMBL, STITCH). For AOPs incorporating QSAR models to identify chemical initiators, such as in case study 1, chemical databases (e.g., PubChem, Chemspider, OECD toolbox) can provide identifiers essential for linking between data sources in addition to structural information (SMILES, InChI). Other online tools can aid qualitative identification of additional KEs on the basis of gene-disease interactions, pathway annotations, etc. (e.g., Small Molecule Pathway Database, Comparative Toxicogenomics Database, ChEMBL). Still others offer quantitative empirical support and are more suitable for AOP weight of evidence analyses, as described below.

Table 1.

Examples of Databases useful for AOP development.

Resource, URL	Description	Known AOP component	AOP component to identify	Advantages for AOP development	Limitations for AOP development
ChEMBL, https://www.ebi.ac.uk/chembl	A molecular database containing drug-target data from dose-response studies.	Target	Chemical initiator(s) Relative potency	Vast database containing curated bioactivity of chemicals from many sources. Sources are annotated.	Difficult to identify the chemical (given as chembl ID; no CAS number). Requires filtering of data, etc.
		Target	Relative potency (WoE evaluation)	Easy to obtain bioactivity data (e.g., IC50, Ki, EC50) which may be used for relative potency ranking of chemical initiators. Sources are annotated.	Difficult to identify the chemical (given as chembl ID; no CAS number). Requires filtering of data, etc. Relative potency ranking conducted on results within a study requires review of primary data source.
DrugBank, www.drugbank.ca [131]	A database containing drug-protein interactions, including drug- target, drug-enzyme and drug-transporter info.	chemical initiator	target	Describes intended effects; identifies target(s) with a description of the action on the target and references; easily links to other drugs effecting the same target for identifying additional potential chemical initiators. Query by chemical can also be performed to display drug action pathways via a link to SMPDB database http://smpdb.ca	Pharmaceuticals only; limited to human targets; No potency information.
		target	chemical initiator(s)	Identifies drugs with known interaction at a molecular target, the type of interaction, and references. Links to other targets affected by drug’s target for evaluation of specificity of chemical initiators. Query of a generic target provides information on specific isoforms.
Small Molecule Pathway Database (SMPD), http://smpdb.ca [132]	Interactive, visual database of human small molecule pathways which provide information on relevant organs, subcellular components, protein-complex cofactors.	chemical initiator, target	key events, adverse outcome	Query of drug or target returns links to Drug Action, Physiological and Disease pathways. Drug Action pathways provide a text description as well as pictorial, interactive, pathway indicating involved cellular components and organ systems. Physiological pathways indicate organ systems involved in drug or target perturbation. Disease pathways provide descriptions of adverse genetic and biochemical conditions in humans.	Limited chemical (pharmaceutical) and pathway coverage; visual pathway depictions are difficult to navigate; generic references; human focus.
toxic exposome database, www.t3db.ca	A database containing chemicals and chemical-gene target information curated from various sources; similar format as DrugBank but covers broader chemical space.	chemical initiator	target	Provides target information for chemicals from many classes; AC50 concentrations are reported from chemical screening in target-related ToxCast assays; provides references.	Target interaction type is not specified (activation, binding, antagonism); multiple targets often identified without differentiation of importance.
ECOTOX, http://cfpub.epa.gov/ecotox	A manually-curated database containing single chemical toxicity data for aquatic life, terrestrial plants and wildlife.	chemical initiator	target, key events, adverse outcomes, taxonomic, lifestage and sex appicability, dose-response	Search by CAS #, chemical name, or general chemical category. Batch queries using pre-identified chemical initiators can be performed. The results are available in spreadsheet format and provide information on species, sex and lifestage tested, exposure route, type of endpoint (mRNA, protein, behavioral effect, reproduction, mortality, etc), chemical concentrations and significance, plus relevant references.	Chemical concentrations in various units, many of the recorded responses are generic. Effects of interest must have a corresponding ECOTOX code. Ecological species emphasis. Limited by literature curated and effects coding.
		target	chemical initiator(s), key events, adverse outcomes, taxonomic, lifestage and sex appicability, dose-response	All records reporting chemical effects on a target including type of effect (increase, decrease). Query can be filtered for taxanomic specific results,	Target of interest must have a corresponding ECOTOX code, as identified in biochemistry, enzyme, genetics and hormone effects categories. No cross-referencing for alternative target symbols. Curated literature primarily is focused on in vivo effects measured in toxicity studies, thus minimal target-specific coverage.
		key event, adverse outcome	species, lifestage, sex applicability, dose-response	Efficiently reports species, lifestage and sexes tested, exposure information (route, duration, and concentration), significance and references corresponding to the effect of interest. Queries can be filtered for chemical initiators or species of interest. Results may expedite weight of evidence assembly.	Effect of interest must have a corresponding ECOTOX code. Apical effects such as fertility and mortality may be unrelated to the AOP pathway in development (e.g., majority of mortality reports may be LC50 studies). Ecological focus.
		taxonomy	Key events, adverse outcomes, sensitivity. Assays used for measurements	Taxonomic queries are possible as scientific or common names of species, or as predefined taxonomic groups (e.g., fish, birds), allowing for evaluation of species differential susceptibility. Threatened or endagered species may also be queried.	Most recent studies probably not captured. Apparent preference for in vivo laboratory studies. Variable how endpoints relate, or not, to AOP events.
iCSS ToxCast Dashboard, http://actor.epa.gov/dashboard [74, 75]	A database containing high throughput chemical screening data generated from the Toxicity Forecaster (ToxCast) and Toxicity Testing in the 21st century (Tox21) collaboration	chemical initiator	target	Chemicals (single and batch) can be queried for dose-response activity results in tested bioassays. Assays are described including target and type of perturbation measured. Data are generated from standardized testing procedures, allowing for determination of approximate effect concentrations and relative potency.	Bioassays generally incorporate human or mammalian targets only. The most relevant target may not be captured by the bioassays or the type of perturbation measured in the available assays. No xenobiotic biotransformational capability of assays.
		target	chemical initiator(s)	Biological targets can be queried for relevant screening bioassays with descriptions of the types of perturbations measured. All tested chemicals with a ready filter for those determined to be active in a bioassay are displayed. Redundant assays exist for many targets, allowing for an evaluation of confidence in chemicals identified as interacting with a target. Chemicals are tested using a standardized procedure at multiple concentrations and compared to the response of a reference chemical, allowing for determination of approximate effect concentrations and relative potency.	Limited to tested chemicals. Sensitivity of assays vary.
		target	Relative potency for empirical WoE	Screens large numbers of chemicals for activity at a (generally) molecular target, providing dose-response data. Chemical relative potency may be ranked according to maximum effect, AC50, AC10, area under the curve, or other parameters.	AC50 and AC10 (50% ans 10% maximimun effect concentrations) may have limited utility as measurements of potency unless the response data generated from the assay is normalizing to a reference chemical because of different maximum effects.
International Mouse phenotyping consortium, www.mousephenotype.org	A database of observed phenotypes from knockout embryonic stem cell and mouse models. Searchable by gene, phenotype or human disease.	target	key events, adverse outcomes, essentiality	Target essentiality for the adverse outcome can be evaluated. Specificity of an observed phenotype is reported against all genes evaluated with knock out technology. Identifies other genes responsible for the same phenotypes. Lists homozygous null genes which are embryo lethal.	Mammalian targets only. Addresses only target loss of function (e.g. inhibition/ antagonism) effects. Severity of loss of function phenotype is greater than effects of inhibition/ antagonism of target and may not represent the same pathway (?). Emphasizes developmental effects. Embryo lethal effects do not allow for prediction of effects in adults.
Sequence alignment to predict across species sensitivity (SeqAPASS), https://seqapass.epa.gov/seqapass	Database for assessing available molecular target information to describe protein sequence similarity at the primary amino acid sequence, conserved domain and individual amino acid residue levels.	target	taxonomic applicability	Assesses conservation of protein targets across taxa in relation to user-identified reference species. Three levels of conservation are assessed, as available data allows: primary amino acid sequence, functional domain, and specific amino acid.	Taxonomic differences in physiology are not addressed. No assessment of chemical potency at target.
AOP knowledgebase (AOP KB), https://aopwiki.org	A repository of AOPs in development, with modular, linkable key events to allow for formation of AOP networks.	chemical initiator, target, key event, adverse outcome	chemical initiator, target, key event, adverse outcome	Any AOP event or chemical initiator may be queried for its preexistence in the knowledgebase. In many situations, relevant key events or pathways exist for incorporation into a new AOP or expansion in terms of domain of applicability. Many common events have descriptions, relevant assays, domain of applicability and empirical support already provided.	Variable quality of event descriptions. Modification of existing entries requires contacting original authors. Search term must match existing entered term and retrieves all entries throughout the database.

Biological plausibility

One primary consideration in assessing an AOP’s robustness is the biological plausibility of its KERs. The plausibility of a relationship between an upstream and downstream KE is supported by mechanistic (i.e., structural or functional) knowledge consistent with established biological knowledge [12, 139]. With this consideration, AOP developers need to be aware that many online databases provide empirical support for an association with a chemical exposure (chemical-gene or chemical-phenotype associations), but most are not curated to identify the MIE or causally related KEs that make up KER- type interactions. The structural and/or functional underpinnings of a reported association are rarely detailed. For example, chemical-gene associations in many instances involve general metabolism or compensatory responses, rather than direct effects. Extensive lists of statistically-associated targets without an understanding of the structural or functional relationships, generally increases the effort required for an AOP developer to differentiate the KE-type responses. As a result, databases such as the Comparative Toxicogenomic Database (http://ctdbase.org; [140]) may be useful in suggesting some putative pathways on which to focus, but it is essential that potential relationships be confirmed through primary literature and functional understanding. One database that differentiates chemical-target (MIE) interactions from other biomolecular interactions is DrugBank. There, both the target molecule and kind of interaction are specified, as well as any additional molecular interactions (in the context of side effects). This information, however, usually lacks relative potency data when multiple targets are identified.

With regard to evaluating the plausibility of an MIE in untested taxa, a SeqAPASS evaluation provides a moderate level of confidence based on the degree of target similarity to that of a susceptible species. Moderate confidence, as described by Becker et al., can be derived from analogy to an accepted biological relationship without fully established scientific understanding [12]. While robust target conservation infers plausibility, high confidence in taxonomic relevance would require a detailed understanding of specific physiology. For example, in case study 2, the protein sequence conservation of the 5-HTT of taxa other than molluscs had greater levels of target similarity to the human query. However, research into the unique physiological role of serotonin in mussel catch relaxation and reproduction augmented the plausibility that adverse effects could occur at lower exposure concentrations in mussels than in other (e.g., vertebrate) taxa [44]. In general, whether considering relationships within the AOP or assessing the taxonomic relevance of the AOP, the better structural and functional relationships in terms of normal biological functions are understood, the less empirical information is needed to confidently apply the AOP. For this reason, plausibility ranks highest among the different types of evidence used to support an AOP [9, 139].

Essentiality

A second weight of evidence consideration for assessing an AOP concerns the essentiality of an upstream event causing the downstream event [12]. Evidence of essentiality establishes a direct causal connection between one KE and the next –suggesting that if the upstream event is prevented, none of the downstream KEs in that AOP will be observed (unless acted upon through another intersecting AOP). One approach to establishing the essentiality of the MIE, or KEs is to use knockout/knock-down or overexpression genetic models. For events involving the inhibition or antagonism of a target, the applications of CRISPR/CAS9 technology, which can make targeted genome changes to living cells [141], offers a highly promising source of data to address questions of essentiality. Two online resources which may assist a developer in addressing the essentiality of an inhibitory or antagonistic MIE by curating studies which have used these types of genetic manipulations are the International Mouse Phenotyping Consortium (IMPC; www.mousephenotype.org) and the Zebrafish Model Organism database (ZFIN; http://zfin.org). Although online genomic resources for genetic manipulations are limited to a few model species, rapid development within this field may soon provide similar tools in non-model organisms.

The IMPC website can be searched by gene, phenotype or human disease. For example, a query of ‘Dravet syndrome’ (case study 3) in the IMPC database identified knockout models for scn1a and scn9a and suggested a high phenotypic similarity in the gabra1 (GABA_A receptor) knockout model. Searches conducted for the genes scn1a or gabra1, likewise, describe the epileptic phenotypes associated with loss of function of these targets. The ZFIN database is not as intuitive as the IMPC database, but offers a unique feature in displaying figures (not under copyright restrictions) from, and links to published studies employing the knockout model of interest. For example, a query of the scn1lab gene revealed phenotypes related to abnormal effects in brain, eye, and locomotion, and provided links to four studies which used knockout zebrafish models. One of these studies used morpholino technology to obtain a zebrafish model of Dravet syndrome, but inhibited the epileptic seizures using pharmacotherapy with a serotonin agonist, demonstrating a reversal of phenotype [142].

Studies like the one above, which demonstrate a reversal in phenotype (recovery) upon elimination of an upstream event, are ideal to validate the essentiality of KERs. Most of these kinds of experiments are not easily identified in databases, but require a developer to research primary literature. Based on the proposed pathway, it may be possible to predict the kinds of rescue experiments that may support essentiality of an AOP. For example, a query for ‘Dravet syndrome rescue with GABA’ in PubMed (www.ncbi.nlm.nih.gov/pubmed) or Google Scholar (https://scholar.google.com) revealed a study in which behavioral phenotypes displayed by a scn1a +/− mouse model were completely “rescued” by administration of a GABA_A receptor modulator, clonazepam [143].

Empirical support

The third major type of evidence assembled as part of an AOP evaluation is the extent of empirical support for the KERs. Specifically, the concordance of empirical observations of dose-response, temporal relationships and incidence between upstream and downstream KEs are key to providing a high level of confidence in a KER [12]. Importantly, concordance of dose-response is not satisfied by providing evidence of the dose-dependence of a single KE. Rather, the expectation, if there is a causal relationship between KEs, is that the upstream event should be observed at concentrations equal to or lower than those that cause the downstream event. Generally, this type of support is evaluated using data generated with specific chemical initiators (e.g., aromatase inhibition AOP [12], AOP: 25). However, a common limitation in evaluating dose-response concordance is the inability to compare results across studies. Even if the data are generated using the same chemical initiator, species differences, actual doses tested, routes of exposure (e.g., diet vs intraperitoneal), in vitro chemical bioavailability, etc., can all confound the ability to make direct dose-response comparisons. Additionally, even within a study, not all KEs can be measured with similar sensitivity and precision. Consequently, in evaluating empirical support for a KE, we propose that chemical potency may also be used to infer dose-response concordance, with the caveat that toxicokinetic (ADME) considerations may still impact this comparison. The basic idea is that for any KE-related assays which provide potency estimates for a suite of chemicals, those relative ranks can be compared between the upstream and downstream KEs. If the upstream and downstream KE responses indicate the chemical potencies are in agreement, there is support for dose-response concordance (although chemicals with weak potency in the upstream KE may not elicit effects in the downstream KE).

The ToxCast and ChEMBL databases may be particularly useful for potency-based concordance evaluation because they contain bioactivity data on large numbers of chemicals. For example, in case study 4, one of the putative MIEs for the AOP describing diclofenac toxicity in avian species is the inhibition of COX1. The ToxCast program has currently screened 2075 chemicals for COX1 inhibition using the assay NVS_ENZ_oCOX1 (although few prototypic COX inhibitors were included). Of the screened chemicals, four pharmaceuticals were identified for concordance evaluation and ranked by potency: indomethacin > diclofenac > clofibrate > aspirin. This ranking was based on several ToxCast parameters (Supplementary data; Figure S4, Table S4), and represented as activity concentrations as 50% maximal response (AC50; Table 2). Further, SeqAPASS evaluations confirmed that avian COX 1 is highly conserved with the ovine COX1 used in the ToxCast assay (Supplementary data, Figure S5).

Table 2.

Relative potency of cyclooxygenase 1 (COX1) inhibitors as determined by various assays.

	ToxCast	Kawai 1998	Blanco 1999	Riendeau 1997			Brideau 2001			Ricketts 1998	Toutain 2001
chemicala	sheep AC50	human IC50	human IC50	human - microsomes IC50	human - Platelets IC50	human- transfected CHO IC50	horse IC50	dog IC50	cat IC50	dog IC50	dog IC50	Range IC50	Relative potency
FL				0.001	nd	0.0005	0.060			0.008		0.0005–.06	very potent
KP				0.0028	0.0017	0.0018		0.13		0.029		0.0017–0.13	potent
DF	0.163	0.037	0.611	0.007	0.002	0.004		0.410				0.002–0.611	potent
IM	0.068	0.013	0.063	0.020	0.005	0.017	0.036	0.740				0.005–0.74	potent
CF	1.329											1.33 (AC50)	moderate
IB	inactive	3		0.094	0.154	0.47						0.094–3.0	moderate
MX		4.7	36.6	0.143	0.31	1.81		4.3		0.891		0.143 –36.6	weak- moderate
NM				0.117	1.11	0.78				2.15	20.3	0.117– 20.3	weak- moderate
CP				0.076c	0.61c	0.2c	9.2	65.2	33.08	13.2		0.076–65.2	weak-moderate
PB				1.01	3.7	8	6.15	17.8		> 10b		1.012–17.8	weak
AS	inactive	3.2	3.57	21.5	nd					34.3		3.2–200+	weak

^aFL, flunixin; KP, ketoprofen; DF, diclofenac; IM, indomethacin; CF, clofibrate; IB, ibuprofen; MX, meloxicam; NM, nimesulide; CP, carprofen (racemic mix); PB, phenylbutazone; AS, aspirin

^bno inhibition of COX 1 was observed at the concentrations tested (0.01–10 uM)

^cno stereospecific information provided; R enantiomer is known to be more potent

AC50 and IC50 concentrations provided in micromolar

Additional studies in which multiple chemicals were evaluated for their ability to inhibit COX1 were included in the potency ranking. Specifically, chemical concentrations (IC50) which inhibited 50% of COX1 activity were used to compare potency. Because measured IC50 concentrations may vary widely across different assays, relative potency was assessed using intra-assay results (Table 2). Only where chemicals could not be ranked using results within a given assay, or different studies reported conflicting relative potency between two drugs, were absolute IC50 concentrations used for direct comparison. This process identified the relative potency flunixin > ketoprofen/indomethacin/diclofenac > clofibrate/ibuprofen > meloxicam/nimesulide/carprofen > phenylbutazone/ aspirin (Table 2).

The ranking of COX1 inhibiting chemicals was then compared with in vivo KE data, for example hyperuricemia (Table 3). Because of known species differences in sensitivity, it was necessary to evaluate dose-response concordance within taxonomic groups. Gyps vultures have been identified as uniquely sensitive, but poultry have also been validated as an avian model for NSAID renal toxicity [118]. An evaluation of the empirical evidence available within these groups suggests the following relative potency for causing hyperuricemia: diclofenac > flunixin/ketoprofen/indomethacin > clofibrate/carprofen > meloxicam/phenylbutazone/aspirin (no data on ibuprofen or nimesulide were available). The overall ranking is nearly equivalent to the COX1 inhibition potency, although there are a few notable differences (elevated diclofenac and carprofen, reduced meloxicam). The only study to assess carprofen observed a very weak elevation of plasma uric acid in one of two treated vultures, with no subsequent toxicity, suggesting that the relative potency of this drug for the overall AOP is likely very weak, as predicted. Further, the stereospecificity of the carprofen used in the study was not provided, but the S enantiomer is known to be a more potent inhibitor of COX1 than the R enantiomer or racemic mixture [144]. Generally, however, differences in the toxicokinetics of the various drugs explain why some have elevated toxicity (e.g., diclofenac) and others are perhaps safer than predicted by COX1 inhibitory potency alone (e.g., meloxicam) [134, 135, 145]. While AOPs do not consider toxicokinetics, the marriage of the AOP framework with the AEP framework [6] or similar concepts, would potentially improve these comparisons by taking exposure and ADME-related considerations into account. In cases where apparent inconsistencies in dose-response concordance are readily explained by ADME or other factors, it is important to capture the explanations for the apparent lack of concordance as part of the KER descriptions [12].

Table 3.

Empirical evidence of COX1 inhibitor relative potency in causing hyperuricemia in two animal models: sensitive Gyps vultures and domestic fowl.

COX1 inhibition potency	chemicala	Gyps vultures			Poultry
		referenceb	dose (mg/kg); adminc	affected/exposed; %	referenceb	dose (mg/kg); adminb	affected/exposed; %
very potent	FL	1	1.0; food	(1/2); 50%	2	2.5/d × 4 d; IM	(1/5); 20%
					2	5/d × 4 d; IM	(2/5); 40%
potent	KP	6	0.5–1; food	not significant	7	10/d × 4 d; IM	not significant; 0%
		6	1.3–1.4; oral	not significant
		6	1.5,1.8; oral	(1/2); 50%
		6	5; oral	(4/8); 50%
potent	IM				9	2.5; IM	significant
						5; IM	significant
potent	DF	5	0.1–2.5; food	(2/2); 100%	13	1.25/wk × 5 wk; IM	(2/6); 33%
		10	0.25; oral	(2/2); 100%	13	2.5/wk × 5 wk; IM	(2/6); 33%
		10	2.5: oral	(1/2); 50%	13	5/wk × 5 wk; IM	(2/6); 33%
		11	0.8; oral	(2/2); 100%	13	10/wk × 5 wk; IM	(3/6); 50%
		11	0.8; oral	(3/3); 100%	14	2.5/d × 4 d; oral	(1/10); 10%
		13	0.8; IV	significant	14	5/d × 4 d; oral	(3/10); 30%
		12	0.8; IV	(2/2); 100%	14	10/d × 4 d; oral	(5/10); 50%
					14	2.5/d × 4 d; IM	(3/10); 30%
					14	5/d × 4 d; IM	(4/10); 40%
					14	1/d0 × 4 d; IM	(7/10); 70%
moderate	CF				17	400 on d 1, then 250/d × 4 d; SC	significantd
weak-moderates	MX	18	0.5; oral	(0/5); 0%
		18	1.0; oral	(0/5); 0%
		18	2.0	(0/5); 0%
		18	2.0	(0/14); 0%
		18	2.0	(0/21); 0%
		18	0.03–1.98	(0/6); 0%
		18	1.18–2.45	(0/6); 0%
		18	0.50	(0/3); 0%
		18	2.0	(0/3); 0%
		18	0.50	(0/2); 0%
		18	2.0	(0/2); 0%
		19	2; IM	(0/4); 0%
weak-moderate	CP	1	11.5; food	1/2; 50%
weak	PB	1	1.7; food	(0/2); 0%
weak	AS				21	10/d × 5 d	not significant

^aFL, flunixin; KP, ketoprofen; IM, indomethacin; DF, diclofenac; CF, clofibrate; MX, meloxicam; CP, carprofen; PB, phenylbutazone; AS, aspirin

^bReferences available in Supplementary data

^cAdministration routes: food (field or lab); oral (oral gavage), IV (intra-venous), IM (intra-muscular), SC (sub-cutaneous)

^dCF effects were only significant in chicks co-exposed to propylthiouracil to obtain hyperlipidemic/hypothyroidal phenotype

Evidence of temporal concordance is usually provided by time-course studies where both KEs are measured and the upstream KE is found to occur prior to the downstream. While this type of information is difficult to capture in a database, support is often found in toxicology, as well as pathology and general physiology studies. For example, research on the toxicity of NSAIDs in vultures report elevated levels of uric acid in the plasma generally at 24 h after exposure, followed by mortality at 36–48 h [127, 146, 147]. In terms of case study 1, general endocrinology studies in crustacean species detail how normal molting follows a peak Ec titer concentration during the inter-molt period [148, 149].

Concordance of incidence is an additional consideration when evaluating KERs. This concept posits that upstream KEs should occur with greater frequency than downstream KEs. Initially derived for evaluating causality in epidemiological studies, the greater incidence of the upstream KE reflects the reality of biological diversity and variable susceptibility within a population. Finding evidence of greater incidence of each upstream KE relative to the downstream KE at equivalent doses is likely to be a challenge for developers of ecotoxicological AOPs because of the relatively small number of replicates in most studies. However, where this kind of evidence is available, there is often valuable information to be gleaned. For example, observations of birds with initial elevated uric acid levels which did not succumb to toxicity, compared to those which were susceptible, spotlighted the importance of uric acid clearance rates in predicting survival [118, 145].

Empirical support from multiple test systems (chemical initiators, in vitro and in vivo systems, taxonomic models) also lend support for individual KERs, as well the AOP as a whole [12]. It is recommended that these results be distilled into a table which provides the temporal, dose-response, and incidence data necessary for evaluating the AOP [9, 139]. The provision of such concordance tables (e.g., Supplementary data, Tables S3, S5) may further assist researchers in identifying data gaps, biomarkers for monitoring, additional chemical initiators and linking AOPs.

CONCLUSION

Generally speaking, developing an AOP, in conjunction with evaluation of the degree of confidence in the AOP, requires significant effort, not only in reviewing toxicological data, but understanding the basic biology behind the pathway. From the genesis of the AOP, starting with a working knowledge of one or more components of the AOP, the developer may find the use of various databases (e.g., Table 1) helpful to identify additional elements of a pathway, to define the domain of applicability, and/or to assemble and evaluate weight of evidence. However, it is frequently necessary for the developer to confirm such support in primary literature. As living documents, AOPs are intended to be edited, expanded, and updated as data become available, new applications are envisioned, and additional developers with unique expertise are engaged. In this sense, it takes a community of developers and researchers to build AOPs and AOP networks. This process is facilitated as further guidance on development becomes available, and additional data resources and tools are identified, modified, or built to address AOP development needs.