Abstract
The need for assembled existing and new toxicity data has accelerated as the amount of chemicals introduced into commerce continues to grow and regulatory mandates require safety assessments for a greater number of chemicals. To address this evolving need, the ECOTOXicology Knowledgebase (ECOTOX) was developed starting in the 1980s and is currently the world’s largest compilation of curated ecotoxicity data, providing support for assessments of chemical safety and ecological research through systematic and transparent literature review procedures. The recently released version of the Knowledgebase (ECOTOX Ver 5, www.epa.gov/ecotox) provides single chemical ecotoxicity data for over 12,000 chemicals and ecological species with over one million test results from over 50,000 references. Presented here is an overview of ECOTOX, detailing the literature review and data curation processes within the context of current systematic review practices, and discussing how recent updates improve the accessibility and reusability of data to support the assessment, management, and research of environmental chemicals. Relevant and acceptable toxicity results are identified from studies in the scientific literature, with pertinent methodological details and results extracted following well-established controlled vocabularies and newly extracted toxicity data added quarterly to the public website. Release of ECOTOX Ver 5 included an entirely re-designed user interface with enhanced data queries and retrieval options, visualizations to aid in data exploration, customizable outputs for export and use in external applications, and interoperability with chemical and toxicity databases and tools. ECOTOX is a reliable source of curated ecological toxicity data for chemical assessments and research and continues to evolve with accessible and transparent state-of-the-art practices in literature data curation and increased interoperability to other relevant resources.
Keywords: Ecotoxicology, Environmental Toxicology, Database, Systematic review, New approach methodologies, Controlled vocabulary, Interoperability, Chemical assessment
INTRODUCTION
Rapid and cost-effective methods are needed to evaluate chemicals for risk to human health and the environment. Regulatory agencies worldwide have been increasingly mandated to assess hazards of large numbers of chemicals while researchers seek to further existing bodies of knowledge. Typically, the starting point for hazard assessment and research of contaminants is gathering existing empirical toxicity data. However, there are challenges in identifying, reviewing, and compiling toxicity test data in a thorough and transparent, yet efficient, manner. This is not a new problem, and the US Environmental Protection Agency (USEPA) began development of the ECOTOXicology Knowledgebase (ECOTOX) in the early 1980s with several ecosystem/taxa specific databases to provide regulatory USEPA Program Offices with rapid access to ecotoxicity data.
Furthermore, there was a need for documented systematic and transparent literature searching, acquisition, and curation procedures to provide toxicological data for regulators and researchers across the USEPA and other federal and state agencies. Risk assessors needed a cost-effective means of locating relevant ecological effects data to use in chemical risk characterizations at hazardous waste sites and assisting in the assessment of potential hazards of chemicals while administering legislated mandates associated with the Federal Insecticide, Fungicide, and Rodenticide Act; the Clean Water Act; the Comprehensive Environmental Response, Compensation, and Liability Act/Superfund Amendments and Reauthorization Act; and the Toxic Substances Control Act. Thus, ECOTOX continued to develop and is now an authoritative source of curated, consistently updated, single chemical environmental toxicity data relevant for ecological species, including both aquatic and terrestrial organisms, for use in risk assessment, risk management, and research.
Both toxicity testing and chemical regulation have evolved since the origination of ECOTOX. As the number of compounds and the need for rapid evaluation continue to increase, many new strategies have been developed to meet the demand for increased rate and efficiency in prioritizing, assessing, and predicting hazard and toxicity for both human health and ecologically relevant species. A National Research Council report, Toxicity Testing in the 21st Century: A Vision and A Strategy (2007), outlined a strategy to increase efficiency in evaluation of potential adverse effects of chemical exposure through a shift to greater reliance on high-throughput in vitro assays and computational modeling. This blueprint for change in toxicology contributed to recommendations for future toxicological risk assessments (U.S. EPA, 2014; Krewski et al., 2014). Progress subsequent to the release of this National Research Council report includes advancements in new approach methodologies and technologies, including in vitro high-throughput assays, toxicogenomics, and in silico, structure-activity, and other computational modeling (Krewski et al., 2020; Thomas et al., 2019). While human health assessments have been the driver of these shifts, many of these concepts and methods are relevant and increasingly being applied to ecological assessments (Villeneuve et al., 2019). Recent mandates to reduce animal usage for toxicity testing have further focused efforts on NAMs and led to efforts across the globe to transition to alternatives (ECHA, 2017; Herrmann et al., 2019; ICCVAM, 2018; USEPA, 2020a).
In this shifting toxicity testing paradigm, existing empirical data from in vivo toxicity testing have important roles in decision support on environmental effects of chemicals. At present, in vivo models continue to be the standard for chemical risk assessments. In addition, in vivo toxicity testing provides necessary empirical data for development of computational models and, moving forward, will bridge assessments based on new approach methodologies to those using the more traditionally defined apical endpoints. Verification and validation of new approach methodologies have been and continue to be a major challenge, and most evaluation frameworks require biologically relevant in vivo data for comparison (see Parish et al., 2020). Thus, the identification and curation of toxicity data support the development, evaluation, and adoption of new approach methodologies by providing accessible existing data for use in evaluations and modeling, and identification of data gaps to target with additional testing.
Concurrent with the shifts in testing approaches to support regulatory toxicology, the use of transparent processes for the evaluation of existing toxicity data for use in risk assessment has evolved significantly in the environmental health field over the last two decades with the emergence of evidence-based toxicology, establishment of study evaluation methods, and adoption of systematic review approaches (Guzelian et al., 2005; Hoffman & Hartung, 2006; Klimisch et al., 1997; Moermond et al., 2017; Ruden et al., 2016; Stephens et al., 2013, 2016, 2018; Thayer et al., 2014; Wikoff & Miller, 2018). The systematic review approach is a framework for identifying, evaluating, and synthesizing evidence that enhances transparency, objectivity, and consistency in reviews with a set of standard steps and clear criteria (Hoffman et al., 2017; National Toxicology Program [NTP], 2019; Woodruff & Sutton, 2014). While multiple applications in the human environmental health field have made progress in incorporating aspects of systematic review (see USEPA, 2020b), ecotoxicological assessments have lagged in their adoption of systematic review methods. Through the decades, the ECOTOX team has developed operating procedures for identifying, reviewing, and extracting toxicity data from the literature which are consistent with the attributes of standardized guidelines for systematic reviews (NTP, 2019; Texas Commission on Environmental Quality, 2017) and systematic evidence maps (Wolffe et al., 2019).
To date however, the ECOTOX project and literature review/data curation pipeline has not been described in the peer-reviewed literature and has not previously been framed within the context of current SR lexicon. This publication describes the recently updated ECOTOX (Ver 5), including an overview of the systematic methods used for literature search, review, and data curation, description of the new user interface, summary of the type and extent of currently available data, and examples of applications and tools using data from ECOTOX. With the recent updates, ECOTOX strives to meet the needs of researchers and risk assessors with accurate, updated, and easily accessible information following the FAIR principles of “Findable, Accessible, Interoperable, and Reusable” data (Wilkinson et al., 2016). Although this is an on-going process, ECOTOX Ver 5 represents a significant advancement in transparency and data accessibility, as well as advancements for data reusability and interoperability with other tools and databases. Soon after the release of the redesigned user interface, ECOTOX passed a major milestone and now includes over one million records of curated toxicity data from over 50,000 references. This evergreen database continues to support USEPA research and regulatory activities and is also a substantial source of toxicity records for other applications and databasing efforts including those for toxicity modeling (e.g., quantitative structure-activity relationships [QSARs], bioaccumulation), toxicity predictions (e.g., predicted no-effect concentrations and threshold values), and species sensitivity distributions (SSDs). Thus, this is an opportune time to provide the first comprehensive description of this project which has a long history of using systematic methods to identify and curate ecological toxicity data, with many similarities to contemporary systematic review methods, and recently added processes to increase transparency and consistency with FAIR principles.
METHODS
Over the decades, the ECOTOX team has developed a literature search, review, and data curation pipeline to identify and provide ecological toxicity data with consistency and transparency. Described here is an overview of the systematic methods and procedures from chemical verification and development of search terms through literature review and data extraction. The below section ‘ECOTOXicology Knowledgebase literature search, review, and data curation pipeline’ summarizes the steps in the pipeline and Figure 1 shows the flow chart of study selection, following the PRISMA (Preferred Reporting Items for Systematic Reviews and MetaAnalyses) guidelines from Moher et al. (2009). Each step follows well-established standard operating procedures (SOPs), which are summarized herein. There are currently three SOPs that cover the steps of the pipeline (literature searches, citation identification, and applicability criteria; data abstraction; data maintenance), as well as SOPs for species verification and entry, chemical verification and entry, outreach and technical support, and software development, all of which are available upon request. These SOPs include project- and application-specific documentation, such as the details needed for reviewers to select and navigate through the appropriate interfaces and forms for screening references and extracting data, and are updated quarterly to reflect any changes in procedures, incorporation of new efficiencies and shifts in file paths, and so forth.
Figure 1.
ECOTOXicology Knowledgebase (ECOTOX) literature search and study selection flow diagram, with general steps on left from PRISMA (Preferred Reporting Items for Systematic Reviews and MetaAnalyses [Moher et al., 2009], dark grey) and ECOTOX pipeline (light grey). Each step is described in the section ECOTOX literature search, review, and data curation. CASRN = Chemical Abstracts Service registry number; STN = Scientific and Technical Information Network; EPA/U.S. EPA = US Environmental Protection Agency.
Ecologically-relevant toxicity studies for chemicals of interest are identified through comprehensive searches and review of the open and grey literature (note that the grey literature includes government documents and reports that are not typically included in databases which index peer-reviewed publications). Screening of references includes review of titles and abstracts followed by full-text review. References move forward in the pipeline following the established criteria for applicability (e.g., in terms of ecologically-relevant species, chemical of interest, biological species identified, exposure concentration and duration reported) and acceptability (documented controls and reported endpoints). From each study meeting these criteria (listed in Table 1), relevant details on chemical(s), species, study design, test conditions, and toxicity results are extracted into the structured database (see example data fields in Table 2 and full list in Supporting Table S1). In the following section, we summarize this pipeline and the ECOTOX SOPs.
Table 1.
Criteria for inclusion in the ECOTOXicology Knowledgebase (ECOTOX) shown side by side with these same requirements formatted as a population-exposure-comparator-outcome (PECO) statement.
| Requirements/Inclusion Criteria from ECOTOX SOP | General ECOTOX PECO | |
|---|---|---|
|
P
(Population) |
• Ecologically relevant species • Live, whole organisms • Organism taxonomic information verifiable against standard taxonomic sources • Priority species are wild (test results for terrestrial domestic and laboratory species are used to fill data gaps when needed) • In vitro studies (with viable cells or tissue) flagged for possible inclusion as requested by Programs. • NOT: humans, monkeys, bacteria, viruses, yeast |
Animal: Aquatic and terrestrial species (live, whole organism) of any life stage (including preconception, in utero, lactation, peripubertal, and adult stages). Include vertebrates (wild mammals [e.g. Peromyscus sp.], fish, amphibians, reptiles, birds) and invertebrates (crustaceans, molluscs, insects, spiders, worms, other invertebrates). Bacteria and viruses are not included. Plants: Aquatic and terrestrial species (live), all plants including algal, moss, lichen and fungi species |
|
E
(Exposure) |
• Verifiable Chemical Abstracts Service (CAS) registry number • Single-chemical exposure • Relevant to environmental exposure • Report exposure concentration, dose, or application rate • Report duration of exposure • Sediment studies must have a water concentration reported to be included • NOT: Air pollution studies related to CO2 and ozone |
Relevant forms: Chemical of concern, name and CAS registry number (plus synonyms, tradenames); when requested: metabolites, degradates, parent compound, and related chemicals. Animal: Any exposure to relevant forms of the chemical of concern including via water, injection, diet, and dermal, with reported concentration and duration. Inhalation studies are excluded unless this is the primary route of environmental exposure (e.g., for volatile compounds). Plants: Exposure to relevant forms of the chemical of concern via water or soil, with reported concentration and duration |
|
C
(Comparator/ Control) |
• Must have a control treatment a |
A concurrent control group exposed to vehicle-only treatment and/or untreated control (control could be a baseline measurement) |
|
O
(Outcome) |
• Biological effect measured • Effect concurrent with associated chemical exposure |
All biological effects (including bioaccumulation from laboratory studies with concurrently measured water and tissue concentrations) |
| Publication /Data Format | • Primary source of the data • Study must be a full article in English NOT: Review article or abstract citation only |
Studies with documented controls and statistically-derived endpoints meet the acceptability criteria; however, there are instances in which data are included in ECOTOX without meeting these criteria, under the condition that they occurred in a publication with statistically analyzed endpoints compared to a control. For example, when standard methodologies were cited in the experimental design without explicitly naming a control exposure, LC50s were reported without a control exposure, and histopathological observations reported without calculated endpoints or statistical analysis.
Table 2.
Types of data extracted from each reference for curation in ECOTOXicology Knowledgebase (ECOTOX; if applicable and reported), with category, data fields, and examples of how specific fields in ECOTOX can inform study evaluation questions.
| Category | Data Fields (not all inclusive) | Select study evaluation questions with relevant ECOTOX field(s) |
|---|---|---|
| Chemical | Chemical Name, Chemical Abstracts Service registry number, grade, purity, formulation, carrier Test-specific: Analysis, application type and rate/frequency, number of doses, doses, concentration type (e.g., active ingredient or formulation), concentration/dose associated with each effect and/or endpoint |
Is test substance identified? a,b,c
Required for inclusion in ECOTOX
Is the purity of test substance reported? a,b,c Chemical Purity Were chemical concentrations verified?c Chemical Analysis (e.g., nominal versus measured concentrations) |
| Species | Scientific and common name, taxonomy, lifestage, age, initial and final weight, Gender, Source | Is the species given? a,b,c
Verifiable species (Scientific Name, etc.) required for inclusion in ECOTOX Are the organisms well described? a,b,c Organism Source, Lifestage, Age, Gender, Initial and Final Weight |
| Study Design/Test Conditions | Test method, media type, test location, exposure and study duration, control, experimental design, physical and chemical soil and water parameters | Are appropriate controls performed? a,b,c
A control is required for inclusion in ECOTOX, type described in Control
Is a guideline method (e.g., OECD) used? a,c Test Method Are the experimental conditions appropriate and acceptable for the test substance and organism? a,b,c Test Method, Media Type, Test Location, Experimental Design, Physical and Chemical Soil and Water Parameters (e.g., pH, Temperature, Dissolved Oxygen) |
| Test Results | Effect (observation of a response): general effect groups and specific effect measurements, endpoint (quantification of an observed effect, e.g., LC50), trend, response site, effect percentage, statistical significance and level, observed duration (exposure duration when result observed), bioconcentration (BCF or BAF) with units | Are the reported effects and endpoints appropriate for the purpose, test substance and organism? a
Effect Measurement, Endpoint Is the response/effect statistically significant? a Statistical Significance, Significance Level |
The full list of data fields with definitions is in Supporting Table S1.
Criteria for reporting and evaluation ecotoxicity data – relevance and reliability (Moermond et al., 2016).
Relevance and reliability of experimental data in human health risk assessment (Kaltenhäuser et al., 2017).
ToxRTool, Reliability assessment of in vivo toxicity studies (Schneider et al., 2009).
OECD = Organisation for Economic Co-operation and Development; LC50 = lethal concentration to 50% of test organisms; BCF = bioconcentration factor; BAF = bioaccumulation factor.
Many of the steps in this pipeline are consistent with systematic review methods which have recently been adopted in toxicology and environmental assessments. ECOTOX, however, is an ongoing project that continually identifies and reviews toxicity studies, and as such, the protocols were developed with a generalized approach as compared to a systematic review for a single chemical or set of chemicals. In addition, ECOTOX provides ecologically relevant toxicity results to multiple partners and applications, and thus extracts and manages data suitable for predecisional analysis including sufficient chemical, methodological, and toxicity result details for users (e.g., USEPA Program Offices) to complete study quality evaluations and use the results to reach conclusions.
The procedures described herein have been reviewed and refined based on results from many years of identification of applicable references. The protocols are reviewed and updated on a continual basis, with the steps, sources, exclusionary reasons, and controlled vocabulary updated as needed. Updates and changes to the SOPs are made annually for quality assurance review and tracking, with relevant changes to the processes added to the documentation in both the User Guide and online “Help”.
The ECOTOX Knowledgebase was developed to enter, review, and manage the references and toxicity results identified through the above described pipeline. This Knowledgebase consists of two components: 1) the internal back-end Oracle database and data entry user interface referred to as “Unify” into which the chemical, reference, and toxicity tests and results are entered and managed (see below, ECOTOX Infrastructure: Unify); and 2) the public Oracle database and web-based user interface that pulls information populated from Unify to provide data from studies that met the criteria for inclusion, which is generally recognized by USEPA and our users as “ECOTOX” and recently underwent a complete redesign with added functionality and features (see below, ECOTOX Ver 5.0: Updated web-based application). “Data” within ECOTOX are considered the records that are extracted from ecologically relevant toxicological studies, entered into Unify, and then provided in the public ECOTOX user interface. These records (or “data”) include details on the chemicals, species, study design, test conditions, toxicity results, and statistically-calculated or derived endpoints.
ECOTOXicology Knowledgebase literature search, review, and data curation pipeline
Study selection criteria.
The inclusion criteria for ECOTOX were developed to select ecological toxicity data possibly relevant for ecological risk assessments and subsequent risk management decisions or policy. These inclusion criteria are summarized in Table 1 (second column) and consist of requirements for applicability as well as acceptability. The minimum data requirements for applicability include ecotoxicologically relevant criteria (e.g., ecologically relevant species, quantified chemical exposure amount and duration, biological adverse or neutral response to chemical exposure) as well as basic study reporting and quality requirements such as verifiable chemical and biological species (full descriptions in Supporting Table S2). Wild species have been prioritized for inclusion; however, test results for terrestrial domestic and laboratory species are used to fill data gaps when needed. In addition, acceptability criteria were developed to require documented controls and reported endpoints. These inclusion criteria can be stated in the form of a population, exposure, comparator, and outcome (PECO) statement commonly used in systematic review. The two formats are displayed side by side in Table 1.
Development of search terms.
Most of the details in the inclusion criteria (and PECO statement) are general across all efforts; however, additional information is needed about each requested chemical to complete the exposure (E) section and develop search terms. Each chemical of interest is reviewed through a verification process to unequivocally link the chemical name to a specific Chemical Abstracts Service Registry Number (CASRN) and identify any synonyms, tradenames, and other relevant forms of the compound which should be considered for inclusion in the literature search (e.g., metabolites, degradants, parent compounds, related chemicals). This information is identified with resources, including, but not limited to, Pesticide Action Network, USEPA’s Pesticide Fate Database, and USEPA’s CompTox Chemicals Dashboard. The resulting chemical search terms are reviewed by the curation team and the partners requesting toxicity data for specificity to the chemical of interest and compiled for inclusion in the literature search.
Literature searches and sources of toxicity data.
Literature searches are conducted using selected search terms as described above and shown in Figure 1. The majority of the literature searches are chemical-specific using the names and CASRNs for the target and related chemicals to search multiple databases within a narrow window of days to identify relevant references. The current list of databases includes Scopus, ProQuest, Dissertation Abstractions, PubAg, and Web of Science. This list of databases has remained fairly consistent over the years, although there have been recent shifts based on availability and use of web services (e.g., Scopus was recently incorporated as a replacement for ScienceDirect) and there are unique efforts that require additional sources (e.g., inclusion of PubMed when human health data is also of interest).
Chemicals included in these searches are based on the needs of our partners, such as requests from USEPA Program Offices and researchers within the Office of Research and Development with anywhere from 48 to 90 chemical-specific searches conducted per year. Each literature search is conducted by ECOTOX staff specializing in this area, utilizing automation wherever possible, and documenting the search terms and sources searched, the resultant number of citations from each search, and the disposition (or status) of each reference (applicable/not applicable and reason for exclusion, as identified in the next step) in standardized memos and file formats. This documentation is retained and provided to our partners for requested chemicals.
In addition to chemical-specific literature searches spanning all years using the databases described above, titles and abstracts are collected and reviewed monthly from the table of contents of several journals which historically have had a high frequency of ECOTOX-relevant references. This set of journals was identified based on evaluation of the applicability rate in the results from ECOTOX literature searches and is reviewed annually, with journals added or removed as needed. References identified with this literature search process are also reviewed with the inclusion criteria in Table 1 and often include studies for chemicals that are not currently prioritized by the USEPA’s research or Program Offices. These references are added to the extraction queue, where they remain until chemicals are either prioritized and/or resources are available to complete the screening and data extraction process. Thus, curation includes toxicity data for both target chemicals for the USEPA research and Program Offices as well as those of interest in the broader ecotoxicological community based on studies in recent publications.
In addition to references identified through the literature search process, ECOTOX includes data from several highly relevant toxicity databases. These include the USEPA fathead minnow acute toxicity database (Duluth laboratory), the USEPA Office of Pesticide Programs Ecotoxicity Database, and the US Geological Survey (USGS) Acute Toxicity Database, as well as independently compiled data sets from France, Germany, The Netherlands, and Russia (see User Guide, Appendix F for details). These sources make up a small fraction (<6%) of the total records.
Screening References.
ECOTOX has a tiered system for identifying studies to include (shown in Figure 1). Potentially applicable studies are first identified through a screening of the titles and abstracts from the literature search using the criteria (or PECO) described in Table 1. Studies that pass the title/abstract screening are acquired and the full text of the article is reviewed using the same criteria. References which are excluded at either the title/abstract or full-text review stage are labeled (‘tagged’) with the primary reason for exclusion (e.g., ‘chemical methods’, ‘human health’, ‘survey’, ‘mixture’; see Supporting Tables S2 and S3 for minimum data requirements and a list of exclusionary reasons). The disposition (or status) of each reference is documented (applicable/not applicable, reason for exclusion) in the citation files and in Unify. Therefore, studies identified currently as ‘non-applicable’ are documented and the citations are retained for potential future review for different projects or if there are changes in applicability or acceptability criteria.
Data extraction.
For references that meet ECOTOX inclusion criteria, data are extracted for each reported toxicity test, including details on the test methods, species, chemical(s), and toxicity results (see examples in Table 2, full list in Supporting Table S1). Toxicity tests within a reference are considered unique tests based on differing study designs (e.g., new chemical, new species, new life stage, new exposure type) and are given a computer-generated identification number (“test number”). Each toxicity test typically includes measurements for multiple types of effects, each with one or more associated endpoints (statistical quantification or calculation of the observed effect, e.g., lethal concentration to 50% of test organisms [LC50], no-observed-effect concentration [NOEC], lowest-observed-effect concentration [LOEC]). The combined unique experimental design, effect measurement, and associated endpoint(s) are extracted as separate data records with a unique “result number” assigned to each record. Thus, each reference may include multiple toxicity tests and multiple data records (see examples in Figure 2).
Figure 2.
Examples of data extracted from references that have met applicability and acceptability criteria, with effects and endpoints measured and reported: [A] one study with amphibians and perfluorooctanesulfonic acid, and [B] three studies with aquatic organisms and metals. Each reference could include multiple tests, each with a unique test design with different chemicals, species, life stages, exposure types, and so forth (e.g., Claesson and Tomqvist, 1988). From each reported test within a reference, each effect measurement and associated endpoint(s) are extracted as separate data records in the ECOTOX Knowledgebase. Thus, there can be many data records for each reference. PFOS-K = perfluorooctanesulfonic acid‐potassium; CRF = corticotropin releasing factor; HSP70 = heat shock protein 70; PPAR = peroxisome proliferator–activated receptor; NOEC = no‐observed‐effect concentration; LOEC = lowest‐observed‐effect concentration; NR = not reported; EC50 = effective concentration for 50% of test organisms; LC50 = lethal concentration to 50% of test organisms; LT50 = median lethal time.
For each toxicity test within a reference, up to 90 different study entities (e.g., organism, chemical, media type, type of control, endpoint, statistical significance) are extracted in standardized fields with controlled vocabularies. In order to fully capture the relevant and reported information, many entities include additional fields for specific comments as well as ranges and units, resulting in possible totals of up to 300–400 data fields for terrestrial and aquatic studies, respectively. Extraction of study details and results is based on what is reported by the author(s) and follows SOPs using controlled vocabulary in a form-based data curator interface within the Unify system, which includes standard quality review for all extracted data. Supporting Table S1 includes the list of data fields and associated appendices with controlled vocabularies (where relevant). The data fields and controlled vocabularies with definitions, are also documented in the online “Help” in “Data Fields Definitions and Terms” (https://cfpub.epa.gov/ecotox/help.cfm?sub=wi-definitions) and the “ECOTOX Terms Appendix” (https://cfpub.epa.gov/ecotox/help.cfm?sub=term-appendix).
The established data fields include standardized unique identifiers (e.g., CASRN for Chemicals, National Center for Biotechnology Information’s [NCBI] taxonomy identifier [TaxID] for species) and ECOTOX-specific controlled vocabularies developed through reviewing the ecotoxicological literature (e.g., media type, effect measurements, endpoints, test method and type). Numerical fields (e.g., organism age, number of doses, observed duration, concentration, temperature) include fields for the mean, minimum, and maximum values as well as operators (>, >=, <, =<, =) and units. Also, text “comment” fields are included for additional relevant study details. Toxicity results include reported endpoints (e.g., LC50, LOEC or lowest observed effect level [LOEL], NOEC or no observed effect level [NOEL], effective concentration for x% of test organisms [ECx]) for each effect measurement (e.g., total lipids, biomass, mortality) in a study, with measurements categorized into 23 effect groups (e.g. biochemistry, growth, reproduction, mortality, enzymes, genetics [which includes “omics” data]). Presently, there are 30 separate endpoints and over 5000 effect measurements defined in the ECOTOX controlled vocabularies. The ECOTOX Knowledgebase includes the treatment doses or concentrations associated with these effects and endpoints; however, due to priority given to statistically-derived endpoints and limited resources, full extraction of responses at each treatment (or dose) is not completed. As described above, each reference may include multiple toxicity tests, each with one or more effect measurements and endpoints, resulting in multiple records to represent all measured endpoints (including “no effect” results). Examples of effect groups, effect measurements, and endpoints are shown in Figure 2 for data extracted from individual studies and Figure 3 with data from multiple aquatic studies on aluminum chloride (CASRN 7446–70-0).
Figure 3.
Example data for aluminum chloride (Chemical Abstracts Service registry no.: 7446–70-0) showing results for aquatic species with exposure concentration (y-axis) by type of effects (x-axis) and [A] species group (point color, shape) or [B] endpoint (point color, shape). Currently there are 731 aquatic records for this chemical from 102 publications, of which 492 have exposure concentrations that can be standardized to parts per million and displayed in the “Explore” data visualization plots. Note that darker shades indicate overlapping points. LOEC = lowest‐observed-effect-concentration; NOEC = no‐observed‐effect concentration; NR = not reported; NR-ZERO = 0% mortality; NR-LETH = 100% mortality; LCx = lethal concentration to x% of test organisms; LT50 = median lethal time; EC50 = effective concentration for 50% of test organisms; BCF = bioconcentration factor; ICx = x% inhibition concentration; MATC = maximum acceptable toxicant concentration.
The data extracted into and managed within ECOTOX are useful for scoping or summarizing the extent or type of available data and providing relevant information for evaluating studies for inclusion in analyses, but no decisions are reached within ECOTOX. Data analysis and synthesis, as well as interpretation, are application- and assessment-specific and therefore are conducted outside of the structure of the ECOTOX Knowledgebase web application. However, the ECOTOX criteria for inclusion overlap with common study evaluation questions, and many of the data fields extracted from studies provide the necessary information to answer more detailed study evaluation questions (see Table 2). For example, for the common study evaluation questions “Is the test substance unequivocally identified?”, “Is the test species given?”, and “Were controls included?”, have already been answered as ‘yes’ for studies meeting the ECOTOX criteria for inclusion. Additional fields in ECOTOX that provide information for study evaluation include “test method”, “doses”, “exposure sample number”, and “exposure duration”, which could be used for evaluation of the experimental design either directly (e.g., “Is a guideline method used?”, “Is the number of animals per group given?”, “Is the exposure duration defined?”) or to inform professional judgements (e.g., “Were there an appropriate number of doses?”, “Are test concentrations relevant in the context of addressed problem?”, “Is the duration of exposure appropriate for the studied endpoints?”). These fields can thus be informative for filtering studies for more restrictive inclusion criteria or to complement formal study quality or risk of bias evaluations that require review of the original publications. In such scenarios, we recommend that users of ECOTOX also consult the original publications as not all nuances of study design and toxicity results can be conveyed in the structured database. To aid in study evaluation, we recently released an optional output format which includes the fields most pertinent for evaluating relevance, reliability, and/or risk of bias, based on recent review of evaluation guidelines (Kaltenhauser et al., 2017; Moermond et al., 2016; Schneider et al., 2009).
Data maintenance and updates.
As an evergreen database, ECOTOX continually adds references and records. In addition, previously added records are routinely updated to reflect current recommended terms and correct entry errors. For biological species, changes in taxonomy and scientific names are updated annually using a complete download from the USGS Integrated Taxonomic Information System (IT IS) database (http://www.itis.gov/). Species are updated with any changes in common names, synonyms, geographic terms, taxonomic serial number, and parent taxonomy. Previously recognized species names that are no longer used for taxonomic classification are retained as historical names and can be used in ECOTOX searches (with the currently accepted name provided in the output). Preferred names of chemical, CASRNs, and terms in the controlled vocabulary (e.g., effect measurements, endpoints, exposure types) are also updated as needed to provide consistency and clarity. These updates are typically identified through quarterly review of terms or during incorporation of records from new literature and are documented in the relevant Unify modules and/or terms appendices. When deleted or alternate CASRNs are identified, these are maintained as historical CASRNs associated with the currently accepted/preferred CASRN, and all records updated to the preferred CASRN. Corrections for entry errors are completed as needed. Entry errors are identified internally (through review and quality control [QC] procedures) as well as through reports by users. Any reported errors are reviewed in comparison to the original source, with corrections made and released with the quarterly updates. Documentation of the error and corrections is maintained within the project files.
ECOTOXicology Knowledgebase Infrastructure: Unify
ECOTOX’s Unify system is the platform into which the chemical, reference, and toxicity tests and results are entered and managed. Unify maintains data provenance (all records traceable to the original source), enables quality assurance (QA) processes and QC checks, manages a backlog of potentially relevant references for future review and data extraction, and uses a unified extract-transform-load (ETL) processing platform for populating the ECOTOX public database reliably and efficiently. Unify was developed to modernize and collate the ECOTOX data entry process and initially merged the three separate taxa-specific databases (AQUIRE, PHYTOTOX and TERRETOX) that previously stored all ECOTOX data. This change resulted in our current system, which is comprised of a sophisticated browser-based user interface using a ColdFusion model-view-controller framework on top of an enterprise-quality Oracle relational database with integrated modules for entry, management, and/or querying of chemicals, species, references, and/or toxicity data. The relational online transaction processing (OLTP) data model is optimized for multi-user real-time data entry and referential integrity.
Curation is conducted manually and standardized using a form-based graphic user interface for abstraction (locating and recording of details from published studies). Entry includes controlled vocabularies, drop-down and selection fields, and extensive data validation. The modules and vocabularies used are described as follows:
Chemicals. Chemicals are catalogued by CASRN, with associated chemical name and synonyms from CompTox Chemicals Dashboard (Williams et al., 2017, https://comptox.epa.gov/dashboard/) and other sources (e.g., PubChem, ChemSpider). Chemicals go through a process to verify chemical name, structure, and CASRN using these sources. Chemicals for which a CASRN cannot be located through these sources are labeled as “archived” and associated data are not entered until the chemical can be verified. The Chemicals module includes chemical properties, uses, and general structural groups (e.g., polyaromatic hydrocarbons [PAHs], neonicotinoids, polychlorinated biphenyls [PCBs]). As part of the updates described here, the Distributed Structure Searchable Toxicity Database Substance ID (DTXSID, Grulke et al., 2019) was added to increase interoperability with databases that include physicochemical properties, exposure, uses, and so forth.
Species. Test organisms are identified by their current scientific name, which is verified using the ITIS. If the taxonomic data are not available with ITIS, other sources may be used (including taxa-specific sources such as AlgaeBase [www.algaebase.org] and Xenbase [www.xenbase.org]). For each entry in the “Species” module, the verified name, taxonomic kingdom, nomenclature history, and verification sources are retained in the Unify record. Name synonyms (e.g., previous scientific names) and taxonomic/species groups (e.g., fish, standard test species) are included in the Unify “Species” module and available as ECOTOX query options. As part of the updates described, the NCBI TaxID (www.ncbi.nlm.nih.gov/taxonomy, Federhen 2012) was added and is available for approximately 80% of species in ECOTOX. The addition of the NCBI TaxIDs allows for greater interoperability with the USEPA Sequence Alignment to Predict Across Species Susceptibility (SeqAPASS) tool that uses protein sequence comparisons to make predictions of chemical susceptibility across species (LaLone et al., 2016).
References. Each new reference that advances from title/abstract screening (identified as applicable or needing full-text review) is added to the “References module” for further tracking and processing. This module includes standardized bibliographic information, workflow and tracking of the physical paper, full-text screening and abstraction information for references, and project report tracking and generation. When possible, digital object identifier (DOI), PubMedID, master record ID (MRID), and Health & Environmental Research Online (HERO) ID are added to references in this module as additional identifiers for tracking and cross-referencing.
Toxicity. The “Toxicity” module was developed for efficient online data extraction from each reference, with separate sections for entity entry information of chemical(s), species, study design and parameters, and results. Entry parameters for all fields are defined (Supporting Table S1) and selected from controlled vocabularies developed over the course of decades of reviewing ecotoxicological literature (terms are defined in the ECOTOX Terms Appendix: https://cfpub.epa.gov/ecotox/help.cfm?sub=term-appendix). Efforts to refine and map ECOTOX terms to standardized vocabularies (e.g., NCBI Entrez-Gene IDs, the National Library of Medicine’s Unified Medical Language System) are underway.
Administration. The “Administration” module includes internal lookup codes, users, and roles.
Intensive training and QA procedures are implemented throughout the steps in the ECOTOX pipeline, and trained abstractors follow the well-documented procedures described in multiple SOPs, with close interaction with a Data coordinator. The QA process is an ongoing component of literature abstracting. Most data fields have validation files (lists of specific allowable terms) to control entries, and electronic QA checks to ensure integrity of terms between data fields and consistency of data within fields. All entries are assessed for QA by trained staff to ensure that data have been accurately included and entered. More details on the data curation processes, including Unify modules, controlled-vocabularies, and QA/QC processes are available in the online “Help” section (https://cfpub.epa.gov/ecotox/help) and by request.
ECOTOX Ver 5: Updated web-based application
The curated toxicity data from studies which meet inclusion criteria are loaded from the Unify OLTP database into the public-facing ECOTOX data warehouse via a quarterly ETL process. The Oracle data warehouse is optimized for query operations by employing a star schema design and Oracle materialized views. The ECOTOX Ver 5 web-based application incorporates significant updates which were first implemented in 2018 to meet the data needs of current toxicological assessments, and updates are ongoing in response to increasing demands for rapid access to relevant and reliable data to meet regulatory mandates. These updates include integration of more diverse effect measurements (e.g., gene expression changes and biochemical responses), simplified outputs, and a completely redesigned user interface that incorporate data exploration and visualization tools which allow users to more efficiently search and use the existing toxicity effects data.
Redesign of the user interface was launched in August 2018 with the goal of increased accessibility and functionality for querying and exploring available data. New functionality and features have been added each quarter since our initial release. This change also improved the application architecture by separating the front-end user interface logic from the back-end database query logic. The web application utilizes an open-source development stack including HTML5, CSS, JavaScript and JQuery/Ajax using the Node.js framework. Additionally, the HighCharts JavaScript library was used for the new visualization plots feature.
In this newly designed user interface, data are presented and downloadable in tabular formats, with data query and filter options in the “Search” and “Explore” modules. Data can be queried directly through the “Search” module with options to input specific terms or filter by category (e.g., chemicals, effects, endpoints, species, test conditions, publication[s]). Data are available to download in Microsoft Excel or delimited (.txt) format in the “Search” module, with 22 fields included in the basic default display with the option to include additional data fields for download, if desired. The novel module “Explore” allows discovery and exploration of available data through interactive filters for chemicals, species, effects, measurements, endpoints, and publication years. “Explore” also includes the option to visualize the data in dynamic plots, which display results by exposure concentration, duration, effect, chemical, species, and so forth. Selected query filters in “Explore” can be sent to the “Search” module for further refinement prior to downloading data. Details of how to navigate ECOTOX Search and Explore functions, as well as query and download data can be found on the “Help” page (https://cfpub.epa.gov/ecotox/help) and in the ECOTOX User Guide found therein, and described below (see Strategies for navigating ECOTOX Ver 5 for different applications).
RESULTS & DISCUSSION
Current data landscape
The ECOTOX Knowledgebase captures a robust representation of the published ecotoxicological chemical testing literature for the past several decades. As such, the data can be utilized to reflect trends in the distribution of species, chemicals, and biological effect measurements in cumulative proportions and in shifts over time in the field of ecotoxicology. In addition, reviewing the current data landscape can be used to prioritize literature searches and data curation, as well as identify data gaps for chemical and species groups. At the time of this writing (data update from June 15, 2021), ECOTOX includes extracted data from 52,274 references with toxicity data for 12,326 unique CASRN numbers and 13,610 biological species, for a total of 1,070,215 test records.
Chemicals with toxicity data in ECOTOX include, but are not limited to, groups such as metals and organometals, pesticides and herbicides, major ions, PAHs, polybrominated diphenyl ethers, PCBs, pharmaceuticals, and per- and polyfluoroalkyl substances. The number of references and records varies widely across chemicals (Supporting Figures S1 and S2). Some chemicals have been well-studied and documented in the published literature, such as copper sulfate (CASRN 7758–98-7) with 32,475 records extracted into ECOTOX from 2660 publications, while other chemicals are represented with only a few references (i.e., > 6100 unique chemicals have only one reference in ECOTOX). The list of chemicals included in ECOTOX has been, and will continue to be, driven largely by USEPA regulatory and research needs; however, improved coverage across the chemical universe through emphasis on recent publications (with review of monthly publications from target journals) and prioritizing literature searches for data-poor chemicals is an objective moving forward.
The ECOTOX Knowledgebase includes data for >13,000 biological species, with a distribution that reflects the somewhat uneven taxonomic representation in the ecotoxicological literature. For example, a select set of model species (e.g., rainbow trout, northern bobwhite quail, Norway rat) has historically been used for most toxicity testing, and the 227 ‘standard test species’ in ECOTOX (described under Species Verification in “Help”, https://cfpub.epa.gov/ecotox/help.cfm?sub=wi-documentation) not surprisingly represent 45% of the records. In fact, 100 of the 13,610 species account for 50% of the total number of records, whereas >9000 species have data from only one or two references (Supporting Figure S3). Fish are the most highly represented taxonomic group, with 25% of the records (273,693 records) and >13,000 references in ECOTOX. Taxonomic groups with the fewest records and references are reptiles, moss, and hornworts. Within the taxonomic groups, the amount of available data varies greatly across individual species. For example, looking at the fish species with the highest number of records in ECOTOX, four standard test species [rainbow trout (Oncorhynchus mykiss), fathead minnow (Pimephales promelas), common carp (Cyprinus carpio), and bluegill (Lepomis macrochirus)] represent 0.4% (4/1000) of all fish species in ECOTOX yet account for 33% of both the references (4433/13,281) and records (90,183/273,693) for fish (Figure 4A). Zebrafish (Danio rerio) is among the top five fish species (based on number of records); however, prior to 2009, this species contributed relatively few records to the total number of records for fishes. Currently, zebrafish are ranked second in total number of records for fish species (Figure 4B), in part because of the “crossover” status as a human health model species for neurodevelopmental toxicity testing (d’Amora and Giordani 2018). Zebrafish together with fathead minnow, rainbow trout, common carp, and bluegill reflect 47% (127,520/273,693) of records from fish species. This uneven distribution of available ecotoxicity data highlights the need to conduct “refresh” literature searches for chemicals to potentially fill data gaps for important species or species groups (e.g., threatened and endangered species), as well as to develop linkages to tools that evaluate species similarities (e.g., USEPA’s Web-based Interspecies Correlation Estimation [Web-ICE], SeqAPASS).
Figure 4.
Distribution of data in the ECOTOXicology Knowledgebase for fish species: (A) number of records across all fish species (as of June 2021 update); (B) number of references for five commonly studied fish species by decade of publication.
The types of effects studied and reported has changed substantially over the decades of ecotoxicological research that has been conducted. Many publications now include a wider variety of types of effects (e.g., behavioral, transcriptomics) in addition to the standard growth, development, reproduction, and mortality endpoints historically reported in the literature. The ECOTOX Knowledgebase continually incorporates effect measurement terms to the controlled vocabulary to reflect these shifts in reported toxicity results. Currently, ECOTOX includes toxicity data from 6070 author-reported unique effect measurements, including a great diversity of endpoints beyond the standard apical toxicity measurements (growth, reproduction, and mortality). These include measurements of organ- and cellular level effects (e.g., histopathology, organ-to-body weight indices, red and white blood cells), hormone, enzyme, and other biochemical changes (e.g., thyroxine, cholesterol, aromatase activity), genetic effects (e.g., DNA or RNA concentration, DNA methylation, chromosomal breaks, transcription of a wide variety of genes), chemical accumulation (e.g., body burden, uptake), and behavioral changes (e.g., general activity, feeding behavior, swimming, orientation) (Figure 5A). Measurements of mortality, growth, and reproduction make up 47% of the total records and are included in 39%, 32%, and 12% of references, respectively. Subapical biochemical, cellular, and genetic effects are included in 28%, 9%, and 7% of references respectively; and combined these three types of effects account for 20% of the records.
Figure 5.
Distribution of types of effects data in the ECOTOXicology Knowlegebase (ECOTOX): (A) percent of records by general type of effects (as of June 2021 update); (B) percent of references that include each general type of effect (effect Groups as bars) by decade of publication with cumulative number of records in ECOTOX (black line).
The results in ECOTOX provide insight into historical trends of ecotoxicological testing. Data extracted from literature through the decades show that growth and mortality have been, and continue to be, the most often reported effects. Studies published prior to 1980 primarily report effects on mortality, growth, and physiology, with some reproduction (<10% of references) and population (~20% of references, mostly for algae and plants) effects (Figure 5B). The percentage of references reporting growth and mortality have stayed consistent through time (36–42%), while the number of studies reporting reproduction and behavior have grown from 10% to 15% and 6% to 12%, respectively. With technological advances in analytical, biochemical, and behavioral measurement techniques, measurements in reported toxicity testing have shifted. Increases in reported biochemical effects started in the 1980s, and references with biochemical endpoints doubled from 18% in the 1960s and 1970s to nearly 40% in the 2000s. These changes were followed by increasingly more cellular and genetic effects starting in 2000, with references increasing from approximately 3% to >10%. While traditional apical endpoints (growth and mortality) continue to be included in ecotoxicological testing, technological advances, in combination with chemical regulatory policy directives to reduce animal usage and to identify sub-organismal effects levels for apical endpoints in chemical assessment, have resulted in dramatic increases of biochemical, genetics, and cellular endpoints reported over the last decade. This trend is likely to continue.
Applications and users of ECOTOX
In addition to supporting USEPA research and regulatory activities, data curated into ECOTOX have been used for many applications in government, industry, and academia (Table 3). Within the USEPA, the data in ECOTOX contribute to development of Ambient Water Quality Criteria for Aquatic Life, ecological risk assessments for chemical registration and re-registration, chemical prioritization and assessment under the Toxic Substances Control Act, and for site-specific assessments and emergency response. Based on those that contact the ECOTOX team, other users include several (U.S.) Federal agencies, states, and tribes, international entities, academic researchers and educators, and industry. As no registration or log-in is required to search ECOTOX data, other resources such as Google Analytics and peer-reviewed publications have been relied upon for information on the volume of use, types of users, and applications. Recent volume of use was assessed by Google Analytics from March 2019 to June 2021, during which time there was an average of 19,820 page views per month (range of ~14,400 to ~23,900 per month).
Table 3.
Applications using empirical toxicity data curated in the ECOTOXicology Knowledgebase, with examples, tools, and references.
USEPA = US Environmental Protection Agency; SSL = soil screening level; ICE = Interspecies Correlation Estimation; NOAA = National Oceanic and Atmospheric Administration; CAFÉ = Chemical Aquatic Fate and Effects; QSAR = quantitative structure‐activity relationship; OECD = Organisation for Economic Co‐operation and Development.
There are hundreds of publications that reference ECOTOX with the majority occurring over the past two decades, from a variety of journals related to toxicology, supporting work conducted in numerous countries around the world (Supporting Table S4). Many of these publications use toxicity data from ECOTOX for specific applications, while others, including many book chapters on ecotoxicology, refer to ECOTOX as an example of available toxicity data in general but also for specific compounds. Many tools and applications incorporate data from ECOTOX, including those focused on SSDs (e.g., USEPA’s Web-ICE, National Oceanic and Atmospheric Administration’s Chemical Aquatic Fate and Effects [CAFÉ]), predicted no-effect concentrations and threshold values (e.g., ecological threshold of toxicological concern, EnviroTox, NORMAN), QSARs (e.g., the ECOlogical Structure Activity Relationships [ECOSAR] predictive model, Toxicity Estimate Software Tool [TEST], the Organisation for Economic Co-operation and Development’s [OECD] QSAR Toolbox, Assessment Tools for the Evaluation of Risk [ASTER]), bioaccumulation modeling and validation, and Adverse Outcome Pathway (AOP) development (see Table 3 for examples). The recent updates to the ECOTOX Knowledgebase increase the ease of identifying and accessing relevant data for applications such as those listed above, and as described in the following section.
Strategies for navigating ECOTOX Version 5.0 for different applications: components and examples
This section provides an overview of how to navigate ECOTOX to query or download data, with a focus on examples of how the updates and improvements have increased the accessibility and transparency of ECOTOX data. Extensive details can be found on the ECOTOX “Help” page (https://cfpub.epa.gov/ecotox/help) and in the User Guide (most recent version available online, link in “Help” page). Illustrative examples of applications of ECOTOX are briefly described below (see Examples of EXOTOX applications), with details and associated tables and figures provided in Supporting Information (Supporting Tables S5-S9, Figures S4-S10). Note that the original scientific papers should be consulted for awareness of any nuances of the data retrieved from ECOTOX in the context of decision-making.
Initial landing page and getting assistance.
The initial landing page for ECOTOX is the “Home” page, which provides a general overview and contains links to the other pages within the application (e.g., “Search”, “Explore”, “Help”), as well as information on ECOTOX data statistics (e.g., number of records for each update and as totals) and other databases. Important, yet often overlooked, are the links from the Home page to the “Help” and “Contact Us”. The “Help” page provides a wealth of information in general categories such as “Starting Out” (how to begin navigating and searching within ECOTOX, as well as frequently asked questions and links to the PDF version of the User Guide), “How do I…” (details on how to perform searches, select query parameters, and format output), “What is…” (overview on database including species and chemical verification sources used by ECOTOX curators, details on data field definitions and terms in ECOTOX’s controlled vocabulary), and “More Resources” (glossary of terms and links to related websites). When additional information is needed, or if users have suggestions to improve ECOTOX, comments and questions can be submitted through the “Contact Us” link.
Strategies to identify relevant data in ECOTOX.
The strategies available for users to identify relevant data are one of the most impactful changes from the previous versions of ECOTOX. Database retrievals can now be conducted using either the “Search” module to directly query specific species, chemicals, or effect with expanded options to customize many output fields or the “Explore” module to investigate and visualize the data available. The new “Search” module provides a direct method to retrieve data, comparable to the previous “Advanced Query” in ECOTOX Ver 4, with expanded options to iteratively revise the query criteria, customize output to include specific data fields, and review the data in tabular format before downloading. The query parameters available in “Search” include options for chemicals (e.g., CASRN, chemical name), species (e.g., scientific name, taxonomic group), effects (e.g., type of effect, specific measurements), statistically-derived indicators of effects/no-effects (endpoints such as LC50, EC50, NOEL, and LOEL), test conditions (e.g., study site type [laboratory, field], exposure media [freshwater, soil], route of chemical exposure [oral, diet]), and publication options (e.g., author name, publication year). Selections may be further filtered before continuing a search or updated after reviewing the report (tabular data) based on the initial parameters, and the output can be customized to include or exclude fields for viewing or exporting in Excel or delimited formats. Users can also view and download the list of references associated with records meeting query parameters.
Version 5 of ECOTOX includes the new module “Explore”, which is especially helpful when search criteria are not known and/or if a visual representation of general and specific data trends is desired. This module includes a limited set of query filters and output formats, which focus on helping the user to investigate data availability through interactive queries and data visualizations. At any point, the query parameters in “Explore” can be sent to “Search” for further refinement with additional query parameters and review or export of more comprehensive information with additional data fields. The “Explore” module starts a user with predefined or custom groups for chemicals, species, or effects groups, with query filters for chemicals, species, effect groups, effect measurements, endpoints, and publication years. As parameters are selected, the other parameters and tables respond reactively, which allow the user to refine or expand parameters to identify the most relevant data for a specific question or goal. For example, once a chemical or chemical group is selected in “Explore by Chemical”, the query filters will display the number of species, effect groups, effect measurements, endpoints, and so forth for which there are data in the Knowledgebase. Selection within the query filters is then limited to the categories with data. When any of the options within a query filter are selected, such as limiting the exploration to only fish in “Species Group”, the rest of the query filters will only show options for the chemical(s) selected for which there are data for fish species. This reactive feature in “Explore” allows the users to expand or restrict the query based on this immediate knowledge of data availability.
Data output in “Explore” are presented in three formats: “Group Summary” (summary of number of records and publications by chemical, species, or effect group), “Records” (resulting records summarized by effect and endpoint), and the “Plot View” (dot or scatterplot of records by exposure concentration or duration by effects, endpoints, species, chemicals, and so forth, including a simplified table of records below the plot). Data presented in each of tables and plots can be further refined using the query filters. Plot View allows visualization of the data, which provides a general concept of the range of available data. The plots are interactive with the option to “zoom” into the plot, deselect/select items in the plot legend, and/or display a line at a specified concentration. There are some limitations to the data visualization feature (maximum display of 3000 records, only includes data that can be converted to exposure concentrations of parts per million). Therefore, it is recommended that the user sends the query parameters in “Explore” to the “Search” module before downloading data. Both the “Search” or “Explore” modules are designed to search on all data unless users restrict the search by choosing specific criteria of interest. Further details and examples for querying and exploring the available data are included on “Help” sections.
Data downloads.
The ECOTOX Knowledgebase allows downloads of all data available on the public-facing website. Results from “Search” include the most comprehensive set of data fields and can be downloaded either as a Microsoft Excel spreadsheet or delimited file format. The Excel spreadsheet includes all data records, the list of references from which the data were extracted, and the query parameters applied by the user in these output files for improved documentation and reproducibility of searches. When delimited files are preferred by the user, the data records, list of references, and search parameters are available to download as separate files from the “Search” module. Within the “Explore” module, comma separated files (.csv) can be downloaded for the group summary table, the records table, or the simplified table below the plot; however these do not provide the same level of detail nor the options for customizing output fields that are available in the “Search” module. In addition, users can download delimited ASCII files of the entire aquatic or terrestrial raw data. The ASCII file download does not require specific software although it requires combining various files together to view entire records, as the data are divided into two sections: aquatic and terrestrial. Within each of these sections, users will find data tables, field descriptions, and graphical relations of the data structure.
Examples of ECOTOX applications.
The updated options for querying and downloading data greatly improve the end user’s ability to efficiently retrieve targeted toxicity data, with added benefits of flexibility in the output formats, transparency and documentation of search results, and the embedded information to aid in linking to external databases and computational platforms. In this section, we present examples of how the new ECOTOX Ver 5 can be used to retrieve toxicity data for several common use cases, with brief descriptions (full details in the Supporting Information). Each example is intended only to demonstrate accessing toxicity results through “Search” and “Explore” for identification, filtering, and export of relevant data; further evaluation and analysis of downloaded data is conducted in other platforms. See Table 3 for published studies with similar applications as these examples.
A). Development of toxicity reference values (TRVs):
Toxicity data from ECOTOX can be used in the development of toxicity reference values (TRVs, see Table 3). In this example, we used “Explore” to determine the scope of toxicity data available for various cadmium (Cd) and biological species (Supporting Table S5), and observed potential trends in relative potency among chemical species and develop testable hypotheses (Supporting Figure S4). This exploration of available data allowed us to determine that growth effects data in mammalian species could be used to evaluate how metal speciation affects TRVs. The “Search” module was then used to select and export data (Supporting Table S6) specific to the testable hypothesis: Does metal speciation affect TRVs? After export from ECOTOX, the data records were further filtered based on methodological data fields (e.g., exposure method, number of doses, use of a control, and exposure duration), and dose-response modeling of growth effects (using Toxicity Relation Analysis Program, TRAP V 3.1, https://archive.epa.gov/med/med_archive_03/web/html/trap.html) showed that ECx values for cadmium chloride were 4-fold lower than for cadmium acetate (Supporting Figure S5). Additional details for this example are in the Supporting Information (Supplemental for Example A).
B). Development of SSDs:
Toxicity data from ECOTOX are routinely used in development of SSDs (see Table 3). In this example, we used “Explore” and “Search” to identify empirical toxicity data for the development of a chlorpyrifos SSD for aquatic invertebrates. Through the recently added features in “Explore”, we queried the aquatic data for chlorpyrifos (CASRN 2921–88-2); filtered these results to limit the species group, effect groups, effect measurements, and endpoints to those of interest; and reviewed the scope of available data by species, class, and general species group (Supporting Figures S6 and S7). These query filters were sent to the “Search” module, where the data were exported with the data fields relevant for study evaluation and formatted for import into the SSD Toolbox (Etterson 2020). After export from ECOTOX, the data records were filtered to include only 2- and 4-day studies and reviewed for inclusion in the SSD (Supporting Table S7). The data records which were identified as suitable for inclusion were imported into the SSD Toolbox to create a preliminary SSD that resulted in 5% hazard concentration of 2.024 × 10−5 using the normal distribution fit to the toxicity data using maximum likelihood (Supporting Figure S8). Additional details for this example are in Supporting Information (Supplemental for Example B).
C). Linking chemical detections and concentrations to effects data:
Toxicity data in ECOTOX can provide relevance for potential effects of the chemicals detected in environmental samples. In this example, we used the “Explore” module to determine the scope of aquatic toxicity data available for 109 chemicals detected in surface water samples collected in the Great Lakes Basin (Lee et al., 2015) using the chemical identifiers provided (CASRNs) and summarizing by the types of chemicals (Supporting Table S8). Available data was further reviewed with the visualization options in “Explore” (see example in Supporting Figure S9) and then, after additional filtering, query filters were sent to the “Search” module for exporting the data (Supporting Table S9). Summaries were completed in Excel and R to identify data-poor compounds as well as compare measured concentrations with observed effects concentrations (Supporting Figure S10). Additional details for this example are in the Supporting Information (Supplemental for Example C).
Challenges and limitations
As with any on-going project, ECOTOX has recognized challenges and limitations. One of the largest limitations is the sheer immensity of the ecotoxicological literature; even with thousands of chemicals and species, data currently in ECOTOX are a subset of all of the compounds tested with ecologically relevant species. The identification of studies for inclusion is primarily through targeted chemical-specific literature searches based on the needs and requests from USEPA Program Offices or research teams (as described above, see Literature searches and sources of toxicity data). In an effort to address the issue of underrepresented chemicals and species, monthly review of relevant recent publications, prioritization of species groups with limited data, and the recent incorporation of data analytics tools to decrease screening time been incorporated. These efforts have aided the prioritization of studies with chemicals and/or species that are currently underrepresented. However, there remains a backlog of >60,000 potentially relevant references in the Unify “References” module waiting to be prioritized for full review and data extraction. In addition, ECOTOX is limited to the details and results reported by study authors. This retains the intent of the authors and avoids potentially erroneous extrapolation or interpretation; however, this results in fields being left empty when not reported in a study. This issue of insufficient reporting is across toxicology, as well as other disciplines, and is currently being tackled by several journals and working groups (e.g., Society of Environmental Toxicology and Chemistry 2019). In addition, relying on author-reported details has resulted in potentially duplicative or overlapping terms, in particular for the specific effect measurements and endpoints reported. The employment of controlled vocabularies has enabled use of consistent terminology by relating terms used by authors to the common set of verified terms, and the integration of ontologies will further aid this consistency.
Another major challenge is balancing the need to respond to the evolving toxicology testing and regulatory environment with the responsibility to maintain the current processes to curate toxicity results from traditional in vivo toxicity tests. Continuing the identification and incorporation of in vivo studies is essential for providing structured data for comparison and evaluation of alternative methods. As incorporation of new approaches expands, the recommended data reporting and evaluation criteria for relevant studies have started to become standardized (e.g., OECD, 2017, 2018; Roth et al., 2021). Already, ECOTOX has began incorporating in vitro studies and has added terms to the controlled vocabularies to reflect these methodologies (e.g., exposure type), with additional fields and terms expected to be added as guidance is developed. With both the current and future records, ongoing work is needed to update the previously curated data to meet current standards and use currently recommended terminology. This is not a new challenge and is already incorporated into the ECOTOX processes (see above, Data maintenance and updates).
Interoperability and future directions
The updates made to the ECOTOX Knowledgebase described here increase the efficiency in identifying and retrieving relevant data, with added functionality to explore and visualize available ecological toxicity data, thus supporting the increasing need for rapid access to toxicity test results used for risk assessment and chemical decision-making. This ready access to a wealth of existing toxicity data, along with the ongoing addition of data to the Knowledgebase, will help the USEPA efficiently review possible chemical effects on ecologically relevant species, identify data gaps, prioritize testing, and evaluate new approach methodologies to support the decision-making for the increasingly large universe of chemicals. Future applications and plans for improvements include increased functionality and interoperability to allow for integration with other USEPA databases, new text mining and data visualization programs, as well as support for new and evolving tools for predictive toxicology.
Throughout all the recent updates to ECOTOX, there was a recognized need to incorporate support for data integration and interoperability. Typical applications that use toxicity test results from ECOTOX (e.g., ecological risk assessment, SSDs, site-specific hazard evaluations) incorporate additional toxicity, chemical, and biological data from other sources. In addition, computational toxicology and integration of new approach methodologies require the ability to link multiple disparate data sources (Watford et al., 2019a). Thus, work is being conducted with developers of web portals for chemical information to incorporate linkages into ECOTOX, including the OECD eChemPortal (https://www.echemportal.org/echemportal/, ECOTOX chemical list added in 2020) and USEPA’s CompTox Chemicals Dashboard (https://comptox.epa.gov/dashboard/chemical_lists/ECOTOX_V2, added in 2017).
In addition, information is being incorporated into ECOTOX which will enable the integration and exchange of data with public data resources having similar (e.g., toxicity data in ToxRefDB, Watford et al., 2019b) and different types of data (e.g., physicochemical properties in the CompTox Chemicals Dashboard, biological pathways in Gene Ontology and Kyoto Encyclopedia of Genes and Genomes) following FAIR data principles (Wilkinson et al., 2016). Initial steps to increase the accessibility and ease of locating data within ECOTOX have included incorporation of unique identifiers for biological species (NCBI TaxIDs, added in 2018) and chemicals (DTXSID, added in 2020). Further work for data interoperability and information discovery includes incorporation of unique identifiers for genes and proteins (e.g., NCBI Entrez GeneID, UniProt IDs), and mapping ECOTOX controlled vocabularies to existing toxicology database vocabularies (e.g., Unified Medical Language System) and ontologies (e.g., Chemical Entities of Biological Interest). In combination, these unique identifiers and mappings will enable linking to and from a variety of chemical, toxicological, and biological databases; increasing the accessibility of data in ECOTOX; and supporting advanced querying capabilities. These additions will ultimately contribute to making ECOTOX machine-readable and computable, which in turn supports future goals to advance integration into applications such as adverse outcome pathway development and semantic phenotypic mapping (Ives et al., 2017; Wang et al., 2019; Whaley et al., 2020).
Complementary to the addition of new information for interoperability, new technologies in artificial intelligence, including machine learning and data analytics and software applications, are being incorporated to increase the efficiencies of the pipeline. In 2020, we began testing and utilizing applications for automated literature searching, and machine learning and artificial intelligence techniques for title and abstract screening of references [e.g., AbstractSifter (Baker et al., 2017), and SWIFT-Review (Howard et al., 2016) and SWIFT-ActiveScreener (Howard et al., 2020)]. Preliminary results (not reported here) have demonstrated that these tools provide time-savings for most typical chemical searches and reviews conducted by the ECOTOX team. Efficiencies gained include increased the speed at which ECOTOX personnel can review literature and extract data which, along with enhanced ECOTOX interoperability with other data sources, will aid in meeting the current demand for rapid access to toxicity data for risk assessment and environmental research. As we look to the future, other challenges (e.g., complex mixtures, nanoparticles) are recognized, and possible additions and/or modifications to ECOTOX Knowledgebase are being considered.
SUMMARY & CONCLUSIONS
The ECOTOX Knowledgebase has long provided curated data extracted from ecologically relevant toxicity studies for use in broad applications in environmental research and risk assessments. The ECOTOX Knowledgebase continues to provide needed information for decision support in ecotoxicity, and it is utilized by a wide-range of governmental, academic, and industry researchers and risk-assessors; however, there was a recognized need to update the database content and the user interface to keep pace with changes in reported toxicological results. Therefore, extensive updates were recently completed, including a redesigned user interface for the web-based access to the Knowledgebase; these new features allow the user to explore and visualize toxicity data, give a variety of data output/visualization options, and incorporate standardized identifiers (e.g., DTXSIDs) for increased interoperability and reusability.
The ECOTOX Knowledgebase continues to be a flexible resource for empirical toxicity data as the challenge of providing more information at a faster pace to make regulatory decisions is addressed. The primary focus continues to be thoroughly curated data from robust reviews of the literature, with transparency in the processes and accessibility of the curated data. The updates extend the utility of ECOTOX with increased efficiencies to locate relevant toxicity data while building on the existing strengths. These strengths include the comprehensiveness and transparency of the literature search and review processes, the manual curation of detailed information from each relevant study, the inclusion of all measured endpoints capturing both adverse effects and no-effects data, and the extensiveness of toxicity data housed in the Knowledgebase. This ever-growing and evolving database will continue to add applicable and relevant toxicity data while working towards increased efficiency, transparency, and interoperability to support environmental research and risk assessment.
Supplementary Material
Acknowledgment
We gratefully acknowledge the invaluable contributions of multiple USEPA and contract contributors to ECOTOX over the many years that it has been in development. We also recognize and acknowledge the teams at the USEPA National Health and Environmental Effect Research Laboratory that developed the initial parent databases that were incorporated into ECOTOX (AQUIRE at Mid-Continent Ecology Division; PHYTOTOX and TERRETOX at the Western Ecology Division). We thank our management for on-going support of this project. In addition, we recognize and thank our USEPA and international colleagues for working with us for improved the interoperability. We greatly appreciate the comments and suggestions from Dr. Gerald Ankley (USEPA Office of Research and Development Center for Computational Toxicology) and Dr. Michelle Angrish (USEPA Office of Research and Development Center for Public Health and Environmental Assessment) on an earlier draft of this manuscript. Finally, we thank the USEPA Program Offices and Regions and multiple other end users for their feedback and suggestions to improve the ECOTOX processes and user interface. This work was supported by the U.S. EPA Office of Research and Development and multiple other U.S. EPA Offices for targeted literature searches and reviews. Initial development of the unified interface was funded in 1995 through the U.S. Department of Defense’s Strategic Environmental Research and Development Program (SERDP).
Funding Information:
US Environmental Protection Agency
Footnotes
Supporting Information
The Supporting Information is available on the Wiley Online Library at https://doi.org/10.1002/etc.5324.
Disclaimer:
The views expressed in this paper are those of the authors and do not necessarily reflect the views or policies of the U.S. Environmental Protection Agency, nor does the mention of trade names or commercial products indicate endorsement by the federal government.
Conflict of Interest:
The authors claim no conflicts of interest.
Data Availability
Data pertaining to the present study are available on the ECOTOX website: www.epa.gov/ecotox and at https://edg.epa.gov/metadata/catalog/main/home.page. Data, associated metadata, and calculation tools are also available from the corresponding author (olker.jennifer@epa.gov).
REFERENCES
- Alexander AC, Luis AT, Culp JM, Baird DJ, & Cessna AJ (2013). Can nutrients mask community responses to insecticide mixtures?. Ecotoxicology, 22(7), 1085–1100. 10.1007/s10646-013-1096-3 [DOI] [PubMed] [Google Scholar]
- Bacey J, & Spurlock F (2007). Biological assessment of urban and agricultural streams in the California Central Valley. Environmental Monitoring and Assessment, 130(1–3), 483–493. 10.1007/s10661-006-9438-8 [DOI] [PubMed] [Google Scholar]
- Baker N, Knudsen T, & Williams A (2017). Abstract Sifter: a comprehensive front-end system to PubMed. F1000Research, 6(Chem Inf Sci), 2164. 10.12688/f1000research.12865.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bermúdez-Saldaña JM, Escuder-Gilabert L, Medina-Hernández MJ, Villanueva-Camañas RM, & Sagrado S (2005). Modelling bioconcentration of pesticides in fish using biopartitioning micellar chromatography. Journal of Chromatography A, 1063(1–2), 153–160. 10.1016/j.chroma.2004.11.074 [DOI] [PubMed] [Google Scholar]
- Bettiol C, De Vettori S, Minervini G, Zuccon E, Marchetto D, Ghirardini AV, & Argese E (2016). Assessment of phenolic herbicide toxicity and mode of action by different assays. Environmental Science and Pollution Research International, 23(8), 7398–7408. 10.1007/s11356-015-5958-5 [DOI] [PubMed] [Google Scholar]
- Bezirci G, Akkas SB, Rinke K, Yildirim F, Kalaylioglu Z, Severcan F, & Beklioglu M (2012). Impacts of salinity and fish-exuded kairomone on the survival and macromolecular profile of Daphnia pulex. Ecotoxicology, 21(2), 601–614. 10.1007/s10646-011-0820-0 [DOI] [PubMed] [Google Scholar]
- Bielmyer GK, Tomasso J, & Klaine SJ (2006). Physiological responses of hybrid striped bass to aqueous copper in freshwater and saltwater. Archives of Environmental Contamination and Toxicology, 50(4), 531–538. 10.1007/s00244-005-0131-7 [DOI] [PubMed] [Google Scholar]
- Brock TC, & Van Wijngaarden RP (2012). Acute toxicity tests with Daphnia magna, Americamysis bahia, Chironomus riparius and Gammarus pulex and implications of new EU requirements for the aquatic effect assessment of insecticides. Environmental Science and Pollution Research International, 19(8), 3610–3618. 10.1007/s11356-012-0930-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Claesson A, & Törnqvist L (1988). The toxicity of aluminium to two acido-tolerant green algae. Water Research, 22(8), 977–983. [Google Scholar]
- Connors KA, Beasley A, Barron MG, Belanger SE, Bonnell M, Brill JL, de Zwart D, Kienzler A, Krailler J, Otter R, Phillips JL, & Embry MR (2019). Creation of a Curated Aquatic Toxicology Database: EnviroTox. Environmental Toxicology and Chemistry, 38(5), 1062–1073. 10.1002/etc.4382 [DOI] [PMC free article] [PubMed] [Google Scholar]
- d’Amora M, & Giordani S (2018). The utility of zebrafish as a model for screening developmental neurotoxicity. Frontiers in Neuroscience, 12, 976. 10.3389/fnins.2018.00976 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dardenne F, Nobels I, De Coen W, & Blust R (2007). Dose-response relationships and statistical performance of a battery of bacterial gene profiling assays. Applied Microbiology and Biotechnology, 75(1), 223–234. 10.1007/s00253-006-0808-5 [DOI] [PubMed] [Google Scholar]
- Demetrio PM, Bulus Rossini GD, Bonetto CA, & Ronco AE (2012). Effects of pesticide formulations and active ingredients on the coelenterate Hydra attenuata (Pallas, 1766). Bulletin of Environmental Contamination and Toxicology, 88(1), 15–19. 10.1007/s00128-011-0463-0 [DOI] [PubMed] [Google Scholar]
- Dornfeld CB, Rodgher S, Negri RG, Espíndola E, & Daam MA (2019). Chironomus sancticaroli (Diptera, Chironomidae) as a Sensitive Tropical Test Species in Laboratory Bioassays Evaluating Metals (Copper and Cadmium) and Field Testing. Archives of Environmental Contamination and Toxicology, 76(1), 42–50. 10.1007/s00244-018-0575-1 [DOI] [PubMed] [Google Scholar]
- Dulio V, Koschorreck J, van Bavel B et al. (2020). The NORMAN Association and the European Partnership for Chemicals Risk Assessment (PARC): let’s cooperate!. Environmental Sciences Europe, 32(1), 100. 10.1186/s12302-020-00375-w [DOI] [Google Scholar]
- Etterson M (2020). User’s Manual: SSD Toolbox Version 1.0 EPA/600/R-18/116. https://www.epa.gov/chemical-research/species-sensitivity-distribution-ssd-toolbox [Google Scholar]
- European Chemical Agency (ECHA). (2017). The use of alternatives to testing on animals for the REACH Regulation Reference Report ECHA-17-R-02-EN. 10.2823/023078 [DOI]
- Fai PB, & Grant A (2010). An assessment of the potential of the microbial assay for risk assessment (MARA) for ecotoxicological testing. Ecotoxicology, 19(8), 1626–1633. 10.1007/s10646-010-0548-2 [DOI] [PubMed] [Google Scholar]
- Fay KA, Mingoia RT, Goeritz I, Nabb DL, Hoffman AD, Ferrell BD, Peterson HM, Nichols JW, Segner H, & Han X (2014). Intra- and interlaboratory reliability of a cryopreserved trout hepatocyte assay for the prediction of chemical bioaccumulation potential. Environmental Science and Technology, 48(14), 8170–8178. 10.1021/es500952a [DOI] [PubMed] [Google Scholar]
- Fay KA, Villeneuve DL, LaLone CA, Song Y, Tollefsen KE, & Ankley GT (2017). Practical approaches to adverse outcome pathway development and weight-of-evidence evaluation as illustrated by ecotoxicological case studies. Environmental Toxicology and Chemistry, 36(6), 1429–1449. 10.1002/etc.3770 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Federhen S (2012). The NCBI Taxonomy database. Nucleic Acids Research, 40(Database issue), D136–43. 10.1093/nar/gkr1178 [DOI] [PMC free article] [PubMed] [Google Scholar]
- George TK, & Liber K (2007). Laboratory investigation of the toxicity and interaction of pesticide mixtures in Daphnia magna. Archives of Environmental Contamination and Toxicology, 52(1), 64–72. 10.1007/s00244-005-0271-9 [DOI] [PubMed] [Google Scholar]
- Grulke CM, Williams AJ, Thillanadarajah I, & Richard AM (2019). EPA’s DSSTox database: History of development of a curated chemistry resource supporting computational toxicology research. Computational Toxicology, 12, 10.1016/j.comtox.2019.100096 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guzelian PS, Victoroff MS, Halmes NC, James RC, & Guzelian CP (2005). Evidence-based toxicology: a comprehensive framework for causation. Human & Experimental Toxicology, 24(4), 161–201. 10.1191/0960327105ht517oa [DOI] [PubMed] [Google Scholar]
- Guzmán FT, González FJ, & Martínez RR (2010). Implementing Lecane quadridentata acute toxicity tests to assess the toxic effects of selected metals (Al, Fe and Zn). Ecotoxicology and Environmental Safety, 73(3), 287–295. 10.1016/j.ecoenv.2009.10.006 [DOI] [PubMed] [Google Scholar]
- Herrmann K, Pistollato F, & Stephens ML (2019). Beyond the 3Rs: Expanding the use of human-relevant replacement methods in biomedical research. ALTEX, 36(3), 343–352. 10.14573/altex.1907031 [DOI] [PubMed] [Google Scholar]
- Hoffmann S, & Hartung T (2006). Toward an evidence-based toxicology. Human & Experimental Toxicology, 25(9), 497–513. 10.1191/0960327106het648oa [DOI] [PubMed] [Google Scholar]
- Hoffmann S, de Vries R, Stephens ML, Beck NB, Dirven H, Fowle JR 3rd, Goodman JE, Hartung T, Kimber I, Lalu MM, Thayer K, Whaley P, Wikoff D, & Tsaioun K (2017). A primer on systematic reviews in toxicology. Archives of Toxicology, 91(7), 2551–2575. 10.1007/s00204-017-1980-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Howard BE, Phillips J, Tandon A, Maharana A, Elmore R, Mav D, Sedykh A, Thayer K, Merrick A, Walker V, Rooney A, & Shah RR (2020). SWIFT-Active Screener: accelerated document screening through Active Learning and integrated recall estimation. Environment International, 105623. 10.1016/j.envint.2020.105623 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Howard BE, Phillips J, Miller K, Tandon A, Mav D, Shah MR, Holmgren S, Pelch KE, Walker V, Rooney AA, Macleod M, Shah R, & Thayer K (2016). SWIFT-Review: A text-mining workbench for systematic review. Systematic Reviews, 5, 87. 10.1186/s13643-016-0263-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM). (2018). A strategic roadmap for establishing new approaches to evaluate the safety of chemicals and medical products in the United States Available: https://ntp.niehs.nih.gov/go/iccvam-rdmp. https://doi.org/10.22427/NTP-ICCVAM-ROADMAP2018
- Ives C, Campia. I, Wang RL, Wittwehr C, Edwards S (2017). Creating a structured AOP Knowledgebase via ointology-based annotations. Applied In Vitro Toxicology, 3(4):298–311. 10.1089/aivt.2017.0017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson BM, Kanagy LE, Rodgers JH, & Castle JW (2008). Chemical, physical, and risk characterization of natural gas storage produced waters. Water, Air, and Soil Pollution, 191, 33–54. 10.1007/s11270-007-9605-8 [DOI] [Google Scholar]
- Kaltenhäuser J, Kneuer C, Marx-Stoelting P, Niemann L, Schubert J, Stein B, & Solecki R (2017). Relevance and reliability of experimental data in human health risk assessment of pesticides. Regulatory Toxicology and Pharmacology, 88, 227–237. 10.1016/j.yrtph.2017.06.010 [DOI] [PubMed] [Google Scholar]
- Klimisch HJ, Andreae M, Tillmann U (1997). A systematic approach for evaluating the quality of experimental toxicological and ecotoxicological data. Regulatory Toxicology and Pharmacology, 25(1), 1–5. 10.1006/rtph.1996.1076. [DOI] [PubMed] [Google Scholar]
- Kostich MS, Flick RW, Batt AL, Mash HE, Boone JS, Furlong ET, Kolpin DW, & Glassmeyer ST (2017). Aquatic concentrations of chemical analytes compared to ecotoxicity estimates. The Science of the Total Environment, 579, 1649–1657. 10.1016/j.scitotenv.2016.06.234 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Krewski D, Andersen ME, Tyshenko MG, Krishnan K, Hartung T, Boekelheide K, Wambaugh JF, Jones D, Whelan M, Thomas R, Yauk C, Barton-Maclaren T, & Cote I (2020). Toxicity testing in the 21st century: progress in the past decade and future perspectives. Archives of Toxicology, 94(1), 1–58. 10.1007/s00204-019-02613-4 [DOI] [PubMed] [Google Scholar]
- Krewski D, Westphal M, Andersen ME, Paoli GM, Chiu WA, Al-Zoughool M, Croteau MC, Burgoon LD, & Cote I (2014). A framework for the next generation of risk science. Environmental Health Perspectives, 122(8), 796–805. 10.1289/ehp.1307260 [DOI] [PMC free article] [PubMed] [Google Scholar]
- LaLone CA, Villeneuve DL, Lyons D, Helgen HW, Robinson SL, Swintek JA, Saari TW, & Ankley GT (2016). Editor’s Highlight: Sequence Alignment to Predict Across Species Susceptibility (SeqAPASS): A Web-Based Tool for Addressing the Challenges of Cross-Species Extrapolation of Chemical Toxicity. Toxicological Sciences, 153(2), 228–245. 10.1093/toxsci/kfw119 [DOI] [PubMed] [Google Scholar]
- LeBlanc HM, Culp JM, Baird DJ, Alexander AC, & Cessna AJ (2012). Single versus combined lethal effects of three agricultural insecticides on larvae of the freshwater insect Chironomus dilutus. Archives of Environmental Contamination and Toxicology, 63(3), 378–390. 10.1007/s00244-012-9777-0 [DOI] [PubMed] [Google Scholar]
- Leboulanger C, Bouvy M, Carré C, Cecchi P, Amalric L, Bouchez A, Pagano M, & Sarazin G (2011). Comparison of the effects of two herbicides and an insecticide on tropical freshwater plankton in microcosms. Archives of Environmental Contamination and Toxicology, 61(4), 599–613. 10.1007/s00244-011-9653-3 [DOI] [PubMed] [Google Scholar]
- Lee KE, Langer SK, Menheer MA, Hansen DS, Foreman WT, Furlong ET, Jorgenson ZG, Choy SJ, Moore JN, Banda JoAnn, Gefell DJ (2015). Chemicals of emerging concern in water and bottom sediment in the Great Lakes Basin, 2012—Collection methods, analytical methods, quality assurance, and study data: U.S. Geological Survey Data Series; 910, 14 p., 10.3133/ds910. [DOI] [Google Scholar]
- Lee YG, Hwang SH, & Kim SD (2006). Predicting the toxicity of substituted phenols to aquatic species and its changes in the stream and effluent waters. Archives of Environmental Contamination and Toxicology, 50(2), 213–219. 10.1007/s00244-004-1259-6 [DOI] [PubMed] [Google Scholar]
- Liu AX, Lang YH, Xue LD, Liao SL, & Zhou H (2009). Probabilistic ecological risk assessment and source apportionment of polycyclic aromatic hydrocarbons in surface sediments from Yellow Sea. Bulletin of Environmental Contamination and Toxicology, 83(5), 681–687. 10.1007/s00128-009-9870-x [DOI] [PubMed] [Google Scholar]
- Moermond C, Beasley A, Breton R, Junghans M, Laskowski R, Solomon K, Zahner H (2017). Assessing the reliability of ecotoxicological studies: An overview of current needs and approaches. Integrated Environmental Assessment and Management, 13(4), 640–651. 10.1002/ieam.1870 [DOI] [PubMed] [Google Scholar]
- Moermond CT, Kase R, Korkaric M, & Ågerstrand M (2016). CRED: Criteria for reporting and evaluating ecotoxicity data. Environmental Toxicology & Chemistry, 35(5), 1297–1309. 10.1002/etc.3259 [DOI] [PubMed] [Google Scholar]
- Moher D, Liberati A, Tetzlaff J, Altman DG, & PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine, 6(7), e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nam SH, Lee WM, & An YJ (2012). Derivation of Ecological Protective Concentration using the Probabilistic Ecological Risk Assessment applicable for Korean Water Environment: (I) Cadmium. Toxicological Research, 28(2), 129–137. 10.5487/TR.2012.28.2.129 [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Research Council (NRC). (2007). Toxicity testing in the 21st century: A vision and a strategy National Academies Press, Washington, DC. 216 pp. [Google Scholar]
- National Toxicology Program (NTP). (2019). Handbook for conducting a literature-based health assessment using office of health assessment and translation (OHAT) approach for systematic review and evidence integration https://ntp.niehs.nih.gov/ntp/ohat/pubs/handbookmarch2019_508.pdf (accessed 13 January 2022).
- Organisation for Economic Cooperation and Development (OECD). (2017), Guidance document for describing non-guideline in vitro test methods. OECD Series on Testing and Assessment, No. 211, OECD Publishing, Paris, 10.1787/9789264274730-en. [DOI] [Google Scholar]
- Organisation for Economic Cooperation and Development (OECD). (2018), Guidance document on Good In Vitro Method Practices (GIVIMP). OECD Series on Testing and Assessment, No. 286, OECD Publishing, Paris, 10.1787/9789264304796-en. [DOI] [Google Scholar]
- Ownby DR, & Newman MC (2003). Advances in Quantitative Ion Character‐Activity Relationships (QICARs): Using Metal‐Ligand Binding Characteristics to Predict Metal Toxicity. Quantitative Structure-Activity Relationships, 22, 241–246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Parish ST, Aschner M, Casey W, Corvaro M, Embry MR, Fitzpatrick S, Kidd D, Kleinstreuer NC, Lima BS, Settivari RS, Wolf DC, Yamazaki D, & Boobis A (2020). An evaluation framework for new approach methodologies (NAMs) for human health safety assessment. Regulatory Toxicology and Pharmacology, 112, 104592. 10.1016/j.yrtph.2020.104592. [DOI] [PubMed] [Google Scholar]
- Ra JS, Lee BC, Chang NI, & Kim SD (2008). Comparative whole effluent toxicity assessment of wastewater treatment plant effluents using Daphnia magna. Bulletin of Environmental Contamination and Toxicology, 80(3), 196–200. 10.1007/s00128-007-9344-y [DOI] [PubMed] [Google Scholar]
- Ralston-Hooper K, Hardy J, Hahn L, Ochoa-Acuña H, Lee LS, Mollenhauer R, & Sepúlveda MS (2009). Acute and chronic toxicity of atrazine and its metabolites deethylatrazine and deisopropylatrazine on aquatic organisms. Ecotoxicology, 18(7), 899–905. 10.1007/s10646-009-0351-0 [DOI] [PubMed] [Google Scholar]
- Rial D, Beiras R, Vázquez JA, & Murado MA (2010). Acute toxicity of a shoreline cleaner, CytoSol, mixed with oil and ecological risk assessment of its use on the Galician Coast. Archives of Environmental Contamination and Toxicology, 59(3), 407–416. 10.1007/s00244-010-9492-7 [DOI] [PubMed] [Google Scholar]
- Rosenbaum RK, Bachmann TM, Gold LS, Huijbregts MAJ, Jolliet O, Juraske R, Koehler A, Larsen HF, MacLeod M, Margni M, McKone TE, Payet J, Schuhmacher M, van de Meent D, & Hauschild MZ (2008). USEtox—the UNEP-SETAC toxicity model: recommended characterisation factors for human toxicity and freshwater ecotoxicity in life cycle impact assessment. International Journal of Life Cycle Assessment, 13(7), 532–546. 10.1007/s11367-008-0038-4 [DOI] [Google Scholar]
- Roth N, Zilliacus J, & Beronius A (2021). Development of the SciRAP Approach for Evaluating the reliability and relevance of in vitro toxicity data. Frontiers in Toxicology 10.3389/ftox.2021.746430 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rudén C, Adams J, Ågerstrand M, Brock TC, Poulsen V, Schlekat CE, Wheeler JR, & Henry TR (2017). Assessing the relevance of ecotoxicological studies for regulatory decision making. Integrated Environmental Assessment and Management, 13(4), 652–663. 10.1002/ieam.1846 [DOI] [PubMed] [Google Scholar]
- Russom CL, LaLone CA, Villeneuve DL, & Ankley GT (2014). Development of an adverse outcome pathway for acetylcholinesterase inhibition leading to acute mortality. Environmental Toxicology and Chemistry, 33(10), 2157–2169. 10.1002/etc.2662 [DOI] [PubMed] [Google Scholar]
- San-Segundo L, Guimarães L, Torija CF, Beltrán EM, Guilhermino L, & Pablos MV (2016). Alterations in gene expression levels provide early indicators of chemical stress during Xenopus laevis embryo development: A case study with perfluorooctane sulfonate (PFOS). Ecotoxicology and Environmental Safety, 127, 51–60. 10.1016/j.ecoenv.2016.01.005 [DOI] [PubMed] [Google Scholar]
- Schneider K, Schwarz M, Burkholder I, Kopp-Schneider A, Edler L, Kinsner-Ovaskainen A, Hartung T, & Hoffmann S (2009). “ToxRTool”, a new tool to assess the reliability of toxicological data. Toxicology Letters, 189(2), 138–144. 10.1016/j.toxlet.2009.05.013 [DOI] [PubMed] [Google Scholar]
- Schuler LJ, Hoang TC, & Rand GM (2008). Aquatic risk assessment of copper in freshwater and saltwater ecosystems of South Florida. Ecotoxicology, 17(7), 642–659. 10.1007/s10646-008-0236-7 [DOI] [PubMed] [Google Scholar]
- Shuhaimi-Othman, Nadzifah Y, Umirah, NS, & Ahmad AK (2012). Toxicity of Metals to an Aquatic Worm, Nais elinguis (Oligochaeta, Naididae). Research Journal of Environmental Toxicology, 6(4), 122–132. 10.3923/rjet.2012.122.132 [DOI] [Google Scholar]
- Society of Environmental Toxicology and Chemistry (SETAC). (2019). Technical Issue Paper: Recommended minimum reporting information for environmental toxicity studies [Technical issue paper] [Google Scholar]
- Sorensen MT, Conder JM, Fuchsman PC, Martello LB, & Wenning RJ (2007). Using a Sediment Quality Triad approach to evaluate benthic toxicity in the Lower Hackensack River, New Jersey. Archives of Environmental Contamination and toxicology, 53(1), 36–49. 10.1007/s00244-006-0165-5 [DOI] [PubMed] [Google Scholar]
- Stadlinger N, Berg H, Van den Brink PJ, Tam NT, & Gunnarsson JS (2018). Comparison of predicted aquatic risks of pesticides used under different rice-farming strategies in the Mekong Delta, Vietnam. Environmental Science and Pollution Research International, 25(14), 13322–13334. 10.1007/s11356-016-7991-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stephens ML, Akgün-Ölmez SG, Hoffmann S, de Vries R, Flick B, Hartung T, Lalu M, Maertens A, Witters H, Wright R, & Tsaioun K (2018). Adaptation of the Systematic Review Framework to the Assessment of Toxicological Test Methods: Challenges and Lessons Learned with the Zebrafish Embryotoxicity Test. Toxicological Sciences, 171(1), 56–68. 10.1093/toxsci/kfz128 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stephens ML, Andersen M, Becker RA, Betts K, Boekelheide K, Carney E, Chapin R, Devlin D, Fitzpatrick S, Fowle JR3rd, Harlow P, Hartung T, Hoffmann S, Holsapple M, Jacobs A, Judson R, Naidenko O, Pastoor T, Patlewicz G, Rowan A, … Zurlo J (2013). Evidence-based toxicology for the 21st century: opportunities and challenges. ALTEX, 30(1), 74–103. 10.14573/altex.2013.1.074 [DOI] [PubMed] [Google Scholar]
- Stephens ML, Betts K, Beck NB, Cogliano V, Dickersin K, Fitzpatrick S, Freeman J, Gray G, Hartung T, McPartland J, Rooney AA, Scherer RW, Verloo D, & Hoffmann S (2016). The emergence of systematic review in toxicology. Toxicological , 152(1), 10–16. 10.1093/toxsci/kfw059 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Texas Commission on Environmental Quality (TCEQ). (2017). TCEQ guidelines for systematic review and evidence integration White Paper. https://www.tceq.texas.gov/assets/public/implementation/tox/dsd/whitepaper/srguidelines.pdf (accessed 13 January 2022).
- Thayer KA, Wolfe MS, Rooney AA, Boyles AL, Bucher JR, & Birnbaum LS (2014). Intersection of systematic review methodology with the NIH reproducibility initiative. Environmental Health Perspectives, 122(7), A176–A177. 10.1289/ehp.1408671 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thomas RS, Bahadori T, Buckley TJ, Cowden J, Deisenroth C, Dionisio KL, Frithsen JB, Grulke CM, Gwinn MR, Harrill JA, Higuchi M, Houck KA, Hughes MF, Hunter ES, Isaacs KK, Judson RS, Knudsen TB, Lambert JC, Linnenbrink M, Martin TM, … Williams AJ (2019). The Next Generation Blueprint of Computational Toxicology at the U.S. Environmental Protection Agency. Toxicological Sciences, 169(2), 317–332. 10.1093/toxsci/kfz058 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tousova Z, Froment J, Oswald P, Slobodník J, Hilscherova K, Thomas KV, Tollefsen KE, Reid M, Langford K, & Blaha L (2018). Identification of algal growth inhibitors in treated waste water using effect-directed analysis based on non-target screening techniques. Journal of Hazardous Materials, 358, 494–502. 10.1016/j.jhazmat.2018.05.031 [DOI] [PubMed] [Google Scholar]
- Traoré H, Crouzet O, Mamy L, Sireyjol C, Rossard V, Servien R, Latrille E, Martin-Laurent F, Patureau D, & Benoit P (2018). Clustering pesticides according to their molecular properties, fate, and effects by considering additional ecotoxicological parameters in the TyPol method. Environmental Science and Pollution Research International, 25(5), 4728–4738. 10.1007/s11356-017-0758-8 [DOI] [PubMed] [Google Scholar]
- U.S. Environmental Protection Agency (USEPA). (2014). US Environmental Protection Agency. Next generation risk assessment: incorporation of recent advances in molecular, computational, and systems biology (Final Report) Washington, DC., Environmental Protection Agency. EPA/600/R-14/004. https://ofmpub.epa.gov/eims/eimscomm.getfile?p_download_id=520491 (accessed 13 January 2022). [Google Scholar]
- U.S. Environmental Protection Agency (USEPA). (2020a). New approach methods work plan: Reducing use of animals in chemical testing U.S. Environmental Protection Agency, Washington, DC. EPA 615B2001. https://www.epa.gov/sites/default/files/2020-06/documents/epa_nam_work_plan.pdf (accessed 13 January 2022). [Google Scholar]
- U.S. Environmental Protection Agency (USEPA). (2020b). ORD staff handbook for developing IRIS assessments (Public comment draft, Nov 2020) U.S. Environmental Protection Agency, Office of Research and Development, Washington, DC, EPA/600/R-20/137. https://ofmpub.epa.gov/eims/eimscomm.getfile?p_download_id=541571 (accessed 13 January 2022). [Google Scholar]
- Villeneuve DL, Coady K, Escher BI, Mihaich E, Murphy CA, Schlekat T, Garcia-Reyero N (2019). High-throughput screening and environmental risk assessment: State of the science and emerging applications. Environmental Toxicology and Chemistry, 38(1), 12–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Voigt K, Brüggemann R, & Pudenz S (2004). Chemical databases evaluated by order theoretical tools. Analytical & Bioanalytical Chemistry, 380(3), 467–474. 10.1007/s00216-004-2794-8 [DOI] [PubMed] [Google Scholar]
- Wang RL, Edwards S, & Ives C (2019). Ontology-based semantic mapping of chemical toxicities. Toxicology, 412, 89–100. 10.1016/j.tox.2018.11.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Watford S, Edwards S, Angrish M, Judson RS, & Paul Friedman K (2019a). Progress in data interoperability to support computational toxicology and chemical safety evaluation. Toxicology and Applied Pharmacology, 380, 114707. 10.1016/j.taap.2019.114707 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Watford S, Ly Pham L, Wignall J, Shin R, Martin MT, & Friedman KP (2019b). ToxRefDB version 2.0: Improved utility for predictive and retrospective toxicology analyses. Reproductive toxicology,89, 145–158. 10.1016/j.reprotox.2019.07.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Whaley P, Edwards SW, Kraft A, Nyhan K, Shapiro A, Watford S, Wattam S, Wolffe T, & Angrish M (2020). Knowledge organization systems for systematic chemical assessments. Environmental Health Perspectives, 128(12), 125001. doi: 10.1289/EHP6994. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wikoff DS, & Miller GW (2018). Systematic Reviews in Toxicology. Toxicological Sciences, 163(2), 335–337. 10.1093/toxsci/kfy109 [DOI] [PubMed] [Google Scholar]
- Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten JW, da Silva Santos LB, Bourne PE, Bouwman J, Brookes AJ, Clark T, Crosas M, Dillo I, Dumon O, Edmunds S, Evelo CT, Finkers R, Gonzalez-Beltran A, … Mons B (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3, 160018. 10.1038/sdata.2016.18 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Williams AJ, Grulke CM, Edwards J, McEachran AD, Mansouri K, Baker NC, Patlewicz G, Shah I, Wambaugh JF, Judson RS, & Richard AM (2017). The CompTox Chemistry Dashboard: A community data resource for environmental chemistry. Journal of Cheminformatics, 9(1), 61. 10.1186/s13321-017-0247-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Williams ES, Berninger JP, & Brooks BW (2011). Application of chemical toxicity distributions to ecotoxicology data requirements under REACH. Environmental Toxicology and Chemistry, 30(8), 1943–1954. 10.1002/etc.583 [DOI] [PubMed] [Google Scholar]
- Wolffe T, Whaley P, Halsall C, Rooney AA, & Walker VR (2019). Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environment International, 130, 104871. 10.1016/j.envint.2019.05.065 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Woodruff TJ, & Sutton P (2014). The Navigation Guide systematic review methodology: a rigorous and transparent method for translating environmental health science into better health outcomes. Environmental Health Perspectives, 122(10), 1007–1014. 10.1289/ehp.1307175 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yang Y, Yun X, Liu M, Jiang Y, Li QX, & Wang J (2014). Concentrations, distributions, sources, and risk assessment of organochlorine pesticides in surface water of the East Lake, China. Environmental Science and Pollution Research International, 21(4), 3041–3050. 10.1007/s11356-013-2269-6 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Data pertaining to the present study are available on the ECOTOX website: www.epa.gov/ecotox and at https://edg.epa.gov/metadata/catalog/main/home.page. Data, associated metadata, and calculation tools are also available from the corresponding author (olker.jennifer@epa.gov).