A Critical Review of the Availability, Reliability, and Ecological Relevance of Arctic Species Toxicity Tests for Use in Environmental Risk Assessment

Rebecca J Eldridge; Benjamin P de Jourdan; Mark L Hanson

doi:10.1002/etc.5247

. 2021 Dec 29;41(1):46–72. doi: 10.1002/etc.5247

A Critical Review of the Availability, Reliability, and Ecological Relevance of Arctic Species Toxicity Tests for Use in Environmental Risk Assessment

Rebecca J Eldridge ^1,², Benjamin P de Jourdan ¹, Mark L Hanson ^2,^✉

PMCID: PMC9304189 PMID: 34758147

Abstract

There is a pressing need to understand the impact of contaminants on Arctic ecosystems; however, most toxicity tests are based on temperate species, and there are issues with reliability and relevance of bioassays in general. Together this may result in an underestimation of harm to Arctic organisms and contribute to significant uncertainty in risk assessments. To help address these concerns, a critical review to assess reported effects for these species, quantify methodological and endpoint relevance gaps, and identify future research needs for testing was performed. We developed uniform criteria to score each study, allowing an objective comparison across experiments to quantify their reliability and relevance. We scored a total of 48 individual studies, capturing 39 tested compounds, 73 unique Arctic test species, and 95 distinct endpoints published from 1975 to 2021. Our analysis shows that of 253 test substance and species combinations scored (i.e., a unique toxicity test), 207 (82%) failed to meet at least one critical study criterion that contributes to data reliability for use in risk assessment. Arctic‐focused toxicity testing needs to ensure that exposures can be analytically confirmed, include environmentally realistic exposure scenarios, and report test methods more thoroughly. Significant data gaps were identified as related to standardized toxicity testing with Arctic species, diversity of compounds tested with these organisms, and the inclusion of ecologically relevant sublethal and chronic endpoints assessed in Arctic toxicity testing. Overall, there needs to be ongoing improvement in test conduction and reporting in the scientific literature to support effective risk assessments in an Arctic context. Environ Toxicol Chem 2022;41:46–72. © 2021 The Authors. Environmental Toxicology and Chemistry published by Wiley Periodicals LLC on behalf of SETAC.

Keywords: Bioassays, Strength of methods, Marine toxicity tests, Aquatic toxicology, Reliability, Ecological risk assessment

INTRODUCTION

Environmental risk assessors and others who rely on the ecotoxicological peer‐reviewed literature to aid in regulatory decision‐making have identified data reliability and reporting as a critical issue preventing data uptake (see Borgert et al., ²⁰¹⁶; Hanson et al., ²⁰¹⁷; Harris & Sumpter, ²⁰¹⁵; Thoré et al., ²⁰¹⁹). In the present study, reliability refers to the completeness of reported methodologies—specifically, whether the experimental design is sufficiently described to ensure reproducibility of the method and accuracy of the results (Ågerstrand, Kuster, et al., 2011; Moermond et al., ²⁰¹⁶), with the specific purpose of informing environmental risk assessment (ERA). Well‐designed studies with relevant species and endpoints contribute to the effectiveness of the ERA process (Ågerstrand, Breitholtz, & Rudén, 2011; Ågerstrand, Kuster, et al., ²⁰¹¹; Küster et al., ²⁰⁰⁹). High‐quality data and effective reporting reduce the uncertainty around the potential for effects and give risk assessors greater confidence in their decisions and recommendations. Furthermore, methodologically robust toxicity tests are vital in helping to reduce unnecessary animal testing within ecotoxicology (Burden et al., 2020; Vergauwen, ²⁰¹⁸). Ultimately, well‐designed and ‐performed studies free up resources that could be used to perform additional studies that will further ERAs, instead of repeating studies to confirm or refute inconsistent observations (Hanson & Brain, 2021).

While many regulatory agencies have internal screening criteria for data inclusion in ERAs (e.g., Henry & Pease, ²⁰¹⁶; US Environmental Protection Agency [USEPA], ²⁰⁰³), there is currently no single, standard approach used to critically evaluate the reliability of data being used in ERA. Several approaches have been developed and proposed for ad hoc screening of the peer‐reviewed literature (Ågerstrand et al., 2013; Durda & Preziosi, ²⁰⁰⁰; Hanson et al., ²⁰¹⁹; Hobbs et al., ²⁰⁰⁵; Klimisch et al., ¹⁹⁹⁷; Moermond et al., ²⁰¹⁶; Schneider et al., ²⁰⁰⁹; Van Der Kraak et al., ²⁰¹⁴). For example, Klimisch et al. (1997) proposed an evaluation system whereby data are sorted into one of four reliability categories for use in risk assessment, ranging from reliable without restriction to not reliable or assignable. Criteria regarding thorough reporting of methods, results, and inclusion of appropriate controls are outlined to aid in determining which category is appropriate for a particular study. This approach relies heavily on expert judgment and, thus, could be seen as relatively subjective in its application. Since then, several papers (e.g., Ågerstrand et al., ²⁰¹³; Durda & Preziosi, ²⁰⁰⁰; Schneider et al., ²⁰⁰⁹) have expanded on the relatively brief descriptors outlined by Klimisch et al. (1997) and broadened the scope beyond acute studies. Schneider et al. (2009) offered an instrument called ToxRTool whereby scores are given to different groups of criteria resulting in the assignment of the study to one of the Klimisch categories. This technique helps address the criticisms of subjectivity and lack of transparency that have been identified when using Klimisch categories as a stand‐alone approach in determining data reliability (see Kase et al., ²⁰¹⁶). Ultimately, attempts to objectively assess data quality rely on a combination of using such tools and expert judgment. Beyond assessing the quality of the data, a reasonable and neutral interpretation of results is crucial to ensure that ecotoxicology as a discipline maintains credibility in both the scientific and the public spheres (Brain & Hanson, 2021). Overemphasis of effects not only can have damaging implications for the reputation of the discipline but can also tie up funding and resources in addressing problems that potentially do not require resolution (Hanson & Brain, 2020).

It is important to note that there are numerous reasons why a study may not be recommended for risk‐assessment purposes, and these are not judgments that the research is wholly uninformative or does not contribute to our larger understanding of risk. Experimental design and execution could be excellent, but if the question being asked is not relevant to a risk scenario, the utility of the data for ERA is likely limited. Relevance in this context refers to the applicability of the observations to ERAs and the utility of the data in informing decision‐making (Ågerstrand, Kuster, et al., 2011; Moermond et al., ²⁰¹⁶). Whether specific data are relevant to a risk scenario depends on the risk assessment in question, but the ecological relevance of various endpoints (i.e., mortality being more definitively tied to population‐level effects than biomarker responses) can be assessed independently of a specific risk scenario and was the focus in the present review.

Thorough documentation and transparency are needed to accurately assess the reliability of data, but these elements can be lacking in published studies, leading to difficulty in verifying the methods used and the results. Regulators often rely on studies conducted under good laboratory practices or according to standard methods (e.g., International Organization for Standardization, Organisation for Economic Co‐operation and Development, and USEPA), especially for plant protection products, because these guidelines have been developed to reduce uncertainty in toxicological testing (Borgert et al., 2016). However, it is not always possible to obtain these types of data for all compounds, and other published literature is often the source for deriving acceptable environmental values (Ågerstrand et al., 2013; Henry & Pease, ²⁰¹⁶). This is especially true for tests and assessments for ecological risk where there is a lack of standard test development, such as the Arctic.

It has been long noted in the literature that there is a lack of testing and understanding around Arctic species' sensitivity to contaminants and that this lack of information may result in undue harm to this ecosystem (Chapman & Riddle, 2003). Of the available data, only a fraction may be reliable enough to provide accurate insight into environmental risk. This gap is in part due to the lack of standardized laboratory testing of Arctic species, as noted by others (Chapman & Riddle, 2005). The response of relatively numerous temperate aquatic species to a vast array of contaminants has been characterized (Chapman & Riddle, 2003; USEPA, 2021), but critical differences in the structure and function of Arctic and temperate ecosystems mean extrapolating these results could be a source of significant uncertainty. For example, Arctic ecosystems lack the same functional redundancy that is present elsewhere because there may only be one or two different species that play a similar functional role, while in temperate, tropical, or boreal ecosystems, there may be tens or hundreds of species with similar functional niches (Aune et al., 2018). Therefore, assessing and supporting the conduct of high‐quality Arctic ecotoxicology data are vital to Arctic ecosystem protection.

The presence of contaminants in Arctic marine ecosystems (Sonne et al., 2021) underlines the importance of generating high‐quality, Arctic‐specific toxicity data to provide accurate and robust insight for ERA in the region. Human activities in this region such as mineral extraction, hydrocarbon exploration, tourism, and shipping all pose potential threats to flora and fauna through the release of contaminants into aquatic ecosystems (Arctic Monitoring and Assessment Programme, 2017; Aulanier et al., ²⁰¹⁷; Hsiao, ¹⁹⁷⁸). For example, in 2008, the US Geological Survey estimated that up to 90 billion barrels of oil (~14 trillion L), 1669 trillion cubic feet (~47 trillion m³) of natural gas, and 44 billion barrels (~7 trillion L) of natural gas liquids remain undiscovered in the Arctic (US Department of the Interior, ²⁰⁰⁸). If and when these resources are extracted, these processes bring to light several contaminants of concern; namely polycyclic aromatic hydrocarbon (PAH) compounds that are present in crude oil. As risk assessments move forward in the Arctic, a baseline of data gaps is needed to ensure proper test prioritization.

The objective of this critical review was to quantitatively characterize the current state of knowledge in Arctic ecotoxicological research regarding effects of contaminants on Arctic organisms. Specifically, we sought to objectively and transparently assess the methodological strengths and weaknesses (i.e., reliability) of available toxicity data found in the peer‐reviewed and accessible gray literature through the creation and application of a scoring rubric. The results from the scoring rubric were used to analyze trends in current research practices; identify gaps in study design, conduct, and reporting; and highlight the future work needed to address these deficiencies and contribute to the broader discussion of data reliability, both in Arctic ecotoxicology and beyond. Finally, this approach allowed for the highlighting of knowledge gaps in available Arctic species, endpoints, and contaminants with recommendations to address in future toxicity testing.

METHODS

Literature search and initial screening

An overview of the methodological process for this exercise is presented in Figure 1 (for interpretation of the references to color in these figures, the reader is referred to the web version of this article). Literature searches were conducted from January 2018 to June 2021 to gather relevant studies for assessment in the present review. The University of Manitoba Libraries' database search engines were used to conduct Boolean searches from the Web of Science, Wiley Online Library, TOXLINE, ProQuest, and ScienceDirect databases. A Boolean search was conducted using the search terms Arctic OR polar AND ecotoxicology, toxicology, toxic*, effects, contaminants, test, assay, exposure, NOT Antarctic. These search terms generated relevant articles that were then subjected to backward and forward searching, which entails reviewing relevant sources cited by the obtained article and sources that have cited the article itself (Brocke et al., 2009).

Overview of the process whereby Arctic laboratory toxicity tests were evaluated for the reliability and relevance of their data for use in environmental risk assessment.

Initial screening of retrieved articles was completed to filter out any non‐English studies and those that were not peer‐reviewed or published in an indexed scientific journal (i.e., one with an assigned impact factor). The process of backward and forward searching highlighted data sets available from conference proceedings and other gray literature (e.g., Arctic and Marine Oil Spill Program [AMOP], government technical reports) that were highly relevant to this subject matter. Despite targeting peer‐reviewed publications for this exercise, the sporadic and sparse nature of Arctic species toxicity tests became apparent throughout the metadata collection phase. To address this, technical reports were included in the full evaluation (e.g., Department of the Environment Beaufort Sea Technical Report 11; Percy & Mullin, ¹⁹⁷⁵) where they were publicly accessible through database searches or easily available from a public forum. Where technical reports were not accessible (e.g., US National Marine Fisheries Service technical reports), titles were gathered into a table as references (Supporting Information Table S1) but not scored. The AMOP abstracts from 1978 to 2018 were accessed through personal copies available from the 2019 AMOP conference. At the time of this writing, AMOP abstracts were only available to the authors up until 2018. The inconsistency of including gray literature was outweighed by the importance of incorporating this widely accessible and significant pool of Arctic‐related data. It is also important to note that these publications represented a mechanism that was designed to complement standard peer‐reviewed articles. A significant amount of Arctic ecotoxicology data have been gathered by Canadian government departments and programs (e.g., former Department of Indian Affairs and Northern Development, Newfoundland Oil Burn Experiment, Oil‐in‐Ice Joint Industry Program) where studies were not officially published or peer‐reviewed, but excluding these data on this basis alone would have detracted from the overall purpose of this assessment and resulted in eight fewer studies from which data points could be gathered. For this reason, abstracts that reported results were subsequently included in the analysis if they also met the additional inclusion criteria outlined in the following section.

Inclusion and exclusion criteria

Reports and literature that passed initial screening were organized into a table where studies were further evaluated for their relevance to the present review. The following criteria determined which studies proceeded to the scoring phase:

Must be an aquatic toxicity test (freshwater, marine, or estuarine), terrestrial tests were excluded;
Must be laboratory toxicity data or deliberate field exposure (e.g., mesocosm studies, studies that measured effects in wild populations after accidental contaminant releases were excluded);
Must use a test species endemic to an Arctic region or be conducted in an Arctic context with a species that can be found in an Arctic region (see next paragraph for further details).

In summary, studies returned from the search were excluded if they did not include a deliberate exposure to a test species found in the aquatic Arctic environment. Toxicity tests with seabirds were included as aquatic toxicity tests because seabirds spend >50% of their lives at sea (Brown, 1980). This behavior means that they have the potential to be exposed to marine contaminants both directly (e.g., swimming through contaminated areas) and indirectly (e.g., biomagnification via feeding on contaminated fish; Braune et al., ²⁰¹²; Campbell et al., ²⁰⁰⁵). Some retrieved studies included test species with Arctic distribution but that were not endemic to this region. For example, the blue mussel Mytilus edulis is commonly tested as a temperate species because it has geographical distribution throughout the North Atlantic ocean (see Falfushynska et al., ²⁰¹⁹); however, it has also been tested as an Arctic species because of its presence in the Arctic ocean (see Thyrring et al., ²⁰¹⁵). Similarly, the copepod Calanus finmarchicus exists geographically in both oceans and has been tested as both an Arctic (see Faksness et al., ²⁰¹¹; Grenvald et al., ²⁰¹³) and a non‐Arctic (e.g., Hansen et al., ²⁰¹³; Øverjordet et al., ²⁰¹⁴) species for comparison. In the case of the blue mussel, the study was included if conducted in an Arctic context (e.g., based on study objective, environmental parameters, or exposure scenario) but excluded if conducted in a non‐Arctic context. The copepod C. finmarchicus was included from all papers because its prevalence and importance in Arctic regions are key to ecosystem processes (Hansen et al., 2009). Including all aquatic toxicity tests using a specific species solely because it is found in Arctic regions, although found elsewhere, would broaden the scope of the present review beyond its original purpose of describing the current state of knowledge of Arctic‐specific species within ecotoxicology.

Scoring

The objective of the scoring exercise was to assess the methodological strengths and weaknesses of published studies to determine the reliability of the data for use in ERA. To accomplish this, a scoring system was modified from Hanson et al. (2019), which was developed for primary producer toxicity tests and based on previously published data reliability guidelines and criteria from standard methods (Klimisch et al., 1997; Schneider et al., ²⁰⁰⁹; US Environmental Protection Agency, ²⁰¹²). The aim of identifying scoring criteria a priori was to increase the transparency and objectivity of the process.

Information regarding the exposure scenario, test conditions, and results for each study (i.e., exposure durations, test vessels, environmental parameters, x% effect concentration [ECx] values, statistical analyses performed) was incorporated into a spreadsheet to facilitate scoring and comparison between studies. The spreadsheet required modifications over several iterations to ensure that pertinent information from all assessed studies could be captured (e.g., the inclusion of additional columns for media properties), but the basic scoring approach and weightings did not change. The system was comprised of three groups of criteria based on (1) test substance, (2) test organism and experimental system, and (3) test design, statistics, and results. Within these groups, several key criteria were identified as critical study components (described in more detail in section Data handling), and overall study scores were multiplied by a factor of 0.5 for each critical criterion that was not met (i.e., where the uncertainty introduced by its absence is considered exceptional). Experimental procedures were numerically scored with binary responses according to 15 criteria (i.e., 1 if the criterion was met and 0 if it was not). An additional category called “reviewer judgment” was added as part of the criteria when the method was outlined a priori, in the case where an unforeseen issue not captured in the rubric rendered a study unreliable (e.g., control performance was reported but would not be considered adequate or significant contamination of test compound in controls). This criterion ultimately did not have to be used for any of the studies assessed in the present review. The scoring rubric is presented in Table 1. To be considered reliable for use in risk assessment, we set a minimum overall reliability score threshold of 7.5 out of 15, with all critical study criteria (those in bold in Table 1) being met. Tests that achieved exactly 50% were considered to be reliable. Although the numerical value of 7.5 is arbitrary, an overall reliability score above this value means that the study did not achieve a score of 0 in any of the critical criteria, which was our minimum threshold for being considered reliable.

Table 1.

Scoring criteria, rationale for their use, and elaboration on scoring used to assess the methodological quality of available Arctic species ecotoxicological data^a

Open in a new tab

^{^a}

Criteria presented in bold are considered critical components of a study for reliable use in risk assessment. Each bold criterion scored with a 0 results in a multiplication factor of 0.5 on the overall reliability score of the study.

USEPA = US Environmental Protection Agency; OECD = Organisation for Economic Co‐operation and Development; ASTM = ASTM International; ISO = International Organisation for Standardisation; NOEC = no‐observed‐effect concentration; LOEC = lowest‐observed‐effect concentration; ECx = x% effect concentration; LCx = x% lethal concentration.

Each endpoint that was assessed in each study was additionally given a score based on relevance of the response, which was not a factor in the overall reliability score, but was assessed separately to gauge utility for risk assessment. Endpoints were classified into one of eight categories based on the general descriptor of the response (i.e., biochemical, cellular, physiological, behavioral, growth, mortality, reproduction, and ecosystem‐level) and scored based on potential impacts with increasing levels of biological organization. For example, endpoints that have more relevance to sustaining populations or communities were scored higher, while those with little or no linkage to higher‐level effects were scored lower (US Environmental Protection Agency, ²⁰¹⁶). Though sublethal endpoints such as behavior and biomarker response are important to characterize and add value to our understanding of toxicity, they do not supersede endpoints that have more direct and definitive population‐level effects such as mortality and reproduction. This endpoint weighting approach differs from current Canadian regulatory pass or fail practices but has been integrated into assessments such as the US National Resource Damage Assessment (Office of the Federal Register, National Archives and Records Administration, 2021). A complete list of endpoints assessed in the present study and justification for their inclusion in each category are presented in Table 2.

Table 2.

Scores for relevance of laboratory responses of available Arctic ecotoxicological data^a

Score	General descriptor	Rationale	Endpoints observed in the present study
Score	General descriptor	Rationale	Mammal	Bird	Fish	Invertebrate	Primary producer
0	No known linkage to survival, development, growth, and/or reproduction	If a response has no real or hypothetical linkage to higher‐level effects, then it has little to no value in elucidating ecological risk.	NA	NA	NA	NA	NA
1	Biomarker response such as genomic or metabolomic measures	While informative from a mechanistic perspective, the relevance of these responses to population‐, community‐, and ecosystem‐level effects is considered very low. In many cases, the responses characterized are regular processes to detoxify or adapt to a transient stressor, which in and of themselves are not adverse.	NA	NA	Liver transcriptome changes (1) Gene expression (CYP1A) (27)	Gene expression (CYP330A1) (18) Gene expression (GST mRNA) (18,19,21,39)	Phosphate flux (44) Potential nitrification (44) Potential oxygen consumption (44) Silicate flux (44)
2	Biomarker responses such as enzymatic changes or general physiology	While informative from a mechanistic perspective, the relevance of these responses to population‐, community‐, and ecosystem‐level effects is considered very low. In many cases, the responses characterized are regular processes to detoxify or adapt to a transient stressor, which in and of themselves are not adverse.	Cell viability (6a,6b,14a,14b) Electrolyte changes (8) Enzyme and metabolite changes (8) Erythrocytic changes (8) Hormonal changes (8) Intracellular thiol levels (14a,14b) Leukocytic changes (8) Lymphocyte proliferation (6a,6b,14a,14b,29a,29b,29c,29d) Natural killer cell activity (6a,6b) Nitrogenous compound changes (8) Phagocytosis (6a) Metallothionein induction (14a,14b)	NA	Bile content/analysis (7a,7b) EROD activity (7a,7b,16,40) Osmolality (40) Plasmatic sex steroid measurements (16) Plasmatic vitellogenin (16) PAH metabolites (1,16) Thyroid status (40) Vitamin status (40)	Lysosomal membrane stability (10) Acetylcholinesterase response (47) Acyl‐coenzyme A oxidase response (47) GST response (47)	NA
3	Changes in behavior	Behavior, especially if related to reproduction, can have impacts on populations and communities of organisms. Other behavioral changes, such as predator avoidance, could result in increased mortality.	NA	NA	NA	Attraction–repulsion response (42a) Behavioral response (42a) Curling of tube feet (9) Emergence (9) Feeding/feed response/grazing/ingestion rate (42a,46) Foot protrusion (9) Locomotory activity (43) Loss of attachment ability (9) Loss of cover (9) Loss of equilibrium (45a,45b) Mantle gape (9) Ostial closure (9) Respiration (3b,42a) Retraction of tube feet (9) Siphon retraction (9) Spine droop/rigidity (9) Stimulus response (9)	NA
4	Changes in growth and development, such as mass, other morphometrics, and phenology	These responses are typically highly relevant to the success or sustaining of populations and communities in an ecological context.	NA	NA	Cardiac activity and arrhythmia (34,27) Condition index (7a,7b) Eye pigmentation (13a,13b) Time to hatch (13a,13b) Dry mass (27) Standard length (27) Myotome height (27) Eye diameter (27) Pericardial edema (27) Lipid content (27)	Fecal pellet production/volume (23,25,26,28,35,46,49) Lipid content (49,50,51) Carbon content (50) Nitrogen content (50) Prosome length (50) Fatty acid composition (48)	Chlorophyll‐a content (28,44) Growth inhibition (4,24) ¹⁴C‐incorporation (44)
5	Changes in reproduction, including malformation of young	These responses are typically highly relevant to the success or sustaining of populations and communities in an ecological context.	NA	NA	Malformations (34,27) Underdeveloped embryos (13a,13b)	Clutch number (51) Clutch size (51) Development (4) Egg production (23,25,26,35,38,46,49,51) Fraction reproducing (38) Naupliar development (17) Spawning interval (51)	NA
6	Mortality and other surrogates	Loss of individuals is highly relevant to the success or sustaining of populations and communities in an ecological context.	NA	Survival to 90% of development (2)	Mortality (1,4,5a,5b,11a,11b,13a,13b,15a,27,34,37,42a) Hatching (13a,13b,27,33)	Mortality (3a,3b,3c,3d,3e,4,5a,5b,9,10,11a,11b,15a,15b,17,18,19,20a,20b,20c,21,22a,22b,30,31,32,33,36,37,38,39,48,49,51,42a,45a,45b,46) Hatching (17,23,25,26,38,51)	Biomass‐specific primary production (28) Primary producer biomass (41a,41b,41c)

Open in a new tab

^{^a}

Based on their linkage to effects on organisms at the population and community levels. Study number corresponding to each endpoint is reported in parentheses.

CYP1A/CYP330A1 = cytochrome P450 1A/330A1; GST = glutathione S‐transferase; mRNA = messenger RNA; EROD = 7‐ethoxyresorufin O‐deethylase; PAH = polycyclic aromatic hydrocarbon.

The rubric for the scored studies can be found in the Supporting Information for this article. Other researchers are encouraged to modify criteria depending on their specific use and recalculate scores as needed if certain aspects of the sheet are more or less relevant to their particular scenario. A blank template is also included where criteria can be removed, added, or modified and used for additional studies.

Data handling

Metadata from this exercise were compiled into a spreadsheet for further consideration. The in vitro and in vivo studies followed the same scoring process, and in cases where a cell line was used, the test species and aquatic habitat (freshwater or marine) were categorized based on the source animal (e.g., ringed seal [Pusa hispida] cell tests filed under marine mammal tests). Unidentified test species (e.g., a sediment sample with unidentified bacterial and algal communities in Petersen et al. [2008]) were counted as a discrete species category, as well as tests that identified the species only by genus (e.g., Gammarus sp. was differentiated from Gammarus oceanicus and counted as a different species).

Data are presented for individual papers (peer‐reviewed publications [see Olsen et al., ²⁰¹³], abstracts [see Faksness et al., ²⁰¹¹], or theses [see Moore, ²⁰¹⁶]) collectively under the term studies, as well as by unique toxicity tests (test substance and species combinations, e.g., Calanus glacialis exposed to pyrene [Jensen et al., ²⁰⁰⁸]) or experimental combinations (test substance/species/endpoint combinations, e.g., 7‐ethoxyresorufin O‐deethylase activity in Arctic cod exposed to produced water [Geraudie et al., ²⁰¹⁴]). Test organisms were divided by organism group (i.e., invertebrate, vertebrate, or primary producer) and organism type as defined by the USEPA ECOTOX Knowledgebase (i.e., alga, bird, crustacean, fish, mammal, mollusk, or other invertebrate [US Environmental Protection Agency, ²⁰²1]), with the additional category of microbial consortium included based on findings in the present study. Test substances were organized into three main groups based on their structure and use: oil‐related contaminants, inorganic contaminants (e.g., metals), and organics (e.g., phenols and flame retardants). This was done to easily identify more generally groups of compounds where testing was either strong or lacking. Last, endpoints were categorized into eight effect categories, again based on the USEPA ECOTOX Knowledgebase: behavioral, biochemical, cellular, ecosystem, growth, mortality, physiological, and reproduction (US Environmental Protection Agency, ²⁰²1). This was done to assist in the assignment of relevance scores to individual endpoints.

Oil‐related contaminants in this context are defined as any type or brand of fuel oil or gasoline, products designed to clean up or remediate sites contaminated with fuel oil or gasoline, and single compounds that are found in fuel oil or gasoline when tested in the context of fuel oil or gasoline contamination. Specific test substances grouped within oil‐related contaminants were difficult to categorize because of the extensive number of products (e.g., types of oil and oil spill–remediation technologies), preparation methods (e.g., oil slick, mechanically or chemically dispersed water accommodated fractions [WAFs], in situ burn residues), and additives (e.g., chemical dispersants or shoreline washing agents) used in this testing. To facilitate data analysis and assist in the understanding of knowledge gaps, crude oil was counted as a single test substance regardless of oil type, preparation method, or additives. Oil spill–remediation technologies were counted as a single test substance based on operational usage (e.g., Absorrep K212 and Bioversal both counted as shoreline washing agents, Dasic NS and Corexit 9500 both counted as dispersants).

To investigate whether there was a correlation between the impact factor of the journal in which a paper was published and the reliability of the data presented, historical impact factors were retrieved from the Web of Science InCites Journal Citation Report (Clarivate Analytics, 2021) for the year in which the study was published. Where no impact factor was available for the time of publication, the average from the most recent 5 years was calculated and used in its place.

Although the scoring rubric was designed to be as objective as possible, there remained areas that require clarification for full transparency. For example, Criterion 11 requires a minimum of three replicates per exposure to achieve a score of 1; however, the line between true replication and pseudoreplication is sometimes difficult to discern (e.g., replicate dishes being held in the same bath in Frantzen et al. [2012] or the inclusion of 12 eggs per dose group pooled together for survival calculations in Braune et al. [2012]). Arguments could be made that replicates using the same preparation of stock or test solutions are actually pseudoreplicates or, similarly, experimental units being incubated in the same growth chamber. To limit subjective differences in scoring between studies for this criterion, replicates were simply defined as unique exposure vessels.

Environmental relevance of test substances (Criterion 6) is described in detail in Supporting Information, Table S2. Note that environmentally relevant concentration thresholds listed in this table are based on general literature values, but study‐specific justification may change the environmentally relevant value for a particular study. For example, a general environmentally relevant concentration of PAHs may be 189 µg/L based on data accumulated from an oil spill; however, Petersen et al. (2008) used a measured value of sediment porewater concentrations to justify environmental realism because their study mirrored this type of exposure and, as such, were scored as 1 in this category.

The purity and/or grade of some oil‐related contaminants are difficult to characterize and compare across studies (Criterion 1). Justification for purity or grade of these test substances can be found in Supporting Information, Table S3. Similarly, relevant environmental parameters to include when describing methodologies for toxicity tests can vary depending on test species and whether the test was conducted in vitro or in vivo. Justification for scoring in this criterion can be found in Supporting Information, Table S4.

Statistical analyses

For comparisons, scores from each group of criteria (test substance; test organism and system; test design, statistics, and results) were converted to percentages of the total achievable score in each group. These percentages were used for statistical analyses. Before assessing the data for statistically significant differences between organism groups, assumptions of normality were assessed using the Shapiro‐Wilk test and homogeneity of variances using Bartlett's test (base package in R; R Foundation for Statistical Computing, ²⁰¹²). Then, the statistical significance of differences between organism groups was assessed through Kruskal‐Wallis rank tests. Post hoc Dunn tests were performed with a Bonferroni p adjustment to reveal where significant differences occurred. Spearman's correlation coefficients (rho) were calculated for non‐normally distributed data with unequal variance using linear regression models.

Scores are presented in the Results section as mean ± standard deviation.

RESULTS

Summary of reviewed studies

The search process returned 180 studies, 47 of which passed screening criteria for scoring. One additional PhD thesis (Moore, 2016) was included because it was publicly available and included a large volume of test results from freshwater Arctic fish. This resulted in a total of 48 individual studies (40 peer‐reviewed and 8 gray literature). The four chapters of the unpublished thesis were assessed independently. In total, there were a total of 51 separate scoring sheets in the Excel file (see Supporting Information, Study Scores). There were 253 unique toxicity tests (i.e., test substance and test species combinations), 73 unique Arctic test species (two unidentified groups of microbes not included in this total [Pančić et al., ²⁰¹⁹; Petersen et al., ²⁰⁰⁸]), 39 tested substances, and 95 endpoints, for a total of 596 experimental combinations (i.e., test substance/test species/endpoint combinations).

The volume of research produced in this field steadily declined from the 1970s (n = 45 unique toxicity tests) through the 1980s (n = 18) and 1990s (n = 0) and began to increase again in the first decade of the 2000s (n = 5), finally experiencing a period of rapid growth in the 2010s (n = 185); however, 104 tests of the 185 reported in the 2010s were from a single study (Moore, 2016). Excluding that study, 81 tests were performed in this decade. The relationship between number of unique toxicity tests conducted and study year was not statistically significant (Supporting Information, Figure S1). Moore (2016) was also the driver of the notable increase in testing of inorganic contaminants in the 2010s (Supporting Information, Figure S2); however, with that study excluded, oil‐related contaminants comprised the majority of tests throughout this decade.

Test species

Vertebrates were tested in 141, invertebrates in 104, and primary producers in 8 out of 253 unique toxicity tests (Supporting Information, Figure S3). These organism groups were further broken down by organism type within each category (Table 3). Of these, 107 of 253 discrete experiments used freshwater organisms (42% of all unique tested species), and 146 of 253 used marine organisms (58%; Supporting Information, Figure S4). Crustaceans were the most studied organism type in number of individual studies (n = 29, 60% of all studies), number of species tested (n = 31, 42% of all species), and number of endpoints studied (n = 26, 27% of total endpoints). They were also one of only two organism groups that encompassed both marine (n = 29 species) and freshwater (n = 2 species) organisms. Within this group, the class Malacostraca was the best studied (n = 41, 47% of crustacean tests) and the most diverse class in terms of number of species tested (n = 20, 23% of species within crustaceans). Hexanauplia was the second best‐studied class of crustaceans (n = 34, 39% of crustacean tests), and within Hexanauplia C. glacialis (n = 15, 44% of Hexanauplia tests) and C. finmarchicus (n = 14, 41% of Hexanauplia tests) were the primarily studied species. These two copepods were also the most frequently tested organisms of all crustaceans.

Table 3.

Summary of publicly available Arctic ecotoxicological laboratory tests that passed inclusion criteria for the present study as of June 2021

Organism type	Individual studies (n = 48)	Unique toxicity tests (n = 253)	Species tested (n = 73)^a	Compounds tested (n = 39)	Endpoints assessed (n = 95)
Crustacean	29	87	31	12	26
Fish	14	126	12	24	24
Mammal	4	13	8	8	12
Alga	3	6	6	3	3
Mollusk	5	12	11	5	13
Other invertebrate	4	5	3	2	10
Microbial consortium	2	2	NA	2	13
Bird	1	2	2	1	1

Open in a new tab

^{^a}

Two unidentified groups of microbes not included.

NA = not available.

Fish were the second most frequently tested organism type by number of individual studies (n = 14, 29% of studies) but were the best studied in terms of unique toxicity tests (n = 126, 50% of tests). This group was the second most diverse in terms of number of species tested (n = 12, 16% of all tested species); however, fish tests were entirely comprised of a single organism class (Actinopterygii, or ray‐finned fishes). Both marine (n = 7 species) and freshwater (n = 5 species, anadromous fish included) species made up this group; however, the majority of data points collected using fish were driven by a single study that tested a variety of metals, metal salts, and nitrogenous contaminants on freshwater fish (Moore, 2016). With that study included, Prosopium cylindraceum (round whitefish; n = 25, 20% of fish tests) and Salvelinus alpinus (Arctic charr; n = 25, 20%) were the most frequently tested species. With that study excluded, the keystone species polar cod (Boreogadus saida) was the most frequently tested (n = 11, 50% of remaining fish tests). Fish were tested against the widest variety of test substances compared to other organism groups (n = 24 unique test substances, 62% of all tested compounds), representing all three contaminant groups (oil‐related contaminants, inorganic contaminants, and organic contaminants).

Mammals were the third most frequently studied organism types based on number of individual studies (n = 4, 8% of studies) and number of unique toxicity tests (n = 13, 5% of unique tests). Twelve of the 13 mammalian tests were in vitro studies using orca (Orcinus orca), seal (Pusa hispida, Pagophilus groenlandicus), and polar bear (Ursus maritimus) cell lines to determine cellular, biochemical, and physiological effects. A single test (Engelhardt, 1981) was conducted using whole organisms to assess the clinical effects of exposure to oil pollution on polar bears.

Algal, mollusk, microbial, bird, and other invertebrate tests were a combined total of only 11% of unique toxicity tests (n = 27 tests). Algal tests principally used Arctic diatoms as test species (i.e., Bacillariophyceae and Coscinodiscophyceae; n = 4, 67% of algal tests), but green algae were also tested (Chlorophyceae; n = 1, 17% of algal tests). A single test was conducted on an identified species of phytoplankton (Lemcke et al., 2018). The “other invertebrate” category is composed of invertebrates that did not fall under either of the other organism types (i.e., crustacean or mollusk). This category includes the green sea urchin (Strongylocentrotus droebachiensis), sea spider (Nymphon gracile), and balloon jellyfish (Halitholus cirratus). The microbial consortium category captured two studies (Pančić et al., 2019; Petersen et al., ²⁰⁰⁸) that used an unidentified mixture of benthic algae and bacteria obtained from grab samples as test species. Seabirds were also tested in a single study (Braune et al., 2012). Scores from the studies testing seabirds and microbes were omitted in comparisons of scores between organism types to avoid skewing the data based on one and two studies, respectively, but were included in assessments of the data set as a whole (e.g., included when presenting number of tested compounds and endpoints among all studies but excluded in statistical analyses comparing overall reliability scores of different organism types).

Test substances

Contaminants used as test substances in Arctic ecotoxicological research were categorized into three groups: oil‐related contaminants, inorganic contaminants, and organic contaminants. The majority of experiments using inorganic contaminants were freshwater tests (n = 104 of 114 tests; 91%), whereas the other compound groups were mainly comprised of marine tests (98% of oil‐related contaminant tests [n = 125 of 127 tests] and 92% of organic contaminant tests [n = 11 of 12 tests]; Supporting Information, Figure S5). Oil‐related contaminants were used in 50% of all toxicity tests (n = 127), with crude oil comprising the majority of substances in this group (n = 63, 50% of tests using oil‐related contaminants). Crude oils from both Arctic (Alaska North Slope, Atkinson Point, Norman Wells, Cook Inlet, Kobbe, Troll, North Sea, and Prudhoe Bay) and non‐Arctic (Venezuela, Arabian Light, Lago Medio, Midale, and Pembina) regions were included and used as fresh and weathered, with and without suspended sediment, and in the presence and absence of ultraviolet light. In addition, specific compound groups found in oil (e.g., aromatic, asphaltic, or paraffinic compounds alone) were tested as individual test substances. Preparations of crude oil were either mechanically dispersed (i.e., physically agitated to create a WAF of oil) or chemically dispersed (i.e., crude oil used in conjunction with a chemical dispersant to create a chemically enhanced WAF of crude oil) or used after a period of natural attenuation by microorganisms was allowed to occur, as with in situ burn residues or a slick in mesocosm studies. Single compounds found in oil (naphthalene, 2‐methyl‐naphthalene, and pyrene) were the second most frequently tested group of substances in this category, used in 27% of oil‐related contaminant tests (n = 34). Spill response agents alone (i.e., no oil added) were used in 13% of oil‐related contaminant testing using either chemical dispersants (Corexit [9500, 9500A, or 9527], Dasic NS, Finasol OSR52, or Gamlen OD4000) or shoreline washing agents (Absorrep K212, Bios, Bioversal, Corexit 9580A, or Hela saneringsvaeske). The remaining tests in this category were conducted using heavy fuel oil (n = 2) and produced water (n = 7), an oil extraction by‐product. The vast majority of crustacean, algal, mollusk, other invertebrate, and microbial tests have been conducted with oil‐related contaminants (Figure 2); but there are some data for these substances across nearly every organism category.

Number of unique Arctic species toxicity tests captured in the present review (n = 253 as of June 2021) by organism group, with test substance type within bars (number below each bar is total number of unique tests).

Inorganic contaminants were made up of mostly metals and salts (e.g., copper, zinc, aluminum, sodium chloride, and calcium chloride) and were the second most tested class of substances, used in 45% of all tests (n = 114). More than three‐quarters (n = 104) of all fish tests in particular used inorganic contaminants, most of which were tested in the same study (Moore, 2016). They were also the only class of substances tested on seabirds. This compound group was the most diverse and included 23 unique test substances.

Organic contaminants were the least frequently tested compounds encompassing just under 5% of all toxicity tests (n = 12), all of which were tested on mammalian cell lines (in vitro studies) with the exception of a single test each with crustaceans and fish. Half of the tests that used this group of substances were from a single study (Desforges et al., 2017) where contaminant cocktails were derived from the blubber of wild marine mammals. On analysis, they were determined to be composed primarily of polychlorinated biphenyls (PCBs) and organochlorine pesticides. The remaining compounds tested in this category were various PCB congeners, tetrachlorobiphenyl, and unidentified phenolic compounds from crude oil.

Endpoints

The 95 distinct endpoints were grouped into eight effect categories, as shown in Figure 3. Mortality was the endpoint most frequently assessed, with 54% of all data points being attributed to this effect category (n = 319 of 596 experimental combinations). Within this effect category, crustaceans comprised the majority with 42% being attributed to this group (n = 134, 42% of mortality endpoints), followed by fish (n = 128, 40% of mortality endpoints), mollusks (n = 9), other invertebrates (n = 4), and birds (n = 2). All other effect categories were sublethal, chronic, and/or ecosystem‐based and take up a combined total of less than half of all data points observed (n = 277, 46% of all data points). Behavioral endpoints were the second most frequently assessed, comprising 14% of all tested combinations (n = 85). Within this effect category, crustaceans were again the most popular test organism at 78% of all data points (n = 66), followed by other invertebrates (n = 11) and mollusks (n = 8). The third most frequently assessed endpoint was growth, which was the primary endpoint for algal studies (n = 13 of 17 algal data points; 25% of all growth endpoints). This endpoint was also assessed in crustaceans (n = 19 of 51 growth endpoints; 37% of all growth endpoints), fish (n = 17; 33%), and mollusks (n = 2; 4%) as test species. The remaining five effect categories comprised a total of only 24% of all data points (n = 141). In order of most frequent appearance, they included biochemical endpoints (n = 50; 8% of all endpoints), cellular (n = 37; 6% of all endpoints), physiological (n = 28; 5% of all endpoints), reproduction (n = 20; 3% of all endpoints), and ecosystem‐based (n = 6; 1% of all endpoints) responses.

Number of data points for Arctic species captured in the present review (n = 596 as of June 2021) by effect category, with organism type within bars (number below each bar is total number of reported responses for that category).

A period of monitoring postexposure, framed as either a recovery period or an opportunity to observe latent effects, was included in 25% of individual studies (n = 12 of 48 studies), The terminology used in the study was mirrored in the scoring spreadsheet to present the results for these time points in the same context as originally reported.

Overall methodology scores

Overall reliability scores calculated from the rubric are presented as vertebrates, invertebrates, or primary producers in Figure 4. Scores were variable within and between groups; however, the majority of means, medians, and first and third quartiles for every group were <50% of the total achievable score. Mean scores expressed as a percentage of the total achievable score were greatest for vertebrates (36 ± 15), followed by invertebrates (30 ± 20) and primary producers (30 ± 23). The difference in means of overall reliability scores between vertebrates and invertebrates were found to be statistically significant, while primary producers were not significantly different from either of the other two groups (p < 0.05).

Overall reliability scores (as a percentage out of 15) for all unique toxicity tests reviewed in the present study (n = 253 as of June 2021) by their status as vertebrates, invertebrates, or primary producers (number below each bar is total number of unique toxicity tests for that group). Median values are represented by a black line within each box; lower and upper hinges correspond to the 25th and 75th percentiles, respectively; and whiskers extend to the largest or smallest value within 1.5 × the interquartile range from the hinge. Letters indicate statistical significance between groups (p < 0.05) based on Kruskal‐Wallis rank sum tests followed by post hoc Dunn tests with Bonferroni p adjustments.

The individual toxicity test scores for Groups A (test substance), B (test organism and test system), and C (test design, statistics, and results) and overall are presented by organism type in Figure 5. Birds and microbes have been omitted from this figure and statistical analyses because of the small number of scores obtained from these groups of studies (n = 2 for each); however, mean scores for each criteria group for all organism types can be found in Table 4. Tests achieved the greatest score in Group B (74 ± 28), followed by Group A (55 ± 16), Group C (50 ± 15), and overall across all organism types (34 ± 18). The overall reliability scores were generally lower than the average scores for specific groups of criteria because of the high proportion of tests that did not meet one or more of the critical study criteria outlined in the rubric, resulting in a multiplication factor of 0.5 applied to the overall reliability score for each one not met.

Scores (as a percentage out of 15) for all unique toxicity tests reviewed in the present study (n = 253 as of June 2021) by organism type. Median values are represented by a black line within each box; lower and upper hinges correspond to the 25th and 75th percentiles, respectively; and whiskers extend to the largest or smallest value within 1.5 × the interquartile range from the hinge. Letters indicate statistical significance between groups (p < 0.05) based on Kruskal‐Wallis rank sum tests followed by post hoc Dunn tests with Bonferroni p adjustments. No statistically significant differences were found for Group A (test substance).

Table 4.

Percentage of unique toxicity tests with a score of 1 for each criterion by organism type^a

		Crustacean	Fish	Alga	Mammal	Mollusk	Other invertebrate	Bird	Microbe	All unique toxicity tests
Group A	Criterion 1	79	95	100	54	33	60	100	50	84
	Criterion 2	64	98	33	85	67	40	100	100	80
	Criterion 3	15	7	17	0	33	20	0	50	11
	Criterion 4	14	5	0	8	33	20	100	100	11
	Criterion 5	71	94	100	92	92	100	100	50	86
	Mean score	57 ± 19	52 ± 11	47 ± 22	56 ± 13	60 ± 26	57 ± 28	83 ± 0^b	75 ± 12	55 ± 16
Group B	Criterion 6	94	16	33	100	100	100	100	100	56
	Criterion 7	93	100	100	100	100	100	100	100	98
	Criterion 8	79	97	83	100	100	100	100	100	91
	Criterion 9	15	88	17	0	8	0	0	0	49
	Mean score	53 ± 24	93 ± 18	50 ± 16	73 ± 7	54 ± 10	50 ± 0^b	75 ± 0^b	63 ± 18	74 ± 28
Group C	Criterion 10	23	89	0	92	8	0	100	50	57
	Criterion 11	55	9	100	92	67	40	100	50	37
	Criterion 12	84	100	33	92	100	100	100	100	93
	Criterion 13	0	1	0	0	0	0	0	50	1
	Criterion 14	38	90	83	8	17	20	100	50	61
	Criterion 15	32	87	0	0	17	40	100	0	55
	Mean score	42 ± 17	57 ± 10	43 ± 8	38 ± 6	40 ± 17	40 ± 14	80 ± 0^b	50 ± 14	50 ± 15
	Mean reliability score	30 ± 21	35 ± 13	27 ± 26	48 ± 18	34 ± 16	22 ± 8	80 ± 0^b	38 ± 31	34 ± 18

Open in a new tab

^{^a}

Bolded numbers indicate <50% of tests with a score of 1 for that criterion. Mean reliability scores ± standard deviation for each organism type are calculated per criterion group and presented in italics below each section.

^{^b}

All scores identical.

Within Group A, toxicity tests performed on mollusks had the greatest mean score (60 ± 26), although other invertebrates, crustaceans, mammals, fish, and algae were closely behind with mean scores ranging between 47 and 57%. Tests performed with algae scored the lowest in this criteria group (47 ± 22). Variability in scores was the greatest in this category for other invertebrates (coefficient of variation [CV] = 49%), while fish scores were the most consistent (CV = 21%).

Tests performed with fish had the greatest mean score among organism types for Group B (93 ± 18), followed by mammals (73 ± 7), mollusks (54 ± 10), crustaceans (53 ± 24), other invertebrates (50 ± 0), and algae (50 ± 16). Variability in scores for criterion Group B was greatest for tests using crustaceans (CV = 46%) with scores ranging from 0 to 100%. The least variability was found in the other invertebrate group (CV = 0%).

Within Group C, fish again had the greatest average score (57 ± 10). Algae, crustaceans, mollusks, and other invertebrates all scored in the 40% range (43 ± 8, 42 ± 17, 40 ± 17, and 40 ± 14, respectively); and mammals achieved the lowest average score of 38 ± 6. Variation in scores was greatest for mollusks (CV = 43%) and least for mammals (CV = 14%).

With regard to overall reliability scores (i.e., the sum of Groups A, B, and C multiplied by 0.5 for each critical criterion not met), mammalian tests scored the greatest with an average score of 48 ± 18 and had the greatest percentage of unique toxicity tests that were deemed most reliable for use in risk assessment (e.g., score >50%) according to the criteria outlined in the present review (n = 10 of 13 mammalian tests, 77%). Fish tests scored second greatest overall at 35 ± 13, followed by mollusks (34 ± 16), crustaceans (30 ± 21), algae (27 ± 26), and other invertebrates (22 ± 8). After mammals, algal tests had the second highest proportion of reliable studies with 33% of tests meeting the criteria, although only six unique toxicity tests were observed in total. Twenty of 87 crustacean tests were considered reliable for risk assessment (23%), 1 of 12 mollusk tests (8%), 10 of 126 fish tests (8%), and 0 of 5 tests using other invertebrates. In total, only 46 of 253 (18%) unique toxicity tests passed screening for data reliability for use in ERA.

Finally, statistically significant corelationships were found between score and study year for Groups B and C and overall reliability scores but not for Group A (Supporting Information, Figures S6–S9). The corelationship between overall reliability score and the impact factor of the journal in which the study was published was analyzed and determined to be statistically significant (Supporting Information, Figure S10).

Individual criteria scores

Data pertaining to individual criteria organized by organism type are presented in Table 4 and are visually represented in Supporting Information, Figures S11–S13. Bold numbers indicate that <50% of tests achieved a score of 1 in that criterion across all organism types. Criterion 7 (strain or source of test organism identified) was met the most frequently (n = 247 of 253 unique toxicity tests; 98%); followed by Criterion 12 (statistical methods described; n = 234; 92%); Criterion 8 (initial test organism characteristics described; n = 230; 91%); Criterion 5 (three or more test concentrations included; n = 217; 86%); Criterion 1 (source and purity of test substance reported; n = 212; 84%); Criterion 2 (any analytical confirmation of tested concentrations; n = 206; 81%); Criterion 14 (key measured raw values included; n = 159; 63%); Criterion 10 (relevant environmental parameters identified and reported; n = 148; 58%); Criterion 15 (control values reported and performance criteria met; n = 143; 57%); Criterion 6 (at least one exposure concentration < environmental concentrations; n = 138; 55%); Criterion 9 (standard protocol followed; n = 126; 50%); Criterion 11 (minimum of three replicates per exposure; n = 90; 36%); Criterion 3 (individual initial analytical confirmation of tested concentrations; n = 29; 11%); Criterion 4 (individual final analytical confirmation of tested concentrations; n = 28; 11%); and Criterion 13 (concentration–response model and parameters provided; n = 2; 1%).

Several organism types achieved a score of 1 for specific criteria in 100% of unique toxicity tests. These were algal tests in Criteria 1, 5, 7, and 11; fish tests in Criteria 7 and 12; mammalian tests in Criteria 6, 7, and 8; mollusk tests in Criteria 6, 7, 8, and 12; and other invertebrate tests in Criteria 5, 6, 7, 8, and 12. In addition, almost all fish tests achieved a score of 1 in Criteria 2 and 8 (98 and 97%, respectively). In contrast, several organism types did not achieve a score of 1 in any test for certain criteria. No algal tests achieved a score of 1 in Criteria 4, 10, 13, or 15 and likewise for crustacean tests in Criterion 13; mammal tests in Criteria 3, 9, 13, or 15; mollusk tests in Criterion 13; and other invertebrate tests in Criteria 9, 10, or 13.

Supporting Information, Figure S14 shows overall, among all organism types, the percentage of unique toxicity tests that achieved a score of 1 in the four critical criteria outlined in the rubric (criteria in bold text in Table 1). Of these four criteria considered critical components of a laboratory toxicity test for reliable use in risk assessment, the inclusion of at least three replicates (Criterion 11) was met in the fewest number of unique toxicity tests (n = 90, 36%). Eighty percent of tests (n = 206) included some analytical confirmation of test concentrations (i.e., measured concentrations in stock solution prior to diluting to create test solutions or in test solution stocks prior to allocating into individual experimental units; Criterion 2). The inclusion of a minimum of three test concentrations (Criterion 5) was met in 86% of tests (n = 217), and an appropriate description of statistical methods (Criterion 12) was met in 92% of tests (n = 234). These data are further broken down by organism type in Supporting Information, Figure S15. The strongest study assessed for each organism type and test substance category combination is presented in Table 5.

Table 5.

Arctic ecotoxicology studies with the greatest reliability scores for each organism type tested in the present study^a

Organism type	Test substance category	Study no.	Study	Test substance	Test species	Marine or freshwater	Endpoints	Overall reliability score (%)
Fish	Oil‐related contaminant	34	Nahrgang et al. (2016)	Crude oil	Boreogadus saida	Marine	Cardiac activity and arrhythmia Hatching Malformations Mortality	87
	Inorganic contaminant	30–33	Moore (2016)	Metals, metal salts, and nitrogenous contaminants	Coregonus clupeaformis Prosopium cylindraceum Salvelinus alpinus Salvelinus namaycush Thymallus arcticus	Freshwater	Mortality	33^b
	Organic contaminant	40	Palace et al. (2001)	PCB Congeners	Thymallus arcticus	Freshwater	EROD activity Osmolality Thyroid status Vitamin status	60^b
Bird	Oil‐related contaminant	NA	NA	NA	NA	NA	NA	NA
	Inorganic contaminant	2	Braune et al. (2012)	Methylmercury	Sterna paradisaea Uria lomvia	Marine	Survival to 90% of development	80^b
	Organic contaminant	NA	NA	NA	NA	NA	NA	NA
Crustacean	Oil‐related contaminant	15a	Gardiner et al. (2013)	Crude oil	Calanus glacialis	Marine	Mortality	80
	Inorganic contaminant	39	Overjordet et al. (2014)	Mercury	Calanus glacialis Calanus finmarchicus	Marine	Gene expression (GST mRNA) Mortality	73
	Organic contaminant	45b	Riebell and Percy (1989)	Phenol	Mysis oculata	Marine	Loss of equilibrium Lying on dorsal side Mortality No movement without prodding	12^b
Algae	Oil‐related contaminant	4	Camus et al. (2015)	Produced water	Porosira glacialis	Marine	Growth inhibition	67
	Inorganic contaminant	NA	NA	NA	NA	NA	NA	NA
	Organic contaminant	NA	NA	NA	NA	NA	NA	NA
Mollusk	Oil‐related contaminant	4	Camus et al. (2015)	Produced water	Mytilus edulis	Marine	Development	67
	Inorganic contaminant	48	Thyrring et al. (2015)	Lead	Mytilus edulis	Marine	Fatty acid composition Mortality	30^b
	Organic contaminant	NA	NA	NA	NA	NA	NA	NA
Mammal	Oil‐related contaminant	8	Engelhardt (1981)	Crude oil	Ursus maritimus	Marine	Electrolyte changes Enzyme and metabolite changes Erythrocytic changes Hormonal changes Leukocytic changes Nitrogenous compound changes	6^b
	Inorganic contaminant	14a and 14b	Frouin et al. (2012)	Mercury	Delphinapterus leucas	Marine	Cell viability Intracellular thiol levels Lymphocyte proliferation Metallothionein induction	23^b
	Organic contaminant	6a and 6b	Desforges et al. (2017)	Blubber‐derived contaminant cocktail	Cetacean spp. Orcinus orca Pagophilus groenlandicus Seal spp. Ursus maritimus Cystophore cristata	Marine	Natural killer cell activity Cell viability Lymphocyte proliferation Phagocytosis	60
Microbe	Oil‐related contaminant	44	Petersen et al. (2008)	Pyrene	Unidentified microbial consortium	Marine	Algal ¹⁴C incorporation Chlorophyll‐a content Phosphate flux Potential nitrification Potential oxygen consumption Silicate flux	60
	Inorganic contaminant	NA	NA	NA	NA	NA	NA	NA
	Organic contaminant	NA	NA	NA	NA	NA	NA	NA
Other invertebrate	Oil‐related contaminant	9	Engelhardt (1983)	Crude oil	Strongylocentrotus droebachiensis	Marine	Curling of tube feet Loss of attachment ability Loss of cover Mortality Retraction of tube feed Spine droop Spine rigidity Stimulus response	30
	Oil‐related contaminant	42a	Percy and Mullin (1975)	Crude oil	Halitholus cirratus	Marine	Mortality	30
	Inorganic contaminant	NA	NA	NA	NA	NA	NA	NA
	Organic contaminant	NA	NA	NA	NA	NA	NA	NA

Open in a new tab

^{^a}

Studies presented in bold are considered reliable for use in ecological risk assessment (overall reliability score ≥50%). Study number corresponds to the alphabetically ordered list of assessed studies in Supporting Information, Table S1.

^{^b}

The only study for this category.

PCB = polychlorinated biphenyl; EROD = 7‐ethoxyresorufin O‐deethylase; NA = not available; GST = glutathione S‐transferase; mRNA = messenger RNA.

Ecological relevance scores

Relevance scores for all experimental combinations assessed in the present study are presented by organism type in Table 6. The mean relevance score for fish tests (with the greatest mean relevance score; 5.3 ± 1.4) and mammalian tests (with the least mean relevance score; 2.0 ± 0) were both significantly different from all other organism groups and each other (p < 0.05).

Table 6.

Count of relevance scores for all reported responses assessed in the present study (i.e., test substance/test species/endpoint combinations, n = 596), except birds and microbes, by organism type

Relevance score	Crustacean (n = 276)	Fish (n = 173)	Alga (n = 17)	Mammal (n = 35)	Mollusk (n = 23)	Other invertebrate (n = 15)
0	0	0	0	0	0	0
1	10	2	0	0	0	0
2	2	18	0	35	3	0
3	60	0	0	0	9	11
4	33	17	15	0	1	0
5	20	4	0	0	1	0
6	151	132	2	0	9	4

Open in a new tab

Crustacean tests achieved the second highest mean relevance score (4.8 ± 1.5), followed by mollusks (4.2 ± 1.6), algae (4.2 ± 0.7), and other invertebrates (3.8 ± 1.4). No statistically significant differences were found between any organism groups other than those described. When assessed by test substance type, differences between mean relevance scores for inorganic contaminants (5.5 ± 1.2), oil‐related contaminants (4.7 ± 1.6), and organic contaminants (2.3 ± 1.0) were all statistically significantly different from one another (Supporting Information, Figure S16). Figure 6 provides the relationship between overall and relevance scores for the data set as a whole (n = 596 experimental combinations). The points in the lower left quadrant indicate both low overall and relevance scores (20% of experimental combinations). The upper left quadrant contains studies that used highly relevant endpoints but did not score well overall (59% of experimental combinations). The lower right quadrant indicates high overall reliability scores but with minimal relevance for use in risk assessment (7% of experimental combinations). The upper right quadrant includes studies that are considered highly relevant, achieved a score of 1 in all the critical categories outlined, and were generally well designed, conducted, and reported (14%). This relationship is broken down further by organism group in Supporting Information, Figure S17.

Strength of methods and relevance scores for all reported responses in the present study (n = 596). Size of circle corresponds to number of studies. Numerical values in each quadrant refer to percentage of data points with a relevance score between 0 and 3 (inclusive) and an overall reliability score of <50% (bottom left quadrant) or >50% (bottom right quadrant) and a relevance score between 4 and 6 (inclusive) and an overall reliability score of <50% (top left quadrant) or >50% (top right quadrant).

DISCUSSION

The present review supports the data reliability concerns that have been raised by the ecotoxicological community in recent years (e.g., Ågerstrand et al., ²⁰¹³; Breitholtz et al., ²⁰⁰⁶; Durda & Preziosi, ²⁰⁰⁰; Harris et al., ²⁰¹⁴) pertaining to Arctic species toxicity testing. A specific sensitivity comparison between Arctic and temperate species was not a focus of the present review; however, it is a question that needs to be answered for environmental risk assessors conducting ERAs. Overall, relatively few Arctic species toxicity tests assessed in the present study achieved an overall reliability score deemed minimally acceptable for use in risk assessment (43 of 259 unique toxicity tests; 17%) without significant uncertainty; therefore, there are limited data available to use in a sensitivity comparison. Of those that achieved acceptable scores, there are many important groups of test species, test substances, and endpoints that are highly underrepresented relative to their potential for exposure in the Arctic. The results of the present study highlight gaps in collected data, clearly demonstrate the need for more methodologically robust studies, and provide a set of collected data that could be used to address these relative sensitivity questions in the future. In addition, the present study provides an opportunity to further refine the scoring rubric.

General recommendations for improving the conduct of future toxicity studies

Several methodological deficiencies across studies came to light during the scoring process. First, appropriate controls should always be included with every test, regardless of prior evidence obtained that supports the decision to exclude them. A common approach in the studies assessed in the present study was to indicate that a solvent was used to dissolve the test substance but that no solvent control was included because of evidence in previous studies that such concentrations did not significantly impact the organism. Errors in solution preparation, solvent stock preparation, or increased sensitivity of a particular group of test organisms cannot be definitively ascertained without the inclusion of these controls. Ultimately, each test compound may have a unique interaction with a solvent control that needs to be accounted for in the test design (Christou et al., 2020).

Second, reporting of environmental conditions needs significant improvement to ensure that tests can be repeated and results reproduced. For example, studies often reported the target temperature or lighting of exposure chambers without reporting actual measured values. Issues can arise with growth chambers, from temperature and lighting swings and inconsistencies spatially to faulty light bulbs that stop working midway through exposure to power failures that may not be recognized if no monitoring is conducted. In some cases, monitoring may have been conducted but the results not reported in the publication itself. Although this information may have been retrievable through contacting the authors to confirm whether or not certain monitoring was done or whether measured values were within the predefined range, it is unrealistic and impractical to do this when it could easily be included in the original paper. Just as it is critical to analytically confirm exposure concentrations to have confidence in the results, the user of the data will not have confidence that test conditions were within the predefined range for the duration of exposure if target conditions were not measured as well as reported.

Third, low exposure volumes can lead to an inability to analytically confirm exposure concentrations (e.g., algae and copepod tests in Camus et al. [2015]). The approach taken in the present study was to estimate measured concentrations by using the average ratio of nominal to measured concentrations that were reported for other tests in the same study. This likely provides a more accurate estimation of the actual exposure than nominal concentrations alone; however, it does not give risk assessors full confidence in the results as when the individual unit concentrations were reported. Including larger‐volume surrogate vessels that are exposed for the same duration and under the same conditions as experimental units can be an option for these scenarios.

Finally, and perhaps most importantly, it is imperative to stress that not all of the studies assessed in the present study claimed to have been conducted with the objective of generating toxicity data for use in ERA. Understanding the biological traits of Arctic species through a variety of study designs is equally as critical to achieving the overall goal of environmental sustainability as is generating toxicity data; however, the aim of the present study was to identify studies exclusively for the latter purpose. A poor score achieved through the application of this rubric does not necessarily mean that the study was not well conducted or of high value to science; rather, it implies that the study is not fully reliable for the sole function outlined a priori in the present study. For example, Criterion 5 indicates that a minimum of three concentrations must be tested. This is the minimum number of concentrations required to plot a concentration–response curve to characterize the range of toxicity to an organism. This is useful for ERA but may not be required to produce scientifically valuable results. If a study were assigned a score of 0 for this criterion because only a single concentration was tested to assess the mode of action of a compound, the data may not carry as much weight in the risk‐assessment sphere, though the results are important to science. Understanding the mechanism of action of various contaminants on exposure to Arctic species in Arctic conditions or generating data that will assist in our knowledge of biomarkers of exposure in the field is extremely relevant. These studies contribute to the foundation of knowledge that is required to advance the field of ecotoxicology and ERA and would likely render toxicity data less useful if they were not gathered. We recommend that this type of data continue to be collected, but it would be beneficial if studies were more explicit in their objectives as to the purpose of the data collection so that data collected for the purpose of informing risk assessment could be easily identified.

Specific recommendations to advance Arctic toxicity testing

The present study has highlighted several knowledge gaps and deficiencies in reporting that should be addressed for future published work. The recommendations should be taken into consideration by anyone planning, performing, or reviewing ecotoxicological data including researchers, funding agencies, and risk assessors.

Standardization of Arctic‐specific test methods

Knowledge Gap 1: Lack of standard toxicity and recovery test methods for Arctic species.

Recommendation 1: Strive toward developing standard laboratory tests (whole organism and in vitro), and in the absence of standard tests, provide a thorough description of what was modified from standard methods.

Prior to discussing the data set as a whole, it is important to note that standard toxicity test methods designed specifically for Arctic aquatic species do not currently exist in the literature. There have been techniques introduced by researchers for modeling Arctic‐specific toxicity data that take into account environmental, geophysical, and physiological factors specific to Arctic regions that partially bridge this gap; however, they are novel in their application (see Fahd et al., ²⁰¹⁹). Many of the studies scored in the present review followed standard test methods as closely as possible but with modifications to environmental parameters, test durations, or validity criteria as necessary to accommodate the requirements of Arctic test organisms (see Camus et al., ²⁰¹⁵; Faksness et al., ²⁰¹¹; Hansen et al., ²⁰¹²; Riebell & Percy, ¹⁹⁸⁹). However, several studies reported that standard methods were followed but with “modifications,” without detailing what was modified from the original test conditions. This renders the experiment impossible to reproduce or score effectively. Researchers should at least report a brief summary of the standard method conditions instead of referencing a separate document with or without modifications.

Developing and validating standard in vivo tests for Arctic regions is necessary because data based on modeling or collected via in vitro methods need to be validated by in vivo studies to have confidence in the results. In addition, there are many questions that cannot be answered with tests that do not expose the whole organism, such as behavioral and physiological interactions within the organism. The current regulatory infrastructure relies on whole organism–based endpoints (e.g., mortality) to determine hazardous concentrations in the environment, and thus these data are enormously helpful to ERA. In short, both in vivo and in vitro tests are important to conduct, each offering a range of benefits and drawbacks that the other does not.

Because it pertains specifically to in vitro tests, emphasis should be placed on developing methods where the sensitivity, specificity, and exposure conditions of the assays are clearly understood and defined (Rehberger et al., 2018). The purpose of these tests, including how the data can be extrapolated to the ecosystem level, should be plainly expressed by researchers to facilitate understanding by environmental risk assessors regarding how the data can be used in the context of ERA.

Finally, understanding potential latent effects and/or the capacity of test organisms to recover postexposure is a critical component in extrapolating ecosystem‐level effects from lab‐based assays; however, only a quarter of tests assessed in the present study included postexposure monitoring (Beketoc & Liess, 2009; Pechenik, ²⁰⁰⁶). This may be even more important for Arctic test organisms because a variety of biotic and abiotic factors specific to this ecosystem result in significant uncertainty regarding time to effect (Chapman & Riddle, 2005; Olsen et al., ²⁰¹¹). For example, longer toxicity response times for polar invertebrates relative to temperate ones have been observed for metal exposures. It has been hypothesized that this can be partially attributed to the temperature‐driven low metabolic rate of Arctic species as well as the longer development time of polar organisms relative to their temperate and tropical counterparts. In addition, sea‐ice cover is known to prevent mixing of sea layers, which increases the probability of long exposure times for under‐ice organisms (Chapman & Riddle, 2005).