Quality Criteria for Microplastic Effect Studies in the Context of Risk Assessment: A Critical Review

Abstract

In the literature, there is widespread consensus that methods in plastic research need improvement. Current limitations in quality assurance and harmonization prevent progress in our understanding of the true effects of microplastic in the environment. Following the recent development of quality assessment methods for studies reporting concentrations in biota and water samples, we propose a method to assess the quality of microplastic effect studies. We reviewed 105 microplastic effect studies with aquatic biota, provided a systematic overview of their characteristics, developed 20 quality criteria in four main criteria categories (particle characterization, experimental design, applicability in risk assessment, and ecological relevance), propose a protocol for future effect studies with particles, and, finally, used all the information to define the weight of evidence with respect to demonstrated effect mechanisms. On average, studies scored 44.6% (range 20–77.5%) of the maximum score. No study scored positively on all criteria, reconfirming the urgent need for better quality assurance. Most urgent recommendations for improvement relate to avoiding and verifying background contamination, and to improving the environmental relevance of exposure conditions. The majority of the studies (86.7%) evaluated on particle characteristics properly, nonetheless it should be underlined that by failing to provide characteristics of the particles, an entire experiment can become irreproducible. Studies addressed environmentally realistic polymer types fairly well; however, there was a mismatch between sizes tested and those targeted when analyzing microplastic in environmental samples. In far too many instances, studies suggest and speculate mechanisms that are poorly supported by the design and reporting of data in the study. This represents a problem for decision-makers and needs to be minimized in future research. In their papers, authors frame 10 effects mechanisms as “suggested”, whereas 7 of them are framed as “demonstrated”. When accounting for the quality of the studies according to our assessment, three of these mechanisms remained. These are inhibition of food assimilation and/or decreased nutritional value of food, internal physical damage, and external physical damage. We recommend that risk assessment addresses these mechanisms with higher priority.

Introduction

In the past decade, the body of literature addressing the occurrence and impacts of plastic debris has substantially increased.¹ Particular attention has been given to microplastic particles (MP), generally defined as plastic particles 1 μm to 5 mm²⁻⁷ which have been detected at a wide range of concentrations in various aquatic systems, from remote marine to coastal zone and estuarine areas, as well as in freshwater lakes and rivers.⁸⁻¹¹ Their ubiquity in aquatic systems and their small size has resulted in concerns regarding their effects on aquatic biota for which ingestion has been observed at all levels of biological organization.¹²⁻¹⁴

Characterizing and quantifying the environmental fate and transport of MP requires insight into the influence of various environmental processes and pathways.^8,15,16 The release of MP into the environment can occur either directly, such as via primary emissions from products during their manufacture and consumer-use life cycle, or alternatively, can be generated from the degradation and fragmentation of mismanaged plastic waste, commonly referred to as secondary MP, which results in a heterogeneous mixture of particle types, shapes, and sizes released to the environment.¹⁷ It is generally agreed that secondary sources represent the dominant source of MP.¹⁸ Primary sources are estimated to contribute between 15 and 31% of all plastic in the environment.¹⁹

To assess the ecological risk associated with exposure to MP, there is a need to develop robust toxicological dose–response relationships, which can effectively relate environmentally relevant exposures with effects.²⁰ Because of the heterogeneous presence of MP in the environment of varying concentrations of shapes, sizes, and polymer composition, there is a need to better understand effect mechanisms and the key factors triggering them. For instance, effects observed following exposure to MP on an organism can either be initiated due to sorption of the particles on the external surface of the organism or due to other mechanisms of action being triggered following their ingestion.¹² Effects following exposure to MP, both external and internal, have been assessed in laboratory studies for a wide range of species.²¹⁻²⁵ The ingestion and/or adsorption of MPs has been suggested to cause adverse effects on toxicological end points at various levels of biological organization, generally observed in laboratory test systems at relatively high exposure concentrations.^{21,22,24−26} Furthermore, experimental work has suggested that effects of MPs can occur at the community level (e.g., biodiversity, species composition),^27,28 population level (e.g., abundance),²⁹ individual level (e.g., survival, reproduction, growth, feeding, emergence, embryonic development, mobility, and physiology),^21,22,25,30 or suborganismal level (e.g., inflammation, reduced lysosomal stability in the digestive gland, reduced antioxidant capacity, DNA damage, neurotoxicity, oxidative damage, gut dysbiosis and alteration of the genetic expression, the ionic exchange, and enzymatic activity).^{1,10,26,31−35} Several studies have speculated that elevated MP concentrations can cause physical damage (i.e., blockage of food passage), leading to a feeling of satiation and a reduced feeding.³⁶⁻³⁸ Some studies have attributed the effects to specific properties of the polymer composition, such as the availability of functional surface groups,^39,40 while other studies have assigned effects of MP to the leaching of chemical additives and plasticizers or other hydrophobic organic pollutants.^{28,32,41−43} A limitation identified for studies testing ecotoxicological effects, however, is a lack of consistency and standardization of test methods necessary to characterize dose–response relationships for specific end points. Particularly problematic is the need for standard methods in relation to the dosing of particulates, such as MP, an issue that can result in ambiguous results and considerable speculation regarding the proposed mechanisms of action representative of ecologically relevant exposures.^20,44 Consequently, the weight of the evidence supportive of a quantitative risk assessment for MP remains unclear. Recent reviews have discussed the evidence regarding the occurrence of MP effects and the underlying effect mechanisms.⁴⁵⁻⁴⁷ However, in their evaluations of the literature, the quality of studies was not taken into account, possibly leading to biased assessments. While these reviews underline that the quality of effect studies should improve, and call for more ecologically and environmentally relevant exposure systems in order to better assess the effect of MP on the environment, we argue that the quality of studies should be assessed first, in order to be able to discard unreliable data.

A fundamental element of assessing ecological risk is the availability of a suite of standardized test systems and analytical tools and methods, which enable the application of dose–response relationships relating environmental exposure to effect threshold concentrations that are consistent and of sufficient quality.⁴⁸⁻⁵⁰ This also applies to the relatively young field of MP risk assessment, where many studies have emphasized the need to improve the quality of data needed to inform risks assessment(s).^10,51−58 Efforts to assess the quality of data emerging from studies reporting on exposure concentrations of MPs in biota and in surface and drinking water, adopting methods similar to the existing Klimisch and CRED approaches,^48,49 have recently been developed and applied.^54,55 Whereas these systems and aspects of these systems start to be adopted and recommended in the literature,⁵⁹⁻⁶⁶ currently, a similar evaluation method for assessing the quality of MP effect studies is lacking.

The aim of the present study is to critically review the literature reporting on ecotoxicological effects of MP on aquatic biota, emphasizing quality assurance aspects of studies, and assessing the weight of the evidence (WOE) the studies provide with respect to the effect mechanisms that they report. This is done by first developing a quantitative evaluation method for effect studies and methods employed to assess effects of MP on aquatic biota. The evaluation method is subsequently applied retrospectively to the reviewed studies. Average scores per evaluation criterion are used to prioritize and provide guidance with respect to the analytical and test system protocol that would benefit most from refinement. Based on our analysis, a guidance protocol for testing ecotoxicological effects of MP for aquatic species is provided. Demonstrated and suggested effect mechanisms reported in the reviewed papers are summarized and discussed, with the results of the quality evaluation applied as a method to assess the overall weight of evidence regarding probable ecologically relevant effects of MP.

Methods

Literature Search

Literature was retrieved from the database from the systematic review underlying the SAPEA report.¹ In addition, an extensive literature search accessing the Natural Science Collection database available at ProQuest was performed for ecotoxicological effect studies with MP until November 2019. The following search strings were used: (effect OR impact OR end point OR toxicity) AND (growth OR feeding OR consumption OR survival OR mortality OR behavior OR behavior OR stress OR response(s) OR activity OR reproduction OR inhibition) AND (microplastic(s) OR microbead OR polyethylene (PE) OR polystyrene (PS) OR polyamide (PA) OR polypropylene (PP) OR polyvinyl chloride (PVC)) AND (aquatic OR freshwater OR marine OR estuarine) NOT (chemicals OR additives). Studies were only included when at least one type of MP tested had a diameter between 1 μm and 5 mm. To enable interpretation of particle effects, studies explicitly aiming to study effects of plastic-associated chemicals, or aiming to solely study accumulation, ingestion, and/or egestion of MP were excluded from the analysis.

Assessment of General Study Characteristics

A total of 10 characteristics were extracted from each paper and summarized (see Table S1 in the Supporting Information): size, shape, polymer type, ecosystem (fresh, marine, estuarine), taxonomy categories (class, species), exposure duration, end points studied, end points affected, and effect threshold when reported (as either LCx, ECx, LOEC, or NOEC). When a size range was used, the upper and lower size ranges are noted, however, if an average size was provided together with the range, the average is also recorded. In instances when the average was not given, it is assumed that the particles are uniformly distributed between the upper and lower size limit and that the average can be estimated accordingly. For shapes, the terms “beads” and “spheres” are assumed to be the same and are combined in a single category. As the definition of “irregular” is ambiguous and could include any nonregular shape, it is included as a separate category.

For the analysis of the taxonomic groups we followed De Sá et al. (2018), where classes polychaeta and clitellata are combined in the category “annelida”; classes bivalvia and gastropoda are combined in the category “mollusca”; classes anthozoa and hydrozoa are combined in the category “cnidaria”; classes branchiopoda, hexanauplia, and monogononta are combined in the category “small crustacea”; class malacostraca is renamed “large crustacea”; and class actinopterygii is renamed “fish”.¹² Additionally, classes gammaproteobacteria and cyanophyceae are combined in the category “bacteria”; classes bacillariophyceae, chlorophyceae, trebouxiophyceae, dinophyceae, and mediophyceae are combined in the category “microalgae”; and class liliopsida is renamed “macrophyte”.

Quantitative Quality Assessment

All of the 105 reviewed studies are evaluated based on 20 quality assurance/quality control (QA/QC) criteria in the following categories: particle characterization, experimental design, applicability for risk assessment, and ecological relevance. These categories are consistent with the principles of sound ecotoxicology proposed by Harris et al. (2014), which represent fundamental elements for ensuring quality and reproducibility and are thus critical when designing, applying, and reporting ecotoxicological effect studies for MP.⁶⁷ A summary of the 20 QA/QC criteria is shown in Table 1 and a detailed motivation for each criterion is provided in the Supporting Information (see Methods Continued). Building on the methods developed by both Hermsen et al. (2018) and Koelmans et al., (2019), each criterion is assigned a score of either 2 (adequate), 1 (adequate with restrictions), or 0 (inadequate) points (see Table S2).^54,55 All studies collated as part of this literature review are independently assessed by three of the authors, with scores subsequently tabulated and discussed to reach consensus, sometimes leading to adjustments of the original formulation of a criterion to decrease potential ambiguities. The scores per individual study are provided in the Table S3.

Table 1. Summary of Specific Guidance Proposed Towards the Adoption of Standardized Protocol for Testing the Effects of MP in Aquatic Test Systems for the Purposes of Strengthening the Quality of Data Generated with Respect to Quality Assurance/Quality Control (QA/QC) Criteria^a.

guidance to increase the technical quality of effect tests (1–12)
Particle Characterization
1.	particle size	Size is a crucial factor explaining effects of MP and thus should be reported. If a range of sizes is used; a full (i.e., ≥ 10 bins) size distribution is measured and reported. If a single size is used, that size is measured with an indication of measurement error and reported.
2.	particle shape	Shape is a crucial factor explaining effects of MP and thus should be measured and reported. Shapes are measured with high resolution picture and reported.
3.	polymer type	Polymer type can be a factor explaining effects of MP and thus should be reported. Polymer identity confirmed with, e.g., FTIR, Raman spectroscopy, or similar methods.
4.	source of MP	Specification on where MP stock or solution is bought and/or how it is self-made maximizes reproducibility and thus should be reported. The origin and/or production of MP in own laboratory is reported in detail.
5.	data reporting	Unambiguous units are required to ensure reproducibility of the experiment and to make it possible to compare data across experiments. MP concentrations are reported as mass as well as number concentration.
Experimental Design
6.	chemical purity	In order to test particle toxicity, the toxicity of other chemicals in solution or mixture should be ruled out. This includes additives present in MPs, chemicals associated with food particles and surfactants (e.g., Tween). Chemical effects other than from the polymer or solution/mixtures are ruled out. MPs are cleaned with organic solvent.
7.	laboratory preparation	MP contamination arising from the laboratory (air, water and materials) should be minimized.
		•All materials used (equipment, tools, work surfaces and clothing) should be free of MP. All materials used are thoroughly washed with high quality water (e.g., Milli-Q water).
		•Measures are taken to prevent MP contamination from air.
		•Cotton lab coats were used to avoid microfiber contamination.
8.	verification of background contamination	MP contamination of the exposure systems in the laboratory should be assessed. Level of contamination evaluated and quantified, e.g. with FTIR, Raman or similar method.
9.	verification of exposure	Not only the nominal concentration should be mentioned. The exposure concentration should be measured. Measurement of exposure concentration and evidence that at least 80% of the nominal concentration throughout the test is maintained.
10.	homogeneity of exposure	Verification of homogeneity is crucial for the MP characterization and the assessment of bioavailability.
		•Water as medium: Picture or measurement of MP in water that demonstrated well mixed or dispersion in solution
		•Sediment as medium: Description of method used to obtain homogeneous exposure
11.	exposure assessment	Exposure of the organism to MP should be verified by measurement. Exposure of the organism to MP is measured quantitatively with e.g. FTIR or Raman. In case MPs are ingested additionally a digestion step is included (see criteria 9 and 10 Hermsen, Mintenig, Besseling, & Koelmans54).
12.	replication	For statistical rigor in detecting effect thresholds (e.g., EC50 or EC10), sufficient replicates should be tested. Three or more replicates.

guidance to increase the applicability in ecological risk assessment(13–20)
Applicable for Risk Assessment
13.	end points	End points should be considered that inform ecologically relevant population level risk assessment and clearly reported. End points taken at the community (e.g., bacteria and algae) or individual level (e.g., survival, mortality, growth, development, reproduction).
14.	presence of natural (food) particles	The exposure conditions should be environmentally relevant. Natural particles (at least food) are added to avoid force feeding of MP. Criterion not applicable to algae or bacteria and hence these studies receive 2 points.
15.	reporting of effect thresholds	To enable PEC/PNEC types of comparisons, the effect threshold should be assessed with error of uncertainty using dose- response relationships. Effect thresholds are reported as L(E)Cx with error or uncertainty intervals.
15.	quality of dose–response relationship	For statistical rigor in detecting effect thresholds (e.g., EC50, EC10), sufficient doses should be tested, including a treatment control, covering the full shape of the effect curve and emphasizing the slope for parameter estimation. Multiple doses, at least 6, including a treatment control.
Ecological Relevance
17.	concentration range tested	Concentrations should be motivated (with a reference in the appropriate unit) from measured environmental concentrations (MEC). More than 1 environmentally relevant concentration should be used within the range tested.
18.	aging and biofouling	Aging and biofouling is what occurs in the environment and could affect the uptake of MP; therefore, it is crucial to consider this for an ecological relevant experiment. MP particles should have undergone process to make them more environmentally realistic, accounting for biofouling. Additionally, pictures of altered particles are provided.
19.	diversity of MP tested	In the environment, MPs have a wide variety of shapes and sizes. This needs to be taken into account for environmentally relevant effect assessment. A wide range of sizes (order of magnitude), shapes and densities is used, thereby approaching the diversity of environmental microplastic.
20.	exposure time	It is crucial to use appropriate exposure times to allow for the detection of adverse effects.
		•Bacteria and phytoplankton: 1 week or longer
		•Zooplankton: 21 days or longer
		•Benthic invertebrates: 28 days or longer
		•Fish: 3 months or longer
		•Macrophytes: 28 days or longer

^aA detailed motivation for each criterion is provided as Supporting Information (see Methods Continued).

Consistent with the approach adopted in previous method evaluation papers,^54,55 we emphasize that the scores assigned for each study should not be perceived as a judgment indicative of the relative value of a study, i.e., a paper scoring low on a certain criterion could still provide valuable and reliable information regarding other potential insights. Problem formulation is therefore an important element to understand, in that depending on the purpose of an effect study the results may or may not help to inform the decision-making process with respect to assessing risk. A WOE may be assembled, for instance, regarding an effect mechanism, but the mechanism may not necessarily be ecologically relevant (see Supporting Information Criterion 13 End Points, p 11, Methods Continued). The primary objective of the evaluation criteria developed and applied in this study is directed at providing insight regarding those aspects of MP ecotoxicological effect studies that could be improved in future studies in order to better inform the application of a quantitative environmental risk assessment. The evaluation criteria, however, also provide the opportunity to assess the current WOE of effect mechanisms.

Analysis of Perceived versus Demonstrated Mechanisms Explaining Adverse Effects

Authors’ conclusions with respect to observed adverse effects and the mechanisms explaining them are summarized in the Table S3. In instances where the discussion and conclusions included ambiguous terms, such as, “may”, “could”, “can”, “would”, “postulate”, “suggest”, “might”, “potentially”, “most likely”, “imply” the reported mechanisms are classified under the category “suggested”. If the discussion and/or conclusion used more definitive terminology, such as, “demonstrate”, “observe”, “indicate”, “induce”, “provide”, and “evidence”, the reported mechanisms are classified under the category “demonstrated”. When a combination of both ambiguous and definitive terminology are used in the same sentence to describe an effect mechanism, the mechanism is considered as “suggested”. Terms that imply a mechanism to be either “demonstrated” or “speculated” are reported in italic, whereas keywords indicating the mechanism category are reported in bold. Finally, in addition to classifying effect mechanisms as either “‘suggested” or “demonstrated”, specific categories based on the modes of actions proposed by authors are recorded and numbered accordingly.

Results and Discussion

Study Characteristics

Characteristics of the Tested Microplastics: Size, Shape, Polymer Type

Size

A total of 178 different MP sizes have been tested in the 105 reviewed papers. The cumulative distribution illustrates that about 75% of studies tested the effects for MP < 100 μm, or “small MPs”^55,66 (Figure 1), with approximately 30% of MP having sizes <10 μm. Of the 178 sizes tested, 58.4% corresponded to a size range, while 41.6% consisted of one size only. Moreover, 16.3% of the tested MP included a size range greater than 1 order of magnitude.

Figure 1.

Cumulative frequency distributions for MP particle sizes used in effect tests for aquatic biota. The majority of studies tested a size range, which implies that separate cumulative distributions can be plotted for the minimum (Min), the maximum (Max) and the average size tested across studies.

Species-specific traits, such as size selective ingestion of MP have been demonstrated for aquatic organisms.^25,38,68,69 Size selectivity can potentially help in understanding effect mechanisms that influence the toxicological response of an organism. Mechanistic insight, however, can only be demonstrated when an appropriate range of particles sizes is used. Therefore, when evaluating the effects of MP of only one size, the most detrimental sizes for a specific species may not be included in the analysis, resulting in an underestimation of actual effects across the MP size range. Furthermore, it can be assumed that effects of MP of a certain size will differ in the presence of other sizes of MP, since there can be complex particle–particle interactions that may influence exposure as well as complex organism–particle interactions that can be difficult to account for when limiting testing to one size or narrow size range distributions. The observation that effects testing of MP to date is dominated by particles <100 μm (Figure 1) implies that comparisons between MP sizes used in effect studies and sizes of MP found in the environment are difficult to be made, particularly since the detection of MP < 100 μm represents an ongoing analytical challenge.⁵⁵ Nevertheless, we recommend the use of MP size distributions that are appropriate for the species being tested, which can potentially add greater insight between adverse effects and organism–particle interactions.

Shape

The shapes of MP reported in the 105 studies are dominated by spheres/beads, followed by irregular MP, fragments, and fibers (Figure 2A). We assume that most of the studies reporting the use of irregular MP have tested either fragments, films, foams and sheets, or a combination thereof. Consequently, characterizing MP into distinct categories includes a subjective, qualitative, element that is difficult to enable differentiation, but which could result in greater refinement of shapes divided into more categories that would provide opportunities for better mechanistic understanding.

Figure 2.

Number of studies reporting a particular shape (A) or polymer type (B) for the microplastics used in the exposure tests (from a total of 124 records for shapes and 145 records for polymer types). PS = polystyrene, PE = polyethylene, PVC = polyvinyl chloride, PP = polypropylene, PET = terephthalate, PA = polyamide, N/A = not analyzed, PLA = polylactic acid, PMMA = poly(methyl methacrylate), PC = polycarbonate, PE-Acrylate = polyethylene-Acrylate, EVA = ethylene-vinyl acetate, PHB = polyhydroxybutyrate, ABS = acrylonitrile butadiene styrene, SAN = styrene acrylonitrile resin, and POMH = polyoxymethylene-homopolymer.

When comparing the shapes used in different effect studies with those shapes commonly observed in environmental samples, there is considerable inconsistency. While 58.1% of effect studies have tested MP spheres/beads, this category only represents 6.5% of the MP detected in water and sediment samples.¹⁷ In contrast, only 8.1% of the tested MP in effects studies were fibers, although they are the most abundant shape category detected in water and sediment, typically representing about half of MP detected.¹⁷ Therefore, the use of fibers in effects studies represents a significant opportunity for advancing quantitative data for the purposes of assessing environmental risks.

An important factor to consider in future studies is how the shape of MP might influence their ingestion and egestion by aquatic organisms,^40,70,71 which can potentially influence their relative toxicity. Thus, the use of shapes representative of those detected in the environment has the potential to benefit both the ecological relevance and mechanistic understanding of risks associated with MP commonly encountered in the environment.

Polymer Type

The most common polymer types used in the 105 effect studies reviewed were PS and PE. Together they represent 62.3% of the MP types tested (Figure 2B). The use of these two polymers is relatively consistent with the polymer types typically observed in the environment, whereby the three most commonly detected polymers in surface waters are PS, PE, and PP.⁵⁵ In effect studies, however, the inclusion of PP is limited to only 5.5% of MPs tested. Given that the polymer type can influence the fate of MP in both the test system and ecosystem, depending on its density, surface chemistry, degree of crystallinity, and presence of chemical additives and plasticizers, it is important to include as much detail as possible with respect to the polymer composition.^16,20 Consistent with the need to advance the effects testing and mechanistic understanding of MP with respect to size and shape, as discussed above, there is also a need to strengthen understanding of the influence that the polymer composition may represent toward an observed adverse effect on various species. Insight regarding the relationships between size, shape and polymer composition is important for advancing environmental risk assessment and helping to inform the decision-making process.

Exposed Organisms, Exposure Duration, End Points Studied, And Effect Thresholds Reported

The organisms tested in the 105 studies evaluated consist of 52.4% marine, 42.9% freshwater, and 4.8% estuarine species. The most abundant organisms studied are small crustaceans (which belong to the zooplankton category), followed by mollusks and fish (Figure 3A). The most common exposure durations used were; 24 h, 96 h, 240 h (10 days), 336 h (14 days), 504 h (21 days), and 672 h (28 days) (see Figure S2). The exposure durations generally correspond to the recommended exposure durations of standard ecotoxicity test guidelines for chemicals, implying that exposure durations are also closely linked to standard effect end points, such as mortality, growth, and reproduction. However, there is literature indicating that effects of MP can be time dependent^27,72,73 and standard test protocol guidelines applicable for chemicals may not be applicable for the effect testing of MPs. Nevertheless, chronic effects testing of MP adopting longer study durations does not appear to be well represented, with only 18% of studies using an exposure time >28 d, and <2% (i.e., 2 papers) with exposure times above three months. Consequently, it is recommended that future effects testing include greater emphasis on assessing longer term effects.

Figure 3.

Number of studies evaluating the effects of MP on organisms of a certain taxonomic group (A) and on a particular end point (B) (from a total of 134 records for organisms and 252 records for end points).

Effects of MP on growth are observed to be the most often studied (25.4%), followed by suborganismal end points (21.4%), survival (14.7%), feeding (11.5%), and reproduction (9.9%) (Figure 3B). Population-level end points correspond to only <4% of the total end points studied. From the 105 papers, only about 10% reported effect thresholds (as either LC_x, EC_x, LOEC, or NOEC). Of all the studies providing effect thresholds, 33.3% report them as number concentration (i.e., particles/L), 50% as weight concentration (i.e., mg/L), and 16.7% in both units. In order to assess the environmental risks of MP, effect thresholds are fundamental, preferably in both units, which will also further enable comparisons between studies for use in developing quantitative WOE with respect to effects and risks.

Quality Assessment

The results of the scoring based on the quantitative quality assessment proposed in this study imply that substantial improvements can be made in how MP effect studies are designed and conducted (Figures 4 and 5). As previously stated, the scores obtained should not be interpreted as an absolute value judgment, but as a guide for identifying and prioritizing study-design components that would benefit most in improvement for the purposes of assessing environmental risks. Consequently, we suggest that those studies with relatively high scores represent the most reliable and useful in the context of risk assessment (Figure 5). Individual studies, however, often had other objectives, which were not necessarily consistent with information needed to support an assessment of risk. It is important, therefore, to assess each of the specific criteria and to compare them with other studies rather than simply evaluating the studies based on how they rank on their total score. The first subset of criteria (criteria 1–12) enable the evaluation of the general technical quality of an effect test study. Here, the average score across all studies is 11.3 (range 5–18), of a maximum possible score of 24. In this first subset there are no studies for which positive scores on all quality criteria is assigned. The second set of criteria (criteria 13–20) relates to the relevance of the papers for their use in environmental risk assessment. For these criteria, the average score across all studies evaluated is 6.6 (range 0–14) of a maximum potential score of 16. Again, no studies had positive scores for each of the ecological relevance quality criteria defined. Finally, the total scores combine both the technical quality and ecological relevance evaluation criteria, whereby the total score can be used as part of a quantitative WOE approach in the context of risk assessment. The average total score is 17.8 (range 8–31), from a maximum possible score of 40, indicating that results from effect studies assessing MP are often not fully reliable and/or reproducible. All studies included in this review were assigned a criterion value of 0 in at least one criterion, implying that important QA/QC criteria are consistently poorly addressed in the design and reporting of MP effect studies. With respect to the general technical quality of the effect studies evaluated, 34.8% of the criteria in studies are assigned a value of 0, whereas 50.1% of studies receive the same poor quality score with respect to their ecological relevance. Average scores per criterion ranged from 0.06 to 1.79 (Figure 4). Those criteria that are typically evaluated high across all studies include the reporting of the source of the MP, the use of replicates, reporting on ecologically relevant end points and the inclusion of food particles within the test study. A more detailed evaluation of each category is provided below.

Figure 4.

QA/QC quantitative system scores from n = 105 studies. Average scores per criterion with categories “particle characterization”, “experimental design”, “applicable for RA”, and “ecological relevance”. Each study is assigned a criterion value of either 2 (adequate), 1 (adequate with restrictions), or 0 (inadequate) points, for each of the 20 criteria.

Figure 5.

QA/QC quantitative system scores from n = 105 studies. Scores per study with categories “particle characterization”, “experimental design”, “applicable for RA”, and “ecological relevance”. *Studies with involvement of 1 or more of the authors of the present paper. Detailed scores and full references are provided in Table S3 and the SI reference list, respectively.

Particle Characterization

The category with the highest average score is “particle characterization” (Figure 4). Overall, the majority of studies evaluated is observed to provide satisfactory reporting on particle characteristics (scores >1). Only a limited number of studies (13.3%) fails to report on either one of these specifics. Improvements, however, are suggested, such as related to efforts toward the confirmation of size, shape, and polymer type, as opposed to simply relying on information from the manufacturer. Nonetheless, by failing to provide characteristics of the particles, an entire experiment can become irreproducible. Lastly, it should be noted that approximately 60.0% of studies either do not report a concentration or limit reporting to a mass or number concentration, which further complicates comparison across studies. It is thus suggested that with relatively limited resource toward addressing the shortcomings identified, substantial improvements can be realized within this quality criteria category.

Experimental Design

As a general observation, the majority of studies scored poorly within the category of experimental design (Figure 4). Concern is particularly apparent with respect to the quality evaluation criteria of “laboratory preparation” and “verification of background contamination”, with average scores of 0.18 and 0.06, respectively. While MP are often said to be ubiquitous in the environment, including indoor (laboratory) air,¹ only 3.8% of the reviewed studies thoroughly report how they minimized potential contamination arising from air, water, and all materials used during the experiment. Additionally, only 4.8% of the reviewed papers verified the background contamination (visually).

Only a few, 6.7%, of the evaluated studies included a protocol specifically used to preclean MP with an organic solvent. Additionally, 20% of studies took measures to ensure chemical purity. For instance, Karami et al. (2017) and Romano et al. (2018) measured certain chemical contaminants associated with the MP,^74,75 however, this still does not exclude chemical effects from experimental results. Some studies include a solvent control, but do not account for chemical contaminants that might be present in the MP themselves.⁷⁶ Importantly, the majority of studies (73.3%) do not mention the potential for chemical contaminants influencing observed adverse effects, making it difficult to disentangle particle toxicity from a potential chemical toxicity.

The criteria “verification of exposure” and “homogeneity of exposure” also are observed to score low, with average scores of 0.45 and 0.68 (n = 105), respectively. These criteria are critical for enabling the reproducibility of study results, which further increase the uncertainty associated with reported effect thresholds. Finally, the criterion “exposure assessment” (average value of 0.84) is generally unsatisfactory in the studies evaluated. While most studies (78.1%) include a description verifying that MP have been ingested by test organisms, verification is often (72.4%) demonstrated in either a separate experiment, qualitatively, visually, or without a digestion step.

While it is acknowledged that the resources needed to address the shortcomings identified with the criteria falling under the category of “experimental design” are likely to be high, failing to address the various criterion results in studies with greater uncertainties and which thus fail to add value to broader scientific understanding as well as for strengthening opportunities to assess environmental risk. It is therefore prudent to carefully consider experimental design in future effect studies, with the development and application of standard test protocols applicable to MP identified as an urgent need to better guide researchers.

Applicability to Risk Assessment

An important implication of data reported from ecotoxicity effects studies is their role in assessing environmental risks. Consequently, suggestions for improvement made under this category are perceived to have implications for the regulatory decision-making process. Results from the studies evaluated under the criteria related to applicability for risk assessment imply the need for improvements to “reporting of effect thresholds” and “quality of dose-response relationship”, where average scores of 0.25 and 0.48 were assessed, respectively. As mentioned above, a limited number of studies (10.5%) is observed to explicitly report on effect thresholds with an indication of error. Moreover, only 30.5% of the 105 studies include a sufficient number of concentration doses to ensure statistical rigor in detecting these effect thresholds. The majority (86.7%) of reported end points for MP effects, however, is informative to the risk assessment process, with 84.8% including a source of food to avoid the artifact of force-feeding MP to test organisms.

Ecological Relevance

Apart from the criterion “exposure time’, which shows an average score of 1.11 and was thus evaluated as satisfactory among the 105 studies, all other criteria in this category score low. The criterion “diversity of MP”, with an average of 0.30, is of particular concern. Only 33% of the studies included at least one environmentally realistic concentration, raising concerns regarding the relationship between laboratory-based observations of adverse effects and ecological risks. Most studies (71.4%) assessed the effects of MP using a single MP type or MP with a limited range of characteristics. Only one study used a mixture in their experiment representative of environmental exposure.⁷⁷ Only two studies included the influence of biofouling when assessing the effects of MP, subsequently characterizing the microbiology of the biofilm.^77,78

WOE for Mechanisms Explaining Adverse Effects of Microplastic on Aquatic Biota

Currently, the knowledge on effect mechanisms for MP is limited and there is a need to increase mechanistic understanding of toxicological modes-of-action.⁷⁹⁻⁸¹ Criterion no. 11 “exposure assessment of organism” aims at improving the strategic design of effect testing that might enable results to differentiate between intrinsic physicochemical properties of the MP themselves and how those interact with species-specific biological and physiological traits to influence an observed adverse effect (see Supporting Information, methods continued). Acknowledging that MP represent a complex mixture of particles (shape, size, and type), incorporating strategies that enable effect-assessment to move from a “substance-based” approach to a “mechanism-based” approach may add considerable value in assessing environmental risk, not just for MP but for any other particle-stressor organisms may encounter.^79,80 Knowledge on effect mechanisms will enhance the strategic application of species sensitivity distributions for distinct categories of effects. Finally, advancing scientific understanding of particle effect mechanisms, such as those associated with exposure to MP, will aid in the development of effect models.⁸²

Given the importance of advancing the scientific weight-of-evidence with respect to the effect mechanisms following exposure to MP, each of the 105 studies is reviewed with respect to the mechanisms that authors used to explain the adverse effects they observed. The analysis is based on four considerations. First, we verified whether authors refer to the mechanisms they described using terms such as “suggested” versus “demonstrated” (see Table S3). If authors themselves described a mechanism as “demonstrated”, the WOE is perceived to be stronger. Second, the frequency of reporting certain mechanisms was assessed (Table 2). The more often a mechanism is reported in the literature, the stronger the perceived WOE can be considered to be, in that consistency between studies in relation to observed effect mechanisms is assumed. Third, the relative strength of the WOE supportive of an effect mechanism is further scrutinized based on the criteria nos. 6 “chemical purity”, 14 “addition of food”, and most importantly 11 “exposure assessment of organism”. While all 20 criteria are crucial in order to ensure quality and reproducibility of data from effect studies, the latter three criteria are specifically important in order to successfully assess the mechanisms behind adverse effects. Fourth and finally, the scores from the QA/QC assessment are used to assess the relative credibility of effect mechanisms reported.

Table 2. Tiered Weight of Evidence (WOE) Approach for Effect Mechanisms Reported in 105 Studies, By Number of Studies That (a) Frame a Mechanism as “Suggested”, (b) Frame a Mechanism as “Demonstrated”, (c) Fulfil the Three Quality Assurance Criteria (Score >0) Considered Most Relevant to Identify Effect Mechanisms (Nos. 6, 11, 14), and (d) Average Score According to QA/QC of Studies That Fulfilled Those Three Quality Assurance Criteria.

no.	description of mechanism explaining adverse effect	suggesteda	demonstratedb	number of studies that fulfill criteria nos 6, 11, and 14c	average score of studies that fulfill criteria nos. 6, 11, and 14 QA/QCd
1	inhibited food assimilation and/or decreased nutritional value	32	9	5	21.4
2	internal physical damage	20	7	3	21.0
3	external physical damage	8	4	2	24.0
4	oxidative stress	6	8	1	16.0
5	disturbance of essential processes that affect physiology	8	3	0
6	adjustment of energy metabolism to cope with mp	1	2	0
7	microbial imbalance	2	1	0
8	leaching additives or chemicals	14	0
9	(cellular) stress	8	0
10	effects of surface properties	2	0
	total	100	34	11

Suggested versus Demonstrated Mechanisms for Adverse Effects

From the 105 studies evaluated in this review, 10 separate effect mechanisms are identified as “suggested”, whereas 7 mechanisms are identified to be “demonstrated”, the latter including (1) inhibited food assimilation and/or decreased nutritional value, (2) internal physical damage, (3) external physical damage, (4) oxidative stress, (5) disturbance of essential processes that affect physiology, (6) adjustment of energy metabolism to cope with MP, and (7) microbial imbalance (Table 2). Three additional mechanisms are reported as speculated only: 8) leaching of additives or chemicals, 9) (cellular) stress and 10) effects of surface properties. While 100 times studies describe an effect mechanisms as “suggested”, only 34 times studies describe an effect mechanism as “demonstrated”. The most frequently suggested mechanisms are “inhibited food assimilation and/or decreased nutritional value” and “internal physical damage” with a frequency of 32 and 20 suggested occurrences, respectively. However, it is notable that only 9 and 7 studies have reported these mechanisms as demonstrated, respectively.

1. Inhibited Food Assimilation and/or Decreased Nutritional Value

Within the studies that report on “inhibited food assimilation and/or decreased nutritional value ” as demonstrated, there are five studies that meet the crucial criteria “chemical purity”, “addition of food”, and “exposure assessment of organism” and have therefore reliably concluded on the demonstrated effect explaining the adverse effect, scoring 21.4 points QA/QC on average^{21,24,39,42,83} For instance, Blarer and Burkhardt-Holm (2016) visually quantified the presence of PA fibers in the digestive tract of Gammarus fossarum and showed inhibition of food assimilation.³⁹

2. Internal Physical Damage

Of the seven studies that report on the demonstrated mechanism of “internal physical damage”, there are three studies that also comply with the aforementioned crucial criteria (nos. 6, 11, and 14).^21,84,85 The studies by Redondo-Hasselerharm et al. (2018), Qiao et al. (2019), and Von Moos et al. (2012) are assigned a score of 31, 16, and 16 in the QA/QC assessment, respectively. Wang et al. (2019), scored relatively high with 25 points. Moreover, they were able to verify the exposure of MP to organisms, and also avoided potential system-dependent artifacts by including a protocol for adding food during their experiments. However, they do not include measures to ensure chemical purity, resulting in some caution when interpreting the mechanism as “demonstrated”.⁸⁶

3. External Physical Damage

Although not one of the most often speculated (eight times), the mechanism “external damage”, is concluded to be demonstrated in four studies.^26,87−89 Among these, there are two studies that fulfilled the crucial criteria (nos. 6, 11, and 14). The one with the highest QA/QC score is Ziajahromi et al. (2017) with 30 points, who observed malformations on the carapace of Ceriodaphnia dubia. Additionally, with a score of 18, Kalčíková et al. (2017) showed that microbeads with sharp edges affected the root growth and reduced viability of root cells of Lemna minor.⁸⁷ This study qualitatively assessed the adsorption of MP onto root surface and took measures to ensure chemical purity.

4. Oxidative Stress

Oxidative stress has frequently been framed as a demonstrated mechanism for the effects observed (eight times). There is, however, only one study that complied with the three criteria crucial to reliably assess a demonstrated mechanism (i.e., nos. 6, 11, 14). Qiao et al. (2019) observed inflammation and oxidative stress in the gut of Danio rerio. Besides qualitatively assessing MP in the gut, they also took measures to ensure chemical purity, and fish were fed daily. This study, however, scored relatively low on QA/QC (16 points), rendering the results less reliable. Oxidative stress is a molecular mechanism and can be defined as an imbalance in the production of free radicals and the ability of organisms to deal with them.³⁴ As oxidative stress is also an end point, it is likely that it has often been considered as demonstrated. Moreover, oxidative stress is one of the most commonly measured biomarkers.^79,90 It is, however, not clear if oxidative stress is a response to another MP toxicity mechanism or that the MP toxicity directly works at the molecular level.^35,80 Elucidating on this aspect will aid in choosing relevant end points to use within risk assessment frameworks.⁸⁰

5. Disturbance of Essential Processes That Affect Physiology

The mechanism “disturbance of essential processes that affect physiology” is claimed to be demonstrated three times.⁹¹⁻⁹³ No studies, however, comply with the criteria to credibly ascertain the demonstrated mechanism.

6. Adjustment of Energy Metabolism to Cope With MP

While the mechanism “adjustment of energy metabolism to cope with MP” is suggested once, it is reported as “demonstrated” two times.^94,95 Seoane et al. (2019) showed that MP caused a slight decrease in the growth rate of the marine diatom Chaetoceros neogracile, but also a significant decrease in the esterase activity and the lipid reserves of MP-exposed cells. While scoring relatively well on the overall QA/QC scores (20 points), this study did not take any measures to ensure chemical purity, rendering the result less reliable. Additionally Watts et al. (2016) showed that Crabs were able to overcome minor effects on ion exchange by minor physiological regulation, however did not meet criteria nos. 6 and 14.⁹⁵

7. Microbial Imbalance

Two studies speculate that adverse effects are caused by microbial activity or the presence of bacteria on the MP.^28,43 Additionally there is one study by Jin et al. (2018) that has framed this mechanism as demonstrated. However, no measures were taken to ensure chemical purity or assess MP exposure to the organisms.⁹⁶

8. Leaching of additives or chemicals

‘In 14 studies, leaching of additive or adsorbed chemicals from MP was speculated to be an explanation for the observed effect of MP; however, this mechanism has never been framed as demonstrated. Demonstrating this mechanism can be achieved by simply washing MP with organic solvent thoroughly and repeatedly, subsequently enabling to distinguish particle from chemical toxicity of MP. Interestingly, Cole et al. (2019) only suggested that leaching of chemicals could have played a role, i.e., not claiming the mechanism to be demonstrated. However, they received maximum score of 2 on this criterion (no. 6), meaning that in our view they adequately addressed the issue actually rendering the mechanism to be demonstrated.⁹⁷

9. Cellular Stress

As “cellular stress” is a broad term, hard to specify and hence not easily measurable, it is likely that for this reason it has never been framed as a demonstrated mechanism.

10. Effects of Surface Properties

Only two studies speculate that adverse effects measured in their studies are due to the surface properties of MP.^39,40 No study, however, claims to have demonstrated an effect of surface properties.

Overall final WOE Assessment of Mechanisms Explaining Adverse Effects of MP

When comparing the demonstrated mechanisms according to studies it is apparent that “inhibited food assimilation and/or decreased nutritional value” has been demonstrated most often with relatively high overall QA/QC scores (average = 21.4). Most importantly five out of nine studies comply with the crucial criteria to reliably assess a mechanism, making it a plausible mechanism to explain adverse effects with high overall WOE.

Additionally, the mechanism “internal physical damage” has a relatively high overall WOE. Of the seven studies that managed to demonstrate this mechanism, three fulfilled the crucial criteria (nos. 6, 11, 14) with an average score of 21.0 points. While the mechanism “external physical damage” has been demonstrated less often, effects have been measured with higher reliability than for other demonstrated mechanisms. The two out of four studies that comply with the crucial criteria to reliably assess a mechanism, score an average of 24 QA/QC points, thus also making it a plausible and high WOE mechanism explaining adverse effects.

Perspective and Outlook

Research on the effects of MPs on biota in aquatic and other environmental compartments is a relatively new discipline in the environmental sciences. As a result, approaches to assess these effects vary widely across research groups, with both the nature of effects testing and analytical methods developing rapidly over time. Here, we evaluate the quality of 105 studies that report on the ecotoxicological effects of MPs for aquatic biota. The evaluation includes studies of organisms at various functional groups, such as phytoplankton, macrophytes, zooplankton, benthic invertebrates, and fish. The evaluation criteria developed as part of the evaluation can be used as guidance toward best practices to assess exposure, effects and effect threshold concentrations for MPs, and can provide a quantitative quality assessment of studies reporting adverse effects of MPs on aquatic organisms. Lastly, we summarize and discuss the characteristics of the tests that have been performed thus far (e.g., particle size ranges, concentrations, polymer types, particle shapes, species, end points, test duration) in order to detect knowledge gaps within effect studies, and use information gained from the review of the literature to assess the WOE with respect to the effect mechanisms most likely influenced by exposure to MPs.

When adopting strict quality criteria, an overall lack of reliability is observed in the studies evaluated in this review, particularly for how data from available effect studies can be used to help inform the risk assessment process. This is partly related to technical shortcomings in the experimental design, such as not ensuring chemical purity, prevention and verification of MP contamination in the laboratory, and partly to limitations in the relevance of studies, for instance when studies do not use ecologically relevant particles or testing conditions. This implies that based on the current state-of-the-science, the WOE for ecological effects is very limited and the environmental risk of MPs is difficult to assess. The lack of clear evidence for ecological effects in nature due to relatively poor-quality effects studies available for the risk assessment process is worrying, particularly given concerns raised by the public and decision-makers to provide a quantitative assessment of the risks for MPs. The purpose of the present study is therefore to provide timely guidance on best practices needed to improve and standardize effects testing protocols. This includes the need for access to standardized test methods using reference MPs that can be used between research groups in an effort to strengthen both replication and inter- and intralaboratory reproducibility. We recommend that at least one of these reference materials is an environmentally realistic mixture of particles, i.e., having a realistic range of sizes, shapes, densities, and ages. This way, organisms themselves select the fraction from the mixture that is bioavailable and relevant for them. This would mimic the situation in nature better than tests with single type materials. The adoption of standardized test methods and use of environmentally relevant reference materials would help reduce uncertainties inherent in the effects data and strengthen both environmental risk assessment and mechanistic understanding of the ecotoxicity of MP.

Based on our review of study characteristics, it appears that particle type “fibers” and polymer type “polypropylene” are understudied in effect studies. Ideally, the MP tested should be as realistic as possible, thus representing a broad range of sizes, shapes, densities, and polymer types. The ecological relevance of tests should be increased by extending exposure times, as chronic tests are rarely performed. In order for effect tests to be more informative for risk assessment, the reporting of thresholds effect concentrations should be made more accurate and explicit, preferably as either LC_x, EC_x, LOEC, or NOEC values, with the use of both mass and particle unit concentrations.

Based on the evaluation of the WOE pertaining to effect mechanisms associated with exposure to MPs, we observe that the WOE is strongest for the mechanisms related to ‘inhibition of food assimilation and/or decreased nutritional value’, “internal physical damage” and for the mechanism “external physical damage”. To increase the WOE of ecological effects and effect mechanisms we recommend that the guidance provided in this evaluation study be used to develop studies that explore the mechanistic nature of both MPs and generic particle effects on aquatic organisms more broadly.

Supporting Information Available

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acs.est.0c03057.

• Additional information as noted in the text including three tables and two figures (PDF)

Author Contributions

^§ Equal contribution.

The authors declare no competing financial interest.