Detection of microplastics in fish using computed tomography and deep learning

Abstract

Marine organisms have been observed ingesting microplastic particles, with field analyses indicating fibers and fragments as prevalent forms. Current microplastic detection methods are mainly time-consuming, susceptible to cross-contamination, and expensive. Furthermore, these techniques, being disruptive, do not allow for the exact localization of the microplastic in the sample. This study proposes a new approach using Computed Tomography (CT scan) and Artificial Intelligence for the automatic and non-destructive detection of microplastics in fishes and other species based on the combination of several factors, such as density and shape. The advantages of this methodology include accurate identification of plastic localization, a low risk of cross-contamination, rapid processing, automatic tomographic measurement, efficient data processing, cost-effectiveness, and a high cost-benefit ratio. The herein results highlight how artificial intelligence applied to conventional techniques can significantly improve precision and efficiency in microplastic research. Indeed, the semantic segmentation model clearly recognized the presence of 100 % of the plastic particles, both in their location and in their volume, accelerating the identification process and surpassing the limitations of traditional spectral analysis methodologies.

Highlights

• •New approach using Computed Tomography (CT scan) and AI for automatic MPs detection in fishes.
• •The technique allows to greatly reduce the probability of cross-contamination.
• •Accurate identification of the plastic localization within the sample.
• •Rapid sample processing, low men-hours to work on the process.
• •Low-cost methodology addressing the concern of microplastic contamination in the food chain.

1. Introduction

The issue of plastic pollution has gained increasing attention in recent decades, emerging as a substantial and expanding environmental concern. Unlike materials such as ceramics or glass, plastic waste displays heightened mobility within ecosystems. What was once perceived as a localized problem has evolved into a recognized global hazard, adversely impacting both terrestrial and marine environments. The escalating global consumption of plastic, coupled with its slow degradation rate, leads to its accumulation in coastal and marine sediments. This phenomenon extends its influence across pelagic and benthic biota in a range of oceanic regions, spanning from coastal to open ocean areas at every latitude from the poles to the equator [1]. Plastic materials manifest in various forms and sizes, categorized by their dimensions. These categories include macroplastics, measuring over 5 mm and comprising recognizable items like bottles, bags, and discarded fishing nets. Conversely, microplastics (MPs), smaller than 5 mm, are often indistinguishable within sediment, encompassing fragments, sheets, fibers, beads, foams, pellets, etc. Nanoplastics, with dimensions in the nanometer range, are invisible to the naked eye. Extracting them and determining their typology proves challenging due to technological limitations, resulting in a limited understanding of their distribution in the natural environment [2]. Furthermore, MPs have been observed in various marine organisms [3,4], representing a hazard for human health, as MPs enter the human food chain through the consumption of fish and other seafood.

The primary methodologies employed for the extraction and separation of MPs from biota samples predominantly include chemical digestion and density separation techniques, using several different chemicals [5,6]. In these techniques, the sample undergoes chemical degradation, hindering the comprehensive understanding of MPs position and distribution within tissues (on the skin surface, in the muscles, between the muscles and the skin, etc.).

Nowadays, various techniques are used for MPs detection and identification, each with its advantages and limitations. These methods are often time-consuming, requiring constant oversight by personnel throughout the process. Some techniques lack the capability to determine particle numbers or identify specific MP polymers, while others are better suited for sediment or water samples rather than biological specimens [[7], [8], [9]]. Cross-contamination, originating from various sources starting from sample collection and persisting throughout sample processing, MPs extraction, and subsequent identification, poses a significant challenge. MPs can manifest as microfibers present in the air or embedded within clothing fabrics, potentially influencing the samples during handling [10]. The complexities associated with cross-contamination often hinder accurate estimations of MPs quantities within various matrices.

To address these issues, this study investigates the use of Computed Tomography (CT) combined with Artificial Intelligence (AI). CT provides a non-invasive method to detect MPs and identify their exact location within fish samples, reducing the risk of contamination during sample processing. Additionally, the use of Artificial Intelligence (AI) represents an innovative frontier in MPs identification. The integration of advanced algorithms, such as artificial neural networks and support vector machines (SVM), not only enhances accuracy but significantly accelerates the particle classification process, overcoming the limitations of traditional techniques relying solely on tomography.

This study highlights how artificial intelligence applied to conventional techniques can significantly improve precision and efficiency in MPs research, accelerating the identification process and surpassing the limitations of traditional spectral analysis methodologies. This research proposes a novel approach for detecting MPs in fish and other species. The proposed methodology relies on the combination of Computed Tomography (CT scan) and Artificial Intelligence. The herein exploited methodology enables the automatic detection of MPs based on multifactor information (e.g. density, shapes, etc.) and their precise location within organisms, avoiding the need for physical extraction methods. Notably, this approach is non-destructive, preserving the integrity of the specimen while providing comprehensive insights into the presence and exact location of MPs within the organism.

2. Materials and methods

2.1. Sample preparation

Two specimens of codfish (Merluccius merluccius (Linnaeus, 1758)) were selected for fish sampling. To ensure the possibility of replicating the experiment and facilitate the finding of individuals of the same size and with the same characteristics, the fish samples were taken from the local supermarket (Merluzzetti puliti surgelati, ATHENA). Each specimen measured 220 mm and 250 mm in length and weighed 87.3 g and 92.5 g, respectively. For each specimen, a truncated cone-shaped sample measuring 50 × 40 mm and weighing 21.5 g was extracted, encompassing all anatomical parts (skin, muscle, bones, etc.) (Fig. 1A), so in total two samples were used for this experiment, one sample (A) has been used for training phase, the other one (B) has been used for the detecting phase.

Fig. 1.

A: Truncated cone-shaped fish sample; B: Plastics inserted in the sample; C: Fish sample in the plastic cup with agar agar gel.

To ensure the experiment's target attainment and mitigate the risk of not detecting any plastic fragments within the sample, we deliberately inserted 11 pieces of MPs made of HDPE 2 with a density range of 0.930–0.970 g/cm³. These MPs varied in shapes, ranging from cubic to parallelepiped, and exhibited diverse sizes, measuring between 1x1x1 mm to 2.5 x 4 × 1 mm (Fig. 1B). The 11 MPs inserted in samples A and B were different in size and shape for each sample. The insertion was conducted using tweezers to ensure precise placement into the samples. To maintain the sample's original shape and prevent dehydration or any potential deformation during the scanning process, a preservation technique was employed: the sample was immersed in a plastic glass container filled with a solution of agar-agar gel (Fig. 1C). The gel was prepared by combining a 10 % solution of agar-agar powder and water, heated to 90 °C for 10 min or until the complete dissolution of the powder was achieved.

One sample (A) underwent scanning and was used for training purposes in deep learning techniques. The other sample (B) underwent scanning to assess the performance of the machine learning model and its capability for the automatic detection of MPs.

2.2. Samples CT scan

The proposed approach relies on X-ray computed tomography (CT), enabling the visualization and quantification of an object's internal density distribution using X-ray radiation [[11], [12], [13]].

In this project, the Zeiss Metrotom 1500 [14] system has been used, offering a maximum measurement volume of 350 mm in the x, y, and z directions (Fig. 2).

Fig. 2.

Sample installed in the Tomograph Metrotom 1500.

This system features a fixed tube-to-detector distance of 1500 mm, with the flexibility to adjust the distance along the X-axis between the tube and the center of the rotating plate. This variability allows precise control over voxel size: closer proximity to the tube allows for smaller voxel sizes, achieving a remarkable resolution as fine as 5 μm. This ability to manipulate voxel size is critical, as it enables customization of imaging parameters to capture the smallest details of scanned objects.

Tomographic data acquisition involves two primary stages: data collection during the scanning of the object and the subsequent reconstruction of this data into a three-dimensional model. The object undergoes a 360-degree step-by-step rotation around the axis of rotation during the scanning process, capturing numerous projections from different angles. Following data acquisition, the reconstruction phase begins. The multiple two-dimensional projections acquired from different angles are processed and reconstructed into a comprehensive three-dimensional model. This detailed model offers insights into the internal structure and density distribution of the scanned object. The acquisition duration for this specific scan was 1.5 h, considering an image averaging setting of 3, which influenced the overall duration. Furthermore, the spatial resolution of the acquired data is specified at 40.31 μm, providing a comprehensive volume size of 1641 x 1242 x 1612 voxels.

2.3. Image preprocessing

Tomography generates a 16-bit 3D matrix, where each element (voxel) represents the local X-ray transparency of the material, linked to its density. However, visualizing this 3D data poses challenges due to its vast size, typically involving billions of values, and the inherent complexity of comprehending 3D information, which contrasts with the human eye's familiarity with 2D visuals.

This difficulty arises both from the fact that the amount of data to be visualized is usually very large, on the order of 10⁹ values, and from the nature of 3D information itself, to which the human eyes and brain are unaccustomed. The conventional approach involves presenting volumetric data in slices, akin to scrolling through the object in a direction perpendicular to the slice. This method aligns with human cognition's preference for 2D imagery. Yet, difficulties arise when implementing automated procedures, particularly for neural network processing of tomographic data. In fact, on the one hand, there are many strategies and software packages for image analysis (i.e., 2D maps) in the literature, and on the other hand, neural strategies often rely on a human-guided training phase. This causes visualization difficulties for the human performing the training, affecting the capabilities of the network.

Hence, a decision was made to conduct training and recognition directly on individual slices. Moreover, to enrich the dataset and enhance analysis robustness, slices have been extracted along all three axes - x, y, and z (Fig. 3).

Fig. 3.

Slices in x. y and z directions.

2.4. Deep learning-based MPs semantic segmentation

Given the wide range of plastic densities, it is important to note that density alone (i.e., image grayscale value) is not a sufficient criterion for accurately selecting material properties, as objects with similar densities may be encountered. This phase focuses on training a Deep Learning-based model to automatically select specific tomographic traces with certain characteristics, such as density and shape, that were manually identified in the training dataset. This selection, performed in before the training process, is guided by the known locations of MPs.

A Deep Learning-based model for binary semantic image segmentation has been employed to automatically detect MPs in tomographic images. This model takes an input image, such as a 2D CT image, and outputs a binary image of the same dimensions. In the output, each pixel is assigned a value of 1 if it is predicted to belong to the target object (i.e., MPs), and 0 if it does not.

The binary semantic segmentation model used for the detection of MPs in fish tissue was based on the DeepLabV3+ architecture, in its PyTorch implementation [15], a state-of-the-art Deep Learning based model designed for complex image segmentation tasks. DeepLabV3+ extends the original DeepLabV3 model by incorporating an encoder-decoder structure with Atrous Spatial Pyramid Pooling (ASPP) modules, enabling the model to efficiently capture multiscale contextual information. The decoder's ASPP modules enable the model to analyze the input at multiple scales, improving segmentation performance by considering local and global contexts [16]. This architecture can use diverse types of encoders, selectable from the convolutional neural networks available in the literature.

2.5. Training dataset

The model training process involves three steps. In the first step, a set of images is collected to serve as the ground truth, consisting of images where the exact locations of MPs are known. A total of 3072 CT frames are collected and divided into training and validation sets (sample A) and test sets (sample B) in an 8:1:1 ratio. Sample A is used for training the model, while Sample B is reserved for testing. The frames are manually segmented by an expert operator using a Python-based graphical user interface, where the operator identifies and selects the regions containing MPs. Each training frame is paired with a corresponding binary mask of uniform dimensions, where pixels are assigned a value of 1 if they belong to the manually segmented region (i.e., MPs), and 0 otherwise. As shown in Fig. 4, the region containing MPs is outlined by manually selecting its constituent pixels.

Fig. 4.

Tomographic slice with plastic inclusion on the left and its manually labeled mask on the right.

To isolate only the MPs in test sample B (i.e., test dataset), the segmentation model will rely solely on the provided information about MPs in sample A (i.e., training dataset). This ensures the exclusion of all other objects present in the sample, such as bones and tissues (Fig. 5).

Fig. 5.

Tomographic slices with (A) MPs and with (B) the fish bones and tissue.

2.6. Model training and hyperparameters optimization

After generating the training dataset, the second step is to train the model, which involves selecting various training parameters, known as hyperparameters. To enhance the training performance, a Bayesian-based hyperparameter optimization approach, which has shown effectiveness in previous studies, as described by Giulietti et al. [17,18], is employed. This approach is superior to traditional random or grid search methods for optimal hyperparameter selection. The optimization process defines the hyperparameters and specifies ranges of variability for each. Multiple training iterations are then conducted with randomly chosen sets of hyperparameters, and the Bayesian optimization process refines these values based on the results to improve the model's overall performance [19].

For the evaluation of segmentation model performance, the Jaccard index, also known as Intersection over Union (IoU), has been employed as the metric. Higher IoU values indicate superior segmentation results. The IoU has been calculated using the intersection and union of the predicted and ground truth segmentations, as defined in the following equation, where $gt\cap \mathit{pr}$ represents the intersection area between the ground truth and the predicted region, while $gt\cup \mathit{pr}$ represents the union area encompassing both the ground truth and predicted regions:

$$J\left(gt,\mathit{pr}\right)=\frac{gt\cap \mathit{pr}}{gt\cup \mathit{pr}}$$

Thus, the objective function of Bayesian optimization is given by the following equation, where I is the input image I and gt is the ground truth, representing the manually segmented mask:

$$J=J\left(I,gt,bs,lr,o,bb\right)$$

The parameter involved in the optimization are.

• •Batch Size (bs): This parameter determines the number of sub-samples of training data propagated through the network, with a variation range of 2–16.
• •Optimizer (o): The optimizer influences the algorithm employed to enhance the IoU score. It achieves this by modifying neural network attributes, such as weights and learning rate. Noteworthy optimizer algorithms include Stochastic Gradient Descent (SGD), Adam, RMSprop, Adadelta, and Adagrad [17].
• •Learning Rate (lr): This parameter governs the extent to which the model undergoes changes in response to the estimated IoU score during each optimization step. The learning rate varies within the range of 0.00001–0.1.
• •Backbone (bb): Referring to the internal Convolutional Neural Network (CNN) architecture of the coding path, this parameter offers a selection among architectures such as efficientnet-b7, efficientnet-b6, efficientnet-b5, inceptionv4, and vgg19 [15].

The entire process has been implemented through the development of custom Python-based code, involving the Segmentation Models library [15] for managing model architecture, loss function and backbones, and the Bayesian Optimization library for managing the hyperparameter optimization [18]. To enhance the dataset size, a range of data augmentation techniques has been implemented, including horizontal and vertical flipping, random shifting, random rotation, introduction of Gaussian noise, and perspective transformation [19]. An early stopping criterion [20] is adopted for each training, in which the number of epochs is set to 1000. If the validation loss does not increase for five consecutive epochs, the training is stopped, and the model moves to the next iteration of the optimization process. The iterations of the optimization process are set to 1000.

The model underwent training on a Linux Ubuntu 22.04 desktop equipped with dual NVIDIA GeForce RTX 3090 GPUs, an Intel Core i7-12700F 2.1 GHz processor, and 64 GB RAM, Python 3.9 and PyTorch 2.3.1. The hyperparameter optimization process obtains, at the end of all iterations, a binary semantic segmentation model with an IoU of 0.976 on the test set. This result is achieved with a learning rate of 0.0002, a batch size of 5, the Adam optimizer and an EfficientNet-B7 backbone. To validate the model's robustness, a k-fold cross-validation approach with 3 folds was employed, resulting in an IoU value of 0.975 ± 0.002.

3. Results

A single file (∗.uint16_scv) of about 6 GB was obtained from the CT scan for each sample. Subsequently, each file was cropped to delete all the area without the target image, this made the file smaller and easier to be processed in the next steps. 1024 JPG files per cartesian axis (X-Y-Z, 3072 files in total) were obtained after the uint16_scv file processing and each file was 1024 x 1024 pixels, 92 dpi.

The main result of the proposed procedure is the map of voxels in which plastic inclusions were identified. The model, trained on the sample A, was then applied in the tomography analysis performed on sample B, similar in characteristics, but different in both the characteristics and portion of the fish analyzed and the size and location of the plastic inclusions (Fig. 6).

Fig. 6.

A: Tomographic input image; B: Corresponding image with the predicted mask revealing the presence of a plastic inclusion.

Specifically, shown in Fig. 7 is the 3D map obtained by inference applied to slices in a single direction (slices in the X direction).

Fig. 7.

Plot of the voxels recognized as plastic only in x slices. False positive are highlighted with circles.

The trained semantic segmentation model clearly recognizes the presence of 100 % of the inoculated particles in the B sample, both exactly in their location, shape and volume. Also noticeable is the presence of some false positives, i.e., voxels in which the network detects plastic inclusions that are not actually present in the results of the MPs detection. In Fig. 7 sole false positives are highlighted with circles. These false particles are small in size and could be discriminated by filtering, but it is clear that such an approach heavily compromises the resolution of the technique, since it is not possible to otherwise discriminate such noise from voxels that correctly detect plastic. Instead, the superposition of the inferences made on the obtained slices in the three directions x, y and z is shown in Fig. 8.

Fig. 8.

Plot of the voxels recognized as plastic in x, y and z slices.

The voxels in each direction, recognizable by the color of the plotting, turn out to provide the same indications of the location and size of the inclusions, while the voxels of the false positives are different in the different directions. For cleaning the maps from noise, it is therefore possible to consider as plastic voxels only those identified by all three inferences in the 3 directions. In essence, a logical “AND” operation is performed among the three maps, thus eliminating voxels that are not recognized as plastic by all three inferences. The entire process duration was about 3 h (not considering the time of manual segmentation and training process). The final result image representing only the correctly identified MPs is shown in Fig. 9.

Fig. 9.

Plot of the final result showing the voxels recognized as plastics in both x, y and z slices.

4. Discussion

MPs have been identified in various settings and mediums, including rivers, oceans, sewage, sediments, and soil. Additionally, they have been found in a diverse range of food items, such as beer, canned sardines, honey, and drinking water, ultimately reaching the food table, and making direct contact with humans. The extensive use of plastics and the resulting environmental contamination linked to plastics are likely to result in human exposure to MPs. Humans encounter plastic particles on a daily basis through activities like inhaling nano-plastic containing aerosols, consuming them in their diet, and having dermal contact with them [[21], [22], [23], [24], [25], [26]]. The ingestion of plastic-contaminated foods leads to exposure, resulting in the consumption of 39,000–52,000 plastic particles per person per year. This estimate could even be higher when factoring in the settling dust on plates during mealtimes. Given the widespread contamination of food with MPs, the observed level of intake is not unexpected [[27], [28], [29]].

Nowadays, several techniques are developed for MPs detection, extraction, and identification from organic matrices such as fish or shrimps. Throughout the entire process, there is a high probability of contamination, requiring extreme care in handling the samples and working in an extremely uncontaminated environment free from other sources of plastics. Furthermore, all the techniques used nowadays do not allow for a precise understanding of the exact position of the MPs inside the entire body of the species (e.g., on the skin, inside the muscle, etc.) and the entire methodology for detection, extraction, and identification of MPs from sea species requires a significant processing time, typically around 3 or 4 days per sample.

Trusler et al. [30] explored the use of CT to distinguish various types and sizes of MPs (PP, PET, PE, and PVC) in river-estuary sediments. The CT images showed good contrast with the background sediments, allowing all MPs examined in artificial layers to be observed. However, large MPs (4 mm) were detectable even when randomly distributed in the sediments, while smaller MPs (≤125 μm) could not be detected due to resolution limitations. Funcke et al. [31] described a robust approach for quantifying MPs fragments in aqueous environments. Using a microCT system V|tome|x-M180/300, they successfully analyzed plastic fragments ranging from 0.18 to 0.71 mm in diameter, achieving satisfactory quantitative results with a relative error below 20 %. However, microCT has limitations in contrast resolution, particularly for particles ≤125 μm, which compromises sensitivity at smaller scales. The study conducted by Zhu et al. [32] demonstrates the effectiveness of deep learning in automatically identifying MPs using Fourier-transform infrared (FT-IR) spectroscopy on environmental samples. Introducing “PlasticNet,” a convolutional neural network architecture, PlasticNet's deep learning approach effectively overcomes challenges posed by traditional spectral classification, successfully handling additives, surface modifications, and adsorbed contaminants that can alter spectral shapes. Up to now, no data or papers are available that use the same technique employed in this study to detect MPs in fish or food.

The entire process, from sample preparation to results, takes just a few hours and does not require the constant presence of a researcher, thanks to the automation of the entire process. The sample size that can be processed at a time ranges from a few mm³ to 100 cm³ (30 cm height), which is effectively larger than the limited size processed in the previous techniques. The actual cost of the procedure used in this experiment, considering materials, the gel, and the CT scan working cost, is around 150.00 Euro (160.00 USD) in total. Therefore, the main advantages of the proposed technique lie in the accurate identification of plastic localization within the sample, low cross-contamination probability, rapid sample processing, low man-hours required to work on the process, automatic tomographic measurement, rapid processing of data despite their large size, low-cost methodology, and high cost-benefit.

The herein results highlight how artificial intelligence applied to conventional techniques can significantly improve precision and efficiency in MPs research, accelerating the identification process and surpassing the limitations of traditional spectral analysis methodologies. This procedure allows for accurate and detailed analysis, enabling statistical analysis of the samples by virtue of the possibility of automating the procedure itself.

It is important to note that for future research, the wider the range of MP samples in terms of density, shape and size that the semantic segmentation model is trained on, the more widely applicable this technique will become in different matrices. In addition, the semantic segmentation model can better detect and provide information about the MPs typology. Further research is already underway in the laboratory team to improve the process, testing increasingly smaller and different MPs particles, evaluating the limits of the technique, and developing an integrated methodology for the automatic identification and distinction of the main MPs polymers in food. This is aimed at making the technique user-friendly and entirely shareable.

5. Conclusions

The automated CT scan technique represents a significant advancement in MPs research, offering several key advantages over traditional methods. It enables rapid and accurate identification of MPs within samples. The method's low cost and efficiency make it particularly promising for enhancing food quality control processes and environmental monitoring efforts. Considering the importance of food MPs contamination for human health in modern knowledge and all these advantages in this new methodology, there is a high potential for a development of an automated food quality control processes.

Future research efforts will focus on refining and expanding the technique's capabilities, including testing with smaller MPs particles and developing integrated methodologies for polymer identification, aiming at the technique usability and applicability across diverse matrices and environmental conditions. In conclusion, the automated CT scan technique stands poised to revolutionize MPs research by offering a robust, efficient, and cost-effective solution for detecting and analyzing MPs in biological samples and food.

CRediT authorship contribution statement

Pierluigi Strafella: Writing – original draft, Methodology, Funding acquisition, Conceptualization. Nicola Giulietti: Writing – review & editing, Validation, Software, Methodology, Formal analysis, Data curation. Alessia Caputo: Visualization, Software, Resources, Data curation. Giuseppe Pandarese: Supervision, Methodology, Formal analysis, Data curation. Paolo Castellini: Writing – review & editing, Supervision, Software, Methodology, Funding acquisition, Formal analysis, Data curation.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.