A Systematic Review of Species Classification Using Deep Learning Algorithms and Gender Identification of Tribolium castaneum Using Convolutional Neural Networks

Abstract

Machine learning (ML) constitutes a division of artificial intelligence (AI) that aims to train computers how to perform specific tasks without explicit programming. Traditional ML tools are widely used for classification and identification of animals. However, these methods have some drawbacks because of the extensive manual reliance and the delay in data interpretation. To overcome this, Applied Deep Learning algorithms are used with Artificial Neural Networks (ANN) and Convolution Neural Network (CNN) models introduced to address species classification, characteristics detection, and pattern recognition tasks helping in accurate identification and classification of animals. In this paper, we have tried to compile and deliver a recent comprehensive information on latest available investigations in the field of life sciences particularly used for animal identification. We have also accentuated the diverse applications of machine learning models including other parameters like, features, accuracy gained, database used and their limitations. The red flour beetle, Tribolium castaneum (Coleoptera; Tenebrionidae) is a prevailing and detrimental secondary insect pest of stored grains along with derived products causing 7% to 35% annual loss. Despite of that, nowdays it is also extensively considered as a model organism for genetic disease investigation. While using it in scientific research, exact sex identification of these insects becomes a crucial preliminary step. Generally, pupal stage is used to sort these insects according to their sex and needs expert humans. It is crucial to employ image processing and ML algorithms to quickly identify gender of this insect which is not done yet. We have used a CNN-based smart technique to recognize and categorize gender differences in T. castaneum using microscopic images in order to build an intelligent system for applied research. For this study, a dataset is created by taking 116 microscopic images of both the dorsal and ventral sides of pupae of two different sexes. In this algorithm, a 2D matrix of feature map is selected sequentially and the maximum value in the matrix is selected to generate a pooled feature map. The Rectified Linear Unit (ReLU) activation function is used for the CNN. The classification model has an accuracy between 97 and 98% with an F-score of 0.67. These results demonstrate the robustness of the classification model, which does not rely heavily on manual intervention compared to traditional machine learning (ML) tools and automates the processes of feature extraction and gender classification regardless of the position of the pupae in the images.

BACKGROUND

Tribolium castaneum, (Herbst 1797) commonly known as the red flour beetle and belonging to the Coleoptera order within the Tenebrionidae family, is a prevalent and destructive secondary insect pest that primarily targets stored grains and their derived products. This species exhibits sexual dimorphism (Sokoloff 1974; Rees 2004; Mahroof and Hagstrum 2012). Under optimal conditions, the developmental time for T. castaneum is approximately five days for eggs, twenty days for larvae, and seven days for pupae (Sokoloff 1974; Dawson 1964). Both larvae and adult beetles infest stored food and grains, posing a significant threat to agricultural commodities. Hana (2013) reported that the damage inflicted by these beetles accounts for a considerable percentage, ranging from 7% to 35%, of total agricultural production on an annual basis.

Nowdays, T. castaneum have gained recognition as a valuable model organism for investigating the underlying causes of genetic diseases. Notably, T. castaneum was the first species within the Coleoptera order to have its genome sequenced (Richards et al. 2008; Herndon et al. 2020). This type of insect species is progressively employed across a diverse array of biomedical investigations, covering fields like neurodegenerative ailments (such as Parkinson’s disease), the signaling pathway for diuretics (which involves the function of vasopressin-like peptide and its receptor), interactions between hosts and pathogens (encompassing antagonistic interactions and coevolution), as well as the domains of pharmacology and toxicology (specifically in the analysis of the impacts of psychoactive substances) (Denell 2008; Grunwald et al. 2013; Bingsohn et al. 2016; Adamski et al. 2019). Although there exist certain dissimilarities in cellular characteristics and overall structure between humans and the red flour beetles, a multitude of genetic, physiological, and immunological traits remain consistent. This conservation renders them a valuable model for investigating diverse facets of human biology. T. castaneum’s utilization as a model for practical research extends beyond postharvest management, encompassing inquiries into aging, the environment, and pest control (Thomson et al. 2014; Wijayaratne et al. 2018). Several studies have been conducted on the evolutionary and the pre- peri and post-mating sexual selection behavior of T. castaneum (Michalczyk et al. 2010).

The red flour beetle, known for its reddish-brown coloration and three-segmented clubbed antennae (Bousquet 1990), exhibits distinct sexual dimorphism. In T. castaneum, differentiation between males and females can be established by the presence of genital papillae during the pupal stage and sex patches in the bodies of adult insects. This differentiation is applicable to both pupal and adult stages. Notably, sex determination is most straightforward during the pupal stage (Kramarz et al. 2016). The morphological characteristics of the insect, influenced by both genotype and phenotype, indirectly impact the process of gender classification. Female pupae display pointed genital papillae, whereas male pupae possess stubby and barely noticeable papillae.

Accurate identification of the sex of individuals is a crucial initial step in characterizing the population. As the sizes of the beetles are very small (3–4 mm), it is difficult to be perceived with human eyes. However there is need identify it using microscopic images. Perseverance of large number of microscopic images create fatigue to human subjects which will endup in to incorrect outcome. This repeated task will be very well handled by machine. Thus machine learning plays a very vital role in classification of beetles. There is an urgent need to integrate some fast processing techniques to speed up the experiments going on them.

Machine learning has emerged as a viable alternative to traditional technical methodologies in various domains of science and technology. By leveraging data-driven approaches, machine learning techniques offer the potential to expedite the design process, minimize complexity, and enhance costeffectiveness (Simeone 2018).

Machine learning is a subset of artificial intelligence dedicated to instructing computers in the execution of particular tasks, all without necessitating direct, explicit programming. Computers are fed structured data and ‘learn’ to become better at evaluating and acting on that data over time. A computational framework inspired by the structure of biological neural networks, which forms the foundation of the human brain, is commonly denoted as an artificial neural network. These networks are capable of effectively processing vast quantities of data. Artificial neural networks (ANNs) have been shown to be effective tools for various kinds of tasks, but they also have a number of disadvantages. As ANNs to generalize effectively, a lot of labelled training data is often needed. Overfitting or low accuracy might result from a lack of data or data with poor quality. Neural networks frequently perform the function of “black boxes,” which means they can make accurate predictions but cannot be interpreted. The “curse of dimensionality” refers to the exponential increase in data required to generalize the machine learning model accurately as the number of dimensions or characteristics rises. Additionally, ANN requires features that are handmade for model training and testing.

Very few characteristics enables male and female T. castaneum pupae to be distinguished from one another. Because the pupae body is white in colour and has less texture, it is difficult for conventional machine learning approaches to correctly recognize it. Female and male pupae 117 could be distinguished by the size and shape of genital papillae, located anterior to the 118 urogomphi. Females have larger and finger like papillae whereas males has smaller papillae. To overcome this problems we need deep learning network to classify the beetles. Convolution Neural Network (CNN) model is used to classify beetls automatically. A Convolutional Neural Network (CNN) can comprise tens or even hundreds of layers, with each layer designed to learn and recognize different features within an image. During training, every image undergoes filtering at multiple resolutions, and the outcome of each convolution is employed as input for the subsequent layer. Starting with fundamental attributes such as brightness and edges, these filters can progress towards more intricate aspects, ultimately culminating in features that distinctly identify the object. Deep learning algorithms, notably convolutional neural networks (CNN), have garnered substantial attention due to their capabilities in pattern recognition tasks related to image analysis. Their popularity is notably prominent in the field of biological sciences.

In our proposed study, we aim to devise an intelligent approach utilizing deep learning techniques to discern and categorize disparities observed in both species and gender based on microscopic images captured from ventral and dorsal views. This endeavor seeks to contribute to the advancement of intelligent systems within the realm of applied research.

Literature survey

The table 1 (presented below at the end of manuscript) provides a comprehensive overview of recent investigations in the field of life sciences, highlighting the diverse applications of machine learning models. Predominantly, supervised learning models have been employed to address species classification, characteristics detection, and pattern recognition tasks. An array of machine learning algorithms, including support vector machines (SVM), logistic regression (LG), random forests (RF), gradient boosting (GB), k-nearest neighbors (kNN), decision trees (DT), and deep learning (DL), have been employed to achieve these goals. The utilized datasets encompass both publicly available resources and researchergenerated collections through sample acquisition. The fundamental strategy in constructing classification models entails dividing the dataset into a training set for model development and a test set for the purpose of validating and evaluating the model’s dependability. Accuracy measurements reveal that deep learning variants, including artificial neural networks (ANN) and convolutional neural networks (CNN), demonstrate superior robustness compared to other classification models. Nevertheless, it is important to acknowledge that the accuracy of these models is influenced by certain limitations, such as the quantity and quality of the datasets employed, as well as class imbalance, which arises when multiple species classes are involved in the classification and identification studies. It is widely acknowledged that there has been minimal research conducted so far regarding gender classification using computational methods in T. castaneum. In this proposed study, we aim to utilize a CNN-based intelligent approach to detect and differentiate gender disparities from microscopic images. This endeavor is aimed at contributing to the advancement of intelligent systems within the domain of applied research.

Table 1.

Summery of features, accuracy, model and limitations

The following table offers a comprehensive overview of recent investigations conducted in the field of life sciences (Table 1), highlighting the various applications of machine learning models. Primarily, supervised learning models have been utilized to address tasks such as species classification, characteristics detection, and pattern recognition. To achieve these objectives, a variety of machine learning algorithms, including support vector machines (SVM), logistic regression (LG), random forests (RF), gradient boosting (GB), k-nearest neighbors (kNN), decision trees (DT), and deep learning (DL), have been implemented. The datasets used encompass both publicly available resources and collections generated by researchers through sample acquisition. The fundamental approach to developing classification models involves partitioning the dataset to create a training set for model training and a test set for the purpose of validating and assessing the model’s credibility. Accuracy measurements indicate that deep learning variants, such as artificial neural networks (ANN) and convolutional neural networks (CNN), exhibit superior robustness compared to other classification models.

Researchers have employed various machine learning and deep learning classification models to study insects and pests. For example, Kirkeby et al. 2021, utilized three different methods to classify insect groups and achieved an impressive accuracy rate of 80%. Chola et al. (2022), employed classical machine learning methods to classify the gender of Drosophila melanogaster, achieving a maximum accuracy of 90%. Rabinovich et al. (2021), used machine learning models for experimental design, testing the thermal limits of kissing bugs and achieving an accuracy of 80%. Veiner et al. (2022), employed Support Vector Machine (SVM), Random Forest (RF), and Generalized Linear Model (GLMNET) to characterize and identify genes associated with honeybee waggle dance, achieving accuracy levels ranging from 67% to 100%. Perez et al. (2022), utilized a deep neural network to classify mosquitoes by genus and sex, achieving an accuracy exceeding 94%. Similar studies were conducted by Pataki et al. (2021); Kittichai et al. (2021); Motta et al. (2019); Bellin et al. (2021); and Zhao et al. (2022), who used deep learning models such as ResNet5016 and YOLO, as well as neural network models like ANN and CNN, to classify mosquito species based on insect images. These studies achieved accuracies ranging from 54% to 100%. Themozhi et al. (2019) and Aladhadh et al. (2022), applied CNN models to classify and detect crop pests using images, achieving accuracy levels higher than 95%. Liu et al. (2022), used a deep learning model to classify tomato pests from a collection of images, achieving an accuracy of 95%. Additionally, researchers have employed faster deep learning models like R-CNN for classifying various insects. Ozdemir et al. in 2022 used R-CNN to identify key indicators for insect classification, achieving an accuracy of over 80%. Shen et al. in 2018 and Alsanea et al. in 2022 applied the R-CNN classification model to images of stored grain insects and for the auto-detection of red palm weevil, respectively, resulting in accuracy levels of 88% and 99%. Bisgin et al. (2022) and Tannous et al. (2023), utilized CNN models for automated identification of food-contaminating beetles and insect species, achieving accuracy levels of 90% and 93%, respectively. Dai et al. (2022), employed cascade region-based convolutional neural networks (R-CNN) for the autodetection of citrus psyllids from images with an accuracy level of 89%. Aladhadh et al. in 2022 applied CNN and faster R-CNN models for the autodetection of pests from insect images with accuracy levels between 57% and 98%.

In addition to species classification studies, machine learning and deep learning models have also been applied in various other biological studies. Lee et al. (2020), used support vector machines, random forest (RF), multi-layered perceptron, AdaBoost, gradient boosting (GB), and CatBoost models for malaria diagnosis and achieved accuracy levels between 56.9% and 85.6%. Safavi et al. (2022), employed ExtraTreesClassifier and ANN models for forecasting and predicting the occurrence of LSDV infection, achieving a highest classification accuracy of 97%. Zare et al. (2022), applied a boosting method to train classifiers and extract features from microscopic images for detecting leishmaniasis with an accuracy level of 83%. Petrescu et al. (2021), used decision trees, k-nearest neighbors, support vector machines, and artificial neural network models for fear classification from physiological data, with accuracy levels between

91.7% and 93.5%. Shia et al. (2021), employed a combination of locally weighted learning and sequential minimal optimization for unsupervised classification of malignant breast tumors, achieving an accuracy level of 84.7%. Deep learning methods have been applied by Antipov et al. (2016) and Patel et al. (2020), who used CNN ensemble and R-CNN models for gender prediction from face images and classification of Galápagos Snake Species, with accuracies greater than 96% and 75%, respectively. Zhu et al. 2020, applied neural network models for the identification of three specific types of promoters in the DNA sequences of species including Homo sapiens, Mus musculus, Drosophila melanogaster, and Arabidopsis thaliana, with classification accuracy ranging from 80% to 98%.

However, it is important to acknowledge that the accuracy of these models is influenced by certain limitations, such as the quantity and quality of the datasets used, as well as class imbalance that occurs when multiple species classes are involved in classification and identification studies.

It is well established that the realm of gender classification through computational means in T. castaneum remains significantly underexplored. Given the minute sizes of these beetles, human classification becomes challenging, necessitating the fusion of image processing and machine learning techniques to facilitate species and gender identification, which could speed up the ongoing experiments on them. Here, in this planned work, a machine learning-driven intelligent methodology is developed by utilizing microscopic images (from ventral and dorsal viewpoints) to discern and categorize gender disparities. The ultimate objective is to foster the development of intelligent systems within the applied research domain.

MATERIALS AND METHODS

Insect rearing and image Acquisition

The primary T. casteneum culture was procured from ROSS Lifescience Pvt. Ltd., Pune. It was thereafter cultured in wheat adding 5% yeast at the Zoology department of Savitribai Phule Pune University in an ideal environment at 33 ± °C and 70% relative humidity in a BOD incubator (Halliday et al. 2014). Images of T. castaneum at the pupal stage were acquired using digital stereo microscope (Nikon SMZ1270) and MIchrome 6 (6MP) color microscopic camera. The microscopic picture dataset comprises a total of 116 photographs of pupae of two distinct sexes, male and female, with each class including 58 images. These photos were taken from both the dorsal and ventral sides using constant angle and magnification (40X) with a dark background. Figure 1a–d shows representative photos of male and female T. castaneum pupa. In this study, machine learning model is train and tested with image size 128 × 128 × 3.

Fig. 1.

Specimen photographs of T. castaneum pupae under the microscope. a) and b) ventral and dorsal view of female pupa, c) and d) ventral and dorsal view of male pupa.

Methodology

The methodology for the proposed study is depicted in figure 2. The prepared images, as discussed in previous sections, are given as input to a deep neural network. A total of 116 original images were used in this study. To improve model generalization, data augmentation was applied. Each training image was augmented using techniques such as rotation, horizontal and vertical flipping, zooming, and brightness adjustments, resulting in 10 augmented versions per image. This expanded the dataset to a total of 1,160 images. The dataset was then split into training and testing subsets, with 70% of the images used for training the network and 30% used for evaluating its performance. The trained network classifies the images as male and female pupae. Since the algorithm deals with the classification of the pupae, a Convolutional Neural Network (CNN) is applied, which relieves us from the need to locate the pupae in the given image and focuses on recognizing the gender of the pupae. The important feature of CNN is obtaining the abstract features as the image propagates through the deeper layers of the network. Each image is expressed as 128 × 128 matrix.

Fig. 2.

Architecture of the proposed model.

The first step of the CNN is defining the convolutional layer. Each image matrix is defined with a kernel, which is a set of random numbers also called kernel weights. These weights are adjusted during each training cycle, allowing the kernel to extract significant features. In the applied algorithm, the kernel is of size 2-D (e.g., 2 × 2). The kernel slides over the complete given image matrix, and the corresponding values are multiplied and summed to create a single scalar value.

The next layer is the pooling layer, which reduces the feature map generated by the convolutional layer. The commonly used algorithms for the pooling layer are max pooling, min pooling, and Global Average Pooling (GAP). The applied algorithm works on the max 2-D pooling algorithm. In the max pooling layer, a small window (typically 2 × 2) slides across the feature map, and for each window, the maximum value is selected to create a downsampled (pooled) feature map. Following this, the Rectified Linear Unit (ReLU) activation function is applied to introduce non-linearity by replacing all negative values with zero. This function returns zero if the input is negative or the value itself.

Following this, a fully connected layer is designed as the final layer of the CNN network. In this layer, each neuron is connected to all the neurons of the previous layer. The input to the fully connected layer comes from the last pooling or convolutional layer, and its output represents the final output of the CNN.

We chose to develop this model to overcome the problem of overfitting for this specific test case scenario. During the early stages of the development of the solution to the problem we tried to implement VGG16 architecture but faced a serious problem of overfitting, as the image was microscopic and had fine details which were not necessarily needed for the dectection problem as seen in figure 1. Hence a simpler CNN model proved to be higher performing than more denser models. The architecture of the CNN model includes three convolutional layers followed by maxpooling layers, with the first layer having 32 filters, the second layer having 64 filters, and the third layer having 128 filters, all using a 3 × 3 kernel size and ReLU activation. Each convolutional layer is followed by a 2 × 2 max-pooling layer to reduce the spatial dimensions. After the convolutional and pooling layers, the output is flattened and passed through a fully connected dense layer with 128 units and a ReLU activation function, followed by a dropout layer with a dropout rate of 0.5 to prevent overfitting. We use binary cross entropy as loss function with batch size of 32 which provided us with good number of gradient updates. Figure 2 depicts the detailed architecture of the model. The final output layer uses a softmax activation function to classify the images into the respective categories. This architecture allows the model to learn complex patterns in the images while mitigating overfitting by incorporating dropout and appropriate pooling operations. Reffer to figure 3 for the representation.

Fig. 3.

Procedure and filters for CNN model.

RESULTS

As discussed in the methodology, the first step in identifying key factors for pupal gender classification was the design and implementation of a convolutional layer. The employed model was trained on 812 images in the training dataset and validated and tested on 348 images in the test dataset. To minimize the risk of overfitting with a limited dataset, all images were resized to a resolution of 128 × 128 × 3. This reduction in resolution decreased the input dimensionality, effectively lowering the model’s complexity. Additionally, the resizing process enhanced key morphological features by creating a zooming effect, allowing the network to focus on more prominent and consistent traits rather than fine-grained noise, which can contribute to overfitting in small datasets. While this approach improved the model’s generalization capability, future work will include the application of interpretability techniques such as Grad-CAM to further validate that the CNN is attending to biologically meaningful regions within the images.

Extracted features by the CNN were passed to a dense network to classify the sex of the pupae. Experimental results showed a high classification accuracy. Notably, it was observed that reducing the image resolution improved the model’s ability to differentiate key features. The model was implemented using the TensorFlow library in Python, requiring only 8 MB of memory. It was trained and tested on an Nvidia RTX 3060 using Keras version 3.0.5, CUDA version 11.2, and cuDNN version 8.1, which facilitated efficient processing and inference.

This overall configuration led to a classification accuracy between 97% and 98%, as illustrated by the accuracy and loss plots in figure 4. The model also achieved an F-Score of approximately 0.96 (Fig. 5). These results demonstrate the effectiveness of the proposed model for real-time, automated classification of T. castaneum pupae. A comparison of the model’s performance with other state-of-the-art techniques is provided in table 2.

Fig. 4.

Plots of accuracy and loss of employed model.

Fig. 5.

Confusion matrix for the deployed model.

Table 2.

Comparison of results between proposed and other state of the art methods

DISCUSSION

We have already comprehensively discussed the use of deep learning algorithms for species identification in the literature survey section, so we are excluding that information in this section to avoid repetition and considering only CNN-based methods that are similar to our study. Themozhi et al. (2019) classified and detected 40 different crop pests using photos with greater than 95% accuracy. Tannous et al. (2023) and Cannet et al. (2022) used CNN models to automatically identify Glossina spp. (tsetse flies) and food-contaminating beetles, respectively, with accuracy levels of 93% and 3–100%. Alsanea et al. (2022) achieved accuracy values of 99% by applying the R-CNN classification model to photos of stored grain insects and for the autodetection of red palm weevils, respectively. Cascade region-based convolutional neural networks (R-CNN) were used by Dai et al. (2022) to recognize citrus psyllids from photos with an accuracy of 89%. Zhao et al. (2022) classified mosquito species based on insect pictures using deep learning models like ResNet5016 and YOLO as well as neural network models like CNN. Compared with these results, our experimental set up network performed well with accuracy 97–98% and F1 score 0.67. Nevertheless, identification of gender of pupae in given image challenges minute level feature extraction. The employed network result indicate that cutting-edge systems for tracking the dynamics of pests populations can be created using machine learning methods. This investigation of the population dynamics of important pupae may encourage the creation of novel methods based on a field-based distributed network of automatic monitoring system. This would improve the effectiveness and sustainability of control efforts. With little data pre-processing and a similar design, this approach can be applied to other species of interest.

CONCLUSIONS

This finding can be used in ecological and entomological study as well as a variety of developmental research projects in which determining the gender of T. castaneum will be needed. Automation of current process helps to minimise the efforts and time in sex-specific categorisation of T. castaneum beetles with 97–98% accuracy. To compensate for the lost characteristics in the photographs and attain more accuracy, we intend to further improve our algorithm by including Image Enhancement techniques in future.

List of abbreviations

ML

Machine learning.

AI

Artificial Inteligence.

ANN

Artificial neural network.

DL

Deep learning.

CNN

Convolution neural network.

ReLU

Rectified Linear Unit.

GAP

Global Average Pooling.

NBAIR

National Bureau of Agricultural Insect Resources.

RF

Random forest.

GB

Gradient boosting (GB).

FAO

Food and Agriculture Organization.

LR

Logistic regression.

SVM

support vector machines (SVM).

DNN

Deep neural network.

NB

Naive Bayes.

KNN

K-nearest neighbour.

RIBBIT

repeat interval-based bioacoustic identification tool.

GLMNET

Generalized Linear Model.