Transformer augmented hybrid deep learning for explainable multi class pest classification

Abstract

Agricultural pests continue to impose serious threats to global food security by causing major yield losses across diverse cropping systems, making early and accurate identification vital for effective pest management. With the growing integration of digital technologies in modern agriculture, deep-learning–based pest recognition has become a promising approach to surpass the limitations of manual scouting and conventional monitoring practices. This work presents a comprehensive experimental evaluation of multiple deep-learning architectures for multi-class pest classification via image-level classification covering 19 pest species. The study investigates classical CNNs (MobileNetV2, VGG16), compound-scaled models (EfficientNetB0/B3, EfficientNetV2-B0), residual architectures (ResNet50), automated NAS models (Xception, NASNetLarge), and novel hybrid CNN–Transformer designs including Hybrid InceptionResNetV2, Hybrid ResNet50 + CBAM, Hybrid EffNet-Transformer, and Hybrid EfficientNetV2-S + Transformer. To enhance foreground isolation and reduce background complexity in field images, segmentation-driven preprocessing is employed using GrabCut, Watershed, SLIC, and Felzenszwalb, generating structure-refined image representations for downstream classification. Results show that attention-augmented hybrid models consistently outperform standalone CNNs, with the Hybrid EfficientNetV2-S + Transformer achieving the highest performance with 0.8800 validation accuracy, 0.849 macro-F1, and 0.4560 validation loss. These findings highlight the effectiveness of combining convolutional feature hierarchies with global self-attention for reliable multi-species pest classification and offer meaningful guidance for developing intelligent precision agriculture systems.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-026-41248-1.

Introduction

Pests are one of the main factors that endanger agriculture all over the planet and they cause such losses that they negatively influence food security and economies on the global scale. According to the forecasts for the years 2024–2025, pests alone will account for about 20–40% of the total annual crop yield that will be lost, which would mean that these pests have an economic impact of $220 billion to $290 billion every year on the global economy¹. The huge loss of yield here consists of the damage that insects, weeds, and plant diseases have done, with insect pests being solely responsible for about 38% of these lost crops. The losses might be even bigger if there were no crop protection in place, as some major food crops could suffer losses of up to 70%. Besides, the damage done to crops has a dual effect on the market - less food available and hence its price goes up, and at the same time, it becomes harder for farmers to survive because of the high production costs². The agricultural pest control market is expanding rapidly, with a value of over $8.5 billion and a future growth expectation due to the rise in food demand and the increasing pest problems caused by climate change. The size of the market signifies an urgent need for pest management through improved and sustainable methods³. In any case, traditional pest control methods, particularly those that use chemical pesticides, have the disadvantages of pest resistance, pollution, and the destruction of non-target beneficial species. Consequently, the practice of integrated pest management and the development of biopesticides that are less harmful to the environment have gained even more popularity⁴.

Earlier, pest control methods were mainly based on manual inspection, followed by the extensive spraying of chemical pesticides that took over large areas. Although these methods have been around for decades and are still widely used, they have many drawbacks. Some of the issues caused by these methods are late classification of pests, air quality deterioration, decreased pesticide efficacy, and accidental killing of beneficial insects. Besides, climate change and pest migration have made it even harder to achieve sustainable pest control⁵. Still, in the recent past, AI has been one of the promising solutions to this problem. The pest classification systems powered by AI are using computer vision, deep learning and advanced sensor technologies, to locate the pests quicker, at a higher level of accuracy and in real-time⁶. The presence of large datasets and the deployment of powerful CNN algorithms often enable these systems to attain precision levels that are significantly superior to manual inspection. The benefits of using automated classification along with constant monitoring are the implementation of targeted treatments which results in reduced chemical use and minimal adverse environmental impact. In addition, the partnership of drones, satellite imagery and IoT sensors provides new insights on pest locations, populations and their interactions with crops, thus making pest management decisions eco-friendlier and more informed⁷.

In the recent years, AI technology has emerged as a promising approach to address these limitations. These AI-centric pest classification units implement various methods such as computer vision, deep learning and sensor technology. This makes the process of classification less complex and continuous. Next, the system is working on a huge volume of data and a very complex algorithm such as Convolutional Neural Networks that can attain precision in the range of 85 to 95%, which is way higher compared to manual processes⁸. Another advantage of this technology is that it automatically recognizes and instantly monitors the appeasement of the pest towards precision pest management in the infested areas. This reduces the volume of chemical used, thus saving the pest from contamination. Combining drone and satellite analysis for crop monitoring along with IoT sensor real-time data would create a foundation for the sustainable pest surveillance in terms of the pest population and the environment. The development of a balanced dataset; the application of advanced image segmentation techniques; and optimization must come followed by the changing of the metaheuristic algorithms that will be enhancing the accuracy and adaptability of the classification to the different practical agricultural environments⁹. By the integration of real-time variables of soil moisture and crop pans, AI can provide very accurate pest control management through direct intervention and, thus, deterring high levels of damage. The use of AI has brought a change in the direction of pest control to a more sustainable and efficient future that is also environmentally responsible and promotes the health of both the ecosystem and the crop. This approach offers potential benefits for agricultural applications. Hence the aim of the paper is to identify the most effective deep-learning architecture for multi-class pest recognition by evaluating CNNs, modern compound-scaled networks, automated NAS-based models, and hybrid CNN–Transformer designs on a large field-based pest dataset.

The primary contributions and novel aspects of this study are outlined as follows:

• The paper develops and evaluates a diverse set of deep learning architectures, including classical CNNs, compound scaled EfficientNet variants, residual networks, NAS based models, and hybrid CNN Transformer frameworks to determine their effectiveness for multi class pest recognition.

• The paper provides a novel analysis of architectural factors such as depth, width, receptive-field scaling, convolutional structure, and attention integration, showing how these elements influence the recognition of small, occluded, and visually similar pests.

• Segmentation-based explainability using GrabCut, Watershed, SLIC, and Felzenszwalb is incorporated to understand how models interpret pest boundaries and background complexity.

• The work contributes practical guidance for designing next-generation hybrid architectures tailored for reliable and scalable pest classification in precision agriculture.

Related work

Recent advances in computer vision have led to significant progress in automated pest classification, with many studies exploring deep learning models for classification, classification, and field-level monitoring (Table 1).

Table 1.

Summary of recent studies on pest classification and classification.

Author	Primary Task	Core Methodology	Dataset/Setting	Research Emphasis
Wen et al. (2022)	Detection, counting	YOLO-based detector with focal loss	Pest24 (field images)	Dense and small-object pest detection
Qi et al. (2023)	Real-time detection	Optimized YOLOv5 architecture	Field-acquired pest images	Speed–accuracy balance for real-time use
Liu et al. (2022)	Dataset construction, classification	Curated large-scale dataset with CNN baselines	Forestry pest dataset	Large-scale multi-class pest benchmarking
Peng & Wang (2022)	Classification	CNN backbone with Transformer head	IP102	Global-context modeling for pest recognition
Karar et al. (2021)	Classification (on-device)	Lightweight CNN for mobile deployment	Single-species field data	Embedded and mobile pest monitoring
Guan et al. (2023)	Lightweight classification	EfficientNet-style compact CNN	PlantVillage, IP102	Efficient architectures for constrained devices
Li et al. (2024)	Detection and localization	Lightweight Faster R-CNN with attention	Field pest images	Accuracy–efficiency trade-off in detection
Yuan et al. (2024)	Classification	ResNet with spatial attention	IP102	Attention-enhanced feature learning
Venkateswara & Padmanabhan (2025)	Detection and classification	Augmentation-driven deep models	IP102	Data-centric improvements for pest recognition

Pest-YOLO by Wen et al. (2022)¹⁰ achieved a mean-AP of 69.59% and mRecall of 77.71% when evaluated on the Pest24 benchmark (approximately 20k images, around 190k annotations). Qi et al. (2023)¹¹ improved Pest-YOLO by enhancing classification accuracy (mAP increased from 71.5% to 74.1%) along with increased throughput (79.4 → 104.2 FPS) and decreased FLOPs/model size in their experiments. The Forestry Pest Dataset assembled by Liu et al. (2022)¹² provided a massive and carefully assembled set of pest images in the order of several tens of thousands with well-established class divisions, and although the problem of class imbalance persists, it has turned into a frequently applied reference point for classification research. The Hybrid CNN and Transformer-based classifiers which were introduced by Peng & Wang (2022)¹³ showed to be better in terms of performance on IP102 and related datasets. Moreover, the authors had reported classification accuracies varying from 74.9% to 75.6% depending on the different resolutions of inputs. By their results, the necessity of the Transformer heads to capture the global context cues, which were very important for fine-grained recognition of pests, has been proved. Karar et al.¹⁴ managed to accomplish in-field accuracy of the highest level for certain pest targets, e.g., red palm weevil classification. Guan et al. (2023)¹⁵ found an optimal combination of the model size, the speed of inference, and the accuracy at the field level. Dise-Efficient had shown that it is able to perform well on the PlantVillage dataset, which was designed for controlled experiments, and reach only moderate accuracy on the much more demanding IP102 dataset after applying transfer learning. LLA-R-CNN (2024)¹⁶, which was based on a MobileNetV3 backbone and is equipped with locality and channel attention modules, has claimed to have very good performances versus standard detectors yet to consume significantly less FLOPs and total model size. Meanwhile, GBiDC-PEST, the YOLO-based mobile-friendly method that includes GhostNet, BiFPN, and CBAM, possesses a very compact model with an efficient real-time performance and is highly suitable for mobile use. Furthermore, Yuan et al.’s ESA-ResNet34 (2024)¹⁷ and Venkateswara and Padmanabhan (2025)¹⁸ made new contributions to the field of attention-enhanced classification models and augmentation-driven improvements on the IP102 dataset, respectively.

Despite substantial progress in deep-learning–based pest recognition, the existing literature consistently identifies several open challenges that hinder reliable deployment in real agricultural environments. A key issue is robustness to real-field variability, as pest images captured under natural conditions exhibit significant variation in illumination, background clutter, pest pose, scale, and life stage, often accompanied by complex foliage textures, shadows, partial occlusions, and color similarity between pests and host plants, all of which introduce noise and reduce feature consistency. Closely related to this is the challenge of fine-grained discrimination among visually similar pest species, where subtle morphological differences such as wing venation, body segmentation, or minor color gradients must be distinguished despite high intra-class variability and limited visual prominence when pests occupy small or partially obscured regions. In parallel, the literature reflects an unresolved trade-off between classification accuracy and computational efficiency: lightweight and mobile-oriented models are well suited for real-time and edge deployment but frequently lack sufficient representational capacity for complex field imagery, whereas larger and more expressive architectures improve recognition performance at the cost of increased computational demand, memory usage, and susceptibility to overfitting. Furthermore, many existing approaches rely predominantly on local receptive fields, resulting in limited modeling of global contextual information, even though spatial relationships between pests and surrounding plant structures often provide critical cues for accurate identification when local features are ambiguous. Finally, most prior studies evaluate narrowly focused model families or task-specific optimizations, offering limited insight into how architectural design choices such as depth, width, scaling strategies, and attention integration interact under unified experimental settings, thereby restricting consensus on the most effective principles for robust, scalable, and generalizable multi-class pest classification in realistic agricultural scenarios^10–18.

Methodology

In this context, the entire flow of the proposed pest recognition pipeline is represented in Fig. 1. The fundamentals of the process of normalizing raw images, performing data augmentation, and correcting imbalance are discussed, before focusing on the segmentation phase to get an accurate boundary for the objects using one or more techniques. Such inputs were supplied to the various classifiers in the deep learning domain, using these classifiers to evaluate classes 19 pests for standard evaluation metrics.

Fig. 1.

Overall workflow of the proposed pest recognition system.

Dataset description

The research employed a well-structured multi-class pest image dataset consisting of 34,200 images, comprising 28,500 images for training and 2,850 images each for validation and testing. The dataset is sourced from a publicly available Kaggle repository (PestAI Dataset v2) and contains real field-acquired images representing 19 significant agricultural pest species, including aphids, whiteflies, mites, thrips, beetles, weevils, cutworms, cabbage loopers, and various hemipteran and lepidopteran insects (Fig. 2). Each pest category contains a total of 1,800 images, with approximately 1,440 images allocated to the training set and around 180 images each assigned to the validation and test sets¹⁹.

Fig. 2.

Sample images of pest dataset.

The images were captured under real agricultural field conditions using heterogeneous consumer-grade imaging devices, resulting in substantial variation in background clutter, illumination, pose, and pest life stages. While the dataset documentation confirms real-field acquisition across multiple crop types and environments, detailed metadata such as exact geographic location, capture time span, crop variety, or camera specifications are not consistently provided. As shown in Table 2, the selected pest species cover a wide range of crops and symptom types, highlighting the need for early classification to prevent severe yield losses.

Table 2.

Overview of major agricultural pest species.

Pest class	Latin Names	Main host crops	Key symptoms	Why early classification matters
Aphids	Aphidoidea	Brassicas, cucurbits, solanaceae, cereals, legumes	Curling, honeydew, sooty mold, virus spread	Limits virus spread; allows mild controls
Whiteflies	Aleyrodidae	Tomato, cotton, cassava, cucurbits	Yellowing, honeydew, TYLCV	Reduces vector-driven disease
Thrips	Thysanoptera: Thripidae	Onion, chili, cotton, legumes	Silver marks, deformed blooms, tospovirus	Prevents quality loss, virus issues
Spider mites	Tetranychidae	Vegetables, cotton, fruit trees	Webbing, bronzing, stippling	Cuts miticide use; stops flare-ups
Leafminers	Agromyzidae	Tomato, cucurbits	Serpentine mines, necrosis	Treat early instars; save parasitoids
Tuta absoluta	Tuta absoluta	Tomato, eggplant	Leaf tunnels, fruit bore, frass	Trap-based warning enables quick action
Diamondback moth	Plutella xylostella	Brassicas	Windowing, skeletonization	Helps manage resistance
Fall armyworm	Spodoptera frugiperda	Maize, sorghum	Shot holes, whorl damage	Early larvae easiest to control
Bollworm	Helicoverpa armigera	Cotton, tomato, chickpea	Fruit/boll injury	Avoids broad sprays
Stem borers	Mostly Chilo spp. and Scirpophaga spp.	Rice, maize, sugarcane	Dead hearts, white heads	Must act before tunneling
Brown planthopper	Nilaparvata lugens	Rice	Hopper burn, lodging	Prevents outbreaks and resurgence
Leaf/planthoppers	Cicadellidae/Delphacidae	Rice, pulses, vegetables	Hopper burn, virus spread	Cuts virus transmission
Stink bugs	Pentatomidae	Soybean, cotton, fruits	Piercing injury, misshaped pods/fruits	Edge control limits spread
Fruit flies	Tephritidae	Mango, guava, cucurbits	Punctures, maggot-infested fruit	Baiting prevents big losses
Mealybugs	Pseudococcidae	Cotton, grapes, fruits	Waxy clumps, honeydew	Early removal blocks spread
Scale insects	Coccoidea	Citrus, olives	Scale cover, sooty mold	Act at crawler stage
Weevils	Curculionidae	Stored grains, banana	Boring, stand decline	Prevents structural loss
Leaf beetles	Chrysomelidae	Vegetables, cereals	Leaf chew, defoliation	Thresholds protect predators
Grasshoppers/locusts	Acrididae/Locustidae	Cereals, rangelands	Fast defoliation, swarms	Nymph control stops outbreaks

Prior to model training, the dataset was inspected for exact duplicate samples using filename consistency and hash-based comparisons, and no identical images were retained across the training, validation, and test splits. The dataset was partitioned using random stratified sampling at the image level to preserve balanced class distributions across all splits. Since the dataset lacks location and capture-session details, some visually similar images from the same plant or scene may appear in both the training and test sets. This can lead to a potential data leakage issue, which is commonly observed in open field-image datasets. As a result, the results indicate image-level generalization rather than generalization across different fields or times. Nevertheless, the dataset still offers a realistic and challenging setting for assessing how effectively deep-learning models can separate similar pest species and operate under practical agricultural conditions.

Data preprocessing

It is such a crucial factor for pest classification because there is a large difference in the field images when it comes to lighting, size, and the complexity of the background. Normalization was applied to each input channel using a standard operation , which ensured numerical stability and enabled the network to learn feature patterns instead of raw intensity differences. By the method of Data augmentation, the effective diversity of the training set was increased with the help of applying controlled transformations such as rotations, flips, color perturbations, and geometric scaling. The effective dataset size is enlarged from N to approximately N(1 + k) if in the augmentation pipeline an average of k = 5 transformations per image was applied, thus the model generalization is made stronger under heterogeneous field conditions. One way of dealing with class imbalance was to use weighted loss functions, where each class weight is calculated as , with N being the total number of samples, Cbeing the number of classes, and n_ibeing the number of samples in class i. The cross-entropy loss functions , where the weights were integrated through, thus ensuring that minority classes take part in training by contributing proportionally. Normalization, augmentation, and imbalance-aware weighting practiced together collectively improve the robustness and stability of deep-learning models.

Data segmentation

After performing the preprocessing techniques on image data, it undergoes various segmentation techniques^20–23. All segmentation techniques such as Canny, GrabCut, Otsu thresholding, K-means clustering, Watershed, SLIC, and Felzenszwalb were applied as preprocessing to the dataset prior to training the classification models. Each method was used to generate segmented versions of the pest images, ensuring that the models were trained on inputs with enhanced boundary localization and foreground–background separation. These segmentation outputs provided multiple representations of each pest instance, enabling the classifiers to learn from boundary-refined, background-reduced, and structure-enhanced variants of the images.

As shown in Fig. 3, in the Otsu column, the segmentation of all insects such as the Adristyrannus with its dark body and the brightly colored Aphids reflects the method’s reliance on a single global threshold , where is the optimal threshold chosen from all candidate thresholds . Otsu determines this threshold by maximizing between-class variance

1

Fig. 3.

Segmentation results using Otsu, Canny, and K-Means methods across multiple pest species.

where and are the probabilities of pixel intensities falling below and above threshold , and and are the corresponding class mean intensities. This mathematical structure causes areas with dominant intensity distributions such as the dark wings or vivid green leaves to be grouped into large, homogeneous regions. Consequently, the Beetle’s wings appear distinctly separated from the green foliage, while the Cicadellidae shows strong foreground–background contrast but loses fine structural detail. Likewise, the Canny technique displays dense red contour overlays around every insect, revealing how the method uses the gradient magnitude

2

where and are the partial derivatives of image intensity in the horizontal and vertical directions. This expression detects locations where the image intensity changes rapidly, which is why high-response boundaries appear prominently along the Beetle’s elytra, the articulated body of the Cabbage Looper, and the sharply contrasted regions of the Adristyrannus. As a result, Canny produces extensive thin edge maps that trace nearly every visible boundary and texture change including leaf edges and insect limbs leading to visually cluttered outputs. Finally, the K-means column groups pixels into color-based clusters by minimizing the within-cluster variance

3

where is the number of clusters, is the set of pixels assigned to cluster , represents the feature vector (such as RGB values) of pixel , and is the centroid of cluster . Because insects such as the yellow Aphid, blue-green Beetle, and brown Cicadellidae exhibit distinct color distributions, K-means produces compact color regions that approximate their shapes even when boundaries are not sharply defined. This also accounts for the more “blocky” segmentation visible in background foliage, as K-means interprets the image mainly through its color composition rather than spatial continuity or gradients. Across all insects, the interplay between shape, color distribution, and lighting determines the distinctive segmentation behavior seen across the four techniques.

In the GrabCut results as shown in Fig. 4, the pests appear as cleanly extracted foreground objects because GrabCut minimizes the original energy formulation:

4

Fig. 4.

Segmentation outputs using GrabCut, Watershed, SLIC, and Felzenszwalb methods across multiple pest species.

where the data term is

5

and the smoothness term is

6

Here, E =total energy to be minimized, L= label assignment for all pixels (e.g., class of each pixel), θ = parameters of the likelihood model (e.g., class-specific statistics), z = observed data (e.g., pixel intensities or feature vectors), i = pixel index, z_i = observed feature value at pixel I, L_i = label assigned to pixel I, θ₍L_i₎ = model parameters associated with label Li, P(z_i | θ₍L_i₎) = likelihood of observing zi under label Li γ = smoothness weight (penalizes label discontinuities), (i, j) ∈ N = neighboring pixel pairs, [L_i ≠ Lⱼ] = indicator function = 1 if labels differ, 0 otherwise, β = contrast sensitivity parameter, ‖z_i − zⱼ‖² = squared feature difference (e.g., color contrast),

The first shape of the Adristyrannus, Aphids, and Beetles has smoother outlines, and the gradients of intensity across the watersheds were treated as a topographical surface. Watershed segmentation comes from identifying basins separated by watershed lines. Given a gradient function , the watershed transforms computes regions where

.

based on the gradient magnitude:

7

This is why the second column shows many fine contour boundaries across all insects. In the SLIC column, the grid-like superpixels come from clustering pixels using the original SLIC distance function:

8

Where

9

And

10

Here, G(x, y) = gradient magnitude image or topographic surface, ∇G = gradient, ∂G/∂x, ∂G/∂y = partial derivatives of gradient magnitude, ‖∇G‖ = gradient strength used to form watershed boundaries, D = combined SLIC distance metric, d_c = color distance, d_s = spatial distance, m = compactness parameter, S = approximate grid spacing between superpixel seeds, l, a, b = CIELAB color components, x, y = pixel coordinates.

Because SLIC enforces spatial compactness, the pests in the images are segmented into regular, tile-like superpixels that still align with their boundaries. Finally, Felzenszwalb’s graph-based segmentation (last column) produces irregular region boundaries by merging components based on the original formulation:

11

12

and the merging condition:

13

here, C = image component (region), E(C) = internal edges within component C, w(u, v) = edge weight measuring dissimilarity, Int(C) = internal difference of region C, Dif(C₁, C₂) = difference between two regions, |C| = size (number of pixels) in component C, k = scale parameter controlling region size.

This original formulation explains why the boundaries in the last column adapt naturally to the insects’ shapes and the background texture.

To avoid ambiguity between experimental settings, it is important to clarify that models trained on segmentation-enhanced images operate under a different input regime than those trained on raw RGB images. Segmentation alters the visual characteristics of the data by highlighting object boundaries, suppressing background clutter, and emphasizing foreground regions. As a result, performance differences observed between segmented and non-segmented pipelines should be interpreted as arising from differences in input representation, rather than being attributed solely to architectural design.

Applied classifiers

In the pest dataset, the behavior of each classifier can be directly related to its architectural variables and how these variables control the extraction of texture cues, color variations, occlusions, and fine-grained morphological details^22–26. A consistent training protocol was adopted across all architectures; however, input resolution was adjusted according to architectural capacity and common practice, with lightweight CNNs trained at 224 × 224 and higher-capacity and hybrid models trained at increased resolutions to preserve fine-grained pest details. Consequently, performance differences reflect both architectural design and input-resolution effects. All models used identical dataset splits, optimizer, loss function, learning-rate scheduling, regularization settings, and early stopping criteria unless explicitly stated otherwise. The hyperparameter ranges reported in Table 3 reflect architectural constraints rather than per-model tuning, and no model was granted additional training budget or specialized optimization.

Table 3.

Summary of training hyperparameters used across all architectures.

Hyperparameter	Value/Range	Rationale/Notes
Input image size	224 × 224 (default) / 300 × 300 (mid-size) / 384 × 384 (high-res)	Use 224 for lightweight models and fast training; increase to 300–384 for larger nets or hybrids to preserve small-object detail.
Batch size	32	Balanced memory/stability for common GPUs; reduce if OOM.
Optimizer	Adam	Good generalization and weight-decay support for transfer learning.
Learning rate (backbone/pretrained)	1e-5–5e-5	Small LR for fine-tuning pretrained conv layers.
Learning rate (head/new params)	1e-4–5e-4	Higher LR for newly initialized classifier/transformer layers.
Learning rate schedule	Cosine decay with linear warmup	Smooth warmup + decay stabilizes fine-tuning.
Weight decay	1e-2	Regularization to reduce overfitting.
Epochs	50 (with early stopping)	Train until convergence with monitored early stopping on val metric.
Early stopping patience	10–15 epochs	Prevents overfitting and saves compute.
Loss function	Cross-entropy (with optional class weights)	Standard for multi-class classification; apply class weights if imbalance persists.
Label smoothing	0.1	Reduces overconfidence, helps calibration.
Data augmentation set	Random horizontal flip; random vertical flip; random crop/rescale; rotation (small); color jitter; random brightness/contrast; optional MixUp/CutMix	Augmentations tailored to natural images; MixUp/CutMix improves robustness on small/inexact objects.
Normalization	ImageNet mean & std	Matches pretrained backbone statistics.
Dropout (classifier/FFN)	0.2	Regularizes final layers/FFN in transformer blocks.
Gradient clipping (norm)	1.0	Stabilizes training for large batches or deep nets.

Although Table 3 lists ranges for certain hyperparameters, these values were not freely tuned on a per-model basis. Input image resolution was fixed within architectural groups: lightweight and standard CNN models were trained at 224 × 224 resolution, while larger-capacity and hybrid CNN–Transformer models used higher resolutions (300 × 300 or 384 × 384) to preserve fine-grained pest details, following common practice. All models employed the same base data augmentation pipeline, and optional techniques such as MixUp and CutMix were either disabled or applied uniformly across all models in each experiment. All architectures were trained for a maximum of 50 epochs, with early stopping applied uniformly across models using the same validation-loss criterion and a patience of 10–15 epochs, such that training typically terminated earlier once convergence was reached. No model received additional epochs, data, or architecture-specific hyperparameter tuning beyond this protocol.

MobileNetV2 makes use of lightweight inverted-residual blocks with linear bottlenecks, which leads to a significant reduction in the parameter count and computational width. The design of MobileNetV2 allows the preservation of important edge and texture information even with the limit of a small receptive field, thus it is suitable for pests that have simple shapes or are in strong color contrast. Conversely, an insufficient channel limit and intermediate feature maps with a narrowness restrict the modeling capacity for complex spatial interactions. Therefore, MobileNetV2 (Fig. 5) has a harder time in a variety of scenarios-for instance, overlapping pests, high-frequency background clutter, or subtle interspecies differences with requiring greater contextual integration. From its deep yet identical stacks of 3 × 3 convolutions input, VGG16 represents radically different architectural biases. Although a very deep network without skip connections provides for representing more complex features, it also promotes background relationship memorizing. With refusal to use architectural regularization mechanisms like identity shortcuts, squeeze-and-excitation modules, and attention layers, the low-level textures present in the background of an image such as leaf venation, soil granularity, or shadow gradients-go deep into the hierarchy of features. This, in turn, adversely affects the capacity of VGG16 to maintain interclass separations under visually similar objects, and its bias towards background invariance weakens.

Fig. 5.

Architecture of MobileNetV2.

The architecture of EfficientNetB0 (Fig. 6) is a very well-thought and classified scaling procedure of depth, width, and input resolution, which permits increased diversification of feature learning while keeping computation costs and memory down. This scaling, based fully on compound scaling, acts to stretch the receptive field effectively at one hand while increasing discriminative capability of mid-level features at the other hand, so that species discrimination can be carried out even through very small adjustments across a pest species. This gets stretched gradually by EfficientNetB3, which goes even deeper and wider through the same compound scaling principle, yet providing more detailed anatomy such as antenna segmentation, thorax marking, and wing venation with accuracy. Larger architecture allows stronger connections among multi-scale features, crucial in differentiating species where the distinctions are based on very tiny, subtle-folded morphological characters.

Fig. 6.

Architecture of EfficientNetB0.

The ResNet50’s identity-based residual connections, as the name suggests, not only stabilize the gradient flow over the entire depth of the network but also preserve the low-level spatial information continuity. Thus, in case of the Network, it can detect pests hidden in very noisy biological substrates like, for instance, leaves with colors resembling the ones of the pests, which are considered as a type of camouflage. On the other hand, its downsampling strategy and large feature-map strides result in a very strong spatial smoothing. A great benefit here for the network would be the robustness, but this smoothing might also eliminate some very fine discriminative cues, such as minute color gradients, micro-patterning, or small appendage structures, leading to the occasional misclassification when the class boundaries depend on tiny, localized differences.

EfficientNetV2-B0 features a combination of improved training-aware scaling, fused-MBConv blocks, and optimized progressive learning which not only expands the stability across varying imaging conditions but also maintains the feature consistency. These features are effective in preserving the features even under different lighting, motion blur, mismatched angles and in the presence of cluttered backgrounds. The improved capacity-to-compute ratio is a major factor in helping the model generalize better when pest samples are taken under non-standard or noisy field conditions.

Xception and NASNetLarge are examples of another dimension of architectural diversity. By applying depthwise separable convolutions (SepConv), the model design of Xception (Fig. 7) has good boundary sensitivity that is good for post-processing. Despite that, the parameter count remains relatively low. This means that we can focus more to the fine boundaries of objects and hence the model can precisely tell pest outlines apart even if they are well sketched against complex leaf structures. On the other hand, NASNetLarge, which is the result of neural architecture search, utilizes cell-based motifs with wider filter sizes and bold multi-branch design. Although the model is very expressive, the occupation of high-frequency texture channels can lead to overfitting of the dominant background textures, thus causing confusion when pests are on visually intricate plant surfaces.

Fig. 7.

Architecture of Xception.

The hybrid architectures Hybrid InceptionResNetV2, Hybrid ResNet50 + CBAM, and Hybrid EffNet_Transformer are the ones who got their gain from the merging of convolutional backbones and attention-enhancing modules. Such models utilize still or moving architectural variables like channel-attention weighting, spatial-attention refinement, and multi-head global reasoning shift to divert focus from the environment around the pest to its body. They also improve the selectivity of the features, suppress the noise coming from the background, and assist in the localization of the target by making it more discriminative.

Inception-ResNet-v2, as shown in Fig. 8, is a robust deep convolutional neural network that can perform pest classification with great accuracy. Its training stability along with learning rich and multi-scale visual features are its major advantages. The network, which is a combination of residual connections and Inception modules, can capture the fine-grained features of the pests such as texture, shape, and their structural patterns which are necessary for the distinction of visually similar pest species. The reduction blocks have the effect of increasingly deeper features and at the same time, they compress the spatial information; this enables the hierarchical abstraction from low-level visual cues to high-level semantic representations. Residual learning also contributes to faster convergence and better generalization, which is especially valuable when dealing with limited or imbalanced agricultural datasets. Global average pooling and a softmax classifier are then employed alongside Inception-ResNet-v2, which not only provides but also requires robust and discriminative feature representations; thus, it is a strong backbone for accurate multi-class pest identification in real-time agricultural monitoring systems.

Fig. 8.

Architecture of InceptionResNetV2.

The Hybrid EffNetV2-S + Transformer configuration is achieved the strongest overall performance for offering such a high level of performance because of the mutual benefit of its architectural components. The EfficientNetV2-S backbone is the one that, thanks to the increased depth–width balance and fused-MBConv layers, can capture small-scale color transitions, texture irregularities, and micro-structural features with high resolution. To avoid the situation where the model only sees the local area, the Transformer provides a global receptive field where the long-range relational modeling can take place. This modeling is not possible through convolution alone. The resulting combination is such that even when the pests are obscured, out-of-focus, fragmented in space, or very similar to neighboring classes, the model’s classification remains true.

The differences in performance that were noted are from the mutual influence of the architectural variables such as depth, width, scaling of the receptive field, type of convolution, number of parameters, attention mechanisms, and the strategies applied for feature aggregation. These factors play a fundamental role in determining each model’s ability to pick up details of small objects, its tolerance to background variability, and its capability to separate pests with minor morphological differences.

To formalize the hybrid design integrating a convolutional backbone with a Transformer encoder, the following Algorithm 1 outlines the architectural construction and forward propagation steps used in the proposed Hybrid EfficientNetV2-S + Transformer model.

Algorithm 1.

Construction and forward pass of the hybrid EfficientNet–Transformer architecture.

Results and analysis

In this section, the experimental results of all evaluated models are presented, followed by a detailed analysis of their performance on the pest dataset. The aim is to compare how different architectural choices influence accuracy, robustness, and feature-extraction capability across various pest categories.

Overall performance comparison of models

In Table 4, the differences in performance across these models demonstrate the effectiveness of each model in exploiting fine-grained visual signals that may separate agricultural pests, those features that frequently conveniently range in size, texture, luminance, and background clutter.

Table 4.

Analysis of the models based on accuracy and loss.

Model Category	Model	Training Accuracy	Validation Accuracy	Training Loss	Validation Loss
CNN	MobileNetV2	0.8151	0.7601	0.5756	0.8262
	VGG16	0.7235	0.7431	0.8740	0.8785
	ResNet50	0.8512	0.8041	0.4543	0.7020
	EfficientNetB0	0.8642	0.8371	0.4214	0.5649
	EfficientNetB3	0.8820	0.8705	0.3675	0.4558
	EfficientNetV2-B0	0.8470	0.8382	0.4703	0.5389
	Xception	0.8566	0.8236	0.4385	0.6240
	NASNetLarge	0.8673	0.8469	0.4089	0.5829
Hybrid Networks	Hybrid InceptionResNetV2	0.8132	0.8095	0.5728	0.6484
	Hybrid ResNet50 + CBAM	0.9033	0.8158	0.2958	0.6878
	Hybrid EffNet + Transformer	0.9440	0.8488	0.1689	0.6363
	Hybrid EfficientNetV2-S + Transformer	0.9105	0.8800	0.2670	0.4560

While MobileNetV2 and VGG16 tend to show weaker generalization due to smaller representational power, as they can hardly capture fine-grained morphology details, and so they are more sensitive to noisy backgrounds that may come up with domain shifts in field imagery. On the reverse side, modern families such as EfficientNet, ResNet, Xception, and NASNet stand out owing to deeper integration with depthwise feature hierarchies; for instance, extreme scaling strategy, good convolutional blocks making better spatial patterns, etc. The main direction pursued for these architectures is to balance depth and efficiency, which also enables the modeling of the shape, color patterns, and structural variations of pests more robustly.

Indeed, hybrid architectures combining deep CNN backbones with attention mechanisms or Transformer modules have taken global contextual modeling to yet higher levels. Global models have the unique advantage of modeling all long-range dependencies, as required when a pest occupies a small region, or when discrimination of features depends on some larger context present in the leaf structure. Some of the hybrid models are sporting a symptom of overfitting, failing to generalize to validation data: their model capacity is simply too large. This too has the potential to make them remember the training data over the generalization, particularly if the dataset is imbalanced or limited in representations among the pest classes. Adding CBAM or Transformer layers usually results in models that are more interpretable with greater emphasis on regions of interest, resulting in detecting obstacles in complex environments. Taking this into account, the ones that strike the best balance between model capacity and regularization are generally more generalizable. In conclusion, the overall performance follows a progression: simpler CNNs are not very good at handling fine differences in pest data, strong modern CNNs do this very well, and hybrid attention-driven models would excel, provided they maintain the correct balance between expressiveness and generalization.

Learning curve behavior across architectures

A thorough analysis of epoch-based performances of these models in Fig. 9 reveals how model architecture might impact their ability to learn some of the most discriminative features of biological pest classification, whereas, on the contrary, large-scale variation in space, texture, and background complexity being the challenges. To clarify the slight instability observed in the accuracy and loss curves, the early stopping mechanism used in training was configured with a min_delta value of 0.001. This threshold determines the minimum improvement in validation loss required to continue training. Because min_delta was relatively large, training stopped earlier than optimal, before the curves fully stabilized. Using a smaller min_delta (e.g., 0.0001) would allow additional epochs and result in smoother convergence trends.

Fig. 9.

Learning curve of the applied models.

The Hybrid EfficientNetV2-S + Transformer shows a decent progression: training accuracy goes up well and the validation loss follows the same slope. Accompanied by consistently declining losses, it has displayed well-balanced performance. This trend indicates the capacity of hierarchical convolutional features with the world-wise attention to generalize quite nicely, as opposed to overtraining, for this model now. On the other end, the Hybrid EfficientNet + Transformer sees an increase in the gap between training and validation curves. Its training losses plummet very fast while the training accuracy takes off; the one for validation plateaus with slight oscillations. The high capacity of the hybrid design is a factor: it captures fine-grained patterns during training quite well; it instead really starts to tune memory to be trained-specific but still does not quite encompass the representation of those more benign features across extremely disparate field images.

NASNetLarge seems to generally generalize satisfactorily: both the accuracy and loss move up together at the beginning, while validation loss seems relatively unstable toward minor variations like minor invocation or background noise. The model is crafted from an architecture derived from an automated structure search; it shows occasional signs of being overfitted as seen in the loss in the validation set. An EfficientNetB3 is the first model so far that showed the most coherent and synchronized learning patterns among the CNN-based models. Both training and validation accuracy curves proceed almost parallel, demonstrating that the loss values are smoothly diminishing. Regarding this, EfficientNetB3 being designed on the principle of compound scaling master’s the enhancement of minute visual cues without becoming over-specialized with the training data. Taken together, this group of findings suggests that models that integrate balanced scaling or carefully tuned attention all tend to stabilize generalization in pest classification, while an over-parameterized model demands careful regularization to avoid learning dataset-specific artifacts, in contradiction to the underlying biological patterns.

Detailed performance evaluation using class-level metrics

In addition to this, Table 5 provides generalizable representations which produce balanced precision and recall across pest classes, while architectures that either lack capacity or overfit produce uneven trade-offs between false positives and false negatives. Here, precision reflects how many of the model’s predicted positive cases are correct, recall indicates how many of the true positive cases the model successfully identifies, and the F1 score, being the harmonic mean of precision and recall, summarizes the balance between these two measures. Together, these metrics provide a comprehensive view of each model’s reliability, particularly in situations where class distributions are uneven or certain pest categories are more challenging to distinguish.

Table 5.

Macro Precision, Recall, F1 score of models.

Model Category	Model	Precision	Recall	F1-Score
CNN	MobileNetV2	0.782	0.778	0.772
	VGG16	0.769	0.763	0.761
	ResNet50	0.815	0.805	0.806
	EfficientNetB0	0.846	0.845	0.844
	EfficientNetB3	0.872	0.871	0.870
	EfficientNetV2-B0	0.845	0.844	0.842
	Xception	0.838	0.835	0.834
	NASNetLarge	0.834	0.832	0.831
Hybrid Networks	Hybrid InceptionResNetV2	0.816	0.813	0.812
	Hybrid ResNet50 + CBAM	0.815	0.806	0.804
	Hybrid EffNet + Transformer	0.848	0.844	0.843
	Hybrid EffNetV2-S + Transformer	0.865	0.856	0.849

Since these scores are macro-averaged, they give equal weight to each pest class- So a model might see itself achieve better scores when it maintains residual performance both with rare and common pests as compared to those which do not perform well in the dominant categories. The best-balanced precision and recall, hence, F score, are delivered by the latest and compound-scaled CNNs (EfficientNet family) and the best hybrid that exhibited similar training/validation behavior earlier, endorsing that there are fewer spurious classifications successfully contained by the multi-class instances-such is what we really need in a multi-class pest detector. The allowing regularity of their loss/accuracy profile shows that good feature extraction has taken place without the memorization of the dataset peculiarities, and this stays as consistently high macro metrics.

Some hybrid models seemed to have learned the training data very quickly; however, they were not able to generalize well on the validation set. Despite the gap in poor precision and recall, they still show sufficiently strong precision to ensure that the models are trained very well. This is frequently seen where the model becomes too confident on training examples and hence ends up missing or underestimating harder or less common classes from inferences. Few models, whose overall scores are generally lower, become more likely to misclassify during real-world applications. They are not good at extracting features well when en-tangled with the background, or for unassuming and subtle interfaces between classes: a temptation to both false positives and false negatives. Models in the middle class like those known to one and all by their names, say ResNet, Xception, and NASNet very much exist between these poles. They gain a limited extent of regularity in the representation of a few structural configurations, but still, without much stronger regularizers or more variety in the data for training, those don’t outperform more complex and intricate architectures.

However, in each confusion matrices, the models shown in Fig. 10 follow a consistent pattern which states that they maintain high diagonal values with few off-diagonal errors, showing their ability to differentiate reliable pest species despite the minor differences that exist among classes.

Fig. 10.

Confusion matrix of the applied models.

EfficientNetB3 displays the most consistent conduct, nearly all classes are given high correct classification rates with only small distribution into visually alike categories. This is in accordance with its smooth learning curves and balanced macro metrics, proving that its compound scaling lets it to capture fine-grained morphological cues like wing venation, body segmentation, and texture—that distinguish pests under varying field conditions. On the other hand, NASNetLarge, which is still performing well, shows a bit more scattered misclassifications, this is especially true for pests that have overlapping visual traits or similar color patterns. This is like the ups and downs in its validation loss: even though it has great capacity, its predictions get less stable for classes with lower training representation or higher intra-class variability. The hybrid model integrating EfficientNet features with attention mechanisms presents a similar trend: it performs strongly on many classes, but its errors are somewhat more scattered than those of EfficientNetB3. This reflects what was observed earlier in its training curves—its high expressive power improves representation learning, but mild overfitting causes certain classes to be confused more often, particularly where contextual cues rather than localized patterns are required for distinction.

The ResNet50 + CBAM model is the one that most frequently misclassifies, and the confusion is spread across pest types in several clusters. The use of CBAM in the network enables it to pay attention to the regions that are most important, but the backbone corresponding to it is not as successful as the modern or hybrid architectures in capturing the fine-grained differences. This is in line with its average performance on the macro scale and the train–validation gaps noted earlier.

In summary, these confusion matrices serve to strengthen the explanation interpreted by the learning curves and macro metrics: the models with balanced scaling or careful integration of attention (especially EfficientNetB3 and the well-regularized hybrids) are the ones that generalize more reliably among the different pest species, while the architectures that are either prone to overfitting or limited in representational depth produce errors that are more scattered and class-specific.

Table 6 summarizes how different deep-learning models perform when distinguishing among multiple insect and pest species, with the focus on identifying which class each model predicts most accurately based on precision, recall, and F1-score.

Table 6.

Class-wise performance of models.

Pest Category	Precision	Recall	F1-score	Models where category is top performing
Mediterranean fruit fly	0.993	1.000	0.964	EfficientNetB0, EfficientNetB3, NASNetLarge, Hybrid EffNet
Weevil	0.980	0.987	0.958	VGG16, ResNet50, Hybrid ResNet50 + CBAM
Beetle	0.980	0.960	0.970	MobileNetV2
Aphids	0.952	0.940	0.936	VGG16
Cicadellidae	0.973	0.960	0.973	EfficientNetB3
Field Cricket	0.974	0.987	0.980	EfficientNetV2-B0
Red Spider	0.993	0.980	0.987	Hybrid InceptionResNetV2
Thrips	0.974	0.980	0.977	Xception
Whitefly	0.986	0.993	0.980	NASNetLarge
Earwig	0.993	0.970	0.969	Hybrid EffNet_Transformer
Grasshopper	1.000	0.940	0.958	Hybrid ResNet50 + CBAM

For each model, the pest class associated with the highest precision, highest recall, and highest F1-score is reported to highlight the species that the model recognizes most reliably. Notably, several pest categories such as Mediterranean fruit fly, Weevil, and Beetle—recur as top-performing classes across multiple architectures, indicating consistent recognition strength rather than model-specific bias. This repeated dominance can be attributed to factors such as balanced sample representation, distinctive morphological features, strong contrast against background foliage, and comparatively lower intra-class variability, which facilitate robust feature learning across different network designs. The metrics capture complementary aspects of recognition quality: precision reflects the correctness of predictions for a given class, recall measures the completeness of instance retrieval, and the F1-score provides a balanced summary of both. In contrast, pest categories with higher visual similarity or greater intra-class variation (e.g., thrips, aphids, and beetles under cluttered backgrounds) appear less consistently as top performers, reflecting intrinsic dataset difficulty rather than architectural limitations.

Confidence-based reliability assessment

The per-class confidence value in Fig. 11 describes two complementary aspects of EfficientNetB3 behaviour for each pest: how often the model is correct (class accuracy) and how confident it is when it predicts (mean and spread of confidence for true predictions and for the top-1 prediction regardless of correctness). Reading these together reveals whether the model is (a) reliable and well-calibrated for a class, (b) underconfident, or (c) overconfident and brittle and those patterns tie directly back to the learning-curves and confusion matrices.

Fig. 11.

Per-class accuracy and confidence across pest categories using EfficientNetB3 model.

First, a few classes show high accuracy (green color) together with high mean true confidence (blue color) and low true-confidence variance (light blue color). For these classes, the model both knows and knows that it knows: predictions are correct, consistently confident, and rarely scattered to other labels in the confusion matrices. This indicates clean, distinctive visual features (or plentiful representative examples) that the feature extractor learned robustly. Second, another set of classes have moderate accuracy but still display reasonable mean true confidence (blue), while their top-1 confidence (orange) is noticeably higher than the mean confidence on true predictions. That gap is a hallmark of overconfidence on incorrect predictions: the model often makes high-confidence mistakes for these classes. In the confusion matrices, these classes appear as sources or sinks of off-diagonal errors (confusions with visually similar pests). Practically, this pattern means the model forms sharp decision boundaries around noisy or overlapping feature regions which means it is decisive, but sometimes decisively wrong. Third, several classes have low accuracy (green), low mean true confidence (blue), and high top-1 and true-confidence variability (red and light blue). These classes are both uncertain and inconsistent: the model is frequently wrong, and when it is correct its confidence varies substantially. In the confusion matrices, these classes tend to have diffuse error patterns (confused with many other labels), and in the learning curves they align with cases where validation instability or class-specific variance is observed. Contributing factors include limited or low-diversity training samples, high intra-class variability (pose, background, life-stage differences), or strong visual overlap with other pest species.

These “highest-confidence scores” offer a visual explanation for the quantitative patterns observed earlier across accuracy, confusion matrices, and per-class confidence statistics (Fig. 12).

Fig. 12.

Confidence scores associated with images generated by (a) EffNetV2S+Transformer (b) EfficientNetV2-B0.

For both models i.e. EffNetV2-S + Transformer and EfficientNetV2-B0, the correctly classified, high-confidence images share a set of clear visual properties: strong contrast between the pest and its background, distinctive morphological cues (body shape, color patterning, segmentation, wing structure), and minimal occlusion. When such cues are present, both architectures respond with near-perfect certainty, which aligns with the very high true-class confidence values shown in the per-class confidence table for classes like snail, Mediterranean fruit fly, cutworm, or cabbage looper. The consistency of these high-confidence examples also helps explain why certain classes repeatedly ranked as top performers in macro precision/recall, why their confusion-matrix diagonals were nearly perfect, and why their confidence distributions had low variance. During these classes, the model gets to look at examples of the species that are the “canonical” visual forms, thus, the deep learning backbone can distinguish even small differences in texture and shapes while the Transformer or the improved feature-scaling layers are working to maintain a global consistency across the picture. This teamwork is mainly noticed in classes with very strong outline identity e.g. snail, earwig, weevil where the two models have stable and clear predictions. Meanwhile, these pictures are also revealing the reason why some classes were more difficult: the examples of the highest confidence are surprisingly clean in comparison with those of the classes that have lower accuracy and confidence variance. Insects such as beetles, thrips, mites, or aphids, for instance, usually have their dataset samples mixed up with a lot of other things, have individuals of various sizes overlapping each other, or have very low contrast. The best cases of these insects are still confidently detected, but they make it very clear what the model means by “looking for” highly visible structure and minimum background noise which is a condition that is not always the case in real field imagery. This observation is directly linked to the previous finding that some classes have their model confidence overestimated on the hard or noisy samples or have their validation performance fluctuating.

In summary, these exemplary images are qualitatively confirming the numerically observed trends: classes with clean, distinctive visual signatures consistently driving high confidence and high accuracy across the models, while visually ambiguous classes challenging even advanced architectures. The visuals for practical improvement reveal the necessity of diversifying training data to include more difficult, field-realistic cases and of applying targeted augmentation or calibration techniques to ensure that the model’s confidence better reflects its uncertainty in less ideal conditions.

Discussion

The comparison of classical CNNs, modern deep architectures, and hybrid CNN–Transformer models in this study brings out the fact that the choice of architecture has a very significant influence on the capacity of a model to recognize pests that are very small, overlapped or partly hidden in the real environment of agriculture³⁰. The strengths of VGG16 and MobileNetV2 have even been more clearly seen in that they effectively reveal the limitations of older CNN designs that are deprived of modern regularization and attention mechanisms and also limited in terms of depth and channel capacity. Consequently, such models usually have a hard time ensnaring subtle morphological features and are more prone to noise from the background and slight variations between classes. On the other hand, contemporary compound-scaled architectures like EfficientNetB0/B3 and EfficientNetV2-B0 provide good generalization and stability via the adjustment of depth, width, and input resolution just this way balancing. Likewise, networks such as ResNet50, Xception, and NASNetLarge demonstrate that residual connections, depthwise separable convolutions, and architecture search can significantly enhance multi-scale feature learning.

The best performance is achieved by hybrid models that combine convolutional backbones with attention mechanisms or transformer components. CNNs enable highly accurate local feature extraction, while transformers effectively capture global contextual information, leading to improved discrimination among visually similar pest species. This advantage is clearly demonstrated by the Hybrid EfficientNetV2-S + Transformer model, which achieves the highest classification accuracy and exhibits superior robustness across pest categories. Furthermore, segmentation analysis reinforces these findings, showing that architectures enhanced with attention mechanisms can more effectively focus on pest regions while reducing the influence of complex background noise. Recent CNN–Transformer hybrid frameworks have shown strong performance in fine-grained visual recognition tasks. Investigating additional CNN–Transformer architectures on pest datasets therefore represents a promising direction for future research. However, due to computational constraints and the scope of the present study, such experiments are deferred to future work.

The advantageous findings nevertheless accompanied limitations that somewhat restricted their general applicability. The dataset is diverse but lacks consistency in visual quality. In cases of some pest species, their presence is denoted by clear, high-contrast images while for others, their capturing is done in poor lighting, under heavy occlusion, or amid cluttered backgrounds. Such inconsistencies make certain autotrophic classes difficult to identify, which leads to a noticeable decline in accuracy and a rise in confidence variability, especially for pests like thrips, mites, beetles, and earwigs. Although their performance is second to none, high-capacity hybrid models might eventually get overfitted if trained on little or unbalanced data samples. Besides, the models are still prone to domain shifts like changing regions, different camera types, or varying crop environments, which might lead to a decrease in performance when used in the field unless domain adaptation is employed. The segmentation techniques used in this research were applied as a preprocessing step to generate segmented versions of the images, and the classifiers were trained using these segmentation-enhanced images. Lastly, the study emphasizes single-image classification, overlooks temporal information and multi-view perspectives, both of which could be crucial in real-time pest monitoring and more detailed field diagnostics. The above-mentioned drawbacks also put forth several attractive areas for future investigations. One of the ways to reduce the models’ sensitivity to domain shifts and to support their better generalization is by making the dataset more diverse through the inclusion of images from various parts of the world, various seasons, different crop types, and various camera devices. Data-centric methods such as targeted augmentation, class rebalancing, generative creation of rare pest samples, and curriculum learning can still be used to enhance performance, especially for hard classes. On the architectural side, future research might investigate fully integrated segmentation-classification pipelines, Transformer-based detectors, or multi-stage networks that simultaneously capture fine morphological details and broader ecological context. Knowledge distillation, among others, might help to generate lightweight versions of the high-performing hybrid models for drone, smartphone, and low-power IoT device deployment in the field. It is possible through multi-sensors data fusion to achieve an improved interpretation of pest behavior by combining RGB images with hyperspectral measurements, pheromone traps, or environmental sensors. Besides, the integration of these models into farmers’ decision-support systems that consider the weather patterns, crop growth stages, and pest life-cycle information would not only make pest management predictive and proactive but also enhance the effectiveness of precision agriculture.

Although the segmentation outputs helped visualize boundary structures and improve interpretability, in the present study the segmentation techniques were also applied as preprocessing to generate multiple transformed versions of the dataset prior to model training. Therefore, segmentation functioned as a dataset-level enhancement rather than a post-hoc visualization tool. However, since no controlled ablation experiment was performed to compare raw versus segmented training regimes, the precise performance contribution of segmentation cannot be quantified within this study and will be addressed in future work.

Conclusion

In this research, various deep-learning architecture for multi-class pest classification was evaluated extensively and the impact of various computational designs on the performance of the real agricultural images was clearly shown. The results of comparing the classical CNNs, modern compound-scaled networks, residual architectures, NAS-based models, and attention-enhanced hybrid frameworks revealed that effective pest classification is dependent on both the strength of the feature extractor and the model’s skill in uniting fine details with broader contextual information. The results underline that lightweight traditional networks are often unable to differentiate subtle characteristics among similar pest species while modern and hybrid architectures extract richer multi-scale features to be far better at the task of differentiating between pests with closely related visual characteristics.

Out of all the models tested, the Hybrid EfficientNetV2-S + Transformer gave the best and most reliable results. The model was able to classify pests correctly in a wide range of images, thanks to the combination of hierarchical convolutional features and the global relationship capturing ability of the Transformer. Different methods of evaluation classification metrics, confusion matrices, confidence analyses, and segmentation-based visual checks were all in agreement that the integration of convolutional efficiency with attention-based adaptability resulted in the best-performing models. The results suggest that the combination of CNN and Transformer components in a balanced architecture is a reliable method for the identification of pests with high accuracy. This, in turn, lays the groundwork for future agricultural monitoring systems that can be eco-friendly by supporting the sustainable protection of crops, reducing the use of chemicals and improving decision-making in the field of precision agriculture.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

V.K.V conceptualized the research problem, prepared the dataset, implemented the hybrid attention-based CNN models, and conducted the experiments. A.K. supervised the study, guided the design of the model architectures and attention mechanisms, validated the experimental outcomes, and coordinated all manuscript revisions. V.P. contributed to lesion segmentation, parameter optimization, statistical analysis, and the preparation of performance visualizations. Y.K. contributed to feature engineering and attention module integration, critically reviewed the manuscript and supported the refinement of results and conclusions. All authors reviewed and approved the final manuscript.

Funding

Open access funding provided by Manipal University Jaipur.

Data availability

The datasets analysed during the current study are available in the repository [https://www.kaggle.com/datasets/ibrahimagabardiop/pestaidatasetv2].

Declarations

Competing interests

The authors declare no competing interests.

Ethics

This research did not involve any studies with human participants or animals performed by any of the authors.

Consent to participate

Informed consent was obtained from all individual participants involved in the study.