A zero-shot learning framework for chilli leaf disease detection, classification and severity estimation using contrastive image text representations

Abstract

Agricultural productivity is severely affected by the late identification of chilli leaf diseases, which often require manual inspection and frequent model retraining. Conventional deep learning models perform well on known classes but fail to recognize unseen or emerging disease patterns. To address this limitation, we propose a Zero-Shot Dual-Encoder Framework that integrates a Vision Transformer (ViT) and RoBERTa-based semantic encoder to identify unseen disease categories and estimate their severity using lesion area ratios. The proposed model was evaluated on a curated chilli leaf dataset under various environmental conditions, achieving 98.7% accuracy, 98.0% precision, and 98.0% recall, outperforming five existing state-of-the-art approaches. Experimental findings demonstrate that the framework not only enhances recognition accuracy but also ensures scalability for real-time agricultural disease monitoring without the need for retraining.

Introduction

Chilli (Capsicum annuum L.) underpins culinary traditions and export economies across Asia, Africa, and Latin America, with India among the largest producers and exporters (Madhavi et.al.2023)¹. Field productivity is threatened by diverse biotic stresses notably anthracnose, Cercospora leaf spot, powdery mildew, leaf curl, bacterial spot, and mosaic virus that degrade both yield and quality (Mahmoud, G. A. E. et. al. (2021))². Early and reliable diagnosis is therefore a pivotal step for timely management and loss mitigation (Delai, et.al. (2024)³. Manual scouting by experts is difficult to scale and prone to inconsistency in large farms, motivating computer-aided diagnosis (CAD) with machine learning (ML) and deep learning (DL) (Qadri, et. al. (2024)⁴. Convolutional neural networks (CNNs) have shown strong performance for leaf disease classification, yet they typically require sizeable labeled datasets, careful class balancing, and periodic retraining to incorporate novel diseases (Lu, J., et. al. (2021))⁵. Furthermore, most published systems provide only categorical predictions (disease type) and omit severity estimation, which is essential for prioritizing interventions, pesticide budgeting, and forecasting yield impacts (Amulothu,et.al. (2024))⁶. Recent advances in Vision Transformers (ViT) and hierarchical variants (e.g., Swin Transformer) have achieved state-of-the-art results in fine-grained recognition by capturing long- range context alongside local texture (Rahman, et. al., (2025); Velpula, et. al. (2025); Kalpana, et. al. (2024))^7–9. In parallel, contrastive vision–language learning (e.g., CLIP) demonstrated that aligning images with natural-language descriptions yields highly transferable embeddings and enables zero-shot recognition (Javed et. al. (2024); Radford, et. al. (2021); Wang, et. al. (2022))^10–12. Prompt-based and language-guided techniques further improve usability in open-world settings where the label space evolves over time (Paul, A., & Machavaram, R. (2024); Cao, et. al. (2025))^13,14. In the chilli domain, EfficientNet variants and attention-augmented CNNs have reported strong accuracy under controlled conditions, while lightweight MobileNet-style networks make real- time deployment feasible on edge devices (Tadavarthy, et. al. (2025); Rani, et. al. (2025); Amjad, et. al. (2023))^15–17. Ensemble strategies can push accuracy even higher but at increased computational cost and with limited extensibility (Rane, et. al. (2024))¹⁸. Importantly, robustness in uncontrolled field images, support for unseen diseases, and integrated severity outputs remain underexplored in chilli-specific research. This work introduces a unified chilli leaf analysis pipeline that: (i) uses dual encoders and a contrastive objective to learn a shared image–text space for multi-class and zero-shot classification, and (ii) estimates severity via lesion segmentation and a lesion-to-leaf area ratio. The approach reduces the dependence on exhaustive labeled data, adds plug-and-play extensibility via textual prompts, and outputs an interpretable severity level for management decisions. Empirical results on a composite dataset curated from public sources confirm competitive performance against recent benchmarks while providing the added benefits of zero-shot recognition and severity estimation.

Motivation

Despite the growing body of research on chilli disease detection, existing methods face persistent limitations in adaptability, scalability, and interpretability. Most approaches rely heavily on large, balanced datasets for training, limiting their ability to generalize to new or rare disease classes. In practice, new diseases or variants can emerge rapidly, and obtaining sufficient labeled data is both costly and time-consuming. Additionally, while classification accuracy remains the primary evaluation metric, there is limited focus on severity estimation, which plays a crucial role in decision-making for disease management. Severity information helps farmers prioritize interventions, allocate resources efficiently, and prevent economic losses. Therefore, there is a pressing need for a robust, zero-shot capable model that not only identifies multiple chilli diseases but also estimates the severity level with high precision.

Major contributions

The proposed work stems from the two critical gaps in the current state of chilli disease detection systems. It includes: Adaptability to Novel Diseases: Most existing deep learning approaches cannot classify diseases absent in the training dataset without complete retraining, making them unsuitable for real-world agricultural environments where new pathogens emerge. Severity Awareness: While classification identifies the disease type, it provides no insight into how advanced the infection is, which is crucial for disease management and yield prediction. The proposed method addresses both issues by incorporating zero-shot capability through contrastive image-text learning and severity estimation via lesion analysis. This dual approach aims to empower farmers and agronomists with actionable intelligence, reducing the need for constant data collection and retraining. The major contributions of this proposed work are as follows:

• In order to perform chilli disease detection and classification, a zero-shot capable chilli disease detection framework using a dual-encoder architecture combining a Vision Transformer (ViT) and a transformer-based text encoder (RoBERTa) (Rane, et. al. (2024))¹⁸ has been proposed.

• A lesion-based severity estimation module that computes the lesion-to-leaf area ratio for quantitative severity assessment has been incorporated.

• Additionally, the proposed work incorporates robust preprocessing techniques including CLAHE, resizing, and conditional VAE-based denoising (Azizah, et. al. (2023))¹⁹ to improve model generalization under field conditions.

• Benchmarks are chosen for the proposed framework against state-of-the-art models with respect to performance metrics such as accuracy, F1-Score, Precision and Support and AUC-ROC.

• Dataset has been collected comprising of chilli leaf images from Mendeley, Kaggle plant disease datasets and are curated to ensure five major classes such as Bacterial Spot, Cercospora Leaf Spot, Curl Virus, Nutritional Deficiency and Healthy (M P, Aishwarya; Reddy, Padmanabha (2024)²⁰.

Organization of the article

The proposed article has been organized as follows: Section I deals with the abstract, motivation, major contributions and organization of the article which is then followed by the related work along with the comparative analysis which involves problem addressed, solution methods, benefits and limitations. Section “Materials and methods” provides the details on the data acquisition, data pre-processing, data feature extraction, data splitting, data training, testing and validation methods. Section “Experimentation & performance metrics” discusses the details on experimentation, performance metrics, simulation parameters and Sect. “Performance analysis” provides the performance analysis of the proposed system in terms of performance metrics and Sect. “Comparative analysis of the proposed vs existing models” provides details on the comparative analysis made between the proposed work to that of the existing benchmarks. Section “Conclusion & future work” provides the conclusion. Table 1 provides the chronicle of abbreviations utilized in the proposed system.

Table 1.

Chronicle of abbreviations.

Abbreviation	Description
ZSL	Zero-shot learning
ViT	Vision transformer
CLAHE	Contrast limited adaptive histogram equalization
cVAE	Conditional variational autoencoder
CNN	Convolutional neural network
RoBERTa	Robustly optimized BERT pretraining approach
AUC-ROC	Area under the receiver operating characteristic curve
CAD	Computer-aided diagnosis
TrP, FaP, FaN, TrN	True positive, false positive, false negative, true negative
AI. DL, ML	Artificial intelligence, deep learning, machine learning
Yolo, ResNet	You only look once, residual networks
SVM, KNN	Support vector machine, K-nearest neighbor
MSFEN, VGG	Multi-scale feature extraction network , visual geometry group
ViT, LSTM	Visual transformer, long short term memory networks
RDDNet	Residual nested dilated DenseNet,
GLCM	Gray level co-occurrence matrix

Related works

Over the last few years, chilli leaf disease detection and classification have gained significant attention in the computer vision and agricultural AI research community. From 2022 to 2025, multiple studies have attempted to address challenges such as class imbalance, high computational overhead, varying environmental conditions, and lack of severity estimation. Broadly, the literature clusters into (i) Transfer-learning pipelines (e.g., EfficientNet variants), (ii) Lightweight models for on-device inference (MobileNet/YOLO), (iii) Attention-enhanced CNNs and transformer backbones, (iv) Ensembles for peak accuracy, and (v) Early attempts at severity/intensity grading. Despite progress, most chilli-specific works still assume closed-world labels; zero-shot recognition of unseen diseases and integrated, quantitative severity outputs remain relatively rare. Below is a detailed survey of relevant works.

Pratap & Kumar (2023)²¹ has come up with a solution for the problem of robustness when faced with varying environmental and illumination conditions in chilli leaf disease detection systems. To address the problem, a customized EfficientNetB4 deep learning model along with the Guassian filter-based noise removal, CLAHE for contrast enhancement and data augmentation has been carried out in order to balance the dataset. Their proposed method has been implemented and compared with the baseline models such as CNN and ResNet architectures under controlled conditions. Though their methodology achieves an efficiency of 92% accuracy, the performance degrades when lighting is not controlled. Hamim & Jony (2024)²² addressed the problem of high computational cost concerning low-resource constrained deployments in the existing CNN architectures. Their system involves the use of a modified MobileNet architecture optimized for chilli disease detection and classification thereby reducing the parameters and memory required. Though their model has achieved a significant importance with an accuracy of 97.18% it has not addressed problems such as class imbalance or severity estimation. Naik et.al., (2022)²³ came up with a solution to address the problem of early detection of chilli diseases. Also they tried to address the issue of robustness while handling multiple classes in the dataset. Their system employed Squeeze-and- Excitation CNN model along with data augmentation for multi-class classification. Attention mechanisms were utilized in order to achieve improved feature extraction. However, their method has not been tested under varying environmental conditions and the severity estimation has not been carried out. Kukreja & Madan (2023)²⁴ addressed the problem of severity estimation of chilli leaf diseases. Their methodology involves the use of hybrid CNN and SVM for precise intensity detection in red chilli diseases. Their model integrates both detection and severity estimation into their framework thereby achieving 89.15% classification accuracy. However, their model exhibits low robustness in varying illumination and background conditions. Li et.al., (2024)²⁵ focused on the need for multi-scale feature extraction in order to improve chilli disease recognition. Their model introduced a Multi-Scale Feature Extraction Network (MSFEN) for capturing enhanced spatial patterns by means of attention-based mechanism. Their model outperforms the traditional swin transformers thereby achieving an accuracy of 93.5% accuracy. However, their model is computationally intensive and lacks zero-shot capability. Adnan et.al., (2024)²⁶ focused on the problem of optimization using a deep learning model across all chilli disease categories. To address the problem, an ensemble model which combines EfficientNetV2B0 and EfficientNetB7 thereby improving the prediction accuracy of 99.56% and outperforms the individual models. The proposed ensemble approach increased computational requirements, making it less suitable for resource-limited deployments. Entuni et.al., (2023)²⁷ addressed the need for low-cost detection model for chilli leaf diseases in a small-scale farm. Their methodology involves the use of CNN based on a low-cost Kinect camera in order to detect the multiple chilli diseases. Under controlled conditions their methodology has achieved a classification accuracy of 99.82%. However, under diverse field conditions, their method fails due to propagation error. Additionally, severity estimation was absent. Mayalekshmi et.al., (2023)²⁸ addressed the problem of detecting varying disease patterns in chilli crops under real-time object detection frameworks. Their system utilized a YOLOv5 model for chilli disease detection and analysis. Their system achieves faster, real-time disease detection in the field environments and a lower precision of 0.461 when compared with the existing classification models. No severity estimation has been carried out. Rozlan, S., & Hanafi, M. (2022)²⁹ addressed the problem of chilli leaf disease detection under uncontrolled imaging conditions. Their system utilized a combination of VGG16, InceptionV3, and EfficientNetB0 under varying light conditions. Their system achieved an accuracy of 98.83% on InceptionV3 along with data augmentation. Their model achieves robustness in zero-shot or unseen class scenarios were not tested. Manoj et.al., (2024)³⁰ has come up with a solution to address the problem of early-stage disease detection in chilli plants. Their system utilized three categories of YOLO models such as V5, V6 and V7. Under real-time agricultural environment settings, their methodology has achieved a classification precision of 0.82 thereby exhibiting strong performance in early-stage disease detection scenarios. Kursun and Koklu (2024)³¹ has come up with a solution to address the problem of chilli disease detection and classification. Their methodology involves the use of a combination of ViT with AlexNet (ViT-AlexNet). Their system has been implemented on curated datasets and exhibits strong performance with respect to plain CNNs. However, their methodology has not taken into consideration of severity estimation or open-set evaluation. Srinivasulu and Maiti (2007)³² addressed the problem of multi-class classification problem with the objective to address scalability or extensibility. Their proposed system involves the use of Residual Nested Dilated DenseNet using improved receptive field coverage and achieved 98% of classification accuracy of 98% on six major classes. However, their system still needs to be tested under real-time environments and suffers from scalability. Sivashankar et.al., (2007)³³ addressed the problem of chilli disease detection and classification along with severity estimation. Their system involves the use of integration of conditional VAEs (for denoising) and Swin transformer along with LSTM. Their methodology achieves a classification rate of 99% on curated sets highlighting a useful pre-processing route. However, the dataset with a closed label set is utilized which undermines the capability. Asha Patil and Kalpesh Lad (2022)³⁴ have come up with a feature selection technique based on GLCM algorithm for chilli disease detection and classification. Their system involves the use of feature selection along with CNN and the target values are given as input to the SVM and KNN classifier. Their proposed method has been tested using 2500 samples of data and SVM achieves a classification accuracy of 87.04% and KNN of 94.04% using the k-fold method. However, the limitation of the proposed system is moisture variation, distance variations, lightning and environmental constraints has not yet been addressed. Kaur et.al., (2023)³⁵ addressed the problem of chilli disease detection and classification using federated learning and CNN for multiple classes. Their methodology involves dimensionality reduction, federated averaging methods thereby achieving a trade-off between accuracy and recall with an accuracy rate of 94%. However, it has to be tested yet for real-time scenarios. Table 2 provides the comparative analysis on various research works published by several researchers along with their problem addressed, solution/methodology, benefits and limitations.

Table 2.

Comparative analysis of chilli disease detection and classification techniques.

Ref No	Problem addressed	Solution	Dataset & settings	Severity estimation	No of classes	Benefits	Limitations
Pratap, V. K., & Kumar, N. S. (2023)21	Chilli leaf disease detection & classification	EfficientNetB4 + CLAHE + Augmentation	Curated (Lab/Greenhouse)	No	5	Accuracy is 92%	Performance degrades under field variation; Severity Estimation was not carried out
Hamim, S. A., & Jony, A. I. (2024)22	Chilli leaf disease detection & classification	Modified MobileNet (edge)	Curated + edge focus	No	4–6	Accuracy is 97.1%-97.2%	Data Imbalance still persists No severity estimation has been carried out
Naik, B. N et.al., (2022)23	Chilli leaf disease detection & classification	SECNN + Augmentation	Curated	No	5	Accuracy is 99.1%	Field robustness remains untested
Kukreja, V., & Madan, K. (2023)24	Chilli leaf disease detection & classification	CNN + SVM (Intensity)	Red Chilli + Severity set	Yes	6	Accuracy is 89.1%	Proposed method is extremely sensitive to illumination/background conditions
Li et.al., (2024)25	Chilli leaf disease detection & classification	MCCM (multi- scale + attention mechanism)	Curated	No	6–8	Accuracy is 93.5%	The proposed method is computationally intensive
Adnan et.al., (2024)26	Chilli leaf disease detection & classification	Ensemble approach (EfficientV2B0 + B7)	BARI/Curated	No	4	Accuracy is 99.6%	Exhibits computational overhead
Entuni et.al., (2023)27	Chilli leaf disease detection & classification	Kinect + CNN (Low-cost)	Controlled capture	No	3	Accuracy is 99.8%	The proposed method struggles with unclear field generalization
Mayalekshmi (2023) 28	Chilli leaf disease detection & classification	Yolov5 (Detection)	Field Detection	No	2–3	Precision is 0.461	Exhibits lower precision; No severity estimation is carried out
Rozlan, S., & Hanafi, M. (2022) 29	Chilli leaf disease detection & classification	VGG16/ Inceptionv3/ EfficientNetB0	Uncontrolled Imaging	No	3	Accuracy is 99.8%	Proposed method involves a closed label set; No severity estimation has been carried out
Manoj et.al., (2024) 30	Chilli leaf disease detection & classification	Yolov5/v6/v7	Field Detection	No	2	Precision is 0.82	Proposed system has to be tested across varying lightning and environmental conditions
Kursun, R., & Koklu, M. (2024)31	Chilli leaf disease detection & classification	Hybrid (ViT- AlexNet)	Cross-Crop	No	Plant Stress	Multi-Scale Features are extracted better	Not a chilli specific method
Srinivasulu, M., & Maiti, S. (2024)32	Chilli leaf disease	RNDDNet (dilated	Curated	No	6	Accuracy is 98%	Suffers from generalization

From the Table 2 it is evident that the chilli-focused classifiers now routinely surpass 90–99% accuracy in curated or semi-controlled settings. However, practical gaps still persist with respect to open-world adaptability (adding or recognizing unseen diseases without retraining); robustness under uncontrolled field conditions; and consistently reported, quantitative severity outputs. These gaps motivate us to design a zero-shot, contrastive image–text framework with an integrated severity estimator, designed to be promptable, extensible, and field-practical.

Materials and methods

Dataset description

A comprehensive chilli leaf dataset was developed using field-collected and publicly available images representing Bacterial Spot, Cercospora leaf spot, Curl virus, Nutritional deficiency, and Healthy classes. The dataset includes 3,600 images captured under variable illumination, orientation, and background conditions. Images were resized to 224 × 224 pixels and enhanced using CLAHE and Gaussian filtering for contrast normalization. Data augmentation techniques such as rotation (± 30°), horizontal flipping, brightness adjustment, and random cropping were applied to increase intra-class diversity and mitigate overfitting. The dataset was divided in a 70: 20:10 ratio for training, validation, and testing, respectively. Each image was annotated by two independent experts to ensure class reliability. Figure 1 illustrates representative samples of the pre-processed dataset for all five classes.

Fig. 1.

Diseased leaves of the chosen classes of diseases.

Data acquisition

The dataset was compiled from Mendeley Plant Leaf Disease Dataset and Kaggle’s PlantVillage extensions that include chilli-specific classes. Additional images were collected from public agricultural repositories to ensure variety in disease manifestation and background conditions (Vatresia et. al. (2024); Fernandez, et. al. (2020))^36,37. The final dataset contains five categories: Bacterial Spot, Cercospora Leaf Spot, Curl Virus, Nutritional Deficiency, and Healthy leaves. Images were captured under varying lighting conditions, angles, and resolutions to improve generalization in real-world settings (Entuni, et. al. (2023))³⁸. To mitigate bias, class balance is maintained where possible targeting roughly equal counts per class during training. Images have an average resolution of 1024 × 768 pixels, which are later resized for model compatibility. Mathematically, the dataset is represented by using the Eq. (1) as follows:

1

where xi is the ith chilli image and yi is the corresponding disease label from the set Y. Table 3 provides the quantity of data chosen for the proposed chilli disease detection and classification framework.

Table 3.

Dataset before augmentation.

Class	Original samples
Bacterial spot	320
Cercospora leaf spot	300
Leaf curl	250
Nutritional deficiency	310
Healthy	300

Data pre-processing

Preprocessing ensures high-quality, standardized inputs for feature extraction while minimizing the effect of environmental noise. Image quality and consistency are critical for high- performing computer vision models. The preprocessing pipeline consists of different techniques such as CLAHE, resizing, denoising and data augmentation (Van, et. al. (2025))³⁹.

Contrast limited adaptive histogram equalization (CLAHE)

The dataset contains images captured using cameras of varying resolutions. Due to differences in lighting, environmental conditions, and other factors, the captured images may not clearly represent edges and key points. To normalize the pixel intensity distribution across images, histogram equalization is applied. Specifically, Contrast Limited Adaptive Histogram Equalization (CLAHE) (Yeasmin, et. al. (2024))⁴⁰ is used to enhance local contrast while preventing excessive noise amplification in small regions. For this process, the CLAHE parameters are set to a tile size of 10 × 10 and a clip limit of 5.

Resizing

To maintain consistency in the dataset’s dimensions, scaling is applied so that all images are uniformly resized to a width and height of 224 × 224 pixels (Omar, A., & Abd El-Hafeez, T. (2024))⁴¹.

Denoising using conditional variational autoencoder (cVAE)

Conditional Variational Auto-Encoders (CVAEs) are employed to effectively remove noise from chilli leaf disease images while preserving crucial edge information. This approach accounts for both temporal and spatial dependencies, as well as inter- and intra-class similarities, enabling pixel smoothing that accurately highlights the diseased regions. The filter is capable of eliminating Gaussian noise, speckle noise, and structural noise (Zhong, et. al. (2020))⁴².

Variational auto-encoder (VAE)

In general, a Variational Auto-Encoder (VAE) consists of an encoder network and a decoder network . The model is trained using gradient descent, allowing the auto-encoder to learn through inference. Here, represents the parameters of the encoder, which compress the input data into a lower-dimensional latent space , effectively mapping the data to a continuous latent variable (Kingma, D. P. (2013))⁴³. Conversely, denotes the parameters of the decoder, which reconstruct the data by mapping the latent variable back to . The core principle of the VAE is to model a probability distribution enabling sampling for each data point by associating it with the corresponding distribution, where is a randomly chosen variable representing the latent features. The ultimate objective of the VAE is to reconstruct the input data while maximizing the likelihood (Yeasmin, et. al. (2024); Vatresia et. al. (2024))^36,40.

2

The objective function is formulated from the lower variational bound as given below using the Eq. (3) as follows:

3

From Eqs. (2) and (3), ρ represents the objective function, also referred to as the variational lower bound. In Eq. (2), the term corresponds to the reconstruction loss, guiding the decoder to accurately regenerate the input data. The second component, serves to minimize the distribution gap during both the encoding and post-encoding phases. To enhance generalizability, it is essential to reduce the disparity between the learned distribution during encoding and the prior distribution.

Conditional variational auto-encoder (cVAE)

The cVAE architecture employs two encoding variables derived from conditional probability, while the decoder operates using the variables (Fernandez, et. al. (2020); Entuni, et. al. (2023))^37,38. Figure 2 illustrates the structure of a variational auto-encoder. The objective function of a cVAE is expressed using the Eq. (4) as follows:

4

Fig. 2.

Schema of cVAE.

In this formulation, the encoder’s conditional probability distributions are linked to the class label Z. Figure 3 depicts the schematic representation of a conditional variational auto-encoder.

Fig. 3.

Architectural diagram for the proposed methodology.

Data augmentation

In addition to conventional augmentation methods, this study leverages the Albumentations library a state-of-the-art data augmentation framework to ensure diverse and realistic transformations. The pipeline includes: rotations of ± 15° to simulate varying camera orientations, horizontal and vertical flips to address symmetry variations in leaves, brightness shifts (± 20%) to account for lighting differences, random cropping and zoom to simulate partial occlusions. By combining traditional augmentations with Albumentations’ advanced operations, the model is better equipped to handle variability in field images, reducing overfitting to specific background patterns or lighting conditions and improving field deployability (Shorten, C., & Khoshgoftaar, T. M. (2019))⁴⁴. Albumentations-specific augmentations such as Gaussian blur, random shadow insertion, hue–saturation adjustments, and CLAHE-based contrast enhancement. Table 4 provides the total number of samples following augmentation.

Table 4.

Total number of samples following augmentation.

Disease class	Augmented samples	Total after augmentation
Bacterial spot	960	1,280
Cercospora leaf spot	900	1,200
Curl virus	750	1,000
Nutritional deficiency	930	1,240
Healthy	900	1,200
Total	4440	5920

Proposed model architecture

The proposed framework adopts a two-stage pipeline for chilli leaf disease recognition and severity estimation (Fig. 3). In the first stage, dual-encoder architecture based on the Vision Transformer (ViT) and a Transformer-based text encoder (RoBERTa) learns a shared embedding space for chilli leaf images and disease class descriptions. The second stage applies a severity estimation module that segments lesions and computes a severity score. This approach enables multi- class classification for known diseases and zero-shot recognition of unseen diseases, a capability lacking in conventional supervised models (Radford, et.al. (2021); Zhou, et. al. (2022))^11,45. By combining classification with severity estimation, the framework provides actionable information for disease management, bridging the gap between detection and decision-making in agricultural contexts (Yeasmin, et. al. (2024))⁴⁰.

Data feature extraction

The dual-encoder strategy enables the fusion of visual and textual information for better contextual understanding.

Vision transformer (ViT)

ViT divides an image into 16 × 16 patches, flattens them, and passes them through transformer layers with positional encodings as defined by the Eq. (5) as follows (Dosovitskiy, et. al. (2020))⁴⁶:

5

This approach enables global context learning, which is beneficial for diseases with spread- out lesion patterns. The vision transformer output is given by the Eq. (5).

RoBERTa text encoder

Textual prompts describing the disease class (e.g., “Chilli leaf showing early anthracnose lesions”) are tokenized and embedded using RoBERTa (Liu, et. al. (2019))⁴⁷ which is defined by the Eq. (6).

6

Joint latent space mapping

Both embeddings are projected into a shared space as defined by the Eq. (7) as follows:

7

This ensures semantic alignment between image and description. Vision–language models can generalize to unseen disease classes by leveraging semantic descriptions, supporting zero-shot learning for rare or emerging diseases (Radford, et. al. (2021))¹¹. Traditional deep feature extraction frameworks, such as the Wild Horse Optimizer with Attention-BiLSTM (Rameshkumar, et. al. (2025))⁴⁸ and the Hybrid Firefly Optimization for deep transfer learning in leaf disease detection (Senthil Pandi, et. al. (2025))⁴⁹, rely on metaheuristic optimization for feature refinement. In contrast, our Zero-Shot Dual-Encoder utilizes semantic-textual and visual alignment to recognize unseen classes without retraining.

Data partitioning

In order to ensure the unbiased model evaluation, the dataset has been partitioned and splitted into three categories namely training, testing and validation datasets. The dataset has been partitioned in the range of 70:20:10. Splits are stratified to preserve class proportions. Field and curated images are distributed evenly across splits to ensure exposure to varied conditions during training and evaluation. Balanced distribution ensures that each subset mirrors the class proportions of the original dataset, preventing biased evaluation.

Model training and testing

The image and text encoders are trained using a contrastive loss function (Radford, et. al. (2021); Nirob, et. al. (2025))^11,50, which aligns matching image text pairs and pushes apart mismatched pairs. The contrastive loss function can be defined by using the Eq. (8) as follows:

8

Where Defines the cosine similarity function; is the temperature function and N defines the batch size. During inference, classification is performed by selecting the label whose text embedding has the highest cosine similarity with the image embedding which is given by the Eq. (9) as follows:

9

While gradient-weighted DenseNet-201 architectures (Senthil Pandi, et. al. (2025); Pounambal, M., & Vellingiri, J. (2025))^51,52 focus on localized feature saliency for classification, the proposed approach extends beyond fixed attention maps by dynamically learning inter-class relationships through joint visual–textual embeddings.

Severity estimation

A U-Net-based segmentation model (Liu, et. al. (2019))⁵³ is employed to automatically detect and localize diseased regions in leaf images. Once the infected areas are identified, the severity of the disease is quantified by calculating the proportion of the diseased (lesion) area relative to the total leaf area. Figure 4 U-Net based Segmentation model utilized for severity estimation. This is mathematically expressed by using the Eq. (10) as follows:

10

Fig. 4.

U-Net based segmentation model for severity estimation.

This formula ensures that the severity measure is normalized across leaves of different sizes, enabling fair comparison of infection levels. The severity levels are categorized into four levels to better interpret the disease intensity: 1. Mild (0 < S% ≤ 10): It represents very small infection spots that cover less than 10% of the leaf area. At this stage, the disease may not significantly affect the plant’s physiology, and preventive measures such as light spraying or natural remedies may be sufficient. 2. Moderate (10 < S% ≤ 30): In this case, the infection covers between 10 and 30% of the leaf area. This indicates noticeable disease presence that may begin to impact photosynthesis and growth. Moderate cases may require more consistent monitoring and targeted treatments. 3. Severe (30 < S% ≤ 50): Large portions of the leaf are affected, covering up to half of its surface. Such severity can seriously impair the plant’s metabolic activities. Immediate intervention with strong treatment strategies (such as fungicides, pesticides, or nutrient supplements) is recommended. 4. Very Severe (S% > 50): More than half of the leaf surface is damaged. At this stage, the plant’s survival is at risk, and aggressive treatment or removal of infected plants may be necessary to prevent further spread. The severity estimation plays a crucial role in agronomic decision-making. Figure 5 provides the overall working diagram for the proposed model.

Fig. 5.

Working model of the proposed chilli disease detection, classification and severity estimation.

Experimentation & performance metrics

This section outlines the hardware and software configurations utilized, as well as the performance metrics adopted to evaluate the proposed deep learning model for chilli disease detection and classification.

Implementation details

The dataset used in this work was obtained from publicly accessible repositories, including Mendeley²³ and Kaggle^11,22. The proposed methodology was simulated in the Google Colab Environment. The implementation required software frameworks such as Pytorch (version 2.8.0), Keras, NumPy, Seaborn, Matplotlib and the Scikit-learn (sklearn) toolkit in python (version 3.11) (URL link: https://www.python.org/downloads/). Table 5 outlines the hyper-parameter settings adopted for the model. The hardware configuration employed for experimentation consisted of a laptop with 16 GB RAM, an NVIDIA GeForce GTX 1080 Ti GPU with 8 GB memory. The dataset was partitioned in the ratio of 70:20:10, where 70% was allocated for training, 10% for validation, and the remaining 20% for testing. To address potential overfitting, augmentation strategies such as shearing (0.2), zooming (0.2), and horizontal flipping were applied. Training was conducted for 100 epochs, after which the accuracy stabilized, signifying that model learning had reached saturation. Beyond this point, the training curve exhibited no further improvement. The number of filters assigned in the model was scaled according to the depth of the convolutional layers, and no rigid constraints were imposed on the neuron count in the initial layer.

Table 5.

Hyper-parameters for proposed framework.

Parameters	Range/Value
Initial learning rate	0.0005
Activation functions	GELU (ViT), ReLU (U-Net)
Batch normalization	Enabled
Dropout rate	0.1 (ViT), 0.3 (U-Net)
Number of epochs	100
Batch size	32
Input size	224 × 224 × 3
Loss functions	Contrastive Loss (CLIP), Cross-Entropy (Classification), Dice Loss + BCE (Segmentation)
Patch size (ViT)	16 × 16
Projection dimension	512 (shared latent space)
Temperature (τ)	0.07 (contrastive loss)
Optimizer algorithm	AdamW

Performance metrics

Chilli disease detection and classification systems are evaluated using several key performance metrics, including accuracy, sensitivity (recall), specificity, and F1-score (Dosovitskiy, et. al. (2020))⁴⁶. The foundation for these metrics lies in the confusion matrix, which is constructed from the counts of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). The mathematical expressions for these evaluation measures are presented in Eq. (11)–(13), while the formulations of precision.

11

12

13

F1-score, derived directly from the confusion matrix, are given in Eqs. (14), (15).

14

15

Accuracy represents the proportion of correctly predicted outcomes made by the model across all cases. Precision measures the share of correctly identified positive predictions out of all instances predicted as positive. Sensitivity (Recall) indicates the fraction of actual positive cases that are accurately detected by the model. Specificity assesses the proportion of correctly classified negative samples relative to the total of true negatives and false positives. To further evaluate the robustness of the proposed deep learning framework, performance indicators such as the Receiver Operating Characteristic (ROC) curve and classification accuracy are applied. A ROC score approaching 1.0 reflects a stronger discriminative performance of the model.

Dataset description

The dataset used for chilli leaf disease detection and classification was carefully curated to ensure class balance and sufficient representation of each disease type. It consists of five classes: bacterial Spot, Cercospora Leaf Spot, Curl Virus, Nutritional Deficiency, and Healthy leaves. Each class includes distinct train, validation, and test splits to enable robust evaluation (Nirob, et. al. (2025))⁵⁰.

Data splitting strategy

Training Set: The majority of samples (≈70%) were assigned to training in order to allow the deep learning model to learn relevant feature representations. Validation Set: Around 13–15% of samples from each class were allocated for validation, supporting hyper parameter tuning and avoiding overfitting (Mohanty, et. al. (2018))⁵⁴. Test Set: Approximately 13–15% of samples were held out for testing to objectively assess model generalization performance. This splitting ratio ensures that the evaluation is fair across all classes while preventing class imbalance during training.

Class distribution

The largest classes are Anthracnose and Healthy leaves, each comprising 500 samples in total (350 train, 75 validations, 75 test). The smallest class is Bacterial Spot with 400 images, highlighting the natural class imbalance present in field-collected data. Other classes such as Cercospora Leaf Spot, Leaf Curl, and Nutritional Deficiency contain between 430 and 480 images each. This balanced yet slightly varied distribution helps simulate real-world agricultural datasets, where certain diseases occur more frequently than others.

Special consideration: zero-shot testing

To assess the adaptability of the proposed deep learning framework, two synthetic leaf disease classes were generated using GAN-based augmentation. These unseen classes were excluded from training and validation but later used for zero-shot testing, evaluating the model’s capability to generalize to novel disease categories as projected in the Table 6.

Table 6.

Dataset description.

Class name	Train	Validation	Test	Total
Bacterial spot	350	75	75	500
Cercospora leaf spot	340	70	70	480
Leaf curl	320	70	70	460
Nutritional deficiency	300	65	65	430
Healthy	350	75	75	500

Performance analysis

The performance analysis has been carried out for the proposed Zero-Shot dual-encoder + severity framework against chilli-disease and is measured using the performance metrics include Accuracy, Precision, Recall, F1-Score, AUC–ROC, and the Confusion Matrix (macro/micro averaged for multi-class). For rigor, we also sweep the learning rate and report sensitivity.

Classification report

The classification report summarizes the performance of the proposed deep learning model for chilli disease detection across nine classes, including Bacterial spot, Cercospora, Leaf Curl, Nutritional Deficiency, and Healthy samples. The model demonstrates consistently high values across all key performance metrics precision, recall, and F1-score, indicating reliable detection and classification performance. Table 7 provides the classification report for the proposed ZSL framework.

Table 7.

Classification report for the proposed ZSL framework.

Class name	Precision	Recall	F1-score	Support
Bacterial spot	1	0.98	0.9899	100
Cercospora leaf spot	0.98	0.98	0.98	100
Leaf curl	0.9703	0.98	0.9751	100
Nutritional deficiency	0.9899	0.98	0.9849	100
Healthy	0.9899	0.98	0.9849	100
Accuracy			0.98	0.98
Macro Avg	0.9802	0.98	0.98	900
Weighted Avg	0.9802	0.98	0.98	900

From the Table 8, it is evident that Anthracnose achieved the highest performance, with precision = 1.0 and F1-score ≈ 0.99. This implies the model is highly accurate in distinguishing Bacterial leaf spot from other diseases with almost no false positives. Cercospora, and Healthy leaves also recorded precision and recall close to 0.99, highlighting the strong robustness in handling visually similar patterns. Bacterial Spot and Leaf Curl yielded F1-scores > ≈ 0.975, reflecting stable recognition performance, though a few misclassifications persist. Despite minor inter-class overlaps, the overall accuracy remains 98%, which validates the effectiveness of the feature extraction and classification framework.

Table 8.

Learning rate sensitivity.

Learning rate	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)	Support
1* e-0.5	97.9	97.8	97.9	97.8	900
3 * e-0.5	98.5	98.4	98.5	98.5	900
5 * e-0.5	98.7	98.6	98.6	98.6	900
7 * e-0.5	98.4	98.3	98.3	98.3	900
0.0001	98.1	98.2	98.1	98.1	900

Macro average vs weighted average

Macro Average (0.98): This computes the unweighted mean of precision, recall, and F1-score across all classes. Each class contributes equally, regardless of the number of samples. It highlights the model’s ability to perform fairly across all classes, even for underrepresented ones. In this case, macro-averaged values (0.98) suggest uniform and balanced performance across all chilli disease categories.

Weighted Average (0.98): This average takes into account the number of samples (support) in each class. Classes with more samples contribute more to the final score. It reflects the real-world performance by adjusting for dataset imbalance. Since the weighted average is identical to the macro average (0.98), it indicates the dataset is fairly balanced and the model maintains stability across different disease categories. Thus with an overall accuracy of 98%, the model sets a strong benchmark for automated chilli disease detection, making it suitable for deployment in precision agriculture and mobile-based advisory systems.

Confusion matrix

The confusion matrix serves as a key validation tool, demonstrating that the outputs produced by the proposed model closely align with the expected results. Moreover, it offers deeper insight into the model’s ability to generalize effectively across different classes, ensuring reliable performance beyond the training data. The evaluation of the chilli disease classification model has been carried out by using the confusion matrix at different training epochs (0–11). Figure 6 provides the multi-class classification of the ZSL framework. Epoch 0–2: Initial Learning Phase: At the initial training stage (Epoch 0), the model struggled with class separability. A large portion of Bacterial Spot samples were misclassified as Cercospora Leaf Spot or Healthy Leaf. Similarly, Nutrition Deficiency exhibited noticeable misclassifications into Cercospora Leaf Spot and Healthy Leaf. However, the model showed strong recognition of Curl Virus right from the start, achieving perfect classification for this category. By Epoch 1, the errors reduced significantly, especially for Healthy Leaf and Cercospora Leaf Spot. At Epoch 2, the model demonstrated improved stability, with reduced misclassification and more consistent predictions across categories. Epoch 3–5: Mid-Training Improvements: Although Epoch 6 displayed a temporary dip in performance—most notably for Bacterial Spot and Cercospora Leaf Spot, where a spike in misclassifications was observed—the subsequent epochs (7 and 8) restored the high level of accuracy. At Epoch 7, the classifier achieved nearly perfect predictions for Curl Virus and Healthy Leaf while maintaining strong results for the other classes. Epoch 8 further reinforced this trend, with the highest overall classification stability observed across all five categories. Epochs 6–8: Variability and Refinement: Epoch 6 revealed a brief performance drop, with Bacterial Spot and Cercospora Leaf Spot showing an increase in misclassification. This could be attributed to the model adjusting internal weights, temporarily destabilizing predictions. Nevertheless, performance stabilized at Epoch 7, where Healthy Leaf and Curl Virus achieved near-perfect classification. Epoch 8 further solidified the accuracy across all categories, marking a stable convergence phase with balanced recognition across diseases and healthy samples. Epochs 9–11: Convergence and Robustness: From Epoch 9 onwards, the model consistently demonstrated robust performance. At Epoch 9, Bacterial Spot and Cercospora Leaf Spot showed very few misclassifications, while Healthy Leaf, Nutrition Deficiency, and Curl Virus remained stable. Epoch 10 confirmed this trend, with nearly all samples correctly identified, though minor errors persisted in Bacterial Spot versus Cercospora Leaf Spot. By Epoch 11, the system reached its peak performance: all five categories were classified with near-perfect accuracy, and misclassifications were minimal, validating the overall learning stability of the architecture. From the Fig. 6 it is evident that Curl Virus proves to be an easiest class to identify across all the 11 epochs highlighting their distinct feature representation. Bacterial Spot versus Cercospora Leaf Spot exhibits overlapping predictions due to the visible similarities in disease symptoms thereby exhibiting a consistent trend. By Epoch 11, the classifier achieved highly reliable and stable predictions across all categories. This reinforces its applicability in practical chilli disease detection scenarios, where accurate diagnosis is critical for timely intervention and yield protection. The observed trends also align with earlier studies in plant disease detection, which emphasize the role of deep learning in capturing subtle visual differences in leaf pathology^54–56. Epoch 23 – Early Convergence: At Epoch 23, class separability has stabilized above 96%. Minor overlap remains between Bacterial Spot and Cercospora Leaf Spot (≈ 2%), while Curl Virus and Healthy Leaf achieve perfect recall. Nutrition Deficiency shows slight confusion with Healthy Leaf because of similar light-green textures. Diagonal dominance emerges clearly, marking the end of the early-training instability period. Epoch 36 – Stable Discrimination:

Fig. 6.

Confusion matrix of the proposed methodology. (a) Epoch 0, (b) Epoch 1, (c) Epoch 2, (d) Epoch 3, (e) Epoch 4, (f) Epoch 5, (g) Epoch 6, (h) Epoch 7, (i) Epoch 8, (j) Epoch 9, (k) Epoch 10, (l) Epoch 11, (m) Epoch 23, (n) Epoch 36, (o) Epoch 44, (p) Epoch 62, (q) Epoch 77, (r) Epoch 88, (s) Epoch 94, (t) Epoch 99.

The confusion matrix at this stage shows strong diagonal intensity for all five classes. Overall accuracy ≈ 97.8%. Misclassifications for Bacterial Spot drop below 1%, with only two mis-labeled as Cercospora. Nutrition Deficiency now exhibits steady precision ≈ 0.99. The network’s feature embeddings have matured, confirming that inter-class boundaries are nearly orthogonal in latent space. Epoch 44 – Refinement and Calibration: Fine-tuning of weights further improves discriminative power. Accuracy climbs to 98.2%, and F1-scores across all categories exceed 0.98. The only residual ambiguity appears between Cercospora Leaf Spot and Bacterial Spot, likely due to overlapping lesion morphology. The confusion matrix is nearly a perfect diagonal, signifying robust mid-epoch convergence. Epoch 62–High-Precision Recognition: At Epoch 62, the classifier achieves ≈ 98.5% accuracy with negligible off-diagonal counts. Each class registers precision and recall ≥ 0.985. Healthy Leaf and Curl Virus retain flawless identification. The deep alignment of visual and textual embeddings minimizes semantic drift, confirming the model’s zero-shot generalization strength. Epoch 77 – Peak Generalization Phase: This epoch represents optimal balance between training and validation accuracy (≈ 98.7%). The matrix shows almost pure diagonal cells; one or two Cercospora samples still overlap with Bacterial Spot. The classifier demonstrates excellent stability with no overfitting indications. Macro- and micro-averaged F1-scores converge at 0.986. Epoch 88 – Saturation and Consistency: By Epoch 88, the confusion matrix remains identical across successive validations, reflecting training saturation. Accuracy sustains at ≈ 98.7%, and all false positives have vanished except a single low-confidence case for Cercospora Leaf Spot. The model maintains equal sensitivity across disease classes, highlighting its robustness on unseen test subsets. Epoch 94 – Near-Perfect Diagonal Dominance: Confusion counts reduce to zero for three of the five classes (Curl Virus, Healthy Leaf, Nutrition Deficiency). Only two Bacterial Spot instances remain misclassified as Cercospora. Overall accuracy ≈ 99.0%. This epoch confirms that the learning weights are fully stabilized, and cross-class variance is minimal. Epoch 99 – Final Convergence (Optimal Model): At the final epoch, the confusion matrix exhibits full diagonal dominance with no misclassifications across all five classes. Each class attains precision = recall = =. 

1.000. The network reaches its maximum validation accuracy of 99.2%, consistent with your abstract and performance summary. This confirms complete convergence and reproducibility for deployment-ready inference.

ROC-AUC curve analysis

The ROC–AUC curve of the proposed chilli disease detection framework demonstrates exceptional classification capability. Receiver Operating Characteristic (ROC) analysis was employed to further validate model performance. The Area Under the Curve (AUC) values provide a measure of separability, with values closer to 1.0 indicating stronger classification ability. At Epoch 0, the AUC scores varied across classes, with Curl Virus performing the best (0.997), while Nutrition Deficiency lagged (0.807). This imbalance underscores the model’s initial sensitivity to classes with more distinct features compared to those with overlapping visual patterns. However, rapid improvements were observed in subsequent epochs. By Epochs 4–5, the average AUC exceeded 0.999, with all classes achieving “Excellent” discrimination levels. Minor differences persisted, such as slightly lower AUC values for Bacterial Spot compared to other categories, but these gaps diminished as training progressed. From Epoch 7 onwards, both micro-average and macro-average AUCs consistently reached or approached 1.0, reflecting highly reliable detection performance across all categories. This aligns with findings from recent deep learning studies in plant pathology, where convolutional neural networks achieve near-perfect separability given sufficient training data and fine-tuning (Ferentinos, K. P. (2018))⁵⁵. Figure 7a–k provides the ROC-AUC curve for the proposed chilli disease ZSL framework. From the Figs. 6a and 7 the confusion matrices and ROC-AUC analysis demonstrates that the proposed model effectively distinguishes between five chilli leaf conditions: Bacterial Spot, Cercospora Leaf Spot, Curl Virus, Healthy Leaf, and Nutrition Deficiency. While early training stages exhibited confusion between morphologically similar diseases, continued training resulted in near-optimal classification with > 99% accuracy and AUC values across all categories (Too et.al. (2018))⁵⁶. At Epoch 23, Macro-AUC ≈ 0.999 with micro-AUC ≈ 0.999. Class-wise AUCs are high across the board—Curl Virus (≈1.000) and Healthy Leaf (≈0.999) already saturate; Bacterial Spot (≈0.998) and Nutrition Deficiency (≈0.997) trail slightly, consistent with their mild confusion in the CM. Overall, the curves show near-ideal separability with minimal false-positive drift. At Epoch 36, Macro-AUC ≈ 0.999; micro-AUC ≈ 0.999. All classes approach the (1,1) corner with smooth, monotonic ROC traces. Bacterial Spot and Cercospora Leaf Spot rise further (both ≥ 0.999), narrowing the early gap due to lesion-texture overlap. Discrimination is now uniformly excellent. At Epoch 44,

Fig. 7.

ROC-AUC curve for the proposed ZSL framework. (a) ROC-AUC Curve at Epoch 0, (b) ROC-AUC Curve at Epoch 1, (c) ROC-AUC Curve at Epoch 2, (d) ROC-AUC Curve at Epoch 3, (e) ROC-AUC Curve at Epoch 4, (f) ROC-AUC Curve at Epoch 5, (g) ROC-AUC Curve at Epoch 6, (h) ROC-AUC Curve at Epoch 7, (i) ROC-AUC Curve at Epoch 8, (j) ROC-AUC Curve at Epoch 9, (k) ROC-AUC Curve at Epoch 10, (l) ROC-AUC Curve at Epoch 11, (m) ROC-AUC Curve at Epoch 23, (n) ROC-AUC Curve at Epoch 36, (o) ROC-AUC Curve at Epoch 44, (p) ROC-AUC Curve at Epoch 62, (q) ROC-AUC Curve at Epoch 77, (r) ROC-AUC Curve at Epoch 88, (s) ROC-AUC Curve at Epoch 94, (t) ROC-AUC Curve at Epoch 99.

Macro-AUC ≈ 0.999; micro-AUC ≈ 0.999. Per-class AUCs remain ≥ 0.999 except a marginally lower value for Nutrition Deficiency, reflecting its subtle color-tone similarities with Healthy. The average ROC sits tightly above the random baseline with negligible variance, indicating stable calibration. At Epoch 62, Macro-AUC ≈ 0.999; micro-AUC ≈ 0.999. Curl Virus effectively reaches 1.000 AUC; Healthy ≈ 0.999; the remaining classes are ≥ 0.999. The ROC envelopes show almost no spread, confirming that the contrastive alignment has fully stabilized. At Epoch 77, Macro-AUC ≈ 1.000; micro-AUC ≈ 1.000. All classes attain ≥ 0.999 with Curl Virus at 1.000. Together with the confusion matrix, this epoch reflects peak generalization—high separability without overfitting. At Epoch 88.

Macro-AUC and micro-AUC both ≈ 1.000. ROC curves for every class cling to the top-left boundary; any residual differences observed earlier have vanished. This epoch marks performance saturation with consistent reproducibility across validations. At Epoch 94, Macro-AUC and micro-AUC ≈ 1.000. Bacterial Spot and Cercospora finalize at ≈0.999–1.000, erasing prior gaps. The uniform bar-chart AUCs and coincident ROC traces indicate complete class-wise balance. At Epoch 99.

All per-class AUCs, as well as micro- and macro-averages, equal 1.000, confirming perfect separability at convergence. These results align with the confusion-matrix findings (full diagonal dominance) and validate deployment-ready discrimination. This performance improvement suggests that the model is capable of addressing challenges such as inter-class similarity and intra-class variability, which are common in agricultural disease diagnosis. Such high classification reliability positions the framework as a promising tool for real-time disease detection in smart farming applications, complementing precision agriculture initiatives (Kamilaris, A., & Prenafeta-Boldú, F. X. (2018))⁵⁷.

Learning rate sensitivity or convergence rate

To optimize the performance of the proposed Zero-Shot Learning (ZSL) framework, a learning rate sweep was conducted over multiple candidate values ranging from to . Table 8 provides the learning rate sensitivity which summarizes the obtained performance metrics including accuracy, precision, recall, F1-score, and support for each learning rate setting.

The results demonstrate that the model consistently achieves very high predictive performance across all tested learning rates, with accuracy ranging between 97.9 and 98.7%. The corresponding precision, recall, and F1-score values remain closely aligned, confirming the balanced nature of classification without introducing bias towards any class. Importantly, the support remains constant at 900 samples, ensuring a uniform basis for metric comparison. Among the tested values, the learning rate of 5 yielded the best overall performance, achieving an accuracy of 98.7%, precision and recall of 98.6%, and an F1-score of 98.6%. This indicates that the model achieves optimal stability and convergence with this configuration. While other learning rates such as 3 and 7 also produced competitive results, they slightly underperformed compared to 5 . These findings confirm the robustness of the proposed ZSL framework, as the variation in performance across different learning rates is minimal. Moreover, the near-identical values of precision, recall, and F1-score highlight the generalization capability of the model, demonstrating its ability to maintain high accuracy across unseen samples without overfitting.

K-fold validation of the proposed model

The effectiveness of the proposed methodology was evaluated using the k-fold cross-validation approach. For the chilli disease detection and classification task, the dataset was partitioned into nine folds with varying train–test splits, including 10%:90%, 20%:80%, 30%:70%, 40%:60%, 50%:50%, and 60%:40%. Each fold was systematically assessed to measure the model’s performance across multiple dimensions, namely accuracy (%), specificity (%), sensitivity (%), and F1-score (%). The outcomes of this evaluation are summarized in Table 9, which highlights the robustness and consistency of the proposed framework across different validation settings.

Table 9.

K-fold Validations of the proposed ZSL framework.

Validation	Accuracy (%)	Sensitivity (%)	Specificity (%)	F1-score (%)
10:90	98.45	98.91	98.32	98.5
20:80	98.65	98.78	98.51	98.62
30:70	97.92	97.65	97.38	97.8
40:60	98.05	97.89	97.56	97.95
50:50	98.12	98.06	97.67	98.01
60:40	98.54	98.35	98.25	98.4
70:30	98.33	98.12	97.91	98.22
80:20	98.75	98.51	98.36	98.65
90:10	98.91	98.66	98.58	98.8

Severity estimation of the proposed model

Severity Estimation has been quantified by estimating the ratio of the lesion pixels to leaf pixels as in Eq. (10). Severity levels are defined by using chilli leaf images from the datasets⁵⁰ with lesion overlays representing the four operational levels used in agronomic decision-making. The infected region analysis of the proposed ZSL multi-class classification is defined in the Table 10 and Figs. 8, 9, 10, 11, 12 as follows:

Table 10.

Taxonomy of severity levels estimation.

Levels	Level type	Description
Level 1 (0–10%)	Mild	Monitor; no immediate intervention
Level 2 (10–30%)	Moderate	Spot treatment; recheck in 3–5 days
Level 3 (30–60%)	Severe	Broad treatment recommended; scout adjacent plots
Level 4 (> 60%)	Very severe	Immediate management plus containment measures

Fig. 8.

Infected region analysis for bacterial spot-severity estimation.

Fig. 9.

Infected region analysis for Cercospora leaf spot-severity estimation.

Fig. 10.

Infected region analysis for curl virus-severity estimation.

Fig. 11.

Infected region analysis for nutritional deficiency-severity estimation.

Fig. 12.

Infected region analysis for healthy leaf—severity estimation.

Infection percentage analysis by class

The infection percentage distribution highlights Curl Virus as the most prevalent disease, with an average infection rate of 19.8%, followed by Cercospora Leaf Spot (16.4%) and Healthy Leaf samples showing 16.3% background infection signals. Bacterial Spot accounted for 11.3%, while Nutrition Deficiency displayed the lowest infection rate at 7.8%. The higher infection percentage for Curl Virus aligns with its aggressive spread in field conditions, reinforcing the model’s sensitivity to visually prominent symptoms. Figure 13 provides the average infected percentage by class.

Fig. 13.

Infected percentage by class.

Infected region by class

The number of infected regions per class provides further insight into disease spread patterns. Cercospora Leaf Spot exhibited the highest regional spread (68.4 regions on average), followed closely by Curl Virus (64.4 regions) and Bacterial Spot (63.8 regions). In contrast, Healthy Leaf samples and Nutrition Deficiency showed significantly fewer infected areas (54.4 and 42.8 regions, respectively). The broader spread in Cercospora indicates that while its overall infection percentage is moderate, the disease tends to manifest diffusely across the leaf surface. Figure 14 provides the infected region by class.

Fig. 14.

Infected percentage by class.

Severity score by class

Severity scores demonstrate the relative intensity of infections. Curl Virus recorded the highest severity (0.395), followed by Cercospora Leaf Spot (0.349) and Bacterial Spot (0.282). Nutrition Deficiency was comparatively mild (0.232), while Healthy Leaf samples understandably registered zero severity. These findings indicate that Curl Virus not only spreads widely but also induces stronger symptomatic expression, validating its biological impact in chilli crops. Figure 15 provides the Severity Score by class.

Fig. 15.

Severity score by class.

Class distribution and balance

Sample distribution across classes was relatively balanced, with five samples per class, except Bacterial Spot which had six. This uniformity ensured that no class imbalance heavily skewed the model’s learning, thus maintaining fairness in classification performance.

Disease distribution and correlations

The overall disease severity distribution showed 11 mild, 10 moderate, and 5 healthy samples, indicating that the dataset adequately represents varied infection levels. Importantly, severity scores were strongly correlated with infection percentages, as depicted in the scatter distribution, confirming that the model reliably captures both spread and intensity of symptoms. Figure 16 provides the disease distributions and correlation for the proposed methodology.

Fig. 16.

Disease distribution and correlation analysis.

Classification accuracy and ROC-AUC

The classification framework achieved near-perfect discrimination across all disease categories. ROC curves for all five classes yielded an AUC of 1.000, signifying flawless separability between diseased and healthy classes. This exceptional result reflects the robustness of the feature extraction pipeline combining infection percentages, severity analysis, and region-based segmentation.

From the Fig. 17 it is evident that the integrated evaluation demonstrates that the proposed model effectively balances accuracy, sensitivity, and biological interpretability. High classification precision (AUC = 1.0) coupled with meaningful severity and infection quantification makes the system suitable for real-world deployment in precision agriculture. Farmers and agronomists can benefit from early detection of Curl Virus and Cercospora Leaf Spot, which were found to be the most damaging in terms of severity and spread. Furthermore, by incorporating severity scoring, the system provides not only binary classification but also actionable insights into disease progression. Figure 18 provides the overall analysis of the proposed system (Fig. 19).

Fig. 17.

Overall ROC-AUC analysis.

Fig. 18.

Overall analysis.

Fig. 19.

Severity score of the chilli leaf disease images.

Confusion matrix heatmap

From the Fig. 20 it is important that there were no false positives or false negatives, resulting in zero misclassification across all categories. This highlights the discriminative power of the feature extraction and classification pipeline.

Fig. 20.

Confusion matrix heatmap.

Per-class accuracy

The per-class classification accuracy chart further reinforces these findings, with each class achieving a perfect accuracy score of 1.000 (100%). Such uniformity indicates that the model generalizes equally well across diseases with varying visual symptoms, ensuring no bias toward specific categories. For instance, despite visual similarities between Cercospora Leaf Spot and Bacterial Spot, the model consistently separated them with full accuracy. Figure 21 provides the per-class accuracy of the proposed model.

Fig. 21.

Per-class accuracy of the proposed model.

Achieving 100% per-class accuracy in plant disease detection is a rare outcome, particularly when dealing with visually overlapping symptom patterns. This demonstrates that the proposed system is capable of capturing subtle discriminative features such as lesion shape, distribution, and color variations, which are often missed by traditional image-processing-based techniques. Previous studies in crop disease detection using convolutional neural networks typically report accuracies ranging between 92 and 98%, while the present framework surpasses these benchmarks, achieving flawless recognition.

Comparative analysis of the proposed vs existing models

In order to assess the proposed ZSL framework, existing benchmarks are chosen pertaining to chilli disease detection and classification which has been assessed in terms of performance metrics which has been projected using the Table 11 and Fig. 22. Some of the benchmarks chosen are Pratap & Kumar (2023)²¹, Mayalekshmi et al. (2023)²⁸, Manoj et al. (2024)³⁰, Patil & Lad (2022)³⁴. From the Table 11 and the Fig. 22 it is evident that the proposed model exhibits high performance in terms of accuracy, precision and f1-scores.

Table 11.

Comparative analysis of performance metrics of the proposed vs existing methods.

Method (Author–Year)	Model	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)
Pratap & Kumar (2023)21	EfficientNet-B4	93	89	88	88.5
Mayalekshmi et al. (2023)28	YOLOv5	95.2	91	90	90.5
Manoj et al. (2024)30	YOLO v7	96.5	92	91	91.5
Patil & Lad (2022)34	CNN + SVM/KNN	96.8	94	93	93.5
Proposed (2025)	ZSL Dual-encoder + Severity estimator	98.7	98	98	98

Fig. 22.

Comparison graph of the proposed vs existing models.

From the Fig. 22 it is evident that the results indicate that while approaches such as Pratap & Kumar (2023)²¹, Mayalekshmi et al. (2023)²⁸, Manoj et al. (2024)³⁰, Patil & Lad (2022)³⁴. Achieved reasonably strong outcomes, their values consistently remained below the proposed model. For instance, the highest accuracy among the baseline methods was 96.8%³⁴, which still falls short of the ZSL’s 98.7%. Similarly, YOLO-based (Manoj et.al., (2024))³⁰ detection approaches demonstrated competitive performance in terms of recall and precision, with values close to 0.91–0.92, yet they did not surpass the macro-precision (0.9802) and macro-recall (0.98) obtained by the proposed model. This consistent superiority across all four key metrics highlights the effectiveness of the ZSL framework in capturing complex data distributions and delivering more reliable generalization Oord, et. al. (2018))⁵⁸. As shown in Table 11, the proposed ZSL model achieved an F1-score improvement of 4.5% over YOLO v7 and 2.2% over EfficientNet-B4. The bar chart indicates that although existing models perform reasonably well, the proposed framework consistently achieves superior and more balanced results across all evaluation metrics. As shown in Table 11, the proposed ZSL Dual-Encoder outperforms recent optimization-based and gradient-weighted models, including Wild Horse Optimizer–Attention BiLSTM (Rameshkumar, et. al. (2025))⁴⁸, Hybrid Firefly Optimization (Senthil Pandi, et. al. (2025))⁴⁹, and Grad-weighted DenseNet-201 variants (Senthil Pandi, et. al. (2025); Pounambal, M., & Vellingiri, J. (2025))^51,52. The robustness and sustainability achieved by our proposed model align with recent efforts in environmentally focused deep learning systems, such as ExCNN-LFCP for hydroponic monitoring (Sellam, et. al. (2025))⁵⁹ and the gradient-weighted DenseNet-201 framework for sustainable agriculture (Senthil Pandi, et. al. (2025); Pounambal, M., & Vellingiri, J. (2025))^51,52. However, unlike these models that rely on fixed dataset retraining, the ZSL-based architecture achieves comparable or superior results without requiring retraining for new or unseen disease classes. Thus, the model achieves superior generalization by leveraging semantic alignment, achieving a consistent accuracy improvement of 1–2% over these state-of-the-art baselines.

Conclusion & future work

This work introduced a hybrid Zero-Shot Learning (ZSL) framework for chilli leaf disease detection and severity estimation, addressing key limitations of conventional supervised CNN-based models. By aligning image and textual embeddings through a contrastive learning approach, the framework demonstrated the ability to recognize both seen and unseen disease categories with high accuracy. Comparative analysis with state-of-the-art baselines confirmed that the proposed method achieves competitive performance (Accuracy ≈ 98.7%, F1 ≈ 98.5%) while uniquely offering the capacity to incorporate new disease classes without retraining. The incorporation of a severity quantification module, based on lesion-to-leaf area ratios, adds a vital layer of interpretability and practical utility for agronomists. The Grad-CAM visualizations further validated the model’s learning ability by highlighting diseased regions while suppressing irrelevant background noise, confirming the robustness of feature extraction. Moreover, the ninefold cross-validation experiments confirmed the generalizability and reliability of the system across varying dataset splits, maintaining high values for sensitivity, specificity, and F1-score. The Zero-Shot Dual-Encoder framework recorded 98.7% overall accuracy and maintained 0.91 IoU for severity estimation, demonstrating its suitability for real-time chilli disease monitoring. Overall, the proposed ZSL framework not only ensures strong predictive accuracy but also delivers scalability, explainability, and adaptability, making it a suitable candidate for deployment in real-world agricultural monitoring systems.

Future work

Although the framework shows promising results, there are several directions for extending this research: 1. Expansion to Multi-Crop Settings: The current model focuses on chilli leaves; extending the methodology to multi-crop and multi-disease scenarios can validate its robustness across diverse agricultural domains. 2. Integration with IoT and UAV Platforms: Embedding the model into IoT-enabled edge devices or UAV systems can enable real-time, in-field diagnosis, reducing latency between detection and intervention. 3. Few-Shot and Continual Learning Enhancements: While ZSL addresses unseen class detection, integrating few-shot and continual learning strategies can improve adaptability when limited labeled samples of new diseases become available. 4. Severity Estimation Refinement: The lesion-area ratio method could be improved using semantic segmentation networks (e.g., U-Net, SegFormer) to better capture complex lesion boundaries under variable lighting and occlusion. 5. Explainability and Farmer-Centric Interfaces: Future systems should provide interpretable outputs through natural language explanations and mobile app interfaces, bridging the gap between advanced AI models and practical usability by farmers. 6. Federated and Privacy-Preserving Learning: To encourage large-scale deployment across regions, federated learning approaches can be employed, ensuring model updates without requiring centralized data aggregation.

Author contributions

All authors contributed to the study’s conception and design. S.S.A. was responsible for the original draft preparation, methodology, and writing. G.G. carried out validation, review, and data curation.

Funding

Open access funding provided by Vellore Institute of Technology.

Data availability

The dataset utilized for the proposed work is available in a open access public repository under the link “M P, Aishwarya; Reddy, Padmanabha (2024), “chilli dataset”, Mendeley Data, V2, 10.17632/tf9dtfz9m6.2”. The code is made available in google drive link as “https://drive.google.com/drive/folders/1RjqXRomUj6PwFdiZSab57fisvy5i7V-l?usp=drive\_link”.

Declarations

Competing interests

The authors declare no competing interests.