A novel hybrid approach for microbial detection using deep ensemble learning

Abstract

Microbes play a vital role in healthcare, clinical microbiology, biotechnology, and environmental health, making their accurate detection and classification essential in these fields. The accurate detection of bacteria enables investigators to distinguish various organisms and group them according to requirements of the relevant individuals. The recent advancements in deep learning-enabled models have enhanced image-based microbial analysis with higher accuracy as well as efficiency. In this study, we expound a hybrid ensemble model i.e. DeepBACT, which coalesce three fine-tuned pre-trained models such as VGG19, ResNet50, and InceptionV3 to capture highly discriminative features from microbial images for accurate detection. Our approach is efficient in terms training overhead mainly due to the concept of transfer learning-based hybrid ensemble. The proposed approach employs principal component analysis (PCA)for feature selection and efficient classification via support vector machine (SVM). The approach is trained and tested on a self- created novel dataset i.e. Bacterial Images (BACTI) that contains 320 samples of four types of bacterial images i.e. Priestia aryabhattai strain D5, Bacillus cereus strain NTFB22, Bacillus thuringiensis strain NTFB5, and Bacillus cereus strain NTFB9. The approach is evaluated using a randomized train-test split along with the 5-cross validation that demonstrates efficacy of DeepBACT with an F1-score of 97% and accuracy of 98.61%. The experimental results clearly shows that our method outperforms similar state-of-the-art methods in terms of accuracy as well as efficiency. The proposed approach is effective in classifying the unseen samples of microbe images with promising performance and suitable for real-time deployment.

Introduction

Microorganisms (microbes) are living miniscule organisms that cannot be seen with the naked eye but are visible under a microscope. The plethora of microbes includes viruses, bacteria, fungi, algae etc. The identification of microorganisms from the microscopic images are vital for monitoring environmental changes, facilitating waste decomposition, managing wastewater, and enhancing food processing and ecological health. Since the advent of ML techniques, various studies have been conducted in which ML techniques have been employed in the classification of microbes [1–5]. However, traditional microorganisms image analysis methods rely heavily on the handcrafted features such as shape signatures, texture descriptors, morphological measurements and requires subject experts. These methods suffer from the capturing the complex and spatial patterns present in the microscopic images especially when images exhibit low contrast, uneven illumination and in case of overlapping microbes. Because of the significant morphological similarities among microorganisms, their classification becomes a challenging task. Furthermore, the researchers have explored the various deep learning models such as convolutional neural networks (CNNs) and its variants like MobileNetV2, DenseNet201, ResNet, VGG19, and Xception have achieved state of the art performance in various down streaming tasks like medical imaging, histopathology and cellular phenotype classification [6–10]. Most of the studies have explored CNN model or its variants for end-to-end classification. Moreover, the scarcity of sufficient datasets for learning a deep model for accurate microbes classification is an additional challenge. Thus, lack of a benchmark dataset for microbes detection is a multifarious problem that has been least explored in the literature.

To the best of our knowledge, some studies have done which employs hybrid deep learning approach that combine CNN features with classical classifiers (that is, SVM, XGBoost) for microbial detection [11, 12]. The potential of integrating multiple pretrained CNN’s to extract robust discriminatory representation for microbial image analysis remains unexplored. The main motivation of this work is to explore the potency of existing pre-trained models and to propose a robust hybrid model for ‘microbes’ detection. The aim is to train an ensemble of deep model that yield improved accuracy with low training overhead. Hence, this study proposes a novel hybrid deep learning framework DeepBact for the detection of bacteria from bacterial microscopic images. For this purpose, a pretrained architecture is proposed as a combination of three different models, namely VGG19, ResNet50, and InceptionV3, to extract hierarchical and multi-scale deep features. Additionally, we created a novel dataset of microbes with four classes that contains 320 samples. The scarcity of sufficient number samples was overcome by applying transfer learning as well as data augmentation methods.

The main contributions of the study are summarized as follows:

• i.We propose a hybrid approach DeepBACT for four class microbes detection.
• ii.An ensemble of pre-trained models is employed for deep level features extraction from microbes images.
• iii.Highly discriminative features are selected by using PCA algorithm.
• iv.The approach is efficient as the potency of pre-trained-models is used, which also overcome the issue of scarcity of sufficient amount of dataset.
• v.The method is evaluated on a self-created dataset that demonstrates the promising performance in both known and unknown scenarios.

The rest of the article is organized as follows. Section 2 briefly highlights the related methods for microbial classification. The self-created novel dataset is presented in Section 3. The framework and proposed algorithms are illustrated in details in Section 4. In section 5, we present the various evaluation protocols along with details experimentation as well as comparative analysis. Concluding remarks and future scope of the study are depicted in section 6.

Related work

There have been previous studies in the past that explored microbial detection using machine learning. Authors used KNN, Bayesian Decision rule, SVM and Naïve Bayes used respectively for the classification of bacteria [1–3, 13]. Support Vector Machine and Naïve Bayes Classifier was also employed to detect the bacteria [4, 5]. This study adopts a systematic methodology for bacterial recognition and classification [11]. Gabor transform was used for feature extraction as it is a good texture analysis tool that detects the detailed frequency content of the images. The extracted features were then modelled using XGBoost classifier. The major issue with these feature-based machine learning approaches for microbial detection task is that they need manual feature engineering. Hence the researchers explored deep learning models in this domain. Authors presents a CNN-assisted hyperspectral microscope imaging system for rapid, automated detection and classification of foodborne bacteria [14]. This system combines LSTM, Resnet and 1D-CNN in stacked manner, achieving 98.4% accuracy in identifying bacterial species. The identification of mushroom was done using deep neural networks [15]. A dataset MO106 was curated. The images are collected from the FGVCx2018 challenge dataset and the mushroom observer website. MO106 contains 29,100 images of 106 types of mushrooms. EfficientNetB5 was used in the experiment. Experimental results show that it provides the 92.06% classification accuracy. A convolutional neural network to recognize the fungus infected mango leaves was developed [16]. In this study authors worked on a dataset composed of 3500 images some of which were self-captured and some were taken from the PlantVillage Dataset contains four classes and got 96.16% accuracy. Authors have developed an intelligent system for the diagnosis of the bacterial vaginosis among the females ​ [17]. They developed a deep learning model based Nugent Net. ½ Nugent Net, ¼ Nugent Net and 1/8 Nugent Net. Among these 3 variants ¼ Nugent Net obtained a much higher average accuracy of 75.1% which exceeded the human expert in the relevant field. Authors have proposed a deep learning-based model based on AlexNet, VGG16 and VGG19 for the classification of bacterial infected leaves [18]. For the study they have used dataset containing 23,280 samples for training, 5815 samples for validation and 1082 samples as test samples. The proposed model achieved an accuracy of 98.75%, precision of 98.78%, Recall around the order of 98.75% and F1-Score was 98.75%. A theoretical analysis of the techniques of deep learning in the identification of microorganisms [6]. This study uses a convolutional neural network (CNN) where the base model is InceptionV3 with transfer learning [7]. The dataset of 108 images across five bacterial classes was sourced from the DIBaS database, and preprocessing/augmentation was done to images and achieved 94.42% of average accuracy. The classification of bacterial images is carried out using ResNet-50 architecture in the present work [8]. ResNet-50 model is adjusted by changing the final classification layers. The results indicate that model accuracy is 97.37% with precision = 0.9761, recall = 0.9722, and F-score = 0.9742. Authors proposed a hybrid model for the classification of bacterial species through transfer learning [9]. The Digital Images of Bacteria Species (DIBaS) are used, containing 33 bacterial species. They have used a deep neural network, DenseNet201, to extract features in which the last fully connected layer produces a 1000-feature vector. Then four selection algorithms based on filters information gain, chi-square test, symmetric uncertainty, and Relief F1 to identify the most significant features. These combined features resulted in one distilled feature set of 180 features. Lastly, the Support Vector Machine (SVM) classifier was applied. Results indicated superb performance with an accuracy of 99.78%, precision of 99.91%, sensitivity of 99.88%, and F1-score of 99.89%. The study developed and evaluated deep Convolutional Neural Network (CNN) models for image-based differentiation of bacterial and fungal corneal ulcers using images captured by handheld cameras [12]. Five CNN architectures MobileNetV2, DenseNet201, ResNet152V2, VGG19, and Xception were trained on a dataset of 980 high-quality images from culture-proven cases in South India. The results indicate that proposed model outperform the human expert cornea specialists on both single-center and multicenter test sets. The researchers developed an ensemble learning approach combining VGG19, ResNet50, and DenseNet121 trained on 2,167 anterior segment images from 194 patients [10]. The proposed model achieved diagnostic accuracy with an F1 score of 0.83 and sensitivity of 0.77. A smart framework was introduced in for classification of fungi species [19]. FungiCLEF2022 having 295,938 images with 1604 species has been used as the dataset. In this study ResNeXt and EfficientNet used to extract the deep features used as input to Gradient Boosting Ensemble classifier method to achieved classification accuracy up to 85.11% with F1 score in the order of 50.22%. Deep learning methods based on AlexNet architecture were used as a model for the recognition of five species using CNN and R-CNN [20]. AlexNet, ResNet50 and GoogleNet were used on the dataset in this study. The dataset consists of 623 images having 5 species. The accuracy in classification for ResNet50, GoogleNet and proposed model and AlexNet are 96.50%, 96.50%, 95.50% and 95.00% respectively. The proposed model using CNN and RCNN demonstrated accuracy levels of 98.50% and 95.50%, respectively. Authors have proposed an improved Algae-YOLO object detection approach based on ShuffleNetV2 adapting the ECA attention mechanism to improve detection accuracy [21]. Experiments showed that the Algal-YOLO approach in this study reduces the size of parameters by 82.3%, and the computation is decreased from 16G to 2.9G with less loss of accuracy, and mAP by only 0.007 when compared to the original YOLOv5s. Authors compared the applicability of the YOLO v3 and YOLOv4, models for algal image detection [22]. The dataset consists cell images of 30 algal genera. The model performances were compared using the mean average precision and YOLO v4 got the highest mAP value of 89.8. YOLOv3, YOLOv4, and YOLOv5 were used to classify the four types of algae [23]. 400 real images were used to generate more images using GAN. The mean average precision of YOLOv3, YOLOv4, and YOLO v5 is 75.3%, 83.0%, and 91.0% respectively. Authors proposed a comprehensive review of the use of ML and DL techniques used in microbial detection [24]. An automated system was proposed for detecting bacteria responsible for tuberculosis [25]. Classification task was performed using ResNet18, ResNet34, ResNet50, ResNet101, ResNet152, DenseNet121, DenseNet169 and SqueezeNet. ResNet50 performed better than all such models with an accuracy of 99.74%. The study proposes a hybrid machine learning framework for the classification of MRI images into five categories [26]. This framework integrates AlexNet for feature extraction and an ensemble of machine learning classifiers for accurate multiclass brain tumor detection. It achieves classification performance, reaching about 95.55% accuracy, outperforming several existing methods. An intelligent detection of two bacteria colonies has been done [27]. For this study they have used a dataset composed of 8043 images. Five experiments have been performed on the dataset using SqueezeNet, DenseNet, ResNet18, AlexNet and VGG16 got the accuracy very close to 100%. The proposed deep learning model provides the 96.16% classification accuracy. An automated system for the recognition of harmful algae was developed [28]. Five state of the art deep learning networks VGG16, Alex Net, Google Net, ResNet50 and MobileNetv2 were implemented by the authors to do the recognition task in this study. The dataset contains 7859 images belonging to 159 genera of alga. Out of the five deep learning neural networks employed in the experiment GoogleNet provides the highest accuracy of 96.9% with transfer learning. The study introduces a Faster R-CNN based framework for automated detection and localization of blood cells using microscopic images [29]. The proposed model reports class wise detection accuracy i.e. Red Blood Cells (RBCs) with 91.83%,White Blood Cells (WBCs) with 97.90% and Platelets with 88.36% accuracy. The study proposes a hybrid intelligent framework for accurate histopathological classification of oral cancer [30]. Authors employed Gabor filter to extract texture features, ResNet50 for deep features, and CatBoost for classification and got 94.92% accuracy. An hybrid intelligent system for the OSCC diagnosis was developed by the authors [31]. This study had used traditional GLCM, HOG, LBP for texture descriptors and DenseNet201for extracting deep features. The extracted features were given to PCA for dimensionality reduction, SVM was used for the classification and achieved the best performance at 97% accuracy. A knowledge-enhanced transform-based multimodal classifier (KTBMC) is proposed for microbial keratitis (MK) identification by integrating slit-lamp images and treatment text data [32]. The dataset contains 704 images from 352 patients. The model combines the convolutional neural networks (CNNs) ResNet50, DenseNet121 for feature extraction and BERT for paragraphs. KTBMC model produced a diagnostic accuracy of 97%, higher than the benchmark accuracy of 86%. The proposed study applies deep learning techniques, specifically a convolutional neural network (CNN) model, to detect fungal infections from microscopic images [33]. Dataset is composed of 9,114 pictures of fungi across five classes. They have used Inception-V3 and VGG16 Models. The pre-trained VGG16 received an accuracy of 84.38% but with data augmentation accuracy improved to 95.35% and to 97.19% with fine-tuning. The deep learning approach was developed using ResNet-18 [34]. Multiple instance learning (MIL) approaches are then used to both aggregate and classify the features. The findings show the multi-MIL method produces a high ROC AUC up to 0.972. Microorganism classification system based on deep learning using microscopic images was introduced [35]. Several pre-trained CNN architectures DenseNet201, MobileNetV2, ResNet152V2 and InceptionResNetV2, NasNetLarge were used for classification of parasites, fungi, bacteria and viruses. Experimental results indicated that DenseNet201 was the best performing model for fungi classification (100%), bacteria (86.72%) and viruses (77.43%), whereas the best performance for parasite classification was obtained with InceptionResNetV2 (98.44%). A deep learning-based fungal spores classification approach in grapevine from microscope images is presented in the study [36]. The dataset contains 1320 images of four of the eight species. Individual deep learning frameworks including ResNet-50, VGG-16, MobileNet and InceptionV3 were implemented and MobileNet resulted in having the highest accuracy of 97.40% by applying transfer learning. Deep learning-based technique was proposed for algae detection [37]. In this study a dataset with 3915 objects across 07 classes was created. For the classification task, authors implemented their new approach TOOD, with ResNet101 backbone. This approach enabled them to achieve mAP@0. 5 of order 0.950 and mAP@0. 5:0.95 is 0.57. A smart self-organized identification system was developed for algae classification [38]. The dataset is composed of 28,329 images with 562,512 bounding boxes and of 54 genera. YOLOv5, YOLOv6, YOLOv7 with 10 different variants were used. YOLOv7 has highest mAP@0 among all the variants 5 of the order of 70.6%. A method based on a deep learning model termed AIBISI has been developed for bacterial infection detection [39]. For this study 4073 images were collected of 05 types of infections and got an AUC of 0.755. A deep learning model for the classification of algae was developed from the remote sensing images [40]. In this study authors employed a dual U Net model for the spring and winter season data respectively. The proposed model is Dual position channel attention U Net (DPCAU-Net). The classification accuracy of the proposed method is more than the 90%. This study proposed a deep learning model adopting Big Transfer (BiT) and Graph Laplacian-based data cleaning approach for bacterial classification [41]. The model was trained and validated on the Digital Images of Bacteria Species (DIBaS) dataset, which contains microscopic images for 33 species of bacteria. Its accuracy, precision, recall, and F1 score are 99.31%, 99.11%, 99.09%, and 99.06%, respectively. Mask-RCNN model was implemented by the authors for image segmentation and identification [42]. The evaluation metrics demonstrated also high values, with mAP over 90% and recall above 94%. CNN techniques were used in the study for the automatic classification of soil fungi from microscopic images [43]. Transfer learning was implemented using the ResNet50 model and a majority voting scheme was performed to improve classification accuracy. An overall precision of the classifier was 82%, while 97% with majority voting. This study develops a multi-task deep learning framework for the automated detection and classification of bacteria in microscopic images [44]. Three object detection networks SSD-MobileNetV2, EfficientDet, and YOLOv4 were used for bacteria classification. YOLOv4 performed better than the other three models with mAP of 98%, precision of 86%, recall of 97%, and F1-score of 91%. Authors proposed an GAME-Net for the automated brain tumor segmentation using MRI images [45]. It comprised of an ensemble deep learning framework using a generative autoencoder with attention mechanism, a CNN-based classifier with unsupervised k-means ROI generation, and an attention-augmented U-Net and achieved a dice coefficient of 0.85 and classification accuracy of 87.18%. From the above we found that deep learning is able to automate bacterial identification and it exceeds traditional microscopic analysis and could help in clinical diagnostics (Table 1).

Table 1.

List of methods/techniques used for the bacteria identification from the reviewed articles

Year/Author	Key Concepts	Dataset Details	Performance metrics
2022 Wonohadidjojo, Daniel Martomanggolo [8]	Transfer Learning using ResNet50 along with SGD	Total = 125 Classes = 06	Accuracy = 97.37% precision = 0.9761 recall = 0.9722 F1 score = 0.9742
2022 Nasip, Ömer Faruk and Zengin, Kenan [9]	DenseNet121 with SVM	Digital Images of Bacteria Species (DIBaS) Total = ~ 660 Classes = 33	Accuracy = 99.78%
2022 Redd, Travis K. et al. [12]	CNN Ensemble	Total = 626 Classes = 02	AUC = ~ 0.83
2022 Zachariou, Marios et al. [25]	Modified ResNet 50	Total = 5700 Classes = 02	Accuracy = 99.74% precision = 0.990 recall = 0.967 F1 score = 0.960
2022 Nagro, Shimaa A. et al [27]	ResNet18, AlexNet, VGG16, SqueezeNet and DenseNet161	Total = 8,043 Classes = 02	Accuracy = 98.86%
2023 Borowa, Adriana et al. [34]	ResNet50 based Multiple Instance Learning	Total = 591slides Classes = 04	AUC = ~ 0.755
2023 Zhu, Miaosong et al. [39]	DNN	Total = 4073 Classes = 05	AUC = 0.755
2024 Visitsattaponge, Sarinporn et al [41]	Big Transfer (BiT)	Digital Images of Bacteria Species (DIBaS) Total = ~ 660 Classes = 33	Accuracy = 99.31% precision = 0.9911 recall = 0.9909 F1 score = 0.9906
2024 Zou, Hanbang et al [42]	Mask-RCNN	NA	mAP = 90% recall = 94%
2024 Chin, Shuang Yee et al [44]	SSD-MobileNetV2, EfficientDet, and YOLOv4	Total = 1933 Classes = 33	mAP = 98%, precision = 0.86,recall = 0.97 F1-score = 0.91

Dataset details

For this work we have self-created a novel dataset named BACTI. It comprised of images acquired from School of Bio Technology, University of Jammu under the supervision of the subject experts in cross sensors scenario namely Priestia aryabhattai strain D5, Bacillus cereus strain NTFB22, Bacillus thuringiensis strain NTFB5, and Bacillus cereus strain NTFB9. The images were acquired using Carl Zeiss Primo Star microscope with an AxioCam ERc 5 s digital with have used 100X magnification. These bacteria species are very important for environmental health. These bacterial species play an important role in the growth of the world-famous Basmati Rice plant of Jammu. The dataset consists of a total of 320 high resolution microscopic slides containing four species having 60–95 (approximately) slides per class which are augmented using standard augmentation techniques.

Three independent annotators were employed to annotate the dataset who were domain experts. We compute the Cohen’s Kappa coefficient to measure inter annotator agreement among the independent annotators and we report the statistic to be 0.81, that indicates an almost prefect agreement. The mismatched samples further labelled by another domain expert. The distribution of samples across different classes is shown in the Fig. 1. The bacterial species used in present study were identified using 16S rRNA sequencing. Some sample images are depicted from the BACTI dataset in Fig. 2.

Fig. 1.

Pie-chart of the BACTI dataset

Fig. 2.

Displays sample images of bacteria from the BACTI dataset

Our proposed approach

The potential of local patterns combined with the global features may result in constructing more robust feature vectors that may significantly enhance the accuracy. We proposed a hybrid framework for DeepBACT to accomplish the task of the classification as illustrated in Fig. 3. The key concept of the proposed framework is to function in two phases that comprise of training and testing procedures. The training phase broadly covers a series of activities that are applied to dataset such preprocessing, feature vectors extraction, building an ensemble on base estimators, and tuning parameters of the model.

Fig. 3.

The proposed DeepBACT framework

The goal of the testing phase is to validate the DeepBACT model on a randomly selected set of images covering different range of parameters and datasets for evaluating its accuracy. The dataset goes through a set of image processing operations that prepare and augment these images for features- based model as shown in the Algorithm 1. After that, for decreasing computational complexity we convert the colored images to gray scale. In fact, the purpose of color feature as in our approach not to classify among the existing classes since it gives us already an additional information which increase the complexity by adding extra bytes in the memory. Then to achieve uniformity, images were carved out to 224 × 224 from the microscopic slides in the next stage. We enhance the quality using Histogram equalization (HE) technique to compute the histogram of gray values of the input image, thus stretching the input images intensity into normal distribution, then the effective range of the image is fully normalized.

Algorithm 1 Algorithm for image pre-processing

After the preprocessing phase, input images are provided to three pretrained deep learning models including VGG19, ResNet50 and InceptionV3 that are adapted for bacteria imaging classification. Here, these models work in parallel and extract different features f₁, f₂ and f₃ respectively from the input images that can aid in enhancing classifier performance. Each network maintains its strong learning of features while also fine-tuning to the structure and patterns of bacteria images through transfer learning. Finally, extracted feature maps from all three models are concatenated to produce a unified and comprehensive feature set f, improving the model’s discrimination capacity between bacterial species.

Fine-tuned VGG19

In order to fine-tune VGG 19 model, the convolutional (Conv) and pooling layers of the of the Block 1 to Block 5 are freezed. Whereas, fully connected layers are FC1, FC2 and FC3 are discarded. As a result our fine-tuned model will extract a feature vector f1from the input microbial images. VGG19 is a deep sequential network with convolutional layers generating fine detection of isotropic spatial features which may be advantageous for inferring small morphologies of close clustered bacteria. This structure allows VGG19 to pay close attention to local patterns and textures in an image, which is particularly useful in detecting subtle differences in the morphology of bacterial colonies, including shape, size, and surface texture. The gradual feature extraction is particularly useful for discriminating bacterial species that might have slight structural differences that a shallower network might not detect. Thus, the core spatial information of the bacteria can be preserved following the dimensionality reduction step. Fine-tuning of VGG19 have been done by using the Algorithm 2.

Algorithm 2 Algorithm for fine tuning VGG19

Fine-tuned RESNET50

In order to fine-tune ResNet-50, Conv, Batch Norm, ReLu, Max Pool layers of 3 Res-Block, 4 Res-Block, 6 Res-Block and 3 Res-Block are freezed. Whereas fully connected layer is discarded. As a result the fine-tuned model extract a feature vector f₂ from the input microbial images. Employing residual connections, ResNet50 facilitates deeper feature extraction and addresses the vanishing gradient problem, promoting resilient hierarchical feature learning. In standard deep networks, as we add more layers, the gradient diminishes, meaning the deeper the network gets, the harder it is to train effectively, and this is the primary purpose of ResNet50. It does this using residual connections, which enables the network to skip layers, adding in to a later layer the input of a previous layer. This residual learning approach allows important features to be preserved and propagated throughout the network, achieving even deeper networks that are easier to optimize. This allows for more effective hierarchical feature learning, as deeper layers learn abstract features necessary for differentiating between bacterial species that exhibit small structural variations. Fine-tuning of RESNET50 have been done by using the Algorithm 3. Figure 4 shows the fine-tuned ResNet 50 architecture used in our proposed framework for detecting microbes.

Fig. 4.

Fine-tuned architecture of RESNET 50

Algorithm 3 Algorithm for fine tuning RESNET50

Fine-tuned INCEPTION V3

All the convolutional, max pooling, average pooling, dropout, concatenation etc. of the 3^x Inception Module A, 4^x Inception Module B, 2^x Inception Module B are freezed. As a result the fine-tuned model extract feature vector f₃ from the input microbial images InceptionV3 is a deep convolutional neural network that is optimized to perform efficient feature extraction by using convolutions with multi-scale convolutions in its inception modules. This multi-scale feature extraction enables the network to learn fine-grained textures, mid-level patterns, and high-level structural features simultaneously, which is especially useful in studying bacterial images with different morphologies. InceptionV3 incorporates these multi-scale convolutions to facilitate its detection of the diversity of bacterial structures, such as shapes, arrangements, and surface properties. Because the network learns one model on the whole test set, it determines which features are discriminative at different scales, leading to robust classification against subtle or overlapping bacterial species. Its innovative architecture not only provides regularization but also helps propagate the gradient during updating, avoiding vanishing gradient when stacking many layers, which allows it to reach to the better model in general. This stabilizes the learning of bacterial features on changing resolution. Fine-tuning of Inception V3 have been done by using the Algorithm 4. Figure 5 shows the fine-tuned Inception V3 architecture used in our proposed framework for detecting microbes.

Fig. 5.

Fine-tuned architecture of INCEPTION V3

Algorithm 4 Algorithm for fine tuning InceptionV3

Feature fusion

To maximize classification performance, the feature maps generated by VGG19, ResNet50, and InceptionV3 are concatenated, creating a comprehensive feature set as shown in Eq. 1.

1

where are the set of feature vectors generated from VGG19, RESNET50 and INCEPTION V3 respectively. Where the dimension of is 25,088, whereas include 1,00,352 number of features. Additionally, comprised of 51,200 features. Thus, the dimensionality of features f after combining all the three feature vectors is 1,76,640 This fusion set f integrates spatial, hierarchical, and multi-scale information, leading to a more discriminative representation of bacterial species. To improve the computation efficiency and the classification performance of the CNN ensemble framework, Principal Component Analysis (PCA) is used for reducing the dimension of the feature vectors. The high-dimensional feature map ‘f’ obtained through feature extraction using VGG19, ResNet50, and InceptionV3 contains redundant and highly correlated information. PCA minimizes redundancy by transforming the feature space into an orthogonal set of principal components that capture the most critical variance in the data. PCA does so by focusing on the components that contribute the most to explaining variance in the data, thus allowing dimensionality reduction of features while retaining important discriminative patterns. Not only does this help reduce computational time during training and inference, it also reduces the risk of overfitting, enabling the ensemble model to generalize on bacteria images. After obtaining the image features, by applying the fine-tuned pre-trained models, the next step, is to train a classifier or learner to classifies the bacterial image. A classifier is generally defined as a function, that takes a collection of input features and returns those as classes. We used SVM classifier will be integrated through an ensemble to obtain the classification. The following subsection describes the fundamental principle behind this classifier more in depth. The multi class SVM is used to train the classifier in the DeepBACT. We have employed Support vector machine for our approach mainly due to following reasons:

1. SVM is the most robust classifier which creates a hyperplane of maximum margin as compared to a linear line.

2. The SVM is efficient in terms of training as it creates a hyper plane by using only few support vectors from the training data.

3. SVM is able to solve non-linear separable problems with the help of linear SVM by using a kernel function.

The hyper-plane is built by SVM in such a way that it maximizes the margin of separation between the different class items. First, a training set of microscopic images of target bacteria is selected from our new dataset BACTI. Applying few basic image processing operations in order to get a standard set of images as shown in the algorithm1. In the next step, we apply three VGG19, ResNet50 and Inception V3 on F₀ to extract respective features as , and respectively. In this manner, three feature sets vectors, , and are generated from . After wards , and are concatenated to f, then the dimensionality reduction was done on f, single feature vector using PCA [24]. The label of i^th image is concatenated with the corresponding value of the feature vector to build a dataset (f). As a result, this feature set is used to learn SVM [4] to build meta classifier. Finally, the trained model is fine-tuned to various parameters with proper experimentation at an appropriate search space. The steps for testing algorithm of the DeepBACT method are listed in algorithm 6. The trained DeepBACT framework is validated by presenting images from the testing dataset. A similar sequence of steps such as image pre-processing and feature extraction from three models is applied on test images to generate the corresponding feature sets. Finally, the classification of the test samples is carried out by DeepBACT by assign a class of a bacteria.

Experimental results and discussions

In this Section, the DeepBACT is fine-tuned and the resulting model is tested on our novel dataset called BACTI. To measure the generalization capability of the technique, it is additionally evaluated in cross-data and cross-sensor settings. Additionally, an ablation study assesses the performance of the model under the excisions of different components. Finally, the DeepBACT's verification performance is compared with related state-of-the-art methods. Settings of the hyper-parameters can greatly influence the detection performance of the proposed model. Using a Meta-process, optimal hyperparameters are tuned for the novel dataset. Our framework used early stopping to avoid overfitting, while class weights were applied to handle class imbalance as some classes had few samples. The hyperparameters for DeepBACT such as kernel function, C-value (regularization), gamma -value in SVM, degree -value in SVM and n-value components of PCA were hyper tuned. Table 2 presents the hyper-parameter tuning results of our proposed model. In our DeepBACT, for training of classifier we employed SVM with c = 1 and function = ‘rbf.’

Table 2.

Performance evaluation of DeepBACT at different hyper-parameters

S.No	Hyper-parameters	Search space	Parameter setting	Model performance (%)	Selected value
1	PCA n-Components	[0.80, 0.85, 0.90, 0.95, 0.99]	0.80	93.05	0.99
			0.85	90.27
			0.90	93.05
			0.95	94.44
			0.99	98.61
2	SVM C-value	[0.1, 1, 10]	0.1	98.61	0.1
			1	95.02
			10	93.22
3	SVM kernel-type	['linear', 'rbf', 'poly']	rbf	98.61	rbf
			Linear	20.83
			Poly	20.83
4	SVM gamma-value	['scale', 'auto']	Scale	98.61	Scale
			Auto	95.27
5	SVM degree value	[2, 3, 4] only for poly kernel	2	20.83	2
			3	20.83
			4	20.83

The proposed DeepBACT framework is optimally tuned for various hyperparameters that demonstrate higher accuracy with selected parameters. For SVM classifier, the kernel parameter chosen as linear and a c-value of 0.1 and gamma value of ‘scale’ provides best performance and provides 98.61% classification accuracy. The various performance metrics class-wise in unknown environment are shown in the Table 3. The Confusion matrix is shown in Fig. 6. It is evident from the confusion matrix that our DeepBACT framework classifies class 1 samples with 94.4% accuracy, Class2 samples with 100% accuracy, class 3 samples with 94.3% accuracy and class 4 samples with 97.1 % accuracy. The ROC curve of DeepBACT framework on BACTI is depicted in Fig. 7 which specifies its outstanding performance.

Table 3.

Class-wise performance of DeepBACT framework

S.No	Accuracy	Precision	Recall	F1 score
Class0	94.40 ± 02	0.92	0.94	0.93
Class1	100 ± 03	0.95	1.00	0.97
Class2	94.30 ± 03	1.00	0.94	0.97
Class3	97.10 ± 02	1.00	0.99	0.99

Fig. 6.

Confusion matrix of the DeepBACT

Fig. 7.

ROC Curve performance of DeepBACT

Cross validation of DeepBACT model

The proposed bacterial classification framework DeepBACT is validated for its performance using 5-fold cross-validation for its robustness and generalization capability. In this method, k equal-sized (folds) subsets are randomly partitioned from the dataset. Five folds are used to train the model, with the remaining fold used for testing. It does this 5 times (for 5-folds) creating a different test set every time and finally averages the results across all iterations. This is especially important for microbial image classification, as class imbalance or dataset variability may influence model performance. Finally, this approach guarantees that every sample serves the dual purpose of both training and testing, offering more accurate estimates of critical performance metrics, such as the classification accuracy, precision, recall, F1-score., and more. Table 4 presented the accuracy of proposed model for the 5- cross validation. Data leakage was taken care of with the splitting of data at slides level.

Table 4.

Validation accuracy of DeepBACT model with train-test-split

Sno	5-cross validation		Accuracy of the DeepBACT model
	Training Dataset(%)	Testing Dataset(%)
1	70	30	0.941 ± 0.3
2	75	25	0.954 ± 0.2
3	80	20	0.986 ± 0.3
4	85	15	0.98 ± 0.1
5	90	10	0.98 ± 0.2

Ablation study

Ablation study is an experiment that evaluate the performance of overall framework in terms of various factors such as accuracy, errors and training overhead by ablating some of the components. Moreover, ablation study demonstrates the effectiveness of the sub components of the main microbes detection model. In the need to evaluate the contribution of each individual sub-component, we carry out ablation experimentation in order to investigate DeepBACT under all model components inductively by removing the significant features. Considering the above facts, Table 5 lists the performance as classification accuracy of each test factors different scenario. The ablation study of Inception V3 in DeepBACT using BACTI achieves an accuracy of 97.91% which reduces the performance of the model by 0.7%. Thus, it shows the efficacy of InceptionV3 features which provide more discrimination power to the overall system. The incremental ablative analysis on DeepBACT shows an accuracy drop of 0.8% to existing VGG19 and 1.4% to ResNet50. PCA became considering with BACTI dataset shows the importance of the feature with an ablation and an accuracy improvement of 1.05% of the PCA by reducing noise in fused features. The comprehensive ablation experimentation presents encouraging results of DeepBACT with various contexts. These results confirm that all three CNN models and PCA contribute positively.

Table 5.

An ablation study of DeepBACT framework

S.No	Model	Accuracy	Precision	Recall	F1 score	Error rate
1	VGG19	95.10	0.95			0.49
2	ResNet50	96.40	0.96	0.96	0.96	0.36
3	InceptionV3	96.10	0.96	0.95	0.95	0.39
4	VGG19 + ResNet50 + SVM	97.91	1.00	0.90	0.95	0.20
5	VGG19 + InceptionV3 + SVM	97.21	0.99	0.88	0.96	0.27
6	InceptionV3 + ResNet50 + SVM	97.85	1.00	0.81	0.89	0.21
7	InceptionV3 + VGG19 + ResNet50 + SVM	97.56	1.00	0.91	0.94	0.24
8	InceptionV3 + VGG19 + ResNet50 + SVM + PCA	98.61	1.00	0.94	0.97	0.18

Conclusion

This research work focuses on the four-class bacterial classification detection mechanism by using a hybrid technique. In this study, we present DeepBACT an ensemble based deep learning framework for classifying bacteria, incorporating VGG19, ResNet50 and InceptionV3 to capture extensive features of a bacterium. Dimensionality reduction through PCA is utilized to augment performance while retaining key information. To validate DeepBACT's robustness during the cross-validation process using 5-fold validation, we find that it achieves greater accuracy than any of the individual models combined. The findings validate the efficacy of ensemble learning to improve bacterial classification by capturing diverse morphological features. The major strength of DeepBACT is its demonstrated performance for real-world microbial analysis, which is likely to be enhanced further as more intricate attention mechanisms and self-supervised learning methods are used to make it more robust to context. The proposed approach has been trained and evaluated on a limited dataset and cross-dataset evaluation was not included. In future, the newly created dataset will be expanded by including additional samples of microbes. Besides, the dataset will be expanded with new class of microbes images captured via distinct sensing devices. Moreover, the study can be further explored by including more detailed experimentation on cross-domain scenarios.

Abbreviations

AIBISI

AI-based Bacterial Infection detection using Whole Slide Images

AUC

Area Under the Curve

BACTI

Bacterial Images

BERT

Bidirectional Encoder Representations from Transformers

BiT

Big Transformer

CatBoost

Categorical Boosting

CNN

Convolutional Neural Network

DiBaS

Digital Images of Bacteria Species

DPCAU-Net

Dual position channel attention U Net

ECA

Efficient Channel Attention

FC

Fully Connected Layer

FGVCx2018

Fungi Classification Challenge 2018

GameNet

Graph Augmented Memory Networks

KNN

K-Nearest Neighbors

KTBMC

Knowledge-enhanced transform-based multimodal classifier

LSTM

Long Short-Term Memory

mAP

Mean Average Precision

MIL

Multiple instance learning

MK

Microbial keratitis

ML

Machine Learning

MO106

MO106 is a dataset containing 29,100 mushroom images across 106 classes

MRI

Magnetic Resonance Imaging

PCA

Principal Component Analysis

rbf

Radial Basis Function

RCNN

Recurrent Convolutional Neural Network

ReLU

Rectified Linear Unit

RESNET

Residual Network

ROC

Receiver operating characteristic curve

rRNA

Ribosomal Ribonucleic Acid

SVM

Support Vector Machine

TOOD

Task-aligned One-stage Object Detection

U-Net

"U" structure Neural Network

VGG19

Visual Geometry Group 19

XGBoost

EXtreme Gradient Boosting

YOLO

You Only Look Once

Authors’ contributions

Amit Sharma and Arvind Selwal conceived the original concept. Amit Sharma developed the framework and algorithms of the proposed research work. Amit Sharma and Ayushi Verma contributed in the dataset preparation. Amit Sharma and Arvind Selwal conducted the validation and testing of the approach. Amit Sharma wrote the initial draft of the research work. Arvind Selwal and Devanand Padha supervised and rectified the manuscript.

Funding

No funds, grants, or other support was received.

Data availability

Data available on request/reasonable request.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.