RETRACTED ARTICLE: A MobileNet-based CNN model with a novel fine-tuning mechanism for COVID-19 infection detection

Abstract

COVID-19 is a virus that causes upper respiratory tract and lung infections. The number of cases and deaths increased daily during the pandemic. Once it is vital to diagnose such a disease in a timely manner, the researchers have focused on computer-aided diagnosis systems. Chest X-rays have helped monitor various lung diseases consisting COVID-19. In this study, we proposed a deep transfer learning approach with novel fine-tuning mechanisms to classify COVID-19 from chest X-ray images. We presented one classical and two new fine-tuning mechanisms to increase the model’s performance. Two publicly available databases were combined and used for the study, which included 3616 COVID-19 and 1576 normal (healthy) and 4265 pneumonia X-ray images. The models achieved average accuracy rates of 95.62%, 96.10%, and 97.61%, respectively, for 3-class cases with fivefold cross-validation. Numerical results show that the third model reduced 81.92% of the total fine-tuning operations and achieved better results. The proposed approach is quite efficient compared with other state-of-the-art methods of detecting COVID-19.

Introduction

COVID-19 (COronaVIrus Disease 2019-COVID-19) is a contagious disease caused by coronavirus 2 (SARS-CoV-2) that causes severe acute respiratory syndrome (Ai et al. 2020). The World Health Organization (WHO) officially reported the disease on March 11, 2020, declaring it a pandemic affecting more than 227 countries (https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-968covid19/news/news/2020/3/who-announces-covid-19-outbreak-969a-pandemic). COVID-19 has infected more than 645,630, 482 people worldwide and caused more than 6,634,816 deaths, as reported on December 14, 2022 (https://covid19.who.int/). The first clinical manifestation of SARS-CoV-2-associated COVID-19 disease in which cases were detected was pneumonia (Seibert et al. 2020). In symptomatic patients, clinical signs of the disease, such as cough, pyrexia, nasal obstruction, fatigue, and upper respiratory tract infections, usually begin in less than a week. However, the disease is fatal, with about 75% of patients experiencing severe chest symptoms consistent with pneumonia, as seen on computed tomography during hospitalization (Feng et al. 2020).

The main factors for the rapid spread of the disease are the low efficacy of vaccines with different mutations and inadequate medical testing (Kerr et al. 2021). Therefore, rapid detection and isolation of positive cases are critical to controlling the spread. Given the increase in COVID-19 cases worldwide, more attention must be paid to the containment of the outbreak. The use of reverse transcription-polymerase chain reaction (RT-PCR) is the most commonly used method to detect COVID-19 virus (Susanto et al. 2022). Rapid and accurate detection of the disease is critical to control the epidemic.

An alternative method of diagnosing COVID-19 is the analysis of chest X-rays and CT scans. The equipment required for these images is available worldwide and has been used for years to detect viral infections such as cough, fever, and shortness of breath. In addition, some classification methods have been used to detect lung abnormalities (Chaunzwa et al. 2021), such as tuberculosis, cancer, and pneumonia, that may be related to COVID-19 in chest X-rays in previous studies using deep learning (DL) methodologies. The importance of medical imaging is recognized as a decisive source of information to enable rapid diagnosis (Song et al. 2021). COVID-19 and the combination of AI and chest imaging can help describe and detect COVID-19 complications (Afshar-Oromieh et al. 2021). X-ray images play a critical role in the rapid clinical evaluation of COVID-19 (Wong et al. 2020).

Recently, medical imaging systems have used DL techniques and helped medical personnel make rapid diagnoses (Suzuki 2017). Researchers used convolutional neural network (CNN) models, one of the most popular designs, to detect disease on chest X-ray images (Maior et al. 2021). It is reported that classical computer imaging systems cannot solve complex problems such as image identification, organ recognition, bacterial colony classification, and disease identification (Abiyev and Ma’aitah 2018). However, computer imaging systems that use DL models can solve such problems with a high success rate. Deep learning approaches require powerful GPU systems and a large number of training examples to train the models. To overcome this challenge, transfer learning can be applied to pre-trained CNN models for small datasets by re-designing the last few layers of the model and fine-tuning the model. The CNN model can achieve high accuracy if appropriate hyperparameters are adjusted and efficient fine-tuning approaches are used.

CNNs, which compute weights and extract features during training, were developed to handle multidimensional data such as time series or image data. The name “convolutional” comes from using a convolution operator to solve complex operations. In addition, CNNs are commonly used in health care due to their ability to generate features from images automatically. CNNs can also learn from one task and transfer it to another through transfer learning and fine-tuning. This method has proven successful in classification tasks (Tajbakhsh et al. 2016).

In the literature, many deep learning models have been reported to detect and classify COVID-19 cases from chest X-ray images (Shorfuzzaman and Masud 2020; Degadwala et al. 2021; Iskanderani et al. 2021; Pathak et al. 2020; Kaur et al. 2021; Wang et al. 2020; Misra et al. 2020; Xu et al. 2020; Ozturk et al. 2020; Asif and Wenhui 2020; Varela-Santos and Melin 2021; Boccaletti et al. 2020; Sun and Wang 2020). These studies include different CNN models such as Resnet50, VGG16 (Shorfuzzaman and Masud 2020), MobileNetV2 (Apostolopoulos and Mpesiana 2020), DenseNet (Iskanderani et al. 2021), and several customized new CNN models such as DarkCovidNet (Ozturk et al. 2020) , InstaCovidNet (Gupta et al. 2021)).

However, one of the major problems in studying COVID-19 is the lack of sufficient and reliable data in the first quarter of the pandemic. Because of the limited number of tests, many deaths and viral infections cannot be detected during the beginning of the pandemic. The accessibility of large and high-quality datasets in the ML and DL applications plays a critical role in the reliability of the results. Therefore, researchers had to work with datasets in a limited range by combining available datasets or using data augmentation techniques. As the pandemic progressed, larger data sets were made available to the public (Mahajan et al. 2022). Thus, the most recent studies have used large datasets and ensured the accuracy of their models (Tammina 2022; Bhattacharyya et al. 2022).

In our study, we tested five different pre-trained CNN architectures, namely VGG16, DenseNet, MobileNetV2, InceptionV3, and ResNet, to find a suitable model for our problem. We trained these models in 100 epochs using 80% of the images, and the results show that MobileNetV2 is more successful. Thus, a deep transfer learning approach based on MobileNetV2 with novel fine-tuning mechanisms is proposed to detect COVID-19 diseases from chest X-ray images. We created a relatively large dataset by combining two publicly available datasets containing COVID-19 and pneumonia X-ray images, namely chest X-ray images (Pneumonia) (ChestX-RayImagesPneumonia 2022) and COVID-19 Radiography Database (Covid-19RadiographyDatabase 2022), to evaluate the proposed model. The dataset consisted of three classes: normal, pneumonia, and COVID-19.

Our approach integrates the training phases of the MobileNetV2 CNN architecture, such as data augmentation, transfer learning, and fine-tuning, into a single design. We proposed one classical and two new fine-tuning mechanisms to improve model performance. We also used data augmentation methods such as zooming and rotation to preserve the diversity of the test data. Our theory is that during the training of the network, the last layers are trained more, which increases the ability to generalize to the new problem while at the same time hiding the information learned from the previous training of the first layers, leading to more successful results. In this study, three methods were used for comparative theory implementation. In the fine-tuning phase, the first approach was classical by opening the last 50 layers and freezing the 102 remaining layers of the MobileNetV2 model. The second approach used a step function to fine-tune the model. The third approach used a predefined exponential mathematical function to fine-tune the model. We performed experimental tests with three models and compared the numerical results. The numerical results show that the proposed approach achieves promising results when using raw data without complex preprocessing steps.

The main contributions of this study can be summarized as follows:

1. A new deep transfer learning model was proposed based on MobileNetV2 to detect COVID-19 disease.

2. Two novel fine-tuning mechanisms were integrated into the proposed model.

3. We combined two public COVID-19 X-ray databases to evaluate our model and trained our model on this dataset using two standard training approaches, 30–70% hold-out with five different seeds and fivefold cross-validation.

4. Proposed model reduced 81.92% of total fine-tuning operations and achieved better results.

The remainder of the article is organized as follows. Section 2 contains a literature review. Section 3 contains information about the datasets, the proposed CNN architecture, and its sublevels. Section 4 contains the numerical results obtained from the proposed approach. Section 5 discusses the proposed approach with the works in the literature. Finally, Section 6 presents the conclusions and future work of the paper.

Related works

Deep learning (DL) algorithms and CNN models in image analysis and processing in the biomedical field have recently produced successful results (Bala et al. 2019). Moreover, various CNN-based deep neural networks achieved remarkable results in ImageNet competition (Krizhevsky et al. 2012). Thus, the researchers have focused on using DL to analyze medical images.

Diagnosis of COVID-19 by chest X-ray is associated with pneumonia symptoms. The image classification methods developed by researchers for COVID-19 or pneumonia are divided into the following categories: Machine learning methods (ML) (Kwekha-Rashid et al. 2021), CNN architectures (Minaee et al. 2020; Wang et al. 2020; Varshni et al. 2019), complex CNN models (Wang et al. 2020; Monshi et al. 2021; Khan et al. 2020), adversarial networks (Shams et al. 2020), deep fuzzy model (Song et al. 2022), and transfer learning (TL) methods (Shorfuzzaman and Masud 2020; Pathak et al. 2020; Apostolopoulos and Mpesiana 2020).

More specifically, Stephen et al. developed a DL model to identify pneumonia in chest X-ray images (Stephen et al. 2019). They used a ConvNet model to compute features from images and used these features to detect pneumonia. The dataset consists of 36 validation and 64 training data. Data augmentation and hyperparameter fine-tuning were performed to improve the model. The result was an accuracy rate of 93.73% on a small dataset. In another study, Islam et al. proposed a general model for computer-aided detection of pneumonia on chest X-ray images on a dataset comprising 5863 images of normal and pneumonia patients. Their model classified pneumonia with 97.34% accuracy (Islam et al. 2019). In Ayan et al. (2019), the authors aimed to detect pneumonia from chest X-ray images based on the VGG16 model. They used a pre-trained model and achieved an accuracy rate of 82%. Varshni et al. applied deep learning models such as ResNet, VGG16, and Xception, extracted some features and used these features to classify pneumonia on chest X-rays using Random Forest, SVM, Naive Bayes classifiers, and K-Nearest Neighbors (Varshni et al. 2019). In addition, they used hyperparameter optimization in their model to improve the results.

Concerning the COVID-19 studies, Wang et al. used a DL approach to compute features and achieved a correct detection rate of 79.3% on a database consisting of 325 COVID-19 and 740 pneumonia images (Wang et al. 2021). Apostolopoulos et al. proposed a model to distinguish between normal, viral, and COVID-19 cases. They implemented a model using TL techniques and achieved success rates of 98.66%, 96.46%, and 96.78%, respectively (Apostolopoulos and Mpesiana 2020). In Ucar and Korkmaz (2020), the authors used a DL model to categorize coronavirus-related infections based on X-ray images. They applied Bayesian optimization-based fine-tuning to their model to calculate their results (Ucar and Korkmaz 2020).

In Nour et al. (2020), the authors also proposed a classification model using a Bayesian optimization algorithm and performing TL. In another study implementing a 3-class COVID-19 detection model, the authors used CT images for disease detection and achieved a success rate of 86.7% with a model segmentation-based approach (Xu et al. 2020). Khan et al. integrated pre-trained Xception into a model for automatically classifying COVID-19 on X-rays. Their study achieved an accuracy rate of 87.5% (Khan et al. 2020). In Rehman et al. (2020), the authors developed a DL approach to implement a CNN model to distinguish viral pneumonia, bacterial pneumonia, and normal images. They also used their model to diagnose COVID-19 and achieved a correct classification rate of 98.75%. Narin et al. proposed a model employing five pre-trained CNN models, namely InceptionV3, ResNet50, ResNet101, ResNet152, and Inception-ResNetV2, on chest X-ray images. The proposed approach achieved an accuracy rate of 98.00% (Narin et al. 2021). In Toğaçar et al. (2020), the authors proposed a MobileNetV2-based DL approach to detect COVID-19 infection from X-ray images. They used stacked fuzzy colored and original images to increase their model’s success and achieved an accuracy rate of 97.06%. They also employed the Social Mimic Optimization algorithm to optimize their parameters. Gupta et al. used several pre-trained CNN models to identify COVID-19 and pneumonia by extracting features from healthy chest X-rays. The model had a 99.08% success rate (Gupta et al. 2021). In another study using CNN to calculate features, researchers developed a model that combined two CNN models to classify chest X-ray images and attained a classification success rate of 91.40% (Rahimzadeh and Attar 2020).

Luz et al. presented an EfficientNet-based model to analyze X-ray images. They trained their model on 183 COVID-19 samples and achieved an accuracy rate of 93.9% (Luz et al. 2021). Punn and Agarwal introduced a DNN for coronavirus symptom detection. They used 108 COVID-19 cases in the database and achieved an average classification rate of 97% (Punn and Agarwal 2021).

Feature extraction and deep network design used for diagnosing COVID-19 infection are very effective in the results. Previous studies show that TL and fine-tuning models of DL architectures are widely used to detect diseases such as COVID-19 and pneumonia. Most of the work has been done using data augmentation operations such as image enlargement and resizing. Some reported approaches used hybrid models that combine classical machine learning algorithms with DL approaches to detect coronavirus-associated infections on chest images.

Material and methods

This section briefly discusses datasets, data augmentation, preprocessing, transfer learning, and fine-tuning approaches. In addition, the proposed DL methods are described in detail. A general block diagram of the study is shown in Fig. 1.

Fig. 1.

Block Diagram of proposed approach

We used pre-trained CNN models, VGG16, InceptionV3, MobileNetV2, ResNet50, DenseNet, and EfficientNet, to find the most appropriate design for our problem in this study. These models were trained on 80% of the entire dataset for 100 epochs. Table 1 shows the results of this test. As can be seen from the table, MobileNetV2 was the most effective architecture. Therefore, we determined MobileNetV2 as the main design.

Table 1.

Achievements of pre-trained deep CNN models for COVID-19 disease classification before any fine-tuning

Model	Training loss	Training acc.	Val. loss	Val. acc.
VGG16	0.9279	0.5938	0.9064	0.6250
InceptionV3	0.0857	0.9722	0.0730	0.9375
MobileNetV2	0.0970	0.9657	0.1183	0.9549
ResNet50	0.0445	0.9826	0.2045	0.9488
DenseNet	0.0598	0.9896	0.1710	0.9375
EfficientNet	0.3335	0.9890	0.8310	0.8346

Bold text represents the best validation accuracy

Datasets

The main challenge in training and validating the proposed model is the lack of publicly available labeled datasets. Most studies used small datasets (Shorfuzzaman and Masud 2020; Kaur et al. 2021; Rehman et al. 2020), which does not guarantee that the proposed model is fully trained. Thus, to increase the proposed model’s generalization ability and create a more robust system, it is necessary to work with a larger database. Most of the public COVID-19 datasets have two classes. We combined the two publicly available datasets to increase the number of images and added pneumonia as a third class in this work. The first dataset, the chest X-ray dataset, contains images in two classes (pneumonia and normal) and consists of 5,863 X-ray images (ChestX-RayImagesPneumonia 2022). The second dataset, the COVID-19 Radiography Database, contains normal, viral pneumonia, and COVID-19 images. The dataset consists of 1341 normal, 219 COVID-19, and 1345 viral pneumonia chest X-ray images (Covid-19RadiographyDatabase 2022). Figure 2 shows some sample images from these datasets.

Fig. 2.

Sample images of our database

Data augmentation and preprocessing

Deep CNNs have performed remarkably well on many image classification tasks; however, these models rely on large datasets to avoid over-fitting (Shorten and Khoshgoftaar 2019). We need a large dataset to develop a robust and successful DL classification model, and this is not always possible. Data augmentation methods are used, a set of techniques that improve the size of training datasets. It also provides a diversity of classes in the dataset. Data augmentation was applied to the proposed model to increase the amount of data, avoid over-fitting, and create a more robust model. These methods generate new images by interacting with different variations in the visual properties of images that can significantly change them, such as image rotation, random horizontal flipping, and zooming. The dataset contains chest X-ray images, some of which are 1007x1024 pixels. We need to reduce the size of the images to fit the input shape of the tested models. The input shape of the MobilenetV2 is 224x224 pixels. This study only used the resize operator as a preprocessing process.

MobileNetV2 deep learning model

MobileNetV2 is a CNN architecture that aims to perform well on mobile devices. It is based on a different structure from other CNN structures where its connections are between the bottleneck layers. In addition, the intermediate expansion layer uses deep folds to filter out nonlinear features. The MobileNetV2 architecture includes 32 layers of initial convolution followed by 19 bottleneck layers. The design details of the MobileNetV2 model are shown in Figure 3. In this study, we propose a MobileNetV2 design that uses two novel fine-tuning approaches to identify COVID-19 X-ray images.

Fig. 3.

MobileNetv2 architecture

MobileNetV2 design has some advantages over the other DL designs. For small datasets, it is hard to train the system and the visual classification task becomes prone to over-fitting. The MobileNetV2 architecture hinders this effect, preventing over-fitting, and it is a fast and successful architecture that optimizes memory consumption with a low margin of error. Moreover, MobileNetV2 design provides fast execution of transactions while making experimentation and parameter optimization (Huo et al. 2022).

Transfer learning

Training a CNN model from scratch is an excessively time-consuming and computationally expensive process (Alzubaidi et al. 2021). Therefore, the common approach is to use pre-trained CNN models on datasets. TL is applying the gained knowledge. If training examples are insufficient in a classification problem, pre-trained examples of TL are used. There are two methods of applying a pre-trained model in TL. The first method is to use the pre-trained CNN model as a feature extractor and for classification; the last fully connected layer(s) is changed according to the number of classes in the data set. In the other method, fine-tune the pre-trained CNN model and its retraining of all or part of the layers with specific methods. As a result, a changed architectural design is employed for the new classification task.

Fine-tuning

Fine-tuning, a TL technique, focuses on saving the information obtained while solving a classification task and applying this knowledge to different approaches to a related problem. The major difference between fine-tuning and TL is that only the weights of the newly added classifier layers are optimized in TL, while the whole model is optimized in fine-tuning.

In fine-tuning, unfreeze some of the top layers of a frozen convolutional base model and simultaneously train the last layers of the convolutional base model and the newly added classifier layers. Fine-tuning allows the higher-order feature representations in the convolutional base model to be more relevant for the specific task. In most convolutional networks, the last layers are more specialized than the beginning layers. When you go deeper, the features of the pre-trained model are progressively more specific to the pre-trained dataset. Fine-tuning aims to adjust these specific features to run with the new data rather than overwrite the generic knowledge to adapt to the new classification task. Thus, it is crucial to open how many of the last layers and to train them on how many epochs.

In addition, fine-tuning helps achieve the goal and is an excellent combination for COVID-19 detection, where time to delivery and availability of training data is critical. This method uses pre-trained models to perform the tasks of DL, saving time and afford. It uses the features developed by different algorithms to pass on the learned parameters or information. DL gives decent results when more data can be used, but fine-tuned TL model can achieve the same results with a smaller dataset.

Proposed fine-tuned MobileNetV2 model

In our study, we implemented the MobileNetV2 network (https://www.tensorflow.org/api_docs/python/tf/keras/applications/mobilenet_v2/MobileNetV2) to classify COVID-19 X-ray images. There are two kinds of blocks in MobileNetV2. One is the residual block with stride 1 and the other block is the non-residual block with stride 2, which is used for downsizing (Sandler et al. 2018). Details of the MobileNetV2 model are given in Fig. 3.

MobileNetV2 has 155 layers, including the classification layer. We used this model to transfer information from a related task that has already been achieved. Our proposed model includes 154 pre-trained network layers (convolutional base) and two additional layers, one at the beginning for preprocessing and one at the end for classification of the new task.

As seen in Fig. 4, the classification process was performed by entering the inputs through the layers obtained in the fine-tuning process.

Fig. 4.

Fine-Tuned MobileNetV2 Model

In our approach, we trained the entire model for 50 epochs before fine-tuning. In the first fine-tuning, we opened the last 50 layers of the convolutional base model and created new training loops to train the entire model for 80 epochs, as shown in Fig. 5 (yellow bars). In the second fine-tuning, we started by opening the last layers of the convolutional base model using a step function. In this approach, we reduced the number of opened layers from the last 50 layers by five for every eight cycles, as shown in Fig. 5 (green bars).

Fig. 5.

Fine-tuning number of epoch and layer numbers for Models 1–3 (color figure online)

In our last approach, unlike the other two approaches, we determined the number of epochs and which layers to be opened according to the result using a predefined exponential equation instead of performing the fine-tuning process in a certain decreasing or increasing order. We used the following (Eq. 1) equation to calculate these values:

$$\begin{array}{c}\hfill y=e^{\frac{\frac{-2·x}{8}+9}{2}+1}\end{array}$$ 1

where x is between 1 and 50 corresponding to defining unfrozen top layers of the base model and y is the calculated value for fine-tuning to define the number of epochs. This equation was determined by empirical observation. Therefore, another exponential decreasing equation can be used for that purpose.

According to this equation, the whole system was trained 80 epochs in the last layers and exponentially decreased to 1 epoch in the ${102}^{\mathit{th}}$ layer, as seen in Fig. 5 (blue bars). Thus, our model can preserve more pre-trained generic information. The last layers of CNN models can have special learned properties and the first layers of the CNN models often learn more about generic properties, such as edges, shapes, and textures. This equation provides us with an exponentially decreasing number of training cycles from the last layer to a predetermined depth during the training process. As can be seen in Fig. 5 that in our three approaches, we fine-tuned models at 4000, 2200, and 723 training operations in total for Models 1, 2, and 3, respectively. We used the following training parameters: 0.0001 as the learning rate and Adam as the optimizer.

Evaluation metrics

We used the following evaluation metrics: accuracy, recall, precision, and f1-score as in Eqs. (2)–5.

$$\begin{array}{cc}\hfill & \text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}\hfill \end{array}$$ 2

$$\begin{array}{cc}\hfill & \text{Recall}=\frac{\mathit{TP}}{TP+FN}\hfill \end{array}$$ 3

$$\begin{array}{cc}\hfill & \text{Precision}=\frac{\mathit{TP}}{TP+FP}\hfill \end{array}$$ 4

$$\begin{array}{cc}\hfill & \text{F1 - score}=\frac{2·(\text{Precision}·\text{Recall})}{\text{Precision}+\text{Recall}}\hfill \end{array}$$ 5

where True Positive (TP) is the case where the output of the algorithm outputs YES when the actual state is YES, False Positive (FP) is the case where the output of the algorithm outputs NO when the actual state is YES, True Negative (TN) is the case where the output of the algorithm outputs NO when the actual state is NO, False Negative (FN) is the case where the output of the algorithm YES when the actual state is NO.

Results

In this paper, we implemented three different fine-tuning approaches for classifications of three different classes, namely normal, COVID-19, and viral pneumonia, on X-ray images. We combined two datasets containing 9457 images. We evaluated our model using two common evaluation methods, separating the data 30–70% randomly test-train splits using different seeds and repeating five times, and fivefold cross-validation to validate our models. Experimental tests were conducted on a PC with an Intel I7 6700hq 2.60GHz CPU, NVIDIA GTX970M having 3GB GPU, and 16GB RAM. The proposed approach was implemented using the Keras-Tensorflow library. We evaluated our model using the performance metrics given in Eqs. (2)–(5). Table 3 shows the pre-fine-tuning and after fine-tuning accuracy results of three models. Models 1, 2, and 3 demonstrate proposed classical, step, and exponential fine-tuning models, respectively. We used 50 epochs of training to calculate pre-fine-tuning metrics and 80 epochs of training to calculate after fine-tuning metrics. The number of epochs and learning rate parameters were determined empirically. Previous studies showed that using Adam as an optimizer achieved promising results (Bera and Shrivastava 2020). Thus, Adam was chosen as the optimizer. In addition, we defined the batch size as 32 and the learning rate as 0.0001. All parameters of the model are summarized in Table 2.

Table 3.

Pre-fine-tuning and After fine-tuning accuracy and loss values

Model	Pre-fine-tune acc.	Loss	After fine-tune acc.	Loss
Model 1	0.9409	0.1223	0.9588	0.0969
Model 2	0.9429	0.1335	0.9739	0.0675
Model 3	0.9425	0.1268	0.9781	0.0417

Table 2.

Model parameters

Parameter	Value
Learning rate	0.0001
Optimizer	Adam
Epochs	50+80
Batch size	32

As can be seen from Table 3 that before fine-tuning, models gave about 94% accuracy rates. Although these models are the same in the pre-tuning stage, the results are slightly different because of the random separation of datasets and the small number of training epochs. Results after the fine-tuning show that the proposed fine-tuning mechanisms achieved better accuracies than the classical fine-tuning mechanisms. Furthermore, these models were fine-tuned at 80, 44, and 14.46 epochs on average for models 1, 2, and 3, respectively. Model 3 decreased 81.92% of total fine-tuning epochs and still achieved better results.

Table 4 shows the results of the experiments based on different evaluation metrics: Accuracy, recall, precision, and F1-score for the 30–70% hold-out evaluation model. The tests were repeated five times using different seeds and mean, standard deviation (Std. Dev.), and the best and worst values of evaluation metrics were calculated. When the standard deviation was examined, the proposed model 3 was the most robust. It has outperformed the other models in terms of all evaluation metrics.

Table 4.

Numerical results for proposed models using 30–70% hold-out for five different seeds

Metrics	Model 1	Model 2	Model 3
Accuracy
Mean	0.9261	0.9583	0.9666
Std. Dev.	0.0079	0.0076	0.0017
Best	0.9358	0.9608	0.9689
Worst	0.9143	0.9499	0.9644
Recall
Mean	0.9274	0.9583	0.9665
Std. Dev.	0.098	0.0052	0.0017
Best	0.9400	0.9630	0.9671
Worst	0.9140	0.9500	0.9643
Precision
Mean	0.9275	0.9582	0.9666
Std. Dev.	0.0093	0.0052	0.0017
Best	0.9401	0.9631	0.9672
Worst	0.9143	0.9496	0.9644
F1-Score
Mean	0.9275	0.9582	0.9666
Std. Dev.	0.0097	0.0051	0.0017
Best	0.9401	0.9630	0.9690
Worst	0.9143	0.9606	0.9643

We also employed a fivefold cross-validation technique to evaluate the proposed model and to compare the results to the results of other studies. The results of cross-validation of each model are shown in Tables 5, 6 and 7. As can be seen from the tables, Model 3 performed better results and achieved fivefold mean accuracy, recall, precision, and f1-scores of 97.61%, 97.60%, 97.59%, and 97.60%, respectively.

Table 5.

Numerical results for Model 1 using fivefold cross-validation

K-Fold	Accuracy	Recall	Precision	F1-score
Fold 1	0.9571	0.9571	0.9568	0.9569
Fold 2	0.9582	0.9582	0.9582	0.9582
Fold 3	0.9587	0.9587	0.9582	0.9583
Fold 4	0.9557	0.9577	0.9550	0.9753
Fold 5	0.9513	0.9513	0.9511	0.9511
Mean	0.9562	0.9566	0.9558	0.9559

Table 6.

Numerical results for Model 2 using fivefold cross-validation

K-fold	Accuracy	Recall	Precision	F1-score
Fold 1	0.9635	0.9635	0.9634	0.9634
Fold 2	0.9582	0.9582	0.9579	0.9580
Fold 3	0.9603	0.9603	0.9599	0.9599
Fold 4	0.9640	0.9640	0.9636	0.9638
Fold 5	0.9592	0.9592	0.9592	0.9592
Mean	0.9610	0.9610	0.9608	0.9608

Table 7.

Numerical results for Model 3 using fivefold cross-validation

K-fold	Accuracy	Recall	Precision	F1-score
Fold 1	0.9756	0.9756	0.9756	0.9756
Fold 2	0.9725	0.9725	0.9725	0.9725
Fold 3	0.9793	0.9793	0.9792	0.9792
Fold 4	0.9777	0.9777	0.9776	0.9776
Fold 5	0.9751	0.9751	0.9750	0.9751
Mean	0.9761	0.9760	0.9759	0.9760

Figure 6 presents the average confusion matrices for Models 1–3 for a 30–70% hold-out data split using random seeds of five repetitions. From Fig. 6a, we can note that while 1062.2 COVID-19, 387.2 normal, and 1179 pneumonia samples were accurately classified, 21.6 COVID-19, 96.2 normal, and 92.8 pneumonia samples were misclassified using Model 1. Thus, the correct classification rates of COVID-19 samples were 98.09%, while 80.09% for normal and 92.70% for pneumonia samples. As shown in Fig. 6b, 1076 COVID-19, 429 normal, and 1215.8 pneumonia samples were classified correctly, and 7.8 COVID-19, 54.4 normal, and 56 pneumonia cases were misclassified using Model 2. As a result, 99.28%, 88.74%, and 95.59% correct classification rates of COVID-19, normal, and pneumonia were achieved, respectively. It can also be noted in Fig. 6c that 1076 COVID-19, 438.4 normal, and 1230 pneumonia samples were classified correctly, and 7.8 COVID-19, 45 normal, and 41.8 pneumonia cases were misclassified using Model 3. Therefore, the correct classification rates of COVID-19, normal, and pneumonia were 99.28%, 90.69%, and 96.71%, respectively.

Fig. 6.

Calculated confusion matrices of Models 1–3 using 30–70% hold-out

Figure 7 presents the mean confusion matrices of fivefold cross-validation for Models 1–3. From Fig. 7a, we can note that while 717.4 COVID-19, 275 normal, and 816.2 pneumonia samples were accurately classified, 5.8 COVID-19, 40.2 normal, and 36.8 pneumonia samples were misclassified using Model 1. Thus, the correct classification rates of COVID-19 samples were 99.72%, while 87.62% for normal and 94.23% for pneumonia samples. It is shown in Fig. 7b, 718 COVID-19, 278.6 normal, and 821.2 pneumonia samples were classified correctly, and 5.2 COVID-19, 37.6 normal, and 31.8 pneumonia cases were misclassified using Model 2. As a result, 99.28%, 88.10%, and 96.27% correct classification rates of COVID-19, normal, and pneumonia were achieved, respectively. It can also be noted in Fig. 7c that 832.8 COVID-19, 292.2 normal, and 832.8 pneumonia samples were classified correctly, and 5.8 COVID-19, 23 normal, and 20.2 pneumonia cases were misclassified using Model 3. Thus, the correct classification rates of COVID-19, normal, and pneumonia were 99.20%, 92.76%, and 97.63%, respectively.

Fig. 7.

Fivefold cross-validation mean confusion matrices of Models 1–3

In addition to these evaluation metrics, the Friedman test, a popular nonparametric statistical test used to rank the algorithms based on their results without specifying any statistical difference, was performed for fivefold cross-validation results. We achieved chi-square test statistics of 8.0 and a p value of 0.0183. Friedman’s test results show that the p value is less than 0.05. Thus, there are significant differences in the methods. We also employed the Nemenyi post hoc test, used to determine the statistical difference with method pairs, to evaluate the performance of the proposed methods. Table 8 shows the analysis of the Nemenyi tests on the methods using a significance level of 5%. It can be seen from the table that the results of Model 1 and Model 3 have significant differences, and Model 3 achieves better results.

Table 8.

Statistical test results

Model	1	2	3
1	1.0000	0.3341	0.0129
2	0.3341	1.0000	0.3341
3	0.0129	0.3341	1.0000

Figures 8, 9, and 10 demonstrate the training process of fine-tuned Models 1–3 using 30–70% hold-out method, respectively. The green vertical line in the plots shows the 50th epoch, where fine-tuning started. We previously stated that we trained the classification layer added to the base model for 50 epochs and fine-tuned the whole model for additional 80 epochs in a total of 130 epochs. As shown in Fig. 8, validation accuracy flattened about 94% before fine-tuning, and after fine-tuning, it increased to about 95%.

Fig. 8.

The training process of Model 1

Fig. 9.

The training process of Model 2

Fig. 10.

The training process of Model 3

Colored vertical lines in Fig. 9 show stages of step fine-tuning. It is clearly seen that fine-tuning using Model 2 increased validation accuracy from 94% to about 97%. Beginning of the fine-tuning, validation accuracy fluctuated strongly. In later epochs, it settled down to better rates. It was also observed in Fig. 10 that after fine-tuning, validation accuracy increased sharply in early epochs.

Discussion

This study aims to develop a CNN-based COVID-19 diagnostic model using new fine-tuning mechanisms. It aims to bring novel approaches to the literature by incorporating different proposals into the methodological design. Most recent studies on this topic using DL methods are summarized in terms of accuracy metrics in Table 9.

Table 9.

Comparison of the proposed models with previous studies

Study	Methodology	Accuracy (%)
Rehman et al. (2020)	ResNet101, MobileNet	98.75
Shorfuzzaman and Masud (2020)	VGG16, ResNet50, Xception, MobileNet, and DenseNet121	99.26
Degadwala et al. (2021)	FT CNN	90.70
Minaee et al. (2020)	ResNet, SqueezeNet, and DenseNet121	98.00
Iskanderani et al. (2021)	DenseNet	96.25
Pathak et al. (2020)	ResNet32	96.22
Apostolopoulos and Mpesiana (2020)	MobileNetV2	96.78
Kaur et al. (2021)	AlexNet	99.52
Wang et al. (2020)	ResNet101 and ResNet152	96.10
Misra et al. (2020)	ResNet18	93.90
Punn and Agarwal (2021)	ResNet, Inception-v3, and NASNetLarge	97.00
Ucar and Korkmaz (2020)	Bayes-SqueezeNet(CNN)	98.30
Narin et al. (2021)	ResNets and Inception	96.10
Lee et al. (2020)	VGG16	95.00
Ismael and Şengür (2021)	ResNet50	92.60
Wang et al. (2020)	COVID-Net	94.00
Nishio et al. (2020)	VGG16	86.30
Monshi et al. (2021)	CovidX-RayNet	95.82
Das et al. (2021)	VGG16	97.67
Rahaman et al. (2020)	VGG19	89.30
Moujahid et al. (2020)	VGG19	96.97
Khan et al. (2020)	Coro-Net	95.00
Ohata et al. (2020)	DenseNet201	95.60
Manokaran et al. (2021)	DenseNet201	92.19
Garg et al. (2020)	DenseNet121	94.00
Naronglerdrit et al. (2021)	MobileNet	96.76
Xu et al. (2020)	ResNet + Location Attention	86.70
Ozturk et al. (2020)	DarkCovidNet	87.02
Asif and Wenhui (2020)	InceptionV3	96.00
Luz et al. (2021)	EfficientNet	93.90
Bargshady et al. (2022)	CycleGAN	94.20
Rahimzadeh and Attar (2020)	Xception + ResNet50V2	91.40
Model 1	MobileNetV2+Classical fine-tuning	95.62
Model 2	MobileNetV2+Step fine-tuning	96.10
Model 3	MobileNetV2+Exponential fine-tuning	97.61

As can be seen in Table 9, many studies have been conducted so far that include the CNN architecture. The major advantage of the CNN architecture is that it allows continuous learning. Three approaches are distinguished in the literature: trained from scratch DL models, TL approaches that use pre-trained models with fine-tuning and hybrid techniques that combine DL models with classical machine learning methods.

Some studies achieved high success rates in classification using a pre-trained CNN models (Iskanderani et al. 2021; Pathak et al. 2020; Kaur et al. 2021; Misra et al. 2020; Apostolopoulos and Mpesiana 2020; Luz et al. 2021; Lee et al. 2020; Ismael and Şengür 2021; Nishio et al. 2020; Das et al. 2021; Rahaman et al. 2020; Moujahid et al. 2020; Ohata et al. 2020; Manokaran et al. 2021; Garg et al. 2020; Naronglerdrit et al. 2021; Asif and Wenhui 2020). It is seen that different pre-trained CNN model designs attained varied results on the same data set (Rehman et al. 2020; Shorfuzzaman and Masud 2020; Minaee et al. 2020; Wang et al. 2020; Narin et al. 2021; Punn and Agarwal 2021; Rahimzadeh and Attar 2020). In Rehman et al. (2020) and (Iskanderani et al. 2021), the authors used binary classification (COVID-19, normal) and achieved more successful results.

To summarize these results, Table 9 includes various studies using CNN models that offer promising results. However, these studies use diverse CNN architectures, training samples, the number of classes, and design parameters. Thus, it is unfair to compare these studies directly. It can be seen from the table that some studies with decent results either have fewer training samples (Shorfuzzaman and Masud 2020; Kaur et al. 2021; Rehman et al. 2020) or have a binary classification (Minaee et al. 2020; Rehman et al. 2020). It is common knowledge that the model trained using a small number of training data results in a poor approximation. The model trained using fewer classification classes achieves better results. In other words, if the model contains a small amount of data or fewer classification classes, it will result in an uncertain and high variance estimate compared to its actual performance (Barbedo 2018).

Conclusions

The COVID-19 virus and pandemic might be called the disease one of the most significant challenges in human history. Many groups, including medical communities and scientific researchers, have attempted to find viable solutions to combat the disease. As a result of numerous studies, advances in artificial intelligence and medical imaging have brought hope for anomaly detection in medical images. Thus, many researchers have developed DL-based COVID-19 diagnosis systems. However, there were still some gaps to be developed.

In this work, we attempted to fill the gaps and presented a new TL-based deep CNN model that includes novel fine-tuning approaches to identify COVID-19. We tested three fine-tuning models, two of which differ from the classical fine-tuning approaches, providing decent COVID-19 detection using chest X-ray images. The first approach is the classical fine-tuning approach in which the last 50 layers of the CNN model are trained at 80 epochs, and the remaining layers are frozen. Unlike the classical fine-tuning approach, the second proposed approach relies on training the last 50 layers of the CNN model in descending order using a step function instead of training at once. Thus, the model achieved a higher success rate than the first approach by minimizing data loss and increasing the number of transferred features in fine-tuning operations. In the last fine-tuning approach, we performed training of the proposed model by determining the number of epochs and which layers would unfreeze for training according to a predetermined exponential function. The proposed second and third models attained promising results and achieved the correct classification rate of COVID-19 as 99.28% and 99,20%, respectively, using fivefold cross-validation. In addition, Model 3 achieved this result using only 18.07% of 4000 total fine-tuning training operations, as seen in Fig. 5. Our study offers a robust and reliable model that can be used as a decision support system to detect COVID-19 disease. Thus, computer-aided diagnosis systems can be created, and using those systems can reduce the X-ray evaluation time of the experts. The numerical results show that training more of the bottom layers of the model has the ability to generalize to the new problem and increase the classification results. Our approach could be accepted as an alternative COVID-19 diagnostic tool or contribute to the further development of diagnostic methods. In addition, the proposed fine-tuning mechanisms can also be applied to other medical applications, such as breast cancer and tumor detection.

For future work, meta-heuristics-based hyperparameter tuning (Kıymaç and Kaya 2023) can be implemented to optimize the model and to get better results. In addition, we plan to extend the proposed methodology to some other relevant pulmonary diseases. A lightweight DL model can be trained from scratch to classify COVID-19 cases and implemented on mobile devices for smart health applications.

Funding

Not applicable.

Data Availibility

The datasets analyzed during the current study are available in the chest X-ray (ChestX-RayImagesPneumonia 2022) and COVID-19 Radiography Database repositories (Covid-19RadiographyDatabase 2022), https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia, https://www.kaggle.com/tawsifurrahman/covid19-radiography-database.

Declarations

Conflict of interest

All of the authors declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.