Skip to main content
PLOS One logoLink to PLOS One
. 2023 Jun 7;18(6):e0286873. doi: 10.1371/journal.pone.0286873

Real-time scene classification of unmanned aerial vehicles remote sensing image based on Modified GhostNet

Xiaole Shen 1, Hongfeng Wang 1, Biyun Wei 1, Jinzhou Cao 1,*
Editor: Sunday Adeola Ajagbe2
PMCID: PMC10246849  PMID: 37285360

Abstract

Unmanned Aerial Vehicles (UAVs) play an important role in remote sensing image classification because they are capable of autonomously monitoring specific areas and analyzing images. The embedded platform and deep learning are used to classify UAV images in real-time. However, given the limited memory and computational resources, deploying deep learning networks on embedded devices and real-time analysis of ground scenes still has challenges in actual applications. To balance computational cost and classification accuracy, a novel lightweight network based on the original GhostNet is presented. The computational cost of this network is reduced by changing the number of convolutional layers. Meanwhile, the fully connected layer at the end is replaced with the fully convolutional layer. To evaluate the performance of the Modified GhostNet in remote sensing scene classification, experiments are performed on three public datasets: UCMerced, AID, and NWPU-RESISC. Compared with the basic GhostNet, the Floating Point Operations (FLOPs) are reduced from 7.85 MFLOPs to 2.58 MFLOPs, the memory is reduced from 16.40 MB to 5.70 MB, and the predicted time is improved by 18.86%. Our modified GhostNet also increases the average accuracy (Acc) (4.70% in AID experiments, 3.39% in UCMerced experiments). These results indicate that our Modified GhostNet can improve the performance of lightweight networks for scene classification and effectively enable real-time monitoring of ground scenes.

Introduction

Unmanned Aerial Vehicles (UAVs) as a remote sensing platform are used in various applications, such as search and rescue [1], disaster evaluation, and traffic monitoring [2], wireless communications [3]. Deep learning networks and embedded devices are equipped with UAVs to autonomously carry out tasks of image classification. An autonomous UAV rapidly monitors hazards and disasters by classifying the captured images in real-time, which relies heavily on its onboard sensors and microprocessors [4]. Local embedded devices are superior to cloud storage in scenarios involving privacy, latency, and limited connectivity. However, because embedded devices have limitations in memory and computing power, there are challenges to efficient scene classification.

Scene classification is an important problem for remote sensing image classification. In recent years, numerous methods have been applied to complete the task of remote sensing scene classification. These methods can be divided into three categories [5]: handcrafted feature-based methods [6–10], unsupervised-feature-learning-based methods [11–17], and deep-learning-based methods. Handcrafted feature-based methods include SIFT (scale-invariant feature transform) [18], SURF(speeded-up robust features), HOG (histogram of oriented gradients) [8], etc. The unsupervised feature-learning-based methods automatically learn features from unlabeled images. K-Means [19] is a typical method. Compared with these two methods, deep learning has two characteristics of feature learning and deep structure, which is beneficial to achieving higher classification accuracy [20]. Most deep learning methods are based on Convolutional Neural Networks (CNNs) [21–24]. By deepening the number of network layers, CNNs can improve classification accuracy, such as AlexNet [25], Network in Network (NIN) [26], VGGNet [22], deep residual networks (ResNet) [23], etc.

However, the inference speed of deep learning networks on embedded platforms is impeded by the high computational cost, which is prohibitive from the perspective of memory consumption and real-time performance [27, 28]. The large memory footprint and high latency of deep learning networks are unacceptable for embedded devices in some special situations [29, 30].

Lightweight networks have been extensively studied to address these problems. Iandola et al. [31] first proposed the lightweight network SqueezeNet, whose parameter size can be reduced to less than 5MB, and the network parameter size can be further reduced by network compression. However, the SqueezeNet cannot be used in embedded devices, due to the lack of real-time performance. In 2017, a lightweight network MobileNet V1 [32] specifically for mobile devices, was proposed by the Google team. The Face++ team learned from the idea of MobileNet V1 and proposed ShuffleNet [33]. ShuffleNet used the method of shuffling channels to solve the problem of “sluggish information flow” caused by depthwise separable convolution. Experiments indicated that its classification accuracy is higher than that of MobileNet V1. In subsequent research, MobileNet V2 [34], ShuffleNet V2 [35] and MobileNet V3 [36] were proposed successively. In 2020, Huawei Noah Lab, together with Peking University and the University of Sydney, jointly launched a lightweight network model called GhostNet [37] that uses linear operations instead of convolution operations to analyze the phenomenon of redundancy in feature images. GhostNet is widely used in image classification, semantic segmentation, instance segmentation, target detection, etc.

In this paper, we build a Modified GhostNet based on the original GhostNet that can be deployed on an embedded device for real-time classification of drone imagery scenes. Data augmentation methods are employed to expand the diversity of samples in the UAV image scene dataset, including geometric transformation, pixel color transformation, and compound transformation. These data augmentation methods are verified on three datasets, including UCMerced, AID, and NWPU-RESISC. The Modified GhostNet model is trained based on transfer learning. The weights obtained by training on NWPU-RESISC are used as pre-training weight parameters, and weight transfer training is used for UCMerced and AID. The performance of the Modified GhostNet is evaluated in terms of FLOPs, memory usage, predicted time, and Acc. Compared with MobileNetV3-Small and the original GhostNet, the Modified GhostNet proposed in this paper reduces the amount of computation, thus improving real-time processing rate while reducing memory usage.

Materials and methods

The contribution of this research is the development of a lightweight network based on the GhostNet model using visualization operation and structural compression techniques. We analyze the basic architecture of the GhostNet model and then adjust the structure of the network, developing a Modified GhostNet model that has less network complexity while ensuring the accuracy of the model. The Modified GhostNet model designed in this paper can be deployed on an embedded device for image scenes classification with high requirements for both accuracy and real-time processing.

GhostNet model

The GhostNet model is a lightweight network model jointly launched by Huawei Noah Lab, Peking University, and the University of Sydney in 2020. The model determined that during the training process of the network, there will be feature redundancy in each output layer in the middle. The conclusion is that a linear operation with a lower computational cost can replace a part of the convolution operation with a higher computational cost to reduce the amount of computation and save computing resources. The author proposes the Ghost module, which is characterized by replacing half of the convolution operations with linear operations, to reduce the computational complexity of the model. The structure of original GhostNet model is shown in Fig 1, including ordinary convolutional layers and Ghost module.

Fig 1. Structure of original GhostNet model.

Fig 1

a. The convolutional layer, b. The Ghost module.

The compression ratio between the network model after using the Ghost module and the network model with pure convolution operation can reach 2 times. Suppose n is the number of feature maps that a convolution layer should produce, m represents the actual number of feature maps output by the convolution operation, and s represents the factor of the linear operation. Since the general convolution operation is replaced by a linear operation in this paper, s = 2. Then, according to the principle that the total number of feature maps output by a certain layer should remain unchanged, we know that n = m ⋅ s, which is equivalent to m·(s-1)=ns·(s-1). Suppose the average size of the convolution kernel for each linear operation is d × d. The compression ratio formula between the network model after using the Ghost module and the network model with pure convolution operation can be obtained by calculation, which is shown below, where c is the number of input channels, and k is the size of the convolution kernel of the convolution operation.

rc=n·c·k·kns·c·k·k+s-1s·d·d≈s·cs+c-1≈s (1)

The author designed two residual structures that make up the Ghost bottlenecks of GhostNet model, and two structures with a stride of 1 and a stride of 2. The two structures are shown in Fig 2. Ghost bottleneck is mainly composed of two Ghost modules stacked. For stride of 2, the shortcut path uses the downsampling layer and inserts depthwise convolution with the stride of 2 in the middle of the Ghost module. The original GhostNet model is formed by stacking the bottlenecks of the two sub-network models. The overall architecture is shown in Table 1.

Fig 2. Structure of Ghost bottlenecks.

Fig 2

Table 1. Original overall architecture of GhostNet.

Input Operator Expansion channel Output channel SE Stride
256*256*3 Conv2d 3*3 - 16 - 1
112*112*16 GBneck 16 16 1 2
56*56*16 GBneck 4 24 - 1
28*28*24 GBneck 72 24 - 1
28*28*24 GBneck 72 40 1 2
14*14*40 GBneck 120 40 1 1
14*14*40 GBneck 240 80 - 2
14*14*80 GBneck 200 80 - 1
14*14*80 GBneck 184 80 - 1
14*14*80 GBneck 184 80 - 2
14*14*80 GBneck 480 112 1 1
7*7*160 GBneck 960 160 - 1
7*7*160 GBneck 960 160 1 1
7*7*160 GBneck 960 160 - 1
7*7*160 GBneck 960 160 1 1
7*7*160 Conv2d 1*1 - 960 - 1
7*7*960 GAP 7*7 - - - -
1*1*960 Conv2d 1*1 - 1280 - 1
1*1*1280 FC - 1000 - -

Modified GhostNet model

In this paper, we propose a network model based on structural compression. The feature map redundancy phenomenon is analyzed when the GhostNet model is applied to the UAV image scene classification dataset. We remove the Ghost bottlenecks that generate redundant feature maps in the GhostNet model and replace the final fully connected layer with a fully convolutional layer. The Modified GhostNet has lower memory usage and energy consumption. It is well known that computationally intensive models drain batteries quickly for embedded devices. The same embedded device processes a model with a large amount of calculation. If the battery can only support 30 minutes, then reduce the amount of calculation by half, and it can work for at least 10 more minutes. Therefore, reducing the computational load of network models is crucial for supporting models in embedded devices. The improvement scheme of the GhostNet model is mainly based on the following two points:

  1. The 2nd, 4th, 9th, 10th, 15th, and 17th layers in the GhostNet model are extracted. Through the visualization operation, it is found that there is a large amount of feature redundancy between these layers and their adjacent layers;

  2. The fully connected layer at the end is replaced with a fully convolutional layer to reduce the amount of network computation caused by the fully connected layer.

The structure of Modified GhostNet network model based on these two points is shown in Table 2. K is the number of classification samples, and SE indicates whether to use the Squeeze-And-Excite module.

Table 2. Modified overall architecture of GhostNet.

Input Operator Expansion channel Output channel SE Stride
256*256*3 Conv2d 3*3 - 16 - 2
112*112*16 GBneck 3*3 16 16 1 2
5the 6*56*16 GBneck 3*3 72 24 - 2
28*28*24 GBneck 5*5 88 24 - 1
28*28*24 GBneck 5*5 96 40 1 2
14*14*40 GBneck 5*5 240 40 1 1
14*14*40 GBneck 5*5 120 48 1 1
14*14*48 GBneck 5*5 144 48 1 1
14*14*48 GBneck 5*5 288 96 1 2
7*7*96 GBneck 5*5 576 96 1 1
7*7*96 GBneck 5*5 576 96 1 1
7*7*96 Conv2d 1*1 - 576 - 1
7*7*576 GAP 7*7 - - - -
1*1*576 Conv2d 1*1 - 1024 - 1
1*1*1024 Conv2d 1*1 - K - -

Experiment

Dataset

With the development of neural networks, the number of data samples, the amount of computation, and the algorithm have become the three key factors that have the greatest impact on the neural network model. Among them, sample data is the basis for all research work. When the dataset used in the research is not suitable, that is, the number of samples in the dataset is scarce or the data samples do not have diversity, it will be difficult to carry out the research work smoothly [38]. Brill et al. [39] gave relevant conclusions on this problem. The same problem has almost the same performance on different algorithms. If the sample data is increased, the overall accuracy of the algorithm will be improved to a certain extent.

The quality of the dataset is reflected in the richness and diversity of data samples. For natural images, the same type of samples should have different shapes, angles, sizes, etc. For UAV images, the same type of samples should include different factors such as climate change, illumination change, viewing angle change, and spatial resolution change. The current public UAV image scene datasets have some deficiencies. The commonly used public UAV image scene datasets are shared in Table 3, including UCMerced, AID and NWPU-RESISC [40].

Table 3. The parameters of UAV images dataset.

Dataset Samples Categories Number of per category Image size Spatial resolution
UCMerced 2100 21 100 200 × 200 0.3m
AID 10,000 30 200-400 600 × 600 0.5-0.8m
NWPU-RESISC 31,500 45 700 256 × 256 2-30m

UCMerced is a dataset with 21 categories that was released by the Computer Vision Laboratory of the University of California in 2010, which is the earliest UAV image dataset collected. However, the number of various types of samples in this dataset is relatively small, and the number of pictures in each category is only 100. Additionally, the scene information involved in this dataset is limited to American cities, and the diversity of samples is lacking.

Wuhan University and Huazhong University of Science and Technology released a UAV image dataset called AID. The dataset contains 30 categories of data, a total of 10,000 UAV images, and the resolution of each image is 600 × 600. There are about 220-420 pictures. Although the dataset has some variation in spatial resolution, the number of images per category is inconsistent.

NWPU-RESISC is a large-scale benchmark dataset created by Northwestern Polytechnical University in 2016, and it is the most commonly used dataset for validating network performance. Although there are certain advantages compared to the first two datasets, the number of types of samples in the dataset is relatively small, and the image data of similar samples are basically ideal environments and lack sample diversity.

Data preprocessing

The dataset used in this paper has a small capacity, and the similarity of the data samples is extremely high, which may lead to an overfitting problem in the network model. Therefore, we utilize image augmentation methods to expand the capacity of image scene data. The image augmentation methods are used to enrich the scene information in different environments and improve the generalization capacity of the network. The approaches include geometric transformation, pixel color transformation, and complex transformation. The geometric transformation includes flipping, rotating, translating, scaling, cropping, and other operations. Pixel color transformation includes noise interference and blurring. Combination of different rotation angles and different noise is shown in Fig 3. In addition, the composite transformation is important for enriching the sample characteristics of UAV image datasets. Compound transformation is the combination of geometric transformation and pixel color transformation. Combining rotation and noise interference can simulate complex sample data taken from different directions. The combination of rotation, brightness, and contrast augmentation can simulate pictures in different lighting conditions and in different shooting orientations. The combination of different rotation angles and different brightness and contrast augmentation is shown in Fig 4.

Fig 3. Combination of rotation and noise interference.

Fig 3

a) 90° + high noise, b) 180° + high noise, c) 270° + high noise, d) 90° + low noise, e) 180° + low noise, f) 270° + low noise.

Fig 4. Combination of rotation and different brightness and contrast augmentation.

Fig 4

(a) 90° + α = 1.4, β = 0.6. (b) 180° + α = 1.4, β = 0.6. (c) 270° + α = 1.4, β = 0.6. (d) 90° + α = 0.8, β = 1.2. (e) 180° + α = 0.8, β = 1.2. (f) 270° + α = 0.8, β = 1.2.

We use transfer learning techniques to generate the Modified GhostNet model. Transfer learning means that knowledge learned in one domain (such as knowledge learned in natural scenes) is applied to another domain (such as defect detection or drone images) to improve its generalization ability [41]. The pre-training model in this paper is the model trained on NWPU-RESISC. The weights generated in the model are set as the initial weights of AID and UCMerced, and then the network model is fine-tuned.

The neural network model must have feature redundancy during the training process. This paper uses the GhostNet network model to train using the NWPU-RESISC dataset and visualizes the feature maps of the output channels from the 8th to 11th layers and observes the continuous output layer. There is redundancy of feature maps. Here, 20% of the pictures are randomly selected for comparison, as shown in Fig 5. Feature redundancy is revealed in the training process for the UAV image dataset while using original GhostNet model, which is improved by Modified GhostNet.

Fig 5. Feature maps derived from different layers.

Fig 5

(a) The 8th layer. (b) The 9th layer. (c) The 10th layer. (d) The 11th layer.

Experiment environment

To ensure the training speed of the model, the model training system environment used in this experiment is Ubuntu16.04, and the hardware platform is Quadro P5000. The model was built on the Keras deep learning framework. During the training process, the Adam is selected as optimizer, the initial learning rate is set to 0.001, the training rounds are set to 100, and the batch size is set to 8. The image resolution in the dataset is set to 256 × 256, and the same image augmentation operations are performed, including geometric transformation, pixel color transformation, and composite transformation.

Since the purpose of this project is to apply the lightweight network model to embedded devices, the trained network model is transplanted to the embedded device for prediction. The embedded device selected in this experiment is the Jetson TX2. The test method in this paper mainly adopts the simulation test. The embedded device is connected to a camera through USB, and the camera is treated to recognize the picture, and the recognized category is drawn in the upper left corner of the picture in the form of characters. The display screen is used to display the ground object information captured by the drone. On the left side of the Jetson TX2 device is a monocular camera, which is used to capture the ground object information.

Training strategy

To ensure the training speed of the model, the model training system environment used in this experiment is Ubuntu16.04, and the hardware platform is Quadro P5000. The building of the model was done on the Keras deep learning framework.

In this paper, the capacity of the dataset is expanded through the methods of image augmentation, which prevents overfitting in network training and improves the generalization ability of the network. The image resolution in the dataset is set to 256 × 256. The dataset is shuffled and divided into three parts in a ratio of 6:2:2, including training set, validation set and test set. During the training process, the Adam optimizer is used, the initial learning rate (lr) is set to 0.001, the training rounds are set to 100, the batch size is set to 8, and dropout ratio is set to 5%. The entire training strategy of Modified GhostNet model is shown in Fig 6. Val Loss is the loss of validation set. Training complete means training rounds reach 100 and ε is a constant value.

Fig 6. The entire training strategy of the Modified GhostNet model.

Fig 6

Results

Image augmentation

To verify the generality of the scheme, this paper will conduct experiments on three small UAV image scene classification datasets: UCMercedx, AID, and UCMerced. Each sample data is expanded by 20 times the original, thus the total number of samples in the dataset is also 20 times the original. The division ratio of the training set, validation set, and test set of the dataset is 6:2:2. The number of a single category in training set before and after augmentation is shown in Table 4. The number of training samples is greatly increased, providing rich sample features for model training.

Table 4. The samples of a single category in training set before and after augmentation.

Dataset Samples (before) Samples (after)
UCMerced 60 1200
AID 120 2400
NWPU-RESISC 420 8400

The dataset after image augmentation is employed trained on GhostNet. The average accuracy of GhostNet using dataset after image augmentation is improved, which is revealed in Table 5.

Table 5. The average accuracy of GhostNet before and after image augmentation.

Dataset Acc. (before) Acc. (after)
UCMerced 79.60% 92.80%
AID 85.25% 87.35%
NWPU-RESISC 88.36% 91.45%

UCMerced

The loss and accuracy of the GhostNet model before and after UCMerced image augmentation are shown in Fig 7. The red line represents the change trend after image augmentation, the blue line represents the change trend before image augmentation, and steps represent the training rounds.

Fig 7. The loss and accuracy of the GhostNet model before and after UCMerced image augmentation.

Fig 7

(a) Loss of training set. (b) Accuracy of training set. (c) Loss of validation set. (d) Accuracy of validation set.

In Fig 7, figure(a) indicates that the convergence rate of the network model on UCMerced after image augmentation is slower than that before image augmentation. Figure(c) shows that the convergence of the network model before image augmentation on the validation set is always jittered within a certain range, and the convergence of the network model after image augmentation on the validation set maintains a relatively stable trend after 50 rounds of training. In comparing figure(b) and figure(d), it can be seen that the network model before image augmentation has a very serious overfitting phenomenon. The accuracy of the blue line is equal to the red line in the training set, and its accuracy is close to 100%. However, the accuracy of the blue line in the validation set is less than 80%, and the red line is not much different from the red line in the training set. The experimental results indicate that the preprocessing method of image augmentation is able to alleviate the overfitting phenomenon well.

AID

Before experimenting with AID, this paper reduces the sample resolution in AID to 256 × 256, which is consistent with the resolution of the other two datasets. Fig 8 shows the loss and accuracy of the GhostNet model before and after AID image augmentation.

Fig 8. The loss and accuracy of the GhostNet model before and after AID image augmentation.

Fig 8

(a) Loss of training set. (b) Accuracy of training set. (c) Loss of validation set. (d) Accuracy of validation set.

As shown in Fig 8, compared with the decrease in the loss of the training set before image augmentation, the value loss after image augmentation is lower, indicating that its convergence occurs before image augmentation. In figure(a), the loss before data enhancement is always in a jitter state, which is caused by the imbalance of sample data in the dataset before data enhancement. As shown in figure(b) and figure(d), the AID fitting situation before and after data enhancement is similar, but the overall accuracy after data enhancement is higher.

NWPU-RESISC

Fig 9 shows the loss and accuracy of the GhostNet model before and after NWPU-RESISC image augmentation. The red line represents the change trend after data enhancement, the blue line represents the change trend before data enhancement, and steps represent the training rounds.

Fig 9. The loss and accuracy of the GhostNet model before and after NWPU-RESISC image augmentation.

Fig 9

(a) Loss of training set. (b) Accuracy of training set. (c) Loss of validation set. (d) Accuracy of validation set.

From figure(a) and figure(c), it can be seen that the loss of NWPU-RESISC after image augmentation is lower on the training set and validation set after 70 rounds of training. Figure(b) and figure(d) indicate that the accuracy of the validation set and training set has a small degree of overfitting before and after data enhancement, and the overall accuracy after data enhancement is higher than before image augmentation. Experimental results reveal that image augmentation can, indeed, improve the accuracy of the network to a certain extent.

Comparison of models

To verify the validity of the GhostNet model based on structure compression, we compare MobileNetV3-Small, GhostNet, and the Modified GhostNet models through different evaluation metrics, including FLOPs, memory usage, predicted time, and average accuracy.

FLOPs

In lightweight neural networks, Floating Point Operations (FLOPs) are commonly used to measure the complexity of the network model. FLOPs represent floating point operands, also known as computations. The calculation formula of FLOPs is:

FLOPsCNN=H×W×Cin×Cout×F2 (2)

Among them, H and W represent the height and width of the input feature map, respectively, Cin represents the number of channels of the input feature map, Cout represents the number of channels of the output feature map, and F represents the size of the convolution kernel.

Since the amount of computation generated in the model has nothing to do with the dataset and the structure of the model itself, only the amount of data on one dataset is shown below, and the others are the same. Table 6 shows the FLOPs of different models on UCMerced dataset. The amount of computation of the Modified GhostNet model is reduced by as much as 3 times that of the original GhostNet model. Compared with MobileNetV3-Small, it is also reduced by nearly 3 times.

Table 6. FLOPs of different models on UCMerced dataset.
Model FLOPs (M)
MobileNetV3-Small 6.09
GhostNet 7.85
Modified GhostNet 2.58

Memory usage and predicted time

Embedded devices have high requirements on model memory usage and real-time performance. The reduction of the parameter quantity is accompanied by the reduction of the memory occupancy rate, and the reduction of the calculation quantity is accompanied by the reduction of the operation time. The combined effect of these two factors is important for the predicted speed of the network model. The memory usage and predicted time of different models are shown in Table 7.

Table 7. The memory usage and predicted time of different models.
Model Memory usage (MB) Predicted time on Jetson TX2 (ms) Predicted time on Quadro P5000 (ms)
MobileNetV3-Small 12.7 47.7 18.2
Original GhostNet 16.4 52.5 19.1
Modified GhostNet 5.7 42.6 17

The quantitative results show that the amount of the Modified GhostNet’s parameters is reduced by nearly 3 times that of the original GhostNet model, and the overall prediction time is also reduced from 52.5ms to 42.6ms. The Modified GhostNet model also shows its overall performance advantage over the MobileNetV3-Small model, both in terms of memory usage and forward inference time.

Acc

Except for FLOPS, memory usage, and real-time performance, average accuracy (Acc) of models deployed on embedded devices is one of the most critical metrics. Acc of different models on the three datasets is displayed in Table 8.

Table 8. The average accuracy of different models.
Model Dataset Acc. (%)
MobileNetV3-Small UCMerced 92.38
Original GhostNet UCMerced 92.80
Modified GhostNet UCMerced 96.19
MobileNetV3-Small AID 87.25
Original GhostNet AID 87.35
Modified GhostNet AID 92.05
MobileNetV3-Small NWPU-RESISC 91.36
Original GhostNet NWPU-RESISC 91.45
Modified GhostNet NWPU-RESISC 91.73

Dropout with the ratio of 0.05 and weight transfer are employed in the Modified GhostNet model proposed in this paper. The average accuracy of three models on three dataset indicate that the Modified GhostNet has higher accuracy. On the UCMerced dataset, the average accuracy of Modified GhostNet is 96.19%, which performs better than MobileNetV3-Small and original GhostNet. The average accuracy of Modified GhostNet is 92.05% on AID dataset, which is much higher than MobileNetV3-Small and original GhostNet. The Modified GhostNet has a certain improvement in accuracy compared with the original network structure, and the UCMerced and AID datasets have increased by 3.39% and 4.70% respectively. However, the average accuracy of the three models is similar on NWPU-RESISC dataset, and Modified GhostNet increased by 0.28% compared with original GhostNet.

The effect of dropout position on average accuracy is shown in Fig 10. Experiments are performed on three datasets, and dropout operations are performed on the 5th, 7th, 9th, and 11th layer of the Modified GhostNet. The dropout operation can improve the classification accuracy of the model, and the dropout in different positions has different effects on the classification accuracy of the model. In UCMerced and AID, adding dropout to the 9th layer can greatly improve the accuracy of the model, while in NWPU-RESISC the introduction of dropout at the 7th layer was most effective.

Fig 10. The average accuracy of dropout position.

Fig 10

For small datasets, fine-tuning the network model by weight transfer can usually achieve good results. Weight transfer is a technique commonly used in transfer learning, where a pre-trained model on a large dataset is used as a starting point for training a model on a smaller dataset. NWPU-RESISC dataset is a larger dataset, while UCMerced and AID datasets are smaller. In this experiment, the weights obtained by training on NWPU-RESISC are used as pre-training weight parameters, and weight transfer training is used for UCMerced and AID. The policy tunes the network model to fit the respective dataset. As shown in Fig 11, the figure shows the change in the classification accuracy of the model before and after using transfer learning. The quantitative results show that adjusting the network model by weight transfer can avoid the phenomenon of network overfitting and effectively improve the prediction accuracy of the network.

Fig 11. The average accuracy of the model before and after using transfer learning.

Fig 11

Discussion

The evaluation metrics examined in this study were FLOPs, memory usage, predicted time, and Acc. The values obtained from these criteria in the modified network are as follows. The FLOPs was 2.58 MFLOPs, the memory usage was 5.7 MB, the predicted time on Jetson TX2 was 42.6 ms, the Acc On the UCMerced dataset was 96.19, and the Acc on the AID dataset was 92.05. Based on the basic GhostNet results, the proposed network had an average improvement of 5.27 in FLOPs, 18.86% in predicted time, 3.39% in Acc on UCMerced dataset, and 4.70% in Acc on AID dataset, indicating an improvement in the classification of the proposed model.

In recent years, the employment of artificial intelligence and deep learning methods has become one of the most popular and useful approaches in scene classification. LeCun et al. [42] introduced the LeNet5 convolutional neural network model. Lin et al. [26] proposed Network in Network (NIN). The entire network model was formed by stacking sub-networks and could be changed arbitrarily. Based on the idea of increasing network depth, VGG [22] increased the depth of the CNN model to about 20 layers. The error rate of image recognition has also decreased from 11% to 6.7%, which is gradually approaching the human eye error rate of 5.1%. He et al [23] formulated ResNet, improving the network accuracy by increasing the network depth. However, the real-time performance of these studies still needs improvement. In addition, one of the limitations of these studies was that the computing performance of most embedded devices cannot support the deep convolutional neural network model.

Previous research has focused on improving the accuracy, with a few studies being actually applied to embedded environments. Moreover, no matter how high the classification accuracy is, the experimental conditions are tested on high-performance equipment in the laboratory, and the network model does not go out of the laboratory and into practice. Thus, it can be said that porting neural network models to embedded devices is still an important issue for UAV image scene classification. The main goal of this study is to reduce computational costs and apply the lightweight network model to real-time scene classification of UAV images.

In this study, the GhostNet was modified to improve the challenges of distinguishing ground objects from UAV images in real-time. The Modified GhostNet model can not only reduce the memory and calculation amount of the model, thereby improving the prediction speed of the model, but also improve the accuracy of some data sets. Because the network model with high complexity easily causes overfitting when training on small data sets, the structure compression of the network model can reduce the complexity of the network and alleviate the overfitting phenomenon to some extent. Compared with MobileNetV3 and the basic GhostNet, the Modified GhostNet has faster speed and higher classification accuracy.

Conclusion

Based on the original GhostNet, we built a lightweight neural network, the Modified GhostNet model, which can be transplanted on an embedded device for real-time UAV classification of drone imagery scenes. We utilize image augmentation methods to expand the sample diversity in the UAV image dataset, including geometric transformation, pixel color transformation, and compound transformation. The Modified GhostNet model is transplanted to the embedded device Jetson TX2, which is trained on three datasets based on transfer learning. The performance of the Modified GhostNet is evaluated in FLOPs, memory usage, predicted time, and Acc. Compared with MobileNetV3-Small and original GhostNet, the Modified GhostNet proposed in this paper reduces the amount of computation, improving real-time processing rate while reducing memory usage.

Data Availability

All Files are available from the NWPU-RESISC database (url(s) https://hyper.ai/datasets/5449), UCMerced database (url(s) https://hyper.ai/datasets/5431), and AID database (url(s) https://hyper.ai/datasets/5446).

Funding Statement

Funding: This research is supported by National Natural Science Foundation of China (Grant No. 42001393, 41501370 and 62176165), the 5th College-enterprise Cooperation Project of Shenzhen Technology University (Grant No. 2021010802014), Shenzhen Science and Technology Program (Grant No. JCYJ20220530152817039), and Guangdong Science and Technology Strategic Innovation Fund (the Guangdong–Hong Kong-Macau Joint Laboratory Program, Grant No. 2020B1212030009). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Petrides P, Kyrkou C, Kolios P, Theocharides T, Panayiotou CG. Towards a holistic performance evaluation framework for drone-based object detection. 2017 International Conference on Unmanned Aircraft Systems (ICUAS). 2017; p. 1785–1793.
  • 2.Kyrkou C, Timotheou S, Kolios P, Theocharides T, Panayiotou C. Optimized vision-directed deployment of UAVs for rapid traffic monitoring. 2018 IEEE International Conference on Consumer Electronics (ICCE). 2018; p. 1–6.
  • 3.Affum EA, Adigun MO, Boateng KA, Ajagbe SA, Addo E. Enhancing UAV communication performance: Analysis using interference based geometry stochastic model and successive interference cancellation. In: Computational Science and Its Applications–ICCSA 2022: 22nd International Conference, Malaga, Spain, July 4–7, 2022, Proceedings, Part I. Springer; 2022. p. 232–245.
  • 4.Kyrkou C, Theocharides T. Deep-Learning-Based Aerial Image Classification for Emergency Response Applications Using Unmanned Aerial Vehicles. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2019; p. 517–525.
  • 5. Cheng G, Xie X, Han J, Guo L, Xia G. Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2020;13:3735–3756. doi: 10.1109/JSTARS.2020.3005403 [DOI] [Google Scholar]
  • 6. Haralick RM, Shanmugam KS, Dinstein I. Textural Features for Image Classification. IEEE Trans Syst Man Cybern. 1973;3:610–621. doi: 10.1109/TSMC.1973.4309314 [DOI] [Google Scholar]
  • 7. Ojala T, Pietikäinen M, Mäenpää T. Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns. IEEE Trans Pattern Anal Mach Intell. 2002;24:971–987. doi: 10.1109/TPAMI.2002.1017623 [DOI] [Google Scholar]
  • 8.Dalal N, Triggs B. Histograms of oriented gradients for human detection. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). 2005;1:886–893 vol. 1.
  • 9. Oliva A, Torralba A. Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. International Journal of Computer Vision. 2004;42:145–175. doi: 10.1023/A:1011139631724 [DOI] [Google Scholar]
  • 10.Lazebnik S, Schmid C, Ponce J. Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06). 2006;2:2169–2178.
  • 11. Cheriyadat AM. Unsupervised Feature Learning for Aerial Scene Classification. IEEE Transactions on Geoscience and Remote Sensing. 2014;52:439–451. doi: 10.1109/TGRS.2013.2241444 [DOI] [Google Scholar]
  • 12. Mekhalfi ML, Melgani F, Bazi Y, Alajlan NA. Land-Use Classification With Compressive Sensing Multifeature Fusion. IEEE Geoscience and Remote Sensing Letters. 2015;12:2155–2159. doi: 10.1109/LGRS.2015.2453130 [DOI] [Google Scholar]
  • 13. Zheng X, Sun X, Fu K, Wang H. Automatic Annotation of Satellite Images via Multifeature Joint Sparse Coding With Spatial Relation Constraint. IEEE Geoscience and Remote Sensing Letters. 2013;10:652–656. doi: 10.1109/LGRS.2012.2216499 [DOI] [Google Scholar]
  • 14. Zhong Y, Zhu Q, pei Zhang L. Scene Classification Based on the Multifeature Fusion Probabilistic Topic Model for High Spatial Resolution Remote Sensing Imagery. IEEE Transactions on Geoscience and Remote Sensing. 2015;53:6207–6222. doi: 10.1109/TGRS.2015.2435801 [DOI] [Google Scholar]
  • 15. Lu X, Zheng X, Yuan Y. Remote Sensing Scene Classification by Unsupervised Representation Learning. IEEE Transactions on Geoscience and Remote Sensing. 2017;55:5148–5157. doi: 10.1109/TGRS.2017.2702596 [DOI] [Google Scholar]
  • 16. Fan J, Chen T, Lu S. Unsupervised Feature Learning for Land-Use Scene Recognition. IEEE Transactions on Geoscience and Remote Sensing. 2017;55:2250–2261. doi: 10.1109/TGRS.2016.2640186 [DOI] [Google Scholar]
  • 17. Romero A, Gatta C, Camps‐Valls G. Unsupervised Deep Feature Extraction for Remote Sensing Image Classification. IEEE Transactions on Geoscience and Remote Sensing. 2016;54:1349–1362. doi: 10.1109/TGRS.2015.2478379 [DOI] [Google Scholar]
  • 18. Lowe DG. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision. 2004;60:91–110. doi: 10.1023/B:VISI.0000029664.99615.94 [DOI] [Google Scholar]
  • 19.Zheng J, Cui Z, Liu A, Jia Y. A K-Means Remote Sensing Image Classification Method Based On AdaBoost. 2008 Fourth International Conference on Natural Computation. 2008;4:27–32.
  • 20. Xia GS, Hu J, Hu F, Shi B, Bai X, Zhong Y, et al. AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification. IEEE Transactions on Geoscience and Remote Sensing. 2017;55:3965–3981. doi: 10.1109/TGRS.2017.2685945 [DOI] [Google Scholar]
  • 21. Nex F, Remondino F. UAV for 3D mapping applications: a review. Applied Geomatics. 2014;6:1–15. doi: 10.1007/s12518-013-0120-x [DOI] [Google Scholar]
  • 22.Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556. 2014;.
  • 23.He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016; p. 770–778.
  • 24.Huang G, Liu Z, Weinberger KQ. Densely Connected Convolutional Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017; p. 2261–2269.
  • 25. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Communications of the ACM. 2012;60:84–90. doi: 10.1145/3065386 [DOI] [Google Scholar]
  • 26.Lin M, Chen Q, Yan S. Network in network. arXiv preprint arXiv:13124400. 2013;.
  • 27.Wang Y, Quan Z, Li J, Han Y, Li H, Li X. A retrospective evaluation of energy-efficient object detection solutions on embedded devices. 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE). 2018; p. 709–714.
  • 28.Shafique MA, Theocharides T, Bouganis CS, Hanif MA, Khalid F, Hafiz R, et al. An overview of next-generation architectures for machine learning: Roadmap, opportunities and challenges in the IoT era. 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE). 2018; p. 827–832.
  • 29.Chen W, Wilson JT, Tyree S, Weinberger KQ, Chen Y. Compressing Neural Networks with the Hashing Trick. ArXiv. 2015;abs/1504.04788.
  • 30.Zhao W, Fu H, Luk W, Yu T, Wang S, Feng B, et al. F-CNN: An FPGA-based framework for training Convolutional Neural Networks. 2016 IEEE 27th International Conference on Application-specific Systems, Architectures and Processors (ASAP). 2016; p. 107–114.
  • 31.Iandola FN, Moskewicz MW, Ashraf K, Han S, Dally WJ, Keutzer K. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size. ArXiv. 2016;abs/1602.07360.
  • 32.Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. ArXiv. 2017;abs/1704.04861.
  • 33.Zhang X, Zhou X, Lin M, Sun J. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018; p. 6848–6856.
  • 34.Sandler M, Howard AG, Zhu M, Zhmoginov A, Chen LC. MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018; p. 4510–4520.
  • 35.Ma N, Zhang X, Zheng H, Sun J. ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. ArXiv. 2018;abs/1807.11164.
  • 36.Howard AG, Sandler M, Chu G, Chen LC, Chen B, Tan M, et al. Searching for MobileNetV3. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019; p. 1314–1324.
  • 37.Han K, Wang Y, Tian Q, Guo J, Xu C, Xu C. GhostNet: More Features From Cheap Operations. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020; p. 1577–1586.
  • 38. pei Zhang L, Zhang L, Du B. Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art. IEEE Geoscience and Remote Sensing Magazine. 2016;4:22–40. doi: 10.1109/MGRS.2016.2540798 [DOI] [Google Scholar]
  • 39.Brill E, Lin JJ, Banko M, Dumais ST, Ng AY. Data-Intensive Question Answering. In: Text Retrieval Conference; 2001.
  • 40. Cheng G, Han J, Lu X. Remote Sensing Image Scene Classification: Benchmark and State of the Art. Proceedings of the IEEE. 2017;105:1865–1883. doi: 10.1109/JPROC.2017.2675998 [DOI] [Google Scholar]
  • 41. Pan SJ, Yang Q. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering. 2010;22:1345–1359. doi: 10.1109/TKDE.2009.191 [DOI] [Google Scholar]
  • 42. LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998;86:2278–2324. doi: 10.1109/5.726791 [DOI] [Google Scholar]

Decision Letter 0

Sunday Adeola Ajagbe

24 Apr 2023

PONE-D-23-05971Real-time Scene Classification of UAV Remote Sensing Image based on Modified GhostNetPLOS ONE

Dear Dr. Cao,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 08 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sunday Adeola Ajagbe

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

3. Thank you for stating the following financial disclosure:

“Funding: This research is supported by “National Natural Science Foundation of China (NSFC)”(https://www.nsfc.gov.cn/) (Grant No.  41501370 and 62176165),  and “the 5th College-enterprise Cooperation Project of Shenzhen Technology University” (https://english.sztu.edu.cn/)(Grant No. 2021010802014).”

Please state what role the funders took in the study.  If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

4. We note that Figure 3 in your submission contain [map/satellite] images which may be copyrighted. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, Street View, and Earth). For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

a. You may seek permission from the original copyright holder of Figure 3 to publish the content specifically under the CC BY 4.0 license. 

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

b. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

The following resources for replacing copyrighted map figures may be helpful:

USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/

The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/

Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html

NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/

Landsat: http://landsat.visibleearth.nasa.gov/

USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain): http://eros.usgs.gov/#

Natural Earth (public domain): http://www.naturalearthdata.com/

5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

The application of UAV can be enriched by description of the introduction section. The authors can refer to papers

Affum, E. A., Adigun, M. O., Boateng, K. A., Ajagbe, S. A., Addo, E. (2022). Enhancing UAV Communication Performance: Analysis Using Interference Based Geometry Stochastic Model and Successive Interference Cancellation. In: Gervasi, O., Murgante, B., Hendrix, E.M.T., Taniar, D., Apduhan, B.O. (eds) Computational Science and Its Applications – ICCSA 2022. ICCSA 2022. Lecture Notes in Computer Science, vol 13375. Springer, Cham. https://doi.org/10.1007/978-3-031-10522-7_17

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors should write UAV in full for the paper title.

Under the Abstract section, what does the authors mean by M as the measurement unit? Is it MB or Mb or M what?

Generally,

* Placing the Figures very far from where it was cited in the body of the paper makes it difficult to really see if the

Figures are in order for the purpose mentioned or not. E.g. There are 2 Figures (a & b) on the page referred to

for Figure 1.

* Table 1 content are values for different parameters and not a model.

* From the 2 bottlenecks shown in Figure 2, there seems to be no difference between the 2 bottlenecks,

apart from the number of iterations (2 & 3 respectively). If that is the case, then the authors

can just use one figure and include in the explanation the 2 & 3 iterations.

* First paragraph under Discussion section, the authors should include correct measurement units for their

respective evaluation metrics.

* Correct the 1st phrase in your conclusion section. "Based on the original GhostNet,..."

* The authors should be consistent about their evaluation metrics. Under Discussion section, it was stated that

"The evaluation metrics examined in this study were FLOPs, memory usage, predicted time and Acc...".

However, under Conclusion and Abstract section, it was stated that "The performance of the Modified GhostNet

is evaluated in terms of calculation amount, memory usage, and classification speed."

Reviewer #2: The MobileNet V3 (large or small) models was not used for all possible analysis- it was selective e.g. Tables 6, 7 and 8.

I don't think your co-authors or the paper reviewers should be acknowledged at the end of the paper. Kindly remove.

In Figure 12, the third dataset is not shown.

Limit keywords to five (5).

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dr Olukayode Oki

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Jun 7;18(6):e0286873. doi: 10.1371/journal.pone.0286873.r002

Author response to Decision Letter 0


12 May 2023

Dear Editors and Reviewers:

Thank you for your letter and for the reviewers’ comments concerning our manuscript entitled “Real-time scene classification of unmanned aerial vehicles remote sensing image based on Modified GhostNet”(ID:PONE-D-23-05971). Those comments are all valuable and very helpful for revising and improving our paper as well as the important guiding significance to our researches. We have revised the title on the submission form to make it consistent with the manuscript. We have studied comments carefully and have made correction which we hope meet with approval. Revised portion are marked in yellow in the paper. The main corrections in the paper and the responds to the editors and reviewers’ comments are in the uploaded files.

Attachment

Submitted filename: Response to Reviewers.pdf

Decision Letter 1

Sunday Adeola Ajagbe

25 May 2023

Real-time scene classification of unmanned aerial vehicles remote sensing image based on Modified GhostNet

PONE-D-23-05971R1

Dear Dr. Jinzhou Cao 

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sunday Adeola Ajagbe

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Acceptance letter

Sunday Adeola Ajagbe

29 May 2023

PONE-D-23-05971R1

Real-time scene classification of unmanned aerial vehicles remote sensing image based on Modified GhostNet

Dear Dr. Cao:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Sunday Adeola Ajagbe

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Attachment

    Submitted filename: Response to Reviewers.pdf

    Data Availability Statement

    All Files are available from the NWPU-RESISC database (url(s) https://hyper.ai/datasets/5449), UCMerced database (url(s) https://hyper.ai/datasets/5431), and AID database (url(s) https://hyper.ai/datasets/5446).


    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES