Development of a convolutional neural network to accurately detect land use and land cover

Highlights

• •EuroSat dataset, PNOA and Sentinel-2 imagery are used.
• •Classifications based on CNN are performed.
• •Developed model shows >90 % accuracy.
• •The resources employed enhance the achievement of several SDGs.

Abstract

The detection and modeling of Land Use and Land Cover (LULC) play pivotal roles in natural resource management, environmental modeling and assessment, and ecological connectivity management. However, addressing LULCC detection and modeling constitutes a complex data-driven process. In the present study, a Convolutional Neural Network (CNN) is employed due to its great potential in image classification. The development of these tools applies the deep learning method. A methodology has been developed that classifies the set of land uses in a natural area of special protection. This study area covers the Sierra del Cando (Galicia, northwest Spain), considered by the European Union as a Site of Community Interest and integrated in the Natura 2000 Network. The results of the CNN model developed show an accuracy of 91 % on training dataset and 88 % on test dataset. In addition, the model was tested on images of the study area, both from Sentinel-2 and PNOA. Despite some confusion especially in the residential class due to the characteristics in this area, CNNs prove to be a powerful classification tool.

• •Classifications based on a CNN model
• •LULC are classified into 10 different classes
• •Training and test accuracy are 91 % and 88 %, respectively

Graphical abstract

Specifications table

Subject area:	Environmental Science
More specific subject area:	Land use and land cover (LULC)
Name of your method:	CNN-LULC Predictor
Name and reference of original method:	Helber, P., Bischke, B., Dengel, A., & Borth, D. (2017). EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification. https://doi.org/10.1109/JSTARS.2019.2918242
Resource availability:	Links to the EuroSAT dataset (https://www.eotdl.com/datasets/EuroSAT), to the Sentinel-2 data (https://dataspace.copernicus.eu) and to the National Aerial Orthophotography Plan (PNOA) (https://pnoa.ign.es/)

Background

Application of the CNN in the environmental field

Environmental shifts and obstacles to sustainable development often arise from human activities that exert pressure on land utilization requirements [1]. Worldwide, land use and land cover change (LULCC) is key due to increasing urbanisation, industry and rural abandonment, playing a major role in local climate, hydrogeological conditions, environmental connectivity and environmental sustainability [2]. In that sense, the combination of new remote sensing technologies together with geographic information systems (GIS) have facilitated the examination of changes in LULCs with reduced time investment, cost, and heightened precision [3]. Mapping LULCC is crucial for integrated land-use planning, development and management at all levels. The development of satellite imagery with various spectral, temporal and spatial resolutions from numerous satellites, such as Landsat (moderate resolution) and Sentinel-2 (high resolution) available free of charge, has supported the efficient monitoring of the Earth's surface [4].

In recent years, strategies for LULC classification have shifted towards the advancement of machine learning techniques. For example, the development of methodologies based on artificial neural networks (ANN), [5] support vector machines (SVM) and random forests (RF) [6]. Nonetheless, evaluating LULC involves a prolonged process characterized by distinct temporal and spatial variations, heightening the intricacy of the study by Zhou et al. [7] In particular, the use of convolutional neural networks (CNNs) has emerged as a promising and innovative strategy that achieves comparatively good results, with an overall accuracy (OA) greater than 90 % [8,9]. Nowadays, a large number of neural networks has been developed and selecting the most appropriate method depends on factors such as its intended application, available computational resources, and the volume of training data accessible.

The main objective of this research was to apply and evaluate the efficiency of a CNN-based approach for LULC classification in the Sierra del Cando, Galicia, Spain. To date, this methodology has not been tested in the study area, which is characterized by a high fragmentation of LULC, mainly due to human activity, and a high population dispersion. Furthermore, the proposed methodology is characterized by the classification of up to ten different land uses, thus presenting an accurate and robust model capable of distinguishing between different land use categories in an area designated as a Site of Community Interest (SCI) by the European Union and integrated in the Natura 2000 network. The approach focused on the application of the CNN neural network using the EuroSat dataset, with the aim of providing an effective tool for the monitoring and sustainable management of natural resources in a region characterized by its diversity of land use patterns.

Method details

The method used (Fig. 1) aims to create an accurate and reliable CNN that is environmentally applicable, that classifies the LULC of the study area but that the method can be used to determine the LULCC produced between two different epochs, which is crucial for the management of the protected area and for environmental sustainability. It is also intended that the model produced with the suggested method can be extrapolated to all those areas where the EuroSat dataset or remote sensing data of a similar nature are available and that the LULC coverages defined are adapted to those existing in the terrain to be studied. High accuracies (>90 %) can be obtained for complex remote sensing cases for all terrain of similar nature without the need to modify the proposed CNN architecture.

Fig. 1.

Flowchart of the method used.

Dataset

The EuroSat dataset [10] was used, which has a collection of 27,000 Sentinel-2 images divided into 10 classes (Table 1). This dataset provides us with a large amount of images to train the neural network and also these images are both in “jpg” format, and in georeferenced “tif” format.

Table 1.

EuroSat dataset distribution.

Class	Number of images
Annual Crop	3000
Forest	3000
Herbaceous Vegetation	3000
Highway	2500
Industrial	2500
Pasture	2000
Permanent Crop	2500
Residential	3000
River	2500
Sea/Lake	3000

Convolutional neural network

Python 3.10.9 has been used as the programming language due to its versatility and the numerous libraries available for working with neural networks like tensorflow or numpy. A script was developed for training and obtaining metrics and another to perform test image prediction. The dataset was divided, allocating 85 % for training and validation purposes, with 80 % dedicated to training and 20 % to validation. The remaining 15 % was reserved exclusively for testing. The test accuracy was calculated from the confusion matrix for the data set, to plot the set of statistics and different plots such as accuracy and loss, the matplotlib and seaborn libraries were used.

The model is made up of several layers (Table 2) and was compiled with the optimizer set on 'adam', the loss value set to 'SparseCategoricalCorssentropy' and metrics set to accuracy. The number of epochs was set to 50.

Table 2.

CNN model summary.

Layer (type)	Output	Param#
Rescaling	(None, 64, 64, 3)	0
Data Augmentation	(None, 64, 64, 3)	0
Conv2D (1)	(None, 64, 64, 8)	224
MaxPooling2D (1)	(None, 32, 32, 8)	0
Conv2D (2)	(None, 32, 32, 16)	1168
MaxPooling2D (2)	(None, 16, 16, 16)	0
Conv2D (3)	(None, 32, 32, 32)	4640
MaxPooling2D (3)	(None, 8, 8, 32)	0
Conv2D (4)	(None, 8, 8, 64)	18,496
MaxPooling2D (4)	(None, 4, 4, 64)	0
Dropout	(None, 4, 4, 64)	0
Flatten	(None, 1024)	0
Dense (1)	(None, 128)	131,200
Dense (2)	(None, 10)	1290

The layers used and their function are described in the Keras API [11].

• •Rescaling layer: Preprocessing layer which rescales input values to a new range.
• •Conv2D: Generates a convolutional kernel to convolve with the input layer, producing a tensor of outputs.
• •MaxPooling2D: Reduces the input's spatial dimensions (height and width) by selecting the maximum value over each channel of the input.
• •Dropout: Introduces randomness by setting input units to zero with a specified rate during training, thereby mitigating overfitting.
• •Flatten: Flattens the input. Does not affect the batch size.
• •Dense: Just your regular densely-connected NN layer.

In addition, the second layer in Table 2, called data_augmentation (Sequential), is defined as a set of pre-processing layers for image augmentation created to rotate, flip and zoom the training images (Fig. 2) without the need to create more training data at different locations. This technique slightly decreases the training accuracy percentage, but significantly increases the accuracy of the test dataset and the overall accuracy of the images never seen by the model, which prepares it for a real environment with a wider variety of situations. The data_augmentation (Sequential) technique consists of randomly rotating, panning or zooming the image, with all, some or none of the factors in the image.

Fig. 2.

Data augmentation example, where the Original image represents the unmodified area under study and the Augmented image is rotated 180° from the original image.

The schematic diagram shown in Fig. 3 represents the architecture built for the present methodology, consisting of a total of fourteen layers. The input file after passing through the rescaling and data augmentation layers. goes through several Conv2d and MaxPooling layers in sequence, alternating, until it reaches a Dropout layer designed to avoid overfitting [12]. The data is then flattened by a Flatten layer. After this process, there are two densely connected layers that define the output in 10, representing the classes on which classification will be performed.

Fig. 3.

Architecture of deep learning convolutional neural network.

Testing

A script was developed that loads the previously trained model, and applies it to the desired dataset, displaying a group of images and their predicted classes by the model with their confidence percentages.

Firstly, tests were randomly performed on the 4050 images split for this purpose of the EuroSat dataset, then on Sentinel-2 images of the study area and finally, small image sections of the study area were source from the National Plan for Aerial Orthophotography (PNOA, from its acronym in Spanish) [13] and were used to perform the test in the same way.

Method validation

Study area

The SCI Serra do Cando (Fig. 4) is a mountain range in the autonomous community of Galicia (Northwest Spain) with an area of 5458 ha, it has an average altitude of 712 m, with a maximum of 1000 m. It is the source of the Verdugo River and some tributaries of Lerez. The SCI features an Atlantic climate characterized by minimal temperature variations (with an average annual temperature of 13 °C, monthly minimum of 3 °C, and maximum of 23 °C) along with plentiful rainfall. It is characterized by areas of scrubland and rocky outcrops and oak and riparian forests.

Fig. 4.

Location of Serra do Cando in Pontevedra (NW Spain). The map coordinate system is EPSG:25829 ETRS89/UTM zone 29 N.

High land fragmentation and small landholdings are two prominent features in the study area, and it is important to highlight how these conditions can complicate the development of methodologies for land use classification [14].

It has been observed that overall, the accuracy of the developed methodology is optimal for the 10 classified land use classes as demonstrated in the prediction on the test dataset (Fig. 5). The main confusions in the model are in the classification of rivers and highways due to their shape or in permanent crops and herbaceous vegetation.

Fig. 5.

Confusion matrix drawn from the 15 % test data (4096 images).

In terms of accuracy, the model achieved an accuracy of 91 % on the training dataset (Fig. 6) and a 88 % on the test dataset, It can be seen how little difference there is between the training dataset and the test dataset due to the data augmentation technique mentioned above. These results are similar to other studies, for example, the study by Tenemos et al. [15] where a 94.72 % accuracy rate was achieved with their convolutional neural network for land use and land cover classification. Using the EuroSat dataset in a similar study area and with the same classes, 82 % accuracy was obtained with an RF classifier.

Fig. 6.

Training and validation accuracy and loss.

Once the model was trained and tested, random predictions were made on the test dataset. Successful results were achieved with high confidence percentages provided by the model (Fig. 7).

Fig. 7.

Predictions on the EuroSat test dataset.

Once the model was trained and tested with the test dataset, Sentinel-2 images of the study area were obtained, trying to maintain similarity with the EuroSAT dataset images. These images were processed by the CNN model obtaining coherent results (Fig. 8), despite the contrast of characteristics of the study area with the training data such as the presence of narrow rivers or the non-rectangular shapes in crops.

Fig. 8.

Predictions on the study area (Sentinel-2).

Finally, trying to go one step further and test the model at higher resolutions, images of the study area were obtained from the PNOA, and tested (Fig. 9). PNOA has already been used in some neural networks studies in Spain with obtaining a high percentage of accuracy. For example, Cira et al. [16] developed a CNN with the objective of detecting road networks that obtained an accuracy of 90 %, or Ruiz et al. [17] that their CNN classifies in the region of Valencia, Spain, both the type of crop and its condition with an accuracy of 93 % and 99 % respectively. It should be noted that these high percentages have probably been obtained because their analyzes have focused on very specific land uses, while this study covers a greater number of classes (a total of ten).

Fig. 9.

Predictions on the study area (PNOA).

The results in the study area are generally good but present some problems due to the characteristics of the area such as the high fragmentation of the landscape [18] or heterogeneous land uses. For example, for the residential use, due to the low density of houses and that they are mostly single-family houses, there is some confusion in the classification, especially with industrial use and herbaceous vegetation. These areas are characterized by houses on large properties, which is why various constructions (houses, garages, annexes, etc.) are combined with gardens, orchard areas and more green areas [14]. This confusion is common in other studies like Li et al. and Zhang et al. [19,20] since the high density of the residential class when training can cause errors in less dense areas. There are also some errors when cataloguing roads like rivers due to its shape, as noted Naushad et al. [21], and we must also take into account the characteristics of the roads in the study area, which are not highways like the kind of roads on which the model is trained, but rather 1 or 2 lane roads.

Limitations

The proposed method could have limitations depending on the training data used, if the EuroSAT dataset is trained with an insufficient number of images or with images that do not correspond to the land cover present in the study area, the modeling produced will not be accurate. On the other hand, limitations in the spatial resolutions of the sensors used must also be considered; higher spatial and spectral resolutions (such as hyperspectral cameras) are linked to remote sensors of unmanned aerial vehicles and not to satellite sensors, which increases the initial investment and image pre-processing work [22]. Higher spatial or spectral resolutions are also linked to changes in CNN architectures such as a higher number of neurons [23], so computational efforts also increase. The last prerequisite for the CNN-based model to work properly is the availability of images without any cloud cover in the geographical area of application if satellite remote sensing is to be used, otherwise active satellite remote sensing with synthetic aperture radar (SAR) or LiDAR data should be used [24].

Ethics statements

Each author has actively contributed to the significant work leading to the paper and accepts public responsibility for its content.

CRediT authorship contribution statement

Carolina Acuña-Alonso: Conceptualization, Methodology, Software, Supervision, Formal analysis, Investigation, Writing – review & editing. Mario García-Ontiyuelo: Writing – review & editing, Methodology. Diego Barba-Barragáns: Writing – original draft, Methodology, Software, Visualization, Formal analysis, Investigation. Xana Álvarez: Conceptualization, Investigation, Supervision, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

• https://github.com/dbb-01/LandUseCNN_EuroSAT