Road defect detection based on improved YOLOv8s model

Abstract

Road defect detection is critical step for road maintenance periodic inspection. Current methodologies exhibit drawbacks such as low detection accuracy, slow detection speed, and the inability to support edge deployment and real-time detection. To solve this issue, we introduce an improved YOLOv8 road defect detection model. Firstly, we designed the EMA Faster Block structure using partial convolution to replace the Bottleneck structure in the YOLOv8 C2f module, and the enhanced C2f module was labeled as C2f-Faster-EMA. Secondly, we improved the model speed by introducing SimSPPF instead of SPPF. Finally, for the head, Detect-Dyhead, chosen to replace the original head, significantly improves the representation ability of heads without introducing any GFLOPs. Experimental results on the road defect detection dataset show that the improved model in this paper outperforms the original YOLOv8, with a 5.8% increase in average accuracy (mAP@0.5), and notable reductions of 22.33% in model size, 23.03% in parameter size, and 21.68% in computational complexity.

Introduction

The challenging operational conditions faced by road infrastructure encompass not only the recurrent need to withstand diverse loads but also the influence of dynamic climatic variations. Throughout the course of road usage, a spectrum of cracks emerges, including longitudinal, transverse, and alligator patterns¹. Road maintenance, marked by its complexity, labor-intensive nature, and seemingly straightforward yet crucial tasks, is susceptible to inadvertent oversight². The expansive and intricate nature of the highway traffic network exacerbates the escalating prominence of daily maintenance challenges, thereby posing a potential hazard to roadway safety. To fortify road performance and ensure driving safety, the imperative lies in the adoption of rapid and precise road defect detection methods. Concurrently, the evolving societal landscape amplifies the expectations for the quality of road service, necessitating effective strategies for sustaining roads in optimal condition while adhering to economic considerations. The meticulous detection of road defects within the realm of road maintenance assumes heightened significance to accomplish these objectives³.

Traditional methods for detecting in roads largely depend on manual inspection. It is slow and arduous. With a high risk and subject to the inspectors’ personal biases. After 2014, object detection models based on deep learning began to develop by leaps and bounds. Object detection can be broadly categorized into two types: two-stage algorithms and single-stage algorithms. Two-stage algorithms include: R-CNN⁴, Fast-RCNN⁵, Faster-RCNN⁶, Mask-RCNN⁷, AlexNet⁸ and so on. The two-stage algorithm implements detection in a two-stage process. First, the object region is extracted. Then the regions are classified and recognized. Kortmann et al.⁹ introduce a new method for fast road defect detection based on Faster R-CNN. Faster R-CNN was systematically trained with dataset. The experiments, systematically conducted on a particular dataset, demonstrate the ability to achieve an F1 score of 0.487 across all regions, covering longitudinal and lateral cracks, alligator cracks, and potholes. However, this model has room for improvement in terms of processing speed. It was trained using smaller batch sizes, which may result in longer training times. Nguyen et al.¹⁰ proposed a two-stage CNN where the first stage structure is used to eliminate noise and reduce the crack region in the image, and the second stage is used for crack feature learning, which achieves crack detection with smaller network parameters. Despite achieving a high F1-score, the model divides the crack identification process into two steps, thereby prolonging the inference time.

Although the two-stage algorithm is highly accurate, it does not detect as quickly as the single-stage algorithm. This is because single-stage target detection methods require extracting features only once to achieve target detection. One-stage detection algorithms include You Only Look Once^11–17 series, Single Shot MultiBox Detector (SSD)¹⁸, RetinaNet¹⁹, etc. Researchers have extensively studied one-stage algorithm for crack defect detection and have achieved significant results. For instance, Tran et al.²⁰ proposed a supervised machine learning network model based on RetinaNet to detect and classify various types of defection in asphalt pavements. The RetinaNet trained network needs only 4 days to analyze a 150 km road section compared to a 1-month analysis using manual checking. However, the detection accuracy of the model is not very high, and there are misdetections and omissions. Yan et al.²¹ proposed an improved Single Shot Multi-Box Detector. By adding a deformable convolution to the backbone feature extraction network VGG16. They validate the proposed model on the PASCAL VOC2007 dataset, and the mean accuracy (mAP) of their model is 3.1% higher than the original SSD model. However, this model does not reuse underlying features for high resolution, resulting in poor performance in small crack detection. This problem was only well solved by introducing Feature Pyramid Networks for Object Detection in YOLOv3. Wan et al.²² introduced YOLO-LRDD, a lightweight YOLO algorithm designed for road crack detection. By using a novel backbone network, Shuffle-ECANet, accuracy is maintained without too much variation while increasing the speed of model detection and reducing the model size. Through the incorporation of a novel backbone network, Shuffle-ECANet, the model achieves improved recognition speed for individual images and a reduction in model size. On the flip side, a consequence of this enhancement is an impact on accuracy.

However, while the aforementioned studies contribute significantly to road damage detection tasks, there remains substantial room for improvement in both accuracy and detection speed. Building on the insights gained from previous studies^23–26, this paper adopts YOLOv8s²⁷, the latest one-stage object detection algorithm, as the foundational model for road defect detection. The model’s accuracy and detection speed are subsequently enhanced through optimization.

In this paper, we address the challenges of low detection accuracy and a high number of model parameters in road maintenance crack detection by introducing an enhanced YOLOv8s model. In the publicly available dataset, our model achieves a 5.8% increase in accuracy and a 23.03% decrease in the number of parameters compared to the original model. In summary, our contribution points include:

• We introduce a novel C2f structure into the backbone, drawing inspiration from the FasterNet architecture. Additionally, we incorporate the newly proposed attention mechanism, EMA, before the Concat layer. This augmentation serves to enhance the detection accuracy of the model, while concurrently reducing the number of parameters and expediting the inference time.

• A more concise SimSPPF activation function is adopted to replace the SPPF structure of the original model, which improves the running speed of the model without affecting the accuracy.

• To further enhance detection accuracy, we introduce a dynamic detection head structure. The experiment reveals that the optimal number of repetitions plays a crucial role, striking a balance between higher detection accuracy and reduced module running time.

The remainder of the paper is structured as follows: “Improved YOLOv8 model” section provides a detailed description of our model. “Results and analysis” section covers the dataset, comparisons with other modules and models, ablation experiments, and interpretability experiments. “Discussion” section delves into the discussion of the obtained results. Finally, “Conclusion” section offers a comprehensive summary of the entire work.

Improved YOLOv8 model

In addressing the imperative to augment both accuracy and detection speed, this study undertakes enhancements to the YOLOv8 model employing two distinct methodologies. Initially, for the purpose of optimizing computational efficiency, this investigation introduces modifications to the Bottleneck structure within the C2f module and the activation function within the SPPF module. These alterations involve the substitution of the extant architecture with a lighter structure, resulting in minimal reduction in detection accuracy. Subsequently, in pursuit of heightened detection precision, the Dynamic Head²⁸, introduced in 2021, is incorporated. This approach seeks to unify object detection heads with attention mechanisms, thereby significantly enhancing the representational proficiency of the model’s object detection head with negligible computational overhead.

YOLOv8 model

Object detection is a fundamental task in computer vision, and the YOLOv8 model represents the latest iteration in the YOLO series. This model excels in efficiently detecting objects within images through its well-structured architecture. The YOLOv8 algorithm is structured with key elements, including an input layer, a backbone responsible for feature extraction, a neck for further refinement, and an output layer. Each element plays a distinct role in the model’s overall functionality. The central components of the YOLOv8 network are the backbone and the neck. The backbone network, a critical component in YOLOv8, employs an efficient combination of modules, namely CBS (Convolution, Batch Normalization, SiLU Activation), C2f, and SPPF. This combination allows the model to extract fine-grained feature information from input images.

The neck layer of YOLOv8 critically positions itself between the backbone and the head, optimizing the utilization of features extracted from the backbone. It incorporates FPN²⁹ + PAN³⁰ structure to enhance the model’s feature fusion capability, allowing for more comprehensive and accurate object detection. In a succinct overview of the network characteristics, PAN-FPN undergoes initial down-sampling followed by up-sampling. The network incorporates two cross-layer fusion connections between the up-sampling and down-sampling branches.

The YOLOv8 head is divided into a classification head and a detection head. Task-Alignment Anchor³¹ is used to select samples for training via the loss function. For each instance, m anchors with maximum values are selected as positive samples and the remaining anchors are selected as negative samples. The detection head of YOLOv8 is evaluated using the Bbox Loss, which consists of two parts: CIoU³² and DFL³³. The Intersection over Union (IoU) is defined as the intersection of the predicted bounding box and the ground truth bounding box divided by their union, as shown in Eq. (1). The term $v$ is calculated as $\frac{4}{{\pi }^2}{\left(\text{arctan},\frac{w^{\mathit{gt}}}{h^{\mathit{gt}}},-,\text{arctan},\frac{w}{h}\right)}^2$, where $w$ and $h$ are the width and height of the predicted bounding box and $w^{\mathit{gt}}$ and $h^{\mathit{gt}}$ are the width and height of the ground truth bounding box, as shown in Eq. (2). The coefficient a is given by $\frac{v}{1-IoU+v}$ as shown in Eq. (3). The Complete IoU (CIoU) loss is then formulated as $L_{\mathit{CIoU}}=1-IoU+\frac{{\rho }^2\left(b,,,b^{\mathit{gt}}\right)}{c^2}+av$, where $\rho (b,b^{\mathit{gt}})$ is the Euclidean distance between the central points of the predicted and ground truth boxes, and $c$ is the diagonal length of the smallest enclosing box covering the two boxes, as shown in Eq. (4).The classification head, on the other hand, employs Binary Cross-Entropy (BCE) loss. The use of different loss functions enhances the model’s ability to accurately classify and localize objects. YOLOv8 adopts an anchor-free detection mechanism, predicting the center of an object directly rather than the offset from a known anchor box. This approach reduces the number of box predictions and speeds up the post-processing process. In summary, YOLOv8 introduces a sophisticated architecture, combining efficient modules in the backbone, an advanced feature fusion mechanism in the neck, and a unique anchor-free detection approach in the output head. These innovations collectively contribute to the model’s superior object detection capabilities.

$$IoU=\frac{\mathit{intersection}}{\mathit{union}},$$ 1

$$v=\frac{4}{{\pi }^2}{\left(\text{artan},\theta ,=,\frac{w^{\mathit{gt}}}{h^{\mathit{gt}}},-,\text{artan},\theta ,=,\frac{w}{h}\right)}^2,$$ 2

$$a=\frac{v}{1-IoU+v},$$ 3

$$L_{\mathit{CIoU}}=1-IoU+\frac{{\rho }^2\left(b,,,b^{\mathit{gt}}\right)}{c^2}+av.$$ 4

Lightweight bottleneck

In the realm of deep learning networks, the pursuit of increased computational speed serves as the foundational premise for fostering faster and more robust models. Consequently, the recent advent of the lightweight neural network, FasterNet³⁴, introduces the concept of partial convolution. This innovative approach employs regular convolution to extract spatial features solely on a subset of input channels, leaving the remaining channels unaffected. The subsequent application of a 1 × 1 convolution layer capitalizes on the information from these untouched channels. Hence, we amalgamate the recently introduced C2f structure in YOLOv8 with FasterNet, resulting in the formation of C2f-Faster.This lightweight paradigm not only upholds a commendable computational speed but also mitigates the number of floating-point operations (FLOPs).

While incorporating the modules from FasterNet substantially decreases the number of parameters in the model, there is a noticeable impact on accuracy. Thus, there is a need to introduce a module that does not increase the number of parameters but rather enhances accuracy—the attention module.

In various computer vision tasks, the integration of channels or spatial attention mechanisms is prevalent for enhancing feature clarity. However, cross-channel relationship modeling with channel dimensionality reduction may inadvertently introduce adverse effects in deep visual representation extraction. Addressing this concern, the latest advancement in cross-spatial learning introduces an efficient multi-scale attention module known as the Efficient Multi-scale Attention (EMA) Module. This module preserves valid channel types without necessitating channel dimensionality reduction and aggregates multi-scale spatial structure information. EMA facilitates the transformation of partial channels into batch dimensions and partitions channel dimensions into multiple sub-feature groups, ensuring well-distributed spatial semantic features within each group. Consequently, this augmentation significantly amplifies the feature representation capabilities.

In an amalgamation of the partial convolution concept from FasterNet and the efficient multi-scale attention mechanism, we propose an EMA Faster Block structure to replace the Bottleneck structure in YOLOv8. This innovation enhances the model’s detection accuracy while concurrently alleviating the computational burden. The illustration in Fig. 1 delineates the specifics of this novel structure. Partial Convolution (PConv) is emblematic of the partial convolution, utilizing only 1/4 of the input channels for convolution operations, while leaving the remaining channels untouched. This deliberate reduction in memory access and computational redundancy is instrumental in optimizing the overall efficiency. CBS represents a convolution module comprising convolution, batch normalization, and an activation function. Subsequently, a 1 × 1 convolutional layer is deployed to diminish the number of channels in the CBS output to its pre-expansion state.

Figure 1.

Bottleneck vs. Faster Block vs. EMA Faster Block.

In this study, the original C2f structure undergoes replacement with the proposed C2f-Faster-EMA structure, as depicted in Fig. 2. This novel structure serves as a replacement for the original Bottleneck structure. As illustrated in the example, the instantiation of the C2f-Faster-EMA structure involves the incorporation of three EMA Faster Blocks.

Figure 2.

Structural diagram of the C2f-Faster-EMA module.

SimSPPF model

To further curtail the computational complexity of the model, reduce inference time, and ensure real-time road defect identification, the SimSPPF (Simple Spatial Pyramid Pooling Fusion) structure is adopted in this paper. Originating from YOLOv6¹⁶, the SimSPPF structure proves instrumental in simplifying the model’s complexity, thereby expediting processing time. Unlike its predecessor, the SimSPPF structure employs the ReLU (Rectified Linear Unit) activation function, in contrast to the SPPF (Spatial Pyramid Pooling-Fast) module which leverages the SiLU (Sigmoid Linear Unit) activation function. The mathematical formulations for these activation functions are provided in Eqs. (5) and (6).

$$ReLU:f(x)\left\{\begin{array}{c}x,ifx>0\hfill \\ 0,ifx\le 0\hfill \end{array},=,\text{max},\left(0,,,x\right),,\right)$$ 5

$$SiLU:f\left(x\right)=\frac{x^2}{1+e^{-x}}.$$ 6

Given the exponential nature of arithmetic in the SiLU function, it inherently contributes to increased computational complexity. To address this challenge, the SimSPPF structure opts for the ReLU function, mitigating the issue of gradient vanishing and expediting convergence. Experimental comparisons highlight a notable 18% increase in the execution speed of a single ConvBNReLU module compared to a ConvBNSiLU module. The structural configuration of SimSPPF is delineated in Fig. 3.

Figure 3.

SimSPPF structure diagram.

Detect-Dyhead

The replacement of the original detection head with Detect-Dyhead (Detect-Dynamic head) significantly contributes to the enhancement of the model’s expressive capability without imposing a substantial increase in computational overhead. Given the high dimensions of the tensor, direct learning of the attention function over all dimensions proves computationally expensive. Detect-Dyhead adeptly addresses this challenge by decomposing the attention function into three distinct mechanisms, each concentrating on a specific perspective. These three perspectives include scale-awareness, spatial-awareness, and task-awareness, with each attention mechanism being applied independently. The resulting architecture of the Detect-Dyhead network is elucidated in Fig. 4.

Figure 4.

The architecture of Detect-Dyhead network.

The scale-aware attention module D_L comprises an average pooling layer, a 1 × 1 convolutional layer, ReLU, and a Hard Sigmoid activation function, as defined in Eq. (7). Here, f (∙) denotes a 1 × 1 convolution, and σ(x) represents the Hard Sigmoid activation function.

$$D_L\left(F\right)·F=\sigma \left(f,\left(\frac{1}{\mathit{SC}},{\sum }_{S,C},F\right)\right)·F.$$ 7

The subsequent spatial-aware attention module D_S integrates an index, a 3 × 3 convolutional layer, a sigmoid activation function, and an offset. The index specifies the utilization of deformable convolutions based on different classes of feature maps, unifying them into a medium feature map size. The formula for this module is articulated in Eq. (8), where K signifies the number of sparse sampling locations, p_k + ∆p_k denotes the offset for the region, and ∆m_k is the self-learning factor at location pk, both learned from the preceding input features.

$$D_S\left(F\right)·F=\frac{1}{L}{\sum }_{l=1}^L{\sum }_{k=1}^K{W}_{l,k}·F_l\left(p_k,+,\mathrm{\Delta },p_k\right)·\mathrm{\Delta }{m}_k.$$ 8

And finally, the task-aware attention module D_C, which consists of average pooling Layer, two fully connected layer, ReLU activation function and Normalize. The specific formula is Eq. (9). Where FC (Fully Connected Layer) is the feature slice at c-th channel and $[{\alpha }^1,{\alpha }^2,{\beta }^1,{\beta }^2]$ constitutes a hyperfunction controlling the activation thresholds This module combines total average pooling of features and normalization.

$$D_c\left(F\right)·F=\text{max}\left({\alpha }^1,\left(F\right),·,F_c,+,{\beta }^1,\left(F\right),,,{\alpha }^2,\left(F\right),·,F_c,+,{\beta }^2,\left(F\right)\right).$$ 9

Network structure and parameters

Table 1 presents essential information, including the count of layers, sources, parameter totals, model denominations, and parameters specific to each model. This detailed presentation aids in providing a thorough comprehension of the network architecture and the intricate parameters associated with the enhanced version of YOLOv8. The inclusion of such comprehensive data ensures readers can fully grasp the structural and parametric nuances that contribute to the improvements in this iteration of the YOLO architecture.

Table 1.

Network structure and parameters of improved YOLOv8s.

Layer	From	N	Params	Module	Arguments
0	− 1	1	928	Conv	[3, 32, 3, 2]
1	− 1	1	18,560	Conv	[32, 64, 3, 2]
2	− 1	1	15,472	C2f_Faster_EMA	[64, 64, 1, True]
3	− 1	1	73,984	Conv	[64, 128, 3, 2]
4	− 1	2	88,896	C2f_Faster_EMA	[128, 128, 2, True]
5	− 1	1	295,424	Conv	[128, 256, 3, 2]
6	− 1	2	353,408	C2f_Faster_EMA	[256, 256, 2, True]
7	− 1	1	1,180,672	Conv	[256, 512, 3, 2]
8	− 1	1	967,808	C2f_Faster_EMA	[512, 512, 1, True]
9	− 1	1	656,896	SimSPPF	[512, 512, 5]
10	− 1	1	0	Upsample	[None, 2, ‘nearest’]
11	[− 1,6]	1	0	Concat	[1]
12	− 1	1	373,824	C2f_Faster_EMA	[768, 256, 1]
13	− 1	1	0	Upsample	[None, 2, ‘nearest’]
14	[− 1, 4]	1	0	Concat	[1]
15	− 1	1	93,856	C2f_Faster_EMA	[384, 128, 1]
16	− 1	1	147,712	Conv	[128, 128, 3, 2]
17	[− 1, 12]	1	0	Concat	[1]
18	− 1	1	275,520	C2f_Faster_EMA	[384, 256, 1]
19	− 1	1	590,336	Conv	[256, 256, 3, 2]
20	[− 1, 9]	1	0	Concat	[1]
21	− 1	1	1,098,880	C2f_Faster_EMA	[768, 512, 1]
22	[15, 18, 21]	1	2,863,988	Detect_DyHead	[80, 128, 2, 128, 256, 512]
Summary: 341 layers, 8,591,112 parameters, 8,591,096 gradients, 22.4 GFLOPs

In our improved structure, the convolution kernel size is set to 3 × 3, and the stride for convolution operations is 2. The backbone is seamlessly integrated with the neck through the C2f-Faster-EMA mechanism at the fourth and sixth layers, alongside the implementation of the SPPF structure at the ninth layer. Similarly, the head is linked to the neck utilizing the C2f-Faster-EMA mechanism at the 15, 18, and 21st layers. Our improved model features three dynamic detection heads, each designed to take a single input feature map as input to this layer. These design choices are optimized to enhance computational efficiency and accuracy in specific applications. The integration of dynamic detection heads enables our model to dynamically adapt to varying input sizes, which include 80 × 80 × 256, 40 × 40 × 512, 20 × 20 × 1024, significantly improving detection accuracy for diverse scenarios. The structure of the final implemented YOLOv8 is shown in Fig. 5.

Figure 5.

Improved structure of YOLOv8s.

Results and analysis

Experimental environment

To validate the effectiveness of the proposed approach, we established an experimental platform using Ubuntu 20.04 as the operating system and PyTorch as the deep learning framework. The baseline network model selected for this experimentation was YOLOv8s. The detailed configuration of the experimental environment is outlined in Table 2.

Table 2.

Configuration and training environment.

Environmental parameter	Value
Operating system	Ubuntu20.04
Deep learning framework	PyTorch
Programming language	Python3.9
CPU	Inter(R) Core(TM) i9-13900K
GPU	NVIDIA GeForce RTX 3090 (24 GB)
RAM	64 GB

Consistency in hyperparameter settings is pivotal for ensuring fair and comparable evaluations across all experiments. Table 3 enumerates the specific hyperparameters that remained constant throughout the training process, fostering a standardized and rigorous assessment of the proposed modifications to the YOLOv8 model.

Table 3.

Hyperparametric configuration.

Hyperparameters	Value
Learning rate	0.01
Image size	640 × 640
Momentum	0.937
Optimizer	SGD
Batch size	64
Epoch	200

Dataset and evaluation metrics

In this research, the RDD2020 dataset³⁵ served as the public dataset, comprising road damage images selected from China. The dataset comprises a total of 4378 images, with 2401 images captured by drones and 1977 images captured by on-board cameras. Given the focus on road defect detection in pavement maintenance, six distinct types of crack damage were considered: longitudinal cracks (D00), transverse cracks (D10), alligator cracks (D20), Potholes (D40), road repairs (Repair) and Block cracks (Block crack). The distribution of various detection targets within the crack images from the China is summarized in Table 4. The images are divided into training, validation, and test sets in an 8:1:1 ratio.

Table 4.

Types and quantities of defects in the dataset.

Pavement distress	Distress class	Quantity
Longitudinal cracks	D00	2453
Transverse cracks	D10	1761
Alligator cracks	D20	750
Potholes	D40	228
Road repairs	Repair	803
Block cracks	Block crack	3

In the objective assessment of road defect detection models, a range of comprehensive evaluation metrics was employed to capture various facets of model performance. The evaluation metrics employed encompass GFLOPS (giga floating-point operations per second), which quantifies the execution time of the network model in terms of billions of floating-point operations per second. The parameters, which assess the size and complexity of the model. FPS (Frames Per Second), which gauges the detection speed of the model in frames transmitted per second. mAP (mean average precision), utilized to evaluate the model accuracy, is computed using Eq. (13). The precision (P) is defined as the ratio of true positives (TP) to the sum of true positives and false positives (FP), as shown in Eq. (10). Recall (R) is defined as the ratio of true positives to the sum of true positives and false negatives (FN), as shown in Eq. (11). Average Precision (AP) is the area under the precision-recall curve, as indicated by Eq. (12). The F1-score, which is a weighted average of precision and recall, serves as a measure of the model overall performance and stability. The calculation formula for the F1-score is provided in Eq. (14). This comprehensive set of metrics offers a holistic perspective on the road defect detection model’s performance, covering key aspects such as speed, accuracy, and stability.

$$P=\frac{\text{TP}}{TP+FP},$$ 10

$$R=\frac{\text{TP}}{TP+FN},$$ 11

$$AP=\underset{0}{\overset{1}{\int }}P(R)dR,$$ 12

$$mAP=\frac{1}{n}\sum _{i=1}^{n}AP,$$ 13

$$F1\text{-}score=2\times \frac{Precision\times Recall}{Precision+Recall}.$$ 14

In Eqs. (10) and (11), the terms are defined as follows: TP (True Positive) signifies that a positive sample is predicted to be positive, FP (False Positive) indicates that a negative sample is incorrectly predicted to be in the positive category, and FN (False Negative) represents that a positive sample is incorrectly predicted to be in the negative category. In Eq. (13), ‘n’ represents the number of categories. Regarding Eq. (14), the F1-Score serves as a composite metric, incorporating both Precision and Recall. A higher F1-Score value indicates superior performance and robustness of the model.

Comparison of improved C2f structures and Detect-Dyhead repetitions

Various replacement structures for C2f are compared to assess their impact on model size and accuracy, as detailed in Table 5.

Table 5.

Performance of different models after replacing the C2f structure.

Models	mAP@0.5/%	Module size (M)	Para (M)	GFLOPs	Time (ms)
YOLOv8s	0.699	21.5	11.16	28.6	3.33
YOLOv8s+C3	0.713	17.5	9.03	23.2	2.38
YOLOv8s+C3-Faster	0.685	16.8	9.03	22.9	2.25
YOLOv8s+C3-Faster-EMA	0.663	16.9	8.67	23.0	2.83
YOLOv8s+C2f-DCNV236	0.707	21.6	11.22	28.2	19.01
YOLOv8s+C2f-DCNV337	0.694	20.7	10.68	28.1	35.09
YOLOv8s+C2f-Faster	0.684	19.8	10.25	27.7	2.23
YOLOv8s+C2f-Faster-EMA	0.715	19.8	10.29	28.0	3.46

To validate the performance of models designed after replacing the C2f structure, a uniform replacement of the C2f structure in layer 8 of the backbone with the test modules is conducted. For C3, C3-Faster, and C3-Faster-EMA, the C2f structures are all replaced with C3, and then the original module in layer 8 is substituted with the module to be tested.

The validation results in Table 5 demonstrate that the C2f-Faster-EMA structure can effectively reduce the number of model parameters and computational complexity while improving accuracy. To assess the potential superiority of combining the EMA attention mechanism with C2f-Faster, the mainstream attention mechanism is replaced in our experiments. This experiment also involves replacing the original C2f structure with a different one only at the eighth layer, as outlined in Table 6. The outcomes of these two sets of experiments indicate that the use of C2f-Faster-EMA can enhance accuracy while concurrently reducing the model size, the number of parameters, and computational complexity compared to other alternative structures.

Table 6.

Performance of different attention mechanisms combined with C2f-Faster.

Attention	mAP@0.5/%	Module size (M)	Para (M)	GFLOPs	Time (ms)
None	0.699	21.5	11.16	28.6	3.33
+C2f-Faster-SE38	0.702	19.8	10.26	27.7	2.32
+C2f-Faster-GC39	0.666	19.9	10.26	27.7	2.38
+C2f-Faster-TripletAttention40	0.698	19.8	10.25	27.7	2.65
+C2f-Faster-ECA41	0.682	19.8	10.50	27.7	7.46
+C2f-Faster-GAM42	0.689	23.0	11.89	23.0	5.11
+C2f-Faster-SGE43	0.666	19.9	10.30	27.7	2.35
+C2f-Faster-EMA	0.715	19.8	10.29	28.0	3.46

The impact of varying numbers of Detect-Dyhead repetitions on model size and accuracy is analyzed in Table 7. The data presented in the table indicates an improvement in mAP@0.5 as the number of repetitions increases. However, this improvement is not consistent, and accuracy begins to decline when the number of repetitions reaches four. Additionally, the model size, parameters, and computational complexity increase with the number of repetitions. Therefore, in our improved model, we opt for two repetitions to strike a balance between accuracy and model efficiency.

Table 7.

Performance table of different Detect-Dyhead repetitions under multiple metrics.

Repetitions	mAP@0.5/%	Module size (M)	Para (M)	GFLOPs	Time (ms)
1	0.732	15.7	8.07	21.8	262.23
2	0.757	16.7	8.60	22.4	508.50
3	0.745	17.6	9.06	22.7	748.38
4	0.688	18.6	9.58	23.4	988.96
5	0.705	19.5	10.04	23.8	1179.0
6	0.725	20.5	10.54	24.3	1524.89
9	0.710	23.3	12.03	25.9	2190.23

Ablation experiment

Based on the data presented in Table 8, the substitution of the original C2f structure with C2f-Faster-EMA results in a 24.65% reduction in model size, a 28.94% reduction in parameter size, and a 22.38% decrease in computational complexity. Replacing SPPF with SimSPPF has minimal impact on accuracy, module size, number of parameters, and computational complexity, but it enhances the detection speed by 1.8 FPS. The integration of the dynamic head slightly increases model size, number of parameters, and computational complexity but achieves a 3.6 percentage point improvement in accuracy (mAP@0.5).

Table 8.

Ablation experiments with the modules.

C2f-Faster-EMA	SimSPPF	Detect-Dyhead	mAP@0.5/%	Module size (M)	Para (M)	GFLOPs	FPS
			0.699	21.5	11.16	28.6	384.6
√			0.673	16.2	8.37	22.2	295.2
	√		0.698	21.5	11.16	28.6	386.4
		√	0.735	22.9	11.87	29.3	181.3
√		√	0.734	17.6	9.09	22.9	157.9
√	√		0.715	16.2	8.37	22.2	293.0
√	√	√	0.757	16.7	8.59	22.4	136.7

Combining the C2f-Faster-EMA module with Detect-Dyhead significantly enhances the model’s accuracy while reducing model size, parameters, and computational effort. Overall, the improved YOLOv8s, incorporating C2f-Faster-EMA, SimSPPF, and Detect-Dyhead, surpasses the original YOLOv8s model in terms of detection accuracy, model size, parameter size, and computational complexity. On the same dataset, the enhanced YOLOv8s model achieves a 5.8% improvement in mAP@0.5, a 22.33% reduction in model size, a 23.03% reduction in parameter counts, and a 21.68% reduction in computational complexity.

The analysis of Fig. 6 demonstrates that the improved YOLOv8s model, as proposed, exhibits significantly higher values for mAP@0.5 and mAP@0.5:0.95 compared to the original YOLOv8s model.

Figure 6.

A comparison was conducted to evaluate the mAP values of the improved YOLOv8s model against the original. (a) Comparison of mAP@0.5, and (b) comparison of mAP@0.5:0.95.

Interpretability experiment

In the context of deep learning, models are often perceived as black boxes due to their highly nonlinear nature. This lack of interpretability is a significant challenge, particularly in critical applications such as autonomous driving. Understanding the interpretability of deep learning models becomes crucial for addressing and solving problems effectively. In this study, we have selected both the improved YOLOv8s model and the original YOLOv8s model as validation models. A comparative analysis of their performance is conducted by examining their confusion matrices, as depicted in Fig. 7.

Figure 7.

Confusion matrix of the improved YOLOv8 model and YOLOv8 model. (a) Confusion matrix of improved YOLOv8. (b) Confusion matrix of YOLOv8.

The confusion matrices indicate that both models exhibit a high number of false negatives (i.e., misclassifying targets as backgrounds) and false positives (i.e., misclassifying backgrounds as targets). A detailed analysis shows that both models perform well in recognizing D00 (longitudinal cracks) and Repair (road repairs). However, the original YOLOv8 model demonstrates poor recognition accuracy for D10 (transverse cracks), achieving only 74% accuracy. In contrast, our improved YOLOv8s model significantly enhances the recognition accuracy for D10 by 6%.

Performance comparison of different models

To assess the performance of the enhanced YOLOv8s model, this study conducted comparative experiments with various widely used object detection models on different datasets, including Faster-R-CNN, RT-DETR-l⁴⁴, RT-DETR-x, YOLOv3s, YOLOv3s-SPP, YOLOv3s-tiny, YOLOv5s, and YOLOv6s. The experiments were carried out on the same dataset and under identical experimental conditions.

In Experiment 1, depicted in Fig. 8, a range of models, such as Faster R-CNN, RT-DETR-l, RT-DETR-x, YOLOv3s, YOLOv3s-tiny, YOLOv3s-SPP, YOLOv5s, YOLOv6s, and YOLOv8s, were utilized for crack detection on the Chinese dataset. Notably, YOLOv3s exhibited the highest miss detection rate. While Faster R-CNN successfully detected all objects, its predicted classes were inaccurate. Conversely, our improved model demonstrated superior performance in this scenario, exhibiting minimal missed or false detections, and providing more accurate predicted bounding boxes compared to other models.

Figure 8.

Comparative experiments on visualization results for different models on the Chinese dataset: (a) Faster R-CNN; (b) RT-DETR-l; (c) RT-DETR-x; (d) YOLOv3s (e) YOLOv3s-tiny; (f) YOLOv3s-SPP; (g) YOLOv5s; (h) YOLOv6s; (i) YOLOv8s; (j) Ours.

As illustrated in Table 9, our model achieved a high accuracy of mAP@50 on the Chinese dataset. It is evident that our improved YOLOv8s model outperforms other target detection models. These results highlight that, in comparison to the original YOLOv8s, our enhanced model not only achieved improved accuracy but also reduced the number of parameters and computational complexity by 23.03% and 21.68%, respectively.

Table 9.

The performance table of different models on the Chinese dataset.

Models	mAP@0.5/%	Para (M)	GFLOPs	FPS
Faster R-CNN	0.691	41.36	940.85	12
RT-DETR-l	0.664	56.4	105.5	58.9
RT-DETR-x	0.683	129.2	232.7	43.0
YOLOv3s	0.727	15.35	43.9	110.9
YOLOv3s-tiny	0.683	3.21	5.1	400.1
YOLOv3s-SPP	0.711	15.61	44.1	119.4
YOLOv5s	0.707	9.11	23.8	179.7
YOLOv6s	0.633	16.30	44.0	264.1
YOLOv8s	0.699	11.16	28.6	384.6
Ours	0.757	8.59	22.4	136.7

The detection results of the different models in Experiment 2 can be seen in Fig. 9 on the United States dataset³⁵. The figure visually illustrates that Faster R-CNN, RT-DETR-x, YOLOv3s, YOLOv3-SPP, YOLOv5s, YOLOv6s, and YOLOv8s exhibit missed detections. While RT-DETR-l and YOLOv3s-tiny manage not to miss detections, their average confidence are comparatively low. In contrast, our improved model demonstrates significantly superior detection performance compared to other models.

Figure 9.

Comparative experiments on visualization results for different models on the United States dataset: (a) Faster R-CNN; (b) RT-DETR-l; (c) RT-DETR-x; (d) YOLOv3s (e) YOLOv3s-tiny; (f) YOLOv3s-SPP; (g) YOLOv5s; (h) YOLOv6s; (i) YOLOv8s; (j) Ours.

Table 10 reveals that our model outperforms other models, achieving a higher mAP@50 on the US dataset. Despite having a larger number of parameters than YOLOv3s-tiny, our model significantly excels in accuracy. Comparative experiments highlight a 2.6% improvement in accuracy compared to the original YOLOv8s model. Comparative experiments indicate that our enhanced model exhibits superior detection accuracy, a reduced missed detection rate, and a lower false detection rate across various scenarios. These outcomes underscore the model’s robust performance in diverse situations, showcasing a significant improvement in target detection capabilities.

Table 10.

The performance table of different models on the United States dataset.

Models	mAP@0.5/%	Para (M)	GFLOPs	FPS
Faster R-CNN	0.423	41.36	940.85	12
RT-DETR-l	0.474	56.4	105.5	58.9
RT-DETR-x	0.511	129.2	232.7	43.0
YOLOv3s	0.524	15.35	43.9	110.9
YOLOv3s-tiny	0.476	3.21	5.1	400.1
YOLOv3s-SPP	0.530	15.61	44.1	119.4
YOLOv5s	0.557	9.11	23.8	179.7
YOLOv6s	0.551	16.30	44.0	264.1
YOLOv8s	0.553	11.16	28.6	384.6
Ours	0.579	8.59	22.4	136.7

To verify the robustness and generalizability of our model, we carried out Experiment 3 on the Indian dataset³⁵ and Experiment 4 on the GRDDC (Global Road Damage Detection Challenge) dataset⁴⁵. The outcomes of these experiments indicate that our enhanced model demonstrates superior accuracy and increased speed in detecting road defects compared to different models.

The results of Experiment 3 can be found in Table 11, with visualizations in Fig. 10. The images used for detection were taken from the test portion of the GRDDC dataset. The outcomes of Experiment 4 are shown in Table 12, with visualizations in Fig. 11. The images used for detection in this experiment were sourced from the test set of an Indian dataset. In Experiment 3, the Faster R-CNN model, RT-DETR-l model, and YOLOv3s-SPP exhibited high rates of false detections and misses. Although the RT-DETR-x model successfully identified some cracks, it did so with low accuracy in terms of category classification and confidence levels. In contrast, our model accurately detected cracks without any instances of leakage or misdetection. In Experiment 4, our model demonstrated superior performance by not exhibiting any leakage or misdetection and maintained a high confidence level relative to other models. Conversely, the Faster R-CNN model, YOLOv3s-tiny, and YOLOv5s were all prone to misdetections and false negatives. Among the models evaluated, YOLOv8s showed the poorest performance, failing to detect any road defects.

Table 11.

The performance table of different models on the India dataset.

Models	mAP@0.5/%	Para (M)	GFLOPs	FPS
Faster R-CNN	0.328	41.36	940.85	12
RT-DETR-l	0.299	56.4	105.5	58.9
RT-DETR-x	0.353	129.2	232.7	43.0
YOLOv3s	0.375	15.35	43.9	110.9
YOLOv3s-tiny	0.339	3.21	5.1	400.1
YOLOv3s-SPP	0.411	15.61	44.1	119.4
YOLOv5s	0.403	9.11	23.8	179.7
YOLOv6s	0.341	16.30	44.0	264.1
YOLOv8s	0.381	11.16	28.6	384.6
Ours	0.421	8.59	22.4	136.7

Figure 10.

Comparative experiments on visualization results for different models on the India dataset: (a) Faster R-CNN; (b) RT-DETR-l; (c) RT-DETR-x; (d) YOLOv3s (e) YOLOv3s-tiny; (f) YOLOv3s-SPP; (g) YOLOv5s; (h) YOLOv6s; (i) YOLOv8s; (j) Ours.

Table 12.

The performance table of different models on the GRDDC dataset.

Models	mAP@0.5/%	Para (M)	GFLOPs	FPS
Faster R-CNN	0.498	41.36	940.85	12
RT-DETR-l	0.501	56.4	105.5	58.9
RT-DETR-x	0.573	129.2	232.7	43.0
YOLOv3s	0.638	15.35	43.9	110.9
YOLOv3s-tiny	0.479	3.21	5.1	400.1
YOLOv3s-SPP	0.637	15.61	44.1	119.4
YOLOv5s	0.638	9.11	23.8	179.7
YOLOv6s	0.651	16.30	44.0	264.1
YOLOv8s	0.646	11.16	28.6	384.6
Ours	0.652	8.59	22.4	136.7

Figure 11.

Comparative experiments on visualization results for different models on the GRDDC dataset: (a) Faster R-CNN; (b) RT-DETR-l; (c) RT-DETR-x; (d) YOLOv3s (e) YOLOv3s-tiny; (f) YOLOv3s-SPP; (g) YOLOv5s; (h) YOLOv6s; (i) YOLOv8s; (j) Ours.

In summary, the improvements observed in our model across various experiments can be attributed to several key architectural enhancements. Specifically, the EMA Faster Block structure, which replaces the traditional Bottleneck structure, enhances feature extraction by using partial convolution combined with multi-scale attention. This approach improves spatial feature representation without increasing computational complexity, thereby enhancing detection accuracy and reducing false positives and negatives. For instance, in Experiment 1, this structure contributed to a 4% increase in mAP, enhancing detection accuracy on the Chinese dataset. Additionally, the replacement of the SPPF module with the SimSPPF module further optimizes computational efficiency by utilizing the simpler ReLU activation function, leading to faster processing times while maintaining high accuracy. This change is particularly beneficial in real-time detection scenarios, as evidenced by the increased frames per second (FPS) metrics in our experiments. Furthermore, the integration of the Detect-Dyhead module significantly enhances the model’s ability to dynamically adjust its focus on different scales and tasks. This results in improved precision and recall across various datasets, as the module’s attention mechanisms ensure that relevant features are prioritized during detection. For example, in Experiment 2, the dynamic adjustments provided by the Detect-Dyhead module led to a notable improvement in detection performance on the US dataset, underscoring the model’s robust capabilities.

Overall, these modifications collectively contribute to the observed improvements in model performance, including higher mean average precision (mAP), reduced model size, and lower computational complexity. The enhancements make our model not only more accurate but also more efficient and suitable for deployment in real-time and resource-constrained environments.

Discussion

The improved YOLOv8s model demonstrates enhanced accuracy in road defect detection compared to the original model. However, it still faces challenges, particularly in accurately detecting certain road cracks due to factors like adverse weather and lighting conditions, as well as image blurring from long shooting distances or small target sizes.

Significant improvements in our model’s performance stem from architectural enhancements. The EMA Faster Block structure integrates partial convolution and multi-scale attention mechanisms, significantly enhancing feature representation capabilities while maintaining a lower parameter count. The SimSPPF module optimizes spatial pyramid pooling for improved computational efficiency, crucial for real-time applications. The Detect-Dyhead module dynamically adjusts focus during detection, enhancing precision and recall. These enhancements address limitations of the original YOLOv8 model, improving accuracy, model size, and computational efficiency.

Furthermore, the improved YOLOv8s model shows substantial gains in accuracy (mAP@0.5), reductions in model and parameter size, and improvements in computational complexity compared to its predecessor. However, performance improvements come at the cost of increased detection time due to the adoption of Detect-Dyhead, which adds three attention modules. To mitigate this, the SimSPPF structure with a simpler ReLU activation function is employed to boost detection speed. Incorporating the C2f-Faster-EMA structure with attention mechanisms further enhances overall model performance.

Accurate road defect detection is critical for routine maintenance and road safety. Optimizing the improved YOLOv8s model in terms of size, parameters, and computational efficiency makes it suitable for deployment on mobile or embedded devices in road defect detection equipment.

Conclusion

This paper introduces an improved YOLOv8s algorithm to address the limitations of the original YOLOv8s algorithm in road crack detection, specifically targeting low accuracy and a high number of model parameters. The proposed enhancements include lightening the bottleneck, adopting the C2f-Faster-EMA structure, utilizing SimSPPF with a ReLU function, and employing the Detect-Dyhead module with attention mechanisms for object unification across scale, spatial, and task dimensions.

Experimental results demonstrate the effectiveness of the model in detecting road defects in images captured by UAVs and on-board cameras. Comparative experiments with other object detection models, including Faster R-CNN, YOLOv3, YOLOv3-tiny, YOLOv3-SPP, YOLOv5, and YOLOv6, show improved detection accuracies ranging from 3.0 to 12.4%. Moreover, compared to the original YOLOv8s model, the improved YOLOv8s model exhibits a 5.8% improvement in mAP@0.5, a 22.33% reduction in model size, a 23.03% reduction in model parameters, and a 21.68% reduction in computational complexity.

The improved YOLOv8s model ensures real-time crack detection while enhancing accuracy, providing a valuable reference for the development of road maintenance systems. Its suitability for deployment on memory- and computation-constrained devices, such as mobile or embedded devices, opens avenues for practical applications. The model presented in this paper exhibits improved accuracy and more stable performance. Nevertheless, the imperative to minimize the number of parameters and computational requirements is paramount, especially for running the model on embedded or mobile devices. Subsequent experiments will continue to scrutinize the model’s parameters and computational complexity, ensuring its suitability for deployment on embedded and mobile devices.

Author contributions

All the authors contributed extensively to the manuscript. J.W. designed the experiments and wrote the main manuscript. R.M. revised and suggested the paper and helped with the formatting review and editing of the paper. All authors have read and agreed to the publication of the manuscript. J.W. and R.M. developed the experimental plan and methodology; J.W. conducted the experiments; R.M., Z.Q., and C.N. performed the experimental validation; L.H., L.Z., and Y.H. analyzed the data and generated the graphs; L.H., L.Z., and Y.H. conducted the literature survey and collected the data; All authors reviewed and wrote the manuscript. All authors have read and agreed to the published version of the manuscript.

Data availability

All the images and experimental test images in this paper were from the open source RDD2022 dataset and GRDDC dataset. These datasets analyzed during the current research period can be found at the following website. RDD2022 and GRDDC: https://github.com/sekilab/RoadDamageDetector (accessed on 6 November 2023).

Competing interests

The authors declare no competing interests.