SGPLane: Efficient lane detection via sampled grid points for autonomous driving

Graphical abstract

Abstract

Lane detection is one of the critical tasks for autonomous driving. Earlier works revolved around semantic segmentation and object detection with a special program for lanes. However, most methods still suffer from unstable post-processing algorithms which leads to a gap between camera input and downstream applications. In this paper, we propose a novel detection presentation form for lanes and design a simple network without any complicated post-process. Specifically, we use sampled gird points to express lane lines and construct a network for the special lane format, which is called SGPLane. Therefore, the network learns a regression branch and a confidence branch to realize end-to-end lane detection by setting the threshold confidence value. Our model is validated on the typical dataset and real-world driving scenes. Experiments on lane detection benchmarks show that our method outperforms previous methods with accuracy score of 96.84$\%$ on Tusimple dataset with high FPS and 76.85$\%$ on our real-world dataset.

1. Introduction

Lane detection is a critical challenge in the domain of autonomous driving, with its performance significantly influencing downstream applications [1]. For dynamic maps, positioning and decision making, high quality vectorized lane lines are required. For other applications, such as the crowdsource mapping, the low time cost is a crucial capability. With the rapid advancements in deep learning theories, a plethora of impressive methods have emerged, providing inspiration for robust lane extraction. Several works in lane detection have demonstrated significant progress by employing effective models.

From a representation perspective, lane detection methods can be roughly divided into three categories: point-based [2], [3], [4], [5], [6], line-based [7], and region-based [8], [9]. The point-based methods usually represent lanes as a set of sampled points [6]. The line-based methods usually use a curve line fitting formula to describe the lanes or anchors with rotation angle and offset [10]. The region-based methods usually consider each lane as a set of contiguous regions, usually processed by a classical semantic segmentation model [8]. The region-based representation is closest to the original lane pixels in the image, which suffers from distinguishing lanes at the instance level. In addition, this representation is weak when multiple lanes are very close together. Line-based representations can export lanes directly. Formula-based line methods have no direct connection to the shape of the image, so the results are seriously distorted by parameters that are far from the ground truth [11]. Anchor-based line methods are usually bottlenecked by anchor shapes and anchor filtering [10]. We choose points to represent lane lines mainly because points are flexible enough to characterise line shapes with high accuracy. Points can be combined to form lane instances. However, previous point-based methods [2], [3], [4], [5], [6] are inefficient at learning the exact position of abundant points [12] and associating points that belong to the same lane.

To overcome the above limitations, we formulate a lane detection task as a grid-sampled point regression problem and propose an end-to-end lane detection model. As shown in Fig. 1d, the $y$-axis of the images are sampled with a certain proportion of the grid, so that the distance between the near coordinates is large and the distance between the far coordinates is small. The grid is a set of fixed sampled $y$ value from the $y$-axis as shown in Fig. 1d, which can be formulated as Eqs. 2,3. As the grid $y$ value is preset, we only need to sample the points of the lane line on the grid and regress the corresponding $x$ coordinate values to obtain the lane line results. However, not all preset grids have corresponding lane lines, so the classification value of non-existent sampled points is set to 0 to control.

Fig. 1.

Comparison of our method with previous work on lane detection based on sampled points[2], [3], [6]. (a) FOLOLane [3] predicts the candidate keypoints, constructs associations between them, and then groups the keypoints into lane lines. (b) RCLane [2] models the keypoints of each lane as a relay chain. RCLane predicts the starting point and then flows up and down one by one. (c) GANet [6] directly regresses each key point to its associated lane by predicting the offset between each key point and the starting point of its corresponding lane line. (d) Our model samples the key points of each lane through a grid on the y-axis, and regresses the $x$-coordinate values of sampled key points.

This approach enables a detailed description of the shape changes of each lane through point sampling, even for distant lane lines. The advantage of fixed ordinate is that the network can learn the corresponding normalised abscissa in the order of ordinate sampling, so each group of network outputs corresponds to a lane line, and the refined instantiation lane line result can be obtained directly. The points belonging to a line are output on the same layer of the network so that this model can directly obtain the instantiated lane line. Coordinates are output directly through the sampled design mesh. These two special designs avoid complex post-processing, so the model can be applied to real vehicles with high efficiency. In the process of network training, this ordered sampling design makes the receptive field of the network automatically focus on different positions of the lane line, which is equivalent to an adaptive point anchor. To effectively control the occurrence of redundant points in the lane line, we added a confidence branch to determine whether the points exist with a certain threshold. This allows us to arbitrarily control the number of lane lines and sampling points output. To make this pipeline effective, we need to add Hungarian matching to the training process to avoid complex post-processing such as NMS [13].

The main contributions of this work are summarized as follows: (1) This paper proposes a novel lane line representation based on sampled grid points, which simplifies lane detection to an $x$-coordinate regression problem. (2) We design an end-to-end CNN network model to extract lane lines without additional expensive post-processing.

The experiment results show that our model achieves good performance on Tusimple, with an accuracy of 96.84$\%$, and is suitable for real-world corner cases, which are rare in public datasets.

This paper is organized as follows: Section 2 summarizes the related work of lane detection. Section 3 primarily explains the technical methods at length. Section 4 describes the experiment details and shows the results on the open dataset. Finally, in Section 5 we make a conclusion.

2. Related work

As a classic and widely used computer vision task, lane detection has a long research history. Initially, the lane detection task was considered as a colour thresholding problem [14], an edge detection problem [15] or a Hough transform [16] using traditional digital image processing methods. With the development of deep learning and computing power, training a deep neural network with a large amount of lane data has become mainstream. Thanks to the impressive capabilities of deep learning frameworks, lane detection is becoming increasingly flexible. By designing flexible expression forms of lane lines and corresponding neural networks, it is possible to achieve increasingly accurate and efficient lane detection.

In this section, we will discuss these methods separately from the perspective of lane representation and lane organisation process.

Description of lanes: From the perspective of expression, lanes can be regarded as points, lines, and areas. Corresponding to the expression format, lane detection models are designed particularly. The description of lanes determines the output format of a deep network, which has a great impact on the detection performance. Lanes can be described in wide ways because of their narrow shape.The point-based works module lanes as a set of sample points. Then the task can be considered as a point regression or classification problem. FOLOLane [3], RCLane [2] and GLANet [6] use a set of points to depict the shape of the lane line, and the difference is how to combine points belonging to the same lane line, as shown in Fig. 1. PiNet [5] is one of the point-based methods, which estimates the points of lanes. UFAST [4] selects the location of sampled points row by row. Girds are made to sample lanes, so the module only needs to classify the sampled points. To sum up, this kind of work give the discrete points to describe lanes. The line-based works module lanes as several curves. Then the frameworks usually learn the key parameter of the curves. LaneNet [8] designs a small CNN model to forecast the curve-fitting parameters of each lane. PolyLaneNet [7] models the lanes as curves with the formation of polynomials. This kind of method regards lane detection as a task of parameter regression, which simplifies the amount of regression value compared with the method of regression point directly. The region-based works module lanes as connected domains. This kind of description usually uses semantic segmentation works to discriminate the pixels of lanes. SCNN [9] is one of the most classic lane detection works. SCNN [9] designs a special convolution network that conducts segmentation with convolution in a certain direction. The first step of LaneNet [8] uses a CNN backbone to learn the binary segmentation mask of lanes. The binary mask is the repression of the lane region. IntRA-KD [17] treats the lanes as one of the classes and conducts semantic segmentation to distinguish the lane pixels. The post-processing of the region-based method is complex, inefficient and unstable, and the line-based method is greatly disturbed by the influence of parameters. Among these representations, the key point is the most flexible and accurate form that fits the shape of the lane line.

The procedure of lane detection: From the perspective of the detection procedure, we can roughly divide the framework into two categories: top-down and bottom-up. According to the processing pipeline, lane detection can be roughly divided into top-down methods and bottom-up methods, which is corresponding to the classification of instance segmentation methods. Top-down methods have been a mainstream of lane detection. In the first instance, top-down works give the sketchy proposal of lanes. A set of offset values is then learned to refine the lane shape. The anchor-based method is a typical type of top-down work with fixed anchors or flexible anchors. Inspired by the region anchors in Faster-RCNN [18], line-CNN [19] first sets line anchors to detect lanes. This pipeline has gained in popularity over recent years. Several works such as CurveLane-NAS [20], LaneATT [10], and SGNet [21] have pursued the framework. LaneATT [10] designs the anchor-based feature pooling particularly based on line-CNN [19]. SGNet [21] utilizes bounding boxes as the anchors and learns the offset to the edges of bounding boxes to specify the lane shape. CondLaneNet [22] is another type of top-down framework. It designs a proposal head primarily to give the line proposal, then a conditional shape head is constructed to refine the shape of the lines with the conditional convolution. However, the anchor-based methods have a bottleneck due to the limitation of anchors. The bottom-up method is to fetch the refined lines and extract the lines at the instance level based on the first step. When it comes to the pipeline of bottom-up, the segmentation-based method has been mainstream. LaneNet [8] introduces the bottom-up idea of instance segmentation to lane detection. This work learns the value of embeddings to represent lane instances. In the space of embeddings, pixels belonging to the same lane are very close to each other, while points belonging to different lanes are very far from each other. However, clustering pixels according to embeddings leads to huge post-processing costs. To overcome the shortcomings of clustering post-processing, LaneAF [23] designs the affinity fields to help clustering. Another type of bottom-up method is to first extract the maximum amount of proposed lanes and use NMS to find out the redundant lane proposals. SOLOLaneNet [24] is one of the NMS based methods. No matter whether the extraction method is top-down or bottom-up, the two links of lane line instance extraction and lane line shape extraction need to be organised respectively. Our SGPLane adopts a more efficient framework, which neither needs top-down to screen the redundant lane instance, nor bottom-up to cluster or combine keypoints to lane line. Each lane line instance is obtained through the preset network channel, and the key points belonging to the same lane line are sampled through the preset grid and represented by coordinate values.

To sum up, due to the performance bottleneck of the lane line representation method based on line and region, we select key points as the effective expression method of lane line to express the shape of lane line. For this point representation, we use the preset grid sampled keypoints to design a one-stage end-to-end network SGPLane to avoid the complexity of lane line shape characterization and instance extraction.

3. Methodology

The overall architecture of our proposed Sampled Grid Points Lane Detection Network (SGPLane) is illustrated in Fig. 2. Given a front-view camera image $\mathit{I}\in R^{H\times W\times 3}$ as input, where H and W denote the height and width of the input image, the goal of SGPLane is to predict a set of lanes as the form of sampled points axis. The input image is fed into a CNN backbone, which can extract deep features. We design two branches to get the final output without any post-processing method. One is for the regression of lane points, and the other is for the confidence estimation of corresponding lane points. Through the two branches of sampled points decoder networks, the deep features are translated to the axis and the confidence of each lane point. In the following subsections, we introduce the crucial components of our method.

Fig. 2.

Architecture for our SGPLane network. The image is fed into the encoder to obtain the depth feature, and then the depth feature is fed into the decoding branch structure with the same structure. One branch is used to learn the confidence of the grid sampling points, and the other branch is used to learn the normalised coordinate value of the grid sampling points. Finally, the results of the lane line instance are obtained directly by combining the outputs of the two branches.

3.1. Sampled points representation

To obtain the specific location information of each lane line in the image directly according to the lane instance, and inspired by the idea of DETR [25], we directly design each column of the network output as a lane line information. Therefore, the sampled lane line point representation is the core of this work. Given a front-view camera image $\mathit{I}\in {\mathbb{R}}^{H\times W\times 3}$, we model lanes in the image $\mathit{I}$ as a set of lines:

$$L=\{l_1,l_2,\dots l_N\}$$ (1)

where N is the preset number of lanes in the image.

As shown in Fig. 1d, the $y$-axis is sampled by a preset grid with a fixed start value, a fixed end value and fixed intervals. K is the preset fixed number of sampled grid of the whole image. The sampling intervals are also preset through experience. The fixed start value is set to H-1 because this is the $y$ coordinate value of the image bottom. The fixed end value is set in the range of [0, H/2] as Eq. 3. According to the distribution of lane lines in the driving image, the end of the lane line is basically within this interval, and we usually set it as H/4.

Then we set the end $y$-axis value (the start $y$ value is always set to the bottom edge of the image) and sample $K-2$ other fixed coordinates on the $y$-axis:

$$y=\{y_1,y_2,\dots y_K\}$$ (2)

$$y_1=H-1,y_K\in [0,H/2]$$ (3)

$$y_j-y_{j+1}=I(y_{j+1}-y_{j+2})$$ (4)

where $y_j$ is the $y$ coordinate of the $j$th sampled point on all lane and $I$ is the interval coefficient, which is usually set to 1.1. We use a set of sampled points to represent a lane as:

$$l_i={\{p_1^i,p_2^i,\cdots ,p_K^i\}}_{i=1}^N$$ (5)

where $p_j^i$ is the $j$th sampled point on the $i$th lane line. Then coordinates of $p_j^i$ can be expressed as:

$$p_j^i=(x_j^i,y_j^i)$$ (6)

where $y_j^i$ is the $y$ coordinate of the $j$th sampled point on the $i$th lane line, $x_j^i$ is the $x$ coordinate of the $j$th sampled point on the $i$th lane line. Sicne we fix the sampled grid, the equation can be expressed as:

$$y_j^i=y_j$$ (7)

Therefore, coordinates of $p_j^i$ can be expressed as:

$$p_j^i=(x_j^i,y_j)$$ (8)

Not all lane lines exist on all sampled $y$ grid. For non-existent lane line sampled points, it is necessary to set the value to zero through confidence threshold filtering, so that the structured lane line results with fixed points conform to the real situation. $\theta$ is the set threshold value of confidence to figure out whether there exists a point.

Algorithm 1 is the algorithm to judge the existence of points from the classification branch output, where $\widehat{C_j^i}$ denotes the corresponding confidence value of the $j$th point in the $i$th lane from classification branches. We set the $x$ coordinate of non-existent points to zero.

Algorithm 1.

Judgement of point existence.

Algorithm 2 is the algorithm to set the $x$-axis value of non-existent point.

Algorithm 2.

Distinguish the non-existent points.

Since $y$-axis grid is set and the values are known, the lane line can be simplified as $x$-axis values:

$$l_i={\{{x_1^i}^{\prime },{x_2^i}^{\prime },\cdots ,{x_K^i}^{\prime }\}}_{i=1}^N$$ (9)

Through this representation, the coordinates of the sampling point can be directly corresponding to the network output, so it is only necessary to output each branch of the network as a $K\times N$ matrix.

3.2. Network architecture

The whole framework is a traditional encoder-decoder network. We adopt some available backbones as the encoder module. To enhance the performance of points regression, we immediately link the layer of the encoder and the two output branches as FPN [26]. We choose ERFNet [27] as our backbones to extract deep features of each input image.

Spatial position implicit coding. When we fix the y-axis grid value of sampled points, the attention field is fixed to the longitudinal distribution of the lane line. Therefore, features belonging to each grid only need to pay attention to nearby values and we do not need to associate global image information. The output form of grid is equivalent to implicit lane anchor. The visualization of all the regression branch results is shown in Fig. 3. It can be seen that without classification value, all preset N lanes are N lane implicit anchors, and all lane implicit anchors only care about the adjacent area.

Fig. 3.

Visualization of regression branch results.

Regression head. The regression branch is designed to decode the axis of each sampled points. In order to make the network easier to converge, we use image width W as denominator to normalize all $x$ coordinate of sampled points in this branch.

Classification head. The classification branch is designed to estimate the confidence of each sampled points. The output of this branch represents the probability of point existence. Through the classification branch, our network can ignore the empty lane prediction and filter out points that do not exist in the sampling $y$ grid.

3.3. Hungarian matching and losses

To establish a unique relationship between the predict lane point set and the ground truth point set, we adopt the Hungarian matching method used in DETR [25]. The Hungarian algorithm is the most common algorithm for partial graph matching. The core of this algorithm is to find the augmented path. It is an algorithm for finding the maximum matching of Bipartite graph by using the augmented path. Through the Hungarian matching algorithm, the cost matrix is to find the optimal assignment of detected lane ground truth to potential lanes. The algorithm minimises the total cost by finding the best pairing of lane ground truth and lane prediction, ensuring that each lane is associated with only one lane. Through this matching algorithm, the network can reasonably converge rather than over-fit and remove the redundant lane line prediction value during training. Furthermore, this method avoids the NMS post-processing [13] to reach end-to-end performance.

To train the proposed CNN network, we utilize two kinds of loss function for the two branches. Because the existing points are sparse relative to the preset $K\times N$ matrix output, we adopt focal loss to regress the confidence value of classification branch. Focal loss [28] is a frequently-used loss function to do the key point estimation:

$$\begin{array}{ccc}\hfill {\mathrm{Loss}}_{\text{focal}}& =& \frac{-1}{K\times N}\sum _{i=1}^N\sum _{j=1}^K\hfill \\ & & ·\{\begin{array}{c}{(1-\widehat{C_j^i})}^{\alpha }\log \left(\widehat{C_j^i}\right),\text{if}{C}_j^i=1\hfill \\ {(1-C_j^i)}^{\beta }{\left(\widehat{C_j^i}\right)}^{\alpha }\log (1-\widehat{C_j^i}),\text{otherwise}\hfill \end{array}\hfill \end{array}$$ (10)

where $\alpha$ and $\beta$ are set the same as the value in CenterNet [29]. $\widehat{C_j^i}$ denotes the predicted confidence value of the $j$th point in the $i$th lane from classification branches. $C_j^i$ denotes the corresponding ground truth, where 1 means that the point exists.

Because there are more preset points than are actually visible, there are many more other value than 1s in the matrix representing the true value. Focal Loss is designed for unbalanced samples, so using Focal Loss can speed up convergence.

For the $x$-axis regress branch, L1 regression loss is used to regress the normalized $x$ values. We only compute the loss of matched lanes through Hungarian matching:

$${\mathrm{Loss}}_{L1}=\frac{1}{K\times N}\sum _{i=1}^K\sum _{j=1}^N|X_j^i-\widehat{X_j^i}|$$ (11)

where $X_j^i$ denotes the ground truth of the normalized $x$ value of the $j$th point on the $i$th lane line, $\widehat{X_j^i}$ denotes the prediction result of the corresponding point.

Then the total loss of the network is computed as:

$$\mathrm{Loss}={\lambda }_{focal}{\mathrm{Loss}}_{focal}+{\lambda }_{L1}{\mathrm{Loss}}_{L1}$$ (12)

where ${\lambda }_{focal}$ and ${\lambda }_{L1}$ are the loss weight coefficients to balance the training of the two branches. We adjust the two values based on the experimental results. ${\lambda }_{L1}$ is set more larger because L1 loss is only calculated on the lane line results that match the true value, while focal loss is calculated for each preset point.

3.4. Synthesis of lane line instances

Though the two branches of SGPLane, the ordinate information and the confidence are known. Then we use a threshold $\theta$ to change the confidence branch into one-hot to separate the non-existent points from the visible points. The post-processing method is shown in Fig. 4.

$$\{\begin{array}{c}{T}_j^i=1,\widehat{C_j^i}>\theta \hfill \\ T_j^i=0,\widehat{C_j^i}<\theta \hfill \end{array}$$ (13)

where $\widehat{C_j^i}$ is the value at row $i$ and column $j$ of the classification head output matrix $\widehat{C}\in {[0,1]}^{K\times N}$, which denotes the confidence of the $i$th point on the $j$th lane.

Fig. 4.

Post-processing pipeline for our SGPLane. (a) is the output result of regression branch. (b) is the output result of classification branch. (b) is filtered by the threshold and multiplied by (a), as (a) is normalized value that needs to be multiplied by the image width W to get the final sampled grid points result (c). (d) is the visual diagram.

Then we get the formulate of output value from Algorithm 1:

$${x_j^i}^{\prime }=W*\widehat{X_j^i}*T_j^i$$ (14)

where $\widehat{X_j^i}$ is the value at row $i$ and column $j$ of the regression head output matrix $\widehat{X}\in {[0,1]}^{K\times N}$, which denotes the normalized $x$ coordinate of the $i$th point on the $j$th lane. Because the predicted value $\widehat{X}$ is a normalized quantity, it is necessary to multiple the W value (width of image) to obtain the final result.

4. Experiment

In this part, experiments have been carried on several datasets to verify the practicability of our work.

4.1. Dataset

The Tusimple dataset [30] is a widely used open dataset to demonstrate the feasibility of works. It has 3626 images for training, 358 images for validation and 2782 images for test with excellent visual condition. The main evaluation metric of TuSimple dataset is accuracy, which is formulated as follows:

$$\text{accuracy}=\frac{{\sum }_{clip}{C}_{clip}}{{\sum }_{clip}{S}_{clip}}$$ (15)

where $C_{clip}$ is the number of points correctly predicted by the model and $S_{clip}$ is the total number of points in the clip. Since Tusimple is a dataset organized as a set of points, the basis of quantification is the position relationship of points. A predicted point is regarded correct only if it is within 20 pixtels of the ground truth point. The predicted lane with an accuracy greater than $85\%$ is considered a true positive.

We also conduct experiments on our private driving scene dataset named Shougang dataset. Shougang dataset is consist of the data collected by driving vehicles in Shougang, which focuses on the complex real world scene at the end of the vehicle, including multiple lane lines close to each other, lane lines blocked by guardrails, and large differences in image brightness. Shougang dataset has 4768 images for training, 402 images for validation and 1927 images for test. Because the lanes are very close, a predicted point is regarded correct only if it is within 8 pixtels of the ground truth point. The predicted lane with an accuracy greater than $85\%$ is considered a true positive. The evaluation metric is more strict than TuSimple.

4.2. Implementation details

We choose ERFNet [27] as the backbones to form two different versions of SGPLane.

The size of input pictures is set to $480\times 768$. The number of sampled points of each lane is set as $K=32$ in Shougang dataset. The number of sampled points of each lane is set as $K=48$ in Tusimple dataset. The number of lane instance N is set to 40. The loss weight coefficients ${\lambda }_{focal}$ is set to 1 and ${\lambda }_{L1}$ is set to 40. The hyper-parameters $\alpha$ and $\beta$ in Eq. 10 are set as 2 and 4 respectively. For optimization, we used Adam optimizer and poly learning rate decay with an initial learning rate of 0.001. We train 300 and 500 epochs for Tusimple and Shougang dataset respectively with a batchsize of 32 per GPU. Training are performed on NVIDIA GTX 1080 Ti. Testing are performed on Tesla-V100 GPUs.

In the test phase, we set the threshold $\theta$ of keypoints as 0.4. As the ground truth value of the classification branch is set to 0 and 1, 0 means non-existent, 1 means visible. Due to the focal loss, the distribution of the classification value is basically close to 0 or 1. Therefore, our model can easily filter the lane line sampling points that do not exist in the actual image through the middle threshold value. Obviously, as the threshold $\theta$ increases, more potential points are filtered and FP increases. As the threshold $\theta$ decreases, more possible error points are added, so FN increases. 0.4 is a result that takes into account various indicators, and we hope to detect as many as possible rather than miss them. In fact, the choice of 0.4, 0.5 or 0.6 doesn’t make much difference.

4.3. Results

Some of the visualization results on Tusimple and our driving scene dataset are shown in Figs. 5 and 6. To evaluate the performance of our works, we choose FP, FN, accuracy and the running speed FPS as the quality index. FP and FN are the rate to reflect errors. FP is the number of lanes the model has detected while there is no such a lane. FN is the number of lanes which exist but the model forgets. The accuracy is defined as:

$$\text{Accuracy}=\frac{N_{pred}}{N_{gt}}$$ (16)

where $N_{pred}$is the quantity of the right-predicted points and $N_{gt}$ is the ground-truth.

Fig. 5.

The visualization results of Tusimple.

Fig. 6.

The visualization results of some real world scenes.

Fps is an abbreviation for the number of frames transmitted per second, representing the speed of network computing. FPS is defined as:

$$\text{FPS}=\frac{1}{mT}$$ (17)

where $mT$ is the average time required for the network to process each frame of image.

The comparison results on TuSimple dataset are shown in Table 1. Our SGPLane has good performance and achieves the best accuracy score of 96.84$\%$ with high FPS. The low FP value means our SGPLane is rarely disturbed to produce wrong results. High FPS mainly comes from the simple process pipeline, which illustrates our sampled gird points model is very efficient and effective. Low FP with high FPS means our SGPLane is very valuable for actual driving.

Table 1.

Performance of different methods on Tusimple.

Method	Accuracy	FP	FN	FPS
Region-based
SCNN [9]	$96.53\%$	0.0617	0.018	7.5
LaneNet [8]	$96.38\%$	0.0780	0.0244	52.6
LaneAF [23]	$96.49\%$	0.0280	0.0418	–
Line-based
LaneATT-ResNet34 [10]	$95.63\%$	0.0353	0.0292	171
BezierLaneNet [11]	$95.65\%$	0.051	0.039	–
Point-based
GANet-M [6]	$95.87\%$	0.0199	0.0264	127
RCLane-M [2]	$96.51\%$	0.0224	0.0236	43.8
SGPLane(Ours)	96.84$\%$	0.0183	0.0256	134

The comparison results on our Shougang dataset are shown in Table 2. Our SGPLane outperforms all other methods and achieves the best score among them. Because the Shougang dataset is composed of complex real world lane line scenes, the state-of-the-art performance shows that our SGPLane network can extract lane lines with high accuracy, which is particularly effective for complex scenes such as dense lane lines. Because this dataset is composed of complex lane line scenes, sota performance shows that our network can extract lane lines with high accuracy, which is particularly effective for complex scenes such as dense lane lines. This is because we have formed implicit anchors for individual lane line instances in the training process through preset sampling grids, and the corresponding positions of anchors pay more attention to the close sampling points, making the convolution more effective. This method can not only get the position of the point with high accuracy, but also distinguish the lane line directly through the implicit anchor, so that the error detection of complex scenes is less. As shown in Fig. 7, other methods make it difficult to separate tightly connected lane lines through the mask of lane lines, resulting in multiple lane lines sticking together. Moreover, due to the instability of the pixel area, there is also a lot of noise, and there are also many cases of discontinuous pixels in farther lane lines, which poses a great challenge to post-processing.

Table 2.

Performance of different methods on Shougang dataset.

Method	ACC	FP	FN	FPS
GANet [6]	$68.34\%$	0.0576	0.0459	127
RCLane [2]	$73.96\%$	0.0591	0.0563	43.8
SGPLane(Ours)	76.85$\%$	0.0538	0.0442	134

Fig. 7.

The visualization results of some error detection results.

The visualization results of some complex lane line detection scenarios is shown in Fig. 8. The green points in the graph represent the predicted results, while the red points represent the true values. The complex lane line situation includes multiple lane lines closely connected, with lane lines obstructed by guardrails and closely connected to each other, bends, lane lines merging, and visibility obstructed in foggy weather. It is clear from the images that our network has very good lane detection results even in complex lane situations.

Fig. 8.

The visualization results of some complex lane line detection scenarios.

5. Conclusion

In this paper, we propose a novel expression module of lane lines and construct an end-to-end two-branch network model to correspond with this special sampled grid points. Each key point is directly regressed to normalized $x$ coordinate, then we combine the $x$ coordinates of each lane line into a sequential chain of lane line points in the format of network output. Owing to the simple but effective model, we can realize robust, accurate, clear, end-to-end lane line extraction. Experimental results show our SGPLane reaches competitive performance on public benchmarks Tusimple. Moreover, SGPLane is superior to the existing methods in our real driving dataset.

Declaration of competing interest

The authors declare that they have no conflicts of interest in this work.

Biographies

Xuewei Tang received the BS degree from the Department of Automation, Tsinghua University, Beijing, China, in 2020. She is currently working toward the PhD degree at the School of Vehicle and Mobility of Tsinghua University, Beijing, China. Her research interests include computer vision and high definition map construction.

Mengmeng Yang received the PhD degree in photogrammetry and remote sensing from Wuhan University,Wuhan, China in 2018. She is currently an assistant research professor at Tsinghua University, Beijing, China. Her research interests include autonomous vehicles, high precision digital map, and sensor fusion.

Diange Yang received his PhD in automotive engineering from Tsinghua University in 2001. He is now a professor at the School of Vehicle and Mobility at Tsinghua University. He currently serves as the director of the Tsinghua University Development & Planning Division. His research interests include autonomous driving, environment perception, and HD map. He has more than 180 technical publications and 100 patents. He received the National Technology Invention Award and Science and Technology Progress Award in 2010, 2013, 2018, and the Special Prize for Progress in Science and Technology of China Automobile Industry in 2019.