Toward autonomous vehicles: A survey on cooperative vehicle-infrastructure system

Summary

Cooperative vehicle-infrastructure system (CVIS) is an important part of the intelligent transport system (ITS). Autonomous vehicles have the potential to improve safety, efficiency, and energy saving through CVIS. Although a few CVIS studies have been conducted in the transportation field recently, a comprehensive analysis of CVIS is necessary, especially about how CVIS is applied in autonomous vehicles. In this paper, we overview the relevant architectures and components of CVIS. After that, state-of-the-art research and applications of CVIS in autonomous vehicles are reviewed from the perspective of improving vehicle safety, efficiency, and energy saving, including scenarios such as straight road segments, intersections, ramps, etc. In addition, the datasets and simulators used in CVIS-related studies are summarized. Finally, challenges and future directions are discussed to promote the development of CVIS and provide inspiration and reference for researchers in the field of ITS.

Graphical abstract

Internet of things; Artificial intelligence; Transportation engineering

Introduction

With the rapid increase in the number of vehicles, there are more and more traffic accidents. Around 1.3 million people die globally each year in road accidents, the leading cause of death among young people.¹ These factors are responsible for around 94% in accidents, including distraction, fatigue, and emotional driving according to a statistical survey completed by the National Highway Traffic Safety Administration.² Autonomous vehicles (AVs) were born to relieve these problems. Compared to human-driven vehicles (HDVs), AVs have potential to improve road safety through more precise positioning and speed control and shorter reaction times.^3,4 However, there are still many scenarios that cannot be handled by AVs. For one thing, the perception system of AVs may fail in some special conditions (e.g., wicked weather and system faults), resulting in serious consequences. For another, AVs cannot take into account macro traffic conditions and complex traffic scenarios (e.g., temporary traffic control, chaotic intersections, etc.). These issues can be solved by cooperative vehicle-infrastructure system (CVIS), which can provide AVs with a global view.

In 1986, the concept of CVIS was first proposed by Partners for Advanced Transit and Highways (PATH) of the University of California, Berkeley.⁵ With the development of technology, CVIS becomes an efficient combination of Internet of things and intelligent vehicles.⁶ In CVIS environment, information on roadside perception (e.g., vehicles and pedestrians beyond visual range) can be provided by CVIS for AVs.⁷ In addition, CVIS can exploit a variety of sensors and communication equipment to carry out intelligent information exchange and sharing among vehicles, infrastructure, pedestrians, and the cloud, so as to improve vehicle safety and traffic efficiency while reducing energy consumption.

The key elements of CVIS are vehicles and infrastructure. Due to the increasing intelligence of AVs and roads, CVIS is constantly evolving. As shown in Figure 1, according to SAE J3016, the degree of vehicle automation can be divided into six levels of L0–L5, and most automation degree of currently listed cars reach L2, such as the popular Tesla.⁸ A small number of AVs can reach the L3, such as the Audi A8.⁹ The AVs of L4 are currently under development, which have not yet been introduced to the market in large quantities. According to the European Road Transport Research Advisory Council, infrastructure support levels for automated driving can be divided into five levels ranging from “E. Conventional infrastructure/No AV support” to “A. Cooperative driving”.¹⁰ Currently, the level of road intelligence is at level C. Moreover, CVIS has three stages of S1–S3.¹¹ The S1 is information exchange and sharing, which can be used for road hazard alerts. At present, S1 has been reached in some cities.^12,13,14 The S2 is collaborative perception, which is carried out by both vehicle and roadside sensors. The S3 is the cooperative decision and control, ensuring that vehicles could drive safely and automatically in any situation. The S2 and S3 are still under exploration. In the current situation, improvements in the infrastructure will significantly increase the level of CVIS according to Figure 1. If the infrastructure level can reach level B, L4 autonomous driving can be achieved only with AVs of L3. It will greatly reduce the development cost of high-level AVs.

Figure 1.

Levels of autonomous driving, infrastructure, and CVIS

CVIS has received increasing attention in literature over the past few years. Simulation and test technologies of CVIS were summarized.¹⁵ Communication technologies utilized in CVIS are commonly referred to as vehicle-to-everything (V2X), including vehicle-to-infrastructure (V2I), vehicle-to-vehicle (V2V), vehicle-to-person (V2P), vehicle-to-network, etc. Some general overviews of V2X technologies were provided, including history, applications, benefits, and challenges.^16,17,18 Vehicles in CVIS are called connected vehicles (CVs). They are often referred to as intelligent connected Vehicles or connected AVs (CAVs) if they are capable of autonomous driving. Zhang¹⁹ analyzed the research status of cooperative decision-making mechanism, method, and application scenarios of CVs in CVIS environment. Additionally, Wu²⁰ reviewed studies on the control of CAVs at intersections. Some cases were analyzed to investigate the potential benefits that CAVs could achieve, including vehicle platooning, lane change, intersection management, vehicle energy management, road friction estimation, etc.^21,22 In addition, the infrastructure in CVIS has also been concerned by some studies.²³ Lim²⁴ presented a review of data platforms for CVs and infrastructure. Most of the previous surveys focus on certain technologies or listed a few simple applications of CVIS. CVIS is a complex system closely linked to AVs, and AVs with CVIS functionality can be transformed into CAVs. However, there is a lack of a complete survey of CVIS in improving the performance of AVs to summarize the up-to-date work. In order to fill this knowledge gap, this paper outlines the architecture and components of CVIS, investigates how CVIS can be specifically applied to AVs from the perspective of enhancing vehicle performance, and summarizes the challenges and research points of the current CVIS technology for AVs.

The structure of research is shown in Figure 2. The remainder of the paper is organized as follow. First, we investigates the architecture and components of CVIS, including overall architecture, infrastructure, AVs, and communication. And then, the applications of CVIS in AVs by improving safety, efficiency, and energy saving are summarized. In addition, we analyze the datasets and simulation environments of CVIS utilized by CVIS in support of CAVs. Finally, the challenges and future directions of CVIS are discussed. It provides reference for the research of intelligent transport system (ITS).

Figure 2.

An overview of the contents of this research

The architecture and components of CVIS

CVIS is a complex system composed of multiple components. The architecture and components of CVIS including the base implementation and CVs are presented in this section. In addition, communication connects all the components and the way of communication between different components is discussed.

Overall architecture

A suitable architecture of CVIS is required to make vehicles and infrastructures work well together. Yu²⁵ proposed an architecture including the centralized cloud, gateway (roadside units [RSUs]), and vehicular cloud. Liu²⁶ proposed a four-layer architecture consisting of environment sensing layer, communication layer, mobile edge computing (MEC) server layer, and remote core cloud server layer. Based on the cloud-assisted real-time methods for autonomy (CARMA) project, Montanaro²⁷ proposed an architecture consisting of three layers: CARMA vehicle, CARMA edge, and CARMA core cloud. In addition, Wang²⁸ proposed a four-layer hybrid architecture that consisted of end users, mobile buses, public infrastructures, and remote cloud. Li²⁹ designed an architecture suitable for CVIS, including CVs and other traffic participants, roadside infrastructures, basic cloud control platforms, related 3rd party supporting platforms, V2X network communication links, cloud application platforms, etc.

From the examples, the architecture of CVIS mainly includes three major layers: vehicle, inter-vehicle, and cloud levels. Pedestrians are crucial for traffic scenarios, so we also take pedestrians into account in CVIS. CVIS architecture is summarized in Figure 3. The overall architecture of CVIS mainly consists of four parts: cloud platform, roadside system, vehicle system, and pedestrian. The cloud platform includes cloud control facilities and data access facilities. The former is responsible for sending corresponding instructions to vehicle systems, roadside systems, and pedestrians. The latter is responsible for taking care of accessing the data of the external third-party platforms (e.g., transportation data center). The roadside system would transmit road information to the cloud control platform to model the virtual environment, transmit traffic information and driver assistance information to the vehicle system, and give traffic information and location guidance information to pedestrians. The vehicle system transmits the status of the vehicle to the cloud control platform and the road system. Besides, it also transmits location information to other vehicles and pedestrians. The pedestrians can share their location, intentions, and status to the vehicle system, roadside system, and cloud control system through wearable devices.³⁰ Infrastructure, perception fusion, and communication all play key roles in ensuring the normal operation of the system.

Figure 3.

The overall architecture of CVIS

Introduction to infrastructure

The infrastructure needed for CVIS will be described in this section, including basic roads, roadside sensors, MEC, and RSU.

Basic roads

Roads are the basis for laying out CVIS infrastructure. There are a large number of sensors and RSUs laid out on the road, and these are generally fixed to poles at the roadside. To realize the full utilization of poles, it is necessary to arrange these devices on the same pole based on different purposes. In addition, many of the devices are retrofitted to existing traffic facilities to save costs. Many traffic facilities can be digitized, and their information can be collected and managed in a unified way to enhance the intelligence of the road. For a long time in the future, the roads will be in a state where CAVs and HDVs coexist. Therefore, to take full advantage of CVIS, it is often necessary to create dedicated lanes for CAVs. A dedicated lane can be a lane added to an existing lane or a lane reserved for CAVs. When and where to create dedicated lanes for CAVs is a question worth studying.³¹ The development of dedicated lanes for CAVs at a reasonable CAVs penetration rate will improve capacity without wasting resources and increasing traffic congestion. Yao³² modeled a basic graph of mixed traffic flow with dedicated lanes for CAVs to determine when and how many lanes to dedicate at different CAVs’ penetration rates. The impact of dedicated lanes for CAVs on highway traffic efficiency was studied.^33,34 In addition, Zhang³⁵ explored the creation of a dedicated lane for connected autonomous buses in urban transportation networks, allowing a portion of CAVs to pass through and improve the performance of the transportation system.

Roadside sensors

At present, the commonly used roadside sensors in CVIS mainly include cameras, lidar, and millimeter-wave radar.^36,37 The comparison of various sensors is shown in Table 1. The cameras are able to detect traffic participants, lanes, obstacles, traffic signs, etc. within the field of view.³⁸ The data collected by the camera are in high resolution, rich color, and vast dynamic range, which help to reveal the real scene. In this way, it is an indispensable perception device for CVIS.³⁹ However, the effect of cameras will be affected by light and weather.⁴⁰ In general, because roadside cameras have lower real-time requirements than cameras on AVs, their frame rate requirements are a little lower. In addition, because roadside cameras are mounted in a fixed location, it is easier to control the lighting conditions, and their dynamic range requirements are a little lower.

Table 1.

The comparison of various sensors

	Range measurement	Resolution ratio	Stability	Computational requirements	Price	Application
Camera	++	++	++	+++	+	Vehicle/road
Millimeter-wave radar	++	+	+	++	+	Vehicle/road
Lidar	++	+++	+++	++	+++	Vehicle/road

Note: “+” for weak/low, “+++” for strong/high.

The advantage of millimeter-wave radar lies in the fact that it can realize the perceptual recognition of roads and road participants that are not disturbed by light or weather. Nevertheless, millimeter-wave radar has low resolution, which means that it cannot detect people, two-wheelers, or animals that are slightly farther away.⁴¹ In addition, it fails to recognize road conditions and accurately distinguish between the targets that are close enough to each other.⁴² Unlike on-vehicle millimeter-wave radar, roadside millimeter-wave radar needs to detect targets in multiple lanes, so it needs to have a wider coverage area. The lidar can provide perceptual identification of roads and road participants without interference from light, including accurate distance measurements and speed measurements.^43,44 The disadvantage of lidar is its lower resolution because of the discrete measured points, so the perception of small targets such as people, two-wheelers, and animals that are slightly farther away is weak. Moreover, the cost is relatively high, some up to tens of thousands of dollars, so it is generally used in very complex intersections.⁴⁵ The field of view of the vertical scanning of a road LIDAR is generally negative due to its installation position. In addition, it is desirable to have a detection range of up to 200 m to cover a sufficiently large area. Optimization of roadside sensors placement is essential to use as few sensors as possible while meeting coverage requirements. The particle swarm optimization (PSO) was used to optimize the placement of orientation sensors⁴⁶ and lidars⁴⁷ for maximum effective area coverage. Based on the modeling of the camera, the global greedy search⁴⁸ or genetic algorithm⁴⁹ could be used to optimize the arrangement or number of cameras to cover the target range even if there are some obstacles. The economics of the sensor is also an issue to be considered. Lovisari⁵⁰ used a convex optimization approach to optimize sensors placement by balancing cost and performance.

MEC

The main feature of MECs is push of mobile computing, network control, and storage to the edge of the network so as to enable compute-intensive and latency-critical applications on resource-constrained mobile devices.⁵¹ The MECs can solve two problems, network delay and multi-terminal connection.^52,53 By offloading computing tasks from mobile devices to MEC with sufficient compute resources, network congestion and data transfer latency can be reduced effectively.

The efficient placement of MECs can effectively meet the needs of mobile users to access low-latency and high-bandwidth services. It is a multi-objective optimization problem with multi-constraints. The mixed-integer linear program (MILP)⁵⁴ and intelligent algorithms⁵⁵ were used to optimize the placement of MECs to minimize access latency between MECs and users. In addition to the access delay, the energy consumption of MEC is also an optimization goal. Li⁵⁶ used PSO algorithm to place the MECs to minimize energy consumption of MECs.

RSU

An RSU is a network transmitter that is statically placed along the road to facilitate communication between vehicles and infrastructures.⁵⁷ RSUs can be placed at intersections or along roads, receiving information and passing it on to other vehicles, RSUs, and cloud databases. This allows them to pass information between these systems, allowing information to be spread over longer distances.⁵⁸ The ability of V2I depends on the number of existing RSUs and radio coverage in the vicinity.⁵⁹

The placement of RSUs is a non-deterministic polynomial hard problem.⁶⁰ Too few RSUs will result in poor system performance. However, too many RSUs would be too expensive to install and maintain. Hwang⁶¹ and Lee⁶² optimized the placement of RSUs to reduce the quantity of RSUs with the constraints of covering vehicles at intersections. In addition to controlling the number of RSUs, there are also some studies that simultaneously optimize the number and location of RSUs by means of multi-objective optimization. Olia⁶³ optimized the optimal number and optimal location of RSUs by non-dominated sorting genetic algorithm-II and verified the optimization effect with a test platform.

Policy and maintenance of infrastructure

CVIS is a new field, and policies and regulations for its infrastructure are important.⁶⁴ In a survey of how organizations should choose to invest time or resources in CAV technology readiness, the largest set of answers related to legal, legislative, and regulatory issues and standards.⁶⁵ CVIS will bring issues of security, land use, ownership, data sharing, privacy, and regulation.⁶⁶ Therefore, policymakers need to consider coordination between different government agencies such as transportation, justice, economy, and energy. In addition, policymakers need to be concerned about the privacy of CVIS-related data. It is worth noting that policy development needs to be carried out with the consideration that it will be a state of coexistence of AVs and CAVs for some time to come and will therefore need to be continuously iterated.

CVIS requires the use of a large amount of roadside infrastructure, and the construction and maintenance of this infrastructure needs to be given a large amount of attention. Whether dedicated short-range communications (DSRC) or long-term evolution (LTE) technology is used, the infrastructure will have a high cost of construction and operation, which will affect the full deployment of CVIS.⁶⁷ In 2015, an organization called the Vehicle to Infrastructure Deployment Coalition (V2I DC) was formed in the United States to study the issues associated with V2I deployment during the implementation of CAVs on highways.⁶⁸ This organization summarized some lessons learned about infrastructure. For example, most of the scenarios where CVIS is applied should be related to intersections; equipment that contains many mechanical components should be built with infrastructure; and RSUs need to be careful to set reasonable warranty periods and use reasonable health monitoring techniques. It is possible to determine whether an infrastructure is failing or not by its maintenance schedule, specified lifetime, deterioration model, etc. Additionally, roadside equipment can fall from fixtures, which can lead to traffic accidents, and this needs to be taken into account during maintenance. Since the infrastructure is connected through communication, it can be subject to many attacks, such as unauthorized access or hardware tampering.⁶⁹

Introduction to CVs

CVs, as an integral component of CVIS, refer to vehicles with network connectivity. Vehicles transfer information between on-board units (OBUs) and roadside RSUs or OBUs of other CVs.⁷⁰ The OBU is usually installed on the roof of the vehicle. The main functions of the OBU include the reception and analysis of V2X messages, the reading and analysis of controller area network data, and information security protection. In fact, a large number of CVs in the current roadways are still at L0–L2, requiring manual driving. Although they are not capable of autonomous sensing and planning, they can receive other information through the OBU, such as real-time road traffic conditions and alerts of various hazards, thus improving safety. CVs with autonomous driving capability are referred to as CAVs. Compared to general AVs, CAVs have more advanced autonomous driving capabilities because of their ability to access information passed from infrastructure or other vehicles and to collaborate in sensing and planning. Cooperative sensing involves combining the information received by CAVs from their own sensors and the road sensors in order to enhance the scope and effectiveness of their sensing and produce the ultimate sensing results. In addition, collaborative planning means that CAVs combine the planning results of roadside infrastructure in the trajectory planning process to generate the final planning results. CAVs are seen as one of the most promising directions for vehicle development. As the number of CAVs in the roadway continues to increase, the role of CVIS will be further utilized.

Perception

In CVIS, the perception system needs to have good adaptability, robustness, and high availability to obtain real-time basic dynamic data. CVIS needs to obtain various information through different sensors installed on the vehicle and roadside. There are many limitations to perceive by a sensor at a single location, including occlusion, limited field of view, and low sensing density.⁷¹ With CVIS, vehicle and roadside sensors can observe locations near the same road from different perspectives.

However, the information obtained by different sensors cannot be used directly. It is necessary to convert the obtained data into unified coordinates and compare different data to form a unified structured data.⁷² According to different fusion targets, sensor information fusion can be divided into data-level fusion (such as point cloud and image), feature-level fusion (such as a bounding box from a vision-based object detector), and target-level fusion (such as different targets coordinate information). The existing perceptual fusion methods mainly use feature-level fusion and target-level fusion. Deep learning methods were used to fuse the feature information of multiple 3D lidars (vehicle and roadside) to improve the range and accuracy of perception.⁷³ Compared with object-level perceptual fusion, target-level perceptual fusion methods are simpler to use. Kitazato⁷⁴ sent vehicle information detected by roadside sensors to a sensor database that recognizes the same vehicle data sent from multiple sensors and integrates the data by matching vehicle data such as position, speed, and heading. According to Duan,⁷⁵ an environment perception framework was constructed to send objects detected by roadside multi-sensor systems back to AVs to enhance the perception capability of AVs at intersections. In the T-junction and roundabout scenarios, Arnold⁷¹ compared the two schemes of data-level perceptual fusion and feature-level perceptual fusion and found that the effect of data-level perceptual fusion was better. Existing fusion perception methods can significantly improve the accuracy of object detection and enhance the ability to track objects on the road. Most of them can only focus on object-level or feature-level perceptual fusion, and few focus on data-level perceptual fusion.

Communication

As shown in Figure 4, the current CVIS communication contains cellular V2X (C-V2X) technology and DSRC technology.⁷⁶ C-V2X technology mainly includes LTE-V2X developed from LTE communication technology and new radio V2X (NR-V2X) developed from 5G communication.⁷⁷ DSRC is a wireless communication technology designed for automotive applications that enables direct communication between vehicles and other road users without the need for cellular networks or other communication infrastructures.⁷⁸ C-V2X is an in-vehicle wireless communication technology based on the evolution of cellular network communication technology. Among them, LTE-V2X, which mainly carries basic traffic safety services, began to formulate standards from 2015 and released R14 version in 2017. NR-V2X, based on 5G NR technology, mainly for carrying autonomous driving services, was released in 3GPP R16 at the end of 2019.⁷⁹

Figure 4.

Communication technology of CVIS

With V2X technology, elements such as pedestrians, vehicles, roads, and cloud environments can be connected to enable V2V, V2I, etc. Using the commonly used V2V as an example, DSRC simplifies authentication, association, and data transmission before sending data, enabling vehicles to broadcast relevant safety information directly to neighboring vehicles. LTE-V2X includes cellular communication (Uu) and direct connection communication (PC5). Cellular communication is able to utilize the existing LTE cellular network to achieve V2V communication through forwarding. In addition, the PC5 mode is similar to DSRC and enables direct communication. The applications of V2X can mainly include safety, efficiency, and information service applications. The specific cases are introduced in detail according to different scenarios in the next chapter.

The comparison of various communication technologies is shown in Table 2. After comparing DSRC with LTE-V2X, we found that in terms of technology, LTE-V2X latency is significantly better than DSRC. As an emerging star, C-V2X has a comprehensive performance superior to DSRC in terms of communication range, capacity, vehicle movement speed, and anti-interference.⁸⁰ In terms of landing, the LTE-V2X can be upgraded through the existing LTE network base station equipment to achieve deployment, and DSRC needs to install new roadside equipment.

Table 2.

Differences between communication technologies of CVIS

	Standardization	Network	Latency	Reliability	Data rate	Effective distance
DSRC	IEEE 802.11p and IEEE1609	Access point	10 ms	Under 90%	6 Mbps	100 m
LTE-V2X	3GPP Rel-14/Rel-15	LTE	20 ms	90%, 95%	30 Mbps	300 m
NR-V2X	3GPP Rel-16/Rel-17	5G	3 ms	99.999%	100 Mbps	1000 m

Applications of CVIS in AVs

CVIS can provide AVs with more perception information and status information of surrounding traffic participants. The wide perception range and vehicles’ interaction based on CVIS provide a new possibility for eco-traffic. In the current research studies, CVIS can be applied to improve the safety, efficiency, and energy saving of AVs.

Safety

In different environments, CVIS enhances the safety performance of AVs, by assisting driving or warning the hazardous situation. The following classification is based on the vehicle’s behaviors corresponding to different environments, including straight road segments travel, intersection travel, on-ramp merging and adverse environment.

Straight road segments travel

Straight road segments travel is the most frequently driving condition when a vehicle is moving. As shown in Figure 5, we divided straight-line road travel into four situations, including straight driving, lane change, blind zones, and vulnerable road users.

Figure 5.

Scenarios about straight road segments travel

Some scholars studied the safety of straight driving based on CVIS. For example, Papadoulis⁸¹ used the time to collision (TTC) and post encroachment time parameters in surrogate safety assessment model to calculate the total number of conflicts and verified the positive impact of CAVs on road safety in a highway environment through simulation. The results show that traffic conflicts can be reduced by 12%–47%, 50%–80%, 82%–92%, and 90%–94% when CAV penetration reaches 25%, 50%, 75%, and 100%, respectively. Wen⁸² proposed a rear-end collision warning method for AVs based on a stochastic local multivehicle optimal speed car-following model using the characteristics of CVIS. Due to the existence of information transfer lag in data acquisition, transmission, and processing, Zhao⁸³ considered the inherent error of the system, information transfer delay, and GPS error, and it established an error-compensated safety distance model and rear-end collision warning system. The experimental results in the test field show that the correct warning rate of the system can reach 90% when the system error is 2 m. Moreover, it was confirmed that the maneuver coordination process including the maneuver coordination message of infrastructure could optimize TTC for safe driving.⁸⁴

Predicting lane-changing behavior can improve the safety of AVs. The information of ego vehicle and the vehicles of neighboring lanes can be obtained through V2X cooperation of CVIS. Djahel⁸⁵ collected V2X cooperative awareness messages to coordinate the driving intentions of the approaching vehicles to achieve behavioral safety such as lane change. Besides, considering the traffic congestion and information update cycles of RSUs and OBUs, an assistance application for lane selection was designed.⁸⁶ This study used data utilization from California’s highways for simulations, and the system was able to reduce travel time by 5%–7% compared to the HDVs, in moderate traffic flow conditions. Based on the driving performance, such as relative safety distances, the steering wheel turns, and angular velocities, Xu⁸⁷ adopted the fused Gaussian mixture model and continuous hidden Markov model to predict the behavior, reflecting the relationship between host and remote vehicles through CVIS. It can change lanes with an accuracy of 93.6%. This prediction is usually utilized to design a reference trajectory with a low collision risk.⁸⁸ Prathiba⁸⁹ proposed a collaborative collision avoidance scheme for CAVs during lane changing using CVIS and inverse reinforcement learning, which is able to avoid 87.23% of collisions.

Some of the main scenarios of blind zones are shown in Figure 6. Dynamic detection of blind zones for right-turning vehicles was investigated,⁹⁰ which used roadside lidar to obtain real-time pedestrian trajectory data of high resolution for warning of right-turning vehicles. Simulation results showed that when CVIS covered all vehicles, minor conflicts can be reduced by 80.00% and severe conflicts by 94.81%. In-vehicle sensors, such as radar, lidar and cameras were affected by weather conditions, darkness, and nighttime, which caused the presence of non-line-of-sight regions. To solve such problems, Baek⁹¹ fused the multi-sensor and wireless on-board communication and proposed the approach based on V2X. It could warn 2.5 s before a collision, compared to 1.5 s for vehicles without the system.

Figure 6.

Scenarios about blind zones

(A) The target is out of range of sensors, (B) perception is limited by orientation of the sensor, (C) perception is blocked by obstacles.

The safety of vulnerable road users (VRUs), which refers to pedestrians, cyclists, etc., is a huge challenge for CVIS. The important causes of VRUs accidents include blind spots, the psychological state of drivers’ distraction, and VRUs’ negligence. An end-to-end V2P framework was proposed to cover the VRUs, and then hazardous situations based on current vehicle dynamics were identified using a target classification algorithm.⁹² Nguyen⁹³ identified the activity of VRUs online based on V2P and machine learning, and the information was used to implement the collision detection algorithm. In addition, Ka⁹⁴ developed an intersection pedestrian collision warning system through the CVIS, using a machine learning model to predict pedestrian crossing intentions and then issue a collision warning to the driver. This system is capable of predicting unexpected pedestrian behavior within 1.5 s, which is an improvement of more than 6% compared to a warning system that only uses distance sensors. The early warning system used a wireless communication module to detect the relative speed and position of pedestrians and vehicles to create different warning scenarios.⁹⁵ From another perspective, acceptance of pedestrian warning systems had been studied and it can reach 60%.⁹⁶

Intersection travel

It is a tough challenge for the complex intersection to keep safety between the vehicles of different branches.⁹⁷ To coordinate the behaviors of conflict over the interactions, RSUs relay or signal a collision risk in CVIS if line of sight is obstructed at intersections, to analyze the safety of CAVs by longitudinal conflicts and driving volatility.⁹⁸ The time difference between ego vehicle and target vehicle arriving at the intersection and post encroachment time were taken as the main warning index.⁹⁹ Furthermore, corners of intersections are prone to be blind spots. In order to enhance the reliability of collision warning system, a time division multiple access scheme was designed without reducing the traffic throughput of the road network,¹⁰⁰ which reduced the conflicts of message in vehicular ad hoc networks of CVIS and the number of vehicle collisions at intersections effectively. To address the transmission delay, error, and redundancy of emergency warning messages (EWMs) in intersections, Zhao¹⁰¹ classified vehicles at intersections into different states and reduced EWM propagation through role-switching strategies. Aoki¹⁰² proposed a distributed synchronized intersection protocol and a collaborative perception-based HD map for mixed traffic environments to improve traffic throughput, where human drivers just need to drive normally and follow traffic lights. Shahriar¹⁰³ proposed a CVIS-based traffic safety framework for intersections in the past, and SUMO-based simulation tests showed that the method can reduce traffic accidents by 90%.

On-ramp merging

Traffic operation at a diverging bottleneck area nearby the on-ramp is an important component of highway management. It involves vehicles following mandatory and discretionary lane-change behaviors.¹⁰⁴ CVs with CVIS can obtain information about other vehicles before they reach the ramp, so they can make advance strategies, such as slowing down or accelerating, to ensure safety. On-ramp areas have serious merging conflicts, and the vehicle group collaboration behaviors of the main road and ramps in an on-ramp area are investigated to deal with the dynamic game between two streams of traffic flow.¹⁰⁵ Li¹⁰⁶ and Rios-Torres¹⁰⁷ estimated the TTC of trunk roads and ramps on pairs of vehicles based on their position, the direction of travel, and speed difference through RSUs. In a CVIS environment, Xie¹⁰⁸ developed an optimization-based ramp control strategy and a simulation evaluation platform for AVs.

Adverse environment

The adverse environments that AVs face mainly include harsh weather conditions and transportation. Inclement weather such as rain, snow, and fog usually results in blurred images, color degradation, and disruption of lidar point distribution. Radar denoising,¹⁰⁹ image enhancement,¹¹⁰ and multi-sensor fusion¹¹¹ are all methods to improve roadside sensor perception performance. Sensors mounted on pole-mounted RSUs have a wide viewing angle and are not susceptible to easily being affected by rain and snow. Compared with traditional AVs, AVs with CVIS can obtain a lot of sensory information from different perspectives and are more resistant to adverse environments.¹¹² Some unstructured scenarios are not supported by reliable maps, various obstacles are difficult to track accurately, and the driving space for vehicles is cluttered and narrow. Storck¹¹³ evaluated the use of CVIS in a rural roadway environment, and the results showed that the rate of transmission of information was able to meet the requirements. Gao¹¹⁴ proposed a CVIS-based autonomous mining scheme that ensures the safety of vehicles in harsh environments through the sharing of sensing results among multiple CAVs. CVIS was used in the unloading area of an open pit mine and was able to control multiple vehicles working at the same time, increasing unloading efficiency by 24%.¹¹⁵

Efficiency

On the premise of safety, traffic participants would further expect the traffic experience to be comfortable and efficient, and traffic administrators require a smooth eco-traffic. This section reviews the strategies of CVIS for efficiency improvement in different traffic scenarios: parking, intersection, straight road segment, and on-ramp, as shown in Figure 7. Meanwhile, Table 3 shows the methods of different scenarios for efficiency of CVIS.

Figure 7.

The function of CVIS in different scenarios for efficiency

Table 3.

The methods for efficiency of AV with CVIS

Scenario		Method
Road-segment travel	Vehicle platooning control Priority control of special vehicle	Longitude control MPC116,117 Optimization method118,119,120
Intersection travel	Signal timing	Optimization method121,122,123,124,125,126,127 Reinforcement learning128,129,130
	Signal-based speed guidance	Dynamic program131
		Optimal control132,133,134,135,136,137,138
		Graph search methods139,140
	Signal timing and trajectory optimization	Sequential optimization141,142,143 Joint optimization144,145,146,147
	Signal-free strategy	FCFS policy148 Optimization method149,150,151,152,153,154,155,156
On-Ramp merging		Decentralized optimization157 Centralized optimization158 Reinforcement learning159,160
Parking	Parking-space allocation Parking planning	Optimization method161,162,163 Graph search method164 Optimization method165,166

Straight road segments travel

Based on CVIS, the platooning control for normal vehicles and the priority control for special vehicles can be reached to improve the efficiency of straight road segments travel. Vehicle platooning is a cooperative driving technology based on CVIS where CAVs drive in a string with a close gap.¹⁶⁷ The short inter-vehicle gap can obviously increase the traffic capacity and reduce the aerodynamic drag at the platoon center. The vehicle platooning control mainly includes longitude and lateral control. The former focuses on the control of vehicle speed in the platoon based on the information communication with CVIS, where the issue of string stability has received great attention. The popular method is model predictive control (MPC). Meanwhile, this feedforward control is able to consider the information at a tactical level (e.g., traffic light) for eco-driving.¹¹⁶ Besides, the early research usually assumed platoon vehicles are homogenous to simplify the problem. However, it also inevitably brings error, which should be further explored.¹⁶⁸ In addition, robustness is necessary for suppress widespread disturbances, but its uncertainty always causes problems. Feng¹¹⁷ adopted feedback control for small yet frequent disturbances and feedforward control for large yet infrequent one. While the platoon operation usually requires the cooperation of longitudinal and lateral control based on CVIS, i.e., joining, merging, leaving, and splitting platoon.

The priority control of the special vehicle is essential for traffic management, especially of the emergency vehicle (e.g., police cars, fire trucks, and ambulances), as shown in Figure 7. CVIS can guide special vehicles in real time based on the comprehensive perception of traffic states. Yao¹¹⁹ designed a two-stage planning-oriented signal coordination control model for emergency vehicle paths, which led to a 14% reduction in the average delay time at intersections. With regard to the emergency vehicle, efficiency is more important than disturbance in priority control.¹¹⁸ The lane-clearing strategy is essential for this problem, which needs multi-vehicle cooperation based on CVIS to clear the way of the emergency vehicle. Wu¹³⁴ proposed a merging-trajectory algorithm for the CVs and formulated the clearing strategy as a mixed-integer non-linear program problem.

Intersection travel

The intersection leads to loss of efficiency. As shown in Figure 7, the management of conflict zone is one of the critical parts of CVIS to keep traffic smooth.¹⁶⁹ The traditional control methods of those are short-sighted, while CVIS can broaden the perception range and improve the performance of intersections. In detail, the signal control and trajectory guidance based on CVIS can form a green wave band for vehicles through the intersection without stopping or deceleration. Figure 8 shows the intersection control problems in different intelligent levels.

Figure 8.

Signal timing and trajectory planning

Signal timing is a traffic-control approach of roadside by scheduling the signal phase sequence and duration based on CVIS. The traditional timing methods aim to ensure a fixed time sequence based on historical data,¹⁷⁰ which is difficult to suit for dynamic traffic. On the contrary, CVIS can adjust timing dynamically based on real-time traffic data. Signal timing is always modeled as an optimization problem to minimize traffic delays. The complexity of optimization increases rapidly with the increase of the number of lanes. Therefore, reinforcement learning (RL) methods with strong adaptability have received extensive attention under the CVIS. Wei¹²⁸ regarded all the signal lights as an agent to learn, easily coordinating each signal light. However, such methods are computationally complex due to its centralization. To overcome the issue, Chu¹³⁰ splitted the global agent into independent agents for training and dispersed the computational by increasing the number of agents. Meanwhile, it also brings the issue of local optimal such that how to model the interaction of different agents becomes the key to multi-agent RL.¹²⁹ The priority control of buses aims to alleviate delays based on bus lanes and traffic signal priority control (TSP). The TSP requires the use of CVIS, and its strategies can be divided into passive and active priority.¹³⁴ The former uses historical traffic data to arrange the offline signal timing and cannot respond to dynamic traffic. Therefore, the active priority has received great attention, which is able to provide a real-time guide. The key to priority control for the bus is the trade-off of bus efficiency and the disturbances on normal traffic.^135,136,137

Each vehicle in front of intersection should plan the eco-trajectory based on the signal state of multiple intersections from CVIS. Since the planning here focuses on the longitudinal speed, it is called as speed guidance or trajectory optimization. It is difficult to find an efficient solution due to their complex objective functions and constraints in trajectory planning. The popular method is sampling with the dynamic program (DP) in discrete state space.¹³¹ In addition, some studies simplify the speed patterns through empirical rules, which can improve 10% relative to HDVs.¹³³ It is always considered as a two-stage (i.e., accelerating and decelerating) or a three-stage solution with an additional cruising stage.¹³²

Signal timing and trajectory optimization actually affect each other, and their essence is a two-dimensional optimization. In the context of CVIS, the two problems can be collaboratively optimized to further improve the traffic efficiency.¹⁴¹ In isolated intersection with full CAVs, Feng¹⁴¹ optimized the signal timing by DP and used optimal control to plan trajectory which could reduce vehicle delays by 24%. Compared to the scenario with full CAVs, mixed traffic scenario is more complicated because HDVs cannot directly be controlled by CVIS. Guo¹⁴² extended the research¹⁴¹ in mixed traffic and solved the DP by a shooting heuristic algorithm. It reduces the average travel time by 35.72% compared to adaptive signaling control.

Optimization in stages can simplify the optimization and improve the real-time performance, but inevitably sub-optimal. The MILP is the most common formulation of the joint problems. The research of joint problem focused on a trade-off between accuracy and computation cost by formulation,¹⁴⁴ constraint simplification,¹⁴⁷ restricted solution form,¹⁴⁶ etc. For example, Tajalli¹⁴⁴ decomposed the initial problem into several lane-level problems and linearized the nonlinear constraints. It was able to reduce travel time by 13%–41% in different scenarios compared to the traditional method of fixed signal time. Niroumand¹⁴⁵ introduced a white phase to enforce HDVs in mixed traffic to follow their front vehicle to solve the optimization. The proposed procedure reduced the total delay by 19.6%–96.2% compared to the fully driven signal control optimized by the actual traffic signal timing optimization software.

CVIS presents an opportunity for a signal-free strategy, which is regarded as a potential method to further reduce traffic delays. The classic method for centralized control is a rule-based model for permits distribution, of which the most popular is the first-come-first-served (FCFS) policy.¹⁴⁸ Such methods are simple to realize, but it usually leads to a sub-optimal result. Yu¹⁵⁴ adopted an MILP model to optimize the trajectories of CAVs and additionally considered microscopic vehicle behavior (i.e., car-following and lane-changing). Jiang¹⁵⁵ and Yao¹⁵⁶ proposed a two-stage method to manage the oncoming vehicles, including timing schedule optimization and trajectory optimization. However, the large number of vehicles and their dynamic behavior lead to great difficulty in solving results. Compared to the FCFS approach, this approach reduced vehicle delays by 89.48% for varying traffic demands. Ge¹⁵³ adopted the oriented graph to describe relative priorities between vehicles and limit the scale of the optimization. On the contrary, some researchers replace centralized control with the decentralized one, aiming to improve the solving efficiency and adaptability for different scenarios. Mirheli¹⁵² proposed a distributed cooperative optimization model to obtain each vehicle trajectory where a coordination scheme was also designed to share vehicle state. This method was able to reduce travel time by 43.0%–70.5% compared to conventional methods of signal optimization. Moreover, centralized control might not look after the interest of every vehicle due to its objective of global optimization. Thus, Wang¹⁵¹ modeled the interaction between vehicles as a competing process by game theory.

On-ramp merging

CVIS can help the mainline traffic and on-ramp merging vehicles know the information and intentions of each other and guide them to merge efficiently.¹⁷¹ A simple method used to improve the merging is traffic management (e.g., ramp metering) based on CVIS, which uses traffic signals to control the rate of the vehicle entering the mainline traffic at the macro level. To further exploit the efficiency potential of merging strategy, the optimization methods become popular, including centralized optimization and decentralized one.¹⁵⁷ For example, Zhou¹⁷² formulated the trajectory planning of the mainline vehicle and on-ramp merging vehicle as two related optimal control problems and implemented a recursive framework to accommodate the dynamic environment. However, those centralized methods are hard to consider the uncertainty of HDVs. Liu¹⁵⁹ proposed a lane selection model by RL for on-ramp vehicles to alleviate the congestion of the outside lane and completed the trajectory planning based on optimal control. Compared to the common optimal control method, it can improve the traffic efficiency by 41.2%. Kherroubi¹⁶⁰ additionally adopted an artificial neural network to predict the intentions of drivers and improved the performance of RL.

Parking

Because the vehicle is parked for 94%¹⁷³ on average of its lifetime, parking is an important part for the AVs. Furthermore, parking is one of the biggest consumers of urban land.¹⁷⁴ CVIS can coordinate parking vehicles and parking facilities. For one thing, CVIS is able to guide drivers to find the right parking space quickly and avoid unnecessary energy consumption, congestion, and driver’s worries.¹⁷⁵ Meanwhile, the autonomous valet parking based on CVIS can reduce accidents and improve the driver’s experience obviously. For another, CVIS-based parking arrangement is able to enhance the parking capacity and efficiency.¹⁷⁶

The roadside of CVIS provides the real-time information of parking-space state for drivers and optimizes the parking-space allocation based on the time cost of drivers through the optimal pricing policy.^161,163 The local planner includes sampling-and-search and optimization methods. The former can obtain an available trajectory quickly through discrete solution space, and the latter is able to reach an optimization trajectory.¹⁶⁵ Simulations showed that it was capable of achieving a 100% success rate. In addition, a CAV can adjust its position with the order of roadside and facilitate vehicle movement in or out of parking space,¹⁶⁶ which increases the capacity of the parking lot by 25%.

Energy saving

Transport is fundamental to society and energy is fundamental to transport.¹⁷⁷ Although AVs are mostly electric vehicles, they still need some smart control methods to improve energy efficiency.¹⁷⁸ CVIS can not only improve the safety and efficiency of AVs, but also help AVs save energy, thereby increasing the driving distance of AVs.¹⁷⁹ In Section 3.2, we have introduced the related research on CVIS to improve the efficiency of AVs. In fact, sometimes driving more efficiently can reduce the amount of time spent driving, and thus reduce energy consumption. Compared to efficiency, in the current research, there is less research on how CVIS can help AVs save energy. As shown in Figure 9, this section describes how CVIS can be applied to help AVs save energy, including vehicle formation and multi-vehicle cooperative driving. The former is mainly used more in commercial vehicles, while the latter is used in passenger vehicles.

Figure 9.

Application of CVIS in the energy saving of AVs

Vehicle platooning control

With CVIS, the gap between AVs can be safely closed and the aerodynamic drag coefficient is smaller at shorter following distances, resulting in significant energy savings, especially for commercial vehicles at high speeds. Because the vehicle speed is high under highway conditions, air resistance is an important component of the total resistance suffered by the whole vehicle, which can reach half or even more. In this case, the following vehicle enters the wake area under the barrier effect of the lead vehicle, reducing the air resistance at high speed of the queue, which in turn can reduce energy consumption.¹⁸⁰

One approach often used for platoon control of AVs is to treat the platoon as a whole and obtain the appropriate speed of the platoon. An MPC-based approach is firstly proposed to obtain the optimal speed of the whole platoon of CAVs, and fuel consumption reduction and transportation efficiency improvement are considered in the optimization process, then, a distributed adaptive three-step nonlinear control strategy is investigated to ensure the stability of the whole platoon.¹⁸¹ Another method is to determine the trajectory of the front-most vehicle first, and other vehicles follow the front vehicle. Yang¹⁸² developed an ecological cooperative adaptive cruise control (ECACC) strategy that used DP-based ecological speed trajectory planning for the leading vehicle, while using combined feedforward feedback control for vehicle tracking for the remaining vehicles, reducing the energy consumption of the AVs platoon. In this strategy, the AVs adjust their own speed by acquiring information about the vehicle in front through V2V. The proposed ECACC can reduce energy consumption by up to 38.1% in the energy optimal mode compared to the conventional energy-optimal cruise control system. The aforementioned studies are all for the same type of AVs, but different types of AVs can also form platoons. A new distributed economic MPC algorithm was proposed to reconcile the conflicting tracking, safety, stability, and fuel economy control objectives of heterogeneous vehicle platoons.¹⁸³ This approach enables vehicle energy savings of 4.2% compared to conventional platooning control.

Multi-vehicle cooperative driving

Multi-vehicle cooperative driving refers to treating the vehicles on the road as a flexible formation and realizing information sharing among multiple vehicles through CVIS. In CVIS environment, the deliberate exchanges of intentions between the ego and other vehicles reduce the need to guess the traffic patterns around them, thus enabling a better coordination of activities. The energy consumption of AVs is significantly reduced as CVIS changes the driving behavior of AVs, reducing frequent speed fluctuations and eliminating unnecessary stops. Multi-vehicle cooperative driving can be applied in lane changing, intersection, and ramp scenarios.

Changing lanes is a maneuver that AVs are often prone to when driving in a straight driving. The occurrence of a lane change often interrupts the previous following state and thus affects the change in energy consumed. Awal¹⁸⁴ proposed an efficient collaborative lane-changing algorithm for AVs with CVIS, while minimizing the total travel time, fuel consumption, and pollutant emissions during lane changing. Huang¹⁸⁵ proposed a neural network-based algorithm to obtain the position and speed information of the vehicle in front through CVIS, which can reduce fuel consumption by 15%. The future speed of the local vehicle is predicted by a neural network and the future energy consumption of each lane is output by another neural network. Kamal¹⁸⁶ presented an efficient driving system based on MPC framework for surrounding vehicle prediction, which acquired vehicle information and predicts their future behavior via V2V, generating optimal acceleration and simultaneously making lane-change decisions. Experiments have shown that the system improves fuel economy by 1.5 km/L compared to conventional adaptive cruise control.

AVs often need to slow down or even stop frequently when passing through junctions, resulting in unnecessary energy consumption. Bai¹⁸⁷ proposed an HRL framework that integrated rules-based policy and deep learning, and also used V2I to collect information about intersection signals, to support the connected eco-driving at intersections with mixed traffic signals. Experiments show that this method can reduce energy consumption by 12.70% compared to the state-of-the-art model-based eco-driving method. However, this approach optimizes the energy savings of a single AV as it passes through a junction, without considering the impact of other AVs. An optimization-based centralized intersection controller is proposed to find the optimal trajectory of CAVs through a signal-free intersection for each vehicle, capable of reducing energy consumption by 50%.¹⁸⁸ Yao¹⁸⁹ proposed a two-stage optimization method for CAVs scheduling and trajectory planning that can reduce vehicle fuel consumption by up to 34.36% compared to the FCFS method.

In the common entrance ramp scenario, the vehicle density in the outer lanes of the mainline usually increases sharply due to the inflow of ramp traffic. This stop-and-go driving can cause excessive energy consumption. Rios-Torres¹⁹⁰ defined the optimal coordination problem of CAVs on merging roads as an unconstrained optimal control problem and applied Hamiltonian analysis to derive an analytic closed-form solution to achieve smooth traffic flow. The 30-vehicle simulation demonstrated a 50% reduction in overall fuel consumption. However, this study focused on merging sequence optimization in single main-lane scenarios and failed to fully utilize the capacity of multi-lane roads. CVIS was used to obtain the average lane density and speed, and the central infrastructure controller used a vehicle motion planning algorithm based on time-energy optimal control to make all vehicles follow the optimal trajectory.¹⁹¹ With high entrance traffic flow and high inhomogeneity, this method can improve the fuel economy by 43.5% compared to the common optimal control method. Liu¹⁹² proposed a hierarchical environmental cooperative ramp management system for AVs with CVIS that could coordinate all ramp inflow rates along the corridor based on real-time traffic conditions, while an MPC-based algorithm was designed for detailed speed control of individual CAVs. It can reduce energy consumption by 35.1% (for gasoline-powered vehicles) and 24% (for electric vehicles) compared to conventional ramp metering.

Datasets and simulators

Suitable datasets are crucial for training models, and with some methods of deep and reinforcement learning being applied to cooperative perception, cooperative planning, etc. in CVIS. In addition, in order to verify the generalization and validity of various models and methods, a large number of experiments need to be conducted to validate them. Physical testing of CVIS using actual vehicles and roads requires a lot of equipment and expense, and currently a variety of simulators are mainly utilized to validate some algorithms in a virtual environment. In this section, we summarize some of the datasets and simulators used in research related to CVIS for AVs.

Datasets

Most datasets are currently used primarily for AVs, and datasets used for CVIS are in short supply. Our summary of some of the datasets used in existing CVIS-related studies is shown in Table 4. These datasets can be categorized into simulator-based datasets and real-scenario-based datasets. Simulator-based dataset data are generated in a simulator. Some of the commonly used simulators for generating data include SUMO, CARLA, OpenCDA, etc. However, there are some gaps between simulator data and real-world models. Real-scenario-based datasets can demonstrate more complex traffic behavior and noisy data and can enhance the generalization performance of training algorithms, such as DAIR-V2X, the first large-scale real-scenario-based dataset for V2I.

Table 4.

A summary of existing datasets for CVIS

Dataset	Source	Year	Sensors	Frames	Data scenarios	Application scenarios	Tasks	Link
V2X-Sim Li et al.193	CARLA & SUMO	2021	Cameras, Lidars	10,000	Urban roads	V2I, V2V	Perception	https://ai4ce.github.io/V2X-Sim/
COMAP Yuan et al.194	CARLA & SUMO	2021	Lidars	7,788	Urban roads	V2V	Perception	https://github.com/YuanYunshuang/FPVRCNN
V2XSet Xu et al.195	CARLA & OpenCDA	2022	Cameras, Lidars	11,447	Urban roads	V2I, V2V	Perception	https://github.com/DerrickXuNu/v2xvit
DOLPHINS Mao et al.196	CARLA	2022	Cameras, Lidars	42,376	Urban roads, mountain roads	V2I, V2V	Perception	https://dolphins-dataset.net/
OPV2V Xu et al.197	CARLA & OpenCDA	2022	Lidars	11,464	Urban roads, rural roads	V2V	Perception	https://mobilitylab.seas.ucla.edu/opv2v/
Deep Accident Wang et al.198	CARLA	2023	Cameras, Lidars	57,000	Urban roads	V2I, V2V	Perception, Motion and Accident Prediction	https://deepaccident.github.io/
DAIR-V2X Yu et al.199	Real-world	2022	Cameras, Lidars	38,845	Urban roads	V2I	Perception, Trajectory Prediction, Decision Making and Planning	https://thudair.baai.ac.cn/index
V2v4real Xu et al.200	Real-world	2023	Cameras, Lidars	20,000	Urban roads, highway roads	V2V	Perception	https://mobility-lab.seas.ucla.edu/v2v4real/

Simulators

CVIS-related simulations can be divided into traffic scenario simulations and numerical analysis simulations. Traffic scenario simulation is to simulate real-world traffic scenarios using existing traffic simulators or building your own simulators, generating a large amount of data for training and testing under various scenarios and conditions. SUMO and CARLA are commonly used traffic simulators. SUMO is able to simulate the traffic environment very well, and it is often combined with other simulators, such as the NS3²⁰¹ simulator or CARLA,²⁰² when performing V2X-related simulations. CARLA has many built-in sensors and rich API interfaces and is able to perform V2I- or V2V-related planning and control simulations, but its traffic management does not fully take into account the complexity of the driver.²⁰³ OpenCDA supports the joint simulation of CARLA and SUMO with test scenarios and basic algorithms related to autonomous driving, and is capable of performing simple CVIS-related simulations.²⁰⁴ In addition, PTV-VISSIM²⁰⁵ and AirSim²⁰⁶ are often used for CVIS-related simulations. Some studies also often build their own various simulation environments, such as MATLAB.²⁰⁷ Furthermore, numerical analysis simulations are mostly processed for some datasets to evaluate the algorithms related to CVIS. This approach is generally used in learning-based algorithms. For example, a deep learning approach is used to solve the multi-vehicle planning problem under CVIS.²⁰⁸

Challenges and future directions

With the continuous breakthrough and integration of artificial intelligence, autonomous driving, 5G communication, and other technologies, CVIS will continue to move toward a higher level. Although the CVIS has promising application prospects, its development still faces many challenges. This paper lists some worthy challenges and research directions for researchers to further explore in Figure 10.

Figure 10.

Challenges and future directions of research

Architecture and components of CVIS

The overall architecture and infrastructure of CVIS used are well established. Current research on the architecture of CVIS is mainly in the area of communication. However, latency, infrastructure optimization, and rational utilization of computational resources still deserve further research.

Communication of CVIS

V2X technology is widely used in CVIS. Information redundancy, errors, and transmission delays can cause communication problems. Although great bandwidth and high transmission speed can be provided through the use of NR-V2X technology, long vehicle distances and high traffic volumes place high demands on communication. There are some ways to reduce the delay. It is possible to study and construct a reasonable communication resource allocation system to control the access quantity and distance of each communication point. In addition, it is also very important to unify and improve the communication protocol between different communication devices, thereby improving the efficiency of information transmission.

In addition, in the communication process, there will be many security risks. In CVIS, various sensors acquire a large amount of data, including not only the trajectories of traffic participants and traffic events, but also some sensitive geographic information. The system should be able to resist corresponding cyber attacks, including illegal access and tampering with data. What deserves special attention is the security of identity information. The vehicle side, roadside, cloud side, etc. all have their own identity information and need to prevent identity forgery and control hijacking. It should be considered how to ensure the security of the information through some efficient security protocols, secret keys, and intrusion detection systems.

Infrastructure optimization

A large amount of roadside infrastructures is used in CVIS, and the selection, placement, and invocation of various devices need to be considered in order to construct the optimal solution. However, when there are too many constraints and solution objectives, it is difficult to obtain an objective solution to this problem. The balance between cost and efficiency needs to be considered comprehensively. In addition, the scheduling of sensor resources is also a challenge. Most of the current infrastructure arrangements are based on empirical methods. The optimization object of some infrastructure optimization research is relatively single. Therefore, it is worth exploring to study the interaction between different sensors to synergistically optimize different infrastructures. It is also possible to study how the infrastructure behaves in 3D space to get reliable optimization results, rather than optimizing it on the plane.

Computing resources

The calculation of the CVIS involves in-vehicle computing, roadside edge computing, and cloud computing. Among them, in terms of calculation amount and real-time performance, roadside computing has the highest requirements. In order to utilize computing resources efficiently, it is also very significant to study the adaptive dynamic allocation of computing resources at different levels and at the same level. In view of this situation, the calculation delay is reduced by using some learning algorithms or parallel computing.

Applications of CVIS in AVs

CVIS and AVs are very closely linked, and further collaborative research between CVIS and AVs can be carried out mainly through the following aspects.

Decision of AVs with CVIS

At present, in the field of CVIS, the decision-making research on AV mainly focuses on the vehicles with CVIS function. With the development of CVIS, there will be more and more vehicles with CVIS functionalities on the road. However, there will still be some vehicles without CVIS functionality for quite some time. It is necessary to consider the interaction between the two types of vehicles to construct a vehicle decision and control model under mixed traffic flow. It is also worth investigating how vehicles with and without CVIS interact when driving together, and how this affects decision-making and planning of AVs. For example, when considering the signal-based speed guidance, if the vehicle is in the green wave, the trajectory of the vehicle is planned to take into account the influence of surrounding vehicles and the possible collision risk. Currently, CVIS cannot directly obtain relevant information from VRUs, although CVIS can obtain the trajectory information of VRUs through more sensing devices. However, there is great uncertainty in the movement of VRUs, and corresponding avoidance mechanisms need to be considered. In addition, trajectory prediction based on CVIS can be studied to explore whether the advantages of CVIS can be used to improve the accuracy and time length of trajectory prediction of AVs and other traffic participants. Each AV is an intelligence that can be linked together to form a multi-intelligence system through CVIS. Current cooperation is mainly focused on handling the same task in a single scenario. There is great potential for research on how to motivate vehicles to cooperate when they are in different tasks.

Scenario generation and testing

There are currently few methods on CVIS tests and standards, and relevant datasets are also lacking. However, the algorithm training and testing of CVIS relies on a sufficient number of datasets. In order to solve the problem, at present, some datasets based on roadside sensors or unmanned aerial vehicles have been established. However, the datasets collected by single roadside sensors cannot well simulate the interaction between the vehicles and the road infrastructures. In addition, autonomous driving based on deep learning and reinforcement learning is a current trend in research related to AVs, as rule-based autonomous driving cannot cover all scenarios. Therefore, the scene generation technology can be applied to the virtual simulation test of CVIS to generate a large number of test scenes for related training and testing. However, there is a need to rationalize the introduction of ambient noise in the data to reduce the variability between the generated data and the real data.

Performance optimization

Through previous studies, we can find that most of the studies on CVIS being applied to AVs are for a single performance of AVs, such as security or efficiency. There are fewer studies related to the energy savings of CVIS that help AVs, especially the lack of quantification of the energy savings. Sometimes, CVIS helps AVs improve efficiency to increase energy savings, and sometimes the opposite. Therefore, when applying CVIS to AVs, it is worthwhile to study the balance between efficiency and energy savings while ensuring safety and quantifying the various performance impacts. Furthermore, ChatGPT demonstrates the impressive language comprehension capabilities of large language models (LLMs). CVIS generates large and multimodal data, and if we can transform these data into linguistic information in LLMs, we can effectively improve the efficiency of data processing. Therefore, how to use LLMs with a cross-modal encoder to process traffic data with different modes is a worthy research problem. In addition, suitable training data need to be prepared for the LLM on CVIS to improve the accuracy of the LLM.

Conclusion

This paper provides a comprehensive review of the architecture and components, applications in AVs, the datasets and simulators, and challenges and future directions of CVIS. The architecture and components include the overall architecture, introduction to infrastructure, CVs, and communication technology. CVIS plays a crucial role in improving the performance of AVs according to current research. This paper concludes that CVIS improves the safety, efficiency, and energy saving of AVs according to different scenarios of AVs, such as ramps, intersections, etc. However, the collaboration between CVIS and AVs still needs to be further explored to meet complex traffic scenarios. In recent years, there have been many studies about CVIS, but some key points have been ignored by researchers or have not achieved ideal results. Therefore, we outline the challenges facing CVIS and propose promising research directions. CVIS is an effective method to improve the high-level autonomous driving of AVs, and it is also a hotspot of intelligent transport system. How to improve the current technical level and application effect of CVIS has always been a concern of researchers. We hope this paper will help researchers gain a comprehensive understanding of the current applications and challenges of CVIS and advance the development of AVs and ITS technologies.