Research on multimodal data enhanced SLAM algorithm for global mapping of underground coal mines

Abstract

To resolve the issue of inaccurate global mapping in coal mines, which impedes reliable support for autonomous driving, the paper proposes an enhanced SLAM global mapping method for coal mines based on multimodal data, which consists of LiDAR-inertial-wheel odometry frontend and a multi-factor motivated optimization backend, to realize the pose estimation and map establishment for global mapping of underground roadways. The frontend fuses inertial measurements from the M-SINS, which are compensated for the Earth rotation, with wheel encoder data and LiDAR points using an iterated error-state Kalman filter, mitigating the LiDAR degeneration. The backend utilizes a series of identified spherical targets, sparsely deployed in the tunnel, to introduce global position constrains and unambiguous loop closure detection under a multi-factor pose graph of length distance. Meanwhile, Since the algorithm operates within a globally consistent navigation coordinate system, point-based corrections using GNSS/UWB signals or spherical targets can enhance the accuracy of the global map. The proposed method is evaluated and compared to state-of-the-art approaches through simulation tests in a virtual subterranean roadway and field experiments at an active underground coal mine using our own robot platform. These evaluations demonstrate the method’s performance and practicability in large-scale underground coal mine environments.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-025-21053-y.

Introduction

With the rapid advancements of coal mining technology in recent years, autonomous driving technology is increasingly being applied to coal mine production. This application aims to enhance transportation efficiency, reduce labor costs, and improve operational safety. In some open-pit coal mines, autonomous driving transport fleets have already been implemented in actual production, yielding significant economic benefits and substantial labor savings. However, in underground coal mine scenarios, the application of autonomous driving technology to actual transportation remains rare due to the challenging underground environment.

One of the primary challenges facing autonomous driving in underground coal mines is achieving reliable positioning and navigation of vehicles in subterranean tunnels. Typically, autonomous driving vehicles are equipped with a variety of sensors, such as LiDAR, camera, inertial measurement unit (IMU), and global navigation satellite system (GNSS), which provide complementary sensing capacities for local and global motion estimation. However, underground vehicles usually lack access to absolute positioning sources due to the GNSS-denied environment. Additionally, state estimation of automated vehicles in underground coalmine roadways is often hindered by the self-symmetrical structure with homogeneous and untextured surfaces. Consequently, reliable localization and navigation in large-scale underground coal mines present significant challenges.

Simultaneous Localization and Mapping (SLAM) has been widely used in the domestic robotics, self-driving cars and autonomous exploration to obtain six degree-of-freedom state estimation and environments mapping^1,2. With the growing demand for positioning and navigation in underground coal mine, there has been increasing attention towards underground SLAM, which is a key enabler for navigation in GNSS-denied environments. While building a map is not essential for the autonomous driving, mapping remains a crucial prerequisite for successful underground operation³. In the absence of global localization signals, the underground roadway map built by SLAM can provide a-priori global information. The transportation roadways of underground coal mines are usually long and continue to extend with mining production. An example of narrow roadways of an active real-world underground coal mine is shown in Fig. 1. Accurate a-priori global maps and suitable localization methods are required to enable autonomous driving vehicles to navigate through the tunnel-like environment with degenerate planar features.

Fig. 1.

Roadway map of an active underground coalmine created by the proposed method. The color bar indicates the depth of underground roadways with respect to the ground square. A: Inclined roadway of auxiliary transport; B: Refuge chamber of the inclined roadway; C: Intersection of the inclined roadway and the parallel roadway; D: Spherical target installed in the underground roadway; E: Sensors and platform used in our research.

However, traditional SLAM solutions often exhibit inadequate performance and failure when applied to large-scale and dynamically changing subterranean roadways. Poorly illuminated conditions significantly degrade camera data, rendering visual-based methods unreliable for motion estimation⁴. The structure-less environment (long-straight tunnel) and the presence of obscurants (dust, fog and smoke) cause failures in LiDAR-based approaches that depend on feature or scan matching^5,6. Additionally, the complex road conditions and slippery terrains of subterranean roadways introduce noise to inertial sensors and wheel odometers⁷. Moreover, autonomous vehicles are required to transport goods or personnel from the coalmine surface to the underground roadways. It is essential to establish a globally consistent navigation coordinate system that covers both surface and underground workspace. For surface scenario, global position information from GPS can be used as an optimization factor to estimate vehicle pose in the global coordinate system, as demonstrated by some SLAM methods, like LIO-SAM⁸. However, in underground scenarios, the absence of reliable global position signals means that pose estimation can only be obtained in the local reference frame. Ensuring the consistency of pose estimation across both environments is a critical issue that needs to be addressed.

Although significant progress has been made in SLAM technologies for tunnels and subways², coal mine roadways exhibit unique characteristics that distinguish them from these environments. In addition to the features mentioned above, challenges such as long slopes, frequent sharp-angled turns, dense equipment causing complex electromagnetic interference, susceptibility of wireless positioning failures, and explosion-proof requirements for sensors collectively pose substantial difficulties for SLAM-based mapping and localization in underground coal mines.

In this paper, a multimodal data enhanced SLAM system is proposed to address the problem of state estimation and roadway mapping in large-scale underground coal mine. Specifically, the framework of our approach contains a filter-based frontend and an optimization-based backend. The frontend adopts a tightly-coupled iterated error-state Kalman filter (iESKF) to fuse LiDAR points, inertial measurements and wheel odometer readings, mitigating the problem of LiDAR degeneration in featureless subterranean roadway environment.

Unlike other approaches that use micro-electro-mechanical systems (MEMS)-based IMUs, our method introduces inertial measurements from the mining strapdown inertial navigation system (M-SINS) to obtain vehicle motion estimation. The M-SINS integrates a high-precision fiber optic gyroscope (FOG) instead of a MEMS-based IMU. This allows the M-SINS to accurately sense the acceleration due to gravity and the angular velocity of the Earth rotation in underground coal mines. Consequently, it utilizes accelerometer leveling and gyro compassing to calculate the carrier’s attitude within the geographic coordinate system. Leveraging the advantages of M-SINS, our method employs a north-referenced navigation coordinate system, which is consistent with the geographic coordinate system used by M-SINS. This ensures the consistency of the navigation coordinate system across both surface and underground scenes in coal mines. Additionally, by compensating for the Earth rotation in a tightly fused LiDAR-inertial-wheel odometry, we enhance the system’s accuracy and robustness.

Nevertheless, without global position constraints, the LiDAR-inertial-wheel odometer inevitably accumulates errors that increase with the traversed distance. Loop closure detection is an effective way to mitigate the accumulation of global drift. However, in perceptually degraded subterranean roadways, robust detection of loop closures presents a significant challenge⁹. To address this, our framework’s backend combines artificial and geometric loop closure detection methods by utilizing sparsely deployed spherical targets embedded with ultra-wideband (UWB) tags. These tags store the position information and unique code name of spherical targets. Through point cloud registration, the system can recognize the pre-installed spherical targets at intersections and turning points in underground roadways. The UWB module installed on the approaching vehicle can then retrieve the position and code number information from the tags, triggering global position constraints and facilitating unambiguous loop closure detection.

The main contributions of this work can be summarized as follows:

• We present a multimodal data enhanced SLAM system designed specifically for large-scale autonomous navigation in underground coal mines. The system consists of a tightly coupled LiDAR-inertial-wheel odometry frontend and a multi-factor graph optimization backend, offering enhanced robustness in perceptually-degraded mining tunnels.

• Earth-rotation compensation is incorporated through a mining-grade strapdown inertial navigation system (M-SINS), improving attitude estimation under long-term operation. LiDAR and wheel odometer measurements are fused within a global navigation framework that remains consistent from surface to underground. This is achieved by leveraging the north-finding capability of the M-SINS, with initial global positioning provided by GNSS, UWB, or pre-surveyed landmarks.

• To mitigate cumulative drift over extended traversals, we introduce a practical landmark-based correction mechanism using sparsely deployed spherical targets with embedded UWB tags. These targets serve as globally constrained landmarks via point cloud matching in GNSS-denied areas and help reject incorrect loop closures in perceptually ambiguous tunnels. Both functions are incorporated into a unified optimization backend, significantly improving long-term pose consistency without requiring dense landmark coverage.

• Extensive field experiments and ablation studies in real underground coal mines demonstrate the effectiveness and reliability of the proposed system. Evaluations against state-of-the-art methods confirm the advantages of our approach in challenging mining environments. Additional ablation analyses validate the contribution of each sensor modality and constraint, emphasizing the importance of system-level integration under such demanding operational conditions.

The rest of the paper is structured as follows: Sect. “Related work” provides a brief overview of related work on SLAM systems for subterranean and perceptually degraded environments; Sect. “System overview” presents an overview of the proposed SLAM framework; In Sect. “LiDAR-inertial-wheel odometry frontend”, we detail the iESKF workflow of the system’s frontend, while Sect. “Multi-factor motivated backend” discusses the optimization-based backend of the system; In Sect. “Experiment and results”, the simulation test and field experiment results of the proposed method are presented; The paper ends with a conclusion in Sect. “Conclusion and future work”.

Related work

In subterranean environments with poor illumination, vision-based or visual-centric SLAM approaches tend to perform poorly¹⁰. Conversely, LiDAR-based SLAM solutions exhibit strong performance in pose estimation and environment mapping without the need for external light sources. From early work¹¹ to more recent systems^8,12–15, LiDAR-based SLAM has emerged as the primary solution for mapping complex underground environments. Notably, LiDAR-centric SLAM was the preferred approach for nearly all teams in the DARPA Subterranean (SubT) Challenge, a three-year-long global competition ended in 2021 with the goal of demonstrating and advancing the state of the art in mapping and exploration of complex underground environments³.

To mitigate the issue of LiDAR degeneracy in geometrically self-symmetric and featureless environments, most underground SLAM solutions integrate complementary data from additional sensors with LiDAR. Based on the types of additional sensors used, these methods can be classified into two primary categories: LiDAR-centric methods utilizing local multi-sensor fusion and LiDAR-based methods aided by global constrains.

LiDAR-centric methods utilizing local multi-sensor fusion

To address the pose estimation in sensor degraded environments, multi-sensor fusion has been widely used to combine the advantages of different sensors¹⁶. LiDAR-centric methods utilizing local multi-sensor fusion refer to integrate LiDAR feature points with on-board local sensors data, such as IMU and camera. Koval et al. evaluated state-of-the-art LiDAR-based SLAM algorithms under the dataset of an underground tunnel and concluded that fusing IMU with LiDAR is helpful for correcting the pose estimation in subterranean environment¹⁷. Stefaniak et al.. integrated IMU measurements and dynamic time warping DTW algorithms to locate the underground mine LHD and obtained robust performance^18,19. LINS²⁰ presents an iterative error-state Kalman filter (iESKF) to fuse the LiDAR points in a tightly-coupled scheme that obtained good performance in feature-less scenes. Based on the same filter-based approach, FAST-LIO¹⁴ introduces a new formulation for computing the Kalman gain, leading to significant enhancements in computational efficiency. In its successor, FAST-LIO2¹⁵, the authors improve accuracy and robustness in geometrically challenging scenarios through directly registering raw LiDAR points to the map based on point-to-plane iterative closest point (ICP) without extracting salient features (e.g., plane and edge points). SW-LIO²¹ presents a lightweight tightly coupled LiDAR-inertial odometry based on the sliding window method, using from the previous point cloud frames as a constraint for the current pose, resulting in more accurate state estimation.

In some geometrically uninformative scenes, the texture of environment still offers some visual information. To take the advantage of the complementary data, several LiDAR-inertial-visual odometry systems (LIVO) are proposed to achieve robust state estimation, such as R³LIVE²² and LVI-SAM²³. They usually consist of two subsystems, a LiDAR-inertial odometry (LIO) and a visual-inertial odometry (VIO), that jointly fusing the state vector but separately processing each data without considering their measurement-level coupling. The resultant system takes up significant computation resources. FAST-LIVO¹⁶ uses raw LiDAR points and image pixels to respectively track LiDAR scans and images without extracting any features, resulting in higher computation efficiency. In spite of this, the use cases of the visual camera are limited to well illuminated environment, which limits their applicability. Super odometry⁵ uses an IMU-centric data processing pipeline to receive pose constraints from LIO and VIO, which enables it to overcome potential sensor failures.

However, due to the lack of global position constraints, the cumulative error of LiDAR-centric methods utilizing local multi-sensor fusion will inevitably continue to increase over time. On the other hand, previously proposed methods mainly employ the measurements from a MEMS-based IMU, which suffers significant errors, such as biases and scale factors, resulting in worse accuracy for long-term navigation²⁴. Although in a tightly-coupled LIO or VIO system, the bias of IMU can be estimated and compensated to some extent by the observations from LiDAR or a camera, both LiDAR and camera observations are prone to failure in the perception-degraded environment of underground coal mines. Furthermore, because the detection of the Earth rotation would be drowned out by noise in MEMS-based IMU, a LIO system employing a MEMS IMU typically performs motion estimation within a local reference frame (e.g., the vehicle body frame) rather than an Earth-fixed coordinate system. And the Earth rotation compensation is ignored in these methods, leading to uncorrected errors that can affect long-term navigation accuracy for unmanned transport vehicles in large-scale underground coal mines.

To tackle these challenges, our method integrates the M-SINS with a high-precision FOG into the LIO system, overcoming the limitations of MEMS-based IMUs. While, M-SINS has primarily been predominantly used in underground coal mines for orientation and attitude estimation of excavation machinery such as roadheaders and shearers, its application in underground SLAM remains largely unexplored. The high sensitivity and low noise characteristics of the FOG enable precise detection and compensation of Earth’s rotation, which is critical for maintaining accuracy in long-duration navigation within underground roadways.

This integration facilitates motion estimation in an Earth-fixed coordinate system, enabling consistent navigation across both surface and underground mining operations. The exceptional precision and stability of the FOG ensure reliable performance even in environments where LiDAR and visual sensors are prone to failure. Furthermore, by incorporating wheel odometer measurements, we impose effective velocity constraints within the tightly coupled LIO system based on iESKF, enhancing positioning accuracy and robustness in perceptually degraded conditions.

LiDAR-based methods aided by global constraints

Multi-sensor fusion utilizes each sensor’s strengths to reduce perception degradation in state estimation, but without periodic absolute positioning corrections, cumulative errors accumulate, leading to system divergence and failure. To address this, many underground SLAM methods incorporate global position constraints using wireless positioning signals like ultra-wideband (UWB) and radio frequency identification (RFID), and improve global optimization through loop closure detection.

Over the past decade, UWB has developed into a reliable, inexpensive, and commercially available radio frequency (RF) solution for data transmission, ranging and localization²⁵. UWB is increasingly being utilized to locate personnel and equipment in underground coal mines²⁶. To mitigate accumulated errors, UWB positioning data have been incorporated into underground SLAM systems as global localization constraints. Funabiki et al. introduce a Range-aided pose-graph-based SLAM method to estimate positions in perceptually degraded SubT environment, leveraging sparsely deployed UWB ranging beacons⁹. Zhen et al. develop a probabilistic sensor fusion method that integrates IMU, LiDAR and UWB data to achieve robust localization within long straight tunnels²⁷. Li et al. propose a pseudo-GPS positioning system for underground coal mine, which integrates UWB range measurements and IMU data to provide six degree of freedom (6-DOF) state estimation for coal mine robots²⁸. They also propose an optimal deployment strategy based on an optimal selection mechanism to appropriately deploy UWB anchor nodes within laneways.

UWB positioning accuracy is often compromised by non-line-of-sight (NLOS) signals and multipath effects, especially in large underground coal mines. Precise 3D localization requires more than four non-coplanar range measurements from different UWB anchors simultaneously, necessitating many anchors in long tunnels. Building a comprehensive UWB network for full SubT roadway coverage is challenging and costly. Additionally, the high speed and strict safety standards of underground autonomous vehicles demand further validation and improvement of UWB’s dynamic positioning accuracy and signal stability in SubT environments.

Dong et al. propose analytical and iterative velocity-free localization methods in complex and dynamic mining conditions²⁹. By leveraging active sources, laser rangefinders, proximity sensors together with the proposed localization method, they achieve precise positioning of autonomous rock drilling jumbos and explosive charging vehicles in deep underground mines. However, the method relies on the prior deployment of a sensor network, which significantly increases economic costs and infrastructure investment, particularly when applied to large-scale underground coal mine.

Kim et al. develop an autonomous driving robot that drives and returns along a planned route in an underground mine tunnel through a machine-vision-based road sign recognition algorithm³⁰. However, methods that rely on visual sensors for marker recognition exhibit insufficient reliability and robustness under the poor illumination conditions of underground coal mine.

Ebadi et al. present a large-scale autonomous mapping and positioning method for exploration of perceptually-degraded SubT environments relied on LiDAR-based multi-robot SLAM system^10,31,32. Depending on the centralized multi-robot SLAM system, robust estimate of the trajectories of multiple robots in large-scale, unknown, and complex subterranean environment can be obtained. However, multi-robot cooperative systems in SubT settings still face significant challenges, including stringent requirements for network stability and precise spatiotemporal synchronization.

Additionally, successful loop closure detection is crucial for mapping in SubT environments, as it provides global constraints that optimize pose estimation. Zlot and Bosse present a LiDAR-based SLAM system, consisting of a spinning 2D LiDAR and industrial grade MEMS-based IMU, for mapping of a 17 km underground copper and gold mine tunnel, resulting in a dense and accurate georeferenced 3D surface model³³. To eliminate accumulative errors in the LiDAR-inertial SLAM system, they use a surfel representation to find matches with previous trajectory segments, achieving loop closure detection.

However, the smooth surfaces of underground coal mine tunnels hinder tracking longitudinal motion due to the lack of surface normal along the tunnel. Additionally, the high geometric similarity of tunnel networks often leads to intersections with similar structures, increasing the risk of false loop closures. These false detections can introduce erroneous global constraints, jeopardizing the accuracy and success of the entire mapping process.

To address these issues, we deploy sparse spherical targets along the tunnels, each equipped with a UWB tag containing location information and a unique ID. These targets are easily identifiable by LiDAR. Through point cloud registration, the vehicle can determine its relative position to the targets. By receiving global positions from the UWB tags, the vehicle calculates its own global position, adding absolute position constraints to the pose graph optimization (PGO) framework. This approach enhances pose estimation robustness, eliminates false loop closures, and mitigates low UWB positioning accuracy and the high cost of large-scale UWB base station deployment.

System overview

Our system aims to estimate the position, orientation, and velocity of a mobile vehicle in underground coal mine roadways with low latency and at full sensor rate. The framework of our approach is illustrated in Fig. 2 and includes a filter-based frontend for LiDAR-inertial-wheel odometry and an optimization-based backend with multiple factors.

Fig. 2.

System overview of the multimodal data enhanced SLAM framework.

Frontend introduction

LiDAR-inertial odometry estimation based on iESKF has been proved to be effective in perceptually degraded environments³⁴. However, the LIO system typically provides constraints on position and orientation, while constrains on velocity are indirectly obtained through the integration of IMU measurements. In the absence of direct velocity observations, the accuracy of velocity estimation relies heavily on the correctness of pose estimation derived from point cloud registration. Unfortunately, geometrically self-symmetrical structures and featureless environments of underground tunnels can deteriorate the performance of point cloud registration, leading to optimization divergence along weakly constrained directions³⁵. This issue often manifests as LiDAR-slip when traversing long underground corridors, severely impacting pose estimation performance.

In light of this, the proposed frontend leverages data from LiDAR, M-SINS and wheel encoders to provide 6-DOF state estimation of the vehicle, along with a point cloud map of SubT roadways. The use of multiple sensors in odometry estimation enables the system to handle challenging roadway environments where a single sensor might fail or become partially degenerated. The frontend adopts a tightly-coupled iESKF framework that fuse measurements from LiDAR, M-SINS and wheel encoders. The integration of wheel encoder measurements provides velocity observation, enhancing the accuracy of state prediction for the underground vehicle. Precise inertial measurements from the M-SINS and the algorithmic compensation for the Earth rotation further improve the system’s accuracy and robustness. Of particular importance is that, by utilizing the north-referenced positioning capability of the M-SINS, our LiDAR-inertial-wheel odometry operates within an Earth-fixed coordinate system rather than a local reference frame. This facilitates the use of a unified coordinate system for pose estimation in both surface and SubT environments of coal mines.

It should be noted that although M-SINS has traditionally been larger and more expensive than MEMS-based IMUs, significant cost reductions and substantial miniaturization in recent years have greatly improved both its affordability and deployability, even on compact mining equipment. Some manufacturers now offer products priced below one hundred thousand RMB. With distinct advantages in North orientation, high stability, and strong resilience to magnetic interference, M-SINS is particularly well-suited for GNSS-denied underground environments. Ongoing improvements in cost efficiency and miniaturization are expected to further expand its adoption in underground positioning systems.

Backend introduction

The backend employs PGO to mitigate the accumulative errors introduced by the frontend through global localization constraints and loop closure detections. When global localization signals are available, such as GPS in open ground areas or UWB in underground roadways, GNSS/UWB factors can be added in the PGO framework to effectively limiting system-wide drifts. In the absence of global positioning signals, a common scenario in underground coal mines, accurate loop closure detection, which recognizes when a place has been revisited, becomes a crucial approach to addressing drifts accumulation.

Typical loop closure approaches of LiDAR-based SLAM use techniques that search for physical and geometric structural features matching in an environment based on the method of iterative closest point (ICP)³⁴ or normal distribution transform (NDT)³⁶. However, similar structural scenes and long corridor effect of perceptually degraded underground roadways can cause misalignments in physical structural features matching no matter for ICP or NDT, resulting in the failure of loop closure detection or introducing spurious loop closure. It will negatively impact the overall accuracy and reliability of SLAM. To improve accuracy and robustness of loop closure detection in large-scale subterranean laneways with perceptual degradation, our approach combines artificial and geometric loop closure detection through utilizing spherical targets sparsely arranged in the tunnel.

The spherical targets, depicted in Fig. 1D, are made of high-reflectivity materials. Owing to their uncommon spherical shape and high surface reflectivity in the underground coal mine setting, these targets can be readily differentiated from the tunnel background when exposed to LiDAR illumination. Each target has a precisely known diameter of 300 mm. The point cloud data of these spherical targets will be recorded into the system as a matching sample. Using the sample, the system can perform state estimation based on its position relative to the targets through point cloud registration. To further identify these uniformly shaped spherical targets, UWB tags are embedded in each target, storing the target’s unique code number as well as the positional information of the spherical target’s center. These known-position spherical targets will be preinstalled in the underground roadways at locations prone to loop closure, such as intersections, corners, and merge points. In long straight tunnels, where perceptual degeneracy often occurs, the targets are installed at intervals of 150–180 m at easily accessible locations to provide periodic absolute position updates and suppress odometry error accumulation.

Meanwhile, the hardware of our system includes a UWB module mounted on the vehicle to receive information from spherical targets, as depicted in Fig. 1E. Given the complex electromagnetic conditions and severe NLOS effects in coal mines, UWB-based ranging is highly error-prone. Therefore, the UWB tags embedded in the spherical targets are used solely to broadcast a unique identifier and the pre-surveyed spatial coordinates of the target, rather than for distance measurement. This design significantly mitigates ranging errors and improves reliability under challenging underground propagation conditions.

When the LiDAR distinguishes the spherical target from the roadway background and the UWB module identifies its code number, the corresponding spherical target position is added as a constraint to the pose graph. In our approach, the UWB system is not used for providing location information but rather as an identification tag and data transmission method. This avoids the issue of insufficient positioning accuracy of UWB in underground roadways and eliminates the need for infrastructure investments such as installing positioning base station. Compared to the cost of arranging UWB base stations, the cost of placing spherical targets is almost negligible.

We opted for UWB devices for data transmission rather than lower-cost alternatives such as RFID or QR codes, primarily for two reasons. First, UWB positioning has become the de facto standard technology in the Chinese mining industry, where it is widely adopted. This widespread use offers advantages in terms of existing infrastructure scalability and maintainability. Second, UWB outperforms RFID in terms of transmission range and reliability. In contrast, QR codes are not only difficult to deploy in harsh underground conditions but are also prone to damage. These limitations pose significant risks in applications such as moving equipment positioning, where high reliability and safety are critical.

The global positions of spherical targets can be derived from the measurement data collected during the tunnel excavation process. In the excavation process of underground roadways in coal mines, to ensure excavation accuracy, a series of excavation control points are measured using a total station to obtain their global position under the world geodetic system 1984 (WGS-84)³⁷. These control points are generally spaced at intervals of 20 to 50 m and can be found in the underground tunnels. By utilizing the positional information of these excavation control points, the global positions of the deployed spherical targets can be located and stored in their respective UWB tags.

LiDAR-inertial-wheel odometry frontend

The workflow of the proposed frontend is provided in Fig. 3. The state propagation model is based on M-SINS mechanization, while the measurement model relies on wheel encoders and LiDAR observations. The optimization variables include the position, orientation, and velocity of the underground vehicle, along with the noise and bias of the M-SINS. Initially, global position information from GNSS/UWB or spherical targets is provided to assist with M-SINS initialization. Motion distortion in a LiDAR scan is compensated using M-SINS measurements. The processes of motion prediction and measurement update are executed within the iterated ESKF framework. Unlike the nominal state representation used in the extended Kalman filter (EKF), the error state representation in ESKF offers more advantages³⁸. The error-state system operates close to the origin, where values are consistently small. Additionally, the orientation representation in the error state is minimal, thereby avoiding redundancy and singularity issues.

Fig. 3.

System pipeline of the proposed LiDAR-inertial-wheel odometry frontend.

In the motion prediction process, the state variables to be estimated are divided into two parts: the nominal state and the error state. The accelerations and angular rates measured by the M-SINS are integrated to derive the nominal state. By relegating noise processing to the error state, we can assume that the equations governing the nominal state are noise-free. However, as the motion equations are recursively applied, the error state accumulates under the influence of Gaussian noise. The magnitude of this error can then be quantified by evaluating the mean and covariance of the error states.

In the measurement update process, data from wheel encoders and LiDAR are used to update the posterior mean and covariance of the error state. By formulating residuals as observation equations and incorporating iterative processes during the observation phase, the errors are iteratively refined until they meet convergence criteria. Once convergence is achieved, the error state is merged into the nominal state, resetting the iESKF and completing a cycle of prediction and update.

System initialization and coordinate systems

The initialization is a crucial procedure for the M-SINS. By detecting a zero-velocity condition from the wheel encoders and maintaining a stationary state for one minute, the M-SINS can determine the Earth rotation direction. Subsequently, the relationship between the M-SINS coordinate system and the Earth-fixed coordinate system is established through North orientation and gravity aligning. In our system, the navigation coordinate system is defined using the Earth-fixed east-north-up (ENU) frame, facilitating an initial rough alignment between the M-SINS frame and the world frame. Additionally, the initialization process includes initial positioning, as global position information from GNSS/UWB or spherical targets can be directly incorporated into a unified world frame without the need for coordinate transformations.

As illustrated in Fig. 3, if global wireless positioning information is available, such as GNSS signals on the surface or UWB positioning from an underground base station, the initial positioning within the world frame can be directly obtained. If wireless positioning is not feasible, the initial position can be determined using preinstalled spherical targets. As shown in Fig. 1, the spatial locations of these spherical targets are measured by an electronic total station, based on excavation control points of laneways under the WGS84 coordinate system. The unique position information stored in the UWB tag of each spherical target can be received by the UWB module installed on the vehicle when the target is successfully identified through point cloud registration.

The M-SINS can obtain the initial value of the absolute pose within the Earth-fixed coordinate system through the global-position-aided initialization. Since the vehicle body frame is fixed to the M-SINS frame in our system, the initial absolute pose of the vehicle can also be obtained. The estimated states of the proposed algorithm are relative to the Earth-fixed frame. This paper uses notations listed in Table 1. We use right uppercase letters to define the involved frames, all of which are right-hand coordinate systems. represents the M-SINS frame; G denotes the Earth-fixed frame; denotes the LiDAR frame; denotes the odometer (i.e., wheel encoder) frame. The time offsets among the sensors (LiDAR, M-SINS and wheel encoder) are known. Besides, assuming the three sensors are rigidly attached, the extrinsic parameters defined in Table 1 are pre-calibrated.

Table 1.

Important notations.

Notations	Meaning
	The vector in world frame.
	The vector in LiDAR frame.
	The vector in M-SINS frame.
	The vector in odometer frame.
	The extrinsic of LiDAR frame w.r.t M-SINS frame.
	The extrinsic of odometer frame w.r.t M-SINS frame.
,	The true state and nominal state of .
	The error state between ground-truth and its estimation.

M-SINS mechanization and state propagation

The M-SINS is a fiber optic inertial navigation system specifically designed for use in underground coal mines. It integrates a high-precision fiber optic gyroscope (FOG), enabling it to accurately detect both gravitational acceleration and Earth rotation angular velocity. For high-grade IMU to function effectively, the Earth rotation must be taken into consideration^24,37, and the M-SINS is precisely such a system. Compensating for the Earth rotation is not a negligible factor in our system. To fully utilize M-SINS’s precision, the state propagation of our system is driven by M-SINS mechanization, which accounts for the Earth rotation effects. By establishing the measurement model, the kinematic model of M-SINS can be derived.

M-SINS measurement model

The M-SINS can measure angular rates and accelerations of the carrier. However, M-SINS measurements are subject to various errors, including bias, scale factor, non-orthogonality, installation error and white noise. Errors such as installation errors, scale factor, and non-orthogonality can be quantified through calibration experiments and incorporated into the measurement equation as system errors and inherent attributes. To streamline calculations, this study simplifies the M-SINS error model into two main types: systematic bias b and Gaussian white noise n. The measurement model of M-SINS is as follows:

1

where and are the raw M-SINS measurements of accelerations and angular rates; is the gravity vector in the world frame, the value of which is ; and represent the Gaussian white noises of accelerometer and gyroscope, respectively, , ; the accelerometer bias and gyroscope bias of M-SINS are modeled as random walk process, whose derivatives are Gaussian noises: and ; is the rotation matrix from G to I; is the local Earth rotation rate in the world frame and it can be calculated in

2

where is the constant magnitude of the Earth rotation rate (rad/s); is the geodetic latitude of the initial point.

M-SINS kinematic model

Based on the measurement model, the motion equations can be obtained to predict state propagation of the system. The variables to be estimated include the position , the orientation , the velocity , the gravity, and the gyroscope bias and the accelerometer bias of M-SINS. The M-SINS continuous kinematic model can be described as follows:

3

where and are the position and velocity of the M-SINS frame in the Earth-fixed frame, respectively; the rotation matrix denote the rotation of the M-SINS frame with respect to the Earth-fixed frame; the notation denotes the skew-symmetric matrix of vector that maps the cross-product operation.

Discretizing the continuous model in (3), the system motion can be inferred from the M-SINS measurements. The state variables to be estimated of the system at time can be computed as follows:

4

where is the exponential map from rotation vector to its rotation matrix .

Error-state propagation model

The M-SINS kinematic model in Eq. (4) describes the motion equation of the system true state, which can be represented uniformly by a state vector :

5

The value of the true state is the sum of the nominal state and the error state , represents the generalized addition operator, which defined in Eq. (6).

6

Based on the M-SINS discrete kinematic model, the motion equation of the nominal state can be described as Eq. (7). The motion equation of the nominal state is the same as the true state, except that noise is not taken into account. The noise processing is placed into the motion equation of the error state. The right-hand side of the following equations omitted the to simplify the expression.

7

And the motion equation of the error state can be described as Eq. (8).

8

where ; and represent the Gaussian white noises of velocity and orientation, respectively. The standard deviations of the four kinds of noises in discrete time are as follows:

9

Above all, the prediction process includes the nominal state prediction and the error state prediction. The nominal state prediction is obtained by the M-SINS measurements integration. And the prediction of the error state is summarized in Eq. (10), with the corresponding covariance :

10

where represents the motion noise; denotes the error-state transition matrix, and its expression is depicted as in Eq. (11); is the identity matrix; denotes the covariance matrix of the noise, and its expression is depicted in Eq. (12).

11

12

Measurement model and state update

LiDAR measurement model

Since the feature points obtained in each scan are measured at their respective sampling time, the motion distortion of the LiDAR measurement can lead to the misalignment in the body frame of reference. To address this, motion distortion can be compensated based on the nominal state propagation of M-SINS, utilizing the backward propagation method proposed in¹¹. The motion-compensated points expressed in LiDAR local frame can be viewed as being sampled simultaneously at the same time and used to conduct the residual. Based on the relative estimated pose between the MINS frame and the Earth-fixed frame from the nominal state at the scan end time , the feature point’s estimated pose in the Earth-fixed frame can be represented in Eq. (13).

13

where denotes the known extrinsic of LiDAR frame with respect to the M-SINS frame; m represents the number of feature points.

When registering the scan points to the map, it is assumed that each LiDAR point lies on a neighboring plane where the point truly belongs to, which defined by its nearby feature points in the map. The LiDAR measurement residual is then defined as the distance between the feature point’s estimated pose and its nearest neighboring plane with normal and center point .

14

When transforming the resultant points expressed in the LiDAR frame to the Earth-fixed frame using the true state and considering the LiDAR raw measurement noise , the residual should be zero and the residual can be depicted as below:

15

Approximating the above equation by its first order approximation made at the nominal state , the LiDAR measurement equation leads to:

16

where is the Jacobin matrix of with respect to , which can be calculated according to the chain rule as in Eq. (17); comes from the noise .

17

Velocity observation model

Measurements from wheel encoders are utilized as observations to constrain the velocity. As depicted in Fig. 1 (E), the wheel encoders installed on the left and right wheel of the vehicle separately capture the wheel rotation pulse data . The velocity during the time interval can be calculated in Eq. (18).

18

where denotes the wheel radius, denotes the total number of pulses per rotation.

We utilize the average of the left wheel velocity and the right wheel velocity as the velocity measurement. Assuming that the vehicle is a rigid body and that the odometer frame is rigidly attached to the M-SINS frame, non-holonomic constraints can be applied in velocity observation, i.e., the velocity in the vertical and cross-track should be zero. The vector of velocity observation can be represented as . Based on the state prediction result, the velocity observation model in the Earth-fixed frame can be represented as . The velocity observation residual is then defined as the difference between the wheel encoders measurement in the Earth-fixed fame and the nominal state prediction of velocity at the time .

19

Similarly, if transforming the velocity observation to the Earth-fixed frame using the true state and considering the wheel encoder measurement noise , the residual should be zero vector and the velocity residual can be depicted as below:

20

Approximating the above equation by its first order approximation made at the nominal state , the measurement equation leads to:

21

where is the Jacobin matrix of with respect to , which can be calculated in Eq. (22); comes from the noise .

22

Iterated state update

The observation process contains constraint from point cloud registration, which requires multiple nearest-neighbor iterations before arriving at the correct solution. Therefore, the iteration process is applied into the error state updates to mitigate the effects of nonlinear errors. Assume the current iteration at the time of the iterated Kalman filter is . The iterative process linearizes the error state to update the nominal state until convergence. And then, the true state .

The initial value of the nominal state derived from Eq. (7) (i.e., when ). The value of the error state differs in each iteration, denoting its corresponding covariance as . When , , . represents the corresponding covariance of the error state based on the error state propagation model in Eq. (10). Notice that during the iterated process, the error state to be estimated has been changed from to . The relationship between the two error states can be described as follows:

23

where ; and respectively represent the orientations of and under the Earth-fixed frame; represents the generalized subtraction operator, which is the inverse operation of . The propagated iterated error state and its corresponding covariance impose the prior distribution of the iterated filter as follows:

24

where. Combining the prior distribution, the measurement equation of LiDAR and velocity observation model of wheel encoders, the maximum a posteriori (MAP) estimation is obtained for error state as follows:

25

where .

The above optimization problem can be iteratively solved by Gauss-Newton method. Such an iterative optimization has been proven to be equivalent to an iterated Kalman filter³⁹. Let, , , the updating process can be carried out through the following equations:

26

27

The Kalman gain we used follows the form in¹¹ to avoid large dimensional matrix inversion operation and save the computation. And then the solved updated estimate is used to compute measurement residuals in Eq. (14) and Eq. (19) and repeat the iteration process until convergence (i.e., ). After convergence, the optimal nominal state and the corresponding covariance are used to be the initial state in next time as described in Eq. (28).

28

Multi-factor motivated backend

Although reliable state estimation can be achieved from the LiDAR-inertial-wheel odometry frontend, the system still suffers from accumulative drifts during long-distance underground roadway navigation tasks. To mitigate these errors, global position information from GNSS/UWB and spherical targets are introduced as constraint factors in the pose graph of our back-end optimization module. For the purposes of illustration here, this paper primarily discusses the position constraints from the spherical targets, as GNSS/UWB signals are typically unavailable in the underground roadway environments. Moreover, the position constraints from GNSS/UWB and spherical targets are fundamentally similar.

The pose graph of the backend, depicted in Fig. 4, contains three types of factors and one node to be optimized. The node represents the vehicle’s state at a specific time. The factors include the keyframe odometry factor, the global position factor and the loop closure factor. The node is optimized upon the insertion of these factors using incremental smoothing and mapping with the Bayes tree (iSAM2)⁴⁰. And the specific implementation relies on GTSAM⁴¹ framework, and the Gauss-Newton algorithm is employed to solve the nonlinear least squares problem. The generation of these factors is described as follows:

Fig. 4.

Pose graph of the multi-factor motivated backend.

Keyframe odometry factor

The keyframe is collected from the frontend output odometry when the change in vehicle’s pose exceeds a predefined threshold. Both the pose and point cloud of the keyframe are recorded simultaneously. The state of keyframe pose is then added to the pose graph as a new node . In the context of an underground roadway environment, the threshold is defined as a 1-meter change in position or 10° change in orientation compared to the previous state . The pose transformation from to is represented as .

When the backend receives the frontend output odometry, a series of keyframes is periodically instantiated as pose graph nodes to be optimized. The pose change between adjacent keyframes is added to the pose graph as a local continuity constraint, i.e., the estimated odometry edge shown in Fig. 4. The optimized keyframe pose is updated as: , where is the estimation noise which follows a zero-mean Gaussian distribution with covariance matrix .

Global position factor

A new global position factor is associated with the node in the pose graph when the position signal from GNSS or UWB is available, or when the spherical target can be recognized through point registration. The global position of spherical target is expressed as , which has been previously stored in its UWB tag. When the spherical target is recognized in the roadway environment and its position information is received by the system, the relative pose between the keyframe node and the spherical target can be obtained based on the method of ICP.

The true state of the keyframe node can be inferred as , where represents the pose of the spherical target (if there is only one target can be recognized in a LiDAR scan, ) and is the measurement noise which follows zero-mean Gaussian distribution with the covariance matrix . And then the relative transformation between the true state and the estimated state of the keyframe node can be obtained and added to the pose graph as a measurement edge to optimize the trajectory.

Loop closure factor

In the absence of global localization signals, loop closure detection can effectively eliminate frontend cumulative errors, enabling globally consistent state estimates and high-quality point cloud maps. However, the symmetrical geometry and sparse features of the underground roadway environment often lead to the failure of loop closure detection methods that rely on point cloud descriptors^42,43 or distance-based approach^8,13. In our system, loop closure detection is achieved through the use of spherical targets. Each spherical target has own unique identification and can be recognized by the system, making it easy to determine whether the vehicle has returned to a previously visited area upon receiving the target signal. If the target is revisited, the newly added keyframe searches for pairing previous keyframes within the scope of the target effective region. Loop closure detection is then performed between the two pairing keyframes that are close in Euclidean space but distanced in timestamp.

As depicted in Fig. 4, when a new state is added to the pose graph within the effective region of a revisited spherical target, the prior state close to in Euclidean space is identified as a candidate for loop closure. Meanwhile, to enhance the accuracy of inter-frame matching, point cloud registration is performed by matching the newly added keyframe with a local map. This local map consists of the previous keyframe and five frames before and after it. By matching the LiDAR scan of the current keyframe with the local map using Scan-to-Map point registration, the relative transformation between the current keyframe and the previous keyframe is obtained. This transformation is then added to the pose graph as a loop closure factor.

Experiment and results

In this section, we evaluate and analyze the proposed multimodal data enhanced SLAM system through both simulation tests and field experiments. We compare our framework against state-of-the-art SLAM methods, including LEGO-LOAM, LIO-SAM, and FAST-LIO2. All methods are implemented in C + + and executed on an embedded computing platform equipped with an Intel i9-12900E CPU, running the robot operating system (ROS)⁴⁴ on Ubuntu Linux. The vehicle platform used in our tests is the Trackless Auxiliary Transportation Robot (TAT Robot), a self-developed automatic guided carrier designed for underground coal mine environments, as shown in Fig. 1E.

Simulation test in gazebo

The Gazebo engine is utilized to construct an underground roadway environment and a virtual model of the TAT Robot equipped with relevant sensors. As shown in Fig. 5A, the virtual environment includes nearly 500 m of testing roadway featuring diverse common underground scenarios, such as long straight corridors, T-junctions, right-angle turns, and refuge chambers. Six virtual spherical targets containing global position information are dispersed throughout the underground roadway network. To align with our algorithm, the world coordinate system of the virtual environment is modeled as an east-north-up (ENU) Earth-fixed frame, with the frame orientation depicted in Fig. 5A. The origin of the world frame is set at the start point indicated in the diagram.

Fig. 5.

Simulation Test Scenario based on Gazebo. A: Simulated underground roadway environment: A solid white line represents the traverse through the roadway, while arrows depict the direction of traverse; Yellow circles denote the action area of spherical targets. B: Virtual model of Trackless Auxiliary Transportation Robot.

The virtual model of the TAT Robot is constructed based on the structure and dimensions of the actual robot prototype. Sensors necessary for the test, including LiDAR, camera, wheel encoder and IMU, are incorporated into the model. Since the virtual environment does not account for the Earth rotation, we use an IMU model instead of a M-SINS model. As shown in Fig. 5B, the IMU frame aligns with the robot body frame, fixed at the geometric center of the robot. The x-axis of the IMU frame aligns with the robot’s longitudinal axis, and is perpendicular to the robot’s body. The initial IMU frame serves as the world frame, with the start point as the origin and upward movement in the map representing the East orientation. The accelerometer and gyroscope biases and Gaussian noise are simulated based on M-SINS parameters, as detailed in Table 2.

Table 2.

Parameters used in the simulation test.

Parameters	Value	Units
Gyroscope systematic bias	0.015
Accelerometer systematic bias	50
Gyroscope random walk	0.002
Accelerometer random walk	50
	100
	10
	20

Two LiDAR are symmetrically positioned at the front and rear ends of the robot. The point clouds obtained from these LiDAR are fused and output as a single entity, with the LiDAR frame coinciding with the IMU frame. During the test, the robot travels the entire roadway from the start point, gathering video information, point clouds of the virtual environment, and the robot’s motion parameters. The parameters used in the test are listed in Table 2. The robot moves at a nearly constant speed of 3.0 m/s, with a maximum angular velocity of 0.5 rad/s.

Based on the testing data, various SLAM methods are applied to estimate the robot’s pose and construct the roadway map. Table 3 illustrates the performance evaluation of these methods using the root mean square error (RMSE) of both absolute pose error (APE) and relative pose error (RPE), benchmarked against the ground truth robot pose from Gazebo. All utilized methods are LiDAR-centric SLAM solutions that incorporate data fusion.

Table 3.

RMSE translation error w.r.t ground truth (meters) (Best in Bold).

Methods	LEGO-LOAM	LIO-SAM	FAST-LIO2	Our Method
APE	19.27	2.65	0.80	0.57
RPE	0.285	0.193	0.073	0.068

LEGO-LOAM, a loosely-coupled LiDAR-inertial odometry method, processes LiDAR and IMU measurements separately before fusing their results. During testing, LEGO-LOAM exhibited the poorest performance due to its separate handling of scan registration and data fusion, which often leads to an underestimation of the robot’s motion along the featureless simulation roadway. In contrast, LIO-SAM and FAST-LIO2 are tightly-coupled LiDAR-Inertial odometry methods that fuse raw feature points with IMU data. The test results indicate that the filter-based method, FAST-LIO2, outperforms the optimization-based method LIO-SAM in the featureless tunnel environment, despite LIO-SAM’s loop closure detection capability. This suggests that the iterated error-state Kalman filter-based data fusion method, employed by FAST-LIO2 and our proposed method, is more suited for long symmetric corridors lacking detectable geometric features.

Compared to other representative algorithms, the proposed multimodal data enhanced SLAM framework achieves superior performance in both state estimation accuracy and roadway mapping quality during simulation testing. The RMSE of APE and RPE relative to the ground truth are 0.57 m and 0.068 m, respectively, the lowest among the four tested methods. As shown in Fig. 6, with the assistance of the data from wheel encoder and the positional information of spherical targets, the robot trajectory estimated by our method aligns most closely with the ground truth. Furthermore, as the robot travels greater distances, the translation error and yaw angle error calculated by our method remain consistently low compared to other methods.

Fig. 6.

Pose estimation accuracy comparison for the selected methods.

Comparatively, FAST-LIO2 achieves relative accurate pose estimation, but the results indicate that its translation error gradually increases with traveled distance due to the absence of global position constraints. This issue is not prominent in small-scale, ideal roadway conditions in simulated environment. However, in large-scale, actual underground coal mine roadways, uneven tunnel surfaces and the long corridor effect can exacerbate error accumulation, leading to pose estimation faults and mapping failures.

In our method, the position information of spherical targets is introduced as global position constrains to eliminate accumulative errors. Additionally, placing spherical targets at T-junctions assists the robot in achieving accurate loop closure constraints when it returns to the same target. This approach ensures globally consistent pose estimation and mapping results, as shown in Fig. 7D.

Fig. 7.

Mapping results comparison based on simulation testing data. A: LEGO-LOAM; B: LIO-SAM; C: FAST-LIO2; D: Our Method. The mapping of LEGO-LOAM fails after traveling along featureless corridor due to the underestimated of the robot motion. LIO-SAM establishes a complete roadway map, but its map shows bad global consistency, despite it achieves loop closure detection. FAST-LIO2 outperforms LIO-SAM, in the absence of loop closure detection. However, its map shows displacement when the robot returns to the start point due to the accumulative errors. Our method produces a map that is consistent with the simulated roadway environment.

Field experiments in an active underground coal mine

To validate the practicality of the proposed method, field experiments were conducted in an active underground coal mine, Cunchao Tower Coal Mine, located in northwest China. These experiments were divided into three distinct phases.

First, utilizing the proposed framework, we established a comprehensive environmental map of the test coal mine, covering both the surface ground square and the degraded underground roadways. The localization and mapping were conducted within a globally consistent navigation system based on the Earth-fixed frame, enhanced by surface GPS signals and sparsely deployed spherical targets along the underground tunnels.

Second, to verify the advantages of the proposed algorithm in degraded SubT environments, we conducted quantitative experiments by comparing it against other algorithms in the underground parallel roadway region of the test coal mine, using total station measurements of spherical targets as ground truth. We also evaluated whether the robot could return to the starting point with minimal accumulative drifts.

Third, to further evaluate the effectiveness of integrating wheel odometry and Earth-rotation compensation in the frontend, along with the global position constraints provided by spherical targets, we performed ablation experiments by separately removing the wheel encoder measurements, the Earth-rotation compensation, and the spherical targets from our framework, analyzing the individual contributions of each component to the overall performance of the proposed algorithm.

Globally consistent mapping of both surface and underground roadways

The field experiment was conducted using the prototype of the TAT Robot. As illustrated in Fig. 1E, the TAT Robot is equipped with the necessary sensors, including 3D LiDAR, M-SINS, wheel encoders, cameras, a GPS receiver, and a UWB module. The robot traveled from the start point on the ground square down into its underground tunnel roadways. The sensor arrangement mirrors that of the robot’s virtual model. It should be noted that the cameras were employed primarily for experimental documentation. The visual data was not integrated into our mapping or localization methods due to the challenging illumination and texture-deficient conditions prevalent in the underground coal mine environment.

Specifically, the robot was outfitted with two types of 3D LiDAR: the 128-beam Ouster OS1-128 spinning mechanical LiDAR with a full field of view (FoV) and the DJI Livox HAP solid-state LiDAR with a 120° FoV. Each type of LiDAR was installed in pairs, symmetrically positioned at the front and rear ends of the robot. The OS1-128 LiDAR was mounted above the explosion-proof housing, while the HAP LiDAR was placed within the explosion-proof housing behind explosion-proof glass. Additionally, the M-SINS and the embedded computing platform were housed within the explosion-proof enclosure. The M-SINS coordinate system was set as the robot’s body frame, and the extrinsic parameters and time offsets among the sensors were pre-calibrated. The parameters used in field experiments are listed in Table 4.

Table 5.

Start-to-end translation errors (meters) (Best in bold).

Errors	LEGO-LOAM	LIO-SAM	FAST-LIO2	Ours- Frontend	Whole Framework
X-axis	63.56	1.21	41.94	3.40	1.56
Y-axis	7.60	18.72	12.61	4.89	0.57
Z-axis	4.71	0.18	9.96	8.60	0.24
Total	64.19	18.76	44.91	16.89	1.68

Table 4.

Parameters used in field Experments.

Parameters	Value	Units
Gyroscope systematic bias of MSINS	0.015
Accelerometer systematic bias of MSINS	50
Gyroscope random walk of MSINS	0.002
Accelerometer random walk of MSINS	50
FoV of OS1-128	360
FoV of HAP	120
Standard deviation of wheel encoder	0.3%	/
	100
	10
	20
	1

At the beginning of the experiment, the M-SINS system completed its initialization process at the start point on the ground square, achieving an initial rough alignment between the M-SINS frame and the ENU frame. Our navigation coordinate system was then defined under the ENU frame, with the starting point’s global position obtained from the GPS receiver serving as the coordinate origin. Once the robot commenced movement, its pose estimation was performed under the Earth-fixed frame, and the established map of the coal mine was aligned with the actual world map, as illustrated in Fig. 8A. The robot proceeded from the start point to the entry spot at the wellhead to conduct a safety check, before entering the wellhead from the entry spot. During this phase, it created a point cloud map of the ground square and the inclined roadway of the coal mine, as shown in Fig. 8.

Fig. 8.

Mapping and localization on the ground square of the experiment coal mine based on the proposed algorithm. A: Established map aligned with the google map; B: Point cloud map of the ground square; C: Top view of the ground square and its inclined roadway; D: Side view of the ground square and its inclined roadway.

Subsequently, the robot traversed an inclined long tunnel and entered the relatively flat transport roadways at the bottom of the coal mine, which consisted of two parallel tunnels linked by several connection roadways. A number of spherical targets identified by UWB labels were dispersed along the underground roadways, providing global position constraints for the approaching robot. Based on the data collected from the experiment, the map of the underground roadways traversed by the robot was established under a globally consistent coordinate system using our proposed method, as depicted in Fig. 1. The total distance covered during this experiment exceeded 3 000 m, including nearly 2 400 m of featureless underground corridors.

Mapping and localization in perceptually-degraded underground roadways

The second phase of the field experiment involved a quantitative analysis of our proposed method by comparing it with other representative algorithms. Obtaining ground truth for the robot’s trajectory and the roadway map in large-scale underground coal mine environments is a challenging task. To evaluate our algorithm, we adopted the method of measuring start-to-end drift, wherein the robot starts and stops at the same location within the tunnel. Given that the inclined tunnel roadway is a single, uninterrupted long corridor that makes it difficult to form an effective traveling loop for evaluation, the testing area for our method and other representative algorithms primarily focused on the parallel roadways.

It should be noted that, for the sake of data conciseness, the starting point of this experiment is different from that of the previous experiment. As shown in Fig. 9, the starting point of this experiment is designated as the coordinate origin (0,0,0). The positional coordinates obtained during this experiment are relative to the Start/End point.

Fig. 9.

Underground experiment roadways for testing the proposed method.

As depicted in Fig. 9, the underground experiment field comprises two parallel long tunnels linked by several connecting roadways. Ten spherical targets, each identified by its own UWB label, are dispersed throughout the roadways, primarily installed at the intersections of the tunnels. The positions of these spherical targets relative to the start point were measured using an electronic total station. The 3D positions of these targets are also used as ground truth positioning points for the experiment.

The robot completed the initialization of the M-SINS at the start point and established the navigation coordinate system under the Earth-fixed frame in the GPS-denied underground environment with the aid of the global position information provided by the No. 1 spherical target. After initialization, the robot moved from the start point along the route marked by the red solid line and eventually returned to the start point, forming a closed loop of the traversed path.

The robot’s traversed paths obtained using our method and other representative algorithms in the underground experiments are depicted in Figs. 10 and 11. Due to the environment of long symmetric tunnels lacking detectable geometric features, both loosely-coupled and tightly-coupled LIO methods resulted in erroneous estimates of the robot’s pose, which increased with distance, leading to the failure of path closure. Effective loop closure detection can mitigate cumulative errors by providing global position constraints. However, spurious loop closures are frequent in underground tunnels with repetitive appearances. From the robot trajectory derived by LIO-SAM, it can be observed that false loop closure detection induced by the geometrically self-symmetrical structures of the long underground corridor led to significant distortions in the robot’s estimated trajectory. Hence, rejecting false loop closures is crucial in large-scale underground tunnel environments.

Fig. 10.

Robot traversed trajectories obtained by the used methods during the field experiment. The spherical targets position relative to the start point, which were measured by electronic total station, are also labeled in the graph. The top row shows the bird eye view of the trajectories. Compared to the other algorithms with large end-to-end drifts, the proposed method can obtain a closed traversed path when the robot returned to the start point. The bottom row shows the Z-axis drifts of all used method. The proposed method with optimized backend can effectively relieve the Z-axis drifts that is common in other LIO methods.

Fig. 11.

The comparison of robot pose variation with the traversed time estimated by the used methods in forms of translation in three axes (X, Y, Z) and orientation in three Euler angles (Roll, Pitch, Yaw).

Since there is no available ground-truth of the robot trajectory, we quantitatively compared the algorithms based on the start-to-end translation errors. As shown in Table 5, the total start-to-end translation error of the robot path obtained with the proposed method is 1.68 m which is significantly less than the other algorithms.

With the integration of wheel-encoder data and compensation for Earth rotation, the frontend of our method outperforms other state-of-the-art LIO algorithms. Nevertheless, in the absence of backend optimization and loop closure detection, all the aforementioned algorithms experience severe drifts in the Z-axis direction, which is a common issue for LiDAR-centric methods in degenerated environments. To address this, the backend optimization of our method introduced global position constraints and unambiguous loop closure detection with the aid of spherical targets to eliminate the Z-axis drifts, as shown in Fig. 11.

Since there is no available ground truth for the robot trajectory, we quantitatively compared the algorithms based on the start-to-end translation errors. As shown in Table 5, the total start-to-end translation error of the robot path obtained with the proposed method is 1.68 m, which is significantly less than that of the other algorithms.

Based on the data collected from the field experiment, we established maps of the parallel roadways using our algorithm as well as other representative methods, as shown in Fig. 12. In the absence of loop closure detection, the mapping results from LEGO-LOAM and FAST-LIO2 exhibited noticeable start-to-end drifts and map displacements. The drift is significant due to the presence of featureless corridors, causing ICP to underestimate the robot’s motion along the direction of the tunnel. In contrast, odometry drift was mitigated by correct loop closure detection during the mapping process of LIO-SAM. However, the ambiguous loop closures induced by the long corridor effect ultimately introduced false loop closures into the mapping process, resulting in significant distortion of the entire map.

Fig. 12.

Mapping results comparison of the used methods based on the field experiment data. A: The map obtained by LEGO-LOAM reports the misalignments of all the connecting roadways and severe Z-axis drifts; B: The map of FAST-LIO2 reports the misalignments of the last two connecting roadways and severe Z-axis drifts; C: The map of LIO-SAM reflects the mapping distortion induced by the false loop closure detection; D: The map established by the proposed method without backend outperforms than other competitive algorithms, but still with significant Z-axis drifts; E: The proposed method with backend optimization establishes a closed roadway map with minimum distortion and reflects the geometric structure of the parallel tunnels realistically.

The mapping result from our proposed frontend outperformed other competitive frameworks, establishing a relatively accurate map of the connecting roadways between the two parallel roadways. However, all the mapping results from the mentioned algorithms exhibited severe drifts in the Z-axis direction. In comparison, the map derived from our method, with backend optimization, realistically reflected the geometric structure and positional relationships of the parallel tunnels and their connecting roadways.

Furthermore, during the test, the point cloud data from both the HAP LiDAR and the OS1-128 LiDAR were separately applied in our framework. The results demonstrated that the solid-state LiDAR with a small field of view (FoV) achieved an essentially consistent mapping result with the full FoV spinning mechanical LiDAR based on our algorithm. Although the HAP LiDAR was enclosed in an explosion-proof shell, it exhibited only partial deficiencies in map integrity due to the limited field of view, as shown in Fig. 13. Consequently, the solid-state LiDAR is more suitable for underground coal mine applications due to its low cost and ease of explosion-proof design. One of our ongoing work focuses on optimizing our algorithm according to the characteristics of solid-state LiDAR with explosion-proof design.

Fig. 13.

Underground roadway mapping results comparison of the solid-state LiDAR with a small FoV and the spinning LiDAR with a full FoV based on the proposed framework.

Ablation experiments

To quantitatively analyze the impact of each individual component of the system, we conducted ablation experiments. In these experiments, we systematically disabled one component at a time while keeping the others operational. Specifically, we disabled Earth-rotation compensation in the M-SINS signals, wheel odometry, and global constraints from spherical targets. By comparing the performance of these modified systems against a baseline where all components were active, we were able to isolate and evaluate the contribution and influence of each individual component. This approach provided a clear understanding of the role each component plays in the overall system performance, guiding further optimization efforts.

Figure 14 illustrates the results of the ablation experiments, providing a visual comparison of performance with each component deactivated. The data indicate that the absence of wheel odometry has the greatest impact on the pose estimation, primarily causing inaccuracies along the tunnel direction. This is because, without direct velocity measurements, the system relies on the fusion of LiDAR and inertial measurements, leading to significant degradation in odometry estimation in long-straight underground tunnels.

Fig. 14.

Robot traversed trajectories obtained from ablation experiments. The baseline of these experiments is established using our system with all component activated.

When the Earth rotation is not compensated for, the M-SINS degrades into a regular inertial measurement unit, significantly impacting accuracy. As the robot travels further, the cumulative errors continue to increase. Although global position information from spherical targets can provide some compensation, the sparsity of spherical targets causes the global position optimization to fail once the cumulative errors exceed a certain threshold, resulting in poor accuracy and robustness.

Without global position constraints from spherical targets, the robot poses are estimated based on the LiDAR-inertial-wheel odometry frontend. In the absence of periodic absolute positioning corrections, cumulative errors of the LIO in perceptually-degraded SubT roadways can be reduced through tight coupling with wheel odometry and Earth rotation compensation in the M-SINS signals. However, the system still suffers from cumulative drifts increased with the distance traveled.

Table 6 presents the performance of the ablation experiments using the RMSE of both APE and RPE, benchmarked against the baseline where all components were active. In summary, the ablation experiments reveal the importance and interdependence of each component in our framework under the challenging environment of underground coal mines.

Table 6.

RMSE translation error w.r.t baseline (meters).

Ablation Experiments	Wheel Odometry Disabled	Earth-rotation Compensation Disabled	Spherical Targets Disabled
APE	150.83	73.92	10.54
RPE	0.315	0.24	0.17

Conclusion and future work

This work presents a multimodal data enhanced SLAM system aimed at addressing pose estimation and map construction challenges in large-scale underground coal mine environments. The system comprises a tightly-coupled LiDAR-inertial-wheel odometry frontend and a multi-factor optimization backend. In light of LiDAR degeneracy due to the absence of geometrically informative structures in underground tunnels, the system fuses Earth-rotation-compensated inertial measurements from M-SINS and velocity data from wheel encoders with LiDAR points based on the iterated error-state Kalman filter. Additionally, the frontend operates a globally consistent navigation coordinate system covered from the coalmine ground square to the underground roadways, leveraging the M-SINS capability of auto-navigation and gravity-alignment, with initial positioning provided GNSS/UWB signals or pre-installed spherical targets.

To mitigate cumulative drift over long traversed distances, the backend introduces global position constraints and loop closure detection by constructing a multi-factor pose graph. In subterranean environments lacking global localization, sparsely deployed spherical targets identified by UWB labels provide global position constraints through point cloud registration. These spherical targets also help exclude ambiguous loop candidates, ensuring robust and efficient loop closure detection in underground tunnels with appearances. The combination of artificial and geometric loop closure detection effectively limits global drift, while maintaining manageable computational load, minimal infrastructure, and robustness against perceptual degradation.

The effectiveness of the proposed system is validated through simulation tests in a virtual subterranean roadway and field experiments in a real underground coal mine using the TAT robot platform, both featuring challenging geometrically symmetrical structures. Furthermore, ablation experiments, systematically disabling individual components, reveal importance and interdependence of each component in our framework under the challenging conditions of underground coal mines. Future research will focus on optimizing the placement of spherical targets within the underground roadway and enhancing the real-time performance of the algorithms.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

Mingrui Hao conducted the overall design of the method, data collection, and the overall writing of the paper. Jie Ren was in charge of the experimental design and sensor layout. Xiaodong Ji was responsible for both data collection and analysis.Yueqi Bi handled the data analysis. Sihai Zhao and Miao Wu were in charge of the overall project planning and design.

Data availability

The datasets generated and/or analysed during the current study are not publicly available due to privacy and safety concerns related to the specific infrastructure of the coal mine but are available from the corresponding author (Xiaodong Ji) upon reasonable request.

Declarations

Competing interests

The authors declare no competing interests.