Abstract
Monitoring of the open caissons during the sinking process is a crucial aspect in construction projects. Traditional methods such as Electric Level have limitations in providing continuous monitoring throughout the sinking process. To address this challenge, this paper introduces a novel approach utilizing computer vision and linear intensity change model for 3D reconstruction of open caissons and real-time monitoring of the caisson sinking. In this proposed approach, RGB-D camera was firstly calibrated for the intrinsic and extrinsic parameters. Then 3D coordinates of the open caisson were obtained via the calibrated IR sensors of RGB-D camera. Subsequently, the spatial visual image of the open caisson for display was calculated using RGB sensors and the 3D coordinates. During sinking, multiple RGB sensors combined with a linear intensity change model were used for monitoring settlement displacement. The spatial visual image of the open caisson was updated with the data of the settlement and out-plane displacement. The feasibility and effectiveness of the proposed approach are validated by examples. This innovative monitoring solution has the potential to revolutionize the construction process in open caisson, ensuring timely detection of any deviations and facilitating proactive decision-making for optimal project outcomes.
Keywords: Computer vision, Open caissons, 3D reconstruction, Sinking, Calibration
Subject terms: Natural hazards, Civil engineering
Introduction
An open caisson, made of reinforced concrete and open at both the top and bottom during construction, is first constructed on the ground. It is then lowered to the desired depth by clearing obstructions beneath it (see Fig. 1). The open caisson method is commonly employed in complex geological conditions like sandy soil or areas with a high groundwater table1. This technique minimizes disturbances to surrounding structures and buildings. Many projects opt for this type of foundation, including underground storage and attenuation tanks, pumping stations, bridge piers as well as launch and reception shafts for tunnel boring machines2–7.
Fig. 1.
Sinking of a open caisson.
The sinking process of an open caisson often faces challenges related to controlling the sinking displacement and velocity8–10. This control is achieved by adjusting the velocity of digging obstruction and soil-structure frictional stresses using lubricating fluids. Moreover, an active control method with a data-driven strategy was proposed to predict the sinking displacement8. The sinking displacement of each wall of the foundation is a crucial variable that ensures smooth and accurate construction4.
Techniques for obtaining displacements in civil engineering are discussed in11–23. The conventional approach to monitoring sinking displacement involves using an Electric Level. Initially, height information for measurement points is obtained during the first survey. Subsequently, after the second survey, sinking information is calculated by comparing data from both surveys. While this traditional method is commonly employed, it has drawbacks such as requiring multiple skilled workers, low efficiency, and the inability to continuously monitor sinking.
Typically, the Global Positioning System (GPS) can be utilized for monitoring displacements over a large area with high accuracy14. The process involves designing and installing measuring points in the monitoring area, followed by placing sensors (antennas) on these points to receive coordinate information from GPS satellites. By comparing data at different times, displacement information can be calculated to predict landslide deformations24–26. This method is suitable for large-scale monitoring but has limitations when applied to monitoring the sinking of an open caisson. It requires professional workers, involves contact-based measurements that necessitate equipment installation on the caisson walls, and the permanent measurement points are susceptible to damage during the sinking process11–13.
Another advanced method for monitoring displacements is through the use of three-dimensional (3D) laser scanning technology15. This method entails remotely collecting highly detailed data of the monitoring area to create a series of digital terrain models (DTMs) over time. The DTMs generated have a small point spacing, enabling precise determination of the geometry of large and even hard-to-reach areas15,16. This method is a non-contact approach that typically allows for the simultaneous monitoring of multiple points. However, the equipment required for 3D laser scanning is considerably more expensive compared to traditional monitoring methods. Additionally, the operation of this technology still necessitates skilled professionals to ensure accurate data collection and analysis.
To address the limitations of the traditional method, computer vision (CV) has been utilized for monitoring the displacement of the structures, offering a non-contact alternative27–31. However, this approach focused on the displacement and it has difficulties to 3D mapping. This paper proposed an approach based on RGB-D camera and linear intensity change model for 3D reconstruction and displacement estimation of an open caisson. The proposed approach boasts significant advantages, including the elimination of the need for skilled workers, improved efficiency, and continuous monitoring of sinking information throughout the construction process.
The remainder of this paper is structured as follows: section “Technical background” covers the technical background, followed by an explanation of the proposed approach in section “Development of computer vision-based approach for open caissons monitoring”. The performance of the method is then validated through testing in section “Experimental evaluation”, with section “Conclusion” dedicated to presenting the concluding remarks.
Technical background
3D mapping with a camera
A point 
can be projected onto an image plane using a camera (refer to Fig. 2). The perspective projection equations in physical image coordinates are as follows,
Fig. 2.
Physical image coordinate system32.
 |
1 |
 |
2 |
where x, y, z are the components of point 
, and u, v are the components of point 
on the image plane (see Fig. 2). 
, 
, 
, 
and 
are the intrinsic parameters of the camera. Eqs. (1) and (2) can be recast as follows,
 |
3 |
where 
and 
. 
is defined as follows,
 |
4 |
In Eq. (3), point 
is expressed in the camera frame C. When the camera frame C is different from the world frame W, one need to note that
 |
5 |
where 
represents a rotation matrix defined by three angles, while 
represents a translation vector defined by three coordinates, which collectively form the extrinsic parameters. The coordinates 
correspond to the frame C, and 
corresponds to the frame W. By substituting Eq. (5) into Eq. (3), one obtains:
 |
6 |
Note that 
in Eq. (6) is the point in the world frame W.
Structural displacement calculation by matching measurement points
Now the approach for obtaining the sinking information of the open caisson was described. This method focuses on aligning the grayscale intensity of a subset in the reference image with that of a corresponding subset in the deformed image. A linear intensity change model is utilized33, incorporating both the scale and offset of the intensity. This model is expressed as:
 |
7 |
In the reference image, 
denotes the grayscale intensity value at position 
, whereas 
symbolizes the grayscale intensity value at location 
in the deformed image. The variable n signifies the pixel quantity within the reference subset that encompasses the measurement points. Moreover, a stands for the intensity change scale factor, and b represents the intensity change offset.
Development of computer vision-based approach for open caissons monitoring
Basic principles
A strategy was introduced to utilize an RGB-D camera for determining the 3D coordinates and sinking details of the open caisson during construction. The RGB-D camera provides both depth and RGB images, acquired through a depth (IR) sensor and an RGB sensor, respectively. To extract this information, the following steps must be executed:
Place measurement points (interesting points) on the open caisson wall, as depicted by the square markings in Fig. 3. It is easier to track and calculate settlement displacement when there is a significant difference between the color of the square boxes and the background color. It’s noteworthy that d represents the gap between two rows of measurement points, and to ensure at least one row of measurement points on the image, d should be less than a critical value
.Calibrate the intrinsic and extrinsic parameters of the RGB and IR sensors.
Position RGB-D cameras surrounding the open caisson to record the measurement points, as visualized in Fig. 4. During the settlement of the open caisson, the surrounding soil will be deformed. The range of deformation will vary depending on types of the soil. The camera should be set up outside of the anamorphic range and the soil under the camera is in a stable state during the monitoring period. The distance of the camera from the target can affect the resolution of the image. When the camera is far away from the target object, the image of the object on the image sensor becomes smaller. In the case of a fixed number of pixels, an object of the same size occupies a relatively small number of pixels in an image taken at a distance, and the image will be less detailed from a visual point of view. The accuracy of the monitoring data will be reduced.
Calculate the 3D coordinates of points within the IR camera coordinate system and subsequently convert these points to the world coordinate system (open caisson coordinate system).
Estimate the sinking details (in-plane data) of the open caisson by aligning measurement points across various RGB images captured at different time instances.
Extract the out-of-plane information from the IR sensor.
Display and update the spatial location of open caisson in real-time with the obtained information.
Fig. 3.
Measurement points on the wall of open caisson.
Fig. 4.
Top view of the open caisson and the distribution of RGB-D cameras.
Calibration of RGB-D camera
Camera parameter calibration is crucial before use. This section outlines the process of determining the intrinsic and extrinsic parameters of both RGB and IR sensors. The calibration process revolves around calculating the camera’s perspective projection matrix, 
. Once 
is obtained, the camera’s intrinsic and extrinsic parameters can be derived from its decomposition.
Specifically, suppose that a camera observes n points 
, 
, 
, 
, with known positions in given world coordinate system and their image points 
, 
, 
, 
. According to Eq. (6), one can obtain the following equations,
 |
8 |
 |
9 |
where 
and 
denotes rows of the matrix 
.
Equations (8) and (9) can be recast as follows,
 |
10 |
where 
is defined as follows,
 |
11 |
Equation (10) can be resolved using a linear least-squares method. After obtaining 
, the camera’s intrinsic and extrinsic parameters can be calculated (for computational specifics, consult32). This method is referred to as linear camera calibration. A nonlinear approach can be employed for camera calibration that incorporates all constraints related to a camera. In the nonlinear approach, the camera calibration problem is simplified to minimizing the least-squares error as follows,
 |
12 |
where 
denotes the vector formed by all intrinsic and extrinsic parameters of a camera. This nonlinear approach can be solved with nonlinear least-squares methods such as the Gauss-Newton method and the Levenberg-Marquardt method.
To calibrate the parameters of the RGB and IR sensors, the calibration board should be firstly generated (see Fig. 5). Then, images under different views were collected, which can be done by fixing the camera and moving the calibration board. Next, extreme points should be selected and corner points should be found (see Fig. 5) on all the images. Note that the physical edge size of each square must be known. In the end, one can use the above calibration approach to obtain the intrinsic and extrinsic parameters for RGB and IR sensors. This calibration can be done via the MATLAB toolbox by Jean-Yves Bouget.
Fig. 5.
Calibration board.
After the calibration parameters of each camera were estimated, the stereo calibration can be carried out. This stereo calibration transforms the world coordinate system to IR camera coordinate system. Finally, one can obtain 
for the RGB sensor and 
for the IR sensor.
Obtaining 3D coordinates of physical points
Once the cameras have been calibrated, one can obtain the 3D coordinates of the image points. Based on Eq. (3), one can obtain the following equation,
 |
13 |
where 
. For the IR sensor, the pixel of the image (i.e., 
) is actually the depth information. Thus, Eq. (13) can be rewritten as follows,
 |
14 |
where 
is the 3D coordinates of the points from IR images.
After the 3D points 
were obtained, the following equations can be used to compute the pixel coordinates of its projection using the RGB sensor. Firstly, the homogeneous coordinates can be obtained as follows,
 |
15 |
where 
is the homogeneous coordinates and it has the following form,
 |
16 |
Then based on Eq. (16), 
can be obtained as follows,
 |
17 |
Now a set of 3D points describing a scene were obtained and have projected them onto the RGB image. In this way, the color of a 3D point 
can be determined as the RGB color value of its 2D projection 
Note that several RGB-D cameras are used in monitoring the open caisson and each RGB-D camera has an IR senor. Thus, the 3D coordinates of the points in each IR coordinate system should be transformed to the open caisson coordinate system. When the 3D coordinates of the points are obtained, the spatial location of the open caisson can be displayed. The spatial location is supposed to be used for guiding the construction.
Structural displacement calculation via a linear intensity change model
The assumption is that points in close proximity within a reference subset should remain neighboring in the deformed subset33. The location of the subset encompassing the measurement points in the reference image shifts in the deformed image due to the sinking of the open caisson. The coordinates of points 
within the reference subset are mapped to points 
in the deformed image using the following relationship,
 |
18 |
where 
and 
denote the distances in the u and v axis from the subset’s center to point 
. U and V represent the integer pixel displacement in the u and v directions. 
and 
are sub-pixel displacement. 
are the first-order displacement gradients for the reference subset.
Substituting Eq. (18) into Eq. (7), the following equation can be obtained,
 |
19 |
By disregarding second-order and higher-order terms, Eq. (19) can be approximated using a Taylor expansion centered at 
, resulting in
 |
20 |
where 
is unknown. 
and 
are the u and v directional spatial gradients of the deformed image at location 
For the n
points within the subset, Eq. (20) can be solved directly using a conventional linear least-squares method at a very fast speed. However, the displacement measurement accuracy can be increased considerably using an iterative least-squares algorithm.
Here the RGB sensor in the RGB-D camera is used to take images for different time (see Figs. 6 and 7). the image from an earlier time was selected as the reference image (e.g., image A in Fig. 6). Next, a subset containing the measurement points was selected in the reference image. Later, the subset in deformed images should be pinpointed (e.g., image B in Fig. 7) via the linear intensity change model to obtain the sinking information U, V. Note that here U, V are pixel displacements. Using the calibration parameter, one can easily obtain the sinking displacements 
of the open caisson.
Fig. 6.
Obtaining image A at time 
.
Fig. 7.
Obtaining image B at time 
.
Note that the open caisson is made of concrete, which has a low tensile strength. During construction and the settlement process of the open caisson, deformation or cracks are inevitably produced. In fact, the open caisson is not a rigid body. In order to improve the monitoring accuracy, one can use multiple cameras to monitor different locations of each wall of the open caisson. The sinking displacements from the measurement points can be used to determine the displacements of other points on the open caisson via an interpolation function. In addition, the settlement process of the open caisson is complicated, such as displacement, rotation and other situations may occur. In order to accurately capture these caissons, multiple cameras need to be set up for monitoring each wall.
When the sinking information is obtained, the spatial location of the open caisson can be updated. To clearly display the spatial location, one adds the sinking displacements and the out-plane changes obtained from the IR sensor to the 3D points that are obtained before, which has practical values for guiding the construction.
Experimental evaluation
Flowchart of the application
The implementation of the proposed strategy is summarized by the flowchart in Fig. 8. Firstly, one builds the geometric model of the open caisson via the 3D coordinates. Then, the sinking information can be monitored via tracking measurement points. Next, the sinking displacements and out-plane information were used to update the 3D model for display. Later, one can compare the updated 3D model with the designed model and judge whether the designed condition is satisfied. If the designed condition is not satisfied, advice for constructing will be given. For example, the settlement displacement of the open caisson is directly related to the excavation volume. If one side of the open caisson sinks too much, it indicates that the excavation volume on the opposite side is relatively small. Therefore, based on the different settlement displacements at different locations, it can be determined where to increase excavation and where to reduce excavation. The spatial location of the open caisson will be constantly monitored until the designed condition is satisfied.
Fig. 8.
Flowchart of the application.
Analysis and results
Firstly, the RGB-D camera via checkerboard should be calibrated for the parameters as follows,
 |
Then, the 3D geometric model of the open caisson was reconstructed. For example, the 3D coordinates of points on the wall (see Fig. 9) in the IR camera coordinate system was generated. When one obtains the 3D coordinates of all the walls of the open caisson via several IR sensors, one should transform them into the open caisson coordinate system.
Fig. 9.
3D wall of open caisson.
Later, one can use images taken by the RGB sensor at a different time during the sinking (see Fig. 10) to compute the sinking displacements. Note that the images are generated via a virtual camera. In this case, the sinking displacements are known in advance. After the sinking displacement was obtained, one can compare the results with the real displacements (see Figs. 11 and 12). By comparison, one can find that the predicted results are very close to the real displacements. The relative error of horizontal displacement is less than 
while that of vertical displacement is less than 
Fig. 10.
Images obtained by the RGB sensor at a different time.
Fig. 11.
Vertical sinking in-plane displacement (unit: m).
Fig. 12.
Horizontal sinking in-plane displacement (unit:m).
After the sinking displacement of one certain measurement point on a RGB image was obtained, the out-plane displacement of this measurement point should be calculated. The mapping between 3D points on the wall and points on the RGB image was constructed. The mapping between 3D points on the wall and points on the IR image was constructed. Thus, one can build a mapping between points on RGB images and points on IR images. From this new mapping, one can obtain the coordinates of the measurement points on a IR image. The difference of pixel values of the IR images at the calculated coordinates during a different time are actually the out-plane displacement.
Conclusion
This study proposed a computer vision-based approach for 3D reconstruction and displacement estimation of the open caisson during the sinking. By integrating the RGB-D camera and the linear intensity change model, the proposed approach can monitor the open caisson during construction. The performance of the proposed approach was validated via experiment. The experimental results show that the 3D coordinates of points on the open caisson can be generated via IR camera and the predicted results via the proposed approach are very close to the real displacements. The relative error of horizontal displacement is less than 
while that of vertical displacement is less than 
.
Acknowledgements
This work was financially supported by Academician Innovation Platform Project of Hainan Province (No.YSPTZX202320), Hainan Innovation Center for Academician Team, and the National Natural Science Foundation of China (Grant No.12262012).
Author contributions
Q.Wang: conceptualization, experimental design, data collection, editing, writing. Q.Li: conceptualization, experimental design, data collection, editing, writing. Y.Zhang: conceptualization, experimental design, validation, editing, writing, project administration. All authors reviewed the manuscript.
Data availability
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Competing interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Jiang, B. et al. Analysis of mechanism of sand gushing and sudden sinking of ultra-large deep-water open caisson in sand via field and model tests. Appl. Ocean Res.130, 103436 (2023). [Google Scholar]
- 2.Yang, B. et al. Study on distribution of sidewall earth pressure on open caissons considering soil arching effect. Sci. Rep.13, 10657 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Lai, F., Liu, S., Li, Y. & Sun, Y. A new installation technology of large diameter deeply-buried caissons: practical application and observed performance. Tunn. Undergr. Space Technol.125, 104507 (2022). [Google Scholar]
- 4.Sheil, B., Royston, R. & Byrne, B. Real-time monitoring of large-diameter caissons. In Proceedings of China-Europe Conference on Geotechnical Engineering 725–729 (Springer, 2018).
- 5.Zhang, Z., Leong, E. C., Gong, W., Dai, G. & Li, J. Foundation design of single-span arch bridges in china. Case Stud. Constr. Mater.19, 02239 (2023). [Google Scholar]
- 6.Faustin, N., Elshafie, M. & Mair, R. Case studies of circular shaft construction in London. Proc. Inst. Civ. Eng. Geotech. Eng.171, 391–404 (2018). [Google Scholar]
- 7.Zhang, X. et al. Study on the design calculation method and construction control method of suction drum foundation for guided frame platform. Sci. Rep.14, 1405 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Dong, X., Guo, M. & Wang, S. Active control method for the sinking of open caissons: a data-driven approach based on cnn and time series prediction. Ocean Eng.257, 111683 (2022). [Google Scholar]
- 9.Jiang, B., Wang, M., Chen, T., Zhang, L. & Ma, J. Experimental study on the migration regularity of sand outside a large, deep-water, open caisson during sinking. Ocean Eng.193, 106601 (2019). [Google Scholar]
- 10.Wu, L., Qian, J., Xie, L. & Qin, Y. Dem. modeling direct shearing behavior of sand considering anti-rotation of particle. Sci. Rep.13, 8941 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Chen, G., Cheng, X., Chen, W., Li, X. & Chen, L. Gps-based slope monitoring systems and their applications in transition mining from open-pit to underground. Int. J. Min. Miner. Eng.5, 152–163 (2014). [Google Scholar]
- 12.Iwasaki, T., Takechi, K., Takeishi, A. et al. Web-based displacement monitoring system using gps for the maintenance of roadside slopes. In The 6th International Symposium on Field Measurements in Geomechanics, FMGM03, Oslo 137–143 (2003).
- 13.Jianpeng, W., Jingxiang, G., Chao, L. & Jian, W. High precision slope deformation monitoring model based on the gps/pseudolites technology in open-pit mine. Min. Sci. Technol. (China)20, 126–132 (2010). [Google Scholar]
- 14.Kien, N. T., Nakashima, S. & Shimizu, N. Displacement monitoring using gps at an unstable steep slope and the performance of a new low-cost gps sensor. In Geotechnics for Sustainable Infrastructure Development 1197–1202 (Springer, 2020).
- 15.Jatmiko, J. & Psimoulis, P. Deformation monitoring of a steel structure using 3d terrestrial laser scanner (tls). In Proceedings of the 24th International Workshop on Intelligent Computing in Engineering, Nottingham, UK 10–12 (2017).
- 16.Collins, B. D. & Sitar, N. Application of high resolution 3d laser scanning to slope stability studies. In 39th Annual Symposium on Engineering Geology and Geotechnical Engineering, Butte, Mont (2004).
- 17.Fan, J. et al. Monitoring and analyzing mountain glacier surface movement using sar data and a terrestrial laser scanner: a case study of the himalayas north slope glacier area. Remote Sens.11, 625 (2019). [Google Scholar]
- 18.Liu, X., Wang, Q., Wang, Y. & Dong, Q. Review of calibration strategies for discrete element model in quasi-static elastic deformation. Sci. Rep.13, 13264 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Stark, T. D. & Choi, H. Slope inclinometers for landslides. Landslides5, 339 (2008). [Google Scholar]
- 20.Matsuda, H. & Shimizu, N. Assessment of rock slope stability based on the three-dimensional continuous displacement monitoring by global positioning system. Int. J. JCRM2, 17–20 (2006). [Google Scholar]
- 21.Mancini, S. M., Township, C. & Penxia, J. M. Monitoring verticality of a sinking casson (2011). US Patent 7,921,573.
- 22.Dong, X., Guo, M. & Wang, S. Inclination prediction of a giant open caisson during the sinking process using various machine learning algorithms. Ocean Eng.269, 113587 (2023). [Google Scholar]
- 23.Jiang, C. et al. Influence of external insert angle on structural deformation of long pipe shed with shallow hole. Sci. Rep.13, 14141 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Nigussie Habte, A. & Eshagh, M. Combination of psinsar and gps to estimate three-dimensional crustal displacements over the afar region. J. Afr. Earth Sci.209, 105119 (2024). [Google Scholar]
- 25.Ali, M. Z., Chu, H. T. & Burbey, T. J. Spatio-temporal estimation of monthly groundwater levels from gps-based land deformation. Environ. Modell. Softw.143, 105123 (2021). [Google Scholar]
- 26.Devoti, R., Zuliani, D., Braitenberg, C., Fabris, P. & Grillo, B. Hydrologically induced slope deformations detected by gps and clinometric surveys in the cansiglio plateau, southern alps. Earth Planet. Sci. Lett.419, 134–142 (2015). [Google Scholar]
- 27.Xin, W., Pu, C., Liu, W. & Liu, K. Landslide surface horizontal displacement monitoring based on image recognition technology and computer vision. Geomorphology431, 108691 (2023). [Google Scholar]
- 28.Fan, H. et al. Rockfill material segmentation and gradation calculation based on deep learning. Case Stud. Constr. Mater.17, 01216 (2022). [Google Scholar]
- 29.Ren, B., Wang, H., Wang, D., Guan, T. & Zheng, X. Vision method based on deep learning for detecting concrete vibration quality. Case Stud. Constr. Mater.18, 02132 (2023). [Google Scholar]
- 30.Liu, T. et al. Identification and analysis of seashells in sea sand using computer vision and machine learning. Case Stud. Constr. Mater.18, 02121 (2023). [Google Scholar]
- 31.Lai, Y. et al. Framework for long-term structural health monitoring by computer vision and vibration-based model updating. Case Stud. Constr. Mater.16, 01020 (2022). [Google Scholar]
- 32.Forsyth, D. & Ponce, J. Computer Vision: A Modern Approach (Pearson, 2012).
- 33.Pan, B., Asundi, A., Xie, H. & Gao, J. Digital image correlation using iterative least squares and pointwise least squares for displacement field and strain field measurements. Opt. Lasers Eng.47, 865–874 (2009). [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.