Efficient and accurate projector calibration method based on phase-shifting method with sinusoidal structured light

Abstract

Objective

Projector calibration is one of the most basic and important steps in structured light three-dimensional measurement. However, such as complicated calibration process and low accuracy are still the inevitable problems in the calibration process. In this paper, a projector calibration method based on phase-shifting method with sinusoidal structured light is proposed to improve the accuracy of the calibration and simplify the operation of the calibration.

Methods

Firstly, projecting a set of sinusoidal fringes onto a black-and-white circular calibration board, and collecting the images synchronously by a CCD camera; Secondly, establishing the correspondence between camera pixels and projector pixels by using the consistency of phase information which means that the absolute phase value of an arbitrary point on the calibration board is equal to the absolute phase value of this point in the camera image and virtual projector image; And then a set of virtual DMD images on the projector can be obtained by using the principle above; Finally, using this set of virtual DMD images, the parameters of the projector can be calibrated with camera calibration method based on circular calibration board.

Results

The experimental results show that the maximum reprojection error of the projector calibrated by this method is 0.0419 pixels, and the average reprojection error is 0.0343 pixels. The equipment involved in the calibration process is simple and the experimental operation is easy. The experimental results indicated that this method has high calibration accuracy and efficiency.

1. Introduction

Typical computer vision is generally divided into passive vision measurement based on binocular stereo vision and active vision measurement based on structured light vision [1]. Stereo matching is a difficult and inevitable problem in binocular vision. An effective method to avoid stereo matching is to use structured light vision [2]. Structured light three-dimensional measurement technology has the advantages of non-contact, fast measurement speed and high accuracy. In recent years, it has been widely used in physical modeling design, robot navigation, object recognition, reverse engineering, preservation and restoration of cultural relics and quality inspection [3]. The basic components of structured light 3D measurement system are computer, projector and camera. System calibration is one of the most important factors that affect the accuracy of 3D measurement. Camera calibration and projector calibration are the most basic and key procedures of the system work, and their calibration accuracy directly affects the measurement accuracy of the entire system.

At present, camera calibration methods and technologies are relatively mature [[4], [5], [6]]. In 1987, Tsai proposed the typical two-step method [7], which has good calibration accuracy and calibration speed and is widely used. In 1992, Stephen et al. proposed a theory of self-calibration of a moving camera [8], which does not require calibration board and has strong flexibility. In 2004, Zhang Zhengyou proposed a method for camera calibration with one-dimensional objects [9], which filled up the missing dimensions in the calibration method.

However, due to the projector’s inability to take photos, calibrating the projector becomes relatively difficult. It is hard to obtain the relationship between the object’s position in space and its corresponding position mapped to the DMD of projector directly. Moreover, the projector calibration still has difficulties, such as low calibration accuracy, complex calibration algorithm, large computation and strict requirements for calibration equipment. Therefore, it is necessary to find a simple and reasonable projector calibration method with the advantages of precision and cost.

The existing projector calibration methods can be divided into two categories according to whether they depend on the camera calibration results: one is to calibrate the camera first, and then use the camera calibration parameters to calibrate the projector. This method has a strong dependence on the camera calibration accuracy. In 2008, FALCAO et al. proposed plane-based calibration of a projector-camera system [10]. It is inevitable to bring errors into projector calibration because of depending on camera calibration parameters. In 2012, Ma et al. proposed to use a three-dimensional reconstruction method based on nonlinear iterative optimization [11] to correct errors caused by lens distortion, so as to reduce the impact of lens distortion on contour extraction. The other is to make the projector have the “photographing function” by establishing the corresponding relationship between the projector image and the camera image without depending on the camera calibration parameters, so that the projector can be calibrated using the camera calibration method. In 2012, in the projector calibration method [12] proposed by Anwar et al., the projector was regarded as a reverse camera and calibrated with the camera calibration method. In 2013, Chien et al. proposed a two-stage calibration method [13]. This method needs to assign an initial parameter to the projector, and then continuously modify this parameter by acquiring calibration patterns which generated dynamically. In 2014, Li et al. proposed a calibration method for out-of-focus projectors [14], which requires the virtual creation of a one-to-one mapping between camera pixels and projector pixels in the phase domain. In 2020, the projector calibration method [15] proposed by Tang et al. did not use the phase-shifting method. This method needs to project a black-and-white checkerboard pattern to the calibration board. In 2021, the projector calibration method [16,17] proposed by Yu et al. needs to ensure that the optical axis of the camera is perpendicular to the object plane.

This paper presents a calibration method of projector with high efficiency and high accuracy. First, projecting a set of sinusoidal fringes which have the same period to the black-and-white circular calibration board, and the sinusoidal fringes include four horizontal fringes and four vertical fringes. Then, using the camera to collect images synchronously. With image processing and phase unwrapping algorithm, establishing the pixel correspondence between the projector and the camera. Finally, regarding the projector as a reverse camera model and then using the camera calibration method based on circular calibration board proposed by our research group [18] to calibrate the projector. Zhang’s method in literature [19] used a red-and-blue checkerboard for calibration. In contrast, this method uses a standard black-and-white circular calibration board. The use of circular calibration board improved the calibration accuracy, while avoiding the use of non-standard calibration board can simplify the calibration steps. Moreover, compared with Ji’s double four-step phase-shifting method, the four-step phase-shifting method of single period fringes not only reduced the number of projection pictures and shortened the image processing time, but also had good accuracy and stability. Significantly, using phase-shifting method to calibrate the projector has better calibration accuracy than Tang’s method which based on checkerboard mapping. The method proposed in this paper does not need to ensure that the optical axis of the camera is perpendicular to the object plane as in literature [16,17], which greatly reduced the accuracy requirements of the experimental equipment. The precision does not depend on the accuracy of the camera calibration, and there is no secondary error propagation. It has the advantages of simple calibration equipment and high calibration accuracy.

2. Projector calibration

2.1. Projector model

From the perspective of optical imaging of the projector, due to the reversibility of the light path, the projector can be regarded as a reverse camera, so the projector can be described by the camera’s pinhole model [20,21]. Fig. 1 shows a typical diagram of a pinhole camera model. {Ow; Xw, Yw, Zw} is the world coordinate system which describes the camera position. {Oc; Xc, Yc, Zc} is the camera coordinate system whose origin of coordinates is the optical center. {O; x, y} is the image coordinate system whose origin of coordinates is the midpoint of the image. {o; u, v} is the pixel coordinate system whose origin of coordinates is the upper left corner of the image. P is a point in the world coordinate system and p is the imaging point of P in the image. f is the focal length of the camera.

Fig. 1.

The pinhole optical model of camera.

Since the projector can be regarded as a reverse “camera”, the mathematical model of the projector can be represented by a mathematical model similar to the camera as Equation (1):

$$s\tilde{\mathbf{m}}=\mathbf{A}\left[\begin{array}{cc}\mathbf{R}& \mathbf{T}\end{array}\right]\tilde{\mathbf{M}}$$ (1)

s is the scaling factor, $\tilde{\mathbf{m}}=\left[\begin{array}{ccc}u& v& 1\end{array}\right]$ is the pixel coordinate of a point on the projector object plane, A is the internal parameter matrix of the projector, R and T are the rotation matrix and translation vector of the world coordinate system relative to the projector coordinate system which are the external parameter matrices of the projector, $\tilde{\mathbf{M}}=\left[\begin{array}{cccc}X& Y& Z& 1\end{array}\right]$ represents the world coordinate of a point in projector space. The internal parameter matrix A can be expressed as Equation (2):

$$\mathbf{A}=\left[\begin{array}{ccc}{f}_x& \gamma & u_{\mathit{0}}\\ \mathit{0}& f_y& v_{\mathit{0}}\\ \mathit{0}& \mathit{0}& \mathit{1}\end{array}\right]$$ (2)

fx = f/dx and fy = f/dy are the focal length of the lens that measured by the number of pixels, f is the physical focal length of the lens, dx and dy are the physical dimensions of each pixel in the u and v directions. $\gamma$ represents the angular deviation between the u axis and v axis of the projector image plane, and (u₀,v₀) is the coordinate of the principal point of the projector which is the coordinate of the intersection of the projector optical axis and the image plane.

Nevertheless, the model of actual projector is not the same as the ideal pinhole model completely. Due to the errors in the manufacturing and installation process, the lens usually has optical distortion which brings large errors to calibration and measurement. Radial distortion and tangential distortion are included mainly.

Radial distortion which includes barrel distortion and pillow distortion has the following correction formulas as Equation (3):

$$\{\begin{array}{c}{x}_{corrected}=x\left(1+k_1{r}^2+k_2{r}^4+k_3{r}^6\right)\\ y_{corrected}=y\left(1+k_1{r}^2+k_2{r}^4+k_3{r}^6\right)\end{array}$$ (3)

Tangential distortion which includes thin lens distortion and centrifugal distortion has the following correction formulas as Equation (4):

$$\{\begin{array}{c}{x}_{corrected}=x+\left(2p_{1}y+p_2\left(r^2+2x^2\right)\right)\\ y_{corrected}=y+\left(2p_{2}x+p_1\left(r^2+2y^2\right)\right)\end{array}$$ (4)

where, r² = x² + y², k₁, k₂ and k₃ are radial distortion factors, p₁ and p₂ are tangential distortion factors, and the distortion factors of the lens are also considered as the internal parameters. In general, the distortion factors of the lens are described by a distortion matrix D = [k₁ k₂ p₁ p₂ k₃].

2.2. The algorithm of circle center extraction

In the camera calibration method based on circular calibration board, the circle centers are used as feature points to calibrate the camera. Because of the advantage of obtaining the center coordinates from thousands of edge points of an ellipse, the influence of image noise on edge extraction is reduced. The proposed calibration algorithm is more accurate than many previous calibration algorithms. A method for extracting feature points of circular calibration board includes the following steps:

• Step 1: As shown in Fig. 2, design and make a circular calibration board which contains 7 × 7 black circle marks. The distance between two adjacent circles in the horizontal and vertical directions is 12 mm and the diameter of the circle is 6 mm;

• Step 2: As shown in Fig. 3(a), place the calibration board within the field of view of the CCD camera and collect the image of the circular calibration board;

• Step 3: Implement image binaryzation by adjusting the threshold value;

• Step 4: Use the findContours function in OpenCV to extract all contours in the image;

• Step 5: For each ellipse contour, use the least square fitting algorithm to fit the ellipse, and solve its central coordinates, major axis length and minor axis length;

• Step 6: According to the relatively small distortion of the CCD camera, the ratio of the major axis to the minor axis of the ellipse in the collected image is approximately equal to 1, and the ellipses whose ratio of the major axis to the minor axis is greater than 4/3 are eliminated;

• Step 7: According to the feature that the designed and manufactured calibration board ellipses have the same size, the axes of all ellipses in the image have little difference, and the ellipses whose major axis length or minor axis length are less than 3/4 or greater than 5/4 in the remaining ellipses are eliminated;

• Step 8: Set the rectangular box with 1/4 axis length as side length as the decision range;

• Step 9: Use the mouse cursor to manually point out the center of the ellipse on the image;

• Step 10: Take the light punctuation point as the center of the determination range, and find the center coordinate within the range in the variable containing the center coordinate of the ellipse;

• Step 11: Calculate the relative coordinates of the centers of all feature points according to the center coordinates of the four corners and the circular position distribution of the calibration plate and use the determination range to judge the corresponding serial numbers of all centers;

• Step 12: Finally, as shown in Fig. 3(b), obtain the coordinates of image feature points and circle center extraction is completed.

Fig. 2.

The circular calibration board.

Fig. 3.

The calibration board in certain position and its circle center extraction.

2.3. Phase-shifting method

It is very difficult to obtain phase value with high precision by phase-shifting method through just a frame of fringes, so it is necessary to use phase-shifting algorithm to obtain the phase value accurately. The phase-shifting method includes two-step phase-shifting method, three-step phase-shifting method, four-step phase-shifting method, etc. Generally speaking, more steps of phase-shifting method could bring better stability, stronger reliability and higher accuracy but more time cost. Based on the consideration of stability, anti-interference performance and time cost, three-step and four step phase-shifting method are most commonly used at present.

Generating a set of sinusoidal fringes by computer. The sinusoidal structured light means that the light intensity of projected structured light conforms to the sinusoidal distribution. And the function of light intensity distribution is expressed as Equation (5):

$$I_n=A\left(x,y\right)+B\left(x,y\right)\cos \left[\phi \left(x,y\right)+{\phi }^{\prime }\right]$$ (5)

A fringe image captured by the camera can be represented by I_n, and the subscript n refers to the order of fringe images. A(x, y) is the light intensity of background, B(x, y) is the modulated light intensity, φ(x,y) is the phase of the fringe, and the φ′ is the amount of phase shift. The phase-shifting method requires that the total phase shift amount of n images is 2π and that means the amount of phase shift of each image is φ′ = 2π/n.

The amount of phase shift of each image in four steps is φ′ = π/2. The light intensity expression of the standard four-step phase-shifting method is

$$I_1=A\left(x,y\right)+B\left(x,y\right)\cos \left[\phi \left(x,y\right)+0\right]$$

$$I_2=A\left(x,y\right)+B\left(x,y\right)\cos \left[\phi \left(x,y\right)+\pi /2\right]$$

$$I_3=A\left(x,y\right)+B\left(x,y\right)\cos \left[\phi \left(x,y\right)+\pi \right]$$

$$I_4=A\left(x,y\right)+B\left(x,y\right)\cos \left[\phi \left(x,y\right)+3\pi /2\right]$$

Follows can be obtained after simplification

$$I_1=A\left(x,y\right)+B\left(x,y\right)\cos \phi \left(x,y\right)$$

$$I_2=A\left(x,y\right)-B\left(x,y\right)\sin \phi \left(x,y\right)$$

$$I_3=A\left(x,y\right)-B\left(x,y\right)\cos \phi \left(x,y\right)$$

$$I_4=A\left(x,y\right)+B\left(x,y\right)\sin \phi \left(x,y\right)$$

According to the formulas above, expression could be shown as Equation (6):

$$\phi \left(x,y\right)=\arctan \frac{I_4-I_2}{I_1-I_3}=\arctan \frac{\sin \phi \left(x,y\right)}{\cos \phi \left(x,y\right)},\phi \left(x,y\right)\in \left[-\pi ,\pi \right]$$ (6)

The range of phase’s principal value is [−π,π]. After that, the continuous phase value of the entire fringe area can be obtained through the phase unwrapping algorithm. Fig. 4(a and b) shows two sets of sinusoidal fringes generated by the four-step phase-shifting method. The phase shift amount of each group is π/2, and there are four horizontal fringes and four vertical fringes respectively.

Fig. 4.

Sinusoidal fringe patterns of four-step phase-shifting method.

2.4. The relationship of the phase

The relationship between the camera and the calibration board can be established by shooting the standard calibration board. The difficulty of projector calibration is how to obtain the coordinate correspondence between the projector and the calibration board, because the projector cannot “shoot”, but can only project the image. In order to solve the problems above, we adopt a projector calibration method that does not rely on camera calibration parameters. We treat the calibration board as a bridge between the camera and the projector to establish the corresponding relationship between the projector image and the camera image. This kind of corresponding relationship makes the projector have a “photographing function”, so that the camera calibration method can be used to calibrate the projector. Two sets of generated sinusoidal fringes are projected to the calibration board, and then the camera is used to take this set of pictures at the same time. The point in the sinusoidal fringe which generated by computer, projected by projector and captured by CCD camera has the same absolute phase value when considering the consistency of phase information. When taking advantage of the phase information, the pattern of the calibration board taken by the camera can be mapped to the corresponding pattern on the DMD element of the projector, as if the projector “shoots” a set of calibration patterns. Finally, the parameters of the projector can be calibrated by using the camera calibration method based on circular calibration board.

As shown in Fig. 5, outputting the generated sinusoidal fringe pattern to the DMD plane, and then projecting it onto the calibration board on the workbench through the lens. The fringes need to cover all feature points on the calibration board. Adjusting the position of the camera so that the view of camera can cover the entire projection area, which means the camera should be able to capture all fringes. Point A is an arbitrary point on the calibration board, point B is the virtual corresponding point of point A mapped to the DMD plane of the projector, and point C is the corresponding point of point A mapped to the CCD plane of the camera. It can be seen from the consistency of fringe phase information that the absolute phase values of points A, B and C are equal, which can be expressed as Equation (7):

$${\phi }_A={\phi }_B={\phi }_C$$ (7)

Fig. 5.

The principle of the calibration method.

Based on the above principle, a calibration board without fringes is photographed with a camera, and the center coordinates are calculated by using the center extraction algorithm. Then the phase information on the CCD image plane is obtained by using the phase shifting method, and then the horizontal phase value and vertical phase value of the circle center can be obtained. When the horizontal and vertical phase values are known, the only corresponding point on the DMD plane can be found. By analogy, a group of virtual DMD pictures for projector calibration can be obtained.

2.5. The process of the projector calibration

The projector calibration method is based on the four-step phase-shifting method with single-period sinusoidal fringes. The calibration board with dot is chosen. There are 7 × 7 black dot marks. The distance between two adjacent circles in the horizontal and vertical directions is 12 mm and the diameter of the dot is 6 mm. Preliminary preparations also included the use of a computer to generate a set of single-period sinusoidal fringes, including four vertical fringes and four horizontal fringes. The phase difference between the four vertical fringes is π/2, and the phase difference between the four horizontal fringes is also π/2. And the phase of the generated fringes needs to be extracted and unwrapped.

Fig. 6 shows the projector calibration process. The specific calibration steps are as follows:

• Step 1: Place the calibration board at a certain position, and make sure that all feature points on it are completely covered by the fringes and within the camera’s field of view;

• Step 2: Capture a calibration board image with a CCD camera and obtain the pixel coordinates of the center of the circle through the ellipse fitting algorithm [22];

• Step 3: Project the generated vertical and horizontal single-period sinusoidal fringes onto the calibration board and shot synchronously with the CCD camera. The pictures must cover all sinusoidal fringes;

• Step 4: Phase extraction and phase unwrapping with Goldstein Brunch-cut algorithm are performed on the fringe pattern collected by the CCD camera, and the corresponding phase value data matrix is obtained;

• Step 5: The horizontal phase and vertical phase of the center in the CCD image are obtained from the center pixel coordinates, and then the virtual center coordinates in the generated fringe pattern are obtained from the corresponding horizontal phase and vertical phase of the center. After that, a virtual DMD image is obtained for projector calibration;

• Step 6: The calibration board should be tilted, rotated and translated in a small range within the allowed range. Repeat the above steps and generate a set of virtual DMD images using the calibration board images of different attitudes to achieve the “photographing function” of the projector. The projector is then calibrated using the camera calibration method based on circular calibration board.

Fig. 6.

The process of the projector calibration.

3. Experiment and result analysis

In order to verify the effect of the projector calibration method based on structured light phase-shifting method in reality, a projector calibration system was built in the laboratory. The calibration system consists of Lenovo C113 projector with a maximum resolution of 1280 × 1024 pixels, DH-HV1351UM-M industrial CCD camera with a resolution of 1280 × 1024 pixels from DAHENG IMAGE, 8 mm lens, computer and several optical platform fixings. The experiment system is showed in Fig. 14.

Fig. 14.

The experimental system.

As shown in Fig. 2, a black-and-white circular calibration board with 7 × 7 black dots was printed for the experiment. Fig. 7(a–d) shows four images of the calibration board covered with vertical fringes which is collected by the CCD camera synchronously. As shown in Fig. 9(a and b), the phase of the images is extracted by four-step phase-shifting method. It can be seen that the range of the phase value is [-π,π] and the value is discontinuous. Fig. 10(a and b) shows the results of phase unwrapping of the collected images, which can be seen that it changes continuously. Fig. 8(a–d), Fig. 11(a and b) and Fig. 12(a and b) correspond to the acquisition map, extracted phase map and unwrapped phase map of horizontal fringes.

Fig. 7.

Calibration board covered by vertical fringes.

Fig. 9.

Extracted phase of the vertical fringes.

Fig. 10.

Unwrapped phase of the vertical fringes.

Fig. 8.

Calibration board covered by horizontal fringes.

Fig. 11.

Extracted phase of the horizontal fringes.

Fig. 12.

Unwrapped phase of the horizontal fringes.

By adjusting the position of the calibration board, 12 calibration pictures of different positions were collected, and finally 12 virtual DMD pictures of different positions were obtained for projector calibration. As shown in Fig. 13, Fig. 13(a) is the camera picture of the calibration board at a certain position and Fig. 13(b) is corresponding projector virtual DMD picture. Table 1 describes the data obtained from the calibration board at this position after the above processing. The first column is the center number of 49 circles, the second and third columns are the pixel coordinates of the center obtained through the center extraction algorithm, the fourth and fifth columns are the phase values corresponding to the center points, and the sixth and seventh columns are the pixel coordinates of the center points corresponding to the virtual points on the DMD plane of the projector.

Fig. 13.

Camera picture and its corresponding virtual DMD picture.

Table 1.

The data of the circle centers.

Center number	x/pixel	y/pixel	horizontal phase	vertical phase	u/pixel	v/pixel
1	246.78525	368.93753	32.28020314	100.6414031	237.9835258	383.0648046
2	292.49509	352.246	36.42874411	89.33056706	283.1687158	366.505324
3	338.21613	335.63974	40.42234537	77.90314396	328.8070619	350.5375623
4	384.21957	319.09341	44.263638	66.35687894	374.9329399	335.2099535
5	430.30594	302.54102	48.4663635	54.64743951	421.7221721	318.4265411
6	476.35907	286.04907	52.54769665	42.86804511	468.7660048	302.1035946
7	522.67175	269.5047	56.68372908	30.90896992	516.532157	285.5930496
8	263.53693	414.95212	22.00015398	96.56376406	254.2765264	424.1223505
9	309.1694	398.44034	26.24993444	85.38627425	298.9301944	407.155949
10	354.90866	381.85855	29.89109017	74.01354698	344.3577916	392.6167872
11	400.80948	365.34454	33.75176864	62.40635498	390.7297448	377.2073391
12	446.98605	348.83319	37.90140194	50.58043021	437.9676185	360.6379009
13	492.95248	332.31964	41.82743895	38.695732	485.4307051	344.9498546
14	539.19806	315.8096	45.94945038	26.67886993	533.4242264	328.4644613
15	280.24161	461.01245	11.7158041	92.48040254	270.5862597	465.2246678
16	325.85538	444.57651	15.59928108	81.13473834	315.9183409	449.7030975
17	371.64447	428.14764	19.23266894	69.73631852	361.4398646	435.1930757
18	417.57651	411.59549	23.19192438	58.18576182	407.5672692	419.3799957
19	463.56232	395.08597	27.35917136	46.48544686	454.319188	402.7393287
20	509.5824	378.63724	31.31487663	34.61852689	501.7133645	386.9258677
21	555.73047	362.13928	35.23853144	22.48342868	550.1875624	371.2561485
22	297.01575	506.98328	1.19005364	88.66644091	285.820769	507.2567848
23	342.58328	490.53647	4.969009918	77.25970643	331.3896036	492.1771009
24	388.27396	474.22531	8.648400968	65.62622205	377.8647882	477.4749053
25	434.11484	457.69357	12.69246208	53.88161216	424.7638256	461.3165768
26	480.17154	441.26324	16.55720012	42.074463	471.9222966	445.872672
27	526.12103	424.84406	20.52416818	30.19041231	519.4102232	430.0310427
28	572.21649	408.36566	24.48345216	18.09260784	567.7310972	414.2158064
29	313.81415	552.86871	−9.241023365	84.6873108	301.7071941	548.932544
30	359.37384	536.52655	−5.678768442	73.20517357	347.5890264	534.7008976
31	404.97311	520.26215	−2.041669404	61.62192108	393.8533079	520.1631787
32	450.79166	503.78745	1.948404319	49.87262267	440.7737931	504.2366748
33	496.74924	487.35077	5.734754551	37.88653703	488.6757734	489.1226559
34	542.63477	470.9642	9.618135923	25.83853614	536.7899636	473.5857735
35	588.69397	454.59396	13.71512771	13.54813265	585.886534	457.2258807
36	330.60342	598.63507	−19.90088224	80.63896316	317.9020881	591.5112614
37	376.08224	582.3418	−16.42617489	69.12842534	363.8692144	577.6392516
38	421.68201	566.08197	−12.6312135	57.48266222	410.3879909	562.4589437
39	467.42776	549.80115	−8.747208937	45.72212769	457.3595442	546.9499749
40	513.36536	533.43158	−4.936228908	33.73261922	505.2621243	531.7297973
41	559.25842	517.07874	−1.216884429	21.61119428	553.6845669	516.8763487
42	605.24109	500.68991	2.9669603	9.352659693	602.6434724	500.1756827
43	347.40417	644.23212	−30.74178841	76.59154711	334.0548549	634.8322447
44	392.96893	628.16907	−27.25055536	64.96987165	380.469195	620.867033
45	438.36584	611.99298	−23.34226431	53.24835088	427.3068304	605.255643
46	484.27408	595.71057	−19.76620725	41.42345464	474.5342105	590.9733901
47	530.01056	579.44147	−15.95112627	29.38507387	522.6136012	575.7375246
48	575.75653	563.19604	−12.24142099	17.29574666	570.9150196	560.923087
49	621.672	546.83459	−8.323037439	4.901852905	620.4237004	545.255488

Moreover, in the conversion from camera image to virtual DMD image, the idea of equal-ratio transformation is used to refine the mapping relation between camera and projector pixels into the mapping relation between camera and projector sub-pixels, which greatly improves the calibration accuracy. The actual experiments show that the maximum reprojection error for projector calibration is 0.0419 pixels and the average reprojection error is 0.0343 pixels. The internal parameters of the projector are represented by the internal parameter matrix A, and the distortion coefficient is represented by the distortion matrix D. It can be seen that the projector calibration method based on four-step phase-shifting method with single period fringe has high accuracy in the actual projector calibration.

• Internal parameter matrix

$$\mathbf{A}=\left[\begin{array}{ccc}{f}_x& \gamma & u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]=\left[\begin{array}{ccc}2456.223309& 0& 623.200325\\ 0& 2422.034359& 456.887143\\ 0& 0& 1\end{array}\right]$$

• Distortion matrix

$$\mathbf{D}=\left[\begin{array}{ccc}{k}_1& k_2& \begin{array}{ccc}{p}_1& p_2& k_3\end{array}\end{array}\right]=\left[\begin{array}{cc}\begin{array}{ccc}-0.079395& 0.006016& 0.013040\end{array}& \begin{array}{cc}-0.00164& 2.350521\end{array}\end{array}\right]$$

As can be seen from Table 2, compared with Falcao’s method, the projector calibration method based on the phase-shifting method with structured light adopted in this paper does not depend on the results of camera calibration, thus avoiding the coupling of camera calibration errors. In contrast to Zhang’s method, a standard black-and-white circular calibration board is used, which not only avoids the inconvenience caused by non-standard calibration boards, but also improves the calibration accuracy. Although the four-step phase-shifting method requires two more images to be projected than Zhang’s three-step phase-shifting method, it also brings better stability and accuracy. Compared with Li [23]’s method, the method mentioned in this paper only needs to project 9 pictures, which is relatively reduced by 15 pictures and this has a great improvement of the calibration efficiency of the projector. Compared with Huang’s approach, it reduces the use of additional equipment and simplifies the experimental operation. Compared with the method of Ji [24], using single-period fringes instead of multi-period fringes during the projector calibration can not only avoid the change of period and number of the fringes, but also shorten the time for phase extraction and phase unwrapping. Considering all this, the projector calibration method based on the phase-shifting method with structured light adopted in this paper leads to higher calibration accuracy and higher calibration efficiency.

Table 2.

Comparison of projector calibration.

Method of calibration	Rely on camera calibration	Camera error	Projector error	Number of pictures projected	Additional equipment
			Mean error
Falcao’s method	Yes	0.5	0.8671	0	No
Zhang S’s method	No	0	0.2176	6	No
Li Z W’s method	No	0	0.312	24	No
Huang S J’s method	No	0	0.0704	24	Yes
JI H B’s method	No	0	0.1329	16	No
mine	No	0	0.0911	9	No

4. Conclusion

To address the complexity of the projector calibration process and the low calibration accuracy, a projector calibration method based on a phase-shifting method with structured light is proposed. A set of single-period sinusoidal fringe patterns in the horizontal and vertical directions were first generated by a computer based on the four-step phase-shifting method. Next, fringe patterns are projected sequentially onto a circular calibration board. Then, the CCD camera collects the images on the calibration board in synchrony. After that, the four-step phase-shifting method is used to extract and unwrap the phase information of the collected images, and the one-to-one correspondence between camera and projector pixels can be known. Finally, virtual DMD images for projector calibration can be obtained by the above principle. In contrast to Falcao’s approach, the proposed method does not depend on the parameters of the camera calibration and can avoid error coupling. It also does not require additional auxiliary equipment like Huang’s method, which simplifies experimental operations. Compared with the red-and-blue checkerboard used in Zhang’s method, the standard black-and-white circular calibration board in this method is used to avoid the inconvenience caused by non-standard calibration board and improve the calibration accuracy. Compared with Ji’s double four-step phase-shifting method, the four-step phase-shifting method of single-period fringe proposed in this paper can reduce the number of projected images and has great accuracy and stability. In real experiments, the maximum reprojection error of the projector calibrated in this way is 0.0419 pixels, and the average reprojection error is 0.0343 pixels. Experimental results show that the proposed method can achieve both high calibration accuracy and high calibration efficiency in practical projector calibration.

Author contribution statement

Wenguo Li: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data.

Yongpeng Zhong: Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Manli Tai; Tao Liu: Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data.

Data availability statement

Data will be made available on request.

Declaration of interest’s statement

The authors declare no conflict of interest.