1.

Introduction

High-speed three-dimensional (3-D) shape measurement has been an important technology in industrial manufacturing, such as quality inspection, reverse engineering, 3-D sensing, and object recognition.¹ To get accurate 3-D shape measurement results, many technologies have been researched, such as Moire technique,² phase-measuring profilometry (PMP),^3–6 and Fourier transform profilometry (FTP).^7–9

In general, it is significant to use fewer fringe patterns to get the results in a high-speed measurement.¹⁰ Because one-shot technologies only need one frame to get the wrapped phase, they have the potential to be used in high-speed measurement. It mainly contains color coding,¹¹ FTP,¹² and composite pattern;¹³ however, these methods still have shortcomings. Color coding is encoded with different colors. It is easily affected by the surface color.¹⁴ Fourier transform is insensitive to the color, but it needs the extract from the fundamental frequency component, so the geometry changes are required to be lower than the frequency of the fringe pattern.¹⁵ Composite pattern with different frequencies is projected and different frequencies are filtered to obtain the deformed patterns with different frequencies.¹⁶ However, the computational cost is high and the measurement accuracy needs to be improved. Another problem is that the wrapped phase needs to be unwrapped to obtain the absolute phase; however, the process of the phase unwrapping will also affect the accuracy of absolute phase and it will also increase computing time of the measurement.¹⁷

Traditionally, the wrapped phase is always retrieved based on temporal or spatial phase unwrapping. Temporal phase unwrapping needs multiple frames to get absolute phase that will affect the computation complexity.¹⁸ Spatial phase unwrapping process is a process of integral accumulation point-by-point scanning that will make error spread backward.¹⁹

To balance the accuracy and speed, this paper proposes a high-speed 3-D shape measurement method based on phase measurement profilometry and stereo vision. A three-step phase-shifting measurement only needs three frames-deformed fringe patterns to get the wrapped phase. Using original image matching constraint, a rough parallax is used in the phase matching. To get the accurate corresponding point, phase matching and subpixel parallax optimization are used to preclude the false point. Phase matching is used to confirm the candidate points. When the points are at the edge of wrapped phase where the phase value is $-\pi$ or $\pi$, there are missing points or wrong corresponding points. An average phase value is used to increase the robustness of the high-speed measurement. Subpixel parallax optimization is used to find the true corresponding point based on the coordinate of subpixel. In the process of measurement, the dithering fringe pattern is used to overcome the gamma effect.

This paper is organized as follows: in Sec. 2, the principle of a three-step phase-shifting measurement and the flow of the proposed method are introduced. Section 3 illustrates the principle of phase matching and subpixel parallax optimization. Section 4 verifies the feasibility and accuracy of the proposed method. Conclusion is given in Sec. 5.

2.

Principle

2.2.

Flow of the Proposed Method

The whole process of the proposed method is shown in Fig. 1. It mainly includes five steps.

• Step 1: Preparation before measurement. It includes the generation of dithering pattern and stereo vision calibration. The dithering fringe pattern is insensitive to gamma of projector, which uses an one-bit binary instead of an eight-bit gray information to approximate the sinusoidal fringe pattern. Dithering fringe pattern can be used for high-speed measurement without gamma calibration. Another merit of dithering fringe pattern is that it is suitable to measure objects when wide fringe pattern is used. In this paper, the dithering fringe pattern is generated based on Ref. 20.

• Step 2: Original image matching constraint. The original image can be captured from left and right cameras, respectively. Based on the feature of stereo vision, a rough matching can be implemented. It can be used to provide a rough parallax as a constraint condition.

• Step 3: Calculation of the wrapped phase. PMP is applied to the captured image from left camera and right camera, respectively. Then wrapped phase is obtained based on Eq. (4).

• Step 4: The wrapped phase matching. Based on the original image matching constraint, the wrapped phase matching can be performed to find the candidate points. It is difficult to match the points at the boundary of wrapped phase, where the phase is or $-\pi$ or $\pi$ so the average phase is used to detect the edge point.

• Step 5: Subpixel parallax optimization. To find the corresponding points precisely, the subpixel parallax optimization is used. The optimized parallax will correspond the target point. Once the stereo vision system is calibrated, the height of object can be calculated.

Fig. 1

The flowchart of the proposed method.

In this paper, steps 4 and 5 are the main proposed methods, so the principle of them will be introduced in the following part.

3.

Principle of the Proposed Method

3.1.

Phase Matching

Traditionally, wrapped phase cannot be used to find the corresponding points, because it ranges from $-\pi$ to $\pi$ with periodical change. Because dithering fringe pattern is used in this system, gamma effect of projector can be neglected. The original image matching constraint is to narrow the range of candidate points within the epipolar line. Without original image matching constraint, the false points will be considered and reconstructed, which will occupy the process time. Traditionally, absolute phase is required to find the correct corresponding points. The algorithm of retrieving absolute phase can be classified into the spatial method and temporal method. But both methods have demerits. Spatial method cannot be used to retrieve the phase of isolate objects and the phase error will spread along the direction of phase unwrapping. Temporal method needs multiple frames to unwrap the phase that will occupy the measurement speed. The process of wrapped phase matching without absolute phase is shown in Fig. 2. Because absolute phase is not used, there are some false corresponding points with the same phase value. These points can be defined as candidate points.

Fig. 2

The process of wrapped phase matching without absolute phase.

The target of matching is to find the corresponding points in two cameras. In the theory of stereo vision, the rough parallax can be obtained by making a difference between the twoimages that are captured from left camera and right camera, respectively. Then, the rough parallax is applied to the phase matching. Because structure light increases the texture features, it can confirm the corresponding points accurately. For the stereo vision system, the point $(x_L,y_L)$ in the left camera corresponds to the point $(x_R,y_R)$ in the right camera. The original image matching constraint provides a rough corresponding parallax Par, which can be expressed as

Eq. (5)

$$\mathrm{Par}=x_R-x_L.$$

The parallax is used in the wrapped phase to find the target phase. When the phase of point $(x_L,y_L)$ in the left wrapped phase is ${\text{Phase}}_L(x_L,y_L)$, the corresponding phase of right wrapped phase is ${\text{Phase}}_R(x_R,y_R)$, as shown in

Eq. (6)

$${\text{Phase}}_L(x_L,y_L){=\text{Phase}}_R(x_R,y_R){=\text{Phase}}_R(x_L+\mathrm{Par},y_R).$$

In the proposed method, the point $(x_R,y_R)$ and adjacent points are set to candidate points. Considering the computation complexity, adjacent candidate points are shown in Fig. 3.

Fig. 3

The relationship between the points in the left camera and right camera.

To find the true point, the phase of left point ${\text{Phase}}_L(x_L,y_L)$ subtracts the phase of these candidate points ${\text{Phase}}_R(x_R+s,y_R)$ and the absolute difference can be expressed as

Eq. (7)

$$\mathrm{\Delta }\text{Phase}=|{\text{Phase}}_L(x_L,y_L)-{\text{Phase}}_R(x_R+s,y_R)|,$$

where $s$ is an integer and $s\in [-\mathrm{2,2}]$. $\mathrm{\Delta }\text{Phase}$ is the difference between the point phase values. The least difference $\mathrm{\Delta }\text{Phase}(x_{R\_\min },y_{R\_\min })$ is used and $(x_{R\_\min },y_{R\_\min })$ is the coordinate of candidate point, which has the least phase difference.

Once the optimal point can be obtained, the parallax ${\mathrm{Par}}_{\min }$ can be calculated as

Eq. (8)

$${\mathrm{Par}}_{\min }=x_{R\_\min }-x_L.$$

Phase value matching belongs to pixel matching and the parallax of subpixel is required.

Although the phase matching can reject most candidates, there are still some challenges. When the points are at the edge of wrapped phase where the phase is $-\pi$ or $\pi$, it is difficult to find the matching points precisely based on $\mathrm{\Delta }\text{Phase}$. To find the corresponding points accurately, the average phase value is used. As shown in Fig. 4, the middle point $(x_R,y_R)$ is on the edge of wrapped phase and the phase ${\text{Phase}}_R(x_R,y_R)$ is required to be calculated. The average phase value ${\text{Phase}}_{\mathrm{ave}}(x_R,y_R)$ can be described as

Eq. (9)

$${\text{Phase}}_{\mathrm{ave}}(x_R,y_R)=\frac{1}{5}\sum _{i=-2}^2\text{phase}(x_R+i,y_R).$$

Fig. 4

The determination of boundary points based on the average phase. (a) Average phase is greater than zero and (b) average phase is smaller than zero.

The phase value is monotonic during a period. When the phase value ${\text{Phase}}_{\mathrm{ave}}(x_R,y_R)$ is greater than zero, it can be designed as $\pi$ or it can be designed as $-\pi$, which can be expressed as

Eq. (10)

$$\text{Phase}(x_p,y_p)=\{\begin{array}{ll}-\pi & \text{if}{\text{Phase}}_{\mathrm{ave}}<0\\ \pi & \text{if}{\text{Phase}}_{\mathrm{ave}}>0\end{array}.$$

3.2.

Subpixel Parallax Optimization

In the wrapped phase, the phase value ranges from $-\pi$ to $\pi$ and it changes periodically. Using this feature, the coordinate of subpixel can be obtained. If the coordinate of original image matching point is $(x_{R\_o},y_R)$, the coordinate of subpixel $(x_{R\_\mathrm{sub}},y_R)$ can be obtained. Because the phase value monotonically increases under every period, the coordinate of subpixel $(x_{R\_\mathrm{sub}},y_R)$ is between the best corresponding point $(x_{R\_\min },y_{R\_\min })$ and candidate point $(x_{R\_o},y_R)$. The phase value of the subpixel in the right-wrapped phase can be set to ${\text{Phase}}_R(x_{R\_\mathrm{sub}},y_R)$. Because ${\text{Phase}}_R(x_{R\_\mathrm{sub}},y_R)$ represents the best matching point in the wrapped phase, it should be equivalent to the phase ${\text{Phase}}_L(x_L,y_L)$ in the left wrapped phase. Then, it can be expressed as

Eq. (11)

$${\text{Phase}}_R(x_{R\_\mathrm{sub}},y_R)={\text{Phase}}_L(x_L,y_L).$$

As shown in Fig. 5, when the original image matching point $(x_{R\_o},y_R)$ is on the left of the best point $(x_{R\_\min },y_{R\_\min })$, it means the $x_{R\_o}<x_{R\_\min }$.

Fig. 5

The original image matching point and the points on the left of the best point.

The coordinate of the subpixel can be expressed as

Eq. (12)

$$x_{R\_\mathrm{sub}}=\frac{{\text{Phase}}_R(x_{R\_\mathrm{sub}},y_R)-{\text{Phase}}_R(x_{R\_o},y_R)}{{\text{Phase}}_R(x_{R\_\min },y_R)-{\text{Phase}}_R(x_{R\_o},y_R)}(x_{R\_\min }-x_{R\_o})+x_{R\_o}.$$

When the point $(x_{R\_o},y_R)$ has the same $x$-coordinate of the best point $(x_{R\_\min },y_{R\_\min })$, it means the $x_{R\_o}=x_{R\_\min }$. As shown in Fig. 6, the coordinate of the subpixel can be expressed as

Eq. (13)

$$x_{R\_\mathrm{sub}}=x_{R\_\min }.$$

Fig. 6

The original image matching point has the same $x$-coordinate of the best point.

Similarly, as shown in Fig. 7, when the original image matching point $(x_{R\_o},y_R)$ is on the right of the best point $(x_{R\_\min },y_{R\_\min })$, it means the $x_{R\_o}>x_{R\_\min }$.

Fig. 7

The original image matching point is on the right of the best point.

Then, the coordinate of the subpixel can be expressed as

Eq. (14)

$$x_{R\_\mathrm{sub}}=\frac{{\text{Phase}}_R(x_{R\_\mathrm{sub}},y_R)-{\text{Phase}}_R(x_{R\_\min },y_R)}{{\text{Phase}}_R(x_{R\_o},y_R)-{\text{Phase}}_R(x_{R\_\min },y_R)}(x_{R\_o}-x_{R\_\min })+x_{R\_\min }.$$

Then, the subpixel parallax ${\text{ParPhase}}_{\mathrm{sub}}$ can be obtained as

Eq. (15)

$${\text{ParPhase}}_{\mathrm{sub}}=x_{\mathrm{sub}}-x_L.$$

Based on the baseline and subpixel parallax map, the height of the object can be calculated.

4.

Experiments

To verify the proposed method, a 3-D shape measurement system is developed. The system contains a projector (Samsung SP-P310MEMX) and two digital CCD cameras (Daheng MER-500-14U3M/C-L). The camera is attached with a 16-mm focal length lens (Computar M1614-MP). The camera resolution is $1024\times 768$. The projector resolution is $800\times 600$ and it has 0.49- to 2.80-m projection distance.

Figure 8 shows the measurement results based on the proposed method. Figure 8(a) shows the reconstructed result when only phase matching is used. Figure 8(b) shows the reconstructed result when phase matching and the average phase calculation are used. Figure 8(c) shows the reconstructed result from Fig. 8(b) when the subpixel parallax optimization is used.

Fig. 8

The images captured from left camera and right camera. (a) The original image captured from left camera, (b) the original image captured from right camera, (c) the deformed fringe pattern from left camera, (d) the deformed fringe pattern from right camera, (e) the wrapped phase from left camera, and (f) the wrapped phase from right camera.

By comparing the measured surface, the effect of every step process is shown. Because the system contains two digital cameras, the stereo vision can be calibrated to reconstruct the measured object surface using phase matching. The measurement result is shown in Fig. 9(a). But it can be found that because there are wrong corresponding points, the surface is coarse. In addition, there are missing points at the edge of the wrapped phase. By adding edge detection into the phase matching, the missing points and blank vertical lines are removed as shown in Fig. 9(b). Compared with Fig. 9(a), Fig. 9(b) has more accurate matching results and less noise. It has smoother surface. But it also can be found that there are still noise. When the subpixel parallax optimization is used, the matching correctness rate is increased and the surface becomes smoother as shown in Fig. 9(c).

Fig. 9

The measurement results based on the proposed method. (a) The result from phase matching, (b) the result from phase matching plus edge detection, and (c) the result from the proposed method.

To better illustrate the effect of the proposed method, a white house is measured. The measurement results are shown in Fig. 10. Original image without fringe patterns is shown in Fig. 10(a). Figure 10(b) shows the measurement result when the wrapped phase is unwrapped. Figure 10(c) shows the result based on the proposed method. From the measurement results, it can be found that the proposed method can generate better results with less noise. To better show the measurement results, the details of the door are shown. Because the proposed method is based on phase matching and subpixel parallax optimization, it can get more accurate corresponding points. In addition, it does not need phase unwrapping; the speed of measurement is higher.

Fig. 10

The measurement result of a white house with complicated surface. (a) Original image without fringe patterns, (b) the measurement result and details from absolute phase, and (c) the measurement result and details from the proposed method.

To further quantify the matching accuracy of the proposed method, different masks’ surfaces are measured. The original masks are shown in Fig. 11. The measurement results are shown in Fig. 12. The measurement results from absolute phase matching are used as Ref. 18. The percent of correct matching points, the percent of wrong matching points, and the percent of missing matching points are calculated, respectively. These data are shown in Table 1.

Fig. 11

The original masks without deformed fringe pattern. (a) Monkey mask, (b) Iron Man mask, and (c) Santa Claus mask.

Fig. 12

The measurement results of different masks. (a) The measurement results of monkey mask, (b) the measurement results of Iron Man mask, and (c) the measurement results of Santa Claus mask.

Table 1

Matching precision based on the absolute phase (Unit: percentage %).

From the data in Table 1, we can find that the proposed method can provide high correctness rate that ca be above 98%. Compared with Iron Man and Santa Claus, monkey has larger error and missing rate because monkey has the more holes on the surface. Based on our analysis, we think that the wrapped phase is obtained from one-shot frame fringe pattern and it is hard to get high-quality phase value at the edge of the holes. These holes will decrease the quality of the wrapped phase and signal-to-noise ratio. Iron Man has less holes and it has less error rate and less missing rate. Because there are no holes on the Santa Claus, it has the highest correctness rate in addition it has the lowest error rate and missing rate.

To further verify the property of the proposed method for the step-like objects, discontinuous blocks and continuous blocks are measured and the experimental results are compared with the results from absolute phase matching, which are shown in Fig. 13. Figures 13(a) and 13(b) show the original images. Figures 13(c) and 13(d) show the measurement results from absolute phase matching. The measurement results from the proposed method are show in Figs. 13(e) and 13(f). The height of measured blocks are shown in Tables 1Table 2Table 3–4.

Fig. 13

The measurement results comparison from discontinuous blocks and continuous blocks. (a) The original image of discontinuous blocks, (b) the original image of continuous blocks, (c) the measurement results of discontinuous blocks from absolute phase matching, (d) the measurement results of continuous blocks from absolute phase matching, (e) the measurement results of discontinuous blocks from the proposed method, and (f) the measurement results of continuous blocks from the proposed method.

Table 2

The discontinuous blocks measurement from absolute phase matching (Units: mm).

Table 3

The continuous blocks measurement from absolute phase matching (Units: mm).

Table 4

The discontinuous blocks measurement from the proposed method (Units: mm).

From Tables 2–5, the ideal height, average height, RMS, average error, and maximum error are listed for different blocks. From these data, we can find that the height from the proposed method is closer to the ideal height. The height errors from the proposed method are smaller than that from absolute phase. This further verify the success of the proposed pattern. From the data, we can also find that the proposed method can be employed to measure discontinuous objects and continuous objects.

Table 5

The continuous blocks measurement from the proposed method (Units: mm).

5.

Summary

This paper propose a high-speed 3-D shape measurement algorithm based on PMP and stereo vision. It can use one-shot fringe pattern to get wrapped phase and reconstruct the 3-D shape without phase unwrapping. The original image matching constraint and subpixel phase matching are used to find the correct corresponding points. To increase the robustness and reduce missing rate of the boundary of the wrapped phase where phase value is $\pi$ or $-\pi$, the average value is used based on the adjacent phase value. By comparing the matching precision, the proposed method can get high-quality surface. Because the process of measurement can use any three consecutive deformed fringe patterns to get the wrapped phase and the matching process does not need phase unwrapping, the proposed method can be used in fast measurement.