Enhanced Calibration Method for Robotic Flexible 3D Scanning System

1. Introduction

Complicated components with numerous small key local features (KLFs) are commonly used in modern advanced manufacturing industries such as aerospace, marine engineering, and automotive sectors. Automated and accurate 3D shape measurement of these features is critical for ensuring quality control, enhancing product reliability, and minimizing manufacturing costs [1,2,3]. Three-dimensional (3D) shape measurement techniques are generally classified into contact and non-contact approaches. Among contact-based methods, coordinate measuring machines (CMMs) equipped with tactile probes [4] offer exceptional precision. Nevertheless, they are constrained by low efficiency in acquiring high-density 3D point cloud data. In recent years, structured light sensors [5,6,7], known for their non-contact nature, high precision, and efficiency, have been widely used in industrial applications and integrated with CMMs for 3D shape measurements. However, the combined method, which integrates CMMs with a structured light sensor [8], is not suitable for online measurements due to its restrictive mechanical structure and limited measurement efficiency. In contrast to CMMs, industrial robots are well-suited for executing complex spatial positioning and orientation tasks with efficiency. Equipped with a high-performance controller, these robots can also transport a vision sensor mounted on the end-effector to specified target locations. During the online measurement process, the spatial pose of the vision sensor varies as the robot’s end-effector moves to different positions. Nevertheless, the relative transformation between the sensor’s coordinate system and the end-effector remains fixed—an essential concept known as hand–eye calibration. Consequently, the accuracy of this calibration plays a pivotal role in determining the overall measurement accuracy of the system.

For hand–eye calibration, techniques are generally classified into three categories based on the nature of the calibration object: 2D target-based methods, single-point or standard-sphere-based methods, and 3D-object-based methods. One of the most widely recognized hand–eye calibration methods based on a two-dimensional calibration target was introduced by Shiu [9]. The fundamental constraint of the calibration process is derived by commanding the robot to move the vision sensor and observe the calibration target from multiple poses. Consequently, the hand–eye calibration problem is formulated as solving the homogeneous matrix equation AX=XB. Since then, a wide range of solution methods have been developed for these calibration equations, which are generally categorized into linear and nonlinear algorithms [10,11]. Representative approaches include the distributed closed-loop method, the global closed-loop method, and various iterative techniques. However, these methods tend to be highly sensitive to measurement noise. Compared to typical calibration methods, Sun et al. [12] introduced a flexible hand–eye calibration method for robotic systems. This approach estimates the hand–eye relationship using only the robot’s predefined motion and a 2D chessboard pattern. In contrast, Xu et al. [13] proposed a single-point-based calibration method that utilizes a single point—such as a standard sphere—to compute the transformation, offering a relatively simple and practical implementation. Furthermore, hand–eye calibration methods utilizing a standard sphere of known radius have been extensively adopted to estimate the hand–eye transformation parameters [14]. These methods provide an intuitive and user-friendly solution for determining both rotation and translation matrices. Among the 3D-object-based calibration methods, Liu et al. [15] designed a calibration target consisting of a small square with a circular hole. Feature points were extracted from line fringe images and used as reference points for the calibration process. However, the limited number of calibration points and the low precision of the extracted image features resulted in suboptimal calibration accuracy. In addition to hand-eye calibration methods based on calibration objects, several approaches that use environmental images, rather than calibration objects, have been successfully applied to determine hand-eye transformation parameters, as reported by Sang [16] and Nicolas and Qi [17]. Song et al. [18] proposed a robot hand–eye calibration algorithm based on irregular targets, where the calibration is achieved by registering multi-view point clouds using FPFH-based sampling consistency and a probabilistic ICP algorithm, followed by solving the derived spatial transformation equations. However, the methods mentioned above do not account for the robot positioning errors introduced by kinematic parameter errors during the hand-eye calibration process.

To overcome this limitation, many researchers have proposed various hand-eye calibration methods that account for the correction of robot kinematic parameter errors. Yin et al. [19] introduced an enhanced hand-eye calibration algorithm. Initially, hand-eye calibration is conducted using a standard sphere, without accounting for robot positioning errors. Then, both the robot’s kinematic parameters and the hand-eye relationship parameters are iteratively refined based on differential motion theory. Finally, a unified identification of hand-eye and kinematic parameter errors is accomplished using the singular value decomposition (SVD) method. However, the spherical constraint-based error model lacks absolute dimensional information, and cumulative sensor measurement errors further limit the accuracy of parameter estimation. Li et al. [20] introduced a method that incorporates fixed-point information as optimization constraints to concurrently estimate both hand-eye transformation and robot kinematic parameters. However, the presence of measurement errors from visual sensors substantially compromises the robustness and generalizability of the resulting parameter estimations. To further optimize measurement system parameters, Mu et al. [21] introduced a unified calibration method that simultaneously estimates hand-eye and robot kinematic parameters based on distance constraints. However, this method does not adequately address sensor measurement error correction during the solution of the hand-eye matrix, resulting in accumulated sensor inaccuracies that significantly degrade the precision of the derived relationship matrix.

To address the aforementioned limitations, we propose an accurate calibration method for a robotic flexible 3D scanning system based on a multidimensional ball-based calibrator (MBC) [22]. This method constructs a distance-based calibration model that concurrently considers measurement errors, hand-eye parameter errors, and robotic kinematic errors. Specifically, by incorporating angular-constraint-based optimization for compensating coordinate errors of the measurement points, a preliminary hand-eye calibration method is introduced based on a single virtual point determined via the barycenter technique. Subsequently, a distance-constraint-based calibration method is developed to further optimize both hand-eye and kinematic parameters, effectively associating system parameter errors with deviations in the measured coordinates of the single virtual point.

The remainder of this paper is organized as follows. Section 2 introduces the model of the measurement system. Section 3 outlines the proposed calibration methodology. Section 4 presents the experimental setup and accuracy validation. Finally, Section 5 concludes the study with a brief summary of the findings.

2. Measurement Method and Principle

2.1. Measurement Method

The proposed robotic 3D scanning system integrates an industrial robot with a fringe-projection-based 3D scanner, which is mounted on the robot’s end-effector using a custom fixture. For accurate system calibration and validation, a specially designed multidimensional ball-based calibrator (MBC) is utilized. The MBC features eight non-collinear spheres, whose spatial relationships are pre-calibrated using a CMM. As illustrated in Figure 1, the system comprises three coordinate frames: the robot base coordinate system$O_b-X_b{Y}_b{Z}_b$(BCS), the robot end-effector coordinate system$O_e-X_e{Y}_e{Z}_e$(ECS), and the 3D scanner coordinate system$O_s-X_s{Y}_s{Z}_s$(SCS).

During the measurement process, the robot adjusts its pose to guide the scanner in acquiring the target features. The acquired data are subsequently transformed from SCS to BCS. Based on the coordinate transformation principle, the measured coordinates of the visual points${}^S\mathit{P}_c$in the SCS can be expressed${}^B\mathit{P}_c$in the BCS as follows:

$$\left[\begin{array}{c}{}^B\mathit{P}_c\\ 1\end{array}\right]=\left[\begin{array}{cc}\mathit{R}_E^B& \mathit{T}_E^B\\ 0& 1\end{array}\right]\left[\begin{array}{cc}\mathit{R}_S^E& \mathit{T}_S^E\\ 0& 1\end{array}\right]\left[\begin{array}{c}{}^S\mathit{P}_c\\ 1\end{array}\right]$$

(1)

where${\mathit{R}}_E^B$and${\mathit{T}}_E^B$represent the rotation matrix and translation vector, respectively, from ECS to BCS, while${\mathit{R}}_S^E$and${\mathit{T}}_S^E$represent the rotation matrix and translation vector, respectively, in the hand-eye relation.

2.2. Robot Kinematic Error Model

Due to structural and manufacturing imperfections, deviations arise between a robot’s actual and nominal kinematic parameters, leading to discrepancies in its actual and theoretical poses—commonly referred to as positioning errors. A kinematic error model quantitatively characterizes the relationship between these parameter deviations and the resulting positioning errors. In the study, the error model is formulated based on the Modified Denavit-Hartenberg (MD-H) convention and rigid-body differential motion theory, and is expressed as follows:

$${\mathit{H}}_E^B+d{\mathit{H}}_E^B={\displaystyle \prod _{i=1}^N\left(\mathit{H}{}_i{}^{i-1}+d\mathit{H}_i^{i-1}\right)}$$

(2)

where$d{\mathit{H}}_E^B$is the deviation in the homogenous transformation matrix between ECS and BCS, and${}_i{}^{i-1}\mathit{H}$is the homogenous transformation matrix between the adjacent joints.

By expanding Equation (2) and neglecting the second-order term, and combining it with the differential kinematics model$d{\mathit{H}}_E^B={\mathit{H}}_E^B\delta {\mathit{H}}_E^B$, we obtain:

$$\begin{array}{l}{\mathit{H}}_E^B+d{\mathit{H}}_E^B={\mathit{H}}_E^B+{\mathit{H}}_E^B\delta {\mathit{H}}_E^B\\ ={\mathit{H}}_E^B+{\displaystyle \sum _{i=1}^n\left(\frac{\partial {\mathit{H}}_E^B}{\partial {\theta }_i}\Delta {\theta }_i+\frac{\partial {\mathit{H}}_E^B}{\partial d_i}\Delta d_i+\frac{\partial {\mathit{H}}_E^B}{\partial a_i}\Delta a_i+\frac{\partial {\mathit{H}}_E^B}{\partial {\alpha }_i}\Delta {\alpha }_i+\frac{\partial {\mathit{H}}_E^B}{\partial {\beta }_i}\Delta {\beta }_i\right)} \end{array}$$

(3)

where$\Delta {\theta }_i$,$\Delta d_i$,$\Delta a_i$,$\Delta {\alpha }_i$and$\Delta {\beta }_i$are small link parameters errors.

Consequently, the relationship between the robot positioning error and kinematic parameter

$$\Delta \mathit{E}=\left[\begin{array}{l}\Delta \mathit{D}\hfill \\ \Delta \mathit{\Theta }\hfill \end{array}\right]=\left[\begin{array}{l}{\mathit{M}}_1\hfill \\ {\mathit{M}}_2\hfill \end{array}\right]\Delta \mathit{\theta }+\left[\begin{array}{l}{\mathit{M}}_2\hfill \\ 0\hfill \end{array}\right]\Delta \mathit{d}+\left[\begin{array}{l}{\mathit{M}}_3\hfill \\ 0\hfill \end{array}\right]\Delta \mathit{a}+\left[\begin{array}{l}{\mathit{M}}_4\hfill \\ {\mathit{M}}_3\hfill \end{array}\right]\Delta \mathit{\alpha }+\left[\begin{array}{l}{\mathit{M}}_5\hfill \\ {\mathit{M}}_6\hfill \end{array}\right]\Delta \mathit{\beta }=\mathit{J}\Delta \mathit{X}$$

(4)

errors can be expressed as:

where$\Delta \mathit{E}={\left[\Delta e_x \Delta e_y \Delta e_z \delta e_x \delta e_y \delta e_z\right]}^T$and$\Delta \mathit{X}=\left[\Delta \mathit{\theta } \Delta \mathit{d} \Delta \mathit{a} \right.{\left.\Delta \mathit{\alpha } \Delta \mathit{\beta }\right]}^T$;$\Delta \mathit{D}$and$\Delta \mathit{\Theta }$represent respectively the differential translation vector and differential rotation vector;$\Delta \mathit{\theta }$,$\Delta \mathit{d}$,$\Delta \mathit{a}$,$\Delta \mathit{\alpha }$and$\Delta \mathit{\beta }$represent the kinematic parameter errors vector,${\mathit{M}}_1\sim {\mathit{M}}_6$are the$3\times n$error coefficient matrices,$\mathit{J}$is the Jacobian matrix.

3. Measurement System Calibration

To improve the accuracy of hand-eye calibration, the paper proposed an online accurate calibration method using a multidimensional ball-based calibrator (MBC) to simultaneously identify both hand-eye transformation and robot kinematic parameters. A schematic of the calibration procedure is presented in Figure 2. First, Section 3.1 introduces an initial hand-eye calibration method that compensates for measurement point errors using an angular-constraint-based optimization method. This method is based on a virtual point calculated via the barycenter algorithm. Subsequently, Section 3.2 presents a distance-constraint-based calibration strategy to further refine the estimation of both hand-eye transformation and kinematic parameters.

3.1. Initial Calibration of the Hand-Eye Parameters

3.1.1. Preliminary Hand-Eye Calibration Method based on a Virtual Single Point

During the initial stage of hand-eye calibration, the MBC is stably positioned within a suitable workspace. A single virtual point, serving as the calibration target, is calculated based on the coordinates of four measurement points optimized using an angular-constraint-based optimization method. Specifically, the initial hand-eye transformation is then determined by capturing this virtual point from multiple robot’s poses using the 3D scanner. Furthermore, the position of the single virtual point relative to the BCS remains constant. According to the principle of barycentric coordinates, an initial (non-optimized) virtual point${}^S\mathit{P}_{gi}^m={\left({}^{S}x_{gi}^m,{}^{S}y_{gi}^m,{}^{S}z_{gi}^m\right)}^T$, as illustrated in Figure 2, is computed based on the initial coordinates of four measurement points${}^S\mathit{P}_{si}^m={\left({}^{S}x_{si}^m,{}^{S}y_{si}^m,{}^{S}z_{si}^m\right)}^T$, and is formulated as follows:

$${}^S\mathit{P}_{gi}^m={\left({\displaystyle \frac{1}{4}}{\displaystyle \sum _{i=1}^4{}^{S}x_{si}^m},{\displaystyle \frac{1}{4}}{\displaystyle \sum _{i=1}^4{}^{S}y_{si}^m},{\displaystyle \frac{1}{4}}{\displaystyle \sum _{i=1}^4{}^{S}z_{si}^m}\right)}^T$$

(5)

When the 3D scanner moves to the i-th and j-th poses, the following equations considering the measurement deviation can be obtained according to Equation (1):

$$\left\{\begin{array}{l}{}^B\mathit{P}_{gi}=\mathit{R}_{E^{0i}}^B{\mathit{R}}_S^E\left({}^S\mathit{P}_{gi}^m+\Delta {}^S\mathit{P}_{gi}^m\right)+\mathit{R}_{E^{0i}}^B\mathit{T}{}_S{}^E+\mathit{T}_{E^{0i}}^B\hfill \\ {}^B\mathit{P}_{gj}=\mathit{R}_{E^{0j}}^B{\mathit{R}}_S^E\left({}^S\mathit{P}_{gj}^m+\Delta {}^S\mathit{P}_{gj}^m\right)+{\mathit{R}}_{E0j}^B\mathit{T}{}_S{}^E+\mathit{T}_{E^{0j}}^B\hfill \end{array}\right.$$

(6)

where$\Delta {}^S\mathit{P}_{gi}^m$, computed by the optimized method introduced in section 3.1.2, is the deviation between the nominal and actual coordinates of the single virtual point in the SCS.

By translating the 3D scanner multiple times to measure the points, a matrix equation with form of${\mathit{R}}_S^E\mathit{A}=\mathit{b}$is given by:

$${\mathit{R}}_S^E{\left[\begin{array}{c}\left({}^S\mathit{P}_{g1}^m+\Delta {}^S\mathit{P}_{g1}^m\right)-\left({}^S\mathit{P}_{g2}^m+\Delta {}^S\mathit{P}_{g2}^m\right)\\ ⋮\\ \left({}^S\mathit{P}_{g1}^m+\Delta {}^S\mathit{P}_{g1}^m\right)-\left({}^S\mathit{P}_{gn}^m+\Delta {}^S\mathit{P}_{gn}^m\right)\end{array}\right]}^T={\left({\mathit{R}}_{E0i}^B\right)}^T{\left[\begin{array}{c}{\mathit{T}}_{E02}^B-{\mathit{T}}_{E02}^B\\ ⋮\\ {\mathit{T}}_{E0n}^B-{\mathit{T}}_{E02}^B\end{array}\right]}^T$$

(7)

The unknown matrix${\mathit{R}}_S^E$is determined using the singular value decomposition (SVD) algorithm. Subsequently, the unknown translation vector${\mathit{T}}_S^E$is calculated via the least squares method. Since measurement errors at the target point substantially affect the overall calibration accuracy, it is essential to optimize the coordinates of the measurement points acquired by the 3D scanner.

3.1.2. Measurement Error Identification Method

To identify and compensate for the measurement errors of the measurement points, an adjustment optimization method based on the angle constraints is introduced, and the diagram of a spatial angle constructed by two nonzero vectors is shown in Figure 3.

Based on the property that the angle between two vectors in Euclidean space is independent of the coordinate system, the angle values formed by the target points on the MBC are considered as the reference true values. These points are pre-calibrated using a high-precision CMM. The angular values derived from the measured coordinates on the MBC are then compared with the reference true to establish an angular error equation.

According to the angle information composed of the initial measurement points in the MBC, the arccosine function is given by:

$${\theta }_{\mathit{i}}=\arccos \left({\displaystyle \frac{\mathit{a}\cdot \mathit{b}}{‖\mathit{a}‖‖\mathit{b}‖}}\right)$$

(8)

where${\theta }_{\mathit{i}}$is the angle between the vectors a and b.

Next, the prior angle values are selected as the reference true values. The angular values derived from the field measurements of each target point are then subtracted from the reference true values to form the angular error equation. Taking angle${\theta }_{ai}$as an example, we apply a Taylor expansion to Equation (8), neglecting second-order and higher-order terms, resulting in the linearized equation for the angular constraint:

$${\widehat{\theta }}_{ai}={\theta }_{ai}^0+\left({\displaystyle \frac{\partial {\theta }_{ai}}{\partial {}^{D}x_{s1}^S}}\Delta {}^{D}x_{s1}^S+{\displaystyle \frac{\partial {\theta }_{ai}}{\partial {}^{D}y_{s1}^S}}\Delta {}^{D}y_{s1}^S+{\displaystyle \frac{\partial {\theta }_{ai}}{\partial {}^{D}z_{s1}^S}}\Delta {}^{D}z_{s1}^S+\dots +{\displaystyle \frac{\partial {\theta }_{ai}}{\partial {}^{D}z_{sn}^S}}\Delta {}^{D}z_{sn}^S\right)$$

(9)

where$\left(\Delta {}^S\stackrel{⌢}{x}_{si}^m,\Delta {}^S\stackrel{⌢}{y}_{si}^m,\Delta {}^S\stackrel{⌢}{z}_{si}^m\right)$represent the coordinate corrections of the initial measurement points, the nominal angle${\stackrel{⌢}{\theta }}_i$is calibrated by CMM, and${\stackrel{⌢}{\theta }}_i^0$is the measured angle.

The angular error is characterized by the following mathematical expression, which quantifies the relationship between measurement variables and angular deviation:

$$v_{ai}={\widehat{\theta }}_{ai}-{\theta }_{ai}^0$$

(10)

To further facilitate analysis, the above equation is reformulated into a matrix form as follows:

$${\mathit{W}}_a={\mathit{B}}_a\Delta {\stackrel{⌢}{\mathit{X}}}_a-{\mathit{d}}_a$$

(11)

where$\Delta {\stackrel{⌢}{\mathit{X}}}_a={\left[\Delta {}^{D}x_{s1}^S,\Delta {}^{D}y_{s1}^S,\dots ,\Delta {}^{D}z_{sn}^S\right]}^T$denotes the vector of coordinate corrections;${\mathit{d}}_{\mathit{a}}$represents the angular error vector;${\mathit{B}}_{\mathit{a}}$is the coefficient matrix.

According to the principle of least squares adjustment, the normal equations are derived as follows:

$$\left({\left({\mathit{B}}_a\right)}^T{\mathit{U}}_a{\mathit{B}}_a\right)\Delta {\stackrel{⌢}{\mathit{X}}}_a={\left({\mathit{B}}_a\right)}^T{\mathit{U}}_a{\mathit{d}}_a$$

(12)

where${\mathit{U}}_a$is weight matrix.

In the paper, the ridge estimation method [23] was employed to calculate the optimal parameters. As a result, after compensating for measurement errors, an optimized single virtual point${}^S\tilde{\mathit{P}}_{gi}^m$is derived from the adjusted measurement points.

3.2. Accurate Calibration of Robot Kinematic and Hand-Eye Parameters

Discrepancies between the theoretical and actual kinematic parameters of the robot lead to deviations in the end-effector’s pose. Additionally, the robot motion involved in the initial hand-eye calibration process introduces inevitable positioning errors, further compromising calibration accuracy. Measurement errors from the 3D scanner also impose constraints on improving calibration precision. To address these issues, this section presents a joint calibration method based on a stereo target, aiming to simultaneously compensate for both hand-eye and kinematic parameter errors. Specifically, a distance error model is derived for associating the parameter errors with the deviations in the measured coordinates of the single virtual point. By reducing the impact of scanner measurement noise and kinematic deviations, the proposed approach significantly enhances the online calibration accuracy of the integrated robot-scanner system.

Assume that${}^S\mathit{P}_c^m={\left[{}^{S}x_c^m,{}^{S}y_c^m,{}^{S}z_c^m,1\right]}^T$represents the homogeneous coordinates of a measurement point in the SCS, and the corresponding measurement result in the BCS can be denoted as:

$${}^B\mathit{P}_{gi}^m={\mathit{H}}_{Ei}^B{\mathit{H}}_S^E{}^S\mathit{P}_{gi}^m$$

(13)

where${\mathit{H}}_{Ei}^B$and${\mathit{H}}_S^E$represent the theoretical values of the robot end-effector pose matrix and the hand-eye transformation matrix, respectively.

Considering the influences of hand-eye calibration errors, robotic kinematic parameter deviations, and scanner measurement errors, the actual position of the measurement point in the BCS can be expressed as:

$${}^B\mathit{P}_{ci}^r=\left({\mathit{H}}_{Ei}^B+\Delta {\mathit{H}}_{Ei}^B\right)\left({\mathit{H}}_S^E+\Delta {\mathit{H}}_S^E\right)\left({}^S\mathit{P}_c^m+\Delta {}^S\mathit{P}_c^m\right)$$

(14)

where$\Delta {}^S\mathit{P}_c^m={\left[\Delta {}^{S}x_c^m,\Delta {}^{S}y_c^m,\Delta {}^{S}z_c^m,0\right]}^T$represents the scanner measurement error,$\Delta {\mathit{H}}_{Ei}^B$denotes the robotic end-effector pose error, and$\Delta {\mathit{H}}_S^E$corresponds to the hand-eye calibration error.

By subtracting Equation (14) from Equation (13) and neglecting higher-order terms beyond the second order, the deviation between the actual and measured values of the point in the BCS is obtained:

$$d{}^B\mathit{P}_c={\mathit{H}}_{Ei}^B{\mathit{H}}_S^E\Delta {}^S\mathit{P}_c^m+{\mathit{H}}_{Ei}^B\Delta {\mathit{H}}_S^E{}^S\stackrel{⌢}{\mathit{P}}_c^m+\Delta {\mathit{H}}_{Ei}^B{\mathit{H}}_S^E{}^S\stackrel{⌢}{\mathit{P}}_c^m$$

(15)

where${}^S\stackrel{⌢}{\mathit{P}}_c^m={}^S\mathit{P}_c^m+\Delta {}^S\mathit{P}_c^m$is the corrected coordinates of the scanned measurement point. The first term on the right-hand side can be expressed as:

$${\mathit{H}}_{Ei}^B{\mathit{H}}_S^E\Delta {}^S\mathit{P}_c^m={\mathit{H}}_{Ei}^B{\mathit{H}}_S^E{\left[\Delta {}^{S}x_c^m,\Delta {}^{S}y_c^m,\Delta {}^{S}z_c^m,0\right]}^T={}^S\mathit{L}_i\Delta {}^S\mathit{T}_c^m$$

(16)

According to differential kinematics, the hand-eye relationship error can be expressed as:

$$\Delta {\mathit{H}}_S^E={\mathit{H}}_S^E\delta {\mathit{H}}_S^E$$

(17)

where$\delta {\mathit{H}}_S^E$is the differential operator.

$${\mathit{H}}_{Ei}^B\Delta {\mathit{H}}_S^E{}^S\stackrel{⌢}{\mathit{P}}_c^m= {\mathit{H}}_{Ei}^B{\mathit{H}}_S^E\delta {\mathit{H}}_S^E{}^S\stackrel{⌢}{\mathit{P}}_c^m={\mathit{M}}_i\Delta \mathit{S}$$

(18)

Thus, the second term on the right-hand side of Equation (15) can be simplified as:

Similarly, the error model for the robot end-effector pose can be expressed as:

$$\Delta {\mathit{H}}_{Ei}^B{\mathit{H}}_S^E{}^S\stackrel{⌢}{\mathit{P}}_c^m={\mathit{N}}_i\Delta \mathit{E}={\mathit{N}}_i{\mathit{J}}_i\Delta \mathit{X}$$

(19)

where${}^B\stackrel{⌢}{\mathit{P}}_c^m={\mathit{H}}_{Ei}^B{\mathit{H}}_S^E{}^S\stackrel{⌢}{\mathit{P}}_c^m$represents the coordinates of the corrected scan measurement point transformed into the robot base coordinate system, and$\Delta {\mathit{E}}_i={\mathit{J}}_i\Delta \mathit{X}$represents the robot end-effector pose error model from Equation (4).

By neglecting second-order high-order terms, the hand-eye relationship model incorporating scanner measurement errors and robot kinematic parameter errors is obtained as follows:

$$d{}^B\mathit{P}_c={}^S\mathit{L}_i\Delta {}^S\mathit{T}_c^m+{\mathit{M}}_i\Delta \mathit{S}+{\mathit{N}}_i{\mathit{J}}_i\Delta \mathit{X}={}^S\mathit{L}_i\Delta {}^S\mathit{T}_c^m+{\mathit{G}}_i\Delta \mathit{K}$$

(20)

where${\mathit{G}}_i=\left[\begin{array}{cc}{\mathit{M}}_i& {\mathit{N}}_i{\mathit{J}}_i\end{array}\right]$is the coefficient matrix corresponding to the i-tℎ calibration pose of the robot, while$\Delta \mathit{K}={\left[\begin{array}{cc}\Delta \mathit{S}& \Delta \mathit{X}\end{array}\right]}^T$is a vector comprising system parameter errors, including hand-eye parameter errors and robot kinematic parameter errors.

Building upon the preceding research, a system parameter identification model is further developed based on distance error analysis. In Euclidean space, the theoretical distance between two measured points should remain invariant across different coordinate systems. However, discrepancies arise between the theoretical and measured distances of two points in the BCS due to scanner measurement errors, inaccuracies in hand-eye parameters, and deviations in robot kinematic parameters—collectively referred to as distance errors. In the paper, the distance error is defined as the difference between the calibrated distance and the measured distance between the center points of two spheres on the stereo target, as illustrated in the Figure 4.

Let${}^B\mathit{P}_{ca}^r=\left({}^{B}x_{ca}^r,{}^{B}y_{ca}^r,{}^{B}z_{ca}^r\right)$and${}^B\mathit{P}_{cb}^r=\left({}^{B}x_{cb}^r,{}^{B}y_{cb}^r,{}^{B}z_{cb}^r\right)$represent the actual coordinates of points a and b on the stereo target in the BCS. Let${}^B\mathit{P}_{ca}^m=\left({}^{B}x_{ca}^m,{}^{B}y_{ca}^m,{}^{B}z_{ca}^m\right)$and${}^B\mathit{P}_{cb}^m=\left({}^{B}x_{cb}^m,{}^{B}y_{cb}^m,{}^{B}z_{cb}^m\right)$represent the measured coordinates of the corresponding points. Similarly, let${}^B\mathit{l}_{ab}^r$be the actual distance vector between the two points, and${}^B\mathit{l}_{ab}^m$be the measured distance vector. The error vectors between the measured and actual values of the two points are denoted as$d{}^B\mathit{P}_{ca}$and$d{}^B\mathit{P}_{cb}$, respectively. Then, we have:

$$\left\{\begin{array}{l}{}^B\mathit{l}_{ab}^r={}^B\mathit{P}_{ab}^r-{}^B\mathit{P}_{ab}^r\\ {}^B\mathit{l}_{ab}^m={}^B\mathit{P}_{ab}^m-{}^B\mathit{P}_{ab}^m\\ \mathit{d}{}^B\mathit{P}_{ca}={}^B\mathit{P}_{ca}^r-{}^B\mathit{P}_{ca}^m\\ \mathit{d}{}^B\mathit{P}_{cb}={}^B\mathit{P}_{cb}^r-{}^B\mathit{P}_{cb}^m\end{array}\right.$$

(21)

$$\Delta l_{ab}=‖\Delta {\mathit{l}}_{ab}‖=‖{}^B\mathit{l}_{ab}^r‖-‖{}^B\mathit{l}_{ab}^m‖$$

(22)

Thus, the distance error$\Delta l_{ab}$between the two points can be expressed as:

where$\Delta {\mathit{l}}_{ab}$is the distance error vector.

Then, we obtain:

$$‖\Delta {\mathit{l}}_{ab}‖={\displaystyle \frac{{\left({}^B\mathit{P}_{ca}^m-{}^B\mathit{P}_{cb}^m\right)}^T}{‖{}^B\mathit{l}_{ab}^m‖}}\times \left(d{}^B\mathit{P}_{cb}-d{}^B\mathit{P}_{ca}\right)$$

(23)

Substituting Equation (20) into the above expression yields:

$$\underset{\Delta \mathit{l}}{\underbrace{\Delta l_{ab}-{\displaystyle \frac{{\left({}^B\mathit{P}_{ca}^m-{}^B\mathit{P}_{cb}^m\right)}^T}{‖{}^B\mathit{l}_{ab}^m‖}}\left({}^S\mathit{L}_b\Delta {}^S\mathit{T}_{cb}^m-{}^S\mathit{L}_a\Delta {}^S\mathit{T}_{ca}^m\right)}}=\underset{\mathit{Q}}{\underbrace{{\displaystyle \frac{{\left({}^B\mathit{P}_{ca}^m-{}^B\mathit{P}_{cb}^m\right)}^T}{‖{}^B\mathit{l}_{ab}^m‖}}\left({\mathit{G}}_b-{\mathit{G}}_a\right)}}\Delta \mathit{K}$$

(24)

To refine the kinematic model and hand-eye transformation matrix for improved end-effector scanning accuracy, the position of the stereo target was varied, and the above process was repeated multiple times to establish a system of linear equations. The Levenberg-Marquardt (L-M) algorithm was then employed to solve for system parameter errors, resulting in a more accurate representation of the system.

4. Experimental and Discussion

To verify the effectiveness of the proposed calibration method, a robot-scanner experimental system was established, and corresponding validation experiments were conducted. As illustrated in Figure 4, the system mainly comprises a KUKA industrial robot with a repeatability of 0.04 mm and an LMI 3D scanner. The calibration experiments are presented were carried out in Section 4.1, while Section 4.2 assesses the calibration accuracy by measuring a metric artifact in accordance with VDI/VDE 2634 Part 3.

For precise calibration and performance evaluation, a customized multidimensional ball-based calibrator (MBC) was developed specifically for 3D scanner measurements. As illustrated in Figure 5, the calibrator is constructed from carbon fiber reinforced polymer and incorporates eight matte stainless steel balls (MSBs). The spatial relationships among all balls were pre-calibrated using a coordinate measuring machine (CMM) with an accuracy of 0.001 mm.

4.1. Calibration Experiments

First, an initial hand-eye calibration experiment was conducted following the method introduced in Section 3.1. A multidimensional ball-based calibrator was positioned within the robot’s workspace, and the robot was programmed to execute six translational movements and six orientation changes. To prevent singularities, the end effector was translated along the X, Y, and Z axes of the robot’s base coordinate frame during the six translational movements. At each pose, the 3D scanner performed multiple measurements of the target spheres on the stereo target. Using the first six sets of measurements, the rotation matrix${\mathit{R}}_S^E$was computed according to Equation (7). Subsequently, the translation vector${\mathit{T}}_S^E$was determined from the remaining six sets of data, yielding the initial estimate of the hand-eye transformation matrix.

Subsequently, a high-precision hand-eye calibration experiment was performed. The MBC, mounted on a tripod, was successively placed at eight distinct vertical positions within the workspace. At each position, the robot’s end effector executed eight unique orientation changes. A 3D scanner was employed to capture the target points and extract the coordinates of the sphere centers. Based on these measurements, a system parameter error identification model was constructed to estimate the kinematic parameter errors, as presented in Table 1. The refined hand-eye transformation matrix was then accurately determined through this high-precision calibration process, as shown below:

${ \stackrel{⌢}{\mathit{H}}}_S^E=\left[\begin{array}{cccc}-0.0169& -0.9998& -0.0045& 1.2621\\ -0.9999& 0.0173& -0.0011& -0.5610\\ 0.0003& 0.0052& -1.0000& 264.7998\\ 0& 0& 0& 1\end{array}\right]$

4.2. Accuracy Evaluation of Accurate Calibration

To validate the calibration accuracy, a standard scale was employed, as illustrated in Figure 6. The errors between the measured and actual distances of sphere centers on the standard scale were computed at ten different spatial positions, using system parameters obtained both before and after calibration. The corresponding results are presented in Table 2. Following calibration, the maximum (MPE) and mean errors (ME) were reduced from 1.041/0.809 mm to 0.428/0.394 mm, respectively. These results satisfy the accuracy requirements for scanning critical and hard-to-reach features, thereby confirming the reliability and effectiveness of the proposed method.

5. Conclusions

In this study, a flexible measurement system integrating a fringe-projection 3D scanner with an industrial robot was developed to enable efficient measurement of large-scale surfaces. To improve overall measurement accuracy, we proposed an online calibration method employing a multidimensional ball-based calibrator to simultaneously compensate for hand-eye transformation and robot kinematic errors. Accuracy verification results show that the maximum and average measurement errors were reduced from 1.041 mm / 0.809 mm to 0.428 mm/ 0.394 mm, confirming the effectiveness of the proposed approach. Future research will focus on quantitative validation of measurement accuracy for large-scale components.