Pre-Alignment of 3D Surfaces

containing the reconstructed optical rays Li from every image point is defined by

S3D _{= {L}

i}ni=1. For j = 1 to n iterations the following steps are repeated:

Silhouette-based Correlation ICP Algorithm

1. Extract the 3D silhouette SIL3D _{∈ M}3D _{and the virtual silhouette SIL}2D_.

2. Compute the local features of the projected silhouette points_P(Xi) with re-

spect to SIL2D and define the set of profile vectorsVsil = {osili }ni=1.

3. For every image point xj ∈ S2D with profile vector oimgi ∈ Vimg, find its corre-

sponding projected silhouette point_P(Xi) with the search criterion

corr(oimg_{i , V}sil) = max j_=1,···,n



corr(oimg_i , osil

j )

, and define the correspondence sets{Li, Xi}ni=1or{xi, Xi}ni=1.

4. With the defined correspondence sets, find the pose parameters M with the 2D-3D minimization constraint

0 = λM X_i Mf _{× Li}_{· e}+

or with the projective pose minimization constraint 0 = λ



xi − P(MXiMf)

 .

5. Actualize the position of the surface model points _M3D _{with the computed}

pose parameters M.

6. Compute the average error err = d (S2D_{, SIL}2D_{) between image points and}

silhouette points. If err < thres, exit; else goto 1.

5.3

Pre-Alignment of 3D Surfaces

Pre-alignment approaches are commonly used in the context of 3D surface registra- tion problems. Sensor information is matched to a model object by applying variants of the classical ICP algorithm in most of the cases. If the distance between sensor and model data is too large, the ICP algorithm can not be applied anymore. In this case, the problem is divided in two steps. First, a rough approximation of the pose is computed by aligning model and sensor data in 3D space. Once that model and sensor data are aligned, the classical ICP algorithm can be applied to find the exact pose. The last implies that a pre-alignment step is used to gain tracking assump- tion conditions in order to ensure the computation of the real pose by applying the

ICP algorithm. In practice, the computation of the pre-alignment and the exact pose imply the estimation of two rigid body motions by two minimization processes.

When a pre-alignment approach is needed, global features of model and sensor data are used. In general, the global orientation and position of a set of 3D points are defined by its major and minor distribution axes and their respective centers of mass, see [20]. For specific pose estimation scenarios, thinning algorithms in 3D are applied to sensor and model points in order to reduce them to a 3D skeleton, see [78]. The sets of arms and joints defining the skeleton are used to find the needed correspondences to compute the rough pose.

In this section, the pre-alignment problem for the monocular pose estimation is introduced. According to the correspondence search strategies and pose estima- tion constraints defined in this and the last chapters, a variation of the classical pre- alignment approaches is introduced for the monocular pose estimation of surface models.

5.3.1

Pre-alignment for the Monocular Pose Estimation

Classical pre-alignment approaches are in general proposed for pose scenarios where model and sensor data are defined in 3D space. For the monocular pose estimation problem, some considerations must be done to apply these approaches. Let us re- member that 3D-2D and projective pose minimization constraints are available. On the other hand, the correspondence search problem for contour and surface models can be solved in 3D space or in the image plane. Similarly, finding correspondences to perform the pre-alignment can be done either in 3D space or in the image plane.

The first variant is shown in figure 5.14. In this case, it is assumed that the major and minor distribution axes of the model in 3D space are known. Similarly, the main orientation axes of the detected image contour are obtained. Then, a plane Pm is re-

constructed from the image major axis as it is shown in the figure. The rough pose is computed by aligning the model major axis to its corresponding reconstructed plane. Correspondence pairs are formed between points lying along the main axis of the model and the plane Pm. With this set, a 3D point-plane minimization con-

straint can be applied to compute the rough pose, see [87]. If only the major axis is aligned, the pre-alignment is computed with certain uncertainty. Let us remember that normal pre-alignment approaches align three distribution axes in 3D. Never- theless, this is not possible for the monocular pose estimation problem since only two axes are obtained from the image plane. If the minor axis of the image contour is also considered to reconstruct a second plane, a better alignment is obtained.

The pre-alignment can be also done in the image plane. The main idea is shown in figure 5.15. Once that the 3D silhouette of the surface is projected onto the image plane, its main axes are obtained. Similar to the pre-alignment in 3D space, the rough pose is computed by the alignment of the image and silhouette main axes. In

5.3. Pre-Alignment of 3D Surfaces 117

Fig. 5.14: Alignment in 3D space. The model is aligned to a plane which is recon- structed from the image.

this case, 2D point-point correspondences are defined with points along both axes. Finally, the rough pose is computed by the projective pose estimation constraints.

Fig. 5.15: Alignment in the image plane. The model is projected onto the image plane, where a 2D alignment is performed.

In both cases, the pre-alignment is done with respect to projected or extracted data from the image plane. In consequence, the computed motor Mr = exp −θ₂L

describing the rough pose is formed by rotation and translation with respect to the

image plane. As can be seen in figure 5.16, the rotation axis of the motor L is per- pendicular to the image plane. On the other hand, the translational component of the motor is parallel to the image plane.

Fig. 5.16: Rotation axis and translation vector of the computed rough pose with re- spect to the image plane (left) and different positions of the camera with respect to the 3D surface model (right).

When the surface model moves parallel to the image plane, the result of the alignment with respect to the image may be better conditioned. To clarify that, let us suppose that the surface moves within a plane P as shown in the right picture of figure 5.16. At the camera position C1, the computed rough pose results in a rigid

body motion around the rotation axis L₁which is perpendicular to image plane and to the plane P. At this position, the rough pose corresponds to the real movement of the surface model as long as the model is within the view range of the camera. At the position C2, the model is aligned by a rotation around the axis L2. This rotation

aligns the main axes of the model with the main axes of the image contour. Never- theless, it rotates the surface model outside of the plane P . Therefore, the computed alignment does not correspond to the real movement of the surface anymore.

Let us remember that it is only possible to reconstruct planes from the image or to project the silhouette onto the image plane. Both cases imply that the pre-alignment is done with respect to only two orientation axes. Therefore, the pre-alignment for the monocular pose estimation is performed with larger uncertainty than the pre- alignment in 3D space.