5.1 Introduction
6.4.3 A Comparison With A Surface Based Registration Technique
6.4.3.1 Methods
In this section, an experiment is described in which the TricorderTM S4m system is
used to create input data for the algorithm. As mentioned in the previous chapter, the TricorderTM S4m system takes sets of video images, and reconstructs a texture mapped
surface. A single `grab' for the S4m captures four video images, from four cameras, with the scene illuminated with a pseudo-random speckle pattern and four video images illuminated with plain white light. As the four video cameras are accurately calibrated, a surface can be reconstructed from the patterned light images, and texture mapped with information from the four plainly lit video images.
The following experiment was devised. A series of 56 sets of images was captured by the S4m system whilst the volunteer moved slowly within the eld of view. For each of the 56 sets, the corresponding surface was reconstructed. The rst surface was then taken, clipped to remove spurious surface data, and registered to the remaining 55 in
6.4 Experiments
174
the order they were taken. The algorithm used was an independent implementation [Maurer Jr. et al., 1996] of the iterative closest point algorithm [Besl and McKay, 1992]. The registration from surface one to two, was used as the starting estimate for the registration from surface two to three and so on.
Subsequently, the rst clipped surface was taken and registered to the remaining 55 sets of plainly lit video images using the proposed texture mapped tracking algorithm. This was repeated using the non-texture mapped algorithm of the previous chapter. The surface based, texture mapped and non texture mapped algorithm were compared by measuring the 3D error between the texture mapped and surface based transformations, and the non texture mapped and surface based registrations over the sequence of 55 sets of images.
Note that the surface based registration may have errors for two reasons. Firstly, the surface based registration minimises the distance between surfaces, which in itself does not guarantee a correct registration. Consider the case of registering a hemisphere to a sphere of equal radius. The distance between each surface could be zero, but there are still an innite number of possible, incorrect registrations. However, surface based registration is widely used, and in this case where the two surfaces are generated by the same device, captured within twenty seconds of each other, and have featuredness or curvature like the face, should register well. Secondly, the reconstructed surface is formed from the images that were illuminated with the pseudo-random dot pattern. There is approximately a two to three second delay between the capture of the patterned images and the plainly lit images using the TricorderTM system. Therefore the volunteer could
have moved between the capturing of these two sets of images, and so even if the surface based registration was perfect, it would never match the registration produced by the texture or non-texture mapped algorithms. It is assumed that the movement of the volunteer between the capturing of the images illuminated with the pseudo-random dot pattern and the images illuminated with the plain white light is small compared to errors in the registration algorithm because the time delay is small.
6.4 Experiments
175
6.4.3.2 Results
Figure 6.14 illustrates the tracking algorithm. The left column represents the rst frame in the tracking sequence, the middle column represents the 14th frame, and the right column represents the 36th frame. The TricorderTMsystem always captures four images
at a time, one from each of four cameras. In this gure, all the images represent the images from the same view, i.e. the top left camera from the volunteers viewpoint. Images (a), (b), and (c) are the plainly lit video images to which the proposed texture mapped tracking algorithm registers. Images (d), (e) and (f) are the surface reconstructions created by the TricorderTM system, viewed from the same direction.
It can be seen that the surface in image (d) is aligned with image (a), surface (e) aligned with image (b) and surface (f) aligned with image (c). This is because the surfaces were reconstructed directly from similar patterned light video images, and so should t well. In gure (g), the surface in green was clipped, and shown in red. This red surface was then registered to each reconstructed surface which included the surfaces shown in gures (e) and (f), using a surface based registration [Maurer Jr. et al., 1996]. Figures (h) and (i) show that the surface based registration was successful at frame 14 and 36 as the red surface ts the green surface well. Figure (j) shows a wireframe representation of the surface overlayed on the rst video image. Recall that the texture-mapped tracking algorithm matches the texture mapped surface directly to the video images, i.e. an intensity based match. Figure (k) shows that the texture mapped tracking algorithm works well up until frame 14, but gure (l) shows that at frame 36, the algorithm has failed.
The performance can also be assessed by measuring the 3D error between the surface based registration estimate for each video frame, and the texture mapped tracking es- timate for each frame. This is shown in the graph in gure 6.15. The texture mapped tracking algorithm tracks well up until frame 14. After frame 14, the algorithm fails between frame 15 and 25, recovers between frame 25 and 31 and fails from 31 to 36. After frame 36 the algorithm was stopped, as the registration was lost. By comparison, the non-texture mapping algorithm fails completely as the error is always
>
10 mm and after frame 25, the 3D error increases rapidly.6.4 Experiments
176
(a) (b) (c)
(d) (e) (f)
(g) (h) (i)
(j) (k) (l)
Figure 6.14: Comparing texture mapped and surface based tracking. See text, section 6.4.3.2.
6.4 Experiments
177
0 20 40 60 80 100 0 10 20 30 40 50 3D error in mm Frame number "without_texture_mapping" "with_texture_mapping" (a)Figure 6.15: Graph of 3D error in mm between the surface based tracking [Maurer Jr. et al., 1996] and the proposed texture mapped tracking.
6.4.3.3 Conclusions
In the previous experiment it was concluded that the algorithm failed to track because the change in registration transformation between video frames was too large. Here, more care was taken to make the transformation between each frames small. The head movement of the volunteer consisted of a rotation to the left, rotation up, rotation to the right, rotation down, and rotation back to the centre position. The texture from the initial image was mapped onto the surface and the initial surface used to track throughout the sequence. As the video images are taken with one single plain white light source, there is noticeable shading. The illumination of each surface point on the volunteers face will change as he moves relative to the camera. The texture mapped onto the initial surface however, will not. It was concluded that the algorithm failed when the volunteer's head had rotated too far to the left, and to the right. Frame 14, was where the algorithm tracked to, which represents a total rotation of 25 degrees to the left. Frame 25 - 31 represented rotations of 17 - 19 degrees to the right. Therefore it can be concluded that this algorithm only works well for rotations of approximately
20 degrees from the initial position. Beyond this, the shading on the texture map is too dierent to match to the next video image.