Calibration of Multi-Camera Systems

So far, we have assumed the mutually relative poses between the cameras of the multi- camera system to be known. In case the relative poses Mc are unknown, the approach

allows to perform a system self-calibration to additionally find best estimates bMc for rel-

ative poses besides best estimates bMt and bXi.

In the following, we illustrate the calibration of three multi-camera systems and report the achieved precision of the estimated calibration. The camera systems differ in the configuration of the single-view cameras and the type of lenses.

Calibration with overlapping views. We now describe the calibration of the camera system shown in Figure 3.15 with highly overlapping views, which is used for 3D recon- struction of vines. In order to determine the relative poses of the multi-camera system,

Figure 3.15: Multi-camera system consisting of five overlapping perspective camera views: Infrared camera on top, RGB camera in the middle and three monochromatic cameras. The distances from the RGB camera to the others are about 10 cm.

(a) (b)

Figure 3.16: Illustration of the estimated scene points and poses of the reference camera in (a). The red line denotes the known length on a poster for scale definition. The estimated relative poses of the multi-camera system are shown in (b).

we apply the bundle adjustment to 100 images of a wall draped with highly textured posters. The images were taken with the camera system from 20 different perspectives in a synchronized way. We use aurelo without considering the mutual stable relative poses between the cameras to obtain an initial solution for all 100 camera poses and the scene points. The dataset contains 593,412 observed image points of 63,140 scene points.

Starting from an a priori standard deviation for the image coordinates of σl = 1 pel,

the a posteriori variance factor is estimated with bσ2

0 = 0.112 indicating the automatically

extracted SIFT points to have an average precision of approximately 0.1 pel. This high precision of the point detection results mainly from the good image and camera calibration quality, and the highly distinctive scene structure. Figure 3.16 illustrates the estimated scene points and poses as well as the estimated relative poses.

3.7 Experiments 69

(a) Ladybug 3 on robot. (b) Estimated camera poses.

Figure 3.17: The Ladybug 3 mounted on a robot (a) executing a circular movement. The poses given by aurelo of each camera are shown in (b). Note that aurelo applies no constraints for a rigid camera system and the scale is chosen arbi- trarily.

erence camera is 0.1–0.2 mdeg around the viewing direction and 0.4–0.8 mdeg orthogonal to it. We scale the photogrammetric model by using a measured distance of 1.105 m with an error of about 0.1 %. The uncertainty of the estimated relative translations is 0.02–0.04 mm in viewing direction and 0.1–0.2 mm orthogonal to it.

Multi-camera system Ladybug 3. The omnidirectional multi-camera system Ladybug 3 consists of six cameras, five of which are mounted in a circular manner, one showing up- wards, together covering 80 % of the full viewing sphere. Neighboring images only have a very small overlap, which is too weak for system calibration without additional informa- tion. We have mounted the omnidirectional multi-camera system Ladybug 3 on a robot, see Figure 3.17 (a), which executes a circular movement with a radius of 50 cm in a highly textured room while the Ladybug 3 is taking synchronized images. This ensures over- lapping images of different cameras at different times of exposure. Initial values for this image sequence, consisting of 150 images taken by the five horizontal cameras at 30 differ- ent poses, are obtained with aurelo that provides 135,012 image points of 24,078 observed scene points. The resulting 150 camera poses are shown in Figure 3.17 (b).

After applying our bundle adjustment, the estimated a posteriori variance factor amounts to bσ2

0 = 0.252 using a priori stochastic model with σl = 1 pel for the image

points, indicating the automatically extracted SIFT points to have a quite good precision. Two parallel walls with known distance of 7.01 m can be estimated out of the estimated scene points. We use the distance to define the scale between the estimated relative camera poses. The estimated rotation parameters show a very high precision and the maximum deviation to the manufacturer’s calibration parameters amounts to 0.6◦. The

−5 −4 −3 −2 −1 0 1 2 3 4 5 0 1 2 3 4 5 6 7 8 9 y−axis [mm] z−axis [mm] c=5 c=2 c=3 c=1 c=4

Figure 3.18: Comparison of relative poses: estimated (solid) and manufacturer given (dashed).

estimated precision of the rotations and translations between the cameras are in the order of 0.0015–0.0025◦ and 0.1–0.2 mm, respectively.

To compare the estimated poses with the ones provided by the manufacturer, we apply a rigid transformation, which minimizes the distances between the estimated and given projection centers. The resulting estimated relative poses in Figure 3.18 show translational deviations in the order of 1–4 mm compared to the manufacturer’s calibration parameters. The reason for these deviations remains unclear.

The interior angles differ from a regular pentagon, where each interior angle is 108◦_,

by up to 13◦. Possible reasons for the deviations are too few observed scene points near the camera system and that we used an interior orientation for each camera from our own calibration, which is different from that of the manufacturer.

Multi-camera system with fisheye lenses. We make a similar investigation on an im- age sequence consisting of 96 images taken by four synchronized cameras with Lensagon BF2M15520 fisheye lenses having a field angle up to 185◦_{. As described in Sec. 1.2, the}

cameras are mounted on an UAV to generate two stereo pairs, one looking ahead and one looking backwards, providing a large field of view, see Figure 3.19. The UAV moves along a circle at a height of 5 m above a parking lot while rotating around its own axis, providing four overlapping images at each time of exposure.

In order to find corresponding points using the SIFT operator, we need to use a projec- tion of overlapping images which is not too far from a conformal projection, i.e. one that preserves angles, because of the severe fisheye-specific distortions at the image boundaries as the SIFT operator is only translation, scale and rotation invariant. For this reason, we transfer the original images into images following the stereographic fisheye model. This en- sures a conformal mapping between two different images when observing a scene at infinity as they themselves are conformal mappings of the spherical image of the scene. We obtain