Camera placement

(a) Poor constraint (b) Better constraint

Figure 4.5: Impact of camera placement on spatial quality of the visual hull. (a) Imbalanced placement of 4 cameras provides insufficient constraint. (b) Balanced placement of 4 cameras leads to better constraint.

The placement of cameras and the shape of the object being modelled need to be considered. As a general approach, positioning cameras so that they point towards

the object from as many different directions as possible better constrains the visual hull than clustering them together. Figure 4.5 demonstrates this principle. In both cases a subset of 4 cameras from a set of 10 is selected for reconstruction. 4.5a shows the effect on that selecting cameras from insufficient directions has on the reconstructed form. 4.5b shows that, by swapping 2 of the 4 cameras for ones pointing at the object from more diverse directions, better form constraint is achieved.

Objects with holes passing through them can be modelled by shape-from-silhouette, but these holes must be clearly visible from at least one camera and also correctly masked by background segmentation. Small holes can be problematic in terms of camera placement, particularly if the size of the hole is much smaller than the thickness of the object it is passing through. Such a scenario necessitates careful alignment of camera to hole to ensure that the background is visible through the hole.

Moving objects provide further challenges for 3D reconstruction systems, partic- ularly in respect of camera positioning. Any camera placement carefully consid- ered for particular regions of the object being modelled, becomes meaningless if the object is to be allowed to move freely. Employing the strategy of placing cam- eras so that they point at the object from as many different directions as possible can only be achieved for a region within which the object is allowed to move. Furthermore, as the object moves within this region, the size of it within camera image planes will vary as the object moves towards or away from a particular camera. This makes it difficult to make strategic decisions about camera image resolution and the complexity of objects being modelled. Providing cameras are pointing towards the object from many different directions, however, this problem can be mitigated by the object’s size increasing in one camera whilst it diminishes in another.

CHAPTER 4. QUALITY OF 3D RECONSTRUCTION 75

Camera placement and visual hull set definitions

By the classic definition of the visual hull, regions falling within it are defined as those projecting within the silhouette in all camera images:

V H = T Images ( S Silhouettes ( T Contours V))

where: V H is the visual hull formed, and V is the viewing cone formed by a

contour

This definition gives rise to a bounding volume defined by the intersection of regions visible by all cameras simultaneously. An object overlapping the edge of this volume will be clipped in the reconstruction, and those falling outside of the

volume will not be reconstructed at all. [39] defines a new visual hull set definition

in which the visibility domain of each camera is considered:

V HC= S Images ( T Silhouettes ( S Contours DV))

where: V HC is the visual hull complement formed, and DV is the visibility of

viewing cone complement relative the the image from which it is formed.

Considering the visual hull or its complement are the same, because the surface of the hull separates the region defined by regions falling inside or outside the intersection of silhouette cones. The difference lies in the complete visual hull set that is formed from all camera images in the first case, or only those visible in the second case. Regions projecting within the silhouette for all cameras, in which that region is visible, contribute to the visual hull in the second case. Using this definition relaxes the constraints on camera placement, allowing objects under reconstruction to be partially visible by a single camera, or completely outside the visible area of one or a subset of cameras.

4.3.3

Camera calibration

Camera calibration is the process that determines the relationship between the po- sition of objects in the real world and their mapping onto the camera 2D image plane. A number of different processes are described in the literature (Section 3.3.9), resulting in the output of a set of 3x4 projection matrices P, that map 3D world coordinates to 2D coordinates on the camera image plane through the rela- tionship in equation A.0.2 Through the process of camera calibration, two mea- sures of calibration accuracy may be defined. The 2D pixel re-projection error defines the RMS error in camera image pixels of projecting 3D coordinates onto the camera image plane using the matrix P. The 3D reconstruction error is the spatial error in the position of a 3D point, reconstructed from a pair of 2D points, each on a camera image plane projected into 3D space. The pixel re-projection error can be used to determine the calibration quality of an individual camera. Values under half a pixel are desirable for all cameras in a set. Values over a pixel can result in the wrong camera image pixel being selected for a particular point in space, giving rise to spatial and texture distortion in the reconstruction.