A surface constructed from 3D volumetric data facilitates the rendering of the object. This chapter explores a method which constructs triangular patches from an octree, and a robust construction is achieved by assuming two properties of a 3D object. The connec- tivity property presumes a surface covers all area of an object tightly without unattached object segments. The continuity property assumes an object as piecewise convex and a local convexity is similar in shape to the adjacent convexity if they are connected.
The proposed local hull-based surface construction (LCH) estimates a surface from local convexities. The best VMC result is used as its initial surface vertices, and slices are prepared from them. The sliced data is clustered based on a cluster conditional
Figure 6.18: Octree, MC, VMC, and LCH results of (a) the dummy; (b) a school model and (c) courgette. The four objects are shown in Figure 2.10
pdf which is estimated from its octree, and the clusters in each slice are connected to its neighbouring clusters in adjacent slices by the Bayesian decision making rule in order to define local convexities. Finally, a convex hull algorithm creates local surfaces which are combined to complete the surface construction.
The experimental results show that LCH produces quality surfaces with good performance. Its approximation efficiency is better than those produced by other algo- rithms, e.g., MC, VMC, DT and LDT, requiring a reasonable CPU time. However, its peak memory usage is higher than CH and DT because the method needs to store local connection data. Also, any concavity in the xy plane may disappear in LCH, i.e., the concavity of a cluster in a slice is regarded as a 2D convex. This problem can be allevi- ated by dividing a 2D non-convex cluster into convex regions - it is like a 2D version of LCH. Another issue of the proposed method is the quality of the triangular patches, i.e., elongated or thin patches are caused by using a small slicing level and the convex hull
algorithm. However, it can be improved by inserting refining points into the side of the thin triangles.
Chapter 7
Conclusion and future work
7.1
Conclusions
This thesis presents a robust surface modelling method of a volumetric data reconstructed from multiple views. To achieve this goal, it investigates various computer vision algo- rithms in each chapter, e.g., robust image segmentation (Chapter 2), 3D reconstruction from multiple views (Chapter 2), non-linear model optimisation for a projection transform estimation (Chapter 3), 2D Delaunay graph for data clustering (Chapter 4), Hausdorff distance for cluster matching (Chapter 4), image descriptor for robust matching (Chap- ter 4), 2D convex hull algorithm (Chapter 5), the epipolar transfer in an image triplet (Chapter 5), MC, 3D convex hull, and 3D Delaunay for surface construction from volu- metric data (Chapter 6). As a result of the research, this thesis proposes new approaches, which are required to accomplish the final goal, such as the projection matrix estima- tion from an approximate circular motion (Chapter 3), similarity invariant point cluster matching (Chapter 4), affine invariant clique descriptor matching (Chapter 4), unknown image calibration from point correspondences in an image triplet (Chapter 5), and 3D surface construction from non-convex volumetric data using the Bayesian decision making on sliced volumetric data (Chapter 6).
The volumetric reconstruction method presented in Chapter 2 approximates a 3D shape from multiple back-projection of silhouettes, and a result obtained by this approach is collectively referred to as SfS. Thus, the first process of SfS is the preparation of
silhouettes, which are produced by thresholding the background from an object image. For a more robust silhouette detection, the largest segment in the binary image is considered as a silhouette candidate, to which the seed-fill algorithm is applied to remove internal noise. As a volumetric representation, this thesis exploits an octree structure of voxels, and two octree construction algorithms are introduced in Chapter 2. The octree construction method I provides a straightforward implementation of an octree construction, i.e., a volumetric result is constructed by octants only from the last generation. Thus, this method is fast when an object has complex shape. Method II is more memory efficient and faster when an object has simple shape, because it exploits the intermediate octants results. Additionally, Chapter 2 also reviews the latest developments in SfS, such as plane sweeping, voxel colouring and space carving.
The octree construction in SfS should have the projection transforms associated with silhouette images. Thus, Chapter 3 reviews a linear projection model of a digital camera, including some useful approximation of the projection models (e.g., affine pro- jection models). The projection is a mapping process from a point in a 3D space to a point in a 2D image domain. Thus, once a projection model is decided, the DLT al- gorithm estimates the linear solution of a projection model from data observations, i.e., 3D-to-2D point correspondences. Moreover, this solution can be further optimised by the LM non-linear optimisation. To facilitate the estimation of projection transforms in a multiple-view system, Chapter 3 introduces a method which relates the given knowledge of camera motion to the projection estimation. Since a SfS technique normally captures silhouette images from a circular motion, projection matrices are parameterised in terms of a rotation angle from the reference position. When some of the images in a circular motion violates the assumption of the pure rotation, the proposed method modifies them. This modification are estimated from 2D point correspondences between views. There- fore, Chapter 4 explores distinctive image features and a matching algorithm, which are particularly robust to a 3D camera motion.
It is generally assumed that a projection of a plane-like surface is well modelled as an affine transform, so that adjacent views in SfS can be related by a 2D affine transform. Thus, image matching in SfS should address an affine distortion, and as a solution for this, two image matching algorithms are proposed in Chapter 4. Both algorithms have
been developed from the Hausdorff distance which copes with cluster matching, because the proposed methods are motivated by a hypothesis that clustered features can enhance matching performance more than conventional point matching methods. The first match- ing method is invariant up to a similarity transform, and it extracts matching features from the distribution of detected point features (i.e., image texture is not investigated in this matching). It exploits the geometric attributes of a local point cluster, such as angles and local distances, and the result is better than other PPM algorithm. A lo- cal cluster (called a clique in this thesis) is determined by a 2D Delaunay graph, which uniquely determines a triangular graph from points. However, since a Delaunay graph is only invariant up to the similarity transform, the proposed Delaunay graph based match- ing cannot deal with significant affine distortion. To address this, the second matching method, called the clique descriptor matching includes affine invariant feature descriptor. The proposed clique descriptor matching searches for affine invariant regions by a MSER detector, and these regions are then described by SIFT descriptor. The proposed clique descriptor matching detects more tentative correspondences when images are affected by a 3D camera motion.
The affine invariant feature detector presented in Chapter 4 can also be used to improve the quality of an initial VH when additional images are provided. Since each additional image constructs a silhouette cone if a projection matrix is given, an initial VH can be more confined by new views of an object. However, this approach requires projection matrices for the additional new views. Moreover, these new projection matri- ces are defined on the same 3D world frame used in the initial reconstruction. To solve this problem, an image triplet (i.e., two images involved in an initial reconstruction and an additional new image) is investigated to calibrate a new image. In the proposed image calibration algorithm, 3D-to-2D point correspondences are located by a linear triangu- lation and the epipolar transfer, after the clique descriptor matching determines the 2D point correspondences between every two views. Consequently, the proposed calibration method in Chapter 5 shows how to collect point data for the calibration from the stereo reconstruction and three-view geometry.
Once the volumetric data is ready, it is better to extract surface meshes from the VH for the efficient 3D visualisation. Therefore, in Chapter 6, some fundamental surface
construction algorithms (e.g., MC, 3D convex hull, and 3D Delaunay) are explained as a literature review, and the various surface results of SfS by these methods are compared. In general, MC performs well unless there is noise in silhouettes and projection transforms. Otherwise, the surfaces extracted from SfS often fail to construct a closed surface, i.e., the resulting surface contains artefacts such as open meshes or unattached 3D segments. This is because the traditional MC algorithm premises a status of a vertex of a voxel (i.e., an inside or an intersection octant) is clearly classified. On the other hand, even when an image contains some noise, the corresponding volume data is almost intact, since the oc- tree construction is more robust to these errors than the surface construction. Therefore, Chapter 6 proposes a robust surface construction method for non-convex object, which is especially useful when accurate modifications of silhouettes and projection transform are not available. In the proposed method, the volumetric data obtained from SfS are sliced, and the connection of clusters on each slice are estimated. This a priori knowledge of slice connections is exploited when clustering imperfect surface vertices locally. A local 3D point cloud defines a local convexity to which the 3D convex hull algorithm is applied. Finally, these local surfaces are combined to complete the surface construction.