CoFFrS builds upon and depends on the results of the segmentation pipeline described in the previous chapter. This section provides a number of examples showing the behaviour of the segmentation process on a number of indoor scenes which are either taken from previous work by Nan et al. (2012) or are newly acquired. The first three columns in Figure 7.6 illustrate the segmentation process carried out on point clouds representing two separate chairs and an indoor office scene (Nan et al., 2012). The third column illustrates the *·planar segments resulting from this process visualised using different colours.
Figure 7.4: Top left corner shows office room from previous literature (Nan et al., 2012), top right corner shows table top scanned using the Asus Xtion sensor with the Skanect software, middle row illustrates the point cloud of an office acquired using the Structure sensor and Skanect software and, bottom row illustrates the point cloud (synthesized from a triangle mesh) of the conference room
Figure 7.6: Segmentation process on two chairs (single objects) and office (multiple objects in enclosed space) scenes with columns from left to right (a) all points, (b) edge type points, (c) segmentation result - shown as coloured surfaces, (d) close-up view
PaRSe first labels points, then partitions P using a region growing algorithm and applies a RanSaC plane fitting process over resulting regions. This is partic- ularly useful in the case of indoor scenes where noise in the acquired point cloud and variability in point density can lead to scenarios where points which should be tagged as edge are actually tagged as surface and vice-versa. In the former case, this leads to situations where a surface segment spans over multiple object surfaces. Figure 7.7 on the left, illustrates an example where only one surface segment is created for the right-most blue couch following the region growing process. This segment effectively contains all the points on the couch and could be useful on its own. However, in order to be able to describe a couch in terms of segments and connectivity between them, all couches in the scene need to be composed of a similar set of segments. Figure 7.7 on the right, illustrates the segments resulting from the RanSaC process. Note how on both couches, the seat is further split in twosurface·planar segments to match the slight curvature. PaRSe produces a set partition S = {s1, s2, s3, . . . , sn} of P consisting of
elements with type surface·planar,edge·planar, surface·complex oredge (§5.1.2). A structure graph G is built over these segments using adjacency information obtained during the region growing process and OBB (§2.5.3) intersection tests during RanSaC plane fitting. Segments of type *·planar play a critical role in our scene understanding approach. Transitions in G between these segments are augmented with properties in the form of hkey, valuei pairs, e.g. hdot,0.02i to indicate that the nodes connected by this arc are nearly orthogonal. hkey, valuei
pair properties are also attached to nodes and include number of points, plane orientation, area spanned by OBB, points coverage on surface and spatial context
Figure 7.7: Scene from Nan et al. (2012) - a) over segmentation of right sofa and b) new segments created after RanSaC plane fitting
{ seat } { seat ,
hand rests }
{ seat , hand rests, back }
{ seat , hand rests ,
back , front legs }
region growing segment
*.planar segments
Figure 7.8: Region growing segment includes the whole armchair, whereas RanSaC plane fitting subdivides this region into four*·planar segments.
information. Spatial context is used to determine the approximate location of the
*·planar segment along its normal direction within the object or scene, and is set to eitherboundary,central orboundary central. Figure 7.9 illustrates an example showing how spatial context is computed. In the case of object segmentation, which is described in the next section, spatial context is computed with respect to the entire point cloud. In the case of scene segmentation, spatial context for
*·planar segments is also computed with respect to the region in which they belong. Points coverage is used to measure how points are distributed over the
*·planar segments. This is done to further discriminate between segments which might have similar OBB areas but with points clustered in specific parts of the OBB. Figure 7.10 illustrates a number of examples showing how point coverage is computed. The upper part of the figure shows four OBBs enclosing the side of a chair, a triangle, a circle and a rectangle. The areas of the OBBs of the rectangle
boundary = { , }
central = { , , }
centr
al
boundary boundary boundary
centr al boundary centr al boundary central = { }
Figure 7.9: Spatial context indicates the approximate location of each segment within the point cloud being scanned which could either represent a scene of an individual object. In the case of a scene, if the planar segment forms part of a region induced by the region-growing algorithm, spatial context is also computed with respect to the points in the region.
and chair are very similar, however the point coverage for the latter is lower. Similarly, for the triangle and circle shapes, where the circle has a slightly higher point coverage value. Point coverage is a normalised value and is computed by tracing orthogonal rays from a moving virtual camera above the area spanned by the OBB. A low discrepancy sequence, generating a number of camera positions above the OBB, is used to make sure that sampling is distributed over the OBB. Point coverage is calculated as the ratio of rays traced from the moving camera over the number of point intersections. If no intersections occur, points coverage for the segment is assigned a value of 0, whereas if all rays intersect a point in the OBB, point coverage for the segment is assigned a value of 1.
The bottom left hand side image of Figure 7.5 shows an up-sampled version (§2.6.2), from ∼200K to ∼600K points, of the middle row point cloud. Clearly, simply increasing the number of points by interpolation does not contribute ad- ditional information. Note however, that there might be situations where up-
High Coverage
Low Coverage
Figure 7.10: An OBB does not provide information about the distribution of enclosed points. Coverage is used to provide an indication.
sampling can affect the outcome of PaRSe. If interpolated points are added over a planar surface, for instance the top of a table, then these will not make any difference in terms of segments produced but only increase the number of points in specific segments. However, when interpolating points on a curved surface, additional planar segments may be introduced thus affecting the resultant set partition and the mappings carried out by CoFFrS. In the results section, no up- sampling is carried out and CoFFrS is directly applied on the raw point clouds produced by the scanner.
Figure 7.11: Example point cloud from Nan et al. (2012) representing five chairs with different poses. Point in raw data are first assigned to either Ps or Pe. Second im-
age from top is showing points in Pe. Region growing process generates the regions
shown using different colours in the third image and finally RanSaC produces∗.planar segments.