Algorithm Implementation

The algorithm described in this section can be categorized as a hybrid approach that consists of three steps: classification, region growing and merging. In the first step, the points in the data are classified into edge, corner and surface points. Then in the second step, region growing is performed on the identified surface points. Our method using robust saliency features minimizes both over and/or under segmentation. Over segmentation occurs when more surface regions are detected than actually exist. Under segmentation occurs when more than one true region are combined. We prefer over segmentation in the second step as the problem of over segmentation can be overcome by merging incomplete regions in the third step. The three steps of the segmentation algorithm are now detailed in the following three subsections.

5.4.1.1 Classification

Classification means the separation of the point cloud data into sharp features i.e. edge and corner points, and surface points. If the neighbourhood of a point come from a noise or outlier-free planar surface, then the least eigenvalue λ0 estimated by the neighbourhood should have a value of zero or near zero. Usually, the points on or near the edge have a neighbourhood from different adjacent feature surfaces, and will have λ0 values considerably larger than λ0 values from the surface points. We can consider edges and corner points as the outlying cases compared with the surface points. We follow the general rule for finding outliers i.e. a point is considered as an outlier if:

where a = 1 or 2 or 3; depending on the data and needed for getting larger λ0 values. We consider the ith _{point that has}

λ0 >mean(λ0) + 1×StD(λ0) (5.5)

to be an edge/corner point. Although, the mean and StD are not robust measures, we wish to see the influence on them of outliers. We could use the median and MAD as robust alternatives to mean and StD respectively, and since they are robust measures, the influence of the outliers on them will be reduced because of their robustness quality. In Eq. (5.5) we want to be sensitive to the influence of outliers and/or sharp features. It is reasonable that the measure for edge and/or corner point detection depends on the data and neighbourhood size, so we cannot recommend using a specific threshold. We can determine the threshold values by using the percentile values of λ0 for the respective histograms and/or index plots such as for the example shown in Figure 5.9. The rest of the points with smaller λ0 values are considered as the surface points.

5.4.1.2 Region Growing

To extract planar surfaces, it is reasonable that points in the same region should have similar normal orientation and low bias angle θ between the neighbouring points (Hoppe et al.,1992;Powell et al.,1998;Woo et al.,2002;Mitra and Nguyen,

2003). If we remove the edge and corner points for the multiple planar surfaces, then we can easily find the points in the same planar surface using the region growing approach. It is sufficient to fix a small threshold for the bias angles between the normals of two neighbouring points on a surface. The bias angle between two neighbouring points is defined as:

θ =arccos|nˆ1·nˆ2|, (5.6)

where ˆn1 and ˆn2 are the two unit normals for the ith point and one of its neighbours. We exclude the edge points from region growing. The region growing starts by searching for a seed point, which has the lowest λ0 value in the data after eliminating edge points. It is reasonable to consider the ith _point as the seed point, which corresponds to the lowest λ0 value because the ith point and its neighbours should make the most planar surface for the whole data. We consider the least λ0 because we are using normals for region growing, and each

normal is related to the least eigenvalue for a point, and the least eigenvalue measures the variation along the normal to the tangent plane. Removing the edge points can give us an estimate of where the junctions between separating planes are in the data prior to region growing. We assume S is the surface points set and region growing commences from the seed point Spi with the lowest λ0 for the current region Rc. The points whose bias angles are less than an angle threshold θth are added to Rc and used as the next seed points for Rc and will be removed from S. We fix an appropriate θth so that we can avoid under segmentation and bias the region growing to over segmentation. If necessary, the problem of over segmentation can be overcome by merging. After getting a complete region, we select seed point Spi for the next region from the remaining points in S that has the minimum λ0. If a region contains greater than or equal to a minimum number Rmin of points it will be considered as a useful region, otherwise it will be considered as an insignificant region. The same process of region growing will be continued until the surface point set S is empty. The region growing process is summarized in Algorithm 5.2.

Algorithm 5.2: Region Growing for Multiple Planar Surface Extraction

1. Input: surface points set S (excluding edge points), normals, λ0,θth and Rmin.

2. Find initial seed point Spi from the S, which has minimum λ0, and put it into

current regionRc, current seed point list Sc, and remove from S.

3. Findk nearest neighbourhoodN pi for each seed point inSc.

(a) Calculateθ betweenSpi and its neighbours.

(b) Find the points in S that haveθ < θth.

(c) Put them into Rcand Sc, and remove fromS.

4. If size Rc is larger than or equal toRmin, insertRc into the region listR.

5. Repeat Steps 2 to 4 untilS is empty. 6. Sort the regions in R.

7. Output: sorted regions listR.

5.4.1.3 Region Merging

This step merges the neighbouring co-planar regions that are the consequence of over segmentation and should belong to the same feature or object surface. It is assumed that merging a specific region with a larger and most appropriate or

closest neighbouring region should change the Mean Squared Error (MSE) less than the MSE for merging with any other distant or inappropriate region. The MSE for a region can be defined as inPoppinga et al. (2008):

MSE = 1 l l X i=1 (ˆn·pi+d)2, (5.7)

where l is the size of the sample, ˆn is the unit normal, and d is the bias or distance from the origin to the plane for the region. To avoid faulty merging, a threshold is fixed based on knowledge of the data or determined from experiments for similar types of data so that the least Difference in MSE (DMSE) does not exceed a DMSE threshold DMSEth. The merging procedure is summarized in Algorithm5.3.

Algorithm 5.3: Region Merging for Over Segmented Regions

1. Input: sorted regions in R from the Algorithm5.2, and DMSEth.

2. Find neighbouring regions Rij that have the same edge or border points for each

regionRi ∈R.

3. Calculate the MSE forRi and Ri∪Rij.

4. Calculate DMSE =|MSE(Ri)−MSE(Ri∪Rij)|, for all Rij. (5.8)

5. MergeRij withRi for which DMSE≤DMSEth, and remove Rij fromR.

6. Sort the regions in R.

7. Output: sorted regions listR after merging.

5.4.2

Experiments

The proposed segmentation (classification, segmentation and merging algorithms) method are evaluated through simulated and real MLS point cloud datasets. We perform the algorithms for edge detection or classification of points, region growing and merging based on the saliency features: normal and λ0, that are estimated by using PCA, RANSAC and RDPCA. We label the segmentation results as PCA, RANSAC and RDPCA depending on the saliency feature estimation methods used in the proposed segmentation process.

5.4.2.1 Simulated Data

Dataset 5.1: Unit cube dataset

We simulate a unit cube as shown in Figure5.8a that has six surfaces consisting of 6,000 points, for which every surface is generated as an individual planar surface in 3D. We perform PCA, RANSAC and RDPCA to getλ0 and the normal for every point with a neighbourhood size k= 30. Using Eq. (5.5), we determine the edge points for the cube as shown in magenta in Figures 5.8(b, c and d). Results show that PCA (Figure 5.8b) and RANSAC (Figure 5.8c) identify many surface points wrongly as edge points. However, Figure 5.8d shows RDPCA identifies edge points more accurately. We perform Algorithm 5.2 with θth = 2◦ and Rmin = 10 to extract the six surfaces. The results in Figures 5.8(e, f and g) show that the algorithm properly segments all the planar surfaces. The segmentation results shown in Figure 5.8g that are based on RDPCA normals andλ0 values are better than PCA (Figure 5.8e) and RANSAC (Figure 5.8f). Even RANSAC produces several over segments as shown in the ellipses in Figure 5.8f.

Figure 5.8 (a) Simulated unit cube. Edge (in magenta) detection: (b) PCA, (c) RANSAC, (d) RDPCA. Segmentation results: (e) PCA, (f) RANSAC, over segmented regions are shown by ellipses, and (g) RDPCA.

5.4.2.2 Real MLS Data

Dataset 5.2: Road-kerb-footpath-fence dataset

We consider a small point cloud dataset shown in Figure 5.9a that consists of 2021 points acquired by using a moving vehicle-based mobile mapping system. It has four planar surfaces that are parts of a road pavement, kerb, footpath and fence. We name this the ‘road-kerb-foothpath-fence’ dataset. We find the edge points of the data. Based on the empirical study, we set the neighbourhood sizek = 30 and perform RDPCA Algorithm 5.1 to get robust saliency features. We also perform PCA and RANSAC for comparison. In Figure 5.9b, boxplots show that the λ0 values from RDPCA are much more robust than the λ0 values from PCA and RANSAC. Figures 5.9(c, d and e) and Figures 5.9(f, g and h) are the histograms and index plots of λ0 values for PCA, RANSAC and RDPCA respectively. We use Eq. (5.5) to draw the cut-off lines in Figures 5.9(f, g and h) for identifying the edge points (magenta) in Figures5.10(a, b and c). Results show that PCA (Figure 5.10a) and RANSAC (Figure 5.10b) cannot identify edge points properly with many surface points wrongly identified as edge points. However Figure 5.10c shows that RDPCA identifies edge points more accurately.

Figures 5.10(d,e and f) show the visual orientation of the normals for the detected edge points. We see PCA normals for the edge points in the blue marked box in Figure5.10d, which are smooth and too similar to differentiate them according to their position and to know from which planar surface the points come from. The normals for PCA look smoothed out because the neighbourhood for those points have many common points from two different surfaces. Since RANSAC is a robust method and saliency features are based on a consensus set within a local neighbourhood, it gives better results than PCA but Figure 5.10e shows differentiation is not satisfactory. In Figure 5.10f, normals from RDPCA are clearly separated into two directions that indicate from which points and their neighbours they come from, and represent the respective surface. Results for RDPCA also prove that the Algorithm 4.1

(Chapter 4) finds the maximum consistent set (MCS) accurately and produces robust saliency features for identifying edge points and proper segmentation.

Figure 5.9(a) Real MLS point cloud data, (b) boxplots ofλ0values for PCA, RANSAC

and RDPCA. Histograms ofλ0values: (c) PCA, (d) RANSAC, and (e) RDPCA. Index

plots of λ0 values (red lines indicate cut-off lines): (f) PCA, (g) RANSAC, and (h)

Figure 5.10Identification of edge points (magenta in colour): (a) PCA, (b) RANSAC, and (c) RDPCA; normals (red quiver) for the selected edge points: (d) PCA, (e) RANSAC, and (f) RDPCA.

We now use the proposed Algorithm 5.2 to segment the

road-kerb-foothpath-fence dataset shown in Figure 5.9a. Based on empirical

investigation on similar real MLS data, we set the parameters θth = 5◦ and Rmin = 10 to perform segmentation. Figure 5.11d shows that using RDPCA based robust saliency features the segmentation algorithm properly segments all the planar surfaces, whereas PCA based results shown in Figure 5.11a are over segmented with three segments for the kerb and fence surfaces. Over segmentation i.e. a single surface erroneously split into multiple surfaces, occurs because the calculated PCA normals have equal weight for all the regular and outlying points in the respective neighbourhood and hence may be influenced by outliers. On the other hand, RDPCA normals are based on the MCS, which can avoid the influence of the outlying cases in the neighbourhood. In the case of RANSAC, Figure 5.11c shows many points in the footpath surface that disappear from the segmentation results because they are wrongly identified as edge points and were removed before region growing.

Using Algorithm 5.3 and setting DMSEth = 1.0e−04, we perform the merging task for the over segmented regions for PCA in Figure 5.11a. After merging the regions, the PCA based segmentation results are shown in Figure 5.11b. We see all the regions in the fence and kerb surfaces shown in Figure 5.11a are accurately

merged in Figure 5.11b. However the RDPCA based segmentation results in Figure5.11d do not need any merging and are significantly better than the results in Figures 5.11b for PCA and 5.11c for RANSAC.

Figure 5.11 Region growing and segmentation: (a) PCA region growing, (b) PCA merging and segmentation, and (c) RANSAC segmentation, and (d) RDPCA segmentation.