Previously when estimating planes, the value assigned to d was arbitrary, only providing scale. When estimating multiple planes in the same scene, this condition only applies to one of them, hereafter referred to as the “reference plane”. All additional planes introduce a further parameter to the estimation procedure – their scale relative to d. Therefore the set of parameters to be solved within a bi-planar system is now (α, θ0, ψ0, θ1, ψ1, d1).
Including d1in the estimation does not in itself prevent drift between the planes; it merely
allows the second plane to vary in scale. To this end, a further prior is included in the error function for estimation. Let u1 and u2 be two image points on the line of intersection
between the two planes. In addition, let x∗1 and x∗2 be those points back-projected with the estimated parameters for the reference plane. The prior dictates that the second plane, when reconstructed using the estimated parameters, passes through these two points. In the following, let n1 be the normal for the second plane, and d be its distance from the
−5 −4 −3 −2 −1 0 1 2 3 4 5 −3 −2 −1 0 1 2 3 −9 −8 −7 −6 −5 y x z Estimated Plane 1 Estimated Plane 2 True Plane 1 True Plane 2
Figure 4.9: Given clean data and using the SVM based line estimation method a perfect estimation of the true planes can be calculated (see figure 4.8). However; introducing variation in intra-trajectory speed skews the estimation causing an imperfect estimation. Under these conditions the two planes drift apart from each other, indicating some crite- rion is required to enforce alignment at the plane boundary.
−0.5 0 0.5 −0.4 −0.20 0.20.4 −1.6 −1.4 −1.2 −1 −0.8 −0.6 −0.4 −0.2 0 y (world) x (world) z (w o rl d ) (a) −0.5 0 0.5 −0.4 −0.20 0.20.4 −1.6 −1.4 −1.2 −1 −0.8 −0.6 −0.4 −0.2 0 y (world) x (world) z (w o rl d ) (b)
Figure 4.10: By applying a prior to the optimisation function, dictating that the plane should pass through a point on the boundary line (rectified using the hypothesis for the reference plane), the reference and second planes now intersect at the boundary. (a) shows the estimation without the prior and (b) shows the same data but with the second plane estimated with the prior applied. The camera is illustrated by the dark-grey pyramid.
distboundary=
∑
i∈(1,2)
|n1x∗i − d|
kn1k
(4.5)
Figure 4.10 illustrates the effectiveness of this prior on an estimation of parameters using input data under 20% speed variation. The second plane is estimated such that it cor- rectly intersects the back-projected boundary line and the two planes are suitably aligned. Whilst this method proves to be effective, it is inefficient. The system to be solved is being constrained by the prior, but the same number of parameters must still be solved to estimate the orientation of the second plane.
Instead reconsider the situation. Given an approximation of the reference plane and the boundary between it and a connected plane, the orientation of the second plane is simply a rotation of the unit-normal of the reference plane about the back-projected boundary line. In this manner, intersection at the boundary can be enforced and the number of unknowns to be solved can be minimised. The following chapter will detail how this was achieved and how accurate the solution was.
4.5
Conclusions
The contribution of this chapter has been an iterative framework for estimating the rela- tive position and orientation of planes in the world, given the prior knowledge that there are exactly two in the view of the camera. The presence of multiple planes introduces problems that do not arise in a single-plane world – namely one can no longer use input trajectories from across the entire scene and that finding the location of the intersection line between the two planes relies on reasonable estimates for the plane orientation pa- rameters and the camera focal length.
Several methods were discussed for locating the intersection line. Exhaustive search is a possibility, but it is computationally expensive, even when simplifying assumptions are introduced to minimise the search space. Including the boundary parameters into the non- linear optimisation works to some extent with perfect data; however as the boundary line and plane parameters are dependant on one-another, noisy data such as that likely to arise from real-world situations badly affects the quality of the result. The most promising method involves training a Support Vector Machine, using a minimal-error independently labelling assigned to each region. The SVM approach was the most robust to noise as shown in results on controlled simulated data.
The limitation of this framework as currently described is the necessity for there to be exactly two planes in the scene. Instead it would be more useful in a real-world scenario to automatically determine the number of planes prior to segmentation. This problem is the focus of the following chapter, in which the existing framework is generalised to account for any number of flat planes in the scene.
Reconstruction of Hinged Planes
The previous chapter discussed various methods to determine the parameters of two planes and the boundary line between them. It also highlighted the issues associated with multi-planar estimation, namely that given error in the input data, the two planes rectified with estimated parameters may not align correctly. The end of the chapter al- luded to an alternative approach to solving the multi-planar estimation problem whereby the parameters of the second plane are calculated as a rotation of the reference plane’s unit-normal about the boundary between the two planes. This chapter will discuss this concept in the domain of a bi-planar world, before extending the technique to apply to a more generalised world where the number of planes is not known in advance.
5.1
Hinging in a Bi-Planar World
In section, 4.3, a framework was introduced for determining the intersection line between two planes using a variety of methods. Hereafter, the SVM-based method shown in sec- tion 4.3.3 is used to find the boundary line. Once an estimate for the reference plane and the boundary line are obtained, the estimation of the second plane is reposed using the hinged formulation discussed below.
Given the estimated parameters for the reference plane θ and ψ, it is trivial to obtain the
−0.2 0 0.2 −0.6 −0.4 −0.2 0 0.2 0.4 −2 −1.5 −1 −0.5 0 0.5 (a) −0.2 0 0.2 −0.6 −0.4 −0.2 0 0.2 0.4 −2 −1.5 −1 −0.5 0 0.5 (b)
Figure 5.1: Ground-truth (left) and estimated (right) reconstruction of a bi-planar scene using the hinged estimation framework. Input data here was “perfect”, i.e. there was no speed variation. Quantitative analysis of the estimated parameters is given in table 5.1.
−0.2 0 0.2 −0.6 −0.4 −0.2 0 0.2 −2 −1.5 −1 −0.5 0 (a) −0.2 0 0.2 −0.6 −0.4 −0.2 0 0.2 −2 −1.5 −1 −0.5 0 (b)
Figure 5.2: Ground-truth (left) and estimated (right) reconstruction of a bi-planar scene using the hinged estimation framework. Input data was under the influence of 20% intra- trajectory speed variation. The result is still very close to the ground-truth although there is some orientation error. Quantitative analysis of the estimated parameters is given in table 5.1.
Table 5.1: Quantitative comparison of true and estimated parameters for a two-plane scene at 20% speed variation, pictured in figures 5.1 and 5.2.
Plane 1 Plane 2
θ ψ θ ψ α
Ground Truth 30.0 15.0 49.2 75.1 0.0014
0% Speed Variation 32.2 12.1 50.0 78.7 0.0014
20% Speed Variation 39.4 16.0 42.7 69.6 0.0017
estimated normal to the reference plane, n0 using the mapping presented in chapter 3,
equation 3.1. The boundary line rectified with the estimated parameters is denoted by the direction vector b and a point upon it xb∗.
Assume the angle of rotation φ about the boundary line, between reference plane and the second is known. Finding n1, the normal of the second plane, is simply a case of rotating
n0about the boundary. In order to solve for d1, a point known to lie on the rotated plane
is needed. As the boundary line is known to lie upon the second plane, any point upon it xb∗ can be used. Therefore, given n1 and xb∗, d1 can be found by the plane equation
n1◦ xb∗= d1.
Finding the optimum rotation is a simple extension of this method. Searching values of φ between 0 and 180◦ allows us to find the angle that produces a plane orientation minimising the trajectory error function given in the previous chapters. Not only does this method of search improve upon the previous iterative method discussed in chapter 4, it is less computationally expensive as there is only one additional unknown to search instead of three.
Qualitative results are given on perfect data in figure 5.1 with figure 5.2 showing the result at 20% speed variation. The results are quantified in table 5.1. Note that whilst the estimations obtained by this algorithm under the influence of speed variation are lower (since the system has to skew both planes to allow them to align correctly), the actual reconstruction is closer to the true solution due to the hinging constraint.