After having estimated Λx, the uncertainties in features position from (5.9) and
(5.10), and the uncertainties in the initial rotations and translations ΛR and ΛT from
(6.28) and (6.29) respectively, robust L∞ motion estimation is then performed. For
each consecutive image pair, the whole algorithm can be depicted in the following operations:
• Estimating the 3D position for each point Xi using robust convex L∞ tri-
angulation and including the covariance matrices Λxi, ΛT and ΛR (Section 6.8.2);
• Estimating the covariance matrix ΛXi for each recovered 3D scene point Xi using (6.32) (Section 6.7);
6.8. Robust L∞ Motion Estimation Solution 145
• Estimating the camera parameters using robust convex L∞ resectioning and
including the covariance matrices Λxi and ΛXi (Section 6.8.3).
These operations are repeated until the L∞ re-projection errors reach a satisfactory
minimum.
6.8.1
Computational cost
It is known that the computational complexity of the Levenberg-Marquardt algorithm is: O((m + n)3), where m is the number of cameras and n is the number of 3D scene
points [82]. In our algorithm, at a given time step, we will be dealing with either the triangulation problem or the camera parameters recovering problem for which they are solved using a bisection algorithm for feasibility checks. The triangulation problem has a computational complexity of O(m1.5) and a memory requirement of
O(m), where m is the number of cameras in which the triangulating point is visible [130, 135]. Similarly, the camera parameters recovering problem has a computational complexity of O(n1.5) and a memory requirement of O(n), where n is the number
of scene points. Therefore, for one whole update of the proposed solution, when dealing with n scene points and m cameras, the computational complexity is given by: O(mn(√m +√n)) and the memory requirement is O(max(m, n)) [130].
The number of iterations is crucial in this kind of problems, where a bisection search is performed. In our solution, defining the initial diameter of the second order cones plays a very important role in determining the number of the required iterations. Hence, the upper and the lower parameters of the bisection algorithm are chosen so the search area is reduced. A memorable search, based on the previous iteration parameters is performed, where these parameters are chosen in relation with the previous iteration results. This technique significantly reduces the number of iterations and hence the time consumption. This makes our solution comparable to the classical L2 Bundle Adjustment (BA) in terms of time consumption and in the
same time it recovers the global minimum of the cost functions, which is a valuable advantage over the classical L2 BA.
6.8.2
Robust L
∞triangulation with uncertain data
In order to provide an optimal solution to subsequent L∞ robust motion estimation,
a need for a robust and efficient optimal triangulation algorithm will be crucial. Our approach is based on robust convex optimisation with L∞ norm, taking in
consideration all sources of uncertainty. Our uncertain data are bounded, therefore, we will be looking for optimal solutions, which are feasible for any realisation of these data.
146 Chapter
6.
Robust L∞ Convex OptimisationAs described in Section 6.7, the function f in (6.30) relates all parameters of the triangulation between two cameras positions Pi, i = 1, . . . , m, where m is the number
of cameras. In this problem, the aim is to recover the value of X that minimises the maximum of the re-projection error across all images:
εi = d(ˆxi, PiX), i = 1, . . . , mˆ (6.33)
where d denotes image-space Euclidean distance between two points in the image plane, the measured and the projected points.
Given the camera matrices Pi = [Ri|Ti], where Ri = [ri1, ri2, ri3], Ti = [ti1, ti2, ti3],
ˆ
X = [X, 1] and their corresponding image points ˆxi = (x⊤, 1)⊤= (ui, vi, 1)⊤; and by
considering the uncertainties in the image points positions ∆xi = (∆ui, ∆vi), in the rotation matrices ∆Ri and in the translation vectors ∆Ti from covariance matrices Λxi, ΛTi and ΛRi respectively; the L2 norm of this re-projection error function is
given by: Fi(X) = (ui+ ∆ui) − (r⊤i1+∆r⊤i1)X+(ti1+∆ti1) (r⊤i3+∆r⊤i3)X+(ti3+∆ti3) , (vi+ ∆vi) − (r⊤i2+∆r⊤i2)X+(ti2+∆ti2) (r⊤i3+∆r⊤i3)X+(ti3+∆ti3) 2 (6.34) It is shown in [80] that this type of cost function clearly has multiple local minima. This creates a problem to iterative minimisation methods, such as the Levenberg-Marquardt (LM) algorithm, as they converge with high probability to a local minimum. To get around this issue, this problem is formulated within a quasi-convex optimisation framework by using the L∞ norm instead. Then, the
projection error will be then given by:
G(X) = maxiFi(X) (6.35)
For a scene point X to be visible as image points xi, it must obviously lie in front
of all cameras Pi, i = 1, . . . , m. This implies the constraint g(X, Pi) > 0 for all i,
where g(X, Pi) = (ri3+ ∆ri3)⊤X + (ti3+ ∆ti3). Our optimisation problem is then
given by:
min
X G(X)
subject to g(X, Pi) > 0 ; ∀i = 1, . . . , m
(6.36)
The m error residuals in (6.33) give the error vector ε = (ε1, . . . , εm)⊤. The
estimated scene point is then the vector X that minimises the norm of this error vector. Section 3.7.1 (Chapter 3, page 69) gives a detailed proof that the problem is a quasi-convex optimisation problem. This optimisation problem is solved using a
6.8. Robust L∞ Motion Estimation Solution 147
sequence of robust SOCP feasibility problems similar the problem given in (6.3). This robust SOCP is capable of dealing with the feature uncertainties and recovering a robust solution [116]. This leads toward using a bisection search to find the minimum value of δ, for which the optimisation problem is feasible.
The recovered 3D points using this robust L∞triangulation and their uncertainties
are then used in the optimisation algorithm for recovering the camera parameters, which is described in the next section.
6.8.3
Robust L
∞camera resectioning with uncertain data
Camera resectioning problem is detailed in Appendix A (Section A.2, page 281). Firstly, let us assume that n scene points ˆXi are given, with their uncertainties
∆Xi, which are mapped to image points xi with their uncertainties ∆xi, via the camera projection matrix P . This mapping is written as ˆxi = P ˆXi, for i = 1, . . . , n.
According to Definition 8 (page 281), the problem of camera resectioning is the problem of recovering the 3×4 matrix P , such that ˆxi = PiXˆi, for all i, that minimises
the maximum of the re-projection error across all points:
εi = d(ˆxi, P ˆXi), i = 1, . . . , n (6.37)
where d denotes again the image-space Euclidean distances between two points in the image plane, the measured and the projected points. By introducing the uncertainties in feature position ∆xi = (∆ui, ∆vi), and those in their 3D corresponding points
∆Xi; the L2 norm of this re-projection error function is given by:
F (X) = (ui+ ∆ui) − p1⊤(X i+∆Xi) p3⊤(X i+∆Xi) , (vi+ ∆vi) − p2⊤(X i+∆Xi) p3⊤(X i+∆Xi) 2 (6.38)
where pj denotes the jth row vector of P .
Again, this cost function is highly non-linear and has multiple local minima, where iterative algorithms, such as Levenberg-Marquardt (LM), can easily get trapped in one of these local minima. Similarly to the triangulation problem, to get around these drawbacks and to recover robust and global solutions, the L∞ re-projection error is
used. The global minimum can be obtained by solving the following optimisation problem:
min
X maxi Fi(X)
subject to p3⊤(Xi+ ∆Xi) > 0 ; ∀i = 1, . . . , n
(6.39)
This is again a quasi-convex optimisation problem and can be solved using technique presented in Section A.2 (Appendix A, page 281).
148 Chapter
6.
Robust L∞ Convex OptimisationThe recovered camera parameters and their uncertainties are then used again in the subsequent robust convex L∞ triangulation algorithm and so on as detailed in
Section 6.8.