they just used default values (V0[xik] = diag(1, 1, 0)). In this work however, features covariances are estimated for each RGB channel, and then the covariance intersection filter is applied to fuse the three estimates. Therefore, feature uncertainties are reduced and consequently more features, which are optimally matched as explained in Section 5.6.1, will be able to contribute to the final estimate of the fundamental matrix. This method will lead to more robust and accurate estimates.
After robustly estimating the fundamental matrix, motion parameters (translation vectors and rotation matrices) are then extracted using our techniques introduced in Section 4.5 (Chapter 4, page 83).
5.7
Experimental results
This section presents the experimental evaluation of the proposed method using covariance intersection in motion estimation. Uncertainties in feature positions have been taken into consideration in this implementation. First, we illustrate the need for separately dealing with each RGB channel of colour images, and then using the covariance intersection filter. To do that, the error function used to evaluate the performance of the proposed solution is defined by:
fF = 1 n n X k=1 d (xjk, F xik) 2 + dxik, F ⊤ xjk 2 (5.25)
where d(xjk, F xik) is the Euclidean distance error between the point xjk and F xik. Note that the quantity F xik is in fact the epipolar line corresponding to xik (Section C.1, page 293). Since the fundamental matrix defines a point-line mapping, the error is the average over all n matches of the squared distance between the epipolar line of a point and its matching point in the other image.
The implementation of the proposed technique is conducted on real data from different environments (Section 1.6, Chapter 1, page 8), showing the diversity of the image qualities. The first dataset is gathered from a vehicle travelling in an urban environment in the city of Karlsruhe [68] (Figure 5.8a). The second is a sequence of images taken from our laboratory (Figure 5.8b) and the third one is a collection of data gathered at a Mars/Moon analogue site on Devon Island, Nunavut [64] (Figure 5.8c). Few matching features along with their uncertainties are shown in this figure (Figure 5.8).
The corresponding results of these experiments are given in Figure 5.9. For each environment, residual errors are shown against the number of matched points used to compute F . Note that these errors are calculated over all matched points
118 Chapter
5.
Robust Motion Estimation(a) Urban environment
(b) Indoor environment
(c) Moon/Mars analogue environment
Fig. 5.8 Examples of image pairs used for comparison. Features and their uncertainties in these environments are extracted using the SIFT algorithm. Few matched points are illustrated. Error bounds for each channel are given as ellipses coloured according to the channel from which they were estimated.
5.7. Experimental Results 119 0 10 20 30 40 50 60 70 80 90 100 0 1 2 3 4 5 6
Uncertainty evaluated from CI of RGB channels Uncertainty evaluated from grey levels
Number of points
Errors
(pixels)
(a) Urban environment
5 10 15 20 25 30 35 40 45 50 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 Number of points Errors [pixels] (b) Indoor environment 0 10 20 30 40 50 60 70 80 90 100 0 1 2 3 4 5 6 Number of points Errors [pixels]
(c) Mars/Moon analogue site
Fig. 5.9 Errors plotted against the number of points used to compute the fundamental matrix. Dashed lines (blue) show the results using covariance matrices evaluated from the grey levels. Solid lines (red) show results using covariance intersection to fuse uncertainties from all the three RGB channels of colour images. In all environments, using covariance intersection is noticeably better than using grey-levels, though the latter also shows good results.
120 Chapter
5.
Robust Motion Estimation and not just the ones used to compute F . To illustrate the advantage of using colour information, comparison between using the covariance intersection over the three RGB channels for estimating the feature uncertainties and using just the grey levels is shown. Figure 5.9 shows the significant impact on the final estimate of the fundamental matrix when using the decreased feature location uncertainty via the covariance intersection. One can see that the improvement is not considerable in the Moon/Mars analogue environment. This is due to the image qualities and also to the nature of the landscape. However, this is not the case for the remaining environments, where considerable improvement can be noticed. Therefore, including features position uncertainties yields to remarkably better results. Obviously, for all environments, the accuracy is improved as the number of matched points used to estimate F is increased.These results are summarised as well in Table 5.1.
Table 5.1 Average residual errors in pixels for each environment using covariance in- tersection for feature uncertainties of all RGB channels (ERGB) and using covariances
from grey levels (EGray)
Egray ERGB
Urban environment 1.88 1.16
Indoor environment 1.36 0.74
Moon/Mars analogue environment 2.17 2.02
Comparison with similar algorithms for estimating the fundamental matrix, such as the 8-point algorithm, is conducted. Figure 5.10, shows the residual errors from the urban environment experiment as an illustrative example. The proposed solution uses the renormalisation technique with reduced feature uncertainties. This uncertainty reduction is obtained by employing the covariance intersection to the feature location errors. This technique iteratively removes bias of weighted least squares, instead of minimising a cost function.
Both methods, the 8-point algorithm and the proposed technique, use the same well matched points. It is clearly shown in Figure 5.10 that when using the covariance intersection, more features will be able to contribute to the final solution. Thus, this leads to better estimates of the fundamental matrix.
Comparative experiments when using two feature extractors, such as the Harris corner detector and the SIFT algorithm, are conducted as well in this experiment. The two main investigated tasks in this part are the matching robustness and the uncertainty estimation of each detector. We have seen that the Harris corner detector relies on the image intensity changes to detect corners using the second moment matrix. Matching in the Harris corner detector algorithm is performed using the
5.7. Experimental Results 121
Number of points
Errors
(pixels)
Including uncertainty evaluated from CI of RGB channels Using Normalised 8-point algorithm
1 0 2 3 4 5 6 10 20 30 40 50 60 70 80 90 100 0
Fig. 5.10 The fundamental matrix estimation using the covariance intersection al- gorithm. Residual errors are plotted as a function the number of deployed points. Dotted blue line shows the results using the normalised eight-point algorithm. Solid red line shows results using the normalisation technique with reduced feature un- certainties by employing the covariance intersection. The residual errors from the covariance intersection algorithm are substantially smaller than those from the normalised eight-point algorithm, regardless the number points.
cross correlation technique between local image patches (Section C.2.1, Appendix C, page 299). This means that only features that correlate most strongly with each other in both directions are accepted. The SIFT features on the other hand, use the Euclidean distance between the feature descriptors as a similarity criteria, and use the nearest neighbour algorithm to match features (Section C.2.2, Appendix C, page 300). This technique has proved its efficiency among the computer vision researchers [128]. Note that while the Harris corner detector is able to match larger number of features in relatively shorter time, this significantly compromises its robustness. The SIFT robustly performs that, but relatively in more time.
Our experiments reveal that, even though, features uncertainties using the Harris corner are relatively smaller than those estimated using SIFT, the latter detector is more representative to the real uncertainties in the features positions. In our experiments, the average of the residual errors using the Harris corner detector is 1.48 pixels, while it is 1.16 pixel when SIFT is used. The obtained results are plotted in Figure 5.11. These results confirm that uncertainties from SIFT features are less conservative than Harris. Hence, more robust estimates are obtained, which justify our preference for the SIFT algorithm. A summary graph is illustrated in Figure 5.12 comparing the proposed solution using CI of uncertainty in the three RGB
122 Chapter
5.
Robust Motion Estimation 0 10 20 30 40 50 60 70 80 90 100 0 1 2 3 4 5 6CI of RGB channels using SIFT CI of RGB channels using Harris
Number of points
Errors
(pixels)
Fig. 5.11 Comparison between using uncertainties from the SIFT extractor and from the Harris corner detector features with the covariance intersection technique to estimate the fundamental matrix. Performance with the SIFT features is better than with Harris corner detector, though the latter algorithm also gives encouraging results.
channel of colour images with the solution using uncertainty from grey-level images, and with the traditional eight-point Algorithm.
5.8
Conclusion
In this chapter, a technique for robust and accurate estimation of the fundamental matrix is presented. In most vision applications, colour images are converted first to gray-level images leading to a serious loss of information. In our solution, however, each RGB channel of colour images is processed separately. Then, a fusion mechanism is employed to combine these information. After having estimated the uncertainties in feature locations in each channel, covariance intersection filter is used. This results on a reduction of the measurement error, leading to more accurate estimates of the fundamental matrix. The iterative technique for this matrix estimation is adopted, which takes as inputs the uncertainties in feature locations. Before estimating the fundamental matrix, the available feature uncertainty information is used as well to improve the matching task rather than using only the standard RANSAC algorithm.
Through several experimental results in different environments, we showed that including feature uncertainties from all three RGB channels of images leads to more accurate estimates of the fundamental matrix and consequently to more accurate estimates of the motion parameters. A comparative study between employing feature
5.8. Conclusion 123 0 10 20 30 40 50 60 70 80 90 100 0 1 2 3 4 5
6 Uncertainty evaluated from CI of RGB channels Uncertainty evaluated from grey levels
8-point algorithm
Errors
(pixels)
Fig. 5.12 A summary graph comparing the proposed solution using the covariance intersection algorithm on the uncertainty in the three RGB channel of colour images, the solution using uncertainty from grey-level images, and the traditional eight-point Algorithm.
uncertainties extracted from the Harris corner detector and the SIFT algorithm is given as well, where the latter exhibits better performance.
Chapter 6
Robust L
∞
Convex Optimisation
for Monocular Motion Estimation
In chapter 4, we presented a solution for motion estimation using convex optimisation in the triangulation task and the H∞ filter in the scale ambiguity problem. In Chapter 5, we introduced a solution that incorporates feature position uncertainties in estimating the fundamental matrix before recovering the camera motion parameters. In the present chapter, a more global and robust L∞ norm-based optimisation solution
for monocular motion estimation systems is presented. In addition to exploiting the uncertainty estimation techniques from the previous chapter, this solution propagates this uncertainty through the multiple-view geometry algorithms and incorporates it at each stage of the solution.
More precisely, we introduce in this chapter the robust convex optimisation notion in our solution, where the propagated uncertainties to the relative rotations and translations, and to the 3D scene points contribute in improving the global motion estimation. Rather than using the H∞ filter to solve the scale ambiguity problem in our monocular system, we set up a robust least squares algorithm using the SOCP approach, capable of handling the system uncertainties. Experimental evaluations showed that robust convex optimisation with the L∞ norm under uncertain data and
the robust least squares via the SOCP clearly outperform classical methods based on least squares and Levenberg-Marquardt algorithms.