Tests specificities

First, it was taken into account that, if it is necessary to test localization architectures, it is indeed needed some kind of system that can give a relatively good estimate of the vehicle position, so a comparison between the estimated pose and this estimate could be done, giving an estimation error. But, reliable systems as Vicon or Optitrack were not available, so an alternative was considered: the use of a marker-based localization system. The used package was ar track alvar¹, which is a ROS application presented as an evolution of AR Toolkit.

The usage of the marker tracking package was done the following way: it was chosen an AR marker (Appendix B - Figure B.1(b)) to identify the quadrotor, which was placed over the vehicle. Then, another marker was chosen (Appendix B - Figure B.1(a)) so to identify the local origin of the area.

4.1.1 Testing area

Testing area 1

The first chosen testing area was a space in an indoor area with more or less 3x5 meters. In that area, it was installed a RGB-D camera at the top (3.5 meters above the ground), pointing to the ground with a certain angle.

This area had to be confined for a reason: given the camera position, the camera vision angle is not large enough so it cover all the attic area to track markers. So, it was defined a 2x2 meters area in the centre of the focal area of the camera, so the quadrotor could be tracked inside that area. Then, it was

1ROS Wiki. AR Track Alvar. http://wiki.ros.org/ar_track_alvar. Online.

decided to define a path the MAV must follow. The path is a simple square that follows the area defined for tracking (Figure 4.1(b)).

(a) Indoor area overview (b) Marked course seen from the installed camera

Figure 4.1: Indoor tests area

The movement starts in the top left corner, follows in front, then keeping the same yaw orientation, it keeps the movement along the edges of the delimited area, until it reaches the same spot. As it can be seen in (Figure 4.1(b)), the local origin was placed 2.5m ahead of the starting point of the quadrotor, since it is well a backlit area (there are now artificial light sources available) and it is in a place where its tracking is not lost, given that the quadrotor could cross the line of sight between the camera optics and the marker.

All the datasets were made with the MAV handheld. The reasons are the following:

• Given that the area covered by the camera for tracking the quadrotor, a not well calibrated flight, which includes the consequences of the vehicle dynamics and the ground vortex effect, can result in a long failure of the tracking system, which invalidates the test

• Since the area of testing is very short and there is the presence of an obstacle rich surrounding environment, which again, given the dynamics of the quadrotor, and given that is not easy to control, can result on damaging the quadrotor or the obstacles around.

• Since the purpose is to evaluate the performance of the localization algorithms against a ”ground truth” system and knowing the a pre-established path, handholding the quadrotor allows a better control of the movement of this and a better path structure to compare to the ground truth and the defined path on the floor.

Testing area 2

A second testing area had to be chosen. The main reason is the following: to the optical flow module to operate, given the software architecture implemented [26], it needs to have good readings. Those readings imply that the CMOS camera must get data readings from a high texturized surface so that the feature extractor algorithm running on the module can calculate the linear motion of the camera frame

relative to the surface plane. Those readings are evaluated by a ”quality” variable, which values vary between 0 and 255. The acceptable quality threshold is above the 120.

In the case of the floor on the indoor area, it was impossible to use the optical flow measurements since the floor is made of reflective tiles, without any kind of texture. So all the indoor tests were made without the optical flow module velocity measurements being fused on the position estimator of the FCU firmware.

So that the horizontal velocity measurements could also be considered in the position estimation, it was chosen as a testing area an outdoor area, where the floor is made of cobble stone, which pattern is rich enough for getting acceptable “quality” values of the measurements. The testing area (Figure 4.2) was used to evaluate the performance of a vision-based localization system using optical flow velocity calculations and a system without it.

(a) Outdoor area overview (b) Marked course seen from the installed camera

Figure 4.2: Outdoor tests area

4.1.2 Localization modules parameters

The graph-based SLAM algorithms, as seen, give numerous possibilities of tweaking the backend and the frontend framework applications. CCNY RGBD only allows the tweaking of the frontend, since it does not have a mean of optimizing the graphs.

So, for a fair comparison between the SLAM algorithms, it was chosen the following constants and thresholds for both:

• Feature detector: SURF

• Max distance between keyframes: 15 cm

• Max angular distance between keyframes: 10

• Number of features per image: 400

• Max number of iterations: 10

Off course this parameters can be tweaked in a way they can be neat optimal for the considered testing conditions, but that was not take into account, as the main purpose is to test the difference between each type of pose estimation framework.

The main difference between the SLAM frameworks is that CCNY RGBD uses ICP for pose esti-mation and no post graph optimization, while RGBD SLAM uses RANSAC with transform verification by EMM - beam-based environment measurement model - and uses graph optimization with the g2o library².

In the case of DEMO, there is an option of using the bundle adjustment for path correction or not.

After some analysis, and given that there is a ”slimmed down” version available for embedded systems like the Odroid, it was decided to do all the tests without the bundle adjustment. The analysis made also allowed to note that the visual odometry for itself, i.e. without bundle adjustment, is capable of running slightly fast on both laptop and Odroid, which then leave to the conclusion that, given the system purposes, a bundle adjustment is not applicable.

4.1.3 FCU estimators parameters

As presented, the position estimator judges the sensor data depending on relative weights of the different sensor sources. It is known that the barometer measurements are very noise in indoor environments, so the default weight on the position estimator was reduced. The sonar weight was increased, but since this one, as already explained, only takes role on the height offset correction and it is not used as a primary altitude source, the weight increase will not do that much effect. Also, as the sonar readings are unstable, noisy and very subject to interference from highly reflective surfaces, it is comprehensible not to use the sonar as the main source of altitude.

So, considering the PX4 Firmware API, the following parameter weights were defined on the FCU:

(note that the minimum weight is 0 and the maximum weight is 10):

• Vision XY weight: 7

• Vision Z weight: 4

• Sonar weight: 10

• Barometer weight: 0.01

Then, it must also be considered the weight of the optical flow measurements. As stated and verified, since the indoor testing floor area is a reflective kind of floor, without any kind of texture, the “quality”

measurement variable of the sensor is always low (below the 100, in a scale between 0 and 255). So it is considered a value of 0, since the sensor cannot be used.

In the case of the outdoor testing, the case is different, given that the cobble stoned floor has enough texture to be considered feature reach, and so the quality variable increases, and the measured lin-ear velocities are considered in the position estimator. So, for comparison between a non-velocity-fed estimator and a velocity-fed estimator, it was considered the same weights as above, plus:

2OpenSLAM. g2o. http://openslam.org/g2o.html. Online.

• Optical Flow XY weight: 0/5

Which means for the case of evaluating the system without the use of velocity estimation, it is used a 0 weight value, and a value of 5 for the opposite situation.

4.1.4 Datasets

Dataset 1: The first dataset was done on the indoor area, with the main purpose of evaluating and