Figure 3.13: Improved stereo after integratingLIDARdepth information. Top left shows raw dis- parity image, top right shows projectedLIDARscan using calibrated transform, lower left shows the depth image withLIDARinterpolated fill for missing centre area, lower right shows rawLIDARscan onX−Zaxis, radius grid spacing is 2m. The grey dashed
circle shows a discrepancy between the interpolated and expected road contour.
plane they do not tend to form orderly rows of data points, instead they are scattered across the image in the vertical direction due to the low vertical angular resolution. The top right section ofFigure 3.13shows how sparse the data from the LUX is once projected. In this situation it is not practical to exclude large gaps during the horizontal interpolation stage, otherwise the result would be a single interpolated line through the middle, with no reference points available to expand the interpolation vertically. Instead horizontal interpolation is done whenever there are two or more points on the same row of the image which allows a large region of road to be depth estimated. This is only applied across large gaps when there is not also a large discontinuity in depth to prevent filling regions with false gradients. Since roads are generally wide flat regions this approximation
3.8 theoretical stereo noise performance 79
tends to produce good results.Figure 3.13shows a particularly good example of the benefit provided by the fusion, in this frame the image has been over exposed causing the road to reach white saturation and, as a result, the disparity algorithm has failed to produce meaningful depth information in the region just behind the car. Fortunately theLIDAR
has scanned the road just behind, ahead and each side of the region allowing a fully interpolated representation to be created. TheLIDARinterpolated depth image covers
only a small patch of the image but, in this case, it has replaced the missing and noisy (random colour areas) regions just behind the car with a smooth gradient which by appearance looks correct.
There is a slight discrepancy inside the grey dashed circle where the interpolatedLIDAR
region meets the original stereo depth map which gives the impression of a dip in the road which is not actually present. Since there are noLIDARreference points at this particular
vertical location in the image the interpolated path does not perfectly match up with the true curvature of the road.
3.8
theoretical stereo noise performance
The theoretical error in stereo measurements is proportional to the square of the distance measured as shown inEquation 3.9. Using complete, dense disparity images a short comparison was conducted to observe the extent of error between distances measured by the LIDAR and the same distance measured by the stereo to get an idea of the calibration accuracy when applied to fusion.
fs = fr
ε = z 2c
3.8 theoretical stereo noise performance 80
where (including physical values from Bumblebee technical manual where appropriate):
b baseline distance, value0.12Inc. [85] f normalised focal length, value1.27[85]
r horizontal resolution of image, value512(images scaled by half) c root-mean-squared accuracy of correlation in pixels, value0.2[85] fs focal length of stereo in pixels
z distance of a point in question, m ε distance error (accuracy), m
Without the necessary tools to create a ground truth comparison of the range data from each sensor this method can give some idea of the performance. Taking10frames from
the “Berkswell manual” dataset (typical rural roads, seeSection 4.3.4) where the disparity coverage was sufficiently dense to cover the same image points as the projectedLIDAR
data the distance measured by each device was recorded. To do this the disparity image was converted to anSRIthen the range of each depth-image point, covered by aLIDAR
point, was extracted for comparison to the reported range a eachLIDARpoint.
Figure 3.14shows a scatter of distance differences between the two distance measures. There are many factors which would lead to discrepancy between the two measurements in addition to assumed measurement noise such as alignment shift in the sensors and more significantly temporal misalignments i.e. the data from each sensor being captured several milliseconds apart. WhileFigure 3.14(a) does show some similarity to the curve produced usingEquation 3.9the true pattern of error against distance looks to be linearly bounded. This is because the error measured is not true error but the error between the two devices, while the stereo error increases with the square of the distance theLIDAR
error will be more uniform.
Figure 3.14(b) shows the correlation of the measurements which, if each sensor was perfectly accurate, would be a straightx =ygraph. The distribution of points clearly
3.8 theoretical stereo noise performance 81 0 5 10 15 20 25 30 35 40 45 50 0 0. 5 1 1. 5 2 2. 5 3 3. 5 4 Stereo Distance, m
Error (scatter), Theoretical Accuracy (line), m
Distance Difference at Stereo Distance
LIDAR to Stereo Distance Error Theoretical Stereo Accuracy
(a) Comparison of theoretical stereo distance error computed usingEquation 3.9with actual distance error compared againstLIDARmeasurements.
0 5 10 15 20 25 30 35 40 45 0 5 10 15 20 25 30 35 40 45 50
Distance Measurement Correleation
LIDAR Distance Measure, m
Stereo distance Measure, m
(b) Correlation of distance measurements between the two devices.
Figure 3.14: Selection of graphs showing a comparison between the theoretical and observed errors in stereo distance data when compared toLIDARmeasurements.
3.8 theoretical stereo noise performance 82 0 0 . 3 5 0 . 7 1 1 . 0 6 1 . 4 2 1 . 7 8 2 . 1 3 2 . 4 9 2 . 8 4 3 . 2 0 3 . 5 6 3 . 9 1 0 5 0 0 1 0 0 0 1 5 0 0 2 0 0 0 2 5 0 0 3 0 0 0 Error, m Count
Error Magnitude Distribution
Figure 3.15: Histogram of error magnitude shown inFigure 3.14(a) and (b).
shows the errors growing at the longer ranges. According to the ibeo LUX handbook [64] theLIDARhas a range independent accuracy of10cm therefore it reasonable to assume
the error is mostly attributed to the stereo camera.
The most important factor to consider here is the average severity of the errors as shown inFigure 3.15. Since most of the errors are between0cm and70cm and at least10cm
should be expected at any range due toLIDAR uncertainty the accuracy is sufficient
for establishing basic geometric information. At longer ranges the errors are within theoretical limits and the quantity of relatively large errors shown on the histogram is small compared to the majority, indicating they are simply noise. Together this information indicates that the calibration is holding well across the frames examined.