The proposed horizon detection algorithm was implemented in C++ using OpenCV
with IPP optimisations. The method was tested on an Intel i5 3.30 GHz Processor
and the achieved processing times for different image sizes and thresholding process
are shown in Table 4.7. The algorithm is very efficient, a processing rate of 42 Hz
of meeting the desired requirements for the USYD Jabiru J400 system when a fixed
threshold is used. The adaptive thresholding algorithm (with up to 3 iterations) also
meets the desired requirements. The timings for the different detection stages for an
1024×768 image are shown in Table 4.8. The majority of the calculation time is spent in the smoothing stage and the spatial filtering stage. The maximum image precisions for
the algorithm are limited by the camera and lens and are not limited by the algorithm.
These precisions are shown in Table 4.9.
The proposed horizon detection method results for a number of example images are shown in Figure 4.17. The algorithm can detect and extract the horizon interface profile
correctly in many of the situations where the other methods fail. A number of examples
are also shown in Figure 4.18, which highlight the effects of the two different thresh-
old selection techniques. Each example shows the detected horizon using an adaptive
threshold and a fixed threshold as explained in Section 4.5. The adaptive threshold
produces the best extracted horizon interface profile, while the fixed low threshold is
very sensitive to noise and obstructions. There are situations when the proposed hori-
zon detection method can return erroneous results. Two examples of such situations
are shown in Figure 4.19. The adaptive threshold process finds a gradient spanning the image at a higher threshold value than the value required for correct horizon de-
tection. This can happen in cases when the horizon only has a weak gradient response
and there exists a stronger edge feature spanning the image. Another challenge for the
proposed horizon detection algorithm is when part of the horizon is discontinuous. The
discontinuity can cause the adaptive algorithm to saturate towards the lower threshold
limit and therefore search for any continuous gradient spanning the image. Figure 4.20a
shows the situation when part of a strong horizon is obstructed with white buildings.
Due to the colour and shape of the buildings, they have no vertical gradient response. Once the buildings leave the image frame, as shown in Figure 4.20b, then the adaptive
algorithm can correctly find the strong horizon.
For comparison with the other methods, a number of example frames have been
chosen and the various methods have been used to detect the horizon. These comparison
Table 4.7: Proposed Horizon Detection Method Timings
Image Size Timing Rate Hz/GHz1 Pixel Hz/GHz1
Obtained2 1024× 768 24 ms 42 Hz 13 30,670,848 Obtained2 800× 600 17 ms 59 Hz 18 25,920,000 Obtained2,3 640× 480 10 ms 100 Hz 30 27,648,000 Obtained (N = 2) 1024× 768 31 ms 32 Hz 10 23,662,080 Obtained (N = 5) 1024× 768 61 ms 16 Hz 5 11,831,040 Obtained (N = 10) 1024× 768 90 ms 11 Hz 3 7,098,624 Obtained (N = 2) 800× 600 17 ms 59 Hz 18 25,920,000 Obtained (N = 5) 800× 600 30 ms 33 Hz 10 14,400,000 Obtained (N = 10) 800× 600 50 ms 20 Hz 6 8,640,000 Obtained (N = 2)3 640× 480 11 ms 91 Hz 28 25,804,800 Obtained (N = 3)3 640× 480 14 ms 71 Hz 22 20,275,200 Obtained (N = 4) 640× 480 17 ms 59 Hz 18 16,588,800 Obtained (N = 5) 640× 480 20 ms 50 Hz 15 13,824,000 Obtained (N = 10) 640× 480 34 ms 29 Hz 8 7,372,800 Desired4 640× 480 20 ms 50 Hz 21 19,200,00 1
Approximation based on linear scaling.
2 Using a fixed threshold.
3
Meets the desired requirements.
4 Based on USYD Jabiru J400 hardware requirements.
Table 4.8: Proposed Method Timing Breakdown for 1024×768 Image Size
Stage Timing Percent
Smoothing Stage 9.88 ms 46.96%
Colour Binary Calculation 2.05 ms 9.74%
Gradient Calculation 1.46 ms 6.94%
Gradient Culling 0.28 ms 1.33%
Spatial Filtering Stage (per iteration) 6.89 ms 32.75%
Horizon Extraction Stage 0.48 ms 2.28%
Total 21.04 ms
Table 4.9: Proposed Horizon Detection Method Precisions
Image Size Image FOV Pixel Hz/GHz1 Maximum Precision
Obtained2 1024× 768 60◦ 30,670,848 0.017 Degrees/Pixel
Obtained2 800× 600 60◦ 25,920,000 0.075 Degrees/Pixel
Obtained2 640× 480 60◦ 27,648,000 0.09 Degrees/Pixel
1 Approximation based on linear scaling.
2
(a) (b)
(c) (d)
(e) (f)
Adaptive Threshold (tg = 18) Low Threshold (tg = 2) (a) Adaptive Threshold (tg = 5) Low Threshold (tg = 2) (b) Adaptive Threshold (tg = 22) Low Threshold (tg = 2) (c) Adaptive Threshold (tg = 19) Low Threshold (tg = 2) (d)
Figure 4.18: Proposed Horizon Detection Method Threshold Algorithm Effects
Adaptive Threshold (tg = 10) Low Threshold (tg = 2) Correct Threshold (tg = 5) (a) Adaptive Threshold (tg = 11) Low Threshold (tg = 2) (b)
Adaptive Threshold (tg = 1)
Low Threshold (tg = 2)
(a) Incorrect Horizon Detection Due to Obstruction
Adaptive Threshold (tg = 29)
Low Threshold (tg = 2)
(b) Correct Horizon Detection Without Obstruction
Figure 4.20: Proposed Horizon Detection Method Incorrect Detection Due to Obstruc- tion
(a) Proposed Method Example 1 (b) Proposed Method Example 2
(c) Dusha Method Example 1 (d) Dusha Method Example 2
(e) Ettinger Method Example 1 (f) Ettinger Method Example 2
(a) Proposed Method Example 3 (b) Proposed Method Example 4
(c) Dusha Method Example 3 (d) Dusha Method Example 4
(e) Ettinger Method Example 3 (f) Ettinger Method Example 4
4.8
Summary
The developed proposed horizon detection method outperforms the past Dusha and
Ettinger methods investigated in Section 4.3. It is the only algorithm investigated
that meets the desired performance requirements and all the the design objectives.
A summary of the relevant performance parameters and a summary of the design
objectives are shown in Table 4.10 and in Table 4.11 respectfully. The Dusha method
performance was too slow and it required a distinct horizon edge in the image, so that
it was not able to detect weak horizons. The Ettinger method performance was also too
slow and it required a distinct difference between the colour distributions of the sky and
the ground. This limited the operational range of the algorithm to very low altitudes only. The proposed method can handle both of these situations while maintaining a
higher level of performance.
Table 4.10: Horizon Detection Method Performances
Image Size Image FOV Hz/GHz1 Maximum Precision
Dusha 640× 480 60◦ 6 0.09 Degrees/Pixel Ettinger 640× 480 60◦ 4 0.25 Degrees/Pixel Proposed2 640× 480 60◦ 30 0.09 Degrees/Pixel Proposed (N = 3) 640× 480 60◦ 22 0.09 Degrees/Pixel Desired3 640× 480 - 21 - 1
Approximation based on linear scaling.
2
Using a fixed threshold.
3 Based on USYD Jabiru J400 hardware requirements.
Table 4.11: Horizon Detection Method Design Objectives Comparison
Objective Modified Modified Proposed
Dusha Ettinger
Ability to Extract Horizon Interface 4 4 4
Ability to Estimate Horizon Strength 7 7 4
Ability to Use a priori Information 7 7 4
Robust or Automatic Tuning 7 4 4
Fault Detection 4 4 4
Chapter 5
Infinite Horizon Line Attitude
Determination
5.1
Introduction
Once the horizon profile has been detected in the image, a method of calculating the
explicit attitude of the aircraft from this detected horizon is needed. There are a number
of different ways to do this, each method resulting in a different level of accuracy.
The most common assumption, linking the appearance of the horizon in the image frame to the attitude of the aircraft, has been to assume that the horizon is a straight
line. Attitude determination using this horizon line assumption is investigated in this
chapter. This straight line assumption stems from the fact that under perspective
projection, a plane forms an vanishing line at infinite distance away from the camera.
This vanishing line is only dependant upon the attitude of the camera and not the
translation. Equating the infinite vanishing line to the horizon line in the image allows
for a succinct way of approximating the explicit attitude of the camera. This horizon
model is referred to as the infinite horizon line model due to the assumption that the
horizon is effectively infinitely distant. A number of other assumptions are also made in this model, which are:
1. The horizon is infinitely far away from the camera.
2. The horizon is a straight line in the image.
and position).
4. The altitude has no effect on the horizon appearance (since the ground is assumed
to be infinitely planar).
5. The body frame Fb and the camera frame Fc are aligned (i.e. a forward facing
camera), so that the attitude of the camera frame is also the attitude of the body
frame.
This chapter develops an infinite horizon line model based upon the above assump-
tions. The attitude sensitivities to different sources of errors (such as violation of the
base assumptions or horizon detection errors) and model uncertainties (such as camera
calibration) are investigated to deterministically evaluate the uncertainties of attitude
measurements from the infinite horizon line model. The complete attitude determina-
tion process from the horizon detection stage to the final attitude measurement stage
(including fault detection checks for robustness) is described. Attitude determination
results using the infinite horizon line model for a flight test sequence are presented. These results include a comparison of the three different horizon detection methods to
highlight the accuracy advantages of the proposed horizon detection method developed
in Chapter 4 over the other two methods.
The infinite horizon line is one of the most basic attitude determination models.
It is presented here in depth, so that it can be investigated and understood. In later
chapters of this thesis, the infinite horizon model is extended to become the:
• Infinite Horizon Plane Attitude Determination Model (Chapter 6) • General Horizon Plane Attitude Determination Model (Chapter 6)
• Simultaneous Horizon Profile Mapping and Attitude Determination Model (Chap-
ter 7)
• Terrain-Aided Horizon Profile Attitude Determination Model (Chapter 8) • Terrain-Aided Horizon Profile Attitude and Localisation Model (Chapter 8)
The understanding of the limitations and assumptions of infinite horizon line model is
important. This highlights the needs and advantages of the higher derived models. A
Section 5.2 develops an infinite horizon line model, linking the attitude of the camera directly with the line parameters of the horizon line in the image. The horizon
model is derived using the pin-hole camera model rather than using the usual
normalised focal length form which has been used in the past literature. This
allows for the direct calculation of the attitude from the horizon line. The effects
that camera parameters have on the attitude measurements can also be directly
observed from the equations.
Section 5.3 derives the sensitivities of the infinite horizon line attitude determination model with respect to fitting errors and camera parameters. These equations allow
the attitude uncertainty to be calculated. This is a deterministic approach to the
calculation of the attitude measurement uncertainties, which has been lacking
in past approaches. This allows the accuracy of the attitude measurements to be determined and allows for correct data fusion in a Extended Kalman Filter.
Fault detection techniques can also be applied once the measurement uncertainty
is known.
Section 5.4 studies the causes and effects that violations of the different model as- sumptions have on the attitude measurement. This is used to highlight and in-
vestigate the sources of error for this horizon model so that they can be removed
or addressed the later derived horizon models.
Section 5.5 describes in detail the complete attitude determination process from the horizon detection stage to the final attitude measurement stage. Horizon line
fitting, attitude determination, camera offset correction, uncertainty estimation
and fault detection checks are developed and described in detail.
Section 5.6 presents attitude determination results for a flight test sequence. The accuracy of the infinite horizon line model is evaluated for the three different
horizon detection methods described in Chapter 4. The proposed horizon detec-
tion method produces the most accurate results.