Vision plays an integral part in a pilot’s ability to navigate and control an aircraft.
Visual Flight Rules (VFR) have been developed around the pilot’s ability to see the
environment outside of the cockpit in order to control the attitude of the aircraft, to
navigate and to avoid obstacles [1]. This thesis investigates the design, development
and implementation of a robust vision-aided navigation system which is capable of
extracting visual information from the environment for use in navigation, guidance
and control. This system essentially mechanises the VFR process, trying to replicate
the processes of how a human pilot visually navigates an aircraft in order to aid pilot navigation workload or to be used as another navigation system on Unmanned Aerial
Vehicles (UAVs).
Current aircraft navigation systems usually consist of a sensor-aided Inertial Navi-
gation System (INS). In these systems, high-rate inertial measurements from an Inertial
Measurement Unit (IMU) are integrated with respect to time to provide continuous po-
sition and attitude estimates for the aircraft. These integrated inertial estimates are
accurate over small time intervals but are subject to drift over the longer term. This
Dead-Reckoning (DR) or inertial error needs to be corrected by fusing measurements from an aiding sensor with a finite accuracy. This fusion process can constrain the
drift, which then allows for accurate navigation over the long term operation of the
system. GNSS (Global Navigation Satellite System) or GPS (Global Positioning Sys-
however, this thesis investigates the incorporation of visual measurements to generate
a visual navigation system or vision-aided INS.
In VFR conditions, pilots can successfully navigate an aircraft without reliance on
any external systems or infrastructure, such as VORs (VHF Omni-Directional Radio
Range) or NDBs (Non-Directional Beacons) or GPS which are typically used in IFR
(Instrument Flight Rules) conditions. UAVs on the other hand have a high dependence
on external systems to navigate and land. These external navigation systems, such
as GPS, can be jammed/spoofed (in the case of military applications) and satellite signals can be obstructed which can be common in manoeuvring flight. In such cases,
the navigation precision of the INS can drop dramatically as it then depends upon
on the accuracy of the inertial sensors which are subject to drift when using dead-
reckoning navigation. It is partly the reliance on these systems which prevent UAVs
from becoming as truly independent as a human pilot, due to the primary reason that
human pilots are able to navigate in adverse situations due being able to ’see’ where
they are. This is the driving motivation behind the inclusion of visual navigation into
inertial navigation systems.
There are two main processes that pilots use to navigate and control the aircraft in VFR conditions. Pilots are trained to control the attitude of the aircraft by orientating
the aircraft with respect to the visual horizon. This allows the pilot to maintain a
desired bank and pitch angle while also maintaining a desired heading angle by keeping
track of distant features (such as mountains). To localise the aircraft, pilots observe
distinct ground features such as roads, bodies of water, buildings, terrain and other
natural or manmade features and correlate what they see with maps. Both of these
activities require a constant and considerable amount of concentration by the pilot do
to the need to maintain an accurate position estimate during flight. This increases the overall workload for the pilot. Automatic detection of visual ground features and the
correlation of them to a known database would allow for position estimates to be made
automatically if operating in a known environment. This would reduce the workload
for a pilot. Conversely, new or unknown features could be mapped and used for short-
term relative localisation if operating in an unknown environment. While short-term
relative localisation would not provide absolute position measurements required for
such time that a known landmark is observed. This process of simultaneously mapping
features and using them for localisation is commonly referred to as SLAM (Simultaneous
Localisation and Mapping).
Since the visual horizon and visual ground features are the primary references a pilot
uses to navigate the aircraft, it is therefore logical to try to mechanise both of these
visual processes in a vision-aided navigation system. Specifically, this thesis investigates
visual attitude determination and visual localisation from the observed horizon profile,
as well as visual localisation from ground based features.
The problem of visual attitude estimation and visual localisation is that they are
cross-coupled. The accuracy of each process directly affects the accuracy of the other.
The attitude accuracy affects the time evolution of the navigational accuracy from an
INS (by limiting the drift rate) and limits the maximum possible accuracy of a visual
localisation process (by affecting the accuracy of a feature’s azimuth and elevation, and
hence position estimate). This can be seen in Figure 1.1, in both situations the visual
measurement angles are the same, however attitude errors cause the resolved position
to be biased. ε ε ¢µ R R
Figure 1.1: Attitude Error Effects on Visual Navigation
A 1◦ attitude error induces a 1.75% of range position error while a 5◦ attitude error induces a 8.27% of range position error. At an altitude of 5000f t a 1◦ attitude error
would turn into 26.6m position error observing a feature directly below the aircraft or
37.6m when observing a feature at 45◦ declination. The reverse effect is also true, the
can be identified in the attitude estimation system. The cross-coupled system means
that increasing the attitude estimation accuracy will directly increase the positional
estimation accuracy and vice versa. Figure 1.2 shows the various cross-coupling effects
between attitude and position estimation processes. This cross-coupling effect is in-
vestigated and demonstrated in this thesis, highlighting the navigational advantages of
fusing attitude measurements from horizon detection.
Visual Feature Measurements Visual Horizon Measurements Position Estimate Attitude Estimate Inertial Solution Updates Position Updates Attitude Updates Attitude Constrains Attitude Bias Estimation Constrains Visual Positional Accuracy Predicts Position Predicts Attitude Accuracy Limits Positional Drift Rate
Figure 1.2: Visual Attitude and Localisation Estimation Cross-Coupling Effects
Visual sensors are a strong source of aircraft attitude and positional information,
which have many advantages over other sensors traditionally used for flight guidance,
navigation and control. The reliable extraction and processing of navigational data from
a visual sensor at the required rate to generate useful measurements for INS aiding has
been a major limiting factor. The use of visual measurements in airborne navigation
systems has only recently become feasible for implementation due to advances in com-
putational processing power, hardware miniaturisation and reducing hardware cost. This has opened up a new field of researching image processing techniques and vision
systems for application in the aeronautical environment. The sensors and hardware
required are relatively cheap, which allows these advanced systems to be implemented
on a wide range of platforms at low cost. Hence presents many advantages for general
aviation and sports aircraft markets. The ability to use cheaper hardware and achieve
the same level of performance as more expensive equipment is a important outcome
of these vision systems. Visual navigation is also a completely passive sensor scheme
(unable to be detected or jammed), and it is able to provide accurate relative and ab-
solute positional information, assuming VFR conditions. These concepts can also be extended to use Infra-Red sensors, so that they may operate in IFR conditions.
As this is still a developing field, there are many challenges to overcome. These
risk control applications, such as aeronautical platforms. They all must be addressed
in order to produce a feasible aeronautical vision-aided navigation system. The main
problems this thesis addresses are computational processing power, reliability, robust-
ness and accuracy. The problem of processing power is addressed by developing efficient
algorithms and techniques for the processing of the visual data and assessment of their
temporal performance. Robustness is addressed by developing fault-detection checks
and techniques to identify and handle bad or erroneous data. Reliability and accuracy
issues are addressed by the fusion of multiple information sources and by the inves- tigation of the sensitivities of the visual processes. Investigation of the sensitivities
allows deterministic probability uncertainties to be found and the optimum camera
configuration for maximum estimation accuracy to be identified.
The overall motivation behind the development of a visual navigation system in this
thesis is to aid in manned flight and to provide an automatic passive navigation system
for UAVs. The visual system is not designed to replace other sensor-aided inertial
systems (such as barometric, magnetometer, or even GPS aided), but rather work in
conjunction with them to take advantage of each sensor system when that system is
operating correctly to provide the best possible navigation solution. Visual navigation systems can be relatively inexpensive and easily installed on manned aircraft to aid the
pilot in navigation in the case of pilot assisted flight, or to be used in conjunction with
stability augmentation systems to improve the dynamic performance of the aircraft.
Both of which would reduce the workload for the pilot and improve handling qualities.
When visual navigation systems are installed on UAVs, dependance on external systems
and infrastructure (such as GPS) for navigational performance is reduced, increasing
the autonomy of UAVs.