Beacon Image Processing

For the actual flight tests, the video images were processed using a video-interrupt driven task under real time Linux. High pixel-count operations were undertaken using Intel’s SIMD instruction set known as MMX, which speeds processing by a

94 Control of Hover 0 10 20 30 40 50 60 70 80 90 100 −6 −4 −2 X (m)

Beacon Estimate Actual State

0 10 20 30 40 50 60 70 80 90 100 −1 0 1 Y (m) 0 10 20 30 40 50 60 70 80 90 100 1 2 3 Z (m) 0 10 20 30 40 50 60 70 80 90 100 −4 −2 0 Roll (deg) 0 10 20 30 40 50 60 70 80 90 100 −2 0 2 Pitch (deg) 0 10 20 30 40 50 60 70 80 90 100 −20 0 20 Yaw(deg) 0 10 20 30 40 50 60 70 80 90 100 −0.5 0 0.5 Vx (m/s) 0 10 20 30 40 50 60 70 80 90 100 −0.5 0 0.5 Vy (m/s) 0 10 20 30 40 50 60 70 80 90 100 −1 0 1 Vz (m/s) Time (sec)

§5.2 Beacon Hover 95

factor of four over the normal instruction set. The image size was 344x288 pixels. The camera was an inexpensive security camera rated at 400 TV lines with a compa- rable wide-angle lens giving a horizontal field of view of 120◦. Due to the inexpensive nature of the optics, the image produced by the system was highly distorted and unacceptable for this precision positioning application. In order to overcome these issues, Tsai’s camera calibration algorithm [201, 202] was applied. A non-linear op- timisation of camera radial distortion parameters is performed based on comparing ideal pixel locations to measured pixel positions. Using this technique, there was no sign of visible distortion in the image. Rather than apply a look-up-table to the entire image on every frame, all coordinates measured from the screen were trans- formed by the camera calibration parameters prior to passing the data to the beacon solution.

The images were initially pre-processed using a centre surround also known as

Mexican hat operator [203], which is essentially a bandpass filter on the image. The effect of this operator was to remove regions of the image that did not contain high contrast. The beacons were made from styrofoam, and were thus very bright. Certain sites where testing was undertaken were very bright in the near infrared wavelengths, as is typical of vegetation in sunlight. Short pass filters were placed over the camera to accentuate the intensity of the beacons. The filters used did not cut wavelengths longer than 700nm (Kodak Wratten filters), so only undesirable visible wavelengths could be blocked.

After pre-processing to eliminate regions of no contrast, a threshold was applied to segment the beacons from the background. Figure 5.7 shows a typical view of the beacons from the helicopter during a hover trial and a view of the beacons after a threshold has been applied. Due to the contamination with IR, thresholds of as little as 16 out of 256 grey levels were used, rather than the expected 40 or 50. Since the essential configuration of the beacons was known, it was possible for the tracking system to search for the beacons from the bottom of the screen towards the top, labelling the beacons in sequence. In order to qualify as being a beacon, a certain threshold number of white pixels was required to be within a 60x60 pixel window. The size of each beacon was measured based on the first moment of intensity (ie. measuring only white pixels) within each tracking window.

The tracking system was a standard alpha, beta, gamma tracker, with online adjustable tracking parameters [204]. Values of alpha, beta and gamma of 1, 0, and 0.04 were found to be successful, and no modifications were made after early tests. Essentially the tracking stage was not required. In addition to position, the size of each window was also tracked while tracking, so as to reduce the effect of additional ‘speckle’ (particularly dandelions) around the beacons. The window size was adjusted by assuming that the beacon was elliptical, and thus that the first moment in each major axis would define the shape. The size of the window was adjusted to be 1.2 times the size of the moments of the beacon, on each major axis. Track parameters for the window resizing operation were an alpha of 0.1 and the other parameters all zero. Using this system, track was held for all beacons under almost all situations until a beacon departed the 120◦ field of view of the camera.

96 Control of Hover

Figure 5.7: Use of visual landmarks to control hover. View of targets from helicopter (top), view of beacons after thresholding (middle), external view of helicopter hovering over targets (bottom).

§5.2 Beacon Hover 97

mined for each beacon into an azimuth angle Ψ and elevation angle Φ. Elevation angle is measured from the aircraft z axis such that a point directly underneath the aircraft would have an elevation of 0◦. The azimuth angle of the helicopter is measured by projecting the line between the beacon and camera into the aircraft xy plane. The angle between this projection and a line through the nose of the aircraft is the azimuth angle. Points on the right hand side of the helicopter range in azimuth from 0◦ at the nose to 180◦ at the tail. Points on the left hand side of the helicopter vary in azimuth from 0◦ to−180◦.

The beacons were arranged in an equilateral triangle such that each side of the triangle was 5m. The base of the triangle was orientated with magnetic East-West so that the heading calculated by the beacon algorithm could be easily compared against the magnetic heading measured by the magnetometers. The position of the helicopter in global coordinates was defined such that the origin was at the centre of the beacons, the x-axis was aligned with North, the y-axis aligned with East and the z-axis was vertically straight down.

Position information is fused with information from accelerometers, magnetome- ters and gyroscopes to determine a complete state estimate suitable for use in control.