Hardware for Real-Time M achine Vision

We have seen th a t a large number of different algorithms have been developed to compute optical flow. Some have been developed to solve a specific vision task, others as an attem pt to model biology and yet others as novel solutions to a difficult problem. However, there are few algorithms th a t have been designed with a real-time implementation in mind during development. The m otivation in developing more efficient ways to compute motion has often been in an attem pt to cut down the com putation time from minutes to seconds for the sake of convenience. Until recently, the cost of real-time machine vision systems had been prohibitive in term s of the price of hardware, required development times and required expertise. Fortunately, the computer vision industry is

now at the point where relatively inexpensive image processing hardware is readily available along with a large number of software development tools. In fact, there are now a great many hardware options available on which to implement computer vision algorithms. Since the choice of hardware will influence the im plementation strategy used in this thesis, it is im portant to examine the possibilities. The options considered and discussed in the following sections are:

VLSI Chip Development D SP’s for Image Processing F P G A ’s

General Purpose Image Processors Personal Computers

VLSI Chip Development

Neuromorphic engineering has focused on the exploitation of the properties of m iniature analogue circuitry in VLSI chips for some time now [Marino01][Indiveri99] [Arias96] [Mead90]. There are many advantages to designing a dedicated processor for a custom algorithm and the hardware yields the fastest throughput possible. The physics of the analogue devices can be exploited to achieve robustness and low energy consumption [Mead90]. However, VLSI is only suitable for simple algorithms th a t can be implemented w ithout many stages of com putation, otherwise the algorithm becomes too complicated to fit onto one chip. Unfortunately, the VLSI route is expensive in term s of design time and fabrication costs and the finished hardware is difficult or impossible to modify.

Commercial DSP’s

There are a number of products on the m arket which use Digital Signal Processing chips (DSPs) such as the Texus Instrum ents TMS320C40 or custom ASICs for image

processing operations such as convolution, and often come as parallel processing modules. The disadvantage is th a t in order to write custom algorithms an in depth knowledge of the workings of the DSP and programming software is usually required in order to make the most of the functionality of the chip. There is usually a limit to the size and complexity of the program th a t can be executed by general purpose DSPs.

Field Programmable Gate Arrays

Field Programmable Gate Arrays (FPGA) are a relatively recent technology - Xilinx produced the first FPG A ju st over ten years ago and are still the dominant supplier. FPGAs are VLSI chips th a t are re-programmable at run time and are able to

offer comparable speed to dedicated VLSI with much more flexibility. The main

lim itation is th a t there are a limited number of logic units (gates) on the chip th a t in turn limits the maximum complexity of an algorithm. The number of gates in early models was fairly low (typically 300K) and only very simple algorithms could be implemented e.g. analogue to digital conversion and video synchronisation signal stripping (where the m ajority of FPGAs are still employed). W ith the number of gates now approaching lOmillion and running at clock speeds of SOOMhz it has become possible to implement algorithms such as convolution, Fourier transforms and basic optic flow algorithms in real time [Zuloaga98]. The main drawback is th a t multiple FPGA s are required in order to

implement complex algorithms, which can become expensive. However, useful image-

processing boards based upon FPG A technology are becoming more m ature and widely

available and the programming tools are improving. At present, the hardware and

software needed to program an FPG A from scratch is still very expensive compared to machine vision systems, but precompiled software for performing basic image processing routines (convolution, averaging, F F T etc) is available for significantly less expenditure. It could be th a t in the near future FPG A based image processing boards become the hardware of choice for development, obviating the need for complicated VLSI chip design.

Embedded General Purpose Image Processors

General purpose image processing boards are available with a wide range of abilities. Although not usually as fast as dedicated D S P 's, their general purpose architecture permits implem entation of more complicated algorithms. The advantage of these image processing boards usually comes from a custom high bandwidth bus with fast image memory th a t passes acquired images to the processor engine far quicker than a PC. Two of particular interest are the “Genesis” board from M atrox and the “M amba” board

from Goreco. The Genesis board consists of a general purpose DSP (the Texas

Instrum ents G80 or G4Power PG) supported by accelerator chips capable of speeding up many image processing functions. The Genesis can be programmed easily using a set of software libraries and is very flexible and a custom 400MB/sec bandwidth bus improves speed for bandwidth limited algorithms. The M amba uses an Intel P entium ll processor running embedded NT, as such it appears in every way the same as a conventional PG. W ith the Mamba, software can be w ritten for Microsoft Windows operating systems and copied to the M am ba's disk-on-chip to be executed as any normal program. The drawback is th a t the M am ba's processor is not as fast as a dedicated DSP (currently P entium lll 750Mhz), there is no dedicated d ata bus to the GPU and the board is relatively expensive compared to a standard PG.

Desktop PC

W ith companies such as Intel and AMD producing ever faster general purpose processors, real-time image processing using relatively cheap and readily available

hardware has become commonplace. Until recently, a major drawback of PG

architectures was th a t image d ata could not be accessed as faat as required using the main memory bus and most image processing algorithms were limited by this factor. Recent introduction of high-bandwidth buses to main memory (e.g. RAMBUS) and processors with dedicated instructions for fast image processing operations (e.g. MMX, SSIMD) have improved the viability of PCs in real time image processing applications immensely. Real­

time image acquisition hardware is now available for a few hundred pounds th a t is capable of passing image d ata directly to main memory w ithout any CPU loading, and since the clock speed of the general-purpose processors are often an order of magnitude faster than those in D SP’s, they can fulfil many of the tasks previously only possible on specialised hardware. The major drawbacks are their size and power consumption, which prohibits their use in many situations.