Benefits of Functional Image Processing

9.2.1 Im age processing in functional style

Image processing can be described as a certain function applied to input images or descriptions which yields output images or descriptions, i.e. input—^process—^output. Therefore, the basic style of functional programming:

output = function (inputs)

naturally describes image processing operations; for example, describing a series of operations, such as image file I/O functions to convert data from /to a long list of characters (Section 7.2.1) and a series of geometric transformations (Section 7.4.2)'.

However, the real benefit of functional programming is not only its syntactic style, but its strong capability to construct complex operations from simple ones in a m odular fashion, as described in the following.

9.2.2 H igher-order functions and im age processing

Images usually have a regular and common structure, such as a 2D array of pixels, and higher- order functions are particularly useful to handle various operations on the same structure. This statement may be quite obvious and has previously been mentioned, e.g. [Breuel92a,

1. Many fm^ctions are used in a curried form including the collection of examples presented to show the/&pability of pointwise and convolution functions (Sections 3.3 and 3.5).

Parsons87a]. However, what we have achieved by the local neighbourhood operations implemented in Section 3.5 seems quite remarkable and has practical value.

We have demonstrated that a large collection of local neighbourhood operations can be constructed from only three basic functions: pointwise, translation and convolution. There are three kinds of usage of the convolution function:

• Linear operations, such as neighbourhood averaging, Laplacian, and the Sobel edge operator. These operations can be implemented by replacing a mask. Most languages can cope with this category, as data can be given as an argument to a function.

• Non-linear but separable operations, such as minimum and maximum filters. Because the convolution works on a "ID first then 2D" basis and functions such as min and max can be calculated in any order within a local neighbourhood, those operations can be implemented by replacing the arguments for additive and multiplicative operations. • Non-linear and non-separable operations, such as median filtering. In this case, the

convolution is used as an access method to neighbourhood pixels. The neighbourhood pixels are concatenated to form a list of pixels at the position, which is a separable operation that our convolution can deal with. Then the actual operation, such as taking a median, is mapped onto the list image in a pointwise manner.

Using the same technique, other local neighbourhood operations such as morphological operations may be implemented without much difficulty. When an image processing package is to be designed using a functional language, the convolution proposed in this thesis will be a strong candidate as a member, which allows users to implement various local neighbourhood operations without writing low-level code.

The general methodology would be not to implement an individual algorithm directly, but to grasp the common structure of operations and define these structures as higher-order polymorphic functions. Individual algorithms can be implemented later at a much higher

level. This principle has been termed algorithm categorisation in [Parsons87a].

9.2.3 Polym orphic pixel types

In defining a representation of images, we have used the technique to leave the type of a pixel as a polymorphic type using a type variable. This may be a general method to allow processing

- 162-

various categories of images in a unified manner, and has previously been used in [ Allsop91a]. Using this technique, programmers do not have to write almost identical code for each different image category, and the concrete type can be inferred when the arguments (data and functions) to a function have been specified. The spatial conditional operation (Section 3.3.4) would be a good example of combining a boolean image and polymorphic images. In short, from the programmers' point of view, polymorphic typing allows a modular way of programming by reusing polymorphic functions as program parts.

However, a serious drawback has been pointed out when polymorphic data are used to conununicate with the outside world (Section 72.2), as I /O is an essential operation in image processing. In such a case, programmers have to be aware of which concrete types are being processed, and they have to apply an explicit type conversion for each type. It has been suggested that, to overcome this problem, overloaded functions should be defined for I/O which call an appropriate conversion depending on a member in the type class. In addition, if a language has a facility to incorporate such low level functions written in an appropriate language such as assembler or C, it may overcome the I/O efficiency problem. Haskell may be able to cope with both suggestions.

9.2.4 C om posing complex data and operations

An image is typically represented as a regular N dimensional array and functional languages allow construction of higher-dimensional data and operations using lower ones. This would be a general benefit in expressing image processing algorithms in functional languages and was mentioned in [Breuel92a]. However, our contribution is that we have presented a very clear methodology to realise this benefit which would be practically useful for implementing many data structures and operations for image processing.

The principle is to define a common data structure for any dimension using

polymorphic types, then to implement an operation on the ID version making it as polymorphic and higher-order as possible, and lastly to apply the ID operation to itself to compose higher-dimensional operations. If the data structure is a plain list, composing the operations for a list of lists is straightforward as in [Allsop91a]. But using our method, more

elaborate structures can be made common for all dimensions. We have defined interval as the

common structure which is a tuple of parameters to describe the "shape" of the interval and the data in the interval, and regarded a row as an interval of pixels and an image as an interval

of rows. As concrete examples, we have implemented i n t e r v a l * (C hapter 3), i n t e r v a l l

* (Chapter 5) and i n t e rva IT * (Chapter 6), and shown that the methods to compose higher­

dimensional operations are very straightforward regardless of the basic data representations being lists or trees. We have not implemented 3D or higher dimensions, but they may be constructed using the same principle. This is another good example of modularity gained from the features of functional languages.

However, ease of programming may not always bring efficiency, since data struchires and operations are not optimised for each dimension. As discussed in Section 6.6.1, the binary tree version is much easier to program than the quadtree version, but in terms of space usage the quadtree version should be more efficient. We have not tested the practical effect of this trade-off in real image processing applications, which is left as future work.