Image processing - segmentation by thresholding

There is a variety of segmentation methods in the image processing literature and text books but one simple way a segmentation can be produced is by thresholding. In its simplest form applied to grey-scale images, thresholding produces a binary image by labelling each pixel in or out of the region of interest by comparison of the grey-level with a threshold. If colour imagery is used, a threshold may be applied to the overall intensity I(x, y) = R(x, y) + G(x, y) + B(x, y), to the pixel’s red attribute values R(x, y), or to the G or B values. In particular, although the grey-level image was used for segmentation in the malaria thin-film image work at the University of Westminster by Rao, the green channel was used for extraction of stained objects and the difference B − R for detecting mature parasites [146]. Other convenient attributes that can be computed from the RGB colour signal may be used such as the chromaticities or the hue and saturation (c.f. the work of Hengen et al [79]). As might be expected, thresholding using colour attributes often works better than thresholding on the grey-level intensity, especially in applications where it is desired to produce a segmentation into regions similar to those a human would designate. Automatic thresholding analyses the grey value distribution in an image, usually by using a histogram of the grey values (or of the colour attributes), and uses

knowledge about the characteristics of the application to select the most appropriate threshold [89]. Automatic thresholding also helps overcome the difficulty of simultaneously choosing several thresholds when the values in each colour channel are to be separately analysed and the results combined.

In principle, a good way to choose a threshold is to separate the distributions of some property values characteristic of the populations of each type of object or image component to be distinguished [168]. An example which as we shall see later maximizes the between-class variance of the grey-levels – a measure of the separation of the between-class conditioned mean grey-levels – is provided by the following iterative algorithm [89] originally due to Otsu [131]:

1. Select an initial threshold T , usually chosen as the average intensity µ of the image if no other information is available.

2. Segment the image into two regions using T , i.e. under an ideal scenario separate the histogram into two peaks.

3. Calculate the mean grey values µ(1) and µ(2) of the regions, i.e. separately for each of the two parts of the histogram.

4. Calculate a new threshold T = (µ(1) + µ(2))/2.

5. Repeat from 2 until the means µ(1) and µ(2) do not change.

Since the algorithm lies at the heart of our work a full discussion of it will be given in chap-ter 4. A derivation of it, including generalisation to the multi-category case, and its relationship to the Fisher discrimminant is given in appendix A.

If more than two classes are required, multiple thresholds may be used, for example on the grey-level intensity as in the multi-class version of the Otsu algorithm described in appendix A.

However, such an approach may be insufficiently sensitive to minority classes so a sequential application in a tree-structured, hierarchical classification system is more usual. The latter is the thresholding approach adopted in the malaria work of Ross al. [151], except that the thresh-olding is applied to the green channel for detection of pixels belonging to RBCs (erythocytes) followed by detection of pixels belonging to parasites.

In addition, whatever thresholding algorithm is used, thresholding usually does not take into account the spatial characteristics of groups of neighbouring pixels of an image. In some circumstances, in particular when trying to segment objects of a very variable and distributed (filamentary) shape as for example the stained nuclear material in an infected blood cell, this can

be an advantage but when segmenting compact objects such as the red blood cells themselves, ignoring the spatial structure may be a disadvantage. Nevertheless, we stress that a segmentation based on inappropriate spatial information, such as a poorly chosen or poorly parameterised Markov random field [15] is likely to make systematic errors and produce worse results than one which uses no spatial information.

Work by [44], though for a very different application to TV video imagery, tried to extend the thresholding method from grey-level to colour images. Their proposed RGB thresholding method can be summarised as:

1. Apply a grey-level thresholding method to each of the R, G, B colour channels. The chosen grey-level thresholding method was Otsu’s method. This results in eight clusters.

2. Calculate the mean for each class (i.e. cluster).

3. Calculate the within-class variance and the between-class variance for each class.

4. Two classes can be merged if either of their within-class variances is greater than or equal to the between-class variance.

The above is repeated until only two classes remain. Though attractive this procedure may not always result in maximizing the Fisher discriminant, say, of the final two classes.

This paper also illustrates two other points. First that manual manipulation of several thresholds may be tricky so automatic procedures become more attractive in such situations. If manual processing is essential, colour transformations for example to HSV , Hue-Saturation-Value, may be used to reduce the dimensionality of the feature-space and the number of thresh-olds required. Tek [175] does just this in his thesis to determine ground-truth segmentations of the RBCs and of the stained pixels. Second it illustrates the problem of over-segmentation in which the image is broken-up into too many, often small and fragmentary, regions. As in the algorithm summarised above region growing processes are frequently subsequently used to try to fix such problems. Similarly, under-segmentation in particular when multiple objects are included within a single segmented region can be a problem. A variety of methods are used to split such regions, many of them, as we shall see in the malaria work reviewed in the next section, tailored to the specifics of the application. Combined split and merge techniques may be used to tackle both problems. However, it is difficult to design such processes that will ef-fectively fix segmentation errors without introducing further errors of their own. In general it is better if possible to design an accurate initial segmentation.

Finally, we recall Tek’s critique of global thresholding. As we mentioned in section 1.3.3 Tek ascribes the problem to non-uniform illumination and, in general, uses morphological top-hats to overcome it which leads to a kind of locally adaptive thresholding. Other kinds of locally adaptive thresholding, for example utilising a parametrically defined spatially varying thresh-old function, have been used in other applications also to overcome problems caused by non-uniform illumination [169]. The Fisher-Otsu algorithm we have developed is applied globally to each of our images but such problems do not seem to trouble us. As already indicated we ad-vocate monitoring this problem and checking for such non-uniform effects within our images.

We also note that our approach could if necessary be used in a locally adaptive,‘windowed’

manner.