Application to colour image segmentation

Application of the above multi-dimensional extension of Otsu’s algorithm to segmentation of the red-blood cells in images of thin-film slides is straightforward and leads to very pleasing results such as those illustrated in figures 4.12 and 4.13. Putative RBC pixels displayed as the white foreground in the figures were selected automatically as the minority class. Visualisation of the vector a (computed from the whole image) as iteration proceeds indicates considerable change from the direction of principal variance used to initialise the process. This arises because there are more pixels belonging to the blood plasma background than to the foreground of the red-blood cells, resulting in the distribution in colour space being weighted towards the top of the pixel distribution and the initial decision surface being similarly displaced towards this point as shown in figure 4.14. This configuration produces an imperfect segmentation as illustrated in the figure.

As iteration proceeds (figures 4.15 and 4.16) the decision surface swings around until it is slicing the distribution of pixel colours more longitudinally than laterally as shown in figure 4.16. This was somewhat surprising, but even though the segmentation is poor at this stage, the orientation of the decision surface continues to change and the procedure finally produces a very good segmentation as depicted in figure 4.14. Convergence of the direction of a and of the segmentation is quite rapid in 15 iterations.

Once the direction of vector a and the segmentation have settled down, it is easy to produce

Figure 4.12: The colour image shown originally in monochrome in figure 4.2 (top) alongside the final segmentation produced by applying the Otsu 3D algorithm. Its pixel distribution is shown by occupancy of the RGB triplets in colour space (middle) with (bottom) the final decision surface separating foreground, putative RBC pixels (red) from the background (blue).

Figure 4.13: Detail from figure 4.12 showing (left) the segmentation produced by applying the Otsu 3D algorithm to the bottom-left portion of the colour image (right) displayed in monochrome in figure 4.2.

a final ROC curve for segmentation of the image as shown in figure 4.17. The area under the ROC curve, operating point, and error rate are given in table 4.2. The results are very similar to those obtained from combination of the outputs of the separate thresholding on the three colour channels and also similar to the best obtained with a single colour channel (green) shown in table 4.1. The computation time is small (2 secs for processing a 1300 × 1030 image on a Pentium M 1.8 GHz processor using a Matlab implementation. This is only a small multiple of the time required for the one-dimensional thresholding, unlike the search for the optimal combination described in section 4.2 which was computationally intensive and took 57 secs on the same machine.

To assess how well the various algorithms cope with variability within and between images we used the subdivision of one image into nine regions as described in section 4.1.3 and similar regions extracted from eight other images. Areas under the ROC curves, the operating points selected by the Otsu algorithms, and corresponding error rates were calculated. There was little variation within the selected image shown in figure 4.2 that was analysed in detail but, as we might expect from the fact the mean colour of the images varied by almost a factor of two, greater variation between images. Results, averaged over the selected regions from the nine images are summarized in the last column of table 4.3. On average, performance of the 3D extension of the Otsu algorithm is usually a little better than was obtained on the image shown in figure 4.2, in particular with a higher value of tpr. The error rates are a little lower and the two types of error somewhat better balanced. Evidently the image (figure 4.2) analysed in detail was somewhat atypical.

Figure 4.14: Bottom left: the initial decision surface in 3D obtained from the first iteration initialised by PCA. The decision surface is superimposed on the pixel distribution in RGB colour space shown by occupancy of the RGB triplets as in figure 4.12 but in this case in red for pixels belonging to the background and blue for pixels belonging to foreground, i.e.

supposedly to RBCs. Top: the corresponding segmentation. Bottom right: the ROC curve obtained by varying the threshold on the distribution projected onto the initial feature vector a obtained from PCA.

3 × 1D 3D

A 0.9858 0.9863 tpr 0.9574 0.9669 f pr 0.0324 0.0345

 0.0750 0.0676

Table 4.2: Performance of the system obtained by combination of the separately thresholded colour channels (3×1D) and of the multi-dimensional extension of the Otsu algorithm (3D).

Figure 4.15: As in figure 4.14 but showing an intermediate segmentation obtained after 5 itera-tions (top) together with its decision surface (bottom left) and the ROC curve produced at this stage (bottom right).

R G B 3 × 1D 3D

A 0.9880 0.9917 0.9929 0.9928 0.9930 tpr 0.9700 0.9808 0.9740 0.9768 0.9777 f pr 0.0345 0.0364 0.0270 0.0327 0.0274

 0.0646 0.0556 0.0530 0.0560 0.0497

Table 4.3: Performance measures evaluated by averaging over the seventeen selected one-ninth regions from nine different images as described in the text for thresholding algorithms applied to each of the separate colour channels (R,G,B); for the optimal combination of these algorithms (3 × 1D); and for the multi-dimensional extension of Otsu’s algorithm (3D).

Figure 4.16: As in figure 4.15 but after 10 iterations when the orientation of the decision surface is changing most rapidly. The final decision surface is shown in figure 4.12.

Figure 4.17: The ROC curve showing the variation of the true-positive rate tpr (y-axis) as a function of the false-positive rate f pr (x-axis) obtained by varying the threshold on the dis-tribution projected onto the final feature vector a produced on convergence of the 3D Otsu algorithm applied to the whole colour image in figure 4.12 (front-most) superimposed on the ROC curves shown in figure 4.9 obtained by application of the Otsu algorithm in 1D to the intensity (back), green, red and blue (front) channels, respectively.