Overlapping Cell Segmentation

Recent approaches focus on the complete segmentation of individual cytoplasms and nuclei of overlapping cells with varying degrees of overlap among them. Beliz el al. [132] propose a methodology based on a locally constrained watershed transform. The results shown in that paper present limited evidence of the efficacy of the proposed technique. In particular, it is not clear the extent of cell overlap their methodology can successfully handle when segmenting cytoplasm and the nuclei

of overlapping cells. Another methodology proposed is from Lu et al. [110, 133], who propose a method that utilizes a joint optimization of multi-level set functions constrained by the length and area of each cell and the shape of the cell. The method first detects the cell clumps and all nuclei within those clumps, then it involves several levels set functions for each cell within a clump, which interact with each other using both unary (intra-cell) and pairwise (inter-cell) terms. The unary constraints are based on contour length, edge strength, and cell shape, while the pairwise constraint is computed based on the area of the overlapping regions. Nosratiet al. [134] propose a continuous variational segmentation framework using directional derivatives to segment overlapping cervical cells in Pap smear images, incorporating a star-shape-prior with the level set method. However, these shape priors are too simplified to approximate the real shape of the cervical cells, this approach is only applied to the segmentation of objects with a well-defined and consistent appearance, and defining a shape prior for overlapping cells is not a straightforward process.

Although the method presented in Chapter 3 and Chapter 4 provided promis- ing results in segmenting medical images, however, we can not apply them directly to overlapping cells. This is because overlapping cells are characterised by weaker edges. Instead, we explore another way of applying edge-based active contour using a patch-based approach where we let an open counter evolves independently within small patches.

This chapter proposes a framework capable of segmenting the cytoplasm of each individual cell depicted within an image of overlapping cervical cells. The proposed framework uses a patch-based approach where an active contour detects, on a patch-by-path basis, the cytoplasm boundary of each overlapping cell. The proposed framework also uses a supervised classifier to separate cell clumps from the background. Moreover, it uses feature detection algorithm, maximally stable extremal regions (MSER) algorithm [135], to detect the nucleus of each cell in each

clumps. The centriod of each detected nuclei is used to define the major possible region of each cell in the clump. Then, the framework proceeds to allocate the cytoplasm region of each cell. The active contour within the patch deforms under the influence of GVF forces computed based on the local edges depicted in each patch region. This is important to reduce the computational cost and to provide precise features instead of computing this over the whole image domain where small edge features are neglected. The main goal of our framework is to provide fully segmented cells with high-accuracy compared to ground truth and other methods that also segment overlapping cervical cells [110, 136].

The detection and segmentation of overlapping cells are complicated tasks because several layers of cervical cells are present on a glass slide, which means that cells in an upper layer can partially obscure cells lying underneath [137]. This makes the automated detection and segmentation of overlapping cells more complicated. The cytologist, in a manual examination, uses the depth cue that focus provides in order to assist in the interpretation of the overlapping cells. However, the separation of transparent layers from different fields of view (FOVs) is both computationally intense and difficult [138], because overlapping cells are subject to poor contrast and are located at similar focal depths. Therefore, extended depth of field (EDF) methods propose to tackle this issue by producing a single image where all objects are in focus [139]. This approach is more efficient than analyzing a stack of image with overlapping cells. The proposed framework analyses a single EDF image where all objects are in focus.

The proposed framework can be divided into two steps: an initial clump segmentation followed by a detailed segmentation of each individual cell. The first step consists of the following stages: (i) detecting cell clumps using a supervised classifier; (ii) using MSER for nuclei detection; and (iii) estimating the maximum cytoplasm region of each cell. The second stage consists of a segmentation using patch-based parametric active contour based GVF forces [36] as the main force for

curve deformation. GVF, in general, is a static force [36] when it is computed over the image domain. However, in the framework, the GVF computed for each patch which is different from that computes over the whole image, for the same patch region. Results show that the proposed methodology for cytoplasm segmentation leads to more accurate segmentation results compared to the current state-of-art methods [110, 136].

The rest of the chapter is organized as follows. Section 2 details the proposed methodology. Our proposed external force is detailed in Section 3. Experimental results for segmentation of real EDF images are presented in Section 4. Discussions of the segmentation results is presented in Section 5. Finally, the summary is set out in Section 6.