Multiscale Normalised Cuts Segmentation 3.3

In a graph-based segmentation, a graph 𝐺 = (𝑉, 𝐸, 𝑊) represents the relationship of nodes (vertices) 𝑉 connected by edges 𝐸 with weights 𝑊 associated on each edge. In the MNcut, the graph 𝐺 has the pixels as graph nodes 𝑉, and pixels within distance less than 𝑅 are connected by a graph edge in 𝐸. The relationship of pixel connection is computed through a mixed weight value matrix 𝑾𝑴𝒊𝒙𝒆𝒅 (equation (3.21)), which measures the likelihood of pixels 𝑎 and 𝑏 belonging to the same coherent region. The 𝑾𝑴𝒊𝒙𝒆𝒅 is a combined two grouping cues, pairwise pixel affinities of intensity 𝑾𝑰 (equation (3.19)) and pairwise pixel affinities of intervening contour 𝑾𝑪 (equation (3.20)).

𝑊𝐼(𝑎, 𝑏) = ⁡ 𝑒− ∥𝑋𝑎−𝑋𝑏∥2 𝜎𝑥 ⁡−⁡ ∥𝐼𝑎−𝐼𝑏∥2 𝜎𝐼 (3.19) 𝑊_𝐶(𝑎, 𝑏) = ⁡ 𝑒−𝑚𝑎𝑥𝑥∈𝑙𝑖𝑛𝑒(𝑎,𝑏)∥𝐸𝑑𝑔𝑒(𝑥)∥ 2 𝜎𝐶 (3.20) 𝑊𝑀𝑖𝑥𝑒𝑑(𝑎, 𝑏) = √𝑊𝐼(𝑎, 𝑏) × 𝑊𝐶(𝑎, 𝑏) + ⁡𝛼𝑊𝐶(𝑎, 𝑏) (3.21)

where 𝑋𝑎 and 𝐼𝑎 is the location and intensity of pixel 𝑎; 𝜎𝑥, 𝜎𝐼, and 𝜎𝐶 is a variance of pixel location, intensity, and edge; 𝑙𝑖𝑛𝑒(𝑎, 𝑏) is a straight line joining pixels 𝑎 and 𝑏; 𝐸𝑑𝑔𝑒(𝑥) is the edge strength at location 𝑥; 𝛼 is a parameter, contributing to the magnitude of 𝑊𝑀𝑖𝑥𝑒𝑑(𝑎, 𝑏).

The bipartition of a graph 𝑉 = 𝐴⁡ ∪ 𝐵 is based on the Normalised Cuts algorithm, partitioning the graph by maximising the ratio of affinities with a group to that across groups, as defined in equation (3.22) and (3.23).

𝑁𝑐𝑢𝑡(𝐴, 𝐵) = 𝐶𝑢𝑡(𝐴, 𝐵)

𝑉𝑜𝑙𝑢𝑚𝑒(𝐴) × 𝑉𝑜𝑙𝑢𝑚𝑒(𝐵) (3.22)

where 𝐶𝑢𝑡(𝐴, 𝐵) = ⁡ ∑ 𝑊

𝑎∈𝐴,𝑏∈𝐵

(𝑎, 𝑏) _(3.23)

The binary group vector is defined as 𝑋𝐴 ∈ {0,1}𝑁, with 𝑋𝐴,𝑎 = 1 if pixel 𝑎 belongs to segment 𝐴, otherwise 0 if the pixel belongs to segment 𝐵. The segmentation follows the criteria: 𝑚𝑎𝑥𝑖𝑚𝑖𝑠𝑒⁡1 2∑ 𝑿_𝒍𝑻_𝑾𝑿 𝒍 𝑿_𝒍𝑻_𝑫𝑿 𝒍 2 𝑙=1 (3.24)

where 𝑫 is a diagonal matrix, 𝐷(𝑎, 𝑎) = ⁡ ∑ 𝑊(𝑎, 𝑏)𝑗 , 𝑾 is the mixed weight values of affinity graph.

In multiscale graph segmentation, the graphs are decomposed into different scales and are connected by three different types of matrices: the multiscale affinity matrix, the cross-scale interpolation matrix, the cross-scale constraint matrix. To connect between scales, pixels are interpolated by using the cross-scale interpolation matrices 𝑪𝒔,𝒔+𝟏 defined in equation (3.25). 𝐶𝑠,𝑠+1(𝑎, 𝑏) = ⁡ { 1 |𝑁𝑎|⁡⁡𝑖𝑓⁡𝑏 ∈ ⁡ 𝑁𝑎 0⁡𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (3.25)

where 𝑁𝑎 is sampling neighbourhood of 𝑎.

To partition graph into different scales, the matrix 𝑪𝑠,𝑠+1 together with the cross-scale constraint matrices 𝑪 are employed to control the relationship between nodes in layer 𝑰𝒔 and 𝑰𝒔+𝟏. The matrix 𝑪 has values in a diagonal direction as shown in equation (3.26).

𝑪 = ⁡ (𝑪𝟏,𝟐 0 ⁡ ⋱ ⁡ −𝑰_𝟐 𝑪𝑺−𝟏,𝑺⁡ ⋱ ⁡ 0 −𝑰_𝑺⁡⁡⁡) (3.26)

The cross-scale constraint equation satisfies the condition in equation (3.27).

𝑪𝑿 = ⁡0 (3.27)

where 𝑿 = ⁡ ( 𝑋₁

⋮

𝑋_𝑠)⁡ and 𝑋𝑠⁡ ∈ {0,1}⁡

𝑁𝑠∗⁡×𝐾 _{is the partitioning matrix at scale 𝑠, 𝑁}

𝑠∗ = ⁡ ∑ 𝑁𝑠 𝑠,

𝑋𝑠+1(𝑎) = ⁡_|𝑁1

𝑎|∑𝑏∈𝑁𝑎𝑋𝑠(𝑏). The neighbourhood 𝑁𝑖 specifies the projection of 𝑎 ∈ 𝑰𝒔+𝟏⁡on the finer layer 𝑰𝒔.

The multiscale affinity matrix covers the full range of affinity matrix in each scale and is defined as the addition of affinity matrix from each scale as shown in equation (3.28).

This 𝑾𝑭𝒖𝒍𝒍 is memory inefficient and become unmanageable when an image size is increasing. As a result a compressed multiscale affinity matrix 𝑾𝒓𝒆𝒄𝒐𝒏𝒔𝒕𝒓𝒖𝒄𝒕 is reconstructed by using cross-scale interpolation matrix 𝑪𝒔,𝒔+𝟏 and recomputing 𝑾𝒔 by either sub-sample image or sampling values from the affinity matrix 𝑾𝒔−𝟏. The reconstruction affinity matrix is computed by using equation (3.29).

𝑾_{𝒓𝒆𝒄𝒐𝒏𝒔𝒕𝒓𝒖𝒄𝒕}= 𝑾_𝟏+ 𝑪_𝟏,𝟐𝑻 _𝑾 𝟐 𝒄_𝑪

𝟏,𝟐+ ⋯ + 𝑪𝒔,𝒔+𝟏𝑻 𝑾𝒔𝒄𝑪𝒔,𝒔+𝟏 (3.29) For the multiple partition, a generalized K-way Ncut function can be similarly defined using 𝑿 = [𝑋1, … 𝑋𝐾]. The optimal multiscale normalised cut partitioning can be solved by computing the 𝐾 eigenvectors, corresponding to the 𝐾 largest eigenvalues, with the maximising criteria of equation (3.30). The 𝐾 largest eigenvalues is determined by the number of segments altered by a user. Selection of 𝐾 values depends on the complexity of tumour structure. The higher the number of 𝐾 values, the finer the composition is acquired. It is obvious that the higher the 𝐾 values, the longer the computation time.

𝑚𝑎𝑥𝑖𝑚𝑖𝑠𝑒⁡1 𝐾∑ 𝑿_𝒍𝑻𝑾𝑿𝒍 𝑿_𝒍𝑻_𝑫𝑿 𝒍 𝐾 𝑙=1 (3.30)

Graphically, one-dimensional view of multiple-scale graph decomposition with 𝑅 = 1 is decomposed into three scales, 𝑠 ∈ {1, 2, 3}, as shown in Figure 3.4. At each scale, pixels are connected by the relationship defined by the affinity matrices: 𝑾𝟏, 𝑾𝟐 and 𝑾𝟑 for scales 1, 2 and 3 respectively. The connections between scales are defined by the cross- scale interpolation matrices: 𝑪𝟏,𝟐 and 𝑪𝟐,𝟑. Pixels at each scale are sampled at (2𝑅 + 1)𝑠−1_{. The value 𝑅 determines the relationship of two pixels in a graph, i.e. the two pixels} are connected if they are within distance 𝑅. The length of 𝑅 offers distinct segmentation results. Generally, a larger 𝑅 produces better segmentation because a long range graph

connections facilitate propagation of local grouping cues across larger image regions. However, the larger 𝑅 requires longer segmentation time.

Figure 3.4: Three scales of the MNcut graph decomposition with 𝑅 = 1. Relationship between pixels at each scale and cross scales are defined by affinity matrices 𝑊𝑠 and cross-scale interpolation matrices 𝐶𝑠,𝑠+1.

Evaluation of Segmentation