Spatial Domain Defined Filtering Features

With respect to the two-stage model, the filtering operation is regarded as the first stage. Response matrices are considered as higher order statistics as the positional information in the matrix (image) is implicitly retained. The spatial extent exploited by spatial do- main defined filtering featues in this stage is the size of the masks or filters involved. The post-processing of the response matrices is taken as the second stage. However, for those features designed for texture segmentation, the high dimensional output of this stage is not applicable for image-based similarity measurement (or estimation).

Discrete Cosine Transform Based Channel Filters (DCT)

Ng et al. [1992] interpreted the local linear transform, e.g. the discrete cosine transform (DCT), as a multichannel spatial filtering approach. Although this feature set is inspired by the DCT which has many similarities with the FFT, it is implemented in the spatial domain. Nine 3×3 filter masks (see Figure 3.3 (a)) are obtained from three 1D DCT ba- sis vectors:

{ } , { } , and { } . (3.2)

After the filtering operation is performed by convolving the filter masks with the image, local variance matrices are computed from response matrices based on 15×15 local windows and are utilised as feature maps.

(a) (b)

Figure 3.3: (a): Nine 3×3 DCT masks; and (b) nine 3×3 eigen masks obtained from a texture image.

Stage 1 The filtering operation is conducted in the first stage. Response matrices are higher order statistics and the spatial extent used for each operation is 3×3 pixels.

47

Stage 2 The computation of local variance feature maps is performed in the second stage. However, these feature maps are not directly applicable for image-based similari- ty measurement.

Eigen Filters

Given that is an image and is the expectation function, a 9×9 matrix

[ ] (3.3)

is estimated, and eigenvectors and eigenvalues are computed for each texture image [Ade, 1983]. Each 9×1 eigenvector is considered as a 3×3 eigen mask (see Figure 3.3 (b), termed as “EIGENFILTER”). The mean of the absolute values of the differences between pixel values and the local mean within a 15×15 window are also calculated for each position in a response image. The means computed from all response matrices are employed as the features of the pixel at the corresponding position.

Stage 1 The filtering operation is performed in the first stage. Response matrices are higher order statistics and the spatial extent utilised for each operation is 3×3 pixels. Stage 2 The computation of feature maps is conducted in the second stage. However, these feature maps are not applicable for image-based similarity measurement.

Gabor Energy Filters

Fogel and Sagi [1989] represented a texture image by computing the Gabor power spec- trum of micro-patterns. The Gabor function (termed as “GABORENERGY” is defined as:

| _, (3.4)

where is the Gaussian width, is the filter orientation, is its frequency, is its phase angle, and is the centre of the filter. Given that represents an input image and stands for a Gabor filter, then ( means the convolution opera- tion) can encode spectra for different orientations and shifts. The sum of the squares of two response matrices is computed for each pixel. Thus, only power information is

48

used. The feature map is utilised for the representation of one micropattern image or montaged image.

Stage 1 The filtering operation is performed in the first stage. Response matrices are higher order statistics and the spatial extent utilised for each operation is 17×17 pixels. Stage 2 The computation of the feature map is conducted in the second stage. However, the feature map is not applicable for image-based similarity measurement.

Laws Masks

Laws [1980] convolved an image with a set of 2D masks (referred to as “LAWS”) to extract texture features. In total, 25 2D masks can be obtained by convolving a vertical 1D mask with a horizontal 1D mask, given five 1D masks: L5 = [1 4 6 4 1], E5 = [-1 -2 0 2 1], S5 = [-1 0 2 0 -1], W5 = [-1 2 0 -2 1] and R5 = [1 -4 6 -4 1]. The mean of the ab- solute values of responses or the square root of the sums of the squares of responses within a 15×15 windows, i.e. “texture energy measure”, is computed for each position in one response image. These “energy measure” maps are finally used as features in- stead of the response images.

Stage 1 The filtering operation is conducted in the first stage. Response matrices are higher order statistics and the spatial extent utilised for each operation is 5×5 pixels. Stage 2 The computation of “energy measures” feature maps is performed in the second stage. However, these feature maps are not applicable for image-based similarity meas- urement.

Localised Gabor Filters

Considering different textures possess distinct dominant characterising frequencies, Bo- vik et al. [1990] introduced a type of complex 2D Gabor functions (referred to as “GABORBOVIK”) which is expressed as

, (3.5) where , √ , is the cen- tral frequency which is chosen from the frequency at which one spectral peak of a tex- ture occurs, and

49

⁄ . (3.6) Consequently, can be taken as a complex sinusoidal grating modulated by a 2D Gaussian envelope with an aspect ratio , scale parameter , and the major axis orients at an angle from the x-axis. A post-processing, including a nonlinear process and a linear process, is also applied on each response matrix in sequence.

Stage 1 The filtering operation is conducted in the first stage. Response matrices are higher order statistics and the spatial extent utilised for each filtering operation is 85×3, 43×3, 21×3, 11×3 and 11×3 pixels at five different resolutions respectively.

Stage 2 The post-processing is performed in the second stage. However, the feature maps are not applicable for image-based similarity measurement.

Dyadic Gabor Filter Bank

Jain et al. [1991] developed a multi-channel filtering scheme using real-valued and even-symmetric Gabor filters (see Figure 3.4). The impulse response of an even- symmetric Gabor filter (termed as “GABORJFSD”) is expressed as

{ [

]} _, (3.7)

where stands for the frequency of a sinusoidal grating along the x-axis, and and are constants of a Gaussian envelope along x and y axes respectively. A nonlinear process is also performed on each response matrix.

Figure 3.4: Even-symmetric Gabor spatial filters at nine different orientations. For dis- play purposes, each filter is padded into a square matrix.

Stage 1 The filtering operation is performed in the first stage. Response matrices are higher order statistics and the spatial extent utilised for each operation is 409×329, 205×165, 103×83, 51×41 and 27×21 pixels at five different resolutions respectively. Stage 2 The nonlinear process is conducted in the second stage. However, the feature maps are not applicable for image-based similarity measurement.

50 Leung-Malik Filter Bank

A hybrid filter bank (termed as “LM”, see Figure 3.5), including 36 Gaussian derivative filters (1st- and 2nd-order derivatives at six orientations and three scales), eight Laplaci- an of Gaussian filters and four Gaussian low pass filters, was utilised by Leung and Ma- lik [2001]. Given that is a Gaussian function, 1st-order Gaussian derivative fil- ters are defined as

and . (3.8) where is the scale (standard deviation).

Stage 1 The filtering operation is conducted in the first stage. Response matrices are higher order statistics and the spatial extent utilised for each operation is 49×49 pixels. Stage 2 In the original publication, response matrices were used to extract textons and accumulate texton histograms. Since we only use the response matrices, there is no op- eration in the second stage. In addition, the response matrices are not applicable for im- age-based similarity measurement.

Figure 3.5: LM (spatial) filter bank [Varma and Zisserman, 2005].

Schmid Filter Bank

Schmid [2001] utilised a bank of 13 rotation-invariant isotropic “Gabor-like” filters (see Figure 3.6, termed as “S”) to obtain grey level descriptors. These filters are defined as

√ _, (3.9)

where is the number of cycles of the harmonic function enclosed by the Gaussian en- velope of a filter, is the scale (standard deviation) and is added to the func- tion to obtain a zero DC (Direct Current) component.

51

Figure 3.6: 13 isotropic (spatial) filters used by the Schmid filter bank [Varma and Zis- serman, 2005].

Stage 1 The filtering operation is performed in the first stage. Response matrices are higher order statistics and the spatial extent utilised for each operation is 49×49 pixels. Stage 2 In the original publication, the response matrices were used to cluster centroids. Since we only use the response matrices, the second stage is null. Besides, the response matrices are not applicable for image-based similarity measurement.

Root Filter Set and Maximum Response Set

Varma and Zisserman [2005] constructed a hybrid filter bank (Root Filter Set, i.e. RFS, see Figure 3.7) which involves 36 Gaussian derivative filters (see Equation (3.8)), one Gaussian low pass filter and one Laplacian of Gaussian filter. Furthermore, filter re- sponses obtained at different orientations but the same scale are “collapsed” and only the maximum filter response over all orientations at each scale is kept, in order to achieve approximate rotation invariance. Finally, only six maximum filter responses and two isotropic filter responses, namely, maximum response set (MR8), are used for each pixel. Motivated by Weber’s law, the filter response at each pixel is normal- ised as

(

)

, (3.10)

where ‖ ‖ is the magnitude of the filter response vector at that pixel. Stage 1 The filtering operation is performed in the first stage. Response matrices are higher order statistics and the spatial extent utilised for each operation is 49×49 pixels. Stage 2 In the original publication, normalised response matrices were used to extract textons and texton histograms. Since we only use the normalised response matrices, the normalisation operation is regarded as the second stage. However, the normalised re- sponse matrices are not applicable for image-based similarity measurement.

52

Figure 3.7: Root Filter Set (spatial filters) [Varma and Zisserman, 2005].