The Conditional Distribution Representation: Second-

The multivariate distribution representation given in Section 2.2.2 assumes the conditional independence of intensities in a pixel neighborhood given the center pixel’s intensity. Let x

be the intensity of a pixel and let y1, . . . , yn be the intensities of x’s neighboring pixels. In Section 2.2.2, it was shown that p(x, y1, . . . , yn) can be rewritten as p(x)·Qn_i=1p(yi|x) us- ing Bayes rule and conditional independence. This is exactly the equation used by Paget to describe second-order Strong-MRF models [Pag04]. Paget describes Strong-MRF models in detail and demonstrates that they capture sufficient information for the synthesis of some natural textures. The Strong-MRF model simplifies the MRF model by additionally assuming that a neighborhood’s cliques are independent. This allows a neighborhood’s joint intensity distribution to be reduced to the product of the clique probabilities. The equation above cor- responds to a second-order Strong-MRF model since it is constructed from the neighborhood’s first-order (single pixel) and second-order (two pixel) cliques.

Second-order Strong-MRF models are also related to gray level co-occurrence matrices; both only capture pairwise pixel information. Specifically, p(x)·Qn

i=1p(yi|x) consists of the

first-order distribution p(x) and n GLCMs p(yi|x) at specific pairwise spatial relationships. The representation of each p(y|x) given in Section 2.2.2 can be viewed as an alternative, QF based representation of GLCMs. This representation is typically much more compressed than the full histogram of pairwise intensities, and it has a Euclidean distance related to the EMD. Figure 2.13 on page 46 depicts the representation of eachp(yi|x). Each conditional distri- bution is represented by j×k bins, where x is divided into k quantiles. For each of these k

conditions,j-bin QFs ofyiare estimated. Typically,j= 4×k= 4 bins are used for each condi- tional distribution. These are well estimated from the approximately 40,000 sample empirical distributions estimated from each image. Each conditional distribution is an independently estimated local feature, much like each filter response used in Section 3.2.

sults using this texture model are presented.

Multi-scale Neighborhoods

This section describes an MRF texture model that uses compact pixel neighborhoods of sizes 3×3, 5×5, and 7×7, with 9, 25, and 49 local texture features, respectively. While this model has a much smaller spatial extent than filter bank methods, it quickly produces more features as neighborhood size is increased. This quadratic increase in the number of local features can become computationally prohibitive. Therefore, I also explore multi-scale neighborhoods to more compactly increase spatial extent. Pioneering work on multi-scale image representations was done in [HB95].

I define a pixel’s multi-scale neighborhood to include the original 3×3 local neighborhood. Then, I use a Gaussian filter with σ=√2 to generate successively blurred images with pixels that summarize progressively larger spatial extents. At each scale, a 3×3 neighborhood is defined by doubling the distance between each of these 9 pixels and the center pixel.

Results

Classification results are now presented using this Strong-MRF model on the experiment discussed in Sections 3.1 and 3.2. As in Section 3.2, unless otherwise specified all results are averaged over 100 random training and target splits, cross-validation is performed to estimate the common projection error across classes, and training uses 46 images per class.

Table 3.3 summarizes the classification accuracy of QF-NN and QF-QDA for different neighborhood sizes 1. As found in Section 3.2, QF-QDA consistently outperforms QF-NN. QF-QDA achieves the high accuracy of 98.24% using a 3×3 neighborhood and accuracies

>99% for larger neighborhoods. These results use 4×4 bins for each conditional distribution, which constructs 16, 144, 400, 784, 288, and 432 dimensional vector representations of each image for 1×1, 3×3, 5×5, 7×7, 2 scale 3×3, and 3 scale 3×3 neighborhoods, respectively. Compared with the MR8 QF-QDA results from Section 3.2, the Strong-MRF results using neighborhoods larger than 3×3 surpass the results using the original MR8-1 and MR8-2 filters

1_{An early version of these results where presented in [Bro05], wheren×1, instead ofn×n, neighborhoods} were computed due to a programming error.

Table 3.3: Classification accuracy of QF-NN and QF-QDA using second-order Strong- MRF texture features. The results use 4×4 bins for each conditional distribution, 4 QF bins for each of 4 conditions.

Neighborhood Size Strong-MRF QF-NN Strong-MRF QF-QDA

1x1 63.05 ±1.72 65.66 ±1.57 3x3 89.19 ±1.16 98.24 ±0.58 5x5 93.12 ±1.01 99.31 ±0.41 7x7 94.58 ±0.87 99.43 ±0.34 3x3, 2 Scales 93.33 ±1.01 99.33 ±0.38 3x3, 3 Scales 95.04 ±0.93 99.55 ±0.35

and are equivalent to results using the hand-tuned MR8-3 filters. These results also surpass the previous best MR8 filter bank based results, the 98.46% achieved by Hayman’s SVM classifier [HCFE04].

The results summarized in Table 3.3 can also be compared to previous MRF based texture models. Varma & Zisserman constructed several MRF models that, like their method based on the MR8 filter bank, estimate the joint distribution of a neighborhood’s intensities through clustering [VZ03]. Their best model clusters in the joint space of a pixel neighborhood without the center pixel. Then for each cluster the univariate distribution of the center pixel’s intensity is modeled with a histogram. This model with a NN classifier achieves an accuracy of 95.87%, 97.22%, and 97.47%, using 3×3, 5×5, and 7×7 neighborhoods, respectively. These results use 610 textons and 90 bins for the center histogram, for a 54,900 dimensional representation. Their best classification result of 98.03% uses 2440 textons (219,600 values) and a 7×7 neighborhood. The results presented in Table 3.3 are more accurate and compact.

Good classification results are also achieved by the Strong-MRF QF-QDA classifier when modeling each conditional distribution with fewer than 4×4 bins. For example, the 3-scale, 3×3 strong-MRF QF-QDA classifier achieves an accuracy of 99.47% using 1×4 bins. This model, which only computes the mean of 4 conditions for each conditional distribution, constructs a more compact, 108 dimensional vector representation for each image.

Figure 3.8 shows the accuracy of the 3-scale, 3 ×3 Strong-MRF QF-NN and QF-QDA classifiers as training set size and the size of each conditional distribution is varied. The right graph shows that for smaller training set sizes, the Strong-MRF QF-QDA classifier performs

1 2 4 8 16 32 64 20 60 70 80 85 90 95 100

Bins per condition and marginal

Classification Accuracy (%) 1 Condition 2 Conditions 4 Conditions 8 Conditions 4 10 20 30 40 46 60 75 85 90 95 100

Training set size per material

Classification Accuracy (%)

S

Strong-MRF QF-QDA Strong-MRF QF-NN

Figure 3.8: Left: Varying the number of QF and conditional bins for the 3 scale, 3x3 Strong-MRF QF-QDA (solid lines) and QF-NN (dashed lines) classifiers. Right: Varying the number of images available during training, using 4×4 bins. The MR8-3M QF-QDA and QF-NN results are in black.

similarly to but not as accurately as the MR8-3M QF-QDA classifier. The left graph shows that for the 3-scale, 3×3 conditional distributions, only a small number of conditions and QF bins are required. QF-QDA achieves an accuracy of 96.4% using 1 condition per distribution, which only models the three multi-scale, marginal intensity distributions.

The Strong-MRF QF-QDA model achieves excellent classification results for small neigh- borhood sizes. However, compared to the MR8 QF-QDA classifier, it is not as compact, and it does not perform as well for smaller training set sizes. The Strong-MRF model also requires 2 parameters for each conditional distribution to be specified. These issues are addressed by the second MRF model presented next in Section 3.3.2. The accuracy and compactness of these MRF models are discussed further at the end of Section 3.3.2.

3.3.2 The PCA Based Projections Representation: Learning a Linear Filter