7.2 Content-Based Image Retrieval of Different Museum Databases
7.2.2 Query by Texture
The multiscale-based and the segmentation-based approaches are both evaluated on the three museum databases using the procedures and parameters described in chapter 5 and 6 respectively. The three databases carry quite a different challenge. The Victo- ria and Albert Museum database has largely an image of an object, hence is not too complex, but the database size is the biggest. The National Gallery database contains more complex images because all of its images are paintings of a scene. Finally the C2RMF database offers the most difficult challenge where most of its images are quite a smudgy painting (because they are originally for use in restoration research) and also not many textures are significantly visible. Due to limitation in space, it is not possible to give too many examples of the retrieval performance of the algorithms. Nonetheless all the observations from the experiments are summarized below as well as suggested improvements to the algorithms. Figure 7.4, 7.5 and 7.6 show examples of retrieval results using both algorithms on the National Gallery, Victoria and Albert Museum and C2RMF databases respectively.
The most important observation from the experiments is that both algorithms work well with all three museum databases. Even for the C2RMF database, both algorithms manage to retrieve visually similar texture although its performance is not as good as for the other two databases. This is because the smudgy nature of images within the database brings a high level of confusion. The other two databases on the other hand show a much better retrieval result. Even for a database size of 16000 images for the Victoria and Albert Museum database, both query by texture algorithms manage to retrieve similar textures to the query. Both algorithms can be said to be applicable in the content-based image retrieval system.
The next observation is to compare the performance between the multiscale-based ap- proach and the segmentation-based approach. From the experiments, it was observed that the multiscale-based approach gives better performance than the segmentation- based approach. One example can be seen from Figure 7.1. In this particular example, the multiscale-based approach manages to retrieve two images which have exactly the same texture as the query (images ranked first and third) while the segmentation-based approach can only manage to retrieve one of them. This is probably caused by a default in the segmentation process or the lack of multiscale property within the segmentation- based approach. It was also observed that the patches retrieved by the multiscale-based approach are also much more similar to the query. The segmentation-based approach
Figure 7.4: Retrieval example of National Gallery database (top) query image, (mid- dle) result using multiscale-based approach, (bottom) result using segmentation-based
approach
however has a much faster retrieval speed. This is because the number of feature vec- tors of the segmentation-based algorithm for a particular image is much lower than the multiscale-based algorithm. Overall, the advantages and disadvantages of both algo- rithms can be summarized as follows.
The multiscale-based approach has the advantage of much better accuracy at the expense of computational load. The multiscale nature of this approach also adds to its advantages as it has been proved to be useful in capturing both coarse and fine textures. The
Figure 7.5: Retrieval example of Victoria and Albert Museum database (top) query image, (middle) result using multiscale-based approach, (bottom) result using
segmentation-based approach
algorithm however can suffer from some odd retrieval result. For example when two textures are captured by a sub-image, the resulting feature vector might be similar to a feature vector of a completely different query texture, and hence will be retrieved as one of the top matches. However this is only a minor problem as it does not affect the overall performance of the algorithm. An improvement to the multiscale-based approach will mainly be on reducing the computational load. One of the possibilities is to use thecase 1 overlapping for sub-image coverage instead ofcase 2 overlapping (see section 5.3.1), although the accuracy might drop quite drastically as well. Another possibility is to introduce the texture identifier like the one used for the segmentation process to
Figure 7.6: Retrieval example of C2RMF database (top) query image, (middle) result using multiscale-based approach, (bottom) result using segmentation-based approach
decide whether a particular sub-image is textured or not. The feature vectors are then created only for the textured patches, hence reducing the total number of feature vectors greatly. Finally a suitable multidimensional indexing algorithm can also be associated to help speed up the matching process.
The segmentation-based algorithm has the advantage of retrieval speed, although the accuracy is not as good as the multiscale-based approach. The computational load for the feature extraction process is also lower than that of the multiscale-based approach. Although this approach lacks the multiscale property, it can on the other hand provide the shape of the segmented objects. This might be useful for later purposes such as object identification or shape-from-texture processes. On the downside, this approach can suffer from a default in segmentation process, where either a texture is classified as insignificant or a texture is classified together with another texture. An improvement
to the algorithm means improving the texture segmenter itself to make it more accurate and reliable. If the multiscale property is desired, then a way for integrating it to the segmentation-based algorithm can be investigated.
Finally the choice between the multiscale-based and the segmentation-based approaches depends on application. If the user is willing to compensate a longer time for a better accuracy, then the multiscale-based approach will be suitable. However if the user wants a quick retrieval response and is willing to tolerate a slightly riskier outcome, then the segmentation-based approach will be more suitable. An algorithm which combines the advantages of the multiscale-approach with the advantages of the segmentation-based approach therefore needs to be explored.