Content Based Image Re-ranking using Indexing Methods

(1)

International Journal of Emerging Technology and Advanced Engineering

Website: www.ijetae.com (ISSN 2250-2459,ISO 9001:2008 Certified Journal, Volume 5, Issue 8, August 2015)

447

Content Based Image Re-ranking using Indexing Methods

Rachana C. Patil

1

, Prof. S. R. Durugkar

2

1_{M.E. Student, Dept. of Computer Engg., SNDCOE & RC, Yeola, Savitribai Phule university, Pune, India} 2

Head of Computer Engg. Department, SNDCOE & RC, Yeola, Savitribai Phule university, Pune, India

Abstract— With increase in number of digital images,

retrieval of images efﬁciently becomes an important topic for research. Traditional methods for image retrieval used meta-data associated with images, commonly known as keywords. These methods empowered many World Wide Web search engines and achieved reasonable amount of accuracy.

A novel image re-ranking framework propose, which offline learns different visual semantic spaces automatically for different entered query keywords by using keyword expansions. We have projected the visual features of images into their associated visual semantic spaces, to generate the semantic signature. Images are re-ranked by comparing the semantic signature of images in online stage. CBIR uses contents of image, such as shape, colour, texture or any other information that can be derived from image itself. Although there are many problems associated with CBIR method. Amongst them semantic gap with image features has received a lot of attention. Images are represented by low-level features and it is important to reduce semantic gap between high-level and low-level features of images to retrieve visual similar images and re-ranking of images. This study proposes latent semantic indexing (LSI) method to re-rank images that are retrieved using image retrieval method.

A modified approach of LSI; PLSI evaluates the association degree of each document to each subject, and then categorize the search results into subjects by using that information. In this project we propose a Markovian Semantic Indexing (MSI) is offered in the framework of an online image retrieving system. For Annotation-Based Image Retrieval (ABIR) tasks, the properties of MSI make it particularly suitable when the per image annotation data is limited. In the context of online image retrieval systems the characteristics of this method is mostly relevant.

Keywords—Image search, semantic space, semantic

signature, image re-ranking, keyword expansion, LSI, PLSI, MSI

I. INTRODUCTION

As the variety and size of digital image collections produce exponentially, effective image retrieval is becoming increasingly important. Image search is the process of retrieving and displaying relevant images based on user‟s queries from a database. Recently, saleable web scale image search engines accomplish only keywords as queries. There are certain types of images are displayed when users enter query keywords.

The web scale search engine continues with number of images ranked by the given keywords take out from the surrounding text. But the result of text-based image search suffers from the ambiguity of query keywords. Hence, many of retrieved images are noisy, disordered, or inappropriate. Thus, it is difficult for users to exactly define the visual content of mark images only using keywords. For example, using “Sony” as an input query keyword, then the recovered images is a member of different categories, such as “Sony TV”, “Sony Handset”, and “Sony Laptop”.

(2)

International Journal of Emerging Technology and Advanced Engineering

448

Contents of an image can be of several forms like, texture, shape and color etc. CBIR approach is more robust and makes it easier for image retrieval.

When user entered a query keyword as an input to search images, then according to a stored word-image index ﬁle, set of images which relevant to the query keyword are first retrieved by using the search engine. By asking a user to select an entered query image, which reﬂects as per the user‟s search intention, from the pool, on the basis of visual similarities of images with the query image these remaining images are re-ranked. This system proposes latent semantic indexing (LSI) method to re-rank images that are retrieved using CBIR method to improve image retrieval as per user‟s intention. Additionally, we propose a Markovian Semantic Indexing (MSI) method for automatic annotation, indexing and annotation-based retrieval of images which is presented in the context of an online image retrieval system. The new method is shown to possess definite theoretic rewards and also to accomplish well Precision versus Recall results in Annotation-Based Image Retrieval (ABIR) tasks when compared the Latent Semantic Indexing (LSI) with Probabilistic Latent Semantic Indexing (PLSI) methods.

II. RELATED WORK

For calculating image similarity content-based image retrieval concept is used to find out the visual features of images. Significance feedback was broadly used to learn visual similarity metrics to accomplish users‟ search intention [1]. Cui et al. [2, 3] projected an image re-ranking approach which narrows users‟ effort to just on one click response. By using web-scale image search engines such as Bing and Google recently, simple image re-ranking approach has been adopted for example the “search related images” task. The fundamental aim of image re-ranking is to compute the visual similarities among images. Several image features [5, 4, 6] have been established in recent years. Recently, for overall image recognition and corresponding, there have been a number of works on using predeﬁned concepts or attributes as image signature. Rasiwasia et al. [8] plotted visual features to a universal concept vocabulary. Lampert et al. [7] used predeﬁned features with semantic meanings to detect novel object classes. Some approaches transferred knowledge between object classes by calculating the similarities among new object classes and known object classes (called reference classes). Totally these reference-classes were universally applied to all the images and their training data was physically carefully chosen.

They are more appropriate with lower diversity (such as animal databases [7] and face databases [9]) in offline part such that object classes better share resemblances. An enormous set of concepts or reference classes are essential, to model all the web images, which is unreasonable and ineffective for online image re-ranking.

Recently, in excess of 200 content-based retrieval systems have been established [4], which are based on low level features. In precise, they can be categorized into 2 groups: 1) on the basis of examination for textual information associated to images, perform semantics mining, and 2) based on the removal of low-level visual features of images in collections. The first category is determined by on strenuous annotation, while the second method typically cannot successfully capture semantics. Mori et al [10] were the first to model a method for annotating image using grids in co-occurrences where each word assigned to the image is inherited by region inside the image. An orientation work for this task, Duygulu et al. [11] proposed a new method that preserved image explanation as a machine transformation which translates from textual keywords into the pictorial keywords. In both text-based queries and image captions, ABIR (Annotation Based Image Retrieval) systems include more effective semantic content that is several documents retrieval and indexing techniques achieved such as LSI [12], PLSI [13] were incorporated into ABIR systems.

A programmed image annotation model for latent space models is projected by Florent Monay et al [16] such as Latent and Probabilistic Latent Semantic Analysis. Annotation by straight match and LSA are based on comparison and annotation. Zhen Guo et al [14] projected citation-topic (CT) model for demonstrating associated documents. Firstly, apply LSA on multimedia documents for indexing and recovery, this purpose is found by Trong-Ton Pham et al [15]. They have reflected the outcome of LSA on multimedia document recovery and automatic image annotation.

(3)

International Journal of Emerging Technology and Advanced Engineering

449

LSI is a different method to search an image which is closer to query image based on underlying semantics of images. For matrix computation which drop out irrelevant images and locate images that have similar semantics nearer to each other in a multidimensional space used the concept of LSI which customs a mathematical technique called “Singular Value Decomposition (SVD)”.

ABIR (Annotation-Based Image Retrieval) systems include effective semantic content into text-based queries as well as for image captions. That methods are primarily established for document recovery may be appropriate for ABIR systems, as well. Latent Semantic Indexing was originally established for document recovery. Hofmann, based on the Aspect Model [19] presented the probabilistic Latent Semantic Indexing (pLSI) [20] as an alternative to projection (LSI) or clustering approaches for document recovery. Latent Dirichlet Allocation (LDA) was projected by Blei et al. [21] to report the limits of pLSI concerning simplification and appropriate while Griffiths and Steyvers combined a Markov chain Monte Carlo technique to LDA.

III. OUR APPROACH

A new context is proposed here for web image re-ranking. Rather than creating a general concept dictionary, for different query keywords it acquires different visual semantic spaces automatically and separately. When the images are re-ranked using the query keyword provided by the user then they can be signiﬁcantly narrowed down. For example, if an input query keyword is “Sony”, the semantic concepts of “mountains” and “Paris” are uncertain to be significant and can be unnoticed. As an alternative, the semantic concepts of “computers” and “TV” will be used to acquire the pictorial semantic space associated to “apple”.

The query speciﬁc visual semantic spaces can be used to exactly model the images which are re-ranked, hence they have removed other possibly infinite set of non-relevant concepts, which assist only as noise and decline the performance of re-ranking in relations of both computational cost and accuracy. For getting the semantic signatures, visual features of images are estimated into their associated visual semantic spaces. When we are comparing the semantic signatures of images which obtained from the visual semantic space of the given query keywords then images are re-ranked, at the online stage. Our experimentations display that the semantic space of an input query keyword can be defined by just 20 − 30 concepts (which is also referred as “reference classes”). Hence, the online image re-ranking becomes extremely efﬁcient and the semantic signatures are very short.

[image:3.612.329.585.194.456.2]

As of the large set of keywords and the active variants of the web, the visual semantic spaces of entered query keyword essential to be automatically learned. Rather than manually deﬁned, this is completed through keyword expansions in this framework.

Fig. 1 Architecture

(4)

International Journal of Emerging Technology and Advanced Engineering

450

This methodology is founded from the concepts of mining user questions by using Markov Chain model. The features of this method mark it also mostly valid in the framework of online image retrieval systems.

This paper describes the augmentations done to the base project effort by containing LSI methodology to increase image retrieval accuracy and efficiency. Few difficulties of LSI can be overwhelmed by probabilistic latent semantic indexing (PLSI) and Markov Chain semantic indexing (MSI) methods. PLSI method is a way to précis the search results of entered query keyword. Due to the procedure of estimation results, our method achieves well in classifying search results and decreasing the redundancy among summaries. We introduced the Markovian Semantic Indexing (MSI), a new method for automatic annotation and image retrieval based on annotation. The necessity of returning to external taxonomy systems for allocating significance metrics among keywords, and, thus, face the problem of assessing the compatibility stuck between those methods and the semantics behind the actual users that are using the system is improved in MSI. This experiment have exposed that MSI achieves better retrieval results in sparsely annotated images from the large image database.

IV. METHODOLOGY

A.Latent Semantic Indexing

Latent Semantic Indexing method is used to perform re-ranking of retrieved images. LSI method applies synonymy concept to given query keyword and then compares it with title and description of images to produce resultant similarity value in fraction (percentage). By using this similarity co-efficient image is re-ranked.

Example: Steps

a.Search query : „Water‟

b.Get all synonyms of query „water‟ from dictionary and store it in array. Like,

c.SynonymQuery[H2O, body of water, irrigate, watery, ….]

d.Compare these synonyms with array of title plus description of retrieved images.

e.RetrievedImages[title1, title2, title3,….,desc1, desc2, desc3,….]

f. Count ++

Where, words length = total length of word for comparison and Counter = total match found

B.Marrkovian Semantic Indexing:

On the other hand Markovain Semantic Indexing method obtains synonyms of image title and description and then compares it with the query keyword to retrieve similarity co-efficient for retrieved images and using these similarity co-efficient images are re-ranked.

Example: Steps

a.Search query : „Water‟

b.Get all synonyms of title plus description of retrieved images from dictionary and store it in array. Like, c.Synonym RetrievedImages [a, b, c…]

d.Compare these synonyms with given search query „water‟.

e.Count ++

Where, words length = total length of word for comparison

Counter = total match found

C.Probabilistic Latent Semantic Indexing:

Probabilistic Latent Semantic Indexing method is used to perform the re-ranking on the retrieved images. PLSI method is used to compute the synonyms of query keywords as well as synonyms of title and description of images and then perform the comparison and display the result in between 0 to 1. As per the resultant value this system performs the re-ranking on images.

Example: Steps

a. Search query : „Water‟

b. Get all synonyms of query „water‟ from dictionary and store it in array. Like,

c. Syn_Query[H2O, body of water, irrigate, watery, ….] d. Get all synonyms of title plus description of retrieved

images from dictionary and store it in array. Like, e. Syn_Images [a, b, c…]

f. Compare Syn_Query with Syn_Images. g. Count ++

Where, words length = total length of word for comparison

(5)

International Journal of Emerging Technology and Advanced Engineering

451

V. SYSTEM IMPLEMENTATION

In this system, when user enter the query keyword and click on the search button then this system searches the set of images as per the given query keyword and display the relevant images. The numbers of images are displayed on the basis of maximum count selected by the user. When user clicks on one particular image from the retrieved images then he/she can view the visually similar images as per the selected image as shown in fig.2.In fig. 3,displayed the resultant images as per the given query keyword with their title and description from both Google and local database. On the basis of displayed images title and description match found are displayed for each image as per the given query keyword.

Fig.2 Search images using query keyword

Fig.3 Result using Semantic Analysis

(6)

International Journal of Emerging Technology and Advanced Engineering

452

Fig.4 Re-rank the images using PLSI

Fig.6 Re-rank the images using MSI

VI. PERFORMANCE EVALUATION

With the aim of to evaluate the performance of the proposed system, we have used two recovery measurements which are Precision and recall. Precision and recall are the greatest standard metrics for associating CBIR, are also extensively used for estimating the efficacy of automatic image annotation methods. The ratio of the number of images that correctly retrieved to the total number of images retrieved in every image search is called as Precision.

When precision value is increased then more relevant images are retrieved, even though high recall specifies that rare relevant images are lost. An additional way of offering the performance of this system is by plotting precision and recall graph, in which precision and recall values are designed against the search ID. Following table shows the experimental results of this system.

(7)

International Journal of Emerging Technology and Advanced Engineering

453

Fig.7 Recall Graph

VII. CONCLUSIONS

This system have reviewed a novel image re-ranking framework by learning the query-specific semantic spaces it helps to significantly expand both the efficiency and effectiveness of online image re-ranking. Through keyword expansions the visual features of images are estimated into their related visual semantic spaces automatically, at offline stage.

This project introduced a system using which user is able to retrieve images based on query keyword from Google using web service and from local image database. This system also provided functionality where user can view the ranking provided to the image result set based on Latent Semantic Indexing (LSI), Probabilistic Latent Semantic Indexing (PLSI) and Markovian Semantic Indexing (MSI) methods. This system also provided precision and recall (P&R) graph where user can compare the performance of these ranking methods.

Currently this system does not use image clustering of retrieved images based on image similarity in terms of family of images, size, shape and colours, etc. I would like to address this functionality development on local image database in future work.

REFERENCES

[1] Xiaogang Wang, Ke Liu, and Xiaoou Tang, Web, Image Re- Ranking Using Query-Speciﬁc Semantic Signatures, IEEE Transactions on Pattern Analysis and Machine Intelligence (Volume:36,Issue:4) April 2014, DOI:10.1109/TPAMI.201 [2] J. Cui, F. Wen, and X. Tang. Intentsearch: Interactive on-line image

search re-ranking. In Proc. ACM Multimedia. ACM, 2008. [3] J. Cui, F. Wen, and X. Tang. Real time Google and live image

search re-ranking. In Proc. ACM Multimedia, 2008.

[4] N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Proc. CVPR, 2005.

[5] D. Lowe. Distinctive image features from scale-invariant keypoints. Intl Journal of Computer Vision, 2004.

[6] J. Philbin, M. Isard, J. Sivic, and A. Zisserman. Descriptor Learning for Efcient Retrieval. In Proc. ECCV, 2010.

[7] C. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In Proc. CVPR, 2005.

[8] N. Rasiwasia, P. J. Moreno, and N. Vasconcelos .Bridging the gap: Query by semantic example. IEEE Trans. on Multimedia, 2007. [9] Q. Yin, X. Tang, and J. Sun. An associate-predict model for face

recognition. In Proc. CVPR, 2011.

[10] Kobus Barnarda , Pinar Duygulub, and David Forsythc. Recognition as Translating Images into Text.

[11] Rong Jin, Joyce Y. Chai , Luo Si. Effective Automatic Image Annotation Via A Coherent Language Model and Active Learning, MM04, October 1016, 2004, New York, New York, USA.

[12] Pratap Kuraganti,Pitchaiah Repudi ,Image Retrieval: Automatic Annotation And Annotation-Based Accessing, International Journal Of Professional Engineering Studies, Volume IV/Issue3/NOV2014. [13] Z. Guo, S. Zhu, Y. Chi, Z. Zhang, and Y. Gong, A Latent Topic

Model for Linked Documents, Proc. 32nd Intl ACM SIGIR Conf. Research and Development in Information Retrieval (SIGIR), 2009. [14] Himali P. Chaudhari, Prof. Dinesh D. Patil, Improved Markovian

Model for Annotation Based Image Retrieval using Multiple and Synonymous Keywords ,International Journal of Computer Science and Mobile Computing, Vol.3 Issue.7, July- 2014.

[15] Michael W. Berry, Zlatko Drmac, Elizabeth R. Jessup ”Matrices, Vector Spaces and Information Retrieval” SIAM Review, 41(2), 336-362, Year1999.

[16] Gerard Salton, Harry Wu, E. A. Fox, Extended Boolean Information Retrieval Communications of ACM, Volume-26, Issue 11, Year-1983

[17] Wenhao Lu, Jingdong Wang, Xian-Sheng Hua, Shengjin Wang, Shipeng Li, Contextual Image Search,MM11, November 28December 1, 2011, Scottsdale, Arizona, USA.

[18] Mayuri D. Joshi, Revati M. Deshmukh, Kalashree N. Hemke, Ashwini Bhake and Rakhi Wajgi, IMAGE RETRIEVAL AND RE-RANKING TECHNIQUES, An International Journal (SIPIJ) Vol.5,No.2, April2014.

[19] T. Hofmann, “Probabilistic Latent Semantic Indexing,” Proc. 22nd Int‟l Conf. Research and Development in Information Retrieval (SIGIR ‟99), 1999.

[20] T. Hofmann, “Unsupervised Learning by Probabilistic Latent Semantic Analysis,” Machine Learning, vol. 42, no. 1/2, pp. 177- 196, 2001.

[21] D.M. Blei and A.Y. Ng, and M.I. Jordan, “Latent Dirichlet Allocation,” J. Machine Learning Research, vol. 3, pp. 993-1022, 2003.