Chapter 03 Semantic Query Interpreter for Image Search & Retrieval
4.3 Proposed Semantic Ranking Framework
4.3.1 Semantic Intensity
The Semantic Intensity can be defined as the “concept dominancy factor with in the image”. While image is the combination of different objects, these objects constitute to form different semantic idea. Different combination of objects depicts different concepts. The images can depict different semantic idea simultaneously. However, these semantic ideas have
different dominancy degree. Some of the ideas in the image are more dominant than the other as shown in the Figure 4.6.
Figure 4.6: The image is taken from the LabelMe dataset. Image depicts a list of concepts like road, vehicles, signs, buildings, sky, trees, umbrella, buildings, street, cross walk, highlight, flags etc. and some hidden concept like rain. Among all the concepts some are more dominant like street, building etc.
We have implemented a semantic Intensity concept on the LabelMe dataset which is open source dataset available for academic and research, the object in the LabelMe dataset images is represented by a set of points known as polygon. The polygon may be either a regular or irregular polygon.
Figure 4. 7: (a) Regular Polygon
The area A of a regular n-sided polygon having side s, apothem a, and circum-radius r is given by
(4.15)
While area of the irregular polygon is
∑
)
(4.16)
The is the object dominancy degree with in the image. The greater the
value greater will be the object dominancy degree.
(4.17)
The Concept Dominancy for the given object can be calculated as
=
(4.18)
Where , represents size of the image.
The Semantic Intensity (SI) for a particular concept relevant to the given query is calculated by the following expression.
(4.19)
Where are the expanded query terms with their appropriate semantic similarity
concepts. Some of the concepts in the query are more dominant then the other. We have calculated the dominancy level of the different concepts in the query by using the Semantic Query Interpreter. The Semantic Similarity value retrieved with the selected expanded terms depicts the dominancy degree of the concepts with the query.
Let‟s consider the scenario of the simple query car. The car query processed by the Semantic Query Interpreter module and the results are retrieved by using the VSM, we have taken some of the top ranked results as shown in the Figure 4.8.
In the Figure 4.8 it is clear that semantic query interpreter retrieve most of the relevant results, however the ranking is not appropriate some of the less relevant results come before the more relevant ones. Let‟s consider the scenario of the third and sixth one. The sixth one is more relevant to the query than the third one. The semantic intensity of the sixth image is greater than the third image as shown in the Figure. Hence the greater the SI value of the image relevant to the query, the higher is the rank.
Figure 4.9: Semantic Intensity of the images
The proposed model works on the principle of the semantic Intensity for the ranking of a result. Let the initial user query, which is the combination of different keywords be applied on the Semantic Query Interpreter for expanding the query lexically and conceptually.
⋃
(4.20)
Where Q is the query with set of terms„t‟.
)
)
)
⋃ )
(4.21)
W
hile is the expanded or enhanced query with their semantic similarity values.After the expansion of the user query from the SQI, the query is applied on the corpus
C. The system must return a subset of images from the corpus C, where C is set of images
with their annotation represented by the following equation. The images are represented by M.
Where
is the number of images in the corpus.
Where ⋃ , then equation (4.22) become
Where
⋃
⋃
)
(4.23)
The returned results are then passed to the SemRank module to rank the output on the basis of the Semantic Intensity rather than the frequency or visual similarity. The over-all algorithm of the SemRank is given below.
Proposed Algorithm 4.1: SemRank Input: 𝑄 → ⋃ 𝐾𝑡′ 𝑆𝑆)𝑖 𝑖 C→ ⋃𝑛𝑗 ⋃𝑚𝑥 𝑂𝑥)𝑗 Output: 𝑅 𝑆𝑒𝑚𝑅𝑎𝑛𝑘 𝑅𝑒𝑠𝑢𝑙𝑡 Method:
// applying enhance query on the corpus C
R ←𝑄 𝐶, where R 𝜖 𝑄 𝐶 and 𝐶 ≤ 𝐶
For each 𝐶 . Image in R . 𝐶
For each 𝐶 . Image . Object in R . 𝐶 . Image
//Calculate object dominancy OD for each object OD ← ∑𝑛 𝑖 𝑥𝑖𝑦𝑖 𝑥𝑖 𝑦𝑖)
//Calculate the Concept Dominancy of each concept tag with the object in the image CD ← 𝑂𝐷𝐼
𝑠 , Where 𝐼𝑠 𝐻 𝑊 of the image
// calculate the Semantic Intensity (SI) for concepts relevant to the query SI ← R . 𝑄 𝑆𝑆 𝐶𝐷
// Where R . 𝑄 𝑆𝑆 is the semantic similarity value of each term // Calculate netSI at R.𝐶 𝐼𝑚𝑎𝑔𝑒𝑙𝑒𝑣𝑒𝑙
R.SetSI ← ∑𝑛𝑖 𝑆𝐼)𝑖
// Where n is the number of concept tag with object per imageSort the result in descending order