Content-Based Result Generation

Unlike with concept-based result generation, it is not possible for content-based retrieval methods to divide the images in the database into relevant and irrelevant sets as response to a specific query. Instead, the prime aim of content-based result generation is to rank the images in a database in order of the similarity to the query image(s). The similarity of two images is thereby calculated by comparing the features extracted from the query image (e.g.a feature vector) with the features of the images in the database (i.e. in the feature index).

A number of similarity functions based on empirical estimates of the distribution of features exists to quantify such comparison. These so-calleddistance orsimilarity measures strongly depend on the feature space; the choice for a particular measure will most certainly affect the retrieval performance significantly.

This section will cover some similarity measures commonly used in CBIR. Let

D(I, J) denote the distance measure between two imagesI andJ,fi(I) the number

of pixels in bin iof the image I, andB the total number of bins in the histogram.

Histogram Intersection

The histogram intersection was one of the first measures that was introduced to quantify the difference between histograms in the context of VIR. It was first de- scribed by [436] and is defined as follows:

D(I, J) = B X i=1 min(fi(I), fi(J)) B X i=1 min(fi(J)) (2.6)

Although histograms (and histogram intersection) are mostly used to compare colour features, they could theoretically also be used for texture or shape prop- erties as long as each dimension of the image feature vector is independent and of equal significance. Further, research has shown that histogram intersection is fairly insensitive to changes in image resolution, histogram size, viewing point, depth or inclusion [249, 405].

Histogram intersection has been used in many early CBIRS, including [187, 326, 428], and is often blurred using a low-pass filter because it is very sensitive to small shifts [391], or a histogram cross correlation is performed,e.g. [129].

Minkowski-Form Distance

The Minkowski-Form distance (MFD) is the most widely used metric for image retrieval and, like the simple histogram intersection, treats all the bins of the his- togram independently. This distance is defined as:

D(I, J) = X

i

|fi(I)−fi(J)|p

!1_p

(2.7) Whenp= 1, thenD(I, J) corresponds to theManhattan (L1) distance, whenp= 2,

then D(I, J) complies with the Euclidean (L2) distance, and when p = ∞, then

D(I, J) is L∞ respectively.

Examples of the use of L1 include [318]; L2 was used in [369] to compute the

similarity between texture features, in Blobworld [43] for texture and shape com- parison, and was further employed in other systems such as [187, 318]; andL∞ was

used in [476] to compute the similarity between texture features. Sometimes, dif- ferent forms of the Minkowski distance are used for different features. For example, Netra [256, 257] makes use ofL2 for colour and shape and L1 for texture.

Quadratic Form Distance

The independent treatment of histogram bins, as employed by simple histogram intersections or the Minkowski form distances, ignores the fact that certain pairs of bins can correspond to features which are perceptually more similar than other pairs, which might lead to many false negatives [433]. One answer to this problem is thequadratic form distance (QFD), which is defined as follows:

D(I, J) = p(FI−FJ)TA(FI −FJ) (2.8)

whereFIandFJ are vectors that list all entries infi(I) andfj(J),A= [aij] denotes

a similarity matrix,aij the similarity between i and j, and T the transpose of the

Since the QFD considers the cross similarity of histogram bins (e.g.colours), it can yield perceptually better results than the distance measures named above [249]. As a consequence, many retrieval systems such as [158, 318] have used the QFD as a similarity measure, especially for colour features.

Mahalanobis Distance

When all dimensions in an image feature vector are dependent on each other and exhibit different levels of importance, then theMahalanobis distanceis a good choice as a similarity measure. For example, this measure seems appropriate to quantify the similarity of features describing salient points [40]. It is defined as

D(I, J) =p(FI−FJ)TC−1(FI−FJ) (2.9)

whereC is the covariance matrix of the feature vectors.

Hausdorff Distance

One of the most studied similarity measures for shape features is the Hausdorff distance [180, 449, 459], which is defined as

H(I, J) = max n

~h(I, J), ~h(J, I) o

(2.10) where ~h(I, J) denotes the directed Hausdorff distance from I to J, which is the lowest upper bound (supremum) over all points inI of the distances to J,

~h(I, J) = sup

i∈I

inf

j∈JD(i, j), (2.11)

with D(i, j) the underlying distance such as the Euclidean distance L2. Unfor-

tunately, the Hausdorff distance is very sensitive to noise as a single outlier can influence the distance value. A review of other shape similarity measures can be found in [459].

Kullback-Leibler (KL) Divergence

The Kullback-Leiber divergence (KLD) is a measure mainly used for texture simi- larity with the following definition:

D(I, J) =X i fi(I) log fi(I) fi(J) (2.12)

It describes the level of compactness in which one feature distribution can be coded if the other distribution is used as a “codebook” [249].

Other Similarity Measures

Other similarity measures include, for example, theJeffrey Divergence or theBhat- tacharyya, Maximum Likelihood and Fr´echet distances. An interesting alternative was examined by utilising text retrieval features such as term frequency and collec- tion frequency also in image retrieval systems such as [428, 483]. Excellent overviews of similarity measures in general and for shape matching in particular can be found in [405] and in [459] respectively.