HISTOGRAM TO HISTOGRAM MATCHING

I n t e r n a t i o n a l J o u r n a l o f E n g i n e e r i n g R e s e a r c h a n d S p o r t s S c i e n c e _Page1

Himadri Nath Moulick Aryabhatta Institute of Engineering, Durgapur Anindita Chatterjee Tata Consultancy Services, Kolkata, West Bengal Dr. Chandan Koner Dr. B. C. Roy Engineering College, Durgapur

ABSTRACT

Histogram Matching (HM) is a common technique for finding a monotonic map between two histograms. However, HM cannot deal with cases where a single mapping is sought between two sets of histograms. This paper presents a novel technique that finds such mapping in an optimal manner under various histograms distance measures. Also we present a method to perform histogram matching between two color images based on the concept of harmonic mapping between Riemannian manifolds. The key idea is to associate the histogram of a color image to a Riemannian manifold. In this context, the energy of the matching between the two images is measured by the Dirichlet energy of the mapping between Riemannian manifolds. Then, we assimilate optimal matching to critical points of the Dirichlet energy. Such points are called harmonic maps.

__________________________________________________________________________________________________________

I. INTRODUCTION

In many scientific disciplines there is a need to determining a monotonic mapping between two discrete sequences (i.e. a mapping that maintains the internal order of the sequence elements). A classic example is color calibration between two images. In this case, one wishes to remap the tones (intensities) of one image to another image, while maintaining the shadow regions darker than the highlights. When two images are captured from the same view-point but at different times or using different camera parameters, color calibration between the images can be found directly from their color pixels. In this case, one wishes to find a monotonic color map that optimally fits the pixel colors in the source image to their corresponding pixel colors in the target image. As pixel correspondences between the two images are used for calculating the color map, spatial information is exploited. We consider such solutions as spatial approaches. In more challenging cases, the images were acquired under different view-points, thus, pixel correspondence is not directly available and difficult to extract. In such cases, color mapping is commonly calculated from the images’ histograms where only statistical information is considered while spatial information is ignored. We consider solutions of this type as statistical approaches. Clearly, such a solution is inferior to the spatial approaches and may suffer from inaccuracies. In this paper we propose to exploit spatial as well as statistical information for color calibration. The main idea is to divide the images into a set of local regions and construct local histogram for each region. Such local histograms are insensitive to small geometric deformations. We construct a histogram correspondence such that for each local histogram in one image a corresponding histogram in the other image is determined. Histogram correspondence can be obtained either by calculating a rough geometric

transformation between the images or by corresponding image regions using local features such as SIFT or SURF. Given two sets of local histograms, one set for each image, along with a correspondence between the histograms, an optimal monotonic color mapping is calculated to simultaneously satisfy the entire set of histogram pairs. This approach enables, on one hand to handle inaccurate geometric transformation by considering statistical information (histograms), while on the other hand to exploit spatial information as the histograms describes local statistics. Histogram Matching (HM) is a common approach for finding a monotonic mapping between a pair of histograms. Given two histograms the algorithm finds a color mapping that optimally transforms one histogram towards the other. HM is a simple yet very effective algorithm, however it suffers from a number of shortcomings: First, it is limited to only two histograms and cannot deal with multiple histograms simultaneously. Second, HM approximates the optimal solution with respect to the L1 norm over the cumulative histogram pair but is unable to provide an optimal solution for other metrics. Finally, the HM solution is designed for continuous PDFs and may produce non optimal solutions in the discrete histogram case.

In this paper, we propose a new algorithm that generalizes HM in a number of ways. First, the algorithm can find a single monotonic mapping between multiple pairs of histograms such that the mapping will satisfy all pairs simultaneously. Second, the algorithm can work with any distance metric as long as it is additive (such as L1, L2, KL, χ2, histogram intersection, etc.). Finally, the algorithm can work with distance metrics defined over histograms as well as cumulative histograms providing the optimal solution. Since the algorithm generalizes the traditional Histogram Matching we term it Generalized Histogram Matching (GHM).

HISTOGRAM TO HISTOGRAM MATCHING

COMPUTER SCIENCE

I n t e r n a t i o n a l J o u r n a l o f E n g i n e e r i n g R e s e a r c h a n d S p o r t s S c i e n c e _Page2

II.THEGENERALIZEDHISTOGRAMMATCHING

Say one matrix M each column of the mapping matrix M must have exactly one indicator. Due to the monotonicity of M, each row must satisfy the following two conditions:

First, given an indicator existing at M (i, j), no indicator can appear neither at the lower-left sub-matrix of M (i, j) nor at its upper-right sub-matrix.

The second condition is that if there are several indicators in a certain row, they must be consecutive; otherwise the previous condition is violated.

The above two conditions of the mapping matrix M allows for an efficient calculation of the optimal path of indicators. The outline of the algorithm goes as follows:

During the process two auxiliary matrices, T and C are maintained. The matrices are of the same size as M. We use T as the trace-back matrix, helping to keep track of the optimal path of indicators, while C is used as the cost matrix such that each cell, C (i,j), represents the best cost achievable by a partial path of indicators starting at the first row and ending at the ith row given that the right-most indicator in this path is at column j. A row-by-row scan of matrix C is performed, starting at row i=1, and calculating the cost values for every entry C (i , j). In the first row, C(1,j) is evaluated as follows:

That is, Row Cost ( i,j….k) calculates the additional cost of mapping rows j…..k in A into row i in B. In matrix M this is represented as a sequence of consecutive indicators in M ( i , j ….k) For the following rows, each entry C( i,j),i=2…n,j=1….m, is calculated by seeking the optimal indicator path admissible by C( i,j). At this point we exploit the additivity of the distance measure and calculate the optimal path using the values in C( i-1,.). For each C( i, j) , the optimal connecting indicator path starting at row i-1 and ending at row i sought. This search is performed by scanning all values of C ( i-1,j') , 1<=j'<=j, and adding the cost for completing the path with consecutive indicators in row i:

C( i, j)= min { C(i-1,j') + Row Cost(i,(j'+1)…j)}

In order to keep track of the optimal path, the optimal index of j′ is stored in the trace-back matrix,at T (i,j). Since evaluating a row in T and C only requires data from the previous row, C and T can be evaluated using a linear top-down and left-to-right scan. Figure shows a situation where for C(i, j), the optimal

connecting indicator path was chosen to initiate from (i-1, j′). Thus, T(i, j) was associated with j′. As a result, if location (i, j) will be included in the final path, a sequence of j j′ indicators will be inserted in the i’ th row of M. The process terminates when all costs are filled in C. The optimal cost over all possible paths can be determined from C(m, n), and the optimal path of indicators is constructed by tracing back from T(m, n) up to the first row of T. The transformation matrix M is then populated by filling in indicators according to the constructed path. Since the distance is assumed to be additive, partially optimizing the distance function (over only some of the rows of M) still leads to a global minimum of the total cost at the end of the procedure, as performed in standard Dynamic Programming schemes. Therefore, this algorithm can work with any additive distance and outputs the optimal transformation matrix.

a) Information propagation in the C matrix. The cell (3,4) considers the 4 cells above it as optional sources.

(b) Computing the optimal indicator path for entry (3, 4)

a) Information propagation in the C matrix. The cell (3,4) considers the 4 cells above it as optional sources.

(b) Computing the optimal indicator path for entry (3, 4) given the choice of (2, 1) as the source of the path, requiring 4 - 1 = 3 consecutive indicators in row 3. (c) In a monotonic permutation matrix, every indicator has no indicators in its upper-right and lower-left sub matrices (shadowed). Additionally,

the indicators in every row are consecutive. (d) An example of mapping matrix M and its corresponding mapping matrix applying to CDF.

III.HISTOGRAMMATCHINGWITHTHE

CDF-MATCHINGALGORITHM

Given a candidate image B, whose histogram is to be matched to that of a model image A, the CDF-matching algorithm proceeds in four steps:

1. The normalized histograms H_A and H_B of images A and B, respectively, are computed by dividing the histogram values by the total number of pixels in the image.

I n t e r n a t i o n a l J o u r n a l o f E n g i n e e r i n g R e s e a r c h a n d S p o r t s S c i e n c e _Page3

Each CDF then operates as a LUT, where the indices of the LUT at each index is the value of the CDF.

3. For each pixel in image B of associated gray level i is determined via the LUT.

The corresponding gray level value j in image A such that (CDF_A)j equal (CDF_B)I must then be found.

4. The final step is to substitute gray level j for gray level i in image B.

The specific implementation of the CDF-matching algorithm will impact the computational speed. We shall estimate the minimum number of operations that are required to perform CDF matching under the assumption that all computations are executed on one processor. We shall also assume a bin size for histogram computation of one, where the gray levels of the image are binned into 256 different levels spanning 0-255 in increments of one. Any smaller binning of the gray level values would require a larger number of operations for CDF computations. We shall use K to denote the number of bins used in histogram computation and N to denote the number of pixels in the image. A time analysis of this algorithm breaks the steps into several parts: N operations per image to compute the histogram; K-1 operations to make the CDFs; on average K/2 operations to find the CDF_A value that is closed to that of the CDF_B value; and N operations to update the gray level values of the image. The K/2 matching operation is done for each of the N update operations. In summary, the total time analysis yields an O(N+K+KN) or O(KN) algorithm.

IV.HISTOGRAMMATCHINGWITH

SORT-MATCHINGALGORITHM

The sort-matching algorithm utilizes sorting to implement an exact histogram matching rather than creating histograms as was done in the CDF-matching algorithm. By "exact histogram matching" we mean that upon completion of the algorithm, the number of pixels of a particular gray level is the same for both images. The assignment is such that the lowest gray level pixel of the candidate image is assigned the value of the lowest gray-level pixel of the model image. The next-to-lowest gray gray-level pixel of the candidate image is assigned the value of the next-to-lowest gray level pixel of the model image. This process is repeated until all the pixel values have been assigned. Let us again presume that B represents a candidate image we wish to match to a model image A of the same size. Rather than the two usual indexing of the images, we will assume that the pixel values are arranged into one dimensional arrays. Bold characters are employed to denote vector quantities. This lexicographical indexing of the images, which is generally trivial to implement in most computing languages (e.g., C, Fortran), allows multidimensional arrays to be reduced to one dimension in procedures and functions. In higher-level languages such as IDL, re-dimensional arrays to be reduced to one –dimensional lexicographical index. The size of the linear arrays is the total number of pixels in each image. The algorithm is implemented in two simple steps:

1. The pixel values of the two images are sorted in

ascending order. The lexicographic indices corresponding to the sorted gray levels are denoted IND_A and IND_B for images A and B respectively.

The first element of the list IND_A contains the index of the lowest gray level value of A. The second element of IND_A contains the index of the next-to-lowest gray level value of A, etc. The same applies to IND_B. Any of the standard sort routines can be used for this step. We used the generally accepted fastest sort algorithm, Quick Sort. The average time complexity of Quick Sort is O ( N log2 N).

2. The candidate image B is then assigned the gray level values of the model image A according to the sorted gray level values. We can define this step by the following equation:

B (IND_B)=A(IND_A)

This sequence of sorting and substitution yields an image B whose histogram is matched with that of A. An illustration of the sort-matching algorithm can be found in Rolland et al.

Illustration of one scale level of the steerable pyramid transform used in the texture synthesis algorithm

V.RESULTSOFTWOALGORITHMS

Synthesis textures using either the OCM, the conventional CDF matching, or the sort-matching algorithms are shown in the figure for two different model texture images, respectively. Two realizations of image synthesis were generated for each texture model using two different realizations of the input noise image. Given one realization of a noise image, the synthesized images are perceptually identical using the three methods. Their histograms, however, are slightly different because only the sort-matching algorithm performs exact histogram matching as shown. The CDF-matching algorithm performs only a relatively close match. A comparison of performance based on the same noise realizations for each method allows assessment of potential artifacts (e.g. random scrambling of pixel values) that could have been introduced by an approach (e.g. the sort-matching algorithm).

I n t e r n a t i o n a l J o u r n a l o f E n g i n e e r i n g R e s e a r c h a n d S p o r t s S c i e n c e _Page4

Results of texture synthesis by two methods of histogram matching for two textures.

VI.HISTOGRAMFLOWFORHISTOGRAM

MATCHING

From color image to Riemannian manifold

Given an image RGB, the so called

tensor of structure of I is called the 2x2 matrix field.

The eigenvalues and eigenvectors fields of the upper matrix provide information about the local behaviour of the image. Indeed, the eigenvalues measure the highest and lowest directional variations of I and the corresponding eigenvectors give the directions of these optimal variations. The tensor of structure has been widely used for color image processing tasks.

It turns out that the tensor of structure of the upper matrix is quite close to a Riemannian metric. Indeed, if we denote by the graph of the image I, i.e. the map

then determines the embedding of the manifold ^5, and the Euclidean metric on induces a Riemannian metric g on of the form

in the frame ( The couple ( forms a compact Riemannian manifold of dimension 2.

In, an anisotropic diffusion based on the Laplace-Beltrami operator associated to the metric is performed for color image denoising/regularization purposes.

In the following paragraph, we extend this construction to manifolds of dimension 3 in order to encode the local behaviour of color images histograms into Riemannian metrics.

VII.EXTENSIONTOCOLORHISTOGRAM

Let I: R2 RGB be a color image. We denote by HI the function that maps any color (r; g; b) to the number of pixels of color (r; g; b) in I. Let be the graph of the function

: (r; g; b) (r; g; b; HI (r; g; b))

The map determines the embedding of the compact manifold RGB of dimension # into RGB x Then the Euclidean metric on RGB x induces a Riemannian metric h RGB of the form

in the frame ( . The couple ( RGB ,h forms a compact Riemannian manifold of dimension 3.

Remark1. Notice that such a construction can be done for images with any number of channels.

Assuming that the manifolds (M, g) and (N ,h) are respectively of the form ( RGB, h and ( RGB, h we deduce that the harmonic flow converges towards a harmonic map if the sectional curvature of the Riemannian manifold (RGB. h is non positive. Let us distinguish two cases:

1. The case

I n t e r n a t i o n a l J o u r n a l o f E n g i n e e r i n g R e s e a r c h a n d S p o r t s S c i e n c e _Page5

sectional curvature of (RGB, h is zero, and theorem 1 applies. In this case we obtain the heat flow of the Laplace-Beltrami operator since the terms in the tension field vanish.

2. The case

In this case, we do not know if (RGB. h has a non-positive sectional curvature or not. However, we do have a necessary condition for the sectional curvature to be non-positive. Indeed, it is mentioned in that Riemannian manifolds that are hyper surfaces of , n >=4, cannot have strictly negative sectional curvature. Hence, because the manifolds (RGB. h are hyper surfaces of , we deduce that a necessary condition is that the sectional curvature of (RGB. h

vanishes at least once. If not, then the theorem 1 does not apply.

CONCLUSION

We have presented an application to color transfer. The manifolds considered in this paper only encode histogram information of images, i.e. global information, meaning that color transfers modify pixels of same color in the same way, without taking into account their location or the values of the neighboring pixels. The examples presented have revealed the limits of such a global approach. However, the general framework introduced in this paper allows us to construct mixed and local approaches for color transfer by the construction of Riemannian manifolds that take into account local information of images. This is the subject of current research. We proposed a new algorithm, termed Generalized Histogram Matching (GHM), to find a monotonic mapping between two sets of images using their histograms. It extends Histogram Matching in three ways: (1) it can handle multiple histograms (2) it can work with any additive distance metric (3) it can work either with PDFs or CDFs.

REFERENCES

[1] Michael Herrmann, Muhammad Ahtisham Aslam, Oliver Dalferth, Applying Semantics (WSDL, WSDL-S, OWL) in Service Oriented Architectures (SOA), Universität Leipzig, Germany, Technical report, 2005.

[2] Microsoft Healthcare [online], http://www.microsoft.com/ industry/healthcare, (Jun, 2007).

[3] J. Treadwell, Open Grid Services Architecture Glossary of Terms, Hewlett-Packard, January 25, 2005.

[4] Organization for the Advancement of Structured Information Standards (OASIS), “Service Oriented Architecture (SOA) Reference Model,” Public Review Draft 1.0, February 10, 2006. [5] Workflow definition [online], http://en.wikipedia.org/wiki/

Workflow (June 2007).

[6] Latha Srinivasan and Jem Treadwell, An Overview of Service-oriented Architecture, Web Services and Grid Computing, , HP Software Global Business Unit, November 3, 2005.

[7] Armin Haller, Juan Miguel Gomez, Christoph Bussler, Exposing Semantic Web Service Principles in SOA to Solve EAI Scenarios, May, 2005.

[8] Rick Robinson, Understand Enterprise Service Bus scenarios and solutions in service-oriented architecture, IBM, Jun 15, 2004.

[9] Patterns: Implementing an SOA Using an Enterprise Service Bus (IBM Redbooks), IBM.Com/Redbooks, 2004.

[10]WS Specifications [online], http://www.w3schools.com/ webservices/default.asp, (Jun, 2007).

[11]Jeffrey Hasan, Expert Service-Oriented Architecture in C#: Using the Web Services Enhancements 2.0, Apress, 2004. [12]SOAP Specifications [online], http://www.w3schools.com/ soap,

(Jun, 2007).

[13]OSGi Service Platform, The Open Service Gateway Initiative, IOS Press, 2003.

[14]OSGi Tutorial A Step by Step Introduction to OSGi Programming Based on the Open Source Knopflerfish OSGi Framework , Sven Haiges, http://www.knopflerfish.org /tutorials, October 2004.

[15]S.C. Oprhanoudakis, E. Kaldoudi, and M. Tsiknakis,

“Technological Advances in Teleradiology”, Eur. J. Radiology, vol. 22, 205-217, 1996.

[16]S.C. Orphanoudakis, “Supercomputing in Medical Imaging”

IEEE Eng Med Biol, vol. 7, 16-20, 1988.