Image Compression Via Sparse Representation

2017 2nd _{International Conference on Computer Science and Technology (CST 2017)} ISBN: 978-1-60595-461-5

Image Compression Via Sparse Representation

Zhen-zhen SU

1,a

_{and Long YE}

2

Key Lab of Media Audio & Video of Ministry of Education, Communication University of China, Beijing 100024, China

a_{[email protected]}

Keywords: Compression, Sparse representations, K-SVD, Reconstructed image, OMP

Abstract. In recent years sparse representation has become a hot topic in the research of image representation model and there has been a growing interest in the study of image compression based on sparse representation. However, when coding different images, we have to train the dictionaries that correspond to these images. In the paper, we use three different kinds of image databases to train a group of bases which respectively reflects the characteristics of the three kinds of image, these bases are formed a big over-complete dictionary. Experimental results show that the proposed dictionary is superior to image compression based on the dictionaries trained by single image database.

Introduction

Usually the still images take a large amount of spatial redundancy, so we exploit their vast spatial redundancy to compress images. In the traditional image compression processing, orthogonal transformation method is used, such as Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT). However, image sparse representation theory that increase a lot of flexibility and applicability allow higher sparsely of signal representation than orthogonal transformation. Given over-complete dictionary D, sparse representation of signal y can be solved by the following optimization problem:

0

m in . .

x x s t y = D x (1)

where x is a sparse coefficient matrix. The problem (1) can be solved by lots of algorithms, including Basis Pursuit (BP) [1], Matching Pursuit (MP) [2] and Orthogonal Matching Pursuit (OMP) [3].

For a good sparse representation, the design of an appropriate over-complete dictionary is the key step. The majority of works on constructing over-complete dictionary D can be roughly parted into two main categories. One is called as the fixed dictionaries such as DCT and Wavelet–Packet dictionary. They are not data-driven and a linear combination of their atoms, which cannot provide best sparse description of the original signal. The other is termed as the learning dictionary such as K-SVD [4], ANOST [5] and ISI[6], which are adaptively constructed by machine learning techniques from a group of training image datum. Compared to the fixed dictionaries, the learning dictionary can obtain better effect on image compression. At addition, many methods of the design of the over-complete dictionary have been proposed [7][8][9] [10].

represented by the different types of sparse dictionaries, for example, when compressing a face image, the dictionary trained from the face image database is much fitter for coding the image [4]. In this paper, we design an over-complete dictionary that consists of the characteristics of different kinds of images so that the dictionary can be adapted to natural images that consist of face and background and is beneficial to improve the quality of compression.

The compression process that is used to validate our method consists of the following steps: firstly, we find three different kinds of image databases such as FERET_BMP database which consists lots of face images, Brodatz texture database and images-view database which consists lots of landscape images. Respectively, we divide the images in them into numerous 8x8 blocks to compose three new databases; secondly, given each database, we train a set of bases using the K-SVD algorithm and these bases are formed over-complete dictionary that has the characteristics of each database; finally, we make the spare coefficients to be quantization and coding.

This paper is organized as the following. Section II begins with more background on the parse representation and the K-SVD Algorithm used in section III. Section III presents a novel over-completes dictionary design method, and introduces the framework of image compression. Experimental results are demonstrated in section IV. Finally, conclusions are given in section V.

Previous Work

Background on Sparse Representation

In recent years, image sparse representation has become the mainstream of image representation model. The advantage of image sparse representation is that it can make the coefficients representing the image as little as possible, and it has a better stability on processing error [11]. For a given the dictionary N K

D∈R × that contains K atoms for

columns, the signal N

y∈R that can be expressed as a sparse linear combination of the atoms from D. That is either y=Dx or approximation, y≈Dx. The vector K

x∈R is the sparse representation coefficients of y.

The goals of the sparse representation algorithm are to find a vector x and minimize

the number of non-zero coefficients at the same time. To achieve the goals, we need solve the following optimization problem:

2

min x _F s t. . y−D x ≤ε (2)

where, ⋅_F (F {0,1,2}) is the F

l norms, and D is a N x K matrix which is assumed

rank(D)=N; x is a sparse coefficient matrix.

The K-SVD Algorithm

K-SVD is the algorithm which adaptively learns the dictionary that makes the input signal sparse representation. It contains two basic steps. The first one is finding the sparse matrix x for the fixed dictionary and the second step is the dictionary update.

K-SVD method is achieved through the K singular value decomposition (SVD), and K here refers to the number of dictionary atoms and its optimization target equation is

where, Y is the signal matrix, X is the sparse coefficients matrix, T0 is the upper limit

of the number of nonzero.

In the first stage, we assume that D is fixed. Penalty terms are expressed as

following: 2 2 2 1 N i i F i

Y DX y Dx

=

− =



− (4)

Then we divide (3) into N problems:

2

0

2 0

, ₁

min . . ,

N

i i i

D X i

y Dx s t i x T

=

 

− ∀ ≤

 





 (5)

These problems could be solved by the known algorithms such as OMP, BP and others.

The second stage is to update the dictionary D. D is updated according to the

column and in each step only one atom (column in D) dk will be updated. We assume that D and X are fixed. k

T

x is a row corresponding with dk, so we could rewrite (4) as

2 2

2 2

1

j j k k

j T j T k T k k T

F F

j _F j k _F

Y DX Y d x Y d x d x E d x

= ≠

− = −



= −



− = − (6)

where, k T

x is the kth row in X, Ek represents the error for N examples while the kth atom is removed.

When updatingdk, we don’t conduct the sparsity constraint, which leads to arise some errors, so we define the following collection:

( )

{

1 , k 0

}

k T

w = i ≤ ≤i N x i ≠ (7)

In order to use the collection including indexes of vectors

_{{ }}

yi which use the atom

k

d , we define Ωk as a matrix of size N×wk , which the

(

w i ik

( )

,

)

entries and the other points are zeros. k k , k k , k

R R k R R k R k k

X =X Ω Y = ΩY E = ΩE are all the results of shrink by discarding of the zero entries.

Then the problem posed in (6) can be transferred the following problem:

2

k k k

k k k T k _F R k T

EΩ −d x Ω = E −d x (8)

And, this problem can be solved directly by singular value decomposition (SVD). We decompose k

R E to:

k T

R

E = ΔU V (9)

The last step in the dictionary update stage is just replacing the updated dictionary columnsdk with the first column of U and the corresponding row of

k T

Compression Process

Designing Over-Complete Dictionaries

Our objective is to design an over-complete dictionary which is adapted to different images and consists of the characteristics of different images. We use three different image databases such as FERET_BMP database, Brodatz texture database and images-view database. Respectively, divide the images of them into numerous 8x8 blocks to compose three new databases (FERET, Brodatz, view), then

(1) In FERET, for all images , train a dictionary 64 1

K

D ∈R × based on K-SVD,

(2) Brodatz and view are the same steps as FERET, gaining D₂ (D₂∈R64×K) and

3

D (D₃∈R64×K);

(3) Finally, we can get an over-complete dictionary D=

[

D D D1, 2, 3

]

(

64 3K

D∈R × ).

The Compression Process

In order to get a good compression effect, the sparse representation of image data and the design of the overcomplete dictionary are bound to become important steps in the compression process.

[image:4.612.173.440.421.622.2]

In this paper, we trained the overcomplete dictionary from three different databases based on the K-SVD algorithm, and we adopt Orthogonal Matching Pursuit (OMP) to handle image sparse decomposition using the designed overcomplete dictionary; then sparse representation coefficients is 8 bit quantization and operated based on entropy coding where Huffman coding is used; decoding stage use the reverse process, finally, we complete the whole image compression process. The Compression process is presented in Fig.1.

Experimental Results

Experimental Environment

[image:5.612.116.494.172.255.2]

The experimental simulation in this paper is under the environment of MATLAB. The standard test image “Lena” is adopted in our experiment. The size is 512x512, and the images’ format is BMP.

Fig.2 presents six different dictionaries, and we choose K=128.

(1) (2) (3) (4) (5)

Figure 2. (1) The proposed dictionary. (2) The overcomplete DCT dictionary. (3) The dictionary trianed by k-svd using texture images (4) The dictionary trianed by k-svd using view images. (5) The dictionary trianed by k-svd using face images.

Experimental Results Shown

In this section, experiments demonstrate the performance of the proposed algorithm on PSNR. In Figure 3, Dictionary1 represent the result of the dictionary we propose, Dictionary2 represent the result of over-complete DCT dictionary, Dictionary3 represent the result of the dictionary trianed by K-SVD using Brodatz texture database, Dictionary4 represent the result of the dictionary trianed by K-SVD using images-view database, Dictionary5 represent the result of the dictionary trianed by K-SVD using FERET_BMP database. Figure 4 showed the comparison between different dictionaries when the bit rate is fixed.

0 0.5 1 1.5 2 2.5 3

25 30 35 40 45

Rate-BPP value

PS

NR

[image:5.612.203.400.445.606.2]

Dictionary1 Dictionary2 Dictionary3 Dictionary4 Dictionary5

[image:6.612.145.470.67.159.2]

Figure 4. Sample compression results.

Conclusion

Image compression based on a combined over-complete dictionary is an efficient method. In this paper, we design a novel over-complete dictionary which changes the method that a piece of graphic designs a corresponding dictionary, and it is a dictionary applies to different natural pictures at the same time. For “lena” image, face accounts for only a small part of the picture, so the effect of the dictionary trained by FERET_BMP database is not as good as the other two. And the experimental results indicate that the proposed method is superior to image compression based on the other trained dictionaries.

Acknowledgement

This work is supported by the National Natural Science Foundation of China Nos. (61201236, 61371191), and Nation Science and Technology Support Program No. (2012BAH39F01-05).

References

[1] S. Chen, D.L. Donoho, and M. Saunders, “Atomic decomposition by basis pursuit,” SIAM Journal on Scientific Computing, vol. 20, no. 1, pp. 33-61,1999.

[2] S. G. Mallat, and Z. F. Zhang, “Matching Pursuit with time-frequency dictionary,” IEEE Trans on Signal Processing, vol.41, no.12. pp.3397-3415,1993.

[3] Y. C. Pati, R. Rezaiifar, and P.S. Krishnaprasad, “Orthogonal Matching Pursuit recursive function approximation with applications to Gabor decomposition,” in Proceedings of the 27th annual asilomar conference on signal, 1993, pp.40-44

[4] Aharon, M., Elad, M., and Bruckstein, A. M. “The K-SVD: an algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Transon Signal processing, 2006, 54(11) :4311-4322.

[5] Z. Y Tang and S. X. Ding. “Sparse representation of image via overcomplete dictionary learned by adaptive non-orthogonal sparsifying transform,” in 2010 Third International Conference on Intelligent Networks and Intelligent Systems, 2010, pp.120-123.

[7] K. Engan, K. Skrettinga and J. Husoy, Family of iterative LS-based dictionary learning algorithms, ILS-DLA, for sparse signal representation, Digital Signal Processing, Vol. 17, pp. 32–49, 2007.

[8] Z. Xiea and J. Feng, KFCE: A dictionary generation algorithm for sparse representation, Signal Processing, Vol. 89, pp. 2072–2077, 2009.

[9] B. V. Gowreesunker, Learning Sparse Representation Using Iterative Subspace Identification, IEEE Transactions on Signal Processing, Vol. 58, pp. 3055–3065, 2010. [10] K. Skretting, Recursive Least Squares Dictionary Learning Algorithm, IEEE Trans. on Signal Processing, Vol. 58, pp. 2121–2130, 2010