Vol 12, No 1 (2014)

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 11

A Survey on Sparse Representation

based Image Restoration

Dr. S. Sakthivel

Professor, Department of Information Technology Sona College of Technology,

Salem, India.

M. Parameswari

PG Scholar, Department of Information Technology Sona College of Technology,

Salem, India.

ABSTRACT

In recent field of engineering, digital images gaining popularity due to increasing requirement in many fields like satellite imaging, medical imaging, astronomical imaging, poor-quality family portraits etc. Therefore, the quality of images matters in such fields. There are many ways by which the quality of images can be improved. Image restoration is one of the emerging methodologies among various existing techniques. Image restoration is a process that deals with methods used to recover an original scene from degraded observations. The primary goal of the image restoration is the original image is recovered from degraded or blurred image. The main aim of this survey is to represent different methodologies of restoration that provide state-of-the-art results. The motivation of the literature originates from filter concept, iterative methods and sparse representations. The restoration methods of filter concepts are evaluated with the help of performance metrics SNR (signal-to-noise-ratio). These ideas can be used as a good reference in the research field of image restoration.

Keywords: Image Denoising, Image Deblurring, Sparse Representation, Restoration.

1. INTRODUCTION

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 12

In addition to these blurring effects, noise also corrupts any recorded image. Image Restoration can be modeled by the system as shown in equation 1,

yHx v (1)

Where xЄRN

is the unknown high quality original image, HЄRM×N is the degradation matrix, vЄRN

is the additive noise and y is the observed measurement. When H is specified by Kernel, then image reconstruction is the problem of image blurring.

The solution for the de-blurring problem can be obtained by solving the optimization problem as shown by equation 2,



 



2

arg minx .

X  yHx J x (2)

In the past decades, different methods and filters have been used for the purpose of image restoration. These methods do not hold to be proven to restore the image in case of additive white noise and Gaussian noises. Sparse representations approximate an input vector by using a sparse linear combination of atoms from an over complete dictionary. Sparse based methods have been verified to perform well in terms of Mean Square Error (MSE) measure as well as peak signal-to-noise ratio (PSNR). Sparse based models are used in various image processing fields such as image de-noising, image de-blurring, super resolution, etc.

2. IMAGE DENOISING AND DEBLURRING TECHNIQUES

Reginald L. Lagendijk and Jan Biemond [9] describe about the basic methods and filters for the image restoration. Linear Spatially Invariant Restoration Method is basic restoration filters were used. The author described blurring function act as a convolution kernel or point spread function d(n1,n2) that does not vary spatially. It is also assumed that the statistical properties of mean and correlation function of the image and noise do not change spatially. Modelling assumption can be denoted by f(n1,n2) spatial discrete image that does not contain any blur or noise then the recorded image g(n1,n2) is shown in the equation 3 ,

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 13

Two blur models were used. They are linear motion blur and uniform Out-of-focus blur. In linear motion blur, the relative motion between recording device and the scene results in several forms of motion blur that are all distinguishable. In Uniform Out-of-focus blur when camera captures the 3D image onto 2D image some parts are out of focus. These out of focus can be calculated by spatial continuous point spread function.Yusuf Abu Sa'dah

et.al [14] discussed in image enhancement that Low pass filters blur the images which result in noise reduction, whereas high pass filters used to sharpen the images. Butterworth filter and Gaussian filter can be used to sharpen the images and also high pass filter reside in the shape of the curve. Therefore any one of the high pass filters can be used to sharpen the images in restoration algorithm.

Jan Biemond et al.,[1] discusses the iterative restoration algorithms for the elimination of linear blur from images that tainted by pointwise nonlinearities such as additive noise and film saturation. Regularization is projected for preventing the excessive noise magnification that is associated with ill-conditioned inverse problems such as deblurring problem. There are various basic iterative solutions such as inverse filter solution, least squares solutions, wiener solution, constrained least squares solution, kalman filter solution. Inverse filter is a linear filter whose point spread function is the inverse of blurring function. It requires only the blur point spread function. Least Square filters are used to overcome the noise sensitivity and Weiner filter is a linear partial inverse filter which minimizes the mean-squared error with the help of chosen point spread function. Power spectrum is a measure for the average signal power per spatial frequency carried by the image, that is estimated for the ideal image. Constrained least squares filter for overcoming some of the difficulties of inverse filter and of wiener filter and it also estimates power spectrum. Regularization methods associated with the names of Tikhonov and Miller. For both the non-iterative and iterative restorations based on Tikhonov-Miller regularization analysed using eigen vector expansions.

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 14

directional redundant transforms are introduced including curvelet, contourlet, vedgelet, bandlet and steerable wavelet. Matching pursuit and basic pursuit denoising give rise to ability to address image denoising problem as a direct sparse decomposition technique over redundant dictionaries. In sparseland model Bayesian reconstruction framework is employed for local treatment on local patches to global patches. This K-SVD cannot be directly deployed on larger blocks even if provides denoising results

Priyam Chatterjee and Peyman Milanfar [10] proposed K-LLD: a patch based locally adaptive denoising method based on clustering the given noisy image into region of similar geometric structure is proposed with the use of K-LLD. To perform clustering, employ the features of local weight function derived from steering regression. Dictionary employed to estimate the underlying pixel values using a kernel regression. With the use of stein unbiased risk estimator (SURE) local patch size for each size can be chosen. Kernel regression framework uses the methods such as bilateral filter, nonlocal means and optimal spatial adaptation. Denoising can be learned with a suitable basis function that describes geometric structure of image patches. Image denoising can be first performed by explicitly segmenting the image based on local image structure and through efficient data representation. Clustering based denoising (K-LLD) makes use of locally learned dictionary that involves three steps:

1. Clustering: Image is clustered using the features that capture the local structure of the image data.

2. Dictionary selection: We form an optimized dictionary that adapts to the geometric structure of the image patches in each cluster.

3. Coefficient calculation: Coefficients for the linear combination of dictionary atoms are estimated with respect to the steering kernel weights.

Fig 2.1:Block diagram of the iterative version of algorithm

:

Steering Weights

Coefficient Calculation Stage

Clustering Stage

Dictionary Selection Stage

Class 1

Class K

Noisy Img

Y

Denoised Img

Y

W

Φ

Z

Y

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 15

K. Dabov et al.,[6] proposed a novel image denoising strategy on an enhanced sparse representation in transform domain. Sparsity is achieved by grouping similar 2D image fragments into 3D data arrays called groups. Collaborative filtering procedure developed to deal with 3D groups. It involves three steps i.e 3D transformation of group, Shrinkage of transform spectrum, inverse 3 D transformation. Some of the methods used to denoising are transform domain denoising method, sliding window transform domain image denoising. To apply shrinkage in local transform domain sliding window transform domain is employed. Sharp adaptive transform can achieve a very sparse representation of true signal in adaptive neighborhoods. Collaborative filtering for image denoising algorithm involves 2 steps:

1. Block estimate: In step one Block wise estimate is done for grouping and thresholding which follows aggregation.

2. Final estimate: In step two Block wise estimates is done for grouping and filtering which also follows aggregation.

Fig 2.2: Flowchart of the proposed image denoising algorithm.

Noisy Image

Grouping by block Matching

Basic Estimate of true image

Grouping by block Matching

3D Transform Hard Thresholding Inverse 3D Transform Block Wise Estimate

Aggregation

3D Transform Wiener Filtering Inverse 3D Transform Block Wise Estimate

Aggregation Final Wiener _Estimate

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 16

The proposed approach can be adopted to various noise models such as additive colored noise, non-Gaussian noise etc by modifying the calculation of coefficients variances in the basic and wiener parts of the algorithm. This method can be modified for denoising 1-D signals and video for image restoration as well as for other problems that benefit from highly sparse signal representations.

Julien Mairal, Michael Elad, and Guillermo Sapiro [8] used prior knowledge K-SVD algorithm for grayscale image processing is extended for color Image Restoration. Techniques used in color image restoration are Markov Random Field (MRF), Principal Component Analysis (PCA). An iterative method that incorporates the K-SVD algorithm for handling non homogeneous noise and missing information is used. Extension of denoising algorithm can be used for the proper handling of nonhomogeneous noise results better in correlation between the RGB channels. To capture the correlation between the different color images K-SVD algorithm can be adopted. This algorithm uses orthogonal matching pursuit (OMP) or basis pursuit (BP) as part of its iterative procedure for learning the dictionary. At each iteration, the best atom is selected from the dictionary that maximizes its inner product with the residual (minimizing the error metric) and updating the residual by performing an orthogonal projection. In denoising of color image that is represented by column vector and white Gaussian noise is added to each channel. Color spaces are often used to handle the chroma and luma layers differently. The proposed method results better in the application of color image denoising, demosaicing, and inpainting.

Noise in image is unavoidable, to estimate a true signal in noise, the most frequently used methods are based on the least squares criteria. Proper norm for images is the Total Variation (TV) norm. Closed form linear solutions are easily computed, nonlinear is computationally complex. Constrained minimization algorithm as a time dependent nonlinear PDE, where constrains are determined by the noise statistics. TV (L1) philosophy used to design hybrid algorithms. Nonlinear partial differential equations based denoising algorithms [2]. Novel image enhancement technique called shock filter is used. This algorithm yields more details of the solution in our denoising procedure.

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 17

Four different iterative methods are used for the acceleration algorithms are,

Richardson Lucy (R-L): It is an iterative technique used for the restoration of astronomical imagery in the presence of Poisson noise. Maximum entropy (ME) deconvolution: is a means for deconvolving “truth” from an image and point-spread function (PSF).In a perfectly focused, noiseless image there is still a warping caused by a point-spread function (PSF). The PSF is a result of atmospheric effects, the instrument optics, and anything else that lies between the scene being captured and the CCD array.

Gerchberg-Saxton (G-S) magnitude: The Gerchberg Saxton (GS) algorithm is one popular method for attempting Fourier magnitude or phase retrieval. This algorithm can be painfully slow to converge and is a good candidate for applying acceleration. Phase retrieval algorithms: The new method is stable and an estimated acceleration factor has been derived and confirmed by experiment. The acceleration technique has been successfully applied to Richardson-Lucy, maximum entropy and Gerchberg- Saxton restoration algorithms and can be integrated with other iterative techniques. There is considerable scope for achieving higher levels of acceleration when more information is used in the acceleration process.

Sparse representation of image signals admits a sparse decomposition over a redundant dictionary for handling sources of data. The problems of learning dictionaries for color images and extend the K-SVD based grayscale image denoising algorithm was described by Elad and Aharon (2006). Marial et al., [7] forwards the work for handling non homogenous noise and missing information in application such as color image denoising, demosaicking and inpainting. Sparseland model suggests dictionaries exist for various classes of signals and that the sparsity of signal decomposition is a powerful model. The removal of additive white Gaussian noise with gray-scale images uses the K-SVD for learning the dictionary from the noisy image directly. The extension to color can be easily performed by simple concatenation of the RGB values to the single vector and training on those directly which gives better results than denoising each channel separately. The steps involved in K-SVD algorithm are: Sparse Coding Step, Dictionary update, Reconstruction.

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 18

Reconstruction: A simple averaging between the patches approximations and the noisy image.

To extend the K-SVD algorithm for denoising of color images is to denoise each single channel using a separate algorithm with possibly different dictionaries. To take the advantage of the learning capabilities of the K-SVD algorithm to capture the correlation between the different color channels. In denoising of RGB color images, represented by column vector with some white Gaussian noise has been added to each channel. In sparse coding stage, greedy algorithm selects the best atom at each iteration, from the dictionary and then updating the residual by orthogonal projection. Multiscale framework focuses on the use of different sizes of atoms simultaneously. A large patch of size n pixel is divided along the tree to sub-patches of fixed size. A dictionary is built for multiscale structure of all atoms.

It characterizes soft edge smoothness based on a novel softcut metric by generalizing the geocuts methods. Shengyang Dai and Yihong Gong [11] proposed Soft edge smoothness measure can approximate the average length of all level lines in an intensity image. From this the total length of all level lines can be minimized. It presents a novel combination of this soft edge smoothness prior and the alpha matting techniques for color image SR ( Super Resolution) by adaptively normalizing image edges according to their α- channel description. To minimize the reconstruction error, original high resolution (HR) image can be recovered for low resolution (LR) inputs. This is called as inverse process. Reconstruction error can be optimized by back-projection method in iterative way. Interpolation methods such as bilinear or bicubic interpolation tend to produce HR images. To measure and quantify edge smoothness Goecut methods is employed. Goecut method approximates the length of a hard edge with a cut metric on the image grid. In softcut method, a soft edge cut metric measures smoothness of soft edges. To handle various edges in color images alpha matting technique is proposed from computer graphics literature. To prevent cross edges interpolation edge directed interpolation infer sub pixel edges. To extend color image SR, an adaptive softcuts method is proposed based on a novel α- channel image description. It enables a unified treatment of edges with different contrasts on α channel. Promising results for a large variety of images are obtained by this algorithm.

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 19

by selecting one from a prespecified set of linear transforms or adapting the dictionary to a set of training signals. Aharon et al., [4] proposed an algorithm for adapting dictionaries to achieve sparse representations. K-SVD is an iterative algorithm that alternates between sparse coding based on current dictionary and a process of updating the dictionary atoms to better fit the data. The algorithm can also be accelerated by updating dictionaries combined with the update of sparse representations. In sparse signal representation, to overcome the representation problem Matching Pursuit (MP) and Orthogonal Matching Pursuit algorithm is adopted to select the dictionary atoms sequentially. In designing dictionary k- means clustering is done. In clustering, a set of descriptive vectors is learned and each section is represented by one of those vectors. Vector Quantization (VQ) coding method called gain-shape VQ where coding coefficient is allowed to vary. In k-means, at each iteration, two steps are involved.

1. Given {dk}, assign the training examples to their nearest neighbor.

2. Given the assignment, update {dk}.

At step one, it finds the coefficients given the dictionary that is called sparse coding. Then the dictionary is updated assuming known and fixed co-efficient. K-means is used to derive the K-SVD an effective sparse coding and Gauss Seidel like accelerated dictionary update method. This algorithm finds the best co-efficient matrix using pursuit method and the calculation of co-efficient supplies the solution. This algorithm provides better results in minimum number of iterations than other methods and used in various applications such as filling in pixel missing, compression etc.

To reconstruct the degraded image, the sparse coding coefficients should be as close as possible to those of those of the unknown original image with the given dictionary. If only the local sparsity of the image is considered, the sparse coding co-efficient are often not accurate. To make the sparse coding more accurate, both the local and nonlocal sparsity constraint is considered. In centralized sparse representation modelling [12], Sparse Coding Noise (SCN) υα = αy – αx is added to the original image. The sparse coding of x is

based on y is given by equation 4 and 5,





2

2 1

arg max

y y H



       (4)





2

2 1

arg max

x x



       (5)

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 20

dictionary. Iteratively PCA dictionary can be used to code the patches for each cluster and dictionaries are updated along with the regularization parameters. The centralized Sparse Representation model can be given by the equation 6,

2

2 1

arg max

y i i i p

y _{i i}

y H a

 

  _         _







Where γ is a constant and lp norm is used to measure the distance between αi

and βi.

Sparse representation model (SRM) based image deblurring approaches shown promising deblurring results. SRM don’t utilize the spatial correlations between the non-zero sparse coefficients, the SRM based deblurring methods fail to recover the sharp edges. Weisheng Dong, Guangming Shi, and Xin Li, [13] proposed structured sparse representation model is employed to exploit the local and nonlocal spatial correlation between sparse codes. Image deblurring algorithm uses patch based structured SRM. In regularization based deblurring approach, the construction of effective regularization is importance. Sparsity based regularization can be solved by iterative shrinking algorithm. For high dimensional data modeling the low rank approximation is used. Algorithms used in structured based SRM are Patch based low rank approximation structured sparse coding (LASSC), Principle Component Analysis (PCA) and iterative threshold algorithm. The intrinsic connection between the structured sparse coding and the low rank approximation has been exploited to develop an efficient singular value thresholding algorithm for structured sparse coding. In CSR model each patch is coded individually for the PCA dictionary. Instead of coding each patch individually, simultaneous sparse coding techniques code a set of patches simultaneously for the sparse code alignment. Since patches share similar edge structures, over complete dictionary is not needed, a compact dictionary PCA. In image blurring using the patch based Structured Sparse Coding model, structured sparsity over the grouped nonlocal similar patches can be enforced, patch clustering is updated for iterations. An effective deblurring methods using the patch based LASSC image deblurring produces the state-of-the-art image deblurring results.

3. FILTERS AND RESULT

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 21

An adaptive filter does a better job of de-noising images compared to the averaging filter. The fundamental difference between the mean filter and the adaptive filter lies in the fact that the weight matrix varies at iterations in the adaptive filter while it remains constant throughout the iterations in the mean filter. Adaptive filters are capable of de-noising non-stationary images, that is, images that have abrupt changes in intensity. Such filters are known for their ability in automatically tracking an unknown circumstance or when a signal is variable with little a priori knowledge about the signal to be processed. In general, an adaptive filter iteratively adjusts its parameters during scanning the image to match the image generating mechanism. This mechanism is more significant in practical images, which tend to be non-stationary.

Compared to other adaptive filters, the Least Mean Square (LMS) adaptive filter is known for its simplicity in computation and implementation. The basic model is a linear combination of a stationary low-pass image and a non-stationary high-pass component through a weighting function. Thus, the function provides a compromise between resolution of genuine features and suppression of noise. A median filter belongs to the class of nonlinear filters that follows the moving window principle as same as mean filter. The median of the pixel values in the window is computed, and the center pixel of the window is replaced with the computed median. Median filtering is done by, first sorting all the pixel values from the surrounding neighborhood into numerical order and then replacing the pixel being considered with the middle pixel value. The median value must be written to a separate array or buffer so that the results are not corrupted as the process is performed.

The selection of the denoising technique is presentation dependent. So, it is necessary to learn and compare de-noising techniques to select the technique that is appropriate for the application in which we are interested. A technique to calculate the signal to noise ratio in images has been proposed which can be used with some approximation. This method adopts that the discontinuities in an image are only due to noise. For this reason, experiments are done on an image with very little deviation in intensity. The following Table 1 shows the Signal-to-noise Ratio (SNR) values of the input and output images for the filtering approach.

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 22

(a) (b)

(c) (d)

Fig 3.1: De-noising performance Comparison for the photograph image with standard deviation of σ=0.05 when Gaussian Noise is added. (a) Original Image with Noise, (b) Result image using Mean Filter approach, (c) Result image using LMS Adaptive approach, (d) Result image using Median Filter.

Table 1: SNR Results with Gaussian Noise and Standard Deviation σ= 0.05

From the Table 1 it is shown that the median filter provides increased SNR value of 22.79 than mean and adaptive filters. Median filter can be applied for higher denoising performance in case of restoring the degraded original image.

4. CONCLUSION

Image denoising and deblurring had been a major problem in the image restoration methodologies. Different types of algorithms are studied for the deblurring, denoising of degraded images and different type of filters are also analyzed. Sparse representations have been found to provide the better

Method SNR value of input image

SNR value of an output image

Mean Filter 13.39 21.24 LMS Adaptive

Filter 13.39 22..40

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 23

results of image restoration than other representations. Therefore based on sparse representation, local and non-local methods can be used to restore the degraded version of images effectively. Experimental result on filters shows that median filter performs better than other types. By consolidating the review and filter concepts, median and Gaussian filters can be applied for sparse based representation of image denoising.

REFERENCES

[1] Jan Biemond, Reginald L. Lagendijk, and Russell M. Mersereau, “Iterative methods

for image deblurring”, Proceedings of the IEEE, Vol. 78, No. 5, pp. 856–883, May 1990.

[2] Leonid I. Rudin, Stanley Osher, and Emad Fatemi, “Nonlinear total variation based

noise removal algorithms”, Phys. D, Vol. 60, pp. 259–268, November 1992.

[3] David S. C. Biggs and Mark Andrews, “Acceleration of Iterative Image Restoration

Algorithms”, APPLIED OPTICS, Vol. 36, No. 8, pp. 1766-1775, 10 March 1997.

[4] Michal Aharon, Michael Elad, and Alfred Bruckstein, “K-SVD: An Algorithm for

Designing Over complete Dictionaries for Sparse Representation”, IEEE Trans. on

Signal Processing, Vol. 54, No. 11, November 2006.

[5] Michael Elad and Michal Aharon, “Image denoising via sparse and redundant

representations over learned dictionaries”, IEEE Trans. Image Process., Vol. 15, No. 12, pp. 3736–3745, Dec. 2006.

[6] Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian, “Image

denoising by sparse 3-d transform-domain collaborative filtering”, IEEE Trans. Image

Process., Vol. 16, No. 8, pp.2080–2095, Aug. 2007.

[7] Julien Mairal, Michael Elad, and Guillermo Sapiro, “Sparse learned representations for

image restoration”, In Proc. of the 4th World Conf. of the Int. Assoc. for Statistical Computing (IASC), Yokohama, Japan 2008.

[8] Julien Mairal, Michael Elad, and Guillermo Sapiro, “Sparse representation for color

image restoration”, IEEE Trans. on Image Processing, Vol. 17, No. 1, pp. 53–69, Jan.

2008.

[9] Reginald L. Lagendijk and Jan Biemond, “Basic Methods for Image Restoration and

Identification”, In: A. C. Bovik, the Essential Guide to Image Processing, Academic Press, United States of America, pp. 326 – 330, 2009.

[10] Priyam Chatterjee and Peyman Milanfar, “Clustering-based denoising with locally

learned dictionaries”, IEEE Trans.Image Processing, Vol. 18, No. 7, pp. 1438–1451, July 2009.

[11] Shengyang Dai, Mei Han, Wei Xu, Ying Wu, Yihong Gong, and Aggelos K.

Katsaggelos, “Softcuts: a soft edge smoothness prior for color image super-resolution”,

IEEE Trans. Image Process., Vol. 18, No. 5, pp. 969– 981, May 2009.

[12] Weisheng Dong, Lei Zhang and Guangming Shi, “Centralized Sparse Representation

for Image Restoration”, in Proc. IEEE Int. Conf. on Computer Vision (ICCV), 2011.

[13] Weisheng Dong, Guangming Shi, and Xin Li, “Image Deblurring with Low-rank

IJCSBI.ORG

ISSN: 1694-2108 | Vol. 12, No. 1. APRIL 2014 24

[14] Sa'dah Y, Nijad Al-Najdawi and Tedmori S, “Exploiting Hybrid Methods for

Enhancing Digital X-Ray images”, The International Arab Journal of Information

Technology, Vol. 10, No. 1, January 2013.

This paper may be cited as: