Matrix value Linear Regression for Image Denoising

2017 International Conference on Mathematics, Modelling and Simulation Technologies and Applications (MMSTA 2017) ISBN: 978-1-60595-530-8

Matrix-value Linear Regression for Image Denoising

Weirong Zhou

_{, Yunjie Zhang}

_{and Yongchao Liu}

*Corresponding author

Keywords: Image denoising; Matrix-value regression; Global linear operator.

Abstract. Since images (or patches) can be inherently represented by matrices, in this paper, the image denosing task is addressed into learning global linear operator by using matrix-value linear regression strategy. From given training noisy images (or patches) and their corresponding clear ones, a series of linear mappings are potentially defined by those image pairs, and then integrated as a global linear operator on training set. Matrix-value linear regression can be employed to learn the global linear operator. The proposed algorithm can commendably overcome the disadvantages of the vector-based methods. Empirical experiments illustrate that the proposed algorithm is feasible and efficient, as well as easy to be implemented.

Introduction

Image denoising is an important image processing task whether as a process itself or a preprocessing in image process. In most cases, it is assumed that the noisy image is the summation of original image and a noise component. The goal of image denoising is to remove the noise component while retaining the important details as much as possible [1]. Over the past few decades, numerous image denoising algorithms have been proposed for the quality improvement of images corrupted with some kind of noise model, and in general can be categorized as spatial domain, transform domain, and dictionary learning based according to the image representation [2]. Spatial domain method take advantage of the correlations among distinct pixels or patches in the image to reduce noise level. Many spatial domain based algorithms have been developed for noise reduction, such as adaptive median filter, Wiener filter, steering kernel regression, nonlocal means filter, nonlocal median filter, etc. Transform domain methods are based on the assumption that the images (or image patches) can be well represented by the few transform basis with a series of coefficients. The basic idea is to remove or shrink smaller coefficients as far as possible because they are related to image details and noise. Within this category, a few typical algorithms are discrete cosine transform, wavelets transform, curvelets transform, contourlets transform, etc [1,2]. The dictionary learning based methods attempt to construct or learn an appropriate over-complete dictionary that can accurately fit the local structures of images, such that the estimated image can be expressed as a linear combination of only few atoms chosen out from this dictionary. Several representative algorithms in this category are the K-clustering with singular value decomposition (K-SVD), learned simultaneous sparse coding (LSSC), clustering-based sparse representation (CSR), and nonlocally centralized sparse representation (NCSR). The dictionary learning based methods have now become a trend for

image denoising [3].

the sample number. To avoid these defects, some works such as [4,5] have made some attempts at directly using the original image (or image patches) matrices for image denoising.

To overcome the ill posed problem of image denoising, image priors are used to regularize it in order to find an approximate solution effectively . Explicit image priors derived from the image itself, such as piecewise smoothness, patch self-similarity, structural similarity and sparse representations, have been widely used in previous works [3-5]. Although explicit image priors have been quite successful for denoising, little effort has been devoted to implicit image prior which is difficult to make clear.

Regression has been applied to image denoising in recent years. Typically, Takeda et al. [6] developed an adaptive kernel regression method, in which a Taylor series up to some order is regarded as a local representation of the regression function. Chaudhury et al. proposed a non-local patch regression method, in which patch similarity is regressed by using matrices rather than vectors. Lu et al. [5] presented a tree-based locally linear regression approach, in which denoising operators is learned by using the external patch database and the self-similar patches. In these methods regression is either implemented by vectorizing the image patches, or only by using explicit image priors.

According to the fact that images (or image patches) can be inherently represented by matrices, Tang et al. have recently developed the matrix-value linear regression strategy for single-image super-resolution [7-10]. Within their framework, implicit image priors can be described by a set of training image pairs, which are made up of low-resolution images and their corresponding

high-resolution images [11], and expressed by using matrices. Then the maps training low- to

high-resolution images (or image patches) are deemed to be the matrix-value operators based on the fundamental of multi-task learning, meanwhile regression method is employed to learn the matrix-value operator.

Following the ideas of Tang et al, we attempt to propose an image denoising algorithm, abbreviated MVLR, based on matrix-value linear regression strategy. The presented algorithm represents an image (or image patch) as a matrix, and learns a matrix-value operator from the given training set including noisy images (or image patches) and their corresponding clear ones. Such a matrix operator can implicitly express both the explicit and the implicit image priors existing in the training set. The rest of this paper is organized as follows. Main algorithm and theoretical analysis are reported in Section 2. Experimental results are shown in Section 3. Finally, conclusions are drawn in Section 4.

Main Algorithm

It is generally known that the degradation model for the denoising problem can be described as:

y = x + n (1)

where y is a observed degraded image obtained from the unknown clean image x, n is the random

noise that is independent with respect to x. Image denoising aims to reconstruct the original sharp

image x from the noisy observation y, and is thereby seen a inverse process of model (1). A solution

to this inverse problem is an approximationxˆ of the unknown clean image x from noisy observation y.

Algorithmically, most of the existing denoising methods solve model (1) with tricks of vectorization.

Differently, in our context variables x, y and n are represented in matrix form.

For an image pair (x, y) satisfying model (1), each entry (pixel) of approximationxˆ of the image x is

often considered as a value from the mapping defined on entry (pixel) set of y. Consequently,

according to image pair analysis in [8], the image pair (x, y) potentially defines an operator A which

satisfies

x = A(y) (2)

In particular, by restricting the operator A to be linear, equation (2) would become that

That means explicit and implicit image prior described by image pair x and y can be simply

formulized as y = Ax.

Denote the matrix space as Rdh_{, where both d and h are positive integers. Let X, Y}  _Rdh _be

separately the clean and noisy image (or image patch) spaces, where d and h represent the height and

width of an image (or image patch), respectively. Let Sn = {(xi, yi) | xiX, yiY, i = 1, 2, …, n} XY

be a training set with n pairs of training samples. Then, for each i{1, 2, …, n}, there is a matrix

AiRdd such that

xi = Ai yi (4) and this task can be addressed into the framework of supervised learning in terms of matrix

regression. On the other hand, if the noise type in model (1) is given, the internal relation between x

and y should be stable. It implies that, for any image pair (xi, yi) in training set Sn, corresponding

mechanism described by equation (4) is homologous. According to the theory of multi-task learning,

a set of related learning tasks xi = Ai yi (i = 1, 2, …, n), can be considered at the same time, and then

there exist a global linear operator A* from Y to X such that xi = A* yi, i = 1, 2, …, n. Therefore,

solving the inverse of model (1), i.e. getting an approximationxˆ of the clean image x from noisy

observation y, becomes finding a global linear operator A* from Y to X.

Let A = {A | A: Rdd_Rdd _{} be an operator space. Using least square regression technique, the}

global linear operator A* can be learned from the training set Sn by the following optimization

problem

2 F 1

arg min n || _{i i}||

A i

A x Ay

 





A

(5)

where ||||F is Frobenius norm. Denote the objective function in equation (6) as F(A), then

2 F T

T T T T T T

( ) || ||

tr(( ) ( ))

tr( )

i i

i i i i

i i i i i i i i

F A x Ay

x Ay x Ay

x x x Ay y A x y A Ay

 

  

   

Algorithm 1 Matrix-Value Linear Regression (MVLR) for Image Denoising

Required variables:

Training set Sn = {(xi, yi) | xiX, yiY, i = 1, 2, …, n}.

Noisy observation z.

Step 1: Using training sets to estimate the global linear operator









  _ n

i n i i i i

iy y y

x A

1 1

T

T_{( )}

Step 2: Segmenting the noisy observation z into a set of image

patches with the size of dd: zz = {z1, z2, ..., zl}.

Step 3: Denoising the s-th image patch zs for s = 1, 2, …, l





s A z x y y y z

x

1 1

T

T_{( )}

ˆ

Step 4: Merge allxˆs, s = 1, 2, …, l, to obtain an approximationxˆof the

clean image from noisy observation z.

Setting the derivative of F(A) to be zero, in terms of the first-order necessary conditions of the

T T T T T T

T T T T T T

T T

1 1

( ) 0 tr( ) 0

tr( ) tr( ) tr( ) tr( ) 0

i i i i i i i i

i i i i i i i i

n n

i i i i

i i

F A x x x Ay y A x y A Ay

A A

x x x Ay y A x y A Ay A

Ay y x y

 

 _{       }

 

 

  

 _    _









Consequently, the global linear operator A* from Y to X can be represented as

1 1

( )

n n

T T

i i i i

i i

A x y y y 

 







where x+ is the generalized inverse of the matrix x.

Based on the equation (6), a so-called MVLR algorithm for image denoising is summarized in Algorithm 1.

Experimental Results

To verify the feasibility and effectivity of the proposed MVLR algorithm, some experiments are conducted to evaluate the effect of image denoising on general images. In every test, the proposed algorithm is implemented by using MATLAB (version R2014a), and then the denoising image quality is measured by Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measurement (SSIM).

Datasets and Experimental Setup

All original images used in experiments are selected from a number of natural color images, and

converted to grayscale ones. Each image is resized to 78102 pixels, and divided into nine patches

with size of 2634. In order to compare the effect of different noise environments, those gray images

are superimposed to Gaussian white noise with six intensities (0.04, 0.08, 0.16, 0.32, 0.5, 0.64), salt & pepper noise with three intensities (0.04, 0.08, 0.16, 0.32, 0.5, 0.64), as well as a variety of mixed noise among them, so that the noisy images are obtained. Similarly, all noisy images are also

segmented into 26×34 image patches. In addition, three training sets are generated by dividing the

randomly selected image pairs (20, 32, 44), thus the three training sets contain 180, 288, 396 pairs of

image patch respectively, and some of the training samples are shown in Figure 1, the test set is

shown in Figure 2.

Figure 1. Training images.

Experimental Results and Analysis

As mentioned in the Introduction, this paper tries to use matrix-value linear regression strategy to construct an image denoising algorithm. Therefore, the experiments focus on the feasibility and effectivity of the proposed MVLR algorithm, and only two classical algorithms, Wiener and Adaptive Median filter, are used here for performance comparison.

Figure 3 shows the visual comparisons under a training set containing 44 samples, which illustrates denoising results of the first test image by the three algorithms. More quantitative results are reported in Tables 1 to 3, in which the PSNRs and SSIMs of different algorithms in various noise environments are summarized. It should be easy to see that the proposed MVLR algorithm is obviously both feasible and efficient, and more effective than two classical algorithms. This is also demonstrated by the visual realism shown in Figure 3, since the results generated by MVLR algorithm are cleaner than the other two. Furthermore, as the training set size gets larger, the PSNRs and SSIMs of MVLR ascends, leading to better performance.

Figure 3. Visual comparisons of denoising results on test image 1. The first row is the original color image and corresponding grayscale image. From row two to eight, the first column contains noisy images with Gaussian noise (0.16), Salt & pepper noise (0.16), mixture of two noises, respectively; column two to four shows denoising images by

three methods (Adapt median, Wiener, MVLR).

Table 1. PSNRs and SSIMs of test image 1 with Gaussian noise.

Noise Levels 0.04 0.08 0.16 0.32 0.5 0.64

Original PSNR 21.7273/0.5780 19.7262/0.5771 15.5894/0.5570 10.7441/0.4975 7.7797/0.4058 6.3813/0.3312

Adaptive Median 22.4149/0.6083 20.2423/0.6103 15.7679/0.5918 10.7671/0.5164 7.7856/0.4120 6.3827/0.3319

Wiener 24.5767/0.7375 21.3074/0.7289 16.0533/0.6808 10.8489/0.5651 7.8124/0.4275 6.3930/0.3322

MVLR # 1 25.1407/0.6932 24.5240/0.6866 22.8198/0.6643 19.8455/0.6267 16.5753/0.5301 14.7586/0.4467

MVLR # 2 25.3075/0.6936 24.5636/0.6835 22.9274/0.6615 20.0138/0.6247 16.7545/0.5296 14.8501/0.4449

MVLR # 3 25.4168/0.6926 24.5775/0.6835 23.2400/0.6649 20.4049/0.6245 17.0306/0.5269 15.0791/0.4464

Table 2. PSNRs and SSIMs of test image 1 with Salt & Pepper noise.

Noise Levels 0.04 0.08 0.16 0.32 0.5 0.64

Original PSNR 21.6113/0.5784 18.4255/0.4152 15.7646/0.2892 12.5864/0.1483 10.8381/0.1063 9.7317/0.0555

Adaptive Median 25.6598/0.7148 21.5737/0.5483 18.2749/0.3911 14.3417/0.1960 12.3733/0.1440 11.1675/0.0814

Wiener 24.6858/0.6708 22.5150/0.5698 20.1352/0.4532 16.7540/0.2946 14.7303/0.2274 13.2859/0.1428

MVLR # 1 24.7268/0.6789 22.6516/0.5875 20.6969/0.5137 17.4846/0.3547 15.5380/0.3023 14.0673/0.2154

MVLR # 2 24.8576/0.6777 22.7436/0.5849 20.7230/0.5075 17.5440/0.3529 15.6056/0.3024 14.1326/0.2182

Table 3. PSNRs and SSIMs of test image 1 with Mixed (Salt & pepper, Gaussian) noise.

Noise Levels 0.04 0.08 0.16 0.32 0.5 0.64

Original PSNR 18.9379/0.4390 16.7930/0.3675 13.5109/0.2554 10.1319/0.1554 8.1771/0.1034 7.2491/0.0749

Adaptive Median 20.9354/0.5308 18.3755/0.4542 14.4898/0.3299 10.5604/0.1964 8.3340/0.1256 7.3241/0.0950

Wiener 22.7399/0.6246 19.7581/0.5430 15.5440/0.4258 11.1489/0.2830 8.6921/0.2116 7.5318/0.1747

MVLR # 1 23.0669/0.6174 21.2211/0.5568 18.1721/0.4594 15.1375/0.3390 13.4512/0.2728 12.5077/0.2175

MVLR # 2 23.1385/0.6114 21.3000/0.5544 18.2915/0.4530 15.2528/0.3359 13.4967/0.2771 12.5577/0.2139

MVLR # 3 23.3747/0.6146 21.5739/0.5532 18.5926/0.4582 15.5023/0.3370 13.6354/0.2764 12.6759/0.2124

Note that the MVLR # 1, # 2 and # 3 in the tables above denote the results of the training set containing 20, 32 and 44 samples, respectively.

Conclusion and Future Works

In this paper, an image denoising algorithm using matrix-value linear regression strategy is presented. The proposed MVLR algorithm focuses on the inherent structural information existing in image, as well as the implicit image priors reflected by a pair of noisy and noisefree images. To avoid defects of the vector-based methods, matrixing of images (or image patches) is used as the foundation of the proposed algorithm. In order to reflect image priors causing by image pair satisfying the noise model, the linear mapping between the image and its noise observation is established, and further converted to learn a linear operator. Noticing that those parallel linear operators can be treated as a shared representation according to the theory of multi-task learning, the image denoising is transformed into a learning global linear operator from the training matrices, which leads to a matrix-value linear regression algorithm.

Obviously, the algorithm presented in this paper is elementary. As future works, the mapping between image pair should be considered as a bilinear one, which can be viewed as a combination of row denoising and column denoising. Furthermore, regularization of global linear operator should also be taken into account in the modeling. In view of Bayesian maximum a posteriori, mean-squared error (MSE) between noisy and noisefree images corresponds to the likelihood, while the regularization corresponds to the image priors.

Acknowledgement

This research was financially supported by the NSFC Grant (11571056).

References

[1] Tanmay Sinha Roy. Image Denoising Algorithm. International Research Journal of

Engineering and Technology, 2016, 3(4): 1809-1816.

[2] Ling Shao, Ruomei Yan, Xuelong Li and Yan Liu. From Heuristic Optimization to Dictionary

Learning: A Review and Comprehensive Comparison of Image Denoising Algorithms. IEEE

Transactions on Cybernetics, 2014, 44(7): 1001-1013.

[3] Diwei Li, Yunjie Zhang and Xin Liu. A Modified NCSR Algorithm for Image Denoising. In:

Hanning Yuanet et al. (eds) Geo-Spatial Knowledge and Intelligence. Communications in Computer

and Information Science, 2017, 698: 377-386.

[4] Xianhua Zeng, Wei Bian, Wei Liu, Jialie Shen, Dacheng Tao. Dictionary Pair Learning on

Grassmann Manifolds for Image Denoising. IEEE Transactions on Image Processing, 2015, 24(11):

4556-4569.

[5] Xin Lu, Zhe Lin and Hailin Jin. Tree-based Locally Linear Regression for Image Denoising.

[6] Hiroyuki Takeda, Sina Farsiu and Peyman Milanfar. Kernel Regression for Image Processing

and Reconstruction. IEEE Transactions on Image Processing, 2007, 16(2): 349-366.

[7] Yi Tang and Hong Chen. Matrix-Value Regression for Single-Image Super-Resolution. In:

Proceedings of the 2013 International Conference on Wavelet Analysis and Pattern Recognition

(ICWAPR), Tianjin, China, 2013, 215–220.

[8] Yi Tang and Yuan Yuan. Image Pair Analysis with Matrix-Value Operator. IEEE Transactions

on Cybernetics, 2015, 45(10): 2042-2050.

[9] Yi Tang, Hong Chen, Zhanwen Liu, Biqin Song and Qi Wang. Example-Based

Super-Resolution via Social Images. Neurocomputing, 2016, 172: 38-47.

[10] Yi Tang and Ling Shao. Pairwise Operator Learning for Patch-Based Single-Image

Super-Resolution. IEEE Transactions on Image Processing, 2017, 26(2): 994-1003.

[11] Yi Tang, Pingkun Yan, Yuan Yuan and Xuelong Li. Single-Image Super-Resolution via Local