Blind Single-Image Super Resolution
Reconstruction with Gaussian Blur and Pepper &
Salt Noise
Fengqing Qin
Institute of Computer Science and Technology in Yibin University, Yibin, China, 644000 Email: qinfengqing@163.com
Lihong Zhu, Lilan Cao and Wanan Yang
Institute of Computer Science and Technology in Yibin University, Yibin, China, 644000 Email: {yibinwuyu @qq.com, 66243190@qq.com , ywaly@126.com}
Abstract—To improve the spatial resolution of low resolution image with Gaussian blur and Pepper & salt noise, a blind single-image super resolution reconstruction method is proposed. In the low resolution imaging model, the Gaussian blur, down-sampling, as well as Pepper & Salt noise are all considered. Firstly, the Pepper & Salt noise in the low resolution image is reduced through median filtering method. Then, the Gaussian blur of the de-noised image is estimated through error-parameter analysis method. Finally, super resolution reconstruction is carried out through iterative back projection algorithm. Experimental results show that the Gaussian blur is estimated with high accuracy, and the Pepper & Salt noise are removed effectively. The visual effect and peak signal to noise ratio (PSNR) of the super resolution reconstructed image is improved. In addition, the importance of Gaussian blur in single-image super resolution reconstruction is justified in an experimental way.
Index Terms—Blind; Single-image; Super resolution; Gaussian blur; Pepper & salt noise; Iterative back projection
I. INTRODUCTION
In many electronic imaging applications, high resolution (HR) images are often desired and required. HR means that the pixel density within and image is high, and can offer more details. To improve the spatial resolution of image, the most direct way to improve the precision and stability of the imaging system with expensive cost and some technical difficulties. Super resolution (SR) method is an efficient way with lower cost than hardware method.
In general, SR includes video SR [1] and image SR. Video SR refers to reconstructing a higher resolution video form a low resolution (LR) video by utilizing the redundancy information between the adjacent frames and the prior information of the imaging system. Image SR refers to reconstructing a higher resolution from one image or a set of images acquired from the same scene. On the basis of the number of the LR images, image SR image SR mainly includes multi-image SR [2-4] and single-image SR [5-7]. Multi-image SR is commonly researched, in which the movement with sub-pixel precision is estimated and utilized to reconstruct a HR image. Image registration is very important in
multi-image SR. If the movement between the LR images is estimated with low accuracy, the misregistration phenomenon will be very obvious, and the SR reconstruction quality will degrade greatly. But, in some cases, multiple image of the same scene can not be acquired. HR image should be reconstructed from a single LR image. In resent years, approaches have changed the classical SR paradigm with multiples images, evolving towards the use of information from a single low resolution image [8].
In many practical applications, the image restoration problem is always blind, which means that the PSF is most likely unknown or is known only to within a set of parameters [9]. In the IBP reconstruction algorithm, the more accurate the estimation of the imaging model is, the better the quality of the reconstructed image will be. However, in most of the current algorithms, the blur is assumed to be a known Gaussian point spread function (PSF) with given parameters, and the blur is not considered at all in some algorithms, which does not meet the real imaging model of optical devices. Thus, the blind image SR reconstruction [10-12] problem arises naturally and is expressed as estimating a HR image and the PSF simultaneously, which is one advanced issue and challenge in image restoration. The foremost difficulty of blind de-blurring is rooted in the fact that the observed image is an incomplete convolution. The convolution relationship around the boundary is destroyed by the cut-off frequency, which makes it much more difficult to identify the blurring function.
In addition, the process of noise is seldom considered, which restrained the quality of the SR reconstructed image [13]. The noise can worsen the quality of images and bring some difficulty to image analysis. Thus, noise should be considered in the framework of single-image SR reconstruction.
In this paper, a framework of blind single image SR reconstruction method with Gaussian blur and Pepper and Salt noise is proposed. In the LR imaging model, the processes of Gaussian blur, down-sampling, as well as noise are all considered. The Pepper and Salt noise is reduced through median filtering method. The Gaussian blur of the de-noised LR image is estimated through error parameter analysis method. The SR image is reconstructed through iterative back projection (IBP) algorithm.
II.FRAMEWORK OF BLIND SINGLE-IMAGE SR RECONSTRUCTION WITH GAUSSIAN BLUR AND PEPPER &
SALT NOISE
The framework of single-image SR reconstruction with Gaussian blur and Pepper & Salt noise is shown in Fig.1. Firstly, the Pepper and Salt noise in the low resolution image is reduced through median filtering method. Then, the Gaussian blur of the de-noised image is estimated through error-parameter analysis method. Finally, super resolution reconstruction is carried out through iterative back projection algorithm.
A. The LR Imaging Model
In the LR imaging model, the Gaussian blur, down-sample, as well as Pepper & salt noise are considered, as shown in Fig.1. The mathematical description of LR imaging model of single-image SR reconstruction may be expressed as follows:
Y=BDF+N (1) Where, Y represents the LR image; F is the HR image; B is the blur function; D is the down-sample process; N is the noise.
The real scene may be expressed by a high resolution (HR) image. Firstly, the HR image is blurred by convolving with a point spread function (PSF). The blur mainly includes the Gaussian blur induced by the optical devices of the imaging system, the motion blur caused by the movement of the scene or the camera, as well as the defocus blur bringing by the false focus while imaging, etc. As Gaussian blur is the most common and is considered here. Secondly, the blurred image is down-sampled by a given integer factor. Here, the down-sampled image is gained by taking the neighborhood average gray value of the blurred image. Thirdly, the down-sampled is noised to generate the LR image. Here, the Pepper & Salt noise is considered. B. Iterative Back Projection Method
Among the current SR reconstruction methods, the iterative back projection (IBP) method has the virtues of small computational amount, fast convergent rate, good reconstruction effect, and so on. In addition, the estimated information about the LR imaging model can
be well utilized in the IBP algorithm. If the LR imaging model is estimated more accurately, the SR reconstructed image will achieve better quality.
In IBP algorithm, by back projecting the estimation error between the estimated LR image and the original image onto the HR image grid, the estimation error is gained to modify to estimated HR image. Repeating the above process until the iteration time is greater than a given number or the estimation error is less than a threshold, the SR image will be gained.
According to this idea, the IBP algorithm may be expressed as follows:
1
ˆ
ˆ
BP(
ˆ
)
k k k
f
+= −
f
λ
H
y
−
y
(2)Here, the initial value of the estimated HR image is taken as the interpolated image of the LR image by Bilinear interpolation algorithm. According to the LR imaging model proposed in this paper and the idea of IBP algorithm, the framework of the single-image SR reconstruction method with noise is shown in Fig.1. Here, k is the iteration time;
f
ˆ
is the estimated SR image;y
is the observed LR image;y
ˆ
is the simulated LR images off
ˆ
passed through the LR imaging model; B and D are the matrix forms of the motion blur and down-sampling respectively;n
is the system noise;B
−1,1
D− and n−1 denote the inverse operation of B ，D and n;
H
BP is the back projection operation;y
ˆ
−
y
is the difference of simulated LR image and the de-noised LR image;λ
is the gradient step.C. Gaussian Blur Estimation
Gaussian PSF is the most common blurring function of many optical measurements and imaging systems. Generally, the Gaussian blurring function may be expressed as follows:
others
2 2
2
1 exp{ 1 ( )( , )
( , ) 2 2
0
m n m n C
h m n πσ σ
⎧ − + ∈
⎪ =⎨ ⎪⎩
(3)
Where, σ is the standard deviation; C is a supporting ˆk
y
k
Z
ˆk k
f −λZ
n
BP
H
ˆk y −y
1 ˆk f+ 1 n− 0 y 1 D− 1 B−
region. Commonly, C is denoted by a matrix with size of K×K, and K is often an odd number.
Thus, two parameters that are the size (K) and the standard deviation (σ) need to be identified for the Gaussian blurring function. Because the Fourier transformation of a Gaussian function is still a Gaussian function, it is impossible to identify the parameters by the zero-crossing point in the frequency domain. But in many cases, the isolated point and the intensity edges in the observed image may provide the necessary information to identify the blurring function.
The parameters of the Gaussian PSF may be estimated according to the error-parameter curves based on Wiener filtering method. The Wiener filtering is usually approximated by the following formula:
* 2 H Y X H
γ
=+ (4)
Where, X, Y and H are the Fourier transformations of the real image (x), the blurred image (y) and the blurring function (h) respectively; * denotes the conjugate operation; γ is a positive constant, and the best value of
γ
is takes as thereciprocal of the SNR.In the Wiener filtering algorithm, in order to restrain the parasitic ripple induced by the boundary cutoff, the image needs to have circular boundary. For the observed image (y) with size of M×N, reflection symmetric extension is performed on it, and the size of the extended image becomes 2M×2N. Then, calculate the Fourier transformation of the extended image (Y). Given a size (K) of the PSF, the error-parameter curve is generated at different standard deviations ( σ ). According to error-parameter the curves at different sizes, these two parameters can be estimated approximately. The identification process is expressed as follows:
•Step 1 Select a range of the standard deviation given
by the minimum value (
σ
min) and the maximum value (σ
max);•Step 2 Set a searching number (S), and we will get:
max min
( ) /S
α
σ
σ
Δ = − ;
•Step 3 Set different sizes (K) of the PSF, repeating Step 4
•Step 4 For i=1:S, repeat Step 4.1~ Step 4.5
Step 4.1 Compute the current standard deviation:
min
( 1)
i
σ σ
=
+ − Δ
α
;Step 4.2 Generate the Gaussian blurring function h according to K and σ;
Step 4.3 Add zeroes to h and make it to be the size of 2M×2N, then compute its Fourier transformation (H);
Step 4.4 According to equation (4), estimate the Fourier transformation of the real image (X);
Step 4.5 Compute the estimation error:
2
E= Y−XH ;
•Step 5 Plot the error-parameter (
E
−
σ
) curves atdifferent sizes.
According to the
E
−
δ
curves, the approximate sizeand standard deviation of the blurring function can be estimated. The size where the distance between the curves decreases greatly is assumed to be the estimated size, and the standard deviation where the corresponding curve increases obviously is assumed to be the estimated standard deviation.
In order to estimate the parameters of Gaussian PSF automatically, two thresholds T1 and T2 are set. Firstly,
given an estimation error e, the curve where once the distance between curves is smaller than T1 gives out the
estimated size (
K
ˆ
) of the Gaussian PSF. The distance isdefined as the absolute difference of the cycle number (j) of standard deviation at e. Then, by calculating the slop of the estimation error at different standard deviations on the estimated curve, the deviation value can be estimated. The deviation once the slop is greater than the threshold T2 is the estimated deviation (
σ
ˆ
).D. Median Filter Algorithm
The median filter algorithm is widely used due to good smoothing performance for noise with long-tailed probability distribution and some image detail preserving capability. It is used to remove Pepper and Salt noise here.
In median filter algorithm, the isolated noise points are eliminated by making the gay values be close to their real values. The gray value at a given point is replaced by the median gray value in the sliding window. In the sliding window, the gray values are sorted in ascending or descending way. The median filter may be expressed as follows:
0
( , )
{ (
,
),( ,
)}
y i j
=
med y i
−
m j
−
n
m n W
∈
(5)Where, y0 (i,j) is the noised image; y(i,j) is the de-noised image; W is a two dimensional sliding window. The sliding window is often taken as a L ×L rectangle, where L is often an odd number.
III.EXPERIMENTS
Experiments are performed on simulated LR image to test the algorithm objectively and subjectively. The HR image ‘lena.bmp’ with size 256×256 as shown in Fig.2 is passed through the LR imaging model as shown in Fig.1. Firstly, the HR image is convolved by a Gaussian PSF with size of 7 and standard deviation of 1 respectively. Secondly, the blurred image is down-sampled by a factor of 2 in horizontal and vertical direction. Finally, Pepper and Salt noise with a density of 0.05 is added. The generated LR image with size of 128×128 is shown in Fig.3.
Secondly, utilizing the error-parameter analysis method, the parameters of the Gaussian PSF is estimated approximately. The sizes (K) of the Gaussian PSF are taken as 3, 5, 7, 9 and 11 respectively. The range of the standard deviation (σ ) is taken as [0.5, 2]. The searching time is taken as 100. The threshold T1 and T2 are taken as 2 and 0.7 respectively. The generated error-parameter (E-σ) curves of the LR image at different sizes are shown in Fig.7.
Figure 7. The error-parameter curves of the de-noised LR image. Figure 6. The Bilinear interpolated image of de-noised LR image.
Figure 5. The Bilinear interpolated image of LR image. Figure 4. The de-noised LR image.
Figure 2. The HR image.
Figure 3. The simulated LR image.
By analyzing the relationship of the multiple curves, the estimated size (
K
ˆ
) is 7, and the estimated standarddeviation (
σ
ˆ
) is 0.92. The estimated Gaussian PSF is shown in Fig.8. The relative estimation error of the size and standard deviation are as follows respectively:0 ˆ / 0 7 7 / 7 0
K −K K = − = ,
0 ˆ / 0 1 0.92 /1 0.08
σ σ σ− = − =
Thirdly, SR image is reconstructed through IBP algorithm. When the estimated size and standard deviation of the Gaussian PSF are 7 and 0.92, the SR reconstructed image is shown in Fig.9.
In addition, in order to justify the importance of Gaussian blur estimation in SR single-image reconstruction, in the case of the estimated size of Gaussian PSF is 7, and the estimated standard deviation is taken from 0.1 to 3 with an increment of 0.1, the PSNRs of the estimated SR images are shown in Fig.12. The SR reconstructed images when the standard deviations are 0.1 and 3 are shown in Fig.10 and Fig.11 respectively.
Relative to the HR image as shown in Fig.2, the mean square error (MSE) and PSNR of the reconstructed image gained by different methods are shown in Tab.1.
The experimental results show the effectiveness of the proposed method. The Pepper & Salt noise is restrained in the SR reconstructed image. The SR reconstructed image has better visual effect and higher PSNR than other methods. When the Gaussian PSF is near the real value, the SR reconstructed image has better visual effect and higher PSNR. When the estimated is smaller than the real value, the SR reconstructed image is ambiguous. When the estimated is larger than the real value, the SR reconstructed image has obvious ring effect.
Figure 11. The SR reconstructed image. (
σ
ˆ
=3) Figure 10. The SR reconstructed image. (σ
ˆ
=0.1)Figure 9. The SR reconstructed image (
σ
ˆ
=0.92).IV. CONCLUDS
A framework of blind single-image SR reconstruction with Gaussian blur and Pepper & Salt noise is proposed. The degrading processes Gaussian blur, down sample and Pepper & salt noise are considered in the LR imaging model. The gained LR image is de-noised by median filter. The Gaussian blur of the de-noised LR image is estimated through error-parameter analysis method. The SR image is reconstructed through IBP algorithm, in which the LR imaging processes are considered. The Pepper & Salt noise is well reduced in the SR reconstructed image. The visual effect and PSNR of the SR reconstructed image is improved. When the Gaussian blur is around the real value, the visual effect and the PSNR is better than other cases. The proposed framework may be applied in other SR image reconstruction cases, such as including defocus blur or motion blur, Gaussian noise, etc.
ACKNOWLEDGEMENTS
I wish to thank for Lihong Zhu, Lilan Cao and Wanan Yang. This work is supported in part by the National Nature Science Foundation of China (Gant no.61202195, 61202196), the Sichuan Provincial Education Department project (Gant no.11ZA174, 11ZA223, 11ZA224), the Application Fundamental Research Project of Sichuan Provincial Scientific and Technology Department (Gant no.2011JY0139), the key project of Yibin Science and Technology Bureau (Gant no.2011SF016, 2013ZSF009).
REFERENCES
[1] X.H. Zhang, M. Tang, and R.F. Tong, “Robust super
resolution of compressed video,” Visual Computer, vol.28,
pp.1167-1180, 2012.
[2] D.C.Garcia, “Super resolution for multiview images using
depth information,” IEEE Transactions on Circuits and
System for Video Technology, vol.22, no.9, pp. 1249-1256,
2013.
[3] L. Zhang, Q.Q.Yuan, H.F. Shen, and P.X. Li, “Multiframe
image super-resolution adapted with local spatial
information,” J. Opt. Soc. Am. A, Vol.28, no.3, pp.381-390,
2011.
[4] A. Giannoula, “Classification-Based Adaptive Filtering for
Multiframe Blind Image Restoration,” IEEE Transactions
on Image Processing, Vol.20, pp.382-390, 2011.
[5] S.Y. Yang, M. Wang, Y.G. Chen, and Y.X. Sun,
“Single-image super-resolution Reconstruction via learned
geometric dictionaries and clusters sparse coding,” IEEE
Transactions on Image Processing, vol.21, no.9, pp.
4016-4028, 2012.
[6] K.I. Kim, and Y. Kwon, “Single-image super-resolution
using sparse regression and natural image prior,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, Vol.32, no.6, pp.1127-1133, 2010.
[7] D. Chris, “Single image super-resolution using
self-examples and texture synthesis,” SIViP, Vol.5,
pp.343-352 2011.
[8] A. Rueda, N. Malpica, and E.Romero, “Single-image super
resolution of brain MR images using overcomplete
dictionaries”, Medical Image Analysis, vol.17, pp.113-132 ,
2013.
[9] M.Y. Zou, “Deconvolution and signal recovery,”
Beijing:Defense Industry Publishing, 2004
[10]F.Q. Qin, Z. L, L.H. Zhu, and LiL. Cao, “Blind
single-image super-resolution reconstruction based on
motion blur,” Advanced Materials Research, vol.225-226,
pp.895-899, 2011.
[11]J.P. Qiao, J. Liu, and Y.-W.,Cheng, “Joint blind
super-resolution and shadow removing,” IEICE Trans. INF.
& Syst.,vol.E90-D, no.12, pp.2060-2069, 2007.
[12]F.-Q. Qin. Blind image super-resolution reconstruction
based on PSf estimation, IEEE International Conference
on Information and Automation, vol.II, pp.1200-1203,
2010.
[13]Fengqing Qin, Lihong Zhu, Lilan Cao, Wanan Yang,
Single-Image Super Resolution Reconstruction with
Pepper and Salt Noise, 2013 International Conference on
Computer Science, Electronics Technology and Automation, unpublished.
Feng-qing Qin is born in Nanchong,
Sichuan, China, in 1976. She received her BS degree in computer science and technology from Sichuan Technique University, Zigong, Sichuan, China, in 1999, and her MS degree and PhD degree in communication and information system from Sichuan University, Chengdu, Sichuan, China, in 2006 and 2009 respectively.
She is currently an assistant researcher and vice director in the college of computer and information engineering, Yibin University. Her current research interest is image and video processing. E-mail: qinfengqing@163.com.
Lihong Zhu is born in Ziyang, Sichuan,
China, in 1978. She received her BS degree in computer science and technology from Chongqing Normal University, Chongqing, China, in 2001, and her MS degree in software engineering from Chengdu Electronic Technology University, Chengdu, Sichuan, China, in 2007.
She is currently a lecturer in the college of computer and information engineering, Yibin University. Her current research interest is image processing. E-mail: yibinwuyu @qq.com.
TABLE I.
THE MSES AND PSNRS OF THE IAMGE GAINED BY DIFFERENT METHODS
Performance
parameters MSE PSNR(in dB) Figure 5 40.0994 32.0994
Lilan Cao is born in Yibin, Sichuan,
China, in 1978. She received her BS degree in computer science and technology from Sichuan Technique University, Sichuan, China, in 2001, and her MS degree in software engineering from Chengdu Electronic Technology University, Sichuan, China, in 2007.
She is currently a lecturer in the college of computer and information engineering, Yibin University. Her current research interests are image and video processing. E-mail: 66243190@qq.com.
Wanan Yang is born in Nanchong,
Sichuan, China, in 1977. He received his BS degree in information engineering from Southwest Petroleum University, Sichuan, China, in 1999, and his MS degree in computer application technology from Southwest Petroleum University, Sichuan, China, in 2005, as well as his PhD degree in computer software and theory, Chinese Science and Technology University, in 2010.
