2017 International Conference on Computer, Electronics and Communication Engineering (CECE 2017) ISBN: 978-1-60595-476-9
An Improved SL0 Algorithm Based on the SESOP Method
Fu-ping YU, Di SHEN, Zhe LI
and Dan LV
Air Force Engineering University, Xi’an, Shaanxi, 710051, ChinaKeywords: Sequence subspace optimal, Smoothed l0 approximation (SL0) algorithm, Signal-to-noise ratio (SNR), Sparse coefficient, sparse decomposition.
Abstract. The unconstrained formula of the smoothed l0 approximation is proposed based on the basis pursuit (BP) and the improved smoothed l0 approximation (ISL0) algorithm, and then an
improved SL0 algorithm with the sequence subspace optimal method is provided to solve the unconstrained formula. The algorithm translates the constrained sparse decomposition problem into the unconstrained sparse decomposition problem of the smoothed l0 approximation, and the
unconstrained problem is solved with the SESOP method. The experimental results show that the proposed algorithm has the better effect of sparse decomposition and the denoising effect, the sparse coefficient has the better accuracy, and the algorithm is efficient to the proposed unconstrained problem.
Introduction
Sparse decomposition algorithm is a key problem in the field of signal sparse decomposition, which is related to the application of sparse decomposition in practice. Therefore, it is still an urgent problem to study the fast algorithms of sparse decomposition and find new ideas for better approximation of theoretical results of sparse decomposition. Literature [1] replaces l0 norm with
approximate function and proposes the algorithm of smoothed l0approximation (SL0) for sparse
decomposition. The decomposition results arising from this are of more sparsity, which meanwhile open up a brand-new research orientation of sparse decomposition algorithm. An improved smoothed
0
l approximation (ISL0) algorithm is proposed in Literature [2]. The algorithm is an extension and development of Literature [1]. The problem of smoothed SL0 sparse decomposition under constrained conditions is turned into the smoothed SL0 sparse decomposition problem under unconstrained conditions. Literature [3] applies SL0 algorithm for dictionary learning in sparse decomposition and achieves good results. In Literature [4], an improved SL0 algorithm based on zero space is proposed, which avoids the gradient projection operation. In Literature[5], the SL0 algorithm is used and meanwhile the penalty term is introduced to enhance the sparsity constraint so as to realize the accurate estimation of the complex sinusoidal signal frequency. SL0 algorithm is used with accelerated sparse recovery in [6]. In this paper, a smoothed SL0 algorithm based on sequential subspace optimization (SESOP) method is proposed by referring to the sparse solution idea of BP algorithm in sparse decomposition theory.
Unconstrained SL0 Algorithm
In order to consider the influence of noise, an approximate decomposition method is proposed in the process of getting sparse solution through the BP algorithm in the sparse decomposition theory, which is to turn the minimum l1 norm under the constrained conditions in BP algorithm:
1
2
2 1
min yAx n x (2) Therefore, by using the method of getting the minimum value without constraints and ISL0 algorithm of BP algorithm in sparse decomposition, the solution problem of sparse decomposition of the smoothed SL0 algorithm is turned into the unconstrained problem with penalty term.
First, the approximate function is
2
2
( ) 1 exp( )
2
x f x
(3) Because
0
0, 0
lim ( )
1, 0 x f x x
(4) Or approximate approach
0, ( ) 1, x f x x
(5) Then, define the function F x( ) as
1
( ) m ( )i
i F x f x
(6) It can be obtained by the formula (4) and (5): for smaller , x0F x( ). and
0 0
limF x( ) x
(7)
Therefore, the solving problem of sparse decomposition of the unconstrained smoothed SL0 algorithm with penalty term is:
2 2
min yAx F x( ) (8) is the penalty factor. determines the tradeoff between accuracy and error, affecting the accuracy of the calculation.
SL0 Algorithm Principle of SESOP Method
The sequential subspace optimization method is suitable for solving large-scale smooth unconstrained problems. The SESOP method optimizes the complexity of the worst case , and the storage requirement is low. The computational burden is smaller at each iteration, which makes the SESOP method a useful tool to solve the unconstrained optimization problem. In Formula (8), the function F x( ) is the smoothing function whose smoothness is determined by the parameter. Therefore, for the solution of Formula (8), the method of sequential subspace optimization is proposed.
In each iteration, the SESOP method searches for the minimum value of the objective function on the subspace that is extended by the current gradient and the early iterative direction. To define the subspace structure, the following series of directions are used:
1. The current gradient: ( )g xk , g x( )k represent the gradient at the xk of thek point;
(1) (1) 0 0
( )
k k k i i k id
x
x
d
w g x
(9)
in which 2
1 1 1 2 4 1 0 0 k k k w
w k
;
3. The previous direction:
p
k i
x
k i
x
k i 1,i
0, ,
s
1;4. The previous gradient:
g
k i ,i
1, ,
s
2.Among them, any subset of the three directions including the Nemirovski direction, the previous direction, and the previous gradient constitutes the subspace of the SESOP method together with the current gradient direction, and the current gradient direction becomes the necessary direction of the subspace. For quadratic equations, this method is equivalent to the conjugate gradient method when only ( )g xk and
p
k are used to constitute two-dimensional optimized subspace. When the Nemirovski direction is added to the two-dimensional subspace, the worst case optimization is ensured. When the subspace continues to increase in the previous directionp
k i
x
k i
x
k i 1 andthe previous gradient direction
g
k i , the iteration consumption is property increased, while the times of iterations is further reduced. Therefore, the SESOP method is to select K directions as the column vector to form a subspace matrix B; is the correlation coefficient of the subspace matrixB. Through the iterative search, the vector of B is expanded to the new direction B. Its specific stepsare:
1. Initialize:
x
k x0,B
B
0 g x( )0 ; 2. Orthogonal matrixB;3. Look for * arg min ( )
k f x B
4. Update the current iteration: *
1
k xk B
x
5. Update the matrix Baccording to the selected subspace direction;
6. Convergence judgment: convergence, subspace construction is completed; otherwise, go to Step 2.
Kis the equilibrium parameter for calculating the loss and the times of iterations in each iteration calculation, and Kis generally not more than 10 times or 100 times of the direction. At the same time, the basic SESOP method uses only the first derivative of the objective function. In many practical applications, the objective function has a second derivative, so the Newton method is often used in the subspace optimization of Step 3. Since the functionF x( )in Formula (8) has a second derivative, the Newton method is used in Step 3 when the unconstrained sparse decomposition problem with penalty term is solved using the SESOP method.
Analysis of the Experimental Result
In order to study the improved effect of the SESOP method on the unconstrained SL0 algorithm, the sparse decomposition effect of the signal under noiseless interference and that under noisy interference are simulated and analyzed respectively. A randomly generated signalxwith a sampling
point of 256 and a sparsity of 0.03 is used as the sparse original coefficient to randomly generate an over-complete dictionaryAof 128 * 256 in size; according to yAx, get the signalythat needs sparse decomposition. For signal with disturbed noiseynoise Ax
n
y n
,n
is Gaussian whitewhich yAx is the noise-free interference signal; the signal-to-noise ratio between the decomposition coefficient and the original real coefficient is SNR10log (10 x 22 (xˆx) )22 .
In order to study the effect of the sparse decomposition of the signal through SL0 algorithm based on SESOP method in the presence of interfering noise, the influence of the parameters in the Formula (8) on the sparse decomposition results is analyzed first. On this basis, the decomposition effects under the different signal-to-noise ratios are analyzed.
Figure 1 shows the signal-to-noise ratio of the reconstructed signal corresponding to the different penalty parametersat the signal-to-noise ratio of 5dB through 100 iterations. It can be seen from the figure that when the penalty parameter is around 0.09, the reconstructed SNR is the largest, the anti-interference ability is stronger, the signal reconstruction error is the smallest and the decomposition accuracy is higher. As the penalty parameter increases or decreases, the reconstructed SNR becomes larger and the reconstruction accuracy decreases. So in the subsequent simulation experiments, the penalty parameter takes the value of 0.09.
Figure 2 shows the contrast between the decomposition coefficient and the original real coefficient obtained by sparse decomposition of the noise interference signal using the SL0 algorithm based on the SESOP method when the penalty parameter is 0.09 and the SNR is 20dB. It can be seen from the figure that the coefficients obtained by sparse decomposition of the signal with noise interference through the SLO algorithm based on SESOP method are sparseness. At the same time, by comparing with the original real coefficients, the sparse coefficients well reflect the information and characteristics of the original coefficients, and the sparse coefficients are highly reliable. And the signal-to-noise ratio between the decomposition coefficient and the original real coefficient is 40.897dB, and the signal-to-noise ratio is improved greatly.
[image:4.612.131.488.374.511.2]Figure 1. Reconstructed Signal-to-Noise Ratio in Line with Different Penalty Parameters.
Figure 2. Sparse Decomposition Coefficient of SESOP Method.
Table 1. Reconstruction Signal-to-noise Ratios Corresponding to Different Input Noise Amplitude.
Noise Amplitud
e
Reconstruction SNR SESOP
Method
ISL0 Method
0.1 10.1482 7.6641
0.09 11.1223 9.4043
0.08 13.2552 9.883
0.07 14.5682 10.25576 0.06 16.4120 11.8489 0.05 19.7952 13.8579 0.04 24.904 15.8162 0.03 29.622 18.4658 0.02 38.477 22.1827 0.01 44.720 28.2529
[image:5.612.220.391.372.504.2]Figure 3 is a comparison of the decomposition coefficients obtained with the improved SL0 algorithm using the different methods and the original real coefficients. It can be seen from Figure 3 that the decomposition coefficient obtained from the SL0 algorithm based on SESOP method is of sparseness, and it is very close to the original real coefficient, the error is very small and the original coefficient of information is fully reflected. The decomposition coefficient obtained from ISL0 algorithm has interference information and its sparsity is affected, but the main feature information of the original coefficient has also been fully reflected. At the same time, the operation time required by the SL0 algorithm improved by the SESOP method in 100 cycles is 0.325579s and that of ISL0 algorithm is 0.33083s. The difference is very small, but the computation speed of SL0 algorithm improved by SESOP method is faster.
Figure 3. Decomposition Coefficients Obtained through Different Methods of SL0 Algorithm.
Summary
On the basis of putting forward the solution of sparse decomposition through the unconstrained smoothed SL0 algorithm with penalty term, this paper further proposes to solve the above problems by the SESOP method. The simulation results show that compared with ISL0 algorithm, the improved Sl0 algorithm based on SESOP method has better sparse decomposition effects on signals with noise interference, the sparsity of the decomposed coefficient is better and the error better the decomposed coefficient and the original real coefficient is smaller, thus well reflecting the information and features of the original real coefficient; it also has higher accuracy. At the same time, the SL0 algorithm based on SESOP method is more efficient.
Acknowledgement
References
[1] Mohimani H., Babaie-Zadeh M., Jutten C. A fast approach for over- complete sparse decomposition based on smoothed l0norm [J]. IEEE Trans. on Signal Processing, 2009, 52(1): 289-301.
[2] Hyder Md. M., Mahata K. An Improved Smoothed l0 Approximation Algorithm for Sparse
Representation [J]. IEEE Transactions on Signal Processing, 2010, 58 (4): 2194-2205.
[3] Mahdi Ataee, Hadi Zayyani, Massoud Babaie-Zadeh, et, al. Parametric dictionary learning using steepest descent [C]// Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2010: 1978–1981.
[4] Qiu Wei, Zhao Hong-zhong, Chen Jian-jun, Fu Qiang. High-resolution Radar One-dimensional Imaging Based on Smoothed l0Norm [J]. Journal of Electronics & Information Technology, 2011,
33(12): 2869-2874.
[5] Cui Zhi-fu, Zhang Hang, Lu Wei. Subspace iterative algorithm for sparse decomposition based on improved smoothed l0-norm [J]. Journal of PLA University of Science and Technology Natural
Science Edition, 2011, 12(5): 409-413.
[6] Han Wang, Qing Guo, Gengxin Zang. Thresholded smoothedl0norm for accelerated recovery [J].
