Abstract —In this paper, we consider an example of the lifting scheme and present the results of the simple lifting scheme implementation using lazy transform. The paper is tutorial-oriented. The results are obtained by testing several common test signals for the signal denoising problem and using different threshold values. The lifting scheme represents an effective and flexible tool that can be used for introducing signal dependence into the problem by improving the wavelet properties.
Keywords — Dual and primal lifting step, lazy transform, lifting scheme, wavelets.
I. INTRODUCTION
HE wavelet transform (WT) has become very popular in signal representation and analysis. The incapability to have a satisfying insight into a nonstationary signal in frequency domain using the Fourier transform, as seen in frequency spectrum (Fig. 1) or having the satisfying both time and frequency resolution using the Short-Time Fourier Transform, as seen in spectrograms (Fig. 2), brings out the advantages of wavelet transform and its discrete (DWT) and continuous (CWT) versions. In Fig. 3 scalogram representations of a nonstationary signal are presented. Particularly, DWT transform has been used extensively for signal feature extraction.
Lifting scheme (LS) is a flexible tool for building and modification of perfect reconstruction (PR) filters ([1]- [2]). It has different purposes, from building traditional wavelet filter banks to construction of wavelets on different domains. One of the most important purposes is the capability to introduce signal dependence into the problem we are solving. Such an approach can be significant in wavelet based denoising, signal representation and feature extraction.
The lifting scheme was first proposed as a framework for wavelet construction by Sweldens ([3]-[7]). Besides the standard construction of wavelets and their implementation using translations and dilations (the first generation), the second generation wavelets that were
A. M. Gavrovska, PhD stipendiary of the Ministry of Science and Technological Development, Republic of Serbia, is with the Faculty of Electrical Engineering, University of Belgrade, Bulevar Kralja Aleksandra 73, 11120 Belgrade, Serbia; (phone: 381-11-3370143; e-mail:
M. P. Paskaš is with the Innovation Center of the Faculty of Electrical Engineering, University of Belgrade, Bulevar Kralja Aleksandra 73, 11120 Belgrade, Serbia; (phone: 381-11-3370143; e-mail:
I. S. Reljin is with the Faculty of Electrical Engineering, University of Belgrade, Bulevar Kralja Aleksandra 73, 11120 Belgrade, Serbia;
(e-mail: [email protected]).
constructed using the lifting scheme has become more suitable for the situations where we want to avoid the standard implementation, as well as introducing wavelets using the Fourier transform. In other words, using the first generation wavelets we build a joint time-frequency domain based on the Fourier transform, where in the case of using the second generation we investigate the correlation structure of the signal in the spatial domain.
The paper is organized as follows. In Section II an introduction to lifting scheme and a short explanation on its advantages over the standard wavelet transform (WT) is presented. A simulation that uses Lazy transform and a simple way to implement LS can be found in Section III Section IV is dedicated to simulation results on several test signals. Finally, several concluding remarks and observations are presented in Section V.
Fig. 1. Time and frequency domain of (a) stationary and (b) nonstationary signal.
500 1000 1500 2000 2500 3000 3500 4000 -2
0 2 4
t
x(t)
0 0.1 0.2 0.3 0.4 0.5
0.1 0.2 0.3 0.4
f (a)
|Y(f)|
500 1000 1500 2000 2500 3000 3500 4000 -1
-0.5 0 0.5 1
t
x(t)
0 0.1 0.2 0.3 0.4 0.5
0.02 0.04 0.06 0.08 0.1 0.12
f (b)
|Y(f)|
Wavelet Denoising within the Lifting Scheme Framework
Ana M. Gavrovska, Member, IEEE, Milorad P. Paskaš, Member, IEEE, and Irini S. Reljin, Senior Member, IEEE
T
Fig. 2. STFT: (a)spectrogram with better frequency than time resolution and (b)spectrogram with better time than
frequency resolution.
Fig. 3. DWT and CWT scalogram representations.
II. THE LIFTING SCHEME
The lifting scheme represents a way to improve wavelet properties using so called lifting steps. The selection of wavelet may not be the final obstacle to get satisfying results in signal denoising, even in the cases where choice of the filters is not appropriate.
Lifting generalizes the idea of multiresolution and wavelet transform. A general filter bank implementation of the wavelet transform is presented in Fig. 4. The transform is typically iterated on the output of LP (low-pass) band,
generating the series of detail coefficients at different scales (HP (high-pass) bands) and approximation coefficients at the final scale (the last decomposition level). In this way, the standard wavelets (the first generation) are implemented, generated by translations and dilatations of one or more basic functions. LS is a generalization relative to translations and dilations and is used for adjusting the wavelets according to signal's needs.
It is considered that neighboring data and frequencies are far more correlated than those that are further away from each other.
In Fig. 5 a typical structure of the lifting scheme is presented starting from splitting the signal into odd and even indexed samples. In this way, a trivial wavelet transform is applied, called Lazy wavelet transform (LT or LWT). There are two different types of lifting steps: the dual lifting or predict (P) step and the primal lifting or update (U) step. The transform using lifting is typically iterated in a similar way as standard wavelet transform.
This is a two-band decimated filter bank for obtaining a series of detail coefficients, as well as approximation coefficient from the last decomposition level.
Existing wavelet filter banks can be factored into basic building blocks such as lifting steps [8]. The order and their number depend on the particular implementation, not only to factor existing banks but also to construct wavelet bases that are appropriate for the signal processing application. In the presented structure of lifting scheme, a predictor is used for estimating the odd indexed samples from the even indexed samples. An approximation is constructed based on the update step.
Prediction step deals with spatial correlation, but simple subsampling in LT makes an approximation inappropriate, so the primal lifting step (update) is needed. In this step, an update operator is applied on previously calculated detail coefficients and added to even indexed samples constructing the LP band, as presented in Fig. 5.
Signal-dependent lifting could lead to better results, e.g.
using short wavelets for high frequency components like singularities' neighborhood and longer ones for low frequency components. Making the approach adaptive is based on modifying the transform according to some signal properties. Nevertheless, in order to obtain such results, there is still an open question on several settings, such as the used criteria for satisfying solution(s), level selection, modification technique of obtained coefficients, etc. In this paper, we present an easy-to-understand example of the LS and its structure and give some simulation results.
Fig.4.Filterbankimplementationofthewavelettransform.
500 1000 1500 2000 2500 3000 3500 4000 -1
0 1
t x(t) Scale
DWT
t
500 1000 1500 2000 2500 3000 3500 4000 CWT
Scale
500 1000 1500 2000t 2500 3000 3500 4000
Fig. 5. A lifting scheme.
III. SIMULATION
We present a simple simulation on several common- used test signals for the purpose of denoising. This example is more thoroughly explained in Section III.A.
The Section III.B is dedicated to remaining simulation settings.
A. An Example of LS Implementation
In order to simulate LS, we select the most famous, simple LT with a single prediction (dual) and a single updating (primal) step as elementary lifting steps (ELSs).
By splitting the input into even indexed samples )
2 ( n
aj and odd indexed samples aj(2n+1), starting from index n=0, the LT is applied (initial level j=0).
Let both dual and primal steps be simple. The prediction filter can be an average of two even indexed neighbors. By calculating the errors of prediction, the wavelet (detail)
coefficients, dj+1(n), are obtained:
( )
(
(2 ) (2 2))
,2 ) 1 1 2 (
) 2 2 ( ), 2 ( ) 1 2 ( )
1(
+ +
− +
=
+
− +
+ =
n a n a n
a
n a n a P n a n d
j j
j
j j j
j (1)
where P denotes the prediction operator. It can be also a function of a set of adjacent even indexed samples.
Subsampling in LT makes LP band inappropriate, and a simple constraint for introducing the updating step can be preserving the running average, reducing the aliasing problem. The loss of 1
2 1
+
dj that exists after subsampling is added to two even neighborhood samples (for each
4 1
1
+
dj ). In Fig. 6 a geometric interpretation of two lifting steps is presented. Using
( )
(
( 1) ( ))
,4 ) 1 2 (
) ( ), 1 ( )
2 ( ) (
1 1
1 1
1
n d n d n a
n d n d U n a n a
j j
j
j j
j j
+ +
+ +
+
+
− +
=
− +
= (2)
where U denotes the update operator, it is possible to obtain approximation (scaling) coefficients, aj+1(n), based on previously calculated detail coefficients.
An example of signal decompostion for final level decompostion J=4, ( j=0,1,...,J ), can be found in Table 1. Obtained coefficients for each level are marked red. The calculation is performed in-place.
TABLE 1:AN EXAMPLE OF IN-PLACE CALCULATION.
Signal 3 8 56 4 9 6 4 8 0 6 4 9
Split#1 3 8 56 4 9 6 4 8 0 6 4 9
P 3 -21.5 56 -28.5 9 -0.5 4 6 0 4 4 7
U -2.375 -21.5 43.5 -28.5 1.75 -0.5 5.375 6 2.5 4 6.75 7
j #1 -2.375 -21.5 43.5 -28.5 1.75 -0.5 5.375 6 2.5 4 6.75 7
Split#2 -2.375 43.5 1.75 5.375 2.5 6.75
P -2.375 43.8125 1.75 3.25 2.5 5.5
U 8.5781 43.8125 13.5156 3.25 4.6875 5.5
j #2 8.5781 -21.5 43.8125 -28.5 13.5156 -0.5 3.25 6 4.6875 4 5.5 7
Split#3 8.5781 13.5156 4.6875
P 8.5781 6.8828 4.6875
U 10.2988 6.8828 4.6875
j #3 10.2988 -21.5 43.8125 -28.5 6.8828 -0.5 3.25 6 4.6875 4 5.5 7
Split#4 10.2988 4.6875
P 10.2988 -0.4619
U 10.1833 -0.4619
j #4 10.1833 -21.5 43.8125 -28.5 6.8828 -0.5 3.25 6 -0.4619 4 5.5 7
- signal - result of decomposition for level j - additional marks - approximation coefficients
for level j
- detail coefficients for level j
- result of splitting (2nd channel)
Fig. 6. A geometric interpretation of linear prediction and update in the case of piecewise linearity.
TABLE 2:AN EXAMPLE OF THE LIFTING SCHEME.
i ELS
type
Lifting filter coefficients of
length
=2 Ni
d Filters
1.
dual lifting ('d')
[
−0.5 −0.5]
1 −P(z)=−0.5z−0.52.
primal lifting ('p')
[
0.25 0.25]
0 U(z)=0.25+0.25z−1Elementary lifting steps ((1)-(2)) can be rewritten in the sense of Laurent polynomials of degree d and set of coefficients C(k), k=1,...,Ni,
( )z =C(1)zd +C(2)zd−1+...+C(Ni)zd−Ni+1.
H (3)
Table 2 presents the ELSs that were used in this case, as well as applied filters. The parameter d besides being the maximal degree of the polynomial, represents the delay of using the neighborhood in the calculation.
B. Simulation Settings
For testing purposes, six typical test signals (blocks, bumps, heavy sine, doppler, quadchirp, mishmash) of length 1024 samples are used ([9]-[10]). To each test signal, Gaussian noise is added with signal-noise ratio (SNR), 2dB-32dB, where input SNR is defined as
10log .
2 2 10
n sig input
SNR σ
= σ (4) and where σ and sig σ represent standard deviations of n
the tested signal and added noise, respectively. In Fig. 7 several original signals (targets) are presented with their noisy versions for SNRinput =14dB.
Both hard and soft thresholding are used for obtaining results by applying them on detail coefficients from each scale (selected fixed threshold is not local and level- dependent). Equally distant threshold values are tested (value 0-4). The criterion for the "final denoised" signal is the least mean square error with regard to a specific target (or e.g. it can be chosen the one that as a result has the least value of the sum s of absolute errors between target and the preliminary denoised signal, sum of squared errors, etc.).
The choice of level is of importance in order to retain valuable low-frequency information. Here, the level of decomposition is set to be 3.
Fig. 7. The bumps, doppler and quadchirp signal and their noisy versions for
SNR
input= 14 dB
.IV. SIMULATION RESULTS
To each of tested signals Gaussian noise of 2-32dB signal-noise ratio is added and values of thresholds were tested.
Fig. 8. Results for bumps signal for (a) soft and (b) hard thresholding, respectively (SNRinput =14dB).
0 200 400 600 800 1000
0 2 4
t(s)
Input Target
0 200 400 600 800 1000
-0.5 0 0.5
t(s)
Input Target
0 200 400 600 800 1000
-1 0 1
t(s)
Input Target
0 200 400 600 800 1000
0 1 2 3 4 5
t(s)
Input Result
0 200 400 600 800 1000
0 1 2 3 4 5
t(s) (a)
Result Target
0 200 400 600 800 1000
0 2 4
t(s)
Input Result
0 200 400 600 800 1000
0 1 2 3 4 5
t(s) (b)
Result Target
Obtained results versus noisy input forSNRinput =14dB (σn =0.30722) and target are shown in Fig. 8 for the case of soft and hard thresholding, respectively. The results are obtained using thresholds that give the minimal MSE (mean square error) compared with the target. Values of MSE while changing threshold values are presented in Fig. 9 for different SNRinput.
The new SNR is calculated for noisy original signal x and denoised signal as a result r , using:
( )[ ] [ ] [ ]
( ) ,
log
10 2
2 10
= −
i output i
i r i x
i x
SNR (5)
where x− represents estimated noise. In the case r presented in Fig. 8 new calculated SNRs are 15.29dB and 14.46dB for soft and hard thresholding, respectively.
Fig. 9. Bumps signal simulation results for different
input
SNR
values using (a) soft and (b) hard thresholding.Greater the SNRinput it is expected to have a lower threshold value. The comparison between input parameter SNR and calculated SNRoutput is shown in Fig. 10 and Fig. 11. The criterion of choosing a minimum MSE in Fig.
11 has showed to be a better choice than the criterion of absolute errors in Fig. 10. Other possibilities are also available (as in [9]).
The fact that hard thresholding may seem more natural to non-statisticians [10] can be seen in Fig. 8.
Nevertheless, soft thresholding is selected as a general choice for denoising purposes because of statistical properties of the obtained results. In Fig. 9 the criterion functions are smoother and generally give lower or similar MSE values.
As a result, generally speaking, final SNRs are often higher in the case of soft thresholding without sudden changes as with hard thresholding. It is believed that in the signals where the morphology of the signal is of importance this represents a valuable fact.
Fig. 10. Calculated SNR for different noisy test signals compared to initial SNR using (a) hard and (b) soft
thresholding. Minimum sum of absolute errors is used as a criterion.
V. CONCLUSION
The lifting scheme has showed to be a valuable tool for developing more complicated structures based on LS. This paper only emphasizes the simplicity of using the second generation wavelets on the example of LT. In [8] it is shown that every wavelet filter pair can be decomposed into LS. LS ability for making different adaptive constraints and easy perfect reconstructions is worth taking into account for the purpose of denoising and feature extraction. The advantage of working in a spatial domain may be considered as a crucial one.
0 0.5 1 1.5 2 2.5 3 3.5 4
0.2 0.4 0.6 0.8 1 1.2 1.4
Threshold value (a)
MSE
SNR=2dB SNR=5dB SNR=8dB SNR=11dB SNR=14dB SNR=17dB SNR=20dB SNR=23dB SNR=26dB SNR=29dB SNR=32dB
0 0.5 1 1.5 2 2.5 3 3.5 4
0.2 0.4 0.6 0.8 1 1.2 1.4
Threshold value (b)
MSE
SNR=2dB SNR=5dB SNR=8dB SNR=11dB SNR=14dB SNR=17dB SNR=20dB SNR=23dB SNR=26dB SNR=29dB SNR=32dB
5 10 15 20 25 30
-5 0 5 10 15 20 25 30
SNRinput [dB]
(a)
SNRoutput [dB]
blocks bumps heavysine doppler quadchirp mishmash
5 10 15 20 25 30
-5 0 5 10 15 20 25 30
SNRinput [dB]
(b)
SNRoutput [dB]
blocks bumps heavysine doppler quadchirp mishmash
Fig. 11. Calculated SNR for different noisy test signals compared to initial SNR using (a) hard and (b) soft thresholding. Minimum MSE is used as a criterion.
The future work will be oriented towards finding the adaptive techniques in multiresolution analysis and wavelet transform parameter selection for specific implementations. The lifting steps both in signal and image processing may considerably help in introducing signal/image dependence.
REFERENCES
[1] D. P. Radunovic. Wavelets from Math to Practice, Academic Mind, Serbia, Springer-Verlag, Germany, 2009.
[2] J.L. Starck, F. Murtagh, and J. Fadili, Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity, Cambridge University Press, Cambridge (GB), 2010.
[3] W. Sweldens, “Wavelets and the lifting scheme: A 5 minute tour,”
Zeitschrift für Angewandte Mathematik und Mechanik, vol. 76 (Suppl. 2), pp. 41-44, 1996.
[4] W. Sweldens, “The Lifting Scheme: A new philosophy in biorthogonal wavelet constructions”, Wavelet Applications in Signal and Image Processing III, pp. 68-79, Proc. SPIE 2569, 1995.
[5] B. Jawerth and W. Sweldens, “An Overview of Wavelet Based Multiresolution Analysis”, SIAM Rev., vol. 36, no. 3, pp. 377-412, 1994.
[6] W. Sweldens, “The lifting scheme: A construction of second generation wavelets”, Siam J. Math. Anal, vol. 29, no. 2, pp 511-546, 1997.
[7] J. Kovacevic and W. Sweldens, “Wavelet Families of Increasing Order in Arbitrary Dimensions”, IEEE Trans. Image Proc., vol. 9, no. 3, pp. 480-496, 2000.
[8] I. Daubechies and W. Sweldens, “Factoring Wavelet Transforms into Lifting Steps”, J. Fourier Anal. Appl., vol. 4, no. 3, pp. 247- 269, 1998.
[9] D.L. Donoho and I.M. Johnstone, "Ideal spatial adaptation by wavelet shrinkage," Biometrika, vol. 81, pp. 425-455, 1994.
[10] D.L. Donoho and I.M. Johnstone, "Adapting to unknown smoothness via wavelet shrinkage," JASA, vol. 90, pp. 1200-1224, 1995.
5 10 15 20 25 30
5 10 15 20 25 30 35
SNRinput [dB]
(a)
SNRoutput [dB]
blocks bumps doppler heavysine mishmash quadchirp
5 10 15 20 25 30
5 10 15 20 25 30 35
SNRinput [dB]
(b)
SNRoutput [dB]
blocks bumps doppler heavysine mishmash quadchirp
