Error Estimation and Assessment of an Approximation in a Wavelet Collocation Method

Error Estimation and Assessment of an Approximation

in a Wavelet Collocation Method

Marco Schuchmann, Michael Rasguljajew University of Applied Science, Darmstadt, Germany

Email: [email protected]

Received January 17,2013; revised February 25, 2013; accepted March 23, 2013

Copyright © 2013 Marco Schuchmann, Michael Rasguljajew. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

ABSTRACT

This article describes how to assess an approximation in a wavelet collocation method which minimizes the sum of squares of residuals. In a research project several different types of differential equations were approximated with this method. A lot of parameters must be adjusted in the discussed method here. For example one parameter is the number of collocation points. In this article we show how we can detect whether this parameter is too small and how we can assess the error sum of squares of an approximation. In an example we see a correlation between the error sum of squares and a criterion to assess the approximation.

Keywords: ODE; Sinc Collocation; Shannon Wavelet; Wavelet Collocation; Error Estimation

1. Introduction

In the wavelet theory a scaling function  is used, which has properties that are defined in the MSA (multi scale analysis). Through the MSA we know, we can con-struct an orthonormal basis of a closed subspace Vj, where Vj belongs to a sequence of subspaces with the following property:

 

2

1 0 1 ,

V_ V V L R

    

 

 



j k, t



_{k Z} is an orthonormal basis of Vj with

 

2





, 2j 2j

j k t t

   k

j k

. We use the following approximation function:

 

max

 

min

, :

k

j k

k k

y t c  t







 , with 1

 

.

C R

 

min

k



t t0, endand



kmax depend on the approximation interval Now we can approximate the solution of an initial value problem

.



,



y  f y t and y t

 

₀ y₀ by mini-

mizing the following function (  is the Euklid norm)

 

 



 



2

 

2

0 0

1

,

m

j i j i i j

i

Q c y t f y t t y t y









   . (1)

For m k_maxk_min we get an equivalent problem:

 



 

,



j i j i

y t  f y t ti , with i1, 2, , m and

 

0 0 j

y t y .

The advantage of calculating by minimizing Q is

that we can choose more collocation points i as shown in the following example. In that case we apply the least squares method to calculate . Many simulations had shown that if _min was very small then the approxima-tion yj would be good. An even better criterion for a good approximation

c

t

c Q

j

y is a (see (3)). Moreover, the equations have been ill-conditioned in several examples.

Q

Analogously we could use boundary conditions in- stead of the initial conditions. This method can be even used analogously for PDEs, ODEs of higher order or DAEs, which have the form



, ,



0.

F y y t 

If y  f y t

 

, is an ODE system, then we use the

approximation function:

 

 

 

 

max max max

min min min

T

,1 , , ,2 , , , , f ,

j

k k k

k j k k j k k n j k

k k k k k k

y t

c  t c  t c  t

  

 

  













For the i-th component of the solution y, we use the

notation yj as usual. We use for the i-th component of j

with the approximation yj out of Vj, so it will be always distinguished whether the approximation yj

0 or the i-th component of y is used.

We use the collocation points , with ti ti  t i h and





end

maxk . 0

t t

m



min

k

h m (2)

Simulations have shown that even with

max min

m k k we getgood approximations.

For the assessment of the approximation we use the value Qa, with

 



 



2

 

2

0 y0 1

,

a

m

a j i i i j

i

Q y  y t









 f y_j  ,  (3)

0

i t i h a

    , a and is an integer. For big we should weight with

m  a m

a

Q

1

a

a 1a.

Remarks 1: 1) We get

 



max



0, ,

 

k



min

,

 

 

1 2

max

min

k k

Q Q

for  1   , because of: max max

k k2

k

 

 

 





 

 



1 2

max max

min, , _max 0 min,

k k

k _k k

Q c c 1 Q c  1



max

,c 

0 Analogously for smaller min.

2) The sums in (1) and (3) could start with i , too.

3) y tj

 

0 y0 2 in (1) could also be used as a con- straint if the initial value should be fulfilled. But in all good approximations, y tj

 

0 y0 2 was very small.

In the examples we use the Shannon wavelet. Al- though it has no compact support and no high order, in many examples and simulations we got a much better

approximation than using other wavelets (f.e. Daubechies wavelets of order 5 to 8), even with a small n. The Meyer

wavelet yields good results, too.

We even get a good extrapolation outside the interval



t t0,end



.

Example 1:

1) We use the following ODE

 

, 0 1

y  t y y  .

The exact solution is

 

_e1 2t2

y t   .

We approximated the solution on the interval

 

0,1 and chose kmax  kmin, like in all examples.

With a we could see in all our simulations, if the approximation was good. We got a linear relationship between

Q

 

Q2

ln and ln

 

sse . In Figure 1 we see the graph of a linear regression (with an R squared of

0.991196) of ln

 

sse against ln

 

Q₂ with the points

   



ln Q2 ,ln sse



, which have been calculated with dif- ferent j j



0,1,2 ,



kmax kmin



kmax 15, 20, 25



and



max x



with the ODE and I of the

example 1. max , 2k ,3k_ma

m mk

sse is the mean squared error

 

 





100 ₂

0 i j i

i

sse y t y t







   with tii100. Now we see a regression table (Table 1) of ln

 

sse

on ln

 

Q2 , which shows a linear dependency in our

example and the graph of the linear regression function. Here is a graph of the regression function and the graphs of the functions yi and j for j = 0, kmax = 15 and

y y

30

m on the approximation interval

 

0,1 (see

Figures 2 and 3) and on the interval



1, 2



(see Fig- ures 4 and 5). In Figures 4 and 5 we see that we get even a good extrapolation.

[image:2.595.127.470.510.715.2]

0.0 0.2 0.4 0.6 0.8 1.0 t

0.7 0.8 0.9 1.0 y_j

　

t

　

,y

　

t

　

j 　 0

kmax　 15

Qmin　 3.35451 　10　29

Q2　 1.64716 　10　26 mse　 6.88252 　10　30

[image:3.595.72.530.52.746.2]

m　 30

Figure 2. Graph of y₀, kmax = 15, m = 30.

0.2 0.4 0.6 0.8 1.0 t

6.　10　15 4.　10　15 2.　10　15 2.　10　15 4.　10　15

yj

　

t

　

　y

　

t

　

j

　

0

kmax

　

15

Qmin

　

3.35451

　

10　29

Q2

　

1.64716

　

10　26 mse

　

6.88252

　

10　30

m

　

30

Figure 3. Graph of y0y, kmax = 15, m = 30.

1.0 　0.5 0.5 1.0 1.5 2.0 t

0.4 0.6 0.8 1.0 y_j

　

t

　

,y

　

t

　

j

　

0

kmax

　

15

Qmin

　

3.35451

　

10　29

Q2

　

1.64716

　

10　26 mse

　

6.88252

　

10　30

[image:3.595.347.502.85.283.2]

m

　

30

Figure 4. Graph of y₀, kmax = 15, m = 30.

1.0 0.5 0.5 1.0 1.5 2.0 t

　5.　10 9 5.　10 9 1.　10 8

yjt y t

j 0

kmax

　

15

Qmin

　

3.35451

　

10　29

Q2

　

1.64716

　

10　26 mse

　

6.88252

　

10　30

[image:3.595.308.538.331.384.2]

m

　

30

Figure 5. Graph of y₀y, kmax = 15, m = 30.

Table 1. Linear regression table of ln

 

sse on ln

 

Q2 .

Estimate SE T Stat P Value

Intercept −5.34843 0.882147 −6.06297 2.46251 × 10−6

Slope 0.962059 0.0181342 53.0523 3.2288 × 10−27

2. Error Estimation and Assessment of the

Approximation

In the example we used the Shannon wavelet. For this wavelet we have additional information about the error in the Fourier space from the Shannon theorem. For a good approximation with a small j the behavior of Y

 



with growing  is important, because (if yi is an or- thogonal projection from y on Vjand kmax  kmin )

 

 

 

 

 

 

2π

i i

2 π 2 π

i i

2π

1 _{e d} 1 _{e d}

2π 2π

1 1

e d e d .

2π 2π

j

j j

j

j

t t

t t

y t y t

Y Y

Y Y

 

 

   

  



 

 





 

 











With the Parseval theorem we get

 

 

2

2π

2 2

2π

d d

j

j

j L

y y Y   Y  ,

 



 

_



_



so

 

 

1 2

1

2π 2 π

2 2

2 π 2 π

d d

j j

j j

j L

d Y   Y 

 







_



_



With the Riemann-Lebesgue theorem we get:

2

lim 0

[image:3.595.310.514.501.703.2] [image:3.595.95.249.514.730.2]

For the approximation error the decay behaviour of the detail coefficients , ,

j

k j

d  y k is important:

2

2

, 2

s

j _L s k s

s j s j k 2

k

L L

s k s j k

y y d d

d               



 

 

On the other side: we have got in many simulations with the Shannon wavelet better approximations (with the described collocation method) than with higher order wavelets.

Remarks 2:

1) For a theoretical multi resolution analysis we could consider instead of , because when is in

then 1I is in

1Iy y y

 

2

L I y L2

 

R , if we need an ap-

proximation on I. Here 1_I is the indicator function of

the interval I.

2) For interpolating wavelets there are a number of publications with error estimates and also for the ap- proximation of the solutions of initial value problems and boundary value problems (for ordinary and partial dif- ferential equations) see [1,2], as well as to the sinc col- location method (see [3-5]) with special collocation points (“sinc grid points”, see [5]).

Theorem 1 (for the decay behaviour): The wavelet  has the order ,

with and is Lipschitz continu- ous. Then exists a independent from b with

 

2 ,

p yL R

 

r

yC R rp

0

c

 r

y

 

3 2

, r .

y

W a b c a 

y

W is the wavelet transform of y with

 

1 2

 

, d

y

t b

W a b a y t t

a          _{  }  



.

A proof is in [6]. So we get for the detail coefficients an appraisal because

 









2 ,

, 2 2

2 , 2

j j

k j k

IR

j j

y

d y y t t k

W k

 

 

  





j d

t

and so



_{2 , 2}



₂ j r 3 2_.

j j j

k y

d W  k   c   

Now we saw that the decay of the detail coefficients depends on the order of a wavelet.

From the Gilbert-Strang Theory (see [7]) we know ad- ditionally an upper bound of the approximation error in dependency of the order : if the wavelet is of order then the approximation error has the order

p p



2 j p



O   if

 

2

p

L

y   and (if yj is an orthogonal projection from y on V _j and kmax  kmin )

 

2 2 2

p jp

j L L

yy C_   y .

If a wavelet is of order the scaling function p 

even has an interpolation property, because then we can construct the functions with over a linear combination of

r

t r0,1, ,_ p1



t k



  (see [7]). That’s also a property of the so called interpolating wavelets. For in- terpolating wavelets we find error estimations in [8] and [9].

Remarks 3:

1) Error estimations for the sinc collocation with a transformation can be found in [4] and [5].

2) Although the approximation error is depended on the order of a wavelet in many simulations the Shannon wavelet led to much better approximations than Daube- chies wavelets of higher order, if the approximation function yj was calculated by minimizing the sum of squares of residuals Q. Even when comparing the ex-

trapolations the Shannon wavelet was significantly bet- ter.

The reason is, that we do not calculate an orthogonal projection on Vj like in the appraisal above and the function y is in general case not quadratic integrabel on R

(we consider only a compact interval I).

The following appraisal takes account of the fact that we calculate the approximation function by the minimi-zation of Q. We first need a theorem, which follows from

the Gronwall-Lemma. Theorem 2:

Assumptions: we have a initial value problem

 

,

y  f y t with y t

 

₀ y₀ and

   

 



 



0 0 ,

,

j

j j

y t y t

y t f y t t M



 

   (4) and

 



,





 

,



 

 

with 0.

j j

f y t t f y t t L y t y t

L

   

 (5)

Then we get for tt₀:

 

 

_eL t t 0



_eL t t 0



j

y t y t    M L  1

For a proof see [10]. Theorem 3:

With the assumptions from Theorem 2 we get (if

 

0 0 j

y t y ):

   

2





2



 0



2

1 1

e i 1

L

m m

L t t

j i i

i i

C

sse y t y t M L 

   



 _  _  



   



So we get the follow inequality for ln



sse , which is

 

   

2 2

 

ln sse ln M ln L ln CL (6) points (we get a approximation function yj) we must not calculate a second minimization for the calculation of

. a

Q



2

ln M



depends on y_j and only

on the initial value problem and the collocation points. We write

 

 

2

ln CL ln L





2

ln M instead of 2ln

 

M because in

example 2 we set _ln

 

2

M on the x-axes so we have a

comparison with example 1 where we set ln

 

Q2 on

the x-axes.

 



 

,



j i j i i will be in general (for ) less than M, because we use the collocations points ti and so

y t  f y t t i0

 



 

,



j i j i i

min

Q

y t f y t t



is very small at these points (see the next graphic). was in many good simula- tions less than 10−16_.

Remark 4:

We get with yj

 

t0 y0

 



 



2 ₂

min 1

, .

In many simulation j

 

i



j

 

i , i is relative big between to collocation points (or at the edge of I if

we start with i = 1 in the sum (1)).

y t  f y t t

m

j i j i i

i

Q y t f y t t m M









   _{In Figure 6 we see the graph of}

 



 



 





2

,

j j

d t  y t f y t t

If additionally yj

 

ti f y



j

 

ti ,ti



M



m

for one (or more) i



1, 2, ,_ we get:

in example 1 for j2,k_max 10 a H

re

nd m ere a too

small m sults in a very bad approximation.

4  . min

M  Q  mM

We see that Q_min could be very small with a too

small m, but ₂ is very big here. In the graph we see

that d is very small at the collocation points Q

0.25 i

t  i

but between them d is very big. That’s the reason be-

cause we could identify with ₂ a worse approximation in any our simulations. On the other hand a big is an indicative of a too small j.

Q

min

Q

This is analogously right for instead if

with a

Q Q_min

 



 



2

1

,

a

m

a j i j i i

i

Q y  f y  











and i  t0 i h a m, a a m and an integer . Qa is an upper bound for min min

1

a

: a

Q Q Q . With _a we could

assess in all simulations the quality of an approximation and in linear regressions from

Q



ln

So we can approximate M here with the maximum of

 



 

,



j i j i i



sse on ln

 

Q₂ we

got in almost all simulations a (R squared) greater

than 0.99 (see next example). Only if all approximations have been bad, then was less than 0.99 (but we still have a dependency). If

2 R

2 R

j

y is the exact solution, then

. Because we get not only a approximation with 0

a

Q 

y   f y   at the points _i  t₀ i h a

with a2 like we do it in the next example.

Now we want to apply the result from theorem 2. Fur- thermore we will see a correlation between an approxi- mation of _ln

 

2

M and ln



sse



 

2

ln Q

in this example like we saw it before between and ln

 

sse .

0.25 0.5 0.75 1 t 5

10 15 20

d

　

t

　

j 2

kmax　 10

Qmin　 5.27296 　10　31

Q2　 35.0546

mse　 0.194116

[image:5.595.151.448.463.736.2]

m　 4

Example 2:

nitial value problem and the approxima- tio

in Figure 8. In most simulations We use the i

ns with the different parameters j, k_max and m of

example 1. If  0 than follows from rem 2 (un- der the assumptions rom this theorem):

theo f

 



 



 





 

0 0 2

1

:

ln ln e i e i 1

m

  

L t t L t t

i

O M

sse M L



  





 _     _







 



Here we get (see (6)):

 

ln

 

2

 

2

 

0 ln ln L

O M  M  L  C

We now apply a linear regression of ln

 

sse on

 

_ˆ2

ln M with the approximation

 



 



2

2

ˆ _{max , ,}

j i j i i

M  y   f y  

from M2 with    i t0 i h2,i0,1,2, ,200 (the points ning with

from Q2 begin i0). sse and O have

been calculated with the points tii100 d the summation indices i0,1,2, ,100 ).

Here is the regr

(an ession table (Table 2) (with a R

sq

a graph from (in red), the gr

uared of 0.986877).

In Figure 7 we see O0 ) a

aph of the regression function (in blue nd the regres- sion points



_ln

 

ˆ2 _,ln

 



M sse . yj

   

t0 y t0 was not

considered (this means we set  0) because he graphs of

it was very small.

Here are t O

and  0  was

an use less than 10−16_.

Generally we c

 



 



2

2

ˆ _{max ,}

a j i j i

M  y   f y  i ,

0

i t i h a

    and 0,1, ,i _ am

ation of 2 (with an integer

for an ap 1

a ) proxim M . Here we know the

wing relation: follo

2

ˆ

a a

M Q

3. Conclusions

le with which you can evaluate We defined a variab Qa

any

an approximation. In m simulations and in the exam- ples of this article we saw that we get good results with

2

a . A linear relationship between ln

 

Q₂ and

 

ln sse was shown in example 1. It is also shown that

the approximation can be used to extrapolate outside the approximation interval.

Using Theorem 2 we derive an estimate (see theorem 3). Then it is shown how to detect a too great step size using Q2. In example 2 we show that the deduced esti- mate represents a straight line (in the coordinate system with _ln

 

2

M on the x-axes and ln

 

sse on the y-

[image:6.595.97.499.475.717.2]

axes), which runs approximately parallel to the regres- sion line (it is approximately parallel because the regres- sion function is an estimation, theoretically it must be parallel because it cannot cross the upper bound line). In a research project we got analogous results in many with _{ 10}10_{,  10}15

Table 2. Regression table of on

 

Mˆ2

ln .

 

sse

ln

Estimate SE T Stat P Value

Intercept −5.27413 1.08064 −4.88058 0.0000507742 Slope 0.952356 0.0219645 43.3589 4.75173 × 10−25

against

 

Mˆ2

ln .

 

sse

[image:7.595.155.441.83.250.2]

O with 1010, 1015 and 0.

Figure 8. Graphs of

mulations, even with system

si s and higher order odes.

It is shown that 2

M (the size of the estimate) can be

ap_{proximated via} ˆ2

M , and this approximation has Q2 as upper bound. The r gression of the points e

   



ln Q2 ,ln sse



returns a slightly larger R2 than the regression with the points



_ln

 

_ˆ2 _,ln

 



M sse . As a con-

sequence, Q2 is well suited to assess, especially as you can estimate the approximation of 2

M with Q2 and in

Q2 more information is included. Moreover we can com- pare Q2 with Qmin to assess the approximation (see Fig- ure 6).

REFERENCES

[1] O. V. Vasily Second-Gene

Wavelet Collocev and C. Bowman, “ation Method for the Solution of Partiaration l Differential Equations,” Journal of Computational Phys- ics, Vol. 165, No. 2, 2000, pp. 660-693.

https://wiki.ucar.edu/download/attachments/41484400/va silyev1.pdf

doi:10.1006/jcph.2000.6638 [2] S. Bertoluzza, “Adaptive Wav

the Solution of Burgers Equa

elet Collocation Method for tion,” Transport Theory and Statistical Physics, Vol. 25, No. 3-5, 2006, pp. 339-352.

doi:10.1080/00411459608220705

[3] T. S. Carlson, J. Dockery and J. Lund, “A Sinc-Colloca- tion Method for Initial Value Problems,” Mathematics of Computation, Vol. 66, No. 217, 1997, pp. 215-235.

doi:10.1090/S0025-5718-97-00789-8

wo-Point Boundary

ntial Transformation in the [4] K. Abdella, “Numerical Solution of T

Value Problems Using Sinc Interpolation,” Proceedings of the American Conference on Applied Mathematics, Applied Mathematics in Electrical and Computer Engi- neering, 2012, pp. 157-162.

[5] A. Nurmuhammada, M. Muhammada, M. Moria and M. Sugiharab, “Double Expone

Sinc-Collocation Method for a Boundary Value Problem with Fourth-Order Ordinary Differential Equation,” Jour- nal of Computational and Applied Mathematics, Vol. 162, No. 2, 2005, pp. 32-50. doi:10.1016/j.cam.2004.09.061 [6] C. Blatter, “Wavelets—Eine Einführung,” 2nd Edition,

Vieweg, Wiesbaden, 2003.

[7] G. Strang, “Wavelets and Dilation Equations: A Brief In- troduction,” SIAM Review, Vol. 31, No. 4, 1989, pp. 614- 627. doi:10.1137/1031128

[8] Z. Shi, D. J. Kouri, G. W. Wie and D. K. Hoffman, “Generalized Symmetric Interpolating Wavelets,” Com- puter Physics Communications, Vol. 119, No. 2-3, 1999, pp. 194-218. doi:10.1016/S0010-4655(99)00185-X [9] D. L. Donoho, “Interpolating Wavelet Transforms,” Tech-

nical Report 408, Department of Statistics, Stanford Uni- versity, Stanford, 1992.