Matrices Associated with Moving Least Squares Approximation and Corresponding Inequalities

(1)

http://dx.doi.org/10.4236/apm.2015.514080

Matrices Associated with Moving

Least-Squares Approximation and

Corresponding Inequalities

Svetoslav Nenov, Tsvetelin Tsvetkov

Department of Mathematics, University of Chemical Technology and Metallurgy, Sofia, Bulgaria

Received 17 November 2015; accepted 25 December 2015; published 28 December 2015

This work is licensed under the Creative Commons Attribution International License (CC BY).

http://creativecommons.org/licenses/by/4.0/

Abstract

In this article, some properties of matrices of moving least-squares approximation have been pro- ven. The used technique is based on known inequalities for singular-values of matrices. Some in-equalities for the norm of coefficients-vector of the linear approximation have been proven.

Keywords

Moving Least-Squares Approximation, Singular-Values

1. Statement

Let us remind the definition of the moving least-squares approximation and a basic result. Let:

1.  be a bounded domain in d; 2. xi∈, i=1,,m; xi≠xj, if i≠ j;

3. f :→ be a continuous function;

4. pi:→ be continuous functions, i=1,,l. The functions

{

p1,,pl

}

are linearly independent in

 and let l be their linear span;

5. W: 0,

(

∞ →

) (

0,∞

)

be a strong positive function.

Usually, the basis in l is constructed by monomials. For example:

( )

11 d

k k

l d

p x =x x , where

(

x1, ,xd

)

=

x  ,

1, , d

k  k ∈, k1+ + kd≤ −l 1. In the case d=1, the standard basis is

{

}

1

1, ,x,xl− .

(2)

a fixed point x is the value of p*

( )

x , where p*∈l is minimizing the least-squares error

(

)

(

( )

)

2 1

m

i i

i

W p f

=

− −

∑

x x x x

among all p∈l.

The approximation is “local” if weight function W is fast decreasing as its argument tends to infinity and interpolation is achieved if W

( )

0 = ∞. So, we define additional function w: 0,

[ ) [ )

∞ → 0,∞ , such taht:

( )

(

)

(

( )

)

( )

(

)

1

, if 0 or 0 and 0 ,

0, if 0 and 0 .

r r W

W r w r

r W

 _> ₌ _{< ∞}

 = 

 ₌ _{= ∞}



Some examples of W r

( )

and w r

( )

, r≥0:

( )

2 2

2W r =e−αr exp-weight,

( )

2

Shepard weights,

W r =r−α

(

)

2 2

2

, e r McLain weight,

i

w x x =r −α

(

)

2 2

, i e r 1 see Levin s works.

w x x = α − ′

Here and below: ⋅ = ⋅ ₂ is 2-norm, ⋅₁ is 1-norm in d

; the superscript t

denotes transpose of real matrix; I is the identity matrix.

We introduce the notations:

( )

1 1 2 1 1 1

1 2 2 2 2 2

1 2

, ,

l l

m m l m m

p p p a

E

p p p a

   

   

=_ _ =_ _

   

  _ _

 

x x x

a

x x x

 

   



(

)

(

)

(

)

( )

1 1

2 2

, 0 0

0 , 0

2 , .

0 0 , m l

w p

D

w p

   

   

= _ _ =_ _

  _ _

  _ _

 

x x x

c

x x x

 

   



Through the article, we assume the following conditions (H1): (H1.1) 1∈l;

(H1.2) 1≤ ≤l m; (H1.3) rank

( )

Et =l; (H1.4) w is smooth function.

Theorem 1.1. (see [2]): Let the conditions (H1) hold true. Then:

1. The matrix E D Et −1 is non-singular;

2. The approximation defined by the moving least-squares method is

( )

1

ˆ m _,

i i i

L f a f

=

∑

x (1)

where

(

)

1

1 1

0 and 0 .

t

A A D E E D E− − −

= =

a c (2)

3. If w

(

xi−xi

)

=0 for all i=1,,m, then the approximation is interpolatory.

(3)

receive (for convenience we suppose d=1 and standard polynomial basis, see [5]):

( )

( )( )

( )

*

( )

1

ˆ ₁ m _,

i i

f x L f x f x p x _∞ a =

 

− ≤ − _ + _



∑

 (3) and moreover (C =const.)

( )

*

( )

1

{

( )1

_{( )}

}

max l : .

l

f x p x Ch+ f + x x ∞

− ≤ ∈ (4)

It follows from (3) and (4) that the error of moving least-squares approximation is upper-bounded from the 2- norm of coefficients of approximation ( a₁≤ m a ₂). That is why the goal in this short note is to discuss a method for majorization in the form

(

)

2 ≤Mexp N − i .

a x x

Here the constants M and N depend on singular values of matrix t

E , and numbers m and l (see Section 3). In Section 2, some properties of matrices associated with approximation (symmetry, positive semi-definiteness, and norm majorization by min

( )

t

E

σ and max

( )

t

E

σ ) are proven.

The main result in Section 3 is formulated in the case of exp-moving least-squares approximation, but it is not hard to receive analogous results in the different cases: Backus-Gilbert wight functions, McLain wight functions, etc.

2. Some Auxiliary Lemmas

Definition 2.1. We will call the matrices

(

)

1

1 1

1 0 and 2 1

t t t

A =A E =D E E D E− − − E A =A −I

1

A-matrix and A2-matrix of the approximation Lˆ, respectively.

Lemma 2.1. Let the conditions (H1) hold true. Then, the matrices A D1 1

−

and A D2 1

−

are symmetric.

Proof. Direct calculation of the corresponding transpose matrices. Lemma 2.2. Let the conditions (H1) hold true.

Then:

1. All eigenvalues of A1 are 1 and 0 with geometric multiplicity l and m l− , respectively;

2. All eigenvalues of A2 are 0 and −1 with geometric multiplicity l and m l− , respectively.

Proof. Part 1: We will prove that the dimension of the null-space dim null

(

( )

A2

)

is at least l.

Using the definition of 2 1

(

1

)

1

t t

A =D E E D E− − − E −I , we receive

(

)(

)

1

1 1

2 0.

t t t t t

E A = E D E− E D E− − E −E =

Hence,

( )

2

( )

im A ⊆null Et .

Using (H1.3), Et is

(

l m×

)

-matrix with maximal rank l (l<m). Therefore, dim null

(

( )

Et

)

= −m l. More- over, dim im

(

( )

A2

)

= −m dim null

(

( )

A2

)

. That is why m−dim null

(

( )

A2

)

≤ −m l or l≤dim null

(

( )

A2

)

.

Part 2: We will prove that −1 is eigenvalue of A2 with geometric multiplicity m l− , or the system

2 1 0

Aη= − ⇔η Aη=

has m l− linearly independent solutions. Obviously the systems

(

)

1

1 1

1 0

t t

Aη=D E E D E− − − Eη= (5) and

0

t

(4)

are equivalent. Indeed, if η0 is a solution of (5), then

(

)

1

(

)

1

1 1 1 1

0 0

0

0 0

0,

t t t t t

t

D E E D E E E D E E D E E

E

− −

− − _{= ⇒} − − ₌

⇒ =

η η

η i.e. η0 is solution of (6).

On the other hand, if η0 is a solution of (6), then

(

)

(

₁ ₁ 1

)

(

₁

(

₁

)

1

)

(

)

0 0 0,

t t t t

D E E D E− − − E η = D E E D E− − − Eη = i.e. η0 is solution of (5). Therefore

( )

(

1

)

(

( )

)

dim im dim im t .

A = E = −m l

Part 3: It follows from parts 1 and 2 of the proof that 0 is an eigenvalue of A2 with multiplicity exactly l and

1

− is an eigenvalue of A2 with multiplicity exactly m l− .

It remains to prove that 1 is eigenvalue of A1 with multiplicity at least l, but this is analogous to the proven

part 1 or it follows dirctly from the definition of A1=A2+I.

The following two results are proven in [6].

Theorem 2.1 (see [6], Theorem 2.2): Suppose U, V are

(

m m×

)

Hermitian matrices and either U or V is positive semi-definite. Let

( )

1 U m U , 1 V m V

λ ≥ ≥_ λ λ ≥ ≥_ λ

denote the eigenvalues of U and V, respectively. Let:

1. π

( )

U is the number of positive eigenvalues of U; 2. ν

( )

U is the nubver of negative eigenvalues of U;

3. ξ

( )

U is the number of zero eigenvalues of U. Then:

1. If 1≤ ≤k π

( )

U , then

( )

{

1

}

( )

{

( )

}

1

min i k i k min i m k i .

i k λ U λ+ − V λ VU k i m λ U λ + − V

≤ ≤ ≥ ≥ ≤ ≤

2. If π

( )

U <k≤ −m ν

( )

U , then

( )

0.

k VU

λ =

3. If m−ν

( )

U < ≤k m, then

( )

{

}

( )

{

( )

1

( )

}

1

min i m i k k min i i k .

i k λ U λ + − V λ VU k i m λ U λ+ − V

≤ ≤ ≥ ≥ ≤ ≤

Corollary 2.1. (see [6], Corollary 2.4): Suppose U, V are

(

m m×

)

Hermitian positive definite matrices. Then for any 1≤ ≤k m

( ) ( )

( )

( ) ( )

1 U 1 V k VU m U m V .

λ λ ≥λ ≥λ λ

As a result of Lemma 2.1, Lemma 2.2 and Theorem 2.1, we may prove the following lemma. Lemma 2.3. Let the conditions (H1) hold true.

1. Then A D1 1

−

and A D2 1

−

− are symmetric positive semi-definite matrices. 2. The following inequality hods true

(

1

)

_{( )}

max 1

min

1 .

A D

D λ

λ − _≤ Proof. (1) We apply Theorem 2.1, where

1 1

, .

U =D V =A D−

(5)

( )

U

( )

U 0

µ =ξ = , if x≠xi, i=1,,m.

The matrix V is symmetric (see Lemma 2.1).

From the cited theorem, for any index k

(

k=1,,m=π

( )

U

)

we have

( )

(

1

)

( )

{

( )

}

1 1

1

min .

k k k i m i k i k

A A D D VU U V

λ λ − λ λ λ

+ − ≤ ≤

= = ≤

In particular, if k=m:

( )

1

{

( ) ( )

}

1min .

m i i

i m

A U V

λ λ λ

≤ ≤

≤ (7)

Let us suppose that there exists index i0

(

i0=1,,m−1

)

such that

( )

₀

( )

₀

( )

1 V i V 0 i 1 V m V .

λ ≥≥λ ≥ >λ + ≥≥λ (8) It fowollws from (8) and positive definiteness of U, that

( ) ( )

{

}

01

( )

01

( )

1min≤ ≤i m λi U λi V ≤λi+ U λi+ V <0.

Therefore (see (7)), λm

( )

A1 <0. This contradiction (see Lemma 2.2) proves that the matrix 1 1

A D− is positive semi-definite.

If we set U=D, V A D2 1

−

= − then by analogical arguments, we see that the matrix A D2 1

−

− is positive semi-definite.

(2) From the first statement of Lemma 2.3, V A D1 1

−

= is positive semi-definite. Therefore (see Corollary 2.1 and Lemma 2.2):

( )

1

( )

{

( ) ( ) ( ) ( )

}

1≥λk A =λk VU ≥max λm U λk V ,λm V λk U

for all k=1,,m. Moreover, all numbers λk

( )

U , λk

( )

V are non-negative and

( )

max D 1 U m U min D , 1 V m V .

λ =λ ≥ ≥_ λ =λ λ ≥ ≥_ λ

Therefore

( ) ( ) ( ) ( )

{

1 1

}

1 max≥ λm U λ V ,λm V λ U , or

(

1

)

( )

_{( )}

max 1 1

min

1 1

.

m

A D V

U D

λ λ

− ₌ _≤ ₌

□

In the following, we will need some results related to inequalities for singular values. So, we will list some necessary inequalities in the next lemma.

Lemma 2.4. (see [7][8]): Let U be an

(

d1×d2

)

-matrix, V be an

(

d3×d4

)

-matrix.

Then:

( )

max max max

2σ UV ≤σ U σ V , (9)

( )

1

_{( )}

max 1 2

min

1

, if , det 0,

U d d U

U σ

σ

− ₌ ₌ _≠

(10)

( )

max V min U max UV , if d1 d2 d3,

σ σ ≤σ ≥ = (11)

( )

max U min V max UV , ifd4 d3 d2.

σ σ ≤σ ≥ = (12)

If d1=d2 and U is Hermitian matrix, then U =σmax

( )

U , σi

( )

U = λi

( )

U , i=1,,d1.

Lemma 2.5. Let the conditions (H1) hold true and let x≠xi, i=1,,m.

Then:

( )

1

min

1 ,

A D

D λ − _≤

(6)

( )

1

(

1

)

max A1 min D max A D1 ,

σ σ − _≤σ −

(14)

( )

max 1

min

1 A D .

D σ σ

≤ ≤ (15)

Proof. The matrix A D1 1

−

is simmetric and positive semi-definite (see Lemma 2.3 (1)). Using the second statement of Lemma 2.3 and Lemma 2.4, we receive

(

)

(

)

_{( )}

1 1 1

1 max 1 max 1

min

1 .

A D A D A D

D

σ λ

λ

− ₌ − ₌ − _≤

The inequality (14) follows from (12) (d4=d3=m).

From (14) and (10), we receive

( )

(

)

( )

1

max 1 max

max 1 ₁

min min

.

A D D

A

D D

σ _σ

σ

σ σ

−

≤ =

Therefore, the equality A1 = σmax

( )

A1 implies the right inequality in (15).

Using 1

t t

E =E A and inequality (9), we receive

( )

max max max 1 ,

t t

E E A

σ ≤σ σ

or 1≤σmax

( )

A1 = A1 2, i.e. the left inequality in (15).

The lemma has been proved. □

3. An Inequality for the Norm of Approximation Coefficients

We will use the following hypotheses (H2): (H2.1) The hypotheses (H1) hold true; (H2.2) d=1, x1<<xm;

(H2.3) The map c is 1

C -smooth in

[

x x1, m

]

;

(H2.4) w x

(

−x_i

)

=exp

(

α

(

x−x_i

)

2

)

, i=1,,m. Theorem 3.1. Let the following conditions hold true: 1. Hypotheses (H2);

2. Let x∈

[

x x1, m

]

be a fixed point;

3. The index k0∈

{

1,,m

}

is choosen such that

{

}

0 min : 1, , .

k i

x−x = x−x i=  m

Then, there exist constants M M1, 2>0 such that

( )

x ≤

(

( )

xk0 +M x1 −xk0

)

exp

(

M x2 −xk0

)

.

a a

Proof. Part 1: Let

(

)

(

)

(

)

1

2

2 0 0

0 2 0

,

0 0 2 m

x x

x x H

x x α

α

 − 

 ₋ 

 

=  

 

 ₋ 

 

 

  

 then

1

d d

, .

d d

D D

HD HD

x x

−

(7)

We have (obviously D=D x

( )

, H=H x

( )

, and c=c

( )

x )

( )

₍

₎

(

)

(

)

(

)

(

)

(

)

(

)

(

)

(

) (

)(

)

1 1 1

1 1 1 1 1 1

1 1 1 1

1 1 1 1 1 1 1 1

1 1

1 1 1 1 1

d d

d d d

d d

t

t t t

t t t t t

t t t

x

D E E D E

x x

D E E D E D E E D E D E E D E

x x x

HD E E D E D E E D E E D E E D E D E E D E x

H D E E D E E HD E E D E D E α − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − =     =_ _ + _ _ +         = − + _− _ _ _ +     = − + + a c

c c c

a c

(

)

(

)

(

)

(

)

1 1

1 1 1 1

2 0 d d d d d . d t

t t t

E D E x

D E E D E E I H D E E D E x

A H A x − − − − − − − − = − + = + c a c a c

Therefore, the function a

( )

x satisfies the differential equation

( )

2 0 d d . d d x

A H A

x = + x

a

a c (16)

Part 2: Obviously

(

)

(

)

2 1 1 1 .

A H = A −I H ≤ A + H

It follows from (15) that

( )

max 1 min . D A D σ σ ≤

Here σmax

( )

D ≤2 exp

( )

αr2 , r=xm−x1, and σmin

( )

D ≥2. Hence

( )

2

1 exp .

A ≤ αr

For the norm of diagonal matrix H, we receive

2 .

H ≤ αr

Therefore A H2 ≤M2, where

( )

(

2

)

2 2 1 exp .

M = αr + αr

We will use Lemma 2.4 to obtain the norm of A0.

Obviously, 0 1

t

A E =A . Therefore by (12) (m=d4≥d3=l), we have

( )

max 0 min max 1 ,

t

A E A

σ σ ≤σ

i.e.

( )

max

( )

0 min min 1 . t D A D E σ σ σ ≤

Therefore, if we set

( )

2

11 min t M M E σ

= , then A0 ≤M1.

Let the constant M12 be choosen such that

( )

12

[

1

]

d

, ,

(8)

and let M1=M M11 12.

Part 3: On the end, we have only to apply Lemma 4.1 form [9] to the Equation (16):

( )

(

)

(

)

0

0 0

0 0 0

0 2

1 2

d

d exp d

d exp . k k x x k x x

k k k

x x A x A H x

x

x M x x M x x

    ≤ +     ≤ + − −

∫

a a c

a

Remark 3.1. Let the hypotheses (H2) hold true and let moreover

( )

1

1 1, 2 , , , 1.

l l

p x = p x =x p x =x− l≥

In such a case, we may replace the differentiation of vector-fuction

( )

1 2 1 1 l l p x

p x x

x

p x x−

            =_ _{ }= _      _{  }   c   by left-multiplication:

( )

(

)

(

)

( )

2 2 2 3 1 2

0 0 0 0 0 0 0

1

1 1 0 0 0 0 0

2 0 2 0 0 0 0

d

3 0 0 3 0 0 0 .

d

2 0 0 0 2 0 0

1 0 0 0 0 1 0

l l l l x x x x x x x x

l x l

x

l x l

− − − −                             =  =  = ∂               − −    _ _    ₋   ₋      c c              

The singular values of the matrix ∂ are: 0,1,,l−1. Therefore ∂ = l−1. That is why, we may chose

(

)

{

( )

}

1

22

1

1 max max .

m

i i l x x x

M l p x

≤ ≤ < <

= −

Additionally, if we supose x1 ≤ xm , then

( )

1

max , 1, , .

m

i i m

x< <x x p x p x i l

= = _

Therefore, in such a case:

(

)

{

( )

}

22

1

1 max i m .

i l

M l p x

≤ ≤

= −

If we suppose − ≤ ≤ ≤1 x1 x xm ≤1, then obviously, we may set

22 1.

M = l−

References

[1] Alexa, M., Behr, J., Cohen-Or, D., Fleishman, S., Levin, D. and Silva, C.T. Point-Set Surfaces.

http://www.math.tau.ac.il/~levin/

[2] Levin, D. The Approximation Power of Moving Least-Squares. http://www.math.tau.ac.il/~levin/ [3] Levin, D. Mesh-Independent Surface Interpolation. http://www.math.tau.ac.il/~levin/

[4] Levin, D. (1999) Stable Integration Rules with Scattered Integration Points. Journal of Computational and Applied

Mathematics, 112, 181-187. http://www.math.tau.ac.il/~levin/

(9)

[5] Fasshauer, G. (2003) Multivariate Meshfree Approximation. http://www.math.iit.edu/~fass/603_ch7.pdf

[6] Lu, L.-Z. and Pearce, C.E.M. (2000) Some New Bounds for Singular Values and Eigenvalues of Matrix Products.

An-nals of Operations Research, 98, 141-148. http://dx.doi.org/10.1023/A:1019200322441

[7] Merikoski, J.K. (2004) Ravinder Kumar, Inequalities for Spreads of Matrix Sums and Products. Applied Mathematics

E-Notes, 4, 150-159.

[8] Jabbari, F. (2015) Linear System Theory II. Chapter 3: Eigenvalue, Singular Values, Pseudo-Inverse. The Henry Sa-mueli School of Engineering, University of California, Irvine. http://gram.eng.uci.edu/~fjabbari/me270b/me270b.html [9] Hartman, P. (2002) Ordinary Differential Equations. Second Edition, SIAM, Philadelphia, Pennsylvania, United States.