Radial Basis Functions on Scattered Data
5.1 Nonsingularity of interpolation matrices
5.2.2 The reproducing kernel (semi-)Hilbert space
There is a close connection between the radial basis functions we study in this book and the notion of reproducing kernels. All of the radial basis functions we have mentioned give rise to such reproducing kernels with respect to some Hilbert space and/or semi-Hilbert spaces, and, more generally, the conditionally positive definite functions which occurred in the previous section give rise to reproducing kernels and Hilbert spaces and/or semi-Hilbert spaces. A semi- Hilbert space can also be made by considering a Hilbert space which is addi- tionally equipped with a semi-norm and semi-inner product as defined above, where it is usual to require that the semi-norm of any element of the space is bounded above by a fixed constant multiple of its norm.
5.2 Convergence analysis 109
Definition 5.2. Let X be a Hilbert space or a semi-Hilbert space of real-
valued functions onRn , equipped with an inner product or a semi-inner product
(·, ·) , respectively. A reproducing kernel for ( X ,(·, ·)) is a function k(·, ·): Rn ×Rn → R such that for any element f ∈ X we have the pointwise equality
with respect to x
( f ,k(·, x )) = f ( x ), x ∈ Rn.
In particular, k(·, x ) ∈ X for all x ∈ Rn.
Standard properties of the reproducing kernel are that it is Hermitian and non- negative for all arguments from X (see, for instance, Cheney and Light, 1999). There are well-known necessary and sufficient conditions known for the linear space X to be a reproducing kernel (semi-)Hilbert space, i.e. for the existence of such a reproducing kernel k within the Hilbert space setting. A
classical result (Saitoh, 1988, or Yosida, 1968, for instance) states that a repro- ducing kernel exists if and only if the point evaluation operator is a bounded operator on the (semi-)Hilbert space. If we are in a Hilbert space setting, the reproducing kernel is, incidentally, unique. In our case, a reproducing kernel exists for the subspace X ˆ of all functions g that vanish on the K -unisolvent subset ˆ because the condition |g( x )| = O(g X ) = O(g∗) from the pre- vious subsection gives boundedness, i.e. continuity, for the linear operator of point evaluation in that subspace ˆ X .
In our example above, the boundedness of the point evaluation functional amounts to the uniform boundedness of any function g that is in X =
D−k L2(Rn) and vanishes on ˆ, by a fixed g-independent multiple of its semi-
norm g∗. That this is so is a consequence of the two following observations, but there are some extra conditions on the domain .
The first one is related to the Sobolev inequality. We have taken this particular formulation from the standard reference (Brenner and Scott, 1994, p. 32) and restricted it again to the case that interests us. We also recall, and shall use often from now on, the special notation for the semi-norm
| f |2k , :=
|α|=k α∈Zn + k ! α! | D α f ( x )|2 d xas is usual elsewhere in the – especially finite element – literature, and, as a further notational simplification, | f |k := | f |k ,Rn. Accordingly, we shall some-
times use the superspace D−k L2() of X = D−k L2(Rn) that contains all f
with finite semi-norm when restricted to the domain . If f is only defined on
110 5. Radial basis functions on scattered data
Sobolev inequality. Let be a domain in n-dimensional Euclidean space with Lipschitz-continuous boundary and let k > n/2 be an integer. Then there is a constant C such that for any f ∈ D−k L2(Rn) the pointwise inequality
(5.5) | f ( x )| ≤ C
k
j =0
| f | j,
holds on the domain , where C only depends on the domain, the dimension of the underlying real space and on k.
The second one is the Bramble–Hilbert lemma (Bramble and Hilbert, 1971). We still assume that is a domain in Rn and that it has a Lipschitz-continuous
boundary ∂. Moreover, from now on it should additionally satisfy an interior
cone condition which means that there exists a vector ξ ( x ) ∈ Rn of unit length
for each x ∈ such that for a fixed positive ˆr and ϑ > 0,
⊃ { x + λη|η ∈ Rn, η = r , ηˆ · ξ ( x ) ≥ cosϑ, 0 ≤ λ ≤ 1}.
Note that the radius and angle are fixed but arbitrary otherwise, i.e. they may depend on the domain. In fact, Bezhaev and Vasilenko (2001) state that the cone condition is superfluous if we require a Lipschitz-continuous boundary of the domain. We recall the definition of H k from the end of the fourth chapter.
Bramble–Hilbert lemma. Let ⊂ Rn be a domain that satisfies an interior
cone condition and is contained in a ball which has diameter ρ. Let F be a linear functional on the Sobolev space H k () such that F (q) = 0 for all q ∈ Pk −1
n .
Suppose that the inequality
|F (u)| ≤ C 1
k
j =0
ρ j −n/2|u| j,
holds for u with finite semi-norm |u| j, , j ≤ k, with a positive constant C 1 that
is independent of u and ρ. Then it is true that
|F (u)| ≤ C 2
ρk −n/2|u|k ,
,where C 2 does not depend on u or ρ.
For our application of the Bramble–Hilbert lemma we let k > n2 be a positive integer and the operator F map a continuous function f to the value of the interpolation error by polynomial interpolation on ˆ with polynomials of total degree less than k . Thus, it is zero if f is already such a polynomial, as a result of ˆ being unisolvent. Therefore the first assumption of the Bramble– Hilbert lemma is satisfied. The second assumption is a consequence of the Sobolev inequality, where it is also important that f vanishes on ˆ, since the
5.2 Convergence analysis 111
interpolating polynomial for f is zero if f ∈ X ˆ , as is the case for our setting. Therefore, the Bramble–Hilbert lemma implies
| f ( x )| ≤ C | f |k , ≤ C f ∗,
as required, the constant C being independent of f . To this end, we also recall that the polynomial interpolant of degreek − 1 itself vanishes when differenti- atedk times, which is why the polynomial parts of the interpolation error expres- sion disappear in both the middle and the right-hand side of the above inequality. Now that we know about the existence of a reproducing kernel in our partic- ular example (as a special case of the general radial basis function setting), we wish to identify it explicitly in the sequel. Once more, we begin here with the general case and, becoming progressively more specific, restrict to our examples for the purpose of illustrating the general case later on.
The reproducing kernel has an especially simple form in our setting. In order to derive this form, we may let, because of the condition of unisolvency of ˆ, the functions pξ , ξ ∈ ˆ , be polynomial Lagrange functions, i.e. such that pξ
span K and satisfy
pξ (ζ ) = δξ ζ , ξ , ζ ∈ .ˆ
Since we are looking for a reproducing kernel on a subspace, our reproducing kernel k has a particular, K -dependent form. Specifically, for our application, we can express the property of the reproducing kernel by the identity
(5.6) f ( x ) =
f , φ(· − x ) −
ξ ∈ˆ
pξ ( x )φ(· − ξ )
∗
, x ∈ Rn, f ∈ X .ˆ
We have therefore the reproducing kernel of the form
k( y, x ) = φ( y − x ) −
ξ ∈ˆ
pξ ( x )φ( y − ξ )
and φ is a function Rn → R which is, notably, not necessarily itself in X . This
is why φ does not appear itself on the right-hand side of the expression above but in the shape of a certain linear combination involving the pξ . This particular shape also includes the property that it vanishes on ˆ. We will come to this in a moment. In the case that the function f does not vanish on ˆ, then we subtract from it the finite sum of multivariate polynomials
ξ ∈ˆ
f (ξ ) pξ ,
so that the difference does, and henceforth use instead the difference as follows:
(5.7) f −
ξ ∈ˆ
112 5. Radial basis functions on scattered data
The same principle was applied to the shift of φ above to guarantee that the kernel kvanishes on ˆ. If the null-space K of the semi-inner product and of the semi-norm is trivial, no such operation is required – the sums in the preceding two displays remain empty for K = {0}. The fact that the reproducing kernel
k( y, x ) is a function of the difference of two arguments (that is, k( y, x ) =
˜
k( y− x )) is a simple consequence of the shift-invariance of the space X (see the interesting paper by Schaback, 1997, on these issues), because any invariance properties such as shift- or rotational invariance that hold for the space X are immediately carried over to properties of the reproducing kernel k through Definition 5.2. The proofs of these facts are easy and we omit them because we do not need them any further in this text.
Most importantly, however, we require that the right-hand side expression of the inner product in (5.6) above is in X . We make the hypothesis that φ is such that, whenever we form a linear combination of shifts of φ, whose coefficients are such that they give zero when summed against any element of K , then that linear combination must be in X :
ξ ∈
λξ φ(· − ξ ) ∈ X if
ξ ∈
λξ q(ξ ) = 0∀q∈K .
We will show later that for the X which we have given as an example above, this hypothesis is fulfilled.
For the semi-inner product in the above display (5.6), the sum over the coefficients of the linear combination above against any p ∈ K is indeed zero:
p( x ) −
ξ ∈ˆ
pξ ( x ) p(ξ ) = p( x ) − p( x ) = 0,
because of the Lagrange conditions on the basis elements pξ , ξ ∈ . Therefore,
assuming our above hypothesis on φ being in place for the rest of this section now, the right-hand side in the inner product in the display (5.6) is in the re- quired space X . Moreover, since f ( x ) vanishes for x = ζ ∈ ˆ on the left-hand side in the above identity (5.6), so must the expression
φ(· − ζ ) −
ξ ∈ˆ
pξ (ζ )φ(· − ξ )
on the right-hand side, which it does because pξ (ζ ) = δξ ζ , and therefore we
have verified that it is, in particular, in ˆ X .
We may circumvent the above hypothesis on φ which may appear strange at this point, because, as we shall see, it is fulfilled for all our radial basis functions (4.4). More specifically, given a fixed basis function φ, we shall construct a (semi-)Hilbert space X whose reproducing kernel has the required form (5.6).
5.2 Convergence analysis 113
This can be done as follows. In the case of all the radial basis functionsφ(·) we use in this chapter, the constructed X is the same as the above X we started with as we shall see shortly in this section. We follow especially the analysis of Schaback (1999).
To begin with, we define now a seemingly new semi-inner product for any two functions f : → R and g: → R that are finite linear combinations of
shifts of a given general, continuous function φ: Rn → R,
f =
ξ ∈1 λξ φ(· − ξ ) and g =
ξ ∈2 µξ φ(· − ξ ),where1 and2 are arbitrary finite subsets of ; those sets need not (but usually do) agree. The functionφ is required to be continuous and conditionally positive definite of order k , so that we let K = Pk −1
n . Therefore we have an additional
condition on the coefficients in order to define the semi-inner product which follows, that is we require that for the given finite-dimensional linear space K , the coefficient sequences must satisfy the conditions
ξ ∈1 λξ p(ξ ) = 0 and
ξ ∈2 µξ p(ξ ) = 0 for all p ∈ K . Moreover, we let LK () denote the linear
space of all functionals of the form
λ: h →
ξ ∈1
λξ h(ξ ),
with the aforementioned property of the coefficients and with any finite set 1. Its Hilbert space completion will beLK (). Soλ is a linear functional onC ().
Now define the semi-inner product for the above two functions, (5.8) ( f , g)∗ := (λ, µ)∗ :=
ξ ∈1
ζ ∈2 λξ µζ φ(ξ − ζ ),where we identify functions f and g with linear functionals λ and µ in an obvious way defined through their coefficients. So functionalsλ and functions
f are related via λ → f = λ x φ( x − ·), where the superscript x refers to the variable with respect to which we evaluate the functional. Note that we can always come back from one such functional to a function in x by applying it in this way to φ( x − ·).
As a consequence of the conditional positive definiteness of φ, this semi- inner product is positive semi-definite as required. Further, let ˆ still contain a
K -unisolvent set and define the modified point evaluation functional δ( x ) by
δ( x ) f = f ( x ) −
ξ ∈ˆ
114 5. Radial basis functions on scattered data
Here, pξ are Lagrange polynomials. Then the modified point evaluation func-
tional is in LK () for every x ∈ , which the ordinary Dirac function δ(· − x ),
i.e. the usual function evaluation at a point x , is not , as it does not annihilate the polynomials in the space K unless K = (0). Now letX be the completion of the range of
( LK (), δ( x ))∗, x ∈ ,
that is in short, but very useful, notation the space generated by all functions from LK () with the semi-inner product taken with δ( x ), always modulo K . This is a Hilbert space of functions defined for every x ∈ .
In particular, for all µ and λ from LK (), we get µ((λ, δ( x ))∗) = (λ, µ)∗, due to our definition of the inner product in (5.8). Here it is also relevant that the polynomials which appear in the definition of the modified point evaluation functional are annihilated by the semi-inner product
(δ( x ), λ)∗ = (δ( x ), f )∗ = δ( x ) f = f ( x ) −
ξ ∈ˆ
pξ ( x ) f (ξ ),
to which then µ has to be applied in the usual way. Thereby, a semi-inner product and a semi-norm are also defined on the aforementioned range X . All
functions in X vanish on our K -unisolvent point set ˆ. To form X , take the
direct sum of K and X . That is the so-called ‘native space’. Further, let f ∈ X . Thus there exists an f -dependent functional λ such that
f ( x ) = (λ, δ( x ))∗ = ( f ,(δ(·), δ( x ))∗)∗,
the second equality in this display being a consequence of the definition of ( f , g)∗ above. That display provides the reproducing kernel property. Indeed, it
is exactly the same identity as before, because, according to our initial definition of the bilinear (·,·)∗ as a semi-inner product on LK (),
(δ(·), δ( x ))∗ = δ(t ·)δ z( x )φ(t − z),
where the superscript fixes the argument with respect to which the linear func- tional is applied. The above is the same as
δ(t ·)
φ(t − x ) −
ξ ∈ pξ ( x )φ(t − ξ )
= φ(· − x ) −
ξ ∈ pξ ( x )φ(· − ξ ) −
ζ ∈ pζ (· )φ(ζ − x ) +
ζ ∈ pζ (·)
ξ ∈ pξ ( x )φ(ξ − ζ ).5.2 Convergence analysis 115
When inserted into the semi-inner product, the last two terms from the display disappear, since pζ (· ) ∈ K and any such arguments in the kernel are annihi-
lated. Therefore the two expressions for the reproducing kernel are the same.