Radial Basis Functions on Scattered Data
5.3 Norm estimates and condition numbers of interpolation matrices
5.3.2 Bounds on eigenvalues
Surprisingly, several of the results below give bounds on the aforementioned lowest eigenvalues that are independent of the number of centres; instead they depend solely on the smallest distance between adjacent centres. The latter quantity divided by two is termed the ‘separation radius’ q of the data sites
ξ , and this will appear several times in the rest of this chapter. On the other hand, it is not surprising that the bounds tend to zero as that distance goes to zero as well. They must do, because eventually, i.e. for coalescing points, the interpolation matrix must become singular, as it will then have at least two identical rows and columns.
Lemma 5.11. Let φ(·):Rn → Rbe strictly conditionally positive definite of
order zero or one and, in the latter case, suppose further that φ(0)is nonpositive. If for all d = (d ξ )ξ ∈ ∈ R ,
ξ ∈
ζ ∈ d ξ d ζ φ(ξ − ζ ) ≥ ϑ
ξ ∈ |d ξ |2 = ϑ × d 2,138 5. Radial basis functions on scattered data
and ϑ is positive, then the inverse of the invertible interpolation matrix
{
φ(
ξ−
ζ
)}
ξ,ζ∈
is bounded above in the Euclidean matrix norm by ϑ−
1.The proof of this lemma is simple; it relies on exactly the same arguments as we have applied in Chapters 2 and 5 (e.g. Theorem 5.1) of this book when we have shown nonsingularity in the above cases – in this case here, we only need to take ϑ
×
d
2 as the uniform lower bounds on the positive or conditionally positive quadratic forms that occur there. In fact, the proof is entirely trivial for order zero, because the definition of eigenvalues immediately provides the desired result which is then standard for positive definite quadratic forms. This lemma is central to this section because the radial basis functions which are conditionally positive definite of order one or zero provide the most important examples. Therefore we now wish to apply this auxiliary result to our radial basis functions, notably to the negative of the ubiquitous multiquadric function, that is φ(r )= −√
r 2+
c2.Thus, for any radial basis function φ
∈
C ([0,∞
)) to be such that the expression−
dt d φ(√
t ), t > 0,is completely monotonic, we note the following straightforward condition. It is necessary and sufficient that there exists a nondecreasing measure µ such that for some positive ε
(5.19) φ(r )
=
φ(0)−
∞
0 1−
e−
r 2t t d µ(t ), r≥
0, ∞
ε d µ(t ) t <∞
,
ε 0 d µ(t ) > 0.For the sufficiency of the form in the display above, we only need to differentiate once and apply the Bernstein representation theorem which shows that the above indeed is completely monotonic when differentiated once.
Further, the above representation is necessary: we outline the argument as follows. For showing that, we apply once again the Bernstein representation theorem to the derivative of φ in the display before (5.19) and integrate once, the first of the two extra conditions in (5.19) being responsible for the existence of the integral after the integration of the exponential function therein. We also know that it is necessary that the measure µ is nondecreasing and nonconstant, and as a consequence it is in particular necessary that for some positive ε
ε0
d µ(t ) > 0,
5.3 Norm estimates and condition numbers 139
We recall as an important example that thus the above representation (5.19) applies to the multiquadric function; it is conditionally positive definite of order one if augmented with a negative sign, i.e. φ(r )
= −√
r 2+
c2, c≥
0. We wish to use Lemma 5.11 and (5.19) which imply that it suffices, in order to establish a lower bound ϑ for the smallest eigenvalue in modulus, simply to find a lower bound on the positive definite quadratic form with kernele−
r 2t , i.e. to prove an inequality of the form(5.20)
ξ∈
ζ∈
d ξ d ζ e−
t
ξ−
ζ
2≥
ϑ(t )
ξ∈
|
d ξ|
2=
ϑ(t )
d
2.Then, ϑ in Lemma 5.11 can be taken from (5.19) and (5.20) by integration against the measure
ϑ
=
∞
0
ϑ(t )
t d µ(t ) > 0,
because this is how the radial basis function is defined using the representation (5.19). When the above integral exists, the (with respect to r ) constant terms
φ(0) and 1/t disappear because we require the sum of the components of the coefficient vector
{
d ξ}
to vanish. It is therefore, in the next step of the analysisof the condition number, necessary to find a suitable ϑ(t ) for (5.20). This we do not do here in detail, but we state without proof that a suitable ϑ(t ) is provided by Narcowich and Ward (1991) and what its value is. It is their work that was instructive for the presentation here. They define, for a certain constant δn
which depends on the dimension of the underlying space but whose value is immaterial otherwise, and for the aforementioned separation radius q of points
ξ and ζ from ,
(5.21) q
=
12 minξ
=
ζ
ξ−
ζ
,the quantity ϑ(t ) for the desired estimate (5.20) as a constant multiple of (5.22) t
−
n2 q−
n e−
δ2
n/(qt ), t > 0.
Note that we are leaving out a constant factor which only depends on the dimension, our bounds being dimension-dependent in all sorts of ways anyhow; what is most interesting to us is always the asymptotic behaviour with respect to parameters of the radial basis function and possibly the number and spacing of centres.
This gives a ϑ through integration as above, whose reciprocal bounds the required 2-matrix-norm by integrating (5.22) with respect tod µ. Further, it is easy to bound the 2-condition number now, since one can bound the 2-norm
140 5. Radial basis functions on scattered data
of the matrix A = {φ(ξ − ζ )}ξ,ζ ∈ itself for instance by its Frobenius norm
AF :=
ξ,ζ ∈φ(ξ − ζ )2.
This, in turn, can for example be bounded above by the following product which uses the cardinality of the centre-set:
|| × max
ξ,ζ ∈ |φ(ξ − ζ )|.
Here, || is still the notation for the finite cardinality of the set of centres
which we use.
We give a few examples of the quantitative outcome of the analysis when the radial basis function is the multiquadric function and its parameter c is set to be one. Then the measure d µ is defined by the weight function that uses the standard -function (Abramowitz and Stegun, 1972),
(5.23) d µ(t ) = e
−c2t t α−1
(α) dt .
We let, for the multiquadric function, c = 1, n = 2 and in order to simplify notation further
p := q2/(1 +
1 + q2/4) ∼ q 22 , q → 0,
for small q. Then the Euclidean matrix norm of the inverse of the interpolation matrix is at most 24e(12/ p)/ p, as follows from (5.22) and the above. This is asymptotically
48
q2 e
(24/q2), q → 0. For n = 3 one gets 36e(16/ p)/ p and asymptotically
72
q2 e
(32/q2),
q → 0.
For q → ∞ one gets in two and three dimensions the asymptotic bounds 12/q
and 18/q, respectively. Indeed, one expects quickly – here exponentially – growing bounds for q → 0 because naturally the matrix becomes singular for smaller q and the speed of the norm’s growth is a result of the smoothness of φ, the multiquadric being infinitely smooth. Conversely, for large q, one expects to get bounds that are similar to the bound for c = 0, because in that case, c
5.3 Norm estimates and condition numbers 141
an upper bound that is a constant multiple of
√
n/q on the size of the inverse of the interpolation matrix for c=
0.These bounds have been refined in Ball, Sivakumar and Ward (1992) to give for multiquadrics, general c and all n the upper bound for the Euclidean norm of the inverse of the matrix
(5.24) q
−
1 exp(4nc/q),multiplied by a universal constant that only depends on the dimension. This again matches nicely with the aforementioned result due to Ball. We note in particular the exponential growth of this bound when q diminishes. We note also its important dependence on the parameter c in the exponent.
5.3.3 Lower bounds on matrix norms
Other lower bounds on the 2-norms of the inverse of the interpolation matrix have been found by Schaback (1994). Because they confirm our earlier upper bounds, they give an idea of the typical minimum loss of significance we must expect in the accuracy of the coefficients for general right-hand sides which are due to large condition numbers and the effects of rounding errors. We give three examples of the results of this work below.
Theorem 5.12. Let q be theˆ maximal distance maxξ,ζ
∈
ξ−
ζ
between thefinite number of data points
⊂
. Let m= |
|
be the finite cardinality of the set of centres.(i) For φ(r )
=
√
r 2+
c2 , the interpolation matrix satisfies the asymptotic bound
A−
1
2≥
C exp
c[(12n!m)1/n
−
12]/qˆ
m , m
→ ∞
.(ii) Let φ(r )
=
r 2k logr and n be even. Then the interpolation matrix satisfies the asymptotic bound
A−
1
2≥
Cm 2k n−1−
1, m→ ∞
.(iii) Let φ(r )
=
r 2k+
1 , and let n be odd. Then the interpolation matrix satisfies the asymptotic bound
A−
1
2≥
Cm 2k n+1−
1, m→ ∞
.We observe, e.g. by an application of Stirling’s formula, that the bound in (i) compares favourably with the upper bound on the matrix norm given above, except for a power of q which has little effect in comparison with the exponential
142 5. Radial basis functions on scattered data
growth of the upper and lower bounds as ˆq diminishes. Also (iii) compares quite well with the stated upper bounds for k = 0, although we are usually interested in 2k ± 1 > n.
We remark that the above statements show that the minimum norm becomes larger with a larger number of centres (m → ∞). This is, of course, no contra- diction to the results in the previous subsection, because the minimal separation distance becomes smaller with a larger number of data in the same domain and the ˆq above is the maximal, not the minimal distance between centres.
It has been noted already that, while the upper bounds on2-matrix-norms of the inverse of interpolation matrices above do not depend on the cardinality of
, the bounds on the condition numbers do, as they must according to Buhmann, Derrien and LeM´ehaut´e (1995). This is also because the bounds on the norm of the interpolation matrix itself depend on the number of centres. An example is given by the following theorem which states a lower bound on the condition number; it applies for instance to the multiquadric radial basis function.
Theorem 5.13. If −φ is conditionally positive definite of order one and φ(0) ≥
0 , then the 2-condition number of the interpolation matrix with centres is bounded below by || − 1.
Proof: The 2-condition number is the same as the – in modulus – largest eigenvalue divided by the smallest eigenvalue. We note that φ is necessarily nonnegative. Now, because the interpolation matrix has only nonnegative en- tries, the largest eigenvalue is bounded from below by
min
ζ ∈
ξ ∈
φ(ζ − ξ ) ≥
|| − 1
φ(2q),the separation distanceq still having the same meaning as before. Moreover, the smallest eigenvalue can be bounded from above by φ(2q) − φ(0) ≤ φ(2q). This is because we may apply Cauchy’s interlacing theorem to the two-by- two principal submatrix of the interpolation matrix which has entries φ(0) on the diagonal and φ(2q) elsewhere. Concretely, this means that the smallest eigenvalue is bounded by φ(2q), which gives the result, recalling our lower bound on the largest eigenvalue from the above display, namely (|| −1) times
φ(2q).
However, this Theorem 5.13 only applies when −φ is a conditionally posi- tive definite matrix of order one; for other radial basis functions (Narcowich, Sivakumar and Ward, 1994) are able to give 2 (actually, the statement is more generally for p, 1 ≤ p ≤ ∞) bounds on condition numbers that only depend on q, as defined in (5.21). A typical result is as follows.
5.3 Norm estimates and condition numbers 143
Theorem 5.14. Let φ be conditionally positive definite of order zero and the
separation radius q be fixed and positive, let Dq be the collection of all finite
subsets of centres of Rn whose separation radii are at least q. Thensup A p
over all ∈ Dq isfiniteforall1 ≤ p ≤ ∞ , where A is as usual theinterpolation
matrix for the given radial basis function and centres.
There is another highly relevant feature of radial basis function interpolation to scattered data which we want to draw attention to in the discussion in this book. We have noted the importance of the norm estimates to the interpolation matrices and their inverses above, and of course we know about the importance of the convergence estimates and the approximation orders therein. What we wish to explain now, at the end of this chapter, is that there is a remarkable relationship between the convergence order estimates that have been presented earlier in this chapter and the norm estimates above (precisely: the upper bounds on 2-norms of inverses of interpolation matrices). It is called the uncertainty principle.