A particular form of regularization

A p a rtic u la r form of re g u la riz a tio n to w hich o th e r form s can be reduced is obtained when the constraint L(f) is set equal to II f \ \ 2 in (2.5.2). An example of such a reduction can be found in C hapter 6.

W ith L(f) = II/*II2, th e reg u larized solutions f a are given by th e m in im iz atio n

m in \ \ g - X f \ \ 2 + a \ \ f \ \ 2 (2.5.5)

f e F

where a is a strictly positive scalar. The solution to (2.5.5) is given by the norm al equations (Golub, 1983)

f = {%*X + a i r ' X g (2.5.6)

a

value decomposition (Baker, 1977) w ith singular values or converging to zero, an d complete sets of orthonorm al singular functions } an d {<Z>r }, r = 1,2 ,... which satisfy th e relationships

‘K V = G O

^ r r r

= a W

^ r r r

(2.5.7)

If the d a ta g is expanded in term s of the singular functions Or , i.e.

g = X

i =1

use of (2.5.6) and (2.5.7) yields im m ediately the following solution for f a

f

I (

Equation (2.5.8) shows th a t regularization acts as a low-pass filter since

Gflj Vj f rij/Gj for Gj large

G.2 + a g. + a /G . 0 for g. sm all

j j j J

In other words, th e low frequency components of f a corresponding to the la rg e r s in g u la r v alu es re m a in re la tiv e ly u n affected w hile th e h ig h frequency com ponents corresponding to th e sm all sin g u la r v alu es are filtered out. Note th a t th is fea tu re characterizes filterin g in general. For example, filtering of tim e series th a t have been F ourier-transform ed often am ounts to considering a n expression sim ilar to (2.5.8) to replace th e original expansion for f.

At th is stage, it is w orth perform ing a simple erro r analysis sim ilar to th a t presented in Section 2.3. F or th is purpose, assum e the d a ta g have been perturbed by some random noise e given by

The induced perturbation f a and f a now sa tisfy the follow ing relationship

HZ'.-

I'

Xc

It is instructive to compare (2.3.10) and (2.5.9). Equation (2.3.10) showed that if the eigenvalues Xt converge to zero faster than ei , the error in the solution can become arbitrarily large even though the error e may be converging to zero. This feature cannot happen anymore for equation (2.5.9). Furthermore, if the singular values g- are ordered so that

< J i 2 > g2 2 > ... > 0 , it is easy to see that (2.5.9) yields the inequality

1 1 4 - 4 II2 £ Wg - g W2 (2.5.10)

a 2

Equation (2.5.10) demonstrates that I I f a - f a I I remains bounded and

this independently of the rate of convergence to zero of the singular values

G-. In essence, (2.5.10) ensures that the solution f a depends continuously on the data g and thus results in stable algorithm for its computation.

An im portant issue in regularization is the com putation of the parameter a. Several methods have been proposed in the literature with generalized cross-validation (Golub et a l, 1979) appearing to be one of the most popular. In particular, generalized cross-validation is used in the numerical method for aquifer identification presented in Chapter 6 and the methodology is presented in some detail there.

2.6 PRACTICAL IMPLICATIONS

In the previous sections we have presented some of the tools that can be used to analyse the difficulties that are generally associated with ill-posed or ill-conditioned problems. From this analysis it is possible to make some statem ents about problems often encountered in model development for hydrological and hydrogeological systems. Two examples of such 'practical implications' relevant to the material presented in this thesis are now given:

1) Identification of the kernel in a convolution integral:

In Section 2.2 it was m entioned th a t convolution in teg rals are often used to model hydrological system s. For such system s, a m ajor difficulty is t h a t w hile a convolution s tru c tu re is often hyp o th esized a p rio ri th e associated kernel is often unknow n as the physical processes tak in g place in the w atershed or subsurface system s are not fully understood and/or are poorly docum ented. This m eans th a t th e k ern el h a s first to be estim ated from input/output data. Since the input/output d a ta are usually only known a t d isc rete p o in ts over fin ite tim e in te rv a ls , th e u s u a l m eth o d of deconvolution via th e L aplace tra n sfo rm m ay n o t be feasible. As a consequence, one m ay a ttem p t to use (2.2.4) directly to identify the kernel

k(t). In such a case the in p u t d a ta f plays the role of the kernel. In order to quantify the m easure of ill-conditioning of such an equation it is useful to w rite (2.2.4) in the following form

l g(s) = J /f s - t) Ht) dt o with _{h - t )} f f( s — t) if s > t l 0 if s < t (2.6.1)

This illu strates th a t, in addition to errors in the d a ta g, th ere will be errors in th e model since f is obtained from noisy in p u t d a ta f. We m ay th u s expect difficulties in recovering k(t). However, the kernel f is not continuous a t t = s if f(0) * 0 so th a t in general such a deconvolution is going to be only mildly ill-posed w ith singular values typically decaying as

0 ( 1 /r). As a consequence, even if th e errors in the d a ta and th e model are som ew hat large, it should be possible to recover a t le a s t th e firs t components in an expansion of k(t).

This approach h as been used in C h ap ter 4 to identify the kernel of a convolution integral involving a stream -aquifer interaction. In th is case we know th a t on physical grounds th e k ern el k(t) of th e system decays w ith tim e. T herefore k(t) h as been approxim ated w ith a few functions of the form exp(akt) where ak is a complex num ber w ith real p a rt 9te(ajJ < 0 to ensure th a t the exponential decays w ith t.

2) O ptim um param eterization:

W hen attem p tin g to solve an ill-posed problem, a n a tu ra l discretization schem e is th a t of a spectral decomposition of th e q u a n titie s involved in term s of th e operator's eigenfunctions. Indeed, it can be show n th a t such a discretization is optim al in the sense th a t the m agnification of errors will be m inim al (Newsam, 1982). However, for m ost p ractical situ atio n s th e eigendecom position can only be obtained n u m erically an d to do so a d isc retisatio n by know n, easily com puted functions is still needed. In addition, th ere m ay be situations where the unknow n q u a n tity is p a rt of th e o p erato r so th a t in such a case th e eigendecom position can n o t be o b ta in e d . An exam ple is tr a n s m is s iv ity id e n tific a tio n w h e re th e tran sm issiv ity T appears explicitly in the diffusion operator.

Since we know th a t only a smooth recovery of the solution f is possible in the presence of noise, we m ay still expand f in term s of a few smooth basis functions taken, for example, from a set of complete and orthonorm al functions such as the Legendre or Chebyshev polynomials. Such a spectral ex p an sio n h a s been u sed in C h a p te r 6 for a q u ife r tra n s m is s iv ity identification.

Chapter 3

OVERVIEW OF THE PARAMETER ESTIMATION