### Bertrand Gauthier, Xavier Bay

### To cite this version:

### Bertrand Gauthier, Xavier Bay. Spectral approach for kernel-based interpolation. Annales de

### la Facult´

### e des Sciences de Toulouse. Math´

### ematiques. S´

### erie 6, Universit´

### e Paul Sabatier Cellule

### Mathdoc 2012, 21 (3), pp.439-479.

### <

### http://afst.cedram.org

### >

### .

### <

### hal-00915529v3

### >

### HAL Id: hal-00915529

### https://hal.archives-ouvertes.fr/hal-00915529v3

### Submitted on 13 Mar 2014

### HAL

### is a multi-disciplinary open access

### archive for the deposit and dissemination of

### sci-entific research documents, whether they are

### pub-lished or not.

### The documents may come from

### teaching and research institutions in France or

### abroad, or from public or private research centers.

### L’archive ouverte pluridisciplinaire

### HAL, est

### destin´

### ee au d´

### epˆ

### ot et `

### a la diffusion de documents

### scientifiques de niveau recherche, publi´

### es ou non,

### ´

### emanant des ´

### etablissements d’enseignement et de

### recherche fran¸cais ou ´

### etrangers, des laboratoires

### publics ou priv´

### es.

### Bertrand

### Gauthier

### & Xavier

### Bay

Abstract. — We describe how the resolution of a kernel-based interpolation problem can be associated with a spectral problem. An integral operator is defined from the embedding of the considered Hilbert subspace into an auxiliary Hilbert space of square-integrable functions. We finally obtain a spectral representation of the interpolating elements which allows their approximation by spectral truncation. As an illustration, we show how this approach can be used to enforce boundary conditions in kernel-based interpolation models and in what it offers an interesting alternative for dimension reduction.

Résumé(Approche spectrale pour l’interpolation à noyaux). — Nous décri-vons comment la résolution d’un problème d’interpolation à noyaux peut être associée à un problème spectral. Un opérateur intégral est défini à partir d’un plongement du sous-espace hilbertien considéré dans un espace de Hilbert auxiliaire composé de fonctions de carré intégrable. On obtient une représentation spectrale des élé-ments interpolants permettant leur approximation par troncature du spectre. À titre d’exemple, nous montrons comment cette approche peut être utilisée afin d’intégrer des informations de type conditions aux limites dans un modèle d’interpolation et en quoi elle offre une alternative intéressante pour la réduction de dimension.

Contents

1. Introduction. . . 2 2. Optimal interpolation in Hilbert subspaces. . . 3 3. Problem-adapted integral operators. . . 5 4. Representation and approximation of the optimal interpolator. . . 12 5. Finite Case. . . 16 6. Application to Gaussian process models. . . 18

Key words and phrases. — kernel-based interpolation, integral operators, Hilbert subspaces, RKHS, Gaussian process models, spectral decomposition, infinite data set, boundary conditions, optimal approximation.

To quote this article. —B. Gauthier and X. Bay,Spectral approach for kernel-based interpolation, Annales de la Faculté des Sciences de Toulouse, Tome 21, numéro 3 (2012), p. 439-479.

7. Example of application. . . 22 Acknowledgments. . . 32 References. . . 32

1. Introduction

This work is devoted to the study of kernel-based interpolation methods (see for instance and among others [Wah90, Ste99, BTA04, RW06]). In order to cover a

relatively wide class of problems, we consider the general framework of interpolation
in a separated topological real vector space E. We denote by E′ _{the topological dual}
of E and by_{h·},_{·i}_{E,E}′ the associated duality bracket. For a linear subspace M of E′

ande_{∈}E, we say that f _{∈}E is aninterpolator ofe forM (oronM) if

∀e′_{∈}M,_{h}f, e′_{i}_{E,E}′ =he, e′i_{E,E}′.

In this context, we focus on the two linked kernel-based methods that areoptimal interpolation in Hilbert subspaces ofE(seee.g., [Att92]) andGaussian process models

based on the conditioning of zero-mean Gaussian processes with sample paths in E. We consider interpolation problems associated with general sets M, including more particularly the case whereM is infinite dimensional (infinite data set). Such a situ-ation may for instance occur when one aims at enforcing boundary values conditions in an interpolating functions problem.

We propose and analyze an overall process which associates the resolution of a kernel-based interpolation problem with the spectral decomposition of an integral operator. This leads to a spectral representation of the solutions of the considered interpolation problem (Theorem 4.1). By applying spectral truncations, we obtain ap-proximations of the interpolating elements which are optimal in a given sense (Propo-sition 6.4).

From a theoretical point of view, the spectral properties used in this article are well-known and related to extensions of the Mercer’s Theorem and Karhunen-Loève’s Theorem. From an applied point of view, the use of spectral methods in approxima-tion and learning is not new either. We can for instance quote the article of F. Cucker and S. Smale [CS02], where recalls and discussions concerning Mercer kernels and their applications in learning theory can be found. One can also refer to the works of E. Parzen [Par62] (also mentioned in [BTA04, Section 2.4]), or among others, to the articles [Wal67, Kue71, Raj72]. Also, remark that our approach is based on

the embedding of the considered Hilbert subspace into an auxiliary Hilbert space of square-integrable real-valued functions. The various applications and structures we consider can in this sense be compared with the ones appearing in the work of M.Z. Nashed and G. Wahba [NW74]. The main objective of the present article is to give

a theoretical description, in the general context of topological vector spaces, of the processes involved in the association of a kernel-based interpolation problem with a spectral problem. We also aim at showing the potential interests of such an approach.

The first part of this article (Section 2) is devoted to the description of optimal interpolation in Hilbert subspaces. In Section 3, we define the notion ofregular embed-ding adapted to an interpolation problem. We also show how this embedding defines an integral operator, which is referred to as problem-adapted. In Section 4, we use the spectral decomposition of this operator in order to study the initial interpolation problem and its approximation by spectral truncation.

In Section 5, we consider the case where M is finite-dimensional. Explicit calcu-lations are carried out in order to illustrate the use of spectral considerations in this particular case. Section 6 is next devoted to Gaussian process models. The spectral representations considered in the previous sections are extended to the conditioning problem. In particular, we show the IMSE-optimal character of the approximation by truncation in this context.

We finally develop (Section 7) a theoretical example of application involving a Hilbert subspace of continuously differentiable real-valued functions on R2. We

con-sider a particular class of kernels and show how to enforce Cauchy-type constraints on a circle (conditions on both values and normal derivatives) in the associated interpola-tion models. The difference between approximainterpola-tion by truncainterpola-tion and discretizainterpola-tion is illustrated.

2. Optimal interpolation in Hilbert subspaces

2.1. Hilbert subspace and RKHS. — The L. Schwartz theory of Hilbert sub-spaces [Sch64] (or Hilbertian subspaces) is an equivalent formalism for the more widespread theory of reproducing kernel Hilbert spaces (RKHS), introduced by N. Aronszajn in [Aro50], this equivalence is for instance discussed in Remark 2.1. The

abstract formalism of L. Schwartz is adapted to the framework of topological vec-tor spaces; it also draws interesting parallels with operavec-tor theory (see for instance Proposition 3.7).

2.1.1. Hilbert subspace. — The general framework of the Hilbert subspaces of E

requires the (real) topological vector space E to be also locally convex and

quasi-complete (see for instance [Rud95, Sch64]), what we assume thereafter. Remark

that these properties are verified by most of the classical functions spaces and in
particular by all Fréchet and Banach spaces. We denote by E′ _{the topological dual}
space ofE.

AHilbert subspaceHofEis a linear subspace ofEendowed with a Hilbert structure
such that the inclusion of the Hilbert space H into E is continuous. We use the
notationH ∈Hilb(E). We then denote byT_{H} theHilbert kernel naturally associated

with H ∈ Hilb(E). We remind thatT_{H} : E′ _{→ H ⊂} E is a linear, symmetric and

positive application,

i.e. _{∀}e′ andf′_{∈}E′,_{h}T_{H}e′, f′_{i}_{E,E}′ =hT_{H}f′, e′i_{E,E}′ and hT_{H}e′, e′i_{E,E}′ >0.

The kernelT_{H} is in particular characterized by therepresentation property,
(2.1) _{∀}h_{∈ H},_{∀}e′_{∈}E′,_{h}h, e′_{i}_{E,E}′ = (h|T_{H}e′)_{H},

where (·|·)_{H} is the inner product of H. If {hj:j∈J} is an orthonormal basis of

H ∈Hilb(E), the Hilbert kernel T_{H} can be written under the form

(2.2) T_{H}=X
j∈J
hj⊗hj, i.e. ∀e′∈E′, THe′=
X
j∈J
hhj, e′i_{E,E}′hj.

2.1.2. Reproducing kernel Hilbert space. — A RKHS Hof real-valued functions on
a setX is a Hilbert subspace of RX (space of real-valued functions on _{X}) endowed

with the topology of the pointwise convergence (see [Aro50, Sch64, BTA04]).
Thereproducing kernel K(_{·},_{·}) ofH is then linked with the Hilbert kernel T_{H} by
the relation, for allxandy_{∈ X},

(2.3) K(x, y) =_{h}T_{H}δx, δyi_{E,E}′,

where δx is the evaluation functional for x ∈ X, that is hf, δxi_{E,E}′ = f(x) for all
f _{∈}E =RX. This definition for a RKHS (as Hilbert subspace) is therefore exactly

equivalent to the more common definition, namely that _{H}is a Hilbert space of
real-valued functions on_{X} such that, for allx_{∈ X}, the linear mapLx:H →R,h7→h(x),
is continuous.

Remark 2.1. — The RKHS theory can first appear to be a particular case of the
Hilbert subspaces one. In reality, this two theory are equivalent and only differ by
their formalism (see [Sch64, Gau11]). Indeed, one can for instance considerE as a
linear subspace of RE′ (since E _{⊂}E′′_{) and then assimilate a Hilbert subspace of}_{E}
with a RKHS of real-valued functions onE′_{.}

2.2. Optimal interpolation. — Let_{H ∈}Hilb(E)andM be a linear subspace of
E′_{. For a given}_{ϕ}_{∈ H}_{, the set of the elements of}_{H}_{which interpolate}_{ϕ}_{on}_{M} _{can be}
easily described thanks to the Hilbert subspace structure of _{H}. In what follows, we
resume some of the main results concerning such a study. Throughout this article,
we will frequently refer to theinterpolation problem associated withH ∈Hilb(E)and
M, without necessarily mention the element ofHwhich has to be interpolated.

Let us introduce the set

M0=ne_{∈}E:_{∀}e′ _{∈}M,_{h}e, e′_{i}_{E,E}′ = 0

o

.

We defineH0=M0∩ H=TH(M)⊥, whereTH(M)⊥ denotes the orthogonal, in H,
ofT_{H}(M) ={T_{H}e′_{:}_{e}′_{∈}_{M}_{}}_{. Then, for a fixed}_{ϕ}_{∈ H}_{,}

ϕ+ M0_{∩ H}

is the set of all interpolators, in_{H}, of ϕforM.

ϕ+ M0_{∩ H}is a non-empty closed affine subspace of_{H}and is therefore also

con-vex. Thusϕ+ M0_{∩ H}admits a minimal norm element, which we denotehϕ,M and
callminimal norm interpolator, or optimal interpolator. hϕ,M is then the orthogonal
projection of 0ontoϕ+ M0_{∩ H}_{. Let us remark that this first characterization of}

the optimal interpolator is essentially non-constructive, in the sense that it does not allow the construction ofhϕ,M from the only knowledge of the values of ϕonM.

By definition of the orthogonal projection, hϕ,M−0 is orthogonal toH0, i.e. hϕ,M ∈ H⊥0 =

T_{H}(M)⊥⊥=T_{H}(M)H=HM,

withHM the closure, inH, of the linear space spanned by theTHe′, fore′∈M. This
introduces the orthogonal decomposition H = _{H}0+HM and implies in particular
thathϕ,M is the only interpolator, inHM, ofϕforM.

Finally, let P_{H}M be the orthogonal projection of H onto HM. We know that

ϕ_{−}P_{H}M[ϕ]is orthogonal toHM, thus ϕ−PHM[ϕ]∈ H0,i.e. PHM[ϕ]interpolates

ϕforM. We finally obtain

hϕ,M =PHM[ϕ]

and this second characterization is suitable for the construction ofhϕ,M from the only
knowledge of_{h}ϕ, e′_{i}_{E,E}′, fore′ ∈M.

H0
ϕ+_{H}0
ϕ
hϕ,M
HM
H
0

Figure 1. _{Schematic representation of optimal interpolation in a Hilbert subspace.}

The Hilbert kernelT_{H}M of the Hilbert subspace HM,(·|·)_{H}, is linked withTH by
the relation

T_{H}M =PHMTH.

Hence, the knowledge ofT_{H}Mdefines the orthogonal projectionPHM and reciprocally,
this result staying true for any closed linear subspace of_{H}. This implies in particular
that the Hilbert kernelT_{H}0 ofH0,(·|·)_{H}, is given byTH0=TH−THM.

3. Problem-adapted integral operators

Let_{H ∈}Hilb(E)and let M be a linear subspace of E′. Throughout this section,

we consider the interpolation problem associated with_{H} and M. We use the same
notations and definitions as Section 2. Let us in particular remind the orthogonal
decomposition H=H0+HM.

In what follows, we introduce the notion ofregular embeddings associated with an interpolation problem and study the integral operators they naturally define. This

leads to the construction of specific orthonormal bases ofHM which are (in the sense of equation (3.14)) suitable for the resolution of the considered interpolation problem. The results of Sections 3 and 4 hold for any Hilbert subspaceHofE, separable or non-separable. However, if His non-separable, the existence of a regular embedding requires HM to be separable (see Remark 3.4). Let us mention that complementary considerations concerning Section 3 can be found in [Gau11].

3.1. Regular embedding and parameterization. — Let(_{S},_{A}, ν)be a general
measured set with ν a σ-finite measure. We denote by L2_{(}

S, ν)the Hilbert space of square-integrable real-valued functions on S with respect to ν. Let us remind that

L2_{(}_{S}_{, ν}_{)} _{is in fact a quotient space. Nevertheless, we make the widespread abuse}

of notation which consists in assimilating elements of L2_{(}_{S}_{, ν}_{)}_{with functions on} _{S}

(instead of considering equivalence classes of ν-almost everywhere equal functions).
We will sometime refer toL2_{(}_{S}_{, ν}_{)}_{as the}_{auxiliary Hilbert space}_{.}

Let(·|·)_{L}2 andk·k_{L}2 be respectively the inner product and the norm of L2(S, ν).

We recall that

∀f andg_{∈}L2(S, ν),(f_{|}g)_{L}2 =

Z

S

f(s)g(s)dν(s).

Let us consider an application γ : _{S →} E′. For all h _{∈ H}, we then define the
function

(3.1) Fh:S →RwithFh(s) =hh, γs_{i}_{E,E}′ for alls∈ S.

We now introduce conditions concerning L2(_{S}, ν),γ and_{H}, namely:

C-i. for allh_{∈ H}, the functionFh:S →Ris measurable,

C-ii. the function(s, t)∈ S × S 7→ hT_{H}γs, γt_{i}_{E,E}′ = (T_{H}γs|T_{H}γt)_{H} is measurable,

C-iii. τ=

Z

Sk

T_{H}γs_{k}2_{H}dν(s)<+_{∞}.

Proposition 3.1. — Under Conditions C-i, C-ii and C-iii, we have Fh_{∈}L2_{(}_{S}_{, ν}_{)}
for allh_{∈ H}and

(3.2) kFh_{k}2_{L}26τkhk

2

H.

Hence, the linear applicationF:_{H →}L2(_{S}, ν),h_{7→}Fhis continuous.

Proof. — Representation property (2.1), Cauchy-Schwarz inequality applied to(·|·)_{H}
and finally Condition C-iii give

(3.3)
Z
Sh
h, γs_{i}2_{E,E}′dν(s) =
Z
S
(h_{|}T_{H}γs)2_{H}dν(s)6τ_{k}h_{k}2_{H},

each integral being well-defined thanks to Conditions C-i and C-ii.

We now consider the orthogonal decompositionH=H0+HM and add the following
condition on the applicationF:H →L2_{(}_{S}_{, ν}_{),}

Definition 3.2. — We call regular embedding of HM into L2(S, ν) adapted to the

interpolation problem associated with H and M an application F : H → L2_{(}_{S}_{, ν}_{)}

defined from a parameterization γ : S → E′ via equation (3.1) and which verifies Conditions C-i, C-ii, C-iii and C-iv.

LetF :H →L2_{(}_{S}_{, ν}_{)}_{be a regular embedding of} _{H}

M into L2(S, ν). We consider

the linear subspaceF(H)ofL2_{(}_{S}_{, ν}_{)}_{given by}_{F}_{(}_{H}_{) =}_{{}_{Fh}_{:}_{h}_{∈ H}}_{(the image of}_{H}

through F). From C-iv, F(H0) = 0, hence F(H) =F(HM). We endow this space of the following inner-product,

(3.4) ∀handg_{∈ H}M,(Fh|Fg)F(H)= (h|g)H.

Proposition 3.3. — Let F : H → L2_{(}_{S}_{, ν}_{)} _{be a regular embedding of} _{H}

M into

L2_{(}_{S}_{, ν}_{)}_{, then} _{F}_{(}_{H}

M), (·|·)F(H) is a Hilbert space. It is isometric to HM, (·|·)_{H},

the isometry being the restriction toHM of the regular embeddingF.

In addition, the inclusion of the Hilbert space F(_{H}M), (·|·)F(H) into L2(S, ν) is
continuous. In other words,F(HM)∈Hilb L2(S, ν)

.

Proof. — The fact that F(_{H}M)is a Hilbert space isometric to HM directly follows
from its construction. Further, from Proposition 3.1, we have for allh_{∈ H}M,

kFh_{k}2_{L}2 6τkhk
2
H=τkFhk
2
F(H),
thusF(_{H}M)∈Hilb L2(S, ν)
.

Remark 3.4. — One can for instance consult [For85] for a discussion about the
conditions appearing in Definition 3.2. The ones we use here are of similar type but
specially adapted to the study of the interpolation problem associated with _{H} and

M. Indeed, letFbe a regular embedding of_{H}M into L2(S, ν)and consider
(3.5)

Z

S

f(s)hh, γs_{i}_{E,E}′dν(s) = (f|Fh)_{L}2,

wheref is a fixed element ofL2_{(}

S, ν)andh_{∈ H}. From Condition C-iv, the value of
expression (3.5) does not vary if we replacehbyh+h0, withh0_{∈ H}0. This quantity

thus only depends of the values ofhonM (i.e. of hh, e′_{i}

E,E′ fore′∈M), which are

the only available informations when considering an interpolation problem associated
withM and an elementhof_{H}.

When_{S} is a topological space (endowed with its Borelσ-algebra) and Conditions
C-i, C-ii and C-iii are already verified, C-iv will for instance be realized if for all

h_{∈ H}, the functionsFhare continuous and if M = span{γ(supp(ν))}(i.e. M is the
linear subspace ofE′ _{spanned by}_{γ}_{(supp(}_{ν}_{)), with}_{supp(}_{ν}_{)}_{the support of}_{ν}_{).}

Finally, note that the existence of a regular embedding F associated with the

interpolation problem defined byHandM implies in particular thatHM is separable, see for instance Proposition 3.8.

Example 3.5. — Let us consider a RKHSHof continuous real-valued functions on a topological spaceX and the problem consisting in the interpolation of an element

One can for instance define a regular embedding for this problem by introducing
the measure ν = Pn_{i}=1wiδxi (with wi >0) on S = X (endowed with its Borel σ
-algebra) and the parameterizationγ:x_{7→}δx(another possible parameterization for
this problem is given in Section 5).

More generally, if we suppose that the values of ϕare known on a closed subset

D ofX (i.e. M = span_{{}δx, x∈D}) while keeping the same parameterizationγ, one

just has to consider a measureν onX whom support isD and such that Conditions C-iii is also verified.

Note that in this particular example, if one identifies elements of L2(_{S}, ν) with
functions on_{S}=_{X}, then the applicationFis in fact the identity operator on_{H}, with

Fh(x) =h(x), for allh_{∈ H}andx_{∈ X}.

Proposition 3.6. — Let F : _{H →} L2(_{S}, ν) be a regular embedding of _{H}M into

L2(_{S}, ν) and consider its adjoint operator t_{F} _{:} _{L}2_{(}

S, ν) _{→ H} defined by equation

(3.7) hereafter. Then, for all f _{∈} L2(_{S}, ν), t_{Ff} _{∈ H}

M and we have the following

integral representation,

(3.6) _{∀}f _{∈}L2(_{S}, ν),t_{Ff} _{=}

Z

S

f(s)T_{H}γs dν(s),
this expression having to be understood in the sense of equation (3.8).
Proof. — We remind thatt_{F}_{:}_{L}2_{(}_{S}_{, ν}_{)}_{→ H}_{is defined by}

(3.7) _{∀}h_{∈ H},_{∀}f _{∈}L2(_{S}, ν), h_{|}tFf

H= (Fh|f)L2.

From C-iv, we directly deduce thatt_{Ff} _{∈ H}

M for allf ∈L2(S, ν). Next, by applying
the preceding equation toh=T_{H}e′ withe′_{∈}E′, we obtain

t_{Ff, e}′
E,E′ =
Z
S
f(s)_{h}T_{H}e′, γs_{i}_{E,E}′dν(s)
=
Z
S
f(s)_{h}T_{H}γs, e′_{i}_{E,E}′dν(s),
(3.8)

which corresponds to equation (3.6) (one can refer to [Bou59] for details concerning the notion of vectorial integral).

3.2. Integral operator defined by a regular embedding. — We still consider the same interpolation problem associated withHandM. Thanks to Proposition 3.3, we know that a regular embeddingFofHM intoL2(S, ν)defines a Hilbert subspace

F(_{H}M), (·|·)F(H) of L2(S, ν). Hence, from the Hilbert subspaces theory, it admits a

unique associated Hilbert kernel. If we identify the continuous dual ofL2(_{S}, ν)with
itself (Riesz-Fréchet representation Theorem), the Hilbert kernel ofF(_{H}M)relatively
toL2_{(}

S, ν)is the unique linear application

Lγ,ν : L2(S, ν)

′

=L2(_{S}, ν)_{→}F(_{H}M)⊂L2(S, ν)
which verifies the representation property, for allh_{∈ H}andf _{∈}L2(_{S}, ν),

Proposition 3.7. — Let F : H → L2_{(}_{S}_{, ν}_{)} _{be a regular embedding of} _{H}

M into

L2_{(}_{S}_{, ν}_{)} _{and let} _{L}

γ,ν be the kernel of the Hilbert subspace F(HM) of L2(S, ν), then Lγ,ν =FtF, i.e. for all t∈ S andf ∈L2(S, ν),

(3.10) _{L}γ,ν[f](t) =

Z

S

(T_{H}γs_{|}T_{H}γt)_{H}f(s)dν(s).

Proof. — By combining equations (3.9), (3.7) and (3.4), we obtain that, for h_{∈ H}

andf _{∈}L2(_{S}, ν),
(Fh_{|}f)_{L}2 = h|tFf
H= Fh|F
t_{Ff}
F(H)= (Fh|Lγ,ν[f])F(H).

We finally deduce equation (3.10) from the integral expression of t_{F}_{given in }
Propo-sition 3.6 (more precisely, equation (3.6)) by applying the preceding relation to

h=T_{H}γt_{∈ H}, witht_{∈ S}.

Let us remark that the Hilbert subspaceF(_{H}M)ofL2(S, ν)can be assimilated to
the RKHS of real-valued functions onS associated with the reproducing kernel
(3.11) _{∀}(s, t)_{∈ S × S},_{K}(s, t) = (T_{H}γt_{|}T_{H}γs)_{H}.

Hence, Lγ,ν can be seen as a classic integral operator on L2(S, ν) defined by the
symmetric and positive kernel K(_{·},_{·}) on S × S (see for instance [Sch79, §10] and
[For85])

We deduce from the theory of integral operators that Lγ,ν is a Hilbert-Schmidt

operator and therefore acompact operator. So_{L}γ,ν :L2(S, ν)→L2(S, ν)is
diagonal-izable and its eigenvalues are positive. We denote by λi those eigenvalues (repeated
according to their algebraic multiplicity) and by φei ∈L2(S, ν) the associated
eigen-functions, withi_{∈}I_{(and where}I_{is a general index set). We remind that}n_{φ}e_{i}_{:}_{i}_{∈}Io

forms a orthonormal basis of L2_{(}_{S}_{, ν}_{)} _{and that the set of all}_{strictly positive} _{}

eigen-values is at most countable.

Proposition 3.8. — Let F : _{H →} L2(_{S}, ν) be a regular embedding of _{H}M into

L2_{(}

S, ν)and consider its associated integral operator

Lγ,ν =FtF:L2(S, ν)→F(HM)⊂L2(S, ν).

Denote by_{{}λn:n∈I+} the at most countable set (i.e. I+⊂N) of its strictly positive
eigenvalues (repeated according to their multiplicity) and let φen ∈ L2(S, ν) be their

associated eigenfunctions. For all n_{∈}I_{+, we define}

(3.12) φn =
1
λn
t_{F}_{φ}_{e}
n =
1
λn
Z
S
e
φn(s)T_{H}γs dν(s)_{∈ H}M.

Then √λnφn :n∈I+ is an orthonormal basis of the Hilbert space HM endowed

with the inner product(_{·|·})_{H}.

Proof. — First remark that from Proposition 3.6, the elements φn of HM are
well-defined. By definition, we have that, for alln_{∈}I_{+} _{and for all}_{h}_{∈ H}_{,}

(3.13) φn
h
H=
1
λn
t_{F}_{φ}_{e}
n
h
H=
1
λn
e
φn
Fh
L2.

As for allm_{∈}I_{+},Fφm=φem, the preceding equation (3.13) applied toh=φm, gives
that√λnφn :n∈I+ is an orthonormal system ofHM.

From Proposition 3.7 and the properties of Hilbert kernels, we know that the linear subspace spanned by the Lγ,ν[f], f ∈L2(S, ν), is dense in F(HM), (·|·)F(H) (and in

particular n√λnφen :n∈I+

o

is one of its orthonormal bases). Hence, by isometry
betweenF(_{H}M),(·|·)F(H)andHM,(·|·)_{H}, the linear spacespan√λnφn:n∈I+ is

dense inF(_{H}), which concludes the proof.

In our context of interpolation, the main interest of the elementsφn ofH,n∈I+,

appearing in Proposition 3.8 is that (see equation (3.13)),
(3.14) _{∀}h_{∈ H},(φn|h)_{H}=
1
λn
Z
S
e
φn(s)_{h}h, γs_{i}_{E,E}′dν(s).

So, as for equation (3.5) of Remark 3.4, the evaluation of the inner product(φn|h)_{H}
can be directly obtained from the only knowledge of the values ofhonM.

We are now able to formulate our representation Theorem 4.1, which simply con-sists in the use of this particular orthonormal basis and of equation (3.14) in order to describe the orthogonal projection ofHontoHM.

Before this, we conclude this section by some additional remarks concerning the structures and applications we have introduced. This will be useful for the rest of our study.

3.3. Some important remarks. — The definition of a regular embedding F of HM into L2(S, ν) allows the construction of various applications and structures in addition to the ones studied until now. This section aims at introducing a few of them. Let us mention the article [NW74], where a similar framework is studied.

3.3.1. Operator on _{H} defined by a regular embedding. — In the same way as an
embeddingF:_{H →}L2(_{S}, ν)defines an integral operatorLγ,ν =FtFonL2(S, ν)(see
Proposition 3.7), it also defines an operator on_{H}.

Proposition 3.9. — Let F : _{H →} L2(_{S}, ν) be a regular embedding of _{H}M into

L2(_{S}, ν) and consider the framework of Proposition 3.8. We define the following
linear operator on_{H},

(3.15) ∀h_{∈ H}, Lγ,ν[h] =tFFh=

Z

Sh

h, γs_{i}_{E,E}′T_{H}γs dν(s).

Lγ,ν is a continuous symmetric and positive Hilbert-Schmidt operator on the Hilbert

spaceH,(·|·)_{H}. It is diagonalizable, the eigenspace associated with the null eigenvalues
isH0and√λnφn,n∈I+, are the eigenvectors (with

√λnφn H= 1) associated with the eigenvaluesλn,n∈I+.

Proof. — The properties of symmetry and positivity ofLγ,ν are obvious. Let us give a direct proof of the fact that it is a Hilbert-Schmidt operator. Let{hj:j∈J}be an

orthonormal basis ofH. Using equation (2.2) and Fubini’s Theorem, we obtain,
(3.16) X
j∈J
Lγ,ν[hj]
2
H=
Z
S
Z
S
(T_{H}γs_{|}T_{H}γt)2_{H}dν(s)dν(t)6τ2,

the last inequality being a consequence of the Cauchy-Schwarz inequality applied to the inner product ofHand of Condition C-iii.

For allh0_{∈ H}0, we obviously havetFFh0= 0. In addition, for h∈ Handn∈I+,

(3.17) t_{FFφ}
n
h
H=λn
_{1}
λn
t_{F}_{φ}_{e}
n
h
H
= (λnφn|h)_{H}.

This equation combined with Proposition 3.8 completes the spectral decomposition ofLγ,ν and also proves its continuity.

3.3.2. Two additional Hilbert structures. — We introduce two Hilbert spacesF(_{H})L

2

andHM γ,ν

which naturally appear when considering a regular embeddingF ofHM. Note that these two structures will be useful to us for the application of our approach to Gaussian processes conditioning in Section 6.2.

F(H)L

2

is the closure inL2_{(}_{S}_{, ν}_{)}_{of the linear subspace} _{F}_{(}_{H}_{). Let us notice that}

n e

φn, n∈I+

o

is obviously one of its orthonormal bases for the inner product(_{·|·})_{L}2.

Let us now define HM γ,ν

. We start by introducing the following symmetric and
positive bilinear form onH, for allhandg_{∈ H},

(3.18) (h_{|}g)_{γ,ν} = FhFg_{L}2 =
Z

Sh

h, γs_{i}_{E,E}′hg, γsi_{E,E}′dν(s).

We also setkh_{k}2_{γ,ν} = (h_{|}h)_{γ,ν}. Condition C-iv implies that the nullspace of(·|·)_{γ,ν} is
H0(i.e. forh∈ H,khk_{γ,ν} = 0if and only ifh∈ H0) andHM endowed with(·|·)γ,ν is
hence a pre-Hilbert space. We then denote byHM

γ,ν

the completed ofHM fork·kγ,ν. Remark that the operatorLγ,ν (considered as an operator onHM) can be naturally extended toHM

γ,ν

by continuity. Lγ,ν is then a Hilbert-Schmidt operator onHM γ,ν

,
(·|·)_{γ,ν}. It is symmetric and positive definite, its eigenvalues areλn,n_{∈}I_{+} and each

one is associated with the eigenvectorφn (andkφnkγ,ν = 1).

3.3.3. Isometries. — We are finally in presence of four isometric Hilbert spaces,

HM,F(H),F(H)

L2

andHM γ,ν

.

As we have seen in Proposition 3.3, the isometry between HM, (·|·)_{H} and F(H),
(·|·)_{F(}_{H}_{)}is the restriction ofFtoHM. The continuous extension of this first isometry
defines the isometry between_{H}M

γ,ν

, (_{·|·})_{γ,ν} andF(_{H})L

2

,(_{·|·})_{L}2.

The isometry betweenF(_{H})L

2

,(_{·|·})_{L}2 andF(H), (·|·)_{F(}_{H}_{)}is given by

∀n_{∈}I+,φen→

p

It is in fact the restriction of the square-root ofLγ,ν to F(H)
L2
, with
L12
γ,ν
"
X
i∈I
αiφei
#
=X
i∈I
αi
p
λiφei,
wherePi∈Iαiφei∈L
2_{(}
S, ν). We obviously haveLγ,ν =L
1
2
γ,ν ◦ L
1
2
γ,ν.

3.3.4. Pseudoinverse of a regular embedding. — Let us consider the framework of
Proposition 3.8. We can define the pseudoinverse (or generalized inverse)F† ofFby
(3.19) _{∀}n_{∈}I_{+},F†φen=φn=

1

λn
t_{F}_{φ}_{e}

n

and, fori_{∈}I_{\}I_{+} _{(}_{i.e.} _{λ}_{i}_{= 0),}_{F}†_{φ}e_{i}_{= 0. Then,}_{F}† _{is well-defined from}_{L}2_{(}_{S}_{, ν}_{)}_{onto}

HM
γ,ν
and
(3.20) ∀f _{∈}L2(_{S}, ν),F†f = X
n∈I+
f_{e}φn
L2φn∈ HM
γ,ν
.
The restriction of F† to F(H)L
2

defines the inverse of the isometry between HM γ,ν andF(H)L

2

. In the same way, its restriction toHM gives the inverse of the isometry betweenHM andF(H). We have in particular

(3.21) P_{H}_{M} =F†F,

which is in fact an equivalent formulation of Theorem 4.1.

4. Representation and approximation of the optimal interpolator

ForH ∈Hilb(E), we consider the optimal interpolation problem in Hdefined by

ϕ_{∈ H}and a linear subspaceM ofE′. In order to apply Section 3 results, we suppose

M such that_{H}M is separable (thus, M can be an arbitrary linear subspace of E′ if
His itself separable).

4.1. Spectral representation for optimal interpolation. — Once an orthonor-mal basis of HM known, one can easily express the orthogonal projection ofHonto HM. Then, in order to compute the optimal interpolator ofϕ∈ H forM (see Sec-tion 2), we need to be able to evaluate the inner-product inH betweenϕ and each elements of the considered basis of HM, and this from the only knowledge of the values ofϕonM (which are, in our context of interpolation, the only available

infor-mations concerningϕ). This is precisely the property of the orthonormal basis of_{H}M
associated with a regular embeddingF, its elements indeed verify equation (3.14).

Remark that in order to be applied to a given interpolation problem (associated
with_{H}andM), our approach requires the preliminary choice of a measurable space
(S,_{A}, ν)and of a parameterization γ :S → E′ _{allowing the definition of a regular}
embeddingFofHM into the auxiliary spaceL2(S, ν). Some considerations concerning
this choice are discussed in Section 4.3.

Theorem 4.1. — Let F be a regular embedding of HM into L2(S, ν) and consider

the orthonormal basis √λnφn :n∈I+ ofHM associated withF. Then, forϕ∈ H,

we have
(4.1) P_{H}_{M}[ϕ] = X
n∈I+
φn
Z
Sh
φn, γsiE,E′hϕ, γsi_{E,E}′dν(s).

Proof. — We just have to consider Proposition 3.8 and equation (3.13),

P_{H}M[ϕ] =
X
n∈I+
p
λnφn
p
λnφn
ϕ
H=
X
n∈I+
λnφn
e
φn
Fϕ
F(H)
= X
n∈I+
φn
e
φn
Fϕ
L2.

See also equation (3.21) for an equivalent formulation of this result.

Let us remark that the sum appearing in equation (4.1) converges by construction
in _{H}. Since _{H ∈}Hilb(E), it also converges for the initial topology of E and for its
weak topologyσ(E, E′_{). Then, in particular, for all}_{e}′ _{∈}_{E}′_{,}

hP_{H}_{M}[ϕ], e′_{i}_{E,E}′ =
X
n∈I+
hφn, e′iE,E′
Z
Sh
φn, γsiE,E′hϕ, γsi_{E,E}′dν(s).

Finally, because of the continuous inclusion of_{H}M,(·|·)_{H}intoHM
γ,ν

, the considered sum also converges fork·kγ,ν.

4.2. Truncated approach and approximation. — In practice, even if the

spec-tral decomposition of the operator Lγ,ν =FtF is known, it is not always possible,
for instance for numerical reasons, to consider all the terms appearing in the Mercer
decomposition of T_{H}M,

∀e′_{∈}E′, T_{H}Me

′_{=} X

n∈I+

λnhφn, e′iE,E′φn.

A classic alternative simply consists in not considering all the terms that appear in the previous sum. In this case, one currently speaks about spectral truncation and these are usually the largest eigenvalues which are conserved. The following section aims at illustrating the use of this alternative in our context. Note that additional considerations concerning the optimal character of the approximation by truncation based on the largest eigenvalues are developed in Section 6.4.

Remark that we have to keep in mind that, in the most part of applications, the true analytical spectral decomposition ofLγ,ν would be unknown. So, the study of the behavior of the proposed approach when dealing with approximated eigenpairs is of importance in regards of such applications.

4.2.1. Spectral truncation. — Assume that we dispose of an approximated kernel
defined from a subsetI_{trc}ofI_{+}, that is, for alle′ _{∈}_{E}′_{,}

T_{H}trc
M e

′ _{=} X

n∈Itrc

λnhφn, e′iE,E′φn.

Forϕ_{∈ H}, we then obtain an approximation of the optimal interpolatorP_{H}M[ϕ]that
we denote byP_{H}trc
M [ϕ]. We have
∀e′ _{∈}E′,DP_{H}trc
M [ϕ], e
′E
E,E′ =
ϕ_{}T_{H}trc
Me
′
H
= X
n∈Itrc
hφn, e′iE,E′
Z
S
e
φn(s)hϕ, γs_{i}_{E,E}′dν(s),
(4.2)
whereHtrc

M is the closure in Hof the subspace spanned by theφn, forn∈Itrc.

Theorem 4.2. — Consider the general framework of Section 4.2 and introduce the
setI_{err}_{=}I_{+}_{\}I_{trc}_{, then}

(4.3) ∀e′ _{∈}E′,DP_{H}
M[ϕ]−PHtrc_{M} [ϕ], e
′E2
E,E′ 6kϕk
2
H
X
n∈I_{err}
λnhφn, e′i
2
E,E′
(4.4) and _{}ϕ_{−}P
Htrc
M [ϕ]
2
γ,ν
6_{k}ϕ_{k}2_{H} X
n∈I_{err}
λn.

Proof. — By definition, we have for alle′_{∈}E′,
(4.5) DP_{H}
M[ϕ]−PHtrc_{M} [ϕ], e′
E
E,E′ =
ϕ_{} X
n∈I_{err}
λnhφn, e′i_{E,E}′φn
H.

To obtain expression (4.3), we just have to remark that the Cauchy-Schwarz inequality applied to equation (4.5) gives

D
P_{H}
M[ϕ]−PHtrcM [ϕ], e
′E2
E,E′ 6kϕk
2
H
X
n∈Ierr
λnhφn, e′iE,E′φn
2
H
and that, from Proposition 3.8,

X n∈Ierr λnhφn, e′iE,E′φn 2 H= X n∈Ierr λnhφn, e′i 2 E,E′.

Next, from equation (4.3) and the definition ofk·k2γ,ν (see section 3.3.2), we have,

Z
S
D
P_{H}
M[ϕ]−PHtrc_{M} [ϕ], γs
E2
E,E′dν(s)6kϕk
2
H
X
n∈I_{err}
λn
Z
Sh
φn, γsi2_{E,E}′dν(s).

Sincekφnk2γ,ν = 1for alln∈I+, we obtain

P_{H}_{M}_{[}ϕ]−P_{H}trc
M [ϕ]
2
γ,ν
6_{k}ϕ_{k}2_{H} X
n∈Ierr
λn,
which concludes the proof.

Let us remark that, by definition ofk·kγ,ν,
(4.6) ∀ϕ_{∈ H},_{}P_{H}
M[ϕ]−PHtrc_{M} [ϕ]
2
γ,ν =
ϕ_{−}P
Htrc
M [ϕ]
2
γ,ν.

We call Pn∈Ierrλn the spectral error term. It will be in practice evaluated by
considering
X
n∈Ierr
λn =
X
n∈I+
λn−
X
n∈Itrc
λn
=
Z
Sk
T_{H}γs_{k}2_{H}dν(s)_{−} X
n∈I_{trc}
λn.
(4.7)

A good indicator (see also Section 6.4) of the overall quality of the obtained approxi-mation can classically be found in the ratios

P
n∈Itrcλn
P
n∈I+λn
= 1_{−}
P
n∈Ierrλn
P
n∈I+λn
.

4.3. About the choice of the parameterization. — In this section, we men-tion some general consideramen-tions concerning the choice of the parameterizamen-tion. By parameterization, we mean here the overall process leading to the construction of a regular embedding ofHM into an auxiliary Hilbert space.

4.3.1. Computational aspects. — In the interpolation context of Theorem 4.1, the parameterization can just appear as a tool allowing to obtain the representation for-mula (4.1). No matter its choice from the moment it allows the definition of a regular embeddingFofHM.

Nevertheless, if one envisages the computation of the elements constituting the orthonormal basis of HM associated with the considered embedding (Proposition 3.8), this choice takes importance. Indeed, it in part determines the operator which has to be diagonalized. It then appears reasonable to try to make a choice that defines a simplest as possible spectral problem. One will for instance try to be in a position allowing an analytical resolution, or the use of a particular numerical method.

In such a context, an illustration of what appears to us as relatively judicious choices of parameterizations can be found in [Gau11, Section 3.3]. In this

particu-lar example, appropriate choices allow to obtain analytical expressions for many of the involved quantities and prediction formulae concerning the conditioning of a two parameters Brownian sheet are, in so doing, obtained in an original way.

4.3.2. The approximation case. — In addition of this first consideration, the parametrization choice has a direct influence on the behavior of the optimal inter-polator approximation obtained by spectral truncation in equation (4.2). Indeed, different choices of parameterization for a same problem lead to different approxima-tions of the optimal interpolator, and this even if the spectral ratios of the considered truncations are equal.

The parameterization directly influences the way P

Htrc

M [ϕ] approximates

P_{H}

M[ϕ] onM. So, it offers a way to modulate the accuracy of the approximation in function

of the elements ofM. This characteristic of tunable precision could offer interesting possibilities in regards of applications.

5. Finite Case

5.1. Context and notations. — We suppose that M is of finite dimension,i.e.

M = span_{{}µ1,_{· · ·} , µn}, withn∈N∗. Let us define the matrixT∈Rn×n by,

(5.1) for16i, j6n,Ti,j= (THµi|THµj)_{H}.

For simplicity and without loss of generality, we assume that the µi ∈ E′ are such that the symmetric and positive matrixTisinvertible. For convenience, we introduce the following matrix notations,

µ= (µ1,· · · , µn)T andT= T_{H}µ|T_{H}µT

H=

T_{H}µ,µT=µ, T_{H}µT,

whereT_{H}µ= (T_{H}µ1,· · · , T_{H}µn)T is a column vector. Hence, forϕ∈ H, the optimal

interpolator ofϕforM can be written under the form (5.2) hϕ,M =THµTT−1hµ, ϕi,

with hµ, ϕi =

hϕ, µ1_{i}_{E,E}′,· · ·,hϕ, µniE,E′

T

. Remark for instance that with our
notations,hµ, ϕiT =ϕ,µTand, fore∈E ande′ ∈E′,he, e′i_{E,E}′ =he, e′i=he′, ei.

So, we will write, fore′ _{∈}E′,hhϕ,M, e′i_{E,E}′ =

e′, T_{H}µTT−1hµ, ϕi.

The aim of this section is to prove, by explicit calculations, that the expression of the minimal norm interpolator given in equation (5.2) is equal to the one given in equation (4.1) of Theorem 4.1.

5.2. Parameterization. — We first introduce a trivial parameterization of this

problem. LetS={1,_{· · ·}, n_{}}and consider the measureν onSwhich assigns a weight

wi>0to eachi∈ S={1,· · ·, n}. The auxiliary spaceL2(S, ν)can then be identified
to the space Rn endowed with the inner-product_{(}x_{|}y)W =xTWy, where x and y

are two column vectors ofRn and whereWis the matrix

W= diag (w1,_{· · ·} , wn).

Notice that we identify a vector ofRn_{with the column vector of its coefficients in the}
canonical basis ofRn_{.}

We next consider the application γ : _{S →} E′, given by γi = µi, for all i _{∈}

{1,_{· · ·}, n_{}}. The associated application Fis trivially a regular embedding associated

with our problem. It verifies, for allh_{∈ H},Fh=_{h}µ, hi ∈Rn(andFh(i) =hh, µiiE,E′,

for alli_{∈ {}1,_{· · ·}, n_{}}=_{S}). If we identifyL2(_{S}, ν)withRn_{, then for}

α∈Rn,

(5.3) t_{F}

α=T_{H}µTWα∈ HM.
We finally obtain thatLγ,ν =FtFis given by

In the same way, we have (see Proposition 3.9),
∀h_{∈ H}, Lγ,ν[h] = tFFh=
Z
Sh
h, γs_{i}_{E,E}′T_{H}γs dν(s)
=
n
X
i=1
wihh, µiiE,E′T_{H}µi=THµTWhµ, hi.
(5.5)

The symmetric and positive bilinear form(_{·|·})_{γ,ν} on_{H}, associated withFvia equation

(3.18), is given by, forhandg_{∈ H},
(h_{|}g)_{γ,ν} =
n
X
i=1
wihh, µiiE,E′hg, µiiE,E′ =
h,µTWhµ, gi.

5.3. Spectral decomposition. — Let λ1 >0,_{· · ·}, λn > 0 be the eigenvalues of

TWand letv1,_{· · ·},vn be their associated eigenvectors,i.e. TW=PΛP−1 with

Λ= diag (λ1,_{· · ·} , λn) andP= (v1| · · · |vn).

Note that _{{}v1,_{· · ·} ,vn}forms an orthonormal basis ofRn,(·|·)W,i.e.

(5.6) PTWP= Idn_{×}n, whereIdn×n is then×nidentity matrix.

Fork_{∈ {}1,_{· · ·}, n_{}}, we define
(5.7) φk =
1
λk
t
Fvk=
1
λk
T_{H}µTWvk ∈ HM.

Proposition 5.1. — Under the assumptions of Section 5, we have, for allϕ_{∈ H},
T_{H}µTT−1hµ, ϕi=
n
X
k=1
φk
Z
Sh
φk, γsiE,E′hϕ, γsi_{E,E}′dν(s).
Proof. — We have
T_{H}µTT−1hµ, ϕi = T_{H}µTWW−1T−1hµ, ϕi
= T_{H}µTWPΛ−1P−1hµ, ϕi.

Let us study the terms appearing in this last expression. First, from equation (5.7),

T_{H}µTWPΛ−1= (φ1,· · ·, φn).

Next, using equation (5.6), we find

P−1_{h}µ, ϕi= PTWP−
1

PTW_{h}µ, ϕi=PTWhµ, ϕi.

To conclude, we remark that, fork_{∈ {}1,_{· · ·} , n_{}}, the term

Z
Sh
φk, γsi_{E,E}′hϕ, γsi_{E,E}′dν(s) =
n
X
i=1
wihϕ, µii_{E,E}′hφk, µii_{E,E}′

6. Application to Gaussian process models

Optimal interpolation in Hilbert subspaces and Gaussian process models are in-trinsically linked. In this section, we recall some of the main properties concerning the conditioning of Gaussian processes in the framework of topological vector spaces. We also apply the spectral approach developed in Sections 3 and 4 to the conditioning problem. The IMSE-optimal character of the approximation by truncation is finally addressed in Section 6.4.

6.1. Notations an recalls. — Let H be a separable Hilbert subspace of E.
Throughout Section 6, we assume that_{H}is the Cameron-Martin space of a centered
Gaussian process Y defined on a probability space (Ω,_{F},P). We denote by H the

Gaussian Hilbert space associated with Y (see [Jan97]). We remind that H is a

closed linear subspace of the Hilbert space L2_{(Ω}_{,}

F,P_{)} of second order real random

variables (r.v.) on (Ω,_{F},P_{)} (we identify random variables that are equal P-almost

surely).

For the sake of simplicity, we assume that E is a Banach space (see [DFLC71, Tal83] for more general frameworks). We also consider thatY takes its values inE

with P-probability 1 (_{i.e.} the triplet _{j,}_{H}_{,}_{H}Eis an _{abstract Wiener space}, where

HEis the completed of_{H}inEand wherejis the continuous inclusion of_{H}into_{H}E,
see [Gro67]).

We denote byI:H →H the isometry between_{H}andH, it verifies
E_{(}_{I}h_{I}g) = (h_{|}g)_{H},

whereE_{(}_{I}_{h}_{I}_{g}_{)}represents the inner-product in_{L}2_{(Ω}_{,}_{F}_{,}P_{)}between the two centered

random variables_{I}hand_{I}g_{∈}H. Let us also add, fore′_{∈}_{E}′_{,}
hY, e′_{i}_{E,E}′

(notation)

= Ye′ =I(T_{H}e′).

One can consult, among others, [Bax76, TV07, Gau11] for more details about the previous notions.

For a linear subspaceM ofE′,P_{H}M denotes the orthogonal projection ofHonto
HM. We then introduce the orthogonal projectionPH_{M} of H ontoH_{M} =I(H_{M}).

We have the commutative diagram

(6.1) H I //
PH_{M}
H
PHM
HM I //HM

The applicationT_{H}M =PHMTH is the Hilbert kernel of HM, hence, by isometry,
(6.2) ∀e′ _{∈}E′,_{I}(T_{H}_{M}e′) =PH_{M}[Y_{e}′]

(notation)

= E_{(}Ye′|Y_{f}′, f′∈M).

For all e′ _{∈}_{E}′_{, the r.v.} _{P}_{H}

M[Ye′]is called the conditional mean ofYe′ knowingYf′

for f′ _{∈}_{M} _{and} _{T}

H0 is the associatedconditional covariance kernel. The notion of

6.2. Spectral approach for conditioning. — We now consider the general frame-work of Section 3.

Proposition 6.1. — Let us consider the centered Gaussian process(Yγs)_{s}_{∈S}. Under
the assumptions of Theorem 4.1, the sample paths of (Yγs)_{s}_{∈S} are in L2_{(}_{S}_{, ν}_{)} _{with}

P_{-probability 1. In addition,}
Eh Z
S
(Yγs)2dν(s)i= X
n∈I+
λn =τ
.

Proof. — For s _{∈ S}, we remind that Yγs = I(THγs). Let {hj:j∈J} be an
or-thonormal basis of H. As in equation (2.2), (Yγs)_{s}_{∈S} admits the Karhunen-Loève
expansion

(6.3) ∀s_{∈ S}, Yγs=

X

j∈J

hhj, γsi_{E,E}′I(hj),

where the I(hj) = ζj, j ∈ J, form by isometry an orthonormal basis of H (such
independentN(0,1) r.v. are sometime calledorthogaussian, see [Dud10]). We then
deduce from Condition C-i that the sample paths of (Yγs)_{s}_{∈S} are measurable (as
real-valued functions onS) withP-probability 1.

One can obviously choose the orthonormal basis {hj :j∈J} of H such that it coincides onHM with the orthonormal basis√λnφn:n∈I+ associated with the

considered regular embeddingF (Proposition 3.8). Then, from C-iii and C-iv,

X
j∈J
Z
S
Eh _{Fh}_{j(}_{s}_{)}_{ζ}_{j}2i_{dν}_{(}_{s}_{)} _{=} X
j∈J
kFhjk2_{L}2
= X
n∈I+
pλnφen
2
L2=
X
n∈I+
λn <+∞.
Thus, the sum P_{j}_{∈}JkFhjζjk2_{L}2_{(}_{ν⊗}_{P}_{)} is convergent, which, form Tonelli’s Theorem,

implies the convergence in L2(ν_{⊗}P_{)}_{of}

(s, ω)_{7→}X

j∈J

Fhj(s)ζj(ω), withs∈ S andω∈Ω and finally completes the proof.

Theorem 6.2. — Let Y be a centered Gaussian process with values in E and
co-variance kernel T_{H}. Let M be a linear subspace of E′. Under the assumptions of
Theorem 4.1, we have, for alle′ _{∈}E′,

(6.4) E_{(}Ye′|Y_{f}′, f′∈M) =
X
n∈I+
hφn, e′iE,E′
Z
S
e
φn(s)Yγsdν(s).

In addition, the centered Gaussian process with covariance kernel T_{H}M (that is the

process corresponding to E_{(}Ye′|Y_{f}′, f′∈M), withe′ ∈E′) takes its values inH_{M}

γ,ν

Proof. — We have to verify that the right member of equation (6.4) is well-defined.
From Proposition 3.8, we know that, for alle′_{∈}_{E}′_{,}

T_{H}Me

′_{=} X

n∈I+

λnhφn, e′iE,E′φn, this series being convergent inH, so, by isometry,

(6.5) _{I}(T_{H}Me

′_{) =} X

n∈I+

λnhφn, e′iE,E′I(φn).

Forn_{∈}I_{+}_{, we have, thanks to equation (3.12),}

φn =
1
λn
Z
S
e
φn(s)T_{H}γs dν(s).

Now, Proposition 6.1 assures that the expression
(6.6) I(φn) = 1
λn
Z
S
e
φn(s)I(T_{H}γs)dν(s) = 1
λn
Z
S
e
φn(s)Yγsdν(s)

keeps sense under our working hypotheses. At last, from expansion (6.4),
Proposi-tion 6.1 and the isometry between F(_{H})L

2

and HM γ,ν

, we deduce that the process

E_{(}Ye′|Y_{f}′, f′∈M)takes its values inH_{M}

γ,ν

withP-probability 1.

6.3. A note on regular conditional probabilities. — We now study the con-ditional laws of the process Y relatively to the knowledge of the values take by its sample paths onM. Because, in our study,HM can be infinite dimensional, the con-struction of a regular conditional probability requires some precautions. We consider a case where such a conditional probability exists (see for instance [TV07], this is for instance always true ifHM is finite dimensional). We finally introduce an additional assumption which assures the existence of a spectral representation for the mean of the conditional laws, this representation corresponding to a natural extension of the one obtained in equation (4.1).

Let us assume that

(6.7) HE=HM

E

⊕ H0

E

,

which means that_{H}E=_{H}M
E
+_{H}0
E
with_{H}M
E
∩ H0
E
=_{{}0_{}}.

Condition (6.7) assures the existence of the linear continuous projectionP ofHE ontoHM

E

parallel toH0

E

(i.e. Ph0= 0 for allh0_{∈ H}0

E ).

We then consider the family of Gaussian measures onHEwith meanP[Ψ]∈ HM E

,
forΨ_{∈ H}E and covariance kernel T_{H}0. From [TV07, Theorem 3.11], such a family

defines a regular conditional probability over HE relative to the knowledge of Y on

M. In such case, the following notation is often used:

(6.8) P[Ψ] =E Ye′ Yf′ =hΨ, f′i E,E′, f′ ∈M .

Note that if we denote by µY the Gaussian measure on H E

associated with Y, the preceding regular conditional probability corresponds to thedisintegration ofµY re-latively toP.

Now, let us also suppose thatHM E

can be continuously injected intoHM γ,ν , what we write (6.9) HM E // HM γ,ν .

We next consider the extension _{F}ˇ _{of}_{F} _{to} _{H}E

defines by ˇ_{Fh0} _{= 0, for all} _{h0} _{∈ H}_{0}E
and, on HM

E

, by the continuous extension of the restriction of F to HM. From
Section 3.3.3 and condition (6.9), ˇ_{F}_{is well-defined from}_{H}E

ontoF(H)L

2

.

Then, as forP_{H}M =F†F, we haveP =F†Fˇ (remark that this last expression is
well-defined in regards of the definition ofF†given in Section 3.3.4). We finally obtain
the spectral representation formula,

(6.10) ∀Ψ∈ HE,P[Ψ] = X
n∈I+
hφn, e′iE,E′
Z
S
e
φn(s)hΨ, γs_{i}_{E,E}′ dν(s).

Remark 6.3. — We have the well-known equality, forϕ_{∈ H}and e′ _{∈}_{E}′_{,}
(6.11) hP_{H}M[ϕ], e′iE,E′ =E Ye′Y_{f}′ =hϕ, f′i

E,E′, f′∈M

.

6.4. Optimal approximation. — The results and considerations of Section 4.2 can also be extended to the Gaussian processes case. In this section, we discuss the optimal character of the approximation by truncation.

LetY be a centered Gaussian process with values inE and covarianceT_{H}. Under
the assumptions of Theorem 6.2, we consider a subset I_{trc} _{of} I_{+} _{composed of the}

largest eigenvalues of_{L}γ,ν, in the sense that,

(6.12) ifi_{∈}I_{err} _{=}I_{+}_{\}I_{trc}andn_{∈}Itrc, thenλi 6λn.

Proposition 6.4. — Let HappM be any closed linear subspace of HM and denote by

T_{H}app

M the associated Hilbert kernel. Fore

′_{∈}_{E}′_{, we introduce the two approximations}

of Ze′=E Y_{e}′Y_{f}′ =hϕ, f′i
E,E′, f′∈M
,
(6.13) Z_{e}trc′ =I
T_{H}trc
M e
′ _{and}_{Z}app
e′ =I
T_{H}app
M e
′_{.}
If_{H}trc
M andH
app

M have the same finite dimensionN ∈N∗, then under equation(6.12),
(6.14) E _{Y} _{−}_{Z}trc2
γ,ν
6E _{k}_{Y} _{−}_{Z}app_{k}2
γ,ν
.
Proof. — From Theorem 4.2 and Proposition 6.1, we have

E Ztrc2_{γ,ν}=E
Z
S
Z_{γs}trc2dν(s)
=
Z
S
T_{H}trc
M γs
2
Hdν(s) =
X
n∈Itrc
λn
(6.15) andE _{Y} _{−}_{Z}trc2
γ,ν
=E _{k}_{Y}_{k}2
γ,ν
−E _{Z}trc2
γ,ν
.

Letf1,_{· · ·}, fN be an orthonormal basis ofHappM and consider its decomposition in the

orthonormal basis associated with the considered regular embedding, that is

∀i_{∈ {}1,_{· · ·} , N_{}}, fi=
X
k∈I+
αi,k
p
λkφk, withαi,k∈R

and where, foriand j_{∈ {}1,_{· · ·}, N_{}}, P_{k}_{∈}I+αi,kαj,k=δi,j (Kronecker delta). Thus,

we easily obtain that

E _{k}_{Z}app_{k}2
γ,ν
=
N
X
i=1
X
k∈I+
α2_{i,k}λk.

Next, using for instance convex combinations arguments, we remark that

(6.16) X n∈Itrc λn> N X i=1 X k∈I+ α2i,kλk. We finally conclude thanks to equation (6.15).

In the same way as in equation (4.6), one can remark that

(6.17) E _{k}Y _{−}Zapp
k2γ,ν
=E _{k}Z_{−}Zapp
k2γ,ν
.

Thus, in regards of _{k·k}_{γ,ν} (i.e. in the sense of equation (6.14)), for alle′ _{∈}_{E}′_{,} _{Z}trc
e′

is the best approximation, based on N elements of HM, of the conditional mean

E _{Y}_{e}′Y_{f}′ = hϕ, f′i

E,E′, f′ ∈ M

. One can hence speak about a certain IMSE-optimality (Integrated Mean Square Error) of the approximation by truncation. This point is illustrated in Section 7.5.3.

7. Example of application

7.1. The problem. — Let_{X} =R2 and _{H}be the RKHS of real-valued functions

onX (see Section 2.1.2) associated with the kernel (squared exponential or Gaussian
kernel, see for instance [RW06]), forxandy_{∈ X},

K(x, y) =e−kx−yk

2

σ2 _{,} with_{σ >}_{0}and

k · kthe euclidean norm.

Form_{∈}N, let_{E}m_{⊂}RX be the subspace of functions of class _{C}m endowed with

the topology of the uniform convergence on the compact subsets of _{X} for all the
derivatives of order 6m (of general order if m= +∞). From [Sch64, Proposition

25], for all m_{∈}N(and also form= +∞),His a Hilbert subspace of Em_{. In what}
follows, we will considerHas a Hilbert subspace ofE=E1_{.}

Letx= (x1, x2)∈ X, we will use polar coordinates, i.e. x= (rxcosαx, rxsinαx)

withrx∈R+ andαx∈[0,2π). Forx∈ X, we defineδx∈E′ andηx∈E′ by

∀f _{∈}E,_{h}f, δxi_{E,E}′ =f(x)and hf, ηxi_{E,E}′ =

∂f ∂rx

(x),

δx is the Dirac measure centered on x and ηx corresponds to the evaluation of the radial derivative atx.

Let C ⊂ R2 be the circle of center_{0} and radius R > 0. We consider the linear
subspaces ofE′

and MC = MD+MN (D and N stand for Dirichlet and Neumann conditions, C for Cauchy). The aim of this example is to approximate the kernel K0C(·,·)of the subspaceH0C of functions h∈ Hsuch that,

∀t_{∈ C},_{h}h, δtiE,E′ = 0and hh, ηtiE,E′ = 0.

Nevertheless, let us remark that what follows contains all the necessary informations
to tackle the general interpolation problem associated withH, an elementϕ_{∈ H}and

MC (see [Gau11]).

We present a two steps methodology. The first step (Section 7.2) consists in con-sidering independently the interpolation problems inHassociated withMN andMD. Thanks to the study of a third operator (Section 7.3), we finally combine our re-sults in Section 7.4 and obtain a model in which both values and radial derivatives are controlled on the circle (Cauchy condition). Numerical computations are finally presented in Section 7.5.

7.2. The two independent problems. — Let us introduce the linear subspaces ofHnaturally associated withMD andMN,

HMD =H
⊥
0D = span{K(t,·), t∈ C}
H
and
HMN =H⊥0N = span
∂K
∂rt
(t,_{·}), t_{∈ C}
H
.

We denote by T_{H}_{MD} and KMD(·,·)the Hilbert kernel and the reproducing kernel of
HMD respectively. We use similar notations for the kernels associated withMN.

7.2.1. Parameterization. — LetS = [0,2π)endowed with its natural Lebesgue mea-sure (up to the multiplicative constantR) and consider the Hilbert spaceL2([0,2π)) of square-integrable real-valued functions (with respect to the Lebesgue measure) on

[0,2π), endowed with the norm

∀f _{∈}L2([0,2π)),_{k}f_{k}2_{L}2 =

Z 2π

0

f(θ)2Rdθ,

L2([0,2π))plays the role of the auxiliary Hilbert space defined in Section 3. We posesR,θ = (Rcosθ, Rsinθ)∈ C and introduce

γD: [0,2π)→MD, θ7→δsR,θ and γN : [0,2π)→MN, θ7→ηsR,θ.

We easily verify thatγD andγN define two regular embeddingsFD andFN ofHMD
and _{H}MN into L

2_{([0}_{,}_{2}_{π}_{))} _{(see also Remark 7.2) and which are given by, for}_{h}

∈ H
andθ_{∈}[0,2π),

FDh(θ) =hh, γDθi_{E,E}′ andFNh(θ) =hh, γNθi_{E,E}′.

The associated integral operators_{L}D =FDtFD and LN =FNtFN are defined on

L2_{([0}_{,}_{2}_{π}_{))}_{and}

(7.1) _{L}D[f](α) =

Z 2π

0

(7.2) _{L}N[f](α) =
Z 2π
0
∂2K
∂rs∂rx
(xR,α, sR,θ)f(θ)Rdθ,
wherexR,α= (Rcosα, Rsinα)∈ C,α∈[0,2π)andf ∈L2([0,2π)).

Remark 7.1. — Letx= (rxcosαx, rxsinαx)and y = (rycosαy, rysinαy)be two
points ofX, we have
K(x, y) = exp
−_{σ}12 r
2
x+r
2
y−2rxrycos(αx−αy)
,
∂K
∂rx
(x, y) =−_{σ}22 rx−rycos(αx−αy)
K(x, y)and
∂2_{K}
∂ry∂rx
(x, y) = 2
σ2cos(αx−αy)K(x, y)
+ 4
σ4 rx−rycos(αx−αy)
ry−rxcos(αx−αy)
K(x, y).
7.2.2. Spectral decomposition. — Using parity arguments, we obtain that the
eigen-values ofLD are, forn>0,

λD_{n} =Re−2R
2
σ2
Z 2π
0
e2R
2
σ2 cosθ_{cos(}_{nθ}_{)}_{dθ.}

The ones of_{L}N are, forn>0,

λN_{n} =
Z 2π
0
Acosθ+B(1 + cos2θ)e−2R
2
σ2 (1−cosθ)_{cos(}_{nθ}_{)}_{Rdθ,}
withA= _{σ}22 −
8R2
σ4 andB=
4R2
σ4 .

The two operators _{L}D andLN admit the same eigenfunctions. λD0 and λN0 are of
multiplicity 1 and associated with

(7.3) φ0e : [0,_{α}2π) −→ R

7−→ _{√}1

2πR

Forn>1,λD

n andλNn are ofmultiplicity 2 and associated with, forα∈[0,2π),
(7.4) φec_{n}(α) = _{√}1
πRcosnαandφe
s
n(α) =
1
√
πRsinnα.

Remark 7.2. — The two spaces FD(_{H})L

2

and FN(H) L2

are the same and corre-spond to the linear subspace of2π-periodic functions ofL2

loc(R)(restricted to[0,2π)).
As the set of all eigenfunctions of each operator_{L}DandLN coincides with the classical
discrete Fourier basis,LD andLN do not admit other non-null eigenvalue.

Concerning the operatorLD, one can for instance consult [MNY06] where similar

We are now able to express the orthonormal bases of HMD and HMN, associated withFDandFN respectively (see Proposition 3.8). ForFD, we introduce the elements

φD

0,φcDn andφsDn ∈ HMD,n>1, with for instance

(7.5) ∀n>1,_{∀}x_{∈ X}, φsD_{n} (x) = 1
λD
n
Z 2π
0
K(sR,θ, x)
sin(nθ)
√
πR Rdθ.

ForFN, we introduceφN0,φcNn andφsNn ∈ HMN, n>1, with
(7.6) ∀n>1,_{∀}x_{∈ X}, φcN_{n} (x) = 1
λN
n
Z 2π
0
∂K
∂rs
(sR,θ, x)
cos(nθ)
√
πR Rdθ.

Examples of numerical computations are presented in Section 7.5. We are now going to study the behavior on the circleC of this two families of functions.

7.3. An interesting operator. — ForxR,α ∈ C, let us consider the values

φN_{0}, δxR,α
E,E′,
φcN_{n} , δxR,α
E,E′ and
φsN_{n} , δxR,α
E,E′ forn>1,
φD_{0}, ηxR,α
E,E′,
φcD_{n} , ηxR,α
E,E′ and
φsD_{n} , ηxR,α
E,E′ forn>1.

It appears that those ones are all linked with an operatorJν given by,
(7.7) ∀f _{∈}L2([0,2π)), Jν[f](α) =
Z 2π
0
∂K
∂rs
(xR,α, sR,θ)f(θ)Rdθ.
Indeed, we have for instance, forn>1,

φcN_{n} , δxR,α
E,E′ =
1
λN
n
Z 2π
0
∂K
∂rs
(sR,θ, xR,α)
cos(nθ)
√
πR Rdθ,
φcD_{n} , ηxR,α
E,E′ =
1
λD
n
Z 2π
0
∂K
∂rx
(sR,θ, xR,α)
cos(nθ)
√
πR Rdθ
and ∂K
∂rs
(sR,θ, xR,α) =
∂K
∂rx

(sR,θ, xR,α). The operatorJν is self-adjoint but not

posi-tive. For,n>0, its eigenvalues are

ρn=
Z 2π
0
−2R
σ2 (1−cosθ)e
−2R2
σ2 (1−cosθ)cos(_{nθ})_{Rdθ.}

We remark in particular that ρ0 <0. The eigenvalue ρ0 is of multiplicity 1 and is

associated with the same eigenfunctionφ0e thanλD0 andλN0. Forn>1, theρn are of multiplicity 2 and are also associated with the same eigenfunctions φecn andφesn than

λD

n and λNn. The same argument than the one used in Remark 7.2 assures that Jν does not admit other non-null eigenvalue.

The spectrum of the operator Jν has a particular behavior since the number of
its negative eigenvalues depends of the ratio between R and σ2_{. The values of} _{ρ}_{n,}

Concerning the orthonormal basis of HMN associated withFN, we finally obtain, forxR,α = (Rcosα, Rsinα)∈ C,

(7.8) φN 0, δxR,α E,E′ = ρ0 λN 0 e

φ0(α)and, for alln>1,

φcN_{n} , δxR,α
E,E′ =
ρn
λN
n
e
φc_{n}(α)and φsN_{n} , δxR,α
E,E′ =
ρn
λN
n
e
φs_{n}(α).

For the one of_{H}MN associated with FN, we find
(7.9)
φD_{0}, ηxR,α
E,E′ =
ρ0
λD
0
e

φ0(α)and, for alln>1,

φcD_{n} , ηxR,α
E,E′ =
ρn
λD
n
e
φc_{n}(α)and φsD_{n} , ηxR,α
E,E′ =
ρn
λD
n
e
φs_{n}(α).

7.4. Double Constraint. — We combine the results of the two preceding sections in order to obtain a model that both takes account of the values of the function and of its radial derivative onC. We present an original way to express the kernelK0C(·,·) of the subspaceH0C of functions h∈ Hsuch that,

∀t_{∈ C},_{h}h, ηti_{E,E}′ = 0and hh, δti_{E,E}′ = 0.

We use Section 7.2 in order to describe the kernels T_{H}0_{N} of H0N andTH0_{D} ofH0D.
We next consider the interpolation problem associated withT_{H}0_{N} andMD(arbitrary

choice, we could equivalently consider the problem associated with T_{H}0_{D} and MN,

see Remark 7.4).

Remark 7.3. — This two approaches consist in considering the two decompositions

P_{H}0_{C} = PH0_{D}PH0_{N} or PH0_{C} = PH0_{N}PH0_{D}, where PH0_{C}, PH0_{D} and PH0_{N} are the

orthogonal projections ofHontoH0C,H0D andH0N respectively.

We consider the kernelK0N(·,·)ofH0N, the subspace of functionsh∈ Hsuch that ∂h

∂rt(t) = 0, for allt∈ C. We recall that

K0N(x, y) =K(x, y)−KMN(x, y)with
KMN(x, y) =λ
N
0φN0(x)φN0 (y) +
X
n>1
λN_{n} φcN_{n} (x)φcN_{n} (y) +φsN_{n} (x)φsN_{n} (y).

Thanks to the parameterization γD and the kernel K0N(·,·), we define a regular
embedding of _{H}MN ∩ H0N into L

2_{([0}_{,}_{2}_{π}_{)). We finally obtain the integral operator}

LC1defined by, forf ∈L2([0,2π))andα∈[0,2π),

LC1[f](α) =

Z 2π

0

K0N(xR,α, sR,θ)f(θ)Rdθ.

From the study of the operatorJν (see Section 7.3), we obtain that the eigenvalues

λCn1,n∈N, ofLC1are given by
(7.10) λC_{n}1=λD_{n} _{−} ρ
2
n
λN
n
.
The eigenvalue λC1

0 is associated with the eigenfunction φ0e of equation (7.3). For n>1, λC1

We finally introduce the elementsφC1

0 ,φcCn 1andφsCn 1,n>1, which are associated withLC1

ν via Proposition 3.8. Straightforward calculations give

(7.11)
φC_{0}1= 1
λC1
0
λD_{0}φD_{0} _{−}ρ0φN_{0} and, forn>1,
φcC_{n} 1= 1
λC1
n
λD_{n}φcD_{n} _{−}ρnφcNn
andφsC_{n} 1= 1
λC1
n
λD_{n}φsD_{n} _{−}ρnφsNn
.

Remark 7.4. — Instead of first consideringMN and next MD, one can operate in an inverse way. This leads to the study of the operator

LC2[f](α) = Z 2π 0 ∂2K0D ∂rs∂rx (xR,α, sR,θ)f(θ)Rdθ,

which is associated with K0D(·,·)and MN via the parameterization γN. Using the results of Section 7.3, we find that the eigenvaluesλC2

n ,n∈N, ofLC2 are
(7.12) λC_{n}2=λN_{n} _{−} ρ
2
n
λD
n
.
λC2

0 is of multiplicity 1 and is associated withφe0. In a same wayλCn2, forn>1, is of multiplicity 2 and is associated withφec

n andφesn. We finally obtain

(7.13)
φC02=
1
λC2
0
λN0φN0 −ρ0φD0
and forn>1,
φcC_{n} 2= 1
λC2
n
λN_{n}φcN_{n} _{−}ρnφcDn
andφsC_{n} 2= 1
λC2
n
λN_{n}φsN_{n} _{−}ρnφsDn
.

Remark 7.5. — If one conserves the same parameterizations γD and γN and the
same auxiliary space L2([0,2π)), one can in fact obtain similar results for any
sta-tionary covariance kernel with required regularity (in the sense that it defines a Hilbert
subspace of E=_{E}1_{) and of the type}

K(x, y) =

Z R

e−iξkx−ykdκ(ξ),

with k·kthe euclidean norm of R2 (and i2 _{=}_{−}_{1) and where} _{κ}_{is a finite symmetric}

positive measure on R (see the Bochner’s Theorem, for instance in [CS02]). In
particular, from the same arguments of parity than the ones leading to the spectral
decomposition of the operators LD, LN and Jν of Sections 7.2 and 7.3, we deduce
that the eigenfunctions of the integral operators associated with such a kernelK(_{·},_{·})
are still the same functions φ0e , φec

n and φesn, for n > 1, of equations (7.3) and (7.4) (discrete Fourier basis). Only the eigenvalues change.

7.5. Numerical application. — In this last section, we compute some of the

involved quantities for R = 3 and σ2 _{= 2. All computations have been performed}

with the free software R [_{R D08}]. In particular, the implied integrals have been