Contents lists available atScienceDirect
Journal
of
Computational
Physics
www.elsevier.com/locate/jcp
Multi-fidelity
Gaussian
process
regression
for
prediction
of
random
fields
L. Parussini
a,
D. Venturi
b,∗
,
P. Perdikaris
c,
G.E. Karniadakis
d aDepartmentofEngineeringandArchitecture,UniversityofTrieste,ItalybDepartmentofAppliedMathematicsandStatistics,UniversityofCaliforniaSantaCruz,USA cDepartmentofMechanicalEngineering,MassachusettsInstituteofTechnology,USA dDivisionofAppliedMathematics,BrownUniversity,USA
a
r
t
i
c
l
e
i
n
f
o
a
b
s
t
r
a
c
t
Articlehistory:Received17July2016
Receivedinrevisedform10November2016 Accepted23January2017
Availableonline11February2017
Keywords:
Gaussianrandomfields Multi-fidelitymodeling Recursiveco-kriging Uncertaintyquantification
Weproposeanewmulti-fidelityGaussianprocessregression(GPR)approachforprediction ofrandomfields basedonobservations ofsurrogate modelsorhierarchies ofsurrogate models. Our method builds upon recent work on recursive Bayesian techniques, in particular recursiveco-kriging, and extends it to vector-valued fields and various types ofcovariances,including separable and non-separableones.The framework wepropose is general and can be used to perform uncertainty propagation and quantification in model-based simulations, multi-fidelity data fusion, and surrogate-based optimization. We demonstrate the effectiveness of the proposed recursive GPR techniques through variousexamples.Specifically,westudythestochasticBurgersequationandthestochastic Oberbeck–Boussinesqequations describing naturalconvectionwithinasquareenclosure. InbothcaseswefindthatthestandarddeviationoftheGaussianpredictorsaswellasthe absoluteerrorsrelativetobenchmarkstochasticsolutionsareverysmall,suggestingthat theproposedmulti-fidelityGPRapproachescanyieldhighlyaccurateresults.
©2017ElsevierInc.Allrightsreserved.
1. Introduction
High-fidelity numericalsimulations of complexstochastic dynamical systems require substantial computing time and datastorageeveninmodernparallelarchitectures.Thisinherentlylimitsthenumberofsystemstateswecanreliably sim-ulate, therebyaffectingaccuracywheninferringstatisticalpropertiesofanyphasespacefunctionsuchastheperformance of acertain engineeringdesign.Thisbasicobservationhasrecentlydriven anexplosive growthoffundamental and prac-ticalresearch attheinterfaceofhigh-performancescientific computing, probabilitytheory,andappliedmathematics. One of the mainfeatures of such research isto replace expensivecomputational modelswith cheap surrogates orhierarchies ofsurrogates,andthencomeupwithmathematicaltechniquesleveraging oncross-correlationsbetweentheoutput of dif-ferentsurrogates toinferaquantity ofinterest.Thisyieldsamulti-fidelityapproachcomputationalmodeling,whichwas first proposedbyKennedyandO’Hagan[19,20]inaBayesiansetting,andsincethenusedinmanydifferentdisciplines(see,e.g., [25,9]). Forexample,in [3]theauthors presenteda non-intrusive framework basedon treed multi-outputGaussian pro-cesses, inwhichtheresponsestatistics areobtainedthroughsamplingaproperlytrainedsurrogate modelofthe physical system. Thetreeisbuiltinasequentialwayandits refinementdependsontheobservationsthroughaglobalmeasureof
*
Correspondingauthor.E-mailaddress:[email protected](D. Venturi). http://dx.doi.org/10.1016/j.jcp.2017.01.047 0021-9991/©2017ElsevierInc.Allrightsreserved.
theuncertaintyintheprediction,theinferredlengthscales,aswellastheinputprobability distribution.Gaussianprocess regressionhasalsobeenstudied inthecontext ofhierarchicalmulti-scalemodelingofmaterials. Forexample,in[22] an adaptivemoving kriginginterpolationmethodwas proposed toreduce thenumberofmodelevaluationsatthefine-scale and informthe coarse-scale model with essential constitutive information.A multi-fidelity approach for minimizing the numberofevaluationsofexpensivehigh-fidelitymodelswasalsoproposedin[23],wherethestatisticsofthehigh-fidelity model arecomputed basedon realizations ofa corrected low-fidelity surrogate.The correction function can be additive, multiplicative,oracombinationofthetwoanditmaybeupdatedoccasionallybyhighfidelitymodelevaluations.
In this paper we propose a new multi-fidelity Gaussian process regression (GPR) approach for predicting finite-dimensional random fields based on observations of surrogate models or hierarchies of surrogate models. The key idea relieson representing themap betweenthe spaceof (random)Fourieror other spectral coefficientsassociatedwith any series expansion (relativeto a spatial basis)and the physicalspace in which therandom field develops.In this setting, themulti-fidelityinferenceproblemforarandomfieldreducestoaninferenceproblemofamultivariaterandomvectorof Fouriercoefficientsgivendata,i.e.,vector samplesproducedbysurrogatemodels atdifferentlevelsoffidelity. Toperform such inference, onecan apply therecursive co-kriging technique recentlydeveloped byLe Gratiét et al.[15,13] (see also [25])to each componentofthe vector ofFouriercoefficients. From aBayesian standpoint, thisisequivalent to assuming independent priors foreach model output, which mayresult in loss of information.To overcome this issue,we extend the recursiveco-kriging technique toa multivariate setting(statistical modelswithvector outputs),to capturethe cross-correlationstructureamongdifferentvectorcomponents.Weconsiderbothseparableandnon-separablepriorsandquantify theadvantagesandtrade-offsofeachapproach.Weremarkthatalthoughthemainfocusofthispaperisrevolvingaround uncertaintypropagationandquantificationinmodel-basedcomputations,theproposed framework canbe readilyapplied tomoregeneralparametricstudiessuchasmultivariateinverseproblemsandmulti-objectivesurrogate-basedoptimization. Thepaperisorganized asfollows.InSection2we introducethegeneralframeworkalongwithsometheoretical back-ground.InSection3wegiveabriefoverviewofGaussianprocessregression(GPR)methodsforvector-valuedrandomfields and introduce ourmulti-fidelity recursive GPR technique. In Section 4 we evaluate the accuracy ofthe proposed multi-fidelityGPRapproachbyapplyingittothestochasticBurgersequationandastochasticthermalconvectionproblem.Finally, inSection5wesummarizeourmainfindings.
2. Methodology
Considerascalarrandomfieldu
(
x,
ξ
)
dependingonasetofcoordinates(ordesignvariables)x∈
R
n,aswellasonaset ofrandomparametersξ
∈
R
d.Thefielducouldbe,e.g.,thesolutiontoapartialdifferentialequationinwhichtheboundary conditionsare settobe randomandrepresentedinterms ofξ
.Suppose thatu(
x,
ξ
)
isina separableHilbertspace.This allowsustowritetheseriesexpansionu(x
,
ξ
)
∼
=
ki=1
ai
(
ξ
)L
i(
x),
(1)where Li
(
x)
arebasisfunctionsdepending onthecoordinates(or designvariables)x whileai(
ξ
)
are functionsofrandom variablesξ
.Ifu(
x,
ξ
)
isthesolutiontoastochastic PDEmodel,then Li(
x)
areusuallysetapriori(spatialbasis functions), while thefunctions ai(
ξ
)
are determined by the PDE,e.g., by computingits solutionat specific valuesofξ
throughthe probabilistic collocation method[1,8,4].At thispointwe posethefollowing question:canwe determinea modelforthe randomvectorfielda
(
ξ
)
= [
a1(
ξ
)
· · ·
ak(
ξ
)
]
(2)basedondatacollectedataspecific nodesinthe
ξ
-space?Thisquestionisobviouslynotnewandresearchershavebeen workingonitfordecades.Forinstance,onecanusepolynomialinterpolation ofeach ai(
ξ
)
atChebyshevsparse grids[2]. However,thisimplicitlyassumesthatai(
ξ
)
isamultivariatepolynomial(whichwedonotknowforsure),andalsothatwe canpredictthevalueofai(
ξ
)
atanon-observedlocationwithprobability 1,i.e.,withnouncertainty.Thisisobviouslynot correctfroma statisticalstandpoint. Ina morerobust setting,ai(
ξ
)
should beconsidered asa randomfield withknown values atobserved points. Following theclassical literature [18,19,6,29,12], we shall assume that the distributionof the vector a(
ξ
)
, conditional to the realization{
a1=
a(
ξ
1),
...,
an=
a(
ξ
n)
}
, is Gaussian with mean m(
ξ
)
and (matrix-valued) covariancefunctionC(
ξ
i,
ξ
j)
,i.e.,a
(
ξ
)
|
a1, ...,
an∼
GP
m
(
ξ
),
C(
ξ
i,
ξ
j)
.
(3) As we shall see in the subsequent sections, this setting allows usto build a multi-fidelity Gaussian process regression framework,inwhichobservationsofa(
ξ
)
obtainedfrommodelswithdifferentlevelsoffidelityarecombinedinaseamless waytoyieldahighlyaccurateGaussianpredictorofa(
ξ
)
.2.1. Calculationofstatisticalmoments:measuringtheuncertaintyofuncertainty
Inthetraditionaluncertaintyquantificationsetting,ai
(
ξ
)
areconsideredasdeterministic functionsoftherandom vari-ablesξ
,e.g.,multivariateorthogonalpolynomialsselectedfromtheAskeyscheme[35,33,7,32],orgeneralizedbi-orthogonal seriesexpansion [5,30].Thisallows usto evaluatethestatisticalmoments ofu(
x,
ξ
)
intermsofintegralswithrespect to theprobabilitydensityfunctionofξ
(assumingitexists),i.e.,Eξ
u(x,
ξ
)
n=
u(x
,
ξ
)
np(ξ
)d
ξ
.
(4)Asubstitutionof(1)into(4),yieldsthefollowingexpressionsforthefirstandsecondmoment
Eξ
[u(x,
ξ
)
]=
k i=1 Li(
x)
ai(
ξ
)
p(ξ
)d
ξ
,
(5)Eξ
u(x,
ξ
)
2=
k i,j=1 Li(
x)L
j(
x)
ai(
ξ
)a
j(
ξ
)p(
ξ
)d
ξ
.
(6)Next,supposethatai
(
ξ
)
areGaussianrandomfields,withmeanmandcrosscovarianceC (seeEq.(3)).Underthis assump-tion,thefirst- andsecond-ordermoments(5)–(6)arenolongerdeterministicquantities,buttheyareratherrandomfields. In fact, theexpectationEξ
[·]
becomes a conditionalexpectation, i.e.,Eξ
|a[·]
(integralwithrespect toξ
given arealization of the Gaussian random field a(
ξ
)
). Thisyields a Gaussian distribution for the mean field (5) (infinite sum ofGaussian randomvariables),whilethesecond-ordermomenthasthedistributionofanindefinitequadraticforminGaussianrandom variables[26,28].Withsuchdistributionswecanmeasuretheuncertaintyofuncertainty,i.e.,howaccuratewecanbewhen we compute the statisticalproperties ofthe random field u(
x,
ξ
)
given that we are using a stochastic (Gaussian) model forthefunctionsai(
ξ
)
,whichdependontherandomvariablesξ
.Theseobservationscanbegeneralized, andnewwaysof approximatingintegralsandderivativesoffunctionsbasedonstochasticinterpolantscanbebuilt(see,e.g.,[16]).Ifwetake theaverageof(1)relativetotheconditionaldistribution(3),thenweimmediatelyobtainEξ
|a[u(x,
ξ
)
]=
k i=1 mi(
ξ
)L
i(
x),
(7)Eξ
|a u(x,
ξ
)
2=
k i,j=1 Ci j(
ξ
,
ξ
)L
i(
x)L
j(
x),
(8)where Ci j is the cross-covariance of the random fields ai
(
ξ
)
andaj(
ξ
)
. In the next section, we discuss different Gaus-sian process regression(GPR) methods –includingtheproposed multivariate recursiveco-kriging approach –to inferthe conditionaldistribution(3)inamulti-fidelitysetting.3. Gaussianprocessregression(GPR)
Inthissection,wegiveabriefoverviewofGaussianprocessregressionmethods(GPR)forscalarandvector-valuedfields dependingonanarbitrarynumberofvariables.Wealsointroduceourmulti-fidelityrecursiveGPRmethod.
3.1. Krigingandrecursiveco-krigingforscalarfields
Consider a deterministic real-valued scalarfield A
(
ξ
)
depending onk variablesξ
=
(ξ
1,
...,
ξ
k)
. We thinkof A(
ξ
)
asa realizationofaGaussianrandomfielda(
ξ
)
intheforma(
ξ
)
=
m(ξ
)
+
y(ξ
),
(9)wherem
(
ξ
)
isa deterministicfunction,usually calledtrendfunction,representingthemeanofa(
ξ
)
,while y(
ξ
)
isa zero-meanGaussianrandomfieldwithcovariancefunctionC(
ξ
,
ξ
;
θ
)
=
σ
2r(ξ
,
ξ
;
θ
).
(10)Here
σ
2 isascaleparameter –theprocessvariance–andrisasymmetricandpositivedefinitekerneldependingonaset ofhyperparametersθ
.Thefunctionm(
ξ
)
in(9)canhavedifferentforms.Insimplekrigingweassumem(
ξ
)
=
μ
,whereμ
is aknownconstant. A morepopular choiceism(
ξ
)
=
β
,whereβ
isa regressionparameterlearnedfromdata(ordinary kriging).Moregenerally,wecanmodelm(
ξ
)
intermsofseriesexpansionsrelativetoknownbasisfunctionshj(
ξ
)
(universal kriging)as[13]m(
ξ
)
=
pj=1
β
jhj(
ξ
).
(11)Thebasisfunctions
{
h1(
ξ
),
...,
hp(
ξ
)
}
areoftenchosentobepolynomials,1whiletheexpansioncoefficientsβ
= [
β
1· · ·
β
p]
T dependondataandtheyareusuallylearnedthoughmaximumlikelihoodestimates.Now,supposewehaveavailable obser-vationsof A(
ξ
)
atndistinctnodes(
ξ
1,
...,
ξ
n)
,i.e.,observationsoftherandomfield(9)atξ
i (i=
1,
...,
n).Itiswell-known that the(conditional) posteriordistribution ofa(
ξ
)
givendata A= [
A(
ξ
1)
· · ·
A(
ξ
n)
]
T andthe regressionparametersθ
,β
and
σ
2 isGaussianwithmeanma
(
ξ
)
=
h(
ξ
)
β
+
rT(
ξ
)
R−1(
A−
Hβ
) ,
(12) andvariance sa2(
ξ
)
=
σ
2 1−
rT(
ξ
)
R−1r(
ξ
)
+
q(
ξ
)
HTR−1H −1 q(
ξ
)
T.
(13)Inequations(12)and(13)wedefinedh
(
ξ
)
= [
h1(
ξ
)
· · ·
hp(
ξ
)
]
,q
(
ξ
)
=
h(
ξ
)
−
rT(
ξ
)
R−1H,
(14) and r(
ξ
)
=
⎡
⎢
⎣
r(ξ
,
ξ
1;
θ
)
..
.
r(ξ
,
ξ
n;
θ
)
⎤
⎥
⎦
,
R=
⎡
⎢
⎣
r(
ξ
1)
T..
.
r(
ξ
n)
T⎤
⎥
⎦
H=
⎡
⎢
⎣
h(
ξ
1)
..
.
h(
ξ
n)
⎤
⎥
⎦
(15)Allparameters
θ
,β
andσ
2 are learnedfromdatabyclassical maximumlikelihoodestimation.Weremarkthatequations(12)–(13)holdfortheuniversal krigingmodel,i.e., whenm
(
ξ
)
is givenin(11).Similarequationscanbe obtainedforthe simplekrigingbysimplyintegratingoutβ
fromtheconditionalposterioroftheuniversalkriging.3.1.1. Multi-fidelityrecursiveco-krigingforscalarfields
Co-krigingwasoriginallyproposedforenhancingtheaccuracyofahigh-fidelitysurrogateusingsupplementary observa-tionsoflow-fidelity models.Inessence,itaimsatexploitingthecross-correlationbetweentwo ormoreGaussianprocess surrogatesthroughastochasticauto-regressivescheme,andityieldsapredictiveposteriordistributionforthehigh-fidelity modeloutput that encodesthe contributionoflower fidelity levelswithquantified uncertainty. Co-krigingwas first pro-posedbyKennedyandO’Haganinthelandmarkpaper[19],andsincethenithasbeenusedinmanydifferentapplications, such assurrogate-basedoptimization[9].Unfortunately, thealgorithm described in[19] hasan unfavorable cubicscaling withthenumber ofobserveddata andthenumberofsurrogate models.To overcomethisissue,Le Gratiétet al.[15,13] recentlyproposedarecursiveversionoftheco-krigingalgorithm,whichallowsustobreakupthecomputationofposterior distributionintoasequenceofdistinctinferencesofsmallerdimensions.Theadvantageofthisapproachisclear:the inver-sionofthesmallcovariancematricesassociatedwiththesequenceofinferencesislessexpensiveandbetterconditioned thantheinversionofthelargecovariancematrixassociatedwiththefullinferenceproblem.
Todescribe therecursive co-krigingapproach, supposewe haveavailable multiplemodels,say s-models, providingan estimateofthereal-valuedscalarfield
A
:
R
d→
R
ξ
→
A(ξ
)
Wedenoteby A(1)
(
ξ
),
...,
A(s)(
ξ
)
theoutputofsuchmodels,rankedintermsofincreasingfidelity,2 A(1)(
ξ
)
beingthemodel withlowest levelof fidelity.Usually, thecomputational cost ofa modelis proportionalto its fidelity, i.e., thehigherthe fidelitythe higherthe computationalcost.Thismeans thatwe cannot usuallyaffordsampling A(s)(
ξ
)
extensively.Onthe otherhand,wecanusuallyaffordmanysamplesof A(1)(
ξ
)
,butsuchsampleswillnotbeaccurate.Therecursiveco-kriging technique proposed in[14,13] aims at constructing an accurate estimate of A(
ξ
)
by leveraging on data fromall models A(j)(
ξ
)
(j=
1,
...,
s),inacomputationallyefficientway.ThekeyideaistorepresenteachmodeloutputintermsofaGaussian randomfielda(j)(
ξ
)
andthenmakethehypothesisthatsuchfieldsarerelatedtoeachotherbytheautoregressivemodela(j)
(
ξ
)
=
ρ
(j−1)(
ξ
)
a˜
(j−1)(
ξ
)
+
z(j)(
ξ
),
j=
2, ..,
s (16)1 Forexample,alinearbasisinonedimensionish(ξ )= [1,ξ].
where a
˜
(j−1)(
ξ
)
andz(j)(
ξ
)
are appropriate Gaussian random fields, whileρ
(j−1)(
ξ
)
are deterministic scaling fields(see[15,14,13]forfurtherdetails).Itcanbeshownthattheposteriordistributionofa(j)
(
ξ
)
,conditionalontheobservationsand parametersofalllower-fidelitymodels,isGaussianwithmeanandvariancegivenbym(j)
(
ξ
)
=
ρ
(j−1)(
ξ
)m
(j−1)(
ξ
)
+
Z(j)(
ξ
),
(17) s(j)(
ξ
)
2=
ρ
(j−1)(
ξ
)
2s(j−1)(
ξ
)
2+
W(j)(
ξ
),
(18) where Z(j)(
ξ
)
andW(j)(
ξ
)
arefunctionsthatcanbeexplicitlycomputedbasedonunivariateGPRresultsatfidelitylevels j and j−
1 (see[15,14,13]forfurtherdetails).Notethatthemean(17)andthevariance(18)satisfyaMarkovproperty,which inturnallowsustobreakuptheco-krigingproblemasformulatedin[19] intoasequenceofunivariate GPR(single-level kriging).Thisisveryconvenientfromacomputationalviewpoint.In[13],afullyBayesianformulationofrecursiveco-kriging which incorporatesprior informationin themaximumlikelihood estimationofthe modelhyperparameters isgiven. This allowsustospeeduptheestimationprocessbyleveragingonanalyticalexpressions.3.2. Multi-fidelityrecursiveco-krigingforvector-valuedfields
Inthissection,wegeneralizetherecursiveco-krigingapproachdiscussedintheprevioussectiontosystemswith multi-ple(vector)outputs.Tothisend,considerthereal-valuedvectorfield
A
:
R
d→
R
k,
ξ
→
A(
ξ
).
Suppose we have available s different models of A, i.e., A(1)
(
ξ
),
...,
A(s)(
ξ
)
, ranked in terms of increasing fidelity, withA(1)
(
ξ
)
beingthemodelwithlowestfidelity.Asbefore,weassumethat thecomputationalcostofamodelisproportional toitsfidelity,i.e.,thehigherthefidelitythehigherthecomputationalcost.Werepresenttheoutputofeachmodelinterms ofavector-valuedGaussianrandomfielda(j)(
ξ
)
andmakethehypothesisthatsuchfieldsarerelatedtoeachother bythe autoregressivemodela(j)
(
ξ
)
=
ρ
(j−1)(
ξ
)
◦
a˜
(j−1)(
ξ
)
+
z(j)(
ξ
),
j=
2, ..,
s (19) where◦
denotesthecomponentwiseproductbetweentwovectorsormatrices,i.e.,theHadamardproduct,ρ
(j−1)(
ξ
)
=
Ik
⊗
gTj−1(
ξ
)
α
j−1 (20)is a deterministic scaling vector field represented in terms of known basis functions gj−1
(
ξ
)
= [
gj−1,1(
ξ
)
· · ·
gj−1,p(
ξ
)
]
(⊗
thetensorproductoperator, Ikisthek×
kidentitymatrixandα
j−1 areregressioncoefficients).Also,weassumethatz(j)
(
ξ
)
∼
GP
Ik⊗
fTj(
ξ
)
β
j,
C(j)(
ξ
,
ξ
)
,
(21) a(1)(
ξ
)
∼
z(1)(
ξ
),
(22) andthata˜
(j−1)(
ξ
)
hastheconditionaldistribution˜
a(j−1)
(
ξ
)
∼
a(j−1)(
ξ
)
z(j−1)(
Dj−1)
=
A(j−1)(
Dj−1),
β
j−1,
α
j−2,
Cj−1(
ξ
,
ξ
;
θ
j−1).
(23) In theseequations, fj(
ξ
)
= [
fj,1(
ξ
)
· · ·
fj,q(
ξ
)
]
isa vector ofknown basis functionswhoselinear combinationrepresents themeanofz(j),Cj isthecovarianceofz(j)andDj−1 isasetofnodes{
ξ
n}
inwhichweevaluatetheoutputofthemodelA(j−1)
(
ξ
)
.Weassumethatsuchsetsarenested,i.e.,Ds
⊆
Ds−1· · · ⊆
D1.
(24)The hypothesis (23)allowsusto breakupthe fullmulti-fidelity GPRregression forrandomvector fieldsintoa sequence ofindependentGPRsateachleveloffidelity.Theposteriordistributionofa(j)
(
ξ
)
,conditionaltodataatlevel j andmodel parametersisGaussian,i.e.,a(j)
(
ξ
)
z(j)(
Dj)
=
A(j)(
Dj),
β
j,
α
j−1,
Cj(
ξ
,
ξ
;
θ
j)
∼
GP
m(j)
(
ξ
),
C(j)(
ξ
,
ξ
)
.
(25)Themeanandvariancehaveexpressions
m(j)
(
ξ
)
=
ρ
(j−1)(
ξ
)
◦
m(j−1)(
ξ
)
+
M(j)(
ξ
),
(26)C(j)
(
ξ
,
ξ
)
=
ρ
(j−1)(
ξ
)
2◦
C(j−1)(
ξ
,
ξ
)
+
V(j)(
ξ
,
ξ
).
(27) NotethatthemeanandthevarianceofthepredictivedistributionhaveaMarkovianstructure,whichallowsustoperform GPRacrossdifferentlevelsoffidelityindependently.Regarding M(j)(
ξ
)
andV(j)(
ξ
,
ξ
)
,alengthycalculationshowsthat,M(j)
(
ξ
)
= ˆ
fTj(
ξ
)
β
j+
cTj(
ξ
)
Cˆ
−j1A(j)(
Dj)
−
Pj−1◦
A(j−1)(
Dj)
−
fTj(
Dj)
β
j,
(28) V(j)(
ξ
,
ξ
)
=
Cj(
ξ
,
ξ
;
θ
j)
−
cTj(
ξ
)
Cˆ
jcj(
ξ
),
(29) whereˆ
fj(
ξ
)
=
Ik⊗
fj(
x),
cj(
ξ
)
=
Cj(
ξ
,
Dj;
θ
j),
Cˆ
j=
Cj(
Dj,
Dj;
θ
j),
(30) and Pj=
Ik⊗
gj(
Dj)
Tα
j.
(31)Weremarkthatequation(29)holdsforsimplekriging.WehavealsoobtainedtheanalyticalexpressionofV(j)
(
ξ
,
ξ
)
corre-spondingtouniversalkriging (notshownhere).Sofar,wehavemadenoassumptiononthecovariancemodelCj(
ξ
,
ξ
;
θ
j)
weuseforGPRateachleveloffidelity.Inthenextsubsectionwediscusstwoimportantcases.3.2.1. Separablecovariancefunction
SupposethatcovariancemodelCj
(
ξ
,
ξ
)
isseparable,inthesensethatCj
(
ξ
,
ξ
;
θ
j)
=
rj(
ξ
,
ξ
;
θ
j)
j
,
(32) whererjisacorrelationfunctionand jisamatrixwithfixedentries.Theassumption(32)impliesthatthereisonlyone spatial correlationfunctionrepresentingthecrosscovariancebetweenallcomponentsofthemodela(j)(
ξ
)
.The separable covariancehasaconjugatepriorfor j,whichallowsustointegrateout j analyticallyfromtheposterior(25).Thisyields amultivariateStudent-tconditionalposterior[11,24]withmeanandvariancethatcanbecomputedanalytically.3.2.2. Non-separablecovariancefunction
Unliketheseparablecase, anon-separable covariancefunctioncanhaveadifferentspatial correlationfunctionforeach componentofthemodeloutput.Inparticular,weconsiderherethelinearmodelofcoregionalization(LMC),where
Cj
(
ξ
,
ξ
;
θ
j)
=
Bdiag
r1(
ξ
,
ξ
;
θ
1), ...,
rk(
ξ
,
ξ
;
θ
k)
BT
.
(33) Asiswell known[21],depending onthematrix B we canhavedifferentmodelsofcoregionalization.Forexample,ifthe chooseBtobediagonal(withpositiveentries),thentheLMCiscalledindependent;ontheotherhand,ifweassumethat BissymmetricandpositivedefinitethentheLMCiscalleddependent.Inthelattercasewecanusethespectraldecomposition ofB BT torepresentBefficiently.Inbothcases,thecoefficientsofthematrix Bbecomeadditionalparametersthathaveto beestimatedwhenperformingGPRateachleveloffidelity.
4. Numericalresults
InthisSectionweprovidenumericalresultsandstudytheaccuracyofthemulti-fidelityGPRapproachwepresentedin thispaper.Tothisend,wefirststudyasimplepedagogicalexample,i.e.,arealfunctionin
[
0,
2π
]
dependingononerandom parameterξ
.Thisallowsustovalidateourmethodsandassesstheiraccuracyandcomputationalefficiency.Subsequently, weapply ourmulti-fidelityGPRapproachtothestochastic Burgersequationin1D andtoastochastic thermalconvection problemin2D.4.1. Apedagogicalexample
Considera realfunctionu
(
x,
ξ )
,periodicinx∈ [
0,
2π
]
,anddepending ononerandomparameterξ
,whichweassume tobeuniformlydistributedin[−
1,
1]
.SuchfunctioncanbeapproximatedbytheFourierseriesu(x, ξ )
=
k i=1ai
(ξ )L
i(x),
(34)whereLi
(
x)
aretrigonometricpolynomials(oddexpansion)([17],p. 29).Thenumberofmodeskheredefinesthe“fidelity” ofthemodel,i.e.,higherkcorrespondstohigherfidelity.Ourgoalistopredictu(
x,
ξ )
basedonsamplesintheξ
-space ob-tainedfromseriesexpansionswithdifferentnumberofmodes,i.e.,differentfidelity.Inparticular,weconsiderthefollowing prototypelow-fidelitymodeluL
(x, ξ )
=
kL i=1 a(iL)(ξ )L
i(x),
(low-fidelity model) (35) wherekL=
5 andFig. 1.Prototype high-fidelity (left) and low-fidelity (right) models defined in equations(36)and(35), respectively. a1(L)
=
cos(
1.
5ξ ) ,
a2(L)= −
ξ
2,
a(3L)=
0.
5ξ
2+
1 −1,
a(4L)=
e−ξ,
a(5L)=
ξ
2−
ξ.
Ontheotherhand,thehigh-fidelitymodelisdefinedasuH
(x, ξ )
=
kH i=1 a(iH)(ξ )L
i(x),
(high-fidelity model) (36) wherekH=
9 and a(jH)=
kL i=1 a(iL)(ξ )L
i(x
j)
−
0.
5ξx
j+
0.
2 j=
1, . . . ,k
H.
(37)Notethat thecoefficientsa(jH)are definedintermsofa(jL).Thismakesthelow-fidelityandhigh-fidelitymodels(35)–(36) correlatedinbothxand
ξ
.InFig. 1weplotuL(
x,
ξ )
anduH(
x,
ξ )
versusxandξ
.Todeterminetheaccuracyoftheproposed multi-fidelityGPR3methods,wehavecomputedtheabsoluteerrorofboththepredictedcoefficientsaj(ξ )
andthepredicted random function u(
x,
ξ )
relative to a(jH)(ξ )
and uH(
x,
ξ )
, respectively. In particular, we have used the root-mean-square errors(RMSE)definedasR M S E(aj
)
=
1 Np Np p=1 a(js)(ξ
p)
−
a(jH)(ξ
p)
2 j=
1, . . . ,k
H,
(38) and R M S E(u)=
1 NpNq Np p=1 Nq q=1 us(x
q, ξ
p)
−
uH(x
q, ξ
p)
2.
(39)Inthelasttwoequations,
{
xq}
q=1,...,Nq and{
ξ
p}
p=1,...,Np areevenlyspacednodesin[
0,
2π
]
and[−
1,
1]
,respectively,while Np=
Nq=
51.a(js)indicates thesurrogateofthe j-th coefficienta(jH) andusthesurrogateofhigh-levelfunctionuH.Next, wecomparetheperformanceofdifferentGPRapproachesinreconstructingthehigh-fidelitymodeluH basedonsamplesof uL anduH.Specifically,weconsider1. Univariateco-kriging(Section3.1);
2. Multivariateco-krigingwithseparableornon-separablecovariancefunction(Section3.2).
Ineachapproachwestudiedtheeffectsofdifferentcovariancemodels,i.e.,Gaussian,5/2-MatérnandWendland[27,34], aswell asthe effectsofdifferentorders intheregressors ofthetrend function(see Section 3.1).Such effectsare shown in Fig. 2andFig. 3 inthe caseofunivariate krigingandco-kriging, respectively.It isseen thata high-order regressorof the trendfunctionyield betteraccuracy, i.e., predictorswithsmallerstandarddeviation(see Fig. 2). Regardingthe choice of thecovariancemodel,ournumericalresultssuggestthat theGauss andthe Wendlandmodelsyieldthe mostaccurate
3 Inthisexamplewehaveonlytwolevelsoffidelitydefinedbythemodelsu LanduH.
Fig. 2.Effectofthetrendfunctionintheunivariatekrigingmodel.Weplot themeanandthestandard deviationofthe Gaussianpredictorofa3(ξ ) obtainedwith4evenly-spacedsamples,5/2-Matérn covarianceanddifferentpolynomialorderinthetrendfunction(11):zero-orderuniversalkriging (left),second-orderuniversalkriging(right).
Fig. 3.Convergenceoftheunivariateco-krigingmodelwithrespecttothenumberoflow-fidelitysamplesfordifferentcovariancekernels.Weplot Root-mean-squareerrors(38)ofthecoefficienta3(ξ )versusthenumberoflow-fidelitysamples(NL).Weshowresultsobtainedbyusingadifferentnumberof high-fidelitysamples(NH)anddifferentcovariancekernels:Gaussian,5/2-MatérnandWendland.
Fig. 4.Multivariaterecursiveco-krigingwithnon-separable covariance.Shownareresults forthecoefficienta1(ξ )obtainedwith NH=5 high-fidelity samplesandadifferentnumberoflow-fidelitysamples.ItisseenthatasweincreaseNL theco-krigingpredictorbecomesmoreandmoreaccurate(the standarddeviationofthehigh-fidelitypredictordecreases).
predictions,butintermsofcomputationaltime5/2-Matérnismoreefficient.Also,the5/2-MatérnandWendlandcovariance modelsarebetterconditionedthantheGaussian models.Asiswellknown,theaccuracyofGPRdependsonthefunctions we aimto predictas wellason thelocation of thesample points. Therefore,the conclusions wehave justdrawnabout accuracyandefficiencymaynotbeextendedtoothercases.
Next,we comparetheaccuracyofdifferentGPRmodelsinreconstructingthehigh-fidelity modeluH basedonsamples ofuL anduH.Weassumea5/2-Matérncovarianceandanorder2fortheregressorsofthetrendfunctions. A typicalplot weobtainedwheninferringthemodesa(iH)
(ξ )
frommulti-fidelitydataisshowninFig. 4.Weseethatforafixedvalue of high-fidelitysamples,themulti-variaterecursiveGPRwithnon-separablecovarianceconvergestothe high-fidelitymodelas weincreasethenumberoflow-fidelityruns.InFig. 5wecomparetheaccuracyoftheproposedGPRapproaches.Itisseen that univariate co-kriging and multivariate co-kriging with non-separable covariancefunctions yield similar convergenceFig. 5.AccuracyofdifferentGPRapproachesinreconstructingthehigh-fidelitymodeluHbasedonsamplesofuLanduH.ShownareRMSEs(39)versus thenumberoflowfidelitysamplesNL.Eachcurvecorrespondstoadifferentnumberofhigh-fidelitysamples.
rates.The multivariateco-krigingwithseparablecovariancefunction inthiscaseyields aplateauintheerrorslopes.This is due to the fact that the coefficients ai
(ξ )
have different correlation lengths, which cannot be effectivelycaptured by a single correlation function (see Section 3.2.1). Regarding the computational time, univariate GPR isusually fasterthan multivariate GPRwithnon-separablecovariances. Themain bottleneckofthe multivariateGPRis theoptimizationofthe likelihood function, whichdependson the modelparameters of all functionsai(ξ )
. Onthe other hand,multivariate GPR withseparablecovariancefunctioncanbeveryefficientinsituationswherethereisastrongstatisticalcorrelationbetween differentai(ξ )
,i.e.,whenallmodesai(ξ )
canbemodeledbyasinglecorrelationfunctionateachfidelitylevel.4.2. StochasticBurgersequation
Considerthefollowinginitial/boundaryvalueproblemfortheBurgersequation
⎧
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎩
∂
u∂t
+
u∂u
∂x
=
1 2∂
2u∂
x2+
f(x,t)
x∈
[0,
2π
] t≥
0 Periodic B.C. u(x,0,
ξ
)
=
u0(x
;
ξ
)
(40)whereu0
(
x,
ξ
)
isarandominitialconditiondependingontwouniformlydistributedrandomvariables(ξ
1,
ξ
2)
(uniformin[−
√
3,
√
3]
)u0
(x,
ξ
)
=
1+
ξ
1(
ω
)
sin(x)
+
ξ
2(
ω
)
cos(x)
(41)and f
(
x,
t)
isadeterministicforcingtermdefinedasFig. 6.StochasticBurgersequation:Samplelocationsin(ξ1,ξ2)-spaceforeachleveloffidelitysuperimposedtothecontourplotsofthefielda35(1,ξ1,ξ2) (seeEq.(43)).
Fig. 7.StochasticBurgersequation:Univariaterecursiveco-kriging.Predictormeanandstandarddeviationofthecoefficienta35attimet=1.Todetermine therecursiveco-krigingmodel,wecomputed11high-fidelity,29medium-fidelityand64low-fidelitysolutionsamplesof(40)–(42).Thesamplelocations inthe(ξ1,ξ2)-spaceareshowninFig. 6.
Wewouldliketousemulti-fidelityGPRtoinferthestatisticsoftherandomfield u
(
x,
t,
ξ
)
attimet=
1.Tothisend, we representuintermsoftheFourierseriesu(x,t, ξ1
, ξ
2)
=
N/2
q=−N/2aq
(t, ξ
1, ξ
2)e
iqx (43)and run stochastic simulations of (40) by using the probabilistic collocation method with a different number of spatial modes N, i.e., a differentresolution inspace. Specifically, we considered N
=
15 (low-fidelity), N=
20 (medium-fidelity) andN=
60 (high-fidelity).Westudytheaccuracyandthecomputationalcostofmulti-fidelityGPRinmodelingthesolution to(40)–(42)att=
1.Inparticular,we considerthe univariaterecursive co-krigingapproach,4 inwhichwe determine(in a multi-fidelitysetting)each functionaq(
1,
ξ
1,
ξ
2)
(q= −
N/
2,
...,
N/
2)appearing in(43)independently oftheothers.We remarkthat inprinciple one canderive theanalytical solutionto theinitial/boundary value problem(40)anddetermine aq(ξ
1,
ξ
2,
t)
.Inthisstudy,however,werelyontheaccuratenumericalstochasticsolutionweobtainedonatensorproduct Gauss–LegendreLobattogridof40×
40 pointsin(ξ
1,
ξ
2)
(seealso[25]).Todetermine therecursive co-krigingmodel ofeachaq
(
1,
ξ
1,
ξ
2)
, wecomputed11 high-fidelity (N=
60), 29medium fidelity(N=
20) and64low fidelity(N=
15) solution samplesofequation (40).InFig. 6 we showthe sample locations foreachleveloffidelitysuperimposedtothecontourplotsofthefielda35(
1,
ξ
1,
ξ
2)
weobtainedfromhigh-fidelityruns.InFig. 7weplotthemeanandstandarddeviationofthe Gaussianrandomfield representingthecoefficienta35
(
1,
ξ
1,
ξ
2)
we obtainedformtherecursiveco-krigingapproach.Itisseenthatunivariate GPRyieldsapredictorwithgoodapproximation properties,withrelativelysmallstandarddeviation.Similarresultsareobtainedforothercoefficientsaq(
1,
ξ
1,
ξ
2)
.InFig. 8 weplotthe meanandstandarddeviationofthevelocityfieldu(
x,
t,
ξ
1,
ξ
2)
att=
1 weobtainedfromGPRmodeling. 4.3. StochasticRayleigh–BénardconvectionIn this section we study stochastic Rayleigh–Bénard convection of a Newtonian incompressible fluid within two-dimensional square enclosures. In particular we consider steady state solutions corresponding to a random (uniformly
4 Inthis casemultivariaterecursiveco-krigingisnotpractical.Infact, ifwesetN=60 (high-fidelitysimulation),weobtainasetof121 functions aq(1,ξ1,ξ2),whichyieldsamultivariatecovariancemodelrepresentedbyamatrix7381 timesbiggerthantheunivariatecase(ifwesampleeachaqatthe samesetofnodes).
Fig. 8.StochasticBurgersequation:Univariaterecursiveco-kriging.Shownarethepredictormean,thepredictorstandarddeviationandtheabsoluteerror ofthepredictormeanrelativetotheexactsolutionto(40)–(42)att=1 andfortwodifferentsections:x=0 (firstrow),x=π(secondrow).Todetermine therecursiveco-krigingmodel,wecomputed11high-fidelity,29medium-fidelityand64low-fidelitysolutionsamplesof(40)–(42).Thesamplelocations inthe(ξ1,ξ2)-spaceareshowninFig. 6.
Fig. 9.(a)Schematicofdimensionlessgeometryandtemperatureboundaryconditions.Thesidewallsofthecavityareassumedtobeadiabaticwhile
thehorizontalwallsarekeptatconstanttemperature.Thevelocityboundaryconditionsareofno-sliptype,i.e.∂ψ/∂x=0 and∂ψ/∂y=0 atthewalls. (b) Spectralelementmeshweusedforthelocalmodelingofthetemperature field.
distributed)Rayleighnumber[31].SuchsolutionsaregovernedbytheOberbeck–Boussinesqequations
∂ψ
∂
y∂
∇
2ψ
∂
x+
∂ψ
∂
x∂
∇
2ψ
∂
y= −
P r∇
4ψ
+
Ra(ω
)
P r∂
T∂
x,
(44)∂ψ
∂
y∂
T∂
x−
∂ψ
∂x
∂
T∂
y= ∇
2T, (45)where
ψ
denotes the dimensionless stream function, T the dimensionless temperature field, while Ra and P r are the RayleighandthePrandtlnumbers,respectively.A sketchofthegeometrytogetherwiththeboundaryconditionsassociated withthesystem(44)–(45)isshowninFig. 9.Werepresentthetemperatureandvelocityfieldsintermsofseriesexpansions relativetogloballydefinedeigenfunctionsψ
n(
x,
y)
andm
(
x,
y)
as(see[31])ψ (x,
y)=
Nv n=1 An(Ra) ψ
n(x,
y) , T(x,
y)=
NT m=1 Bm(Ra)
m(x,
y) . (46)Fig. 10.StochasticRayleigh–Bénardconvection.Steady-statetemperaturefieldcorrespondingtoone-rollconvectionpatternswithinawiderangeofRayleigh numbers,i.e.,between2.6×103and105.Shownaresimulationsresultswithdifferentresolutionsinphysicalspace.
Fig. 11.StochasticRayleigh–Bénardconvection.GPrepresentationofthecoefficienta(loc)12 (Ra)intheexpansionofthelocaltemperaturefield(47)within thespectralelement25showninFig. 9.Weplotthepredictormeans(lines)andstandarddeviations(bands)obtainedwithunivariaterecursiveco-kriging andmultivariaterecursiveco-krigingwithnon-separablecovariancemodel.
Ourgoalistorepresentthesteady-statetemperaturefieldcorrespondingtotheone-rollflowpatternbyusingGPRwithin awiderangeofRayleighnumbers,i.e.,between2
.
6×
103 and105.Tothisend,weperformedvariablefidelitysimulations ofthe system(44)–(45)both in physicalspace (i.e.,by varying Nv and NT in (46)) as well asinprobability space, (i.e., bysamplingRaatadifferentnumberofcollocationpoints).Inparticular,weconsideredthreelevelsoffidelityinphysical space:thelow-fidelitysolutionisobtainedbysettingNv=
NT=
20 in(46),themediumfidelitysolutionisobtainedwith Nv=
NT=
50 andthehigh-fidelitysolutionisobtainedwithorderNv=
NT=
100 (seeFig. 10).TomodelthetemperaturefieldwithintheGPRframeworkwefirstdecomposethespatialdomain
[
0,
1]
2 into49 square spectral elements ofequal size.Ineach element we approximatethetemperature fieldby usinga fourth-order Legendre polynomialexpansion.Thisyields NTl=
25 localdegreesoffreedom,i.e.25functionsoftheRayleighnumberrepresenting thelocaltemperatureasTloc
(x,
y)=
NTlj=1
a(jloc)
(Ra)φ
j(x,
y), (47)where
φ
i(
x,
y)
arepolynomialbasisfunctions(tensorproductofLegendrepolynomials).ToperformGPRinmulti-fidelitysettingwesampledthesolutionof(44)–(45)atdifferentRayleighnumberswithinthe range
[
2.
6×
103,
105]
. Inparticular, we computed5equally-spacedsamples ofthehigh-fidelity model (Nv
=
NT=
100), 9 equally-spaced samples of medium-fidelity model (Nv=
NT=
50) and17 equally-spacedsamples of the low-fidelity model.InFig. 11weplotonecoefficientintheexpansion ofthelocaltemperaturefield(47),i.e.,a(12loc)(
Ra)
inthespectral element 25 (see Fig. 9), versus the Rayleigh number.Other coefficientshave similar trend.We also compare such exact resultswithGPRmodelsobtainedbyusingunivariaterecursiveco-krigingandmultivariaterecursiveco-kriging with non-separable (independent coregionalization)covariance. It isseen that both theseGPRapproaches yield relatively accurateFig. 12.StochasticRayleigh–Bénardconvection.MeanandstandarddeviationoftheGaussianprocessrepresentingthetemperaturefieldasafunctionof theRayleighnumber.Shownareresultsobtainedwithunivariaterecursiveco-krigingandmultivariaterecursiveco-krigingwithnon-separablecovariance.
results.BytakingthelinearcombinationofallGPsrepresentingthecoefficientsa(jloc)
(
Ra)
wecanrepresentthelocal tem-perature field (47) aswell asthe globalone in terms ofGPs. This isdone in Fig. 12for GPRmodels obtainedby using univariaterecursiveco-krigingandmultivariaterecursiveco-krigingwithnon-separablecovariancefunctions.Itisseenthat inbothcasesthemaximumstandarddeviationofthepredictoriswithin3% ofthemaximummeantemperaturewhilethe absoluteerrors relativeto thehigh-fidelity benchmark solutionare smallerthan1.5e-1(univariate co-kriging)and8.6e-2 (multivariateco-kriging withnon-separablecovariance),respectively.Theabsoluteerrorsherearecomputedbytakingthe absolute value of difference betweenthe high fidelity solutionand the meantemperature field obtained fromGPR. The standard deviationof univariate co-krigingpredictor, however,isvery smallcompared withtheabsoluteerror, especially for Ra5600. Onthe other hand, the multivariate predictor produces uncertainty estimates which are in much better agreementwiththeabsoluteerror.5. Summary
Inthispaperweproposedanewmultivariaterecursiveco-krigingapproachforpredictionofrandomfields.Ourmethod buildsuponrecentworkonrecursiveco-kriging[12]andextendsittovectorvaluedfunctionsandvarioustypesof covari-ances,includingseparableandnon-separablecovariances.Theframeworkweproposedisgeneralanditcanbeemployedto
performmulti-fidelitydatafusion,i.e.,predictionofvector-valued fieldsbasedonvariable-fidelitymodels.Weappliedthe proposed multivariaterecursive GPRapproachto thestochastic Burgersequation andthestochastic Oberbeck–Boussinesq equations.Inbothcasesthepredictormeanandstandarddeviationaswellastheabsoluteerrorrelativetothebenchmark stochastic solutionare very small, suggestingthatthe proposed methodscan yield highly accurateresults. Regardingthe computationalcost,themultivariateGPRapproachwithnon-separablecovariancefunctionsismoreexpensivethanthe uni-variateGPRapproachappliedtoeachcomponentofthevectoroutput.Themainreasonisthatinthemultivariateapproach wemodelallcross-correlationsbetweendifferentvectorcomponents,whichyieldsalargematrixofcross-covariancesthat needstobetrainedondata.ThereisnoconsensusonwhethermultivariateGPRcanachievebetteraccuracythanunivariate GPRappliedindependentlytoeachcomponentofthevectoroutput(see,e.g.,[21]).Thenumericalresultspresentedinthis paperindeed show that they havecomparable accuracy. Onthe other hand,ifthe components ofthe vector output are highlycorrelatedthenthemultivariateGPRapproachwithseparablecovariancefunctioncanoutperformunivariateGPRas wellasmultivariateGPRwithnon-separablecovariances(see[10]),bothintermsofaccuracyandcomputationalcost.The mainreasonisthatintheseparableapproachthesamecovariancemodelisappliedtoallcomponentsofthevectoroutput, yieldingsignificantcomputationalsavingswhenlearningthemodelparametersfromdata(ithasthesamecostofasingle univariateGPR).Thisapproach,however,isaccurateonlyifthecomponentsofthevectoroutputarehighlycorrelated. Acknowledgements
This work was supported by DARPA EQUiPS grant N66001-15-2-4055 and by the Army Research Office (ARO) grant W911NF-14-1-0425.
References
[1]I.Babuska,F.Nobile,R.Tempone,Astochasticcollocationmethodforellipticpartialdifferentialequationswithrandominputdata,SIAMJ.Numer. Anal.45 (3)(2007)1005–1034.
[2]V.Barthelmann,E.Novak,K.Ritter,Highdimensionalpolynomialinterpolationonsparsegrids,Adv.Comput.Math.12(2000)273–288.
[3]I.Bilionis,N.Zabaras,Multi-outputlocalGaussian processregression:applications touncertaintyquantification,J.Comput.Phys.231 (17)(2012) 5718–5746.
[4]H.J.Bungartz,M.Griebel,Sparsegrids,ActaNumer.13(2004)147–269.
[5]H.Cho,D.Venturi,G.E.Karniadakis,Karhunen–Loéveexpansionformulti-correlatedstochasticprocesses,Probab.Eng.Mech.34(2013)157–167. [6]S.Conti,A.O’Hagan,Bayesianemulationofcomplexmulti-outputanddynamiccomputermodels,J.Stat.Plan.Inference140 (3)(2010)640–651. [7]O.G.Ernst,A.Mugler,H.-J.Starkloff,E.Ullmann,Ontheconvergenceofgeneralizedpolynomialchaosexpansions,ESAIM:Math.Model.Numer.Anal.
46 (2)(2012)317–339.
[8]J.Foo,G.E.Karniadakis,Multi-elementprobabilisticcollocationmethodinhighdimensions,J.Comput.Phys.229(2010)1536–1557. [9]A.I.J.Forrester,A.Sóbester,A.J.Keane,Multi-fidelityoptimizationviasurrogatemodelling,Proc.R.Soc.A463 (2088)(2007)3251–3269.
[10]T.E.Fricker,J.E.Oakley,N.M.Urban,MultivariateGaussianprocessemulatorswithnonseparablecovariancestructures,Technometrics55 (1)(2012) 47–56.
[11]T.E.Fricker,J.E.Oakley,N.M.Urban,MultivariateGaussianprocessemulatorswithnonseparablecovariancestructures,Technometrics55 (1)(2013) 47–56.
[12]L.LeGratiét,Buildingkrigingmodelsusinghierarchicalcodeswithdifferentlevelsofaccuracy,in:11thAnnualMeetingoftheEuropeanNetworkfor BusinessandIndustrialStatistics,2011.
[13]L.LeGratiét,Multi-fidelityGaussianprocessregressionforcomputerexperiments,Ph.d.thesis,2013.
[14]L.LeGratiét,Bayesiananalysisofhierarchicalmulti-fidelitycodes,SIAM/ASAJ.Uncertain.Quantificat.1(2013)244–269.
[15]L.LeGratiét,J.Garnier,Recursiveco-krigingmodelfordesignofcomputerexperimentswithmultiplelevelsoffidelity,Int.J.Uncertain.Quantificat. 4 (5)(2014)365–386.
[16]P.Hennig,M.A.Osborne,M.Girolami,Probabilisticnumericsanduncertaintyincomputations,Proc.R.Soc.A471 (2179)(2015)20150142. [17]J.S.Hesthaven,S.Gottlieb,D.Gottlieb,SpectralMethodsforTime-DependentProblems,CambridgeUniv.Press,2007.
[18]D.R.Jones,Ataxonomyofglobaloptimizationmethodsbasedonresponsesurfaces,J.Glob.Optim.21 (5)(2001)345–383.
[19]M.C.Kennedy,A.O’Hagan,Predictingtheoutputfromacomplexcomputercodewhenfastapproximationsareavailable,Biometrika87 (1)(2000) 1–13.
[20]M.C.Kennedy,A.O’Hagan,Bayesiancalibrationofcomputermodels,J.R.Stat.Soc.,Ser.B,Stat.Methodol.63 (3)(2001)425–464.
[21]J. Kleijnen,E.Mehdad,Multivariateversusunivariatekrigingmetamodelsfor multi-responsesimulationmodels,Eur.J.Oper.Res.236 (2)(2014) 573–582.
[22]J.Knap,N.R.Barton,R.D.Hornung,A.Arsenlis,R.Becker,D.R.Jefferson,Adaptivesamplinginhierarchicalsimulation,Int.J.Numer.MethodsEng.76 (4) (2008)572–600.
[23]L.W.T. Ng, M.S. Eldred, Multifidelity uncertainty quantification using nonintrusive polynomial chaos and stochastic collocation, in: 53rd AIAA/ASME/ASCE/AHS/ASCConferenceonStructures,StructuralDynamicsandMaterials,Honolulu,Hawaii,2012.
[24]J.K.Ord,FamiliesofFrequencyDistributions,Griffin’sStatisticalMonographs&Courses,HafnerPub.Co.,1972.
[25]P.Perdikaris,D.Venturi,J.Royset,G.E.Karniadakis,Multi-fidelitymodelingviarecursiveco-krigingandGaussianMarkovrandomfields,Proc.R.Soc.A 471 (2179)(2015)1–22.
[26]S.B.Provost,H.-T.Ha,D.Sanjel,Onapproximatingthedistributionofindefinitequadraticforms,Statistics43 (6)(2013)597–609. [27]C.E.Rasmussen,C.K.I.Williams,CovarianceFunctions,TheMITPress,2006.
[28]S.O.Rice,Distributionofquadraticformsinnormalrandomvariables-evaluationbynumericalintegrations,SIAMJ.Sci.Stat.Comput.1 (4)(1980) 438–448.
[29]M.L.Stein,InterpolationofSpatialData:SomeTheoryforKriging,SeriesinStatistics,Springer,NewYork,Berlin,1999.
[30]D.Venturi,Afullysymmetricnonlinearbiorthogonaldecompositiontheoryforrandomfields,PhysicaD240 (4–5)(2011)415–425. [31]D.Venturi,X.Wan,G.E.Karniadakis,StochasticbifurcationanalysisofRayleigh–Bénardconvection,J.FluidMech.650(2010)391–413.
[32]D.Venturi,X.Wan,R.Mikulevicius,B.L.Rozovskii,G.E.Karniadakis,Wick–Malliavinapproximationtononlinearstochasticpartialdifferentialequations: analysisandsimulations,Proc.R.Soc.A469 (2158)(2013)1–20.
[33]X.Wan,G.E.Karniadakis,Multi-elementgeneralizedpolynomialchaosforarbitraryprobabilitymeasures,SIAMJ.Sci.Comput.28 (3)(2006)901–928. [34]H.Wendland,Piecewisepolynomial,positivedefiniteandcompactlysupportedradialfunctionsofminimaldegree,Adv.Comput.Math.4 (1)(1995)
389–396.
