1.3 Surrogate Modelling
1.3.2 High-dimensional model representation
Another type of surrogate modelling techniques consist on variance-based ap- proaches. These type of methods are known as functional ANOVA (Analysis of Variance) in statistics, and high-dimensional model representation (HDMR) or Sobol’s representation in the UQ community. This methodology assumes the output from the simulator to be a second order stationary process. That is, the model can be characterized by the first and second statistical moments. In particular, a second order stationary process has constant mean and a covariance function that depends only on the distance of the indices. The HDMR approach considers the randomness of the output a consequence of the randomness in the input variables. The assumption is that input random variables are independent from one another. Thus, the joint distribution of X ⊆ Rd is a product d marginal
distributions. Assuming that the output of the simulator η(x) is square integrable with respect to the joint distribution for X , its Sobol decomposition is given by
η(x) = f0 + d X i=1 fi(xi) + X 1≤i<j≤d fij(xi, xj) + · · · + f1,...,d(x), (1.11)
where f0 represents the mean response of η(·), fi the individual contributions
of each variable, fij the interaction and contribution from pairs of variables i
and j, and so on. In general, this type of methods have proven to be effective for sensitivity analysis. That is, to identify the most influential variables in the variance of a simulator.
Polynomial chaos expansions
A surrogate model for η(·) belongs to the class of polynomial chaos expansions if it is written as
f (x, θ) =X
α∈A
θαΨα(x), (1.12)
where A is a set of multi-indices α = (α1, . . . , αd), {Ψa} is a set of multivariate
family of polynomials orthogonal with respect to the inner product defined by the density of X , and θα are the unknown coefficients associated with each polynomial.
Each of the multivariate polynomials Ψα can be obtained as a tensor product of
univariate polynomials
Ψα(x) = ψα(1)1(x1) × · · · × ψ
(n)
αn(xn), (1.13)
where ψα(j)j denotes the α-th order polynomial for the j-variable. Note that the orthogonality condition on the family of polynomials makes the expansion completely dependent on the assumed joint distribution for the input variables. For example, if the assumed probability distribution is a Gaussian, then the family of polynomials used is the Hermite polynomials. This assumption plays an important role in the methods used to find the unknown coefficients. For example, quadrature rules actively exploit the orthogonal property of the polynomial basis by approximating the integrals associated to the inner product. Quadrature points are chosen based on the assumed probabilistic model and these points define the set of configuration points to run the expensive simulator. This strategy is known as a non-intrusive method for solving the polynomial chaos expansion. Note that this renders the estimation of the polynomial expansion coefficients, and more importantly the design of simulator runs, completely dependent on the assumed prior distribution of simulator inputs. See [174] for a detailed discussion on polynomial chaos and non-intrusive methods.
The formulation of the surrogate in (1.12) is an infinite series in the space of polynomials, which in turn, is an infinite dimensional Hilbert space. For computational convenience, it is common to truncate the series and keep only the polynomials of degree up to p. This means that the total number of unknown coefficients for the expansion is
(d + p)!
d! p! . (1.14)
Thus the number of coefficients to be estimated grows exponentially fast as the the number of dimensions in the simulator’s input increases. Several regression-based approaches to overcome this limitation have been proposed in [33,34]. Nonetheless, the main drawback when compared to the Gaussian process model is still the computational burden of running the simulator to generate a training dataset. The reason is that the assumptions on the prior distribution of the simulator’s input determines the training dataset to be used to learn the polynomial chaos. Such is the case of using a non-intrusive method based on quadrature rules. Alternatively, a regression-based approach does not guarantee that the chosen polynomial family is a good mathematical object to approximate the simulator output. A more thorough discussion on Polynomial chaos can be found in [93,94]. A summary and a comparison of Polynomial chaos expansions and Gaussian processes as computer code emulators can be found in [175]. O’Hagan [174] provides a thorough discussion on polynomial chaos surrogates from a statistician perspective.
Low rank tensor approximations
Low rank tensor approximations have been recently developed as an alternative approach to polynomial chaos expansions [134]. In particular, they have been used in sensitivity and reliability analysis [136, 135]. The simulator is represented as a sum of a finite number of rank-one functions. That is, the surrogate is referred as a canonical decomposition with rank equal to the number of rank-one components in the model. A rank-one function is written as
ωl(x) = d
Y
i=1
vl(i)(xi), (1.15)
where v(i)l denotes a univariate function of xi. The surrogate model is defined as
f (x, θ) =
R
X
l=1
θlωl(x), (1.16)
Lastly, the family of univariate functions vl(i) is chosen as an orthonormal basis of polynomials with respect with the marginal distributions for xi. This implies an
expansion on each univariate function as v(i)l (xi) =
pi X
k=0
z(i)k,lPk(i)(xi), (1.17)
where Pk(i) is the k-th degree polynomial in the i-th input, pi is the maximum
degree of Pk(i) and zk,l(i) additional unknown coefficients for each univariate function. The total number of unknown parameters to be estimated is
R
d
X
i=1
(pi+ 1), (1.18)
which scales linearly with the number of dimensions. This provides an advantage with respect to polynomial chaos expansions. Details on the estimation of the parameters for a low rank tensor approximation can be found in [136].