Dynamical Gaussian Processes

When the simulator is predicting dynamic outputs, i.e. time histories a of particular quantity, it may be beneficial to incorporate temporal knowledge into the GP emulator. There are several approaches that exist within the literature, categorised broadly into two main approaches: Autoregressive (AR) and state-space formulations. An AR approach models the next output in a times series as some mapping from past observations, whereas a state-space framework describes the outputs as a Markov

process with one or more states evolving in time by means of a transition function. Frigola-Alcalde provides a detailed overview in [139].

The AR approach leads to the formulation of a Nonlinear Auto-Regressive eXogenous inputs (NARX) (where exogenous are external inputs) model, where previous outputs of the model become inputs to the next time point. In the GP formulation these past observations become part of the input set to a GP, mapping the nonlinear transition to the next time point [139–142].

A nonlinear state-space approach can be formulated in a variety of ways [139, 143, 144]. Firstly, either the transition or observational models or both can be assumed a GP. These prior assumptions are used to reflect the belief that the nonlinearity is contained within either the system dynamics or the measurement respectively [139]. Additionally both the transition and observational models can be GPs (known as the full GP-state space model), however this can lead to non-identifiability problems, as both models are flexible nonlinear functions [139, 144]. Key challenges to this approach are computational complexity, identifiability issues and interpretability.

4.5

Conclusion

Simulators are used throughout engineering and are an integral part of forward model-driven SHM. The majority of statistical methods and optimisation techniques that analyse or incorporate simulators require numerous evaluations. These methods may not be practically feasible when the simulator is computationally expensive to run. For this reason emulators, computationally efficient surrogates of a simulator, are employed.

A variety of tools have been implemented as emulators throughout the literature, notably ANNs, PCE, BLA and GPs. It has been discussed that only BLA and GPs quantify the uncertainty associated with replacing the simulator with an emulator — known as code uncertainty. Moreover, both ANNs and PCE can overfit, and without providing code uncertainty the user is unaware when this occurs. BLA is an approximation of Bayesian inference and only considers the mean and variance. In contrast GPs have closed form solutions to Bayesian inference, when the function can be assumed Gaussian distributed. For these reasons, when the outputs are considered jointly Gaussian, a GP emulator will be the most rigorous form surrogate model,

4.5. CONCLUSION 99

and therefore is utilised in this thesis.

The chapter has outlined derivations of a GP for the purpose of emulating a deter- ministic simulator, along with methods for dealing with numerical issues associated with the ‘noise-free’ assumption. In addition, diagnostics have been implemented on a numerical example presenting a framework for validating an emulator. An emulator must be constructed from a finite set of simulator runs, and a GMLHD has been demonstrated to improve GP predictions.

When the number of input variables N is large, GPs can become numerically intractable as they rely on the inversion of an N × N matrix which has a time complexity of O(N3). Sparse GP methods have been proposed to reduce the time complexity to O(N M2_{) and considerations for implementation in an emulator context} have been outlined. Finally, other GP extensions within the literature have been presented, such as multivariate, heteroscedastic and dynamical GPs.

Chapter 5

Bayesian Calibration and Bias

Correction

In the previous chapter model discrepancy, that occurs due to model form errors, was outlined as a problem in generating predictions from simulators that accurately represent real world observations. This is a particular issue for forward model-driven SHM as a key objective is to generate statistically representative outputs of observa- tional damage states from a simulator. This means that the calibration procedures implemented in forward model-driven SHM must consider model discrepancy as a source of uncertainty. Bayesian Calibration and Bias Correction (BCBC) is one such approach, seeking to calibrate simulator parameters whilst inferring the model discrepancy functional distributions.

The following chapter begins with a discussion of the literature before outlining the BCBC methodology. Subsequent sections demonstrate the technique on two case studies; a three story and a five storey building structure, providing a discussion on the benefits and challenges with the formulation. Lastly, conclusions are presented outlining the methodologies effectiveness within a forward model-driven framework.

5.1

Literature Review

BCBC (also known as the ‘Kennedy and O’Hagan approach’ or a modular Bayesian technique) was developed in 2001 by Kennedy and O’Hagan [68] as part of a discussion

about the correct procedure for calibrating deterministic simulators in a Bayesian manner. The key development of the paper was to outline the sources of uncertainty within a computer simulation, highlighting that model discrepancy should be inferred, along with parameter uncertainties, and proposing that it could be modelled using a GP prior. Their proposed statistical model for calibrating a simulator is as defined in Eq. (5.1).

z(x) = ζ(x) + e = ρη(x, θ) + δ(x) + e (5.1)

This statistical model provides a belief about the relationship between observations z(x) and simulator η(x, θ) that depends on a set of inputs x and parameters θ. The model assumes that the combination of simulator and model discrepancy δ(x) are equivalent to the ‘true’ process ζ(x), where model discrepancy is assumed to have a functional form. In the original formulation a regression parameter ρ is used to weight the evidence provided by the simulator relative to the model discrepancy; with this parameter informing the relative weighting between the model discrepancy and simulator — although some more recent formulations remove this term. Lastly, the observational data z(x) is modelled as the ‘true’ process with the addition of independent observational uncertainty (a Gaussian homoscedastic noise).

The framework has been applied and adapted several times within engineering. Bayarri et al. implemented the methodology on a spot weld FEA model where they discuss the differences between a modular Bayesian approach and full Bayesian analysis, stating similarities in the results [69]. Higdon et al. proposed a multivariate formulation using principle components modelled as GPs [103]. The method was demonstrated on a simulator modelling implosion in a cylinder, where output pre- dictions were demonstrated to fit the data well. However, comments in the paper indicate non-identifiability issues between the parameters and model discrepancy as well as problems in scenarios where the simulator cannot be modelled as a standard GP. The framework’s approach to model discrepancy is also discussed in a general review of model updating [16] without any definitive conclusions. In engineering de- sign, Arendt et al. present an application of the univariate method clearly indicating the problems associated with non-identifiability between the parameters and model discrepancy when non-informative or inadequate priors are used [105]. The issues with non-identifiability are approached again by Arendt et al. where multivariate GPs with separable covariances ([104]) are incorporated in order to better define