The different types of approximation each have their use and successful applica- tions. The scope of this thesis is restricted to automated methods for construction of (mostly) data-driven approximations for (simulations of) complex systems. These types of data-driven approximations are typically referred to as surrogate models, metamodels, replacement models, emulators or Response Surface Models (RSMs). Hereafter, the simulation of the complex system being approximated is referred to as the reference model to maintain generality: simulators are a prime example of an expensive evaluation target, nevertheless this term can be interpreted broadly. Training a model with supplied hyperparameters, performing real-life experiments, or capturing data from an operating complex system may also be considered simu- lations. These approaches however result in some output uncertainty. To further restrict the scope of this dissertation we make following assumptions:
1. The reference model has static input/output behaviour, which does not evolve over time. This excludes time series and prediction of future states.
2. For most of the contributions of this dissertation, the output is produced by computer simulations which are deterministic and quasi noise-free (with the exception of some numerical and discretisation noise). Occasionally, the role of output uncertainty (noise) is further investigated. One major exception in this dissertation can be found in AppendixB, in which data is observed directly from a running complex system (the real world) without a computer model in-between.
3. A single evaluation of the reference model is expensive to obtain, and limited information on the inner workings is available (black-box).
1 0 x1 -1 -1 -0.5 0 x2 0.5 0 300 250 200 150 100 50 1 Response x1 -1-0.8 -0.6 -0.4 -0.20 0.20.40.60.8 1 x 2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Simulation routine (black box) Simulation routine (black box) . . . Design space
Figure 1.2: Surrogate modelling: an experimental design with the design space is evaluated by the reference model. Its responses are used to obtain a data-driven approximation of the input/output behaviour.
The workflow to develop a surrogate model typically consists of the following steps:
1. Formulate model: this involves a better understanding of the system by inquiring from domain experts, and mapping their knowledge into bounded input/output parameters. This results in the formulation of the design space, and any prior knowledge which may influence other choices regarding the experimental design, model fitting or quality assessment.
2. Experimental design: select the type of experimental design to use and, if chosen, how sequential design is to be applied. This results in a set of samplesto be evaluated by the reference model. Selection of a set of samples is discussed further in Section2.5and Chapter3.
3. Model fitting: define the approach for model fitting. A specific model type may be chosen, or this may be done automatically. Also the choice of the model parameters and their optimisation is a relevant question for this step. A formal approach is given in Section2.2.
4. Model quality assessment: define how the model quality will be quantified and validated. This is one of the hardest questions to answer as it involves several stakeholders. Sections2.2and2.3provide a detailed overview.
5. Gain insight: Applying the trained model for its task results in insight into the system. Complex system Reference model (Simulation) Surrogate model Input Output
Figure 1.3: Surrogate modelling: modelling hierarchy.
The surrogate model mimics the response behaviour of the reference model. Typical tasks or use cases involving use of surrogates include:
• Optimisation
• Perform exploration over a large design space to focus further development on some specific areas.
• Visualisation of the design space, trade-offs, feasibility, optimality etc. • Replacement of legacy reference models
All of these tasks can be achieved by training a globally accurate model over the entire design space. The surrogate model then replaces the underlying system or reference model for evaluations. However, for some of these use cases, more specific and more efficient (in terms of number of required evaluations of the reference model) approaches exist.
1.6.1
Surrogate-based optimisation
Surrogates excel in the optimisation of expensive objective functions [14]. This discipline is often referred to as Surrogate-Based Optimisation (SBO). A globally accurate surrogate model can be built and optimised using traditional optimisation methods such as gradient descent, or meta-heuristics such as particle swarm optimi- sation [15]. Although this approach is correct and works faster than evaluating each call of the objective function on the reference model, it is not necessarily the most efficient methodology. When seeking a minimum, less samples can be devoted to regions that clearly show to be the opposite. Applying sequential sampling to explore the search space for optima and exploit the available knowledge to refine optima results in a more optimal process.
1.6.2
Sensitivity analysis
A different use-case of surrogate models is sensitivity analysis of the complex system. Especially when many input parameters are present, it is very difficult to achieve global accuracy due to the exponential growth of the input space. Fortu- nately not all input parameters contribute equally to the output variability, in fact some might not have any impact at all [16]. The surrogate models can be used directly for evaluation-based sensitivity analysis methods such as Sobol Indices [17], Interaction Indices [18] or gradient-based methods.
For some kernel-based modelling methods analytical computation of sensitivity measures is possible resulting in faster and more reliable estimation schemes, even before global accuracy is achieved [19]. This intuitively makes sense as model belief on the input sensitivity may be interpreted as a necessary precondition to high accuracy.
1.6.3
Inverse surrogate modelling
All tasks described so far were forward tasks, mapping samples from a design space to the output or objective space. It is also possible to reverse the method, referred to as inverse surrogate modelling, which can be interpreted as identifying the areas of the design space corresponding to a certain desired or feasible output range.
complex system
input output
Forward modeling Inverse modeling
Figure 1.4: Forward versus inverse surrogate modelling
Typical approaches involve training a (forward) surrogate model first, then optimis- ing the model using an error function between the output and the desired output as objective function. This optimisation is typically preferred to be robust to account for the error of the forward surrogate model [20]. Next, specific sampling schemes to identify these regions directly were also proposed [21]. Finally, it is possible to translate the inverse problem into a forward problem involving discretising the output (feasible/infeasible point) and learning the class boundaries, using the approaches described in Chapter3.