Basic concepts

The choice of a suitable sampling scheme is a problem common to many enquiries in science and engineering, including modelling studies, and the design requirements are often satisfied using response surface methodology (explained below) or similar techniques, for which there is now a large standard literature (e.g. Box et al., 1978; Box and Draper, 1987; Kleijnen, 1998; Law and Kelton, 1991; Vose, 1996; Wu and Hamada, 2000). The review of terminology and the related points here are therefore drawn from these references.

Responses, factors and sample points

In any experiment, whether the experimental unit is a real system or a model thereof, output (the response) is obtained under a particular set of conditions (a run). The response may be expressed in terms of different measures or ‘metrics’ (e.g. drainage density, sediment yield). The experiment, however, should identify whether variables changed during the run - the

factors - have any effect on the response. In this context, the input parameters in a

numerical simulation are equivalent to the factors changed in any experiment. Likewise, the parameter values used are equivalent to the levels of those factors. The concept of factor levels is crucial in experiment design (e.g. Box et al., 1978; Wu and Hamada, 2000) and will be returned to in more detail in subsections 3.3.4 to 3.3.6. In this thesis, the term

parameter case is used to mean any combination of parameter levels (or values) used in

running a full simulation or in making a prediction with a metamodel2.

Where two or more parameters’ levels are varied, the particular combination of levels can be represented as a single point, called a design or sample point. The point can be thought of as being located at the end of a vector in the model’s parameter space, the opposite end being at the origin, which represents all the central (base case) parameter levels. Where the model has two or three parameters only, the parameter space can be drawn as an area or volume, and the position of the design point is not difficult to visualise. As LEMs typically have twenty or more process parameters (Table 2.2 q.v.), their parameter spaces are hyper- dimensional, but the concept of a design or sample point still applies.

The use of appropriate combinations of design levels should therefore not only demonstrate which parameters are influencing the response, but also allow those influences to be

quantified, including their strength, and any characteristic trend or form in the relationship between the response and the parameter values. (e.g. Box et al., 1978; Box and Draper, 1987; Wu and Hamada, 2000). Quantifiable influences of this kind, which are generally related to a particular reference time, are usually called effects, and should be distinguished carefully from the wider, more usual use of the term ‘effect’ in geomorphology. Strictly speaking, in the sense used herein, quantified effects of geomorphological variables are not therefore identifiable if a simulation experiment uses only one value for that variable. The

2

The author finds it helpful here to distinguish between parameter cases and parameter sets. Thus, two different LEMs will require different parameter sets, these comprising the coefficients, exponents and so on used in the different models; similarly, the same LEM may be applied using different parameter sets if the LEM permits alternative formulations to be used to implement the same process. By contrast, a parameter

point is considered again, in the discussion in Chapter 7, where the wider implications of this research are considered. The concepts applicable to quantified effects, in the manner being considered here, are explained next.

Main effects and interaction effects

If a parameter (or factor) has an influence which is clearly discernible in the response (i.e. it can be isolated from the influences of the other parameters or factors), then it is said to have

a main effect (e.g. Box et al., 1978; Law and Kelton, 1991; Wu and Hamada, 2000).

Accordingly, in a LEM, to estimate whether a parameter has a linear main effect, at least two design points are required, with the parameter at one level in the first simulation, and at another level in the second (ibid.). To estimate effects which are non-linear, including those demonstrating curvature, more levels need to be used for each parameter, and the parameter space must be sampled with more design points (ibid.). It follows that although it is

generally assumed that main effects are continuous i.e. with no break or sudden jump in the response as the parameter values are changed across the range of interest, additional

sampling may be required to identify irregular and non-continuous effects.

If the size of the main effect of one parameter is increased or reduced by changing the levels of another, the two factors are said to have an interactive effect, or more simply an

interaction (e.g. Box et al., 1978; Law and Kelton, 1991). Where the interaction involves

two factors only, this is called a two-way or two-factor interaction; likewise, a higher-way interaction may involve three factors or more. As with main effects, interactions may also be linear, curvilinear, or discontinuous.

One way to visualise the way in which two parameters affect a model’s response is to plot the response as a surface. The resulting surface is called a response surface, and methods which exploit properties of the surface form the basis of response surface methodology (‘RSM’) (e.g. Box and Draper, 1987). Figure 3.2 shows an example of a 2-D response surface.

Figure 3.2: Example of a 2-D response surface, formed by the response, y, plotted against two variables, x1 and x2. (Arbitrary scales throughout)

Figure 3.2 shows a smooth, curved response surface, but responses may be more

complicated than this, with steps and irregular features. A response surface also need not be confined to two variables or factors, but may exist in many dimensions mathematically, even though it is not possible to plot or visualise simply.

RSM has been widely applied in optimisation studies, where attention is focused on finding a region of the response surface with useful properties (e.g. Box and Draper, 1987; Wu and Hamada, 2000). However, it can also be used to obtain a better understanding of a system generally, and this may be achieved by spreading the sample points more widely, including to the intended limits of the parameter space. In these circumstances, and through the use of

a response surface model, prediction of the response across the full (unsampled) parameter

space may be based on interpolation of the response surface between the sampled points. Consequently, as more points are sampled, a more complete impression of the response surface is acquired.

In deriving a response surface model, the analysis will usually reveal a number of main effects and interactions which appear to be important, although some will be more

statistically significant than others. If a researcher wishes to derive a parsimonious model demonstrating a good fit to the data, as evidenced by a high target R2 score for example, there may be difficulties in deciding between main or interaction effect terms of broadly the

Response, y

Variable x1

same significance. To simplify the choices, Wu and Hamada (2000) list three general principles to follow, listed below:

• Hierarchical Ordering. This principle comprises two elements, namely that (i) lower order effects are more likely to be important than higher order effects, and (ii) effects of the same order are equally likely to be important. For example, a linear term in a

response surface model is to be preferred over a quadric or cubic term if the explanatory powers of each appear to be broadly the same; similarly, widely applied functions, such as logarithms or exponentials, are preferred to long and complex polynomials.

• Effect sparsity. This principle states that the variation exhibited in the data is primarily attributable to fewer of the effects, and therefore to fewer of the factors, rather than to all of them. This conforms with the idea of parsimony, or “Ockham’s razor”3.

• Effect heredity. This principle states that before including an interaction effect in a model, at least one of its contributing factors should demonstrate a significant main effect. Wu and Hamada (2000) comment that this is primarily useful in the model building phase of a study, as it provides a consistent reason for eliminating many of the possible two-factor and higher-way interactions from the analysis.

Taking these principles together, initial sampling should therefore give priority to estimation of main effects, two-factor and lower-way interactions, and to linear trends or simple

curvature in either. Having covered these basic points and principles, specific methods of sampling are now reviewed.