In this chapter the general aspects concerning optimization and some of the available methods are discussed.
The introductory sections present some of the preliminary concepts of optimization, and a classification of objective functions.
Brief accounts of some of the available methods, their development and their applicability are given in the remaining sections of the chapter.
The purpose of this chapter is to lay the theoretical foundations for later descriptions of interactive computer graphics used in the testing, evaluation and application of optimization methods.
2.1 Preliminary Aspects of Optimization
The requirement for methods of optimization may be said to arise from the mathematical complexity necessary to describe the theory of systems. Optimization methods are used to explore the local region of operation and predict the way that the system’ s parameters should be adjusted to optimize system performance.
In an industrial process, the criterion for optimum operation is often in the form of minimum cost, where the product cost can depend on a large number of inter-related control parameters. . In the scientific fie ld , the performance criterion could be to minimize the integral of a squared difference. In both cases i t is required to minimize a single quantity by manipulation of a number of variables.
Since the beginning of this century many solution methods have
been developed to solve linear and non-linear optimization problems. These methods aim at extremisation of some entity while satisfying side conditions known as constraints.
• Linear programming methods are concerned with optimization of a linear function subject to linear constraints and their main feature is that they always lead to global optimal solutions for well formula ted linear problems. In contrast to linear programming, however, only special types of non-linear problems can be solved with an
assurance of reaching the global optimum. Depending on the non-linear problem, and the starting values assigned to the variables, a given non-linear algorithm may converge to any one of several different local optimal solutions.
In practice, most problems are constraint bound, but this type of problem can often be reduced to an unconstrained one by a suitable transformation of variables. In consequence, this thesis is mainly concerned with methods for unconstrained problems, some of which are described in this chapter and applied later on. However, the poten tia l of interactive computer graphics in the solution of constrained problems is demonstrated in the last two chapters.
2.2 Mathematical Models
In general the objective function, F, depends on n real indepen dent component variables, x-j, ^ . . . xn , often assembled for abbrevia
tion into an n-component column vector, or point, X.
- ( x - j 9 X 2 * . . . x ^ )
In many practical optimization problems there are constraints on the values of the parameters which restrict the search region. An objective function may be subjected to two types of constraints;
(a) Explicit constraints, which are defined as follows:
£ x . U. i = 1, . . . N
where and are the lower and upper bounds on the indepen dent variables x, ... x .
1 N
(fa) Implicit constraints, which are expressed as:
' A, « F. (X) « E. j = 1, ... N
J J vj
A. and 5. are the lower and upper bounds of the im plicit con-
J J
straint function F..
vJ
2.3 Local and Global Optima
As was indicated in section 2.1, an important part of non-linear optimization is the concept of local and global optima.
The parameters X ^ which give an optimum value F ^ ., of the objective function within a local area of search is termed a local optimum. The global optimum of a problem is the local optimum of all possible optima.
In practice5 especially for non-linear functions, i t is very d iffic u lt to determine i f the optimum obtained is a global one or not. Normally, a point w ill be accepted i f its position approximates to the expected location of the optimum as determined by the nature of the problem. Otherwise* further searches from a different in itia l point w ill need to be initiated.
Most optimization problems tend to aim at minimization of the objective function. Since maximization and minimization are equiva lent problems, i.e.the minimum of (F) and the maximum of (-F) occur at the same point, Min (F) = Max (-F ), only minimization is considered here.
2.4 Classification of Objective Functions
Objective functions can be grouped into four classes depending on whether they are finite or continuous, and whether their firs t or second order partial derivatives are available or not. The four classes are:
(i) Objective functions with a finite number of discontinuities, ( ii) Continuous objective functions.
( i ii ) Objective functions which possess fin ite and continuous firs t partial derivatives.
(iv) Objective functions which possess finite and continuous firs t and second partial derivatives.
The above classification gives an insight to the applicability of the available methods of optimization.
An objective function with two variables can be envisaged as the surface of a hilly landscape. The problem of minimization is to locate the lowest point on this surface with constraints defining the bounds within which the search is to be made. If the function is assumed to have a continuous and finite firs t derivative there will be at least one minimum in the feasible region. If there is no feasible minimum the required minimum must lie on the constraints. 2.5 Methods of Optimization
A non-linear problem can be approached by two sets of methods; (i) Analytical methods.
( ii) Numerical methods. -
Figure 2.1 shows a genetic tree of the methods which will be
%
described in the following sections.
CO