Parameter Optimisation

4.5.3.1 Simulated annealing

Simulated annealing (SA) is a randomised local search method based on the simulation of annealing a metal, which was firstly described by Kirkpatrick et al in 1983 [Kirk 83]. The application of SA has been widely used to address optimisation problems with/without constraints. The principle of SA models is to heat and lower system temperature iteratively to reduce system defects and to enable the system to reach a minimum energy status.

In order to search for optimum solution, SA randomly generates a trial solution based on the current solution at each step. The trial solution is evaluated by an objective function to decide if the trial solution can be accepted to replace the current one, which is called an acceptance probability. The system favours to go downhill by lowering the temperature when the temperature is high. Whereas uphill moves enables the system to escape from the local minima, without which, the system would be trapped into a local minimum. This probability is controlled by temperature in such a manner that at the beginning of the procedure the temperature is sufficiently high, in which a high probability is available, and as the calculation proceeds the temperature is gradually decreased, lowering the probability. The procedure of SA consists of two nested loops: an outer loop and an inner one, where the cooling and the equilibrating processes are performed, respectively. This search procedure is iteratively repeated and won’t stop until the terminated conditions reach.

The SA routine is represented in pseudocode format in Algorithm 1, and implementation details can be found in the following.

Require:

choose the initial solution; calculate the cost (fitness); repeat

outer loop (cooling) repeat

inner loop (equilibrating) perturb the current solution; calculate the cost;

accept the trial solution as a new solution by acceptance probability; until

iteration limit is reached;

temperatureis lowered according to the cooling schedule;

until termination condition is satisfied; return Optimal solution

Algorithm 1: Pseudocode representation of simulated annealing

4.5.3.2 Implementation of parameter optimisation

The selected parameters were optimised by constraining them so that the simulation would reproduce qualitative features of the experimentally observed PCP phenotypes. For example, one case involves simulating clones of cells lacking fz function surrounded by wild-type cells, and constraining the parameters to make sure at the end of the simulation, reversed hair polarity is produced in a wild-type cell that is distal to the mutant cells. The degree of asymmetry was scored in this cell based on the asymmetry distribution of the FFD concentration at the end of a simulation for a given set of parameters.

Sf z = −

[F F D]distal_total − [F F D]proximal_total

[F F D]total

Then the constraint was implemented as a quadratic penalty function with weight, Kf z, that was nonzero when the polarity in this cell was not reversed above a threshold target

value, Tf z.

Ff z =

(

0 Sf z > Tf z

Kf z× (Tf z − Sf z)2 Sf z < T f z

This form of the penalty function has the advantage of having a continuous derivative and does not require that the initial parameter set be feasible. Two feature constraints corresponding to the characteristic PCP phenotypes listed in Table 4.3 were used to train the models. A fitness function evaluation of the SA procedure described in Algorithm 1

was applied to different cases. The parameters were selected by iteratively running

the optimisation routine after adjusting the weights of the penalty functions for each of the individual feature constraints, cooling and equilibrating processed until the desired behaviour of the system was achieved. In order to reduce the amount of computational effort required for each fitness function evaluation, cases involving mutant clones were simulated on lines of wild-type cells extending parallel to the proximal-distal axis and interrupted by mutant cells (see Figure 4.6) using models in a size of either 6 or 18 cells. However, it still demands very heavy computational power to run the optimisation routine (see Figure 4.7) due to the search space for the rate parameters and the number of underlying ODEs of the simulated model to solve (e.g. 6-cell model comprising of 1,086 underlying ODEs, while 18-cell consists of 3,258 underlying ODEs) each time manipulating the penalty function.

The same SA optimisation routine was applied to the family of models generated in Section 4.4 .

Model Fitness Constraint description

Biased model with a fz clone Ff z > 0 in cells distally

adjacent to the fz-

clone; Ff z < 0 in other

wild-type cells far away from the clone

Distal domineering

nonautonomy [Gubb 82]

Table 4.3: Characteristic PCP phenotypes used as constraints to fit the HCPN models.

Figure 4.6: Cell configuration for parameter optimisation: (a) simulated cell array comprises 6 cells: four cells distally to the mutant cell are chosen to be fit (labelled by x), (b) simulated cell array comprises 18 cells: five cells are selected to fit (labelled by x). Mutant cells are labelled in red.

identified, they were verified over a larger two-dimensional honeycomb lattice to see if the model could still exhibit the desired behaviour which was used to produce the feature constraints. The selected parameters are shown in Table A.2.

The parameter optimisation routine (SA) was implemented in Matlab 7.11.0R

[MATL 11] because Matlab facilitates to plot and examine the fitness function values makes it easier to follow the progress of the parameter optimisation. A procedure was followed in order to implement the optimisation routine: (i) exporting ODEs of an unfolded Petri nets at a required size from Snoopy [Hein 12b, Rohr 10]; (ii) converting these ODEs to a Matlab readable format; (iii) applying the SA routine to optimise the parameters in the ODE model. The computational effort required by the optimisation algorithm is small compared to the effort required in evaluating the fitness functions due to the time-costing of solving

0 2 4 6 8 10 12 14 16 18 1000 iterations 5000 iterations R u n ti m e (d ay s)

Runtime vs Iterations: 6- & 8-cell models

6-cell model 18-cell model

Figure 4.7: Runtime for parameter optimisation (time unit is in days). It shows the

relationship between the runtime of SA routine and the number of iterations as well as the number of ODEs to be solved (representing by the size of the model): (i) the more iterations the SA routine implements, the longer runtime is; and (ii) the larger the model is, the more computational effort is required.