Bifurcation diagrams

The trajectory of an N-dimensional system can be thought of as a path through

N-dimensional space. For any N >3, this trajectory will be difficult to visualise. A straightforward option is to plot the activation over time of each element of the system independently (Figure 4.1, upper plots). Each of these plots represents one possible trajectory of a dynamic system and already the amount of information generated is difficult to visualise in a meaningful fashion. This information can be condensed by plotting the average activation of all the nodes at each time point (Figure 4.1, lower plots):

hai= 1

N(a0+. . .+aN−1). (4.1)

A common technique for exploring the behaviour of a dynamic system is to investigate the behaviour of a family of parameterised functions. For example a family of linear systems may be described by the function fm(x) = mx where m

is varied. It is then possible to observe how the dynamics of the system change as the function is changed. The same technique may be applied to investigate the behaviour of dynamic networks by the inclusion of a parameter W that scales the

4.2 Study 1: Tools for exploring network dynamics 61

Figure 4.2: The effect ofW on the slope of the sigmoid function. AsW is increased, the sigmoid function passes from the linear range, through the nonlinear range and approximates a Boolean step function when W is very large.

net input inet into the node activation function σ

fW(x) =σ(W ∗inet). (4.2)

The scaling parameter W affects the slope of the sigmoid function. When W

is very small, fW(x) is linear. AsW increases, fW(x) passes through the nonlinear

range, eventually saturating and approximating a Boolean function when W is very large (Figure 4.2).

To obtain an insight into the dynamic behaviour of a network with a particular pattern of interactions, the trajectories originating from a single initial condition were recorded as interactions were scaled from very weak to very strong. For the examples in this section, a fully connected DRGN with 20 regulatory nodes was created with weights and biases drawn from a Normal distribution with mean 0 and variance 1.

1. The state of the DRGN was initialised toI= (I0, . . . , IN−1) where In was a

uniform random value in the range [0,1].

2. The scaling factorW was initialised to a small value (0.01).

3. The DRGN was iterated 1,000 steps to ensure that the trajectory was located on an attractor.

4. The system was iterated a further 500 steps and the average activation (Equa- tion 4.1) at each step was recorded.

62 The Dynamics of Cell Differentiation

5. The network was then reset to the initial state I, and W was incremented by 0.01.

6. Steps 3 to 5 were repeated until W = 20.0.

A qualitative picture of the dynamics of the parameterised network was ob- tained by plotting the average activation of each of the states visited in the attrac- tor orbit (Figure 4.3).

Several general statements can be made about DRGN dynamics on the basis of an orbit diagram. It is possible to clearly distinguish three different types of long-term dynamic behaviour (i.e., after discarding the first 1,000 time steps to eliminate transient fluctuations):

1. if the trajectory is located on a point attractor, all 500 values in the set are identical, and a single point appears on the plot (e.g., when W <1.0); 2. if the trajectory is located on a periodic attractor, each of the states visited

appear as discrete points (e.g., the period 2 cycle that appears when 1.0 < W <1.5);

3. if the trajectory is located on a chaotic attractor, a smear of points is pro- duced as the system visits a series of unique points within a given neighbour- hood (e.g., as occurs when 2.3< W < 5.0).

In general, the location of a basin of attraction in dynamic space moves in a gradual fashion as W is varied. At some values of W bifurcations occur and the nature of the attractor changes (e.g., around W = 1.0 and W = 1.5 the attractor bifurcates into a two-cycle and a four-cycle respectively). Between some adjacent values of W, the system dynamics change in a discontinuous fashion, suggesting that the trajectory may be jumping between different basins of attraction (e.g., aroundW = 5.2 and W = 6.0).

While bifurcation diagrams provide a qualitative picture of how the type of attractor changes asW is varied, they do not give any indication of the stability of that attractor. One possible measure of dynamic stability, the Lyapunov exponent, is investigated in the following section.

4.2 Study 1: Tools for exploring network dynamics 63 Weight Scale (W) 0 2 4 6 8 10 12 14 16 18 20 <a> 0 0.2 0.4 0.6 0.8 1

Figure 4.3: Orbit diagram for a fully connected network with 20 nodes. Each point represents the average activation hai for a single trajectory state. Each vertical slice represents all states visited on the trajectory originating from a single initial condition. Qualitative features that are visible include fixed point, cyclic and chaotic attractors, bifurcations and discontinuities. Full details of the generation and interpretation of the diagram are provided in the text.

Figure 4.4: An example Lyapunov diagram for the network used to generate Fig- ure 4.3. Each trajectory is associated with a single Lyapunov value: positive values indicate diverging trajectories (chaotic attractors) and negative values in- dicate converging trajectories (fixed point and periodic attractors). Full details of the generation and interpretation of this diagram are provided in the text.

64 The Dynamics of Cell Differentiation