Persistent Mutual Information in Dynamical Systems

PMI is fundamentally a probabilistic notion. The need for something fulfilling its role arises naturally in the context of stochastic systems, where descriptions of states at different times are done on the level of measures over the state space. The future is not fully determined by the present, and so uncertainty enters the system through the evolution rule. It is this factor that invites exact statements and leads to quantities such as PMI, entropy rate, ex- cess entropy, and others.

Deterministic systems, on the other hand, do not allow for any doubt in evolution. In order for PMI to make sense in this setting we need to let some aspect of the system admit uncer- tainty. On the more philosophical ground this introduction of probability can be interpreted as working with incomplete knowledge about the given aspect.

The usual definition of a dynamical system as a state space combined with an evolution rule gives at least two levels where this uncertainty may enter (Crutchfield and Packard [1983]), plus a combination of the two. The first leads to ‘noisy’ systems defined by evolution rule supplemented with an error, studied in for example White et al. [1981]. This could also, of course, be interpreted as incomplete resolution of the state space. We take up a similar idea, but consider this partial knowledge as being a feature of the observer, and not the map. In short, we ask the question of what information a limited resolution of the initial state can provide about the final outcome,defined with the same level of uncertainty. In this second level uncertainty enters the dynamical system at the level of knowledge of the initial condition. This notion is supported in Farmer et al. [1980]: “prediction must be discussed in terms of ensembles of initial conditions rather in terms of the behaviour of individual points”.

We now define these concepts more rigorously. Let (X, F) be a (discrete) dynamical system. LetP be a partition on X into M cellsC such thatP =_{Ci :i= 1..M} and

X = M G i=1

Ci. (2.7)

With some suitableσP define a measure space (P, σP, µ).

We also admit a prior measure µp. This allows the definition of measure of the evolved system as follows.

Let

µ(Ci) :=µp(Ci). (2.8)

The evolved measure µτ

p is defined for allA⊆X through

µτ p(A) :=µp F−τA . (2.9) Then µτ(Ci) :=µτp(Ci) (2.10)

We define the joint measureµJ through the conditional, such that for any (A, B)⊆

XxX,

µJ(A, B) =µ(A)µpτ(Fτ(A)∩B). (2.11) In the cases we study the state space X _⊂ Rd, and prior measures will be the

Lebesgue measures. This means that the joint prior is indeed the product of the marginal priors, and the two will cancel. PMI will then be a function of the ‘past’ marginal only.

We begin withN i.i.d. pointsX0

i ∼ρ0(whereρ0 is the density associated with ‘past’

measure definedat a certain resolution), and evolve each withFτ _{to obtainX}τ

i (below we talk about the methods used to estimate PMI using the set ofXi = Xi0, Xiτ

,i= 1..N as data).

In this methodology we essentially reduced the semi-infinite block defined by the stochastic process approach as the ‘past history’ onto a single variable. It is possible because PMI be- ing the function of entropies, it does not manipulate values from support spaces, rather the measures of the subsets defined on the latter. Here the fully-deterministic system ensures that a pointX0

i ∈X is associated with a unique orbit which, if a symbolic block variable is required, can be rewritten in terms of indices of cells housing its consecutive elements, in a manner similar to the process of finding the metric entropy described in the Introduction. Metric entropy and other functions of blocks of variables are based on the fact that there is not a unique correspondence between the initial block and consecutive blocks. Here, on the other hand, we rely on the block corresponding to the initial point. This ensures that the measures we sample by considering only the initial and final points are the same as we would have sampled had we considered sequences of points or their symbolic representation.

Interpretation Fig. 2.1 earlier provides a visual explanation of what it means to discuss PMI in the ostensibly deterministic context of dynamical system. Without loss of gener- ality we can view the graphs by imagining the pictured axes as discretised state spaces of some deterministic map, the very setting of symbolic dynamics (and the ‘effective’ symbolic dynamics of our method). Consider a system whose marginal spaces are the same, and with the same partitioning, such that the flat past measure induces a flat future measure - as will be the case with the standard map. Simply looking at the marginal measures gives no indication of the extent to which the map loses initial information. This is exactly what the joint captures (or rather a function related to the lower limit of the rate of information loss. Orbits can be different in the intervening times but close together at some time τ, whereas if orbits differ at τ any differences in the intervening times will not ‘lessen’ the difference noted by PMI).

Consider a subset of the past marginal with some measure. We populate the subset with points whose relative number is defined by the measure of that value, and is either equidis- tributed, or, if a further partitioning exists, subdivided again. The points then get evolved by the map for τ times, and their final positions go towards contributing to the ‘future’ marginal measures. If the motion is somehow predictable or ‘causal’, then the points that were close together will tend to stay close together. Their relative distances will not de- crease. That is indeed the case in the right hand side of the figure, under the assumption that the thickness of the line is somehow indicative of the size of cells.

If, on other hand, trajectories diverge in a chaotic manner, so that any knowledge of the initial condition is lost, it is likely that there will be a much greater variation in the distri- bution of the points initially in one subset. That is what the first subfigure shows. So even when evolution is deterministic it is still possible to ask the question of how drastic a small inexactitude in the initial condition will, on average, turn out to be.

In the framework usually employed by Tsalliset al (see the next few citations for example), PMI can be viewed simply as an aggregate related to entropy production. In works such as Baldovin et al. [2003], A˜na˜nos et al. [2005], incidentally also focusing on the standard map, a number of points start equidistributed in a cell and their evolution is traced. Their positions at someτ is then added to make up the overall distribution at that time, obtained by also averaging over the location of the initial cell, which is made to be arbitrary in the state space. Evolution of the individual cell then corresponds to the horizontal movement in the figure given. The difference between approaches is also clear - whereas the authors

marginalise by averaging in the horizontal direction to get the future distribution, here we look at the evolution without losing track of the relative location of the joint points.