bandwidth in human perception, while control theory can model the action dynamics. Such an integrated view could address effectively the key question of how constraints on both sides of the feedback loop interact to bound performance.
The interplay between sensors and actuators is commonly described as a transfer of information involving three steps: estimation, decision and actuation. In the first step sensors are used to gather information from the controlled system regarding its state. This information is then processed according to a control strategy in order to determine which control dynamics is to be applied, to be finally transferred to the actuators which modify system’s dynamics, typically aiming at decreasing uncertainty. In closed-loop or feedback control techniques, actuators rely explicitly on the information provided by sensors to apply the actuation dynamics, and this relation is shown to be a zero-sum game, i.e. each bit of information gathered from a dynamical system by a control device can decrease the entropy of that system by at most one bit additional to the reduction of entropy attainable without such information [108, 109].
Information and uncertainty represent complementary aspects of control. Closed-loop methods obtain information about system variables, and use that information to decrease the uncertainty about the values of those variables. Therefore, in a control process information must constantly be acquired, processed and used in order to maintain the system trajectory. Entropy is a suitable candidate for characterising uncertainty as it offers a precise measure of disorderliness or missing information by quantifying the minimum amount of resources (bits) required to encode unambiguously the ensemble describing the system. Decreasing entropy stabilises dynamical systems from disturbances associated with environmental noise, motion instabilities, and incomplete specification of control conditions. The goal in system control is typically to reach a low-entropy final state, starting from a high-entropy (random) initial state. Performance criteria, such as distance to target or energy consumption, are used to determine the optimal control, however they do not consider the information processing cost required for decision making.
2.6
Information as a Utility
Perception, information processing and actuation are usually treated as individual processes, however considering them as an ensemble introduces the perception–action feedback cycle, which is described by Fuster [28, 29] as ‘the circular flow of information between an organism and its environment in the course of a sensory guided sequence of actions towards a goal’.
There are various ways of modelling the perception–action cycle, which poses a challenge for a unified quantitative treatment. One universal approach is the information-theoretic view
inspired by Ashby [3] and developed further by many others [14, 51, 53, 108, 109]. It is general, conceptually transparent and can be post-hoc imbued with the specific constraints of particular models. On the informational level scenarios with different computational models can be directly compared with each other. The informational treatment also allows to impose constraints on the information processing capacity, and enables the consideration of the informational cost.
The concept of information, as introduced by Shannon, derives its power from the strict rejection of semantic elements in its formalism, and initially it has been doubtful whether semantics could be treated within such framework at all [7, 31]. However, emerging evidence suggests that exploring the intrinsic dynamics of information can provide various interesting utility concepts. Important information about a system’s structure can be obtained by measuring to what extent individual components contribute to information production and at what rate they exchange information between each other. Several authors emphasise the utility of having a measure for a flow of information [5, 50, 51, 114].
Various methods for studying the dynamics of information shared between processes have been proposed, which aim to detect the directionality of coupling and quantify the degree of asymmetry [84, 94]. Their goal is to assess the interaction between two subsystems by analysing the interrelation between the two signals at their outputs. In order to determine the direction of causality between two variables Granger introduced measures of causal lag and causal strength in an explicit and testable fashion [32]. Mutual information has been used widely to measure the overlap of information content between two (sub-) systems, as a natural way to quantify for deviation from independence of two processes, however it contains neither dynamical nor directional information. Shannon entropy and mutual information are properties of the static probability distributions, while the process dynamics are represented by the transition probabilities. Another utility, introduced by Schreiber [87] and called transfer entropy, aims to quantify the information transfer between two systems, by characterising the statistical coherence of the systems evolving in time.
Ay and Polani [6] proposed a measure for the strength of a causal effect, which captures essential properties of a Shannon-type quantity, while realising a flow-like philosophy different from the correlative nature of mutual information. Their concept of information flow, based on causal Bayesian networks, can be seen as an information-theoretic counterpart of the probabilistic formalism of Pearl [75]. A measure of causal flow of information could enable the quantification of a number of phenomena in the areas of synchronisation, game dynamics and the perception–action loop. For example, cooperative behaviour of coupled complex systems is related to synchronisation phenomenon. When two players adapt their strategies over time, the game dynamics could move towards cooperative or antagonistic