Methods

transmitted is given by the product

T·(S+ 1),

where it is seen that (S+ 1) is a measure of the maximum capacity that the circuitφ, seen as a communication channel, can achieve. Note that T and S correspond to the time and space complexity of the functionf. LetC(f) andC(φ) denote the minimum number of logic gates needed to implement the functionsf and φrespectively. Then,

C(f)≤T ·C(φ) =κ·T ·S,

because C(φ) = κS for some positive κ and because the implementation of f as a sequential processing machine cannot use less logic gates than the direct implementation of f. Hence, the order of the time-space product is lower-bounded by the circuit complexity of f. In some sense, it seems that:

Computer Science Information Theory Computef(x) ⇐⇒ Communicatex and f although the author is unaware of any proof of this claim.

The point is that, if we are willing to accept the time-space product as an ap-propriate measure of the computational complexity, then the computational resources can be related to information-theoretic resources—and as such, they are governed by information-theoretic principles. In other words, the computational resources corre-spond to the amount of bits that have to be transmitted to communicate a “compu-tation”. While this interpretation is intuitively appealing, there are still many open questions left. For instance, it is unclear what “computation” means in this sense, and it is also unclear how much computation is necessary in order to compute a given function (Sipser, 1996; Papadimitriou, 1993).

thermodynamic work that the receiver has to carry out in order to record the choice.

Furthermore, we have sketched a relation between the amount of bits and the required time-space product of the associated computation.

We argue that this way of thinking has many advantages: It greatly simplifies the analysis of resource costs, because we only have to deal with changes in probability distributions. This allows us abstracting away from the algorithmic details in order to reason about computational costs. The objective of this section is to explain how this formalization can be used to calculate the resource costs of running and constructing an agent.

6.3.1 Cost of Interaction

We have formalized autonomous systems as probability distributions over interaction sequences. According to the information-theoretic arguments presented in this chapter, this means that we are implicitly assigning resource costs for interactions.

This makes sense from an intuitive point of view. The implementation (or embod-iment) of an agent facilitates some interactions while it hampers others. For instance, biologists can infer a great deal of the habits of a species by studying its anatomy. The rationale behind this is that animals manifest energy-efficient behavior more frequently than energy-intensive behavior. This kind of reasoning acquires an extreme form in paleontology, where behavior is mainly inferred from fossilized animals. Conversely, in engineering, systems are designed such that they minimize the resource costs of fre-quent or desirable operations and uses. This is visible in the designs of cars, aeroplanes, buildings, algorithms, advertising campaigns, etc.

From an information-theoretic point of view, an agent interacting with an environ-ment is communicating an interaction sequence. Whenever the agent interacts with its environment (either producing an output or reading an input), its “physical state” or

“internal configuration” changes as a necessary consequence of the interaction—simply because the two instants are distinguishable. In other words, if the instants before and after the interaction cannot be told apart, then we are forced to conclude that they are empirically the same. This change in “physical state” or “internal configuration”

can take place in many possible ways: for instance, by a chemical reaction; by up-dating the internal memory; by consulting a random number generator; by moving to another location; or even by simply advancing in time (i.e. by changing the value of the time-coordinate). Hence, in this context, the semantics of “physical state” or “internal configuration” corresponds to an abstract information state: it is a description that exhaustively characterizes a situation of an agent. We call such a description a state of the agent.

To make this notion of states concrete, we introduce the following model. We assume that a change in state occurs whenever the agent either issues an action or reads an input. We start out from a blank binary tape that we will use to record the agent’s experience. Then, we iteratively append a new binary string that encodes the new input or output symbol experienced by the agent (Figure 6.10). In this model, the appended binary strings are proxies for the changes in state that the agent experiences

a1 o1 a2 o2 a3 aT oT

0 1 0 1 1 1 0 1 0 0 1 1

6 6

3 3

8 8

3 3

a) b)

Figure 6.10: State Model. The agent has an I/O domain given byA:=O:={1,2, . . . ,10}. So far, the agent has experienced four I/O symbols (Panel a). The state of the agent is constructed by iteratively encoding the four I/O symbols into binary strings (Panel b).

during its interactions with the environment. In this way, we abstract away from the inner workings of the agent by simply representing every change by a binary string.

Note that this scheme does not allow an agent returning to a previous state, hence the agent cannot “jump back in time”. Additionally, we want the content of the binary tape to be uniquely decodeable, such that we can recover the whole I/O history the agent has experienced so far at any given time by decoding the content of the tape.

This model highlights the correspondence between resources, codeword lengths and probabilities. Therefore, the behavior of the agent can be thought of as a reflection of the underlying resource costs.

6.3.2 Costs of Construction

Remark 19 This subsection is of speculative nature. ₂ Designing and constructing an agent has a cost. Whether we are thinking to find a solution to optimality equations, running a search algorithm, or assembling mechatronic parts, we are always spending resources during the conception of an agent. These resource costs can be thought of as arising from the change of the distribution over the possible agents, where the cost of this change is given by the relative entropy.

Let Θ denote the index set parameterizing the possible agents. Furthermore, let Pr(θ) denote the belief we have about θ being the optimal parameter. Consider the relative entropy

ρ=X

θ

Pr^′(θ) logPr^′(θ)

Pr(θ), (6.4)

where Pr^′(θ) is the (partial) choice made by the sender. This quantity correctly mea-sures the number of bits we are going to receive over a noiseless channel from a sender that picks out the right θ.

However, there is caveat: The previous calculation represents a situation where we passively receive the optimal answer, which is not possible unless the sender is an

“oracle” who guesses the firstρbits of the optimalθ. To correctly calculate the required number of bits in this communication problem, we have to account for the fact that we do not know the sender. Not knowing the sender has important implications, since this uncertainty might lead to significant resource costs that we are not accounting for.

Unfortunately, the author is not aware of any widely accepted way of dealing with this situation. However, one canspeculate that the amount of information is

2^O(ρ),

that is, exponential in the amount of information needed when the sender is known.

Roughly speaking, the justification for this intuition is based on results from computa-tional complexity, where a non-deterministic machine can be simulated by deterministic machine but incurring exponential cost (Sipser, 1996; Papadimitriou, 1993). In the next chapter, this claim is made precise for a special case.