This section discusses compositionality as a generic conceptual framework for learn- ing and computation in higher brain areas and across species.
Compositionality (or Frege principle) is a principle rooted in mathematics, logic and linguistics. Very generally, this principle states, that the meaning of a complex expression is a function of the meanings of its constituent expressions (Hintikka, 1984). As such it is related the divide-and-conquer algorithm introduced previously which can be seen as one algorithmic implementation of this principle applicable to a family of problems. In the context of sensory processing, the constituent expres- sions can be considered to be locally available sensory cues of the environment and a complex expression can be thought of as a macroscopic problem or objective of interest, for example an odor plume or a complex visual scene. Recombination of the constituent expressions allows to reason about the complex macroscopic prob- lem. Different recombinations of the same cues may refer to different macroscopic problems which means compositionality can also be context sensitive.
Compositionality can be achieved in different forms. For example the layered hierarchical structure of neural systems can be considered as spatial or structural compositionality, as it is the case for the processing layers of the olfactory system used in Rapp and Nawrot, 2020. It is also the major form of compositionality used in Deep Learning. Another form is on the level of representations, for example in the sequential inspection strategy studied in Rapp, Nawrot, and Stern, 2020. Here the overall image representation is transformed into a sequence of representations of smaller images.
However, the most natural form is compositionality in time, which might be one principle used by nervous systems. For example, the simultaneous movement of both eyes between phases of fixation points (saccades), can be considered as tempo- ral compositionality (of focus) to perceive visual scenes (Gegenfurtner, 2016). The sequential inspection strategy used by insects and in Rapp, Nawrot, and Stern, 2020 to solve numerical cognition tasks, can also be seen as temporal compositionality as well as the sensory experience of odor filaments to reason about odor plumes as used in Rapp and Nawrot, 2020. In general, by introducing the additional dimen- sion of time any (static) input can be transformed into a sequence of smaller inputs or expressions. Thus, there are many ways how to define what makes a constitutent expression within the framework of compositionality. However, the challenge re- mains to determine what is a suitable constitutent expression, such that the amount of information related to the original problem increases when recombining them. This raises the hypothesis if biological systems successfully use and combine mul- tiple forms of compositionality, in particular temporal compositionality as time is inherent to biological organisms, for example by development and experience based learning throughout life-time.
Additionally, this principle is one possible solution to overcome the generaliza- tion issue introduced by the iid assumption in statistical learning as discussed in 4.5. The counting MNIST task introduced in Rapp, Nawrot, and Stern, 2020 shows, that by learning the concept of a single digit one, instead of the distribution of possible occurrences, allows to generalize counting to images that have not been included in the training data. In general, it allows to generalize from iid to OOD, as new images are just a composition of previously learned individual instances. The same prin- ciple can be applied to more abstract concepts. For example a face is composed of eyes, mouth, nose and ears. By learning the individual concepts of eye, mouth, nose, ear and how to recombine them, allows to generalize to faces (human and animal) without explicitly learning a distribution over all possible faces.
In summary, the principle of compositionality is general and applicable to many computational aspects of biological systems, including sensation, learning, mem- ory and cognition. Revealing the underlying computational mechanisms and neural
34 Chapter 4. Discussion
olfactory system Drosophila Honeybee Mouse
# of receptor types ∼52 ∼160 ∼1800
# of stimuli combinations (cues) 252 2160 21800
simple visual system Drosophila Honeybee Mouse
# of object detectors
2
bright & dark objects
5
bright & dark objects + 3 geometric shapes
??
# of stimuli combinations (cues) 22 25 2??
size of intrinsic universe 254 2165 >>21800
TABLE4.1: Concept of an organisms intrinsic universe equipped with two sensory systems, exemplary for Drosohila melanogaster, Honeybee and mouse. An olfactory system that can detect a set of odorants and combinations thereof (based on by the number of glomeruli found in Drosophila (Vosshall and Stocker, 2007), honeybee (Galizia and Men- zel, 2001) and mouse (Potter et al., 2001)) and a simple (hypothetical) visual system that can sense the presence or absence of a fixed num- ber of objects. The expressive power in terms of total possible stimuli combinations that can be sensed by each sensory system is given as powers of two when considering the the binary case where a sensory cue can only be present or absent. The intrinsic universe is defined as the combined expressive power of all sensory systems. In the binary case the size of the intrinsic universe is given by summation of the ex- ponents of the two sensory systems. The size of an organisms intrinsic universe follows a combinatorial explosion with each additional sense.
implementations that could allow this principle to emerge, can provide fundamen- tal insights about higher order brain computations and has the potential to explain many aspects of intelligence, in biological and artificial systems.