What behaviour is about

Behaviour involves perceiving relevant aspects of the physical world and acting upon them and in relation to them. It might also involve awareness of aspects of the environment that are not immediately present to the senses, such as remembering situations lived in the past, planning, or imagining. To make sense of how the flow of behaviour is produced (if not by manipulating mental representations), I will discuss the notion of ‘environmental information’, its relation to abilities to perceive and act, and the role of intentions. I hope to show that the notion of environmental information is quite different from ‘information’ (meaning representation) as used in the cognitivist approach.

4.3.1

The information available in the environment

The environment of an animal usually includes non-living components such as the medium (water for aquatic animals, air for terrestrial animals), the ‘natural’ objects and materials such as mountains, hills, rocks, and water bodies, as well as the objects and materials produced by the niche-constructing activities of organisms such as body fluids and waste materials, oxygen and carbon dioxide, nests, bee hives, termite mounds, shelters, and the myriad of human-made artefacts. These objects and materials furnish the different places in the environment which might provide animals with different behavioural opportunities or affordances.

The flow of behaviour 73 Consider how the ‘information’ available for visual perception is created (J. J. Gibson, 1979/2015). In an illuminated environment, light coming from the sun or other sources of illumination is thoroughly scattered by bouncing back and forth between reflective surfaces and the medium, and it achieves an equilibrium or steady state. Usually, the light arriving at a fixed point of observation is different from different directions in a way that is specific to the surfaces surrounding that point. In other words, there is a structured rather than a homogeneous flux of photons arriving at any point of observation.

In his ground-breaking work, James Gibson conceived the structure of the optic array as a nested set of solid angles, with their apices at the point of observation and their bases at distinct surfaces. Neighbouring surfaces, or environmental texture elements, can be distinguished by the intensity and/or spectral composition of the light they reflect. The structure or pattern at the proximal point of observation specifies – is specific to, and thus can inform about – the distal surfaces which constrained its structure. It is in this sense that the ambient light array reaching a fixed point of observation and, especially, the systematic changes in the array produced by moving along a path of observation (the visual flowfield), can be said to create ‘environmental information’. If there was no structure in the light arriving at the point of observation, as in a room filled with white fog, there would still be stimulation of the sensory neurons but there would be no stimulus information available to be detected.

According to J. Gibson, “Information… refers to specification of the observer’s

environment” (J. J. Gibson, 1979/2015, p. 131, emphasis in the original). It emphatically should not be taken to mean a piece of knowledge conveyed in communication, a representation, or a computer-like data structure. He was very much aware that the computer metaphor was becoming increasingly popular, with the expression information processing being used in place of the traditional notion of input processing by mental operations. J. J. Gibson (1979/2015, p. 240) argued that the move was hardly innovative:

But it seems to me that all they [researchers using the computer metaphor in the study of perception] are doing is climbing on the latest bandwagon, the computer bandwagon, without reappraising the traditional assumption that perceiving is the [mental] processing of [meaningless, sensory] inputs. I refuse to let them

74 Chapter 4 pre-empt the term information. As I use the term, it is not

something that has to be processed. The inputs of the receptors have to be processed, of course, because they in themselves do not specify anything more than the anatomical units that are triggered.

Environmental information should also be distinguished from how the term ‘information’ is commonly used in the context of communication technology. “The information for perception is not transmitted, does not consist of signals, and does not entail a sender and a receiver” (J. J. Gibson, 1979/2015, pp. 56-57). In another passage, Gibson explicitly distances his use of the term from the mathematical concept. “The information for perception, unhappily, cannot be defined and measured as Claude Shannon’s information can be” (J. J. Gibson, 1979/2015, p. 232).

Gibson’s use of the term ‘information’ has influenced later work by situation theorists, which I mentioned in Chapter 2. Barwise and Perry (1981, p. 668), for example, explicitly say that their view “was profoundly influenced by [ecological psychologist Michael] Turvey and others working in the tradition of ecological realism”. According to situation theory, information is created by ‘situations’ and informational relations between situation tokens should be understood in terms of ‘constraints’ (lawful or normative regularities) between situation types. To use Dretske’s (1981) expression, the informational content of a situation is the ‘what-it-is-you-can-learn’ about one part of the world (situation, signal, state of affairs) by consulting or detecting some other part of the world. Situation theorists were initially interested in linguistic phenomena. Translated to perception, we can say that the structure of the environment and the structure of the ambient array of light (as well as sound, chemicals, surfaces, and so on) are linked by regularities or constraints. Thus, the animal can learn about, or become aware of, its environment by detecting the ambient arrays that specify it.

4.3.2

Environmental information guides behaviour

When an animal occupies a point of observation, its sensory organs can be stimulated by the structured ambient arrays of light, sound, chemicals, surfaces, and so on. Moreover, its body takes part in structuring the ambient light together with the other objects and, therefore, the information available at that point is as much about the situations it involves as it is about the animal itself. That is, “the perceptual systems are propriosensitive as

The flow of behaviour 75 well as exterosensitive” (J. J. Gibson, 1979/2015, p. 108). The vague boundaries of the field of view, for example, specify the body occupying that location – more specifically, the nose, the edges of the eye socket, the eyebrows, and the cheekbones. Other parts of the body such as arms, hands, abdomen, legs, and so on, are also directly specified in the ambient light array. Information about the animal is available to all perceptual systems: “An individual not only sees himself, he hears his footsteps and his voice, he touches the floor and his tools, and when he touches his own skin he feels both his hand and his skin at the same time. He feels his head turning, his muscles flexing, and his joints bending.” (J. J. Gibson, 1979/2015, p. 108).

Suppose you look at a rectangular table by keeping one eye shut and positioning the open eye on a line perpendicular to its surface. The form of the envelope of the solid angle arriving at the retina, which specifies the outline of the face of the object as seen from this point of observation, will be rectangular (or nearly so, for the retina is not a plane). If you now move a bit to the side, the form arriving at the retina is no longer rectangular but trapezoidal. This is the case because the angles at the table corners and the proportions of the sides change from one point of observation to another. At the same time, there are relations among the angles and relations among the proportions, described by projective geometry, that remain unchanged under transformations. These invariants, as Gibson argued, are specified in the optic flowfield and available to a moving observer. The suggestion is that, by looking at the table along a path of observation, your perceptual system can detect the underlying invariants and you therefore perceive the form of the table as a persistent feature of the environment.

This perception is direct in the sense that the patterns are present in the visual flowfield and therefore it is not necessary to posit mental operation such as representing the table’s angles and side lengths and then performing computations to derive the invariant quantities. Rather than computing the invariants, perceiving is about detecting them. More recently it has been suggested that pattern detection can be achieved, without recourse to algorithmic operations, by spontaneous (self-organising) processes of dynamic pattern formation in nonequilibrium systems (Haken, 1996; Kelso, 1995). In the case of visual perception, such processes may result from the perturbations produced in the ongoing neurophysiological dynamics by the causal interactions between photons and

76 Chapter 4 photoreceptors in the retina. Saying this does not solve the problem of how visual perception occurs but it offers an alternative way to conceive the problem in non- representational, non-computational terms. Much more empirical and modelling work is required to understand the details of how direct perception is achieved. It should be noted that saying that the brain really computes the invariant quantities (instead of sensing it directly) does not solve the problem either, but rather defines the alternative computational framework which might be used to investigate it.

When an observer moves along a path of observation, the optic structure arriving at the retina (i.e. the environmental information thus made available to be picked up by the visual system) changes in some respects – what Gibson called the ‘perspective structure’ – and remains unchanged in other respects – the ‘invariant structure’. Each unique point of observation in the environment is specified by a corresponding perspective structure. Each line segment connecting adjacent points of observation, i.e. each path of possible displacements in the environment, is specified by a corresponding set of transformations in the perspective structure. Therefore, the information made available in the time- evolving perspective structure, which in part depends on the activity of the animal, specifies the current position and the path of locomotion of the observer in the environment. That is, a perspective structure flowing in time means locomotion along a specific path, and an arrested perspective structure means rest. The optic structure that remains invariant under transformation requires activity to be made available and sensed. This invariant structure is common to a set of points of observation and specifies, not the observer, but the persisting features of the environment.

Events occurring in the environment are specified by local disturbances in the ambient structured arrays. For example, the displacement of an object against a background is specified by the progressive deletion of optic structure at one border and accretion of structure at the opposing border. The approach of an object is specified by

looming, i.e.the magnification of the optic form that specifies the object with progressive deletion of optic structure outside its contour. Elastic events such as the overt behaviour of other animals are specified by the deformation of the optic form that specifies their body parts. Vocalisations structure the mechanical waves in the medium and are therefore specified in the sound array. The elimination of substances, such as releasing pheromones

The flow of behaviour 77 in the atmosphere or leaving chemical trails on the ground, structure the available chemical arrays. And so on. It is invariants such as these that animals have available to detect the presence and activity of other animals, and which they might use to guide their own behaviour. J. J. Gibson (1979/2015) argued and reviewed empirical evidence that people are indeed able to detect invariants, and research programs inspired by Gibson have provided further support (Lee, 1976, 1998; Lee & Reddish, 1981; Turvey & Carello, 2011).

Note that the information available in the perspective structure is propriospecific, i.e. is about the relative position of the animal in the environment. In contrast, the information available in the invariant structure is exterospecific, i.e. is about the components in the environment, including other animals. They are concurrent and each imply the other. By picking up both kinds of information, animals might perceive the environment and themselves in it at the same time.

Animals in different locations, and animals following a similar path at different times, might detect the same patterns. The fact that environmental information is thus publicly available to animals with similarly tuned perceptual systems is crucial to understand processes of social learning without recourse to the metaphor of ‘transmission of information’ (representations) between animals. Animals that are closely related genealogically, and whose ontogenies unfolded in similar developmental niches, might develop functionally similar perceptual systems. Consequently, they might detect the same information available in their common environment.

Also crucial to social learning is that animals playing the role of facilitators can, by means of their presence and activity, direct the learners’ attention to information specifying task-relevant aspects of the environment as well as create information that would otherwise not be available to them. When facilitators perform the target task, vocalize in response to a perceived threat, or provide verbal instructions, for instance, they structure the ambient arrays of light, sound, etc, thus creating information that might be perceived by others around them. In this view, social learning is a historical and relational process of developing abilities, in which ontogenies unfold in time and are intertwined, rather than a computational process of acquiring representations. I will return to this below.

78 Chapter 4 In the ecological approach, perception is conceived of as a continuous activity of sampling the ambient arrays of light, sound, chemicals, materials, and so on, that specify features of the environment and the perceiver at the same time. Some of the information available in the environmental situation is especially relevant for behaviour because it specifies the affordances of surfaces, objects, places, and events. For example, if we perceive that an object has opposing faces separated by a distance smaller than our hands, we perceive that the object affords grasping. The affordance graspability is specified in the flowing optic array and can be perceived directly, i.e. without invoking algorithmic transformations of representations. To argue that affordances can be perceived directly does not imply that animals ‘just do it’. The abilities required to detect environmental information and perceive affordances, like all abilities, must of course develop, and can also be changed consequent on experience, i.e. through learning (E. J. Gibson & Pick, 2000).

Animals with appropriately tuned perceptual systems can sample the patterns available in their current environmental situation and use them to adjust their flow of behaviour accordingly. This includes guiding their behaviour in relation to aspects of the environment that are currently present to their senses. For example, the optic flowfield being such and such might inform the animal about the presence of a fruiting tree ahead. The skilled animal can use the patterns in the optic flowfield to adjust the direction of its locomotion towards it. Similarly, when linguistically skilled people engage in conversation, the mechanical (sound) waves being such and such inform them about what the other wants to convey and they can adjust their own utterances in response.

Animals can also use the patterns available in their current situation to become aware of the past, present, and future situations they imply, because of the regularities linking them. Skilled animals can therefore guide their behaviour in relation to aspects of the environment that are not currently present. Imagine a group of hunters searching for game as they move along a path in their environment. On the one hand, the (proximal) patterns in the optic flow as they move along specify, and can therefore inform them

about, the (distal) patterns in the mud on the ground. Depending on their previous experience, detecting these patterns might inform them about the nearby presence of a deer, even if they cannot see it at that moment. This way people (and other animals) can

The flow of behaviour 79 plan and guide their behaviour in relation to an expected situation (the presence of a deer), in ways that might be influenced by past, remembered situations (previous hunting expeditions, stories told and heard about similar hunting situations, and so on) and also by imagined situations.

4.3.3

Behaviour as dynamic pattern formation

The hypothesis put forward by Haken (1996) and Kelso (1995) is that the detection of spatiotemporal patterns in the flowing energy arrays, i.e. perceiving the environment, is achieved by the formation of spatiotemporal patterns of neuronal activity. This involves moving the body parts that compose the perceptual systems around to sample the ambient arrays (J. J. Gibson, 1966, 1979/2015). These global patterns of neuronal activity are enabled by the constructive interactions that occur in the sensory surfaces, and how they become integrated in the on-going activity of the rest of the brain-body-environment system. In the case of visual perception, pattern detection/formation might result from the perturbations, i.e. the structured flux of photons arriving at the retina, to the ongoing (neuro)physiological dynamics of distributed neuronal assemblies.

Sensing a pattern available in the structured ambient array corresponds to settling on an attractor. Making this theoretical suggestion does not explain all aspects of what we might want to know about perception. Rather, it offers an alternative starting point for theoretical and empirical studies. The study of spontaneous pattern formation in nonequilibrium systems, as formalised using the mathematical tools of dynamic systems theory, offers a metaphor or framework that is an alternative to the representational- computational framework, one that is consistent with Gibson’s notion of perceptual systems ‘tuning in’ or ‘sensing’ or ‘resonating to’ the patterned energy and material arrays available in the environment by which animals might pick up the invariants that specify relevant environmental features.

Similarly, the production of spatiotemporal patterns of body movements, i.e. controlling overt behaviour, is also achieved by the formation of spatiotemporal patterns of neuronal, and neuromuscular, activity (Kelso, 1995). This involves perceiving the relevant affordances. These global patterns of activity are enabled by the constructive

80 Chapter 4 interactions that occur in the motor surfaces, and how their (feedback) effects in the sensory surfaces become integrated in the on-going activity of the rest of the system.

The cognitivist view commonly describes behaviour in term of a linear sequence such as sense-compute-act. From the radical embodiment view, this suggestion makes no sense. It is certainly the case that some specific perceptual exploration may occur before

some specific course of action. For example, a commuter might see the train arriving at the platform before boarding it. However, visually perceiving the train depends on her moving her eyes, head, etc, to sample the visual flowfield and detect the invariants that specify the arriving train. Thus, perception involves action. On the other hand,