Research on exaptation
4 Novelty regulation Complex feedback system Parallel processing & workspaces
5 Serendipity Complex embodiment Relative perception &
spillover processes
The interesting work starts with iteration two which is focused on improving the knowledge base of the agent. With the room simulation, moving was a process that requires time to execute and conditions to be verify during the execution. The key simulation adds more actions, controlled by the memory of the agent. The actions were simpler and got identified as a problem for research on exaptation. With iteration two, a design was implemented that made all actions become processes like moving. Even the reasoning of the agent is a process. In the simulation, the reasoning happens several times faster than the physical processes. This allows the reasoning to regulate the physical processes. Each process is represented in the knowledge base of the agent. The process description has five variables: do, get, at, on and with. Two variables are conceptual tags relevant for the regulation. The variable 'do' is the name of a particular action and 'get' is the anticipated outcome of the process. The other variables 'at, 'on' and 'with' are the conditions required for executing the process. These variables can have the agent as the default value. The agent would start with a usable knowledge base for the initial environment. The starting knowledge of the agent is as follows:
• do: view get: identifyObject • do: move get: newPosition • do: pickup get: Object at: Object
• do: eat get: stopHunger on: nutmeat
From the agent's need, to stop the hunger, an internal reasoning is resolving the pre-‐ conditions to execute the process 'eat'. This triggers a cascade of required processes to reach its goal. The variable 'on' and 'with' could only get executed if the agents had those objects in its possession (and the agent always has itself in possession). If this condition is not met, the agent adds two processes: 'view' to identify the required object in the environment and 'pickup' to possess the object. Note how the 'pickup' process has an upper capital for both 'get' and 'at'. This was used to indicate a variable (inspired by logical programming). This means that 'Object' could actually be anything. The 'pickup' process shifts the problem of having the object to being close to the object. This brings us to the variable 'at', indicating a condition that a process can only get executed if the agent is close to a particular object (and the agent is always close to itself). If the condition is not met, the process 'move' is required.
The next step is to apply these rules. To stop the hunger the 'on: nutmeat' needs to become concrete, adding 'view' and 'pickup'. The process 'view' has only default conditions and can get executed directly. By the view process the agent identifies object-‐342 as nutmeat that it wants. Now the agent needs to pick up the object, what requires moving to the object. So once the agent starts moving it internally has the following process stack:
1. do: move get: object-‐342 (being executed, condition not yet reached) 2. do: pickup get: object-‐342 at: object-‐342 (pending, waiting for move)
o do: view get: identifyObject (object-‐342, executed) 3. do: eat get: stopHunger on: object-‐342 (pending, waiting for pickup)
It is necessary to provide some insight on the software architecture behind this description. This requires some understanding of object-‐oriented programming. In object-‐oriented programming, everything is an object, what makes 'process' a virtual object of the simulation containing the five variables and some methods. When the 'process' object gets the method-‐call 'execute', it transforms the variable 'get', being a name, to a method-‐call, which is then sent to the object linked to the variable 'on' and 'with'. For example, in the case of 'view' the variable 'on' is the agent itself. The 'agent' object had a method 'view' that sends a method-‐call 'beingViewed' to all objects in the environment. All objects have the method 'beingViewed' that returns a set of variables. Together with the distance, the agent uses these variables to create a selection. In this case, it selected object-‐342.
The software architecture creates the framework for different simulations. The framework contains abstract objects, agents and processes. For example, in the simulation above, only 'eat' is a process unique to the current 'nuts' simulation. It was implemented by extending the general 'agent' object to an object called 'ape'. This technique was not needed for all objects. For example, the 'process' object contains general dynamics that are well developed and did not require extensions. Because the 'process' object transforms a name into a method, those methods had to exist in the object getting the method-‐call. In the 'agent' object, some general methods like 'move', 'view' and 'pickup' were already implemented. While in the 'ape' object, the method 'eat' was implemented. The framework hides common structures and functions. It creates, by bundling concrete extension, clarity for the researcher.
I'm not again going into great detail to describe how other structures and functions are coded. The next section focuses on how the simulation allows new knowledge to emerge from the actions of the agent. To elaborate the growth of the knowledge, the order of the iteration is mixed up. The focus of our research that is to get a simulation on exaptation, enforced a test that triggered a cascade of adjustments by the next iterations. Only with iteration five did we consider the experiment on exaptation to be acceptable. It is therefore necessary to elaborate on the outcome.
Serendipity in flow leading to exaptation 2.1.1.2
To get to an acceptable flow, ambiguity became essential. Only one object existed that represented hybrid states between the nutshell and nutmeat. The object in the simulation had a variable representing the percentage of the shell around the nutmeat, as the example in Figure 2.1 illustrates. At the start, the agent only searches for nutmeat, which are the objects having 0% shell. The agent eats the nutmeat, which changes the landscape.
Figure 2.1 Examples of the interpretation of the 'shell' value in the simulation. Left, the object has 0%
Iteration five improves the simulation by considering the complex embodiment of the objects in the simulations, creating ambiguity in perception and action. Depending on the percentage of the shell the object could get perceived as nutmeat. Of course, the more open the shell the more likely that the agent is going for that object. Objects below 20% shell would just get picked up like the open nutmeat. With a shell between 20% and 40% the shell would linger and fall off early during the pickup. Between 40% and 60% the agent begins to nip the nutmeat out of the shell and by 75% shell the agent needs to peel off the shell. By simply going for the most open nuts, the agents have to adapt to small perturbations to the process, what can be learned by simple sensory-‐motoric corrections. The description shows how a simple adaptation process arises for each action, while over time the action has become very different. This part of the simulation shows the relevance of circumstantial conditions in the environment, making serendipitous effects emerge. For both perception and acting the ambiguity is enabling the change. For the simulation the overlap between the perceptions and the actions were illustrated as gray zones between the concepts (Figure 2.2).
Figure 2.2 The overlap between perceptual concepts on the one hand and processes on the other
hand in the simulation creating ambiguity.
A richer description of the process allows the agent to observe the spillover effects of its actions. In the simulation a simpler solution was used, although it was still developed enough to be the interpretation of the dynamics of the richest description. Consider the situation of the code. In the simulation, the execution of 'pickup', by a method-‐call to object-‐342, creates a chance, relative to the percentage of shell, that object-‐342 executes a process that reduces the value of the shell. In the simulation, this happened by object-‐342 creating a 'process' object and calling the method 'execute' on it:
The variable 'get' is between breakages as it is what the agent uses to label the outcome of the process. It is not really part of the process itself. The spillover process requires observation. Only in the case where the agent was focusing on the object could it perceive the spillover process in action. It is expected that, in reality, many side effects can happen around action and most are ignored. The agent is able to identify serendipity. How this happened is part of the architecture of novelty regulation. It requires a decoupling of internal processes from external processes, as demonstrated in Section 2.2.4. It is now best to just accept that the agent can identify serendipity and stay focused on the simulation at hand. The serendipitous observation in the simulation is that picking up also creates breaking and breaking the nut gives you nutmeat. For the simulation, the spillover was presented as a wave that initially was hidden from the intentional process, but observable. The wave stays hidden if the agent is not focused on the object (Figure 2.3).
Figure 2.3 Creation of spillover processes by the ambiguity of perception and action
In case the agent did make the serendipitous observation, it had now a second way to get nutmeat, by breaking nuts. This creates a significant change allowing close nuts to become considered. More closed nuts would create a distinct sound of breaking. This is less the case with the chestnuts in Figure 2.1, so consider hazelnuts and walnuts. This sound is relatively related to rock fall, which is the process of one rock falling and smashing into another. The first such rock fall would get ignored. It now links to existing knowledge and becomes a serendipitous observation. Allowing the agent to add:
• do: break (get: chip) on: stone with: stone
It is important to note that this is the only process where 'with' is used. It is a simple description, to indicate that two objects are involved to create the process. At first, the observation has no effect on the activity of the agent. While the agent tries breaking the hardest nuts a breakdown occurs, as the agent does not have the force to break the object by hand. The breakdown creates an inconsistency with 'get'. In the simulation, other such inconsistencies were possible, like the agent picking up a nut, initially perceived as nutmeat, but where the chances were bad and the breaking did not occur. In case no more knowledge exists, the situation is confusing and it restarts the whole reasoning. In the case of breaking, more experience was built resulting in an exploration of alternatives. Focusing on the most recent changes, like the sound and force of breaking, the exploration creates alternatives by a recombination of knowledge, as illustrated in Figure 2.4.
The creative process is always an exaptation and this recombined action is an experimental process. The exaptation of breaking replaces the default (the agent) by a stone. It does not always need to be a perturbation triggering the need for exaptation. It could also come from an insight, under pressure, being bored, etc. In general, it is considered to be a challenge. The challenge would lead to an experimental process that has to be tried before it is considered valuable. After executing the experimental process, the agent gets positive feedback that a stone can indeed break the hard nuts and the agent adds the process description:
• do: break get: nutmeat on: nut with: stone
The simulation created a very basic case of exaptation. Problems with the experimental process can result in a cascade of changes, making the agent create a whole new body of knowledge. In fact, it is what happened to our research itself. Iteration three of the simulation was by pressure to get some effect of exaptation in the simulation experiment results in the adjustments for two more iterations before it became an acceptable solution. It is in a way funny, since the research on regulating novelty was created by … regulating novelty. This effect was recognized after the simulation research and got used to bootstrap the research on novelty, like using the insight of the theory to develop an improved methodology.
Basics of novelty regulation 2.1.1.3
Iteration four makes the simple reasoning of the agent become a complex feedback system. Table 3.1 indicates how parallel processes and workspaces were the solution. A short introduction is given in this chapter, while Chapter 4 takes a in-‐ depth look at the complex feedback system. The parallel processes get implemented twice. The easiest implementation is the reasoning being a process many times faster than the physical process, what allows the adaptation to small perturbations. The tricky implementation is how parallel processing makes simple rules become higher-‐level cognition. To elaborate on this, I need to present some more detail on
cognition. The dynamics are described in Section 2.2.4. The parallel processes require a workspace. In the former, the workspace is the environment, in the latter, it is a special memory. It is the workspace that allows the serendipitous observations. The parallel processes take it up in the reasoning and when it creates any relation to existing knowledge it becomes noticed. The relevance and details of workspaces require insights on collective intelligence. Sections 2.3.2 and 2.3.3 gives more detailed insight on workspaces.
The last iteration was not the end of the simulation research, but no concrete next simulation was built. One exploration, to improve the simulation, was to remove the abstraction that mimics the sensory-‐motoric part. Actual sound and actual movement would make the sensory-‐motoric part concrete. This would require a shift to robotics instead of software agents. The investigation concluded that the research in robotics is still a long way removed from the requirements needed for the complex adaptive dynamics of the simulation. Development could be started to narrow the gap, but it would change our actions from research on exaptation to building complex adaptive robotics. Because the exaptation is our focus, an alternative path was taken to continue the research.
The simulation did create the basic understanding on how knowledge can be created by action. In the simulation, the actions are directed externally to the environment, making the simulation about mastering. In addition, internally directed actions can be created to improve the knowledge base. This is described in this PhD dissertation as modeling. Modeling allows for knowledge to become defined abstractly, like the 'pickup' process. Indeed, if we look at babies, we can see them create, by trial and error, an understanding of what object can be picked up and how.
A second path of investigations by agent simulations explored how to replace the abstraction of knowledge. In the simulation the knowledge is essential, but how it got built stayed abstract. Investigating how to build knowledge from experience implies more refined simulations on modeling, also using the social aspect of creating knowledge. Interestingly, it creates the same problems as the simulations for advanced mastering, although the field of application was radically different. To create good cases of modeling, the simulation needs to develop as a multi-‐agent simulation, since modeling is entwined with linguistic activities and social activities. A huge increase of complex adaptive dynamics is again needed for simulations. It is now related to multi-‐agent modeling and complex knowledge.
By the exploration of advanced simulations, the insight was discovered that both mastering and modeling occurred by the same process. The difference is an internal or external orientation. The simulation showed the external orientation. Imagine a complex knowledge structure and an agent reasoning about the knowledge to create abstractions. The serendipitous patterns in the knowledge triggers the development. The agent uses those patterns to create abstract concepts. By default, concepts are links between sensory-‐motoric experiences. Therefore, concepts linked to concepts are actually creating an exaptation. It is expected that the first abstractions may not
be well designed. Mastering is used to get additional insights on how this can be improved.
Consider how mastering happens with the repetition of the process 'eat'. The repetition creates an adaptation path, allowing small adaptive steps, making 'pickup' transform to 'breaking'. For modeling, reflection creates a similar repetitive adaptation path that could make the eventual abstractions become very different from the initial abstraction. For example, many good ideas have first been metaphors that then create whole bodies of knowledge.
The similarity between modeling and mastering led to my focus on one feedback mechanism that can be applied both internally and externally. The adaptation path is through the many small iterations a continuous process, with the exceptional discontinuous events of serendipitous observations and creative exaptations. The whole system becomes fully capable of creating knowledge by a bootstrapping process between mastering and modeling. Figure 2.5 gives a visual description of the whole process.
Figure 2.5 The bootstrapping process combing serendipity, exaptation and adaptations.
Consider that most of the iterations created for the investigation of exaptation are also about the pre-‐conditions of the complex adaptive systems (environment, agents, objects and processes). Simulations are intended to simplify reality in order to examine a particular aspect. For our simulation, it was necessary to gain an insight inot the regulation behind exaptation. This was answered on iteration four and improved with iteration five. What is now needed to continue the research is not a simplified reality, but a niche to grow and continue what was found with the simulations. The niche is to support real people in the development of innovation. It does require some investigation before it becomes clear how the research can continue.
Technology exaptation for cognition
2.1.2
The research shifted to real people and their technology. It is, however, still the same research on novelty regulation. A point of entry is found by considering the breakdown of normal cognitive abilities. In support of disabilities, brain plasticity turns out a way of recreating cognitive abilities by exaptation. First, some cases of
creative exaptation around brain plasticity are given. In subsequent cases, I also consider the relation to serendipitous observations. The final cases show how the mechanism is used for Intelligence Amplification.
MirrorBox artefact