Reilly

The computational model of emotion proposed by Reilly in [155] is focused upon im- plementing an interface for artists that is both easy to understand and use in order to allow for the creation and definition of believable emotional agents. Consequently, Reilly’s research motivation may be classified as being a member of the believable-agent research motive proposed by Burghouts et al. in [25]. It is important to note that Reilly’s emotional model is rather pragmatic in its approach and is designed to resolve complex implementation issues. As a result, some of the design and implementation decisions selected by Reilly may be adopted for my own work so as to facilitate a simpler im- plementation of the emotion modelling framework to be developed. Reilly’s emotional model is predominantly based upon the OCC model (see 4.1.1.1) with the elicitation mechanism, emotions implemented, intensity mechanisms etc. all derived from work

presented in [142]. Since [155] is notably capacious in its scope, I will only consider relevant aspects of the computational model of emotion here.

The lifetime of an emotion in Reilly’s model spans four stages, these are outlined below:

1. Generation of an emotion structure is triggered by an event creating an emotion structure.

2. The emotion structure is stored and decays over time according to a function defined by Reilly.

3. The current set of emotion structures is used to generate additional behavioural features which abstractly determine the actions of an agent.

4. Behavioural features are used to affect specific aspects of the agent’s behaviour.

To generate an emotion a number of inputs are taken into account. I am only inter- ested in a subset of these inputs namely: sense data/sensory memory, goals, standards and attitudes (Reilly defines these latter three concepts as Ortony et al. do in [142]). Sense data and sense memory take into account the agent’s perception of its environ- ment specifically: where artefacts are located, what other agents are doing etc. Such input is especially important to model agents that are emotionally reactive since this information forms the basis upon which an appraisal can be made. These appraisals result in an emotion and make use of the agent’s internal goals, standards and attitudes (these concepts have been explained in detail in section 4.1.1.1 therefore I will not dis- cuss them again here). Reilly uses the “Hap” language (originally designed to write robust, reactive physical behaviours for agents) to construct emotion generators. Some important features of this language are discussed below:

ˆ Daemons: The ability to have rules that fire in certain circumstances is very important for creating emotion generators. Every generator Reilly constructs is expressed as a daemon that waits for a particular emotional situation to occur, and then fires, creating an emotion structure.

ˆ Flexible match language: Used to code the left-hand sides of emotion generation rules (qualitative conditions used to describe the situations in which the generator should fire).

Therefore, each emotion could be said to be under the control of one daemon that appraises perceived events and matches relevant aspects to a set of conditions for that emotion using a flexible match language. If the conditions for an emotion are met, the daemon will generate the appropriate emotional structure for use. What is important about this is that the emotion generation procedure may be simplified to use a set of “if-else-then” conditions for emotional appraisals. Consequently, it is debatable whether an agent may experience opposing emotions simultaneously towards the same situation

or not. It should also be noted that, whilst this part of the emotion elicitation procedure appears to be qualitative, the daemon will convert the appraised event into a numerical value meaning that emotion generation is ultimately quantitative.

This quantitative approach to emotion elicitation facilitates the implementation of notions such as emotion potentials and activation thresholds. In Reilly’s model, the potential of an emotion has to reach a certain threshold value before an associated behaviour is expressed in keeping with the OCC model, Frijda’s model, Scherer’s model and Steunebrink et al.’s logical formalism of emotion (see sections 4.1.1.1, 4.1.1.2, 4.1.1.4 and 4.2.1 respectively). Reilly appears to consider emotions as being functional since the elicitation of an emotion causes the expression of an un/intentional behaviour. This is not to say that Reilly does not view emotions in other ways e.g. generated emotion structures are capable of feeding back into the emotion generation system, but it is important to note that he does take a functional view regardless of this.

Reilly’s simplification of eliciting conditions for emotions needs to be emphasised: based upon his emotional model, it may be sufficient for me to model an emotion such as anger by simply declaring that “an opponent has performed action α towards me, therefore my intensity of some emotion will increase/decrease”. Such a rule is much easier to construct and utilise than a rule which fires if the agent disproves of another’s blameworthy action along with it being displeased about the undesirability of the event. In this case, the agent would have to determine: what the action was, who performed it, the degree of intentionality behind the action, the consequence of the action and whether the consequence was desirable or not. Essentially, the appraisal process is greatly simplified.

Emotional effects are modelled by Reilly as being simple mappings of emotion to behavioural families that are pre-coded and stored in the agent’s action repository. For example: anger could be mapped to aggressive behavioural features but may also be mapped to submissive behavioural features. Ultimately, it is the user of the system who decides these emotion-behaviour mappings but by allowing such flexibility, Reilly permits the creation of distinct personalities for distinct agents that use his emotional model. In addition, behaviours resulting from the elicitation of an emotion may or may not be triggered; an emotion’s associated behaviour may therefore be masked. In other words, even though an emotion’s potential may have surpassed its activation threshold, the behaviour that should result is suppressed (a landmark of Frijda’s emotional model, see section 4.1.1.2). Unfortunately, Reilly does not appear to provide any details with respect to how this masking mechanism works so it is difficult to expound on this.

Reilly’s emotional model is also notable since it makes an attempt to consider the issue of emotional decay (this issue was first encountered in the discussion of Steunebrink et al.’s logical formalism of emotion, see section 4.2.1). Like other aspects of his model, Reilly’s emotional decay function is relatively simplistic, especially when compared with Steunebrink et al.’s method and is not based upon any solid psychological research. The function used normally reduces an emotion’s intensity by 1 every tick, although for some

emotions (fear, for example), the current intensity of the emotion may be halved every tick.

4.3.2 “ACRES”

The Artificial Concern Realisation System or ACRES is a computational system that implements Frijda’s emotional model (discussed in section 4.1.1.2). ACRES is presented by Frijda’s student, Jaap Swagerman in [192] but I will restrict my consideration of the system to the more succinct description provided by both Frijda and Swagerman in [68]. The purpose of ACRES is to ascertain the validity of Frijda’s concern realisation system described in section 4.1.1.2. Frijda and Swagerman’s reasoning is such that if the preceding analysis of concern realisation is valid, it should be possible to construct a concern realisation system. Therefore, the research motivations of Frijda and Swagerman can be classified under the experimental-theoretical research motivation proposed in [25]. With Frijda and Swagerman’s reasoning in mind, emotions in ACRES act function- ally to safeguard concerns that provide the program with some independence in its interaction with its environment. In other words, if ACRES has a concern that is under threat, an emotion will be activated to remove the threat to that particular concern by identifying and implementing some intentional behaviour that is usually directed at the system itself or its operator. So, if ACRES current circumstances threaten its concerns then these circumstances should be avoided resulting in the system acting intentionally to increase the likelihood of its current circumstances being changed. Conversely, if ACRES current circumstances satisfy any of its concerns then the system should act so that the likelihood of current circumstances changing is decreased.

Concerns are represented by a theme and a tariff that represents the desired situation and what counts as an above-threshold undesirable/desirable state of affairs. This is an important point since concerns contain a notion of thresholds for realisation in much the same way as prescribed by the emotional models of Ortony et al. and Scherer (see sections 4.1.1.1 and 4.1.1.4 respectively) and the logical formalisms of Steunebrink et al. and Adam et al. (see sections 4.2.1 and 4.2.2 respectively). To illustrate how these thresholds are represented, Frijda and Swagerman consider the “preserve reasonable waiting times” concern. This concern is composed of two sub-concepts defining what a “long waiting time” and a “short waiting time” are (10 seconds and 2 seconds, for example).

To give an example of how the concern thresholds work imagine a situation where the current system’s response time is greater than or equal to the time specified in the “long waiting time” sub-concept. In this case the “preserve reasonable waiting times” concern is realised and an emotion is elicited that identifies an intentional behaviour to reduce current and subsequent waiting times so that it is less than or equal to the value contained in the “long waiting time” sub-concept and greater than or equal to the value contained in the “short waiting time” sub-concept. From here the intentional behaviour identified may then be implemented by ACRES to remove the threat to the

relevant concern so long as the behaviour’s expected consequences are admissible. To expand upon this point, Frijda and Swagerman state that the expected consequences of intentional behaviours are considered by ACRES before an intentional behaviour is performed. Therefore, it may be true that even though an emotion is elicited, its associated behaviour is not. Frijda and Swagerman use regulation processes to deal with this issue, but details of these processes are not provided.

Essentially, ACRES, for the most part is a reactive system since it only responds to user input but there are times when autonomous actions will be performed (such as asking a user to define an emotion). Generally though, there are only two ways in which an intentional action is initiated: user instruction or concern relevance (detected by the concern realisation functions). Furthermore, plans generated due to user instructions are not always executed: this occurs when another concern’s relevance is detected and its above-threshold value is higher. This functionality is worth highlighting since it demon- strates the relationship between emotional intensity and action precedence: essentially, if one concern is under more of a threat than another or, if that concern is deemed to be more important, then the emotional response associated with the threatened or more important concern will be higher and candidate actions are prioritised. Therefore, the emotion produced by a concern being threatened is associated with a precedence ordering of actions of the agent.

4.3.3 “Cathexis”

TheCathexissystem, designed by Vel´asquez, is noteworthy due to its innovative compu- tational implementation of an appraisal model of emotion (see section 4.1.1). There is no monolithic psychological model of emotion underpinning Cathexis; the system appears to be amalgam of various noteworthy features borrowed from a range of psychological models of emotion. Cathexis is used by Vel´asquez in a number of simulations whose details are outlined in three papers: [207], [205] and [206]. Of these papers I focus on [205] since it provides a concise overview of the Cathexis system and an informative de- scription of a simulation test-bed used to test the internals of the system. The purpose of this testing is to evaluate how useful Cathexis is with respect to enabling the creation of emotional agents therefore, Vel´asquez’s research motivation can be considered to be experimental-theoretical.

Cathexis is a distributed, object-orientated system; emotions are modelled as net- works composed of special proto-specialist agents. This design paradigm appears to be a literal translation of Minsky’s concept of human intelligence outlined in [128]. In this work, Minsky proposes that intelligence is composed of interactions between “mind- less” agents giving the work its title: “The Society of Mind”. Each proto-specialist in Cathexis represents a basic emotion family; “basic” is to be interpreted as Ekman de- fines the term in [44] i.e. there are a number of separate emotions that differ from each other in important ways such as the events that elicit them, resulting bodily expressions, behavioural/physiological responses etc.

Every proto-specialist in Cathexis is endowed with multiple sensors that monitor both external and internal stimuli. External and internal stimuli that are relevant to a particular proto-specialist will either increase or decrease the intensity of the emo- tion family that the proto-specialist represents. Proto-specialists also run in parallel so multiple emotions can be experienced simultaneously by one agent. The concept of proto-specialists and their functionality is significant for two reasons: firstly, the idea draws parallels with notions that have been previously encountered such as Frijda and Swagerman’s concerns (see section 4.3.2) and Reilly’s demons (see section 4.3.1). Also, each emotion proto-specialist would appear to possess an associated intensity value which can change over the course of time (much like the emotional fluents defined by Steunebrink et al. in section 4.2.1). A further salient aspect of Cathexis is that each proto-specialist has three associated threshold values:

ˆ α: controls the activation of the emotion. Once a proto-specialist’s associated intensity surpasses this threshold, an output signal is released to other proto- specialists and Cathexis’ behaviour system. This system then selects an appropri- ate behaviour according to the current state of the emotion systems.

ˆ ω: specifies the maximum intensity for that emotion proto-specialist. This is consistent with real life emotional systems in which levels of arousal will not exceed certain limits.

ˆ Ψ: the intensity decay function of the emotion.

These thresholds are again similar to concepts previously encountered i.e. the thresh- olds mentioned by Ortony et al., Scherer, Steunebrink et al. and Reilly (see sections 4.1.1.1, 4.1.1.4, 4.2.1 and 4.3.1 respectively) and the tariffs outlined by Frijda and Swa- german (see section 4.1.1.2). In addition, the ability of proto-specialists to be activated in parallel allows for various emotions/actions to be experienced/performed simulta- neously and also allows for blends where two basic emotions may be elicited at the same time and the resulting emotional state of the agent is itself distinct (people who experience grief may be a blend of the emotions of anger and sadness, for example).

Each proto-specialist makes use of an intensity decay function that appears to copy Reilly’s decay function (see section 4.3.1): the intensity of an emotion is decremented by 1 every time step until the emotion becomes inactive. This situation, where the decay function proposed is a “best-guess”, is familiar since the decay functions proposed by Steunebrink et al. and Reilly (see sections 4.2.1 and 4.3.1 respectively) are also “best-guess” attempts3.

As mentioned earlier, after a proto-specialist recognises that an emotion’s intensity has surpassed its threshold of activation, an output signal is sent to the behaviour system. This system determines what behaviour is appropriate for the agent to display,

3

“Best-guess” in the sense that none of the decay functions outlined are backed-up by psychological evidence obtained through experimentation

given its current emotional state. Again, the primary output of the emotional model detailed here is some intentional behaviour that is consistent with the emotional state of the agent. This would indicate that Vel´asquez aligns himself with a functional view of emotions i.e. an emotional state serves to alter some aspect of the current environment that is internal/external to the agent that the system is implemented in. Furthermore, behaviours are guaranteed to occur if the associated emotion is elicited, there is no concept of action potentials (see section 4.1.1.2) included in Cathexis.

“Sim´on”, the simulation test-bed created and used by Vel´asquez in [205] consists of a synthetic agent that represents a young child which a user can interact with. Inter- actions provide stimuli to Sim´on which, along with the internal stimuli generated by his motivational systems, will cause him to react emotionally. The user can also ma- nipulate the different parameters of Sim´on’s proto-specialists; distinct configurations of these parameters enable the representation of different emotional reactions, moods and temperaments. Vel´asquez refers to these parameter configurations as the affective style of the agent. To put this another way, a user may create a number of distinct Sim´on agents, each with their own particular emotionalcharacter. Different versions of Sim´on with distinct emotional characters will make use of and produce different emotional in- puts and outputs even though the basic components of the agent are the same. This is an important feature and will be taken forward in my own work.

4.3.4 “The Affective Reasoner”

Elliott’sAffective Reasoner simulator is presented in a number of papers, specifically [46], [47] and [48]. Of these I will focus upon [47] as it is the most concise in its description of the system. In [47], the “Affective Reasoner” is populated with a number of agents each endowed with a set of appraisal frames that are akin to a unique emotional character much like Vel´asquez’s concept of affective styles in “Sim´on” (see section 4.3.3). These appraisal frames represent an agent’s individual goals, principles, preferences and moods i.e. it acts as a knowledge base for the agent. Different combinations of these appraisal frames can be used by the agents to interpret situations that unfold in the simulation consequently, these interpretations may or may not cause the eliciting conditions of particular emotions to be met. However, if an emotion is elicited, it is expressed using behavioural channels which can be perceived by other agents in the simulation and as new simulation events. Elliott therefore appears to view emotions functionally with some form of behaviour being guaranteed to occur when one is elicited. The Affective Reasoner therefore appears to have much in common with the Cathexis system (see section 4.3.3).

A key issue to address here is how Elliott implements emotion elicitation in the