Learning the Permanent Relationship

In Section 5.2.1 we discussed how transient relationships and permanent relationships are re- lated. Here, we describe a mechanism that allows an agent to calculate the type and strength of a permanent relationship through observation of transient relationships.

We begin by defining a structure that captures an agent’s beliefs about the type of permanent relationship between two others. In the earlier discussion, we defined the relationship between transient and permanent relationships, namely that the type of a transient relationship is dictated by the type of permanent relationship present. For example, if two agents are in a permanent competitive relationship, then they are more likely to find themselves interacting in a particular episode under a competitive transient relationship. Therefore, keeping a count of the times when a particular type of transient relationship occurs and when it does not allows an agent to calculate the expectation of it occurring in the next time step. This count must be maintained for all types of transient relationship, and the type that yields the highest expectation can then be deemed to be the type for the permanent relationship that is present.

For reasons outlined in Section 3.3.1, Beta distributions naturally lend themselves to a binary domain and allow effective methods to predict future outcomes based on previous binary out- comes (for example the presence or absence of a particular type of transient relationship). For this reason, the belief about the type of permanent relationship is represented as a set of beta distributions.

More specifically, we define beta distributions that correspond to an agenta3’s belief about each

particular type of permanent relationship that can exist between two agentsa1 anda2. Thus,a3

has a permanent relationship vector, Ra1,a2_{, containing four beta distributions, which represent} the four types of relationships betweena1anda2, as shown below6:

Ra1,a2 _=<(αa1,a2

com , βcoma1,a2),(αcopa1,a2, βacop1,a2),(α_depa1,a2, β_depa1,a2),(α_depa2,a1, β_depa2,a1)>

Now, the parameters of the various beta distributions in the vector correspond to counts of the corresponding types of transient relationshipsa3 believes to have existed in previous interac-

tion episodes between a1 and a2. In particular, initially the permanent relationship vector is

configured with prior values. Then, after observing a particular type of transient relationship, the observer agent (a3) simply increases the α parameter for the corresponding element, and

increases theβparameter for all the other elements.

6

For example, suppose, that agenta3has the following permanent relationship vector7: Ra1,a2 _{=<(1,1),(1,1),(1,1),(1,1)>.}

Table 5.3 shows how this vector changes subject to the transient relationships that agent a3

calculates over time from observing the interaction episodes between agentsa1anda2. The bold

numbers in the table show beta distributions where theαparameter is increased in response to observing the corresponding type of transient relationship.

Time Transient Relationship Ra1,a2

0 <(1,1),(1,1),(1,1),(1,1)> 1 Lˆa1,a2 com <(2,1),(1,2),(1,2),(1,2)> 2 Lˆa1,a2 com <(3,1),(1,3),(1,3),(1,3)> 3 Lˆa1,a2 cop <(3,2), (2,3),(1,4),(1,4)>

TABLE 5.3: Example of how the permanent relationship vector Ra1,a2 _{changes as an agent} observes transient relationships.

At any given point in time, the actual type of permanent relationship between agents a1 and

a2 (La1,a2), as believed by the observera3, is defined as follows. It is of the type which has a

corresponding beta distribution, in the vector, with the greatest expected value. The expected value of a beta distribution is calculated as shown in Equation 5.2.

E = α

α+β (5.2)

For example, if we refer back to Table 5.3, then we can see that the expected values of the beta distributions at timet= 3are<0.6, 0.4, 0.2, 0.2>. The beta distribution with the highest expected value is found in the element corresponding to the relationship typeLa1,a2

com . This means

that at timet= 3agenta3believes that the permanent relationship type between agentsa1and

a2is competitive.

Storing the knowledge about the type of permanent relationship in vector R as a set of beta distributions gives us additional benefits. It provides us with an elegant way of representing the observing agent’s belief about the strength of the permanent relationship. Here, we assume that the strength of the relationship, as perceived by the observer, is equal to the confidence the observer agent has in its belief about the type of the permanent relationship. More formally, we define the confidence metric,δa1,a2_{, as the integral of the corresponding beta distribution around} its expected value (E) by a certain configurable parameter,ε.

Having described how an agent can determine at a given point in time, the type and strength of the relationship between two agents in the multi-agent system, we can now describe how this knowledge can be used in adjusting the opinions provided by agents that are related to a trustee.

7