Emergence and Evolution of Agent-Based Referral Networks

(1)

Yu, Bin. Emergence and Evolution of Agent-Based Referral Networks. (Under the direction of Dr. Munindar P. Singh.)

Numerous studies have shown that interpersonal communication acts as an important channel for gathering information. But if we wish to rely on interpersonal communication, we still need to figure out how to determine the right person to ask. Usually we cannot find the potential expert(s) directly, and we need some assistance from our friends or friends’ friends to locate them. The phenomenon of Six Degrees of Separation indicates that it is possible to use some intelligent software agents, who can interpret the links between people and follow only the relevant one, to find the desired experts quickly.

A computational model of agent-based referral networks was proposed to assist and simplify the users to find potential experts for a specified topic in a person-to-person social network, in which each user is assigned a software agent, and software agents help automate the process by a series of “referral chains”. Unlike most previous approaches, our architecture is fully distributed and includes agents who preserve the privacy and autonomy of their users. These agents learn models of each other in terms of expertise (ability to produce correct domain answers), and sociability (ability to produce accurate referrals). We study this framework experimentally to see the effects that the different variables have on each other and the effi-ciency of the referral networks.

(2)

AGENT-BASED REFERRAL NETWORKS

by

Bin Yu

A dissertation submitted to the Graduate Faculty of North Carolina State University

in partial fulfillment of the requirements for the Degree of

Doctor of Philosophy

COMPUTER SCIENCE

Raleigh

2001

APPROVED BY:

(3)

To my wife and parents

(4)

BIOGRAPHY

(5)

ACKNOWLEDGEMENTS

I would like to express my most sincere gratitude to Dr. Munindar P. Singh, my advisor, for his guidance, inspiration and support throughout the last three years. He has helped shape my research from day one and pushed me to achieve to the best of my ability. Without Munindar, this dissertation would not have happened.

I would like to extend my appreciation to Dr. James Lester, Dr. Carla Savage and Dr. Peter Wurman for their valuable discussions and comments regarding my research. I am also grateful to have Dr. Henry Kautz on my thesis committee board. I thank him for his inquisitive comments and numerous suggestions in evaluation of referral networks, which led to many improvements in this thesis.

Sincere thanks are extended to the faculty and staff of the Department of Computer Science, especially Dr. Gary Stelling and Ms. Margery Page for their patience and kind help in numerous paperwork and system support. I am especially thankful to Mahadevan Venka-traman, Wentao Mo, Paul Palathingal, and Amit Chopra for their help and collaboration on the project and to Zhengang Cheng, Pınar Yolum, Mike Suralik, and Jie Xing for their heated, friendly and enlightening discussions.

Finally I want to express my utmost appreciation to my parents and my wife for their love, support and encouragement throughout my graduate studies.

(6)

List of Figures

2.1 An example referral network . . . 10

2.2 An agent’s behavior while querying and responding . . . 11

2.3 An agent’s internal data structure . . . 14

2.4 Data structure of MARS agents with a close friend list . . . 18

2.5 Redundant referrals in a referral graph . . . 21

2.6 The cut-point node is a leaf agent . . . 22

2.7 The cut-point node is not a leaf agent . . . 23

2.8 An algorithm for constructing a referral g graph . . . 24

2.9 Random rewiring, starting from a regular ring (adapted from [90]) . . . 31

3.1 Probability distribution of QoS of agent Aj . . . 39

3.2 The process of deciding whether to cooperate with another agent . . . 43

3.3 Testimony propagation through witnesses . . . 44

3.4 Testimony propagation algorithm . . . 45

3.5 An example of a referral graph where A4 and A5 are leaf agents . . . 47

3.6 A distribution of referrals in the three-dimensional model . . . 48

4.1 JavaMail layered architecture . . . 51

4.2 The format of the special MARS messages . . . 53

4.3 The Macro Architecture of the whole MARS system . . . 56

4.4 Internal architecture of an agent in the MARS system . . . 58

4.5 The control flow of an agent in the MARS system . . . 59

4.6 Main Window of the MARS system . . . 61

4.7 Sign up Window of the MARS system for new users . . . 62

4.8 Window for incoming queries or answers . . . 63

5.1 Quality of network when interest and expertise vectors are similar . . . 67

5.2 Quality of network when interest and expertise vectors are independent . . . 68

5.3 Quality of network when interest and expertise vectors are opposite . . . 68

5.4 Usefulness of referrals at different weights of sociability (Quality metric) . . . 69

5.5 Improvement in quality due to a pivot at different sociability weights . . . . 70

(10)

5.7 Improvement of quality perceived by a new agent in a stabilized network . . 72

5.8 The relationship between clustering and the quality of the network . . . 73

5.9 The change of clustering starting from a random graph . . . 74

5.10 The change of clustering starting from a random graph . . . 75

5.11 The change of clustering starting from a regular graph . . . 76

7.1 Reputations change of all agents in the bootstrapping stage (from a regular ring) . . . 85

7.2 Reputation buildup and crash of a new agent . . . 86

7.3 Non-cooperative agents in different community sizes . . . 87

(11)

List of Tables

2.1 Learning by X about Y and Z when X asks Y; if Y refers to Z, then X asks Z 25

3.1 The differences between a referral graph and a trustnet. . . 44

6.1 Average number of referrals and experts found for different branching factors 79 6.2 Average number of referrals and experts found in a static referral network . . 80 6.3 Average number of referrals and experts found in a dynamic referral network,

(12)

Chapter 1 Introduction

Traditionally, there are two different sources from which human beings access informa-tion: broadcast and interpersonal communications. Decades of research on the diffusion of information indicate that interpersonal communication acts as an important channel for gathering and disseminating information [9, 35]. For example, suppose that you need to have your car repaired, and want to know if there is a mechanic with a good reputation near where you live. It is highly unlikely you will find information about reputations from the web or from directories. The most efficient way might be to call friends who have had their cars repaired recently or are knowledgeable about local auto mechanics.

(13)

relationships upon which people rely in daily life.

The importance of referrals has been known for a long time [53, 71, 94] but the ideas of referral systems appeared only recently, e.g., MINDS (based on the documents used by each user) [7, 31, 61], and ReferralWeb (based on co-occurrence of names on Web pages and bibliography database) [38, 39]. MINDS is the earliest agent-based referral system of which we have knowledge, but ReferralWeb first explicitly used referral chains to find some people with the needed information[37]. In general, MINDS is giving much attention to learning heuristics for referral generation while ReferralWeb focuses on the problem of how to bootstrap the referral system. Neither of them studies the dynamics of social structure, especially any emergence or evolution of referral networks due to referrals.

An agent-based referral network is simply a set of software agents which are linked to each other. The key feature of a referral network is that each agent can automatically generate and follow referrals upon some queries. The agents can learn on the shoulder of the user, and they can adaptively choose their neighbors or update the neighbors’ information, i.e., expertise, and sociability, upon the feedbacks from the user. Moreover, there is a good chance to find the needed experts in a referral network than in a human social network. Part of the reasons is that the query was routed among the software agents in the referral network, and the user need not worry about disturbing other users, who are shielded from seeing irrelevant messages by software agents.

In this thesis we propose a computational model of agent-based referral networks in which we focus on how to improve the accuracy of referral chains and the emergence of interesting social structure within referral networks. The long-term goal of our research is intended to develop the technology and infrastructure to allow software agents to learn from each other and rely on each other in such a way that they can cooperate together to explore potential experts in the informal person-to-person social network [85, 101].

(14)

explicitly use referral chains, whereas in referral networks there is usually more than one hop between the requesting agent and expert, and the requesting agent has to rely upon referral chains to find potential experts. Second, ReferralWeb models referral systems statically as a graph and considers referrals directly through path search in the graph. However, choosing relevant referrals is not trivial in referral networks and is one of the major topics in the thesis. We model the referral chaining process dynamically in a three-dimensional space and study the strategies of choosing referrals for different queries.

The prototype system MARS (MultiAgent Referral System) is being implemented for use in a practical social network. Unlike most previous approaches, our architecture is fully distributed and includes agents who preserve the privacy and autonomy for each user. These agents learn models of each other in terms of expertise (ability to produce correct domain answers), and sociability (ability to produce accurate referrals). However, in order for both to refine our design and to understand the principles underlying social networks, we have conducted several experiments on a simulation to see the effects that the different variables have on each other [103]. Specifically, we find that under our multiagent learning heuristics, the quality of the network improves with time; the quality is maximized when both expertise and sociability are considered; and pivot agents (who relate different subcommunities) further improve the quality of the network and have a catalytic effect on its quality even if they are ultimately removed.

(15)

by adaptively choosing the referrals in term of expertise and sociability.

Furthermore, a probabilistic model of reputation management is proposed to help agents (users) avoid interaction with non-cooperative participants [100, 102]. Our approach adjusts the ratings of agents based on their observations as well as the testimony from others. Our previous work used a scalar value to represent the reputation ratings and combined them with testimonies using some combination schemes from the Certainty Factor Model. One problem is that certainty factors do not represent measures of absolute belief. Rather, they are meant to represent changes in belief. In the thesis the mathematical theory of evidence is used to represent and propagate the reputation information in an electronic community. Our approach adjusts the ratings of agents based on their observations as well the testimony from others. Social mechanisms are even more important when some centralized reputation management mechanisms, i.e., trusted third parties, are not available. Our specific approach to reputation management leads to a decentralized society in which agents help each other weed out undesirable players.

1.1 Research challenges

There are several important challenges for the success of agent-based referral networks, and some of them are included as follows.

1.1.1 How to improve the accuracy of “referrals”

The accuracy of referrals decides if you can find the expert(s) and how quickly you can find such expert(s) in a referral network. Basically, it relies on the accuracy of referrals generated by each agent upon a query. In order to improve the accuracy of referrals, each agent has to:

(16)

• learn the expertise and sociability of other users so that he can generate some accurate referrals. This includes how to distribute the credits or penalties to all agents along the referral chains upon the return of an answer (good or bad).

Moreover, the requesting agent who sends out the query has to decide who should be asked next step. There could be hundreds of referrals returned, and it is not wise to ask all of agents being referred. In chapter 3 we define the accuracy of referrals in term of

similarity, sociability, and cooperativeness, and each agent only follows the referrals with higher accuracy in case there are a lot of referrals returned.

1.1.2 How to detect non-cooperative participants

For agents to function effectively in referral networks, an agent needs to ensure that another agent (corresponding to the user) is cooperative. In the case of multiagent systems, the agents are assumed to be fully cooperative or they are built as subject to this “rule of experiments” (they help each other because they have been designed that way [73]). In referral networks, agents will not necessarily cooperate with one another and so they must be able to deal with such undesirable participants.

Further, since we consider large distributed systems of autonomous and heterogeneous agents, it is generally inadvisable to assume that there are universally accepted trustworthy authorities who can announce the cooperativeness of different agents. Sometimes even the authorities may not be considered universally trustworthy. This is a fundamental limitation of traditional methods, the so-called “hard security” techniques based on passwords, keys, and digital certificates. What hard security can ensure, at most, is that the given agent obtained credentials from another credentialed party.

(17)

each other weed out undesirable participants.

1.2 Summary of contributions

The research issues described above have been addressed through the design and imple-mentation of the MARS prototype system which has been downloaded by researchers and students at NC State University, IBM Research and University of Washington. The major contributions from our research are summarized in the following four points.

Bring people and agents together

The conventional way to implement a multiagent system is to use specialized agents such as brokers or facilitators [32]. Despite the considerable research that has gone into the theories and architectures for such multiagent systems, there is relatively little experience with building a multiagent system with people together. Referral networks extend the stand-alone multiagent systems with people, and can be used to answer questions, such as “who knows what”, and equally important “who knows who knows what” within the informal person-to-person social network.

Moreover, referral networks enable people to share their knowledge and experience. There are economic, social, and political reasons for which much valuable information will never be made publicly accessible on the web or any other network [37]. In this situation software agents will help to search the social network for an expert with a chain of personal referrals from the searcher to the expert(s).

Organization management

(18)

The rapid change of organizational structure makes referral networks even more im-portant in which workers relying on their personal social networks, rather than unstable, weakening “organization charts” [62]. Referral networks not only create and maintain the personal social networks of users, but also manage their social networks. They can be used to answer questions which traditional communication networks (who knows who) and knowledge networks (who knows what) can answer, but have no limitation on organizational boundary.

Personalized e-commerce environment

Recommender systems, i.e., collaborative filtering, are used by many e-commerce sites, and are evolving from novelties into serious business tools that are reshaping the world of e-commerce [78]. However, most recommender systems are primarily implemented by the sellers, and they generate recommendations based on anonymous opinions. On the other side, most of us are used to an entirely different method of getting recommendations, namely, by asking a person we know. Our friends may not know the answer, but they can often refer us to someone they know. Referral networks automate the whole process and enable the buyers to share their experience of shopping. If we succeed in the endeavor, the work will be a significant contribution to the next generation shopbot [16, 17].

Trust in e-commerce and peer-to-peer networks

(19)

1.3 Organization of the remainder of this thesis

Chapter 2 defines the computational model of referral networks in which we talk about user modeling, multiagent learning in referral networks, and accuracy of referral chains in a three-dimensional space. Chapter 3 describes a social mechanism for reputation management which helps to detect non-cooperative agents in the agent societies.

(20)

Chapter 2 Computational Model

Software agents are computer programs which can perform a set of tasks autonomously for their users. They use techniques from machine learning to discover how a user does the tasks and then provide active assistance to the user [36, 54]. In referral networks each user is associated with such a software agent, which maintains a personal social network for its user. In this network, each member is represented both by expertise (ability to provide good service) and sociability (ability to provide good referrals).

Queries from the user are first seen by his agent who decides the potential contacts to whom to send the query. After consultation with the user, the agent sends the query to the agents for other likely people. The agent who receives a query can decide if it suits their user and lets the user see that query. In addition or instead of just forwarding the query to the user, the agent may respond with referrals to other users. If the agent or user wish, they can discard the query and not respond to it in any way.

(21)

only if the referring agent has some confidence in the relevance of the agent being referred. Figure 2.1 shows an example. Here agentAasks a question of two of its neighborsB and

C, who send back two referrals apiece. Let’s say these referrals terminate inE (a dead-end) and Phoebe, which returns a suitable answer. B and C’s rating will get credited for Phoebe

and debited forE.

A

B

C

E

D

Phoebe

Figure 2.1: An example referral network

In order to interact successfully with other agents, each agent has the following informa-tion:

• user profile: The user’s own areas of expertise to decide whether or not to accept incoming queries.

• neighbormodels: Models of (some) neighbors to decide whether to send them a given query. Each neighbormodel (modeling for a neighbor) contains the neighbor’s contact information, expertise, and a rating of its sociability.

• cache: Models of additional neighbors. The neighbors in the cache are not directly contacted for a query, but may be contacted if they are referred by someone else.

(22)

of other agents as well. The models of these other agents are considered the agent’s cache. The space of neighbormodels is limited, but the size of the cache is unlimited. The models in the cache are learned from interactions with the other agents, e.g., when they ask or answer a query. The cache agents are not directly contacted for a query, but may be contacted if they are referred to by someone else. By maintaining additional models in the cache, the agent in effect tracks some acquaintances who may becomes closer associates. In section 2.2.3, we introduce some strategies of neighbors selection, in which we adjust the position of each neighbor in neighbormodels and cache depending its quality of services.

Send out queries

Collect responses

Referrals?

Answers?

Evaluate the answers and update the models

No

Yes

Querying

Stop

Responding

Incoming query

Update interest of requester

Have expertise?

Send answer

Have relevant contacts?

Stop

Send referrals Yes

Yes

No No

Figure 2.2: An agent’s behavior while querying and responding

(23)

or cache. Based on accuracy of referrals, it may decide to actually follow up a referral. When the agent receives an answer, it uses the answer as a basis for evaluating the expertise of the agent who gave the answer. This evaluation affects its model of the expertise of the answering agent, and its models of any agent who may have given a referral to this answering agent. As a result, if an agent sends in a good answer, the evaluation of its expertise goes up as does the sociability of any agents who referred to it (through any path).

Figure 2.2 sums up the entire process. Note that in our framework, the referring process is different from that in peer-to-peer networks. In referral networks, all referrals are sent back to the requesting agent. For example, agentAasksB a question, and B does not know the answer but one of B’s neighbor C knows. In peer-to-peer networks, B will go to ask C

and then return the answer toA, whereas in referral networks B return the referral toAand

A goes to ask C. There are two advantages for this: (1) the requesting agent will learn his neighbors quickly; (2) the requesting agent can control when to stop the referring process.

2.1 User modeling

One of the concerns in the thesis is how to model the knowledge/expertise learned from the user and other agents. An important prerequisite for this task is the ability to represent and analyze documents in a computer system. There are several machine learning approaches that can be used to learn a user profile, such as Bayesian Networks, Neural Nets and Vector Space Model [14, 49, 50, 64]. In this section the commonly used vector space model [74] will be explained, along with the particular formulation used for our systems.

2.1.1 Vector space model

(24)

Let D = {d1, d2, . . . , dn} denote a collection of documents. Let {t1, t2, . . . , tp} be the “dictionary” - a set of all of the words of the collection. We represent each document d as a vector in a p-dimensional vector space so d=hdt1, dt2, . . . , dtpi where dti is the weight for wordti in documentd.

How are such vectors derived? The first step is to extract the constituent words from a document. Secondly, very common English words from a standard stop list are removed. Ex-amples of stop words areand, is,the, etc. Extracted words are reduced to their stems using the Porter suffix-stripping algorithm [21]. For instance, computer, computers, computing, and computability are all reduced to comput.

Words are then weighted using a “TFIDF” scheme: the weight dti of a word ti in a document d is derived by multiplying a term frequency (“TF”) component by an inverse document frequency (“IDF”) component:

dti = (0.5 + 0.5

tfi

tfmax

)(log n

dfi )

where tfi is the number of times wordti appears in documentd(the term frequency),dfi is the number of documents in the collection which containti (the document frequency),n is the number of documents in the collection and tfmax is the maximum term frequency over all words in d.

Furthermore, the vector for each document is normalized to be of unit length. Variants of this formula are commonly used by IR systems — the aim is to give high weights to words which occur frequently in the document in question but are rare in the rest of the collection.

2.1.2 Modeling the expertise and sociability

Vector space model has been used in many news and information filtering systems recently [5, 14]. We adapt vector space model to locate people rather than documents. In our formulation, there are two kinds of user models: those learned from the user, called user profile; and those learned from other agents, called neighbormodel1_{. The expertise for each}

1_{Note that there is not a suffix} _s_after _{neighbormodel}_{, and a neighbormodel could be in neighbormodels}

(25)

neighbor and the expertise for the user itself (in a user profile) are modeled as an ontology (When a user sends out a query, MARS will ask the user to explicitly provide labels of expertise domain for each query). Due to the dynamics of users’ expertise, we also represent expertise in a profile or neighbormodel as a vector, calledexpertise vector [95].

Cache NeighborModels

Profile Agent

NeighborModel

Contact Info Expertise

Contact Info Expertise Sociability

Figure 2.3: An agent’s internal data structure

Figure 2.3 illustrates the data structure of a profile or neighbormodels/cache. In the above representation, the expertise of each user was modeled as an expertise vector and a set of scalars LCi for each expertise domain Ci. The single-vector scheme may at first

appear similar to the standard way to represent a query for an ad hoc information retrieval system. However, the underlying assumptions are quite different due to the fact that a query is specified by a user whereas a user profile is learned by the system from both positive and negative feedback. Formally, the profile of user Pi or any neighbormodel for user Pj (in the neighbormodels or cache) having pexpertise domain is represented as,

prof ilePi ={EPi,hLC1, LC2, . . . , LCpi}

(26)

The query vector is generated from the initial query. The expertise vector for each user (including the user itself and its neighbors) is estimated based on previous interactions and referrals. We systematically compare a query vector with the expertise vectors of other neighbors in the same domain to find the neighbor whose expertise is the most similar to the query.

Given a query vector and an expertise vector, the similarity between the two vectors is defined as the cosine of the angle between those vectors. This model has been used as the basis for many ranking retrieval experiments, in particular the SMART system under Salton and his associates [75]. In traditional vector space model, the similarity between a query vector Q=hq1, q2, . . . , qni and an expertise vector EPi =he1, e2, . . . , eni is defined as:

P_n

t=1qtet

qP_n

t=1(qt)2

P_n

t=1(et)2

We give a slightly different definition. This definition too captures the cosine of the angle, but scales it by the length of the expertise vector. This is basically the projection of the expertise vector on the query vector in the n-dimensional vector space. We further normalize this expression by dividing by the maximum length possible. The idea is that a person whose expertise for a query is twice as large as another person is twice as desirable as a source of information. Under the traditional model, two users would be considered equally desirable as long as the angles of their expertise vectors were the same.

Definition 1 Given a query vector Q = hq1, q2, . . . , qni and an expertise vector EPi =

he1, e2, . . . , eni, the similarity between Q and EPi is defined as:

Q3EPi = P_n

t=1qtet

q

nPn t=1(qt)2

For example, given a query vector Q=h0.1,0.9i and two expertise vectorE1 =h0.5,0.5i

and E2 = h1,1i. In traditional vector space model, the two expertise vectors have equal

similarity with the query vectorQ, but in our framework,E2 is better thanE1sinceE23Q >

(27)

Definition 2 Given a query vector Q with the expertise domain Ci, the relevance of Q to user Pi is computed as max{LCi, EPi3Q}

2_{, where} _E

Pi andLCi are the expertise vector and

domain expertise scalar of user pi, denoted asQ∇CiPi.

When a user agent receives a query for expertise, it tries to match the query against the expertise vector and domain expertise vector in its user profile. If there is a good match, the query is passed on to its owner directly.

Definition 3 Given a query vector Q with the expertise domain Ci and a threshold ωA, where 0≤ωA ≤1, it says there is a good match between the userPi and the query Qfor a domainCi if max{LCi, Q3EPi} ≥ωA.

The relevance of a neighbor to a given query depends not only on the similarity of the query to the user’s expertise, but also on what weight is assigned to the user’s sociability, which reflects how much we can trust the referrals produced by this principal. Accordingly, we define the relevance as a weighted sum of the expertise and sociability components.

Definition 4 Given a query vectorQwith the expertise domainCi, the relevance ofQ and any neighborPj of the user is computed asQ4CiPj =max{LCi, Q3EPi}(1−η)+ηSj, where

SPj is the sociability of the neighborPj; andηand (1−η) are the weights given to sociability

and expertise, respectively.

Further, the user may be allowed to specify an absolute relevance threshold. The thresh-old can be adjusted to tune the number of purported experts found and to limit the number of referrals that this user will give to other users. Note that usually we have ωR≥ωA.

Definition 5 Given a query vectorQ(from the user himself) with the expertise domainCi and a threshold ωR, a neighbor Pj is relevant toQ if Q4CiPj ≥ωR for a special value η.

2_{Note that we use}_max_{here since}_L

Ci only represents the level of the domain expertise of the user. The

(28)

Definition 6 We define a preference relation Â for a query Q (with the expertise domain

Ci) over a set of neighbors {P1, P2, . . . , Pn} such that for any neighbor Pj and Pk, we say

Pj ÂQ Pk if and only if Q4CiPj ≥Q4CiPj.

One of the disadvantages of such a single-vector representation is the confusion caused when trying to represent multiple domains. In this thesis we introduce a domain specific ontology to represent user’s domain knowledge, which avoids the complexity in the learning algorithm and the neighbor selection strategy, but keep the flexibility of multiple-vector representation.

Other representations in IR, e.g., probabilistic model[72], form an important alternative to the vector space approach. The basic idea is to estimate the probability that a docu-ment is relevant to a query given probabilities over individual words. However, in practice comparisons show the vector space model is either superior or almost as good as the known alternatives (probabilistic approaches) [28]. Furthermore, vector space model is simple and fast. For these reasons, the vector space model is a popular retrieval model nowadays, and used in many state-of-the-art retrieval systems.

2.1.3 Privacy issues

Privacy in referral networks is typically thought of in terms of protecting information got from others, since a user can control his own privacy when he generates the answer upon a query. As we mentioned in the introduction section, privacy is one of the major concerns from the users when they tried to use MARS system. Such privacy concerns happened when (1) one agent generates some referrals and forwards them to other agents, and (2) one agent wants to share part but not all of its profiling information with another agent. Here we proposed to use a list, called a close friend list to address the first concern.

Definition 7 A close friend list of the user Pi is defined as ΠPi ={P1, P2, . . . , Pc}, wherec

(29)

The close friend list will be managed by the user directly. For example, when another user (response user) returns an answer, the system will ask the requesting user if he would like to add the response user to his close friend list. Figure 2.4 shows the structure of MARS agents with a close friend list. The user will use this list to decide whether he will forward a referral to other users. Now we define the rule when the user will forward a referral to the other agents upon a query.

Close Friend list

NeighborModels

Cache one-to-one mapping

Figure 2.4: Data structure of MARS agents with a close friend list

Definition 8 Given a query vector Q (from any other user Pk) with the expertise domain

Ci and a threshold ωR, the user Pi will return the referral of any of his neighbor Pj to Pk if

Q4CiPj ≥ωR for a special value η and Pk ∈ΠPi.

Sometime the query is not from any of his close friends, but referred by one of his friend. In this case it is quite reasonable to answer the query.

(30)

For the profiling information, the user can decide whether he would like to publish part of his profile information on the registration server. We will address the concern again in the chapter of the prototype system MARS.

2.2 Multiagent learning

The neighbormodels and profile must be learned by the agent through the interactions of its users with other users. When a good quality of service is obtained, the expertise of the providing user is revised upwards as is the sociability of the users who gave referrals. When a poor quality of service is obtained, the revisions are downward. A referral graph encodes how the computation spreads as a query originates from an agent and referrals or answers are sent back to this agent.

Note that we will use the terms user Pi and agent Ai interchangeably, except when we are referring to users who will give an answer.

2.2.1 Referrals and referral graph

We use Pr (or Ar) to denote the requesting user (or agent). When user Ar has a query, he will ask some neighbors in his neighbormodels. Some of them may return referrals or answers. We use a referral graph to model the whole process.

Definition 10 A referral r to agent Aj returned from agent Ai is defined as hAi, Aji, we say Ai is a parent of Aj and Aj is a child of Ai.

In convenience we also callhAr, Aiia referral, where the requesting agentAr send a query to one of his neighbors Ai. Let Q be a query from agent Ar, we assume that, after a series of l referrals, this produces a responseR from colleague Aj. The entire referral chain in this case would be hAr, . . . , Aji, with the length l.

(31)

of the referral chain.

Avoid cycles using ancestors.

Obviously, not all agents are contacted at the same time. Some agents are contacted earlier, and others are contacted latter. We use the concept “depth” to represent the order of different agents being contacted in a referral graph, and the root is the requesting agent with depth zero. Sometime, some referrals may cause a cycle if we append them to the referral graph. These referrals do not increase the power of referral graph, but introduce a lot of trouble in propagating the credits in the referral graph. In this section, the concept of

ancestor is introduced to avoid cycles in a referral graph.

Definition 12 Given a referral chain hAr, Ar+1, . . . , Aji from Ar to Aj, then any node Ak (except for Aj itself) on the referral chain from Ar toAj is called an ancestor of agent Aj, denoted as ancestor(Aj).

If any agent Ak is an ancestor of agent Aj, thenAj is a descendant of agent Ak.

Definition 13 A referral graph G is defined as a directed graph G(Ar,Λ, R), where Ar is the requesting agent, Λ is a finite set of agents {A1, A2, . . . , An} and R is a set of referrals

{r1, r2, . . . , rm}.

When appending a referral to a referral graph, we have to keep track the status of each agent, i.e., does it return an answer or not? is it the last node on the referral chain? In a referral graph we denote each agent as a referral node.

Each referral node includes the following information:

- depth (dynamic): the length of shortest path from requesting agent. - a set of parents

- a set of children - a boolean value isLeaf

- a boolean value hasReferral

(32)

For example, the root Ar of the referral graph G(Ar,{Ar, A1, A2},{hAr, A1i,hAr, A2i})

could be {0,∅,{A1, A2}, f alse, true, f alse}.

Note that some of the referrals may be redundant to the referral graph. We say a path is a closed path if it has the same starting and ending nodes. A loop in a directed graph is a closed path in an undirected graph, and a cycle is a closed directed path in a directed graph.

Definition 14 Given a referral graphG(Ar,Λ, R), a referralr=hAi, Ajiisredundantif and only if (1) agent Ai and Aj are some nodes in graph G, and (2)Aj is one of the ancestors of

Ai.

redundant

Ar

A2

A5

A4

A3

A1

Figure 2.5: Redundant referrals in a referral graph

Figure 2.5 shows an example of redundant referrals in a referral graph, where referral

hA4, A1i is redundant since A1 is one of ancestors of A4. Referral hA4, A2i looks like a

(33)

Moreover, if we look at the referrals more carefully, we can find some referrals expand the referral graph deeper, some do not. Here we introduceweak andstrong referrals in order to distinguish them.

Definition 15 Given a referral graphG(Ar,Λ, R), a referralr =hAi, Ajiisweakif and only if agentAi and Aj are some nodes in graph G; otherwise, it is strong.

Appending referrals at the boundary of a referral graph.

Ar A2 A5 A4 A3 A1 Ar A2 A4 A3 A1 A5 A7 A5 A5 A6 limit 4 0 1 2 3 4 1 0 1 2 3 2 1

Figure 2.6: The cut-point node is a leaf agent

We should be careful when we try to append some referrals to a referral chain in case it reaches the length limitation, i.e., four. For example, hAr, A1, A3, A4, A5i is a referral chain

with length four. If agentA5 returns an answer, that would be easy. However, suppose agent

(34)

of chain is already four, we cannot append the referrals hA5, A6i, hA5, A7i to the referral

graph. Can we just ignore these two referrals? The interesting thing is, suppose at this time, another agentA2 refers toA5, where A2 on the chain hAr, A2i. Now the new chain becomes

hAr, A2, A5i and the depth of A5 is changed from f our to two. The question is that if we

need ask agent A5 again. Since Ar already have the referrals returned from A5, we may

introduce a pool for Ar to store the temporary referrals instead of discarding them in case they exceed the limits of four. Each referral graph has exactly one pool, and there could be more than one pool for the requesting agent Ar. So now we just get the referrals hA5, A6i,

hA5, A7i from the pool (see Figure 2.6).

Ar A2 A5 A4 A3 A1 Ar A2 A4 A3 A1 A5 A7 A5 A5 A6 limit 4 0 1 2 3 4 1 0 1 2 2 3 1

Figure 2.7: The cut-point node is not a leaf agent

Definition 16 Given a referral graph G(Ar,Λ, R), and a new referral r =hAi, Aji, Aj is a

(35)

In the above case, A5 is the last agent on the referral chain hAr, A1, A3, A4, A5i. We

calledA5 a leaf agent, since he has no child in the referral graph. It will be more interesting

if agentA2 refers to A4 not A5 (see Figure 2.7), which has been asked before and returns the

referralhA4, A5i(hereA5 is acut-pointnode). Can we just append the referral hA2, A4i and

stop here? The answer is NO. Since the depth of agent A5 becomes three, and it does not

exceed the limitation. The referrals hA5, A6i, hA5, A7i could append to the referral graph.

An operation relax is introduced to check each descendant of the cut-point node or himself, and expand it if needed.

Assumption: An agent who gives the answer won’t make a referral.

Suppose agent Ar is the requester agent, set Γ is the agents being visited. Then given a

series of referral {r1, r2, …, r2}, for each referral rk = < Ai , Aj >, agent Ar will update the

expertise and sociability of other agents according to the following rules, (1) if Aj ∉Γ and Aj returns an answer, then

1) append rk to the referral graph, and Aj into Γ,

2) evaluate the answer and update the expertise of agent Aj and the sociability

of any agent on the referral chain to agent Aj;

3) otherwise go to (2),

(2) if Aj ∉Γ and Aj doesn’t return an answer, then just append rk to the referral graph,

and Aj into Γ, otherwise, go to (3)

(3) if Aj ∈Γ and Aj ≠ ancestor (Ai), then append rk to the referral graph, and Aj into Γ,

relax Aj and descendant (Aj), otherwise, go to (4)

(4) ignore the referral rk.

(36)

Definition 17 The operation relax (referralNodei) is defined as (1) update the depth of nodei;

(2) check if node i is a leaf agent, retrieve the referrals from the pool if any and set their depth as depth(i) plus 1;

(3) if nodei is not a leaf agent, for each child of node i, relax(child(i)).

With the conception of “ancestor”, we can avoid cycles in a referral graph. However, a non-redundant referral cannot guarantee the corresponding agents get rewards/penalties. For example, one referral r = hAi, Aji could be appended to the graph G(Ar,Λ, R) after the ending node Aj was updated, where Aj returned an answer and was evaluated by the user Pr. Therefore we will distribute the rewards/penalties to all relevant agents only when we want to destroy the referral graph. Fig 2.8 shows an algorithm for how to construct a referral graph upon a series of referrals.

2.2.2 Learning from users’ feedbacks

Now we talk about how neighbormodels and profile can be learned through the interac-tions of its users with other users. When a good quality of service is obtained, the expertise of the providing user is revised upwards as is the sociability of the users who gave referrals. When a poor quality of service is obtained, the revisions are downward.

Situation Y’s exp Y’s soc Z’s exp Z’s soc

Y gives a good answer ↑ – – –

Y gives a bad answer ↓ – – –

Y refers to Z

Z gives a good answer – ↑ ↑ –

Z gives a bad answer – ↓ ↓ –

Z gives a good referral – ↑ – ↑

Z gives a bad referral – ↓ – ↓

Table 2.1: Learning by X about Y and Z when X asks Y; if Y refers to Z, then X asks Z

(37)

answer or a referral to a good answer. An ↑ indicates that the corresponding estimate is revised upwards, whereas a↓ indicates that it is revised downwards.

First we introduce an operator Ξ, which was used to update the expertise domain scalar and/or sociability. The intuition behind the formula is that we want to model the change of sociability which is hard to be built up but easy to be destroyed.

Definition 18 Given any two variable X and Y, where 0 ≤ X ≤ 1, the operator Ξ was defined as Ξ(X, Y) =X+Y −XY if 0≤Y, or X+ XY

1+Y, otherwise.

The MARS system requires users to give explicit feedback to each response according to a scale of [0,+1] with 0.5 representing the midpoint. Suppose α is the rating given by the user. Given this information the relevance feedback method is used to update the weights of user profile vector Ei or Ek. The update rule is equivalent to the “Ide Regular” rule and “Standard Rocchio” rule [76] with additional weighting from the user’s evaluation3_.

Given a referral graph G, suppose the domain of query Q is Ci and user Pj returns an answer R, then the requesting agent Ar will update expertise and sociability as follows, where β is the learning rate, and 0 < β <0.5.

• Expertise: On the side of user Pr, agent Ar will update

– the expertise vector for Pr in the profile as (1−β)EPr +βQ, and

– the expertise vector for userPj in the neighbormodels as (1−β)EPj+αβR, and

– the expertise scalar LCi for user Pj in the neighbormodels as Ξ(LCi,(α−0.5)).

• Sociability: Suppose l is the length of shortest referral chain from Ar (corresponds to user pr) to Aj, the following algorithm will propagate the credits to his ancestors according to the distance to him, invoked by backProp (Aj, l−1,(α−0.5)) 4.

3_{In the simulation, the}_α_{value depends on how close of the answer to the interest.}

4_{Note that we give more rewards or penalties to the agents who are near to the response agent, and}

(38)

backProp (referralNode i, int l, double credits){

for each parent of node i{

if depth(parent(i))≤l then {

referralNode k = parent(i);

Sk= Ξ(Sk, credits)

back-prop(parent(k), l-1, credits/2)

}

For example, in Figure 2.7, suppose A7 returns an answer with quality α, then agent A5,

A4, and A2 will get credit α, α/2 and α/4, respectively. A1 and A3 will not get any credit

since they are too far from the responding agent A7. For example, the depth of A3 is two,

which is bigger than l value in the backProp algorithm. In this case l= 1.

If there is no answer, there will be no penalties for expertise of agent Pj, and no penalty for all members of the chain. In chapter 3 we will introduce another parameter, called

cooperativeness. We will decrease its cooperativeness in case of no answer [98].

Our work is related to reinforcement learning [33], in which programming agents are guided by reward and punishment without needing to specify howthe task is to be achieved. Q-learning [15] is the most popular and seems to be the most effective model-free algorithm in reinforcement learning for learning from delayed rewards. It does not, however, address any of the issues involved in an environment without explicit reward or punishment.

2.2.3 Neighbors selection

(39)

tour-naments of agent strategies in the Iterated Prisoner’s Dilemma, where each strategy had to play against all other strategies. Work in social psychology, however, has shown that in such games selection the right strategy to decide with whom to play yields a better performance than to spending the same effort on how to play the game itself.

Especially in the context of open systems like electronic communities, there is at least the chance of an alternative to every interaction partner, so spending resources under consider-ation with whom to play is beneficial, if not crucial. In this context we propose a statistical learning for neighbor selection. We view each response as a service with a quality s ranked from 0 and +1, and we sample the latest L responses to evaluate the quality of the service. Suppose agentAihas the latestHresponses from agentAg, denoted asSj ={sg1, sg2, . . . ,

sgH}. The user of agent Ai will evaluate each service sgk on a scale of [0,1.0], where 1 ≤

k ≤ H and 1.0 means a good service (the quality of service sgk is equal to 0 if there is no response from agent Ag).

We use f(xk) to denote the probability that a particular value xk of quality of services from agentAghappens, wherexk∈ {−0.1,−0.9, . . . ,1.0}. For example, given a specific value

xk, there are three services with that quality in the latest H responses, then f(xk) = 3/H. However, in some cases, agent Ai may get less than H responses from agent Ag. Suppose there are only h responses, whereh < H, thenf(xk) should be 3/h.

For each user Ai, given two thresholds ωi, and Ωi, where 0 ≤ ωi ≤ 1, 0 ≤ Ωi ≤ 1 and Ωi ≤ωi. We use

P₁

xk=ωif(xk) to indicate the possibility that agent Ai will cooperate with

Aj;

P_x_k_=Ω_i

0 f(xk) indicates the possibility that agent Ai will defect against Aj.

Definition 19 Given a series of responses from agent Ag, Sg = {sg1, sg2, . . . , sgH}, and the two thresholds ωi and Ωi of agent Ai, we can get the reputation toward agent Ag: Γ(Ag) =

P₁

xk=ωif(xk)−

P_x_k_=Ω_i

0 f(xk).

(40)

may not so active as before, should we keep these neighbors in the neighbormodels? In this paper we introduce a decay coefficient γ. After a period of time, i.e., one month, we use

γ to multiple the QoS of each service sgk for agent Ag, and recompute the reputations of each neighbor either in neighbormodels or cache, and adjust the position of each neighbor according to the new ranking 5_.

2.3 Emergence of referral networks

The notion of a social networkand the methods of social network analysis have attracted considerable interest from the social and behavioral science community in recent decades [88]. The phrase “social network” refers to the set of social entities (i.e., agents) , and the ties among them. Rather than focusing on attributes of each individual agent, social network analysis focuses on the patterns or structures of ties among agents and how these structural properties influence observed characteristics and associations among characteristics. Tradi-tional social network analysis studied the properties of human society, and in the thesis we focus on the social structure of referral networks.

The idea of graph analysis as a basis to study social networks has a long tradition; one of the earliest studies is Schwartz and Wood [81]. They describe the use of graph-theoretic notions such as cliques, connected components, cores, clustering, average path distances and the inducement of secondary graphs. The focus of their study was to model shared interests among a web of people, using email messages as connections. Such link analysis has been used to extract information in many areas such as in web search engines [41]. With the emergence of the web as a large-scale graph, interest in information network has recently explored, i.e., authorities and hubs [3, 8, 42].

Google essentially models a one-mode directed graph (of web pages) and uses measures involving principal components to compute “page ranks,” Jon Kleinberg’s algorithm goes a 5_{Note that in the simulation we do not have a sampling process. Whether a neighbor will be kept in the}

(41)

step further by viewing the web graph as two modes (called hubs and authorities). A hub is a node primarily with edges to authorities, and a authority is a page that is linked to by many hubs. A hub is similar to a pivot agent in our study. Instead searching in a web graph, we study the importance of pivot and authority agents during referring process, i.e., catalytic effect of pivot agents, and pivot agents on different lengths of referral chains.

2.3.1 Topology of referral networks

Many people first heard ofsmall-world phenomenonfrom the movie (based on the Broad-way play by John Guare) of this name. But the original concept came from work in the 1960s by Stanley Milgram on what’s called the small-world problem: how long a chain of people would it take to deliver a message from anyone in the world to any other person, using only the ’friends of friends’ ? Milgram found that there was a chain of only about 5.5 people (6 people if the ’sender’ and ’receiver’ are of different races) between any 2 people in the world.

The small-world phenomenon tell us that the artificial referral networks will be possible to locate the needed experts even in the world. However, an explanation of the small-world phenomenon [60] remains relatively unexplored until 1998. Watts & Strogatz [90, 89] found that small-world networks are neither fully regular nor fully random. They observe that such graphs are a highly clustered network (like regular graphs) with just a few random short paths (like random graphs). In other words, a small-world network is sufficiently clustered so that most second neighbors of a nodeX are also neighbors of X (a typical ratio would be 80%). On the other hand, the average distance between any two nodes in the graph is comparable to the low characteristic path length of a random graph.

Watts and Strogatz provide the first such characterization of a small-world network in the form of a graph generation model. In this model, they use a regular ring network with

(42)

to the original ring. The starting point in the figure is a regular ring topology of 12 nodes with every node connected to its four nearest neighbors. The structure has a high charac-teristic path length and high clustering coefficient. The average length is the mean of the shortest path lengths over all pairs of nodes. The clustering coefficient is determined by first computing the local neighborhood of every node. The number of edges in this neighborhood as a fraction of the total possible number of edges denotes the extent of the neighborhood being a clique. This factor is averaged over all nodes to determine the clustering coefficient. The other extreme is a random network with a low path length and almost no clustering. The small-world network, an interpolation between the two, has the low path length (of a random network), and retains the high clustering coefficient (of the ring).

Figure 2.9: Random rewiring, starting from a regular ring (adapted from [90])

(43)

essentially converges to a small-world network. The results from our simulation show that the structure of referral network converges to a small-world network from either a regular ring or random graph in term of clustering coefficient and average path length. The goal of the thesis is intended to develop an accurate, dynamic, and evolving multiagent system that can provide personalized recommendations for users. Each agent can find the shortest path to the expert(s) while requiring only local, autonomous decisions by its member agents.

2.3.2 Weak-ties in referral networks

The study of weak ties comes from the question “how do people find out about jobs?” Are relatives, close friends, acquaintances, employment agencies, or newspaper advertisements the most useful sources about jobs? Mark Granovetter, a sociology graduate student, tried to answer this question in his PhD research. He interviewed about 100 and sent questionnaires to about 200 people in the Boston area who had changed jobs or recently been employed. All of these were what he called ”professional, technical, or managerial” workers; they were not blue collar workers.

Granovetter found that many of the paths between the job and the new employee were surprisingly indirect, and some of the paths were surprisingly long. In addition, the person who told them about the job was often someone they did not know very well. Edward found out about the job from his sister’s old boy friend, Norman from another parent in his child’s nursery school. Thus, contact. Using these questions, Granovetter distinguished between ”strong” and ”weak” ties. Strong ties exist when people see each other frequently over long periods of time. The relationship is close and intimate. Weak ties are the opposite; weakly tied individuals see one another infrequently and their relationships are casual rather than intimate.

(44)

Therefore, the information they provide is likely to be redundant; what he hears from one friend he is likely to have heard from others as well because they talk to one another and they are themselves exposed to the same sources of information. On the other hand, one’s acquaintances are likely to come from different social circles. They are likely to be different from the job seeker himself and different from each other. Therefore, they are exposed to different sources of information. Each of them will tell the job seeker things he has not heard from other acquaintances or friends.

(45)

Chapter 3 Detecting non-Cooperative Agents

The worldwide expansion of network access is driving an increase in interactions among people. For example, in e-commerce, it is important to know the trustworthiness of buyers and sellers before initiating any commercial transaction. Different authors have given various definitions for the term trust, which can be broadly classified into two views: the cognitive view and the mathematical view. In the cognitive view, trust is made up of underlying beliefs, and trust is a function of the value of these beliefs [12]. The mathematical view ignores the role of underlying beliefs and uses a scalar metric to model a subjective probability with which an agent will perform a particular action [55].

In the case of multiagent systems, the concept of trust will make our agents more robust with respect to interactions with agents that are not our own. Assuming blindly cooperation is not necessarily the correct approach. In some cases, they may cooperate; in others, their behavior can range from non-cooperation to downright maliciousness. Since we are uncertain what their behavior may be at any time, the notion of reputation proposed in the thesis relies on a judgment based on past experience, coupled with, if available, past knowledge of other agents.

(46)

coopera-tive behavior of a single agent by its environment. The value of trustworthiness is the rating of the agent’s cooperative behavior by its former transaction partners. If no information about the agent is available, it has “no” reputation at all. It should be noted that there is a difference between having a bad reputation, and no reputation.

We consider the important case of interactions in electronic communities, where the agents assist and represent principal entities, such as people and businesses. We propose a social mechanism of reputation management, which aims at avoiding interaction with undesirable participants. Social mechanisms complement hard security techniques (such as passwords and digital certificates), which only guarantee that a party is authenticated and authorized, but do not ensure that it exercises its authorization in a way that is desirable to others. Social mechanisms are even more important when trusted third parties are not available. Our specific approach to reputation management leads to a decentralized society in which agents help each other weed out undesirable players.

3.1 Why Dempster-Shafer’s theory of evidence

We have developed a family of approaches for social reputation management that apply in a decentralized open environment in which agents help each other weed out undesirable parties [98, 99]. We used a scalar value to represent an agent’s belief ratings about another and combine them with testimonies using combination schemes similar to the certainty factor model [98]. One problem with this work is that certainty factors do not represent measures of absolute belief. Rather, they are meant to represent changes in belief [29]. The drawbacks of the certainty factor models led us to consider alternate approaches. Particularly appealing is the mathematical theory of evidence developed by Arthur Dempster and Glenn Shafer [82]. In the thesis we proposed to use Dempster-Shafer’s evidence theory as the framework for social reputation management. We will focus on how to use the Shafer’s theory to combine a collection of belief functions and to propagate the belief functions through a TrustNet.

(47)

Dempster-Shafer calculus handles this notion explicitly, and with more ease [48]. In Dempster-Dempster-Shafer theory one considers the evidence in favor of a hypothesis. There is no causal relationship between a hypothesis and its negation, and so lack of belief does not imply disbelief. Rather, lack of belief in any particular hypothesis implies belief in the set of all hypotheses, which is referred to as the state of uncertainty. This leads to the intuitive process of narrowing a hypothesis [26], in which initial uncertainty is replaced with belief or disbelief as evidence is accumulated.

Before introducing the Dempster-Shafer theory, we attempt to show some simple justifi-cation for the approach. In general, an agent Ai does not know with full certainty whether another agent Aj is trustworthy or not, but Ai may be able to estimate the degree of trust about Aj. Dempster-Shafer theory handles this uncertainty explicitly, and with more ease than the Bayesian model [48]. Moreover, some evidence available to agent Ai may neither support Aj’s being a trustworthy or nontrustworthy agent. Dempster-Shafer theory models this “ignorance” naturally, which is cited as a major motivation for the Dempster-Shafer theory [83].

3.2 Prisoner’s dilemma

To better understand the notion of trust in communities, let’s discuss the famous pris-oner’s dilemma [4]. The prispris-oner’s dilemma arises in a non-cooperative game with two agents. The agents have to decide whether to cooperate or defect from a deal. The payoffs in the game are such that both agents would benefit if both cooperate. However, if one agent were to try to cooperate when the other defects, the cooperator would suffer considerably. This makes the locally rational choice for each agent to defect, thereby leading to a worse payoff for both agents than if both were to cooperate.

(48)

have to repeatly interact with each other. Our observation is that a reputation mechanism sustains rational cooperation, because the better players are rewarded by society whereas the bad players are penalized. Both the rewards and penalties from a society are greater than from an individual.

The proposed approach builds on (and applies in) our work on constructing a referral network for information gathering [97, 103]. In our architecture, each user is associated with a personal agent. Users pose queries to their agents. The queries by the user are first seen by his agent who decides the potential contacts to whom to send the query. After consultation with the user, the agent sends the query to the agents for other likely people. The agent who receives a query can decide if it suits its user and let the user see that query. In addition to or instead of just forwarding the query to its user, the agent may respond with referrals to other users.

When the originating agent receives a referral, it decides whether to follow it up. When the agent receives an answer, it uses the answer as a basis for evaluating the expertise of the agent who gave the answer. This evaluation affects its model of the expertise of the answering agent, and its models of any agent who may have given a referral to this answering agent. In general, the originating agent may keep track of more peers than his neighbors. Periodically he decide which peers to keep as neighbors, i.e., which are worth remembering.

3.3 Dempster-Shafer’s theory of evidence

Aframe of discernment is defined as the whole set of propositions in which each is known to be true. In our example, suppose the frame of discernment Θ contains only T and ¬T, where T stands for the trustworthy relationship between any two agents. The Dempster-Shafer theory assigns a number in the range [0,1] to every subset of Θ (excluding the empty set), called basic probability assignment (bpa). And the sum of all the bpa’s must equal 1.

Definition 20 If Θ is a frame of discernment, then a function m : 2Θ _7→ _[0_,_{1] is called a}

Emergence and Evolution of Agent-Based Referral Networks

AGENT-BASED REFERRAL NETWORKS

To my wife and parents

BIOGRAPHY

ACKNOWLEDGEMENTS

Contents

List of Figures

List of Tables

Chapter 1

Introduction

1.1

Research challenges

1.1.1

How to improve the accuracy of “referrals”

1.1.2

How to detect non-cooperative participants

1.2

Summary of contributions

1.3

Organization of the remainder of this thesis

Chapter 2

Computational Model

A

B

C

E

D

Phoebe

2.1

User modeling

2.1.1

Vector space model

2.1.2

Modeling the expertise and sociability

2.1.3

Privacy issues

2.2

Multiagent learning

2.2.1

Referrals and referral graph

2.2.2

Learning from users’ feedbacks

2.2.3

Neighbors selection

2.3

Emergence of referral networks

2.3.1

Topology of referral networks

2.3.2

Weak-ties in referral networks

Chapter 3

Detecting non-Cooperative Agents

3.1

Why Dempster-Shafer’s theory of evidence

3.2

Prisoner’s dilemma

3.3

Dempster-Shafer’s theory of evidence