Evaluating trust

Enabling agents in open MAS to evaluate the trustworthiness of their peers and, thus, to be able to select reliable ones for interactions is the main aim of this thesis. To this end, FIRE was developed based on a number of potential sources of trust information. These sources include: direct experiences of an agent from its interactions, witness reports, third-party references, and rules provided by end users encoding beliefs or knowledge about the environment. This breadth is im- portant because, given the dynamic factors that inherently exist in an open MAS, some sources may not be available, or adequate, for deducing trust. Moreover, the multiple sources are used in FIRE not only to back one another up, but also to complement one another in order to produce more precise trust values (c.f. just using one of them). In order to combine trust values derived from different sources of information, a generic framework was developed to standardise trust calculations. This includes:

• a standardised rating form: to represent trust information from any source which is used for exchanging trust information. A rating is not given for an agent for its performance in general, but for its performance in a particular interaction. Therefore, each rating is linked with a particular interaction, providing further contextual information (e.g. value of the interaction, ser- vice provided in that interaction) if needed. Moreover, this also eliminates the correlated evidence problem (RequirementR4b) because no overall opin- ion (i.e. opinions given based on results of more than one interaction) is exchanged.

• a general trust formula: to aggregate the trust information (i.e. ratings) that a trust component collects, which is used in all FIRE’s components. However, depending on the information source used, each component can define its own weight function to reflect the quality of each rating taken into account.

• reliability measures: to produce reliability values for each trust values based on the quality of the ratings taken into account and their deviation.

This framework is generic because it works independently of any specific applica- tion and it does not rely on any assumption or information which is not widely available in an open MAS. Thus, FIRE can be instantiated and applied in a wide range of applications (Requirement R3).

Under this framework, each component of FIRE is developed to process trust information from each of the sources mentioned above. The components are: In- teraction Trust, Role-based Trust, Witness Reputation, and Certified Reputation. First, based on the principles of Regret’s Direct Trust component, the IT compo- nent is built to produce trust values from an agent’s own ratings from its direct interactions. A new rating weight function is devised to calculate the reliability of a rating based on its recency. Second, a formalisation of role-based rules is presented in order for the RT to retrieve relevant rules and calculate the role- based trust based on those rules. Third, the referral process was implemented for the WR component to locate witness ratings for witness reputation calculations. Finally, and most importantly, a novel mechanism was developed for making use of third-party references in the CR component, in which the target agents obtain references themselves and present those to the evaluator when requested. The ad- dition of this new type of reputation greatly enhances the serviceability of FIRE, allowing a trust measure to be available in most circumstances because:

• its mechanism addresses the problem of the lack of direct experience (since agents can typically collect a large number of references themselves and they are incentivised to present these to establish new trust relationships) in the IT component, and

• using the CR component, agents are freed from the various costs involved in locating witness reports (e.g. resource, time, and communication costs). Making use of all the four components, FIRE effectively combines their particular strengths in building a robust trust measure: the reliability of direct experiences, the domain knowledge from role-based rules, and the abundance of third-party information via witness reports and certified references. Moreover, having the four sources of information at its disposal (especially certified references thanks to their high availability) means that FIRE can provide a trust measure that is sufficiently precise to be used in a wide range of situations (Requirements R1a

and R1b). Obviously, there are still cases when FIRE cannot produce a trust value. Specifically, those are when a service provider newly joins the system. Hence, it does not have references about its performance and other agents do not have past experience with it. However, in a realistic scenario, in order to promote its service, that provider can join a (popular) scheme/organisation that provides quality assurance about its members’ service. For example, a car dealer can obtain the title ‘authorised dealer’ from a car manufacturer, or a commercial site can assure its potential customers about its security reliability by showing

a ‘HackerSafe’ 1 _{certification. Such (popular) memberships (and inherently their}

quality assurance) can be recognised by other agents (via rules in FIRE’s RT component) and, thus, helps the provider to sell its service.

In addition to the above, a notable characteristic of FIRE is that all of its mech- anisms are decentralised. This means that individual agents can use FIRE to make trust evaluations without the need of a centralised authority. This is im- portant for making FIRE compatible with the ‘no central authority’ of open MAS (RequirementR2a).

In order to verify our claims, empirical evaluation was carried out and it was demonstrated that:

• Agents using the trust measure provided by FIRE are able to select reliable partners for interactions and, thus, obtain better utility gain compared to those using no trust measure. This result was reconfirmed with various types of provider population.

• Each component of FIRE plays an important role in its operation and sig- nificantly contributes to its overall performance.

• FIRE is able to cope well with the various types of changes in an open MAS and can maintain its properties despite the dynamism possible in an environment (RequirementR2c).

• Although decentralised, to suit the requirements of a trust model in open MAS, FIRE still outperforms, or at worst maintains a comparable perfor- mance level with SPORAS, a centralised trust model.

• In our experiments (Chapters 5, 6, and 7), 500, 1000, and 1500 agents using FIRE have been deployed and we observe the execution time of those ex- periments varies linearly to the number of agents deployed. Thus, given its decentralised nature, we believe that FIRE is scalable to the large number of agents that may be present in an open MAS (Requirement R2b).

To sum up, FIRE satisfies all our requirements (plus Requirement R4b, which is dealt with in the following section) for a trust model in open MAS. Its behaviour can be customised via its set of parameters to suit a particular application. Hence, FIRE is ready to be used in real world contexts.

1_{‘HackerSafe’ certifications are provided by ScanAlert (a security company,www.hackersafe.} com) to certify that the certified sites’ servers are regularly tested (by real security attacks) and shown to be hacker-proof.