Qualitative Analysis

This section provides a qualitative analysis of Evaluation Criteria 6 - 8, i.e., that the RMS should make risk explicit to a decision-maker; that the RMS should provide advice based on trust, risk, context, and interaction dynamics to a decision-maker; and that the RMS should adhere to the characteristics of a trust-based decision-making system that unifies the characteristics and properties of human trust. The analysis of each of these criteria is described as follows.

4.4.1 Making Risk Explicit

Both agent- and context-specific risk can be captured and assessed by the RMS design, as explained in section 3.4.9. The risk component evaluates risk based on several factors, i.e., the trustworthiness of the trust target in the current context as expressed by the trust value and the overall risk of interacting in a given context. By explicitly reasoning about risk, the RMS allows users to specify acceptable levels of risk for interactions with other entities in specific contexts. The RMS makes risk explicit to the decision-maker, both in terms of the risk of financial loss due to untrustworthy behaviour based on the calculation of a trust value that is behaviour-specific, contextually relevant, and based on reliable evidence and in terms of the risk of paying an artificially inflated price due to colluding behaviour between a seller and bidder. By analysing evidence on behalf of users, the RMS reduces complexity because a given user is no longer confronted with an abundance of evidence that he could not manually process and evaluate in terms of risk. Moreover, complexity is reduced and usability ensured by the analysis of risk in terms of trust-based evidence and the presentation of that analysis to a user.

4.4.2 Advice Provision

As described in section 3.5.4, the RMS is designed to provide advice based on trust, risk, context, and interaction dynamics to the decision-maker, guiding the user on whether or not to proceed with an interaction, instead of providing a reputation summary or a trust value to a decision-making user. Therefore, a user of the RMS can be guided by advice based on an automated analysis of evidence to make a more informed, and therefore more accurate, decision about whether or not to proceed with an interaction.

4.4.3 Trust-Based Decision Making

This section discusses the ability of the RMS to meet the trust-based decision-making criteria identified in Chapter 2. First, the specification of confidence in outcome expectations is possible. As described in section 3.4.8, the RMS uses a trust measure, i.e., the (s,i,c)-triple, that is expressed as a value that captures combined evidence for and against a given outcome, as well as capturing uncertainty, and implicitly communicates the amount of evidence, i.e., number of interactions, used to determine trustworthiness. We do, however, note a deficiency in that confidence is not explicitly specified or used by our system in this regard. We suggest that this could be easily resolved through the integration of a confidence assessment policy specification capability in the access control component of the RMS. Clearly, further experiments would need to be performed to determine general confidence thresholds, i.e., the amount of evidence required to shift from low to high confidence in an RMS security decision may be subjective and therefore vary across users.

Next, the RMS was designed to capture the diversity dimensions of trust, i.e., trust origin (individual or entity), trust target (individual or entity), and trust purpose. This is done in part through the design of role- and environment-appropriate event structures, described in section 3.4.4, to provide a flexible and extensible model of the types of events a trust origin would be interested in analysing when considering an interaction with a trust target for a given trust purpose.

Because not all entities have the same perception of evidence, the RMS is designed to allow for the subjective specification of trust formation, evolution, and exploitation processes in terms of context, timeliness, and recommendation integrity, as described throughout sections 3.4 and 3.5. For example, Alice might only be interested in the most current evidence about interactions with Bob in one context, while Carl might choose more lax contextual relevance and time-fading policies to assess historical evidence about Bob. Because subjectivity is also based on an entity’s disposition and belief system, our design may easily be extended to allow for policy to be specified such that the eff function may adjust trust values according to various trust dynamics. Furthermore, because the RMS provides a security decision in terms of advice, a user can still act according to his own disposition, either heeding or ignoring the advice of the RMS.

Evidence that is based on past behaviour can be directly observed or indirectly recommended and used to update trust both positively and negatively in a dynamic and non-monotonic manner. A single trust value is created from the combination of direct and indirect evidence, i.e., observations and recommendations. Observations and recommendations are evaluated independently so that a trust origin is able to keep its own direct evidence about a trust target separate from the general population’s opinion about the trust target, and so that recommendation integrity can be assessed and calculated into the final trust value. Furthermore, the processes to collect and analyse evidence are explicit.

Trust is context-dependent, and the RMS captures context, including asymmetrical relationships (role), time, and environmental factors. As discussed in sections 3.4.7 and 3.4.8, the design provides for evidentiary assessment according to contextual relevance of evidence according to role, time, and environmental factors.

Both agent- and context-specific risk can be captured and assessed, and the RMS makes risk explicit to the decision-maker, as discussed above in this section.

The RMS produces a meaningful and usable measure of trust based on the subjective analysis of contextually relevant evidence, and exploits this measure on behalf of a trust origin in two ways, i.e., the measure may be propagated to the community via recommendations (see section 3.4.6), and the measure may be used to making trusting decisions to interact for a given trust purpose with a given trust target in light of associated risk, as detailed in section 3.4.8.

Finally, the RMS significantly reduces the complexity of decision-making in an environment in which uncertainty and risk are present. The RMS is able to automatically process large amounts of evidence

that a human user could not manually process in order to make a timely decision about whether or not to interact. Furthermore, as noted earlier in this section, instead of providing a reputation summary or a trust value to a decision-making user, the RMS outputs actual advice to a user about how best to proceed in an interaction, given trust, context, interaction dynamics, and risk.

Our model helps clear conceptual confusion by representing trust as a broad but coherent set of constructs that incorporate the major definitions from research to date. It draws together the many aspects of trust to make trusting decisions as a function of available evidence, subjective beliefs, and context in the face of risk and uncertainty. This design captures the characteristics and properties of trust that were suggested in Chapter 2, and that the RMS provides a trust-based decision-making process to advise users regarding trusting behaviour. In conclusion, the RMS allows for trust to be made computationally tractable in the domain of reputation management for Internet auctions while retaining a reasonable connection with human and social notions of trust that guide human users to make decisions in that domain.

4.5

Chapter Summary

In this chapter, we evaluated the RMS according to an extensive evaluation plan addressing eight evaluation criteria. After describing the evaluation plan, we evaluated the RMS design in three parts. First, simulations of extra-auction user behaviour allowed us to evaluate our trust value calculation mechanism and its extensions to assess contextual relevance, timeliness, and recommendation weight. Moreover, the results of the simulations assisted in our demonstration that the RMS mechanisms reduce complexity, increase accuracy, and maintain usability in the assessment of evidence to determine trustworthiness and therefore limit exposure to the risk of interacting with untrustworthy entities. Next, we ran our collusion detection algorithms over real Internet auction data which allowed us to demonstrate the ability of the RMS to suspect colluding behaviour types based on the analysis of interaction dynamics. In fact, a highly probable instance of collusion was detected using this mechanism. Finally, we provided a qualitative analysis of the design of the RMS in terms of the trust characteristics and properties identified in Chapter 2, which allowed us to demonstrate that the RMS adheres to the characteristics of a trust-based decision-making system.