Providers and requesters interact with each other through aninteraction protocol, which
specifies the steps to follow during the selection process and the types of messages they exchange. The interaction process starts when a service request is sent from a requester to a provider, as described in the top of Figure 4.2. When an agent identifies a needed service and has a set of candidate providers that can perform that service, it uses the provider selection mechanism to find the one to which it will send the request. Similarly, when an agent receives a number of requests from other agents to perform some service on their behalf, it uses the requester selection mechanism to choose which requests to accept, taking into account available resources for service provision.
If a request is accepted, the provider performs the service, and the requester waits for the service result from the provider. After service execution is finished, both provider
and requester evaluate the service. The interaction process is complete when both
evaluations are computed and stored.
4.8
Conclusion
In this chapter we have presented an overview of the proposed architecture for support- ing cooperative interactions and the decision-making process of self-interested entities regarding service provision and request in cooperative environments with free services. This architecture is composed of four modules: a framework for non-monetary cooper-
ations, a provider selection mechanism, a requester selection mechanism, and an evalu- ation method.
The cooperation framework consists of using reciprocity as the key incentive for cooper- ation in open systems with unpaid services. We propose the use of social techniques to represent reciprocal relationships as part of agent behaviour (instead of being externally imposed by the system). Norms and commitments are not considered here, since they are higher level structures, defining mechanisms to ensure that agents collaborate and reciprocate, but not addressing the basic motivations that direct agents to interact. The evaluation method aims at providing evaluations for services, so that those with good evaluations can be chosen in future interactions and those with poor evaluations can be avoided. Finally, the provider and requester selection mechanisms support decision- making over interaction partners. To achieve this, these mechanisms use information on service evaluation and reciprocal relationships provided by other components in the agent architecture. The proposed architecture thus combines all the required functionalities for effective cooperation in open cooperative systems.
The next chapters describe in full our proposed solutions, which are represented by each individual component of the architecture.
Evaluation Method
5.1
Introduction
As described in the previous chapter, in open cooperative systems in which there is a large number and variety of available services, and where service providers can join and leave, it is likely that a participant requesting services will find many alternative providers for similar services. Similarly, there is a good chance that a service provider in such systems will receive many requests from different participants. This requires a means for participants in open systems to choose among interactions partners. Moreover, when available services are not only diverse in terms of functionalities and results, but are also continuously updated, improved or modified, as described in Chapter 3, the problem of choosing alternatives is made even more difficult.
Clearly, the most important thing here is for service users to be aware of the quality of the results they receive, so that they can make better choices of service in future interactions. Then, based on the principle that a provider that has performed well in the past is likely to perform well again in a similar situation, if services manifest a certain regularity of behaviour, an efficient way to select interaction partners is by identifying partners with good outcomes in previous interactions. In order to determine
the best outcome in this way, however, service users must perform an evaluation of
services after they are executed and the results are received. This evaluation should
reflect the satisfaction of the user with the service outcome, in relation to any number
of criteria, such as the quality of the interface, the provider’s availability to perform the service when requested to do so, the time taken to execute, the quality of the content returned, and so on. Evaluation thus allows a requester to identify, from the set of
possible providers, that particular provider with the best characteristics, like highest
quality of results and lowest time to complete the request.
Similarly, serviceproviders can use service evaluation to select which requests to accept.
However, unlike the requester’s perspective, this evaluation must reflect the effort of the 55
provider with service execution, in relation to such things as processing time, memory usage, etc. In this case, for example, the evaluation allows the provider to identify, among all candidate requests, those which, if fulfilled, require less time to perform.
In this way, evaluation provides a criterion for future decision-making over alternative
interaction partners. Therefore, in this chapter we propose an evaluation method to
analyse service outcomes. Such an evaluation is needed to determine how satisfied a
service user is with a service it has requested and received, and theeffort invested by a
service provider, both of which can be used for future decision-making over alternative interaction partners.
Although the generalfunctionality being offered (or requested) in a distributed system
can be viewed either as an agent or a service, we distinguish both concepts here such
that aserviceis a functionality itself and anagent is a service user or provider (or both),
so that one agent may provide or request several different services.
The chapter starts in Section 5.2 by analysing the key evaluation properties for dynamic services and discussing alternative approaches to evaluation. We then propose, in Section 5.3, a general evaluation method for dynamic services, and the evaluation process to be followed by service providers and requesters is summarised in Section 5.4. In Section 5.5, we use the proposed evaluation method to evaluate bioinformatics services used for protein identification and discuss the evaluation results. A comparison with similar evaluation methods is presented in Section 5.6, and we conclude in Section 5.7.