REGISTRATION AND DISCOVERY 103

4.7.1 Discovery Service Configuration

Unlike traditional Grid schedulers DRIVE does not rely on GRIS as the sole means of resource discovery. The burden of discovery and scheduling is in part shared amongst participants, using reverse discovery techniques. Never the less the discovery system plays an important role in targeting allocation advertisements at appropriate resources and discovering DRIVE services.

Early information service architectures, such as MDS-1 [48] used centralised services for re-source discovery. In this model users and schedulers send information queries to a single cen-tralised index service and resource providers constantly update the Index Service on resource usage. This approach limits scalability and the centralised service may become a bottleneck, it is also clearly a single point of failure. MDS-3 [49] allows information services to be organised in a hierarchy which improves the scalability problems seen in the centralised model. In a hierarchi-cal model information flows in a stream like manner where individual services pass information upstream. Figure 4.15 shows a hierarchical discovery configuration in DRIVE. Service metadata propagates around the system between MDS services. Multiple information sources may register with individual Index Services. Users and other Index Services query directory services to dis-cover resources and services. One can think of this system like DNS in which information can be found by progressively going up the tree. However this approach may still lead to scalability issues when attempting to discover large numbers of registered entities and it may be susceptible to failure if authoritative nodes fail. As the size of the Grid increases centralised and hierarchical models do not guarantee scalability and fault tolerance.

The most widely proposed scalable discovery model for Grid based systems uses distributed hash table (DHT) P2P overlay networks [203, 69, 204, 205]. P2P models have an extensive body of research and multiple implementations are available that can be used to create distributed, scal-able, and fault tolerant discovery infrastructures. These models typically use decentralised P2P overlay networks to share resource information between VOs. Talia et al. [204] use a combina-tion of a VO level P2P network and a VO level hierarchy of Index Services, thus combining the advantages of each method. Generally, in P2P discovery models, each node in the system stores information relating to one or more resources. In order to discover suitable resources a user sends a request to any node in the system, the node then returns matching resource descriptions if they are stored locally, or forwards the request to another node which in turn does the same. While these systems have a number of positive aspects including distribution, scalability, autonomy, and fault tolerance there are however limitations with P2P based approaches including efficiency, security, trust, and reputation [69].

The advantage of the MDS approach is each piece of information is stored in XML resources fa-cilitating standardised access and filtering based on XPath. A new distributed MDS service could be created by leverage existing work on P2P based WSRF containers [206], in which resources can be discovered using the P2P network. This is in effect the same representation of information as in MDS and provides similar resource queries following the proposed P2P protocol. The DRIVE components have been designed with MDS as the discovery architecture due to its widespread

<ServiceMetadata>

Figure 4.15: DRIVE Discovery Topology. DRIVE Metadata and Service Metadata propagates around the hierarchical MDS topology.

use in Grid environments and standardised nature. However the components that perform dis-covery are easily adaptable to other disdis-covery models assuming the nodes in the network expose similar interfaces (registration and discovery) to that of MDS.

4.7.2 Reverse Discovery and Publishing

The use of a hint based discovery mechanism helps to improve auction efficiency at the expense of absolute knowledge. Reverse discovery mechanisms are used in DRIVE to complement tradi-tional provider discovery and ensure all entities have the opportunity to participate in resource negotiation. Allocation advertising in DRIVE is only targeted at suitable providers based on reg-istered profiles as it is not feasible for Allocation Managers to individually notify all members of a VO, instead, allocations are also published in VO wide Publishing Services to allow other providers the chance to discover ongoing negotiations. These discovery mechanisms are illus-trated in Figure 4.16, two providers are notified by explicit advertising (left) and two providers discover the allocation through a Publishing Service (right).

The DRIVE Publishing Service is essentially a bulletin board for the VO, allowing Allocation Managers the ability to share market information with members without explicitly notifying all entities. Publishing Services describe current negotiations and also allocation results to share mar-ket conditions throughout the VO. The data stored in the Publishing Service could be registered

4.8. SUMMARY 105

DRIVE Agent

Economic Protocol

Service Provider

DRIVE Agent

Economic Protocol

Service Provider

DRIVE Agent

Economic Protocol

Service Provider

DRIVE Agent

Economic Protocol

Service Provider

Discover

Agents Publish

Allocation

Figure 4.16: DRIVE Reverse Discovery. The Allocation Manager discovers suitable providers from the information service and then advertises the allocation directly. The providers on the right discover allocations through a Publishing Service and may choose to participate in the allocation.

in the MDS Index service alongside provider metadata, however the use of a separate service allows fine grained control over publishing, extensibility in published data, and it can be imple-mented as a simple obligation service without requiring communication, aggregation or complex topologies. In addition if publishing to MDS was required, the resources stored in the publishing services could be trivially configured to register with MDS.

4.8 Summary

This chapter has presented the DRIVE architecture, including detailed descriptions of the core DRIVE components and the protocols used to facilitate discovery, allocation, and SLA creation in a distributed market. The architecture is based on a co-op model in which a group of distributed DRIVE services provide decentralised meta-scheduling functionality. These services can poten-tially be hosted on participating providers through the use of secure allocation protocols. The architecture is designed to create an open market through the use of arbitrary economic proto-cols. Auction protocols in particular are well suited to distributed resource allocation as they can dynamically and efficiently establish market prices. In an untrusted environment secure auction protocols can be used to establish trust and privacy in the allocations. To facilitate the require-ments of secure protocols DRIVE includes a plug-in protocol interface and a generic Auction

Component. DRIVE uses a novel two phase contract creation architecture to formally describe participant obligations. An incentive model is used to encourage participants to honour estab-lished contracts. One of the major goals of the DRIVE architecture is providing independence from a particular provider type, task model, and economic protocol. This independence has in-fluenced many of the design decisions made, for example the plug-in protocol model, generic task description and term language, and separation from a particular execution model. To verify the architecture a prototype implementation of DRIVE used to create a federated Grid marketplace is presented in the next chapter.

Chapter 5