Distributed computing provides a viable solution for solving problems that do not fit in the memory of a single computer or are suitable to be executed using computational components distributed over a Wide Area Network, possibly in parallel. Collaborative environments even have a social dimension because they facilitate exchange of ideas and knowledge. Within this section we provide an overview of the main architectural styles and related technologies used for building distributed applications except Web Services which are discussed in Section 2.4 and Grids which are discussed in Section 2.5. The disadvantages that the architectural styles presented in this section have makes them less appropriate for building an infrastructure for symbolic computation.
According to Enslow [87], simple distribution of processing elements over a network must not necessarily be considered a distributed system. To fully leverage the advantages that distributed systems can provide, he emphasizes that a truly distributed system has to comply with the following rules:
• Processing elements to handle a task should be dynamically chosen; there should be more than one processing component capable to process a certain task and the system must be able to dynamically select the most appropriate;
• The computational elements must be autonomous and physically distributed over a network; autonomy guarantees a processing unit the freedom to admit or to refuse a request based on internal rationale;
• The system should have a high-level control framework that makes possible inte-gration of distributed components into a whole;
• Services that are offered by the autonomous components should be identified us-ing a namus-ing scheme. A client must use the namus-ing scheme to specify a service request while the control framework is responsible for mapping the request to the processing element. An important difference between network computing and dis-tributed computing is that the latter uses machine names rather than IP addresses to specify the target machine that should handle a request;
• The components of the system should be able to collaborate to solve problems without a specific request coming from the control framework level;
The above definition is extremely restrictive and rules out a wide variety of distributed computing models. Autonomous behaviour is nevertheless of high importance if compu-tational resources that are integrated in the distributed system are governed by separate organizations. Independent resource providers may still want to be able to control the way their computational resources are used. A definition that covers in a more loose way the notion of distributed systems is given by Tanenbaum [175]: “A distributed systems is
a collection of independent computers that appears to the users of the system as a single computer”. As the author notes, two aspects are of great importance when character-ising a distributed system. The first one is that the system is composed of autonomous entities. The second aspect that the compound nature of the system has to be transparent for the users of the system.
The architectural styles described in the previous subsection isolate software architec-tural characteristics from strucarchitec-tural point of view. An important aspect that has to be considered in the context of distributed computing is the communication model used between computation components of the architecture. Variations of old architectural styles and new styles were created to respond to the new architectural constraints and requirements.
The architectural styles developed for distributed environments use as foundation the client-server style. The plain client-server style is based on request-response interactions that occur between the client and the server. The interface of the server components represent single points of entry which makes the server components easier to control and more secure. Software applications needed at the client level may be less complex and therefore easier to install and maintain. In order to execute the client side application fewer resources are required since most of the computational effort is now externalized to the server component. Data is usually stored and manipulated in a centralized way at server level, which eliminates the need for replica management and allows easier management of concurrent access.
The simplicity of the client-server model comes with the price of poor scalability. The maximum number of clients that the server can handle at the same time may be rapidly reached resulting in high response time or even denial of service. Scalability of the system is also poor if the internal components are tightly coupled. Modern distributed systems rely on client-server model to connect their components but the separation of function within the architecture is no longer evident. Some systems may use complex topologies such as centralised, ring, hierarchical, decentralised which are based on the
pure client-server architectural style. Hierarchical or decentralised topologies make no clear separation between clients and servers. An example is the peer-to-peer model in which components may act both as clients and servers.
Main-program-subroutine architectural style is implemented in distributed environments using the client-server style. The resulting style, Remote Procedure Call (RPC), allows a main routine to call a subroutine that is hosted in a different address space, usually a processing element hosted on a different machine. This model is the precursor of several distributed architectural models, the most popular being Web Services. More details on pure implementations of RPC can be found in [176].
Due to its advantages, object oriented programming (OOP) is currently the most used programming model. Similar to the RPC model, an application may use objects that are not necessarily resident on a single machine. The actual invocation of the methods of a remote object is transparent to the user and the communication logic is provided by the distributed framework onto which the application was build. The most popular models that provide support for distributed objects are CORBA [183] and RMI [9].
Common Object Request Broker Architecture (CORBA) is an initiative supported by Object Management Group (OMG), a not-for-profit computer industry consortium. One of the main purposes of CORBA was to create the premisses for inter-operability be-tween applications developed in different languages. CORBA uses the IDL language to describe the public interface of the objects which makes their interface platform in-dependent. The Object Request Broker (ORB) has the responsibility to find the actual object that must be invoked, to activate it if necessary, to pass values of the incoming parameters and to return to the client the result of the computation. Several languages have built in support for CORBA, such as C, C++, Java, Smalltack and several vendors have implemented ORBs. Unfortunately, the main goal of interoperability was not fully achieved because of lack of interoperability between various ORB implementations.
The Remote Method Invocation (RMI) has many similarities with CORBA but it does not offer support for interoperability with components that are implemented in other
Client
Service Registry
Service Provider DISCOVER
INVOKE
SERVICE
ADVERTISE
Figure 2.1: Service Advertising and Discovery
programming languages. Objects having predefined structure and advertised through the special service RMI Registry may be invoked remotely through proxies. Even if the actual implementation is specific, RMI uses the same pattern to enable remote in-vocation. An object design to be remotely invoked is registered with the RMI Registry service. A client queries the RMI Registry and obtains a local representation of the object implemented remotely. Any method invocation on the local representative is translated through the network to the remote object which executes the request and provides the result of the computation.
The general implementation pattern is therefore similar for both RMI and CORBA and it is also used in the case of Web and Grid Services. A client that wants to invoke a proce-dure/method implemented on a remote machine obtains a handle by querying an index service as described by Fig. 2.1. Unfortunately all implementations described above provide limited discovery mechanisms that enable a client to choose the best service to
invoke based on the service description. Web Services were developed to offer similar support while improving interoperability and discovery capabilities. We address Web Services and their advantages in more detail in the following section.
Multi-processor computers and clusters are especially designed as multi-threaded par-allel architectures. Although their components are not geographically distributed they share most of the characteristics of a distributed system. Every processor may have its own memory space and computational elements may be connected through a Local Area Network (LAN). The small communication latency of such systems makes them suitable to solve problems that require intense inter-process data exchange. There are two base models of communication that may be used efficiently for LAN based distributed sys-tems: distributed shared memory and message passing.
The distributed shared memory model provides an extended address space through which processors may access a shared memory pool. This pool is obtained by integrating the local memory of the participating computing elements. The extended address space is transparent for the user and the underlying system mediates all read/write operations.
Message passing may be use in combination with a distributed shared memory environ-ment or stand-alone systems that do not share local memory space. If message passing is used, inter-process communication is achieved through message exchange. Specific programming models for this type of distributed system are PVM [104] and MPI [133].
Both PVM and MPI are libraries that offer the fundamental tools that allow a heteroge-neous collection of machines to be used as single distributed parallel processor. They offer standard APIs and implemented subroutines that facilitate inter-process communi-cation. The main process of an application implemented using PVM or MPI, also called master, controls the initial set-up of the execution environment, uses explicit calls to de-termine parallel execution and controls all message exchange calls. The slave processes explicitly requested to be created by the master process in the initialization phase have the purpose to receive and solve computations received from the master. As opposed to the client-server architectural style where the client and server are autonomous, the
life-time and the behaviour of the slave processes is entirely controlled by the master. Due to these considerations MPI and PVM are less suited for building Internet scale distributed computational environments.
Maximization of computational resource usage, especially of processing time is an im-portant area of research for computational systems developed using clusters and multi processor machines. Software tools such as Condor [74] and PBS [47] manage such resources using scheduling and load balancing techniques to ensure optimal utilization of computational resources. The users do not have to determine themselves the most appropriate machines where their tasks should be executed. They submit the tasks to the task manager which is responsible for planning tasks’ execution on the most appropriate machines. Using resource managers improves resource utilization but it also ensures better response of the system to user needs and eliminates job starvation.