Distributed Computing

Top PDF Distributed Computing:

Operation of the ATLAS Distributed Computing

Operation of the ATLAS Distributed Computing

The ATLAS distributed computing(ADC) [1], [2] was designed to meet the computational and storage needs of the ATLAS experiment. The experiment has produced over 370 PB of data to date and it is adding new data at a rate of 1.5 PB per week (See figure 1). The data is handled by a central data distribution management system based on the Rucio data manage- ment system [3]. It is processed in di ff erent workflows centrally and by end users using the Athena framework. The traditional unit of processing work in ATLAS is the job. Every job is generated by the Panda workflow management system [4] from requests and tasks defined by the task definition system, ProdSys2, and dispatched to one of the 180 computing resources, in over 40 countries, according to the specific job requirements and resource properties. The computing resources at ATLAS’ disposal consist of WLCG GRID sites [5], commercial and private Clouds, High-Performance Computers (HPC) and volunteer computing resources [6]. ADC currently executes on average 1.1 M jobs per day (figure 2) on 347k (figure 3) com- puting slots on average, peaking at 960k slots for several hours. The global average transfer throughput of the system is 12 GB/s (figure 4).
Show more

7 Read more

Distributed computing and communication in peer to peer networks

Distributed computing and communication in peer to peer networks

This is not to say that some comparative work has not already been done between the various grid systems, just not as much as one might expect given the abundance of systems available. Radić and Imamagić (2004) have provided results on the performance of several job management systems and provided comparative results between the Sun Grid Engine, Torque and Condor. The NAS Parallel benchmarks also included some sample results based on earlier parallel machines (Bailey et al 1991). So called “stat races” are also a prominent feature in online, volunteer distributed computing. Many project groups doing processing will provide online statistics regarding total number of users, CPU years, aggregate memory (in terms of RAM) and so on. Many of these statistics are from BOINC projects with a mix of other technologies such as distributed.net, Apple's X-Grid and others
Show more

166 Read more

An Algorithm for Optimized Cost in a Distributed Computing System

An Algorithm for Optimized Cost in a Distributed Computing System

Distributed Computing System (DCS) is a collection of independent computers interconnected by transmission channels that appear to the users of the system as a single computer. Distributed systems are groups of networked computers. The word distributed in terms such as DCS, referred to computer networks where individual computers were physically distributed within some geographical area. The terms are Now days used in a much wider sense. Each node of DCS is equipped with a processor, a local memory,and interfaces. The purpose of the distributed system is to coordinate the use of shared resources or provide communication services to the users. In distributed computing, each processor has its own private memory (distributed memory). The processors in a typical distributed system run concurrently in parallel. The required processing power for task assignment applications in a DCS can not be achieved with a single processor. One approach to this problem is to use (DCS) that concurrently process an application program by using multiple processors. as a means of differentiating between the various components of a project. It can also be understand as usually assigned piece of work to the processor often to be finished within a certain time. A task is a piece of code that is to be executed and task
Show more

5 Read more

Enabling grassroots distributed computing with CompTorrent

Enabling grassroots distributed computing with CompTorrent

Abstract. This paper describes the operational characteristics of “CompTorrent”, a general purpose distributed computing platform that provides a low entry cost to creating new distributed computing projects. An algorithm is embedded into a metadata file along with data set details which are then published on the Internet. Potential nodes discover and download metadata files for projects they wish to participate in, extract the algorithm and data set descriptors, and join other participants in maintaining a swarm. This swarm then cooperatively shares the raw data set in pieces between nodes and applies the algorithm to produce a computed data set. This computed data set is also shared and distributed amongst participating nodes. CompTorrent allows a simple, “home-brewed” solution for small or individual distributed computing projects. Testing and experimentation have shown CompTorrent to be an effective system that provides similar benefits for distributed computing to those BitTorrent provides for large file distribution.
Show more

12 Read more

Unified Analytical Models of Parallel and Distributed Computing

Unified Analytical Models of Parallel and Distributed Computing

In relation to it this paper describes the deriving and testing of a correction factor of used standard analytical models in order to extend behavior analysis of parallel computers with another more precise analytical model. The developed corrected standard model was extensively tested and the results were compared with both standard analytical model and simulation results too. The results clearly show that the correction factor contributes to better results with a negligible increase in processing time. Its advantage, in comparison to possible used simulation method, is its ability to analyze also large existed communication networks of massive parallel computers (MPC). In this way we hope that more precise corrected analytical model would help in effective resource projecting of suggested parallel computers and parallel algorithms (effective PA) too. In this way parallel computing on networks of conventional personal workstations (single, multiprocessor) and Internet computing, awaits to unify advantages of parallel and distributed computing at their performance modeling too. In summary according input technical parameters of parallel computer we can apply to performance modeling of developed analytical models in actually typical following cases
Show more

12 Read more

Experiences with distributed computing for meteorological applications: grid computing and cloud computing

Experiences with distributed computing for meteorological applications: grid computing and cloud computing

To make it as simple as possible for a (meteorological) end user to use distributed computing resources, we make use of a so-called middleware system. ASKALON, an existing mid- dleware from the Distributed and Parallel Systems group in Innsbruck, provides integrated environments to support the development and execution of scientific workflows on dy- namic grid and cloud environments (Ostermann et al., 2008). To account for the heterogeneity and the loosely cou- pled nature of resources from grid and cloud providers, ASKALON has adopted a workflow paradigm (Taylor et al., 2007) based on loosely coupled coordination of atomic ac- tivities. Distributed applications are split in reasonably small execution parts, which can be executed in parallel on dis- tributed systems, allowing the runtime system to optimise resource usage, file transfers, load balancing, reliability, scal- ability and handle failed parts. To overcome problems result- ing from unexpected job crashes and network interruptions, ASKALON is able to handle most of the common failures. Jobs and file transfers are resubmitted on failure and jobs might also be rescheduled to a different resource if transfers or jobs failed more than 5 times on a resource (Plankensteiner et al., 2009a). These features still exist in the cloud version
Show more

12 Read more

A Dynamic Networked Browser Environment for Distributed Computing

A Dynamic Networked Browser Environment for Distributed Computing

As predicted, these where the three prominent areas that have effected distributed computing in the last ten years and it is still constantly evolving and adapting as new technologies are developed. High speed networks have helped this growth and many individuals and organisations are now starting to turn to the idea of distributed computing as a functional and cost effective alternative to the traditional supercomputers used for solving large problems. Of course one factor that has contributed to the success of distributed computing is the fast growth of the Internet in recent years. The Internet provides a strong backbone in which distributed systems can be built, whether they are local systems or a worldwide system.
Show more

74 Read more

Xcache in the ATLAS Distributed Computing Environment

Xcache in the ATLAS Distributed Computing Environment

Abstract. Inherited from the flexible architecture of Xrootd, Xcache allows a wide range of customization through configurations and plugin modules. This paper describes several completed and ongoing R&D efforts of using Xcache in the LHC ATLAS distributed computing environment, in particular, using Xcache with the ATLAS data management system Rucio for easy-to-use and to improve cache hit rate, to replace Squid and improve distribution of large files in CVMFS, to adapt the HPC environment and the data lake model for efficient data distribution and access for the HPCs.
Show more

6 Read more

Title: Overloading in Distributed Computing System- A Review

Title: Overloading in Distributed Computing System- A Review

utilizes a network of many computers, each accomplishing a portion of an overall task, to achieve a computational result much more quickly than with a single computer. A computer program that runs on distributed system is known distributed program. The process of writing such types of languages is called distributed programming [14]. Grid computing and Cluster computing are types of distributed computing systems. A Distributed system consists of a group of independent computers associated through a network and sharing middleware which enables computers to organize their behavior and to share the property of the system so that users identify the system as a single, in corporate computing facility [15]. There are many properties of distributed computing system. First of all each computational entity has local memory. The entities communicate with each others with the help of message passing. Second the system has to tolerate failures in individual computers. The system structure and links may changes during the execution of the distributed programs. Each system is only aware about the input of the system. Resource sharing is the capability to use any hardware, software or data anywhere in the system. Resources in a distributed system, dissimilar the centralized one, are physically encapsulated within one of the computers and can only be accessed from others by communication. Openness is apprehensive with extensions and improvements of distributed systems. New components have to be included with presented components so that the added functionality becomes accessible from the distributed system as a whole [6].
Show more

5 Read more

Computing while Charging: Building a Distributed Computing Infrastructure using Smartphones

Computing while Charging: Building a Distributed Computing Infrastructure using Smartphones

Today, a number of organizations offer their workers with smartphones for varied reasons [1]; a survey from 2011 [2] reports that sixty six of surveyed organizations do therefore and plenty of those organizations have 75–100% of their workers victimisation smartphones. For example, Novartis [3] (with 100,000 workers in a hundred and forty countries)handed out smartphones for its employees to manage emails, calendars, as well as data regarding health issues; Lowe’s [4] did therefore for its workers to own real time access to key product information and to permit managers to handle body tasks .In this paper, we argue that in such settings, an enterprise will harness the mixture computing power of such smartphones, to construct a distributed computing infrastructure. Such an infrastructure may cut back each the capital and energy prices incurred by the enterprise. First, this could cut back the amount of servers to be purchased for computing functions. For example, Novartis awarded a contract of $2 million to IBM to build an information center for his or her computational tasks [5]. If they could exploit the smartphones handed bent on their workers to run some portion of their work, it is conceivable that the value of their computing infrastructure could are reduced. Due to recent advancements in embedded processor design, now a smartphone will replace a traditional desktop or a server running a twin core processor for computation. According to Nvidia, their Quad Core CPU, Tegra 3, outperforms an Intel Core two couple processor in variety crunching [6]; for alternative workloads, one can expect the performance of the 2 CPUs to be comparable. Our second motivation for the smartphone-based computing infrastructure is that the enterprise could profit from important energy savings by closing down its servers by offloading tasks to smartphones. The power consumed by an advert PC processor like the Intel Core two couple is 26.8W [7] at peak load. In contrast, a smartphone CPU will be over 20x additional power economical, e.g., the Tegra 3 has a power consumption of one.2W [7, 8]. Since their computing abilities square measure similar, it is conceivable that one can harness 20 times additional machine power whereas overwhelming the same energy by commutation one server node with a plurality of smartphones. In fact, to harness the energy efficiency of embedded processors, cloud service providers square measure already pushing towards ARM-based information centers [9].
Show more

17 Read more

Overview of the ATLAS distributed computing system

Overview of the ATLAS distributed computing system

Abstract. The CERN ATLAS experiment successfully uses a worldwide com- puting infrastructure to support the physics program during LHC Run 2. The Grid workflow system PanDA routinely manages 250 to 500 thousand concur- rently running production and analysis jobs to process simulation and detector data. In total more than 370 PB of data is distributed over more than 150 sites in the WLCG and handled by the ATLAS data management system Rucio. To prepare for the ever growing LHC luminosity in future runs new developments are underway to even more efficiently use opportunistic resources such as HPCs and utilize new technologies. This paper will review and explain the outline and the performance of the ATLAS distributed computing system and give an out- look to new workflow and data management ideas for the beginning of the LHC Run 3. It will be discussed that the ATLAS workflow and data management systems are robust, performant and can easily cope with the higher Run 2 LHC performance. There are presently no scaling issues and each subsystem is able to sustain the large loads.
Show more

8 Read more

A COMPARATIVE STUDY ON DISTRIBUTED COMPUTING AND CLOUD COMPUTING

A COMPARATIVE STUDY ON DISTRIBUTED COMPUTING AND CLOUD COMPUTING

Distributed Computing: Distributed computing system provide concepts in which multiple client machines are connected on a network to solve a common problem. Each machine has its own task and gives an illusion as if a single computer is working [3]. All client machines are located at geographically different location and communicate through message passing. Distributed computing is also called as distributed system [4].Fast response time, cost effectiveness and incremental growth are some major advantages of distributed systems. Some challenges in distributed computing in [5, 6] are:
Show more

5 Read more

Modifications in Lamport Algorithm for Distributed Computing System

Modifications in Lamport Algorithm for Distributed Computing System

From the above work, it is concluded that the Unified Modeling Language is a powerful tool and can be used to model the complex research problem. In the above mutual exclusion of execution of a process can be represented by the use of UML. It is observed that a well known researcher Lamport has published an algorithm for mutual exclusion for the distributed computing system which was applicable only for the self node or computer system. In the reference of this, the above work is extended when the numbers of the computer systems are attached according to the bidirectional ring and especially for the newly developed static step topology under distributed environment. The work is extended from the self node to the next promising node and execution of process is according to the symmetric property and further extended for the next promising node by using the transitive property. In this manner, the process may be executed inside the static step topology and in all the cases message complexity is of linear order. The sharing of resources for execution of a process is also done in the same manner. The present work can be extended in many directions like for finding the node failures during the transferring of the messages from one computer system to another computer system. The loading, balancing and resilence issues are also the major areas for extending the proposed algorithms.
Show more

8 Read more

Trends of Manufacturing Systems with Distributed Computing

Trends of Manufacturing Systems with Distributed Computing

The industry developed dramatically in the second half of the 20th century, And with it developed and manufacturing systems ranging from manual to fully computerized systems employing information and communication technology (ICT). This fact has made the manufacturing systems to be totally dependent on ICT and therefore these systems have to keep pace with the advancement in ICT. Distributed computing has totally changed the computing paradigm in recent times resulting in rapid employment of these technologies in the manufacturing sector. An important variable in the equation determining the trend of manufacturing technologies is the purchaser choice and preference which has become active recently. To address these heterogeneous user demands, the Autonomous Decentralized System (ADS) concept was introduced five decades ago. The ADS has been a significant development incorporated in modern manufacturing systems and have been standardised as the de-facto standard for factory automation. These systems hold the assure for on-line system maintenance, timeliness and assurance, ensuring greater productivity and cost benefit emerging as the system of choice in automated manufacturing systems. This paper reviews the ADS, its application to a manufacturing system, assesses the state of the art and discusses the future trends.
Show more

7 Read more

Cloud Computing as Evolution of Distributed Computing A Case Study for SlapOS Distributed Cloud Computing Platform

Cloud Computing as Evolution of Distributed Computing A Case Study for SlapOS Distributed Cloud Computing Platform

The cloud computing paradigm has been defined from several points of view, the main two di- rections being either as an evolution of the grid and distributed computing paradigm, or, on the contrary, as a disruptive revolution in the classical paradigms of operating systems, net- work layers and web applications. This paper presents a distributed cloud computing plat- form called SlapOS, which unifies technologies and communication protocols into a new technology model for offering any application as a service. Both cloud and distributed com- puting can be efficient methods for optimizing resources that are aggregated from a grid of standard PCs hosted in homes, offices and small data centers. The paper fills a gap in the ex- isting distributed computing literature by providing a distributed cloud computing model which can be applied for deploying various applications.
Show more

14 Read more

DESIGNING A TASK ALLOCATOR FRAMEWORK FOR DISTRIBUTED COMPUTING

DESIGNING A TASK ALLOCATOR FRAMEWORK FOR DISTRIBUTED COMPUTING

Thus the task allocation problem can be described as “given a distributed computing system made up of n processors (p1, p2,……..Pn) and several tasks (t1, t2,…….tm), each made up of k modules (m1, m2,……mk), 1<=k<=Large Integer, each module may execute on any processor, allocate each module (of all the tasks) to one of each processors such that an objective cost function is minimized subject to constraints imposed by both the systems and application [1]. In other words we can say the problem of task allocation is to map the tasks, represented by tasks graphs, onto the processing nodes such that it takes optimal time to produce the result.
Show more

7 Read more

The JINR distributed computing environment

The JINR distributed computing environment

Abstract. Computing in the field of high energy physics requires usage of heterogeneous computing resources and IT, such as grid, high performance computing, cloud computing and big data analytics for data processing and analysis. The core of the distributed computing environment at the Joint Institute for Nuclear Research is the Multifunctional Information and Computing Complex. It includes Tier1 for CMS experiment, Tier2 site for all LHC experiments and other grid non-LHC VOs, such as BIOMED, СOMPASS, NICA/MPD, NOvA, STAR and BESIII, as well as cloud and HPC infrastructures. A brief status overview of each component is presented. Particular attention is given to the development of distributed computations performed in collaboration with CERN, BNL, FNAL, FAIR, China, and JINR Member States. One of the directions for the cloud infrastructure is the development of integration methods of various cloud resources of the JINR Member State organizations in order to perform common tasks, and also to distribute a load across integrated resources. We performed cloud resources integration of scientific centers in Armenia, Azerbaijan, Belarus, Kazakhstan and Russia. Extension of the HPC component will be carried through a specialized infrastructure for HPC engineering that is being created at MICC, which makes use of the contact liquid cooling technology implemented by the Russian company JSC "RSC Technologies". Current plans are to further develop MICC as a center for scientific computing within the multidisciplinary research environment of JINR and JINR Member States, and mainly for the NICA mega-science project.
Show more

8 Read more

Distributed Networking is a distributed computing network system, said to be "distributed" when the

Distributed Networking is a distributed computing network system, said to be "distributed" when the

The same system may be characterized both as "parallel" and "distributed"; the processors in a typical distributed system run concurrently in parallel. Parallel computing may be seen as a particular tightly coupled form of distributed computing, and distributed computing may be seen as a loosely coupled form of parallel computing. Nevertheless, it is possible to roughly classify concurrent systems as "parallel" or "distributed" using the following criteria:
Show more

7 Read more

A methodology for design coordination in a distributed computing environment

A methodology for design coordination in a distributed computing environment

This paper describes a generic methodology that allows the management and coordination of design analysis tools. A Computer Aided Design tool, namely the Design Coordination System (DCS), has been developed to assist the designer in performing computational analysis in a distributed computing environment. Within the DCS, a collection of design agents act as members of a multi-functional team operating in a cooperative and coordinated manner in order to satisfy the objective of efficiently performing the design analysis.

6 Read more

Distinguishing Parallel and Distributed Computing Performance

Distinguishing Parallel and Distributed Computing Performance

MPI designed for fine grain case and typical of parallel computing used in large scale simulations – Only change in model parameters are transmitted – In-place implementation – Synchroni[r]

26 Read more

Show all 10000 documents...