Use HPC-ABDS on multiple hardware systems
General Discussion of HPC-ABDS: February 2017
... exemplified by the Swift customized scripting system and the growing use of Python. Apache orchestration of this style is seen in ODE and Airavata with latter coming from Grid research at Indiana University. ...
37
Towards HPC ABDS: An Initial High Performance BigData Stack
... software systems such as Hadoop available from the Apache open source community and thus referred to as High-Performance Computing-Big Data Stack (HPC–ABDS) – has two fundamental building blocks: (i) ...
22
HPC ABDS High Performance Computing Enhanced Apache Big Data Stack
... applications use capabilities at layers 14 which we divide into the classic or batch programming in 14A and the streaming area 14B that has grown in attention ...supports multiple data APIs including graphs ...
10
HPC ABDS High Performance Computing Enhanced Apache Big Data Stack (with a bias to Streaming)
... dataflow systems like Tez and projects like Apache Crunch with a data parallel emphasis based on ideas from Google ...of multiple batch and streaming components; a capability that we expect to become ...
85
Comprehensive Resource Use Monitoring for HPC Systems with TACC Stats
... Stampede systems. These systems both use CentOS [23] as their operating systems but have compute nodes with distinct chip ...additional hardware support is for ...the HPC ...
9
Multiple-Antenna Systems: From Generic to Hardware-Informed Precoding Designs
... coupling effect by the use of tunable antenna loads, which further improves the performance of the generic compact multiple-antenna systems. With the deployment of the tunable load impedance, for ...
279
hwloc: a Generic Framework for Managing Hardware Affinities in HPC Applications
... to use hwloc is through its C-programming ...operating systems provide a specific ...some use iterators while some do not, some manage all objects in a uniform manner while others do not, ...
9
Parallel Event Selection on HPC Systems
... of HPC hardware and available compute cycles, our goal is to explore how we can use the capabilities and organization of these big machines to minimize the processing of analysis-level data, ...
8
Record-and-Replay Techniques for HPC Systems: A Survey
... of hardware concurrency, the effective overlap of computation and communication, and overall code complexity make R&R a vital tool for coping with non-determinism in HPC ...in HPC ap- plications is a ...
20
Trends in Data Locality Abstractions for HPC Systems
... to use in high-level code; yet few languages or libraries provide interfaces for the application developer to inform the compiler about expectations of data locality, data layout, or memory ...containing ...
15
A Study on Data Deduplication in HPC Storage Systems
... stored multiple times across different files, for example, multiple versions of a file might exist that are mostly ...that use identical regions refer to the same unique ...
11
The use of Java in large scientific applications in HPC environments
... Unlike other hardware changes that have happened, the move to multi-core processors has mas- sive implications for software development. Although multi-core processors have been common- place for a number of ...
240
APPLICATION OF CLOUD COMPUTING AS HPC PLATFORM FOR EMBEDDED SYSTEMS
... a HPC platform, we must choose the virtualization platform that gives the best ...handle HPC jobs as well (KVM is not compared in this literature ...to use hardware assisted virtualisation, ...
12
Application-Agnostic Framework for Improving the Energy Efficiency of Multiple HPC Subsystems
... most systems receiving packets requires more processing than ...grouping HPC workloads based on their use of available resources, we define six labels: compute-intensive, memory-intensive, mixed, ...
9
Using HPC-ABDS for Streaming Data
... brings HPC to ABDS • Basic Harp: Iterative HPC communication; scientific data abstractions • Careful support of distributed data AND distributed model • Avoids parameter server approach but ...
47
A Global System Architecture Next Generation HPC-ABDS
... • The performance of Spark, Flink, Hadoop on classic parallel data analytics is poor/dreadful whereas HPC (MPI) is good • One way to understand this is to note most Apache systems. delib[r] ...
19
Implementing parts of HPC-ABDS in a multi-disciplinary collaboration
... File systems: HDFS, Swift, Haystack, f4, Cinder, Ceph, FUSE, Gluster, Lustre, GPFS, GFFS Public Cloud: Amazon S3, Azure Blob, Google Cloud Storage 7) Interoperability: Libvirt, Libcloud, JClouds, TOSCA, OCCI, ...
31
HPC ABDS: The Case for an Integrating Apache Big Data Stack
... DACIDR for Gene Analysis (Use Case 19,20) • Deterministic Annealing Clustering and Interpolative Dimension Reduction Method (DACIDR) • Use Hadoop for pleasingly parallel applications, a[r] ...
22
HPC Hardware Overview
... Longhorn Related References User Guide services.tacc.utexas.edu/index.php/longhorn-user-guide General Information www.intel.com/technology/architecture-silicon/next-gen/ www.nvidia[r] ...
59
Towards HPC ABDS: An Initial Experience Optimizing Hadoop for Scalable High Performance Data Analytics,
... HPC-ABDS Background Applications The Harp Library SPIDAL • Harp is an implementation designed in a pluggable way to bring high performance to the Apache Big Data Stack and bridge the differences between ...
52