High Performance Computing Systems and Enabling Platforms

High Performance Computing Systems

and Enabling Platforms

Marco Vanneschi

Department of Computer Science, University of Pisa

Master Program (Laurea Magistrale) in Computer Science and Networking

Master Program (Laurea Magistrale) in Computer Science and Networking

High Performance Computing Systems and Enabling Platforms

Marco Vanneschi

Course Introduction

www.di.unipi.it/vannesch section: Teaching

My activity

• Research area

– Computer Architecture

– Parallel and Distributed Processing, High Performance Computing

– Parallel Programming Models and Tools

– Programmability of various HPC platforms

• Multiprocessor, Cluster, Grid Computing, Multi-core, Pervasive Computing

– Coordination of some National and European Projects (basic research

and industrial research)

• Research group

– Co-leaded with Prof. Marco Danelutto

– Laboratory of Parallel Architectures

SPA: course objectives

• Providing a solid framework of concepts and techniques in

high-performance computing

– Parallelization methodology and models

– Support to parallel programming models and software development tools – Parallel Architectures

– Performance evaluation (cost models)

• Methodology for studying existing and future systems

• Technology: state-of-the-art and trends

– Parallel processors – Multiprocessors

– Multicore / manycore / … / GPU

– Shared vs distributed memory architectures – Programming models and their support

4

SPA: motivations

• Evolution of computer technology towards parallelism and

HPC

• Multi/many core

• Large clusters, cloud, …

• Heterogeneous large-scale enabling ICT platforms • Embedding

• Increasing maturity with respect to “hardware-software

relationship”

• Concurrency and parallelism as first-class citizens in

application development

• In our Master Program: HPC is a fundamental methodology

and technology for integrated ICT infrastructures and

SPA: first course in HPC and enabling platforms

• Distributed Systems: Paradigms and Models

(Prof. Danelutto),

fundamental, 2nd semester

– assumes SPA as prerequisite

• Other complementary courses in this area:

– in Study Plan “Distributed Systems and Applications”.

Course approach: a Computer Science approach

knowledge

technological knowledge

• Computer Science approach

o Parallel computing: computational and programming models o Cost models of parallel computations

o Computing architecture has its own concepts, principles, models, and techniques

o Conceptual framework in common with the other disciplines of Computer Science:

Basic background and prerequisites

• An undergraduate-level course on

structured computer

architecture

– Firmware level structuring

– Assembler level, CPU architecture, compiling – Memory hierarchies and caching

– Interrupt handling, exception handling

– Process level, addressing space, low level scheduling, interprocess communication

– Input/Output processing

• In Pisa: the course “Computer Architecture” (Undergraduate

Program in Computer Science) adopts such an approach

• Precourse on Computer Architecture fundamentals

Precourse: subjects and approach

1. System structuring: level, modules, cooperation models

2. Firmware level fundamentals

3. Assembler level

4. Memory hierachies and caching

5. Process level fundamentals and run-time support

• Approach of Precourse:

– Main concepts and techniques on these subjects

– Exercises: proposed to students and discussed (class-level or individually)

• Recommendation: Precourse-level knowledge is of fundamental

importance for SPA course and exam.

Precourse: teaching material

• Slides

: course notes

– On my personal page

• Students are recommended to take notes during the lectures

• Reference books:

– M. Vanneschi

, “Architettura degli Elaboratori”, PLUS, 2009, in Italian.

– D.A. Patterson, J.H. Hennessy,

“Computer Organization and Design:

the Hardware/Software Interface”, Morgan Kaufman Publishers Inc.

– A. Tanenbaum

, “Structured Computer Organization”, Prentice-Hall.

Working approach: Precourse and SPA

• As in any other course, it is fundamental to acquire skills and

capabilities in concepts and principles, besides knowing the

technologies.

• Critical aptitude must be properly developed.

• Interaction with the teacher is strongly encouraged

– Questions during the lectures

– Presentation and discussion of exercises

– Question time

• Monday, 14:00 – 16:00, in my room: PRECOURSE

• Tuesday, 9:00 – 11:00, in my room: REGULAR COURSE

• and/or by appointment in case of collision with other courses, or other

SPA: Course Program

PART 1 (about 3 CFU)

1.

Methodology for Parallel Application Structuring and Design

Models, parallelization methodology and techniques, parallel paradigms, programming models, cost models

PART 2 (about 3 CFU)

2.

Shared memory architectures

SMP, NUMA, …, interconnection networks, support to concurrency mechanisms, cost models, static and dynamic optimizations, parallel application benchmarks

3.

Distributed memory architectures

Clusters, MPP, …, interconnection networks, support to concurrency mechanisms, cost models, static and dynamic optimizations, parallel application benchmarks

SPA: Teaching Material

• Course notes

– published on my personal page every 1-3 weeks

– (basically: new English version of Vanneschi’s book, Part IV)

• Reference books:

– Patterson: see Precourse references

– D.E. Culler, J.P. Singh, A. Gupta, “Parallel Computer Architecture: a Hardware/Software Approach”, Morgan Kaufman Publishers Inc. – Papers and reports

SPA: exam modality

• Written + oral exam

(in English or in Italian)

• No intermediate tests (no midterms)

• Exam approach: the student must be able to demonstrate a

deep understanding of concepts and techniques

and

of their

interrelationships, as well as to write / to present the answer

in a clear and rigorous way.

• Registration to the exam on the Official Site of Corso di Laurea:

http://compass2.di.unipi.it/didattica

, section Laurea Magistrale in

Informatica e Netwoking, subsection “orari”

Architetture Parallele e Distribuite (ASE)

Esame di ASE (9 CFU), laurea specialistica vecchio ordinamento:

• SPA (6 CFU)

• + integrazione 3 CFU

(Libro PLUS, Parte IV)

– implementazione di programmi paralleli in LC secondo forme di parallelismo note

– complementi di architetture parallele

• Modalità di esame:

scritto.

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 16