High Performance Computing Infrastructure in Japan

(1)

High Performance Computing Infrastructure

in Japan

Kento Aida

National Institute of Informatics

(2)

Kento Aida, National Institute of Informatics

2

(3)

Introduction

n

 

High Performance Computing Infrastructure (HPCI)

Ø national project promoted by Ministry of Education, Culture, Sports,

Science and Technology (MEXT) in Japan

Ø distributed computing infrastructure for high performance computing

ü “K computer”, supercomputers and high performance storage

Ø first production level infrastructure for high performance computing

in Japan

n

 

roadmap

Ø – Mar 2011 basic design

ü network, authentication, user management, shared storage, testbed for advanced software

Ø Apr – Dec 2011 detailed design

Ø Jan – Aug 2012 test operation

Ø Sep 2012 – production level operation

(4)

_{HPCI shared storage}

computer

ü no password

ü run jobs on supercomputers

ü access files on shared storages

(2) single sign-on (1) account registration

Services

4 cert. HPCI account ü input HPCI account and password ü operation through a web browser ü application ü account (3) login to resources

(5)

portal CA system shib. SP shib. SP shared storage single sign-on apply certificate authentication network infrastructure computer resource shib. IdP shib. IdP shib. IdP HPCI acct. HPCI ID registration review proposals user management certificate repository

System Overview

5

More resources will be connected after 2012. AICS, U. Tokyo NII HPCI Secretariat (RIST) acct. registration helpdesk computer

resource computer resource

AICS (K-computer)

Supercomputer Centers in 9 Universities

(6)

As of Nov. 2012 RIKEN AICS： K computer (10.62PF, 1.27PiB/30PiB) Nagoya Univ.： FX1(30.72TF, 24TB) HX600(25.6TF, 10TB) M9000(3.84TF, 3TB) Tohoku Univ.： SX-9(29.4TF, 18TB) Express5800 (1.74TF, 3TB)

Tokyo Institute of Technology：

TSUBAME2.0 (0.24PF/2.4PF, 10TB/ 100TB) RENKEI-VPE : VM Hosting Osaka Univ.： SX-9 (16TF, 10TB) SX-8R (5.3TF, 3.3TB) PCCluster (6.1TF, 2.0TB) Kyoto Univ. XE6 (300.8 TF, 59 TB) GreenBlade8000(242.5TF, 38TB) 2548X(10.6TF, 24TB) Univ. of Tsukuba： T2K (95.4Tflops, 20TB) HA-PACS (802Tflops, 34.3TB) FIRST (36.1TFlops, 1.6TB) Hokkaido Univ.： SR16000/M1(51.6TF/172TF, 6.6TB/ 22TB) BS2000 (5.76TF/44TF, 1.92TB/14TB) RENKEI-VPE: VM Hosting Kyushu Univ.： FX10 (68.1TF/181.6TF, 9.2TB/24TB) CX400 (44.2TF/510.1TF, 16.4TB/184.5TB) SR16000 L2 (25.3TF, 5.5TB) Univ. of Tokyo： FX10 (1.13PF, 150TB) SR16000/M1(54.9TF, 10.94TB) T2K (75.36TF/140TF, 16TB/31.25TB) EastHubPCCluster(10TF/13TF, 5.71TB/ 8.15TB)

GPU Cluster(CPU 4.5TF, GPU 16.48TF, 1.5TB)

WestHubPCCluster(12.37TF,8.25TB) RENKEI-VPE:VM Hosting

source: M. Hirakawa, AICS

(7)

Hokkaido University

Tohoku University University of Tokyo

University of Tsukuba

Tokyo Institute of Technology Nagoya University

Kyushu University

Osaka University Kyoto University

AICS, RIKEN

•  12 PB+ storage

•  10 PB+ storage

HPCI WEST HUB HPCI EAST HUB

Gfarm2 is used as the global shared file

system

Storage

(8)

Network (SINET4)

SINET4: Science Information NETwork 4

(9)

user user user user IX （Tokyo) resource provider IX （Osaka) AICS LAN user

compt. resource storage

univerisity university commercial network non-comercial network CA portal university user

resource provider

university

user

QoS

VPN

SINET4 (cont’d)

n

 

connection to 700+ academic sites

n

 

IX for commercial networks

Ø 134（30Gbps） in Tokyo

Ø 22（11Gbps） in Osaka

9

n

 

80Gbps backbone between

Tokyo and Osaka

(10)

Cloud Service

n

 

VM hosting

Ø repository for research results

Ø pre/post processing

Ø testbed for prototype system software

Kento Aida, National Institute of Informatics 10

(11)

11

(12)

Overview of Authentication System

n

 

access to web portals: Shibboleth

Ø  management of certificates, user support, cloud service

n

 

access to remote computers: GSI

Ø  login to remote computers, access to shared storage

n

 

bridge between shibboleth and GSI: web portal

12

user _portal

IdP, HPCI account pass word

• login to remote computers

• access to shared storage

single sign-‐on

% gsi-ssh host.univ.ac.jp

(1) sign-on to the portal (cert. issuing system) (2) generate a proxy certificate and download

the proxy certificate

(3) ssh login to remote computers

ü no need to give local account name and password

(13)

Architecture

cert. management system CA system (Shib. SP) portal (Shib. SP) proxy cert. repository Shib. DS Shib. IdP browser GSI-SSH client NII

supercomputer centers, RIKEN

SINET 4 ü apply user cert. ü single sigh-on ü login to resources 13 account DB GSI-SSH server portal (Shib. SP) proxy cert. repository supercomputer centers, RIKEN cert. repository

(14)

Architecture (cont’d)

Shib. DS Shib. IdP NII SINET 4 14 browser GSI-SSH client ü apply user cert. ü single sigh-on ü login to resources supercomputer centers, RIKEN cert. management system CA system (Shib. SP) portal (Shib. SP) proxy cert. repository Shib. DS Shib. IdP account DB GSI-SSH server portal (Shib. SP) proxy cert. repository cert. repository

(15)

Software

role system software

Certificate Authority CA system NAREGI-CA

certificate management custom software certificate repository MyProxy

ID federation Shibboleth

Portal

（NII，supercomputer centers）

portal (cert. issuing system) custom software Proxy certificate repository MyProxy

ID federation Shibboleth Identity Provider （supercomputer centers, AICS） ID federation Shibboleth Resource Provider （supercomputer centers, AICS） middleware to access resources GSI-SSH Gfarm

(16)

Summary and Future Plan

n

 

Summary

Ø This talk presents a design of HPCI focusing on the authentication

mechanism.

Ø HPCI started production level operation in Sep. 2012.

n

 

Issues

Ø interoperation with oversea infrastructure

ü review of the operation in HPCI CA to obtain approval of International Grid Trust Federation (IGTF)

Ø federation with other authentication system

ü discussion about the federation with other web authentication systems, e.g. OpenID

(17)

Kento Aida, National Institute of Informatics 17