High Productivity Computing With Windows

(1)

Windows HPC Server 2008

High Productivity Computing

With Windows

Justin Alderson

16-April-2009

(2)

(3)

‘The purpose of

computing is insight

not numbers.’

Richard Hamming

(4)

Current Issues

HPC and IT data centers merging: isolated cluster

management

Developers can’t easily program for parallelism

Users don’t have broad access to the increase in processing

cores and data

How can Microsoft help?

Well positioned to mainstream integration of application

parallelism

Have already begun to enable parallelism broadly to the

developer community

Can expand the value of HPC by integrating productivity and

management tools

Microsoft Investments in HPC

Comprehensive software portfolio: Client, Server,

Management, Development, and Collaboration

Dedicated teams focused on Cluster Computing

Unified Parallel development through the Parallel Computing

Initiative

Partnerships with the Technical Computing Institutes

(5)

“

Provide the platform, tools and broad ecosystem to reduce the complexity of HPC by

making parallelism more accessible to address future computational needs.”

Microsoft’s Vision for HPC

Ease deployment for

larger scale clusters

Simplify management for

clusters of all scale

Integrate with

existing infrastructure

Enable non-technical users to

harness the power of HPC

Address emerging

cross-industry

computation trends

Address needs of traditional

supercomputing

Increase number of parallel

applications and codes

Offer choice of parallel

development tools,

languages and libraries

Drive larger universe of

(6)

Today’s HPC Environment

Public Sector

(7)

(8)

Spring 2008, NCSA, #23 9472 cores, 68.5 TF, 77.7% Fall 2007, Microsoft, #116 2048 cores, 11.8 TF, 77.1% Spring 2007, Microsoft, #106 2048 cores, 9 TF, 58.8% Spring 2006, NCSA, #130 896 cores, 4.1 TF Spring 2008, Umea, #40 5376 cores, 46 TF, 85.5% 30% efficiency 30% efficiency improvement improvement

Windows HPC Server 2008

Windows Compute Cluster 2003

Winter 2005, Microsoft 4 procs, 9.46 GFlops Spring 2008, Aachen, #100

2096 cores, 18.8 TF, 76.5%

Rank Organisation Rmax Cores

10 Shangai Supercomputer Center

180.6 30720 37 NCSA 68.5 9600 59 UMEA University, Sweden 46 5376

(9)

Microsoft’s in-house cluster

Location Tukwila, WA

Hardware – Machines 256 Dual-CPU, quad-core Intel 5320 Clovertown 1.86GHz processors and 8GB RAM

Hardware – Networking Private & Public: Broadcom GigE MPI: Cisco Infiniband SDR incl/ 34 IB switches in leaf&node configuration Number of Compute Nodes 256

Total Number of Cores 2048

Total Memory 2+ TB of RAM Particulars of for current Linpack

Runs

Best Linpack rating so far 11.75 TeraFLOPS Best cluster efficiency so far 77.1%

For Comparison…

Linpack rating from June2007 Top500 run (#106) on the same hardware

8.99 TeraFLOPS

Cluster efficiency from June2007 Top500 run (#106) on the same hardware

59%

Typical Top500 efficiency for Clovertown motherboards w/ IB regardless of Operating System

65-77%

(2 instances of 79%)

30% improvement in

efficiency on the same

hardware between

versions

Less

_tha

n 2 h

ours

to d

eplo

y 25

6 no

des

#106 (Jun 07)

#116 (Nov 07)

#303 (Jun 08)

(10)

(11)

“The Umeå cluster with

Windows HPC Server 2008 performed well enough to

become the

fastest

academic system in Sweden

.

We are very happy with that result.”

-- Bo Kågström, Professor and Director,

High Performance Computing Center North Umeå University

“Ferrari is always looking for the most advanced technological solutions and, of course, the same applies for software and engineering. To achieve industry leading power-to-weight ratios, reduction in gear change times, and revolutionary aerodynamics,

we can rely on

Windows HPC Server 2008

.. It provides

a fast, familiar, high performance

computing platform

for our users, engineers and administrators.”

-- Antonio Calabrese, Responsabile Sistemi Informativi (Head of Information Systems), Ferrari

Customers

“Financial analysts in Europe mainly use Windows systems. As such, the deployment of a

Windows HPC Server 2008 cluster renders our HPC services extreme

ly

attractive

to a large potential user base.”

-- Dr. M. Rosati, Manager of the Computational Materials Science and Finance Group, CASPUR

“

We are really impressed with many of the new features of Windows

HPC

Server 2008.

Microsoft is a pretty young player in the HPC market, but

this is already

a very solid product.

”

-- Christian Terboven, Project Lead for HPC on Windows, Center for Computing and Communication, RWTH Aachen University

(12)

Key Storage Existing Cluster Infrastructure UNIX/Linux System Business Intelligence SQL Server Analysis/ Reporting SQL Server Integration Services Storage Administration Partner Microsoft System Center Configuration Manager Windows Server Update Services Software Protection Services

Windows® HPC Server 2008

Job Scheduler w/ Failover

Compute Nodes Storage SQL Structured Storage Windows Storage Server with DFS Parallel/Clustered Storage Node Manager Applications: WCF, C#, C++, Fortran

New TCP/IP MPI w/Network Direct

HPC Server 2008 HPC Profile 3rdParty Systems Management Utilities Clients/Job Submission Development Tools System Center Operations Manager Windows® HPC Server 2008 Administration Console:

System, Scheduling, Networking, Imaging, Diagnostics Windows Powershell SharePoint Batch Applications CCS Job Console CCS Scripts Visual Studio: C#, C++, WCF, OpenMP, MPI, MPI.NET MPI Debugging Trace Analysis Profiling MPI Tracing Fortran Numerical Libraries WCF Applications Windows Workflow Foundation Excel System Center Data Protection Manager

(13)

• Available Now

– Development and Parallel debugging in Visual Studio

– 3rd_{party Compilers, Debuggers, Runtimes etc.. available}

• Emerging Technologies – Parallel Framework

– LINQ/PLINQ – natural OO language for SQL queries in .NET

– C# Futures – way to explicitly make loops parallel

• For the future: Parallel Computing Initiative (PCI)

– Triple investment with a new engineering team

– Focused on common tools for developing multi-core codes from desktops to clusters