Cloud computing. Intelligent Services for Energy-Efficient Design and Life Cycle Simulation. as used by the ISES project

Intelligent Services for Energy-Efficient Design and Life Cycle Simulation

Project number: 288819 Call identifier: FP7-ICT-2011-7 Project coordinator: Technische Universität Dresden, Germany | Website: ises.eu-project.info

Matevž Dolenc Munich, Germany, 9.10.2013

BuildingSMART BIM week 2013

Cloud computing

as used by the ISES project

Overview

‣ Definitions

-

Cloud computing

-

Engineering in the cloud

‣ Cloud computing

-

^overview

‣ ISES use of the cloud

-

overview, examples, benefits

‣ ^Summary

Definitions

Cloud computing is a model for enabling ubiquitous, convenient, on-

demand network access to a shared pool of configurable computing

resources (e.g., networks, servers, storage, applications, and

services) that can be rapidly provisioned and released with minimal

management effort or service provider interaction.

P. Mell and T. Grance, The NIST Definition of Cloud Computing: Recommendations of the National Institute of Standards and Technology, http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf

Engineering in the cloud is a combination of cloud services and rich

interactive applications that provides integrated, intelligent, self-service

engineering services over and above engineering- application hosting and

computation—allowing engineers to create, explore, and discover better

designs faster .

P. Williams and S. Cox, (June 2009), Engineering in the Cloud: An Engineering Software + Services Architecture Forged in Turbulent Times, http://msdn.microsoft.com/en-us/architecture/aa894305

But it is more or less the same as ...

But it is more or less the same as ...

Auto nom ic co mp uting

Client-s

erver m

Grid computing

odel

Main fram e com putin g

Minitel

Utilit y co mpu

ting

Peer-to-peer

Service-oriented computing

Thin client

Cloud computing landscape and benefits

‣ Cloud computing benefits

-

Access data/services at any place, from any device and at any time

-

Lower cost of entry

-

Reliability, scalability, security and sustainability

-

Minimize infrastructure risk

-

Reduce run time and response time

-

Increased pace of innovation

Cloud deployment modes

‣ ^Public

-

General purpose, pricing; Ex. AWS, Google Apps, Microsoft Azure, ...

‣ ^Private

-

Security, business related features,

infrastructure cost; Ex. CloudStack, OpenNebula, ...

‣ ^Hybrid

-

Mixed employment of private and public cloud, the best of both worlds

-

2013 Cloud Survey predicts in 5 years 75% hybrid cloud sytems

NorthBridge and GigaOM, 2013 Cloud Survey, http://www.northbridge.com/2013-future-cloud-computing-survey-reveals-business-driving-cloud-adoption-everything-service-era-it

Cloud computing features

‣ On-demand self-service

-

Automatic provisioning of comp. capabilities (e.g. server time, storage, ...) as needed.

‣ Broad network access

-

Use of standard networking mechanisms - thin and thick clients.

‣ Resource pooling

-

Computing resources are pooled in a multi-tenant model - location independence.

‣ Rapid elasticity

-

Capabilities are elastically provisioned and released - a sense of unlimited resources.

‣ Measured service

Cloud computing system types

‣ Software as a Service (SaaS)

-

Use of the provider’s applications running on a cloud infrastructure.

‣ Platform as a Service (PaaS)

-

Deploy consumer-created or acquired

applications created using API supported by the cloud platform provider.

‣ Infrastructure as a Service

(IaaS)

-

Provision processing, storage, networks, and other fundamental computing

resources.

Users

Application Extra

Functions

Application

Application Browser /

Client Application

Pltaform Cloud

Local

Users Developers

Software as a service (SaaS) Attached services Platform as a Service (PaaS)

Cloud computing barriers to adoption

‣ Infrastructure

-

^Network

-

Availability of a service

‣ ^Technology

-

Performance unpredictability

-

Scalability (computing, storage, bandwidth, ...)

-

Bugs in large-scale distributed systems

‣ ^Social

-

Reputation, fate sharing

-

Security ⟶  Trust

‣ ^Business

-

Software licensing

-

Business models

-

Data lock-in

-

Data confidentiality and auditability

‣ Computational analyses

-

Parallel applications (HPC)

-

Parametric studies (HTC)

‣ Building Information Modeling

-

Desktop virtualization

-

Data sharing

-

Visualization

-

Collaboration

Engineering in the cloud: examples

!

! !

ISES cloud requirements

‣ Applications

-

Transparent use of Windows / Linux applications

-

Console application

-

Integration with existing services

‣ Types of analyses

-

Stochastic / Parametric studies

-

Parallel applications (MPI)

‣ Scalability

-

Private cloud extensibility

-

Hybrid cloud

‣ ^Storage

-

Integration with public cloud storage systems

‣ User access

-

^{Web based}

-

^API

ISES cloud architecture and testbed

‣ Hardware specs

-

IntelR XeonR Processor (2.26 GHz), 8 GB RAM

-

152 CPU cores

-

Fiber-Channel disk array – 5 TB

‣ ^Software

-

Ubuntu Server 12.04 LTS

-

OpenStack cloud infrastructure, HTCondor, MPI enabled

-

General purpose software: MATLAB, BLAS, LINPACK, …

-

ISES specific applications (energy related)

Web browser

ISES resources local data

external/remote dataexternal/remote results local results

LOCAL REMOTE

AWS resources

ISES Cloud API

Parametric studies

Parallel MPI applications ISES

appISES appISES

app vel.ises.eu-project.info

ISES cloud architecture and testbed

‣ Hardware specs

-

IntelR XeonR Processor L5520 (2.26 GHz)

-

8MB shared L3 cache 8GB

-

Fiber-Channel disk array – 5 TB

‣ ^Software

-

Ubuntu Server 12.04 LTS

-

OpenStack cloud infrastructure

-

HTCondor, MPI enabled

-

General purpose software: MATLAB, BLAS, LINPACK, …

ISES use of the cloud: analysis

‣ Parametric analysis

-

Generating large parametric studies

-

Parametric studies execute one application many times with different sets of input parameters

-

High-throughput computing environment

-

Example: Granlund Riuska

‣ CFD analysis

-

Computational fluid dynamics, time consuming

-

Parallel applications use traditional computational clusters

-

High-performance computing environment

-

Example: Sofistik CFD

ISES use of the cloud: HTC

‣ Granlund Riuska

-

Efficient and versatile comfort and energy simulation application.

-

Standalone solver - Windows application

-

Running on Ubuntu 12.10 LTS (Wine environment)

‣ Parametric studies

-

Use of independent computer systems

-

Example: Run a parameter sweep of F(x,y,z) for 20 values of x, 10 values of y and 3 values of z (20*10*3 = 600 combinations)

‣ Benchmark (IDA curves)

-

Number of analyses: 280

-

Average analysis time: ~13min Number of

computers

Analysis time

[hours] Speed-up factor

1 61.3 1

5 14.7 4.17

10 7.1 8.63

25 2.5 24.52

ISES use of the cloud: HTC

ISES use of the cloud: HTC

ISES use of the cloud: HTC

ISES use of the cloud: HTC

ISES use of the cloud: HPC

‣ Sofistik CFD

-

Parallel CFD analysis tool for 3D unsteady, incompressible, turbulent, buoyancy-driven flows.

-

Complementary tools (geometrical modeler, mesh generation tools, post-processing tools, etc.)

‣ Parallel processing of CFD solver

-

MPI protocol (MPICH2, OpenMPI)

-

64-bit Linux

-

Synchronization - restricted parallelization

-

Small number of large messages

ISES use of the cloud: HPC

‣ CFD simulations in the context of ISES

-

3D air flow inside a room (indoor climate) - coupled flow-thermal problem

ISES use of the cloud: HPC

‣ CFD simulations in the context of ISES

-

3D wind flow around a tall building (outdoor climate)

Numerical mesh 1034367 elements / 287434 nodes

ISES use of the cloud: HPC

‣ CFD simulations in the context of ISES

-

3D wind flow around a tall building (outdoor climate)

ISES use of the cloud: HPC

‣ CFD simulations in the context of ISES

-

3D wind flow around a block of buildings in a city environment (outdoor climate taking into account building's interference (runtime approx. 6 days - 16 CPUs / realtime: approx. 11 minutes )

Numerical mesh 1233206 tetrahedral elements / 239797 nodes

Munich, Germany, 9.10.2013 | BuildingSMART BIM week 2013 | BIM for energy-efficient buildings Matevž Dolenc & Robert Klinc

ISES use of the cloud: HPC

‣ ^Questions

-

How public cloud MPI clusters compare to traditional scientific HPC clusters?

-

What about use of public cloud virtual computers for HTC?

& ) ' 5 2 % . # 3 ! 0 % 2 & / 2 - ! . # %

.0" -0)

We ran the MPI version of NPB (NPB3.3-MPI) Class B on multiple com-

pute nodes on the EC2 provisioned cluster and on the NCSA cluster. For

the EC2 provisioned cluster, we requested 4 high-CPU extra large instances,

of 8 CPUs each, for each run. On both the EC2 and NCSA cluster compute

nodes, the benchmarks were compiled with the Intel compiler with option

flag

-O3

. For the EC2 MPI runs we used the MPICH2 MPI library (1.0.7),

and for the NCSA MPI runs we used the MVAPICH2 MPI library (0.9.8p2).

All the programs were run with 32 CPUs, except BT and SP, which were run

with 16 CPUs.

Figure 2 shows the run times of the benchmark programs. From the results,

we see approximately 40%–1000% performance degradation in the EC2 runs

compared to the NCSA runs. Greater then 200% performance degradation is

seen in the programs CG, FT, IS, IU, and MG. Surprisingly, even EP (embar-

rassingly parallel), where no message-passing communication is performed

during the computation and only a global reduction is performed at the end,

exhibits approximately 50% performance degradation in the EC2 run.

& ) ' 5 2 %

! . $ /6 % 2 , ! ) $

Walker E., (2008) ,Benchmarking Amazon EC2 for high-performance scientific computing, https://www.usenix.org/legacy/publications/login/2008-10/openpdfs/walker.pdf

Summary

‣ Cloud technology

-

Front-end: Website, mobile, API

-

Back-end: virtualization, scalability, management, accounting, storage, ...

-

New business opportunities for software/service providers, infrastructure providers

-

2013 Cloud Survey (in 5 years 75% - hybrid cloud systems)

-

Bring Your Own Device (BYOD)

‣ Engineering in the cloud

-

BIM, analyses (HPC / HTC), collaboration

‣ Start with the requirements / use-cases / user scenarios

Intelligent Services for Energy-Efficient Design and Life Cycle Simulation

Project number: 288819 Call identifier: FP7-ICT-2011-7 Project coordinator: Technische Universität Dresden, Germany | Website: ises.eu-project.info

The End