Cloud Computing mit mathematischen Anwendungen

(1)

Cloud Computing mit

mathematischen Anwendungen

Vorlesung SoSe 2009

Dr. Marcel Kunze

Karlsruhe Institute of Technology (KIT) Steinbuch Centre for Computing (SCC)

(2)

Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze

2

Agenda Cloud Computing

1.

Einleitung

Was ist Cloud Computing? 2.

Grundlagen

Virtualisierung, Web Services,… 3.

Cloud Architekturen

Infrastruktur, Plattform, Anwendung 4.

Cloud Services

Amazon Web Services, Google App Engine 5.

Aufbau einer Cloud

OpenCirrus Projekt, Eucalyptus, Hadoop 6.

Cloud Algorithmen

MapReduce, Optimierungsverfahren, …

Praktische Übungen und Anwendungen

Vorlesung im Web:

http://www.mathematik.uni-karlsruhe.de/mitglieder/lehre/cloud2009s/

(3)

Commercial Cloud Offerings (Small Excerpt)

Problem: Commercial offerings are proprietary and usually not open for cloud systems research and development

(4)

4

Cloud Systems Research

Simple, transparent, controllable cloud computing infrastructure

What types of interfaces are appropriate for clouds?

How should cloud networks be constructed/managed? How are security concerns addressed in “the cloud”? How are various workloads most efficiently transferred? What types of applications can run in clouds?

What types of service level agreements are appropriate/possible?

Research requirements

Perform experiments also on a low system level Flexible cloud computing framework

(5)

Cloud Computing

Cloud computing R&D: OpenCirrus

TM

project

(6)

(7)

OpenCirrus™ Cloud Computing Research Testbed

http://opencirrus.org

An open, internet-scale global testbed for

cloud computing research

Data center management & cloud services Systems level research

Application level research

Structure: a loose federation

Sponsors: HP Labs, Intel Research, Yahoo! Partners: UIUC, Singapore IDA, KIT, NSF Members: System and application

development

(8)

8

Where are the OpenCirrus™ sites?

Six sites initially:

Sites distributed world-wide: HP Research, Yahoo!, UIUC, Intel Research Pittsburgh, KIT, Singapore IDA

1000-4000 processor cores per site New CMU site coming in 2009

HP Yahoo (sf) UIUC Intel (pgh) KIT (de) IDA (sg) CMU (coming in ‘09)

(9)

HC3: OpenCirrus Installation at KIT (July)

2656 Nehalem Cores / 332 Servers / 10 TB Memory

Infiniband QDR Interconnect

192 TB Lustre Filesystem

MSA2000fc

(10)

10

OpenCirrus Global Services

Federation of sites needs common global services

Global SignOn

Single authentication at all sites Based on distribution of ssh keys

Global Monitoring

Based on Ganglia (http://ganglia.info/)

Scalable distributed monitoring system Supports federations of clusters

OpenCirrus service at http://12.161.200.222/ganglia

Global Data

TBD

(11)

Cloud Architecture

(12)

12

OpenCirrus

TM

Blueprint

IT infrastructure layer (Physical Resource Sets) Cloud infrastructure services

Cloud application services

Virtual Resource Sets

(13)

Physical Resource Sets (PRS)

PRS service goals

Provide mini-datacenters to researchers Isolate experiments from each other

Stable base for other research

PRS service approach

Allocate sets of physical co-located nodes, isolated inside VLANs. Leverage existing software (e.g. Utah Emulab, HP OpsWare)

Start simple, add features as we go

Base to implement virtual resource sets

Hardware as a Service (HaaS)

(14)

14

Virtual Resource Sets (VRS)

Basic idea: Abstract from physical resource by introduction of a

virtualization layer

Concept applies to all IT aspects: CPU, storage, networks and

applications, …

Main advantages

Implement IT services exactly fitting customer‘s varying need Deploy IT services on demand

Automated resource management Easily guarantee service levels Live migration of services

Reduce both: CapEx and OpEx

Infrastructure as a Service (IaaS)

Implement Compute and Storage services

(15)

Eucalyptus: A potential VRS Layer

http://eucalyptus.cs.ucsb.edu

EUCALYPTUS - Elastic Utility Computing Architecture for

Linking Your Programs To Useful Systems.

Open-Source software infrastructure for implementing cloud computing on clusters from UC Santa Barbara.

Impelements Infrastructure as a Service (IaaS) – gives the user the ability to run and control entire virtual machine instances (Xen) deployed across a variety of physical resources.

Interface compatible with Amazon EC2 and S3.

Potential to interact with the same tools, known to work with Amazon EC2 and S3.

Linux image hosting ala Amazon Web Services.

Eucalyptus is an important step for archiving an open and

established cloud computing infrastructure standard.

(16)

16

Eucalyptus: A potential VRS Layer

http://eucalyptus.cs.ucsb.edu

Source: R.Wolski

Schedules the distribution of virtual machines to the NC. Collects (free) resource

information.

Collects resource information from the

CC. Operates like a meta-scheduler in the

cloud.

Runs on every node in the cloud. Xen-Hypervisor running. Provides Information about free resources to

(17)

R&D Cloud Installation at SCC/KIT

Eucalyptus R&D Cloud I

2x IBM Blade LS20

Dual Core Opteron (2,4GHz) 4GB RAM

2x IBM Blade HS21

Dual Core Xeon (2,33GHz) 16GB RAM

Eucalyptus R&D Cloud II

5x HP Blade ProLiant BL2x220c Each Blade: 2 Server Nodes

2x Intel Quad-Core Xeon (2,33GHz) 16GB RAM

(18)

18

Comparing Storage Performance between S3 and Eucalyptus

Sequential Output

Per-Character: file is written usindputc() Block: file is written usingwrite()

Rewrite: read()and write()

Sequential Input

Per-Character: file is read usindgetc() Blockwise: file is written usingread()

WOW!

(19)

Realistic values…

The RAM of the Eucalyptus Node Controller was reduced to overcome memory caching. The storage performance of Eucalyptus depends on the available storage sub-system.

(20)

20

Performance of Random Seeks and File Creation

The RAM of the Eucalyptus Node Controller was reduced to overcome memory caching. The storage performance of Eucalyptus depends on the available storage sub-system.

(21)

How is OpenCirrus™ different from other testbeds?

OpenCirrus

TM

supports both

system- and app-level

research

n/a at Google/IBM and EC2/S3

OpenCirrusTM _researchers

will have complete access to the underlying hardware and software platform.

OpenCirrusTM _{allows Intel}

platform features that support cloud computing (e.g. DCMI, NM) to be

exposed, and exploited.

Virtual machines Hadoop Map-Reduce apps Google/IBM cluster Virtual or physical machines Cluster mgmt software

Open Cirrus cluster Hadoop Cloud apps and services Map-Reduce apps Cannot be modified by users Can be modified by users Can be modified by users

(22)

22

Summary

OpenCirrus

TM

offers interesting R&D opportunities

Cloud systems development Cloud application development Accepting research proposals

OpenCirrus

TM

workshop at HP Palo Alto on June 8/9

https://opencirrus.org/content/agenda-open-cirrus-summit-palo-alto-june-8-9-2009

(23)

Karlsruhe Institute of Technology

Thank you for your attention.