SimGrid Cloud Broker: Simulation of Public and Private Clouds

(1)

SimGrid Cloud Broker:

Simulation of Public and Private Clouds

Jonathan Rouzaud-Cornabas CNRS – CC-IN2P3 / LIP (UMR 5668)

(2)

SimGrid Cloud Broker

SimGrid

simulates many different distributed systems: clusters, wide-area and local-area networks, peers over DSL connexions, data centers, etc has models are theoretically and experimentally assessed

is scalable exists for 13 years

Simulation of Clouds

From an user point of view

From a cloud provider point of view Evaluates application(s) running on Clouds Evaluates different policies in Cloud middleware Multi-Clouds (private and public)

(3)

Architecture

(4)

AWS

Amazon Web Services

S3 / EC2 API

All the instance types All the regions of AWS

On-demand and Spot Instances 3 types of storage: local, EBS and S3

Accounting of network, computing and storage resources Models:

Spot instance prices: smart random, file and prediction model Life cycle of VMs

Storage (3 models) Compute

(5)

AWS

Amazon AWS Platforms

(6)

Private Cloud

(7)

Private Cloud

IaaS Computing Simulation

Basic compute API (loosely based on OpenStack) Eucalyptus and OpenStack models (in progress)

(8)

Easy-to-use

Easy-to-use simulator

Easy to use platform description

One XML file to describe all the Clouds

SimGrid users: only few modifications to migrate a SimGrid code to SGCB code

Easy to replace and extend: VM image deployment and VM to PM mapping policies

(9)

Scalability

Studying the scalability of SGCB

Tested on the simulated AWS platform with all the models activated Mid-term goal: 1 millions VMs on 100,000 Hosts

4 types of scenario (1≤N ≤500)

1 _N _instances_m1.large_in_{eu 1}

2 _N _{instances of each instance types in}_{eu 1} 3 _N _instances_m1.large_{in all regions}

4 _N _{instances of each instance types in all regions}

(10)

Scalability

Scalability of SGCB

0 500 1000 1500 0 10000 20000 30000 NBVM SimTime

(11)

Scalability 0 300 600 900 1200 2.5 5.0 7.5 10.0 AMISIZE Dur ation State Boot Deploy Running Shutdown

(12)

Scalability Simple bag of tasks

Simple Bag Of Tasks scenario

BoT Characteristics 1,000 Tasks Input data: 1GB Output data: 500MB

Storage Input and/or Output: Local, EBS ou S3 1 master (CeS): m2.2xlarge

(13)

Scalability Simple bag of tasks od_ebs_ebs od_ebs_local od_ebs_s3 od_local_ebs od_local_local od_local_s3 od_s3_ebs od_s3_local od_s3_s3 si_ebs_ebs si_ebs_local si_ebs_s3 si_local_ebs si_local_local si_local_s3 si_s3_ebs si_s3_local si_s3_s3

0e+00 5e+05 1e+06 Duration Scenar io State_Start RequestVM SendTasks ReceiveResults TerminateVM End

(14)

Scalability Simple bag of tasks od_ebs_ebs od_ebs_local od_ebs_s3 od_local_ebs od_local_local od_local_s3 od_s3_ebs od_s3_local od_s3_s3 si_ebs_ebs si_ebs_local si_ebs_s3 si_local_ebs si_local_local si_local_s3 si_s3_ebs si_s3_local si_s3_s3 0 500 1000 1500 Duration Scenar io StateSendInput ReceiveInput WriteInput ExecuteTask ReadOutput SendOutput ReceiveOutput

(15)

Scalability Simple bag of tasks EBS S3 0 500 1000 GBMonth T ype Scenario od_ebs_ebs od_ebs_local od_ebs_s3 od_local_ebs od_local_local od_local_s3 od_s3_ebs od_s3_local od_s3_s3 si_ebs_ebs si_ebs_local si_ebs_s3 si_local_ebs si_local_local si_local_s3 si_s3_ebs si_s3_local si_s3_s3

(16)

Internet Network VM Intranet Network VM Network S3

0e+00 2e+11 4e+11 6e+11 GB T ype Scenario od_ebs_ebs od_ebs_local od_ebs_s3 od_local_ebs od_local_local od_local_s3 od_s3_ebs od_s3_local od_s3_s3 si_ebs_ebs si_ebs_local si_ebs_s3 si_local_ebs si_local_local si_local_s3 si_s3_ebs si_s3_local si_s3_s3

(17)

Scalability Simple bag of tasks od_ebs_ebs od_ebs_local od_ebs_s3 od_local_ebs od_local_local od_local_s3 od_s3_ebs od_s3_local od_s3_s3 si_ebs_ebs si_ebs_local si_ebs_s3 si_local_ebs si_local_local si_local_s3 si_s3_ebs si_s3_local si_s3_s3 0 500 1000 1500 2000 Price Scenar io Type_EBS Network S3 Network VM S3 VM OD VM SI

(18)

m1.small m1.large m1.xlarge m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge

0.0 0.5 1.0 1.5 0 100 200 300 0 100 200 300 0 100 200 300 0 100 200 300 0 100 200 300 0 100 200 300 0 100 200 300 0 100 200 300 Hour Pr ice Region brasil_1 asia_2 asia_1 eu_1 usa_east_1 usa_west_1 usa_west_2

(19)

Use case

SGCB usage in the Inria Avalon team

Study different resource reservation and task allocation algorithms for Bag Of Tasks on Clouds

Multi-regions applications and inter-region migration on AWS (Jose Luis Lucas – UCM)

Scientific Workflows on Clouds (Dao Van Toan – Master internship) Applications composed of multiple VMs with security requirements (Arnaud Lefray – Doctorant)

Component based applications on federated PaaS (FP7 PaaSage)

(20)

Future work

Validation of the simulation results with experiment on AWS Finer grain models for AWS

Running the same experimentation on a private Cloud

Study the different between OpenStack and OpenNebula on same hardware and applications

Models for private Clouds

Integrate works around live-migration Multi-core for VMs models in SimGrid Increase the scalability