Cloud Federations in Contrail

Cloud Federations in

Contrail

Emanuele Carlini1,3_{, Massimo Coppola}1_{, Patrizio Dazzi}1_,

Laura Ricci1,2_{, GiacomoRighetti}1,2_"

1 - CNR - ISTI, Pisa, Italy"

2 - University of Pisa, C.S. Dept" 3 - IMT – Lucca, Italy "

contrail is co-funded by the EC 7th Framework Programme under Grant

Agreement nr. 257438

Overview

•  What is Contrail

–  Approach to Cloud Federations

•  SLA and QoS in Contrail

–  Security issues

–  Federation as middle-man

•  Architecture of the Federation support

•  Application description model

•  Current issues and Future work

Contrail

•  FP7 IP, 11 partners from 6 countries

•  Academic INRIA, CNR, ZIB, VUA, STFC

•  Industrial HP-Italy, Tiscali, XLAB, EBM websourcing, Constellation, Genias

•  Design, implement, validate and promote an open source software stack for

Cloud Computing Federations

–  Comprehensive Cloud platform integrating a full IaaS and PaaS offer

–  Integrate resources from other Clouds with private infrastructures

–  Provide trusted Clouds by advanced SLA management

–  Break the current customer lock-in situation by allowing live application migration across Clouds

Contrail project

Text

SP5. Use cases and exploitation

Applications and Use Cases

12

Testbeds 13

SP3. Platform as a Service

High level services 8

Runtime environments 9

SP1. Cloud federation management

2 3

SP2. Virtual Infrastructure layer

Virtual Infrastructure Network (VIN)

4 _{Computational Resource}

Management for Virtual Cluster Platforms (VCP)

5

Global Autonomous File System (GAFS)

6 7 SP4. System Engineering System Architecture 10

Integration, testing and release management 11 Project management 1 Demonstrators 15

Contrail Iaas Federation

A Contrail Federation integrates in a common platform multiple Clouds, of public and private kind.

User identities, data, and resources are interoperable within the federation, thanks to

•  common supports for authentication and authorization •  common mechanisms for policy definition, monitoring,

and enforcing

of all aspects of QoS : SLA, QoP, etc.

Federation WP Objectives

•  Develop a Federation support that integrates and

actively coordinates SLA management provided by single Cloud providers

•  Do not disrupt provider’s business model

–  Cloud administration is not Federation management

•  Allow exploiting a Federation as a single Cloud

–  Cloudbursting to and from the Federation

•  Federation Support must be scalable

Contrail Overall Architecture

•  Abstract viewpoint

•  Full providers and dedicated providers

•  Resource providers based on Open Nebula

•  Interfaces to Public Clouds

Resource Provider Federation API Resource Provider Storage Provider Public Cloud Storage Provider Network Provider

Contrail Overall Architecture

•  Coordinate deployment and management

of application on multiple clouds

•  Each app can exploit more providers

Resource Provider Federation API Resource Provider Storage Provider Public Cloud Storage Provider Network Provider A A A A Application

Distributed Architecture

•  Abstract API is replicated onto each Federation

access point

•  FAP act as brokers, but share a common view

–  Security, provider status, user actions

•  FAP not restricted to “local” provider

CGWS 2011 - Cloud Federations in Contrail - M.Coppola et al.

F

F

F

F

•  Policies and auth/authZ are common

•  Contention issues

•  Final resource allocation is on providers

•  Shared info helps management

•  AP either hosted by provider, or on independent HW

Contrail approach to SLA

–  Reuse SLA@SOI framework as a starting point

•  Integration with Contrail internal interfaces and components

•  Integration with domain-specific reasoning/monitoring plugins

–  Extend SLA@SOI with:

•  Federation support •  QoP support

•  Integration of external providers •  Reputation model for providers •  Cost-based QoS enforcement

Holistic approach to QoS

•  Extend the set of characteristics to be

measured on the platform

•  Protection

–  Type of security mechanisms which are in place

•  Auth. Protocols, Encryption mechanisms, Isolation

•  Privacy

–  Guarantees offered by storage holder, network infrastructure

•  Geo-localization

–  Can have deep legal implications

•  More in the future

SLA Requirement Summary

•  Required QoS terms:

–  availability for VMs and data

–  network bandwidth –  web response time, ...

•  Required QoP terms:

–  storage zeroing upon release

–  encryption algorithms –  access control policies –  authentication and

authorization

requirements for services –  logging policies –  data confinement mechanisms –  data location –  recoverability •  Other requirements:

•  penalties for SLA violations •  access control on map/

reduce intermediate data •  pub/sub for monitoring •  keep history of monitored

data

•  accounting and monitoring data isolation per user

•  support custom monitoring metrics and events

•  user to be notified of SLA violations

•  automatic negotiation and automatic monitoring setup •  multi-turn negotiation and

re-negotiation

Different ways of measuring services

•  Observable properties

–  User’s perspective! Provider has direct control –  Performance of a VM can be directly verified –  You cannot check everything

•  Zeroing unallocated space

•  Physical location and security of machines

•  Enforceable properties

–  Provider perspective

–  Beyond best-effort and static configuration –  The provider is also able to take measures to

correct deviations

User trust and Providers

•  As the SLA is signed, the user should be able

to trust resources from the platform

–  But not all Providers may sport same reliability

•  For non observable terms, trust is

organisational

–  User cannot efficiently detect / track violations

•  Providers know all resource state

–  Can optimize usage, but what if too much?

Federation as 3

_party

•  Federation sits in the middle

–  Hierarchy of SLAs

–  Hierarchical SLA negotiation (cfr. SLA@SOI)

•  Aim

–  serve the user, efficiently exploit providers

•  Gains

–  overall efficiency, features, intermediation

•  Means

–  Info from all users and applications –  can afford to test explicitly

–  can trigger corrective actions over several providers

Architecture of a Federation Access Point

GAFS driver VIN driver

CLI External Cloud adapters Core layer Adapters layer VEP driver State SLA Coordination SLA Negotiation SLA Organizer Provider Watcher SLA Management SLA Template Repository Federation Runtime Manager

Mapping _{Attribute Authority}

Policy Administration Point Policy Decision Point

Authentication Security SLA Management External Provider Image Registry Image Manager

CGWS 2011 - Cloud Federations in Contrail - M.Coppola et al. Contrail Provider Interface layer _HTTP REST Federation support User Identity

GAFS driver VIN driver

CLI External Cloud adapters Core layer Adapters layer VEP driver State SLA Coordination SLA Negotiation SLA Organizer Provider Watcher SLA Management SLA Template Repository Federation Runtime Manager

Mapping _{Attribute Authority} Policy Administration Point Policy Decision Point

Authentication Security SLA Management External Provider Image Registry Image Manager Interface Layer Core Layer Adapter Layer State

SLA mng. Runtime Security.

Provider Reputation

•  Management information

–  Available resources per kind –  Features granted

–  Amount of users/apps ongoing

–  State of SLAs controlled by the federation

–  Static level of “trust” given from federation to the provider

•  Past History

–  History of SLA violation per user / type of app –  Average level of satisfaction of SLA

Planning for SLAs

•  Choose the best provider(s) and map the

application on the virtual resources

provided

•  Beside constraints, multiple criteria choice

–  Many user criteria

–  Federation has its own goals

•  balance user satisfaction

•  balance provider satisfaction

•  How do you choose the resources?

•  What if one provider is not enough?

Application and SLA splitting

•  Application deployment on multiple

providers : a federation is more than the sum

of its providers

–  Type and amount of resources needed –  Sudden elasticity

–  Peculiar resource dislocation

•  Tough issue

–  Multi-criteria and problem size

•  Both at SLA negotiation and at run-time

–  Matching application structure and SLA

–  Identifying suitable set of providers and mapping

Matching SLA with VMs at IaaS level

•  Application description formats for Clouds

describe in detail

–  VM images

–  How to deploy them

•  A detailed SLA can provide key information

in order to to split the app

–  We need to add structure to the application –  And link it to SLA requirements

•  Also, stick with standards…

Matching SLA@SOI and OVF

•  Exploit a combination of two properties

•  In SLA@SOI, SLA parts can reference

resources contained in the application

description

•  In OVF, VMs can be arranged in

VirtualSystemCollections

–  Used to represent data-center like structures, can be useful to describe applications

–  Allows us to build a high-level Task Interaction Graphs of the application

Adding Extra Collections

CGWS 2011 - Cloud Federations in Contrail - M.Coppola et al.

•  Additional VirtualSystemCollections allow to

place constraints on group of machines

–  Ordinary QoS terms

–  Aggregate QoS terms (overall memory, elasticity) –  Reciprocal QoS terms (group comm. Bisection

Bandwidth)

•  No deep restructuring

Benefits of adding structure in SLAs

•  Increased granularity of constraints

–  Delegate parts of SLA to provider

–  Independent VMs on separate providers

•  Finding resource mappings is a multi-criteria problem

–  Size reduction (from VMs to Groups)

–  Eases developing heuristics (solution space pruning) –  Makes splitting a group an explicit choice

•  Known parallel structures can be represented

–  Widespread PaaS services (ConPaaS)

•  workflows, map&reduce, bag of tasks…

–  Components, skeletons

–  More sources of heuristics for mapping and SLA management in federations

Conclusions

•  Contrail Federation goals and architecture

–  Still in early implementation

–  Relying on standards and Open source

–  Tight integration with SLA mng. and Security –  Leverage actual service/resource providers

•  Distributed set of coordinated AP

–  Global security & trust –  Scalability

–  Potential gain from multi-provider applications

•  Splitting applications

Future Work

•  Implementation ongoing

–  Security and state synchronization

–  Extension and integration w. SLA@SOI framework –  Integration with Contrail providers

•  VCP (OpenNebula), GAFS, VIN

•  Research ongoing on

–  exploiting vertical integration with PaaS

–  splitting and mapping heuristics to deal with multiple providers

–  defining and exploiting provider reputation –  express user as well as federation optimality

Thanks for you attention

CGWS 2011 - Cloud Federations in Contrail - M.Coppola et al.

Funded under: FP7 (Seventh Framework Programme)

"

"

Project reference: FP7-IST-257438

"

"

EU contribution: 8,3 million euro

"

Execution: From 2010-10-01 till 2013-09-30

"

"

Contract type: Collaborative project (generic)"

contrail is co-funded by the " EC 7th Framework Programme"