• No results found

Cloud Computing and Content Delivery Network use within Earth Observation Ground Segments: experiences and lessons learnt

N/A
N/A
Protected

Academic year: 2021

Share "Cloud Computing and Content Delivery Network use within Earth Observation Ground Segments: experiences and lessons learnt"

Copied!
17
0
0

Loading.... (view fulltext now)

Full text

(1)

J.Farres – EOP-GS ESRIN

6/6/2012

Cloud Computing and Content Delivery Network

use within Earth Observation Ground

(2)

Agenda

1.

Introduction

2.

ESA Experiences

1.

EO Dissemination on Level 3

2.

EO Re-processing on Amazon

3.

Dissemination and Processing on Hetzner

4.

Exploitation platform with Helix Nebula

(3)

Introduction

Processing bursting Dissemination peaks

ICT Costs savings

Collaboration platform Cloud Computing IaaS SaaS Hosting (VPS, Rental) CDN PaaS

A model for enabling convenient, on-demand network

access to a shared pool of configurable computing

resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and

released with minimal management effort or service provider interaction [NIST]

(4)

Agenda

1.

Introduction

2.

ESA Experiences

1.

EO Dissemination on Level 3

2.

EO Re-processing on Amazon

3.

Dissemination and Processing on Hetzner

4.

Exploitation platform with Helix Nebula

(5)

Case 1: EO Dissemination on Level 3 (1)

Purpose

• Allow for very performing dissemination of EO products at the event

of natural disasters.

Project / Service

• Timeframe: 2008-2012

• Provider: Level 3, off-the-shelf services

• Data: 6-7TB, SAR, GOCE

• System: Portal / EOLI + EO products secured by ESA login

(6)

Pros

Cons

Case 1: EO Dissemination on Level 3 (2)

Pros

1. Excellent dissemination

performance

2. from US and Europe

3. Good integration with ESA user

authentification + EOLI

4. Very good monitoring and

reporting

5. allowing to detect and stop

abuse

Cons

1.  High Price

2.  Price based on bytes stored and

bytes downloaded, hence penalizes large EO products

(7)

Case 2: EO Re-processing on Amazon(1)

Purpose

• Fast re-processing of large EO products collections for CalVal

purposes.

Project / Service

• Timeframe: 2009 and 2011

• Provider: Amazon, EC2, S3

• Data: ERS SAR Wave, MIPAS (30,000 products)

• System: 200 Virtual Servers

configured as Working Nodes to an ESA grid.

(8)

Pros

Cons

Case 2: EO Re-processing on Amazon(2)

Pros

1. Excellent processing scalability

2. Efficient bulk-in/out data service

(via HD)

3. “The faster the cheaper” as it

reduces storing costs

Cons

1.  Complex and changing pricing,

e.g. periods with cheaper hosts and free data upload.

(9)

Case 3: Dissemination and Processing on

Hetzner (1)

Purpose

• Couple large processing capability with fast dissemination for low

cost.

Project / Service

• Timeframe: 2011

• Provider: Hetzner (Hosting services)

• Data: 60TB

• System: 1 Head: Catalogue, Processor Register, Cluster Head

n Nodes: Data Dissemination point, Cluster Node Designed as back-end for multiple front-ends

• Usage: GeoHazards SuperSites (38,000 SAR images and 3,000 users)

(10)

Pros

Cons

Case 2: Dissemination and Processing on

Hetzner (2)

Pros

1. Good archive scalability (chunks

of 8TB)

2. Much cheaper than Level 3 or

Amazon services

3. Synergy of processing

-dissemination: processing peaks followed by dissemination peaks

4. Storage costs service two

functions: dissemination and

Cons

1.  System scales storage,

dissemination and processing capabilities simultaneously.

2.  Lower service levels than Level

(11)

Page 11

Case 4: Exploitation platform with Helix

Nebula – SSEP (1)

Purpose

• Pilot a collaborative platform for EO exploitation using multisourced

cloud provisioning.

Project / Service

• Timeframe: 2012-2013

• Providers: ATOS, CloudSigma, Interoute, T-Systems

• Data: GeoHazards data (ESA, CNES, DLR, …)

• Processors: ESA, CNR, Gamma, …

• System (TBD): Large capabilities: storage, dissemination &

processing

Platform for deployment of partner data, processors Based on existing GS sw components

(12)

Case 4: Exploitation platform with Helix

Nebula – SSEP (2)

Expectations

• ESA can distribute data dissemination and processing functions over

several cloud providers (multi-sourcing) at best value.

• Other agencies can do the same on the same platform/cloud.

• Users can provision their ICT from the cloud “near” the data/services

provided by ESA and other agencies.

• Users can deploy EO value added services directly on the cloud, i.e.

(13)

Agenda

1.

Introduction

2.

ESA Experiences

1.

EO Dissemination on Level 3

2.

EO Re-processing on Amazon

3.

Dissemination and Processing on Hetzner

4.

Exploitation platform with Helix Nebula

(14)

Lessons Learnt: ICT provisioning

1.  As soon as ICT needs can be predicted and planned, IaaS is

more expensive than other hosting solutions like (VPS, Renting).

2.  Flexibility of Public IaaS is less appealing when internal

resources are pooled, virtualized and managed as an internal cloud.

3.  On the other hand, IaaS services allow to size down internal ICT

resources to the “fixed” need and ensure their maximum

utilization; e.g. using external provisioning for the “variable” need.

(15)

Lessons Learnt: Service Levels

•  Terms & Conditions in Public Clouds express surprising low

commitment.

•  Cloud opportunities can become risks when applied to critical

systems.

⇒  Develop multi-sourcing

⇒  Plan contingency scenarios for services hosted in Public

(16)

Lessons Learnt: Application Areas in EO GS

•  Dissemination and on-demand processing

–  because is very variable (depending on user demand)

•  Secondary archive and re-processing

–  because is limited in time

•  Temporary resources for integration, testing and demonstration

–  because is limited in time

•  System sizing

–  because needs are unknown

(17)

Lessons learnt: Users

•  User expectations on Cloud Computing are very high

–  All data discoverable and accessible online with same performances as consumer services.

–  Availability of long time series of coherent data from different providers.

–  Availability of collaboration platforms and tools where to exchange experiences and information and where data from different providers and even different disciplines can be accessed, combined and exploited.

–  Users will be able to perform processing directly on the cloud using virtual servers.

•  The Cloud provides a well accepted paradigm for pay-per-use

•  User communities adopt social networks patterns: sharing objects,

providing feed-back

References

Related documents

Juniper’s flexible business edge solution is a comprehensive VPN toolkit that offers service providers and large enterprises a feature rich and standard-based network that

Based upon the existing data on functional and numerical responses of the major aquatic protist taxa, we have iden- tified two key avenues for future studies: (A) The

When analysing changes occurring in the milk yield and composition depending on successive lactation it was concluded that the highest amount of obtained milk, calculated FCM and

must be factored in and considered from scratch, resulting in includ- ing Big Data Analytics as a mode of processing personal data collected as part of the informa- tion provided

In order to address this further, the BIC project are examining how the combination of their International Advisory Group and supporting working groups could

In this thesis, I will explore the question: “ How are social interactions between immigrant-hosts and their visiting friends and relatives understood and interpreted by

Both studies highlight the importance of not only listening to children and young people regarding their experiences and views of how services can better address their mental

Choral Director of Treble Ensemble, Chamber Singers, Male Choir, and Choral Accompanist, Downers Grove South High School.. Downers