Cloudian The Storage Evolution to the Cloud.. Cloudian Inc. Pre Sales Engineering

(1)

Cloudian Inc. | Pre Sales Engineering

Cloudian

(2)

Agenda

• Industry Trends

• Cloud Storage

• Evolu4on of Storage Architectures

• Storage Connec4vity redeﬁned – S3

• Cloud Storage Use Cases

(3)

Machine Data Builds

Zettabyte = 1,000,000,000,000,000,000,000 Bytes

Graph based on IDC and UC Berkeley Data Growth Estimates, Source: IDC & CosmoBC.com:

http://techblog.cosmobc.com/2011/08/26/data-storage-infographic/

Zettabyte = 1,000,000,000,000,000,000,000 Bytes

Graph based on IDC and UC Berkeley Data Growth Estimates, Source: IDC & CosmoBC.com:

http://techblog.cosmobc.com/2011/08/26/data-storage-infographic/ PC Internet Mobile Mainframe Machine Petabyte Terabyte Exabyte Zettabyte DATA VOLUME •  Logs, Files •  Multi-Structured •  Block & File •  Perpetual Streaming Interactions Human Files Transactional Data 2011 1.8 Zettabytes in Enterprise Data 2020 35 Zettabytes in Enterprise Data Machine Generated Data

(4)

Market Trends – Data Explosion &

Shrinking Budgets

Workloads, IT Budgets and Trends

(5)

Data Classiﬁca>on

Structured data

• Structured data typically refers to data in a

database

(Oracle, MSFT SQL Server, IBM DB2, SAP Sybase…)

• Data that resides in a fixed field within a record or file

• Includes data in a rela4onal database or spreadsheet

• Easily stored and analyzed

• Data types include numeric, currency, alphabe4c,

name, data , address

Unstructured data

• Generically refers to everything else and typically

refers to data in ﬁle systems

• Describes any corporate data not in a database

• Can include emails, presenta4ons, word processing

documents, video, audio, photos, webpages

• Requires use of seman4c search in order to locate

informa4on

(6)

• Low cost

• Designed for PB scale

• Separa4on from performance 4ers

• A ﬂat namespace of billions of objects

• Geo distribu4on, protec4on, & access

• Analy4cs based on Smart Data

• User meta data as a ﬁrst-class en4ty

• Storage as a revenue source

• Mul4-tenant access & metering

• So_ware Deﬁned via RESTful API

• Amazon S3, Swi_ and CDMI API

A Cloud Storage plaAorm oﬀers …

OBJECT OBJECT OBJECT OBJECT ₆ OBJECT OBJECT OBJECT OBJECT OBJECT

(7)

18/11/15 Cloudian Conﬁden4al 7 •  Disparate Systems •  Mul4ple Administra4on Points •  Mul4ple Filesystem’s need to be managed and replicated •  Single System Across Loca4ons •  Access to “One” Storage System without provisioning •  Web Based Administra4on and delivery

Cloud Storage Changes the Game…..

L.A. BOSTON LONDON L.A. BOSTON LONDON

(8)

8

Tradi>onal

SAN/NAS

Flash

host/array

_Storage

Object

Scale Out /

Peta Byte

Scalability / Big Data

Scale Up /

Tera Byte

Commodity /

Low Price

Price / GB

High Price /

Proprietary

Higher Latency /

Latency

High Performance /

Low Latency

Unlimited by

Scale

Transac4on Volume

Limited by

Controllers

Distributed

Access

Local /

Locking

Durable /

Ac>ve-Ac>ve

Robustness / DR

Limited /

Ac>ve-Readonly

Storage needs are evolving,

squeezing the tradi>onal model

Cloudian Conﬁden4al

(9)

Op>mal Storage Architecture

Workload Centric

9

Tier 1 Storage – Scale out All Flash Array

Op4mised for $/IOPs to support performance workloads Op4mised for $/GBs to support capacity workloads Tier 2 Storage – Scale out Cloud/object Storage

Backup & Archive Big Data Analytics File Collaboration Web Content Storage Storage As a Service

(10)

S3 – Simple Storage Service

10

Amazon launched S3, in the US in March 2006/Europe in November 2007

S3 is storage for the Internet. It’s a simple storage service that oﬀers so_ware

developers a Programmable API using web service interfaces (HTTP). which oﬀers;

• Highly-scalable

• Reliable

• Low-latency

• Commodity level costs

It is designed to be highly ﬂexible: Store any type and amount of data that you want;

read the same piece of data a million 4mes or only for emergency disaster recovery

*2000 ISVs support for S3 today

(11)

S3 Func>onality

11

• Connec4vity

• Access control

• Data management

• Repor4ng & No4ﬁca4ons

• Advanced Func4onality

• Tiering

A modern storage protocol combining

…. that can be used across the internet.

• Encryp4on

• Billing

• Monitoring

• No4ﬁca4ons

• Replica4on

• Time limited access

Standard Storage

Protocols

S3 API

(12)

S3 Concepts - Objects

12

S3 is a simple key, value store designed to store as many objects as you want. You store these objects

in one or more buckets. An object consists of the following:

• Key – The name that you assign to an object. You use the object key to retrieve the object.

• Version ID – Within a bucket, a key and version ID uniquely iden4fy an object. The version ID is a

string that is generated when an object is added to a bucket.

• Value – The content that you are storing. An object value can be any sequence of bytes.

• Metadata – A set of name-value pairs with which you can store informa4on regarding the object.

You can assign metadata, referred to as user-deﬁned metadata, to objects. S3 also assigns system-metadata to these objects, which it uses for managing objects.

• Access Control Informa>on – You can control access to the objects you store with S3, both

resource-based access controls, such as ACL’s and bucket policies, and user-based access control

is supported.

(13)

What is an object? – X-Ray image

13

Image ﬁle stored as jpg/4f..

File system meta data would include;

Image name

Date/4me created

Image owner

Extended meta data for object could include;

Pa4ent Name

Pa4ent Age

Scan details

Why scan taken?

X-Ray Machine used

Machine operator

How incident occurred

…….

(14)

S3 Concepts - Buckets

14

• S3 buckets are a virtual storage resource which acts as a data repository (similar to a LUN or ﬁle

system).

• Objects are stored in buckets

• Bucket names are globally unique, even in a mul4-region setup.

• The account that creates a bucket is the owner and can assign permissions as appropriate

• S3 supports both virtual-hosted–style and path-style URLs to access a bucket.

• In a virtual-hosted–style URL, the bucket name is part of the domain name in the URL. For

example: hap://bucket.s3.cloudian.com

• In a path-style URL, the bucket name is not part of the domain (unless you use a region-speciﬁc

endpoint). For example: hap://s3.cloudian.com/bucket

(15)

S3 Concepts - Keys

15 Each S3 object has data, a key, and metadata. The Object key (or key name) uniquely iden4fies the object in a bucket. When you create an object, you specify the key name, which is a sequence of Unicode characters whose UTF-8 encoding is at most 1024 bytes long. The following are examples of valid object key names: •  4my-organiza4on •  my.great_photos-2014/jan/myvaca4on.jpg •  videos/2014/birthday/video1.wmv The S3 data model is a flat structure: you create a bucket, and the bucket stores objects. There is no hierarchy of sub-buckets or subfolders; however, you can infer logical hierarchy using keyname prefixes and delimiters. S3 applica4ons supports a concept of folders. Suppose the bucket (companybucket) has four objects with the following object keys: •  Development/Projects1.xls •  Finance/statement1.pdf •  Private/taxdocument.pdf •  s3-dg.pdf The console uses the keyname prefixes (Development/, Finance/, and Private/) and delimiter ('/') to present a folder structure as shown:

(16)

What it takes to be 100% S3 Compa>ble

Basic Opera4ons

Moderately Complex Opera4ons Advance Complex Opera4ons

(17)

S3 vs ..

(18)

• Enterprise backup & store

– 

Backup is the classic applica4on for object storage

– 

Tape replacement

•  Object storage is much more reliable than tape •  Object storage is much faster than tape •  Automated oﬀsite, mul4-site storage •  Restore anywhere from anywhere •  Integra4on with “con4nuous data replica4on”

• Enterprise long term archiving

–  More cost eﬀec4ve than typical disk solu4on –  Far greater reliability than tape –  Overcome corrup4on risks –  Greater reliability and far lower cost than duplica4ng on tradi4onal storage 18

Cloud Storage Use Cases

Backup and Archive

(19)

• Enterprise ﬁle sync and share

– 

Any ﬁle from any device from anywhere

– 

File access, backup, synchroniza4on

– 

Enterprise security and control

– 

Distributed users – no geographic boundaries

• Remote oﬃce ﬁle storage

– 

Lower cost remote oﬃce ﬁle shares

– 

Central administra4on and control

– 

Remote oﬃce backup to central data center

– 

Content sharing and collabora4on

19

Cloud Storage Use Cases

File Distribu>on and Sharing

(20)

• So_ware deﬁned storage – runs on commodity hardware

• Integra4on with Amazon, OpenStack and CloudStack

• Storage for Virtual Machines Templates, Snapshots and backups

• Storage infrastructure as a Service

20

Cloud Storage Use Cases

Private Cloud Storage

(21)

Cloud Storage Use Cases

Analy>cs

21

Consumer Activity

(Events, GPS, WiFi) Device Tracking and Logs Social Media Result of Analysis

Cloudian HyperStore INTERNET OF THINGS BIG DATA Fast Eﬃcient Beaer business decisions Event processing plaAorm Benefits ü  Faster time-to-decision

ü  Analyze more – allows for efficient bulk data analysis in place ü  No redundant storage of data

ü  HyperStore scales out with your data – adding nodes for I/O ü  Take advantage of multi-core CPUs – makes sense for

MapReduce

ü  Can feed smarter data to subsequent analytic systems

(22)

HyperStore 5.2 Smart Data Storage

Private, On-Premises Storage: Soiware or Appliances Public Cloud Hybrid Cloud Storage Backup _Media and Content Distribu>on File Services Collabora>on Smart Data Applica>ons 1 Cent per GB a Month | QoS | Data Center Management | Commodity Servers Technology Partner Ecosystem Analy>cs High Value (do more) Low Cost (with less) 22