Amazon Web Services Building in the Cloud

(1)

(2)

Amazon Web Services

(3)

(4)

AWS Principles

Easy to use Fast Elastic Highly available Secure Pay as you go

(5)

(6)

(7)

(8)

Services Overview

Building block services that allow developers to

innovate and make money

Infrastructure As a Service Amazon Simple Storage Service Amazon Elastic Compute Cloud Amazon Simple Queue Service Amazon SimpleDB

Commerce As a Service Amazon Flexible Payments Service

Fulfillment Web Service

People As a Service Amazon Mechanical Turk

Alexa Web Services Alexa Web Information Service

Alexa Top Sites

Alexa Site Thumbnail

(9)

Amazon S3

Simple Storage Service

Highly scalable data storage in-the-cloud Programmatic access via web services API Simple to get going, simple to use

Highly available and durable Pay-as-you-go:

Storage: $0.15 / GB / month

Data Transfer: starts at $0.17 / GB Requests: nominal charges

(10)

Amazon EC2

Elastic Compute Cloud

Resizable compute capacity in the cloud

Obtain and boot new server instances in minutes

Quickly scale capacity, up or down, as your computing requirements change

Full root access to a blank Linux or Windows machine Simple Web service management interface

(11)

Amazon EC2 Features

Elastic Block Store

Persistent storage, up to 1 TB Backed by S3 Snapshots

Availability Zones

Separate Data Centers Increased Reliability

Elastic IP Addresses

Fixed IP can be quickly attached to an instance Don't need to wait for DNS propagation delays

(12)

(13)

Amazon Machine Images

Public AMIs: Use pre-configured, template AMIs to get up

and running immediately. Choose from Fedora, Ubuntu, Windows configurations and more

Private AMIs: Create an Amazon Machine Image (AMI)

containing your applications, libraries, data and associated configuration settings

Paid AMIs: Set a price for your AMI and let others purchase

(14)

Instance Types

Standard High CPU

Small Large Extra Large

Medium Extra Large

Bits 32 64 64 32 64 RAM 1.7 GB 7.5 GB 15 GB 1.7 GB 7 GB Disk 160 GB 850 GB 1690 GB 350 GB 1690 GB EC2 Compute Units 1 4 8 5 20 I/O Performa nce

Medium High High High High

Firewall Yes Yes Yes Yes Yes

(15)

Amazon SQS

Simple Queue Service

Scalable Messaging - many application threads/servers

can be used without worrying about overloading the messaging system

Unlimited - any number of queues, any number of

messages per queue

Supports reader fail-over -messages become invisible to

other readers when read the first time. Adjustable time before message is again visible to other readers

(16)

Amazon SimpleDB

Simple to use - implements most common / simple

database features behind a set of web service calls

Flexible - doesn't require structured definition of data which

allows you to add new data easily

Scalable - scales easily as your application grows. Simply

add more domains or data within your domains

Fast - quick efficient storage and retrieval of your data

Reliable - uses Amazon's high-availability data centers to

provide strong and consistent performance

Inexpensive - you pay only for the resources you consume,

so you avoid significant up-front costs. Eventual Consistency!

(17)

Using Amazon SimpleDB

item description color material

123 Sweater Blue, Red

456 Dress shirt White, Blue

789 Shoes Black Leather

PUT (item, 123), (description, Sweater), (color, Blue), (color, Red)

PUT (item, 456), (description, Dress shirt), (color, White), (color, Blue) PUT (item, 789), (description, Shoes), (color, Black), (material, Leather)

Query

Domain = MyStore

(18)

Leveraging AWS

Application hosting - Using EC2 to provided an elastic web tier Batch processing - Using SQS, EC2 and S3 to process large amounts of data

Searchable Object Store - Combine S3 and SimpleDB to store metadata data about objects and run searches

Cloud-Bursting - Leveraging EC2 for excess capacity to augment your own data center

(19)

(20)

(21)

(22)

(23)

Using AWS with Java

Web services provide language independence WSDL can be used to build an access layer Popular Open Source Java libraries include;

Jets3t - Amazon S3 access

(24)

Jets3t

An open source library for interacting with Amazon S3 and objects stored there.

Supports both SOAP and REST interfaces. Well tested and very robust.

(25)

S3 API

package org.jets3t.service; Class to know about;

S3Service

S3Bucket createBucket(String name)

S3Object getObject(S3Bucket bucket, String key)

S3Bucket

setAcl(AccessControlList acl)

S3Object

setContentType(String type)

(26)

S3 Code Sample

// code from a servlet that proxies data from S3

s3 = new RestS3Service(new AWSCredentials(awsAccessId, awsSecretKey));

// get the object details from S3

S3Object obj = new S3Object(objectId);

S3Bucket bucket = new S3Bucket(bucketName); obj = s3.getObject(bucket, objectId);

// use the stream from the S3Object to move data to the servlet response stream

OutputStream oStr = response.getOutputStream(); response.setContentType(obj.getContentType()); InputStream in = obj.getDataInputStream();

(27)

Typica

Typica was designed initially to support SQS via the REST interface in a very reliable and thread-safe way.

The initial single digit downloads were inspiring enough to get EC2 support added.

Incorporates patches from users and supports many popular services.

Current popularity means hundreds of downloads for each release.

(28)

SQS API

package com.xerox.amazonws.sqs; Classes to know about;

QueueService messageQueue getOrCreateMessageQueue(queueName) List<MessageQueue> listMessageQueues MessageQueue sendMessage(msg) String receiveMessage() deleteMessage(msgId) deleteQueue(force) Message String getMessageBody()

(29)

SQS Sample Code

QueueService qs = new QueueService(<accessId>, <secretKey>);

MessageQueue queue = qs.getOrCreateMessageQueue("testQueue"); queue.sendMessage("test Message");

Message msg = queue.receiveMessage();

(30)

EC2 API

package com.xerox.amazonws.ec2; Classes to know about;

Jec2 List<ImageDescription> describeImages(imageIds) List<ReservationDescription> describeInstances (instanceIds) ReservationDescription runInstances(imageId, ...) ConsoleOutput getConsoleOutput(instanceId) ReservationDescription List<Instance> getInstances();

(31)

EC2 Sample Code

Jec2 ec2 = new Jec2(<accessId>, <secretKey>);

ReservationDescription res = ec2.runInstances("ami-1234567", 1, 1, null, null, "dak-keypair"); String instanceId = res.getInstances().get(0).getInstanceId();

while (!res.getInstances().get(0).getState().equals("running")) { try { Thread.sleep(5000); } catch (InterruptedException ex) {} res = ec2.describeInstances(instanceId);

}

logger.debug("Instance running : "+instanceId);

ConsoleOutput cons = ec2.getConsoleOutput(instanceId); logger.debug("console output = "+cons.getOutput());

(32)

SimpleDB API

package com.xerox.amazonws.sdb; Classes to know about;

SimpleDB

ListDomainsResult listDomains()

Domain createDomain(String name)

Domain

QueryResult listItems(String query)

QueryWithAttributesResult listItemsWithAttributes(String query)

Item

List<ItemAttribute> getAttributes()

(33)

SimpleDB Sample Code

SimpleDB sdb = new SimpleDB(<accessId>, <secretKey>); Domain dom = sdb.getDomain("testDomain");

logger.info("items:"); String nextToken = "";

while (nextToken != null) {

ListDomainsResult result = dom.listItems(query, nextToken, 10); List<Item> items = result.getItemList();

for (Item item : items) {

logger.info(item.getIdentifier()); }

nextToken = result.getNextToken();

(34)

Useful Links

http://aws.amazon.com/

http://code.google.com/p/typica/