Amazon Web Services. Elastic Compute Cloud (EC2) and more...

Amazon Web Services

Disclaimer



I don’t work for Amazon



I do however, have a small research grant

from Amazon (in AWS$)



Portions of this presentation are

reproduced from slides by Amazon

Agenda

Introducing Amazon Web Services

What is Amazon EC2?

• EC2 Demo

A closer look at Amazon Web Services

• Elastic MapReduce Demo

Computational experiments on EC2

What is Amazon.com?

Amazon.com sold first book in 1995, which

shipped from Jeff Bezos’ garage

Now sells a lot more than just books

Affiliate sellers: almost anyone can sell

almost anything

Huge infrastructure: ~40 fulfilment centres,

IT supporting millions of transactions per day

What is AWS?



Amazon’s own Infrastructure-as-a-Service

(IaaS)

◦

Publicly available via Amazon query (REST) and

SOAP APIs over HTTP

◦

Provides storage, compute, hosting utilities,

content delivery



Various Software-as-a-Service on-top and

supporting the infrastructure

◦

Database

◦

Demand based auto-scaling

◦

Enterprise style application messaging

Evolution of AWS

2002:

E-Commerce

Service

(ECS)

March 06:

Simple

Storage

Service (S3)

Aug 06:

Elastic

Compute

Cloud (EC2)

beta

Oct 08:

EC2 enters

production

Agenda

Introducing Amazon Web Services

What is Amazon EC2?

• EC2 Demo

A closer look at Amazon Web Services

• Elastic MapReduce Demo

Computational experiments on EC2

What is Amazon EC2?

On

demand

Self

service

Pay as

you go

EC2 Terminology

• Instance

Server

• Elastic Compute Units

Processors

• Amazon Machine Image

Boot disk

• Elastic IP

Static IP

• Security Groups

Firewall

• Elastic Block Store

SAN

• Elastic Load Balancer

Load Balancer

• CloudWatch

EC2 Demo

Try

Me!

Agenda

Introducing Amazon Web Services

What is Amazon EC2?

• EC2 Demo

A closer look at Amazon Web Services

• Elastic MapReduce Demo

Computational experiments on EC2

More Amazon Web Services

Compute

• Elastic Compute Cloud • Elastic MapReduce • Auto Scaling

Messaging

• Simple Queue Service • Simple Notification Service

Storage

• Simple Storage Service

• Elastic Block Store • AWS Import/Export

Content Delivery

• Amazon CloudFront

Monitoring

• Amazon CloudWatch

Support

• AWS Premium Support

Database

• Amazon SimpleDB • Relational Database Service

Networking

• Virtual Private Cloud • Elastic Load

Balancing

Web traffic

• Alexa Web Information • Alexa Top Sites

E-Commerce

• Fulfilment Web Service

Payments/Billing

• Flexible Payments Service • Amazon DevPay

Workflow

• Amazon Mechanical Turk

Data Processing with Elastic

MapReduce



Elastic MapReduce

◦

Hosted Apache Hadoop as-a-Service

◦

A framework for distributed divide and conquer

operations on large datasets

◦

Map function applied to each element of the

dataset (in parallel)

Elastic MapReduce Demo

Try

Me!

Simple Queue Service

msg

msg

msg

msg

Producer-Consumer Problem

Agenda

Introducing Amazon Web Services

What is Amazon EC2?

• EC2 Demo

A closer look at Amazon Web Services

• Elastic MapReduce Demo

Computational experiments on EC2

Parametric computing with the

Nimrod tools



Nimrod workflows for robust design and

search

◦

Vary parameters

◦

Execute programs

◦

Copy data in/out

◦

X, Y, Z might be:



Basic datatypes; ints, floats, strings



Files



Random numbers to drive Monte Carlo modelling

Parameter

Space

Solution

Space

X

Y

Z

Mixing Grids and Clouds



A lot of existing grid based infrastructure



Grids provide services above IaaS

◦

E.g., you can build a grid on EC2

◦

Grids provide job and data handling services,

more like PaaS



So, mix them together:

◦

Blair Bethwaite et al., “Mixing Grids and

Clouds: High-Throughput Science Using the

Nimrod Tool Family,” in Cloud Computing, vol. 0

(Springer London, 2010), 219-237

Agenda

Introducing Amazon Web Services

What is Amazon EC2?

• EC2 Demo

A closer look at Amazon Web Services

• Elastic MapReduce Demo

Computational experiments on EC2

What can I do with it?

Computational horsepower for parameter

exploration/estimation and workflows

Data processing

Host a (scalable) web service/site

Development platform

What now?

Try it out, for free!

http://aws.amazon.com/free/

Then, apply for an academic grant:

http://aws.amazon.com/education/

AWS Free Usage Tier

Highlights (Per Month):

• 750 hours of Amazon EC2 Linux Micro

Instance usage (613 MB of memory and

32-bit and 64-32-bit platform support)

• 10 GB of Amazon Elastic Block Storage, plus 1

million I/Os, 1 GB of snapshot storage,

10,000 snapshot Get Requests and 1,000

snapshot Put Requests

• 5 GB of Amazon S3 storage, 20,000 Get

Requests, and 2,000 Put Requests

Thank you!

Presentation by:

Blair Bethwaite

Researcher, Developer, SysAdmin

Monash eScience and Grid Engineering Lab

Feedback/queries: