AWS Performance Tuning

(1)

AWS Performance Tuning

Markus Albe @Percona Fernando Ipar @Percona Ryan Lowe @Square PLNY 2012

(2)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(3)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(4)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(5)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(6)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(7)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(8)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(9)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(10)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(11)

Amazon Web Services

• Cloud Formation

• ^CloudFront

• CloudSearch

• ^CloudWatch

• DirectConnect

• ^DynamoDB

• ^ec2

• ElastiCache

• ^Glacier

• ^IAM

• ^RDS

• ^{Route 53}

• ^S3

• ^SES

• ^SNS

• ^SQS

• Storage Gateway

• ^SWF

• ^VPC

(12)

Regions & Availability

Zones

(13)

Regions

•

^{US (x3)}

•

EU (Ireland)

•

^{APAC (x2)}

•

South America (Sao Paulo)

•

GovCloud (US Gov’t & Contractors)

(14)

AZ Differences

•

Performance Characteristics

•

^Hardware

•

^Pricing

•

Availability

(15)

(16)

AZ Strategy

•

Multi-AZ Deployment

•

Assume Entire AZs fail

•

Smart Software

•

Cost Optimization

(17)

ec2

(18)

(Elastic Compute Cloud) ec2

•

^{The Cloud}

•

On-Demand computers with minimal administration

•

Best for organizations with light uptime requirements, highly variable workloads,

horizontally distributable data and/or little time/skill to manage hosts

(19)

ec2 Instance Types

•

^Micro

•

^Small

•

^Medium

•

^Large

•

^X-XXXXL

•

High-Memory

•

Provisioned IOPS

•

^EBS

•

^SSD

(20)

ec2 Instance Pricing

•

Micro $0.025

•

Small $0.105

•

Large $0.415

•

XL $0.830

•

XXL $1.170

•

XXXXL $2.340 ($1600/month)

(21)

(Elastic Block Store) EBS

•

Persistent Data Storage

•

^{Up to 1TB*}

•

Limited to single availability zone

•

Automatic “Replication”

•

Multiple EBS per ec2 Instance

(22)

EBS Pricing

• Standard EBS

• $0.10 per GB provisioned

• $0.10 per million IO

• Provisioned IOPs

• $0.125 per GB provisioned

• $0.10 per provisioned IOPS-month

• EBS Snapshots to S3

(23)

ec2 First Write Penalty

(24)

ec2 Hardware

•

Measured in Cores with Compute Units

•

“One EC2 Compute Unit provides the

equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor”

•

Performance Highly Variable. Empirical evidence suggests > 1% unusable!

(25)

ec2 Gotchas

•

Many organizations don’t know what their monthly bill will be until it’s too late.

•

Overprovisioning

•

Underprovisioning

•

Forgetting mandatory maintenance

•

Never testing restores

(26)

16G 16T 8F [rw]/sec

(Large w/RAID10)

0 37500 75000 112500 150000

Random Reads Random R/W Random Writes Seq R Seq W

(27)

16G 16T 8F [rw]/sec

(Large w/EBS)

0 75000 150000 225000 300000

Random Reads Random R/W Random Writes Seq R Seq W

(28)

EBS vs. Hardware

IOPS

EBS RAID10 SSD SAN

(29)

Benchmark Scripts

•

https://launchpad.net/~vadim-tk

(30)

RDS

(31)

RDS

(Relational Database Service)

•

SQL Server, MySQL, Oracle

•

If you need “A relational database service with minimal administration”

•

Best for organizations with light uptime requirements & little time/skill to manage databases

(32)

RDS Caveats

•

Mandatory 30-minute maintenance window PER WEEK

•

Magic Backups

•

Auto-Upgrade Minor Versions!

(33)

RDS Performance

•

Relatively Sane Defaults

•

Somewhat limited access to MySQL configuration variables

•

Very close to the corresponding ec2 size

(34)

Monitoring

(35)

Monitoring ec2

(36)

Monitoring Instances

•

Basic Monitoring: 5-minute granularity

•

Detailed Monitoring: 1-minute granularity

•

Custom Metrics & Alarms

•

AutoScaling based on Metrics!!

(37)

AutoScaling

•

To autoscale or not to autoscale?

•

Monitoring/Alerting on cost

•

Detailed monitoring on some %

•

Regular Reviews

(38)

Architecture

(39)

Partition by Function

•

ElastiCache for sessions

•

CloudSearch for ... wait for it ... Search

•

Per-Service data stores

(40)

Shard

•

Active-Active AZs

•

Shard by GeoIP?

(41)

Chaos Monkey

•

https://github.com/Netflix/SimianArmy

(42)

Conclusions

•

No magic scaling bullet

•

Common sense (and $) will get you far

•

Understand all AWS offerings

•

Software Architecture & Design FTW

(43)