CLOUD COMPUTING WITH AWS An INTRODUCTION. John Hildebrandt Solutions Architect ANZ

Full text

(1)CLOUD COMPUTING WITH AWS An INTRODUCTION John Hildebrandt| Solutions Architect ANZ.

(2) AGENDA.

(3) Todays Agenda • • • • •. Background and Value proposition of AWS Global infrastructure and the Sydney Region AWS services Drupal example Q&A.

(4) AWS BACKGROUND.

(5) How did amazon.com….

(6) Cloud Computing Benefits No Up-Front Capital Expense. Low Cost. Pay Only for What You Use. Self-Service Infrastructure. Easily Scale Up and Down. Improve Agility & Time to Market. Deploy.

(7) GLOBAL INFRASTRUCTURE.

(8)

(9) AWS Global Infrastructure. 9 AWS Regions 30+ AWS Edge Locations.

(10) Each day AWS adds the equivalent server capacity to power Amazon when it was a global, $5B enterprise 2003 $5.2B retail business 7,800 employees A whole lot of servers.

(11) Objects in S3. 1.3 Trillion 835k peak transactions per second.

(12) EMR Jobs. 3.7 M clusters 4,000,000 3,500,000 3,000,000. 2,500,000 2,000,000 1,500,000 1,000,000 500,000 0. launched since May 2010.

(13) AWS Regions & Availability Zones. US REGIONS US East (VA) Availability Zone A. US West (CA). Availability Zone A. Availability Zone A. Availability Zone B. Availability Zone C. GovCloud (OR). Availability Zone B. EU (Ireland) Availability Zone A. Availability Zone A Availability Zone C. Asia Pacific (Singapore). Availability Zone B. Availability Zone B. Availability Zone D. US West (OR) Availability Zone A. Asia Pacific (Tokyo). Availability Zone B Availability Zone A. Availability Zone C. GLOBAL REGIONS. South America (Sao Paulo). Asia Pacific (Sydney). Availability Zone B. Availability Zone B. Availability Zone A. Availability Zone B. Availability Zone A. Availability Zone C. Customer Decides Where Applications and Data Reside Note: Conceptual drawing only. The number of Availability Zones may vary.. Availability Zone B.

(14) #1 enterprise question Is the cloud secure for my apps and data?.

(15) Security is Our #1 Priority. SOC 2. Physical Security. Network Security ISO 27001. Platform Security. ISAE 3402 FISMA Moderate. People & Procedures. ITAR. HIPAA. PCI DSS. FIPS 140-2.

(16) Many Customers’ Security Posture Improves In the Cloud “You basically turn yourself into a polymorphic surface to which the attack guy has a much tougher time getting at. That, ultimately, is the real key advantage to drive security and make things much better for us across the board.” Gus Hunt, CTO Central Intelligence Agency. “The improved computer security includes, but is not limited to, greater protection against network attacks and real time detection of system tampering.”. Earl E. Devaney, Chairman Recovery.gov.

(17) SECURITY IS A SHARED RESPONSIBLITY.

(18) Customer. Customer Data •. Platform, Applications, Identity & Access Management. • • • • • • • •. SAS-70 Type II ISO 27001/ 2 Certification Payment Card Industry (PCI) Data Security Standard (DSS) NIST Compliant Controls DoD Compliant Controls FedRAMP Compliant Controls HIPAA and ITAR Compliant. Operating System, Network & Firewall Configuration Client-side Data Encryption & Data Integrity Authentication. Server-side Encryption (File System and/or Data). Network Traffic Protection (Encryption/Integrity/Identity). Foundation Services. Amazon. •. Customers implement their own set of controls Multiple customers with FISMA Low and Moderate ATOs. Compute. AWS Global Infrastructure. Storage. Database. Networking. Availability Zones Edge Locations Regions.

(19) AWS Platform Your Applications Management & Administration Identity & Access AWS IAM Identity Federation Consolidated Billing. Web Interface. Monitoring. Management Console. Amazon CloudWatch. Deployment & Automation AWS Elastic Beanstalk AWS CloudFormation. Application Platform Services Content Distribution Amazon CloudFront. Application Svcs Simple Workflow Service CloudSearch Amazon SNS, SQS, SES. Parallel Processing. Libraries & SDKs. Elastic MapReduce. Java, PHP, Python, Ruby, .NET. Database. Networking. Amazon RDS Amazon SimpleDB Amazon ElastiCache Amazon DynamoDB. Amazon VPC Elastic Load Balancing Amazon Route 53 AWS Direct Connect. Foundation Services Compute Amazon EC2 Auto Scale. Storage Amazon S3 Amazon EBS Amazon StorageGateway. AWS Global Infrastructure. Availability Zones Regions. Edge Locations.

(20) Let’s use an Example – aGov Drupal HA site.

(21) AWS Platform Your Applications Management & Administration Identity & Access AWS IAM Identity Federation Consolidated Billing. Web Interface. Monitoring. Management Console. Amazon CloudWatch. Deployment & Automation AWS Elastic Beanstalk AWS CloudFormation. Application Platform Services Content Distribution Amazon CloudFront. Application Svcs Simple Workflow Service CloudSearch Amazon SNS, SQS, SES. Parallel Processing. Libraries & SDKs. Elastic MapReduce. Java, PHP, Python, Ruby, .NET. Database. Networking. Amazon RDS Amazon SimpleDB Amazon ElastiCache Amazon DynamoDB. Amazon VPC Elastic Load Balancing Amazon Route 53 AWS Direct Connect. Foundation Services Compute Amazon EC2 Auto Scale. Storage Amazon S3 Amazon EBS Amazon StorageGateway. AWS Global Infrastructure. Availability Zones Regions. Edge Locations.

(22) Built to Enterprise & Gov Standards. Physical • Datacenters in nondescript facilities. • Physical access strictly controlled • Must pass two-factor authentication at least twice for floor access • Physical access logged and audited. Hardware, Software & Network • Systematic change management. Certifications and Accreditations • ISO 27001 • SSAE 16 / ISAE 3402 / SOC1 (formerly U.S. standard SAS-70 Type II) • FISMA Moderate & DIACAP Controls; ITAR region • HIPAA applications certified on AWS. • Payment Card Industry (PCI) Data Security Standard (DSS) Level 1. Security & Compliance Resources. • Phased updates deployment. • Security & Compliance Center: http://aws.amazon.com/security. • Safe storage decommission. • Security Overview & Best Practices. • Automated monitoring and self-audit • Advanced network protection systems. • AWS Risk & Compliance Whitepaper • Creating HIPAA Compliant Applications.

(23) Foundation Services Your Applications Management & Administration Identity & Access AWS IAM Identity Federation Consolidated Billing. Web Interface. Monitoring. Management Console. Amazon CloudWatch. Deployment & Automation AWS Elastic Beanstalk AWS CloudFormation. Application Platform Services Content Distribution Amazon CloudFront. Application Svcs Simple Workflow Service CloudSearch Amazon SNS, SQS, SES. Parallel Processing. Libraries & SDKs. Elastic MapReduce. Java, PHP, Python, Ruby, .NET. Database. Networking. Amazon RDS Amazon SimpleDB Amazon ElastiCache Amazon DynamoDB. Amazon VPC Elastic Load Balancing Amazon Route 53 AWS Direct Connect. Foundation Services Compute Amazon EC2 Auto Scale. Storage Amazon S3 Amazon EBS Amazon StorageGateway. AWS Global Infrastructure. Availability Zones Regions. Edge Locations.

(24) Compute. Elastic Compute Cloud. Amazon Machine Image. Auto Scaling.

(25) Compute Amazon Elastic Compute Cloud (Amazon EC2) EC2 Instances = Virtual Servers • Resizable compute capacity in 16 instance types • Reduces the time required to obtain and boot new server instances to minutes or seconds. • Scale capacity as your computing requirements change • Pay only for capacity that you actually use • Choose Linux or Windows • Deploy across Regions and Availability Zones for reliability • Flexible networking (NAT/classic, VPC, Elastic IPs) • Support for virtual network interfaces that can be attached to EC2 instances in your VPC.

(26) Compute Amazon Machine Image • Building blocks of EC2 instances • An AMI is like a template of a computer's root volume. • Can be public or private • Create hardened or gold “Images” of your EC2 infrastructure.

(27) Compute Auto Scaling • Client Defined Business Rules • Scale your Amazon EC2 capacity automatically once you define the conditions (may be 1000’s of servers) • Can scale up just a little…doesn’t need to be massive number of servers (may be simply 2 servers) • Well suited for applications that experience variability in usage • Set minimum and maximum scaling policies • Alternate Use is for Fault Tolerance. "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "AvailabilityZones" : { "Fn::GetAZs" : "" }, "LaunchConfigurationName" : { "Ref" : "LaunchConfig" }, "MinSize" : "1", "MaxSize" : "5", "DesiredCapacity" : { "Ref" : "WebServerCapacity" }, "LoadBalancerNames" : [ { "Ref" : "ElasticLoadBalancer" } ] } },.

(28) Storage. S3. Import/Export EBS. G. Storage Gateway Glacier.

(29) Storage Simple Storage Service (S3) Web-scale Internet Storage • A “Bucket” is equivalent to a “folder” • Able to store unlimited number of Objects in a Bucket • Objects from 1B-5 TB; no bucket size limit • Highly available storage for the Internet (object store) • HTTP/S endpoint to store and retrieve any amount of data, at any time, from anywhere on the web • Highly scalable, reliable, fast, and inexpensive • Over 2 trillion objects stored. • Peak requests 1M+ per second • Ideal Use Cases: • Static web content – often used with CloudFront CDN • Source and output storage for large-scale “Big Data” analytics. • Backup, archival, and DR storage that is always “live”.

(30) Storage Elastic Block Store (EBS) EBS Volumes = Virtual Disks • Use for persistent storage • Can use to create RAID configuration for a server • Off-instance block storage that persists independently • Storage volumes for use with Amazon EC2 instances – create, attach, backup, restore and delete • Can be attached to a running Amazon EC2 instance and exposed as a block device for raw or formatted (filesystem) access. • Volumes behave like unformatted block devices for Linux or Windows instances • Ideas use cases: • OS Boot device / root file system; secondary volumes/filesystems • Typical basis for database storage. • Raw block devices for RAID, some databases.

(31) Database. RDS. DynamoDB. ElastiCache. RDS. SimpleDB.

(32) Database Amazon Relational Database Service (RDS). RDS. • Fully-managed, tuned MySQL, Oracle 11g, or MS SQL databases • Cost-efficient and resizable capacity • Manages time-consuming database admin tasks • Code, applications, and tools you already use today work seamlessly • Automatically patches the database software and backs up your database • Flexible Licensing: BYOL or License Include. "DBInstance" : { "Type": "AWS::RDS::DBInstance", "Properties": { "DBName" : { "Ref" : "DBName" }, "Engine" : "MySQL", "MultiAZ" : { "Ref": "MultiAZDatabase" }, "MasterUsername" : { "Ref" : "DBUsername" }, "DBInstanceClass" : { "Ref" : "DBClass" }, "DBSecurityGroups" : [{ "Ref" : "DBSecurityGroup" }], "AllocatedStorage" : { "Ref" : "DBAllocatedStorage" }, "MasterUserPassword": { "Ref" : "DBPassword" } } },.

(33) Networking. ELB. Route 53. VPC.

(34) Networking Amazon Elastic Load Balancing • Supports the routing and load balancing of HTTP, HTTPS and generic TCP traffic to EC2 instances • Supports health checks to ensure detect and remove failing instances • Dynamically grows and shrinks required resources based on traffic • Seamlessly integrates with Auto-scaling to add and remove instances based on scaling activities • Single CNAME provides stable entry point for DNS configuration.

(35) Networking Amazon Virtual Private Cloud (VPC) • Secure and seamless bridge between a company’s existing private network and the AWS cloud • Connect existing infrastructure to a set of isolated AWS compute resources via a Virtual Private Network (VPN) connection. • Bring your own address space and extend existing management capabilities.

(36) Application Platform Services Your Applications Management & Administration Identity & Access AWS IAM Identity Federation Consolidated Billing. Web Interface. Monitoring. Management Console. Amazon CloudWatch. Deployment & Automation AWS Elastic Beanstalk AWS CloudFormation. Application Platform Services Content Distribution Amazon CloudFront. Application Svcs Simple Workflow Service CloudSearch Amazon SNS, SQS, SES. Parallel Processing. Libraries & SDKs. Elastic MapReduce. Java, PHP, Python, Ruby, .NET. Database. Networking. Amazon RDS Amazon SimpleDB Amazon ElastiCache Amazon DynamoDB. Amazon VPC Elastic Load Balancing Amazon Route 53 AWS Direct Connect. Foundation Services Compute Amazon EC2 Auto Scale. Storage Amazon S3 Amazon EBS Amazon StorageGateway. AWS Global Infrastructure. Availability Zones Regions. Edge Locations.

(37) Management & Administration Your Applications Management & Administration Identity & Access AWS IAM Identity Federation Consolidated Billing. Web Interface. Monitoring. Management Console. Amazon CloudWatch. Deployment & Automation AWS Elastic Beanstalk AWS CloudFormation. Application Platform Services Content Distribution Amazon CloudFront. Application Svcs Simple Workflow Service CloudSearch Amazon SNS, SQS, SES. Parallel Processing. Libraries & SDKs. Elastic MapReduce. Java, PHP, Python, Ruby, .NET. Database. Networking. Amazon RDS Amazon SimpleDB Amazon ElastiCache Amazon DynamoDB. Amazon VPC Elastic Load Balancing Amazon Route 53 AWS Direct Connect. Foundation Services Compute Amazon EC2 Auto Scale. Storage Amazon S3 Amazon EBS Amazon StorageGateway. AWS Global Infrastructure. Availability Zones Regions. Edge Locations.

(38) Web Console. On-demand, Self Service Management Access.

(39) Identity & Access Management • IAM enables customers to create and manage users in AWS’s identity system • Identity Federation with local directory is an option for enterprises • Very familiar security model • Users, groups, permissions • Allows customers to. • Create users • Assign individual passwords, access keys, multi-factor authentication devices • Grant fine-grained permissions. • Optionally grant them access to the AWS Console • Organize users in groups.

(40) Deployment and Management Amazon CloudWatch • Visibility into resource utilization, operational performance, and overall demand patterns • Metrics such as CPU utilization, disk reads and writes, and network traffic • Accessible via the AWS Management Console, web service APIs or Command Line Tools • Add custom metrics of your own • Alarms (which tie into auto-scaling, SNS, SQS, etc.) • Billing Alerts to help manage charges on AWS bill.

(41) Deployment and Management AWS CloudFormation • Create templates of stack of resources • Deploy stack from template with runtime parameters • Templates are simple JSON formatted text files • CloudFormer supports generating templates from running environments "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "Tags" : [{ "Key" : "MyTag", "Value" : "TagValue" }] } },.

(42) aGov Drupal HA Script • Based on sample at: – https://s3-ap-southeast-2.amazonaws.com/cloudformationtemplates-ap-southeast-2/Drupal_Multi_AZ.template. • Leveraged aGov Drupal 7 distribution: – http://agov.com.au/download.

(43) Availability Zone #2. Availability Zone #1 Site Content Slave. Site Content. S3 Static Content: .jpg, .css, .js. Web Server. Web Server Web Auto Scaling Group. User.

(44) Support repeatable processes. Dev Git Subversion Mercurial. Test. Prod Template File Defining Stack.

(45) Useful Resources & Links. • Architecture Center: http://aws.amazon.com/architecture • Security Center: http://aws.amazon.com/security • Whitepapers: http://aws.amazon.com/whitepapers • Resources: http://aws.amazon.com/resources • Case Studies: http://aws.amazon.com/solutions/case-studies. • Solution Providers: http://aws.amazon.com/solutions/global-solution-providers/ • Calculator: http://calculator.s3.amazonaws.com/calc5.html • TCO Calculator: http://aws.amazon.com/tco-calculator/ • AWS Blog: http://aws.typepad.com/ • The Power of 60: http://www.powerof60.com/.

(46) THANK YOU.

(47)

No results found