"Build and Test in the Cloud "

W5

Class 

11/17/2010 10:00:00 AM

"Build and Test in the Cloud "

Presented by:

Darryl Bowler

CollabNet

Brought to you by: 

Darryl Bowler

With more than fifteen years of IT experience, Darryl Bowler leads the consulting services business for CollabNet Lab Management, delivering high-value virtualization and test lab automation consulting services to major enterprises. Darryl is an

accomplished senior systems architect, specializing in software development

methodologies, virtualization, and cloud computing, combined with application lifecycle management and agile best practices. His portfolio of customers includes major financial institutions, a major automotive company, and health organizations. Darryl is currently leading a project to transform the software development lifecycle of a Fortune 50 company.

Darryl Bowler

Professional Services

Built and Test in the Cloud

A Build Masters Guide

What you will learn in the next 90 minutes

• Business and technical reasons

• Overview of a cloud provider

• Amazon EC2

• VPC – Virtual Private Cloud

• Qualities of a cloud (for software development)

• Build tools – Hudson

• What is Hudson?

• Continuous Integration (and deployment)

• Provision build-slaves into the cloud

• Amazon EC2 plug-in

• Artifact repositories (Apache Archiva and Sonatype Nexus)

• The need for a artifact repository

• Selenium and automated web tests

Cloud Providers and Management

Many to choose from

Reasons to Build on in the Cloud

“a production practice that considers the expenditure of resources for any goal other than the creation of value for the end customer to be wasteful, and thus a target for elimination”

THIS IS OUR GOAL!

• Predictable resource pool of compute • Dynamic provisioning of system resources • Pay as you “need” model

opyright 2010 5

Qualities of a Cloud (for software development)

•On-demand provisioning

•Cross platform

• Can provision a multitude of operating systems and stacks

• Rich API

• Ease of integration into “workflow”

• System provisioning become a “service”

•Multi-tenant

• Flexible RBAC to separate duties and operations

•Measurable with a charge back reporting

•Library of pre-built (reusable) images

•Empowers developers • Increased productivity

• Integrate cloud capabilities into their development tools (IDE)

Amazon Web Services

• Comprehensive Management Interface

• Accounting, provisioning and configuration

• Amazon Elastic Compute Cloud (Amazon EC2)

• Heterogeneous system, pay for what you use model

• Amazon Virtual Private Cloud (Amazon VPC)

• Integration with corporate network

• Amazon Simple Storage Service (Amazon S3)

• Amazon SimpleDB & Relational Database Service (Amazon RDS)

Amazon EC2

• Provision a pre-built system image in minutes

• AMI = Amazon Machine Image

• Create your own AMI

• Library of images available to developers

• Security and IP filtering available

• Multitude of Linux distros, Windows Server 2003/2008, OpenSolaris

• Provision in various geographical locations

• Western Europe, USA East and West and SE Asia Pacific

Amazon EC2 Tools

• Command line tool

• Can be used in a build workflow

ec2-describe-images -o self

IMAGE ami-f41afe9d /hudsonbuild/image.manifest.xml 419334764189 available private i386 machine instance-store IMAGE ami-c59676ac hudson_slave_with_swarm/image.manifest.xml

419334764189 available private i386 machine instance-store

ec2-run-instances ami-59967630 -k DBKEY

RESERVATION r-3a454c51 419334764189 default

INSTANCE i-31f8c55b ami-59967630 pending DBKEY 0 m1.small 2010-09-21T01:09:37+0000 us-east-1c

monitoring-disabled instance-store

Hudson

Extensible continuous integration server

• Java build tool

•java -jar hudson.war or servlet container such as Tomcat

•Can run on many platforms

• Fully configurable from GUI, unlike CruiseControl • Monitoring executions of externally-run jobs • Other extensive features

•Change set support, RSS/E-mail/IM Integration, Distributed builds, File fingerprinting and Plug-in Support

•Hudson support tools/processes that your team uses !!

• DISTRIBUTED and SCALABE!

•Master / Slave

2 Copyright © 2008 CollabNet, Inc. All Rights Reserved. CollabNet Confidential.

Â

FEATURES OF CI

• A connection to a version control repository • A build script

• Feedback mechanism

• Email

• SMS

• A process for integrating the source code • Automated inspection and testing

3 Copyright © 2008 CollabNet, Inc. All Rights Reserved. CollabNet Confidential.

Â

THE VALUE OF CI

• Reduced Risk

• Defects detected and fixed sooner

• Reduce repetitive manual processes

• Automated integration, build, inspection and test

• Allows for more productive and creative work

• Generate deployable software at any time

• Health of software is measurable

• Enable better project visibility

• More effective decisions (Quality metrics)

• Establish greater confidence in the software product

4 Copyright © 2008 CollabNet, Inc. All Rights Reserved. CollabNet Confidential.

Â

A TYPICAL CONTINUOUS INTEGRATION

The Case for Continuous Deployment

production or development

• Rapid method of placing features/changes into production • Requires rigorous testing

• Production clustered environment

• Real-time monitoring and altering

• Rollback

• Great to show work developed during and end of sprints • Hudson can deploy to Websphere, JBoss and Tomcat

The Case for Continuous Deployment

Using EC2 as a Dynamic Scalable Build Farm

• Library of pre-built AMI’s representing all build stacks

• Reasons

• Near real-time builds

• Pay for what you use

• No waiting on resources

• Automation and reuse implied

Hudson Amazon EC2 Plug-in

Web Testing with Selenium

• Cross operating system and web browser support

• An API that can be “driven” from various languages

• C#, Perl, Java, PHP Python and Ruby

• Can be used within a (automated) testing framework

• Bromine, JUnit, NUnit , RSpec (Ruby)

• 3 step process

• Write and run testing a Firefox plug-in (IDE)

• Use Selenium remote control to customize to execute in different browsers and platforms

• Use Selenium grid to execute distribute parallel testing across platforms and operating systems

The case for an Artifact Repository

“an extensible repository management software that helps

taking care of your own personal or enterprise-wide build artifact repository. It is the perfect companion for build tools such as Maven, Continuum, and ANT”

Quote from http://archiva.apache.org/

• Act as highly configurable proxies between your organization and the public Maven repositories

• Save hundreds of requests over HTTP, and, in very large multi-project builds, this can shave minutes from a build.

• Provide an organization with a deployment destination for your internally generated artifacts

• Central (Enterprise wide) software artifact repository

• FOSTERS REUSE, TRACEABLY and CONSISTENCY!!!

Further contact

• Email: dbowler@collab.net