OpenStack and Eucalyptus on FutureGrid

(1)

Science Cloud Summer School 2012 https://portal.futuregrid.org

OpenStack and Eucalyptus on

FutureGri

https://portal.futuregrid.org/tutorials/openstac Slides are at

https://portal.futuregrid.org/projects/24

Presenters:

Javier Diaz

(2)

Before you can use OpenStack

and Eucalyptus

• Please make sure you have a portal account

o https://portal.futuregrid.org

• Please make sure you are part of a valid FG project

o You can either create a new one or

o You can join an existing one with permission of the

Lead

• Please make sure the project you have is approved and valid

(3)

OpenStack vs Eucalyptus

• Both OpenStack and Eucalyptus are IaaS cloud frameworks (similar functionality)

• We are going to focus on OpenStack during this tutorial

• We make use of the EC2 interface thought Euca2ools

(4)

OpenStack

• OpenStack is a collection of open source

components to deliver public and private IaaS clouds

• Components: Nova, Swift, Glance, Keystone, and Quantum

• IaaS Cloud Services allows users to manage:

(5)

OpenStack Functionality

• Amazon AWS Interface Compatibility

• Flexible Clustering and Availability Zones

• Access Control List (ACL) with policies management • Network Management, Security Groups, Traffic

Isolation

• Cloud Semantics and Self-Service Capability

– Image registration and image attribute manipulation

• Bucket-Based Storage Abstraction (S3-Compatible) • Block-Based Storage Abstraction (EBS-Compatible) • Hypervisor support: Xen, KVM, VMware Vsphere,

(6)

OpenStack Testbed

• OpenStack is available to FutureGrid Users on

the India cluster

• Users can make use of a maximum of 24 nodes

on India

• Instances Types

https://portal.futuregrid.org

Type Memory VCPUs Root (vda) Ephemeral

(vdb)

m1.tiny 512MB 1 0GB 0GB

m1.small 2048MB 1 10GB 20GB

m1.medium 4096MB 2 10GB 40GB

m1.large 8192MB 4 15GB 40GB

m1.xlarge 16384MB 4 20GB 40GB

(7)

Log into India cluster

• Example: My username is jdiaz

$ ssh <username>@india.futuregrid.org

(8)

• Credentials in your $HOME directory under .futuregrid/openstack/

• (In Eucalyptus this was .futuregrid/eucalyptus)

• Load environment variables

User Credentials

$ unzip ~/.futuregrid/openstack/openstack-essex-jdiaz-india.zip -d ~/openstack

(9)

Install/Load Euca2ools

• Euca2ools are the command line clients used to interact with OpenStack.

• If using your own platform, install euca2ools bundle from

http://www.eucalyptus.com/downloads/euca2o ols

o Instructions for various Linux platforms are

available on the download page.

• On FutureGrid, load the Euca2ools module

(10)

Euca2ools

• Testing your setup. Use euca-describe-availability-zones to test the setup

• List the existing images using euca-describe-images

$ euca-describe-availability-zones

AVAILABILITYZONE india available

$ euca-describe-images

IMAGE ami-00000012

centos2012/euca-centos-2012.1.14-x86_64.img.manifest.xmavailable public x86_64 machine aki-00000010 ari-00000011 instance-store

IMAGE ami-00000013

debian2011/euca-debian-2011.07.02-x86_64.img.manifest.xml available public x86_64 machine aki-0000000e ari-0000000f instance-store

(11)

Key Management

• Before you instanciate a VM, you need to create at

least one key pair. This key pair will be injected into the VM, allowing you to SSH into the VM

• Create a keypair and add the public key to

OpenStack (Replace userkey with your username

i.e. jdiazkey)

• Fix the permissions on the generated private key$ euca-add-keypair userkey > userkey.pem

$ chmod 0600 userkey.pem $ euca-describe-keypairs

(12)

Image Instantiation

Now we are ready to start a VM using one of the pre-existing images.

We need the ami-id of the image that we wish to start (euca-describe-images)

We use the euca-run-instances command to start a VM with the image ami-00000017

$ euca-run-instances -k userkey -n 1 ami-00000017 -t m1.small

RESERVATION r-zvtbbj8j default

(13)

Monitoring

euca-describe-instances shows the status of the VMs

$ euca-describe-instances i-0000073f

RESERVATION r-cbcagzl4 461884eef90047fbb4eb9ec92f22a1e3 default INSTANCE i-0000073f ami-00000033 server-1855 server-1855 pending

userkey 0 m1.small 2012-07-09T15:49:46.000Z india aki-0000000e ari-0000000f

Shortly after…

$ euca-describe-instances i-0000073f

RESERVATION r-cbcagzl4 461884eef90047fbb4eb9ec92f22a1e3 default INSTANCE i-0000073f ami-00000033 149.165.158.159 server-1855 running

(14)

VM Access

• Please, exit from the VM to continue with the Tutorial

The ssh private key that was generated earlier can now be used to log into the VM

$ ssh -i userkey.pem [email protected]

(15)

VM Access

• VMs also have a private IP that can be used for

internal communication between VMs (OpenStack Essex does not show the private IP)

• On India you can identify the private IP by executing

The ssh private key that was generated earlier can now be used to log into the VM

(16)

VM Access

• Side note

–By default, in Ubuntu images you need to login as

ubuntu user (NOT our case)

(17)

Create Access Rule

(it should not be needed)

(18)

Nova Volumes

• Nova-volume provides persistent block

storage compatible with Amazon’s Elastic Block Store

• The storage in the instances is non-persistent and it gets lost when the instance is terminated

• Persistent volumes keeps data generated

during instance life time after the instance is terminated

(19)

Create Volumes

• Create a 1GB Volume in the India zone

• List available volumes

$ euca-create-volume -s 1 -z india

VOLUME vol-00000031 1 india creating 2012-07-10T15:15:47.244Z

$ euca-describe-volumes

VOLUME vol-00000027 100 india in-use 2012-06-06T21:39:47.000Z

ATTACHMENT vol-00000027 i-0000070f /dev/vdc attached

VOLUME vol-00000028 50 india available 2012-06-06T21:44:30.000Z

(20)

Attach Volume

• Attach volume to a running instance

• A volume can only be attached to one instance.

Euca-describe-volumes will show the volume status as “attached”

• An additional SCSI disk is created in the instance

• Although we specified the device, it may differ if

that device already exists (look into /dev,

/var/log/syslog, /dev/log/messages or use fdisk -l to find the new device)

(21)

Using the New Disk

• Log into the instance again

• Format disk (skip this step if you want to reuse

data stored)

• Mount disk

• Exit from the instance to continue

# mkfs /dev/vdc

# mount /dev/vdc /mnt

(22)

Detach Volumes

• Volumes are automatically detached when

instance is terminated

• Detach volume

• If you detach the volume while instance is

running, it loses access to the disk. Make sure that you umount (umount /mnt) the disk before you

detach the volume.

• If you terminate the instance, the volume is

automatically detached

(23)

Volume Snapshots

• Snapshots are useful to create backups or replicate volumes in different zones

• Create Snapshot

• List Snapshots

$ euca-create-snapshot -d ’Testing snapshot’ vol-00000027

SNAPSHOT snap-00000001 vol-00000027 creating 2012-07-16T14:22:21.728Z 0% Testing snapshot

$ euca-describe-snapshots

(24)

Create Volume from Snapshot

(currently it does not work in OpenStack Essex)

• The snapshot must be in available status (100% completed)

• New volume can be bigger if desired

• Create a 2GB volume from snapshop

$ euca-create-volume -s 2 --snapshot snap-00000001 -z india

VOLUME vol-00000032 2

(25)

Image Registration (1/4)

We will use an CentOS 5 image to test the image registration

o Download the gzipped tar ball

Uncompress and Untar the archive

$ wget i120/test-image/centos5.tgz

(26)

Image Registration (2/4)

• Next we bundle the image with a kernel and a

ramdisk using the euca-bundle-image command

– We will use a kernel already registered.

– euca-describe-images returns the kernel and ramdisk IDs that we need

$ euca-bundle-image -i centos5.img kernel aki-0000000e --ramdisk ari-0000000f

Checking image Encrypting image Splitting image...

Part: centos5.img.part.00 ……..

Part: centos5.img.part.35

(27)

Image Registration (3/4)

• Upload the image (euca-upload-bundle)

• Register the image (euca-register)

$ euca-register jdiaz-bucket/centos5.img.manifest.xml

IMAGE ami-00000034

$ euca-upload-bundle -b jdiaz-bucket -m /tmp/centos5.img.manifest.xml

Checking bucket: jdiaz-bucket Creating bucket: jdiaz-bucket Uploading manifest file

Uploading part: centos5.img.part.00 ….

Uploading part: centos5.img.part.35

(28)

Image Registration (4/4)

• You cannot run instances until the image is in

available status

• Check status

$ euca-describe-instances ami-00000034

(29)

Image/Volume Deletion and

Instance Termination

• You can delete your volume

• You can delete your image (Please do NOT delete other users’ images)

• $ euca-deregister ami-00000034You can terminate your VM

(30)

More Information

• We keep this tutorial updated in the FutureGrid portal