Private Clouds and
OpenStack Introduction
Luis Tomás
Department of Computing Science
Today
1. Private Clouds
2. Introduction to OpenStack
– What is OpenStack?– Architecture and Main components – Demo: basic commands
1. Private Clouds
• IaaS infrastructure
• Built for exclusive use of one client – full
control over infrastructure, data, quality
of service, etc.
– Private
• Company owns infrastructure – may be
located at enterprise or at colocation
– Local (if in-house)
• Built and managed by enterprise IT or
cloud provider
Private Cloud toolkits
• Most have stack in their name:
– Covers the stack from layer 1 (physical) to layer 7 (applications) of the OSI (Open Systems Interconnection) model
• Quite a few toolkits, and even more
debate…
• Examples
– OpenStack – CloudStack – OpenNebula – EucalyptusPrivate Cloud toolkits
• OpenStack
– Started by Rackspace (storage files) and NASA in 2010
– Both Ubuntu and Red Hat distributions – Hypervisors: KVM, Xen, VmWare, …,even
baremetal
• CloudStack
– Apache project– OpenStack owner (Citrix) now own
CloudStack (2009) and dropped OpenStack – More Amazon like
– Hypervisors: KVM, vSphere, XenServer, Oracle VM
More private Cloud toolkits
• OpenNebula
– Developed 2008, Complutense University of Madrid, Spain
– Xen, KVM, VmWare
– Interfaces: EC2, OGF OCCI, vCloud – Large open-source community
• Nimbus
– Developed 2009 – EC2/S3 compatible – Xen, KVM
– Combine with OpenStack, Amazon, others
Even more…
• Eucalyptus
– Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems
• Developed 2008
• VMware, Xen, KVM
• Ubuntu, RHEL, and other flavors of Linux
• Only project based on GPL (GNU general SW license) and not ASL (Apache SW license)
• Implement AWS API on top of Eucalyptus
• AWS agrees to support Eucalyptus, users can migrate workloads between the two, applications compatible with both
Agenda
1. Private Clouds
2. Introduction to OpenStack – What is OpenStack?
– Architecture and Main components – Demo: basic commands
2. Introduction to OpenStack
• Initiated by Rackspace Cloud and NASA in 2010
• Supported by an industry consortium – IBM
– Canonical – AT&T – Rackspace – HP – RedHat – …
2. Introduction to OpenStack
• The Cloud Operation System
• The biggest open source project in the world
• > 18000 people, >400 companies, >140 countries, >20M lines of code, …
• http://www.openstack.org/
What is OpenStack?
• Mission statement
– “To produce the ubiquitous open source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to
implement and massively scalable.”
• Open – Open source – Open design – Open development – Open community
What is OpenStack?
• OpenStack Foundation
– “OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a data center, all managed through a dashboard that gives administrators control while empowering users to provision resources through a web interface.”
OpenStack: Overview
• Written (mostly) in Python
• All code available as open source
– Use Devstack available for more convenient development
– Other tools available, such as RDO from RedHat
– Ongoing projects for quick/easy installation/
configuration (TripleO)
• OpenStack inside OpenStack
• Ongoing effort – Changing rapidly
DevStack template
SERVICE_TOKEN=azertytoken ADMIN_PASSWORD=nomoresecrete MYSQL_PASSWORD=stackdb RABBIT_PASSWORD=stackqueue SERVICE_PASSWORD=$ADMIN_PASSWORD
# ``HOST_IP`` should be set manually for best results if the NIC configuration
# of the host is unusual, i.e. ``eth1`` has the default route but ``eth0`` is the
# public interface. It is auto-detected in ``stack.sh`` but often is indeterminate
# on later runs due to the IP moving from an Ethernet interface to a bridge on
# the host. Setting it here also makes it available for ``openrc`` to include
# when setting ``OS_AUTH_URL``.
# ``HOST_IP`` is not set by default.
#HOST_IP=w.x.y.z
# Logging
# ---
LOGFILE=$DEST/logs/stack.sh.log
…
# Using milestone-proposed branches
#CINDER_BRANCH=milestone-proposed
#GLANCE_BRANCH=milestone-proposed
…
DevStack template
ENABLED_SERVICES=rabbit,mysql,key
ENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cpu,n-cond,n-sch,n-novnc,n-cauth ENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-l3,q-meta,q-lbaas ENABLED_SERVICES+=,s-proxy,s-object,s-container,s-account ENABLED_SERVICES+=,g-api,g-reg
ENABLED_SERVICES+=,cinder,c-api,c-vol,c-sch,c-bak ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng ENABLED_SERVICES+=,trove,tr-api,tr-tmgr,tr-cond ENABLED_SERVICES+=,horizon
ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral,ceilometer- collector,ceilometer-api
… Q_PLUGIN=ml2
ENABLE_TENANT_VLANS=True Q_AGENT=openvswitch
ML2_VLAN_RANGES=physnet1:100:400 PHYSICAL_NETWORK=physnet1 OVS_PHYSICAL_BRIDGE=br-ex PUBLIC_INTERFACE=eth1
….
RDO template
# Set to 'y' if you would like Packstack to install OpenStack Compute
# (Nova)
CONFIG_NOVA_INSTALL=y CONFIG_NEUTRON_INSTALL=y CONFIG_HORIZON_INSTALL=y CONFIG_SWIFT_INSTALL=y CONFIG_CEILOMETER_INSTALL=y CONFIG_HEAT_INSTALL=n CONFIG_CLIENT_INSTALL=y ...
# The IP address of the server on which to install OpenStack services
# specific to controller role such as API servers, Horizon, etc.
CONFIG_CONTROLLER_HOST=192.168.0.1
# The list of IP addresses of the server on which to install the Nova
# compute service
CONFIG_COMPUTE_HOSTS=192.168.0.2,192.168.0.3,192.168.0.4,192.168.0.5
RDO template
CONFIG_NEUTRON_ML2_TYPE_DRIVERS=vlan
# A comma separated ordered list of network_types to allocate as
# tenant networks. The value 'local' is only useful for single-box
# testing but provides no connectivity between hosts.
CONFIG_NEUTRON_ML2_TENANT_NETWORK_TYPES=vlan
# A comma separated ordered list of networking mechanism driver
# entrypoints to be loaded from the neutron.ml2.mechanism_drivers
# namespace.
CONFIG_NEUTRON_ML2_MECHANISM_DRIVERS=openvswitch
# A comma separated list of physical_network names with which flat
# networks can be created. Use * to allow flat networks with arbitrary
# physical_network names.
CONFIG_NEUTRON_ML2_FLAT_NETWORKS=*
….
OpenStack: Overview
Design and coding standards
• Scalability and elasticity
• Asynchronous
• Horizontally scalable
• Share-nothing (or sharding)
• Distribute everything
• Accept eventual consistency
• Test everything
• Enforce PEP-8
• 6 month release cycle
OpenStack: Overview
Release history
• Austin
• Bexar
• Cactus
• Diablo
• Essex
• Folsom
• Grizzly
• Havana
• Icehouse
• Juno
• Kilo
• …?
October 2010 February 2011 April 2011 September 2011 April 2012 September 2012 April 2013 October 2013 April 2014 October 2014 April 2015 October 2015?
OpenStack: Overview
• Why Service Providers Are Adopting?
– Originated by Rackspace and NASA – Designed to scale cost effectively
– Emerging standard backed by large ecosystem – Open source with no lock-in or license – No desire to build proprietary clouds
OpenStack: Overview
Contributors OpenStack: Overview
Contributors
Agenda
1. Private Clouds
2. Introduction to OpenStack – What is OpenStack?
– Architecture and Main components – Demo: basic commands
Conceptual Architecture
Architecture
Consists on several (main) modules:
• Compute (Nova)
• Network (Neutron)
• Image storage (Glance)
• Block storage (Cinder)
• Object storage (Swift)
• Identity (Keystone)
• Metering (Ceilometer)
• Orchestration (Heat)
• Dashboard (Horizon)
Architecture
And many more:
• Database Service (Trove)
• Deployment (TripleO)
• Common Libraries (Oslo)
• Bare metal (Ironic)
• Queue service (Marconi)
• Data processing (Sahara)
• Key management (Barbican)
• …
Logical (Core) Architecture:
Icehouse Modules
• Compute - Nova
– Controls the cloud
computing fabric
– VM management
through the hypervisor
• Launch VM
• Start/Stop/Suspend/Resume/Reboot VM
• Resize/Migrate/LiveMigrate VM
– Format for describing VMs (Flavor)
– Supports popular Hypervisors
• KVM, Xen, or VMWare
• LXC (Container-based)
Modules
• Compute - Nova
– Technology agnostic:
• Virtual machine or container virt.
– Virtualization agnostic
• Libvirt (KVM, QEMU, Xen, LXC), XenAPI, Hyper-V, VMware ESX, PowerVM, Docker, Bare-metal
– Initiates most of the orchestration
activities, such as running or migrating
an instance
Modules
• Networking - Neutron
– Network infastructure
management
– Concepts & User
configurable topology
• Networks, Routers, Subnets, Ports
– Multiple technologies
• OpenvSwitch, Linux Bridge, Vendor plugins
– Floating IP
addresses
• Assigning fixed external IPs to VMs
Modules
• Block Storage - Cinder
– Persistent block storage,
named “Volumes”
– Multiple storage backends
• eg LVM, RBD, Gluster, Sheepdog,...+ more...
– Exposed to compute host via iSCSI:
• Internet Small Computer System Interface
• Internet protocol (IP)-based storage networking standard for linking data storage facilities
• Carry SCSI commands over IP netowrks
Modules
• Identity Management - Keystone
– Single point of integration for
OpenStack policy, catalog, token and
authentication
– Central authentication mechanism
– Directory of users
– Supports user and system policies
– Catalog of OpenStack services
– Can interact with (among others)
• LDAP
• Oauth
• PAM
Modules
• Object Storage - Swift
– Redundant storage system
for static data
• VM Images
• Backups
• Archives
– Data replication and distribution
• Multi-datacenter distribution
– Horizontally scalable
• Add more storage servers if needed – Similar to amazon S3
Modules
• Object Storage - Swift
– Stores and servers objects
(files)
• Accessed via API or HTTP requests
– Employs object level
replication to safeguard data
– Maintains distributed account and
container databases
Modules
• Image Service - Glance
– Support for VM images
• Discovery
• Registration
• Activation
• Snapshots/Backup
– Write-once, read-many storage
of images
– Interacts with backend storage
• OpenStack Object storage
– Image copied on use by Nova
Modules
• Image Service - Glance
– Supports most common
formats
• RAW, VHD, VDI, VMDK, qcow2, OVF, …
– Metadata properties
• eg specify virtual hardware preferences
Modules
• Dashboard - Horizon
– Django application that users can
access in their web browser
– Web UI for administrators
– Manage projects, users, and resources
– Communicates
with each
OpenStack
service via
their API
Modules
• Metering - Ceilometer – Billing and monitoring
• Orchestration - Heat – Template based
• And many more…
Nova
• Main, central component
– VM scheduling – VM creating/migration/
termination/…
Nova Architecture
• Concepts – Instances – Flavours – Virt drivers – Security group
• Dual APIs – OpenStack REST – EC2 compatible
REST
Nova Scheduler
• Places instances on compute hosts
• Filter Scheduler: Pluggable filtering rules – CPU model / architecture
– Virtualization type – PCI device availability
– CPU, RAM, Disk usage
– +more...
Nova Compute
• The nova-compute process is primarily a worker daemon that creates and
terminates virtual machine instances via hypervisor’s APIs
• The process by which it does so is fairly complex but the basics are simple:
– Accept actions from the queue – And then perform a series of system
commands (like launching a KVM instance) to carry them out while updating state in the database (through nova-conductor)
Instance Boot Instance Boot
Instance Boot Instance Boot
Instance Boot Instance Boot
Agenda
1. Private Clouds
2. Introduction to OpenStack – What is OpenStack?
– Architecture and Main components – Demo: basic commands
OpenStack Demo
Basic commands A few commands to:
• List VMs, volumes, network, flavors
• Create keys, volumes and networks
• Create VMs
• Attach/Detach volumes to VMs
• Attach floating IPs to VMs
• Stop/Remove VMs
• Live-migrate VMs
OpenStack demo
List existing
• VMs
• Volumes
• Networks
51
OpenStack demo
52
Flavor list Flavor attributes
OpenStack demo
Create and show keys
OpenStack demo
Create volumes
OpenStack demo
Create virtual networks
55
OpenStack demo
Create virtual networks
56
OpenStack demo
Create VMs
57
OpenStack demo
Network connectivity and floating IPs
OpenStack demo
Network connectivity and floating IPs
59
OpenStack demo
Attach/Detach Volumes
60
Upcoming
• Next (and last) lecture
– Cloud Application Development #2 (2*45 min) – Course summary for exam (45 min)
• Exam 2015-03-03
– Register no later than 2015-02-21 (tomorrow!) – 13-17, lecture room N450
– Old example exam to be added to course web
• Assignment 3 (project) to be posted today.
3 deliverables:
– (Short) project plan. Due: 2015-02-27 – Presentation and live demo. Due: 2015-03-23 – Project Report. Due 2015-03-27