Alessandro Pilo- CEO Cloudbase Solu.ons
@cloudbaseit
OpenStack & Hyper-V
Cloudbase Solutions
• Company started in Italy as .Net / Linux interop dev and consulting
• Branch started in Timisoara in 2012 to hire devs and provide colocation services
• Started using OpenStack for the internal IDC
• Added Hyper-V support
• OpenStack business exploded J
Agenda
• OpenStack + Hyper-V integration
• FreeRDP HTML5 dashboard
• OpenVSwitch on Windows
• Cloudbase-Init
• Windows Server 2012 Eval Image
• Crowbar integration
• Cross Cloud Migration
• Havana plans
Hyper-V 2012 availability
• FREE edition
– Full Hypervisor
– Minimum OS support
• Minimum impact on security updates, etc
• Windows Server 2012
– Just enable the related role
• Windows 8
– For workstation / testing / development usage
Hyper-V / Windows
Openstack components
• Nova Compute driver
• Quantum plugin
• Cinder Volume driver
• Windows Cloud-Init
• Ceilometer Agent (Havana)
Hyper-V Nova Compute
• Python Application installed as an
application/service on the Hyper-V node.
• OpenStack compute utilizes key features baked into the Hyper-V Virtualization
Platform
• Does not require windows clustering services
• Does not require shared storage
Hyper-V Nova Compute
• Supported features (anything missing? J)
– Launch / reboot / terminate – Resize
– Pause / unpause – Suspend / resume
– Attach / detach volume – Live migration
– Snapshot
– Guest / host info – Glance integration – Config Drive
– Rescue (Havana)
– RDP console (instead of VNC)
• Deprecated Nova network is not supported beyond “flat”
networking
– All Hyper-V networking features are part of Quantum
Volume Attach/Detach
Requirements
• Windows iSCSI initiator service enabled and running on hyper-v compute node.
• Configuration can be scripted
• Create, attach detach volumes using the
OpenStack Dashboard or Nova commands.
• OS must be installed directly onto the created volume for “boot from volume”
instances
Live Migration
Requirements
• Each compute node must have Active Directory Domain Membership
• Nova-compute service must be running
under an Active Directory Domain account.
• Shared nothing live migration can be
enabled via simple powershell commands or from the Hyper-V management GUI.
• Live Migration initiated via nova live- migration commands
Resize / cold migration
• Resize / cold migration are supported in Grizzly
• The root VHD is resized to the size specified by the flavor
• CoW VHD disks are automatically merged with the base disk as VHD differencing
disks cannot be resized.
– This will not be required by VHDX (Havana)
Glance Integration
• Native glance client support on Windows.
• Upload images directly from the windows compute node
• VHDx support planned for Havana
Quantum
• Hyper-V plugin is part of Quantum since Jan 2013
• Supported network types:
– VLAN – Flat – Local
– NVGRE (Havana)
• Plugin / agent model
– Plugin runs in quantum-server (controller) – Agent runs on each Hyper-V compute node
Quantum OVS interop
• Quantum plugin / agent AMQP RPC protocol is compatible with OpenVSwitch!!
• You can use the OVS plugin with Hyper-V agents (or vice versa)
– Limited to compatible L2 protocols: Flat / VLAN
• You can use the L3 and DHCP agents with the Hyper-V plugin
– Using the OVS L2 agent on the networking nodes
• Great for interoperability (e.g. KVM + Hyper-V)
Hyper-V OpenVSwitch!
We officially started porting
OpenVSwitch to Windows!!
Why OpenVSwitch?
• De facto standard for SDN
• Great interoperability
– Hyper-V / KVM / etc
• OpenFlow
• Tunnelling
– VXLAN – GRE
• Quantum OVS plugin
– No need for a different one
OpenVSwitch porting
• Porting Posix calls to Windows API
• Same CLI as on Linux
• Open Source
• Windows kernel drivers
– NDIS Hyper-V extensions
• VXLAN, GRE
– Best performance
• Support for Hyper-V 2012 and above
OpenVSwitch
Windows Kernel drivers
Typical Quantum setup
L2 Agent Nova Compute
Network Node
Data Network VLAN Compute Node
Quantum Server
Keystone Nova Services
...
L3 Agent
L2 Agent DHCP Agent Controller Node
Management Network
TenantCNet2 10.0.1.0/24 VLAN ID:3
TenantANet1 10.0.0.0/24
VLAN ID:1 External
Network 30.0.0.0/24
eth0
TenantANet1 10.0.0.0/24
VLAN ID:2
Controller
ENABLE_TENANT_VLANS=True TENANT_VLAN_RANGE=2000:3000 PHYSICAL_NETWORK=physnet1 OVS_ENABLE_TUNNELING=False MULTI_HOST=True
Q_USE_SECGROUP=False disable_service n-net
disable_service n-cpu
SCREEN_LOGDIR=$DEST/logs/screen enable_service q-svc
enable_service quantum
DATABASE_PASSWORD=Passw0rd RABBIT_PASSWORD=Passw0rd SERVICE_TOKEN=Passw0rd
SERVICE_PASSWORD=Passw0rd ADMIN_PASSWORD=Passw0rd
Network server
HOST_IP=192.168.209.196
ENABLED_SERVICES=quantum,q-agt,q-dhcp,q-l3,q-meta SERVICE_HOST=192.168.209.195
MYSQL_HOST=$SERVICE_HOST RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292 SCREEN_LOGDIR=$DEST/logs/screen
OVS_BRIDGE_MAPPINGS=physnet1:br-eth1 OVS_PHYSICAL_BRIDGE=br-eth1
PUBLIC_INTERFACE_DEFAULT=eth2 GUEST_INTERFACE_DEFAULT=eth1 SERVICE_TOKEN=Passw0rd
SERVICE_PASSWORD=Passw0rd ADMIN_PASSWORD=Passw0rd OVS_ENABLE_TUNNELING=False RABBIT_PASSWORD=Passw0rd
Compute KVM
HOST_IP=192.168.209.197
ENABLED_SERVICES=n-cpu,quantum,q-agt,n-novnc,n- xvnc,n-cauth
SERVICE_HOST=192.168.209.195 MYSQL_HOST=$SERVICE_HOST RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292 Q_HOST=$SERVICE_HOST
SCREEN_LOGDIR=$DEST/logs/screen
OVS_BRIDGE_MAPPINGS=physnet1:eth1 GUEST_INTERFACE_DEFAULT=eth1
SERVICE_TOKEN=Passw0rd
SERVICE_PASSWORD=Passw0rd ADMIN_PASSWORD=Passw0rd RABBIT_PASSWORD=Passw0rd
VLAN drivers issues
• BEWARE: Some Windows NIC drivers disable VLAN access by default!
• HKEY_LOCAL_MACHINE\SYSTEM
\CurrentControlSet\Control\Class\{4d36e972- e325-11ce-bfc1-08002be10318}
• Look in all the child keys xxxx, e.g. 0001, 0002 etc for a value "VLanFiltering". Make sure that if
present it's set to 0.
• In case of changes, reboot the server or restart the corresponding adapters.
Dashboard integration
• Hyper-V uses RDP for accessing the console instead of VNC
• By default it accepts connections on port 2179
– Not the RDP connection on 3389!
– Access to any guest: Windows, Linux, FreeRDP, etc
• The VM id is provided as part of an RDP protocol additional buffer called PCB (Pre Connection Buffer)
• Authentication is performed against the host, not the guest!
FreeRDP
• FreeRDP is an amazing RDP open source client.
– Apache 2 license – Multiplatform
• Linux
• Mac OS X
• Windows
• It provides support for the PCB parameter required by Hyper-V
• We are using it as a standalone client on the free Hyper-V server and on Linux, Mac OS X
FreeRDP-WebConnect
• An HTML5 RDP client based on canvas and websockets
– Similar to NoVNC
• Uses a C++ demon (wsgate) to handle
websocket connections and connect to the RDP server
– Better performance compared to the Python proxy used by NoVNC
• We provided support for Hyper-V
Dashboard integration
• We extended Horizon to provide support for
multiple protocols (not only VNC) based on the compute node host requirements
• wsgate has been extended with a “plugin”
based architecture
• Tokens validated with nova-consoleauth (like NoVNC does) to get the connection parameters
– Hyper-V host, port and VM id
• Python runtime executed in our C++ wsgate plugin
– Better performance
Nova Compute Installer
• Independent Python environment to avoid conflicts with existing applications
• Installs and registers all the required dependencies
• Generates dynamically a nova.conf file based on the parameters provided by the user
Nova Compute Installer
• Creates a new Hyper-V external switch if required
• Registers nova-compute as a service and starts it
• Registers quantum-hyperv-agent as a service and starts it (optional)
• Enables the Microsoft iSCSI Initiator service (optional)
Nova Compute Installer
• Enables and configures Hyper-V Live Migration (optional )
• FreeRDP for Hyper-V console access
• Installs a command prompt shortcut in the applications menu for a ready made
OpenStack prompt (optional)
• Can be executed fully unattended and automated
Unattended mode
• Can be installed in fully unattended mode, great for:
– Chef, Puppet, Group Policies…
• msiexec /i HyperVNovaCompute.msi /qn /l*v log.txt
ADDLOCAL=HyperVNovaCompute,QuantumHyperVAgent,iSCSI SWInitiator,OpenStackCmdPrompt INSTALLDIR=C:\OpenStack
\Nova GLANCEHOST=glancehost GLANCEPORT=9292 RABBITHOST=rabbithost RABBITPORT=5672
RABBITPASSWORD=12345678
NOVASQLCONNECTION=mysql://user:password@host/nova INSTANCESPATH=C:\HyperV ADDVSWITCH=0
VSWITCHNAME=external1 LIMITCPUFEATURES=”"
USECOWIMAGES=1 LOGDIR=C:\log ENABLELOGGING=1
VERBOSELOGGING=1 QUANTUMURL=http://quantumhost:9696 QUANTUMADMINTENANTNAME=service
QUANTUMADMINUSERNAME=quantum QUANTUMADMINPASSWORD=12345678
QUANTUMADMINAUTHURL=http://keystonehost:35357/v2.0
Nova Compute Installer
From our web site J
Windows Cloud-Init
• Cloudbase-init (aka Windows Cloud-Init)
– Similar to the Linux cloud-init – License: Apache 2
• Porting Cloud-Init to Linux was not possible
– It requires some major refactoring
• Supported data sources:
– OpenStack HTTP metadata – ConfigDrive V2
– EC2 HTTP metadata (contributed by Mirantis)
Windows Cloud-Init
• Plugin based architecture
• Supported plugins:
– SetHostNamePlugin – CreateUserPlugin
– NetworkConfigPlugin
– SetUserSSHPublicKeysPlugin – UserDataPlugin
– ResizeFSPlugin
Userdata format
• Windows Batch
• Powershell
• Bash
• Multi-part (Havana)
– Heat support
Password generation
• New Grizzly feature
1. Password is randomly generated
– Can be forced to the value injected by Nova
2. Password in encrypted with the SSH public key
3. Password is POSTed to the metadata service 4. Password can be retrieved by the client and
decrypted with the SSH private key
– nova get-password vm1 .ssh/id_rsa
Windows Installer
Unattended mode
• Like all our installers, it can be executed in fully unattended mode:
• msiexec /i CloudbaseInitSetup.msi /qn / l*v log.txt
NETWORKADAPTERNAME="Intel(R) PRO/1000 MT Network Connection"
OpenStack Windows
Server 2012!!
OpenStack WS2012
• WS 2012 Standard - Evaluation edition
– Testing only!!
• Available for:
– Hyper-V – KVM
– XenServer / XCP
• www.cloudbase.it
OpenStack WS2012
• Complete with:
– Drivers / tools:
• VirtIO / XenServer Tools
– Cloudbase-Init – Sysprepped
• Eval edition can be upgraded with a simple uder_data script:
– DISM /online /Set-Edition:ServerStandard /
ProductKey:XXXXX-XXXXX-XXXXX-XXXXX-XXXXX / AcceptEula
– Make sure to respect the eval license!!
Cinder Windows installer
• Cinder includes a driver for Windows Storage Server 2012
• We created an installer that deploys a
complete Python environment with all the required dependencies
• Registers and starts cinder-volumes
• Key community contributor:
– Pedro Navarro Perez (Tx! )
Crowbar
• Big part of Dell’s cloud strategy
• Bare-metal deployment
• vendor independent
• PXE booting
– Sledgehammer image boots and gets configuration
• Provisioning via Crowbar web site
• Barclams
– Configuration – Chef recipes
Crowbar + HyperV
Cross Cloud migration
• Vendor lock-in anybody? J
• Just a few clicks to migrate virtual workloads to and from:
– Amazon AWS – Azure
– OpenStack
• Including RackSpace
– VMWare vSphere – MS System Center
• Currently in beta
Havana!!
• Planning for Havana now!
– More Quantum: NVGRE, security groups, etc – Ceilometer Hyper-V agent
– Heat Windows Cloud-Init support – VHDX support
– Ephemeral storage
– Support for more image formats (e.g. AMI/ARI/AKI) – Nova rescue
– Active Directory Keystone scalability – VDI support
– Fibre channel
– VM dynamic memory support – Cinder improvements…
Resources
• Cloudbase Solutions
– http://www.cloudbase.it – @cloudbaseit
• Hyper-V
– http://www.microsoft.com/en-us/server-cloud/
hyper-v-server/
