TUT8155 Best Practices: Linux High Availability with VMware Virtual Machines

(1)

TUT8155

Best Practices:

Linux High Availability with

VMware Virtual Machines

Jeff Lindholm

SUSE®

Sr. Systems Engineer [email protected]

(2)

Agenda

SUSE® Linux Enterprise High Availability Extension 12 OS Level clustering use case

VMware configuration best practices

SUSE Linux Enterprise High Availibility Extension / Linux Clustering in VMware

SUSE Linux Enterprise Server - High Availability Cluster Demo

(3)

3

Challenge

SUSE® Linux Enterprise High Availability Extension

• Faults will occur

– Hardware crash, flood, fire, power outage, earthquake?

• Can you afford a service outage or worse, loss of data?

– You might afford a five second blip, but can you afford a longer outage?

• How much does downtime cost? Murphy's Law is Universal

(4)

Use Case: Linux Clustering

(5)

5

Version 12 –

Key Features

SUSE® Linux Enterprise High Availability Extension • Major code refresh to

latest upstream versions

• Pacemaker ‒ Object tagging

‒ Significant CIB performance • Cluster Shell:

‒ Health evaluation

‒ Improved error reporting

and syntax ‒ Support corosync configuration • hawk ‒ Improved wizards ‒ History explorer • Geo extension ‒ Improved algorithm

‒ Per-site attributes in CIB ‒ DNS-based IP fail-over • GFS2 now supported in

r/w mode

• New, additional

(6)

SUSE Linux Enterprise High

Availability Extension + VMware

• SUSE Linux Enterprise High Availability Extension

complements VMware host-level HA solution for mission critical applications

• Features

‒ Application level HA protects active memory contents

‒ Scripts for monitoring open source services (eg, Apache, MySQL,

NFS, PostgreSQL, Tomcat, KVM, Xen) and 3rd party applications

(eg, SAP, Oracle, IBM DB2, WebSphere)

‒ Policy-driven cluster resource

manager

‒ Cluster-aware file system and

volume management

‒ Continuous data replication

(7)

7

Optimized vSphere Guest Performance

• VMware tools and drivers integrated with SUSE Linux

Enterprise Server 12 for best out-of-the-box experience

‒ open-vm-tools: eliminates the need to separately install VMware Tools

and reduces operational expenses and virtual machine downtime

‒ vmware_balloon: physical memory management driver ‒ vmw_vmci, vmw_vsock: provide for fast and efficient

communications between guest virtual machines and hypervisors

‒ vmxnet3: next generation of a paravirtualized NIC

designed for performance

‒ vmw_pvscsi: driver for paravirtualized SCSI device which improves

disk performance

‒ vmwgfx: kernel driver for 3D graphics

(8)

Example: SAP HA Cluster Interface

Interfaces to integrate our HA solution in SAP

(9)

9 pacemaker/OpenAIS SAPDatabase SAPInstance FS Ext3 LVM MD-Raid Multipath Network Bonding Network

Linux system / Kernel

Oracle SAP IP FS

IPAddr2 SBD MD LVM

Example: HA Stack for SAP

Application Layer

Resource Agent Layer Cluster Layer

I/O Layer

(10)

VMware HA and SUSE Linux

Enterprise High Availability Extension

zz

VMware ESX VMware ESX (VMware ESX)

VMware HA and DRS Cluster SUSE Linux Enterprise

High Availability Extension

DB OS APP SCS OS DB OS APPOS DB OS APPOS APPOS

* Both SLE HA Nodes running on ESX server 1 * ESX Server 3 is powered down

(11)

SUSE Linux Enterprise 11/12

VMware Virtual Machines

(12)

Optimized vSphere Guest Performance

• VMware tools and drivers integrated with SUSE Linux

Enterprise Server 12 for best out-of-the-box experience

‒ open-vm-tools: eliminates the need to separately install VMware Tools

and reduces operational expenses and virtual machine downtime

‒ vmware_balloon: physical memory management driver ‒ vmw_vmci, vmw_vsock: provide for fast and efficient

communications between guest virtual machines and hypervisors

‒ vmxnet3: next generation of a paravirtualized NIC

designed for performance

‒ vmw_pvscsi: driver for paravirtualized SCSI device which improves

disk performance

‒ vmwgfx: kernel driver for 3D graphics

(13)

13

Virtual Disk Configuration

VMware Para-virtual SCSI drivers (vmw_pvscsi) are included with SUSE Linux Enterprise Server 11 and 12 Para-virtual SCSI drivers are recommended when

(14)

Virtual Network Configuration

VMware vmxnet3 network drivers are default, recommended, and built into both SUSE Linux Enterprise Server 11 and 12

(15)

VMware / SUSE Linux Enterprise

High Availability Best Practices

(16)

SUSE Linux Enterprise High Availability Extension

Clustering with VMware

• SUSE Linux Enterprise High Availability Extension on

VMware is supported by SUSE

• Fencing is accomplished by Stonith Block Device (SBD) • Unicast heartbeat configuration is recommended for two

node configurations

• Mixed physical and virtual cluster nodes are supported

• Shared Storage using SCSI Raw Device Maps to VM ‒ Or

-• VMFS Datastore with simultaneous write

(17)

17

The Dos and Don'ts

Things you should consider

Keep cluster configuration simple

Use SBD for node fencing (STONITH)

Define and perform tests for all failure scenarios Follow our best practices

(18)

The Dos and Don'ts

Things you should avoid

Build Cluster cluster without node fencing (STONITH)

Go live without tests planned and done Go live without proper operations manual Cluster resource (like SBD and STONITH) timings shorter than SAN timings

(19)

19

Considerations for SBD / Shared

Storage on VMware ESXi datastores

• Disable Simultaneous write protection for shared disk

devices: (multi-writer flag)

‒ http://kb.vmware.com/kb/1034165

• Enable by-id disk presentation inside the virtual

machine:

‒ Add disk.EnableUUID = "TRUE" to cluster node .vmx config files

• Enable softdog module for SBD operation in boot.local

prior to initial cluster setup / installation: (each node) echo 'modprobe softdog' >> /etc/init.d/boot.local

(20)

Multi-writer Flag Supported and

Unsupported Actions or Features:

(21)

21

Other multi-writer Limitations

• When using the multi-writer mode, the virtual disk must be

eager zeroed thick; it cannot be zeroed thick or thin provisioned. For more information, see A virtual machine fails to power on with the error: Thin/TBZ disks cannot be opened in multiwriter mode. VMware ESX cannot open the virtual disk for clustering.

(1033570). http://kb.vmware.com/kb/1033570

• Sharing is limited to 8 ESXi/ESX hosts with VMFS-3 (vSphere

4.x) and VMFS-5 (vSphere 5.x) in multi-writer mode. On ESXi 5.x with VMFS-5, you can still share the virtual disks with 32 hosts for read-only access (that is, for View, linked clone, and fast provisioning use cases)

• Hot adding a virtual disk removes Multi-Writer Flag. For more

information, see Hot adding a virtual disk in ESXi 5.5 removes the multi-writer flag (2078540). http://kb.vmware.com/kb/2078540

(22)

Multi-Writer Configuration

(23)

23

Multi-Writer Configuration

(24)

Multi-Writer Configuration

Add the multi-writer setting for each virtual disk that you want to share. For example, to share four disks, the

configuration file entries look like this: scsi1:0.sharing = "multi-writer"

scsi1:1.sharing = "multi-writer" scsi1:2.sharing = "multi-writer" scsi1:3.sharing = "multi-writer"

Edit the vmx configuration file or change the

configuration parameters using the vSphere client or web administration

(25)

25

Disabling Simultaneous Write

Protection on VMware ESXi

• Cluster ready storage configuration and disk

management are REQUIRED to avoid multiple nodes concurrently mounting shared storage on boot

• To disable auto-activation of cluster / shared storage

volumes on boot – disable boot.lvm and/or edit

/etc/sysconfig/lvm to specify what LVM volume groups are activated at boot vs. activated by the cluster

software

• Optional – The OCFS2 file system includes a

distributed lock manager and will safely allow multiple cluster nodes to concurrently block mount shared

storage (Max 32 nodes are supported by SUSE, Max 8 nodes RW limitation per VMware)

(26)

zz

* Both SLE HA Nodes running on ESX server 1 * ESX Server 3 is powered down

VMware HA and SUSE Linux

(27)

27

zz

* VM is migrated to ESX server 2 without * SLE HA cluster interference

APP SCS

OS

vMotion

VMware HA and SUSE Linux

(28)

zz

DB OS APP SCS OS DB OS APP OS DB OS APP OS APP OS

* SLE HA cluster now runs on different ESX servers to have HA against

Hardware failures...

VMware HA and SUSE Linux

(29)

29

zz

* SLE HA cluster now runs on different ESX servers to have HA against

Hardware failures...

VMware HA and SUSE Linux

(30)

zz

DB OS SCS DB OS DB OS APP OS DB OS APP OS APP OS

* ... This was just in time, because Unfortunately a ESX hardware system fails * SLE HA migrates the Database and

optionally shutdown an Application Server

VMware HA and SUSE Linux

(31)

31

zz

(VMware ESX) VMware ESX VMware ESX

SCS DB

OS APPOS APPOS APPOS

* ESX server 1 is now in hardware Maint. * VMware DPM powers up ESX server 3 * Failed Virtual Machines get started by VMware HA

OS

VMware HA and SUSE Linux

(32)

zz

SCS DB

* One of the virtual machines with an SAP application server is migrated to ESX server 3

* SLE HA starts the SAP application Server on the cluster node

APP OS

vMotion APPOS

VMware HA and SUSE Linux

(33)

33

zz

SCS DB

* Migration is ready with complete business continuity

APP OS APP

OS

VMware HA and SUSE Linux

(34)

How Do We Do This?

Learn High Availability

(35)

35

Why Invest in SUSE with VMware ?

• Alliance partnership for 10+ years • Joint certification and support

• Integrated VMware tools and drivers • Supported in VMware public cloud

• Supported for OpenStack private clouds

• Recommended for SAP virtualized on VMware

• SUSE Linux Enterprise High Availability Extension

complements VMware HA for mission-critical virtualized environments

(36)

Start Now

• Visit the SUSE-VMware Alliance website at

https://www.suse.com/partners/alliance-partners/vmware/

‒ Solution briefs ‒ White papers ‒ Case studies

• Download SUSE Linux Enterprise Server:

https://www.suse.com/products/server/eval.html

• Download SUSE Linux Enterprise High Availability

Extension:

https://www.suse.com/products/highavailability/

(37)

Corporate Headquarters Maxfeldstrasse 5 90409 Nuremberg Germany +49 911 740 53 0 (Worldwide) www.suse.com Join us on: www.opensuse.org 37

(38)

This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE.

Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.

General Disclaimer

This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole

discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.