Virtual Networking with z/vm Guest LANs and the z/vm Virtual Switch

Virtual Networking with z/VM Guest LANs

and the z/VM Virtual Switch

Alan Altmark, IBM

Note

References to IBM products, programs, or services do not imply that IBM intends to make these available in all countries in which IBM operates. Any reference to an IBM product, program, or service is not intended to state or imply that only IBM's product, program, or service may be used. Any functionally equivalent product,

program, or service that does not infringe on any of the intellectual property rights of IBM may be used instead. The evaluation and verification of operation in

conjunction with other products, except those expressly designed by IBM, are the responsibility of the user.

The following terms are trademarks of the International Business Machines Corporation in the United States or other countries or both:

IBM IBM logo eServer zSeries z/VM z/OS DB2

Other company, product, and service names may be trademarks or service marks of others.

Topics

Guest LANs

Virtual Network Interface Card

Virtual Switch

What features are supported in what releases

Multi-DMZ Network

web

web web _web

internet

app app app

web _web db

A DMZ (demilitarized zone) is a subnet that insulates critical network components

Multi-DMZ Network on zSeries

web

web web _web

internet

app app app

web _web db

Multi-DMZ Network with Guest LANs

web web web web

app

app web

z/OS

DB2

LPAR 1 LPAR 2

z/VM

z/VM

PR/SM

Guest LAN vs. Virtual Switch

Guest Guest

Guest

Guest LAN

Ethernet LAN

Guest Guest

Virtual Switch

Virtual router is required

Different subnets

No virtual router

z/VM Guest LAN

A simulated LAN

Ethernet: IPv4 and IPv6 HiperSockets: IPv4

Unicast, Multicast, and Broadcast

No built-in connection to outside network

As many as you want

Owned by system or individual user

Is not a device - it is a system object

Created in SYSTEM CONFIG, directory, or by CP

Guest Guest Guest Guest Guest Guest Guest LAN #1 LAN #2

Why Guest LAN instead of Dedicated Hardware ?

Dedicated network connections may be best for some

environments:

When intense network activity is expected

When external connectivity is required

z/VM Guest LAN may be better for other environments:

When network hardware is limited

When multiple nodes are guests in the same z/VM host image

When network activity must be isolated from primary network

(e.g. test environments, student labs, application server access to

database servers)

Guest LAN Attributes

Name

Owner

Type

Permission

Maximum number of connections

Maximum frame size

Accounting

LAN Name and Owner

The LAN name is a simple 1-8 character token

The LAN owner is a VM user ID or “SYSTEM”

(name, owner) is unique within the system

Needed to connect (COUPLE) a NIC to the LAN

Notes about LAN ownership:

The LAN belongs to the system, not to the owner

A Class G LAN owner can modify the LAN access list

A Class G LAN owner can delete the LAN

System vs. User Guest LANs

System

owned by “SYSTEM”

Created by

–SYSTEM CONFIG –Class B DEFINE LAN

Modify or delete by Class B

SET LAN or DETACH LAN

User

owned by a specific z/VM user

Created by

–SYSTEM CONFIG

–Class B DEFINE LAN with OWNERID option

–Class G DEFINE LAN

Modify or delete by Class B or

Class G owner SET LAN or DETACH LAN

HiperSockets vs. QDIO LAN

TYPE HIPERsockets | QDIO

HiperSockets

Synchronous

Low latency

Slightly smaller path length in CP (less CP time)

OSA-Express in QDIO mode

Asynchronous

Higher latency than HiperSockets

Higher CPU cost

Unrestricted vs. Restricted LANs

Unrestricted

Any user can connect (couple) to this LAN

Hint: CP QUERY LAN can show you who is connected

Restricted

Only users in the access list can connect (couple) to this LAN

LAN owner uses CP SET LAN to GRANT or REVOKE access

CP QUERY LAN can show you the current access list

LAN MAXCONN

MAXCONN INFinite | nnnn

Represents the maximum number of simultaneous connections

permitted for this LAN

Decimal number 1-1024 sets a specific limit

INFINITE means no limit is defined for this LAN

When the MAXCONN limit is reached, subsequent COUPLE

commands issued by adapter (NIC) owners will fail

LAN MFS

MFS 16K | 24K | 40K | 64K

Simulates CHPID OS=

value

specification in IOCDS for

HiperSockets (TYPE=IQD) chpids

Does not apply to QDIO

Largest MTU specification = (MFS - 8K)

Hint:

If LAN is isolated, use large MFS and large MTU

If LAN has external gateway, use MFS 16K and match external

MTU (e.g. 1492)

LAN ACCOUNTING

ACCOUNTING ON | OFF

Accounting ON

Accounting is enabled for adapters connected to this

LAN

Directory options determine which guests:

–

NetAccounting enables general network accounting

–

NetRouter enables network accounting as a router

Hint: Both LAN and USER must be set to create

accounting records

Persistent vs. Transient LAN

Persistent / Transient is inferred from other attributes

Any LAN owned by user “SYSTEM” is persistent Any LAN created by SYSTEM CONFIG is persistent All other LANs are transient

A

persistent

LAN must be explicitly deleted by CP DETACH LAN

A

transient

LAN is automatically deleted when the last user uncouples

from the LAN

Setting Guest LAN defaults and limits

Set global VM LAN attributes in the SYSTEM CONFIG file:

VMLAN LIMit PERSistent INFinite|maxcount

VMLAN LIMit TRANSient INFinite|maxcount

VMLAN ACNT|ACCOUNTing SYSTEM ON|OFF VMLAN ACNT|ACCOUNTing USER ON|OFF VMLAN MACPREFIX 020000-02FFFF

VMLAN MACIDRANGE SYSTEM x-y [USER a-b] New for

Setting defaults and limits

Modify global guest LAN attributes with SET VMLAN:

Use CP QUERY VMLAN to see current values

No LAN is deleted by this command if you set limit < current (it just prevents creation of new LAN until current < limit)

CP SET VMLAN LIMit PERSistent INFinite|maxcount

CP SET VMLAN LIMit TRANSient INFinite|maxcount

CP SET VMLAN ACNT|ACCOUNTing SYSTEM ON|OFF CP SET VMLAN ACNT|ACCOUNTing USER ON|OFF

Create a Guest LAN at system IPL

Automated with SYSTEM CONFIG file statements:

DEFINE LAN

name

_[OWNERid

ownerid

_]

[TYPE HIPERsockets|QDIO]

[MAXCONN INFinite|

nnnn

_]

[MFS 16K|24K|40K|64K]

[ACCOUNTing ON|OFF]

[UNRESTricted|RESTricted]

[GRANT

userlist

_]

Examples:

DEFINE LAN HIPER1

DEFINE LAN DELTA TYPE QDIO

Grant Guest LAN permission at IPL

Specify after DEFINE LAN statement in SYSTEM CONFIG to

add users to access list

MODIFY LAN name

[OWNERid ownerid_]

[GRANT userid] Example:

DEFINE LAN HIPER1 OWNER SYSTEM RESTRICTED MODIFY LAN HIPER1 OWNER SYSTEM GRANT LNX01 MODIFY LAN HIPER1 OWNER SYSTEM GRANT LNX02

Create a Guest LAN dynamically

Interactive with CP DEFINE LAN commands:

CP DEFINE LAN

name

_[OWNERid

ownerid

_]

[TYPE HIPERsockets|QDIO]

[MAXCONN INFinite|

nnnn

_]

[MFS 16K|24K|40K|64K]

[UNRESTricted|RESTricted]

[ACCOUNTing ON|OFF]

Examples:

CP DEFINE LAN HIPER1 OWNER SYSTEM RESTRICTED CP DEFINE LAN DELTA TYPE QDIO

Modify a Guest LAN dynamically

Modified by CP SET LAN commands:

CP SET LAN

name

[OWNERid

ownerid

_]

[ACCOUNTing ON|OFF]

[GRANT userid]

[REVOKE userid]

Examples:

CP SET LAN HIPER1 OWNER SYSTEM ACCOUNTING OFF CP SET LAN CSC201 OWNER TCPMAINT GRANT LNX01

A simulated network adapter

OSA-Express QDIO HiperSockets

3 or more devices per NIC

More than 3 to simulate port sharing on

2nd-level system

Provides access to Guest LAN or Virtual Switch

Created by directory or CP DEFINE NIC

command

Virtual Network Interface Card (NIC)

Virtual Machine

Virtual NIC - User Directory

May be automated with USER DIRECT file:

SPECIAL

vdev

_{{HIPERs|QDIO} [}

devs

_[

owner name

_]]

NICDEF

vdev

_{[TYPE HIPERS | QDIO]}

[LAN

owner name

_]

[CHPID xx]

[MAC xxyyzz]

Example:

SPECIAL 1100 QDIO 3 SYSTEM SWITCH1

or

Virtual NIC - CP Command

May be interactive with CP DEFINE NIC and COUPLE

commands:

CP DEFINE NIC

vdev

_{[[TYPE] HIPERsockets|QDIO]}

[DEVices

devs

_]

[CHPID

xx

_]

CP COUPLE

vdev

_[TO]

owner name

Example:

CP DEFINE NIC 1200 TYPE QDIO

NIC Virtual Device Address

base virtual device address where this NIC is installed in your

virtual I/O configuration

A block of contiguous devices addresses is allocated to this

NIC, beginning with “vdev” (see DEVICES parameter)

One I/O subchannel ID is allocated to EACH I/O device

beginning with the first available subchannel

One virtual CHPID (Channel Path ID) is allocated for this virtual

NIC

NIC DEVICES parameter

DEVICES

devs

Specifies the (decimal) number of I/O devices created as part

of this NIC, starting with the specified “vdev”

The default (and minimum) is 3 devices:

Read-Control

Write-Control

NIC TYPE parameter

TYPE HIPERsockets or QDIO

NIC CHPID parameter

CHPID xx

Specifies the Channel Path ID number (in hex) to use for this NIC

Default is any available unused real CHPID number

Needed when guest manages CHPID numbers (e.g. z/OS)

Notes for z/OS configuration:

This is a virtual CHPID number (not a real hardware CHPID) It must be a CHPID number that is NOT in use by hardware Hint: An easy way to find out what CHPID is available is:

– LOGON the guest and “DEFINE NIC xxxx HIPER” – “CP QUERY VIRTUAL xxxx” to get SCHIB (ssss) – “CP DISPLAY SCHIB ssss” to see the CHPID

What’s a ‘switch’ anyway?

•It’s a box that you plug cables in to create a LAN

•Cable/DSL multi-port and wireless switches for home are simplified versions of commercial switches (less configurable)

•It has its own IP address for management purposes

2 1

3

T 1 1

1 1 1 2 2 3 2

z/VM Virtual Switch

A special-purpose Guest LAN

Ethernet IPv4

Built-in 802.1q bridge to outside network

Same subnet as OSA connection

Each Virtual Switch has up to 3 separate

OSA-Express connections associated

with it

Created in SYSTEM CONFIG or by

guest guest

CP

Router AIX

Virtual Switch Attributes

Name

Associated OSAs (maximum 3)

A controlling virtual machine (VM TCP/IP stack server)

Controller not involved in data transfer Do not ATTACH or DEDICATE

User needs IUCV *VSWITCH authorization

User needs VSWITCH CONTROLLER statement in PROFILE TCPIP

Similar to Guest LAN

Owner SYSTEM Type QDIO

Create a Virtual Switch at system IPL

Automated with SYSTEM CONFIG file statements:

DEFINE VSWITCH

name

[RDEV NONE | cuu [cuu [cuu]] ]

[CONNECT | DISCONNECT]

[CONTROLLER * |

userid

_]

[NONROUTER | PRIROUTER]

Example:

Modify a Virtual Switch at IPL

Specify after DEFINE LAN statement in SYSTEM CONFIG to

add users to access list

z/VM 4.4 supports “VLAN ANY”, but don’t use it

MODIFY VSWITCH

name

_GRANT

userid

[VLAN vid1 vid2 … ]

Example:

CP

IEEE VLAN support

guest guest guest guest

•VLAN creates multiple logical LAN segments on a single physical LAN segment

•Multiple VLANs are assigned to OSA vswitch

Multi-DMZ Network on zSeries - Reloaded

web

web web _web

internet

app app app

web _web db

Multi-DMZ Network on zSeries with outboard firewall

web

web web _web

app app app

Multi-DMZ Network with VSWITCH (A)

web web web web

web

z/OS

DB2

LPAR 1 LPAR 2

z/VM

VSWITCH 2

z/VM

app

app app

Multi-DMZ Network with VSWITCH (B)

web web web web

web

z/OS

DB2

LPAR 1 LPAR 2

VSWITCH

z/VM

app

app app

Network Configuration

In general, configure a Guest LAN network like any other network

Subnet routing

Use the VSWITCH whenever possible

Minimize number of VSWITCHes; exploit IEEE VLAN if you can

By having virtual and real configurations be the same, you can easily

test network configuration before deployment with real hardware

Built-in Diagnostics

CP QUERY VMLAN

to get global VM LAN information (e.g. limits) to find out if service has been applied

CP QUERY LAN ACTIVE

to find out which users are coupled

to find out which IP addresses are active

CP QUERY NIC DETAILS

to find out if your adapter is coupled to find out if your adapter is initialized

Support Summary for z/VM 4.2.0

HiperSockets NIC / LAN

HiperSockets IPv4 unicast data transfer

Required Service:

Apply CP PTF UM30225 (APAR VM62938)

Apply TCPIP PTF UQ61461 (APAR PQ51738)

Apply TCPIP PTF UQ65226 (APAR PQ60093)

Support Summary for z/VM 4.3.0

HiperSockets

IPv4 multicast accounting data

OSA-Express QDIO

IPv4 unicast, multicast, broadcast accounting data

Required Service:

Apply CP PTF UM30359 (APAR VM63085) Apply CP PTF UM30743 (APAR VM63261)

Support Summary for z/VM 4.4.0

Virtual Switch

IPv4 Ethernet

Requires OSA-Express QDIO

DEFINE VSWITCH and SET VSWITCH used to establish and

modify virtual switch settings

Virtual OSA-Express QDIO

IPv6

Coming in z/VM 5.1…

ESM control for all guest LANs and VSWITCHes, including

VLAN ID control

RACF: Class VMLAN, Profile owner.lanname or owner.lanname.vid All LANs and VSwitches are restricted

Layer 2 (MAC) communications

Fulfillment of Statement of Direction All types of traffic, not just IP

Virtual NIC MAC appears on network

VMLAN updates to allow specification of ranges used for automatic and

static MAC address assignments

Coming in z./VM 5.1…

IEEE 802.1q compliance changes

VLAN ANY is gone

VSWITCH can be defined as VLAN-aware (or not). Default is

“not”.

When a NIC couples to a VLAN-aware VSWITCH, it will be

assigned a PORTTYPE attribute

– ACCESS: VLAN tags not given to or accepted from guest – TRUNK: VLAN tags are given to and expected from guest

Default PORTTYPE comes from DEFINE VSWITCH

References

Publications:

z/VM CP Planning and Administration SC24-6043 z/VM CP Command and Utility Reference SC24-6008

z/VM TCP/IP Level 430 Planning and Customization SC24-6019 VSE/ESA Version 2 Release 7.0 Release Guide SC33-6713 Linux for S/390 SG24-4987

Linux on zSeries and S/390: ISP/ASP Solutions SG24-6299 zSeries HiperSockets SG24-6816

Linux on IBM eServer zSeries and S/390: TCP/IP Broadcast on z/VM Guest LAN

(Form REDP3596)

Contact Information

By e-mail:

[email protected]

In person:

USA 607.429.3323

On the Web:

http://ibm.com/vm/devpages/altmarka

Mailing lists:

[email protected]

[email protected]

[email protected]