Alexander Paul IBM Certified Advanced Technical Expert (C.A.T.E.) for Power Systems Certified Cisco Systems Instructor CCSI #32044

37 

Loading....

Loading....

Loading....

Loading....

Loading....

Full text

(1)

Network Virtualization

Deep dive and Network Troubleshooting in a

virtualized Environment

Alexander Paul

paulalex@de.ibm.com

IBM Certified Advanced Technical Expert (C.A.T.E.) for Power Systems

Certified Cisco Systems Instructor CCSI #32044

(2)

Physical networking

I‘m a physical

switch.

(3)

Physical networking

A physical network consists of real hardware devices with

embedded logic:

– Switches

– Routers

– Network interface cards

– Cable

[…]

Physical network devices have their own operating

environment.

Physical network devices are independently manageable via…

– a console port

– or an ip based management interface.

(4)

Hypervisor based Virtual Networking

Virtual networks are…

– …in software realized networks.

– …a consolidation of networking logic.

– …minimizing the need for physical links.

– …centrally supervised.

– …dependent on a central operating environment.

Benefits of network virtualization?

– Decreases the time spent by cabling physical servers.

– The number of adapters, switchports, wires… can be reduced.

– Platform for low latency in-the-box communication

(5)

Virtual Ethernet

Virtual Ethernet

– Standard technologie in near

all host virtualization products.

– Hypervisor implemented

layer 2 switch.

– In-box packet delivery by

memory-to-memory copy

– Generated MAC addresses

Client 1

Hypervisor

Client 2

Virtual I/O Server

ent0 (Vir) en0 (if) ent0 (Vir) en0 (if) Ent0 (Phy) Shared Ethernet Adapter ent1 (Vir)

VLAN-Aware Ethernet Switch

Ethernet Switch

(6)

8 Bit Slot ID 20 Bit Random 20 Bit CEC ID

Virtual Ethernet - MAC Address calculation

ent0

(Virt)

HMC

Create

Virtual Ethernet Adapter

Calculating MAC Address

7 2 : E C : F C : F 5 : B 6 : 0 B

LPAR

ABCDEF123456

Frame is blocked by

Hypervisor

(7)

H

y

p

e

rv

is

o

r

Virtual Ethernet Performance

Virt. Eth. Virt. Eth. LPAR 1 LPAR 2 PVID 1 PVID 1 Traffic direction

Jumbo Frames

879 Mbits/sec

Virtual Ethernet scales with processor entitlement.

MTU=1500

270 Mbits/sec

(8)

VPD card 2 x 1Gb Eth Serial 2 Serial 1 Base Offering: #5636

2 Port 1 Gb

VPD card Serial 2 10Gb Eth 10Gb Eth 10Gb Upgrade Offering: #5637

2 Port SX 10 Gb

VPD card 4 x 1Gb Eth Serial 2 4 x 1Gb Upgrade Offering: #5639

4 Port 1 Gb

Integrated Virtual Ethernet

Physical adapter with

virtualization capabilities

No hypervisor work for

frame bridging

Network virtualization

without the need of

hypervisor bridging

Removes software packet

forwarding overhead from

hypervisor

Provides low latency, low

cpu consuming in-box

communication

(9)

IVE Logical Components Diagram

H

y

p

e

rv

is

o

r

AIX 1 lhea0 ent0 lphea lhea1 ent1 lphea AIX 2 lhea0 ent0 lphea lhea1 ent1 lphea AIX 3 lhea0 ent0 lphea lhea1 ent1 lphea HEA Logical Ports (LHEA) Physical Port Physical Port

Virtual Layer 2 Switch Virtual Layer 2 Switch en0 (if) en1 (if) en0 (if) en1 (if) en0 (if) en1 (if)

(10)

IVE System Architecture

Low Latency Design

– GX+ bus attachment

– Immediate data in descriptors (reduced memory access)

– Direct user space per-connection queuing (OS bypass)

– Up to 3X throughput improvement over current 10 Gbps

solutions

– Additional acceleration functions to reduce host code path

length.

– Provides direct I/O virtualization support

– Allows 10 Gbps port to replace up to 10 dedicated PCI 1

Gbps adapters in a partitioned system

IVE offers the following virtualization functions

– Sixteen MAC addresses are assigned to each IVE port group

– Each logical port can be owned by a separate LPAR

– Direct data path to LPAR

– Default send and receive queues per LPAR

– Ethernet MIB and RMON counters per LPAR

– VLAN filtering per logical port (4096 VLANs * 32 Logical

Ports)

– Internal layer 2 switch for LPAR to LPAR data traffic

– Multicast / Broadcast redirection to Multicast / Broadcast

manager

System

Memory

POWER6

Chip

P 5 IO C 2

IVE

GX interface

2 x 10 Gbps or

4 x 1 Gbps Ethernet

(11)

Throughput Benchmark Host Ethernet Adapter

T1

T2

T3

T4

HEA MCS 1 HEA MCS 4

T1

T2

T3

T4

Virt. Eth. HEA MCS 1 HEA MCS 4 Virt. Eth. HEA MCS1 HEA MCS 4 Virt. Eth. HEA MCS 1 HEA MCS 4 Virt. Eth. p570 no 1 p570 no 2

Quad Port

IVE

Quad Port

IVE

LPAR 1

AIX 6.1 SP 1 EC 0.3, capped

LPAR 2

LPAR 1

LPAR 2

AIX 6.1 SP 1 EC 0.3, capped

(12)

Throughput benchmark results

0

100

200

300

400

500

600

700

0

10

20

30

40

50

TCP sessions T h ro u g h p u t [M b it /s ]

(13)

Virtual LANs

Each VLAN appears as a

independent and isolated network.

Each VLAN represents a dedicated

security domain.

VLAN membership is controlled

from a central point and is

transparent to the client.

Broadcast traffic affects only

clients within the same VLAN.

(14)

VLAN trunking

VLAN

trunk

VLAN 1

VLAN 2

Link enablement to transfer

multiple VLAN traffic

through a single port.

In outgoing direction each

frame must be labelled with

its VLAN membership.

VLAN trunks are good for…

– …Switch uplinks

– …connecting to Routers

– …connecting to Firewalls

– …connecting to Hypervisors to

support Virtual

(15)

Inter VLAN routing

VLAN trunks are used to carry

frames to the router

Virtual Gateway

VLAN1

Virtual Gateway

VLAN2

Virtual Gateway

VLAN2

Router

Switch1

Switch2

Dot1q VLAN trunks

VLAN 1

VLAN 2

Dynamic

routing

updates

Core Router

(16)

Data & Control Plane virtualization: VRF

The VRF:

V

irtual

R

outing and

F

orwarding instance

VLAN Trunk, physical

interfaces, tunnels, etc.

VRF 3

Logical or

Physical Int

(Layer 3)

Logical or

Physical Int

(Layer 3)

VRF 2

VRF 1

Each VRF = separate

forwarding table

Each VRF = separate

forwarding table

(17)

Spanning Tree

Physical network topology

STP network topology

Bandwidth Cost 4 Mbps 250 10 Mbps 100 16 Mbps 62 45 Mbps 39 100 Mbps 19 155 Mbps 14 622 Mbps 6 1 Gbps 4 10 Gbps 2

Hierarchical Star Network Architectures

– Easy to implement

– Little reliable

Single point of failure

(in the star centre)

Need for high reliability

– Alternate paths demand

– Problem:

bridging loops

bridging loops

(18)

Gateway redundancy

I cannot

reach my

Gateway!

... but i

could route

you to the

backbone!

Backbone

The Problem…

(19)

Gateway redundancy

Backbone

The idea…

Virtual

Router

Cluster

(20)

Gateway redundancy

Typical Redundancy techniques in mission

critical applications:

– Local Area Networks

• Backup

layer 2 paths with Spanning Tree Protocol

layer 2

– Wide Area Networks

• Backup

layer 3 paths and dynamic routing algorithms

layer 3

– Default Gateways can become

single point of failures:

single point of failures

• Gateway Redundancy Protocols:

HSRP (Hot Standby Router Protocol): proprietary - Cisco

NSRP (NetScreen Redundancy Protocol): proprietary - Juniper

VRRP (Virtual Redundancy Routing Protocol): Standard

(21)

Link Aggregation: EtherChannel

More then one link can be grouped to form a Channel

– Generally used for switch interconnection

– Sometimes used to connect a switch to…

• …a router.

• …a server.

• …a hypervisor.

“Etherchannel” is seen by the switch as

a single physical link

a single physical link

Benefits

– More bandwidth available with load sharing

– Redundancy & Better availability

– Fast recovery in case of failure

ent0 ent1 ent2 Negotiation

AIX

Negotiation

(22)

Packet distribution for Cisco port channels

Availability of other load balancing methods depends on switch model

Cat_3560_2(config)#port-channel load-balance ?

dst-ip

Dst IP Addr

dst-mac

Dst Mac Addr

src-dst-ip

Src XOR Dst IP Addr

src-dst-mac

Src XOR Dst Mac Addr

src-ip

Src IP Addr

src-mac

Src Mac Addr

X

X

X

IP and TCP / UDP

Cisco 6500

PFC3C/XCL

X

Layer 3 + VLAN ID

Cisco 6500

PFC3C/XCL

X

X

X

TCP / UDP

Multi Layer Models

(C3560, C3750,

C4900. C6500)

X

X

X

IP

X

X

X

MAC

Layer 2 Models

(C2900)

Source XOR

Destination

Destination

Source

Protocol

Model

(23)

If you do it wrong…

%SW_MATM-4-MACFLAP_NOTIF: Host 001a.6484.b012 in

vlan 65 is flapping between port Gi1/0/19 and port

Gi2/0/19

ping www.google.de

PING www-tmmdi.l.google.com (216.239.59.103): 56 data bytes

64 bytes from 216.239.59.103: icmp_seq=0 ttl=49 time=46 ms

64 bytes from 216.239.59.103: icmp_seq=2 ttl=49 time=46 ms

64 bytes from 216.239.59.103: icmp_seq=4 ttl=49 time=44 ms

64 bytes from 216.239.59.103: icmp_seq=6 ttl=49 time=45 ms

www-tmmdi.l.google.com ping statistics

---8 packets transmitted, 4 packets received, 50% packet loss

Mar 22 18:57:46 ent12 I ECH_CHAN_RCVRY

Mar 22 18:57:42 ent3 I GOENT_RCVRY_EXIT

Mar 22 18:57:39 ent12 P ECH_CHAN_FAIL

Mar 22 18:57:39 ent3 T GOENT_LINK_DOWN

Mar 22 18:23:55 ent12 I ECH_CHAN_RCVRY

Mar 22 18:23:51 ent3 I GOENT_RCVRY_EXIT

Mar 22 18:23:48 ent12 P ECH_CHAN_FAIL

Mar 22 18:23:48 ent3 T GOENT_LINK_DOWN

(24)

Cisco Virtual Switching System® (VSS 1440)

Operational Manageability

– Two Catalyst 6500s share a single point of management, single gateway IP

address, and single routing instance

Non-Stop Communications

– Delivers deterministic, sub-200 millisecond layer 2 link recovery through

inter-chassis stateful failovers and the predictable resilience of Etherchannel

Scales to 1.4 Tbps

(25)

Multichassis EtherChannel® (MEC) with Virtual I/O Server

Layer 2 multipathing technology

Creates simplified loop-free topologies

Supported Protocols:

– Cisco Port Aggregation Protocol (PAgP)

– 802.3ad Link Aggregation Control protocol (LACP)

– “ON” Manual Etherchannel

phy

Virtual I/O Server

phy LA

MEC

VSL

Active

Active

SEA virt

(26)
(27)
(28)
(29)

How does Quality of Service (QoS) work?

Traffic marking

Priority of network traffic is maintained by

additional header information at frame or

packet level:

– Layer 2 marking: IEEE 802.1P via Class of

Service (CoS)

– Layer 3 marking: Differentiated Service

(ToS, DSCP or DiffServ)

Network traffic is marked accordingly to the

type of service it needs.

– Very important traffic should have a high

priority value.

– Less important traffic should have a lower

priority value.

– Best effort traffic can be forwarded without

any marking.

Traffic queuing

Traffic is scheduled with different

importance.

Importance of traffic depends on priority:

– CoS value on Layer 2

– ToS value on Layer 3

Different queuing methods can be used:

– FIFO

– Weighted Fair

– Weighted round robin

– Low Latency …

Shared Ethernet Adapter supports

bandwidth adoption in conjunction with

IEEE 802.1p.

(30)

PRI

CoS

CFI

VLAN ID

Destination

MAC

Source

MAC

Power Hypervisor supports Layer 2 CoS marking

802.1q

Tag

Type

Data

FCS

TPID

voice < 10 ms latency and

jitter

6

network control

7

video < 100 ms latency and

jitter

5

controlled load

4

excellent effort

3

spare

2

background

1

best effort

0

Typical Application

CoS

(31)

How does priority queuing works?

(relative time of arrival)

Sending Direction

(one packet at a time)

(32)

How does priority queuing works?

(relative time of arrival)

(one packet at a time)

Priority

Queue

Best Effort

Queue

(33)

Shared Ethernet Adapter CoS queuing

Virt

Virt

LPAR 1 LPAR 2 VLAN X LPAR 1 Tag LPAR 1 Tag LPAR 2 Tag LPAR 2 Tag LPAR 1 Tag LPAR 1 Tag

ent2

(SEA)

ent1

(Vir)

en2

(if)

ent0

(Phy)

SEA supports

CoS queuing

VIOS

Low CoS

priority

High CoS

priority

(34)

Management

SAN B

SAN A

LAN

FCoE

Ethernet

Today

Converged I/O Today

Today

Parallel LAN/SAN Infrastructure

Inefficient use of Network

Infrastructure

5+ connections per server –

higher adapter and cabling

costs

Adds downstream port costs;

cap-ex and op-ex

Each connection adds additional points of

failure in the fabric

Longer lead time for server

provisioning

Multiple fault domains –

complex diagnostics

Management complexity –

firmware, driver-patching,

(35)

Converged I/O:

Reduction of server adapters

Simplification of access layer

and cabling

Gateway free implementation—

fits in installed base of existing

LAN and SAN

L2 Multipathing Access—

Distribution

Lower TCO

Fewer Cables

Investment Protection (LANs

and SANs)

Consistent Operational Model

Management

SAN B

SAN A

LAN

FCoE

Ethernet

Converged I/O Today

FCoE

Switch

(36)

#5708 10Gb FCoE PCIe Dual Port Adapter for IBM Power Systems

#5708 is a CNA (Converged Network Adapter)

Dual 10Gb ports

– Physically are Ethernet ports

– Each port can run all NIC, all FC, or mixed

NIC/FC traffic

– SR optical fiber cabling

SOD for NPIV function through VIOS

AIX & Linux support

–AIX 5.3 with the 5300-11 Technology Level, or later

–AIX 6.1 with the 6100-04 Technology Level, or later

–SUSE Linux Enterprise Server 10 Service Pack 3 or later

–Red Hat Enterprise Linux 5.4 or later

VIOS support

–VIOS 2.1.2.0 or later

PCIe 8x Gen 1 Adapter

CCIN = 2B3B

(37)

Thank you !

Alexander Paul

Figure

Updating...

References

Updating...

Related subjects :