BRKACI-2006 ABSTRACT

(1)

(2)

Integration of Hypervisors

& L4-7 Services with ACI

Maurizio Portolani, Distinguished Technical Marketing Engineer Azeem Suleman, Architect Engineering

(3)

(4)

• Introduction to ACI • Hypervisor Integration

• Hypervisor Integration Overview

• VMWare vCenter Integration

• Microsoft SCVMM & Azure Pack Integration

• OpenStack Integration

• Service Insertion

• Service insertion versus Service Graph

• Example

• Design Options

• How to configure

• Ecosystem

(5)

ACI Fabric

Scale-Out Penalty Free Overlay

App DB Web Outside (Tenant VRF) QoS Filter QoS Service QoS Filter Application Policy Infrastructure Controller APIC

Cisco ACI

(6)

 Extend the principle of Cisco UCS®

Manager service profiles to the entire fabric

 Network profile: stateless definition of application requirements

- Application tiers

- Connectivity policies

- Layer 4 – 7 services - XML/JSON schema

 Fully abstracted from the infrastructure implementation

- Removes dependencies of the infrastructure

- Portable across different data center fabrics

## Network Profile: Defines Application Level Metadata (Pseudo Code Example)

<Network-Profile = Production_Web> <App-Tier = Web>

<Connected-To = Application_Client>

<Connection-Policy = Secure_Firewall_External> <Connected-To = Application_Tier>

<Connection-Policy = Secure_Firewall_Internal & High_Priority> . . .

<App-Tier = DataBase> <Connected-To = Storage>

<Connection-Policy = NFS_TCP & High_BW_Low_Latency> . . .

App Tier DB Tier

Storage Storage

Web Tier

Application

The network profile fully describes the application connectivity requirements

ACI Network Profile

(7)

Multi-Hypervisor-Ready Fabric

 Integrated gateway for VLAN and

VxLAN, networks from virtual to physical

 Normalization for VXLAN, and VLAN

networks

 Customer not restricted by a choice of

hypervisor

 Fabric is ready for multi-hypervisor

Virtual Integration Network Admin Application Admin PHYSICAL SERVER VLAN

VXLAN VLAN VLAN_VXLAN VLAN

ESX Hyper-V KVM Hypervisor Management ACI Fabric APIC APIC VMware Microsoft Red Hat

(8)

ACI Layer 4 - 7 Service Integration

Centralized, Automated, And Supports Existing Model

 Service insertion architecture for:

 physical

 virtual services

 Helps enable administrative separation between application tier policy and service definition

 APIC as central point of network control with policy coordination

 Automation of service bring-up/tear-down through programmable interface

Web Server Web Tier A Web Server Web Server App Tier B App Server Application Admin Service Admin S e rv ic e Gr a p h

begin Stage 1 ….. Stage N end

P ro v ide rs inst inst … Firewall inst inst … Load Balancer …….. Se rv ic e Pro fi le

“Security 5” Chain Defined Chain

(9)

Basics of ACI Forwarding

How to Create a L2 domain?

• Create a Bridge Domain

• Disable Unicast Routing on the

Bridge Domain

• Associate the Bridge Domain with a

VRF

• The association with the VRF is

because of the object model

• The hardware won’t program any

VRF if the Bridge Domain is

configured only as L2 Bridge Domain 1

(10)

Basics of ACI Forwarding:

How to Create a L2 domain with Routing?

• Create a Bridge Domain

• Enable Unicast Routing

• Associate the Bridge Domain with

the VRF

• Configure the default gateway by

creating the “Subnet” on the Bridge Domain

Bridge Domain 1 VRF

(11)

How does the Bridge Domain Forward Traffic?

• Based on the mapping database

• This is configured by configuring

“hardware-proxy”

• and by disabling ARP flooding

• Or With flood and learn

• This is configured by enabling

unknown unicast flooding

(12)

Bridge Domain Configuration

Enables or disables Routing

Enables or disables the use of the Mapping Database

(13)

• Introduction to ACI

• Hypervisor Integration

• Hypervisor Integration Overview • VMWare vCenter Integration

• Microsoft SCVMM & Azure Pack Integration • OpenStack Integration

• Service insertion versus Service Graph • Example

• Design Options • How to configure • Ecosystem

(14)

Hypervisor Interaction with ACI

The main advantage of “Integrated mode” is the dynamic allocation of the encapsulation

• ACI Fabric as an IP-Ethernet Transport • Encapsulations manually allocated • Separate Policy domains for Physical

and Virtual

VLAN 10 VLAN 10 VXLAN 10000

Non-Integrated Mode

• ACI Fabric as a Policy Authority • Encapsulations Normalized and

• dynamically provisioned

APP WEB DB

Integrated Mode

(15)

vCenter DVS SCVMM

 Relationship is formed between

APIC and Virtual Machine Manager (VMM)

 Multiple VMMs likely on a single

ACI Fabric

 Each VMM and associated Virtual

hosts are grouped within APIC

 Called VMM Domain

 There is 1:1 relationship between a

Virtual Switch and VMM Domain

VMM Domain 1

Hypervisor Integration with ACI

Control Channel - VMM Domains

vCenter AVS

(16)

VXLAN VNID = 5789 VXLAN VNID = 11348 NVGRE VSID = 7456 Any to Any 802.1Q VLAN 50 Normalized Encapsulation Localized Encapsulation IP Fabric Using VXLAN Tagging Payload IP VXLAN VTEP

• All traffic within the ACI Fabric is encapsulated with an extended VXLAN header

• External VLAN, VXLAN tags are mapped at ingress to an internal VXLAN tag

• Forwarding is not limited to, nor constrained within, the encapsulation type or encapsulation ‘overlay’ network

• External identifies are localized to the Leaf or Leaf port, allowing re-use and/or translation if required Payload Payload Payload Payload Eth IP VXLAN Outer IP IP 802.1Q Eth IP Eth MAC Normalization of Ingress Encapsulation

ACI Fabric – Integrated Overlay

(17)

EP EP EP EP EP EP _EP EP EP EP EP EP EP EP

VMM Domain 1: 4K EPGs VMM Domain 2: 4K EPGs

16M Virtual Networks  VLAN ID only gives 4K EPGs (12

bits)

 Scale by creating domains of 4K EPGs

 Map EPGs to VMM Domain based on scope of live migration

 Place VM anywhere

 Live migrate within VMM domain

Hypervisor Integration with ACI

(18)

Hypervisor Integration with ACI

VMM Domains & VLAN Encapsulation

EP EP EP EP VLAN 5 VLAN 16 16M Virtual Networks VNID 6032

 VLAN ID only gives 4K EPGs

(12 bits)

 Scale by creating pockets of

4K EPGs

 Map EPGs to VMM Domain

based on scope of live migration

 Place VM anywhere

 Live migrate within VMM

domain

(19)

Hypervisor Integration with ACI

Endpoint Discovery DVS Host APIC VMM Control (vCenter API) Data Path

 Virtual Endpoints are discovered for reachability & policy purposes via 2 methods:

 Control Plane Learning:

- Out-of-Band Handshake: vCenter APIs

- Inband Handshake: OpFlex-enabled Host (AVS, Hyper-V, etc.)

 Data Path Learning: Distributed switch learning

 LLDP used to resolve Virtual host ID to attached port on leaf node (non-OpFlex Hosts)

OpFlex Host

Control (OpFlex)

(20)

Policy Provisioning

• Resolution Immediacy:

• Determines when the EPG policies are pushed to the software (Policy Element) the leaf switch: can be Immediate or On-Demand

• Deployment Immediacy:

• Determines when the Policy Element programs the policies in the hardware:can be Immediate or On-Demand EPG Static Node/Port VMM Domain Resolution Immediacy Deployment Immediacy Deployment Immediacy

(21)

(22)

ACI and vSphere Integration

• Three options:

• Deployment with vDS

• Deployment with AVS

• Deployment with vShield

• Configuration Steps:

• Preparing the fabric

• Defining the VMM

• Adding Hosts uplinks to the APIC vDS

(23)

VMWare Integration

Three Different Options

+

Distributed Virtual Switch

(DVS) vCenter + vShield

Application Virtual Switch (AVS) • Encapsulations: VLAN • Installation: Native • VM discovery: LLDP • Software/Licenses: vCenter with Enterprise+ License • Encapsulations: VLAN, VXLAN • Installation: Native • VM discovery: LLDP • Software/Licenses: vCenter with Enterprise+ License, vShield Manager with vShield License

• Encapsulations: VLAN, VXLAN

• Installation: VIB through VUM or Console

• VM discovery: OpFlex • Software/Licenses:

vCenter with

(24)

(25)

Hypervisor Integration with ACI 11.0

EPG Classification via Port-Groups

 VM’s are placed within the port-group defined for each EPG

 Traffic is encapsulated with the specific VLAN or VXLAN assigned to that port-group on that port and forwarded upstream to the TOR

VXLAN VNID = 5789 VXLAN VNID = 11348 802.1Q VLAN 50 Payload IP GBP VXLAN VTEP VXLAN Leaf VTEP 802.1Q vSwitch WEB PORT GROUP APP PORT GROUP vSwitch WEB PORT GROUP APP PORT GROUP 802.1Q VLAN 125 Payload IP IP Payload Port-groups Created for Each EPG +

(

(26)

(27)

Supporting Multiple vCenter DVS

• The “VMM controller” represents a Datacenter within a vCenter • A VMM Domain represents a DVS inside a vCenter.

• Same vCenter can be added into different VMM domains.

• Each will create a corresponding DVS.

Virtual Distributed Switch Domain1 VMware vCenter, Datacenter Domain1 VC IP 1, DC1 Domain2 VC IP 1, DC1 Virtual Distributed Switch Domain2

(28)

Can I have more than 1 APIC vDS per Datacenter? Yes

Can I have more than 1 vCenter Datacenter per APIC? Yes

• vCenter as Seen by APIC

• APIC vDS as seen by vCenter

• vDS1 under Datacenter 1 • vDS2 under Datacenter 1 • vDS3 under Datacenter 1 VMM Controller 1: vCenter1, Datacenter1 VMM Controller 1: vCenter1, Datacenter1 VMM domain 1 VMM domain 2 VMM Controller 1: vCenter1, Datacenter2 VMM domain 3

(29)

APIC Admin

VI/Server Admin Instantiate VMs,

Assign to Port Groups

L/B EPG APP EPG DB F/W EPG WEB

Application Network Profile Create Application Policy

Web Web

Web App

HYPERVISOR HYPERVISOR

VIRTUAL DISTRIBUTED SWITCH

WEB PORT GROUP APP PORT GROUP DB PORT GROUP

vCenter Server / vShield 8 5 1 9 ACI Fabric Automatically Map EPG To Port Groups

Push Policy

Create VDS

2

Cisco APIC and VMware vCenter Initial Handshake

6

DB DB

7 Create Port

Groups

ACI Hypervisor Integration – VMware DVS/vShield

APIC

3

Attach Hypervisor to VDS

4 Learn location of ESX

(30)

APIC Admin

VI/Server Admin Instantiate VMs,

Assign to Port Groups

L/B EPG APP EPG DB F/W EPG WEB

Web Web

Web App

Application Virtual Switch (AVS)

WEB PORT GROUP APP PORT GROUP DB PORT GROUP

vCenter Server 8 5 1 9 ACI Fabric Automatically Map EPG To Port Groups

Push Policy

Create AVS VDS

2

Cisco APIC and VMware vCenter Initial Handshake

6

DB DB

7 Create Port

Groups

ACI Hypervisor Integration – AVS

APIC

3

Attach Hypervisor to VDS

4 Learn location of ESX

Host through OpFlex

(31)

Fabric Preparation for ACI integration with vDS

• AEP associates Domain to interfaces and switches

• Provides:

• Span of VLAN pool and Domains to the leaf ports

• Physical Interface Policies for vCenter vDS. E.g. LACP, LLDP, CDP, MTU Leaf Ports + Policies AEP VMM Domain Physical Domain External Domain VLAN Pool 1 VLAN Pool 2 VLAN Pool 3 vCenter Virtual Distributed Switch

(32)

ACI – Configuration of VMM Domain for vDS

Name of VMM Domain

Type of vSwitch (DVS or AVS)

Associated Attachable Entity Profile (AEP) VLAN Pool

vCenter Administrator Credentials

(33)

DVS Uplink Interface Policies

• LLDP/CDP:

• vDS supports only one of CDP/LLDP, not both at the same time

• LLDP takes precedence if both LLDP and CDP are defined.

• To enable CDP , AEP should contain AccPortGrp with LLDP disabled and CDP enabled

• By Default LLDP is enabled and CDP is disabled

• LACP:

• Enables/disables LACP on uplink port-group

• Changes the Load-balancing algorithm on VM port-groups

LACP Mode Tenant Port group Hash Algorithm

Active Ip-hash

Passive Ip-hash

Off Ip-hash

(34)

(35)

vmk AVS

 The vmk interface provides the VTEP

 The VTEP IP address is obtained via a DHCP request

 The AVS becomes a member of the infrastructure Tenant

 You need to configure the infrastructure Tenant for DHCP relay

Preparing the Fabric for VXLAN integration: configuration steps for AVS

and vShield

Infrastructure VLAN VTEP on infra VLAN

vmk gets its address via DHCP

(36)

(37)

VTEP addresses are allocated to nodes in the fabric automatically based

on the pool configured at fabric inititalization

WithoutAVS, these addresses are internal to the fabric and won’t overlap

with anything in the existing network. Leaf 1 10.0.104.95 Leaf 2 10.01.104.96 Leaf 3 10.0.104.97 Spine 1 10.0.104.92 Spine 2 10.0.104.93 Infra range: 10.0.0.0/16

(38)

With AVS or with vShield

Leaf 1 10.0.104.95 Leaf 2 10.0.104.96 Leaf 3 10.0.104.97 Spine 1 10.0.104.92 Spine 2 10.0.104.93 Infra range: 10.0.0.0/16 ESXi AVS VMK1: 10.0.104.98

With AVS, a VMK interface is created on the host and an address from the ACI infra pool is used.

The ESXi host puts a route to the infra network in its table, pointing to the new VMK interface. ~ # esxcfg-route -l

VMkernel Routes:

Network Netmask Gateway Interface

10.0.0.0 255.255.0.0 Local Subnet vmk1

(39)

DHCP configuration

(40)

Southbound OpFlex API VM VM VM VM AVS vSphere Hypervisor Manager

 OpFlex Control protocol

- Control channel

- VM attach/detach, link state notifications

 VEM extension to the fabric

 vSphere 5.0 and above

 BPDU Filter/BPDU Guard

 SPAN/ERSPAN

 Port level stats collection

Application Virtual Switch (AVS)

(41)

AVS forwarding

• Unicast Traffic:

- Locally originated unicast traffic to local destinations is forwarded by the AVS - Locally originated traffic destined to unknown MAC is forwarded to the Leaf

• If encap type is VXLAN, packet sent to the Fabric tunnel end point (FTEP) IP

• Multi-Destination Traffic:

• Traffic from the leaf with source MAC belonging to a local endpoint is dropped

• Locally originated broadcast and multicast traffic are flooded locally and forwarded to the leaf after adding appropriate encap

• If encap type is VXLAN, destination IP of the outer packet is set to VMM GIPo multicast address

(42)

Multicast configuration

• Multicast Configuration is required on ACI in order to support Broadcast and

Multicast from the VMs

• Broadcast and Multicast traffic from a VM is sent to a unique GiPO mcast group – this is

VMM-wide

• Broadcast and Multicast traffic to a VM is sent to an EPG-specific mcast group

• That is why you configure a pool of mcast IPs

• Multicast Address: Assign a multicast address that represents the AVS

VMM-wide. This is used from the VM to the fabric

• Multicast Address Pool: Create a unique Multicast Address Pool large enough to

(43)

Hypervisor

VM VM

EPG Web

EPG Classification via VM Attributes

VM Attribute Guest OS VM Name VM (id) VNIC (id) Hypervisor DVS port-group DVS Datacenter Custom Attribute MAC Address IP Address v Cen te r VM A ttribu te s VM T raffi c A ttr ib u tes

(47)

AVS with ACI 11.1

EPG Classification via VM Attributes (2)

VM Attribute Guest OS VM Name VM (id) VNIC (id) Hypervisor DVS port-group DVS Datacenter Custom Attribute MAC Address IP Address v Cen te r VM A ttribu te s VM T raffi c A ttr ib u tes

 Two categories of Attributes supported with the 11.1 release

VM Attributes (set by the server administrator at the creation time of the VM)

VM Traffic Attributes (VM MAC/IP address or L4 port being used by the application)

 Any endpoint placed within a Port Group on the vSwitch can be further classified based on the specific VM Attributes

(48)

AVS with ACI 11.1

EPG Classification via VM Attributes

vSwitch (AVS)

Port Group EPG == VM

Attribute ‘x’ _{Attribute ‘y’}EPG == VM

APIC Admin Create an EPG == VM Attribute ‘x’ on VMM Domain ‘A’ 3 4 APIC Distributes VM Attribute Policies to Leaf nodes

AVS notifies Leaf of VM Attach via OpFlex Channel 6 Leaf Determines Attribute to EPG Classification 7

Leaf Pushes EPG encapsulation binding to AVS via

OpFlex Channel 8

802.1Q VLAN 50 AVS forwards traffic

with the correct EPG label (encapsulation) 9

APIC Retrieves Hypervisor State

(VM State & VM Attributes) & Initiate

a Listener Process for any changes/updates 2 Administrator Creates new vDS (AVS) 1 VI/Server Admin

Boot new VM with desired VM Attributes and connect them to the

“Base port-group” 5

(49)

Win EPG

EPG Classification via VM Attributes

Use Case 1 - Isolate a Malicious VM

 Problem: Vulnerability is detected in a particular type of operating system (e.g. Windows). Network security administrator would like to isolate all Windows VM

 Solution: Define Security EPG with criterion as “Operating System = Windows”. No contracts are

provided or consumed by this EPG. It will stop all inter-EPG communication for the matching VMs

 No VM attach/detach or placement of VM to a different port-group is needed

Web _Web01

Linux Web02Linux Web03Win

App _App01

Linux App02Linux App03Win

DB _DB01

Linux LinuxDB02 DB03Win

X

CriterionAttribute

(50)

HR-Web

Sales-Web

EPG Classification via VM Attributes

Use Case 2 - Security across zones

 Problem: VMs belonging to different departments (e.g. HR, Sales) or different roles (Production, Test) are placed in the port-group. But isolation across departments are required. (e.g. HR-Web-VM should not be able to talk to Sales-Web-VM)

 Solution: Define EPGs, which match if the VM Name contains a matching string (e.g. HR, Sales etc).

 Each Attribute based EPG can have their own security policies.

Web

Web01 _Web01HR- _Web01

Sales-App

App01 App02 App03

DB DB01 DB02 DB03 Criterion Attribute (VM name contains HR) Criterion Attribute

(VM name contains Sales)

(51)

 ACI Leaf

Reflexive ACL in the hardware is programmed to allow TCP packets only if ACK flag set This is done when setting the ‘stateful’ flag in the filter

 AVS vSwitch

Maintains connection table to track the flow

In receiving the first TCP SYN packet, AVS creates a flow table entry

Packets belonging to a flow that does not have an entry are dropped by the AVS

• In a typical Firewall solution, the first packet is always sent to a policy engine. In ACI fabric, hardware acts as a policy store, so it doesn’t incur performance penalty for policy lookup

(52)

Hardware Assisted Stateful firewall

Provider B

Consumer A

Src class Src port Dest Class Dest port Flag Action

A * B 80 * Allow

B 80 A * ACK Allow

• Create flow table entry • Forward packet to Leaf

Leaf evaluates stateless policy

Hardware policy permits the packet

Create flow state only for TCP SYN packet received from PNIC

Deliver packet to destination VM

• Packet received from VM • Lookup flow table

VLAN Proto Src ip Src port Dst IP Dst port

A tcp IP_A 1234 IP_B 80

A tcp IP_B 80 IP_A 1234

VLAN Proto Src ip Src port Dst IP Dst port

B tcp IP_A 1234 IP_B 80

B tcp IP_B 80 IP_A 1234

On flow table hit forward packet to Leaf

Policy Enforcement done at Leaf Connection

Tracking at AVS

Response from VM

Perform flow table lookup

Reflexive Hardware policy permits the packet

(53)

What is the ACI vCenter Plugin?

 A VMware vCenter Web Client plugin (vSphere 5.5) for ACI

 Empowers virtualization admins to define network connectivity independently

of the networking team while sharing the same infrastructure

Virtualization admin is able to configure network connectivity (subnets, port-groups) with tenant isolation

• Focused on simplicity

User does not need to understand the ACI Policy Model Follows the vCenter Web Client GUI standards

No configuration of “in-depth” networking stuff – this is done through APIC by the networking expert

(54)

ACI-vCenter Interactions

Current Model

Network Team Virtualization Team vCenter ACI Port Group ESXi Hypervisors APIC Cluster

(55)

ACI-vCenter Interactions

vCenter Plugin Model

Network Team Virtualization Team vCenter ACI Plugin ACI Port Group ESXi Hypervisors APIC Cluster 1 2 3 Virt. Admin

(56)

(57)

(58)

(59)

Summary

Two deployment modes with VMware

• Using ACI with vDS

• You don’t need AVS in order to

deploy ACI

• ACI interacts with Vmware vDS to

create port-groups that provide the extension of EPG

• You can have one tier of switches

between the servers and the leaf

• Using ACI with AVS

• You can get some additional

features with AVS

• You can have more hops in the path

between the ESX servers and the leaf

(60)

(61)

Microsoft Interaction with ACI

Two modes of Operation

• Policy Management: Through APIC • Software / License: Windows Server with

HyperV, SCVMM • VM Discovery: OpFlex • Encapsulations: VLAN • Plugin Installation: Manual

Integration with SCVMM

APIC

Integration with Azure Pack

APIC

• Superset of SCVMM

• Policy Management: Through APIC or through Azure Pack

• Software / License: Windows Server with HyperV, SCVMM, Azure Pack (free) • VM Discovery: OpFlex

• Encapsulations: VLAN, NVGRE • Plugin Installation: Integrated

(62)

(63)

Microsoft Interaction with ACI

• ACI with SCVMM only without Azure Pack: In this case the APIC controller

and SCVMM communicate with each other for network management and Azure Pack is not involved. Networks are created in APIC and are published as VM Networks in SCVMM. Compute is provisioned in SCVMM and can consume these networks.

• ACI with SCVMM and Azure Pack: ACI integrates in Azure Pack to provide a

self-service experience for tenant. ACI resource provider in Azure pack drives APIC controller for network management. Networks are created in SCVMM and are available in Azure Pack for respective tenants. ACI L4-L7 capabilities for Load balancers (F5 and Citrix) and stateless firewall are provided for tenants.

(64)

APIC Admin SCVMM Admin Instantiate VMs, Assign to VM Networks L/B EPG APP EPG DB F/W EPG WEB

MSFT SCVMM 8 5 1 9 ACI Fabric Automatically Map EPG To VM Networks Push Policy Create Virtual Switch 2

Cisco APIC and MSFT SCVMM Initial Handshake

6

ACI Hypervisor Integration – MSFT SCVMM

APIC

3 _{Attach Hypervisor}

to Virtual Switch

4 Learn location of HyperV

Host through OpFlex

OpFlex Agent

HYPERV VIRTUAL SWITCH 7 Create VM

Networks

OpFlex Agent

WEB VM NETWORK APP VM NETWORK DB VM NETWORK

(65)

Building Blocks for ACI Integration with Microsoft

Hyper-V (1)

• APIC Agent on SCVMM

• This component is a Windows System Service (EXE) that runs on SCVMM with

administrative credentials. It communicates with the APIC controller over the ACI REST interface and with SCVMM through powershell interface. It performs the following functions:

• Setup APIC logical switch, networks and encapsulations

• Add\Delete VMNetworks in SCVMM based on the information in the APIC controller.

(66)

Building Blocks for ACI Integration with Microsoft

Hyper-V (2)

• Opflex Agent in Hyper-v Host

• This component is a Windows System Service (EXE) that runs on each host with

administrative credentials. It communicates with the directly attached ACI leaf via REST interface. It uses the Infra-channel and runs on the VTEP interface in the host. The IP for the VTEP interface is given by APIC DHCP service. This agent performs the following functions:

• Send VM attach and detach events to the leaf

• Send inventory information like VM name, NIC name, statistics to the leaf which is then displayed in the APIC UI

• The APIC agent is installed on each Hyper-v host and it communicates with the nearest ACI leaf over Opflex Protocol. The leaf on receiving EP attach and detach notifications downloads policies from the policy manager and applies it to local leaf ports.

(67)

Microsoft Azure Pack Integration

 Integration with Microsoft requires:

- Windows Server 2012

- Systems Center 2012 R2 with SPF

- Windows Azure Pack

 Azure Pack provides single pane of

glass for Definition, creation,

management of their cloud service

 Divided into Provider (Admin) portal

and Consumer Self-Service (Tenant) portal

 Cisco ACI Service Plugin enables

management of Network Infrastructure through APIC REST API

R2 w/ Service Provider Foundation Web Sites Service Plans Users Provider Portal Consumer Self-Service Portal Web Sites Apps Database VMs ACI

Service Provider Customer

(68)

APIC Admin (Basic Infrastructure)

Azure Pack Tenant

3 6 ACI Fabric Push Network Profiles to APIC

Pull Policy on leaf where EP attaches

Indicate EP Attach to attached leaf when VM starts

1

2

HYPERVISOR HYPERVISOR HYPERVISOR

ACI Azure Pack Integration

APIC

Get VLANs allocated for each EPG

Create Application Policy

7

Azure Pack \ SPF

SCVMM Plugin

APIC Plugin _{OpFlex Agent} _{OpFlex Agent} _{OpFlex Agent}

Instantiate VMs 5 1 4 Create VM Networks 4

(69)

Microsoft Azure Pack Integration

Admin Experience

Add & Configure service providers for this

deployment (APIC IP Address, Login Credentials, etc.)

Usage & Billing statistics per user and other admin functions

(70)

Microsoft Azure Pack Integration

Tenant Experience

Services this account

has access to Resources of ACI service currently created and

consumed by this tenant

Application Network Profiles are created through Azure Pack, and pushed to APIC using REST

(71)

ACI as a Resource Provider

• The ACI resource provider has three sub-components:

• Tenant Portal Extension:

• This implements the user interface and APIs for the tenant portal to expose ACI capabilities

• Admin Portal Extension:

• This implements the user interface and APIs for the admin portal

• Resource Provider Backend.

• This is the component that interacts with the SCVMM via Service pack Foundation power shell and with APIC using the Rest interface.

(72)

(73)

OpenStack Networking

Neutron Model

Tenant Network Security Group Security Group Rule Network: external Router Port

Subnet Abstraction Description

Logical Network L2 / Broadcast domain Logical Router L3 domain

Subnet Subnet (OpenStack IPAM / DHCP)

(74)

OpenStack with ACI

APIC ML2 Driver Mapping

Neutron Object APIC Object

Project Tenant

Network EPG + BD

Subnet Subnet

Security Group + Rule IP Tables (outside of APIC)

Router Contract

(75)

APIC Admin (Performs Steps 3)

OpenStack Tenant

(Performs Steps 1,4) Instantiate VMs

Create Application Policy

• Intent-based API for describing application requirements

• Separates concerns of tenants and operators

• Captures dependencies between tiers of an application

• Plugin model

• Built by a community of

developers across a range of companies

Policy Rules Set

Web Group Classifier Action FIREWALL DB Group Classifier Action Service Chain

(79)

ACI OpenStack Integration with Group-based Policy

2

ACI Admin (manages physical network, monitors tenant

state) L/B EPG APP EPG DB F/W L/B EPG WEB

3

5 ACI

Fabric Push Policy

OpenStack Tenant

(Performs step 1,4) Instantiate VMs

Web App Web App Web Web

4

Create Application Network Profile

1

DB DB

HYPERVISOR HYPERVISOR HYPERVISOR

NOVA NEUTRON Automatically Push Network Profiles to APIC L/B EPG APP EPG DB F/W L/B EPG WEB

Application Network Profile

(80)

(81)

10.0.0.x 20.0.0.x

outside router no ip routing_{enable ARP} flood

no ip routing

enable ARP flood no ip routing_{enable ARP flood}

peer to the outside

In order not to learn outside IPs you need to use a l2ext

You can do service chaining with ACI without any

service graph

(82)

With service graph the operational model changes

• Without Service Graph:

• Network admin: configures the

ports, VLANs to connect the FW or the LB

• FW admin day 0: configures ports

and VLANs

• FW admin day 1: configures ACLs

and so on

• The three configurations are spread

over multiple phases / days

• With Service Graph

• ACI admin:

• configures the ports, VLANs to connect the FW or the LB

• FW admin day 0: configures ports and VLANs

• FW admin day 1: configures ACLs and so on

• All configurations are performed in a

(83)

With service graph the operational model changes

• Without Service Graph: • With Service Graph

configuration configuration for fabric

(84)

Device Definition and Package

APIC requires a Device Package to communicate with service devices. A Device Package is a zip file containing two parts

Device Specification (xml): The configuration of the APIC is represented as an object model consisting of a large number of Managed Objects (MOs). A Device type is defined by a tree of MOs with a Meta Device (MDev) at the root.

DeviceScript (py): The integration between the APIC and a Device is performed by a DeviceScript, which maps APIC events function calls defined in Device Script

Device Script

APIC

Configuration through UI or North Bound

APIs PackageDevice

BIG-IP Physical or VE EPG level L4-L7 config Service Graph Function Node level

L4-L7 config Python iControl/South Bound API Device Specification <devtype= “f5”> <service type= “slb”>

<paramname= “vip”> <dev ident=“210.1.1.1” <validator=“ip”

<hidden=“no”> <locked=“yes”>

(85)

With the Service Graph APIC communicates with the L4L7 Device

Device APIs

def deviceValidate (device, version)

def deviceModify (device, interfaces, configuration) def deviceAudit (device, interfaces, configuration) def deviceHealth (device, interfaces, configuration) def deviceCounters (device, interfaces,

configuration)

Cluster APIs

def clusterModify (device, configuration) def clusterAudit (device, configuration)

Service APIs

def serviceModify (device, configuration) def serviceAudit (device, configuration) def serviceHealth (device, configuration) def serviceCounters (device, configuration)

When Concrete device is added, APIC invokes

deviceValidateAPI to validate the device as per the specification in Device package.

Device must have appropriate software version and vendor information

When the graph is attached to a contract or any parameter change event occur in APIC , it invokes the

serviceModifyAPI callouts to the service device When Logical device is added, APIC invokes

clusterModifyAPI to let script perform one time cluster configuration function

Upon successful validation, APIC invokes

deviceModifyAPI to let script perform one time device configuration function

(86)

Service Graph Definition

Abstract graph concept mapping to Service Graph

• Service graph is an ordered set of functions between a set of terminals e-g; Firewall Function, Load balancer Function

• A function has one or more connectors

• Network connectivity like VLAN/VNID tag is assigned to these connectors

Functions rendered on the same device

• A function within a graph may require one or more parameters

• Parameters can be scoped by an EPG or an application profile or tenant context

• Parameter values can be locked from further changes

Service Graph: “web-application”

Func:

SSL offload Load BalancingFunc: Func:

Firewall

Connectors Terminals

Terminals Firewall params

Permit ip tcp * dest-ip <vip> dest-port 80 Deny ip udp *

SSL params

Ipaddress <vip> port 80 Load-Balancing paramsvirtual-ip <vip> port 80 Lb-aglorithm: round-robin

EXT

EXT EXT EXT

EPG - EXT

WEB

WEB WEB WEB

EPG - WEB

(87)

Purpose of the Service Graph

• By using the Service graph you can

• install a service, like the ASA firewall once and

• deploy it multiple times in different logical topologies

• Each time the graph is deployed ACI takes care of changing the configuration on

the firewall to enable the forwarding in the new logical topology.

• ACI takes care of dynamically provisioning VLANs, IP addresses while re-using

the same graph template

• The benefits of the service graph are:

• a configuration template that can be reused multiple times

• a more logical / application-related view of services

(88)

(89)

“Users” “Files”

ACI Fabric

Configurations with Service Graph

• All configurations performed in a

single operation:

• Fabric configuration: Bridge

Domains, VLANs, Routing, EPGs

• Firewall configuration: VLANs,

vlans are dynamically allocated and derived from the physical domain You don’t need to configure them

IP addresses on the firewalls and security rules depend on the EPG/ Bridge Domain

(95)

Example Use Case

BD 1 BD 2 10.10.10.x 20.20.20.x BD 3 BD 4 30.30.30.x 40.40.40.x EPG1 EPG2 EPG1 EPG2

(96)

Applying Service Graph (1)

10.10.10.5 20.20.20.5

subinterface and VLANs are automatically created

BD 1 BD 2

10.10.10.x 20.20.20.x

EPG1 EPG2

L4L7 rules are configured (e.g. virtual address)

(97)

Applying the same service graph (2)

subinterface and VLANs are automatically created

30.30.30.5 40.40.40.5

30.30.30.x 40.40.40.x

EPG1 EPG2

L4L7 rules are specific to this set of EPGs

(98)

If the L4L7 device is a virtual appliance

ACI creates “shadow EPGs” (port-groups)

30.30.30.x 40.40.40.x

EPG1 EPG2

vNIC1 vNIC2

“shadow” EPG on the consumer side

“shadow” EPG on the producer side

ACI automatically connects the vNICs to the correct port-groups

(99)

Service Configuration before the Service Graph

192.168.1.1 192.168.1.100 10.1.1.1 172.16.1.1 192.168.100.1 HTTP (TCP/80) HTTPS (TCP/443) DCERPC (TCP/135) SSH (TCP/22) ICMP

access-list OUT permit tcp host 192.168.1.1 host 10.1.1.1 eq 80 access-list OUT permit tcp host 192.179.1.1 host 10.1.1.1 eq 443 […]

access-list OUT permit icmp host 192.168.1.100 host 192.168.100.1

30 ACL Rules

172.18.20.13

access-list OUT permit tcp host 172.18.20.13 host 10.1.1.1 eq 80 access-list OUT permit tcp host 172.18.20.13 host 10.1.1.1 eq 443 […]

access-list OUT permit icmp host 172.18.20.13 host 192.168.100.1 15 ACL Rules

45 ACL Rules

Network Admin Security Admin

Add client 172.18.20.13, call Security Admin to

enable access

Remove client 192.168.1.1, “no other action necessary”

Add ASA rules for client 172.18.20.13

Original ASA rules never change 4 1 2 2 3 4 Files Users

(100)

Automatic endpoint addition/removal with ACI

10.1.1.1 172.16.1.1 192.168.100.1 Servers 192.168.1.1 192.168.1.100 172.18.20.13 HTTP (TCP/80) HTTPS (TCP/443) DCERPC (TCP/135) SSH (TCP/22) ICMP Source EPG

Leaf 1, port 1 Users Leaf 1, port 10 Users

Destination EPG

Leaf 3, port 2 Servers Leaf 4, port 8 Servers Leaf 5, port 12 Servers Leaf 2, port 12 Users

Network Admin

Add client 172.18.20.13, use existing ASA instance

Remove client 192.168.1.1

Security Admin Insert ASA instance in the service

graph with desired policies

Same 5 service rules and actions

ASA1 Clients

Port Rules

access-list OUT permit tcp any any eq 80 access-list OUT permit tcp any any eq 443 access-list OUT permit tcp any any eq 135 access-list OUT permit tcp any any eq 22 access-list OUT permit icmp any any

(101)

(102)

ACI Terminology

Consumer Side Provider Side

Concrete Device

Logical Device

consumer EPG Provider EPG

(103)

Bridge Domain Outside Bridge Domain Inside

Client EPG _{Server EPG}

Service Graph

Contract

Provider Consumer

For Consistency with ACI Policy Model

ARP Flooding

Unknown Unicast Flooding No IP Routing

ARP flooding

Uknown unicast flooding No IP Routing

Provider Side Consumer Side

Default Gateway for the Servers

(104)

Bridge Domain Outside Bridge Domain Inside

Client EPG _{Server EPG}

Service Graph Contract

Provider Consumer

External Router IP

Default gateway for the servers

Firewall or Load Balancer in Transparent Mode

ARP Flooding

ARP flooding

Uknown unicast flooding No IP Routing

(105)

Bridge Domain Outside Bridge Domain Inside L3Out

L3InstP Server EPG

Service Graph

Contract Provider Consumer

VRF

ARP Flooding

Subnet: Default Gateway for L4-L7 Device Hardware Proxy

Default Gateway for the Servers

(106)

Bridge Domain 1 Bridge Domain 2

L3Out

L3InstP Server EPG

VRF

Subnet Is Default Gateway for Server

SVI

Service Graph

SVI

Bridge Domain 3

Subnet Is Default Gateway for Load Balancer

Unicast Routing Enabled SVI

(107)

(108)

APIC

Install the Device Package on the APIC

• APIC interfaces with the device using python scripts

• APIC calls device specific python script function on various events

• APIC uses device configuration model provided in the device package to pass appropriate configuration to the device scripts

• Device script handlers interface with the device using its REST or CLI interface

Device Spec (XML) Device Script (Python / CLI) Uses Device’s native API

(109)

Make sure that the L4L7 Device has management

access

Enable SSH Enable HTTP access Configure Credentials No need to configure: Interfaces VLANs IPs

(110)

“Users”

Prepare the Fabric

• Define a VLAN pool

• Define a Physical domain for

Physical Appliances

• Define a Virtual domain for Virtual

Appliances

• Map them to the ports where the

(111)

BD1 BD2 BD3

EPG client EPG Server

Bridge Domains Configuration

Creation of the link between the L4L7 Devices

Service Graph

(112)

Bridge Domain Configuration

(113)

Bridge Domains Configuration

Setting the correct flooding and routing parameters

• Bridge Domains used to connect Service Devices are configured as follows:

• Disable Unicast Routing

• (Make sure that the Fabric doesn’t provide the SVI on this bridge domain)

• Enable Unknown unicast flooding

• Enable ARP flooding

• The subnet definition on the Bridge Domain is irrelevant

(114)

Bridge Domains Configuration

Establishing the relation with the VRF

Bridge Domain 1 Bridge Domain 2

Association with VRF

BDs association with the VRF is because of the Object Model.

The VRF is not doing anything unless the admin enables routing on the BD and configures the Subnet for the default gateway function

(115)

Configure The Concrete Device

Routed or bridged Physical or Virtual Domain Firewall credentials

Select the port-grups for producer and consumer side

(116)

Concrete and Logical Device Definition

Configure the Connectivity between the L4L7 device and ACI

E.g. Po11

(must be a syntax which works on ASA)

Both producer and consumer vPC

(117)

Concrete and Logical Device Definition

On APIC enter the port-channel configuration for the L4L7 device

(118)

Concrete and Logical Device Definition

(119)

Function Profiles are XML schema similar to iApp with F5

Configuration of the “Function/s” on the L4L7 Device

Create Function Profiles

Function Profiles can be:

• WebProfile • HTTPS • Application-1 Click to configure L4-L7 Service Node Configurations

(120)

Configuration of the “Function/s” on the L4L7 Device

Choose a Template

Templates gives you option to choose simple Service Graph based on your requirement

(121)

Configuration of the “Function/s” on the L4L7 Device

(122)

Configuration of the “Function/s” on the L4L7 Device

Select the EPGs

(123)

Function Node configuration will be pushed from this pane

Configuration of the “Function/s” on the L4L7 Device

Enter the L4L7 Parameters

(124)

Function Profiles are XML schema similar to iApp with F5

Or use Function Profiles

Function Profiles can be:

• WebProfile • HTTPS • Application-1 Click to configure L4-L7 Service Node Configurations

(125)

Configuration of the “Function/s” on the L4L7 Device

Device Configurations versus Functions Configurations

Device Config: configuration pertain to the BIG-IP partition, for example, pool config that can be utilized by multiple virtual servers

Function Config: configuration specific to the virtual server, like the VIP; or it can reference device config, in this case, the Pool

(126)

Configuration Checklist

• These are the steps to verify to successfully configure the Service Graph:

• Verify that you created bridge domains to create the path between the service devices

• Make sure that the bridge domain has an association with a VRF

• Verify that the bridge domain is operating in flood and learn mode

• Make sure you created a physical or virtual domain to enable the VLANs on the leaf devices

• Make sure that there is a client and a server side EPG

• Make sure that the service graph is associated with a contract that is connecting the client and the server side EPG

(127)

(128)

Device Package

(129)

Upcoming L4-L7 device packages

Partner ETA Certified APIC release

Q3 CY15 FCS+6 Q3 CY15 FCS+6 Q3 CY15 FCS+9 Q3/Q4 CY15 FCS+6 TBD FCS+6 TBD

(130)

Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at

CiscoLive.com/Online

•

Give us your feedback to be

entered into a Daily Survey

Drawing. A daily winner

will receive a $750 Amazon

gift card.

•

Complete your session surveys

though the Cisco Live mobile

app or your computer on

(131)

Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Table Topics

• Meet the Engineer 1:1 meetings

(132)

(133)

(134)

Data Center / Virtualization Cisco Education Offerings

Course Description Cisco Certification

Cisco Data Center CCIE Unified Fabric Workshop (DCXUF);

Cisco Data Center CCIE Unified Computing Workshop (DCXUC)

Prepare for your CCIE Data Center practical exam with hands on lab exercises running on a dedicated comprehensive topology

CCIE®_{Data Center}

Implementing Cisco Data Center Unified Fabric (DCUFI);

Implementing Cisco Data Center Unified Computing (DCUCI)

Obtain the skills to deploy complex virtualized Data Center Fabric and Computing environments with Nexus and Cisco UCS.

CCNP®_{Data Center}

Introducing Cisco Data Center Networking (DCICN); Introducing Cisco Data Center Technologies (DCICT)

Learn basic data center technologies and how to build a data center infrastructure.

CCNA®_{Data Center}

Product Training Portfolio: DCAC9k, DCINX9k, DCMDS, DCUCS, DCNX1K, DCNX5K, DCNX7K

Get a deep understanding of the Cisco data center product line including the Cisco Nexus9K in ACI and NexusOS modes

For more details, please visit: http://learningnetwork.cisco.com

(135)

Cloud Cisco Education Offerings

Course Description Cisco Certification

Designing the FlexPod Solution (FPDESIGN); Implementing and Administering the FlexPod Solution (FPIMPADM)

Learn how to design, implement and administer FlexPod solutions FlexPod Design Specialist; FlexPod Implementation & Administration Specialist UCS Director (UCSDF) Learn how to manage physical and virtual infrastructure using

orchestration and automation functions of UCS Director.

Cisco Prime Service Catalog Learn how to deliver data center, workplace, and application services in an on-demand, automated, and repeatable method.

Cisco Intercloud Fabric Learn how to implement end-to-end hybrid clouds with Intercloud Fabric for Business and Intercloud Fabric for Providers.

Cisco Intelligent Automation for Cloud Learn how to implement and manage cloud deployments with Cisco Intelligent Automation for Cloud

For more details, please visit: http://learningnetwork.cisco.com