Cloud Networking Disruption with Software Defined Network Virtualization
Ali Khayam
In the next one hour
Let’s discuss two disruptive new paradigms in the world of networking:
– Network Virtualization
– Software Defined Networking
Why virtualize the network?
Let’s first ask:
Why is server virtualization so popular?
Virtualizing compute, I/O and memory on a machine as a pool of physical resources which can be assigned to
different VMs on-demand.
Hypervisor
Why is server virtualization so popular?
Avoids underutilization of physical resources.
Allows safe application deployment and multitenancy on the same machine.
Reduces operational costs as VMs can be spun up, shut down and migrated at will
User experience is not compromised as a VM looks and feels exactly like a local machine.
Why virtualize the cloud network?
Server virtualization provides multitenancy, but
networking resources are generally hidden from the tenants
– Basic primitives for VM connectivity are supported… and that’s about it
What if a cloud’s physical networking resources
(switches, routers, firewalls) could be segmented and given to tenants on-demand?
Why virtualize the cloud network?
Multitenacy
Virtualizing the network resources would require defining an overlay network on top of the actual physical cloud network
– E.g., VM1 thinks it communicates with VM2 directly (one hop), but there are several physical hops between them.
The overlay concept is similar to p2p networks, but network virtualization must go beyond that because:
– A tenant’s traffic ownership and semantics must be preserved en-route.
– A tenant’s traffic must be segmented from other tenants.
– A tenant should be allowed to define custom overlay topologies.
Why virtualize the cloud network?
Maximize utilization of networking resources
Cloud operators would want to maximize the usage of networking resources:
200,000 servers in a public cloud data center Fanout of 20 10,000 switches
$5k commercial switch $50M
Networking CAPEX in 10 data centers = $500M
Oversubscription of networking resources is effective only when traffic is spread out through the data center
– Cloud traffic is unpredictably bursty and its not easy to spread VMs out in real-time.
Why virtualize the cloud network?
Fault Localization
If a tenant’s virtual network breaks down for any reason, services of other tenant’s should not be impacted.
Application Agility
Network virtualization would allow easy orchestration and deployment of new applications and services
– Lower hardware costs
– Lower IT installation costs
What is network virtualization?
What is network virtualization?
A network comprises of a bunch of networking devices:
– Switches, routers, firewalls, load balancers, etc.
These are physical networking resources.
Let’s think about how we can virtualize these resources so they can be segmented and given to different
tenants on-demand.
But first one slide on what we would expect from such a network virtualization solution.
What would be expect from a network virtualizer?
Multi-tenancy through network segmentation
Different tenants’ network segments should be completely isolated:
– Allowing overlapping topologies, IP addresses, etc.
– No spill over from one tenant’s traffic domain to another.
Tenants should be allowed to define their own topologies:
– Example 1: I want my 300 VMs to be connected to the same switch.
– Example 2: I want my VMs to be connected to one switch, but the servers should be connected to a router.
Uncompromised user experience:
– Same user experience as if the resources were present locally.
What is network virtualization?
Back to how we can virtualize these resources so they can be segmented and given to different tenants on- demand.
First things first:
How does a physical switch work?
Machine 1 Machine 2 Machine 3
How does a physical switch work?
M1 M2 M3
1. Pkt from M1 for M3
2. Switch doesn’t know where M3 is connected, so it floods on all the ports.
3. M3 responds and switch learns the MAC addresses of M1 and M3.
How can we build a virtual switch?
Rack1
Server1
VM1
Rack2
Server2
VM2
Rack3
Server3
VM3
VM1, VM2 and VM3 should feel like they are connected to the same
(virtual) switch
How can we build a virtual switch?
Rack1
Server1
VM1
Rack2
Server2
VM2
Rack3
Server3
VM3
1. Pkt from VM1 for VM3 2. Switch doesn’t know where VM3 is
connected, so it unicasts to all the VMs. 3. M3 responds and both switches learn the MAC addresses of VM1 and VM3.
How can we build a virtual switch?
VM1 VM2 VM3
Virtual Switch
How can we build a virtual switch?
We can also extend this basic virtual switch to provide the advanced switching functionality:
– VLANs – QoS
– Secure MAC learning – Trunking
– Tunneling
– CLI- and web-based configurations – SNMP …
Using network virtualization, we “can”….
Virtualize any networking resource:
– Routers, firewalls, load balancers, etc.
Design networking elements with a large number of (possibly infinite) ports
– E.g., an entire network connected to a ‘big’ switch, router or firewall
Using network virtualization, we “should”….
Be able to define arbitrary interconnects of virtual networking elements contained within a Virtual Network Domain
– i.e. a self-contained network abstraction equivalent to a Virtual Machine in the hypervisor domain
The Dream
Decoupling of virtual data centers from physical data centers
Extensible technology to deliver new Network Functions
Changing gears to Software Defined Networking….
Why software-defined networking?
What is SDN?
SDN moves networking technology from hardware boxes to software modules
A purist SDN solution should software from a 3rd party to programmatically control traffic on another its
network devices.
Note that network virtualization is not necessarily software defined
– There are plenty of hardware based network virtualization solutions in the market right now
Why do we need SDN?
Agility in network design
At the moment, it is difficult (at times impossible) to design/test new network protocols, architectures and
algorithms
Freedom from Vendor Lock-in
Vendor lock-in stifles innovation and drives costs up
Agility in
Innovation
How is software-defined networking implemented?
Data Plane
Switch/Router
Control Plane
Data Plane
Switch/Router
Control Plane
Data Plane
Switch/Router
Control Plane
How is SDN being implemented?
Data Plane
Switch/Router
Control Plane
• Break CP-DP association
• Open up the platform
Control Plane
Control Plane Control Plane
Control Plane
Switch/Router
Switch/Router Switch/Router
How is SDN being implemented? (The Openflow Model)
Switch/Router Data Plane
Data Plane Data Plane
Data Plane
• Provide an API/protocol to access it
• Introduce simple homogeneous data plane definition
How is SDN being implemented? (The OpenFlow Model)
Switch
Port MAC
src
MAC dst
Eth VLAN type
ID
IP Src
IP Dst
IP Prot
TCP sport
TCP dport
Rule Action Stats
1. Forward packet to port(s)
2. Encapsulate and forward to controller 3. Drop packet
4. Send to normal processing pipeline 5. Modify Fields
Packet + byte counters
The most prominent SDN DP API Standardization effort is OpenFlow OpenFlow 1.0 Flow Format
Now let’s ask…
Are we free from vendor lock-in?
Has innovation in networks become more agile because of SDN?
Are we free from vendor lock-in?
How to insert a new (3rd party) network function in the OF SDN model?
Data Plane Controller Controller Closed Network
Functions
Closed Network Functions
Southbound API + Extensions
Management APIs
Northbound APIs
Hardware Box basednetworking vendors Software basednetworking vendors
Has innovation in networks become more agile?
How to insert a new (3rd party) network function in the OF SDN model?
Data Plane SDN Controller SDN Controller Closed Network
Functions Closed Network
Functions 3rd party Network
Function
3rd party Network Function
How to talk to a closed NB API?
SB API is available, but how to introduce new DP changes?
From one vendor lock-in
to another
Has innovation in networks become more agile?
How do we support new wire protocols in the DP?
How do we support new architectures that require functionality to be split between CP and DP?
How do we avoid race conditions if a 3rd party network function talks directly to a DP that is already talking to a proprietary controller?
. . .
How can software defined network virtualization save the day?
Let’s go back to the basics and ask…
How can the SDN and the network virtualization co- evolution be poised for agility not conformity?
How can SDN solve vendor lock-in once and forever?
How can SDN support agile virtualized network functions for protocols of the past, the present, and the future?
Design for agility, not conformity
Let’s learn from similar experiences in other technologies
“designed for innovation”.
1 9 6 3
1 9 7 0 ATT builds Unix and C
1 9 8 0
Widespread adoption and enhancements from community
1 9 8 9 ANSI C Standard
1 9 8 7
1 9 9 7 Unix SUS Standard
Standardization is the final step, as it stifles innovation
Break vendor lock-in by seamlessly support protocols of the past, the present, and the future
Change mindset from APIs to flexible DP Languages which allow easy definition of new protocols and architectures.
Languages transcend beyond business constraints:
If the language is semantically rich then the community can write compilers and use it on different platforms.
This code is also inherently portable across platforms.
Data Plane
Switch/Router DP API DP API
Data Plane
Switch/Router DP API DP API
Data Plane
Switch/Router HAL HAL
Control Plane Control Plane
Break vendor lock-in by seamlessly support protocols of the past, the present, and the future
Data Plane
Switch/Router DP API DP API
Distributed Messaging fabric
Software Defined Control Plane with Open NB Interfaces
Software Defined Data Plane
Thank you
PLUMgrid’s Islamabad Office is Hiring!
hr.isb@plumgrid.com
How do you implement network virtualization?
What are we missing here?
① There needs to a central database which know about the location and owner of every single VM in the data center and can then communicate this information to the switches.
② Packet headers will be changed by intermediate
network entities (switches, routers, NAT, proxy) while the packet is passing through the network.
Centralized Control
① There needs to a central database which know about the location and owner of every single VM in the data center and can then communicate this information to the switches.
Solution:
This central database is maintained at an entity called the controller.
When a VM’s state changes (migration to another server, power on/off), the central controller encodes the right rules in the right switches.
OpenFlow is a protocol that allows the controller and switches to talk to each other.
Centralized Control
Data Path (Hardware)
Control Path OpenFlow (Flow Tables) Data Path (Hardware)
Control Path OpenFlow (Flow Tables)
Data Path (Hardware)
Control Path OpenFlow (Flow Tables)
Control Path OpenFlow (Flow Tables)
OpenFlow Controller
OpenFlow Protocol (SSL) OpenFlow Protocol (SSL)
OpenFlow 1.0: Rule Format
Switch Port
MAC src
MAC dst
Eth type
VLAN ID
IP Src
IP Dst
IP Prot
TCP sport
TCP dport
Rule Action Stats
1. Forward packet to port(s)
2. Encapsulate and forward to controller 3. Drop packet
4. Send to normal processing pipeline
+ mask
Packet + byte counters
OpenFlow 1.0: Limitations Single table support Rigid flow definition
Single controller support
OpenFlow 1.2/1.3
Pipelined flow tables
Table 0 Table 1 Table n
Action set = {}
Packet In
Action set = {a1}
Action set = {a1, a2, …}
Execute Actions
Packet Out Packet + ingress
port + Metadata Packet + ingress port + Metadata
Packet + Metadata
OpenFlow 1.2/1.3
OpenFlow Extensible Matching (OXM)
class fields mask
OpenFlow 1.2/1.3: Multiple Controllers’ Support
Flow Table 0 Controller 1
Flow Table n Group
Table Channel
Switch
pipeline
Controller 2 Controller k
What are we missing here?
① There needs to a central database which know about the location and owner of every single VM in the data center and can then communicate this information to the switches.
② Packet headers will be changed by intermediate
network entities (switches, routers, NAT, proxy) while the packet is passing through the network.
What are we missing here?
② Packet headers will be changed by intermediate
network entities (switches, routers, NAT, proxy) while the packet is passing through the network.
Solution:
Tunnel the packets so that all the L2, L3, L4, L7 headers of the VM’s packet are preserved.
Tunneling: NVGRE
Outer Ethernet Header
Outer IP
Header GRE Header Inner Ethernet Header
Inner IP Header
|0| |1|0| Reserved (10 bits)
Tenant Network Identifier (TNI) (24 bits)
Reserved (8 bits) Version
(3 bits) Protocol Type 0x6558 (16 bits)
Tunneling: VXLAN
Outer
MAC Outer IP Outer UDP L2 packet Outer
CRC VXLAN
header
R|R|R|R|I|R|R|R| Reserved (24 bits)
Virtual Network Identifier (VNI) (24 bits)
Reserved (8 bits)
Tunneling: STT
VM 1
VM 2 Soft Switch NIC VM 1
Soft Switch VM 2
NIC
Switch-Switch Tunnel Network
Tunneling: STT
Source Port 16 bits
Destination Port 16 bits
|U|A|P|R|S|F|
Reserved |R|C|S|S|Y|I| Window
|G|K|H|T|N|N|
84 bits Data Offset
12 bits
Sequence Number * 32 bits
Acknowledgement Number * 32 bits
Checksum 16 bits
Urgent Pointer 16 bits Options
24 bits
Padding 8 bits
TCP Header
Tunneling: STT
Version 8 bits
Flags 8 bits
L4 Offset 8 bits
Reserved 8 bits Maximum Segment Size
16 bits
PCP 8 bits
V 1 bit
VLAN ID 12 bits
Context ID 64 bits Padding
16 bits Data
STT’s TCP-Like Header
Host Hypervisor Guest VM
Tunneling: STT
Application TCP/IP stack
Soft Switch
NIC Virtual NIC
Network Network
Application DataStream
Application DataStream TCP IP
Application DataStream TCP IP L2
Application DataStream TCP IP L2 STT IP
Data STT IP L2 Data STT IP L2
Port