State and Specification Abstraction: Implemented by SDN controllers

(1)

Communication Systems Group (CSG)

State and Specification Abstraction:

Implemented by SDN controllers

Bernhard Ager

Parts are based on slide sets by Scott Shenker, Nick Feamster, Rob Sherwood, Jeremy Stribling, or as attributed

(2)

OpenFlow programming is not easy

§ Parallel tasks hard to implement (e.g., routing, monitoring)

§ OpenFlow is a very low-level abstraction

§ Controller only sees packets when the switch does not have a rule

(3)

Why SDN?

§ Why is SDN the right choice for the future?

§ Obviously efficiency, scalability, security, functionality, versatility, …

§ Well, not directly …

§ What are the fundamental aspects of SDN?

§ Obviously OpenFlow …

(4)

„Northbound API“ helps

§ Abstracts away low-level details § Enables quick customization using popular languages § Vendor independent https://www.sdncentral.com/technology/the-sdn-gold-rush-to-the-northbound-api/2012/11/

(5)

Abstractions should separate 3 problems

§ Constrained forwarding model

§ Distributed state

§ Detailed configuration

(6)

Forwarding abstraction

§ Flexible forwarding model

§ Behaviour specified by control program, not by forwarding abstraction

§ Abstract away forwarding hardware

§ For evolving beyond vendor-specific solutions

§ Flexibility and vendor-neutrality both valuable

§ One architecturally, the other economically

(7)

State distribution abstraction

§ Control programs should not have to deal with problems caused by distributed state

§ Complicated, source of errors

§ Abstraction should hide state distribution details

§ Natural abstraction: global network view

§ Control program works on global view

§ Input: global view, e.g., network graph

(8)

Specification abstraction

§ Control program should express desired behaviour

§ Control program should not be responsible for implementing that behaviour

§ Natural abstraction: work on simplified model of network

(9)

Network Operating Systems (NOS)

§ Distributed system that creates network view

§ = State abstraction

§ Runs on servers in the network

§ Communicates with forwarding elements

§ Get state from forwarding elements

§ Send control directives to forwarding elements

§ Utilizes forwarding abstraction

§ Control program works on view of network

§ Doesn‘t have to be a distributed system

(10)

NOX: Towards an OS for Networks

§ The first take on a NOS

§ Targeted to implement the abstractions described before

§ Implemented in C++, Python runtime on top

Natasha Gude, Teemu Koponen et al, SIGCOMM CCR, 2008.

(11)

(12)

NOX programming interface

§ Network view and namespace

§ Writes to network view should be limited

§ Monitoring DNS allows mapping host names to flows

§ Events

§ Event handlers handled according to priority

§ Control: OpenFlow

§ Higher-level services

§ System libraries for routing, packet classification, standard services (DHCP and DNS), network filtering

(13)

NOX‘s network view

Network view contains

§ Switch-level topology

§ Locations of users

§ Locations of hosts, middleboxes, other network elements

§ Bindings between names and addresses

§ DNS snooping

Not in network view

(14)

NOX design overview

§ Granularity: Scalability vs. flexibility

§ Aware of switch-level topology, user locations, middleboxes, services, …

§ Forwarding management on flow level

§ Switch abstraction and operation

§ Adopts OpenFlow model

§ Scalability:

§ Leverage parallelism for flow arrivals

§ Maintain centralized network view: indexed hash tables, distributed access with local caching

(15)

NOX usage example: Policy lookup

Enables

§ User-based VLAN tagging

§ User-based traffic isolation

§ Ethane

(16)

Similar event-based controllers

§ POX: NOX in Python

§ Diverse (reusable) examples included in source code

§ Ryu: Python

§ Support for OpenStack

§ Web-based GUI

§ Replication support

§ Supports OF, SNMP, Netconf, OpenVSwitch, …

§ Trema: Ruby/C

§ comes with built-in simulator for testing

(17)

POX programming example

def _handle_PacketIn (self, event): msg = of.ofp_packet_out()

msg.buffer_id = event.ofp.buffer_id msg.in_port = packet_in.in_port

msg.match = of.ofp_match.from_packet(packet)

action = of.ofp_action_output(port = of.OFPP_FLOOD) msg.actions.append(action)

self.connection.send(msg)

(18)

These controllers don‘t solve all the problems

§ Managing state

§ Strong consistency?

§ Persistence?

§ Scalability?

§ Reliability and robustness?

§ Generality

(19)

ONIX

§ Network operating system (closed-source)

§ Design goals: Generality, Scalability, Reliability, Simplicity, Performance

§ Used in the Google backbone

§ Speculation: asset for Nicira deal (1.26 Billion $)

(20)

ONIX components

(21)

ONIX contributions

§ Centralized logic on distributed system

§ Scalability and robustness

§ „Useful“ tools to application developers

§ Simple, general API

§ Flexible state distribution mechanisms

§ Flexible scaling and reliability mechanisms

(22)

Onix API: Data-centric

§ Program on a network graph

§ Nodes are physical network entities

§ Attributes on edges and nodes

§ … represented by Network Information Base (NIB)

§ External changes imported to NIB, NIB notifies application

§ Local changes exported from NIB to network elements

(23)

Onix state abstraction(s)

§ Different requirements depending on application

§ Strong consistency vs. eventual consistency

§ Different storage options

§ Replicated transactional (SQL) storage

§ Distributed hash table (DHT)

§ Storage requirements specified at startup

§ But during run-time only interact with NIB

§ Strong consistency for critical, stable state

(24)

Queries to the NIB

Category Purpose

Query Find entities

Create, destroy Create and remove entities Access attributes Inspect and modify entities

Notifications Receive updates about changes Synchronize Wait for updates being exported to

network elements and controllers

Configuration Configure how state is imported to and exported from the NIB

Pull Ask for entities to be imported

(25)

Distributed Coordination

§ Distributed control programs need to synchronize NIB access themselves

§ Zookeeper integration

§ Apps can use it for:

§ Leader election

§ Distributed locking

(26)

Onix scaling

Support for distributed controllers

§ Multiple dimensions for partitioning

§ By task

§ By subsetting NIB: only subset of NIB locally available

§ Switches connect to subset of ONIX controllers

§ By aggregation

§ Partition network, e.g., by prefix, VLAN, MPLS, network area, …

(27)

Example: ARP server

(28)

Achieving reliability

Network Element &

link failures Application‘s responsibility Connectivity infrastructure failures Assume reliable management connectivity; implement multi-pathing protocol

Onix failures Distributed coordination facilities for app failover

(29)

Use cases

§ Ethane

§ Enterprise-level security control

§ Per-flow decision of embedding

§ Resulting high load handled with DHT

§ Distributed Virtual Switch

(30)

Use cases (2)

§ Multi-tenant data center

§ Virtual L2 networks with data center

§ Isolation in addressing and resource usage

§ Translates to managing tons of tunnels

§ Requires partitioning of NIB due to size

§ Scale-out IP router

§ Scales with number of physical switches

(31)

ONIX does not solve all problems

§ Not clear how to write applications

§ That work on the same header space

§ That need to perform non-local decisions

§ Not clear if NIB abstraction is abstract enough

(32)

§ Open-source Java-based OpenFlow Controller

§ Apache licensed

§ Support by community and Big Switch

§ Offers a module loading system that make it simple to extend and enhance

§ Supports a broad range of virtual and physical switches (OpenFlow and non-OpenFlow)

§ Support for OpenStack cloud orchestration platform

(33)

§ Fork from Beacon OpenFlow Controller

§ Production-level performance but steep learning curve

§ Good documentation

§ Powerful REST API that allows applications to interact with it/switches

(34)

Communication Systems Group (CSG) Floodlight Architecture = Controller + Applications

(35)

§ Controller modules implement functions that are of common use to a majority of applications, such as:

§ discovering and exposing network states and events (topology, devices, flows)

§ enabling controller communication with network switches (i.e., the OpenFlow protocol)

§ managing Floodlight modules and shared resources such as storage, threads, tests

§ a web UI and debug server (Jython)

(36)

(37)

(38)

§ Not just a controller :

§ “The OpenDaylight Project is a collaborative open source project that aims to accelerate adoption of Software-Defined Networking (SDN) and create a solid foundation for Network Functions Virtualization (NFV) for a more transparent approach that fosters new innovation and reduces risk”

§ Industry supported platform

§ Robust and extensible open source codebase

§ Common abstractions for northbound capabilities

§ Integration with OpenStack

§ Cloud applications

§ Production-level performance

(39)

§ Java chosen as an enterprise-grade cross platform compatible language

§ Java interfaces are used for event listening, specifications and forming patterns

§ Maven : Build automation system for Java

§ OSGi :

§ Allows dynamically loading bundles

(40)

(41)

(42)

§ MD-SAL

§ Model-Driven Service Abstraction Layer

§ Performs event routing

§ YANG: XML-based language for service abstraction

§ State: Hierarchical database in MD-SAL

§ Contains all kinds of information, including flow forwarding rules, topology information, etc

§ Plugins can register for change notification events, e.g., the Flow Routing Manager (FRM) registers to flow forwarding rules

(43)

(44)

Network programming languages

Many proposals (and implementations)

§ Frenetic

§ Pyretic

§ Merlin

§ Maple: Algorithmic policies

§ NetCore: predecessor to NetKAT

§ NetKAT: Kleene algebra with test (KAT)

(45)

(46)

Reading state from the network

§ Switch counts pkts/bytes for every switch rule

§ Controller can poll statistics

§ E.g., for traffic engineering

§ Rules can be complex/complicated

§ „All traffic to 192.168.1.1 not targeted at port 80“ needs two flow entries:

1. dstip = 192.168.1.1, dstport = 80

2. dstip = 192.168.1.1 # for monitoring

§ Predicates can be automatically translated

(47)

Further useful primitives

§ Grouping by feature: to construct histograms

§ Limited space in switch tables

§ Run-time can dynamically generate monitoring rules as traffic arrives

§ Example: GroupBy(srcip): One new rule per src IP

§ Limit number of events

§ E.g. more packets arriving while OF rule is installed: Programmer specifies: Limit (1)

(48)

Frenetic: SQL-like query language

§ Example:

§ Select(bytes)

* Where(in:2 & srcport:80) * GroupBy([dstmac])

* Every(60)

Every 60s provide statistics on port 80 traffic via switch port 2

§ Familiar abstraction

§ Result is a stream that can be further processed

§ E.g., to route heavy hitters differently

(49)

Applications interact

Modular SDN programming with Pyretic. J. Reich, C. Monsanto, N. Foster, J. Rexford, and D. Walker. USENIX NSDI 2013

(50)

Explanation of interaction problems

§ Monolithic control programs need to interleave tasks

§ Even simple policy requires programmer‘s finesse

§ Put traffic to 10.0.0.0/8 on output port 2

§ Poll statistics for traffic coming from Internet every 5 seconds

§ Traffic over VLAN 12 should be load balanced every 2 minutes

§ Hard to debug, test, reuse …

§ Problem cannot be solved with slicing of the network!

§ Solution: Modularization

(51)

Approach: Composition

Solve combinatorial problem programmatically

§ Parallel composition: Perform tasks in parallel

§ _{monitor() + forward()}

§ Sequential composition: Perform one task after the other

(52)

Virtualization with Pyretic

§ Pyretic works on „Packets“

§ Actually, these are flow tuples

§ Virtual header fields

§ Label pushing allows to virtualize network

§ Eg. use NAT for tunneling traffic = push IP address: [ 192.168.0.1 ] -> [ 10.0.0.1, 192.168.0.1 ]

§ Useful for certain applications

§ Anonymise packet while packet is inside network

(53)

Topology abstraction

§ Network objects virtualize network topology

§ Map virtual network ports to underlying network ports

§ Perform forwarding between underlying ports

§ Usage examples:

§ “Big switch” comprising the whole network

§ Split one switch into many virtual switches

§ Work on subgraph of actual topology

(54)

Example: “Big Switch”

V

A

C

MAC-Learning Switch Module

Edge links in underlying network (= port in virtual switch)

Packet ingress/egress

Routing in virtual switch/ fabric { switch: A, inport: 3, vswitch: V, ... } { switch: [V, A], inport: [4, 3], vswitch: V, ... }

(55)

Constructing policies

§ Policies control packet delivery

Actions: A ::= drop | identity | fwd(port) | flood

| push(h=v) | pop(h) | modify(h=v)

Filters: F ::= all_packets | no_packets | match(h=v)

| _ingress | _egress| P _& P | (P _| P) | _~P

(56)

Primitive Actions

§ Receive a packet with location information as input and returns a set of located packets as output.

§ Example: Input is _{{ switch: A, inport: 3 }}, output could be _{{{ switch: A, outport: 4 }}}.

§ _drop produces the empty set (no packet is output)

§ _passthrough produces the input packet

§ _fwd(port₎ changes outport

§ _flood floods packet using minimum spanning tree

(57)

Predicates

§ Needed to define conditions on packets

§ If C is a policy and P is a predicate, then C_[P_] means to apply C to all packets for which P is true

§ _{all_packets}, _{no_packets} return true or false, resp.

§ _ingress, _egress return true if packet enters (leaves)

§ _match(h₌v₎ return true if header h has value v

§ P _& P, P _| P, _~P: composition of predicates:

(58)

Query Policies

§ Direct information from phys. network to controller

§ Packets aren’t moved to phys. port on phys. switch

§ Rather, they are put into buckets on controller

§ _counts: packet goes to counts bucket

§ _packets: packet goes to packets bucket

§ Applications are informed about arrival of packets

§ _packets: entire packets, counts: packet counts

§ _{packets(1, [‘srcip’])} passes each packet with new source

address

§ _{counts(every, [‘srcip’])} calls listeners every every seconds with the number of times each source ip has been seen

(59)

Policy Composition

§ Policies can be simple actions (flood etc)

§ Or query policies (not discussed here)

§ Or conditional policies P[C]

§ Applies policy C if the packet satisfies predicate P

§ Composed so that they either act in parallel

§ C1 + C2 (C1 and C2 together)

§ or in sequence

§ C1 >> C2 (first C1, then C2)

§ Or conditionally

(60)

Policies by Example (1)

§ Broadcast every packet entering the network

§ _flood

§ Broadcast all packets entering switch s2 on port 3

§ _{match(switch=s2,inport=3)[flood]}

§ Drop all HTTP packets on switch s3

§ _{match(switch=s2,dstport=80)[drop]}

§ Forward packets on s2 from port 2 to port 3

§ _{match(switch=s2,inport=2)[fwd(3)]}

§ Drop all packets matching some predicate P

(61)

Policies By Example (2)

§ Forward packets from port 2 to port 3 if they fulfill some predicate P

§ _{(match(switch=s2,inport=2) & P)[fwd(3)]}

§ _{if_(P, drop, passthrough)}

>> match(switch=s2,inport=2)[fwd(3)]

§ The combination of _{if_}, _drop, and _passthrough is very convenient for composing policies!

(62)

Examples (1)

from pyretic.lib import * def main():

(63)

Examples (2)

from pyretic.lib import * def printer(pkt): print pkt def dpi(): q = packets(None, []) q.when(printer) return match(srcip=‘1.2.3.4’)[q]

(64)

Examples (2), Detail

§ Process ALL the packets!

def dpi():

q = packets(None, [])

§ When I get a packet, print it!

q.when(printer)

§ Print all packets matching source IP 1.2.3.4

(65)

Examples (3)

From pyretic.lib import * def learn(self): def update(pkt): self.P = if_(match(dstmac=pkt['srcmac'], switch=pkt['switch']), fwd(pkt['inport']), self.P) q = packets(1,['srcmac','switch']) q.when(update) self.P = flood + q

The first time you see new source MAC on a switch, update the policy

(66)

Example (3): Decision tree

self.P =

With every new

MAC address, insert one new decision at the root of the tree.

(67)

Pyretic limitations

§ Dynamicity is limited

§ Re-routing flows not easily possible

§ What does this imply for link-failures?

§ Only available for POX

§ Multiple controllers?

(68)

Network OS and controller conclusion

§ Abstractions are essential to simplify management of complex systems

§ SDN = forwarding, specification, distribution abstractions

§ Network operating systems

§ Simplify managing a network by providing interfaces

§ Don‘t control the network themselves, apps do that!

§ Line between NOS and controller is not sharp

§ Floodlight, Opendaylight, and others call themselves controllers, but provide more services than, e.g., NOX