https://indico.cern.ch/event/401680/
P2P Service Trial Session Notes
Amsterdam LHCONE meeting, Oct. 28-29, 2015
1 Introduction – Gerben van Malenstein, SURFnet
The general activities of the service trial are to 1) Connect sites to AutoGOLE for P2P trials
2) Investigate middleware integration of the virtual circuit The situation P2P service as of the Amsterdam meeting was
For sites supporting OpenFlow, Caltech has an LHCONE P2P prototype For sites using IP routers, NORDUnet has a BGP-based approach
2 Dynamic P2P Service with BGP Route Servers – Magnus Bergroth,
NORDUnet
Current situation: P2P circuits mostly connect sites rather than applications
Dynamic P2P circuits have end points that typically terminate on a site aggregation router o In other words, P2P circuits are typically used to connect two sites
eBGP is configured on the interface at each site
o Reachability is advertised after the P2P link is up and BGP established There are several problems with this
o A full mesh of BGP sessions must be maintained o Extensive configuration is required
o BGP sessions on short-lived P2P circuits are down most of the time which causes alarms Proposal: Use a Route Server based approach
One BGP session per site minimizes configuration BGP session is always up
Prefixes are learned via BGP only when dynamic P2P is up
o Use BFD connection up/down condition to prevent BGP advertisement if condition (“up”) is not met o Put BGP with conditions in route server
o Fast detection when dynamic P2P goes down
If a link goes down, how long for route server to detect? MB: seconds. Route server provides one RIB per site with steering done using communities Edoardo Martelli experimented with
o Creating a route server at CERN
2 o Route server stores, for each peer, a Next-Hop Information Base (NHIB) giving reachability for all
next-hops
o Client routers can verify connectivity to other sites using BFD (Bidirectional Forwarding Detection) Several project are under way that could implement this:
o BIRD Internet routing daemon – a project of Charles University, Prague, CZ
BIRD is fairly widely used in Europe as a route server as well as the routing table manager in some open-source routers. See https://en.wikipedia.org/wiki/Bird_Internet_routing_daemon for an
introduction.
o Quagga routing software suite – http://www.nongnu.org/quagga/
o Several issues of interest to our P2P usages have been communicated to the BIRD and Quagga developers for changes
3 Progress on SDN, OVS and dynamic circuits – Ramiro Voicu, Caltech
Current situation: OpenFlow based P2P circuits and application to application experienceUsing dynamic circuits in PhEDEx allows for more deterministic workflows, useful for co-scheduling CPU with data movement
o Integrating circuit awareness into the FileDownload agent
o Application is backend agnostic; No modifications to PhEDEx DB o All control logic is in the FileDownload agent
o Transparent for all other PhEDEx instances
o The approach is to create a shim in the storage access stack for PhEDEx so that all circuit control logic is transparent to the application, including circuit failure.
A testbed consisting of AutoGOLE sites interconnected by OSCARS NSI circuits
Progress in OpenFlow+SDN technology
Open vSwitch (“OVS”) is a production quality, multilayer virtual switch o OpenFlow protocol support (1.3)
3
o Fine grained QoS
Ingress qdisc (packet scheduling), HTB (a qdisc class that allows for packet filters), HFSC, (Hierarchical Fair Service Curve – classful shaping and scheduling) /2/
o Interoperates with OpenStack (which manages a pool of hardware resources in a Cloud Computing environment)
o Open Virtual Network “OVN” – “…complements the existing capabilities of OVS to add native support for virtual network abstractions, such as virtual L2 and L3 overlays and security groups.” /3/
Using OVS for end-host orchestration Integrating PhEDEx with Dynamic Circuits for CMS
o Standard OpenFlow (or OVSDB) protocol for end-host network orchestration (no need for custom SB (switch and router control communication protocol /4/)
o Simple procedure to migrate to OVS on the end-host. SDN controller not required in the initial deployment phase
Traffic Shaping with Open vSwitch (OVS) WAN tests over NSI o OVS 2.4 with stock kernel
o NSI circuit Caltech -> UMICH (~60ms) o Very stable up to 7.5Gbps
o Fairly good shaping above 8Gbps (small instabilities)
Open vSwitch (OVS) benefits
o Standard OpenFlow (and/or OVSDB) end-host orchestration o QoS SDN orchestration in non-OpenFlow clusters
o OVS works with stock SL/CentOS/RH 6.x kernel used in o HEP; works out-of-the-box on SL7/CC7
o OVS bridged interface achieved the same performance as the hardware (10Gbps) o No CPU overhead when OVS does traffic shaping on the physical port
o Traffic shaping (egress) of outgoing flows may help performance in such cases when the upstream switch (or ToR) has smaller buffers
Open Daylight (ODL) controlling Open vSwitch via OVSDB and OpenFlow
o OVSDB – Open vSwitch Database Management is for managing and configuring Open vSwitch instances /5/
4 o Used to create the virtual bridges
o Virtual bridges can use standard OF to speak with the controller o Normal routing if the controller is down
There follows a discussion of
o OpenFlow topology discovery in a network with non-OpenFlow islands o OpenFlow islands over WAN & NSI circuits
o Possible internetwork architecture with a “single” controller Glossary
/1/ Qdisc, …: See “Components of Linux Traffic Control” at http://tldp.org/HOWTO/Traffic-Control-HOWTO/components.html#c-filter
/2/ HFSC: see “HFSC - Linux traffic shaping's best kept secret” at https://gist.github.com/bradoaks/940616 /3/ OVN: see “OVN, Bringing Native Virtual Networking to OVS” at
http://networkheresy.com/2015/01/13/ovn-bringing-native-virtual-networking-to-ovs/
/4/ SB: “In a software-defined network (SDN) architecture, southbound application program interfaces (APIs) are used to communicate between the SDN Controller and the switches and routers of the network. They can be open or proprietary.” See https://www.sdxcentral.com/resources/sdn/southbound-interface-api/
/5/ RFC 7047 on The Open vSwitch Database Management Protocol: “Open vSwitch is an open-source
software switch designed to be used as a vswitch (virtual switch) in virtualized server environments. A vswitch forwards traffic between different virtual machines (VMs) on the same physical host and also forwards traffic between VMs and the physical network. Open vSwitch is open to programmatic extension and control using OpenFlow and the OVSDB (Open vSwitch Database) management protocol. This document defines the
5
OVSDB management protocol. The Open vSwitch project includes open-source OVSDB client and server implementations.
The schema of the Open vSwitch database is documented in [DB-SCHEMA]. This document specifies the protocol for interacting with that database for the purposes of managing and configuring Open vSwitch instances.
4 GÉANT BoD Service Evolution; Introducing SDN capabilities in backbone -
Mian Usman, GÉANT
Current
BoD implementation
o JunOS Space is the device controller for BoD and does not work well enough to use in production o Deployment is in a set of waves parallel to the production network
o Path can be mapped to NSI DynPaC
o New service manager for introduction of new services into network Move flows to alternate paths, disaggregation, resiliency
o Will migrate to ONOS /1/ controller
WEJ: What does this imply for support of BGP-based hybrids that are currently the norm because very few sites are production SDN capable?
Glossary
/1/ ONOS: Open Network Operating System, is a newly released open-source SDN controller that is focused on service provider use-cases. Carrier organization is developing.
http://sdnhub.org/tutorials/onos/
“What difference between ONOS and ODL?"
“1) The major differences between the two is that ODL focuses on bringing legacy (BGP, SNMP, and such) and new networks (i.e., OpenFlow and SDN) together whereas ONOS focuses on the performance aspects and the clustering to increase the availability and scalability. I heard that ODL recently started to enhance their
performance and Lithium release begins to support clustering. So I think both of them can learn and benefit from each other. So I can safely say the major difference is their focuses, but they could change overtime. “2) ONOS focuses more on carrier-grade networks and many telcos are involved in their projects. ODL has more vendors like Cisco, Juniper, and NEC than ONOS. Real field deployments are not very common for both of them as far as I know.
- Justin Park, SDN researcher, Daejeon, Republic of Korea.”
5 Automated GOLE update – Gerben van Malenstein, SURFnet
AutoGOLE fabric delivers dynamic network services between GOLEs and networkso Based on NSI Connection Service v2.0 and TLS o Using DDS /1/ service between aggregators o Used for numerous projects
Work in progress includes
6 AutoGOLE Dashboard
Prototype exists – e.g. http://dashboard.lab.uvalight.net:8000/dpm o Supporting LHC Sites
Supporting LHC sites that are connecting to the AutoGOLE o More redundant control plane
Forwarding requests to another aggregator if a certain uPA cannot be reached Future work
o Getting rid of old common VLAN range
Support for VLAN retagging needed in all GOLEs and networks
Ultimately using ENNI /2/ ports between networks (requires a new service definition) o Further data plane improvements, e.g. full dynamic control over long-haul links
o Shifting from manual/NOC operation to ‘dynamic provisioning by default’
Glossary
/1/ DDS - Document Distribution Service: A simple peer-to-peer flooding protocol for exchange and distribution data documents between NSA within the interconnected control plane or “document space”. Supports both polling and subscription based notification mechanisms for exchange of documents. Documents that might be distributed: topology descriptions, NSA identities/locations, Service Definition, etc. See “Network Services, Interface Document Distribution Service” John MacAuley, ESnet, Open Grid Forum presentation. https://redmine.ogf.org/dmsf_files/13404?download=
/2/ External Network-to-Network Interface (ENNI) for service availability Simple 2-carrier and multi-carrier service models
Ethernet Virtual Connection (EVC) between UNIs spread across multiple networks Operator Virtual Connection (OVC) for each EVC segment
o Key ENNI requirements
Services: p2p and mp EVC types, single and multiple CoS per EVC Encapsulation: Standard S-Tag frame format
Scalability: 1 and 10 Gbps PHYs
ENNI Protection: 2-link LAG, active/standby, LACP
o See, “Developing Ethernet Services Offerings with MEF Specifications,” Dmitry Dergalov, Technical Director, RAD Data Communications-Russia.
