Extensible Networking Platform
1
1 - Lockwood - Evaluation and Test of Networks with Quality of Service
John W. Lockwood,
Christopher Neely
Christopher Zuver
[email protected]
Reconfigurable Network Group
Applied Research Laboratory
Computer Science and Engineering
Washington University in St. Louis
http://www.arl.wustl.edu/arl/projects/fpx/reconfig.htm
Presented for Boeing QoS Summit, February 16, 2004 – Not classified, Not proprietary
Evaluation and Testing of
Quality of Service in Realistic,
High-Performance Networks
Extensible Networking Platform
2
Quality of Service
Quality of Service
Enables real-time applications
to run over Internet-style networks
Ensures bandwidth properly
allocated among multiple applications
– Can provide virtual link with end-to-end
guarantee on bandwidth and delay
– Can drop or re-order packets under heavy load
Is a Zero-sum game :
– Some flows receive better service
– Other flows receive degraded service
Extensible Networking Platform
3
3 - Lockwood - Evaluation and Test of Networks with Quality of Service
Example of QoS - RSVP
RSVP daemon
Makes reservation
Admission control
Checks for resources
Policy control
Checks permissions
Packet classifier
Determines packet QOS
Packet scheduler
Order packets for QOS
Packet
Scheduler
Packet
Classifier
Data
Admission
Control
Policy
Control
RSVP
Daemon
Applications
Extensible Networking Platform
4
QoS in Distributed and Mobile Networks
Challenges with QoS in distributed and mobile networks
Bandwidth can change on wireless links due
to variable channel noise and radio errors
Network topology can change as network
component added, relocated, or destroyed.
Applications adapt to make best use of available resources
Copyright John W. Lockwood All Rights Reserved
Copyright John W Lockwood All Rights Reserved
Switch Scheduler Control Scheduler Authentication Address Allocation Traffic Monitoring User Registration Wireless Link Wireless Link Wireless Link
...
...
Bandwidth Manager User Interface Internet IntranetCopyright John W Lockwood All Rights Reserved
Extensible Networking Platform
5
5 - Lockwood - Evaluation and Test of Networks with Quality of Service
Quality of Service with Wireless Links
Everything as before, Plus:
Middleware
Provides common functions
for higher-level services
– Publish / Subscribe
Wireless
Link Monitor
Tracks available
bandwidth of radio
channel
Adaptive
Packet Scheduler
Order packets for QOS
Packet
Classifier
Data
Admission
Control
Policy
Control
QoS
Manager
Middleware
Applications
Wireless
Link
Monitor
Adaptive
Packet
Scheduler
Control
Extensible Networking Platform
6
Flow Descriptors
Flowspec: Desired QoS
Tolerable Delay
Average Throughput
Maximum Burstiness
Filterspec: Select Packets to Receive QoS
Sender Address
Destination Address
Protocol
UDP/TCP ports
Extensible Networking Platform
7
7 - Lockwood - Evaluation and Test of Networks with Quality of Service
Example Internet Packet
Source Address = 128.252.5.5 (dotted.decimal)
Destination Address = 141.142.2.2 (dotted.decimal)
Source Port = 4096 (decimal)
Destination Port = 80 (decimal)
Protocol = TCP (6)
Payload contains keywords { CONTENT (0), CONTENT (1) }
Content Vector = “00000011” (binary) = x”03” (hex)
7
103
71
39
Src IP (hex) =
80FC0505
Dest IP (hex) =
8D8E0202
Src Port = 1000 Dest Port = 0050 Proto = 060
8
40
72
All values shown In hex Con-tent = 03111 104
Extensible Networking Platform
8
Sample Filter Specification
Source Address = 128.252.0.0 / 16
Destination Address = 141.142.0.0 / 16
Source Port = Don’t Care
Destination Port = 80
Protocol = TCP (6)
Payload includes general SPAM (List 0)
7
103
71
39
Src IP (hex) =
80FC0505
Dest IP (hex) =
8D8E0202
Src Port = 1000 Dest Port = 0050 Proto = 060
8
40
72
Src IP value =
80FC0000
Dest IP (hex) =
8D8E0000
Src Port = 0000 Dest Port = 50 Proto = 06Src IP (hex) =
FFFF0000
Dest IP (hex) =
FFFF0000
Src Port = 0000 Dest Port = FFFF Proto = FFValue
Mask:
1=care
0=don’t care
IP Packet
Con-ten t= 01 Con-ten t= 01 Con-tent= = 03Extensible Networking Platform
9
9 - Lockwood - Evaluation and Test of Networks with Quality of Service
Packet Classifier
CAM MASK [1] CAM VALUE [1] CAM MASK [2] CAM VALUE [2] CAM MASK [3] CAM VALUE [3] CAM MASK [N] CAM VALUE [N] Flow ID [1] 112 bits Flow ID [2] Flow ID [3] Flow ID [N] Flow ID. . .
. . .
. . .
16 bits Value Comparators Mask Matchers Priority Encoder Resulting Flow Identifier Flow ListSource Address Destination Address 16 bits Payload Match Bits Source Port Dest. Port Protocol CAM Table -Bits in IP Header
Extensible Networking Platform
10
Queue Manager & Packet Scheduler
Flow Buffer
Queue Manager
Tail
Next
Tail
Packet Scheduler
P0 P1 P2 P3 P0 P1 P2 P3
Enqueue
FSM
Dequeue
FSM
SRAM
Controller
Flow
ID
Head
Head
Next
Extensible Networking Platform
11
11 - Lockwood - Evaluation and Test of Networks with Quality of Service
Technology for Processing Network Data
General Purpose Microprocessors
9
Fully Reprogrammable
Optimized for general purpose purpose computing
Sequential Processing
Network Processors
9
Fully Reprogrammable
• Some concurrent processing (8-32 cores)
9
Some hardware optimized for applications
Extensible Network Hardware
9
Fully Programmable
9
Highly concurrent processing
9
Hardware optimized
... ... ... ... ... ... ...Custom Packet Processing Hardware
9
Highly concurrent processing
9
Optimized for specific networking application
Static Functionality
Extensible Networking Platform
12
Previous Work on Megabit-rate Packet Schedulers
Wireless Base Stations
Packet Scheduler
Wireless LAN Adapter
– Netwave Airsurfer+
Wireline LAN Adapter
– Fast Ethernet
Mobiles
Linux Laptops
References
Implementation of the Multiclass Priority Fair Queuing (MPFQ) Algorithm for
Extending Quality of Service in Existing Backbones to Wireless Endpoints, by
Jay Moorman and John Lockwood; IEEE Global Telecommunications
Conference (GLOBECOM 1999), December, 1999
Extensible Networking Platform
13
13 - Lockwood - Evaluation and Test of Networks with Quality of Service
CBR:
MPFQ
<
FQ
rtVBR:
MPFQ
< FQ
nrtVBR:
MPFQ
>
FQ
Extensible Networking Platform
14
Current work on Gigabit-Rate Packet Schedulers
Hard real-time
Reconfigurable
hardware
Scalable
Gigabit/second
throughput
Extensible
Packet classification
Flow scheduling
Traffic Monitoring
Protocol Wrappers
Processes
Internet packets
Data Interfaces
Read/write
packets to network
Buffer data in
off-chip memory
UDP/TCP IP Processor Cell Processor Frame ProcessorPacket
Classification,
QoS Scheduling, &
Traffic Monitoring
Circuits
Data
Output
Data
Input
Interface
to packet
storage
memories
Extensible Networking Platform
15
15 - Lockwood - Evaluation and Test of Networks with Quality of Service
Gigabit Rate Packet Processing Architecture
Flow
Buffer
Free List
Manager
Queue
Manager
SchedulerPacketPayload Match Bits
Content
Scanner
TCAM
Filter
Flow ID
Interfaces to
Off-Chip Memories
SRAM 1
Controller
SDRAM 1
Controller
SDRAM 2
Controller
Network
Statistics
Module
Xilinx XCV2000E FPGA
Layered Protocol Wrappers
Extensible Networking Platform
16
Extensible Networking Platform
17
17 - Lockwood - Evaluation and Test of Networks with Quality of Service Network Interfaces Swi tch Port Li ne Card
Field Programamble Port Extender (FPX)
Reprogrammable
Application Device (RAD)
Dynamic circuit
implemented on FPGA
SRAM and SDRAM
Reprogrammable over
network
Network Interface
Device (NID)
Static circuit
implemented on FPGA
Controls FPX
Full packet switch with
per-flow routing
Programs RAD over net
Rec on fig ur at io n Cont ro l Log ic Network Switch Reconfiguration Interface Control Logic
NID
Cache SRAM ZBT SRAM Bitstream ZBT SDRAM 64 MB 64 MB SDRAM UDP/TCP IP Processor Cell Processor Frame Processor UDP/TCPUDP/TCP IP ProcessorIP Processor Cell ProcessorCell Processor Frame ProcessorFrame ProcessorPacket Classification, QoS Scheduling, &
Traffic Monitoring Circuits
Extensible Networking Platform
18
Photograph of the FPX
Evolvable Internet Hardware Platforms, by John W. Lockwood,
NASA/DoD Workshop on Evolvable Hardware (EHW'01),
Long Beach, CA, July 12-14, 2001, pp. 271-279
Extensible Networking Platform
19
19 - Lockwood - Evaluation and Test of Networks with Quality of Service
Platform for Generating, Scheduling,
and Montoring Network Traffic
Extensible Networking Platform
20
Evaluation and Test of QoS-Enabled Networks
Provide flexibility to emulate
multiple network topologies
Provide scalability to process
large amounts of traffic
Real-time traffic generation
Real-time traffic measurement
Enable real-time interaction between
middleware and networking hardware
Extensible Networking Platform
21
21 - Lockwood - Evaluation and Test of Networks with Quality of Service
FPX
Traffic
Monitor
FPX
Traffic
Generator
FPX
Packet
Schedulers
GigE
Control
Host
Gigabit
Ethernet
Traffic
Source
Wireless
Link
PDA PDALogical View of Boeing QoS Testbed Plan
Control Flow Data Flow
Gigabit
Ethernet
Traffic
Source
Wireless
Link
Traffic Multiplexer
Traffic Demultiplexer
Extensible Networking Platform
22
Physical View of hardware for Boeing Experiments
FPX
WUGS
FPX
FPX
FPX
Extensible Networking Platform
23
23 - Lockwood - Evaluation and Test of Networks with Quality of Service
Control and Configuration
Communication to FPX
can be performed via
– Web / CGI
– Sockets
– Middleware
Control Software
– NCHARGE provides
API to control and
configures FPX hardware
Allows multiple control
devices to remotely
connect and control
common FPX resources
Middleware Apache NCHARGE ControlFPX Software CGI Web-based Control Application Distributed, QoS-Enabled Applications Extensible Network Hardware Live, Real-time, Network TrafficExtensible Networking Platform
24
Flow Specification
Sets the CAM rule Send UDP Control Packet GoQoS Control Applet
Flow Specification Class Allocations Flow Parameters Bandwidth Flow # Netmask Source IP Add 255 255 255 0 Src. Port Mask Netmask Destination IP 255 255 0 0 141 142 2 2 105 Modify Delete Any Protocol TCP Dest. Port Mask Any
Extensible Networking Platform
25
25 - Lockwood - Evaluation and Test of Networks with Quality of Service
Per-Flow Bandwidth Parameters
Go QoS Control Applet
Flow Specification Class Allocations Flow Parameters Bandwidth
Flow #
Unspecified Bit Rate: 105 Flight Status Channel
Avg. Rate
Peak Rate Mbps Mbps Variable Bit Rate: 500 Mbps 950 Mbps Constant Bit Rate: 45 Mbps
Maximum Allowable Latency: seconds
Extensible Networking Platform
26
Transmit Bandwidth Adjustment
Adjust the output bandwidth
Go QoS Control Applet
Flow Specification Class Allocations Flow Parameters Bandwidth
100 Kbps1 Mbps 10 Mbps100 Mbps1 Gbps Pacer Control
Full-Speed Low Bit Rate
Extensible Networking Platform
27
27 - Lockwood - Evaluation and Test of Networks with Quality of Service
WU Laboratory Testing
Live Traffic Testing
of network systems with
Quality of Service at
Gigabit/second Rates
Extensible Networking Platform
28
Mobile System
FPX-based Processor
Standalone operation
Gigabit Ethernet
Extensible Networking Platform
29
29 - Lockwood - Evaluation and Test of Networks with Quality of Service
Summary
Extensible networks provides flexibility to
Emulate large networks
– Topology set via soft configuration of switches
Classify packets using fields of header or payload
Schedule packets for transmission over links
FPX provides scalability to:
Classify and schedule packets at high speed
Generate traffic at Gigabit per second rates
Measure traffic with high accuracy
– Accurate to 16 billionths of second
Networking software provides
Standard, socket-based Application Programming Interface (API)
– Web-based interfaces for control and configuration
– Ability to interact with middleware
Overall System
Provides platform to test and evaluate live traffic in real time.
Extensible Networking Platform
30
Current and Future Work
Network testbed
Gigabit/second switching, packet scheduling, and filtering
Wireless LAN links connected to laptop and PDA hardware
Generate components for network instrumentation
Traffic generation of cross traffic
Traffic monitoring and packet time stamping
Develop and test new switching and
packet scheduling technology
QoS-enabled
Gigabit/second speed
Integrate signaling between hardware with middleware
Work with Boeing to deploy and test QoS-enabled services
Extensible Networking Platform
31
31 - Lockwood - Evaluation and Test of Networks with Quality of Service
More Information
Wireless Packet Schedulers
Implementation of the Multiclass Priority Fair Queuing (MPFQ) Algorithm for Extending Quality of Service in Existing Backbones to Wireless Endpoints, by Jay Moorman and
John Lockwood; IEEE Global Telecommunications Conference (GLOBECOM 1999), 1999. Pgs. 2752-2757
Wireless LAN Networks
Implementation of Campus-wide Wireless Network Services using ATM, Virtual LANs, and Wireless Basestations, by John W. Lockwood; IEEE Wireless Communications and Networking Conference (WCNC). September 21-23, 1999, Paper TA3.3.
»
FPX Platform
Evolvable Internet Hardware Platforms, by John W. Lockwood, NASA/DoD Workshop on Evolvable Hardware (EHW'01), Long Beach, CA, July 12-14, 2001, pp. 271-279.
An Extensible, System-On-Programmable-Chip, Content-Aware Internet Firewall, by John W. Lockwood, Christopher Neely, Christopher Zuver, James Moscola, Sarang Dharmapurikar, and David Lim; Field Programmable Logic and Applications (FPL), Paper 14B, Sep 1-3, 2003.
Automated Tools to Implement and Test Internet Systems in Reconfigurable Hardware, by John W. Lockwood, Chris Neely, Chris Zuver, Dave Lim;
SIGCOMM Computer Communications Review (CCR), vol 33, no 3, July 2003, pp 103-110.
Network Security with the FPX
Internet Worm and Virus Protection in Dynamically Reconfigurable Hardware; by John W. Lockwood, James Moscola, Matthew Kulig, David Reddick, Tim Brooks, Military and Aerospace Programmable Logic Device (MAPLD), Washington DC, 2003,
Paper E10, Sep 9-11, 2003
On-line References
PDF versions of all of these papers and more can be downloaded on-line as:
