CoreLab
: An Emerging Network Testbed
towards Network Virtualization
Network Virtualization Research Lab
Akihiro NAKAO
Associate Professor University of Tokyo
PlanetLab
The largest and most popular overlay network test-bed Currently consists of 900+ nodes at 450+ sites
800+ Projects/ 1000+ Researchers
Brief History of PlanetLab
PlanetLab 1.0 (2002-3)
UNIX account slivers
PlanetLab 2.0 (2003-4) Vserver slivers PlanetLab 3.0 (2004-2006) PLCAPI 1.0 PlanetLab 4.0 (2007-) MyPLC1.0 Federation Idea PlanetLab 4.2 (2008-9)
Federation (PLE, PLJ, (PLK, PLC,=>PLA?))
New Development using Private PlanetLab
(RSpec/GENI-wrapper, New Node Type, Monitoring, QA,VINI)
What have we learned from
PlanetLab
?
Concept
Slice-based experimental facility
“Slice” Mechanism
Resource Isolation through Virtualization
Management Framework
Node : Physical Machine Sliver : Virtual Machine
Slice : A collection of Virtual Machines RSpec: Resource Description
Slivers
Virtual Machine Monitor (VMM)
Node Mgr
Owner
VM VM1 VM2 … VMn
Linux kernel (Fedora Core)
+ Vservers (namespace isolation) + Schedulers (performance isolation) + VNET (network virtualization)
Auditing service Monitoring services Brokerage services Provisioning services
- From http://www.planet-lab.org/files/presentation-2007-05-01-planetlab.ppt
PlanetLab Architecture
PLC Slice1 Slice2 Sliver Sliver Sliver Sliver Node NodeLimitations in PlanetLab
Flexibility is sacrificed over Scalability/Performance
Sliver : Container-based Resource Isolation Shares crucial resources
• Single kernel (Linux with VServer patch)
• Network stack and modules (NAT, Firewall, etc) • Network resources (IP address and ports)
• Vnet (virtual network device ; need work for multi-homing)
Cannot run arbitrary kernel/network stack & modules
Software Engineering Issue
Cannot keep up with the latest kernel (VServer)
• The latest VServer patch 2.6.26 (PlanetLab 2.6.22) • Misses out support for new devices/chip features
Design Principle for a New Test-bed
Performance
Network I/O
Scalability
The number of slices
Isolation
Resource must be isolated per slice
Flexibility
Arbitrary kernel/network stacks/resources
Code-Reusability
Catch up with the latest technology Minimize software engineering effort
VEE Candidate for Sliver Implementation
Design Principles
Performance Scalability Security Isolation Performance Isolation Flexibility Code Reusability Resource Container + + + - Hyper- visor + + + + - Hosted- VMM ? ? + ? + + Hardware VMM VM VM GuestOS GuestOS Apps apps VM MngOS Manager Hypervisor-Based Hardware HostOS Apps VM VM GuestOS GuestOS Apps Apps Host-Based Hardware HostOS Apps VE VE Apps Apps Resource Container advantageous disadvantageous
CoreLab
Design
Sliver VEE : Hosted Virtual Machine Monitor VM Disk Image Transfer : Multipath & Caching Network Configuration : Multihoming
Management Tools: Reuse PlanetLab’s design
(GENI-wrapper/Federation possible)
Comparison Chart PlanetLab CoreLab
Sliver Resource Container Hosted VMM
Kernel Shared Separated
Guest OS Various Linux Distros Any OSes
IP & Port Shared Shared (managed) Privileged System Calls Shared Isolated
CoreLab Prototype
Implementation
Sliver VEE : KVM + QEMU on Kernel 2.6.28.7
VM Disk Image Transfer : BitTorrent
Network Configuration : TunTap / Managed IP/Port
Management Tools: MyPLC
Optimizations
• Compression of VM Image • Increase Sharable Buffers
• Decrease Influence of VM Boots
Misc.
• SSH Login (permitopen, authorized_key) and VNC
KVM Hosted Virtualization
Network Configuration
VM Image Dissemination
Introduce helper nodes • BT Node Master
• BT Node Slave
Introduce components • BTManager
Scalability Check (1/2) pkt-size=
64B
Max at 48kpps (slightly worse than Xen by Trellis) Little overhead in term of # of VMs (16 – 224 VMs) Great resource fairness among VMs (~1.8% deviation) Total packet receive rate (Kpps)
Offered Load via pktgen (Kpps)
Probably our system is CPU bound
CoreLab Deployment (on-going)
Phase1 : 12 Sites over JGN2Plus + SINET + others
10 in operation
Phase2 : Asian Region
HP DL 580 (4U)
Intel Xeon 16-core (quad 4-core) 128GB (Max 256GB) memory 300GB RAID5
CoreLab : In Action…
Conclusions
Contributions
Identify design principles for alternative test-beds
• Performance, Scalability, Isolation • Flexibility, Code-Reusability
Implement prototype test-bed CoreLab
• Hosted VMM as sliver to achieve
Report various optimization and preliminary evaluations
• Throughput 30-70% of Vanilla Linux and PlanetLab OS
Future Work
Build Full-Fledged CoreLab
More Future Directions
CoreLab Enhancement
Geographically (Japan to Asia)
Public PlanetLab
J-Lab as a regional Public PlanetLab
Federation
Federate CoreLab, J-Lab with PlanetLab, OneLab2 GENI Spiral 1 Solicitation 2
Virtual Router Project
Extreme Enhancement to CoreLab OpenFlow / NetFPGA
Overlay to Network Virtualization
Virtualize not only edges… …but also network core
VMM/Hypervisor Mgmt S/W VM(Slice) VM(Slice) VM(Slice)
A Slice of
Computation Resources
PC Server
Planetary-Scale Network Virtualization
Virtual Links
Virtual Core Routers
Virtual End System
Substrate Link (could be any exisiting technology)
Virtual Edge Routers
Roadmap
AKARI Architecture Project
Network Virtualization Lab
(CoE on Network Virtualization)
2nd Phase
Private PlanetLab Research PlanetLab
CoreLab: PlanetLab Enhancement + VR
Public PlanetLab (Federation; Migration)
2007 2008 2009 2010 2011
Fiscal Year
Public PlanetLab
J-lab : Private PlanetLab
(JGN2+SINET+…)
Deploy(1st Phase)
Deploy Federate Nov.
Service Platform Advanced Research Center
Contact Information
Network Virtualization / Overlay is one of the key
technologies for proceeding further in defining NwGN architecture(s)
Test-bed designs are crucial part of this research…
