2 of 130
VTN VN2
High Level Design
May 2009
3 of 130
Document Revision history
Version Date Author Comment / Changes
V0.1 29/12/08 Ronald Lee, Leonard Tan First draft for comments V0.2 9/1/09 Ronald Lee, Ma Shaowen Second draft for comments
V0.3 22/1/09 Ronald Lee Third draft for comments
V.0.4 20/1/09 Ronald Lee, Leonard Tan, Anh Tuan Chu, Ed Spencer
Fourth draft for comments
V.0.5 1/3/09 Ronald Lee, Ed Spencer Fifth draft for comments V0.6 14/03/09 Ed Spencer, Leonard Tan, Anh
Tuan Chu
Sixth draft version
V0.7 20/03/09 Leonard Tan, Ma Shaowen, Anh Tuan Chu
Seventh draft version
V0.8 04/05/09 Leonard Tan, Ma Shaowen Incorporated inputs from the 22Apr meeting
Document Distribution
4 of 130
Table of Contents
Document Revision history ... 1
Document Revision history ... 2
Document Distribution... 2
Table of Contents... 3
1 Scope ... 5
2 Assumptions... 5
3 Introduction ... 5
4 Network Design Requirements ... 5
5 Physical Network Topology ... 5
5.1 Network Architecture... 5
5.2 Existing Network Architecture Caveats ... 5
5.3 POP Types and POP design... 5
5.3.1 Type A1 ... 5 5.3.2 Type A2 ... 5 5.3.3 Type A3 ... 5 5.3.4 Type A4 ... 5 5.3.5 Type A5 ... 5 5.3.6 Type A6 ... 5 5.3.7 Type B1 ... 5 5.3.8 Type B2 ... 5 5.3.9 Type B3 ... 5 5.3.10 Type B4 ... 5 5.3.11 Type B5 ... 5 5.3.12 Type B6 ... 5 5.3.13 Type C1... 5 5.3.14 Type C2... 5 5.3.15 Type C3... 5 5.3.16 Type C4... 5 5.3.17 Type C5... 5 5.3.18 Type D... 5 5.3.19 HNI ASBR ... 5 5.3.20 HCM ASBR ... 5 5.3.21 DNG ASBR... 5
5.4 Aggregated Core plane facing uplinks ... 5
5.5 Network elements types and positioning. ... 5
6 Logic 6.1 al Network Topology ... 5
IP Addressing and naming convention. ... 5
6.1.1 IP Addressing ... 5
6.1.2 Node Naming ... 5
6.1.3 Port Naming ... 5
6.1.4 Network Interface Naming ... 5
6.1.5 Service Distribution Point Naming ... 5
6.2 IP Routing Recommendations ... 5
6.2.1 Overview ... 5
5 of 130
6.4 End-to-End Logical Topology ... 5
6.4.1 IGP ... 5
6.4.2 IGP in Metro Networks ... 5
6.4.3 In-band management to Metro Networks... 5
6.4.4 BGP... 5
6.5 IP Multicast Recommendations... 5
6.5.1 Rendezvous Point (RP) placement... 5
6.5.2 Reverse Path Forwarding (RPF) considerations... 5
6.6 MPLS Topology and Signalling Overview. ... 5
6.6.1 Overview ... 5 6.6.2 Usage ... 5 6.6.3 Backbone MPLS... 5 6.6.4 RSVP Signalled LSP Topology... 5 6.6.5 Traffic Engineering ... 5 6.6.6 Edge MPLS ... 5 6.7 High Availability... 5
6.7.1 Border Router related Failure ... 5
6.7.2 PE upstream related Failure ... 5
6.7.3 P router RSVP Label Switch Path Recovery ... 5
6.8 Network QoS Design ... 5
6.8.1 QoS Overview ... 5
6.8.2 Scheduling... 5
6.8.3 QoS Design ... 5
6.8.4 QoS on Juniper router ... 5
6.9 Network Security Recommendation ... 5
6.9.1 Generic Node Access... 5
6.9.2 Authentication, Authorization, and Accounting (AAA) ... 5
6.9.3 User Accounts and Passwords ... 5
6.9.4 Packet Filtering Toolset ... 5
6.9.5 Securing Nodes ... 5
6.9.6 Securing Client Services ... 5
6.9.7 Juniper network management and security ... 5
7 Service Network Topology... 5
7.1 Core and MANs interconnect ... 5
7.2 Service Architecture ... 5
7.2.1 Ethernet Leased line service using L2 VPN... 5
7.2.2 Wide Area Switched LAN Service using VPLS ... 5
7.2.3 Wide area Routed LAN service using VPRN ... 5
7.2.4 High Speed Internet Service... 5
7.2.5 IPTV Video Core distribution using multicast ... 5
8 Network Management ... 5
8.1 Alcatel-Lucent 5620 SAM Element Manager (5620 SAM-E) ... 5
8.2 Alcatel-Lucent 5620 SAM-Assurance (SAM-A) ... 5
8.3 Alcatel-Lucent 5620 SAM Provisioning (SAM-P)... 5
8.4 Alcatel-Lucent 5620 SAM Open Interface (SAM-O) ... 5
8.5 Alcatel-Lucent 5620SAM Redundancy... 5
8.5.1 Activity switches, failovers, and switchovers... 5
6 of 130
8.5.3 Database switchovers ... 5
8.5.4 Database failovers... 5
8.5.5 Events associated with an activity switch. ... 5
8.6 Direct 5620 SAM Client Access ... 5
8.7 IP Addressing and Naming convention ... 5
8.8 Bandwidth Requirement... 5
8.8.1 Bandwidth Requirements SAM to Network Elements ... 5
8.8.2 Details on the Bandwidth Requirements ... 5
8.8.3 Possible consequences of insufficient bandwidth ... 5
8.9 Server Hardware ... 5
8.10 Security ... 5
8.10.1 TACACS+ Configuration... 5
8.10.2 Communication Port Requirements for 5620-SAM Components ... 5
8.11 Network Time Protocol... 5
9 NMS WANDL IP/MPLSView Solution ... 5
9.1 What Can WANDL Solution Offer ... 5
9.1.1 Overview ... 5
9.1.2 Online Operation Management... 5
9.1.3 Offline Planning Simulation ... 5
9.1.4 Provisioning & Service Management ... 5
9.2 Design of WANDL NMS Solution for VN2 ... 5
9.2.1 Overall Architecture ... 5 9.2.2 Machines ... 5 9.2.3 User Admin... 5 9.2.4 Backup ... 5 10 Conclusion ... 5 References. ... 5 Glossary... 5
7 of 130
1 Scope (quy mô)
The scope of this document is to provide the High-Level Design considerations to ensure a stable and scalable Network implementation using the Alcatel-Lucent
7750 Service Router series. Detail is given regarding architectural high-level design principles, approaches and technology selections, however no attempt is made to detail explicit comman0064-line configuration.
8
2 Assumptions
This document assumes that the reader is reasonably familiar with IP/MPLS, Layer-2 and Layer-3 VPNs.
1 2
3 Introduction
3 4
VNPT will implement a next generation network (NGN) and Alcatel-Lucent has 5
been selected to be the IPCORE Provider Edge (PE) router & Autonomous System 6
Border Router (ASBR) equipment supplier for the VN2 Network. 7
8
The VN2 network is being envisaged to house all of VNPT‟s existing networks, 9
inclusive of all Metropolitan Networks, High speed Internet delivery Core and evolve 10
into a new unified next-generation Core network. 11
12
VN2 is to be built and expanded over time to accommodate all of VNPT‟s existing service 13
offerings and support the introduction of new services such as IP Television and Video-14
on-Demand. 15
16
This document provides high level design (HLD) & recommendations for the VN2 project 17
specific to the PE and ASBR routers. Topics & specifications mentioned in this high level 18
design document will provide the foundation as implementation specifications for the low 19
level design (LLD) phase, which will translate the mentioned implementation specifications 20
into machine specific configurations. 21
4 Network Design Requirements
VN2 is poised to be the national NGN backbone network with the capabilities of delivering the following services as depicted by VNPT:
• High Speed Internet (Broadband access) • Voice over IP (VoIP)
• Multicast Video services (e.g. IPTV, Video Conferencing) • Unicast Video services (e.g. Video on Demand)
• Layer 3 IP-VPN services
5 Physical Network Topology
This section describes the proposed physical network architecture.
5.1
Network Architecture
The VN2 is based on a dual Core Router plane design with 5 main core regions. These are namely: HNI, HCM, HPG, DNG & CTO. Provider edge routers within these 5 regions will be connected to the 2 Core routers located in the regional Central Office (CO).
Core Routers are T-1600 supplied by Juniper Networks. Provider Edge routers and Autonomous system border routers are 7750SR-7 supplied by Alcatel-Lucent. There are a total of 10 Core routers, 79 Provider edge routers and 5 ASBR routers in the VN2 IPCore Network. It is expected that the PE routers will have high speed connectivity to BRAS and Metro Ethernet networks to be realised in separate tenders.
Figure 1: VN2 IP Core Network
Enhancement to the requested VN2 network architecture have been proposed in this high level design document to provide increased service availability and improve the level of optimum route path selection. These enhancements are essentially inter-PE connectivity for PoP with dual PE routers as well as inter- ASBR connectivity for sites with dual ASBR PE routers.
For dual PE router POPs, Alcatel-Lucent strongly recommend that a series of Inter- PE link be added so as to provide increased service availability in the event of link failure to the P router.
5.2
Existing Network Architecture Caveats
Figure 2: Failure Scenario: Inter POP VPLS
In the above failure scenario, a VPLS service is in service between HPG POP and NDH POP. Failure of the STM-64 link of HPG PE1 to HPG P1 will render the VPLS service be split into 2 isolated VPLS islands. As the failure occurred on the MPLS network interface towards HPG P1, the connected MAN-E will not be aware of the failure until the MAC address ages out.
This failure scenario can be mitigated by introducing inter PE connection so that VPLS isolation does not occur during a link failure with the P router node. If the inter PE link is unavailable, the failure scenario can alternatively be mitigated by connecting the VPLS instance to an additional VLAN created within the MAN-E Metro Core to support the redundancy, RSTP will have to be configured on the VPLS for the purpose of breaking the layer 2 loop created. This will be the temporary solution until the inter-PE link is made available.
The recommended bandwidth for the Inter-PE node shall be at least 3.3Gbps assuming a protection ratio of 1:3. Therefore, this will translate to a 4 x 1GE LAG between the PE nodes.
ha v e 2 0 S R ype nterf A2 ce s O as well as unit ual AN si n ho us med ng S o TM 64 u te g 4 ac e
5.3
POP Types and POP design
This section describes the various type of Point of Presence. 5.3.1 Type A1
Type A1 POP have 2 units of 7750 SR7 dual homed to MAN using 10GE interfaces as well as dual homed to P routers using STM-64 interfaces. Alcatel- Lucent recommends 4 x 1GE links between the PE router pair for increased availability. Alcatel-Lucent understands that this could only be added in subsequent expansion/variation order. 5.3.2 Type A2
T
P P
s of 775
7 dual
t M
x 1GE
i
a
d
homed to P routers
i
-
in rf
s.
However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 2 x 10GE interfaces are planned for connectivity to BRAS pair. Alcatel-Lucent recommends 4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.
ha v e 2 0 S R ype nterf AN si n ho us 5.3.3 Type A3
Type A3 POP have 2 units of 7750 SR7 dual homed to MAN using 5 x 1GE interfaces as well as dual homed to P routers using STM-64 interfaces. However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 1 x 10GE interface is planned for connectivity to BRAS pair. Alcatel-Lucent recommends 4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.
5.3.4 Type A4
T A4 POP units of 775 7 dual med to M u g 5 x 1GbE i aces as well as dual homed to P routers ing STM-64 interfaces. However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 2 x 10GbE interfaces are planned for connectivity to BRAS pair. Alcatel-Lucent recommends
4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.
5.3.5 Type A5
Type A5 POP have 2 units of 7750 SR7 dual homed to MAN using 3 x 1GE interfaces as well as dual homed to P routers using STM-64 interfaces. However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 2 x 10GE interfaces are planned for connectivity to BRAS pair. Alcatel-Lucent recommends 4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.
5.3.6 Type A6
Type A6 POP have 2 units of 7750 SR7 dual homed to MAN using 6 x 1GE interfaces as well as dual homed to P routers using STM-64 interfaces. However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 3 x 10GE interfaces are planned for connectivity to BRAS pair. Alcatel-Lucent recommends 4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.
5.3.7 Type B1
Type B1 POP have 1 unit of 7750 SR7 dual homed to P routers using 2 x STM-16 interfaces. In addition, 1 x 1GE interface to MAN and 1 x 1GE interface to BRAS are planned.
5.3.8 Type B2
Type B2 POP have 1 unit of 7750 SR7 dual homed to P routers using 2 x STM-16 interfaces. In addition, 1 x 1GE interface to MAN and 2 x 1GE interface to BRAS are planned.
5.3.9 Type B3
Type B3 POP have 1 unit of 7750 SR7 dual homed to P routers using 2 x STM-16 interfaces. In addition, 2 x 1GE interfaces to MAN and 2 x 1GE interfaces to BRAS are planned.
5.3.10 Type B4
Type B4 POP have 1 unit of 7750 SR7 dual homed to P routers using 2 x STM-16 interfaces. In addition, 2 x 1GE interfaces to MAN and 3 x 1GE interfaces to BRAS are planned.
5.3.11 Type B5
Type B5 POP have 1 unit of 7750 SR7 dual homed to P routers using 2 x STM-16 interfaces. In addition, 3 x 1GE interfaces to MAN and 3 x 1GE interfaces to BRAS are planned.
5.3.12 Type B6
Type B6 POP have 1 unit of 7750 SR7 dual homed to P routers using 2 x STM-16 interfaces. In addition, 3 x 1GE interfaces to MAN and 4 x 1GE interfaces to BRAS are planned.
P rou un
, 3 5.3.13 Type C1
Type C1 POP have 1 unit of 7750 SR7 dual homed to P routers using 3 x STM-16 interfaces. In addition, 3 x 1GE interfaces to MAN and 1 x 10GE interface to BRAS are planned. 1GE interfaces are absent from the initial order, it is understood that VTN will be putting up additional order in time for deployment.
5.3.14 Type C2
Type C2 POP have 1 it of 7750 SR7 dual homed to ters using 3 x STM-16 interfaces. In addition x 1GE interfaces to MAN and 3 x 1GE interfaces to BRAS are planned.
5.3.15 Type C3
Type C3 POP have 1 unit of 7750 SR7 dual homed to P routers using 3 x STM-16 interfaces. In addition, 2 x 1GE interfaces to MAN and 4 x 1GE interfaces to BRAS are planned.
5.3.16 Type C4
Type C4 POP have 1 unit of 7750 SR7 dual homed to P routers using 3 x STM-16 interfaces. In addition, 3 x 1GE interfaces to MAN and 4 x 1GE interfaces to BRAS are planned.
5.3.17 Type C5
Type C5 POP have 1 unit of 7750 SR7 dual homed to P routers using 3 x STM-16 interfaces. In addition, 4 x 1GE interfaces to MAN and 4 x 1GE interfaces to BRAS are planned.
5.3.18 Type D
Type D POP have 1 unit of 7750 SR7 dual homed to P routers using 4 x STM-16 interfaces. Interface requirement for Mobile Access/Core is yet to be determined.
5.3.19 HNI ASBR
Hanoi ASBR POP will have 2 units of 7750 SR7 dual homed to P routers using 5 x STM-64 interfaces as well as 5 x 10GE interfaces to VDC1. Alcatel-Lucent recommends minimum of 1 x 10GE links between the ASBR router pair for increased availability and improved routing convergence.
It was understood that Inter-ASBR link could only be added in subsequent
expansion/variation order. Therefore, 10GE interfaces originally planned for connectivity to VDC has been reconfigured to allow for interconnectivity between ASBR routers. Alcatel-Lucent would request for VTN to plan for the replenishment of these 10GE interfaces where possible.
5.3.20 HCM ASBR
Ho Chi Minh ASBR POP will have 2 units of 7750 SR7 dual homed to P routers using 5 x STM-64 interfaces as well as 6 x 10GE interfaces to VDC2. Alcatel- Lucent recommends minimum of 1 x 10GE links between the ASBR router pair for increased availability and improved routing convergence.
It was understood that Inter-ASBR link could only be added in subsequent
expansion/variation order. Therefore, 10GE interfaces originally planned for connectivity to VDC has been reconfigured to allow for interconnectivity between
ASBR routers. Alcatel-Lucent would request VTN to plan for the replenishment of these 10GE interfaces where possible.
5.3.21 DNG ASBR
Da Nang ASBR POP will have 1 unit of 7750 SR7 dual homed to P routers using 4 x STM-64 interfaces as well as 4 x 10GE interfaces to VDC3.
5.4
Aggregated Core plane facing uplinks
For the purpose of supporting P router plane planning, the below table illustrated the aggregated bandwidth from various PE routers.
Plane 1 Plane 2 STM64 STM16 STM64 STM16 HNI 4 24 4 20 HPG 1 6 1 5 DNG 3 13 3 13 CTO 1 16 1 12 HCM 5 10 5 7 Bandwidth 138880 171120 138880 141360 Aggr BW 310000 280240
5.5
Network elements types and positioning.
The offered Network Provider edge (PE) router and Autonomous System Border Router (ASBR) are both Alcatel-Lucent 7750SR-7 service routers. PE routers are equipped with 2nd generation I/O modules (IOM-2) with 40Gbps/slot capacity giving a total of 200Gbps capacity per system. PE routers can be upgraded in the future with the replacement of IOMs and MDAs to achieve greater performance of up to
100Gbps/slot. ASBR routers are equipped with 3rd generation I/O modules (IOM-
3XP) and Media dependent Adapters (MDA-XP) with 100Gbps/slot capacity giving a total of 500Gbps capacity per system.
The 7750 SR-7 chassis is a fully redundant system and has a total of seven front access slots. Two card slots are dedicated for
redundant common equipment, each of which holds one Switch Fabric/Control Processor Module (SF/CPM). Only one SF/CPM is required for full, non- blocking operation at 500 Gbps. A second SF/CPM provides complete
redundancy of the fabric and the control processors. When two SR-7 SF/CPMs are installed, the traffic is load shared across the switch
fabrics. The remaining five slots are used for Input / Output Module (IOM) baseboards.
The 7750 SR-7 chassis supports redundant power and fan trays and is designed to be NEBS Level-3 compliant. Power options for the SR-7 include -48V DC or
120/240V AC power, supporting 1+1 redundancy. All power connection are made in the rear via 2 separate Power Entry Modules (PEMs). In addition to the PEMs, there are also DC line conditions that must be installed in the front of the chassis. System cooling within the SR-7 uses right to back airflow. Two (2) separate fan trays provide 1+1 cooling redundancy. Both fan trays are inserted in the back of the SR-7 chassis. The dimensions for the SR-7 are 14”H x 17.5”W x 23.5”D, meaning five SR-7s will fit in a 7 foot Telco rack with room
for a breaker panel.
The Juniper Networks T1600 Core Router is positioned in the VN2 network in the role of the P router. The T1600 is an (8) slot chassis, supporting up to 100Gigabit/sec per slot. Line card options to be used in the VN2 network include the
100Gigabit/sec T1600 Type-4 FPC,
supporting two 4-port STM-64 PIC modules, and the 40Gigabit/sec Type-3 FPC, supporting up to four 1-port STM-64 or 4-port STM-16 PIC modules.
The T1600 is a fully redundant system. Switch Interface Boards (SIB) redundancy is available as N+N, with a
minimum redundancy of 4+1 available at full system capacity, great redundancy available at lower utilization,
and a capability of graceful capacity degradation in the event of multiple failures. The Routing Engine (RE) used in VN2 is the RE-A-2000, and redundancy is 1+1, with a graceful failover capability available. Power Entry Modules (PEM) are also
1+1 redundant with load sharing capability and with multiple inputs per PEM providing a greater level of protection from individual circuit failure.
6 Logical Network Topology
6.1
IP Addressing and naming convention.
This section addresses the recommendation for IP addressing and general naming convention to be used on the nodes.
6.1.1 IP Addressing
All nodes in VN2 (P, PE, ASBR and RR) will use public IP for loopback and interface addresses. The MAN nodes will use public IP for loopback addresses and private IP for interface addresses.
MAN should use separate contiguous IP address blocks for the loopback and interface addresses.
6.1.2 Node Naming
Node names will be standardized and have a fixed length. The nodes will be named based on the function that they are performing, the node model and their physical location. The format will be as follows:
<location><function><number>
<location> Three-letters defining a site location; for example HNI for Hanoi or HPG for Hai Phuong.
<function> A three-letter code indicating the function of the node; NPE for Multi Service Network PE router, NPR for Network Core router, ASR for ASBR Router, ACC for Access Router, SAM for 5620SAM Application Server & SDB for 5620SAM Database Server.
<number> A two-digit number representing the node number at this location. Where a location has multiple nodes of the same function then there will be a requirement for each node to be uniquely identified. For example:
HNINPE01 First 7750 SR Multi Service router in Hanoi HPGAGG01 First 7450 ESS Aggregation router in Hai
Phuong All naming should be represented in UPPER CASE. 6.1.3 Port Naming
Access/Network port names/descriptions should be standardised in terms of representation. A proposal for the format is as follows:
<a-end site> Node name representing the location of the a- end of the link.
<b-end site> Node name representing the location of the b- end of the link.
<port> Port number representing the port on the b-end. For example:
HNINPE01:HPGNPE01:1/1/1
• A 7750 SR router in Hanoi connected to the port 1/1/1 on a 7750 SR router in Hai Phong
6.1.4 Network Interface Naming
A network interface is a logical IP routing interface that can be associated with either a physical or logical port or an SONET/SDH channel. Note that the interface name can be up to 32 characters, but must start with a letter. For ease of re- configuration it is useful to avoid the logical port being tied in name to the physical. Therefore it is recommended that the network interface name should include the originating node name and the destination node name followed by a link iteration in the event of more that one network interface exists between the same end points.
<a-end site>:<b-end site>:<number>
<a-end> Node name representing the location of the a- end of the link.
<b-end site> Node name representing the name of the b-end of the link.
<number> Two-digit number representing the iteration of this a-end and b-end where multiple network interfaces exist between two sites.
For example:
HNINPE01:HPGNPE01:01
• The first interface connection on a 7750 SR router in Hanoi defined towards another 7750 SR router in Hai Phuong
All naming should be represented in UPPER CASE. 6.1.5 Service Distribution Point Naming
Distributed VPN services across PE routers use service distribution points (SDPs) to direct traffic to another PE router through a service tunnel. SDPs are created on each participating PE router, specifying the origination address (the PE router participating in the service communication) and the destination address of another PE router. SDPs are then bound to a specific customer service.
Without the binding process, far-end PE router devices are not able to participate in the service (there is no service without associating an SDP with a service).
In the creation of SDP, the SDP identifier is an integer with a value of between 1-
17407. A proposal for the SDP naming convention is to use the last three digits of the destination system address in decimal. Decimal zeroes must be included. For example, a far-end destination address of 10.220.11.5 would give an SDP name of
005 using the last three digits. Equally, a far-end destination address of 10.220.11.26 would give an SDP name of 026 using the last three digits.
Figure 3: Alcatel-Lucent Service Routing Architecture
Full mesh SDP will be defined to provide 100% reachability to all PE nodes and ASBR nodes.
6.2
IP Routing Recommendations
6.2.1 Overview
The logical topology of the VN2 network consists of the device and interface names, the interface, link, and virtual addresses, and the suite of routing and control protocols, including both their individual scopes and interrelations, which are used to organize and bind together the physical components into a working network.
6.3
Routing and Control Protocols
A combination of (4) primary routing and forwarding control protocols have been selected for the VN2 network. The protocols are:
• Multiprotocol BGP • IS-IS
• LDP • RSVP
IS-IS is the baseline routing protocol, running on all devices, and enabled on all core facing physical interfaces, enabling reachability among all devices. IS-IS does not enable any services related traffic to traverse the networks, but each of the other protocols builds on top of it. Details of the use of IS-IS are given in the IGP section.
RSVP is the next layer, using the information in IS-IS to build MPLS paths among the P routers, across the core WAN portion of the network. LDP is then layered on top of that, by tunneling LDP through the RSVP signaled paths in the core of the network. LDP runs natively between the edge and core to build a full mesh of MPLS paths among all edge routers. This enables some services, and provides the next building block for others. Details of the use of RSVP and LDP are given in the MPLS section.
BGP is the top layer. Its operation is abstracted for the forwarding path, making use of centralized route reflector nodes, which are accessed using routing information from the underlying protocols. BGP provides routing information required to enable most services, in the form of a destination route with a next-hop which is reached through the LDP signalled MPLS path. Details of the use of BGP are given in the BGP section.
6.4
End-to-End Logical Topology
BRAS
BRAS
Level 1 Level 1
PE PE
P P
ISIS Level 1 ISIS Level 2 ISIS Level 1
ISIS Area ID 3 ISIS Area ID 1 ISIS Area ID 2
MAN MPLS Domain Dot1q Backbone MPLS Domain Dot1q MAN MPLS Domain
LDP (tunneled)
Single hop RSVP RSVP full mesh Single hop RSVP
Figure 4: End-to-End Logical Routing Architecture
An illustration of all network device types and the scope of each protocol running on each device:
6.4.1 IGP
The proposed IGP for VN2 IP/MPLS backbone and the connected MANs is Integrated Intermediate System to Intermediate System (IS-IS) and requires that participating network elements support the use of OSI IS-IS for routing in TCP/IP and dual environments
(RFC1195), Dynamic Hostname Exchange Mechanism (RFC2763), Support for Intermediate System to Intermediate System Cryptographic Authentication (RFC3567), and Support for Intermediate System to Intermediate System Extensions for Traffic Engineering (RFC3784).
The following are reasons for the selection of IS-IS over OSPF as the other IGP of choice: 1. Scalability. Far better scaling for the number of router nodes in a single level of more than 1,000 routers vis-à-vis 250 OSPF router nodes in a single area. This will primarily mean that future expansion for VNPT would be greatly simplified as there is a whole lot more headroom for expansion without having to constantly relooking at creating more areas.
2. Security. IS-IS protocol is non-IP based, it is considered more difficult to spoof or attack from the Internet and therefore is relatively more secure in an ISP environment.
3. Lesser Resource Usage. IS-IS uses single LSP per router. OSPF has different SLA types and has one destination prefix per LSA.
4. Faster Convergence. IS-IS uses less packets to propagate routing information.
On top of these, for high availability interworking, Graceful Restart will be enabled. 6.4.2 IGP in Me tro Networks
After the meeting held on the 22nd of April 2009, it has been decided by VNPT that the IGP domain of the MANs be part of the IGP domain of VN2. The MANs will be IS-IS level 1 areas. Please refer to figure 4.
As some of the existing MANs are already actively serving customers and are running OSPF as the de-facto IGP, we would recommend that planning should begin for those MANs to be migrated to run IS-IS. This would greatly maintain consistency
throughout the network, which is very important in maintaining an operational network. 6.4.3 In-band management to Metro Networks
As the MANs and VN2 belong to the same IGP domain, in band management of the nodes in the MANs is possible from anywhere within VN2.
6.4.3.1 IS-IS NSAP Addressing
Alcatel-Lucent proposes IS-IS as the IGP of choice because of it scalability, hierarchical capabilities, good convergence rate. The IS-IS Network Entity Title (NET) will use the OSI NSAP format. The format of the NET is composed of the following:
IS-IS NSAP format has two main components: • Initial domain part (IDP)
Each of these are broken down as: IDP
DSP
• Authority and format identifier: AFI • Initial domain identifier: IDI
• High order domain specific part (HO-DSP) • System Identifier (SysID)
• NSAP Selector (NSEL) The following parameters shall be set
• AFI shall be “49” (Private Addressing). • IDI shall be set to “00”
• The High order DSP shall be international country STD code as defined ITU E164, hence “84”.
• The system ID shall be encoded in hex from the router‟s 32 bit system interface.
• The NSEL shall be “00”.
The system ID is a 6-octet field and defines an intermediate system (IS) within a particular area. Within the IS-IS domain, the system ID is formed from the IPv4 system (loopback) address. For example, the system IP address 172.16.0.101 will take the form
1720.1600.0101.
The first part of the NET consists of an Area Address. A variable length field defining the area to which an Intermediate System belongs.
6.4.3.2 ISIS Hierarchy
IS-IS support node population in excess of more than 1000 nodes, IS-IS hierarchy is implemented as illustrated in figure 4. The MANs and BRAS will be in IS-IS level 1 areas. The VN2 PE and P nodes will be in a IS-IS level 2 area. The VN2 PE nodes will be configured as IS-IS level 1/2. Each MAN area, together with the backbone area would be in different IS-IS areas.
6.4.3.3 Convergence
In order to utilize IS-IS as a trigger for other fast convergence/High Availability protocols (such as IBGP Multipath/BGP peer tracking) it is necessary to manipulate key timers
1. Shortest Path First (SPF) interval. Each Intermediate System creates a topology map using Dijkstra‟s SPF algorithm. The topology is calculated as a Shortest Path Tree (SPT) with itself as root. Aside from initialisation, an SPF calculation is executed when the Shortest Path Tree (SPT) topology is changed. This may include a link failure, or a change in link metric.
2. IS-IS Link State PDU (LSP) Generation Interval. Is simply the time an Intermediate System should wait before generating and transmitting an LSP.
In order to achieve fast re-convergence however, the SPF interval needs to be
configurable to a sub-second value. In addition, an exponential back-off is required in order that that Intermediate Systems are not continually computing shortest paths (and consequently using valuable CPU cycles) when subsequent computations are required. All network elements should implement sub-second SPF and exponential back-off and will be configured with an initial SPF delay of 50 milliseconds. This initial wait value is to allow the router to flood the LSP that caused the trigger before actually executing the SPF
calculation, and also to ensure that only one SPF calculation is carried out at receiving intermediate systems. The incremental wait between subsequent SPFs will be configured for 100 milliseconds, whilst the maximum SPF interval should be configured for 2 seconds. Note: In the instance of a failed network link, the Intermediate System at each end of the link will generate an LSP. If the initial wait value is too small, then the potential exists that a receiving Intermediate System may execute an SPF after receiving the first LSP, then execute a second SPF after receiving the second LSP. Increasing the initial SPF wait value gives sufficient time to receive both LSPs even if one (or both) LSPs need are routed sub-optimally (as in the case of failure).
Table 1 details the SPF iterations given the defined values.
Time Iteration
50 msec First
100 msec after completion of first SPF execution Second 200 msec after completion of second SPF execution Third 400 msec after completion of third SPF execution Fourth 800 msec after completion of fourth SPF execution Fifth 2 seconds after completion of fifth SPF execution. Sixth
Table 1 : SPF Execution with Exponential Back-off
For LSP generation, the initial wait should be configured for 0 seconds. That is, LSPs will be generated immediately following the event. The incremental wait between subsequent LSPs being generated should be configured for 1 second, whilst the maximum LSP interval should be configured for 8 seconds.
All network elements should implement sub-second SPF and exponential back-off and will be configured with an initial SPF delay of 50 milliseconds.
Actual values used will be decided during the interop testing with Juniper. 6.4.3.4 Traffic Engineering Extension
RSVP LSPs will be used within the VN2 network. Therefore, IS-IS TE extension will be enabled on all P and PE nodes.
6.4.3.5 Authentication
Within any Service Provider environment, authentication of IGP updates is a desirable attribute. The backbone should be configured to support cryptographic authentication (HMAC-MD5) of IS-IS LSPs. Next to that it‟s worthwhile to use also MD5 authentication for Hello LSPs. The latter provides a mechanism for securely authenticating adjacencies before LSPs can even be exchanged.
Separate commands will be used to define first, the authentication type (since 7750 SR also support simple password or plain text authentication) and secondly the authentication key, which will be used to verify the PDUs sent by neighbouring routers on the interface.
Configuration of MD5 cryptographic authentication of Hellos is implemented under each interface configured under the IS-IS process. The authentication-key and authentication type must be reconciled for all routers on a link/segment :
6.4.3.6 Point-to-Point Broadcast Media
On point-to-point Ethernet, the election of a Designated Intermediate System (DIS) and regular generation of CSNPs is an unnecessary function. Point-to-point Ethernet links should therefore be configured such that no DIS election takes place and reliable flooding on these links is facilitated through generation of a PSNP to explicitly acknowledge received LSPs.
6.4.3.7 BRAS connection
The IS-IS protocol includes both the concept of areas and also makes use of a two level design. IS-IS Level1 routers learn only default routing information from Level1/2 routers in their area. Level1/2 routers pass routes from Level1 up to Level2. It is desirable to support a 2nd tier of edge router downstream of the PE, but without requiring the PE(s) to at each POP to be in separate areas, or to form Level1 adjacencies with each other. To achieve this, the PE should configure links to downstream edge routers as Level1, but in the backbone area.
The BRAS router will take advantage of the 2nd tier edge router scheme for IS-IS in order to learn the loopback address of the upstream PE routers, and to provide the upstream PE with a route to its own loopback. This will require the PE to leak its loopback into Level2. The BRAS will then establish iBGP connectivity to the PE router(s). When dual PE routers and/or dual BRAS uplinks are configured, IS-IS will provide the mechanism for rerouting around a single link or PE router failure without requiring the propagation of a change in the BGP route across the entire network.
6.4.3.8 General Parameters (Suggested) 6.4.3.8.1 MaxAge and LSP Refresh Interval
IS-IS LSPs have a remaining lifetime, which starts at the value of MaxAge and decrements to zero before they are flushed from a link state database. The LSPs must clearly be refreshed before the remaining lifetime expires. In order to reduce the amount of resource a router must dedicate to generating and processing LSPs, this default configuration should be modified to reflect an LSP MaxAge of 65535 seconds (18 hours) and an LSP refresh
interval of 32767 seconds (9 hours).
6.4.3.8.2 Hello Interval and Hello-Multiplier
Once an IS-IS adjacency is established, Hellos are exchanged between the Intermediate Systems that act as keepalive messages. The IS-IS Hello interval should be configured as 1 second on all interfaces, whilst the hold time or hello- multiplier should be set to a value of 3. That is, 3 x Hellos must elapse before the adjacency is declared down.
6.4.3.9 IS-IS Scalability
Table 2 details the performance and scaling characteristics for IS-IS on the 7750 SR platform.
Parameter 7750 SR
Number of adjacencies 252
Number of adjacencies on a single interface 84
Number of LSP(s) 25K
Number of routes 250K
Number of routers in a level 25K
Time to run SPF on 10K routes < 1 second Table 2 : IS-IS Scalability
6.4.3.10 IS-IS timing overview (Suggested)
Default Configured
csnp-interval 10 sec 10 sec
lsp-pacing-interval 100 ms 100 ms
retransmit interval 5 sec 5 sec
lsp-lifetime 12000 sec 65535 sec
spf-wait 10 sec / 1 sec / 1 sec 2 sec / 50 ms / 100 ms
lsp-wait 5 sec / 0 sec / 1 sec 8 sec / 0 sec / 1 sec
IS-IS L1 hello interval 9 sec 9 sec
IS-IS L1 hello multiplier 3 3
IS-IS L2 hello interval 9 sec 1 sec
IS-IS L2 hello multiplier 3 3
overload-on-boot P node disabled disabled
overload-on-boot PE node disabled disabled
overload-on-boot RR node disabled disabled
6.4.4 BGP
This section will describe the use of the Multi-protocol Border Gateway Protocol (M-BGP) protocol for the distribution of routing information within the VN2 IP Core. 6.4.4.1 Usage
BGP is typically used to distribute routing information pertaining to „destination‟ networks. For example: customer networks, application services, or Internet
destinations. Intermediate network infrastructure and devices are typically excluded from BGP.
Specifically, the following Network Layer Reachability Information (NLRI) types will be used, and the following types of routing information will be distributed by each:
IPv4 Unicast NLRI:
• Upstream and Peer Internet Routes • Customer Internet Routes
• Internal Internet and Application-Service Routes IPv4 VPN Unicast NLRI:
• Customer VPN Routes
6.4.4.2 Protocol Features/Parameters • Extended Communities
• Graceful Restart • MD5 Authentication • Default Timers Settings
6.4.4.3 Autonomous System Number
VNPT/VTN has indicated that there would be the requirement to offer inter-service provider VPN services, therefore the Autonomous System number used to deliver BGP/MPLS IP-VPN and Internet services shall ideally be a public ASN.
VNPT has indicated that a Private ASN will be used initially within VN2.
If no BGP router ID is specified, the „global‟ router ID will be used. If no BGP router ID and no „global‟ router ID is specified, the system interface IP address will be used. Alcatel-Lucent recommends configuring the router-id as the system ip-address. 6.4.4.4 Participants and Topology
The network devices that are required to participate in BGP are those forming the border with the above referenced network destinations (customers, services, and
the Internet). Specifically, all BRAS, PE, ASBR, and RR routers will participate in BGP. The transport core P routers do not require BGP routing information.
6.4.4.5 Route Reflector
The key element of the BGP route distribution logical topology is the route reflector function, provided by dedicated RR routers. Route reflectors provide a central
point of BGP peering in the network, and eliminate the need for a complex full mesh of BGP sessions. For scalability route reflectors will be organized into two regions, North and South, with a unique cluster-id for each region. A full mesh of normal iBGP sessions will be built among all RR routers.
P
PE ASBR
RR
North Central & South
iBGP (Non‐Cluster) iBGP, North Cluster iBGP, South Cluster RR PE P P
Figure 5: Route Reflector Architecture in day 1
P
PE ASBR
RR
North Central & South
iBGP (Non‐Cluster) iBGP, North Cluster iBGP, South Cluster RR
PE
P P
On day-one each of two RR routers will establish iBGP sessions to all PE and ASBR routers, and associate these with their regional cluster. In the future both of the two RR routers in each region will establish iBGP sessions with all PE and ASBR router in the same region, and associate them with their regional cluster.
6.4.4.6 ASBR
ASBR routers serve the function of learning routes associated with internet peers or transit providers, and relaying them to the RRs. The ASBRs will establish iBGP sessions with the two RR routers. Each ASBR will establish eBGP sessions with peers and transit providers, learning the full internet routing table. Default BGP behaviour is for eBGP routes to be passed on to iBGP neighbours, so these routes will be passed on to the RRs. Next-hop-self should be used to rewrite the next-hop of external routes to the loopback of the ASBR itself. The ASBRs will learn internal/customer routes from the RRs.
6.4.4.7 PE
PE routers participate in BGP in order to originate customer routes into the network. All PE routers will establish iBGP sessions with the two RR routers. The PE may also
establish eBGP sessions with some transit customer, learning customer routes, and relaying them to the RRs as per default BGP behaviour, but also using a next-hop-self policy. The PE will also be configured to originate customer routes learned via other protocols into BGP, including VPN routes. The PE routers will learn both internet routes and other customer/vpn routes from the RRs
Each PE router, or pair of PE routers at a single POP location, will also be configured as BGP route-reflectors, using a unique cluster-id derived from the system address of the PE. The PE routers will establish iBGP sessions with all directly connected BRAS routers, and associate these sessions with the local cluster. In this way customer routes can be learned from the BRAS routers and relayed to the global RRs. Again, a next-hop-self policy will be used toward the RR so that all customer routes including those originated by the BRAS will be reachable via the loopback of the PE itself.
6.4.4.8 BRAS
BRAS routers will participate in iBGP in order to originate routes for locally configured customer IP pools into BGP. BRAS routers will establish iBGP sessions with directly connected PE routers only, and will be configured to originate customer IP pool routes. They will learn default routes only from the PE routers
6.4.4.9 Authentication
The TCP MD5 signature option for cryptographic authentication defined in RFC2385 should be adopted for both IBGP and EBGP sessions. The TCP MD5 signature option defines a TCP option for carrying an MD5 digest in a TCP segment, acting like a signature for that segment. As BGP uses TCP as its transport, it is inherently secure if this mechanism is adopted. Keys used for Internal BGP sessions should be different to those used for external peering.
Since authentication is performed between neighboring routers, the authentication key is configured on the neighbor level.
6.4.4.10 Policies and Route Selection
The logical topology and flow of routing information described above will result in the following route selections and traffic flow:
• Traffic entering the network through a BRAS router will follow a BGP default route with a next-hop of the directly connected PE router. Normal IP routing will be used to reach that next-hop.
• Traffic entering the network through a PE or ASBR router will follow a specific BGP route to its destination, with a next-hop of another PE or ASBR. That next-hop will be reachable via MPLS. Each PE/ASBR will
learn all routes from two RRs as choose the best path from the two. In most cases these two routes will have the same next-hop.
• The RRs learn all routes from the PE/ASBR routers, and each RR makes a single best path decision. Only the selected best path is relayed by each RR to its cluster members, so any polices required to influence BGP route selection must be implemented on the RR.
• The RRs must participate in IS-IS and LDP in order to have visibility to the routing information for the hops of all BGP routes, especially MPLS next-hops for VPN routes
6.4.4.11 Routing of Internet Traffic
There will be three POPs where connections are made to the VDC network, which acts as an Internet transit provider to VN2. The three POPs correspond to the main points international circuit termination points on the VDC networks. The desired strategy is for VN2 to transport traffic to the POP used as the international exit point for that destination before handing it over to VDC. And conversely for VDC to hand traffic directly back to VN2 at the same POP where it was received, which should result in a symmetrical traffic flow of 'best exit / nearest entry' from the perspective of VN2
6.4.4.11.1 Outbound
For outbound traffic to reach the best exit POP from VN2 to the VDC network each PE must receive the best exit route from the RR. Then the PE will send traffic via MPLS to the ASBR with the next-hop of that route. The BGP route selection to determine the best exit must take place on the RR.
The RR receives routes from each of the (5) ASBRs, and must have a means of
differentiating them. BGP communities should be applied by the ASBR to indicate the POP in which the route was learned, and BGP communities must be sent from VDC to indicate the POP where the routes come in from the international side. Policies can be used the increase the local-preference of the routes learned at the same POP as the international link preferred of that route on the VDC side.
6.4.4.11.2 Inbound
For inbound traffic to take the „nearest entry‟ POP from the VDC network toward VN2, the international facing routers in the VDC network should select routes with the next-hop of the VDC ASBR at the same POP. This behavior can be achieved in several ways, depending on the logical topology used by the VDC network.
6.4.4.11.3 Illustration
The following is a simplified illustration of the physical links, logical BGP protocol topology, and the flow of traffic through VDC to and from the Internet that will result from the routing policies described above. The traffic flow toward ISP B traverses the VN2 network from North to South to reach the exit point closest to ISP B. The traffic flow from ISP A takes the entry point closes to ISP A and does not traverse the VDC network from North to South.
VN2
P PEVD
C
ASBR ISP A North South RRs ASB R PBGP Best Exit traffic flow ISP B
Nearest Entry traffic flow
Figure 7: Internet Traffic flow via VDC
6.4.4.12 BGP timing overview (Suggested)
Default Configured min-route-advertisement 30 sec 30 sec
connect-retry 120 sec 120 sec
hold-time 90 sec 90 sec
keepalive 30 sec 30 sec
min-as-origination 15 sec 15 sec Table 4 : BGP timings
6.5
IP Multicast Recommendations
P2MP LSPs is an alternate method of distributing multicast traffic. It provides the 50ms MPLS FRR capability. This feature will need to be tested and verified between the P and PE nodes to ensure a stable rollout of multicast services using this feature. Until this feature operationally tested, PIM will be used for multicast services.
Multicast traffic in VN2 is classically routed based on PIM-SM, which is best suited multicast routing protocol for sparsely dispersed sources, and more importantly, has less protocol control overhead compared with other multicast routing protocols.
It is advised that each interface has PIM-SM correctly configured, to ensure proper operation and correctly forwarding of the multicast traffic across the backbone. PIM will be enabled on all routers in VN2 IP Core and the MANs. The MAN Agg PE routers will for PIM neighbours with the IP Core PE routers.
IP Core
PIM PIM
IGMP static joins
Duplicated MC Traffic stream
MAN
PIM
Figure 8: PIM Topology
For multicast resiliency, the IP Core PE will use IGMP static joins to distribute duplicate copies of desired multicast streams to the two MAN Agg PEs. With PIM, the MAN is able to intelligently manage duplicated multicast streams effectively by ensuring that the end devices only receive a single stream.
6.5.1 Rendezvous Point (RP) placement
According to the PIM-SM protocol, all join messages are forwarded to a specific IP address, the rendezvous point (RP). It is recommended to place the RP as close as possible to the source.
Considering IPTV as the primary use of multicast in VN2, it is understood that the VHO will be located within the vicinity of the HNI site; therefore, it is recommended that the RP be designated on HNI PE router nodes.
P PE (Anycast RP) IPTV Source (Primary) Edge PIM‐SM North South PE P MSDP PE (Anycast RP) IPTV Source (Backup) IGMP
Figure 9: Multicast RP Topology
However, moving forward should the requirement for multicast VPN increases, it would be advisable to relocate the RP to a more centralise node within VN2 so that join latency is optimum throughout the entire network.
Redundant RP should be implemented using Any-Cast RP whereby both HNI PE routers will be configured with the same RP address on a loopback interface. Redundant RP in accordance to draft-ietf-pim-anycast-rp-0X should also be configured to ensure that both RP will be capable of synchronising source information. This is to ensure that any failure to either RP, the redundant RP have the most current source states and thus minimises convergence delay.
6.5.2 Reverse Path Forwarding (RPF) considerations
Multicast RPF is used in conjunction with PIM-SM to ensure loop-free forwarding of multicast packets. In multicast routing the decision to forward traffic is based upon source address and not on destination address as is the case with Unicast routing. It does this by utilizing either a dedicated multicast routing table or alternatively the router's native Unicast routing table.
When a multicast packet enters a router's interface it looks up the list of networks that are reachable via that input interface. If the router finds a matching routing entry for the source IP of the multicast packet, the RPF check passes and the packet is forwarded to all other interfaces that are participating in multicast for this multicast group. If the RPF check fails the packet will be dropped.
It is recommended that RPF-check should be enabled based on both Multicast & Unicast RIB.
6.6
MPLS Topology and Signalling Overview.
6.6.1 Overview
This section will describe the use of Multi-Protocol Label Switching (MPLS) for forwarding traffic within the VN2 IP Core network.
P P
ASBR
PE Single Hop RSVP RSVP Full
Mesh Between P routers PE POP A P P POP B RSVP Signaled LSP LDP over RSVP Figure 10: VN2 MPLS Topology 6.6.2 Usage
MPLS will be used in two fundamental ways in the VN2 network. The first is as the baseline forwarding scheme across the WAN portion of the network, providing traffic engineering and fault protection. And second tier of MPLS is used as a unified means of connectivity between edge routers, enabling end-user services like Layer3 VPNs, VPLS, and Pseudowires.
Two sets of protocols, participants, and topologies will be used for each, as follows:
Backbone MPLS:
• RSVP and LDP protocols
• Full mesh RSVP LSPs between P routers
• LDP will be enabled for full mesh topology to be dynamically created by default
• Statically configured dual mesh topology (A + B planes) • Traffic Engineering (TE) capabilities
Edge MPLS:
• LDP and RSVP protocols
• Single hop RSVP LSPs between P router and PE or ASBR
• T-LDP sessions between PEs and ASBRs will be tunnelled over RSVP tunnels (LDPoverRSVP)
• LDP will be enabled for full mesh topology to be dynamically created by default
• Follows IGP to determine paths, dependent for re-convergence • Graceful restart and Non stop routing enabled for LDP
6.6.3 Backbone MPLS
The backbone MPLS scheme based on the Resource Reservation Protocol (RSVP) protocol provides a highly configurable framework for forwarding traffic across the core of the network. As RSVP LSPs must be explicitly configured at the origin for each source-destination pair, its use of full mesh is generally confined to the network core, in order to limit the size of the N squared full mesh of LSPs among participating PE routers. In the VN2 network, the full mesh RSVP LSPs will be limited to the P routers.
6.6.4 RSVP Signalled LSP Topology
The following grids give a complete list of the unidirectional RSVP signaled LSPs to be configured among the P routers. Check marks indicate an LSP which corresponds to a direct physical link. Check-plus indicates an LSP with an intermediate hop.
Plane A
From: To:
P1 - HPY P1 - HNI P1 - DNG P1 - HCM P1 - CTO
P1 - HPY 9 9 9+ 9+
P1 - HNI 9 9 9 9+
P1 - DNG 9 9 9 9
P1 - HCM 9+ 9 9 9
Plane B
From: To:
P2 - HPY P2 - HNI P2 - DNG P2 - HCM P2 - CTO
P2 - HPY 9 9 9+ 9+ P2 - HNI 9 9 9 9+ P2 - DNG 9 9 9 9 P2 - HCM 9+ 9 9 9 P2 - CTO 9+ 9+ 9 9 Interlinks From: To: P1 - HPY P2 - HPY P1 - HNI P2 - HNI P1 - DNG P2 - DNG P1 - HCM P2 - HCM P1 - CTO P2 - CTO 6.6.5 Traffic Engineering
RSVP provides a number of features for fine grained control over the path selection for each LSP, collectively providing the Traffic Engineering (TE) capabilities. Those
parameters include a subscribed bandwidth per LSP, admin groups (or „colors‟) per link for inclusion or exclusion, a link metric scheme for TE, and the hop count. Also, RSVP paths can be explicitly defined on a per-hop basis, or allowed to follow the same IGP metric based path selection.
As the traffic levels are expected to be low on the VN2 network on day one, but are
otherwise unknown, it is proposed to use a course bandwidth allocation scheme, well below the actual available link bandwidth. This scheme may be used for
future true bandwidth subscription based TE, or as a means to load share paths across future parallel links.
Otherwise, the largely single-hop LSP topology, and the implied primary/backup paths given by the physical circuit topology, largely negate the need for other types of TE configuration. Only the LSP marked with a check-plus in the above grid
have any intermediate hop which might be subject to a policy based path decision, and these would be expected to be among the lowest bandwidth consumption
paths. As such, no other TE configuration is recommended, and the use of the IGP cost scheme to determine LSP paths should be sufficient.
6.6.6 Edge MPLS
The edge MPLS scheme leverages the multiprotocol capabilities of MPLS to support converged service offerings. Single hop RSVP LSPs will be used to connect the PE and P routers. The RSVP full mesh is only within the P routers. Source PE routers will use LDP over RSVP tunnels to reach the destination PE routers. With properly placed redundant links between the PE routers, this
provides MPLS FRR capability between PE and P routers without the need of a full mesh end to end RSVP tunnels between the PE routers.
As a backup to RSVP, LDP can be used to build a full mesh of LSPs among the PE, ASBR and RR routers.
LDP must be enabled on all of the following: • Links from PE to P routers
• Links from ASBR to P routers • Links from RR to P routers
• Links connecting two PE routers at the same POP • Links connecting two P routers at the same POP
• Via tunnelling over all RSVP LSPs connecting to P routers
LDP uses the IGP to determine the path used, and also counts on the convergence of the IGP in the event of a failure. In the event of a failure on a PE/ASBR/RR to P router link the recovery of the LSP will be dependent on the IGP convergence around the failure.
6.7
High Availability
This section describes high availability features of the core VN2 topology. For VPN service high availability, please refer to Section 7.2 Service Architecture as the topic of offering VPN high availability is being discussed as an extension.
6.7.1 Border Router related Failure
For VN2 ASBR sites with redundant ASBR, single ASBR failure would not constitute to any prolonged failure. Redundant ASBR router will be capable of performing all forwarding should the primary ASBR router fails.
In the rare event of a critical failure in VDC or ASBR site, it will render traffic un- reachable to and from a particular set of ASBR routers. At this point in time, VN2 BGP will converge and HSI traffic will be automatically re-routed to the next administratively nearer ASBR routers.
ailu r
This behaviour covers the following scenarios:
1. Failure of VDC connect – BGP routes gets withdrawn on ASBR, the update is populated to both BGP route reflectors and subsequently to other PE routers and BRAS within VN2. PE and BRAS will reconsider other available routes in the Routing Information Base and installs the new route
destination in the forwarding table for traffic to be re-routed.
2. Failure of ASBR routers – it would be extremely rare for both redundant ASBR routers to fail at the same time, nevertheless, should this occurs, BGP neighbour association with BGP route reflectors would drop and all BGP routes previously learned from the affected ASBR routers will be withdrawn, this update is then populated to PE routers and BRAS within VN2. PE and BRAS will reconsider other available routes in the Routing Information Base and installs the new route destination in the forwarding table for traffic to be re-routed.
Figure 11: F e of ASBRs and/or VDC interconnect
In Figure 11, HPG PE routers re-converges upon BGP notification that the preferred traffic egress via HNI ASBR is to be changed to ASBR DNG due to a major failure on either VDC or the HNI ASBRs.
6.7.2 PE upstream related Failure
In the event of failure on the upstream of the PE router, POP with single PE router and redundant uplinks will converge and uses the alternate path towards its destination as in the example illustrated by Fig 12 for CTO POP.
re 1 P O P 1 PO P 2 LS P B , Pr i. P at h LS P A , F R R P at h LS P A , P ri. P at h
Figu 2: PE upstream related failure
As for POPs that has dual PE routers, such as the HPG POP illustrated, uplink redundancy will have to be handled via the MAN-E. This is a temporary workaround until inter-PE links are made available.
6.7.3 P router RSVP Label Switch Path Recovery
Based on the physical topology of the VN2 network and the proposed dual mesh LSP topology, most RSVP LSPs will correspond to a direct circuit link between two routers, and all POP-to-POP circuit paths are deployed with two parallel physical circuits. As such, most recoverable LSP interruptions will be caused by link failure, and the desired response is for traffic to be shifted from the failed circuit to the parallel circuit.
Physical Links
RSVP LSP and Primary Path RSVP LSP Recovery
Path
This failure scenario lends itself well to link protection using Fast Reroute. The FRR path used would be via neighbouring P routers in the same POP, and over
the parallel circuit, as in the following illustration. This would be expected to be the path of the re-established LSP as well.
6.8
Network QoS Design
6.8.1 QoS Overview
The VN2 provides service differentiation using the Differentiated Service Model. It provides a high-level of flexibility to meet different operating environments, which can sometimes appear daunting and/or complex; indeed, there is a faint line between flexibility and complexity. Prior to detailing the recommended QoS configuration for the VN2 Network, this section provides a high-level overview of QoS and traffic management on the 7750 SR. This section is not intended to detail every aspect of traffic management within the 7750 SR, only the parts pertinent to the proposed QoS model.
6.8.1.1 Traffic Classification
In order to achieve both scalability and service differentiation, the 7750 SR aggregates microflows in up to eight Forwarding Classes (FC). The behaviour of a FC in terms of ingress marking interpretation, egress marking, and queuing/scheduling is configurable in order to define a different Per-Hop Behaviour (PHB) per FC.
By default the Forwarding Classes are classified into three class types; High priority, Assured, and Best Effort (BE). High-priority FCs are serviced before Assured FCs and are intended to be used for real-time delay-sensitive traffic or network control traffic. In turn, Assured FCs are serviced before Best Effort FCs and provide services with a committed rate (Committed Information Rate, or CIR) and peak rate (Peak Information Rate, or PIR). Assured FCs provide the ability to classify ingress traffic as either in-profile or out-of-profile based upon the traffic arrival rate. A queue is considered in the in-out-of-profile state if the rate at which the queue is being serviced is less than its configured CIR. A queue is
considered out- of-profile if the rate at which the queue is being serviced is greater than its CIR, but less than its PIR. After the profile state of the packet is determined at service ingress, the profile state of the packet influences the packets queuing priority and drop preference at all subsequent queuing points. It is worth noting that the in- profile/out-of-profile classification based upon the rate that a queue is being serviced can only be performed on access ingress ports (not network ingress ports). When given some thought this makes sense because at network ingress ports traffic has already been policed (at access ingress).
Table 5 details the eight Forwarding Classes within the 7750 SR SR together with the type of class and its intended use.
FC FC name Class Type Remarks
NC Network
Control Real Time Network Control traffic
H1 High-1 Real Time For a second Network Control traffic or delay/jitter sensitive data
EF Expedited Real Time For delay/jitter sensitive data H2 High-2 Real Time For delay/jitter sensitive data L1 Low-1 Assured Assured traffic
AF Assured Assured Assured traffic L2 Low-2 Best Effort For BE traffic BE Best Effort Best Effort For BE traffic
Table 5: Forwarding Classes
In order to understand the 7750 SR QoS architecture, it is important to understand the concept of Forwarding Classes. As a standalone entity FCs do not provide any QoS capability such as classification, metering, marking or policing. FCs are a conduit and require an input (classified traffic) and an output (one or more queues).
I FC
Figure 14: Forwarding Class Concept
6.8.1.2 Components of the DiffServ Model
Within the 7750 SR DiffServ architecture, there are four constituent components that define the PHB afforded to traffic. These are the Access Ingress, Network Egress, Network Ingress, and the Access Egress.
Ser vice Acce ss Poi nt
Ser vice Acce ss Poi nt
Ser vice (Access) I ng r ess
N etwor k Eg ress
N etwor k I ng ress
Ser vice (Access) Eg r ess
Figure 15: DiffServ PHB Components
Access Ingress Associated with a Service Access Point and responsible for classification of traffic into an FC based upon Layer-2 (i.e.
MAC, 802.1p, Ethertype) Layer-3 (i.e. source/destination IP, protocol, DiffServ Codepoint/IP precedence) or Layer-4 (i.e. source/destination port) criteria. Ingress queues are subsequently associated with each FC and resource such as packet buffer; permissible traffic rates for in-profile/out-of- profile traffic, and scheduling priority (towards the fabric) are allocated to each queue. Network Egress Associated with a network interface and responsible for mapping
each FC and profile (in/out) into an associated MPLS EXP value. Egress network queues are associated with each FC and again resource such as packet buffer and scheduling priority (towards the line) are allocated to each queue.
Network Ingress Associated with a network interface and responsible for classification of traffic into FCs based on MPLS EXP or DSCP bits. Ingress network queues are associated with each FC and resource such as packet buffer and scheduling priority (towards the fabric/other network processors) are allocated to each queue.
Access Egress Associated with a Service Access Point for mapping traffic into egress queues based upon FC or DSCP. Resource such as packet buffer, permissible traffic rates, and scheduling priority (towards the line) are allocated to each queue. Remarking of Layer-2 802.1p bits can also be implemented at the Access Egress (recall that 802.1q bits are not carried over a pseudowire).
6.8.1.3 Buffer Management
Logical default buffer pools exist at the port and media dependent adapter (MDA) levels. Each physical port has three logically associated pools; an ingress access pool, and egress access pool, and an egress network pool. If the mode of a port is configured as access, only ingress access pool and egress access pool are
created for the port. Conversely, if the mode of a port were configured as network, only the egress network pool would be created for the port. The size of the buffer pools is
automatically determined as a function of the MDA type and the port configuration (i.e. mode and speed of the port).
