Jeffrey Haas
JUNIPER
Firewall on Demand Multidomain
S E C U R I T Y V I A B G P F L O W S P E C & A W E B P L A T F O R M
Leonidas Poulopoulos
GRNET NOC
Wayne Routly
DANTE
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
S E C U R I T Y V I A B G P F L O W S P E C & A W E B
P L A T F O R M
Firewall on Demand
L e o n i d a s P o u l o p o u l o s
l e o p o u l @ n o c . g r n e t . g r
G R N E T N O C
( @ l e o p o u l )
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
GRNET NOC
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
3
Staff:
15
Network:
120 devices (40 routers/80
switches)
Juniper-based network
Presence:
90 cities
Clients:
~100
UPSTREAM
NREN
IX
DDoS attack traffic consumes
network capacity
DDoS attack launched from
compromised systems (bots)
DDoS attack targets
applications and services
Victim
DDoS Illustrated
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
4
DDoS facts
<1 1 3 10
17 24
40 49
100
60 60
309
0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 1 0 1 1 1 2 1 3 1 4400
Gbps
Firewall on Demand Multidomain
5
Internet2 Global Summit, Apr 9 2014
Source: Arbor Networks Inc. & Cloudflare
Staying alive…
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
6
acls, firewall filters
RTBH
BGP FLOWSPEC
IETF AND JUNIPER ROADMAP
8
Copyright © 2014 Juniper Networks, Inc. www.juniper.netBGP FLOWSPEC
BGP Flowspec was originally defined in RFC 5575 and has been
part of JUNOS since version 7.3. It permits layer 4 (TCP and
UDP) firewall filters to be distributed in BGP on both a
intra-domain and inter-intra-domain basis.
Flowspec was originally defined to assist in mitigation of DDoS
attacks. Deployments may use native configuration to distribute
the filters. Several DDoS mitigation environments will generate
the filters in support of their detection and mitigation tools.
9
Copyright © 2014 Juniper Networks, Inc. www.juniper.netCURRENT IETF WORK
draft-ietf-idr-bgp-flowspec-oid
Formally permits IBGP origination of BGP flowspec routes without
requiring a longest-match for validation. In practice, operators
have been using policy knobs to permit similar behaviors for
non-eBGP originated flowsec.
draft-haas-idr-flowspec-redirect-rt-bis
Clarifies some issues in RFC 5575 for the “Redirect to VRF”
Route-Target. As currently documented, it’s not possible to have a fully
compatible BGP Flowspec implementation.
10
Copyright © 2014 Juniper Networks, Inc. www.juniper.netCURRENT IETF WORK
draft-ietf-idr-flowspec-redirect-ip adds some exciting features to
BGP flowspec:
Permit redirection of traffic to a specific IP address rather than
requiring tunneling via VRF.
Permit the copying of traffic in a similar fashion.
Some issues with the feature encoding and precedence of rules
are being worked out currently. New draft expected soon.
draft-ietf-idr-flow-spec-v6
Provide for support for IPv6 in flowspec. Necessary changes
include:
(Limited) Support for Next Header.
Flow Label support
Ambiguous case of Traffic Class with regard to ECN still under debate.
11
Copyright © 2014 Juniper Networks, Inc. www.juniper.netJUNIPER ROADMAP
15.1 – Flowspec ISSU/NSR support, draft-oid validation rules
15.2 (tentative) – Redirect-IP
12
Copyright © 2014 Juniper Networks, Inc. www.juniper.netINTO THE REALM OF SPECULATIVE FICTION…
BGP Flowspec provides a convenient encoding mechanism to
permit Layer3+ traffic filters be distributed. Future facing work,
such as Software Defined Networking (SDN), Service
Chaining/Network Function Virtualization or Interface to the
Routing System (I2RS) may be able to leverage flowspec as a
mechanism to distribute custom forwarding behaviors.
BGP community flow
vs.
RTBH
vs.
ACLs
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
13
•
Distributed
across the
network
•
Closer
to the source
•
Fine-grained
even on
core/backbone networks
•
Multidomain
easy
propagation towards the
upstream via BGP
•
Easy automation &
integration
ACL
S
•
Flowspec:
enhancement
of RTBH
•
Does not affect
all traffic
to victim
•
Less coarse
•
More actions
•
Separate NLRI
Firewall on Demand
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
14
G
RANULARITY
:
Per-flow level
A
CTION
:
Drop, rate-limit, redirect
S
PEED
:
1-2 orders of magnitude quicker
E
FFICIENCY
:
closer to the source, multi-domain
A
UTOMATION
:
integration with other systems
M
ANAGEABILITY
:
status tracking, web interface
N
EED
FOR
BETTER
TOOLS
TO
MITIGATE
FoD Architecture
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
15
User Interface
Django MVC
Long Polling
(Gevent)
Job Queue (Celery/Beanstalk)
Caching Layer
(Memcached)
Network Config to
XML proxy (nxpy)
Python NETCONF client
(ncclient)
NETCONF
eBG
P
eBGP
iBGP
iBGP
Shibboleth
•
https://code.grnet.gr/projects/flowspy
•
http://flowspy.readthedocs.org
O
PEN
S
OURCE
https://fod.grnet.gr
FoD Screenshots
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
16
…more during
demo
How it works – Single domain
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
17
•
Customer’s NOC logs in web
tool (shibboleth) &
describes
flows and actions
•
Destination
validated
against
customer’s IP space
•
A dedicated router is
configured (
NETCONF
) to
advertise the route via BGP
flowspec
•
Dynamic firewall filters are
implemented on
all
routers
•
Attack is mitigated
upon
entrance
•
End of attack:
Removal
via
the tool, or
auto-expire
Web
NETCONF
eBGP
iBGP
UPSTREAM
GRNET
Client
Client
IX
FoD
GRNET FoD usage examples
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
18
What now? Idea!
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
19
BGP
is by nature MULTIDOMAIN
Deploy FoD in a
MULTIDOMAIN
Environment
GÉANT
connect • communicate •
collaborate
Firewall on Demand – A
Multi-Domain
Implementation
Wayne Routly
Security Manager
DANTE
connect • communicate • collaborate
GÉANT : Who What How
21
Pan-European Network
…..Transit Network….ISP
30 Physical Pops
50,000 km network
infrastructure on 44 routes
100Gb/s
100s TB of Data
15+ Million IPs
100+ Workstations
Truly Global (50 million users)
10,000 institutions
Interconnects
European NRENs - 40
connect • communicate • collaborate
Today
Little bit of DDoS on the side…..
NTP, DNS, SMTP……. Amplification Attacks
2k DDoS Events (183 pm)
298 vs 929 ….. 1k in 2014, average 300
connect • communicate • collaborate
Today
DDoS Events – CyNet
Target: The University of Cyprus (
www.ucy.ac.cy
)
Port Ranges:
0, 2070 and 3475
Multiple Source
IP’s and source AS’s.
Attack peak: Over 13G over 1G link
connect • communicate • collaborate
Destination ports for 194.42.x.x
Date Seen
Dst Port
Flows (%)
Packets (%) Bytes (%)
2013-09-02 04:58 2070
47268(37.8)
144.2 M(32.7)
182.4 G(35.3)
2013-09-02 04:58 0
46315(37.1)
260.0 M(59.0)
295.4 G(57.1)
2013-09-02 04:58 3475
29714(23.8)
31.3 M( 7.1)
39.2 G( 7.6)
2013-09-02 04:58 771
1348( 1.1)
4.3 M( 1.0)
243.6 M( 0.0)
2013-09-02 04:58 769
145( 0.1)
516000( 0.1)
29.0 M( 0.0)
2013-09-02 04:58 2816
55( 0.0)
199500( 0.0)
16.7 M( 0.0)
2013-09-02 04:58 1024
30( 0.0)
114500( 0.0)
6.4 M( 0.0)
Destination AS 3268 Traffic
Date Seen
Dst IP Addr
Flows (%)
Packets (%)
Bytes (%)
2013-09-02 04:58 194.42.x.x
124919(97.2)
440.6 M(99.2)
517.4 G(99.5)
2013-09-02 04:59 82.116.x.x
129( 0.1)
143000( 0.0)
154.3 M( 0.0)
2013-09-02 05:00 194.42.x.x
128( 0.1)
244000( 0.1)
12.3 M( 0.0)
2013-09-02 04:59 194.42.x.x
114( 0.1)
57000( 0.0)
10.5 M( 0.0)
2013-09-02 04:59 82.116.x.x
90( 0.1)
239500( 0.1)
311.4 M( 0.1)
2013-09-02 04:59 194.42.x.x
81( 0.1)
40500( 0.0)
8.7 M( 0.0)
Today
DDoS Events – CyNet [2]
connect • communicate • collaborate
Date first seen Duration Proto Dst IP
Addr Flows(%) Packets(%) Bytes(%) pps bps bpp 2013-03-13 09:34:05.770 5701.654 any 194.177.211.102 35531( 7.8) 36.1 M(11.3) 53.5 G(11.9) 6330 75.0 M 1481 2013-03-13 09:34:05.782 5701.975 any 194.177.211.100 34632( 7.6) 35.6 M(11.1) 52.6 G(11.7) 6236 73.8 M 1478 2013-03-13 09:33:46.665 5720.961 any 194.177.211.101 34469( 7.6) 35.3 M(11.1) 52.2 G(11.6) 6164 72.9 M 1478 2013-03-13 09:33:14.456 5797.618 any 194.63.239.233 49621(11.0) 31.8 M(10.0) 44.3 G( 9.9) 5478 61.1 M 1394 2013-03-13 09:33:17.612 5753.346 any 194.63.239.234 48220(10.6) 27.1 M( 8.5) 36.7 G( 8.2) 4705 51.1 M 1356 2013-03-13 09:33:12.442 5791.562 any 194.63.239.237 39278( 8.7) 26.1 M( 8.2) 36.5 G( 8.1) 4503 50.4 M 1400 2013-03-13 09:33:11.553 5800.394 any 194.63.239.232 42260( 9.3) 26.1 M( 8.2) 36.4 G( 8.1) 4495 50.2 M 1394 2013-03-13 09:33:16.562 5794.656 any 194.63.239.231 46109(10.2) 26.6 M( 8.3) 36.1 G( 8.0) 4593 49.8 M 1356 2013-03-13 09:33:15.479 5755.473 any 194.63.239.238 44189( 9.8) 26.1 M( 8.2) 35.3 G( 7.9) 4527 49.1 M 1356 2013-03-13 09:33:38.839 5733.229 any 194.63.239.236 39860( 8.8) 25.2 M( 7.9) 34.6 G( 7.7) 4393 48.2 M 1372 2013-03-13 09:33:56.632 5714.286 any 194.63.239.235 38534( 8.5) 23.2 M( 7.3) 31.4 G( 7.0) 4053 44.0 M 1356
Summary: total flows: 452861, total bytes: 449.6 G, total packets: 319.0 M, avg bps: 620.0 M, avg pps: 54994, avg bpp: 1409
Time window: 2013-03-13 09:33:09 - 2013-03-13 11:10:00
Total flows processed: 38411283, Blocks skipped: 0, Bytes read: 2304722444
Sys: 6.808s flows/second: 5641281.6 Wall: 6.723s flows/second: 5713256.9
Date first seen
Dst IP Addr
Flows (%)
Packets (%)
Bytes (%)
2013-03-13 09:34
194.177.211.x
35531( 7.8)
36.1 M(11.3)
53.5 G(11.9)
2013-03-13 09:34
194.177.211.x
34632( 7.6)
35.6 M(11.1)
52.6 G(11.7)
2013-03-13 09:33
194.177.211.x
34469( 7.6)
35.3 M(11.1)
52.2 G(11.6)
2013-03-13 09:33
194.63.239.x
49621(11.0)
31.8 M(10.0)
44.3 G( 9.9)
2013-03-13 09:33
194.63.239.x
48220(10.6)
27.1 M( 8.5)
36.7 G( 8.2)
2013-03-13 09:33
194.63.239.x
39278( 8.7)
26.1 M( 8.2)
36.5 G( 8.1)
DNS Amplification Attack
•
Target:
GRNET
•
Port Ranges:
53 (DNS)
•
Multiple Source IP’s & Source
AS’s.
•
Attack peak: 20G over 10G link
Today
DDoS Events – GRNET
connect • communicate •
collaborate
connect • communicate • collaborate
Today
connect • communicate • collaborate
Strategy
…security solutions that simplify the improvement of
the security status quo…
connect • communicate • collaborate
Requirements - Defining
It must be easy to use
It must be ENHANCE security
Must deliver MEASURABLE VALUE
REDUNDANCY must be incorporated
into existing processes
…accepted by all participants …. conform to BEST PRACTICES & STANDARDS
Must be SCALABLE.
connect • communicate • collaborate
GÉANT Security
Complete Security Solution - NSHaRP
It is a mechanism to quickly and effectively inform affected users of
incidents detected transiting the GÉANT
network dynamically.
It adds value by serving as an extension to an NRENs CERT, by adding
visibility to incidents targeting or originating from your network
Innovative and Unique - Caters for different types of requirements
….is a process that will enhance GÉANT backbone security and will
extend the NRENs ability to protect their infrastructure….
connect • communicate • collaborate
Firewall on Demand …But Why?
…
better tools
to mitigate
transitory
attacks and anomalies
“Better” in terms of
Granularity: Per-flow level
–
Source/Dest IP/Ports, protocol type, DSCP, TCP flag…
Action:
–
Drop, rate-limit, redirect
Speed: More responsive
–
(Seconds / Minutes vs. Hours / Days)
Efficiency:
–
Closer to the source, Multi Domain
Automation:
–
Integration with other systems (NSHaRP)
Manageability
connect • communicate • collaborate
Customer
FoD
NREN A
NREN B
GEANT
Credit: Andreas Polyrakis, GRNET
LEVEL3
•
NSHaRP Customer or GN NOC
logs into web tool and describes
flows and actions
•
Flow destination is validated
against the customer’s IP space
•
Dedicated router is configured to
advertise the route via BGP
flowspec
•
iBGP propagates the tuples to all
GEANT routers.
•
Dynamic firewall filters are
implemented on all routers
•
Attack is mitigated (dropped,
rated-limited) upon entrance
•
End of attack: Removal via the
tool, or auto-expire
Firewall on Demand … Tomorrow
connect • communicate • collaborate
Phase 1
- Test Flow Spec on GN
Athens Router
- Test Propagation to
GN
Gateways
Phase 2
- Deploy Flow Spec
Server
- Web Interface
- Pilot
Phase 2 (b)
- Processes
- API
- Production Service
Firewall on Demand … Roadmap
Today 6 Months 12 Months
GÉANT Tests
Firewall on Demand Multidomain
34
Internet2 Global Summit, Apr 9 2014
GÉANT
CARNet
Victim
GRNET
Flowspec FlowspecFoD
FlowspecAttacker
Click
Apply
6 seconds later…
FoD multidomain principles
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
35
FoD
setup & deploy by every interested domain/NREN
Multidomain FoD
deployed in GÉANT
Multidomain FoD authentication:
eduGAIN
Multidomain FoD authorization:
peer address space
GÉANT
accepts
BGP flowspec rules
from
domains
Policies/filters per peering based on
rule dest. addr.
User belongs to a domain/institution/NREN ::
Peer
Peer is assigned an administrative
IPv4 address space
Rule creation with destination address/network
only
FoD multidomain deployment scenarios
Internet2 Global Summit, Apr 9 2014
Firewall on Demand Multidomain
36
GÉANT
NREN
Flowspec Flowspec FlowspecRTBH
ACL
Possible mitigation
with RTBH, ACL
Victim
NREN
Flowspec
Flowspec Flowspec FlowspecFoD
Legitimate Traffic Flows
Malicious Traffic Flows
Flow spec rule propagation
BGP Peering
Flowspec
Flow spec rules
FoD
