BriefHistory.pdf

(1)

A Brief History of Time

in Software Defined Networks

Tal Mizrahi

Supervised by Yoram Moses

Technion – Israel Institute of Technology

(2)

data plane

control plane

Networks

2 A Brief History of Time in SDN

data plane

switch

Data plane

: forwarding packets between users.

(3)

Software Defined Networks (SDN)

switch

controller

switch

data plane

control plane

OpenFlow:

an SDN protocol.

Control plane updates such as routing changes

•

Based on

network-wide

perspective

•

Frequent

(4)

The First Synchronized Clocks

(5)

Networks: Communications vs. Trains

Network switch

Railway switch

(6)

A Brief History of Time in SDN

1800s

1

st

use of

time in

railway

networks.

1980s

Time and

synchronized

clocks in

distributed

systems.

E.g., [Lamport]

The Network

Time Protocol

(NTP).

[Mills]

~ millisecond

accuracy.

The Precision Time

Protocol (PTP)

[IEEE 1588]

~ microsecond

accuracy.

Birth of SDN:

OpenFlow

Enabling

dynamic and

flexible

networks.

[McKeown et al.]

:

Using

accurate time

to coordinate

SDNs.

(7)

at a Glance

Controller

Switches

Synchronized

clocks

(8)

Accurate Synchronization: It’s Already Here

Open Networking Foundation (ONF)

SDN product directory

Precision Time Protocol (PTP)

[

IEEE 1588 2008

]

~

1

μ

sec

accuracy

~70%

~70% of the

SDN-enabled silicons have

native PTP support.

Mobile backhaul

Power substations

Industrial automation Financial applications

China Mobile: over 1,000,000 PTP-enabled base stations

https://mailarchive.ietf.org/arch/attach/tictoc/pdfsY1ADO.pdf

But what about

(9)

at a Glance

Controller

Switches

(10)

The

Project

Why do we need

time

in SDN?

Scheduling

protocols

Accurate scheduling

methods

SDN Clock

synchronization

Flow Swaps

T

IME

4

(11)

Time

is Optimal for Flow Swaps

[INFOCOM ‘16]

A key benefit of SDN:

Dynamic path allocation based on network load

controller

...

o

2

o

1

(12)

controller

...

o

2

o

1

Time

[INFOCOM ‘16]

A key benefit of SDN:

old

new

Temporary congestion.

If O

₁

, O

₂

are

not

updated

at the

same time

…

(13)

controller

...

o

2

o

1

Time

[INFOCOM ‘16]

A key benefit of SDN:

old

new

If O

₁

, O

₂

are updated

at the

same time

:

No congestion!

T

IME

4

(14)

The

Project

Why do we need

time

in SDN?

Scheduling

protocols

Accurate scheduling

methods

SDN Clock

synchronization

Multi-phase

updates

[SOSR, ‘15]

Flow Swaps

T

IME

4

(15)

Example: Consistent Path Update

0

S

2

S

1

S

3

S

4

S2 S1

S3 S4

S2 S1

S3 S4

S2 S1

S3 S4

S2 S1

S3 S4

1

2

3

0. The ‘before’ configuration.

1. Controller updates S

₁

.

2. Controller updates S

₂

.

3. Controller updates S

₃

.

Sequential update approaches:

[Dionysus ‘14]

[zUpdate ‘13]

[Vanbever et al. ‘11]

[Francois et al. ’07]

S

2

S

1

S

3

S

4

after

before

wait…

X

(16)

Simultaneous Updates?

En-route packets run into a ‘black hole’.

Not consistent!

S2 S1

S3 S4

S2 S1

S3 S4

S2 S1

S3 S4

S2 S1

S3 S4

(17)

Timed

Multi-phase Consistent Updates

[SOSR ‘16]

-

The controller sends timed update messages to S

1

, S

2

, S

3

.

-

Scheduled updates occur at times T

1

, T

2

, T

3

.

T

1

T

2

T

3

Controller does not need to wait between steps!

Timed update



Lower update duration.

S2 S1

S3 S4

S2 S1

S3 S4

S2 S1

S3 S4

S2 S1

(18)

The

Project

Why do we need

time

in SDN?

Scheduling

protocols

Accurate scheduling

methods

SDN Clock

synchronization

OpenFlow

Scheduled Bundles

[INFOCOM ’16, OpenFlow 1.5]

Multi-phase

updates

[SOSR, ‘15]

O

NE

C

LOCK

to Rule

Them All

[NOMS, ‘16]

Flow Swaps

T

IME

4

[INFOCOM, ‘16]

Data Plane

Timestamping

(19)

A Timed SDN Protocol

switch

Time

controller

B

u

n

d

le

C

o

m

it

B

u

n

d

le

O

p

e

n

B

u

n

d

le

A

d

1

...

re

p

ly

B

u

n

d

le

A

d

N

B

u

n

d

le

C

lo

s

e

re

p

ly

re

p

ly

Bundle

[OpenFlow 1.4]

1.

Bundle Open

Challenge: defining a generic protocol for timed updates in SDN.

2.

A set of

OpenFlow

commands.

(20)

A Timed SDN Protocol:

Scheduled Bundles

switch

Time

controller

T

s

B

u

n

d

le

C

o

m

it

(a

t t

im

e

T

s

)

B

u

n

d

le

O

p

e

n

B

u

n

d

le

A

d

1

...

re

p

ly

B

u

n

d

le

A

d

N

B

u

n

d

le

C

lo

s

e

re

p

ly

re

p

ly

B

u

n

d

le

D

is

c

a

rd

Our solution:

Scheduled Bundles

A generic method to schedule

any

OpenFlow command.

Our solution has been incorporated into OpenFlow 1.5

Switch executes

bundle at time T

s

Commit may

include scheduled

time T

s

(21)

at a Glance

Controller

Switches

Synchronized

clocks

(22)

The

Project

Why do we need

time

in SDN?

Scheduling

protocols

Accurate scheduling

methods

SDN Clock

synchronization

T

IME

F

LIP

[INFOCOM ‘15]

O

NE

C

LOCK

[NOMS ‘16]

OpenFlow

Scheduled Bundles

NETCONF

Time Capability

[NOMS ’16, RFC7758]

Multi-phase

updates

[SOSR, ‘15]

O

NE

C

LOCK

to Rule

Them All

[NOMS, ‘16]

Flow Swaps

T

IME

4

[INFOCOM, ‘16]

Data Plane

Timestamping

(23)

SDN Switches

switch

controller

switch

TCAM1 TCAM2 ... TCAMn incoming

packet

(24)

TCAM: Ternary Content Addressable Memory

•

Memory for quick searching

•

Top-down search: first match “wins”

•

Ternary: 0 / 1 / *

•

* = don’t care

action

s

u

, …, s

1

TCAM

Network Device

packet’s search

(25)

TCAM Ranges

•

How do you represent a range?

•

Example:

–

F is a 3 bit field

–

Range F



1

•

Range

expansion

= the number of entries

required to represent a range.

000

001

010

011

100

101

110

111

1**

01*

001

(26)

Problem 1: Atomic Bundles

TCAM

1

TCAM

2

...

TCAM

n

incoming

packet

outgoing

packet

Network device

TCAM

1

TCAM

2

...

TCAM

n

incoming

packet

outgoing

packet

Network device

Atomic Bundle [OpenFlow 1.4]:

A set of updates that need to be applied atomically (‘all-or-none’).

(27)

Problem 2: Time-based Updates

controller

...

o

2

o

1

(28)

T

IME

F

LIP

: Timestamp-based TCAM Lookup

packet’s search

key

action

s

u

, …, s

1

TCAM

Network Device

action

s

u

, …, s

1

TCAM

Network Device

T

s

u

, …, s

1

T



T

0

TCAM

Network Device

action

packet’s search key packet’s search key

The timestamp is used in the TCAM key –

time range

.

COTS switches can synchronize clocks very accurately ~ 1 μsec.

[Using IEEE 1588 Precision Time Protocol (PTP) or GPS]

conventional

TimeFlip

(29)

Atomic Bundles Using T

IME

F

LIP

TCAM

1

TCAM

2

...

TCAM

n

incoming

packet

outgoing

packet

Network device

TT0

(30)

Time-based Updates using T

IME

F

LIP

controller

...

o

2

o

₁

old

new

TimeFlip:

switch accurately

starts using ‘new’ at T

₀

.

T

s

u

, …, s

1

T



T

0

TCAM

Network Device

search

key

‘new’

(31)

What is a Timestamp?

32 bits

48 bits

Nanoseconds

Seconds

IEEE 1588 Time Of Day (TOD):

(32)

Timestamp-based TCAM Lookup

T

s

u

, …, s

1

T



T

0

TCAM

Network Device

search

key

action

Problem:

•

Small number of entries in T



T

₀

.

•

Small number of timestamp bits per entry.

•

Without compromising the scheduling accuracy !

Large # of bits in timestamp field?

(33)

Efficient T

IME

F

LIP

s

(34)

The Punctual Teacher Problem

•

Afternoon lesson

•

Synchronize watches

(35)

•

Afternoon lesson

•

Synchronize watches

•

Write down

date

time

(36)

•

Afternoon lesson

•

Synchronize watches

•

Write down

date

time

2016-05-18 15:38

Flexible + punctual !

Scheduling Tolerance

controller

switch switch

(37)

Scheduling Tolerance – TOL

T

min

time

T

max

•

Assume: we allow the

controller

to schedule the

update to any time in the range [T

min

, T

max

]

•

It

selects

T

₀

that minimizes the expansion of

T



T

₀

•

Without compromising the accuracy !

scheduled time

T

₀

TOL

controller

switch switch

(38)

Optimal Scheduling Algorithm

•

Theorem:

T

SCH

is optimal.

No other scheduling algorithm

produces an extremal range with

a lower expansion.

•

Theorem: the average expansion is:

W := the number of bits in the timestamp field

a :=

log

2

𝑇𝑂𝐿

controller

switch switch

(39)

2016-05-18 15:38

2016-05-18

15:38

T

0

T

1

time

I know the date.

No need to write it.

Periodic Ranges

*

…

*

T

1



T



T

0 TCAM

controller

switch switch

(40)

2016-05-18 15:38:10.251987326

2016-05-18 15:38:10

.251987326

Installation Bounds –

Δ

T

0

scheduled

time

insertion

Assumption: a TimeFlip with

TT

₀

is present during [T

₀

-Δ,T

₀

+Δ)

For example, if

Δ

=0.5 second:

T

0

+

D

T

0

-

D

cleanup

T



T

₀

TCAM

s

u

, ...,s

1,[TT0] a TCAM

s

u

, ...,s

1,*...* a

I know which

second it is.

controller

switch switch

(41)

Periodic Continuation Algorithm

T

₀

time

n

×

2

V

(n+1)

×

2

V

T

1

=T

₀

+2

V-1

-1

A periodic continuation algorithm for

T



T

₀

:

Period=2

V

The

V

most significant bits are

don’t care

!

Period



2

Δ

Nyquist ?

controller

switch switch

(42)

Resource Consumption

•

Theorem: using S

CHEDULE

and B

OUNDED

R

ANGE:

–

The

expansion

is bounded by:

(43)

The Cost of T

IME

F

LIP

: 1 Bit / 1 Entry

•

Theorem: if

𝑇𝑂𝐿 ≥ 2

𝑙𝑜𝑔

2

(∆)

, then the every

timestamp range requires as little as:

–

1 TCAM entry.

–

1 bit in the timestamp field.

* * * *

*

timestamp

field

* * * * * *

function of

TOL

(44)

Efficient T

IME

F

LIP

s

(45)

T

IME

F

LIP

: Would it Work on a Real Switch?

Yes !!

Microbenchmark: TimeFlip was tested on a

(46)

T

IME

F

LIP

: Would it Work on a Real Switch?

Yes !!

T

IME

F

LIP

is a

practical

and

efficient

method

(47)

The

Project

Why do we need

time

in SDN?

Scheduling

protocols

Accurate scheduling

methods

SDN Clock

synchronization

R

EVERSE

PTP

[HotSDN ‘14, ISPCS ‘14]

T

IME

F

LIP

[INFOCOM ‘15]

O

NE

C

LOCK

[NOMS ‘16]

OpenFlow

Scheduled Bundles

NETCONF

Time Capability

[NOMS ’16, RFC7758]

Multi-phase

updates

[SOSR, ‘15]

O

NE

C

LOCK

to Rule

Them All

[NOMS, ‘16]

Flow Swaps

T

IME

4

[INFOCOM, ‘16]

Data Plane

Timestamping

(48)

SDN vs. Clock Sync

controller

switch

master

slave

slave slave

Centralized:

programmable

‘brain’ in the controller

Best Master Clock (BMCA)

fixed

‘brain’ in the slaves

Decentralized:

(49)

R

EVERSE

PTP: SDN-style PTP

controller (slave)

switches

master

controller (slave)

switches

master

Switches

–

master

s

Controller

–

slave

Each switch forms a

separate domain.

Per switch i:

-

offset between local

clock and switch i.

i

(50)

R

EVERSE

PTP: Time-based Update

controller (slave)

switches

master 1 _{master 4}

master 2 master 3

controller (slave)

switches

master 1 _{master 4}

master 2 master 3

T

4

T

3

T

2

T

1

Goal

: a simultaneous update at T

_u

.

Controller

tells switch i to update at T

i

Per switch i:

- offset.

o

i

i u

i

o

T

(51)

R

EVERSE

PTP: SDN-style PTP

controller (slave)

switches

master

controller (slave)

switches

master

‘Dumb’ switches

Smart controller

Programmable

algorithmic logic in controller.

(52)

(53)

Publications

Conference

Paper

INFOCOM 2016

T. Mizrahi, Y. Moses, "Software Defined Networks: It's About Time“.

INFOCOM 2015

T. Mizrahi, O. Rottenstreich, Y. Moses, "TimeFlip: Scheduling Network Updates with Timestamp-based TCAM Ranges".

SOSR 2016

T. Mizrahi, E. Saat, Y. Moses, "Timed Consistent Network Updates".

NOMS 2016

T. Mizrahi, Y. Moses, "OneClock to Rule Them All: Using Time in Networked Applications".

ISPCS 2014

T. Mizrahi, Y. Moses "Using ReversePTP to Distribute Time in Software Defined Networks".

SWFAN 2016

T. Mizrahi, Y. Moses, "The Case for Data Plane Timestamping in SDN".

Journal

Status

Paper

ToN 2016

Accepted

T. Mizrahi, E. Saat, Y. Moses, "Timed Consistent Network Updates in Software Defined Networks".

IJNM 2016

Minor revision

T. Mizrahi and Y. Moses, "ReversePTP: A clock synchronization scheme for software defined networks".

TNSM 2016

Major revision

T. Mizrahi, Y. Moses, "Time4: Time for SDN".

Conference

Short Paper / Poster

HotSDN 2014

T. Mizrahi, Y. Moses, "ReversePTP: A Software Defined Networking Approach to Clock Synchronization".

HotSDN 2013

T. Mizrahi, Y. Moses, "Time-based Updates in Software Defined Networks".

ONS 2014

T. Mizrahi, Y. Moses, "On the Necessity of Time-based Updates in SDN".

INFOCOM 2016

T. Mizrahi, Y. Moses, "Serving Time in the Cloud: Why Time-as-a-Service?".

(54)

Practical Aspects

Open Source Prototypes

Standard Organization Publications

OpenFlow

Scheduled

Bundles

NETCONF

Time

Capability

ReversePTP

OpenFlow

1.5

OpenFlow

1.3.x

ext. package

IETF

RFC 7758

Experimental

Evaluation

Emulab

DeterLab

Amazon

AWS

(55)

The

Project

Why do we need

time

in SDN?

Scheduling

protocols

Accurate scheduling

methods

SDN Clock

synchronization

R

EVERSE

PTP

[HotSDN ‘14, ISPCS ‘14]

T

IME

F

LIP

[INFOCOM ‘15]

O

NE

C

LOCK

[NOMS ‘16]

OpenFlow

Scheduled Bundles

NETCONF

Time Capability

[NOMS ’16, RFC7758]

Multi-phase

updates

[SOSR, ‘15]

O

NE

C

LOCK

to Rule

Them All

[NOMS, ‘16]

Flow Swaps

T

IME

4

[INFOCOM, ‘16]

Data Plane

Timestamping

(56)

(57)

References

[1] T. Mizrahi, Y. Moses, "Software Defined Networks: It's About Time", IEEE INFOCOM, 2016.

[2] T. Mizrahi, Y. Moses, "OneClock to Rule Them All: Using Time in Networked Applications", IEEE/IFIP Network Operations and Management Symposium (NOMS), 2016. [3] T. Mizrahi, E. Saat, Y. Moses, "Timed Consistent Network Updates", ACM SIGCOMM Symposium on SDN Research (SOSR), 2015.

[4] T. Mizrahi, E. Saat, Y. Moses, "Timed Consistent Network Updates in Software Defined Networks", IEEE/ACM Transactions on Networking (ToN), 2016. [5] T. Mizrahi, O. Rottenstreich, Y. Moses, "TimeFlip: Scheduling Network Updates with Timestamp-based TCAM Ranges", IEEE INFOCOM, 2015.

[6] T. Mizrahi, Y. Moses, "Time-based Updates in Software Defined Networks", workshop on hot topics in software defined networks (HotSDN), 2013. [7] T. Mizrahi, Y. Moses, "On the Necessity of Time-based Updates in SDN", Open Networking Summit (ONS), 2014.

[8] T. Mizrahi, Y. Moses "Using ReversePTP to Distribute Time in Software Defined Networks", International IEEE Symposium on Precision Clock Synchronization for Measurement, Control and Communication, (ISPCS), 2014.

[9] T. Mizrahi, Y. Moses, "ReversePTP: A Software Defined Networking Approach to Clock Synchronization", workshop on hot topics in software defined networks (HotSDN), 2014. [10] T. Mizrahi and Y. Moses, "ReversePTP: A clock synchronization scheme for software defined networks", International Journal of Network Management (IJNM), under minor revision,

2016.

[11] T. Mizrahi, Y. Moses, "Time4: Time for SDN", arXiv preprint arXiv:1505.03421, 2016.

[12] T. Mizrahi, Y. Moses, "The Case for Data Plane Timestamping in SDN", IEEE INFOCOM Workshop on Software-Driven Flexible and Agile Networking (SWFAN), 2016. [13] T. Mizrahi, Y. Moses, "Time Capability in NETCONF", RFC 7758, 2016.

[14] T. Mizrahi, Y. Moses, "Serving Time in the Cloud: Why Time-as-a-Service?", IEEE INFOCOM, 2016. [15] T. Mizrahi, Y. Moses, "On the Behavior of Network Delay in the Cloud", IEEE INFOCOM, 2016. [16] Open Networking Foundation, OpenFlow switch specification, Version 1.5.0, 2015.

https://mailarchive.ietf.org/arch/attach/tictoc/pdfsY1ADO.pdf

"Software Defined Networks: It's About Time

"TimeFlip: Scheduling Network Updates with Timestamp-based TCAM Ranges

Timed Consistent Network Updates".

"OneClock to Rule Them All: Using Time in Networked Applications

Using ReversePTP to Distribute Time in Software Defined Networks

"The Case for Data Plane Timestamping in SDN

Timed Consistent Network Updates in Software Defined Networks

"Time4: Time for SDN

"ReversePTP: A Software Defined Networking Approach to Clock Synchronization

"Time-based Updates in Software Defined Networks

"On the Necessity of Time-based Updates in SDN

"Serving Time in the Cloud: Why Time-as-a-Service?

"On the Behavior of Network Delay in the Cloud

Time Capability in NETCONF

, OpenFlow switch specification

, OpenFlow extensions 1.3.x package 2