Advanced Computer Networks. Layer-7-Switching and Loadbalancing

(1)

Advanced Computer Networks

263-3501-00

Layer-7-Switching and Loadbalancing

Patrick Stuedi, Qin Yin and Timothy Roscoe

Spring Semester 2015

© Oriana Riva, Department of Computer Science | ETH Zürich

(2)

Outline

• Last time

– Virtual machine networking – Para-virtualization

– SR-IOV – IOMMU

• Today

– Load balancing – TCP Splicing

– Distributed load balancing

2

(3)

Challenge: accessing services

• Datacenters are designed to be scalable

– Datacenters are replicated

– Each has lots of machines

– Service span (and share) data centers

So:

• What address does, e.g. www.search.ch resolve to?

• What entity does this address refer to?

• What does this entity do?

3

(4)

Requirements

• “Close by” datacenter

• Load balance across machines in a center

• Target machines where the user’s state is kept

• Accessed using TCP (HTTP, SSL, …)

4

(5)

Option 1: IP Anycast

• One IP address refers to multiple destinations – BGP advertizes multiple destinations

– Packets end up at “nearest” destination to source.

Problems:

IP layer  only reliable for stateless protocols (UDP) All packets of a TCP flow must go to the same machine

Service location pushed into BGP  couples routing with end-system provision

5

(6)

Option 1: IP Anycast

• One IP address refers to multiple destinations – BGP advertizes multiple destinations

– Packets end up at “nearest” destination to source.

• Problems:

 IP layer  only reliable for stateless protocols (UDP)

 Service location pushed into BGP  couples routing with end-system provision

• Used for DNS root server location

6

(7)

Requirements

• “Close by” datacenter

• Load balance across machines in a center

• Target machines where the user’s state is kept

• Accessed using TCP (HTTP, SSL, …)

All packets of a TCP flow must go to the same machine

7

(8)

Recall DNS lookup

8

(9)

Option 2: DNS

• Insight: who says the answer is always the same?

• Idea: “smart” DNS server authoritative for service

Query for, e.g.. www.google.com or www.bing.com returns a different “A” record depending on:

– Source address of browser machine – Current state of the service

• Load

• Failures

– A random number

9

(10)

DNS tricks

• One-level of indirection

– Single DNS server returns different Arecs

• Additional level of indirection

– First service resolver returns CNAME

– Regional service resolver can be more specific

• Used for finding the nearest datacenter for a service

10

(11)

Using CNAMEs

11

timeouts

(12)

DNS does not solve the problem

Need IP address for every instance of the service

 100,000 machines

 100,000 globally routable IP addresses – expensive!

 Machine fails

 need to update DNS state

 DNS state changes rapidly

 short TTL on queries

 even higher load on DNS servers

 Slow to react to “hot spots” or other load skews

 Selection of machine can only be made based on address of client's primary resolver

don't know which client this is

12

(13)

Next step: use 1 IP address

• Use Network Address Translation

• Hash source addresses to server machines

(14)

TCP three-way handshake

(15)

TCP three-way handshake

(16)

Stateless hashing

Hash(Source IP)

• Completely static

– No dynamic load balancing Hash(Source IP, Source TCP port)

• Better, but still static

– Limited to 64k destinations per client machine

• Known as a “Layer-4 load balancer”

16

(17)

Stateless hashing

Hash(Source IP)

• Completely static

– No dynamic load balancing Hash(Source IP, Source TCP port)

• Better, but still static

– Limited to 64k destinations per client machine

• Known as a “Layer-4 load balancer”

Basic problem: nothing else is known by the end of the handshake!

17

(18)

Why is static hashing bad?

• Machine failure/upgrade/provisioning

– Can’t update hash function efficiently in switch

• Load balancing

– Can’t avoid a heavily-loaded machine

• Lack of Locality

– Resource being accessed

– Client accessing the resource

18

(19)

19

What else might we

want to hash on?

(20)

HTTP Host: header

• Introduced in HTTP/1.1 – mandatory

• Hosting providers need to switch based on virtual host, not physical host

– Different services have different virtual host – Avoids replicating all service state everywhere

20

(21)

Switching on URL

• Locality:

– Allows state to be partitioned across machines

• Isolation:

– Rare, computationally intensive URLs can be sequestered

– Sensitive data can be kept on more expensive, auditted machines

21

(22)

Hashing on cookies

• Enables partioning of servers by – User state

– Session state

• Critical for scaling online services to billions of users – No need to share state

– No need to synchronize state

22

(23)

How to do it?

• Problem:

– Don’t know the hash key until after the HTTP request

– Typically the first segment after the 3WS

• Solution:

– Don’t establish connection to server until client has sent HTTP request

23

(24)

Late-binding of TCP connection

24

time

Client Server

Port = 3620

Switch

(25)

Late-binding of TCP connection

25

time

Client Server

Port = 3620

Switch

TCP connection setup + HTTP GET

(26)

Late-binding of TCP connection

26

time

Client Server

Port = 3620

Switch

(27)

Late-binding of TCP connection

27

time

Client Server

Port = 3620

Switch

HTTP response HTTP response

(acks not shown)

(28)

Late-binding of TCP connection

28

time

Client Server

Port = 3620

Switch

HTTP response HTTP response HTTP response

HTTP response

(acks not shown)

(29)

Late-binding: Naïve

implementation (SOCKS protocol)

29

(30)

Late-binding: Naïve

implementation (SOCKS protocol)

30

Inefficient: switch needs to copy data between the connections!

(31)

TCP Splicing

• Proposed around 1997 by Maltz & Bhagwat at IBM

• Key idea:

– Take two established TCP connections and splice them – Transfer segments unmodified between them

– Remap port numbers and segment numbers on the fly

• Advantages:

– Very simple calculation per packet

– Not much state to maintain per spliced connection – No segmentation/reassembly

– No buffering/copying

31

(32)

Splicing pseudocode

(from Maltz & Bhagwat)

32

(33)

Splicing in pseudo code

33

queue packets received from server

splice connections, but allow for final

'n' bytes to be transmitted to the client before splicing

'n' bytes message signaling the completion

of the splicing operation

(34)

Splicing in pseudo code

34

(35)

What state is needed?

For each packet, need to do the following:

• IP header operations:

– Rewrite source and destination IP addresses – Update IP header checksum

• TCP header operations:

– Rewrite source and destination port numbers – Apply fixed offset to sequence number

– Apply fixed offset to acknowledgement number – Update TCP header checksum

35

calculated from existing connection state when

splice occurs

(36)

It’s easy to do in hardware

• A10 AX Application Delivery Controller

• Advanced layer 4 / layer 7 server load balancing

• HTTP Proxy

• Layer 7 URL and URL hash switching

• Comprehensive Layer 7 application persistence support

• Load balancing methods:

– Round Robin, Least Connections, Weighted Round Robin, Weighted Least Connections, Fastest Response

• Aggregated throughput: up to 115 Gbps

36

(37)

Problems of single-box load

balancing

• Expensive!

• Scale-up

– Buy bigger (more expensive) load balancer when reaching capacity

37

(38)

Ananta: Load balancing in

Windows Azure

• Windows Azure: Microsoft's cloud computing platform

– Compute, Storage, Databases, etc. in the cloud

• Ananta: Distributed, scalable load balancing running on hosts in a datacenter

– Lower cost

– Scale on demand

38

(39)

Background: Windows Azure

load balancing

39

• Clients connect to service using a virtual IP (VIP)

• Load balancer (LB) load balances traffic to specific server machines using a direct IP (DIP)

(40)

Background: Windows Azure

load balancing (2)

40

• Load balancer is also used when two services communicate within the same data center

(41)

Ananta: Inbound traffic

41

Ananta Manager

(42)

Ananta: Inbound traffic

42

1 2 3

Spread packet to MUX using ECMP Lookup the VIP-to-DIP mapping Tunnel packet to DIP

4 5 6 7 8

De-capsulate and forward to DIP Encapsulate response

Forward to router (bypass MUX)

(43)

Summary

• IP Anycast: select a DNS root server

• Dynamic DNS: locate nearby data centers

• Layer-3-switching: balance connections across machines

• TCP splicing: seamlessly join two connections

• Layer-7-switching: use splicing to late-bind servers to HTTP connects

• Ananta: distributed load balancing

43

(44)

References

• “Host Anycasting Service”, C. Partridge, T. Mendez, W.

Milliken, Internet RFC 1546, November 1993.

• “TCP Splicing for Application Layer Proxy

Performance”, David A. Maltz, and Pravin Bhagwat.

IBM Research Report 21139 (Computer

Science/Mathematics), IBM Research Division, 1998.

• Ananta: Cloud Scale Load Balancing, SigComm 2013

44