Switch Based Load Distribution Design

We propose two designs for the load distribution in our switch: Round Robin and Load Aware Load Balancing. Both designs enable the switch to determine which controller each packet in is sent to in a distributed control plane. Figure 4.2 illus- trates the concept. As traffic flows through the network, the flow requests generated are distributed to specific controllers determined by the switch rather than forward- ing all flow requests to all controllers as the native OpenFlow mandates (replicating each flow request across all connected controllers).

(a) Native OpenFlow Flow Request Replication

(b) Flow Request Distribution

Round Robin: In this system, each switch connects to multiple controllers in EQUAL mode and each controller is assigned a unique ID. When the first packet of a new flow arrives at the switch, instead of sending the flow requests to all controllers, the switch selects one controller to send the request to. Each of the connected con- trollers is selected in turn in a round robin fashion, ensuring that each controller receives 1/n flow requests, where n is the number of controllers the switch is con- nected to. Each controller in this system maintains its own view of the network and as stated before, we rely on the controller communication system implemented in the distributed control plane to ensure they have a consistent view of the network and relevant policies via updates.

We implement the algorithm outlined in Algorithm 1 in Open vSwitch. It assigns each new flow request to a different controller, incrementing the controller ID (to select a new controller at the next request) after each flow request.

Algorithm 1: Round Robin Load Balancer Algorithm for Packet in Dis- tribution

currentControllerID = 1;

foreach flowrequest (f ) do foreach Controller (c) do

if c.ID == currentControllerID then

c.send(f);

else

continue;

currentControllerID++;

if currentControllerID >TotalControllers then

currentControllerID = 1;

else

continue;

A Round Robin algorithm system is ideal for ensuring the load on the controllers are perfectly balanced since it ensures that no controller has more than one addi- tional flow request to deal with than any other controller in the pool at any point in time. A drawback of this approach is that it is simplistic and assumes a perfect network. The network design here assumes homogeneity among the controller pool.

The assumption is that all the controllers are the same, their underlying hardware have the same CPU and memory resources and they are all equidistant from the switch in question. Therefore, each controller takes exactly x time to respond to any flow request. These assumptions would mean that the controllers are truly equal and rotating the flow requests in a round robin manner keeps them all in sync with each other with regards to their load. In reality, if any of those factors are different, it influences the rate at which the controller responds to flow requests. If the controller responses fall out of sync with each other, eventually some controllers become overloaded while others remain well within their capacity. Thus, this system of blindly rotating the controllers the requests are sent to may not work as well as intended in for all architectures. Nevertheless, as we show later, in multicontroller systems which have equal resources, this solution works well to ensure controllers are not overwhelmed.

Load Aware Load Balancer: We therefore propose a second design that may be better suited to the practical drawbacks of an imperfect network. Assuming the controller, the underlying CPU resources and the latency may not all be perfectly homogeneous, we propose a Load Balancer which selects a controller based on its perceived current load.

In similar fashion to the Round Robin system, each switch is connected to mul- tiple controllers in EQUAL mode. In this design, the Load Balancer module of the switch keeps track of both the number of unanswered flow requests for each con- troller and the length of the queue of requests to be sent to the controller.

When the first packet of a flow arrives at the switch, the Load Balancing module reviews the number of outstanding flow requests each controller has and selects the one with the least number of unanswered requests. It then checks the status of that controller’s queue. If a particular controller’s queue is full, the switch drops the next request to the controller. Therefore, if the selected controller’s queue is full, the module selects the next lightest controller (controller with fewest outstanding requests) which has space in its queue. To reduce the likelihood of over-dependence on a single controller, if several controllers have the same (least) number of out- standing requests, it selects randomly one of the controllers from this set. This aims to avoid a single controller being selected more often simply because it is first in

the queue. The module then updates the number of outstanding requests associated with that controller, adding one to the value. When the switch receives a FLOW ADD command from a controller (which adds a flow rule to the switch table in response to a flow request), it decrements the number of outstanding flow requests associated with the controller before installing the rule. In this way, the switch attempts to infer and monitor the load of the controller without adding extra com- munication with the controller. If the controller has a high number of unanswered requests, the switch assumes that its load is higher than one with few unanswered requests. This may be for any number of reasons including higher latency or slower processing capacity. By considering the queue status as well, the algorithm gives the flow request the best chance of being delivered and responded to in a timely manner.

Algorithm 2 outlines the Load Aware load distribution process. Part (a) selects the controller based on the least number of outstanding requests and the queue size and increments the number of outstanding requests. Part (b) decrements the number of outstanding requests upon receiving a FLOW ADD command from a controller.

Algorithm 2:Load Aware Load Balancer Algorithm for Packet in Distri- bution

(a)

foreach FlowRequest(f ) do

least requests = Integer.MAX; Primary controller choice set = null; Secondary controller choice set = null;

foreach Controller (c) do

if (c.out reqs <least requests) && (c.queue != full) then

Primary controller choice set.clear(); least requests = c.out reqs;

Primary controller choice set.add(c);

else if (c.out reqs == least requests) && (c.queue != full) then

Primary controller choice set.add(c);

else if (c.out reqs <least requests) then

Secondary controller choice set.clear(); Secondary controller choice set.add(c);

else if (c.out reqs == least requests) then

Secondary controller choice set.add(c);

if (Primary controller choice set != null) then

controller = Primary controller choice set.random(); controller.send(f);

if (f.sent == true) then

controller.out reqs++;

else

continue;

else

controller = Secondary controller choice set.random(); controller.send(f);

if f.sent == true then

controller.out reqs++;

else

(b)

if c.receive(flow mod) then

c.out reqs–;

else

continue;

4.3.4

Summary

We propose here two designs for load distribution by the SDN switch which en- ables the switch to partition the control plane load it generates among the available controllers. By introducing distribution mechanisms in the switch, we increase its intelligence with the aim of avoiding the issues identified in previous control plane load balancing systems while making the best use of the control plane resources available. The evaluation of these modules in the next chapter aims to show that enabling the switch to autonomously distribute its load among multiple controllers allows for better performance in the network.