Multiple Controllers

The switch may establish communication with a single controller, or may establish communication with multiple controllers. Having multiple controllers improves reliability, as the switch can continue to operate in OpenFlow mode if one controller or controller connection fails. The hand-over between controllers is entirely managed by the controllers themselves, which enables fast recovery from failure and also controller load balancing. The controllers coordinate the management of the switch amongst themselves via mechanisms outside the scope of the present specification, and the goal of the multiple controller functionality is only to help synchronise controller handoffs performed by the controllers. The multiple controller functionality only addresses controller fail-over and load balancing, and doesn’t address virtualisation which can be done outside the OpenFlow protocol.

When OpenFlow operation is initiated, the switch must connect to all controllers it is configured with, and try to maintain connectivity with all of them concurrently. Many controllers may send controller- to-switch commands to the switch, the reply or error messages related to those commands must only be sent on the controller connection associated with that command. Asynchronous messages may need to

be sent to multiple controllers, the message is duplicated for each eligible OpenFlow channel and each message sent when the respective controller connection allows it.

The default role of a controller is OFPCR_ROLE_EQUAL. In this role, the controller has full access to the switch and is equal to other controllers in the same role. By default, the controller receives all the switch asynchronous messages (such as packet-in, flow-removed). The controller can send controller-to-switch commands to modify the state of the switch. The switch does not do any arbitration or resource sharing between controllers.

A controller can request its role to be changed to OFPCR_ROLE_SLAVE. In this role, the controller has read-only access to the switch. By default, the controller does not receive switch asynchronous messages, apart from Port-status messages. The controller is denied the ability to execute all controller-to-switch commands that send packets or modify the state of the switch. For example, OFPT_PACKET_OUT, OFPT_FLOW_MOD, OFPT_GROUP_MOD, OFPT_PORT_MOD, OFPT_TABLE_MOD requests, and OFPMP_TABLE_FEATURES multiparts requests with a non-empty body must be rejected. If the controller sends one of those commands, the switch must reply with an OFPT_ERROR message with a type field of OFPET_BAD_REQUEST, a code field of OFPBRC_IS_SLAVE. Other controller-to-switch messages, such as OFPT_ROLE_REQUEST, OFPT_SET_ASYNC and OFPT_MULTIPART_REQUEST that only query data, should be processed normally.

A controller can request its role to be changed to OFPCR_ROLE_MASTER. This role is similar to OFPCR_ROLE_EQUAL and has full access to the switch, the difference is that the switch ensures it is the only controller in this role. When a controller changes its role to OFPCR_ROLE_MASTER, the switch changes all other controllers with the role OFPCR_ROLE_MASTER to have the role OFPCR_ROLE_SLAVE, but does not affect controllers with role OFPCR_ROLE_EQUAL. When the switch performs such role changes, if a controller role is changed from OFPCR_ROLE_MASTER to OFPCR_ROLE_SLAVE, the switch must generate a controller role status event for this controller informing it of its new state (in many cases that controller is no longer reachable, and switch may not be able to transmit that event).

Each controller may send a OFPT_ROLE_REQUEST message to communicate its role to the switch (see 7.3.8), and the switch must remember the role of each controller connection. A controller may change role at any time, provided the generation_id in the message is current (see below).

The role request message offers a lightweight mechanism to help the controller master election process, the controllers configure their role and usually still need to coordinate among themselve. The switch can not change the state of a controller on its own, controller state is always changed as a result of a request from one of the controllers. Any Slave controller or Equal controller can elect itself Master. A switch may be simultaneously connected to multiple controllers in Equal state, multiple controllers in Slave state, and at most one controller in Master state. The controller in Master state (if any) and all the controllers in Equal state can fully change the switch state, there is no mechanism to enforce partitioning of the switch between those controllers. If the controller in Master role need to be the only controller able to make changes on the switch, then no controllers should be in Equal state and all other controllers should be in Slave state.

A controller can also control which types of switch asynchronous messages are sent over its OpenFlow channel, and change the defaults described above. This is done via a Asynchronous Configuration message (see 6.1.1), listing all reasons for each message type that need to be enabled or filtered out (see 7.3.10) for the specific OpenFlow channel. Using this feature, different controllers can receive different notifications, a controller in master mode can selectively disable notifications it does not care about, and a controller in slave mode can enable notifications it wants to monitor.

To detect out-of-order messages during a master/slave transition, the OFPT_ROLE_REQUEST message contains a 64-bit sequence number field, generation_id, that identifies a given mastership view. As a part of the master election mechanism, controllers (or a third party on their behalf) coordinate the assignment of generation_id. generation_id is a monotonically increasing counter: a new (larger) generation_id is assigned each time the mastership view changes, e.g. when a new master is designated. generation_id can wrap around.

On receiving a OFPT_ROLE_REQUEST with role equal to OFPCR_ROLE_MASTER or OFPCR_ROLE_SLAVE the switch must compare the generation_id in the message against the largest generation id seen so far. A message with a generation_id smaller than a previously seen generation id must be considered stale and discarded. The switch must respond to stale messages with an error message with type OFPET_ROLE_REQUEST_FAILED and code OFPRRFC_STALE.

The following pseudo-code describes the behavior of the switch in dealing with generation_id. On switch startup:

generation_is_defined = false;

On receiving OFPT_ROLE_REQUEST with role equal to OFPCR_ROLE_MASTER or OFPCR_ROLE_SLAVE and with a given generation_id, say GEN_ID_X:

if (generation_is_defined AND

distance(GEN_ID_X, cached_generation_id) < 0) { <discard OFPT_ROLE_REQUEST message>;

<send an error message with code OFPRRFC_STALE>; } else {

cached_generation_id = GEN_ID_X; generation_is_defined = true; <process the message normally>; }

where distance() is the Wrapping Sequence Number Distance operator defined as following: distance(a, b) := (int64_t)(a - b)

I.e. distance() is the unsigned difference between the sequence numbers, interpreted as a two’s com- plement signed value. This results in a positive distance if a is greater than b (in a circular sense) but less than “half the sequence number space” away from it. It results in a negative distance otherwise (a < b).

The switch must ignore generation_id if the role in the OFPT_ROLE_REQUEST is OFPCR_ROLE_EQUAL, as generation_id is specifically intended for the disambiguation of race condition in master/slave transition.