The Global Controller

After the initial bootstrap, the central manager signals the central controller to start the workflow manager thread. The global controller (Fig. 5.4) will launch one workflow manager for each zone. Initially, each workflow manager will try to launch one workflow instance for that workflow. The steps in launching/removing a workflow instance thread include the following:

Global Controller <<Main Thread>> Datacenter virtual compute/storage Controller Thread Intra-datacenter virtual network controller Thread Inter-datacenter virtual Network controller Thread Workflow Manager Thread Global Manager Workflow Instance

#1 Thread Workflow Instance #N Thread

Global Controller

Figure 5.4: Schematic Representation of the Global Controller

• Query for proxy port (pPort) resources: The ingress and egress to every work- flow instance has to be a proxy port or a pPort. The details of the pPort design will be discussed in Chap. 6. For now, let us look at the high-level schematic representation of the pPort as shown in Fig. 5.5. The pPort sits between the AppFabric Gateway Node (AGN) on the external side and AppFabric-aware services composing the App- Fabric service workflow on the internal side. The external side represents external users and third party services that are assumed to be AppFabric-unaware and thus need the AGN to connect them to the platform. The internal side represents the App- Fabric Service Workflow (ASW) comprising of a set of AppFabric-aware services. The pPort may serve as the ingress/egress for multiple workflow instances, as specified in the configuration. The constraint in getting resources for the pPort for the workflow ingress/egress is that it needs to be launched at a datacenter site marked as EDGE (Fig. 5.3). The global controller sends the resource query to each of the local data- center/network controllers of EDGE datacenters. On receiving the replies from the datacenter controllers, the workflow manager selects one of them (potentially from a list of multiple datacenters) based on certain policies that may, for example require maximizing the extent of distribution of the proxies across the available edge datacen- ters. It then signals the selected datacenter to start a pPort and the associated AGN node.

pPort External Internal AppFabric- unaware Service #1 AGN #1 AGN #N AppFabric- unaware Service #N …

AppFabric Service Workflow #1

AppFabric Service Workflow #K

…

Figure 5.5: Schematic Representation of the pPort

• Launch a workflow instance thread: Once the pPort and the associated AGN node has been started, the central controller launches the first workflow instance thread. It then queries the datacenter application controllers, datacenter network controllers and inter-datacenter network controllers (Fig. 5.4) for resources to launch the workflow instance. Note that we have not yet introduced the concept of separate controllers for the application and the network. We will discuss it in the next section. For now, it is sufficient to understand that our controllers are logically separated into two sets of functions- one for directly controlling the OpenADN ports in the data plane (pPort, sPort, and tPort) and the other for managing virtual network resources (virtual switch, virtual WAN links) and virtual network services provided by a cloud or network provider. Once the workflow manager finds the resources to launch the workflow instance, it delegates the responsibility of actually launching, managing and controlling the workflow instance to the workflow instance thread.

• Dynamically scale-up or scale-down: The workflow manager constantly queries each active workflow instance for their load information. If it finds that the workflows are overloaded (actual algorithm will be discussed in our detailed design section), it will start a new workflow instance thread. Conversely, if it finds that the average load across all the workflows is less than a particular lower threshold, it starts de- commissioning the active workflows one at a time (again, the actual algorithm will be discussed in our detailed design section) till the average per-workflow load goes above the threshold.

• Requesting additional resources from the global manager: Whenever, any of the resource request queries fail, the controller requests the central manager to

allocate additional resources. On receiving such a request, the manager either requests additional resource allocations from the currently active resource providers, or chooses a new resource provider from its database and requests resources from it.

Apart from these, each workflow instance thread in the central controller performs the fol- lowing functions:

• Start workflow services: The workflow instance thread launches all the services in the workflow and their associated ports (sPort or tPort) in parallel.

• Configure Layer 4.5 and Layer 3.5 tunnels: After launching the services and their associated ports, it configures the Layer 4.5 and Layer 3.5 tunnels across them. • Configure application-level routing and packet routing in the ports: After

setting up the tunnels, it configures application-level routing in the sports and packet routing in the tPorts. An application-level router consists of a message classifier (over the message content and context) and a forwarding rule while a packet router classifies packets based on the meta-tag carried either in the IP OPTIONS field (for packets traveling between an sport and tPort in the same host) or in the Layer 3.5 tunnel header (for packets traveling between tPorts in separate hosts).

• Attach workflow instance to proxy: After successfully launching the workflow, it attaches the workflow to an ingress/egress proxy service (connected to a pPort). Note that the workflow instance thread is not responsible for launching/managing the proxy and is instead managed by the workflow manager. This is because the proxy is a shared resource that is shared among multiple workflow instances.

• Make the workflow globally accessible: After successfully attaching an ingress/egress proxy to the workflow instance, it registers the workflow instance with the name-server (currently we have our own name-server implementation but DNS can also be used).

The entry is a mapping of the form _{hApplication Name ! IP address of proxyi.}

As you may have noted, all workflow instances connected to the same proxy has the same entry. This is intended. The name server implements a weighted round robin scheme and each duplicate mapping entry is considered a separate entry. For exam- ple, a proxy hosting n workflow instances will have n duplicate mappings n the name

server. Suppose, there is another proxy hosting m workflow instances. Now applying simple round robin, the first proxy will receive n/(n+m) fraction of the traffic while the second proxy will receive (m/n+m) fraction of the traffic; thus automatically the sim- ple round-robin scheme actually implements a weighted round-robin. Load balancing across the workflow instances within the same proxy is done by the pPort.