The Local Controller

Each resource site leasing/contributing virtual resources to the AppFabric platform has a local controller. Therefore, whenever the global controller decides to launch a new deployment site (e.g. cloud/enterprise datacenter or micro-datacenters attached to a network POP), it needs to launch a local controller for managing/controlling the resources for that new site. Also, adding a new site to the application deployment may need adding new wide- area-networking resources to ensure the reachability of the new site from the existing sites. In that case, the global controller may need to launch a new inter-datacenter virtual WAN controller to control/manage these new network links if it involves a new network provider that is not already part of the ADN.

Local Controler <<Main Thread>> VM #1 Controller Thread Global Controller VM #N Controller Thread Intra-datacenter virtual network controller Thread …

Lighthouse Datacenter Controller

Figure 5.6: Schematic Representation of the Local Controller

As shown in Fig. 5.6, the design of the local controller is pretty simple. Its primary function is resource management. Each virtual machine that is launched by the platform runs a host controller agent called the cPort (as shown in Fig. 5.7). The cPort manages the AppFabric

platform ports in the hosts. The AppFabric platform ports are the interfaces through which services connect to the distributed AppFabric platform. There are three types of platform ports:

• pPort: The pPort has already been discussed in the previous sub-section.It connects AppFabric-unaware services to the platform.

• sPort: The sPort connects AppFabric-aware message-level services to the platform. • tPort: The tPort connects AppFabric-aware packet-level services to the platform.

cPort pPort #1 Datacenter Controller sPort #1 tPort #1 AppFabric VM AppFabric- unaware Service #1 AGN #1 AppFabric- aware Message-level Service #1 AppFabric- aware Packet-level Service #1

Figure 5.7: Schematic Representation of an AppFabric VM

Each of these ports are programmable. The cPort is the control plane agent in each virtual machine and is responsible for executing control plane instructions in the virtual machine. The instructions may include commands for launching a new service in the virtual machine and programming the corresponding platform port through which the service is attached to the platform; or replying to queries from the control plane regarding service liveness, resource

availability, load information etc.

When a virtual machine is launched by the global manager, it configures the address of the local controller as a parameter in its global environment. The virtual machine is pre- configured to launch the cPort at startup. The cPort reads the address of the local controller from its environment and sends a registration message to the local controller. On receiving this registration message, the local controller launches a new thread dedicated to managing the virtual machine. This way the local controller manages a pool of threads; each dedicated to managing a single virtual machine. The VM thread in the local controller synchronizes information on load, liveness, resource availability and other performance metrics with the host’s cPort. Whenever the local controller receives any query from the global controller, it queries each of its VM threads to respond to the query. Also, it forwards commands for programming the virtual machine from the global controller to the appropriate VM thread which in turn passes it to the hosts cPort.

VMs are represented at the global controller based on their host IDs. Host IDs are flat identifiers. that unlike IP addresses do not change across host mobility. Also, these IDs are the cryptographic hash of a public key; allowing secure communication between the host and the control plane. The local controller keeps the mapping of the host ID to its locator, thus allowing the VM to migrate within the datacenter without the global controller needing to know about it. However, for inter-datacenter VM migrations, the global controller would require to update its Host ID to datacenter ID mapping.

Note that the current architecture is not very useful to mask local failures from the global controller as the global controller is exposed to individual VM identities. In the next version of the Lighthouse architecture, we plan to mask out this VM-level information from the global controller. The local controller will present the global controller with the abstraction of a single very large host and will be responsible for managing this abstraction by aggre- gating the resources over many di↵erent VMs.

Also note that the local controller behaves a bit di↵erently in managing the network re- sources. This is because the platform can directly run its code inside a virtual machine. However, most virtual network resources (virtual switches, virtual routers, and virtual WAN links) will not allow the equivalent of the cPort and ASP-owned services to be launched over them. These virtual resources generally expose a configuration interface through which they may be programmed to some limited extent. The network controller module of the local controllers will directly access these interfaces to program the virtual network in application- specific ways as required by the ASP.

This brings us to the end of the high-level discussion on the architecture of the Lighthouse control and management plane design. The details of the actual implementation of this design will be presented in the next chapter (Chapter. 6). Note that this chapter presents the reference architecture of Lighthouse that eventually needs to be implemented. Our current prototype implementation does not implement the architecture fully but implements the key components and an extensible framework that will allow later releases to extend and add the missing components pretty easily.

Chapter 6

AppFabric Prototype

In this chapter, we will discuss the details of the AppFabric prototype implementation. The AppFabric prototype has been implemented in C and Python; with most of the control plane (Lighthouse) code implemented in Python while the data plane (OpenADN) is implemented in a mix of C and Python. The total size of the codebase is approximately 10,000 lines of code. As we will explain later in this chapter, the prototype has been implemented to allow future extensions to add new resource drivers, algorithms for resource acquisition, management and control, and new policies for deploying and delivering applications. Also, it may be noted that the current prototype does not implement the complete data plane and control plane architectures discussed in the previous chapters. We will explicitly mention what has not been implemented in the relevant portions of this discussion. The current code base may downloaded from:

https://sites.google.com/site/applicationfabric/6-code-download