As shown in Fig. 5.2, the management subsystem consists of a single entity called the Light- house global manager. It is the first and the only node in the application that has to be started by the administrator manually. The ASP may either run it on-premise or on a cloud
datacenter. However, given some of the sensitive information that this node has access to including billing information, user authorization/authentication and application policies and configurations, it is preferable to host it on-premise. The global manager exposes the north- bound management interface thorough which the ASP administrators can specify the policies for deploying and delivering the application. The manager is connected to a database server from where it may fetch the virtual machine images of the di↵erent Lighthouse controllers and data plane nodes.
Global Controller Central Manager Thread Global Resource Manager Thread Per-Datacenter Resource Driver Thread Per-Network Provider Resource Driver Thread AAS
Application Service Developers Application Architects Application Deployment Administrators
AAW AAC Northbound Interface
Global Manager
The primary function of the global manager in the application runtime is to bootstrap the application deployment. But before discussing the bootstrap process, we need to explain the concepts of zones and sites (Fig. 5.3). In AppFabric application deployments, a large geographical region is divided into several sub-regions called zones. Each zone has several datacenters, also called sites. For example, as shown in Fig. 5.3 United States may be divided into 3 zones -US-E, US-W and US-S. Each zone independently manages the application deployment over the datacenters in that region. Also each datacenter is classified as ’EDGE,’ ’CORE,’ or ’EDGE/CORE.’ The services in the application are also classified under the same labels. An ’EDGE’ service needs to be close to the user access locations and are widely distributed. Examples of edge services include application gateways, firewalls, and data gathering and near-real time control functions in IoT-like use cases. It needs to be deployed on datacenter marked as either, ’EDGE’, or ’EDGE/CORE.’ A ’CORE’ service is located at a somewhat centralized location relative to all the edge datacenters. it is expected that there will much fewer core data centers compared to the number of edge datacenters, with each core datacenter serving multiple edge locations. Examples of ’CORE’ services includes aggregation services running business intelligence logic over the gathered data in IoT use- cases, centralized database for synchronizing the di↵erent edge services, etc. Note that in many cases the core datacenter could be an enterprise datacenter while the edge datacenters could be cloud datacenters or much more distributed and smaller micro-datacenters operated by ISP networks in their network points-of-presence (POPs). A ’CORE’ service may be deployed either on a ’CORE’ datacenter, or a ’EDGE/CORE’ datacenter.
3/13/14 10:09 PM switch Page 1 of 1 file:///users/subharthi/Desktop/switch.svg 3/13/14 10:09 PM switch Page 1 of 1 file:///users/subharthi/Desktop/switch.svg 3/13/14 10:09 PM switch Page 1 of 1 file:///users/subharthi/Desktop/switch.svg 3/13/14 10:09 PM switch Page 1 of 1 file:///users/subharthi/Desktop/switch.svg 3/13/14 10:09 PM switch Page 1 of 1 file:///users/subharthi/Desktop/switch.svg 3/13/14 10:09 PM switch Page 1 of 1 file:///users/subharthi/Desktop/switch.svg 3/13/14 10:09 PM switch Page 1 of 1 file:///users/subharthi/Desktop/switch.svg Zone US-NE Zone US-S Zone US-W Edge Site Core Site
Figure 5.3: Sites and Zones
Now, let us look at the sequence of tasks performed by the global manager during the application bootstrap process.
• Compile user-specified policy scripts: The global manager accepts the policies for cre- ating the application workflows (AAW) and the distributed application cloud (AAC) from the administrator through its northbound interface, verifies and validates the poli- cies for conflicts and/or mis-configurations and then compiles and loads these policies into a local database.
• Start the global controller and select cloud datacenter sites: After loading the policies, it starts the global controller. Again the global controller may either be on-premise or on a cloud datacenter. Generally the manager will choose a location for the global con- troller such that it has (almost) the same latency from all possible datacenters (cloud as well as enterprise) that are candidates for the application deployment. The global controller in turn starts a workflow controller for each workflow in each zone. We will discuss more about this in our discussion on the global controller design. The global
manager, on the other hand, maintains a global database of di↵erent cloud/enterprise datacenter configurations including provider name, location, security (authorization and authentication) and access policies, billing information, and resource access and lease policies. It chooses (based on some policy) an initial set of datacenters in each zone from this list and acquires a pre-configured minimum number of virtual resources (compute, storage and network) from these datacenters. The initial number of re- sources acquired should be enough to launch at-least one instance of each workflow per zone. However, the ASP administrator may choose to override this default behavior and start multiple workflow instances during the bootstrap. This may be necessary to distribute the application geographically within the same zone. These virtual resources are acquired through the southbound management interface. The manager starts a sep- arate resource driver thread for each datacenter. These resource driver threads acquire the necessary virtual resources by calling the appropriate APIs specific to the datacen- ter management stack (such as OpenStack, Eucalyptus, CloudStack, etc).
• Start datacenter controllers: After selecting the datacenter sites and acquiring the required virtual resources on them, it starts a datacenter controller for each site. • Launch virtual resources per datacenter: Next, it launches the data plane nodes in-
cluding AppFabric virtual machines and virtual switches in each of these selected dat- acenters.
• Setup inter-datacenter links: Just like the global database for cloud datacenters, it also maintains a global database of WAN providers that connect these datacenters. After bootstrapping the sites, it calls the APIs of WAN providers to allocate resources for connecting the di↵erent sites.
After bootstrapping the control plane and launching the resources for the data plane, it signals the global controller to start the workflow manager threads. The global controller starts the workflow managers; one per zone per workflow.
These 6 steps complete the initial bootstrapping function of the global manager. It then steps aside and lets the global controller take over the responsibility of actually launching and managing the application. During the application runtime, the global controller may
call the global manager from time-to-time to allocate more resources. These resources may be allocated from the already active sites or it may choose to launch new sites based on the policy specified by the ASP administrator. In our current implementation of AppFabric, our policy language is quite limited in what it can express. Future versions will incorporate much more richer policy specification languages that will allow the ASP administrator to have better control over the application deployment and delivery. The other role of the global manager during the runtime is to create reports and alarms for the ASP administrator based on the runtime logs exported by the control plane entities. Also, the global manager pro- vides an interface to the ASP administrator to manually change the state of the application deployment (overriding the automatic behavior). For example, the ASP administrator may choose to shutdown certain sites or start new sites in order to handle exceptions or other planned or unplanned situations such as security attacks, system maintenance, etc.