From Change Events to Model Updates

From a high-level perspective, the Trans component translates from a change event to a model update in the form of a graph delta, as illustrated in Figure 7.2. The translation has to differentiate between three kind of change events. First, a new object appeared that may result in new nodes and edges in the graph model. Second, an object was removed and the corresponding nodes and edges in the model have to be removed, too. Finally, an object has been modified, i.e., attributes of that object have changed. This may result in attribute changes of nodes in the model too, but it can also leads to the creation or deletion of nodes and/or edges in the model. This categorization aligns well with the events produced by the VMware probe since they contain an attribute indicating the kind of change.

A translation is typically bound to a specific probe and its produced event format, i.e., a VMware translation knows how to translate VMware change events. Therefore, we focus in the following on describing the event translation design with the concrete example of translating VMware change events. The translation needs to handle the three different change events, but also has to deal with the ordering of events due to their dependencies, and with incomplete new objects. Therefore, the output of the translation is either a graph delta, dependency requirements, or a notification that the translation encountered an incomplete, ignored, or unsupported object. If an change event consists of multiple object updates, we merge the produced graph deltas to form a single graph delta for that change event.

7.3.2.1 Translation of an Object Update

We explain and propose a set of algorithms for the successful translation of an Object Update into a graph delta. The handling of a failed translation due to cases such as incomplete objects or dependency ordering will be discussed in Section 7.3.2.3 and Section 7.3.2.4 respectively.

We define a graph delta as∆ = (V+, V−, E+, E−, M), where we differentiate between creator nodes and edges (V+, E+), eraser nodes and edges (V−, E−), and a set M of node attribute modifier in the form of (node, at tribute, v alue). Creators lead to new elements in the graph, erasers remove existing elements

from the graph, and node attribute modifiers change attributes of existing nodes to new values. Enter Object: Creating New Nodes and Edges

We obtain an Enter Object Update for a new object that has been created in the inventory as well as for all the existing objects in the inventory during the initial probe connection. The goal of the translation component is to produce new nodes and edges for the model based on the update.

The fundamental idea is to employ a recursive algorithm that starts at a newly created object and traverses through all its connected neighbor objects. For each object, the technology-specific parts of the translation creates a corresponding model node and populates its attributes with values of the object. It further establishes relations to other created nodes due to the recursion. The output of this algorithm is a set of newly created model nodes and edges, where the edges not only connect to new nodes, but also to existing nodes in the model.

As part of the VMware translation, we distinguish between two types of objects: Managed Entity (ME) and Data. The MEs are elements in the virtualized infrastructure inventory, such as virtual machines (VM) or hosts, and have a reference (Managed Object Reference or MOR). On the other hand, each managed entity may have configuration attributes represented as Data objects, such as virtual devices of a VM. We represent both object types as nodes in the model.

Algorithm 2 shows the recursive translation for VMware objects. The distinction between the two object types is realized through two functions: CreateEntity for Managed Entities (Alg. 2), and CreateData for Data objects (Alg. 3). The algorithm is not shown in its entirety, but parts have been selected to illustrate its concepts. The recursive function CreateEntity takes the Managed Entity that needs to be translated into a model node, as well as the already created nodes and edges of this recursive translation run. If the object has already been translated, i.e., it is in the Cache, we simply return the cached model node. Otherwise, depending on the object’s type, e.g., a virtual machine, the translation creates a new model node with the corresponding type, and extracts and translates the necessary attributes from the inventory object into the model node. An attribute can be another ME, which requires a recursive call and merging of its result, or it is a data object.

For a data object, the function CreateData will return the new node representing the data object. Important for the recursive translation is that we consider data objects as terminators for the recursion, because they do not link to other MEs which would lead to another recursive call. They are leaf nodes in the hierarchy graph, i.e., no outgoing edges.

Algorithm 2: Recursive Construction of Creator Node and Edge Sets. Depends on Algorithm 3 for Data Objects.

Data: New Object o

Result: Creator Nodes V+and Edges E+. CreateEntity(o, V+, E+)begin

Input: Object o, Creator Nodes V+and Edges E+

Output: Extended Creator Nodes and Edges, Created Node if o in Cache then

return V+, E+, C ache(o)

switch o.type do case HostSystem do

host← Host(name = o.name, . . .)

V+← V+∪ host foreach vm in o.vms do V+, E+, v m0← CreateEntity (vm, V+, E+) E+← E+∪ (host, v m0) return V+, E+, host case VirtualMachine do v m← V M(name = o.name, . . .) V+← V+∪ v m

foreach dev in o.devs do

V+, E+, d← CreateData (dev, V+, E+)

E+← E+∪ (v m, d)

E+← E+∪ (C ache(o.host), v m)

return V+, E+, v m

case Network do

/* Ignored, because translated in host’s network configuration. */

return V+, E+, N one

case . . . do otherwise do

/* Unsupported object type */

return V+, E+, N one

/* Calling the recursive function */

return CreateEntity (o,;, ;)

The translation of object updates has to handle two corner cases: Incomplete objects, where the attributes of an object have not been populated fully yet, and the ordering of object updates within the same update set. We will describe the handling of these cases in Section 7.3.2.3 and Section 7.3.2.4, respectively.

Leave Object: Deleting Nodes and Edges

For each object that has been removed from the inventory, we obtain an Leave Object Update. The update contains the MOR of the removed object, and we lookup the corresponding model node identifier and obtain the node by querying the model component. Since in our model a managed entity might have resulted in the creation of multiple nodes, we have to perform a recursive deletion of the dependent nodes of the removed node.

The recursive deletion works as the following. First, given an object that was removed from the inventory, we lookup the corresponding node in the model, and add the node to the eraser node set. For all the deleted node’s neighbors, we place the connecting edges in the eraser edge set. Further, we continue the recursive deletion at the neighbor node if i) the neighbor’s type is a child type in the hierarchy (cf. Section 7.2.1); and ii) the neighbor node is not a managed entity.

Algorithm 3: Construction of Creator Node and Edge Sets for Data Objects. CreateData(d, V+, E+)begin

switch d.type do

case VirtualEthernetCard do

vnic, vport← VNic(. . . ), VPort(. . . ) pg← Cache(d.backing.portgroup)

V+← V+∪ {v nic, v por t}

E+← E+∪ {(v nic, v por t), (pg, v por t)}

return V+, E+, v nic

case . . . do otherwise do

/* Unsupported data type. */

return None

Modify Object: Creating an Entire Graph Delta

Finally, we consider the case that an object has been modified. A Modify Object Update consists of a Property Change and the modified object reference. This property change indicates the type of change, the attribute that has changed, and potentially a new value.

Our algorithm consumes such a property change and produces a graph delta that consists of creator/eraser nodes and edges, as well as a set of attribute modifier in the form of(ob ject, at tribute, v alue). The algorithm has a similar structure as the algorithms for creating or removing objects, and in fact builds upon them. For each object type, we further differentiate between the changed attribute, as well as the operation performed on that attribute. For attribute assignments, we construct attribute modifiers that change the corresponding node’s attribute. For added managed entities or data objects, we rely on the algorithm that handles Enter objects. Similarly, we construct erasers for deleted objects based on the Leave object algorithm.

An example to illustrate a modified object is the creation of a new virtual device, such as a virtual Ethernet adapter, for a VM. In this case, we have to translate the new virtual device into a new model node, and connect it to the existing VM node with an edge. Such an event produces the following creator nodes and edges: V+= {v nic, v por t} and E+= {(v m, v nic), (v nic, v por t), (pg, v por t)}, where v m corresponds to the existing VM, v nic and v por t are created, and the virtual port is connected to the port group p g.

7.3.2.2 Applying the Model Update

Given a graph delta as produced by our set of algorithms based on a change event, updating the graph model is expressed as updating the node and edge sets of a given graph G= (V, E): V0= (V \ V−) ∪ V+ and E0= (E \ E−) ∪ E+. The updated model graph is G0 = (V0, E0). For each node attribute modifier (node, at tribute, v alue) in M we change the attribute of the node in V0_{with the new value.}

The graph delta has to adhere to the following invariants, otherwise the consistency of the model is not ensured. The erasers can only operate on the existing elements in G, i.e., V−⊆ V and E−⊆ E. Elements cannot be created and deleted in the same graph delta. Further, we cannot have two node modifiers (n, a, v ) and (n, a, v0_{) where v 6= v}0 _{in the same set M , since we have a conflicting modification of the}

node n’s attribute a. During our evaluation we have not encountered violations of these invariants.

7.3.2.3 Postponing Incomplete Objects

Resolving further information of an object during the translation may fail when not all relevant attributes have been set yet. For example in VMware, when a VM is created its configuration is only later fully populated. In that case, we are dealing with an incomplete object. We maintain a set of incomplete objects and monitor updates for these objects. If an incomplete object receives an update, we try to

handle it as a new object rather than a modified one. If the translation succeeds, i.e., the object was complete and could be translated, we remove the object from the incomplete set. Otherwise, it remains in the incomplete object set. In practice and during our evaluation, incomplete objects always received modify events when further attributes were populated, usually within a sub-second time span. We employ periodic translation attempts for incomplete objects, in case they receive no further modify events. If an object remains incomplete until a time-out t_{incompl et e}is reached, an alarm is raised that indicates the identifier of the object and when it was first observed.

7.3.2.4 Ordering of Updates based on Dependencies

We are dealing with an asynchronous system and we have to take care about the ordering of the change events. However, two aspects of the VMware probe supports the ordering of events. First, the probe connects over TCP which provides packet ordering for us. Second, VMware employs version numbers for the event discovery, which provides an ordering of events between different versions. However, within one UpdateSet, we may encounter a wrong ordering of changes where one change assumes the existence of an entity that is created by another change in the same UpdateSet. We can order the changes by using a dependency graph, i.e., a directed acyclic graph (DAG) where vertices are changes and directed edges describe dependencies such that the source node fulfills the dependency of the target node. This ensures that changes that produce required entities of other changes are processed first.

We construct such a dependency graph in the following way. A successful translation of an event returns a graph delta of new or modified nodes. In the case of an unsuccessful translation due to a missing dependency, i.e., a node was not found in the current model, the translation returns a requirement in the form of a node type and a predicate on its attributes. Further, the translation of an event may return

potential nodes, which become available once other requirements are fulfilled. Based on the translation

attempts, we try to match new or modified nodes with requirements, and introduce a directed dependency edge from the fulfilling event to the requirement. Potential nodes may also satisfy requirements, thereby building up a dependency graph. A topological sort of the dependency graph will yield an evaluation order.