Changes in the optimization problem and environment have to be detected and assessed in order to be able to react to them appropriately. Depending on the objects affected by changes, different detection methods can be applied. Changes in components and attributes can be captured through active and passive system monitoring, while changes in the optimization definition are known and set externally. In any case, it is necessary to assess the impact a change has on the optimization.
5.2.1
System Monitoring and Profiles
Monitoring can be used to track system inherent dynamics (defined in 5.1) as follows.
Components. When new components are added to or removed from the system, this has a con- siderable impact on the problem instance. A component can be a site, resource, service or deployment, but in fact only the addition and removal of a deployment have an impact. For instance, if a new site and associated resources are added to the system, but no deployment is hosted there yet, then these resources cannot be considered in a solution anyway. Thus, only the status change of a deployment has to be tracked and marked. In the TAG system, this is flagged in the TASK Service Catalog (cf. Section 4.4). For instance, when a deployment is down, its status is changed from “up” to “down” and a flag is set, indicating that a change has occurred. The first optimization process reading this information resets the flag.
For a given point t in time, the system status or system blueprint Sys(t) is defined as the union of the available deployments and links: Sys(t) = D(t) ∪ L(t), where, as defined in Chapter 4, D is the set of deployments and L is the set of links. According to Equations 4.4 and 4.5, the definition of deployments and links include their attributes, as well as the underlying services, resources and sites.
The following operations are defined and feasible on Sys(t):
• Addition of a deployment Di between t and t + ∆t: Sys(t + ∆t) = Sys(t) ∪ Di
• Subtraction of a deployment Di between t and t + ∆t: Sys(t + ∆t) = Sys(t) \ Di
Links are implicitly added and subtracted, as they represent a connection between two deploy- ments. If services, resources or sites are added or removed, this can be mapped to adding or removing deployments attached to the respective component.
Attributes. Monitoring is primarily used to detect changes in attribute values. The monitoring of the TAG system has been described in Chapter 4 (Section 4.4). The latency of detecting a change is defined by the monitoring update intervals. Evidently, there is little chance that a sudden load peak lasting for a few seconds will be detected if the interval of the job monitoring the server load is set to a minute. Additionally, the value for the load is averaged over a time period, resulting in the flattening of peaks. While this better reflects the system status over time, it does not capture with 100% accuracy the system status at time t. There is clearly a trade-off between the accuracy of monitoring information and system load created by the monitoring itself. For the scope of this work, it is safe to assume that the information obtained via the monitoring is up to date and properly describes the system in terms of QoS attributes.
5.2. DETECTING AND ASSESSING CHANGES IN THE ENVIRONMENT 83 Whenever the value of a QoS attribute changes, a particular flag can be set at the appropriate place in the central registry. This flag is unset as soon as the new value is picked up by a new run of the optimization algorithm. This is an easy and straightforward way to mark changes without having to compare values. A simple check for Changed = true thus allows determining if any attribute value has changed since the previous optimization iteration. In the TAG use case, such flags are set and unset in the TASK Service Catalog.
As it is unmanageable to record every small change in an attribute value, levels for the values of the aggregated performance measures are considered. In the terminology of Section 4.2.2,
let q be an attribute and a ∈ Tgt(q) an attribute value. A level (Va)i of the attribute value a
is defined as a range of values of attribute q:
(Va)i= [ai, ai+1[ = {x ∈ R | ai≤ x < ai+1} (5.1)
Attribute level changes are defined as follows:
if a(t) ∈ (Va)i and a(t + ∆t) < ai→ a(t + ∆t) ∈ (Va)j for j < i (5.2)
if a(t) ∈ (Va)i and a(t + ∆t) ≥ ai+1→ a(t + ∆t) ∈ (Va)j for j > i
Implicitly, there is a function f mapping each attribute value to the level it belongs to. Such a mapping is defined per attribute, because each attribute can take different types and thus requires custom levels. These levels can be defined in the attribute ontology and added to the attribute definition (cf. Section 4.2.2):
qi= {id(qi), Dom(qi), Tgt(qi), f : Tgt(qi) → Va:= ∪i(Va)i}. (5.3)
System Usage. Variables quantifying the system usage can be derived from monitoring and logging information. For example, the average number of requests per day can be tracked, as well as the distribution of requests during a day. Taking into account these two values only, occurring deviations can be detected. Usage patterns can be derived with different granularity. For example, on a daily basis, hours of high, moderate and low activity can be detected, and request distributions can be analyzed on a weekly basis. These patterns can be used to predict future workload and accommodate the problem parameters. Additionally, the popularity of services can be analyzed from usage logging information. As a consequence, highly popular
services or use cases can be given priority over less popular ones. Let U = ui, i = 1, ..., n, be
the set of n usage profiles, each indicating a certain usage level.
Based on the above provided definitions, a System Profile SP (t) is defined as a vector associating information about deployment status, attributes, and usage at a given point t or period in time:
SP (t) := [Sys(t), u(t)] (5.4)
where Sys(t) is as defined above, and u(t) ∈ U is a usage level.
A system profile defines and describes the system status at a given point t in time in terms of available deployments, links, related attributes and usage level. If the system profiles at time t and t + ∆t differ, then a system-inherent change has occurred in between. The impact of changes is discussed in Section 5.2.3.
5.2.2
Optimization Profiles
Optimization profiles are used to define and track changes in optimization parameters as defined in 5.1. The following aspects are taken into account:
Objective Weights. Objectives are weighted to express which objectives are more important than others to be optimized. When weights can be determined, a multi-objective optimization prob- lem can be transformed into a single-objective problem by combining the objective functions (cf. 5.4). For example, Badr et al. [12] propose to use a weighted-sum approach to reflect user preferences of non-functional features in the combined objective functions. However, in practice it is difficult to properly assign weights. For example, if in a certain period it is more important to optimize a single request than the system load and throughput, should the two objectives be weighted 0.6 to 0.4, 0.7 to 0.3, or any other values? Thumb rules can be applied, but they can be poor approximations and really good solutions might not be considered. As there exist multi-objective approaches that do not require weights to be defined, this feature is used and four simple cases are distinguished as follows for the general case of q objective
functions, where wi and wj are the weights of objective functions i and j respectively, with
i, j = 1, ..., q and i 6= j, and wi, wj∈ [0, 1],P q
n=1wn= 1.
1. wi wj: Objective i is given priority over objective j.
2. wi∼ wj: Objectives i and j are equally important.
3. wi≺ wj: Objective j is given priority over objective i.
4. wi? wj: There is no preference information/knowledge.
In case (1), random values are assigned to wi and wj as defined above, with the restriction
that the inequality wi > wj must be fulfilled. Case (3) is similar, except that the inequality
wi< wj must be fulfilled. In case (4), no information about objective preferences is available,
thus completely random weights are assigned to wi and wj. In case (2), the objectives i and j
are equally important and the same weight factor is applied to them. In a special scenario, if case (2) applies to all pairs (i, j), then wi= 1q for all i = 1, ..., q. The objective weights vector
o(t) is defined as the set of weight relations valid at time t.
The objective weights vector can be used to express prioritization of use cases. For example, in the TAG system one can define two use cases as follows:
1. Interactive queries (iELSSI)
2. Batch jobs (Extract, Skimming, Event Lookup)
In the TAG example, the prioritization of a use case over another means that for workflows of the prioritized use case the deployments should be chosen such that the response time for the whole workflow is minimized, regardless of load-balancing and resource usage strategies.
Concretely, this maps to case (1) (i.e., w1 w2) of the objective weights, assuming that w1 is
the weight of the objective concerned with minimizing the response time, and w2 the weight
of the objective related to load-balancing and throughput. In the general case, a use case prioritization results in an adapted objective weights vector.
5.2. DETECTING AND ASSESSING CHANGES IN THE ENVIRONMENT 85 Constraints. Regarding constraints, at this level only two cases are distinguished: either the op- timization problem is constrained, or it is not. In the former case, concrete constraints have to be defined on any of the available QoS attributes. The presence or absence of constraints has an important impact on the optimization problem, and the constrained case has to be addressed with specific methods, since solutions can be infeasible (i.e., they do not satisfy the
constraints) and have to be discarded. ci(t), i = 1, ..., m, is defined as the set of inequality
constraints active at time t.
Based on the above definitions, an optimization profile OP (t) at time t is defined as:
OP (t) := [o(t), ci(t)], i = 1, ..., m (5.5)
In summary, the overall system state at time t is defined by SP (t) and OP (t).
5.2.3
Quantifying System Changes
Whereas the previous two subsections discussed general system dynamics and changes in the opti- mization parameters, the following considerations take into account a workflow, i.e., they are made on a concrete instance of an optimization problem rather than on the generic one. If several changes occur between two requests that trigger an optimization run, it is necessary to assess those changes in terms of their (potential) impact on the solutions. Therefore, it is beneficial to be able to ap- proximately quantify a change. As previously mentioned, any change is marked with a flag and its corresponding timestamp in the central registry. It is thus easy to query all changes that occurred since a certain time t. The proposed method to approximate the degree of change (expressed as a similarity index ) is based on a component-level classification as follows. The components (deploy- ments, links and their attributes) of a given solution are compared with the system state Sys(t) at the current time t. The comparison is based on the change flags and timestamps of the components. For example, if a new deployment for a service (that is part of the solution) is available, its creation timestamp is newer than the timestamp of the deployment in the last solution. Therefore, this deployment might not be the optimal one anymore. The same applies to links and attributes. The changes of all components and attributes are rated and aggregated to a similarity index. In a simple setup the changes are rated as follows:
• Components are treated equally: the impact of a single component Ci, impCi, is as listed
in Equation 5.6, where n is the number of deployments and m is the number of links in the workflow.
• Changes of attributes of deployments Di from a service Sj are rated according to the relation
of affected to total deployments of the service, as suggested in Equation 5.7. nD(Sj) is the
number of deployments of service j and xD(Sj) is the number of deployments from service Sj
that changed between t and t + ∆t.
• Changes of link attributes are rated according to the Equation 5.8, where xL is the number of
links affected by changes between t and t + ∆t.
• the overall similarity index of all components is sitotal as listed in Equation 5.9, where n is the
impCi = 1 n + m, i = 1, ..., n + m (5.6) siD(Sj) = impCi· 1 − xD(Sj) nD(Sj) (5.7) siL = impCi· (m − xL) (5.8) sitotal = siL+ n X j=1 siD(Sj) (5.9)
This technique is a first attempt to compare two system states and approximately quantify their similarity. The categories derived from the similarity index however have to be defined. An example is provided in Figure 5.1. A similarity index of 1 means that no system change occurred during the observation period. A system change close to 0 suggests a radical change. There are no absolute boundaries between qualitative categories such at slight, moderate and radical change. This can be determined per scenario and optimization approach. In anticipation of the evaluation, in the present scenario a similarity index of approximately 0.5 is the limit beneath which the change is so important that the reuse of knowledge from previous optimization runs has no beneficial influence. However, as suggested by the dashed lines in Figure 5.1, this is a relative boundary. In fact, it does not only depend on the number of changes, but also on the delta of each change. These classifications are however fair approximations for determining an appropriate adaptation of the optimization approach, as will be discussed in Chapters 6 and 7.
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
moderate change
similar
different
radical change
Figure 5.1: Similarity Index and Change Impact