In this dissertation, we introduce the concept of expectations, capabilities and feedback as a holistic approach to support QoI in EBS as a first-class citizen and enable participants to adapt at runtime based on feedback about their actions and the global system state. We consider the fact that participants operate on resource-constrained devices and Cloud environments. Hence, resource- and data-efficiency are important criteria in our approach.
We enable subscribers to express preferences for QoI properties based on their individual VoI while publishers can associate their capabilities with costs for providing certain QoI properties. As part of our approach, we model QoI-related properties like sampling rate, confidence of de- tection, or latency in a generic way. Actions are associated with a generic property and define how it can be adjusted at runtime: by advising a publisher to adapt, adapting the MOM, or by adapting both publisher and MOM in a coordinated fashion.
Subscribers can express preferences and individual trade-offs between requirements as expecta-
tions in a consistent and information-centric way. Publishers expose their general abilities to
support a generic property, their costs, and the state they are currently operating at to the MOM as capabilities. In a capability, we do not only capture the current state of the publisher but, more importantly, we model the extent to which it could adapt. Thus, a capability describes first and foremost the spectrum of adaptation that would be possible together with the costs arising from this adaptation; the current state is included as well to analyze the still exploitable spectrum. Malleable requirements (expectations) and the ability of the system to adapt (capabilities), de- fined as weighed ranges over generic properties, enable us to automatically negotiate QoI re- quirements at runtime. The algorithms used in this process identify the extent to which the system would have to adapt to satisfy requirements. The system assesses the feasibility by bal- ancing the costs for performing the identified adaptation with the benefits to be derived from it using custom decision- and load-balancing strategies; costs and benefits depend on the current system state, the preference expressed by subscribers and the costs associated with publishers. Based on this assessment, requirements are declined or satisfied by adapting the system. Sub- scribers receive feedback about the state of their expectations while publishers get feedback about the usage of their capabilities and explicit advice to adapt if necessary.
The proposed concept complementary extends the push-based paradigm of an EBS without com- promising the model of indirect many-to-many communication, making it backward compatible. As shown in Figure 1.1, expectations and capabilities are defined independently of advertise- ments, notifications or subscriptions. They are matched only at the MOM, preserving the
Typical Event-based system
Concept of expectations
QoI Properties
Publisher Message-oriented Subscriber
Middleware Advertisements Notifications Notifications Subscriptions Capabilities Feedback Expectations Feedback Type Content a) adaptation advice
b) aggregated metrics a) expectation stateb) aggregated metrics
Figure 1.1.: Our concept extends the model of EBS (top) with capabilities, expectations and bidi- rectional feedback for runtime adaptation (bottom, bold).
anonymity of the associated participants in an EBS that is essential for scalability, flexibility, extensibility and robustness. Individual feedback about expectations or capabilities as well as aggregated feedback about the system enables participants to assess their current situation and adapt their behavior at runtime.
Expressing capabilities in a uniform way and associating generic actions with them, we can connect to the huge body of existing work to manipulate and optimize dedicated lower-level properties such as latency, throughput, or precision. We can utilize these related approaches by decomposing requirements about more abstract generic properties expressed in our approach into lower-level requirements. For example, latency is a property supported by both IndiQoS and Harmony on the transport protocol level. In our approach, we could enforce requirements about this property by associating it with a generic action that maps our definition of latency to the representation used by IndiQoS or Harmony. Depending on the deployment of the system, we could then trigger an underlying instance of either MOM for enforcement.
The fact that QoI requirements or capabilities of a participant can be influenced by changes to its context or state is essential to our approach, as it requires the EBS to react dynamically at runtime. For example, a mobile publisher’s publication latency can temporarily increase due to bad connectivity, forcing the MOM to evaluate the impact of this change on subscribers that rely on low-latency data from this publisher. However, we do not necessarily have to know and understand the current context or state of a participant for this. Neither do we require all participants to be aware of their own context or state in the sense that they are able to infer and interpret their current context based on a specific context model.
A key element of our approach is the realization that QoI requirements and capabilities can change dynamically at runtime. These changes have to be detected by the MOM using runtime monitoring or by participants notifying the MOM about updates of their requirements or capa- bilities. In neither case do we need to know the reason for a change and thus do not require the use of a specific context model. Instead, we provide subscribers with an expressive model that allows them to map their preferences to requirements about objective properties supported by
the system without having to expose the model they use to infer and determine their own context and state. Respectively, publishers can express their current and potential supply for properties depending on their state and context without having to expose their context model either. Thus, approaches to infer and determine the ambient context of a participant are considered to be sec- ondary. Security and privacy aspects are important but orthogonal to our approach; they are out of scope of this dissertation.
By supporting QoI in an EBS in a holistic way, we contribute to the vision of reactive software systems that consist of coequal pull- and push-based software architectures [18, 19, 76, 78]. Pull- based software architectures like SOA operate on persistent data and offer extensive runtime support for SLAs defined over QoS-related properties; efficient monitoring approaches provide usage information about SOA-based systems and SLAs defined for them at runtime. Push-based software architectures like EBS, however, lack a comparably generic concept to support QoI- related properties at runtime. Providing the necessary usage information about an EBS and QoI properties at runtime is still an open issue. We contribute concepts to support and monitor guarantees about QoI-related properties in EBS at runtime.
Referring to the research questions regarding the support of QoI in EBS, the contributions of this dissertation are in particular:
Generic model to express QoI requirements and adaptation capabilities (Q1+Q2). We pro-
pose a generic model to express and manage malleable requirements about QoI-related proper- ties in an EBS at runtime. Subscribers define requirements about an arbitrary set of properties as an expectation in a consistent and information-centric way. Multiple expectations by the same subscriber define a preference order without the need to define an explicit utility function. Pub- lishers expose their support for QoI-related properties to the MOM as capabilities. Each capability describes the spectrum of support a publisher can offer for a specific property when using self- adaptation; the current state it is operating at is contained as well. A capability profile of each publisher is maintained at the MOM and includes capabilities for those properties determinable only by the MOM at runtime. Costs and benefits are essential components of our model: sub- scribers quantify the VoI of their requirements by ranking them with utility values while publishers and MOM manage cost functions for capabilities, e.g., the costs for maintaining a sampling rate of 100 notifications/second vs. a sampling rate of 50. The degree to which a subscriber is satisfied with the results delivered by the MOM is quantified by its fidelity.
Algorithms to negotiate and enforce QoI requirements at runtime (Q3+Q4). We present
algorithms to automatically decide about QoI requirements at runtime, deduce the action plan and execute it. In particular, we discuss algorithms for matching expectations to capabilities, deciding on satisfiable expectations, safeguarding action plans and enforcing the decision. Our approach allows using custom decision strategies and takes into account the costs and utility values provided by participants. Decisions are enforced by transparently adapting the MOM and advising publishers to adapt.
Feedback for self-adaptation to extend the communication model of EBSs (Q1,Q2,Q4). We
extend the unidirectional communication model of push-based systems with bidirectional feed- back from the MOM to participants. At runtime, we provide subscribers, publishers and brokers with individual and aggregated feedback. Individual feedback contains status updates from the MOM about expectations or capabilities of a dedicated participant. Subscribers get updated about the state of their active expectations while publishers receive explicit adaptation advices containing a target value that a capability has to be adjusted to. Thus, we extend the scope of
supported QoI properties to those influenced by publishers. Aggregated feedback is delivered to all participants that have registered interest in aggregated metrics about the system population and state, e.g., the number of active subscribers for a certain type of notification or the average sampling rate of notifications of a certain type.
Novel concept for runtime monitoring of decentralized event-based systems (Q1+Q4). We
introduce a novel approach to monitor the population and dynamics of a large-scale Distributed Event-based System (DEBS) effectively based on the concept of Application-specific Integrated Aggregation (ASIA). We use this monitoring information in three ways to support QoI require- ments: a) to detect significant changes in the global state of the system that require a renego- tiation of expectations; b) to decide on load-balancing; and c) to provide aggregated feedback to participants. Participants can individually balance the measurement costs with the freshness and precision of the reported monitoring information. Our approach exploits relaxations on information precision to limit the propagation of unnecessary updates within the system.
Reference architecture for runtime support of QoI requirements (Q3+Q4). We present the
design of an architecture to support QoI requirements in an EBS at runtime. Our reference architecture consists of an extension to each broker in the MOM as well as additional handlers provided to participants. The extension makes a broker self-adaptive while the handlers enable participants to deal with feedback by the MOM and manage the lifecycle of expectations and capabilities. Our design can be applied to any push-based MOM as it extends the Application Programming Interface (API) of an EBS. The MOM extension consists of platform-independent components for negotiating between expectations and capabilities, monitoring the system state, and coordinating adaptation. Platform-specific components implement MOM-related reactions like routing adaptation or traffic shaping.
Prototypes implemented in Java as proof of concept (Q2-Q4). We present two prototypes of
our reference architecture built on open-source MOMs with different features to show the fea- sibility of our approach: Apache ActiveMQ is representative of a centralized industrial-strength MOM focusing on high performance while the distributed REconfigurable Dispatching System (REDS) allows us to exploit routing strategies in decentralized networks of brokers. Both MOMs have been extended to support expectations, capabilities and provide feedback to participants. Extensions are written in Java without affecting existing code.
Examples for applications implementing our approach (Q1+Q3,Q4). We present examples
for existing open-source applications that we have enhanced with our approach. Using expec- tations, capabilities and feedback, QoI requirements are now supported in the Ganglia monitor- ing system and MySQL master-slave replication by extensions to our self-adaptive Multi-cloud Cost-Aware Transport (McCAT) MOM. We have designed McCAT to combine the self-awareness in distributed and decentralized EBS, as provided by ASIA, with awareness about QoI that is provided by expectations and capabilities. Using McCAT as a transparent transport mechanism for existing distributed applications, we can save up most of the network traffic produced by out-of-the box deployments of Ganglia monitoring and MySQL master-slave replication without violating QoI-requirements defined in these systems. Using McCAT wrappers, we make static dis- tributed systems self-adaptive without having to change existing code. For the evaluation of our approach, we have also extended the open-source benchmarking tools FINCoS and jms2009-PS to include expectations, capabilities and feedback in their workloads.
Evaluation regarding expressivity, benefits, and execution costs (Q3+Q4). We evaluate the
expressivity of our approach by showing that we support arbitrary QoI properties while we also
support the limited sets of QoI properties addressed by related approaches. We show that us- ing expectations, capabilities and feedback for adaptation results in significant benefits for sub- scribers, MOM and publishers in terms of higher fidelity, less network traffic and lower CPU utilization. In terms of execution costs we show that processing and negotiating expectations and capabilities at a single broker is feasible for more than 2000 active participants triggering renegotiations up to every five seconds without degrading the performance of the MOM in terms of throughput and latency. We identify the cost drivers for execution costs and explore their trade-offs using jms2009-PS, which is based on the industry-standard benchmark SPECjms2007. We evaluate the effectiveness of our proposed monitoring approach separately in terms of per- formance, scalability and precision. We show that our approach provides information about the population and dynamics of a DEBS at the individual level of precision defined by the consuming components. Our monitoring approach outperforms external aggregation systems by collecting and distributing information with a limited overhead while we do not significantly impact the performance of the MOM in terms of throughput and latency.