Engineering web services middleware for predictability requires a different approach to engineering for throughput. The overall aim is for the middleware to have more control over the execution of requests. To enable this, the execution of requests is serialised by scheduling requests for execution one after the other in the order of their increasing deadlines. However, a controlled level of throughput is still achieved by implement- ing several lanes of execution where multiple worker threads are be active at a given time, depending on the number of processor cores available for execution. The con- ventional thread-pools in the middleware are replaced by custom designed thread pools that employ real-time threads. More control over their execution is further facilitated by introducing a multiple priority model, containing priority levels that cannot be inter- rupted by housekeeping activities and other processes. Such priorities ensure that the execution of a thread happens uninterrupted. The priority model is used by a real-time scheduler component to execute and suspend worker threads at will.
Moreover, EDF scheduling is implemented within the real-time scheduler in the form of a thread scheduling algorithm. Such fine grain control over thread execution requires the introduction of proper concurrency control using critical sections and semaphores, over the implementation of the admission control check, EDF scheduling and request dispatching in the clusters. The execution deadlines are conveyed to the server using SOAP message headers, and they are conveyed to the admission control check through an internal data structure. These implementation decisions are supported by the use of development platforms, libraries and operating systems that contain real-time features.
The proposed solution involved implementing these techniques in two web services middleware products, namely Apache Axis2 and Apache Synapse. Axis2 is a stand- alone middleware and Synapse is an ESB product that can be used to implement dis- patcher functionality in a cluster. Two case studies based on the enhancements carried out on these products are presented in this chapter as practical examples for achiev- ing predictability of execution. One case study represents a stand-alone web service middleware configuration using Apache Axis2 and the other discusses a web services cluster deployment using Apache Synapse as the dispatcher and Axis2 instances as ex- ecutors. Three possible cluster configurations are presented and their pros and cons are
CHAPTER 5. BUILDING WS MIDDLEWARE WITH PREDICTABLE EXECUTION
discussed. The four dispatching algorithms are implemented in Apache Synapse using its sequence and endpoint extension framework.
The contribution through this chapter are the specialised software engineering tech- niques, algorithms, designs and tools that can be used for achieving predictable execu- tion times. The two case studies are examples on how they could be used in enhancing existing web services middleware. Although specific products were chosen for these case studies the techniques, algorithms and designs are generic enough to be applied for any other web services middleware.
The rest of the chapter is organised as follows. First, we discuss some of the related work found in this area. Next, we present the set of guidelines to follow to achieve pre- dictability of service execution, when enhancing existing middleware products or new ones being built. Thereafter, in Section5.6we give an overview of the implementation and discuss items common to both implementations. Following that in Section (5.7), we present the first case study of enhancing Apache Axis2. It is followed by the case study of the cluster-based implementation and then we conclude in Section6.9.
5.4 Related Work
There has been only a few attempts at introducing predictability into service execution or making it a feature in the middleware. As mentioned earlier, many of the exist- ing middleware products optimise for throughput rather than predictability and thereby introduce features that makes it impossible to achieve predictability of execution. A few of the existing work we found that were purposefully built to differentiate request processing or to have real-time features, are discussed here.
wsBus [Erradi and Maheshwari,2005] is a custom built QoS-aware middleware based on a bus architecture. It has many components that facilitate the use of different trans- ports, request dispatching, service selection and QoS monitoring and has the design of a customised ESB product. It supports the use of priorities for differentiating re- quests and contains an admission control mechanism that controls requests accepted. However, the criteria of admission nor the prioritisation has been explicitly mentioned. Similarly, the way differentiation happens and any evaluation of its performance has not been presented.
CHAPTER 5. BUILDING WS MIDDLEWARE WITH PREDICTABLE EXECUTION
embedded real-time environment is the first attempt we found in literature of such mid- dleware. Web Services are used for communication between different components in the embedded environment. They achieve this by defining behavioural patterns among the components that represent interactions between components and tasks that need ex- ecution as a result. The timing properties, worst case resource requirements of these patterns are figured out at design time of the system. At runtime, these patterns are used to predict and reserve resources for the incumbent tasks. The worst case resource requirements planned at design time ensures that variations in execution and jitter are catered for, by over reserving resources at run-time. However, neither an architecture, implementation nor an evaluation has been presented in the paper.
[Mathes et al., 2009a] presents SOAP4IPC: a real-time SOAP engine designed for in-
dustrial automation. It contains general components as found in a typical SOAP engine that takes care of processing and execution of web services and also components that are designed to represent tasks typically found in real-time systems (as presented in chapter2). The execution engine honours a deadline and caters for both periodic and aperiodic tasks. Given the support for periodic tasks and the not having an admission control check means the middleware is intended to be used with tasks, that properties are known at design time. An approach suggested by the authors is to use a profiling ap- proach of measuring the worst-case execution times of each service. However, there is no mention of the actual scheduling algorithm used or a comprehensive evaluation with realistic services and traffic types. The SOAP4IPC engine is a part of a broader frame- work named TiCS [Mathes et al.,2009c] which stands for Time-Constrained Services framework that is presented as a complete manufacturing execution system that uses web services for industrial automation. The SOAP4IPC engine is at its core handling the execution of services and another layer which is detailed in [Mathes et al.,2009b] works as a service fac¸ade for programmable logic controllers that make up the manu- facturing layer or the overall manufacturing execution system. The TiCS framework is described in detail in terms of its components and their intended functionality, however there is no mention of the actual scheduling algorithm is used or how the deadlines are ensured by the system.
While [Helander and Sigurdsson,2005;Mathes et al.,2009a] and [Mathes et al.,2009c] are mentioned as working solutions in real-time environments they lack important de- tails such as how the requests are scheduled and a comprehensive evaluation of their performance compared against other products and approaches, to make a proper deci-
CHAPTER 5. BUILDING WS MIDDLEWARE WITH PREDICTABLE EXECUTION
sion on their effectiveness. Moreover, both these solutions are intended to be used in closed environments where there is a good understanding of task properties and their re- source requirements. Without having proper admission control and precise scheduling, it is difficult to use their techniques to introduce predictability into web service execu- tion in open systems. The challenge of selecting tasks based on resource availability and making that decision at runtime is an important aspect that has to be met with such open systems.