In addressing the aforementioned research questions, we make the following contribu- tions.
Predictability of execution in stand-alone web services middleware
We present an admission control mechanism and a deadline based scheduling method, to be used in stand-alone web services middleware. We provide an analytical model based on real-time scheduling principles that calculates the schedulability of a request given its deadline requirement and the requests already accepted for execution. The schedulability check considers the laxity property in accepting requests for execution and gives a guarantee on meeting their deadlines. Real-time scheduling principles are typically used in designing static schedules for closed systems. The uniqueness of the proposed solution lies in the fact that they are used in a highly dynamic and open envi- ronment, at run-time. A schedulability check algorithm based on the analytical model selects request based on their laxity and only selects request with execution deadlines that can be met given the current conditions. The selected requests are scheduled for
CHAPTER 1. INTRODUCTION
execution using Earliest Deadline First (EDF) scheduling algorithm. Such purposeful selection and scheduling of requests ensures that execution deadlines of requests can be achieved by the system. With these techniques used for achieving predictability, there is an unavoidable reduction of throughput. However, we ensure the resultant throughput levels to be within an acceptable range, when the nature of application is considered.
Predictability of execution in web service clusters
We present four request dispatching algorithms to be used in cluster base web service deployments that consider the processing deadline as a parameter in dispatching de- cisions. Two of the algorithms dispatch requests in a content-blind manner and the schedulability of a request with a chosen executor is considered prior to dispatch. The remaining two algorithms carry out content-aware dispatching. One uses the task size to match a request to an executor and the other distributes requests among executors to increase the range of laxities at each executor. All four algorithms make use of the laxity property when the dispatching decisions are made. The laxity based request se- lection of the four algorithms ensure that the requests at each cluster server comprise of a wide range of laxities. This enables more requests with overlapping deadlines to be scheduled together by delaying or phasing out the requests of requests with larger laxi- ties. Selected requests are executed at each server using EDF scheduling algorithm. The four algorithms represent a majority of the widely used dispatching techniques that ei- ther balance or unbalance the load of a cluster and therefore are a good example of how predictability attributes such as a laxity could be considered as part of the dispatching decisions.
Building web services middleware with predictable execution
The software engineering aspect of the overall solution is also considered as a main contribution of this thesis. A real-time scheduler component makes use of a priority model and custom made real-time thread pools to achieve fine-grain control over the execution of requests in the servers. Moreover, algorithms and system designs to im- plement the schedulability check and the deadline based scheduling algorithm at the middleware level is also presented as contributions. As, guaranteeing execution level predictability requires a more serialised approach to request execution, separate lanes of execution has been introduced into the web services middleware. Real-time worker
CHAPTER 1. INTRODUCTION
threads with elevated priority levels executing in these lanes gives better control of their execution to the real-time scheduler. In memory logging and debugging techniques that minimises Input/Output activities are used to ensure priority inversions are kept to a minimum. Such software engineering techniques, design patterns, algorithms and tools for achieving predictability is considered to be the main contribution.
Advanced performance modelling of earliest deadline first scheduling in web ser- vices
We present an analytical model based on queueing theory to measure the performance of a priority based preemptiveM/G/1 system using earliest deadline first scheduling
technique. The model is an extension to the work of [Chen and Decreusefond,1996], where the referenced model is extended to a preemptive deadline based scheduling sys- tem. The model approximates the mean waiting time of a request belonging to a par- ticular priority class where the mean waiting time is based on the execution of higher and lower priority requests already at the system, higher priority requests arriving at the system subsequently and having prior service and the mean residual service time ex- perienced by the priority class. The preemptions are approximated as part of the mean completion time experienced by a request of the considered priority class and it is en- capsulated in the definition presented for the mean residual service time experienced by the request. To our knowledge this is the first attempt at approximating the performance of such a system using queueing theory. The significance of this model is that it is independent of the service discipline and therefore is applicable to any scheduling tech- nique. Moreover, the model and its approximations are valid not only for web services, but any other system that uses a similar queue and earliest deadline first scheduling.