Models for Combined Load

Several tasks commonly share a single node in data-oriented systems. To be able to evaluate a given allocation, the global load of a node, i.e., the combined load induced by all tasks that are executed on the node, needs to be estimated. A node’s load determines it’s performance and is hence used to achieve the objective

of the allocation problem and to formulate constraints of the allocation strategy (the correlation of load and performance is discussed in Section 3.3).

In the simplest case, loads induced by tasks are combined by summing them up to derive a node’s global load. This method is referred to as the linear model as it models an ideal system where performance scales linearly with the amount of work that needs to be done. However, in practice, performance often depends on all kinds of workload parameters like request rates, request types, request sizes, and the concurrent execution of requests. Contention on resources caused by concurrent execution may lead to performance the does not scale linearly with the amount of work. Therefore, in addition to the linear model, we propose a non-linear model to combine the individual loads induced by tasks that share a node.

Note that this section discusses how loads, i.e., tasks, are combined. Consequently, this discussion could arguably also be part of the workload model in the following section. In fact, the later allocation strategy will apply the non-linear model to the workload (by increasing the load induced by all tasks) in contrast to applying it to the infrastructure (by reducing a node’s capacity depending on the tasks). However, since the non-linear performance is caused by contention in the infrastructure, we consider it a property of the same and decided to discuss it here.

We casually refer to resources that use non-linear models to combine loads as non- linear resources and in general we refer to non-linear resource usage and non-linear resource behavior. When using these short (though not entirely correct) terms, we refer to the non-linear performance (non-linear in the amount of work) that results from using the resources, e.g., caused by contention.

Non-Linear Resource Usage

A major reason for sub-linear scalability with the number of tasks is contention on resources. Contention can happen on the hardware itself, on operation system resources, or on DBMS components. Disk I/O, as a prime example, has always been considered a non-linear resource caused by characteristics of the underlying hardware. With our focus on in-memory technology, disk I/O is beyond the scope of this thesis. However, other resources like CPU and memory that are sometimes considered to be linear resources for simplification also show non-linear behavior under high load.

Depending on the implementation, high request rates can lead to a high number of context switches in the operating system (process or thread switches). Li et al. (2007) measure direct costs of context switches, e.g., caused by saving and restoring registers and by system calls, and indirect costs of context switches caused by cache pollution. With a tool provided and discussed by Sigoure (2010) in a blog post, we measured the direct costs of a context switch on a current Intel Xeon E7 to be about 32 µs. With a high degree of parallelism, this can add up and deteriorate performance. A high request rate also causes contention on other resources. We have investigated the effect of cache sharing between database workloads in a NUMA system in a previous article (Kiefer et al., 2013). Tözün et al. (2013) investigate cache misses that occur when executing an OLTP system. The authors break down the cache misses and identify components of the system that cause certain types

3.1. Infrastructure Model 0 500 1000 1500 2000 2500 3000 3500 0.2 0.4 0.6 0.8 1 1.2 99th P er cen tile R esp onse Tim e [ms]

Workload (Request Rate)

(a) Experimental Response Time Based on Request Rate 0 500 1000 1500 2000 2500 3000 3500 0.2 0.4 0.6 0.8 1 1.2 99th P er cen tile R esponse Time [ms]

Workload (Request Rate) linear component exponential penalty

(b) Decomposition of Response Time Func- tion

Figure 3.3.: Non-Linear Response Time Behavior, from Schaffner et al. (2011) of cache misses. Blagodurov et al. (2010) conduct detailed experiments on memory controller contention, memory bus contention, and prefetching hardware contention. The data-oriented system itself is another source for contention. As shown in Section 2.1.2 for database-as-a-service systems, the layers of the system stack that belong to the node and are hence captured in the infrastructure model depend on the implementation of the system. For instance, given the Private Database class, the database server is part of the node and contention on DBMS resources is captured in the non-linear behavior of the infrastructure. Different components of relational database management systems like logging facilities, free space management or trans- action management can congest under high load. Johnson et al. (2008) investigate contention on various components of the DBMS that lead to bad scalability. In a related work (Johnson et al., 2009), the authors analyze the storage manager Shore and identify several components, e.g., buffer pool manager, log manager, and lock manager, that cause bad scalability. The optimized Shore-MT improves scalability by optimizing locks, latches, and critical sections. Pandis et al. (2010) use bad scala- bility caused by contention on DBMS resources as motivation for their data-oriented architecture. Although data-oriented systems aim at localizing work and avoiding central data structures, there always remains a portion of central code that may cause contention.

In summary, the detailed behavior of the system strongly depends on details of the hardware, implementation details of the operating system, and on the data-oriented system itself. To grasp the general behavior of the complex system, we assume a simplified resource consumption model that reflects the non-linear performance of the system. In our infrastructure model, each resource has a (relative) capacity and a (possibly non-linear) model to combine loads. We distinguish two types of non-linear behavior, (1) penalized resource usage and (2) general non-linear resource usage. The first type, which is a special case of the second type, is intuitive and leads to optimizations in the allocation strategy.

Penalized Resource Usage The penalized resource usage model is a combination of

of parallelism, the linear usage assumption often holds and the performance can be predicted accurately. However, when a certain load level is reached, contention occurs and the performance does not scale linearly beyond this load level. To account for this contention and the non-linear performance, a penalty is introduced that adds to the resource usage based on the number of concurrent tasks.

Schaffner et al. (2011) did an experiment with the in-memory column database TREX. The authors increase the abstract workload, a combination of data size and request rate, and measure the 99th percentile value of the response time of all requests. Based on multiple experiments, the response time is approximated as a function of the workload. The results of the experiment are shown in Figure 3.3. Figure 3.3a shows the results from the paper for a fixed data size, i.e., when the workload is only a function of the request rate. Figure 3.3b shows that the function for the response time can be split into a linear component and a penalty that starts to affect the response time above a certain load level. The authors in the paper avoid load above this point and argue in favor of a linear performance model. We assume overcommited resources and argue in favor of incorporating the penalty in the infrastructure model.

General Non-Linear Resource Usage In the general case, non-linear resource usage

can be modeled as an arbitrary function2 _{of one or more linear resources. In contrast} to penalized resources usage, the non-linear function here does not have to be a composition of the linearly combined load and a penalty. Furthermore, multiple resources together can be used to describe and thereby influence a single non-linear resource.

Note that the non-linear usage model of resources in the infrastructure model is still a simplification of the actual system behavior. However, given the various sources for contention, we consider it more accurate than the simpler linear model. Given the dependency of the resource behavior on the underlying system, we do not detail any particular non-linear model here but assume the existence of penalty functions or general non-linear resource usage functions. In an actual system, these functions have to be determined from manufacturer manuals, operating system doc- umentation, or experimentally based on low-level experiments using the monitoring capabilities of the system.