The StarPU runtime system

StarPU is a runtime system developed by the STORM (formerly RUNTIME) team at Inria Bordeaux specifically designed for the parallelization of algorithms on heterogeneous architectures. A complete description of StarPU can be found in the work by Augonnet et al. [18].

StarPU provides an interface which is extremely convenient for implementing and parallelizing applications or algorithms that can be described as a graph of tasks. Tasks have to be explicitly submitted to the runtime system along with the data they work on and the corresponding data access mode. Through data analysis, StarPU can automatically detect the dependencies among tasks and build the corresponding DAG. Once a task is submitted, the runtime tracks its dependencies and schedules its execution as soon as these are satisfied, taking care of gathering the data on the unit where the task is actually executed. In StarPU the execution is initiated by a master thread, running on a CPU, which is commonly in charge of submitting the tasks; the execution of the tasks, instead, is performed by worker threads (or, simply, workers) whose number and type (e.g., CPU or GPU) can be chosen by the programmer or by the user at run time. A CPU worker is bound to a CPU core whereas a GPU worker is bound to a GPU and a CPU core which is used to drive the work of the GPU. Note that nothing prevents worker threads from submitting tasks although this does not comply with the Sequential Task Flow model. Because StarPU has full control over a task and the associated data, these have to be declared to the runtime prior to the task submission. A task type can be declared through a codelet which specifies, among other things, the name of the task type, the units where it can be executed (e.g., CPU and/or GPU), the corresponding implementations (one for each type of unit) and the number of input data. Data, instead, is declared through a specific function call where the programmer informs StarPU about the location of the data (i.e., the data pointer) and about its properties such as the size, the rank, the leading dimension. Upon execution of this function call, StarPU returns a handle; once declared the data is not meant to be directly accessed by the programmer anymore but only through the handle and the associated methods. A task can roughly be defined as an instance of a task type coupled with a set of handles which represent the data used by the task itself.

We can more conveniently illustrate the functioning of StarPU using the simple se- quential code in Figure 1.12 as an example. The purpose of the sequential example is to compute every elements of the array denoted y, using the input array x. The computation is done by applying the functions f, g as shown in the example. The right side of the figure

1.4. Programming models and runtime systems

shows the dependencies between function calls in the various iterations of the for loop. Note that in this example we rely on a STF model for parallelization of the sequential code which is explained in Section 1.4.1.

1 for ( i = 1; i < N ; i ++) { x [ i ] = f ( x [ i ]) ; 3 y [ i ] = g ( x [ i ] , y [ i - 1 ] ) ; }

f

g

f

g

f

g

Figure 1.12: Simple example of a sequential code.

The parallel version of our simple example is proposed in Figure 1.13. In this code, ex- ecuted by the master thread, we first declare the codelet structures. A codelet corresponds to the description of the code that is to be executed by a certain type of tasks which is defined by setting, at least, the following fields:

• where enumerates which computational units may execute the task type. The possi- bilities includes STARPU CPU for CPU workers or STARPU CUDA for GPU workers and can be composed when when a task type may be executed by several workers with the following notation STARPU CPU | STARPU CUDA.

• cpu funcs and/or cuda funcs are function pointers referring to the computational kernels (i.e., the actual implementations) to be executed by respectively CPU and CUDA workers.

• nbuffers indicates the numbers of data/handles manipulated by the tasks.

Assuming we want to execute f and g function calls in separate tasks, we need to declare two codelets, one for each type of task. Because a CPU and a GPU implementation exist for function g, the corresponding tasks of type g cl can be executed on either type of units. Next, the data accessed by the tasks have to be registered; because we want each task to work on a single coefficient of the x and y arrays, we have to register each one of them to obtain the corresponding handle. This is done in the loop at line 23. Finally, the tasks are submitted in the loop at line 29 by specifying, for each task, the codelet and the corresponding data. The master thread finally reaches the barrier at line 43 where it sits waiting for all the tasks to be executed by the worker threads. When a task, say, of type f cl is actually executed, the worker thread invokes the f cpu func; this retrieves the x[i] data using the dedicated method on the x handle[i] handle and the calls function f.

As mentioned above, StarPU can transparently handle the data accessed by a task and move it where the task is actually executed. If, for example, for the iteration i of the loop in our example, function f is executed on the CPU and function g on the GPU, the data x[i] is automatically moved to the GPU memory. To avoid unnecessary data transfers, StarPU allows multiple copies of the same data to reside concurrently on several processing units and makes sure of their consistency. For instance, when a data is modified on a computational unit StarPU marks all the corresponding copies as invalid.

/* C o d e l e t d e f i n i t i o n for k e r n e l f */ 2 s t r u c t s t a r p u _ c o d e l e t f _ c l = { 4 . w h e r e = S T A R P U _ C P U , . c p u _ f u n c s = { f _ c p u _ f u n c } , 6 . n b u f f e r s = 1 }; 8 /* C o d e l e t d e f i n i t i o n for k e r n e l g */ 10 s t r u c t s t a r p u _ c o d e l e t g _ c l = { 12 . w h e r e = S T A R P U _ C P U | S T A R P U _ C U D A , . c p u _ f u n c s = { g _ c p u _ f u n c } , 14 . c u d a _ f u n c s = { g _ c u d a _ f u n c } , . n b u f f e r s = 3 16 }; 18 s t a r p u _ d a t a _ h a n d l e _ t x _ h a n d l e [ N ] , y _ h a n d l e [ N ]; 20 s t a r p u _ i n i t () ; 22 /* d e c l a r a t i o n of d a t a h a n d l e s */ for ( i = 0; i < N ; i ++) { 24 s t a r p u _ v a r i a b l e _ d a t a _ r e g i s t e r (& x _ h a n d l e [ i ] , S T A R P U _ M A I N _ R A M , ( u i n t p t r _ t ) & x [ i ] , s i z e o f(d o u b l e) ) ; s t a r p u _ v a r i a b l e _ d a t a _ r e g i s t e r (& y _ h a n d l e [ i ] , S T A R P U _ M A I N _ R A M , ( u i n t p t r _ t ) & y [ i ] , s i z e o f(d o u b l e) ) ; 26 } 28 /* t a s k s s u b m i s s i o n */ for ( i = 1; i < N ; i ++) { 30 s t a r p u _ i n s e r t _ t a s k (& f_cl , 32 S T A R P U _ R W , x _ h a n d l e [ i ] , 0) ; 34 s t a r p u _ i n s e r t _ t a s k (& g_cl , 36 S T A R P U _ R , x _ h a n d l e [ i ] , S T A R P U _ R , y _ h a n d l e [ i -1] , 38 S T A R P U _ W , y _ h a n d l e [ i ] , 0) ; 40 } 42 s t a r p u _ t a s k _ w a i t _ f o r _ a l l () ; 44 s t a r p u _ s h u t d o w n () ;

Figure 1.13: Simple example of a parallel version of the sequential code in Figure 1.12 using a STF model with StarPU.

Among the other features of the StarPU runtime system, in the work described in the following chapters, we will use the following:

1.4. Programming models and runtime systems

Figure 1.14: Two commonly used approaches to the multithreading of the multifrontal method.

• StarPU provides a framework for developing task scheduling policies in a portable way. The implementation of a scheduler consists in creating a task container and defining the code that will be triggered each time a new task gets ready to be executed (push) or each time a worker thread has to select the next task to be executed (pop) as illustrated in Figure 1.14. The implementation of each queue may follow various strategies (e.g. FIFO or LIFO) and sophisticated policies such as work- stealing may be implemented. StarPU comes with a number of predefined scheduling policies suited for different types of architectures and workloads. We will comment more on the development of schedulers in Sections 3.2.2, 4.4 and 6.2.

• If a task is assigned to a worker sufficiently ahead of its execution, the data it needs can be automatically prefetched. This allows for overlapping the data transfers with the execution of other tasks which results in better performance and scalability. We will describe the use of this feature in Section 6.2.

• In StarPU, it is possible to associate a callback function with a task. This is just a function which is executed upon termination of the task itself. We will use this feature in Section 3.2 to express some dependencies among tasks.

• StarPU has several facilities which allow for detailed performance profiling of an ap- plication. For example, it can generate execution traces containing accurate timings of all the executed tasks, of the runtime overhead, of data transfers etc. It can also au- tomatically build performance models for the tasks. These can be used to implement complex scheduling policies (such as the HEFT-like policies described in Section 6.2) or to generate a linear program whose solution provides a lower bound for the per- formance achievable by an application. This feature as well as the execution traces, are used in our performance analysis approach presented in Chapter 2. StarPU can also dump the DAG of an application, which we use to compute the average degree of parallelism (or best achievable speedup) as described in Section 4.3.2.

• It is sometimes convenient to explicitly declare dependencies between tasks rather than letting the runtime system figure them out; StarPU provides methods to im- plement this.