Results of Multiprogramming using Information Sharing

For multiprogram workloads, we have considered three different scenarios to show how the XLL mapper can result in better performance. The error bars on the first two figures (4.5 and 4.6) show how deterministic the results are, while in the third figure (4.7) the error bars illustrate the minimum and maximum running time amongst the 10 identical programs.

First, we have to show why BLL, which is a simple Round-Robin mapping algorithm is inefficient. For this purpose, we have considered three Health programs, each of which with 32 threads (this is based on the best performance achieved for uniprogramming [23]). Two programs have large inputs and one has a small input. The programs enter the system with the interval of 6 seconds. We have previously discussed why Static mapper cannot handle multiprogramming scenarios. The inefficiency of the BLL is also evident from Figure 4.5.

The first scenario clearly shows that the XLL is the winning policy. The scenario is designed in such a way that the program with the small input data set finishes before the second large program enters the system. In the case of BLL, the threads of the first large program are mapped to the first 32 cores of the system. The threads of the small program are mapped to the last 31 cores plus the first core (there are 63 cores to use). Then the small program finishes and the second large program enters the system, but the BLL cannot use the recently freed cores. Instead, based on the Round-Robin algorithm, the threads of the second large program are mapped to the cores 2 to 33, while most of these cores (except one of them) are already busy serving the first large program.

0 10 20 30 40 50 60 BLL XLL Linux Time (s) Mapping Approach 3 instances of the Health application

Small Large1 Large2

Figure 4.5: The first scenario: The inefficiency of the BLL

The second scenario is to run all four programs selected from the BOTS at the same time. Although they start at the same time, their thread creation time is different. This is due to the fact their initialisation phases and memory allocation times are different. Recall that thread creation happens whenever the execution reaches the parallel keyword in the OpenMP code.

4.5. Information Sharing and Multiprogramming 60

According to their uniprogramming performance in [23], the Sort program (50M integers) is limited to 16 threads, the Health program (Medium input) is limited to 32 threads, and both Strassen (2048 × 2048) and NQueens (15 × 15) are run with 63 threads. The result is shown in Figure 4.6.

Once again, the XLL results in better performance. The turnaround times for all 4 programs are smaller when the XLL policy is applied.

0 10 20 30 40 50 XLL Linux Time (s) Mapping Approach

4 programs starting at the same time

Sort (50M)

Strassen (2048) NQueens (15)Health (M)

Figure 4.6: The second scenario: Running selected programs as the same time

0 10 20 30 40 16-Threads 63-Threads Mean Time (s) Number of Threads 10 identical Sort programs (50M),

starting with the interval of 1 Sec

XLL Linux

Figure 4.7: The third scenario: Running 10 identical instances of the Sort program

The third scenario gives a better insight on how the XLL mapper outperforms the Linux scheduler when the system is busy. For this scenario, we have used 10 identical instances of the Sort program arriving the system one after the other with the interval of 1 second. The result is depicted in Figure 4.7.

Figure 4.7 shows that the results with both policies are significantly better when each Sort program uses 16 threads rather than 63. It again verifies that increasing the number of threads

4.6. Summary 61

does not necessarily result in better performance. It is also evident how much our novel XLL mapping technique can outperform the native Linux scheduler in a multiprogramming environment.

4.6

Summary

We compared some of the performance aspects (in particular speedup, CPU balance, and the Total CPU Time) of three well-known parallel programming approaches, Intel OpenMP, Intel Cilk Plus and Intel TBB, on the Xeon Phi coprocessor. For that purpose, we used three differ- ent parallel benchmarks, Fibonacci, Merge Sort and Matrix Multiplication. Each benchmark has distinct characteristics which highlight some pros and cons of the studied approaches in different uniprogramming scenarios. Our multiprogramming scenario on the Xeon Phi was to run all three benchmarks together on the system and observe how the programming models react to this situation.

Based on the results obtained from the uniprogramming scenarios, particularly the Total CPU Time, we predicted that the Intel TBB approach would be more suited to a multiprogramming environment, and our experiment confirmed this. This is just the beginning, and these are our preliminary results. What we learned from these experiments is that keeping the overhead of runtime systems as low as possible in a general-purpose system is crucial, due to its direct impact on multiprogramming performance. The performance results also bold the importance of finding the optimal number of threads, as sometimes increasing the number of threads might lead to performance drop. In the next chapters, we address these problems by designing a new parallel execution model. In GPRM, a big overhead of task scheduling is shifted to the compile time. Moreover, instead of giving attention to both (number of) threads and tasks, the focus is only on the computational tasks.

In addition, since the way Linux deals with multithreaded multiprogramming is sub-optimal, we conclude that there is a need to share additional information between the applications present in the system in order to get better performance. The preliminary experiments on OpenMP applications running on the TILEPro64 were promising. We conclude that in- formation sharing techniques, similar to the technique used in the Extended Lowest Load (XLL) strategy can improve the turnaround times in a multiprogrammed system. This led us to develop this idea inside the GPRM runtime system, denoted by GPRM Global Sharing.

62

Chapter 5

GPRM: The Glasgow Parallel

Reduction Machine

McCool et al. [126] emphasis that none of the most well-known programming languages used today were inherently designed for parallel programming. They also list three desired features for the parallel programming models that intend to enable parallelism: I) Perfor- mance, II) Productivity and III) Portability.

It should be possible to predict good performance, tune it, and scale it to larger systems. Pro- ductivity is not only about expressiveness and composability, but also about maintainability. Supporting a range of targets and operating systems is another desirable property, known as portability.

The Glasgow Parallel Reduction Machine (GPRM) [26] has its origin in Gannet, a system for designing NoC-based SoCs [150] [151]. Gannet was originally designed as a distributed reconfigurable SoC architecture based on the “processing core as a service” paradigm, i.e. a network of services offered by software or hardware cores. The Gannet SoC performs tasks by executing functional task description programs on the Gannet machine. The Gannet sys- tem was based on message passing without shared memory support and the Gannet codebase was intended for a static system with different tasks at each node.

The main contribution of this work is to change the existing Gannet framework [150] to work on shared memory environments. This is mainly achieved by sending data pointers (rather than the data itself) between nodes. Gannet was a static system with different tasks at each node. I had to change the task distribution mechanism such that any task can run on any core. A major contribution is adding an efficient stealing mechanism that matches the execution model of the new framework. Adding support for loop-level parallelism as well as designing a high level coordination language -GPC- are other contributions. Moreover, new APIs, such as parallel loops and parallel lists as well as a new information sharing mechanism have been developed. None of these steps could be finalised without measuring their impact on

63

performance. Therefore, an incremental iterative approach is used for the development of the framework’s components.

GPRM 1 borrows some fundamental concepts such as parallel reduction and intermediate functional representation from Gannet, but has a completely different focus and purpose. The implementation of tiles and their internal states in GPRM has been changed to provide a low- overhead task stealing mechanism for shared memory architectures with per-core caches. GPRM provides a task-based approach to manycore programming by structuring programs into task code, written as C++ classes, and communication code, written in GPC, a restricted subset of C++. The communication language (GPC) describes how the tasks interact using a functional language semantics with parallel evaluation, but with a C++ syntactic veneer. What this means is that it is possible to compile task code and GPC communication code with a C++ compiler and get correct functionality, but without the parallelism.

We use a partial evaluator to transform a GPC code with the specified number of tasks (as its static data) into an Intermediate Representation (IR), called GPIR. GPIR (the task description code) is based on S-expressions that first appeared in the Lisp programming language [152], e.g. (S1(S210) 20) represents a task S1 taking two arguments, the first argument is the

task S2 which takes as argument the numeric constant 10, and the second argument is the

numeric constant 20. GPIR is further compiled into lists of bytecodes, which the GPRM virtual machine (runtime system) executes with concurrent evaluation of function arguments. In other words, the GPRM virtual machine is a coarse-grained parallel reduction machine where the methods provided by the task code constitute the instructions.

The aim of GPRM is to abstract away all the details of threads, such that the users do not need to decide even about the number of them. This is an important issue, as finding the optimal number of threads has always been the most important concern. We contend that by efficient scheduling of tasks, the number of tasks should be the only important factor af- fecting performance, because the tasks are the actual computations, and the threads are only their substrates. The reason why GPRM is the most suitable model to verify our hypothesis is that it is completely task-centric. There is no extra overhead of thread migration, and the user deals only with tasks. GPRM task scheduling combines compile-time (source to in- termediate representation) and runtime (stealing) techniques to provide better performance. In other words, compile-time decisions form the initial distribution of tasks and the runtime system adjusts dynamically. As a result, some threads can be asleep during the execution, which means that the number of active threads are tuned automatically.

Probably, one similarity between GPRM and MPI is that they both implement a message passing environment. However, apart from the fact that the current version of GPRM only supports shared memory architectures and basically only sends pointers, the difference from

5.1. GPRM Architecture 64

the programming perspective is that the users do not deal with the messages in GPRM. Reference packets are generated and dispatched, based on the information obtained from the bytecode. Once the processing results become ready, the callee sends back the results to the caller.

5.1

GPRM Architecture

At the start of the execution, GPRM creates a pool of POSIX threads equal in size to the number of available cores (or hardware threads) in the underlying hardware. Each thread runs a tile, which consists of a task manager (reduction engine) and a task kernel (Fig. 5.1). The task kernel is typically a self-contained entity offering a specific functionality to the system, and on its own is not aware of the rest of the system. The task kernel has run-to- completion semantics. The corresponding task manager provides an interface between the kernel and other tiles in the system.

Since threads in GPRM correspond to execution resources, for each processing core there is a thread with its own task manager. The GPRM system is conceptually built as a network of communicating sequential tiles that exchange packets to request computations and deliver results. In other words, the combined operations of all reduction engines (task managers) in all threads results in the parallel reduction of the entire program.

Figure 5.1: GPRM Architecture: Task Kernel (TK), Task Manager or Reduction Engine (RE) and FIFOs

At first glance, the GPRM model may seem static and intolerant to runtime changes. How- ever as stated, we discuss how it can efficiently balance the load at runtime and thrive in dynamic environments. For load balancing, compile-time information about the task depen- dencies is combined with an efficient task-stealing mechanism. Moreover, by keeping track of the applications that reside simultaneously in the system, GPRM improves the perfor- mance of those applications markedly.