Parallel Control-flow Graph (ParCFG)

1 #pragma omp parallel for

2 for(int idx = 1; idx < N; ++idx) {

3 norm[idx] = vals[idx] / norm(vals, N);

4 }

Adding parallelism to the compiler IR, and thus making it explicit to the compiler, is important to enable optimizations in the presence of parallelism. Imagine, for in-

stance, the OpenMP parallel for loop to the right. Such a construct is typically trans- formed by the compiler frontend and completely invisible to the optimizing middle and backend. Typically, the body of the parallel loop is externalized to a separate function, and the actual loop replaced by a call to the corresponding runtime system. This is called early proceduralization and nearly prohibits all important compiler optimizations, like for instance loop invariant code motion.

But full integration of parallelism is not only a matter of optimizations, and thus perfor- mance. It is also a matter of correctness. In order to keep the engineering overhead as low as possible it has been frequently proposed to integrate parallelism via early proceduraliza- tion, as mentioned above, or by adding meta data or compiler intrinsics to the IR which marks regions of possible parallel execution, but might be ignored by optimizations not aware of the semantics of it. The main motivation of such proposals has been to minimize the necessity to adapt every optimization in order to make it aware of the parallelism, which would be invasive and error-prone. Unfortunately, parallelism is a very invasive concept, as can be seen in Figure 4.7.

Figure 4.7a shows a code region with parallel tasks marked by compiler intrinsics par- allel.task.start(<id>) and parallel.task.end(<id>). Whether we choose intrinsics, meta data or similar minimally invasive constructs to mark parallelism is of minor technical importance. Important however is that an optimization like common subexpression elimi- nation (CSE ), unaware of the parallel semantics, is allowed to produce the result shown in Figure 4.7b. In the example, it is allowed to do so because the standard dominance information, on which many optimizations rely, is not correct: despite the fact that the

1 parallel.task.start(1) 2 3 some_fun(n/2) 4 parallel.task.end(1) 5 ... 6 parallel.task.start(2) 7 other_fun(n/2) 8 parallel.task.end(2) (a) 1 parallel.task.start(1) 2 tmp = n/2 3 some_fun(tmp) 4 parallel.task.end(1) 5 ... 6 parallel.task.start(2) 7 other_fun(tmp) 8 parallel.task.end(2) (b)

Figure 4.7: Illegal transformation due to insufficient integration of parallelism into the compiler IR.

two program parts which are supposed to run in parallel are necessarily written down in a sequential order, the implicit parallel semantics dictates that none of both regions precedes the other. Consequently, none of both regions dominates the other and moving a commonly used subexpression into either of both regions, as done in Figure 4.7b produces a wrong result.

The parallel control-flow graph (ParCFG) resembles the structure of the CFG and adds constructs of parallel execution. This means that we distinguish between sequential and parallel control flow and corresponding edges. Parallelism is introduced into the ParCFG by fork instructions, called πs. πs instructions form the entrance to a parallel section

which in turn is terminated by a join instruction π_e. A parallel section consists of multiple parallel tasks, which contain parts of the code that can potentially be executed in parallel.

The control-flow edges originating in a π_s represent parallel control-flow and end in the first basic block, the so-called head, of a parallel task. A πsis typically succeeded by many

parallel tasks of its containing parallel section. It has exactly as many outgoing edges as the containing section tasks.

Parallel tasks in their basic form are single entry single exit regions (called SESE-Regions or Hammocks in the literature). They are terminated by the one unique πe instruction of

their containing parallel section. The π_e joins, or synchronizes, parallel execution.

Figure 4.8 shows the ParCFG of the performTask method as it is produced by the static part of ParA_τ’s parallelization.

size

πs

call makeList call makeList

call hashList call hashList

call freeList call freeList

πe

mul

ret

Figure 4.8: ParCFG for parallel version P0of the performTask method as automatically

derived by Sambamba. The outer box depicts a so-called parallel region consisting of transactions depicted by the inner boxes. Each parallel region is entered via at least one

πsnode and left via the one πe, which is unique per region. A πsforks parallel execution

and πejoins again after all contained transactions completed.

Parallel control-flow graphs do form an intermediate representation which makes the static parallelism of a function explicit. They are independent of the form of execution, being it parallel or sequential, chosen later on during code generation. Should parallel code be generated, a πs would result in code which produces the necessary communication and

synchronization primitives at runtime and starts parallel execution. The corresponding π_e would be replaced by some form of barrier or synchronization. A simple and straight forward possibility would be to produce a cilk [41] (or Cilk+ [42]) style program: each parallel task t of a parallel section is extracted into a function ft with parameters corresponding

to values used by but not produced within t. The πs is replaced by a number of spawn

instructions potentially spawning all tasks of the section, the πe is replaced by a sync

instruction. It would be as easy to produce an OpenMP [6, 7] or Intel TBB [40] based parallel version, as is to linearize the tasks to produce a sequential version.

The ParCFG forms an essential building block to the modularity and extensibility of Sambamba, which provides the infrastructure for final code generation. Parallelization modules based on Sambamba (like ParAτ and ParAγ) can concentrate on finding the

parallelism inherent in the application and to produce ParCFGs where appropriate. Sambamba provides the infrastructure to generate the parallel code.

Furthermore, ParAτ only profits from a subset of the features of the ParCFG. As Chapter 5

describes, the simple fork/join based parallelism of the ParCFG is sufficient to encode a large range of different forms of parallelism, including many loop parallelization schemes. Recently, and supporting our own efforts, Schardl et al. [96] (also [97]) proposed an extension of the LLVM IR named TAPIR which is based on three constructs introducing parallelism into the IR: detach, reattach and sync, which are of a similar expressive power than the ParCFG parallel sections with their πs and πe constructs. The findings of this

thesis motivated and influenced a group around Johannes Doerfert at Saarland University who is working together with the authors of TAPIR at MIT to introduce proper parallelism constructs, including a well-defined parallel semantics, to the LLVM IR. An important feature of this new IR, which the ParCFG does not fulfill, is the full compatibility to existing analyses and transformations, while preserving correctness of analyses which are unaware of the parallel semantics.