Using Strategies for Parallel Paradigms

This section demonstrates how strategies can be used for indicating parallelism in different parallel paradigms. We will first consider the divide-and-conquer (Section 4.1.4.1) and data-oriented (Section 4.1.4.2) paradigms.

let x = fib (n-1) y = fib (n-2) in x ‘par‘ (y ‘pseq‘ x + y + 1) let x = fib (n-1) y = fib (n-2) in runEval $ do x <- rpar (fib (n-1)) y <- rseq (fib (n-2)) return (x + y + 1) Figure 18: Original Strategies versus New Strategies

4.1.4.1 Task Parallelism

The strict identity monad gives a convenient and flexible notation for expressing evaluation order, i.e. the ordering between applications of rseq and rpar, which is exactly what a programmer needs for expressing basic parallel evaluation [77]. For example, the left hand side of Figure 18 presents a fragment of the parallel Fibonacci function (fib) written using the original par and pseq operators to indicate that x should be evaluated in parallel and guarantee that y is evaluated before the expression. Finally the result (x+y+1) should be returned. The right- hand side presents parallel code of the same function using rpar and rseq. The latter clearly expresses the ordering between rpar and rseq, using a notation that Haskell programmers will find familiar.

payne :: Int -> [(Int,Int)] -> Int

payne 0 coins = 1

payne _ [] = 0

payne val ((c,q):coins)

| c > val = payne val coins | otherwise = (left + right)

‘using‘ strat

where

left = payne (val - c) coins’ right = payne val coins

strat = rnf left ‘par‘ rnf right

coins’

| q == 1 = coins

| otherwise =(c,q-1):coins

Figure 19: Coins Using Original Strategy

payne :: Int -> [(Int,Int)] -> Int

payne 0 coins = 1

payne _ [] = 0

payne val ((c,q):coins)

| c > val = payne val coins | otherwise = res

where

left = payne (val - c) coins’ right = payne val coins

res = runEval $ do l <- rpar left r <- rseq right return (l+r) coins’ | q == 1 = coins | otherwise =(c,q-1):coins

Figure 20: Coins Using New Strategy

can be used is the simple Coins program. The Coins program takes a price and a list representing a set of coins, and determines how many different combinations of coins could be used to pay for an object at the given price.

We start with an original strategy parallel version for the main function in the program (shown in Figure 19). The function just returns the number of results, not the results themselves. If the value of coin (c) is greater than the value (val), then do not pay with this coin. Otherwise, the left branch detects the coin from the value and the right branch computes the number of coins.

Note that the expressions left and right can be evaluated in parallel. The red lines in Figures 19 and 20 describe how left and right branches can be evaluated in parallel without affecting the final result. The left branch is sparked

with rpar construct and bound to the variable l; while the right branch is evaluated sequentially using rseq construct and bound to the variable r.

The examples above explain how a programmer can express parallelism using rpar and rseq. However, a programmer can express parallelism in a smarter way, by using strategies. For example, a fib program can be rewritten in term of strategies, as follows.

fib n = res ‘using‘ strat where

x = fib (n-1) y = fib (n-2) res = (x+ y+1) strat res = do

x’ <- (rpar ‘dot‘ rdeepseq) x y’ <- (rseq ‘dot‘ rdeepseq) y return (x’+ y’+1)

The strat describes how two subcomputations x and y should be evaluated.

A Divide-and-Conquer Pattern: the main power of strategies is the possi- bility of building abstractions over patterns of parallel computation. One way of such abstraction is to define high level functions that can be parameterised to exploit parallelism, commonly known as algorithmic skeletons [32]. A divide-and- conquer skeleton is shown in Figure 21. All coordination aspects of the function are encoded in the strategy strat, which describes how the two subcomputations l and r should be evaluated. The thresholding predicate threshold, provided by the caller, places a bound on the depth of parallelism, and this is used by strat

(b -> b -> b) -> (a -> Bool) -> (a -> (a,a)) -> b divConq f arg threshold conquer divisible divide

| not (divisible arg) = f arg | otherwise = conquer l r

where

(lt,rt) = divide arg

left = divConq f lt threshold conquer divisible divide right = divConq f rt threshold conquer divisible divide (l,r) = (left, right) ‘using‘ strat

strat (l,r)

| (threshold arg) = (evalTuple2 rseq rseq) $(l,r) | otherwise =

(evalTuple2 (rpar ‘dot‘ rseq) (rpar ‘dot‘ rseq)) $ (l,r)

Figure 21: Divide-and-Conquer Skeleton

to decide whether to spark both l and r or to evaluate them sequentially. The evalTuple2 is a strategy that evaluates a tuple according to a given strategy. The definition of diverging achieves a separation between the specifications of the algorithm and the parallelism, the latter being confined entirely to the definition of strat.

4.1.4.2 Data-oriented Parallelism

Data parallelism can be classified into: flat data parallelism and nested data parallelism. A flat data parallelism applies a function to a collection of elements. In this case, it is not appropriate to spawn a thread for each element. Because in most cases it leads to very tiny threads that do not offset their creation, scheduling and other overheads. The appropriate way to parallelise this class of problem is to chunk the collection into suitable blocks, depending on the available processors and the computational size of each element. Nested data parallelism applies a

function over a collection of elements but each of these elements may applies another function over another data structure. Thus, it is irregular parallelism where each task may take a different time.

In strategies, parallelism is expressed by applying a higher order function to collections of data. The map function that applies a function over a list of elements can be parallelised as follows:

parMap f xs = map f xs ‘using‘ parList rdeepseq

The benefits of the strategies are not only the separation of the algorithm from the parallelism, but also the reuse of the map function. The parList function is a strategy that applies f to each element in the list in parallel. It defined as following:

parList :: Strategy a -> Strategy [a] parList strat [] = []

parList strat (x:xs) = do

x’ <- rpar ( x ‘using‘ strat) xs’ <- parList strat xs

return(x’:xs’)

The rpar creates a spark to evaluate the current element. The spark evaluates ( x ‘using‘ strat) which applies a strategy strat to x.

A more advanced strategy like parBuffer n s xs yields a list in which eval- uation of the i th element induces parallel evaluation of the (i + n)th element with the first n elements being evaluated in parallel immediately.

parBuffer :: Int -> Strategy a -> Strategy [a] parBuffer n s = evalBuffer n (rpar dot s)