3·2·3 Solutions

In order to provide effective speculation, information about the relative importance must be known. Further, the difficulties outlined in the previous section must be overcome. To cope with these issues the following strategies should be adopted (and, apart from exception detection, have been implemented):

• threads should be assigned speculative priorities, with the extremes of this priority range irrelevant and mandatory — a priority-based scheduler will ensure the

evaluation of the most important threads at all times;

• a thread’s speculative priority should be variable — it should be possible to upgrade and downgrade a thread’s priority to reflect new information regarding the relationship between that thread’s result and the program outcome; and • exception detection should be possible without aborting the entire script

evaluation.

Priorities could be assigned so that (for example) mandatory threads have a higher priority than speculative threads spawned by a mandatory thread. These in turn could have a higher priority than speculative threads spawned by a speculative thread, and so on. Lastly, those threads suspected to be irrelevant could have the lowest priority assignable so that they are never scheduled.

To illustrate how these strategies would work, consider the example function definitions in Figure 3·1. The main program, main, is defined as the application of the function f

Chapter Three: Speculative Evaluation conditional expression. The antecedent must be executed with the same priority as the function application, i.e. the evaluation of (p x) is mandatory. It will return either Trueor Falsethus setting either the consequent, (g x), or the contraconsequent, (h y), to mandatory and the other to irrelevant. In the meantime, however, both may be

speculatively evaluated. Hence (g x) and (h x) are assigned the highest speculative

priority. Taking this one step further, the antecedent of g, (q x), must be executed if a

result for (g x) is to be found; hence it too must have the highest speculative priority.

But only one of (j x) and (k x) will be required although both could be executed

speculatively — but not with the same priority as the antecedent (q x) since one of

these is certain to be irrelevant. Therefore the second highest speculative priority is initially assigned to (j x) and (k x).

f x y = if (p x) then g x else h y g x = if (q x) then j x else k x main = f 5 7

Figure 3·1: Example functional code.

3·3 Priority Schemes

3·3·1 Preliminary Definitions

In this section four priority schemes are presented. This is the first time these schemes have been collected together and described. The Probability, Percentiles, and

Black-White schemes are novel contributions.

In order to discuss priority schemes, the following formalities need to be presented. Let thread ti contribute unequivocally to the program output. Then thread ti is assigned a priority pm and is a mandatory thread.

Let thread tj contribute nothing to the program output. Then thread tj is assigned a priority p_r and is an irrelevant thread.

Between these two extremes lie speculative threads. A speculative thread is one that is as yet not known to be either mandatory or irrelevant. Let S be the set of speculative priorities and let P be the set of complete thread priorities. Then P=S

∪

The runtime context and frequency of execution of an annotated expression in the user’s program is not usually known. Earlier [Dermoudy 1996b] the notion of absolute priorities was investigated. Under this approach an executing speculative thread is awarded the absolute priority recorded in the annotation — regardless of executable context — and this priority is not able to be changed. This concept was shown to be paradoxical and the idea of absolute priorities has been abandoned.

An expression does not execute in isolation of its context. Consider the function definitions given earlier in Figure 3·1. It is clear that the function main is mandatory

and hence that evaluation of the function f is mandatory. It follows that evaluation of

the predicate p is mandatory and that, at best, execution of the functions g and h are

speculative. If the consequent of f, (g x) was annotated with a mandatory priority it

would compete directly with the predicate p which would be desirable. It is possible

that the outcome of the function f does depend directly on the outcome of the function

application (g x), but the priority of the function application(g x) must be relative to

the priority of f and cannot exceed it.

To re-state this idea, the speculative priority of a sub-expression depends upon the context of its ‘parent’ expression. If the expression

if p then x else y

is being executed at some speculative priority psi then the predicate p should also be executed at priority psi as it is crucial to the evaluation of the expression. x and y should be assigned a different priority, psj(psj<psi) until such time as their relevance is known. If

p returns the Boolean value True then x should be assigned priority p_si while y should

be given priority pr. (The converse is true if p returns the Boolean value False.) Priorities are thus a continuum: pm>ps1>ps2>

…

Note that while it is possible for the set of priorities P to be ternary in nature (consisting of only pm, ps, and pr) this will not provide optimal performance. The consequence of such a priority set is the equal competition between all speculative threads — even those that depend on others with which they are competing. A trade-off exists between the reduction of competition and the increased overhead of thread queue management due to a higher number of speculative priorities. The break-even point depends upon the amount of (nested) concurrency within the user’s program.

Chapter Three: Speculative Evaluation There are two opposing regimes for using annotations: quantitative schemes (in which the value of the annotation is specified with some accuracy that allows it to be used arithmetically in combination with other annotations) and qualitative schemes (in which the value of the annotation is symbolic of its logical relationship with other annotations, but arithmetic operations are inappropriate).

Four distinct schemes for indicating and managing priorities have been identified. These will now be presented.