Different Kinds Of Profiling

There are four different kinds of profiling that can be undertaken in functional programming environments:

i) Program profiling. Measurements relate to the program’s behaviour and are reported with respect to functions in the source code. This is lexical profiling.

ii) Expression/Closure profiling. This is similar to the earlier cost experiment at UCL [Parrott90] and the old Glasgow Cost Centres [Sansom92]. In expression/closure profiling, measurements are based on how the program executes and the results are reported when an expression is evaluated. This is dynamic profiling.

iii) Abstract machine profiling. This measures how effective an abstract machine is by examining the overheads of function calls, function returns, heap management, garbage collection, etc [Hammond91a].

iv) Task profiling. This is particularly relevant in parallel environments where programs are divided into tasks which execute on separate machines. The number and size of the tasks are reported [Parrott92].

In [Runciman90], Runciman and Wakeling provide a good overview of the problems associated with profiling functional programs. They make several suggestions regarding the sorts of information that would be useful to a programmer and provide a more detailed analysis of how such information might be collected. Later in the chapter there is a summary of the issues listed in [Runciman90] and how the lexical profiler addresses these issues.

The research in this thesis has indicated that it is not clear to everyone in the field of functional programming that these different kinds of profiling can be usefully measured separately. Many people in functional programming who are doing measurement are

implementors interested in low level details. They wish to measure when work is done and at what point an expression is evaluated and to observe the effect that lazy evaluation has had on a program. This gives very different results from lexical profiling, which is dissociated from when work happens. Lexical profiling measures whether work happens and how much happens, with results being presented with respect to the source code. The difference is mainly in the way in which lazy evaluation has an observable effect on the program.

The following examples show the difference in the results between dynamic and lexical profiling. Consider the following programs:

Program 1

Program 2

Program 3

Although these three programs are similar, they differ where the expression x is declared and evaluated. Table 5.1 shows the number of primitive operations counted for the functions in each program using both lexical profiling and dynamic profiling. The term equates to the number of primitive operations required to evaluate x . The results of the lexical profiler always show the cost of x being associated with the function in which x is lexically contained. The results of the dynamic profiler highlight the presence and effect of laziness, and the cost of x is associated with the function that required the value of x .

Program

Function in which x is

Number of primitive operations lexical profile dynamic profile

declared reduced f _g h f _g h

1 f h ₁ 1 1 1 1

2 _g h 1 ₁ 1 1 1

3 f _{g 1 +} 1 1 1 ₁ 1

Px is the number of primitive operations

Table 5.1: How the cost o f primitives is attributed by lexical and dynamic profiling

Although most profilers do not count primitive operations as a statistic, these examples highlight the differences in the two styles. Moreover, they indicate that in order to fully appreciate how a program is evaluating, both profilers can be used together to provide a comprehensive view.

Consider another example in which dynamic profiling may give differing results but lexical profiling will give a consistent result. In the program:

f = (g X ) + ( h X )

w h e re x = e x p r e s s io n g X = h X * 10

h X = X + 32

the evaluation order of the primitive + is important. If the evaluation order of + is left to right, then a dynamic profiler will credit g with the evaluation of x , but if the evaluation order of + is right to left, then h will be credited with the evaluation of x . In a parallel system where the load balance and evaluation order are non-deterministic, a dynamic profiler may return different results on different occasions. Lexical profilers do not suffer from either of these problems as results are associated with lexical scope. This provides a static relationship between the source code and the run-time results.

A further advantage of lexical profiling is that because the results are dependent on the source code, it is possible to change the underlying evaluation mechanism and A L W A Y S have meaningful results. As there is not a strong relationship between the source code of a functional program and its evaluation mechanism, one could, for example, replace a graph reducer with a Term Rewriting System [Glauert90]. The results of the lexical profiler would still be associated with the source code. A dynamic profiler for a Term Rewriting System may give very different results and may not fit the model of evaluation that the programmer has. Therefore, with lexical profiling the programmer gets meaningful profiling data for his program regardless of the evaluation mechanism, but data from dynamic profiling is always dependent on the evaluation mechanism.