In this chapter we investigate some aspects of computability pertaining torunning times, i.e. the number of steps that computations take. Two aspects are given special atten- tion: execution of metaprograms, i.e. compilers, interpreters, and specializers, andself- application, e.g. application of a program to itself, and in particular a specializer.
The main purpose of this chapter is not to prove new results in computability theory (although the Futamura projections may be new to some theorists.) Rather, our main aim is to link the perhaps dry framework and results of this book’s material through Chapter 5 to daily computing practice.
This involves relating the time usage of compilation and interpretation; the deleterious effects of multiple levels of interpretation; the use of “bootstrapping” (a form of self- application) in compiling practice to gain flexibility and speed. Last but not least, the Futamura projections show how, using a specializer and an interpreter, one maycompile,
generate compilers, and even generate a compiler generator, again by self-application. Interestingly, the Futamura projections work well in practice as well as in theory, though their practical application is not the subject of this book (see [89].)
Section 6.1 first introduces running times into the notion of a programming language arriving at a timed programming language. Section 6.2 is concerned with with inter- pretation. Section 6.3 describes self-application of compilers, and Section 6.4 introduces
partial evaluation, the well-developed practice of using program specialization for auto- matic program optimization. Section 6.5 shows how it can be applied to compiling and compiler generation, and discusses some efficiency issues, showing that self-application can actually lead to speedups rather than slowdowns.
The final two Sections (which readers focused more on theoretical issues may wish to skip) include 6.6 on pragmatically desirable properties of a specializer for practical applications; and Section 6.7, which sketches an offline algorithm for partial evaluation.
6.1
Timed programming languages
Definition 6.1.1 Atimed programming language Lconsists of 1. Two sets,L−programsandL−data;
2. A function [[•]]L:L−programs→(L−data→L−data⊥); and
3. A functiontimeL:L−programs→(L−data→IN
⊥) such that for anyp∈L−programs
andd∈L−data, [[p]]L(d) =⊥ifftimeL
p(d) =⊥.
The function in 2 isL’ssemantic function, which associates with everyp∈L−programs
a corresponding partial input-output function fromL-datato L-data. The function in 3 isL’srunning time function which associates with every program and input the number of steps that computation of the program applied to the input takes. 2 Much more will be said about program running times in the Complexity Theory parts of this book. In this chapter we discuss time aspects of interpretation, specialization etc. only informally, relying on the reader’s experience and intuition.
6.2
Interpretation overhead
In the first subsection we discuss overhead in practice, i.e. for existing interpreters, and the second subsection is concerned with self-application of interpreters. It will be seen that interpretation overhead can be substantial, and must be multiplied when one inter- preter is used to interpret another one.
Section 6.4 will show how this overhead can be removed (automatically), provided one has an efficient program specializer.
6.2.1
Interpretation overhead in practice
In the present and the next subsection, we are concerned with interpreters in practice, and therefore address the question: how fast can an interpreter be, i.e. what are thelower
bounds for the running time of practical interpreters. Suppose one has anS-interpreter intwritten in language L, i.e.
L S int∈
In practice, assuming one has both an L-machine and an S-machine at one’s disposal, interpretation often turns out to be rather slower than direct execution ofS-programs. If anS-machine is not available, a compiler fromStoLis often to be preferred because
the running time of programs compiled intoL (or a lower-level language) is faster than that of interpretively executedS-programs.
In practice, a typical interpreterint’s running time on inputspanddusually satisfies a relation
αp·timeS
p(d)≤timeLint(p.d)
for all d. Here αp is a “constant” independent ofd, but it may depend on the source
programp. Oftenαp=c+f(p), where constantcrepresents the time taken for “dispatch
on syntax” and f(p) represents the time for variable access. In experiments c is often around 10 for simple interpreters run on small source programs, and larger for more sophisticated interpreters. Clever use of data structures such as hash tables, binary trees, etc. can makeαpgrow slowly as a function ofp’s size.
6.2.2
Compiling (usually) gives faster execution than
interpretation
If the purpose is to executeS-programs, then it is nearly always better to compile than to interpret. One extreme: if S =L, then the identity is a correct compiling function and, lettingq = [[comp]](p) = p, one has timeS
p(d) =timeLq(d): considerably faster than
the above due to the absence of αp. Less trivially, even when S 6= L, execution of a compiledS-program is nearly always considerably faster than running the same program interpretively.
6.2.3
Layers of interpretation
Suppose a Lisp system (called L2) is processed interpretively by an interpreter written in Sun RISC machine code (call this L1). The machine code itself is processed by the central processor (call this L0) so two levels of interpretation are involved, as described in the interpreter diagram in Figure 6.1.
The major problem with implementing languages interpretively is that the running time of the interpreted program is be multiplied by the overhead occurring in the inter- preter’s basic cycle. This cost, of one level of interpretation, may well be an acceptable price to pay in order to have a powerful, expressive language (this was the case with Lisp since its beginnings). On the other hand, if one uses several layers of interpreters, each new level of interpretation multiplies the time by a significant constant factor, so
Two interpretation levels v L2 > intL2L1 v L1 > intL1L0 v L0 L2 ** L1 L2 L0 L1 Time consumption Nested Interpreter application 6
Figure 6.1: Interpretation overhead.
the total interpretive overhead may be excessive (also seen in practice). Compilation is clearly preferable to using several interpreters, each interpreting the next.
Indeed, suppose now that we are given
• An interpreterint10written inL0that implements languageL1; and
• An interpreterint21written inL1that implements languageL2.
where L0, L1, andL2 all have pairing and concrete syntax, and all have the same data language. By definition of an interpreter,
[[p2]]L2(d) = [[int2
1]]L1(p2.d) = [[int10]]L0(int21.(p2.d))
One can expect that, for appropriate constants α01, α12 and any L1-program p1, L2- programp2and data d,
α01·timeL1p1(d)≤timeL0int1 0
(p1.d) and
α12·timeL2p2(d)≤timeL1int2 1
(p2.d)
where α01, α12 are constants representing the overhead of the two interpreters (often sizable, as mentiond in the previous section).
Consequently replacing p1in the first by int21 and d byp2.d, and multiplying the second inequality byα01we obtain:
α01·timeL1int2 1 (p2.d)≤timeL0 int10(int 2 1.(p2.d)) α01·α12·timeL2p2(d)≤α01·timeL1int2
1 (p2.d)
Thusα01·α12·timeL2p2(d)≤timeL0int1 0
(int2
1.(p2.d)), confirming the multiplication of inter- pretive overheads.