The Double Vision Problem

Having considered the well-formedness of recursively dependent signatures, let us now consider the well-formedness of recursive modules. A natural typing rule forrec(X : S.M) would be

Γ,X↑S`M :κ S Γ`rec(X : S.M) :κ S

which checks that the recursive module body matches its declared signature. The new context binding form X↑S is needed because recursive variables are compiled differently from ordinary variables. In particular, X is represented not as a value of signature S, but as a memory location that will eventually be backpatched with a value of signature S. Correspondingly, references to X in M are not values either—they are computations that must implicitly check whether X has been backpatched and, if so, return its contents.

3

In addition, adatatypespecification implicitly specifies a function that enables a value of typetto be pattern- matched. The interpretation of MLdatatype specifications presented here, which is based on Harper and Stone’s treatment of SML semantics, will be formalized explicitly in the language definition of Chapter 9.

rec (X : EXPR BIND. let structure Body =

struct

structure Expr :> sig type t ... end = struct

datatype t = VarExpr of var | LetExpr of X.Bind.t * t | ... ...

fun makeLetExpr (b : X.Bind.t , e : t) : t = LetExpr(b,e) ...

fun eval (e : t) : t = case e of ...

| LetExpr (b,e) =>

let val (vs,es) = List.unzip (X.Bind.eval b) (* vs : var list, es : X.Expr.t list *)

in ...

end ...

end

structure Bind :> sig type t ... end = ... end

in Body :> EXPR BIND

end)

Figure 5.10: The Double Vision Problem Arising in Exprand Bind

Unfortunately, this rule is too simple—it fails to accept many recursive modules whose declared signatures containopaque type specifications, including both theExprBindexample and the boot- strapped heap example from Section 5.1. Consider the code excerpt from the recursive module defining Expr and Bind, shown in Figure 5.10. In this version, the recursive module body is let- bound to a variableBody, which is then sealed with the declared rdsEXPR BIND. I have written the example this way primarily so that we have explicit names for the types defined in the body, e.g.,

Body.Expr.t and Body.Bind.t. The declared signature EXPR BIND is meant to stand for either

one of the rds’s given in Figures 5.8 and 5.9. Whichever rds is used, the type specifications in it are opaque. Consequently, when typechecking the recursive module body, there is no way to connect the abstract type components of the recursive variable X—namely,X.Expr.tand X.Bind.t—with the corresponding type components ofBody. This results in several serious typing difficulties.

In order to understand these typing difficulties, let us first clarify what the typing rule actually requires. Assuming that the rds EXPR BIND has the form ρX.S, the typing rule says that Body must match ρX.S, which means in turn that Body must match the signature S[Body/X].

The first problem is that the Body.Expr.makeLetExpr function does not match its required type. This function constructs a let expression of type Body.Expr.t from a binding of type

X.Bind.tand an expression of typeBody.Expr.t. However, in order forBodyto match S[Body/X],

the first argument ofBody.Expr.makeLetExprmust have typeBody.Bind.t, notX.Bind.t. There is no simple way of addressing this problem—at the point where makeLetExpris defined, the type

Body.Bind.tdoes not even exist yet! One might suggest switching the order ofExpr andBind, so

that Expr can refer to Bind directly, but then the same problem would rear its head again when matchingBody.Bind.makeValBindagainstits required type.

5.2. KEY ISSUES IN THE DESIGN OF A RECURSIVE MODULE EXTENSION ₉₉

A second, more subtle problem arises in typechecking the body of the Expr.eval function. When the input to this function is of the form LetExpr(b,e), the function makes a recursive call to X.Bind.eval in order to process the binding b. The return type of X.Bind.eval is

(var * X.Expr.t) list; this list is then unzipped into a list (vs) of the variables bound by b,

and a list (es) of the value expressions to which they are bound. That the expressions in es have

type X.Expr.tis problematic for several reasons.

For one, the implementation ofevalmay want to deconstruct these expressions. Since they have

typeX.Expr.t, however, the only way to deconstruct them is to call thematchfunctions provided by

X.Expr, which is not as efficient or convenient as case-analyzing a value of thedatatype tdirectly.

Moreover, this problemforces theExpr module to provide deconstructor functions (or else expose

thedatatypedefinition oft) in its interface, regardless of whether it otherwise needs to.

In addition, suppose that the body e of the input expression LetExpr(b,e) has the form

VarExpr(v), wherevis one of the variables bound inb. Presumably, in this case, theevalfunction

should return the value expression in es that corresponds to v. Yet, the type of that expression will be X.Expr.t, whereas the required return type of eval is Body.Expr.t. The interface of X does not provide any way to coerce a value from X.Expr.tto Body.Expr.tor vice versa.

All of these typing difficulties are symptoms of what I call the “double vision problem.” This problem is easiest to understand by thinking ofExprandBindas being written, respectively, by two “agents” (or “principals”) Alice and Bob [24]. Alice knows that the typeExpr.tis implemented by

thedatatypedefinition shown in Figure 5.10, but Bob does not know this because it is not revealed

in the interface for Expr that Alice provides. Similarly, Bob knows how Bind.t is implemented, but Alice does not.

In order to allow recursive references between the two modules, we have introduced the recursive variableX. Intuitively, sinceXwill ultimately be backpatched with the result of evaluating the body, what either agent knows about the type components of X should coincide with what she knows about the definitions of the corresponding type components in the body. Thus, Alice should be allowed to know thatX.Expr.tis implemented in the same way that she has implementedExpr.t in the body, and Bob should be allowed to know that X.Bind.t is implemented in the same way that he has implemented Bind.tin the body. Neither agent, however, should be allowed to know how the other agent’s submodule of Xis implemented. In addition, since Bob can refer to Expr.t in two ways—either directly or throughX—he should be able to observe that the two typesExpr.t

and X.Expr.tcoincide, without knowing what their underlying definition is.

The double vision problem is that the simple typing rule given at the beginning of this section fails to realize this intuition. Alice and Bob are shown the same interface for the recursive variable. If Alice wants to hide the identity of X.Expr.tfrom Bob, she must also hide it from herself. As a result, she “sees double”—that is, she sees X.Expr.tas being distinct from her own definition of

Expr.t, and she cannot tell that they are really one and the same type. Bob also sees two versions

of Expr.t, although he is not privy to a definition for either type.

One way to keep the simple typing rule and avoid double vision is to require that the declared signature of the recursive module be transparent. For example, ifX.Expr.twere revealed in the de- clared signature to equal some type C, then Alice would be able to observe thatX.Expr.tcoincides with its implementation C. Of course, Bob would be able to observe this as well. Transparency addresses the double vision problem, but at the expense of preventingExpr and Bind from hiding type information from each other. Furthermore, this solution assumes that the programmer isable

to write a transparent declared signature. In the case ofExpr andBind, the best that we can do is modifyEXPR BINDso that it exposes the datatypedefinitions of Expr.tand Bind.t. According to ML semantics, though,datatypespecifications are opaque.

functor F Expr (X : EXPR BIND) = ... ...

functor F Bind (X : EXPR BIND) = ... ...

structure Link = rec (X : EXPR BIND. struct

structure Expr = F Expr(X) structure Bind = F Bind(X) end

Figure 5.11: Attempted Separate Compilation of Expr and Bind

The double vision problem is one of the most serious hurdles to overcome in designing a recursive module extension. In Section 5.3, I will discuss the conservative ways in which the existing recursive module proposals deal with it. A key contribution of my own recursive module design, which I describe in Section 5.4, is to provide a more general and effective cure for double vision.