To understand how we map from low-level instrumentation data to high-level derivation trees, we first compare them side-by-side in the instrumented code that type checks abstrac- tions (in Listing 5.4 and Listing 5.5) and function applications (in Listing 5.6 and Listing 5.7). We explain why a naive approach of a one-to-one mapping from the low-level instrumenta- tion to the high-level representation is inefficient in terms of polluting the code space of the compiler. The one-to-many mapping approach taken in this thesis allows us to reduce the number of instrumentation instructions and define high-level goals that hide the unneces- sary details of the implementation. Using a number of examples we illustrate that a one-to- many mapping comes at a cost - ambiguous definitions can lead to different valid mappings where in general we cannot select the most specific one.
Later in Section 5.4 we formally define the one-to-many translation. We show that with a small number of restrictions on the high-level representation we can define rules that non- ambiguously map to them from the low-level instructions representing the type checking.
Towards a high-level representation for type checking functions: An Example
Listing 5.4 presents a simplification of Scala’s actual implementation that assigns types to functions. In the example, thetypedFunctionmethod takes an argument of typeFunction (an AST node for functions), and an expected type of the function. The main instrumentation block (lines 1-12), that creates an instance of a low-level instrumentation classTypeFun, de-
5.3. Mapping between representations
1 def typedFunction(ast:Function, pt:Type): Function =EV.instrument(EV.TypeFun(ast, pt)){
2 val Function(params, body) = ast
3 val (paramsPt, resultPt) = decompose(pt) / parameters and the result type parameters
4 val params1 = (params zip paramsPt).map {
5 case (param, paramPt) => typedParam(param, paramPt) 6 }
7 val body1 = typecheckAst (body, resultPt)
8 ...
9 val ast1 = Function(params1, body1)
10 ... 11 ast1 12 }
13 def typedParam(param:ValDef, pt:Type): ValDef =EV.instrument(EV.TypeFunParam(param)){
14 ...
15 val param1 = typecheckAst (param, pt)
16 ... 17 param1 18 }
Figure 5.4: An example of the instrumented method that type checks ASTs of functions. The...part represents the irrelevant implementation details.
limits the logical block of type checker’s executions, and essentially specifies that any other in- vocation of the instrumentation in between is directly part of type checking the function AST. Similarly, any instrumentation invocation within the second instrumentation block (lines 13- 18), is part of the decision process that type checks the type of the parameter in the abstrac- tion. We recall that the previously defined type checking method,typecheckAst, is already instrumented (visually represented through a gray box around it) and does not have to be placed within the instrumentation block separately.
The method extracts the individual elements of theFunctionAST (parameters and the body of the function in line 2) and type elements of the prototype (line 3); both involve simple pattern matching on the result of the right hand side of the expression. Later, thetypedParam function is applied to the individual parameters of the function, along with their respective prototypes (lines 4-6), in order to determine the types of the parameters (thezipmethod combines the corresponding elements of the two collections into tuples). Importantly, the individual instrumentation blocks that track the typing of the parameters of the function, as well as the instrumentation block for typing the body of the function (line 7), are all direct dependencies of the first, main, instrumentation block.
Now that we’ve seen an example of an instrumentation when type checking functions, we look into the high-level class hierarchy, that can accurately represent it.
Listing 5.5 provides an example of a high-levelTypeFunclass that is a subclass of theTypeGoal. The initial verification of the parameters of the function is represented through theparams
Chapter 5. Lightweight extraction of type checker decisions
abstract class TypeFun extends TypeGoal { type U <: EV.TypeFun
def params: List[TypecheckParam]
def body: Typecheck
}
abstract class TypecheckParam extends Goal { type U <: EV.TypeFunParam
def tParam: Typecheck
}
Figure 5.5: The high-level class hierarchy for representing the typing decisions that infer the type of the function, similarly to the various(abs)rules in the Colored Local Type Inference formalization.
member, and the verification of the body of the abstraction is represented through thebody member. Importantly, the variable number of the possible parameters of the function is ex- pressed through the collection classListtype constructor. TheTypecheckParamhigh-level class requires a single type checking operation - the verification of the type of the parame- ter - in order to be satisfied. Finally, we notice that both high-level classes refine the upper bounds of the high-level classes in order to reflect the types of the low-level events they can represent.
In order to relate the two representations of the type checking, we return to the low-level instrumentation data. These low-level instrumentation events are essentially sequences of type checking events, with an additional hierarchy information gathered from the block de- limitations. By pattern matching on such instrumentation sequences, in a postfix fashion, the mapping groups the sequences into individual categories that correspond to the mem- bers of high-level classes.
Towards a high-level representation for function applications: An Example
Listing 5.6 presents a simplified view on the Scala’s implementation, and instrumentation, for a more involved example that types function applications. In the example, thetypedApplication method takes an argument of typeApply(an AST node for function applications), and an ex- pected type of typeType, and returns an AST with the inferred type of the function applica- tion, i.e., the AST has a value assigned to its type attribute.
The main instrumentation block (lines 2-8) is delimited by the instrumentation classesTypeApp andTypeAppDone. As a result, any instrumented typing decision, executed as part of the type checking function application, is essentially part of such instrumentation block. The already instrumentedtypecheckAstmethod, infers the type of the function, and its type checking decisions are part of the main instrumentation block as well.
The rest of listing 5.6, beginning in line 6, represents the logic that determines the type of the 134
5.3. Mapping between representations
1 def typedApplication(ast: Apply, pt: Type): Tree =
2 EV.instrument(EV.TypeApp(ast, pt), EV.TypeAppDone(_)) {
3 val Apply(funAst, argsAsts) = ast
4 val fun1 = typecheckAst (funAst, WildcardType)
5
6 if (fun1.tpe == ErrorType) typedApplicationFallback(ast, pt)
7 else assignAppType(fun1, argsAsts, pt) 8 }
9
10 def assignAppType(fun1: Tree, args: List[Tree], pt: Type): Tree = { 11 EV <<< TypeApp1(...)
12 val app1 = fun1.tpe match {
13 case MethodType(params, resultTpe) =>
14 val paramsTpes = // ...
15 val args1 = (args zip paramsTpes).map { case (arg, argPt) =>
16 typecheckAst (arg, argPt) }
17 if (hasError(args1)) ... else ...
18 case PolymorphicType(tParams,MethodType(params,resultType)) =>
19 val paramsTpes = // ...
20 val argsPt = argsPtFromResultPt(resultTpe, tparams, paramsTpes)
21 val args1 = (args zip argsPt).map { case (arg, argPt) =>
22 typecheckAst (arg, argPt) }
23 if (hasError(args1)) {
24 ... // fallback mechanism
25 } else {
26 ...
27 inferMethodInstance(args1, fun1, pt, paramsTpes)
28 }
29 case OverloadedType(_, alternatives) => 30 ... 31 } 32 EV >>> EV.Done 33 app1 34 } 35
36 def typedApplicationFallback(ast: Apply, pt: Type): Apply = 37 EV.instrument(EV.InvalidFunApp(...)) {
38 ... // A fallback type checking for an erroneous function type
39 }
40 def argsPtFromResultPt(resultTpe: Type,tparams: List[Symbol], 41 params: List[Type]): List[Type] =
42 EV.instrument(EV.InferTypeArguments(tparams, resultTpe, params), EV.InferredPt(_) {
43 ... // An opportunistic inference of type arguments
44 // from the result type of the prototype
45 }
46 def inferMethodInstance(args: List[Tree], fun: Tree, pt: Type, 47 paramsTpes: List[Type]): Tree = 48 EV.instrument(EV.InferMeth(args, paramsTpes, pt)) {
49 ... // Inference of type variable instantiations
50 }
Chapter 5. Lightweight extraction of type checker decisions
function application. The non-trivial implementation fragment defines type checking steps for different scenarios, as expressed by pattern matching on the inferred type of the func- tion term; a function type having some unresolved local type parameters (PolymorphicType, line 18), a monomorphic function type (MethodType, line 13), an overloaded type represent- ing multiple method alternatives (OverloadedType, line 29), and an erroneous function type that can possibly be adapted (ErrorType, line 6). Importantly, the different paths for type checking the function application will result in the instantiation of different instrumentation classes, and different sequences of low-level events within theEV.TypeAppandEV.TypeApp1 instrumentation blocks.
Now that we’ve seen an example of an instrumentation for a method that type checks func- tion applications, we look into the high-level class hierarchy, that can accurately represent such different execution paths.
Listing 5.7 defines a base classTypeAppfor representing the decisions that type check func- tion applications; the class refines an upper bound of the abstract type member in order to reflect the low-level instrumentation class it links to, and defines a required member,
typecheckFun. ThetypecheckFunmember and its type, determine a single operation that will always have to be executed for the application term - the inference of the type of the function. The subclasses of theTypeAppclass, i.e., theTypeAppFallbackandTypeAppCorrectclasses, correspond to different type checker executions involving erroneous and error-free results of type checking a function. The exposed fallback mechanism of the type checker does reveal some internal details of Scala’s type checking, but at the same time allows us to navigate through the type checking executions of the existing programs.
ClassTypeApplicationMain, which is listed as a type of the membertypeAppin theTypeApp- Correctclass, represents the base type checking decisions that assign the type to the func- tion application, given an error-free function type. The required type checking operations of the mutually exclusive executions paths are specified in the direct subclasses of theType- ApplicationMainclass, i.e., in theTypeAppMonomorphic,TypeAppPolymorphicandTypeApp- Overloadedclasses. The hierarchy of subclasses and their members directly reflects the type checking decisions when no type parameters are present, when some local type parameters have to be inferred or when we deal with multiple method alternatives, respectively. All the subclasses have a member namedtypecheckArgs, corresponding to type checking the argu- ments of the application, but the member itself is not shared through the inheritance. In our approach such member duplication is unavoidable, as the order of the declared mem- bers of the high-level classes has to accurately represent the order of the corresponding type checker’s decisions.
In theTypeAppPolymorphicclass, membertargsFromExpectedTypereveals the instantiation of type variables from the expected type of the function application, prior to type checking the arguments; such opportunistic instantiation of type variables is not formalized in any of the discussed formalizations. By instrumenting the existing compiler we can and have to 136
5.3. Mapping between representations
abstract class TypeApp extends TypeGoal {
type U <: EV.TypeApp def typecheckFun: Typecheck
}
abstract class TypeAppFallback extends TypeApp { def typeAdapted: Typecheck
... }
abstract class TypeAppCorrect extends TypeApp {
def typeApp: TypeApplicationMain
}
– – – – – – – – –
abstract class TypeApplicationMain extends Goal { type U <: EV.TypeApp1
}
abstract class TypeAppMonomorphic extends TypeApplicationMain {
def typecheckArgs: List[Typecheck]
}
abstract class TypeAppPolymorphic extends TypeApplicationMain { def targsFromExpectedType: InferTArgsFromPt
def typecheckArgs: List[Typecheck]
def inferInstance: InferMethodInstance
def typeAppCont: TypeAppMonomorphic
}
abstract class TypeAppOverloaded extends TypeApplicationMain {
def typecheckArgs: List[Typecheck]
def inferAlternative: InferMethodAlternative
def typeApp: TypeApplicationMain
}
Figure 5.7: A fragment of the high-level representation corresponding to type checker decisions necessary to type a function application, under different scenarios.
expose such decisions because they can affect the inferred type of function application (the member corresponds to the instrumentation of theargsPtFromResultPtmethod in Listing 5.6) and further type checking decisions. MemberinferInstanceserves as an entry point to the act of inferring minimal type parameter substitution, while thetypeAppContmember defines the type checking of the function application with instantiated type parameters. The type checking of overloaded methods is represented through theTypeAppOverloaded class; in theinferAlternativemember the type checker infers a single method alternative based on the types of the previously type checked arguments (thetypecheckArgsmember), the inherited expected type and the type of the alternatives, and then repeats the typing of
Chapter 5. Lightweight extraction of type checker decisions
function application, as indicated through the type of the membertypeApp.
In order to relate the low-level instrumentation events to this high-level representation, we consider the type checking executions for the two simple function applications:
val xs: List[Int] = //... xs.filter(x => x > 0) xs.map(x => x + 1)
Both applications manipulate the ‘xs’ collection of typeList[Int]by either removing the el- ements that do not satisfy the ‘> 0’ predicate, or incrementing all the elements of the collec- tion. The ‘filter’ method, as a member of theList[Int]collection has no local type parame- ters, i.e., using our formal notation the type of ‘xs.filter’ is (I nt→ Boolean) → LIST[I nt], while the type checking of the application involving ‘map’ method has to instantiate a local type parameter, i.e., the type of the ‘xs.map’ member selection is∀b.(Int → b) → LIST[b] in our formal notation. Different type signatures of the methods imply different type checker executions. For example, the first function applications will be defined by a context of a single high-level instance of type Typecheckrepresenting the type checking of the anony- mous function ‘x => x > 0’, while the second application will be defined by a sequence of high-level instances of types InferTArgsFromPt, Typecheck, InferMethodInstanceand TypeAppMonomorphic, due to the local type parameter instantiation.
Given the declaration of theTypeApplicationMainclass and its abstract type memberU, the low-level instance of theEV.TypeApp1event can be potentially mapped to three different sub- classes. The mapping of the low-levelEV.TypeApp1event is determined by pattern matching on the types of the instances that constitute the dependencies of the high-level event. The mapping, being a postfix operation, pattern matches on the already mapped dependencies against the types of the members of the high-level classes.
Ambiguous mappings
Our approach uses the types of the declared members of the high-level representation to drive the pattern matching process. While flexible, it can lead to the issue of unpredictable or ambiguous mappings. For example, a member having typeList[T]implies that zero or more type checking decisions of typeT(or a subtype of it) have occurred. The type opens the door for different interpretations of valid pattern matching strategies. Before we define restrictions that avoid the undesired or ambiguously looking high-level representations (Section 5.4), we discuss their examples first.
Ambiguous members within the same class definition
To illustrate one of the problems, Figure 5.8 provides a simpler and more intuitive represen- tation for typing functions than the one given in Figure 5.5. To match the high-level represen- 138
5.3. Mapping between representations
tation the low-level instrumentation would have to be modified. The instrumentation for the typedParammethod in Figure 5.4 involving the low-levelEV.TypeFunParamevent would have to be removed. Consequently, the instrumentation blocks that enclose the type checking of the parameters of the abstraction and its body are all part of the main instrumentation block of typeEV.TypeFun.
abstract class TypeFun extends TypeGoal { type U <: EV.TypeFun
def params: List[Typecheck]
def body: Typecheck
}
Figure 5.8: An ambiguous declaration of the high-level classes representing the type checking of functions.
For the purpose of the example, we assume the existence of three already mapped high-level instances, denoted as { x1, x2, x3}. The runtime type of each of the instances is some subtype
ofTypecheck, and the sequence will serve as a context for our postfix mapping strategy. The types and number of high-level goals offers different possible mappings for the members of the classTypeFun1:
• [params→ { x1, x2} , body→ { x3}] -
The mapping reflects that the compiler type checked two parameters of the function and then type checked the body of the function.
• [params→ { x1, x2, x3} , body→ ε] -
The mapping reflects that the compiler type checked three parameters of the function and the body of the function was not verified, i.e., the instrumentation block that en- closes the type checking of the body of the function was never executed. For example, a type mismatch for one of the type parameters could prevent the type checking of the body.
The lack of mapping for the memberbodycan be either blamed on the insufficient low- level instrumentation that omitted some execution, or lack of coverage of the high-level representation.
In the given example, it is not possible to determine the source of the ambiguity nor the actual type checker execution, when based solely on the types of the members.
Ambiguous one-to-many mappings
A straight-forward approach to instrumenting the type checker’s codebase adds instrumen- tation blocks at:
Chapter 5. Lightweight extraction of type checker decisions
1. The beginning and the end of the typing method.
2. For every type checker’s logic where its execution may diverge, such as for the condi- tional blocks or when pattern matching.
Listing 5.9 illustrates the result of following such a proposal to the letter; needless to say, the instrumentation blocks start to become part of the codebase, rather than only complement- ing it. Such over-instrumentation will significantly affect the maintenance of the type checker. On the positive side, the mapping between the low-level instrumentation events and the high- level representation can now be classified as a one-to-one mapping.
In our approach, we elide some of the instrumentation blocks instead, and infer the different type checker execution paths based on the possible sequences of the low-level events. The already discussed type checking of the function application (Figure 5.7) is one example of such an approach.
To consider the other extreme of the instrumentation spectrum, the under-instrumentation, we remove some instrumentation blocks from our reference instrumentation example in Fig- ure 5.6. The removal of the low-levelEV.TypeApp1andEV.InvalidFunAppevents would lead to a simpler high-level representation, as presented in Listing 5.10. The new high-level hier- archy has four different subclasses of theTypeAppbase class that model various type checker executions. As a result, any instrumentation block with a low-level EV.TypeAppevent will have to be mapped to its high-level counterpart in a one-to-many relation.
The proposed simplification is ambiguous. To illustrate, we consider a mapping context with a sequence of the already mapped high-level instances, { x1, x2}, where the runtime type of
each of the instances is some type S, such that S is a subtype ofTypecheck. The mapping that is based on the given sequence can lead to different possibilities:
• [typecheckFun→ { x1} , typeAdapted→ { x2}] (for theTypeAppFallbackclass)
The low-level events represent a valid mapping for theTypeAppFallbackclass, where some function application had to be typed using the fallback mechanism.
• [typecheckFun→ { x1} , typecheckArgs→ { x2}] (for theTypeAppMonomorphicclass)
The low-level events represent a valid mapping for theTypeAppMonomorphicclass, where some function application did not have to instantiate any local type parameters of the function.
• [typecheckFun→ { x1} , targsFromExpectedType→ { },typecheckArgs → { x2} ,
inferInstance→ { },typeAppCont → { }] (for theTypeAppPolymorphicclass)
The low-level events represent a partial mapping for the TypeAppPolymorphicclass, where mappings for some members could not be satisfied.
• [typecheckFun→ { x1} , typecheckArgs→ { x2} , inferAlternative→ { }]
(for theTypeAppOverloadedclass) 140
5.3. Mapping between representations
1 def typedApplication(ast: Apply, pt: Type): Tree =
2 EV.instrument(EV.TypeApp(ast, pt), EV.TypeAppDone(_)) {
3 val Apply(funAst, argsAsts) = ast
4 val fun1 = typecheckAst (funAst, WildcardType)
5 if (fun1.tpe == ErrorType) typedApplicationFallback(ast, pt)
6 else assignAppType(fun1, argsAsts, pt) 7 }
8
9 def assignAppType(fun1: Tree, args: List[Tree], pt: Type): Tree = { 10 EV <<< TypeApp1(...)
11 val app1 = fun1.tpe match {
12 case MethodType(params, resultTpe) => 13 EV.instrument(...) { 14 ... // same as before 15 if (hasError(args1)) EV.instrument(...) { 16 ... 17 } else EV.instrument(...) { 18 ... 19 } 20 } 21 case PolymorphicType(tParams,MethodType(params,resultType)) => 22 EV.instrument(...) { 23 ... // same as before 24 if (hasError(args1)) EV.instrument(...) { 25 ... // fallback mechanism 26 } else EV.instrument(...) { 27 ...
28 inferMethodInstance (args1, fun1, pt, paramsTpes)
29 }
30 }
31 case OverloadedType(_, alternatives) => 32 EV.instrument(...) { 33 ... // same as before 34 } 35 } 36 EV >>> EV.Done 37 app1 38 } 39
40 def typedApplicationFallback(ast: Apply, pt: Type): Apply = 41 EV.instrument(EV.InvalidFunApp(...)) {
42 ... 43 }
Figure 5.9: Example of the overzealous instrumentation of the function that verifies func- tion applications. In comparison to the initially proposed instrumentation from Figure 5.6, the example instruments every possible type checking path separately.
Chapter 5. Lightweight extraction of type checker decisions
abstract class TypeApp extends TypeGoal {
type U <: EV.TypeApp def typecheckFun: Typecheck