5.2 Type Rules
5.5.3 Transitive Closure
The functiontransthat is defined below is an example that uses potential of a tree. For a binary treet the expressiontrans(t,[])evaluates to a list`such that (x,y) is in`if and only ifxis an ancestor ofyint. In other wordstrans(t,[])computes the transitive closure oft.
attach : (int,T(int),L(int,int)) → L(int,int) attach(y,t,acc) = match t with | leaf → acc
| node(x,t1,t2) → let acc1 = attach(y,t1,acc) in
let acc2 = attach(y,t2,acc1) in (y,x)::acc2; trans : (T(int),L(int,int)) → L(int,int)
trans(t,acc) = match t with | leaf → acc
| node(x,t1,t2) → let acc1 = attach(x,t1,acc) in let acc2 = attach(x,t2,acc1) in let acc3 = trans(t1,acc2) in trans(t2,acc3);
The following types are inferred with the heap-space metric.
attach : (int,T(3,0)(int),L(0,0)(int,int))−−−→0/0 L(0,0)(int,int)
trans : (T(0,3)(int),L(0,0)(int,int))−−−→0/0 L(0,0)(int,int)
According to Lemma 5.1.4, the bound that is implied by the type oftransis h
X
i=2
whereniis the number of nodes on leveli. If the input tree is balanced then h X i=2 ni·3·(i−1) = bl og2nc X i=2 2i·3·(i−1) ≤ 3·(bl og2nc −1)· bl og2nc X i=2 2i ≤ 3·(bl og2nc −1)·(n−1) Note that 3¡n2¢=1.5n2−1.5nis an upper bound for this function.
this difficult science is formed slowly, but it preserves every principle which it has once acquired; it grows and strengthens itself incessantly in the midst of the many variations and errors of the human mind.
JOSEPHFOURIER The Analytical Theory of Heat (1878)
6
Multivariate Polynomial Potential
The univariate polynomial amortized analysis that I presented in Chapter 5 extends linear automatic amortized analysis to polynomial bounds while preserving most of the features that make the linear system practicable. However, the inability of the univariate system to express mixed multiplicative bounds such asn·mhampers both utilization in practice and compositionality.
In this chapter, I describe amultivariate polynomial amortized resource analysis
that extends the univariate system. It preserves the principles of the univariate sys- tem while expanding the set of potential functions so as to express a wide range of dependencies between different data structures. The presentation is based on an article that appeared at the 38th ACM Symposium on Principles of Programming Languages (POPL’11) [HAH11].
Section 6.1 introducesresource polynomials, the multivariate potential functions that I use in the chapter. In Section 6.2, I show how data types can be annotated with resource polynomials. In contrast to the previous chapters, there is one global resource polynomial for tuple types. Section 6.3 contains multivariate shift operations as well as type rules that are used to derive annotated type judgments. In Section 6.4, I prove the soundness of the multivariate analysis. Section 6.5 explains the type inference and Section 6.6 demonstrates the analysis with several example programs.
6.1 Resource Polynomials
A resource polynomial maps a value of some data type to a nonnegative rational number. Potential functions in this section are always given by such resource polynomials.
In the case of an inductive tree-like data type, a resource polynomial will only depend on the list of entries of the data structure in pre-order. Thus, ifD(A) is such a data type with entries of typeA, that is,A-labelled binary trees, andvis a value of type
D(A) then we write elems(v)=[a1, . . . ,an] for this list of entries.
An analysis of typical polynomial computations operating on a data structurevwith elems(v)=[a1, . . . ,an] shows that it consists of operations that are executed for every
k-tuple (ai1, . . . ,aik) with 1≤i1< · · · <ik≤n. The simplest examples are linear map
operations that perform some operation for everyai. Another example are common sorting algorithms that perform comparisons for every pair (ai,aj) with 1≤i<j≤nin the worst case.
Base Polynomials
For each data typeA, I now define a setP(A) of functionsp:A →Nthat map values of typeAto natural numbers. The resource polynomials for typeAare then given as nonnegative rational linear combinations of thesebase polynomials. We defineP(A) as follows. P(A) = {a7→1} ifAis an atomic type P(A1,A2) = {(a1,a2)7→p1(a1)·p2(a2)|pi∈P(Ai)} P(D(A)) = © v7→ X 1≤j1<···<jk≤n Y 1≤i≤k pi(aji)|k∈N,pi∈P(A) ª
In the last clause we have [a1, . . . ,an]=elems(v). Every setP(A) contains the constant function v7→1. In the case ofD(A) this arises fork =0 (one element sum, empty product).
For example, the function`7→¡|`k|¢is inP(L(A)) for everyk∈N; simply takep1=. . .=
pk=1 in the definition ofP(D(A)). The function (`1,`2)7→ ¡|`1| k1 ¢ ·¡|`2| k2 ¢ is inP(L(A),L(B)) for everyk1,k2∈N, and the function [`1, . . . ,`n]7→P1≤i<j≤n
¡|`i| k1 ¢ ·¡|`j| k2 ¢ is inP(L(L(A))) for everyk1,k2∈N. Resource Polynomials
Aresource polynomial p:A →Q+0 for a data typeAis a non-negative linear combina- tion of base polynomials, that is,
p= X i=1,...,m
qi·pi
forqi∈Q+0 andpi∈P(A). We writeR(A) for the set of resource polynomials forA. An instructive, but not exhaustive, example is given byRn=R(L(int), . . . ,L(int)). The setRnis the set of linear combinations of products of binomial coefficients over variables
x1, . . . ,xn, that is,Rn={Pmi=1qiQnj=1¡kxj
i j ¢
|qi ∈Q+0,m∈N,ki j∈N}. These expressions naturally generalize the univariate polynomials from Chapter 5 and meet two conditions that are important to efficiently manipulate polynomials during the analysis. Firstly, the polynomials are non-negative, and secondly, they are closed under the discrete
difference operators∆i for everyi. The discrete derivative∆ip is defined through
∆ip(x1, . . . ,xn)=p(x1, . . . ,xi+1, . . . ,xn)−p(x1, . . . ,xn).
As in [HH10b] it can be shown thatRn is the largest set of polynomials enjoying these closure properties. It would be interesting to have a similar characterisation of
R(A) for arbitraryA. So far, we know thatR(A) is closed under sum and product (see Lemma 6.2.1) and are compatible with the construction of elements of data structures in a very natural way (see Lemmas 6.2.3 and 6.2.4). This provides some justification for their choice and canonicity. An abstract characterization would have to take into account the fact that our resource polynomials depend on an unbounded number of variables, e.g., sizes of inner data structures, and are not invariant under permutation of these variables. It seems that some generalization of infinite symmetric polynomials to subgroups of the symmetric group could be useful, but this would not serve our immediate goal of accurate multivariate resource analysis.
6.2 Annotated Types
The resource polynomials described in Section 6.1 are non-negative linear combinations of base polynomials. The rational coefficients of the linear combination are present as type annotations in our type system. To relate type annotations to resource polynomials we systematically describe base polynomials and resource polynomials for data of a given type.
If one considers only univariate polynomials then their description is straightfor- ward. Every inductive data structure of sizenhas a potential of the formP
1≤i≤kqi¡ni¢. So we can describe the potential function with a vector~q=(q1, . . . ,qk) in the corresponding recursive type. For instance we can writeL~q(A) for annotated list types. Since each annotation refers to the size of one input part only, univariatly annotated types can be directly composed. For example, an annotated type for a pair of lists has the form (L~q(A),L~p(A)). See Chapter 5 for details.
In this chapter, I use multivariate potential functions, that is, functions that depend on the sizes of different parts of the input. For a pair of lists of lengthsnandmwe have, for instance, a potential function of the formP
0≤i+j≤kqi j ¡n i ¢¡m j ¢
, which can be described by the coefficientsqi j. But I also would like to describe potential functions that refer to the sizes of different lists inside a list of lists, etc. That is why I need to describe a set of indexesI(A) that enumerate the basic resource polynomialspiand the corresponding coefficientsqifor a data typeA. These type annotations can be, in a straight forward way, automatically transformed into usual easily understood polynomials. This is done in our prototype to present the bounds to the user at the end of the analysis.
Names For Base Polynomials
To assign a unique name to each base polynomial I define theindex set I(A) to denote resource polynomials for a given data typeA. Interestingly, but as I find coincidentally,
I(A) is essentially the meaning ofAwith every atomic type replaced byunit.
I(A) = {∗} ifA∈{int,bool,unit}
I(A1,A2) = {(i1,i2)|i1∈I(A1) andi2∈I(A2)}
I(L(B))=I(T(B)) = {[i1, . . . ,ik]|k≥0,ij∈I(B)} Thedegreedeg(i) of an indexi∈I(A) is defined as follows.
deg(∗) = 0
deg(i1,i2) = deg(i1)+deg(i2)
deg([i1, . . . ,ik]) = k+deg(i1)+ · · · +deg(ik)
DefineIk(A)={i∈I(A)|deg(i)≤k}. The indexesi∈Ik(A) are an enumeration of the base polyonomialspi ∈P(A) of degree at mostk. For eachi ∈I(A), I define a base polynomialpi∈P(A) as follows: IfA∈{int,bool,unit} then
p∗(v)=1 . IfA=(A1,A2) is a pair type andv=(v1,v2) then
p(i1,i2)(v)=pi1(v1)·pi2(v2) .
IfA=D(B) (in our type systemDis either lists or binary node-labelled trees) is a data structure and elems(v)=[v1, . . . ,vn] then
p[i1,...,im](v)=
X
1≤j1<···<jm≤n
pi1(vj1)· · ·pim(vjm) .
I use the notation 0A(or just 0) for the index inI(A) such thatp0A(a)=1 for alla. We have 0int= ∗and 0(A1,A2)=(0A1, 0A2) and 0D(B)=[]. IfA=D(B) forBa data type then the index [0, . . . , 0]∈I(A) of lengthnis denoted by justn. For convenience, I identify the index (i1,i2,i3,i4) with the index (i1, (i2, (i3,i4))).
For a listi=[i1, . . . ,ik] I writei0::i to denote the list [i0,i1, . . . ,ik]. Furthermore, I writei i0for the concatenation of two listsiandi0.
Recall thatR(A) denotes the set of nonnegative rational linear combinations of the base polynomials.
Lemma 6.2.1 Letp,p0∈R(A) be resource polynomials. Then we havep+p0,p·p0∈R(A), deg(p+p0)=max(deg(p), deg(p0)), and deg(p·p0)=deg(p)+deg(p0).
PROOF By linearity it suffices to show this lemma for base polynomials. For them, the
claim follows by structural induction. ■
Corollary 6.2.2 For everyp∈R(A,A) there existsp0∈R(A) with deg(p0)=deg(p) and
PROOF The proof follows directly from Lemma 6.2.1 noticing that base polynomials
p∈P(A,A) take the formpi·pi0. ■
Lemma 6.2.3 Let a ∈ A and `∈ L(A) be a list. Let furthermore k ≥ 0 and let
i0, . . . ,ik∈I(A) indexes for typeA. Then we have
p[i0,i1,...,ik]([]) = 0
p[i0,i1,...,ik](a::`) = pi0(a)·p[i1,...,ik](`)+p0(a)·p[i0,i1,...,ik](`) .
PROOF Let`=[v1, . . . ,vn]. Writingv0forawe compute as follows.
l ccl p[i0,i1,...,ik](a::`) = X 0≤j0<j1<···<jm≤n pio(vj0)·pi1(vj1)· · ·pim(vjm) = X 1≤j1<···<jm≤n pio(v0)·pi1(vj1)· · ·pim(vjm) + X 1≤j0<j1<···<jm≤n pio(vj0)·pi1(vj1)· · ·pim(vjm) = pio(a)· X 1≤j1<···<jm≤n pi1(vj1)· · ·pim(vjm) + X 1≤j0<j1<···<jm≤n pio(vj0)·pi1(vj1)· · ·pim(vjm) = pi0(a)·p[i1,...,ik](`)+p0(a)·p[i0,i1,...,ik](`)
The statementp[i0,i1,...,ik]([])=0 is obvious as the sum in the definition of the corre-
sponding base polynomial is over the empty index set. ■
Lemma 6.2.4 characterizes concatenations of lists (written as juxtaposition) as they will occur in the construction of tree-like data. Note that we have for instance that elems(node(a,t1,t2))=a::elems(t1) elems(t2).
Lemma 6.2.4 Let`1,`2∈ L(A)be lists of type A. Then we have`1`2∈ L(A)and p[i1,...,ik](`1`2)=
Pk
t=0p[i1,...,it](`1)·p[it+1,...,ik](`2) for all indexesij∈I(A).
This can be proved by induction on the length of`1using Lemma 6.2.3 or else by a
decomposition of the defining sum according to which indices hit the first list and which ones hit the second.
Annotated Types and Potential Functions
I use the indexes and base polynomials to define type annotations and resource polyno- mials. I then give examples to illustrate the definitions.
Atype annotationfor a data typeAis defined to be a family
I say thatQA is ofdegree (at most) k ifqi =0 for everyi ∈I(A) withd eg(i)>k. An
annotated data typeis a pair (A,QA) of a data typeAand a type annotationQAof some degreek.
LetHbe a heap and letvbe a value withHÍv7→a:A for a data typeA. Then the type annotationQAdefines thepotential
ΦH(v:(A,QA))=
X
i∈I(A)
qi·pi(a) .
Usually, I define type annotationsQAby only stating the values of the non-zero coeffi- cientsqi. However, it is sometimes handy to write annotations (q0, . . . ,qn) for a list of atomic types just as a vector. Similarly, I write annotations (q0,q(1,0),q(0,1),q(1,1), . . .) for pairs of lists of atomic types sometimes as a triangular matrix.
Ifa∈ AandQis a type annotation forAthen I also writeΦ(a: (A,Q)) forP
iqipi(a).
Examples
The simplest annotated types are those for atomic data types like integers. The indexes forintareI(int)={∗} and thus each type annotation has the form (int,q0) for aq0∈Q+0. It defines the constant potential functionΦH(v:(int,q0))=q0. Similarly, tuples of atomic types feature a single index of the form (∗, . . . ,∗) and a constant potential function defined by someq(∗,...,∗)∈Q+0.
More interesting examples are lists of atomic types like, that is,L(int). The set of indexes of degreekis then
Ik(L(int))={[], [∗], [∗,∗], . . . , [∗, ...,∗]}
where the last list containsk unit elements. Since we identify a list of i unit ele- ments with the integeri we haveIk(L(int))={0, 1, . . . ,k}. Consequently, annotated types have the form (L(int), (q0, . . . ,qk)) forqi∈Q+0. The defined potential function is
Φ([a1, . . . ,an]:(L(int), (q0, . . . ,qn))=P0≤i≤kqi
¡n i
¢
.
The next example is the type (L(int),L(int)) of pairs of integer lists. The set of indexes of degreekis
Ik(L(int),L(int))={(i,j)|i+j≤k}
if we identify lists of units with their lengths as usual. Annotated types are then of the form ((L(int),L(int)),Q) for a triangulark×kmatrixQwith non-negative rational entries. If`1=[a1, . . . ,an],`2=[b1, . . . ,bm] are two lists then the potential function is
Φ((`1,`2), ((L(int),L(int)), (q(i,j))))=P0≤i+j≤kq(i,j) ¡n i ¢¡m j ¢ .
Finally, consider the typeA=L(L(int)) of lists of lists of integers. The set of indexes of degreekis then
Ik(L(L(int)))=©[i1, . . . ,im]|m≤k,ij∈N,
X
j=1,...,m
Thus we haveIk(L(L(int)))={0, . . . ,k}∪{[1], . . . , [k−1]}∪{[0, 1], . . .}∪ · · ·. Let for instance
`=[[a11, . . . ,a1m1], . . . , [an1, . . . ,anmn]] be a list of lists andQ=(qi)i∈Ik(L(L(int)))be a cor-
responding type annotation. The defined potential function is then
Φ(`, (L(L(int)),Q))= X [i1,...,il]∈Ik(A) X 1≤j1<···<jl≤n q[i1,...,il] Ã mj1 i1 ! · · · Ã mjl il ! .
In practice the potential functions are usually not very complex since most of theqiare zero. Note that the resource polynomials for binary trees are identical to those for lists.
The Potential of a Context
For use in the type system, I have to extend the definition of resource polynomials to typing contexts. I treat a context like a tuple type.
LetΓ=x1:A1, . . . ,xn:An be a typing context and letk∈N. The index setIk(Γ) is defined through
Ik(Γ)=©(i1, . . . ,in)|ij∈Imj(Aj), X
j=1,...,n
mj≤kª.
Atype annotation Qof degreekforΓis a family
Q=(qi)i∈Ik(Γ)withqi∈Q + 0.
I denote aresource-annotated contextwithΓ;Q. LetHbe a heap andV be a stack with
HÍV:ΓwhereHÍV(xj)7→axj:Γ(xj) . The potential ofΓ;Qwith respect toHandV is
ΦV,H(Γ;Q)= X (i1,...,in)∈Ik(Γ) q~ı n Y j=1 pij(axj)
In particular, ifΓ= ;thenIk(Γ)={()} andΦV,H(Γ;q())=q(). I sometimes also writeq0 forq().
6.3 Type Rules
Before I describe the multivariate type system, I formalize some facts about the potential method that are useful to explain some of the ideas I describe later.
If f:A → Bis a function computed by some program andK(a) is the cost of
the evaluation of f(a) then our type system will essentially try to identify resource polynomialsp∈R(A) and ¯p∈R(B) such thatp(a)≥p¯(f(a))+K(a). The key aspect of such amortized cost accounting is that it interacts well with composition.
Proposition 6.3.1 Letp∈R(A), ˇp∈R(B), and ¯p∈R(C) be resource polynomials. Let
f:A → B,g:B → C,K1:A →Q, andK2:B →Q. Ifp(a)≥pˇ(f(a))+K1(a) and
ˇ
Notice that if we merely hadp(a)≥K1(a) and ˇp(b)≥K2(b) then no bound could be directly obtained for the composition.
Interaction with parallel composition, that is, (a,c)7→(f(a),c), is more complex than in the univariate system due to the presence of mixed multiplicative terms in the resource polynomials.
Proposition 6.3.2 Let p ∈R(A,C), ¯p∈R(B,C), f :A → B, andK :A →Q. For each j∈I(C) letp(j)∈R(A) and ¯p(j)∈R(B) be such thatp(a,c)=P
jp(j)(a)pj(c) and ¯
p(b,c)=P
jp¯(j)(b)pj(c). Ifp(0)(a)≥p¯(0)(f(a))+K(a) andp(j)(a)≥p¯(j)(f(a)) holds for allaandj6=0 thenp(a,c)≥p¯(f(a),c)+K(a).
In fact, the situation is more complicated due to the accounting for high watermarks as opposed to merely additive cost, and also due to the fact that functions are recursively defined and may be partial. Furthermore, we have to deal with contexts and not merely types. To gain an intuition for the development to come, the above simplified view should, however, prove helpful.
Type Judgments
The declarative type rules for RAML expressions (see Figure 6.1 and Figure 6.2) define a
multivariate resource-annotated typing judgmentof the form
Σ;Γ;Q`e: (A,Q0)
whereeis a RAML expression,Σis a resource-annotated signature (see below),Γ;Qis a resource-annotated context and (A,Q0) is a resource-annotated data type. The intended meaning of this judgment is that if there are more thanΦ(Γ;Q) resource units available then this is sufficient to pay for the cost of the evaluatione. In addition, there are more thanΦ(v:(A,Q0)) resource units left ifeevaluates to a valuev.
Programs with Annotated Types
Multivariate resource-annotated first-order typeshave the form (A,Q)→(B,Q0) for an- notated data types (A,Q) and (B,Q0). Amultivariate resource-annotated signatureΣis a finite, partial mapping of function identifiers tosets of resource-annotated first-order types.
Like in the univariate and linear cases, a RAML program with (multivariate) resource- annotated types consists of a (multivariate) resource-annotated signatureΣand a family of expressions with variables identifiers (ef,yf)f∈dom(Σ)such thatΣ;yf:A;Q`ef : (B,Q0) for every function type (A,Q)→(B,Q0)∈Σ(f).
Notations
Families that describe type and context annotations are denoted with upper case letters
families are the corresponding lower case letters with corresponding superscripts, that is,Q=(qi)i∈I,Q0=(qi0)i∈I, andQx=(qix)i∈I.
LetQ,Q0be two annotations with the same index setI. I writeQ≤Q0ifqi≤qi0for everyi∈I. ForK ∈QI writeQ=Q0+K to state thatq~0=q~0
0+K ≥0 andqi=q 0
i for
i6=~0∈I. LetΓ=Γ1,Γ2be a context, leti=(i1, . . . ,ik)∈I(Γ1) andj=(j1, . . . ,jl)∈I(Γ2) . I
write (i,j) to denote the index (i1, . . . ,ik,j1, . . . ,jl)∈I(Γ). Like in the other systems, I write
Σ;Γ;Q cf e: (A,Q0)
to refer to multivariate cost-free type judgments where all constantsK in the rules from Figure 6.1 and Figure 6.2 are zero. I use it to assign potential to an extended context in the let rule. More explanations will follow later.
LetQ be an annotation for a contextΓ1,Γ2. For j∈I(Γ2), I define theprojection