In this section we define change structures for sum types, together with the derivative of their introduction and elimination forms. We also obtain support for booleans (which can be encoded as sum type 1 + 1) and conditionals (which can be encoded in terms of elimination for sums). We have mechanically proved correctness of this change structure and derivatives, but we do not present the tedious details in this thesis and refer to our Agda formalization.
Changes structures for sums are more challenging than ones for products. We can define them, but in many cases we can do better with specialized structures. Nevertheless, such changes are useful in some scenarios.
In Haskell, sum types a + b are conventionally defined via datatype Either a b, with introduction forms Left and Right and elimination form either which will be our primitives:
data Either a b= Left a | Right b
either:: (a → c) → (b → c) → Either a b → c either f g (Left a)= f a
either f g (Right b)= g b
We can define the following change structure. data EitherChange a b
= LeftC (∆a) | RightC (∆b)
| EitherReplace (Either a b)
instance (ChangeStruct a, ChangeStruct b) ⇒ ChangeStruct (Either a b) where type∆(Either a b) = EitherChange a b Left a ⊕ LeftC da = Left (a ⊕ da) Right b ⊕ RightC db = Right (b ⊕ db)
Left_ ⊕ RightC _ = error "Invalid change!" Right_ ⊕ LeftC _ = error "Invalid change!" _ ⊕ EitherReplace e2= e2
oreplace= EitherReplace 0Left a= LeftC 0a 0Right a= RightC 0a
Chapter 11. A tour of differentiation examples 77 Changes to a sum value can either keep the same “branch” (left or right) and modify the contained value, or replace the sum value with another one altogether. Specifically, change LeftC da is valid from Left a1to Left a2if da is valid from a1to a2. Similarly, change RightC db is valid from Right b1 to Right b2if db is valid from b1to b2. Finally, replacement change EitherReplace e2is valid from e1 to e2for any e1.
Using Eq. (11.1), we can then obtain definitions for derivatives of primitives Left, Right and either. The resulting code is as follows:
dLeft:: a → ∆a → ∆(Either a b) dLeft a da= LeftC da
dRight:: b → ∆b → ∆(Either a b) dRight b db= RightC db
deither::
(NilChangeStruct a, NilChangeStruct b, DiffChangeStruct c) ⇒ (a → c) →∆(a → c) → (b → c) → ∆(b → c) →
Either a b →∆(Either a b) → ∆c
deither f df g dg (Left a) (LeftC da)= df a da deither f df g dg (Right b) (RightC db)= dg b db deither f df g dg e1(EitherReplace e2)=
either (f ⊕ df ) (g ⊕ dg) e2⊖ either f g e1 deither_ _ _ _ _ _ = error "Invalid sum change"
We show only one case of the derivation of deither as an example: deither f df g dg (Left a) (LeftC da)
=∆ { using variants of Eq. (11.1) for multiple arguments }
either (f ⊕ df ) (g ⊕ dg) (Left a ⊕ LeftC da) ⊖ either f g (Left a) = { simplify ⊕ }
either (f ⊕ df ) (g ⊕ dg) (Left (a ⊕ da)) ⊖ either f g (Left a) = { simplify either }
(f ⊕ df ) (a ⊕ da) ⊖ f a
=∆ { because df is a valid change for f , and da for a }
df a da
Unfortunately, with this change structure a change from Left a1to Right b2is simply a replace- ment change, so derivatives processing it must recompute results from scratch. In general, we cannot do better, since there need be no shared data between two branches of a datatype. We need to find specialized scenarios where better implementations are possible.
Extensions, changes for ADTs and future work In some cases, we consider changes for type Either a b, where a and b both contain some other type c. Take again lists: a change from list as to list Cons a2asshould simply say that we prepend a2to the list. In a sense, we are just using a change structure from type List a to (a, List a). More in general, if change das from as1to as2is small, a change from list as1to list Cons a2as2should simply “say” that we prepend a2and that we modify as1into as2, and similarly for removals.
In Sec. 18.4, we suggest how to construct such change structures, based on the concept of polymorphic change structure, where changes have source and destination of different types. Based on initial experiments, we believe one could develop these constructions into a powerful combinator language for change structures. In particular, it should be possible to build change structures for lists
78 Chapter 11. A tour of differentiation examples similar to the ones in Sec. 11.3.2. Generalizing beyond lists, similar systematic constructions should be able to represent insertions and removals of one-hole contexts [McBride, 2001] for arbitrary algebraic datatypes (ADTs); for ADTs representing balanced data structures, such changes could enable efficient incrementalization in many scenarios. However, many questions remain open, so we leave this effort for future work.
11.6.1
Optimizing filter
In Sec. 11.3 we have defined filter using a conditional, and now we have just explained that in general conditionals are inefficient! This seems pretty unfortunate. But luckily, we can optimize filterusing equational reasoning.
Consider again the earlier definition of filter: filter :: (a → Bool) → List a → List a
filter p= concatMap (λx → if p x then singleton x else Nil)
As explained, we can encode conditionals using either and differentiate the resulting program. However, if p x changes from True to False, or viceversa (that is, for all elements x for which dp x dx is a non-nil change), we must compute ⊖ at runtime. However, ⊖ will compare empty list Nil with a singleton list produced by singleton x (in one direction or the other). We can have ⊖ detect this situation at runtime. But since the implementation of filter is known statically, we can optimize this code at runtime, rewriting singleton x ⊖ Nil to Insert x, and Nil ⊖ singleton x to Remove. To enable this optimization in dfilter, we need to inline the function that filter passes as argument to concatMapand all the functions it calls except p. Moreover, we need to case-split on possible return values for p x and dp x dx. We omit the steps because they are both tedious and standard.
It appears in principle possible to automate such transformations by adding domain-specific knowledge to a sufficiently smart compiler, though we have made no attempt at an actual implemen- tation. It would be first necessary to investigate further classes of examples where optimizations are applicable. Sufficiently smart compilers are rare, but since our approach produces purely functional programs we have access to GHC and HERMIT [Farmer et al., 2012]. An interesting alternative (which does have some support for side effects) is LMS [Rompf and Odersky, 2010] and Delite [Brown et al., 2011]. We leave further investigation for future work.