per-production, rather than per-nonterminal. And so, like default equations, they will need special treatment in how static production flow graphs are created, and (if translated away) how equations are inserted statically.
5.7 Self-evaluation on Silver
This analysis places additional restrictions on the relationships between modules, especially where host language and extension are concerned. Later on, in chapter 7, we construct a C compiler and some extensions that all satisfy this analysis. As evaluations of whether useful extensions are possible for practical host languages, this should be adequate. To evaluate this analysis on an existing attribute grammar, though, we have applied it to the Silver compiler itself, which is written in Silver.
The Silver host language is one of the most complex Silver programs we have. It was implemented in several modules, a division that we found natural prior to the development of this analysis. It has several language features that are implemented not in the “host language” proper, but as extensions, and so should be subject to the restrictions of this analysis. As a further benefit, it may be interesting to note where we do not already meet the requirements of the analysis, given that Silver was largely developed prior to this analysis. It is also the Silver specification that we use the most, and as a result we believe it would have the fewest bugs.
We briefly describe a few of the extensions to the Silver compiler, to demonstrate they are interesting and nontrivial. A “convenience” extension introduces new syntax that greatly simplifies making large numbers of similar occurs declarations, by allow- ing nonterminal declarations to be annotated with a list of them (this is source of our with syntax on nonterminals.) A “testing” extension adds several constructs for
5.7. Self-evaluation on Silver
writing and generating unit tests for the language specification. An “easy terminals” extension allows keyword terminals to be referred to by their lexeme in production signatures instead of using the name the terminal was declared under (using 'to' instead of To_kwd, for example.) We also separate primitive pattern matching (part of the host language, as described for Ag) and a language extension that permits nested pattern matching on multiple values. Finally, the entire translation to Java is implemented as a composable language extension.
A technical report documents the details of the issues raised by the analysis and the changes made to address them [80]. We discuss the interesting aspects that arose from this exercise here.
Silver focuses specifically on language extension, and as a result, we had not previously implemented the monolithic, whole-program well-definedness analysis for attribute grammars. Without the modular analysis, we had simply gotten by without a static completeness analysis at all, relying on dynamic errors instead. The first set of changes to satisfy the analysis were simple bug fixes that could have been caught with a whole-program analysis. We found several legitimately missing synthesized and inherited attribute equations. It also found several productions that should have been forwarding, but were not. These were not exposed because we had written equations for all synthesized attributes that were ever actually demanded. In other words, these were lurking bugs, unexposed until we would have tried to write a certain type of entirely valid language extension.
Another positive set of changes improved the quality of the implementation, even if they did not directly fix latent crash-bugs. We discovered several extraneous attribute occurrences that simply never had equations, and were never used either. Many uses of reference attributes were found to be completely unnecessary and removed.
5.7. Self-evaluation on Silver
One particularly interesting change has to do with how concrete syntax speci- fications are handled in Silver. Silver’s host language supplies a “standard” set of declarations for concrete syntax (nonterminals, terminals, etc), while Copper-specific declarations are kept in a separate module. The analysis raised a simple error: the Java translation attributes for parser declarations (which were part of the “standard” module) were being supplied by the Copper grammar, which is a violation of the rules. There are three modules here: the “standard“ syntax module with the production, the “translation” module with the attribute occurrence, and the “Copper” module, where this equation existed. This declaration does have some “generic” parts: it indi- cates that a parser should be built, and it declares a new function in the environment. However, part of the semantics of this declaration is simply Copper-specific: some- how the function implementation calls the Copper-generated parser. Our solution was to move this parser declaration entirely out of the “standard” module and into the “Copper” module.
Some parts of the Ag language and this analysis were motivated by our attempts to get Silver to conform to the restrictions. We found that we were abusing forward- ing, using it as a way to define default values for attributes where the forwarded-to tree was not, in fact, semantically equivalent to the forwarding tree in the slightest. This was our original motivation for introducing default equations, to remove these abuses of forwarding.
There were two sorts of negative changes made to the Silver specification in order to make it pass the analysis. The first of these resulted from the conservative rules for handling reference attributes. On two occasions, inherited attribute equations had to be supplied whose values are never actually used. In one case, a nonterminal that represents information about concrete syntax has two synthesized attributes, one for
5.7. Self-evaluation on Silver
a normalization process and one for translation. These attributes have different (true) inherited dependencies, but because they internally both made use of references, the analysis required the full set of inherited attributes be supplied in order to access either synthesized attribute.
The second sort of negative changes involved introducing workarounds for code that we already knew needed refactoring, but we did not want to fix, yet. In fact, in many of these cases, the analysis lead us to code that already had “TODO” comments complaining about a design for reasons unrelated to the analysis. The most signifi- cant of these is the use of a single nonterminal (called DclInfo) as a data structure to represent several different types of declarations in Silver (attributes, types, values, occurs-on, etc.) This is a legacy from when Silver did not have parametric polymor- phism and our environment needed to pick a single monomorphic type to return as a result of any query. We could fix this issue by parameterizing our environment lookup functions, and introducing separate types for each namespace (TypeDclInfo, AttrDclInfo, etc.) But to forgo fixing this issue for the time being, we introduced “error” equations for attributes that did not have sensible values otherwise (e.g. at- tributes that make sense for value declarations but do not apply to type declarations.) These directly silence the analysis and leave a potential problem (if these equations were to be demanded), but they’re visibly noisy about this problem in the code and the use of “error” equations remains statically detectable. These error equations are essentially a form of “technical debt” - legitimate problems that we will change later, but for various reasons decide not to do just yet.
In the end, most of the changes necessary were to the host language itself, and the extensions then passed with minimal further effort. The translation code was identified as a pure extension to the host language, which we found surprising initially.
5.8. Extending to circularity