Several principles influenced the ru les used by the SCAN language to invoke macros . Since a pattern might appear nu merous t i mes i n a fi le of text, t he first principle was to apply a macro each time irs pattern is found so that the macro can make multi ple transformations, just like an editor substitution command . Second , i t was important
M A C R O f i n d _ t i m e T R I G G E R { i n t e g e r ' : ' i n t e g e r [ ' · ' i n t e g e r ] } ; D E C L A R E c o u n t : S T A T I C I N T E GE R ;
c o u n t = c o u n t + 1 ;
A N S W E R ' t i m e : ', S T R I N G < c o u n t > ;
E N D M A C R O ;
Figure 1 Example of a Macro
Digital Technical journal 4 1
No. 6 Februarv 1988
Software Product ivity Tools
to be able to speci fy many concurrent substitu tions . Therefore, when a VAX SCAN program is compiled, l in ked , and then run , it behaves much l i ke a batch (rather than a n i n teractive) substitu tion command . The third principle in designing macro invocation rules was that patterns may be of varying degrees of complexity. To manage very complex patterns, the patterns have to be separable i nto pans, much l ike a program can be separated i nto multiple procedures.
Two types of macros are defined to support these principles : syntax macros, and trigger macros . These names reflect the two different macros distinguished by the attributes SYNTAX and TRIGGER with i n a SCAN macro declaration .
Trigger macros provide search and replace semantics. A program may contain any number of trigger macros. Upon executing a START SCAN statement, a program begins scan n i ng the i nput stream of text specified by that state ment for the patterns specified by the trigger macros. Figure 2 i l lustrates a series of trigger macros. M A C R O f i n d _ p a t t e r n _ a T R I GG E R { a _ p a t t e r n } ; A N S W E R r e p l a c e m e n t _ t e x t ; E N D M A C R O ; MA C R O f i n d _ p a t t e r n _ b T R I GG E R { b _ p a t t e r n } ; A N S W E R r e p l a c em e n t _ t e x t ; E N D M A C R O ; M A C R O f i n d _ p a t t e r n _ c T R I GG E R { c _ p a t t e r n } ; A N S W E R r e p l a c e m e n t _ t e x t ; E N D M A C R O ; P R O C E D U R E m a i n - p r o c e d u r e MA I N ; S T A R T S C A N The fi le f i l e _ t o _ b e _ s c a n n e d . d a t , i n this example (specified in the START SCAN state ment) is searched for a _ p a t t e r n , b _ p a t t e r n , and c _ p a t t e r n (speci fied i n trigger macro pat terns) . The file f i 1 e_ t o _ b e _ c r e a t e d . da t (also specified in the START SCAN statement) is cre ated by the program . Its contents will be the orig inal file, f i l e _ t o _ b e _ s c a n n e d . d a t , i nclud i ng the substitutions performed by the macros.
Syntax macros permit patterns to be defined i n terms o f other patterns. That i s , within one pat tern a programmer can request that a pattern described by a separate syntax macro be recog n i zed. This concept of composing a pattern from more elementary patterns is the basis of formal language theory. 1 This topic is expl ored in
greater depth in the next section .
Specifying Patterns
The design of patterns is infl uenced by the fol lowing three sources:
1 . The BNF-style syntax d iagrams customers find i n Digita l 's manuals
I N P U T F I L E ' f i l e _ t o _ b e _ s c a n n e d . d a t ' O U T P U T F I L E ' f i l e _ t o _ b e _ c r e a t e d . d a t ' ; E N D P R O C E D U R E ;
Figure 2 Trigger Macro Example
4 2 Digital Technical journal
2 . Compi ler theory for context-free grammars 3 . The original MACRO language i mplemented
by Sperry Univac Corporation
All three sources recognize a two-level approach for specifying patterns. The lower level recognizes simple constructs, such as numbers, keywords, names, and punctu a tion . The h igher level arranges the lower level constructs i n to statements and groups of statements.
This two-l evel approach has several advan tages. Abstracting pri m i tives into lower level patterns resu lts i n a more uniform use of primi tives . That, i n turn, makes the overal l pattern easier tO remember, which is of great benefit tO a programmer trying to design a language that he is trying ro recognize using the VAX
SCAN product. Compiler theory also states that lower level patterns can be recognized very efficiently i f they conform tO a set of ru les, such as regu lar expressions . 1
To take advantage of those features, the SCAN language provides a two-level descri ption of a pattern . The lower level pattern, called a toke n , groups characters. The higher level pattern that appears in a macro is composed of tOkens.
Figure 3 i l l ustrates several SCAN rokens. The syntax for the patterns comes largely from the conventions used i n Digita l 's software manuals 5 Curly braces surround a sequence that is required, and square brackets surround an optional sequence. An e l l i psis indicates that the prior sequence can occur mul tiple times, and a vertical bar separates alternative choices. Thus in the Figure 3 example, m o r s e _ c o d e _ l e t t e r describes a req u i red sequence of " . " or "_" characters. This required sequence can occur one or more t i mes.
The token i d e n t i f i e r i l lustrates the pattern for an identifier in the SCAN language . The token definition uses two sets, a l p h a and o t h e r , to simplify the specification of its pattern . The definitions of these sets appear i n the exam ple as wel l .
I n macros, higher level patterns are defined using the same operators that are used in roken declarations. Unl i ke a lower level pattern, however, the operands of a macro pattern are rokens and other macros, rather than characters and sets. Therefore , tokens are the buil d i ng blocks of a macro pattern . Referencing a macro within a macro pattern provides a subrout i ne l i ke capab i l i ty for patterns. The p lacement of a macro name i n a macro pattern i nd icates that the pattern of that macro shoul d be recogni zed at the point of reference .
Defining t he pattern of one macro i n terms of other macros adds sign i ficant power ro SCAN's patterns. This power, i l l ustrated in Fig ure 4, shows the syntax for a SCAN token decla ration . The pattern found in a roken declara tion is equivalent to patterns that can be described using regular expressions. Token patterns need to express the precedence of three operators: a lternation , concatenation , and repetition . In addition , a token pattern sup ports nested subpatterns that are either req u ired or opt iona l .
This example uti lizes macros t o provide • Levels of abstraction
• Sharing of patterns • Recursion
Levels of abstraction refers to the process of building a hierarchy of concepts in which each
T O K E N mo r s e _ c o d e _ l e t t e r { { ' _ ' : } . . . } ;
T O K E N b e g i n _ k eywo r d { ' B E G I N ' } ;
S E T a l p h a < ' a ' . . ' z ' O R ' A ' . . ' Z ' > ;
S E T o t h e r < ' 0 ' . . ' 9 ' O R ' $ ' O R ' - ' > ;
TO K E N i d e n t i f i e r { a l p h a [ a l p h a : o t h e r ] . . . } ;
Figure 3 Token Examples
Digital Technical journal 4 3
No. (, Februarv I !)88
Software Productivity Tools
leve l of the hierarchy is bu i l t on the next lower leve l . The SCAN language justifies two levels of patterns, tokens and macros, based on this principle. In Figure 4 , the principle is employed further. T o k e n _ d e c l a r a t i o n i s defined in terms of t o k e n _ p a t t e r n , which is defined i n terms of t o k e n _ o p e r a n d . Each macro describes a level of abstraction that makes a complex pattern both si mpler to write and to understand . In this particular example, the levels expose the prece dence of the three token pattern operators. T _ p a t 3 and t o k e n _ p a t t e r n are examples of
sharing patterns by means of macros. Much as a subromine is a vehicle for sharing code i n a
FORTRAN program , macros are a vehicle for
sharing a pattern i n SCAN .
Recursion - defi n i ng a pattern i n terms of itself - is very useful i n the description of many patterns, especially when patterns can be nested as in the case of a token pattern. In Figure 4 , t o k e n _ p a t t e r n i s defined i n terms of t o k e n _ p a t t e r n because token patterns can be arbitrarily nested using curly braces and square brackets. T O K E N t o k e n _ k e ywo r d . . . . T O K E N i d e n t i f i e r . . . . p a t t e r n s f o r t h e t o k e n s h a v e b e e n om i t t e d T O K E N T O K E N T O K E N T O K E N T O K E N T O K E N T O K E N T O K E N c h a r a c t e r _ s t r i n g . . . . I • I , . . . . I I I • • • • I [ I • • • • I ) I • • • • I { I • • • • I } I 0 0 0 0 I • I ' ... . T O K E N ' . . . ' . . . . ! s y n t a x f o r t o k e n d e c l a r a t i o n M A C R O t o k e n _ d e c l a r a t i o n S Y N T A X { t o k e n _ k e ywo r d i d e n t i f i e r ' { ' t o k e n _ p a t t e r n ' } ' ' ; ' } ; s y n t a x f o r a l t e r n a t i o n M A C R O t o k e n _ p a t t e r n S Y N T A X { t _ p a t 3 [ ' I ' L p a t 3 l . . . } ; ! s y n t a x f o r c o n c a t e n a t i o n M A C R O t _ p a t 3 S Y N T A X { t _ p a t 2 . . . } ; ! s y n t a x f o r r e p e t i t i o n M A C R O t _ p a t 2 S Y N T A X { t _ p a t 1 [ ' . . . ' J } ; ! s y n t a x f o r o p t i o n a l a n d r e q u i r e d p a t t e r n s M A C R O t _ p a t 1 S Y N T A X ! s y n t a x f o r o p e r a n d s t o k e n _ o p e r a n d ' [ ' t o k e n _ p a t t e r n ' l ' ' { ' t o k e n _ p a t t e r n ' } ' } ; M A C R O t o k e n _ o p e r a n d S Y N T A X { c h a r a c t e r _ s t r i n g i d e n t i f i e r } ;
Figure 4 Syntax Macro Example
4 4 Digital Tecbnicaljournal