Language as a Cognitive Process

(1)

Book R e v i e w

N e w E d i t o r ' s N o t e

Books reviewed in the AJCL will be those of interest to c o m p u t a t i o n a l linguists; b o o k s in closely related disciplines may also be considered. The purpose of a book review is to inform readers about the c o n t e n t of the b o o k and to present opinions on the choice of material, m a n n e r of p r e s e n t a t i o n , and suitability for various readers and purposes. T h e r e is no limit to the length of reviews. The appropriate length is deter- mined b y its content.

If you wish to review a specific book, please con- tact me before doing so to check that it is not already under review b y someone else. If you want to be on a list of potential reviewers, please send me your name and mailing address together with a list of keywords summarizing your areas of interest. You can also suggest books to be reviewed without volunteering to be the reviewer.

L y n Bates, B o o k Review Editor

L a n g u a g e as a C o g n i t i v e P r o c e s s . V o l . 1. S y n t a x

T. W i n o g r a d

Addison-Wesley, Reading, MA, 1983. 608 pp., $23.95, ISBN 020108-571-2

General

This book is p r o b a b l y the first ever c o m p r e h e n s i v e , authoritative, and principled description of the intellectual history of natural language processing with the help of computers. It is also a very thorough introduction into the craft of dealing with natural language in the f r a m e w o r k of artificial intelligence or cognitive science, the disciplines that are i n t e r e s t e d in natural language theoretically only to the e x t e n t the latter sheds light on their main object of study: knowledge.

The book is multi-faceted. It is, first of all, a textbook; but it is also a reference book, a c o m p e n d i u m of practical knowledge for grammar and parser writers. This practical knowledge is presented in a " d i g e s t e d " way - no small feat for the author - that is, is organ- ized into conceptual groups and explained in a largely

unified terminology. This approach is very welcome

because "original research papers are o f t e n confusing, since a system is presented as a whole, with its unique features (rather than c o m m o n ideas) emphasized, and with the important ideas mixed with the implementa- tion details" (p. 358). Finally, it is a statement of the linguistic outlook of the a u t h o r and has much to do

with artificial intelligence, c o m p u t e r science, cognitive science (sometimes r e f e r r e d to as theoretical artificial intelligence) and the philosophy of science in general.

These three objectives are present, to some extent, in all the chapters of the book, but one m a y find that the t e x t b o o k material is c o n c e n t r a t e d in C h a p t e r s 2 t h r o u g h 6 - " W o r d P a t t e r n s and W o r d Classes", " C o n t e x t - f r e e G r a m m a r s and Parsing", " T r a n s f o r m a - tional G r a m m a r " , " A u g m e n t e d T r a n s i t i o n N e t w o r k G r a m m a r s " , " F e a t u r e and F u n c t i o n G r a m m a r s " - as well as in Appendix X: " A Language for Describing Objects and P r o c e d u r e s " . The material useful mostly as r e f e r e n c e is c o n t a i n e d in C h a p t e r 7, " C o m p u t e r Systems for Natural Language Parsing", and A p p e n - dixes B, " A n Outline of English Syntax", C, " C u r r e n t Directions in T r a n s f o r m a t i o n G r a m m a r " , and D, " A n ATN G r a m m a r for English". The methodological posi- tion and the linguistic c r e d o of the a u t h o r are explained in C h a p t e r 1 (especially Section 1.3, " T h e computational p a r a d i g m " ) , which is the most theoretically significant part of the volume.

A n o t h e r way to classify the b o o k ' s material is to divide it into the theoretical versus implementational, descriptive versus o p e r a t i o n a l , " w h a t " versus " h o w t o " parts. We shall follow this distinction in our discussion.

DL

Winograd introduces the b o d y of knowledge accumu- lated in the field o v e r the past 20 years not as a chronicle, but rather as intellectual history: ideas have p r e c e d e n c e over people. Thus, for example, the tex- t b o o k starts not with the description of the early at- t e m p t s at dealing with natural language, such as Raphael's SIR or W e i z e n b a u m ' s ELIZA, but with the definition of the notion of patterns and p a t t e r n matching. These are the simplest and the least "intelligent" ways of dealing with natural language formally. The description, h o w e v e r , serves the double p u r p o s e of being the f o u n d a t i o n for a discussion of more complex p a t t e r n matching (e.g.,. transition n e t w o r k s ) at the same time providing a testbed for the introduction of DL, a notation used t h r o u g h o u t the b o o k for defining

entities and describing algorithms and n o n -

deterministic schemata. The decision to devise and

use DL has obviously been a major one in the prepara- tion of this book. T h e r e was a need for it since the prospective readers (as the participants in Winograd's courses) include linguists who do not speak any of the " s t a n d a r d " c o m p u t e r languages. I n d e e d , the b o o k is saturated with DL, and a special 48-page-long appendix is d e v o t e d to DL language specification. The deci-

(2)

Book Review Language as a Cognitive Process. I. Syntax.

sion to use DL will probably prove the most controver- sial issue in the whole book, when viewed from the standpoint of classroom use.

The language will be initially disliked by the people with some background in computing, since it requires time to gain reading fluency.

T e x t b o o k

The "how t o " part of Chapter 2 contains definitions of simple (literal, open, and lexical) and variable patterns, as well as of basic procedures for matching and generating sentences from a pattern. The notions of regular expressions and transition networks (treated as extensions of patterns) are introduced, together with the non-deterministic procedures for recognition. The circle of problems connected with search is also ad-

dressed. Backtracking and parallel processing are

discussed as techniques for traversing transition networks.

The " w h a t " part of this chapter consists of the discussion of word classes, mostly in English. This is a bridge between lexical patterns and transition networks. The organization of dictionaries for computer

systems is discussed. The word classifications and

word class definitions are presented rather brusquely (cf. "anything that does not have another class gets called an adverb", p. 53), but they fill their purpose in providing terminology for discussions in further chapters.

The " w h a t " part of Chapter 3 includes a discussion of the "final products" of syntactic analysis: the types of syntactic structures. The discussion covers the head and modifier approach, the immediate constituent and the slot and filler approach. The functional character of the latter is emphasized ("role names are different from phrase types", p. 79), the importance of which will be felt later. Section 3.3 introduces the notion of parsing (recognition + assignment of structure) and the first schematic representation of the components of a parsing system. Also, this section inaugurates a succession of very important subsections, scattered throughout the book, which deal with the issues of strategy, design, trade-offs and choices in building systems for the automatic processing of natural language. In the first such section Winograd discusses general issues in parser design: uniformity of processing, separate processing of levels and precision, proce- dural alternatives of sequential versus parallel, and top-down versus bottom-up analysis, and the choice of network nodes to be expanded. This book gives the reader a clear understanding that the alternatives are independent of the grammars chosen for a particular analysis, universally applicable and not specific to any system in which they may have been used.

The "how t o " part of Chapter 3 includes the discussion of context-free grammars and derivation of

sentences; context-free parsing; non-deterministic

schemata for the top-down and bottom-up recognition, as well as their realizations: both the backtracking and the parallel algorithms for top-down recognition and the parallel bottom-up one (the remaining algorithm is given as an exercise). Next the augmentation of a recognizer to a parser is discussed, and the chapter is crowned by the introduction of the active chart parser, a technique combining features of a top-down and a bottom-up parser. The material is presented in a concise and very efficient manner, and it is quite easy to understand the idea and the technical details of active chart parsing.

Chapter 4 discusses transformational grammar. There is no "how t o " part. The context-free grammar rules cannot account for discontinuous components, subject-predicate agreement, etc. So, the rules of the grammar are generalized, and an hierarchy of grammar types is presented for the first time. The notions of the finite state (regular) and the context-sensitive grammar are defined. Also included are a procedure for producing a grammar from an equivalent network and some thoughts about the choice of the power of grammars for natural language processing. "In general, the motivations for using more powerful grammars go beyond weak generative capacity (the ability to specify a given language), but are based on a desire to have a grammar that is simple and that produces structures that correspond to our intuitions about other considerations, such as meaning" (p. 147).

Winograd goes on to give a brief explication of the Standard Theory (ST), striving to cover not only the technicalities of the application of transformational rules, but also the philosophy of ST's approach to the study of language (with its emphasis, in different con- texts, on competence, deep structure and interpretive semantics). The transformations themselves are introduced in Winograd's customary lucid and formal way (here he borrowed Akmajian and Heny's (1975) notation); some refinements to ST (bracketing, variables, rule ordering, extensions to the base, etc.) follow the description of the basic transformational derivation. Additional developments in ST are covered in Appen- dix C.

Winograd criticizes transformational grammar for overemphasis on the role of syntax, complete disinter- est in the problems of processing (and processes) and the resulting poor amenability of ttansformational grammar to computer studies of natural language.

Chapter 5 introduces the transition network equiva- lents of the context-free grammars and transformational grammar: recursive and augmented transition networks, respectively. The standard introductions to ATNs (the original paper by Woods (1970) and the excellent introduction by Bates (1978)), while being readable and useful, do not profit from the well- developed context built up by earlier chapters in this book. Winograd, predictably, does not use the tradi-

(3)

B o o k R e v i e w L a n g u a g e as a C o g n i t i v e Process. I. S y n t a x .

tional Lisp-like notation for ATNs, but opts for picto- rial diagrams and a special notation with starting and ending state names in subscript around the arc name and conditions, actions and initializations specified in English. This approach should widen the circle of the readers of the book.

The discussion of the c o m p o n e n t parts of an augmented transition network is very explicit. The notions of the arcs and their classification, conditions and actions, initializations, feature dimensions and role names are lucidly defined. A separate section is de- v o t e d to the use of registers in ATN grammars, and is accompanied by specific examples of problems solved through the use of features u n d e r discussion. Wino- grad discusses in some detail a relatively wide range of syntactic p h e n o m e n a of English and their t r e a t m e n t in ATN terms. A fuller outline of English syntax (though not in the ATN format) is given in Appendix B.

The section d e v o t e d to ATN i m p l e m e n t a t i o n is more ideological than technical and includes the description of the notion of compilation that would be baffling to novices: " T h e ]concept of compiling ... can be loosely formulated as 'Once you know what is actu- ally used, you can write a procedure that does it more efficiently than one designed to handle e v e r y t h i n g ' " (p. 265). This is, however, one of very few flaws of metaphorical oversimplification in the text.

Chapter 6 deviates from the linear progression that established an ascending t r e n d of complexity, and starts to discuss alternative grammar formalisms. The first such alternative is the systemic grammar of Halli- day. Winograd's SHRDLU was built under the influ- ence of systemic grammar, and thus there is a special relationship b e t w e e n the a u t h o r and the approach. Systemic grammar has not f o u n d significant following among linguists; as Winograd himself mentions, its main reason lies in the sociological aspects of language. M a y b e this is the reason why it lacks formali- ty, a c c u r a c y and a unifying organizational principle: the authors of the g r a m m a r had no such intentions, their main audience, at least originally, being second- ary school teachers. The emphasis on classification is too p r o n o u n c e d - designing t a x o n o m i e s is the most superficial way of studying a p h e n o m e n o n , even if functionality is declared as the general goal. T h e structure of the grammar is a loose conglomerate of very interesting issues which are not united by a com- m o n theoretical basis, and are thus relegated to the status of brilliant observations on the nature of language. It seems that the originality value of systemic grammar lies in the fact that its authors tried their best to p r o d u c e something d i f f e r e n t f r o m the " A m e r i c a n transformationalist e m a n a t i o n s " . The interest to language function in systemic grammar ascends not simply to Firth and Whorf, but, more important, to the ideas of Elmslev and Prague Linguistic Circle. Inciden- tally, Sgall, Haji~ov~i and Benegovfi, q u o t e d by Wino-

grad, have but a geographical proximity to the Prague Linguistic Circle; Mathesius spoke a b o u t t h e m e and rheme in the 1940's, not in 1961, as the date in the reference might suggest, thus making this idea roughly c o n t e m p o r a r y to Halliday's work; these notions, and the philosophy of systemic grammar, are part of the legacy of the structuralist paradigm and can be traced back to Saussure.

The introduction to systemic grammar has p r o b a b l y never been formulated as precisely and formally as in this book. It is my conviction that Winograd developed the systemic a p p r o a c h quite b e y o n d its original level. Most of the ideas behind it are sound and ap- pealing, and lack the a p p a r e n t anti-semanticism of the t r a n s f o r m a t i o n a l paradigm; the systemic a p p r o a c h p r e s e n t e d a very good f r a m e w o r k for developing com-

puter programs for language understanding. One of

the most attractive features of this approach for the computational linguists was its relative vagueness and p r e - f o r m a l state, since this p e r m i t t e d quite diverse interpretations and further specification of the t h e o r y in the computational domain. Halliday's description of English clause types and " t r a n s i t i v i t y " must have esspecially a t t r a c t e d the designers of c o m p u t e r systems, since being little more than a list of distinct and real p h e n o m e n a , it w o r k e d as a m e m o r y aid for recol- lecting the various language structures that had to be included into (or consciously excluded from) the sublanguage to be a c c o u n t e d for by the system u n d e r construction.

This impression is c o r r o b o r a t e d by the lack of formal definitions of rules for the systemic grammar and specifically by the observation that in c o m p u t e r programs that used some ideas of systemic grammars, such as W i n o g r a d ' s SHDRLU, the realization rules e m p l o y e d were " i m p l i c i t " , i.e., built into programs which were " c o m p l e x and difficult to m o d i f y " (p. 310).

Next on the agenda are case grammars. The most important observation a b o u t this kind of grammar is that this approach permits one " t o see the sentence structure as a window onto an underlying scenario... The grammar must provide a systematic way to carry out this mapping so the hearer will know ... what kind of scenario is intended. It could be argued that this problem is not properly a part of syntax and should instead be viewed as semantic i n t e r p r e t a t i o n " (p. 313).

Winograd gives a concise account of several case systems for English, including two proposals by Fill- more and the c o n t r i b u t i o n s by Simmons, Schank, Chase and Grimes. This account is much more principled and con~prehensive than, for instance, the chapter on case grammars (by W. Samlowski) in Charniak & Wilks's " C o m p u t a t i o n a l S e m a n t i c s " ( 1 9 7 6 ) although Winograd defers all deliberations on semantics until

(4)

Book Review Language as a C o g n i t i v e Process. I. Syntax.

the second volume of this book, whereas no such re- striction was present in the o t h e r textbook.

The " w h a t " part of this section contains a discussion of " c r i t e r i a for deciding on c a s e s " and " i n t e g r a t i n g cases into f o r m a l s y n t a x " along with a subsection on the relationship b e t w e e n case grammar and systemic grammar. Winograd argues that systemic grammar has all case grammar has to o f f e r and more. It seems, however, that in view of the scientific paradigm of cognitive science the semantically o r i e n t e d cases of, say, Schank are preferable to the t a x o n o m y - m i n d e d c l a u s e - d e p e n d e n t cases (or transitivity patterns) of systemic grammar.

The last part of this chapter is d e v o t e d to functional and generalized phrase structure grammars. B o t h approaches are fairly new and have not yet resulted in the d e v e l o p m e n t of c o m p l e t e large-scale c o m p u t e r applications. T h e s e grammars emphasize n o n d i r e c - tionality (that is, they deal with b o t h parsing and gen- eration), c o r r e s p o n d e n c e with meanings and multiple dimensions of analysis. Thus, f u n c t i o n a l grammars consider the full analysis of a sentence to be made up of 1) constituent structure; 2) functional description; 3) feature description; 4) lexical content; 5) semantic structure; and 6) phonological structure. The goal is noble, but it is doubtful w h e t h e r all these elements can be formally united, by a functional grammar, in one system - in order to implement a " b l a c k b o a r d " - t y p e

parsing arrangement. This discussion, together with

the brief sketches on definite clause, slot, junction, cognitive and relational grammars, constitute a smooth transition f r o m the t e x t b o o k to the r e f e r e n c e b o o k part of Winograd's text. It is impossible to acquire more than a superficial k n o w l e d g e of the g r a m m a r theories and mechanisms from the exposition; but this material was not m e a n t to be a substitute, for one should deal with this material as a source of reference.

Reference Book

Building c o m p u t e r systems that b o a s t a measure of understanding natural language has b e c o m e quite com- m o n and widespread. One feature almost universally present in such systems is a syntactic parser of natural language. C h a p t e r 7 is an e x t e n d e d " q u i c k r e f e r e n c e c a r d " for p e o p l e who build syntactic parsers. T h e chapter contains discussions (in various levels of detail) of 52 systems spanning 19 years of e f f o r t in the field; the discussion p r o c e e d s along c o n c e p t u a l (and not historical or other) lines. A f t e r naming several application areas for such systems (machine transla- tion, q u e s t i o n answering, data base retrieval, t h e o r y justification, etc.), Winograd goes on to the section of the greatest practical importance: "Issues in the design of a parsing s y s t e m " . T h e following crucial issues stand out: a) the choice of the grammar formalism; b) the form of assigned structures; c) the search strate- gies used, and d) the degree of completeness of the

system (what size sublanguage it is supposed to take care of). T h e systems are classified and discussed according to the type of their g r a m m a r formalism ( a u g m e n t e d phrase structures, transformations, charts, ATNs, p a t t e r n matching, or s i t u a t i o n - a c t i o n rules). The emphasis, predictably, is not on technical detail, but rather on the relative strengths and weaknesses of the approaches. One will not be able to implement a parser solely on the basis of the i n f o r m a t i o n in the b o o k (this was not i n t e n d e d ) , but the c h a p t e r is an excellent source for choosing the a p p r o a c h best suited to one's individual needs and tastes.

Some " r a w material" for use in a parsing system can be f o u n d in A p p e n d i x B: " A n Outline of English Syntax". This is a digest of Quirk's English grammar (Quirk et al. 1972), set in a largely systemic terminology and framework. T h e appendix does not purport to give answers to all the grammatical p r o b l e m s of English. M a n y topics are not covered, m a n y more are just sketched with pertinent examples. Some suggest- ed solutions (one example: e m b e d d i n g constraints in .dealing with long-distance dependencies) are transformationalist r a t h e r than systemic, and one could argue that it is next to impossible to reconcile the two phi- losophies, even without trying to i n c o r p o r a t e them in o n e c o m p u t e r system. T h e q u e s t i o n is w h e t h e r re- searchers will be b e t t e r off with this well-structured but rather tendentious digest than with a grammar like Quirk's, or Jespersen's, or Zaliznjak's " N o m i n a l F o r m - F o r m a t i o n in Russian".

Appendix C contains a very concise survey of the post-1965 d e v e l o p m e n t of t r a n s f o r m a t i o n a l syntax. It is a logical extension of C h a p t e r 4. This material is not indispensable for the b o o k , especially since the transformationalist a p p r o a c h has b e e n shown not to be particularly applicable in building c o m p u t e r systems.

A p p e n d i x D is the shortest and the most immedi- ately usable of all. It contains a DL definition of an ATN grammar and one such grammar for English. The n e t w o r k is the summary of results o b t a i n e d in C h a p t e r 5 and is a reasonable starting point for developing a practical parser. It contains 18 states and 51 arcs in three subnetworks.

Methodology and Linguistic Theory

The methodological part of this b o o k is the most im- p o r t a n t one. This seems to be the first forceful at- t e m p t at finding a substitute for the Chomskian transformationalist milieu in the field of linguistics. (Please n o t e the a b s e n c e of the d e s c r i b e r " c o m p u t a t i o n a l " : Winograd significantly considers c o m p u t a t i o n a l linguistics to be a linguistic paradigm, like the structural and the generative ones, not an application area of general theoretical linguistics.) The need for the philosophical and m e t h o d o l o g i c a l justification of the largely application-minded efforts in AI has b e e n realized for a long time but n o b o d y had b e e n eager to spend time

(5)

Book R e v i e w Language as a Cognitive Process. I. Syntax.

devising a m e t a - t h e o r e t i c a l f r a m e w o r k for the field. The necessity of the deviation from transformational- ism is justified in this b o o k in terms of a switch in the scientific paradigm within which research is being con- ducted - the notion was b o r r o w e d from the philoso- pher of science Thomas Kuhn. The reigning paradigm is generative. The computational paradigm is a rebel. Although the generative and the computational para- digms share an interest in the knowledge possessed by an individual who uses language and in formal symbol manipulation, they differ in the degree of attention to process organization (low in the generative paradigm) and the inclusion of non-linguistic knowledge into the sphere of interest of linguistics (liberal in the c o m p u t a - tional paradigm - cf. Raskin ( f o r t h c o m i n g ) and Schank et al. (1982) for two recent expositions of the positions of the adherents of the generative and the computational paradigm, respectively.)

The computational paradigm, of which W i n o g r a d is p r o b a b l y the best explicator, perceives language as " a communicative process based on knowledge .... Theo- retical c o n c e p t s of p r o g r a m and data can form the basis for building precise c o m p u t a t i o n a l models of mental processing" (p. 13). The basic model of communicative processing is perceived in terms of fulfilling c o m m u n i c a t i v e goals (different sets for the speaker and the hearer) t h r o u g h ample use of the stored knowledge of 1) language, 2) world, and 3) situation. Winograd goes on to specify the model of processing done by a language user and to discuss the " n e a r l y d e c o m p o s a b l e " stratified model of the knowledge of language used by a language user. This model contains three rubrics: stored knowledge, processes and assigned structures, each of which contains six parallel levels of rules, processes and structures, respectively (from p h o n o l o g y to, notably, pragmatics), while the stored knowledge also includes two kinds of dictionaries (a syntactic and a semantic one). The model is not discussed in the greatest possible detail simply because it is not yet a full-fledged theory, and also since the genre of the text precludes the u n d u e emphasis on m e t a t h e o r y ( h o w e v e r w e l c o m e such a discussion or theory may be to the field).

The chapter also gives an overconcise and exces- sively metaphorical account of the history of the linguistic science and an overview of the c o m p u t e r applications for natural language. The tone of the discussion here and t h r o u g h o u t the b o o k is refreshingly e v e n h a n d e d and calm.

C o n c l u s i o n

This reviewer taught a o n e - t e r m course based on W i n o g r a d ' s b o o k in Spring 1983 to seniors in C o m p u t - er Science. The course was very successful. The students expressed great enthusiasm a b o u t the topic and the way it was treated, although the course was by no means easy: the participants had 22 h o m e w o r k assign- ments, largely of a computational nature, including an active chart parser and an ATN parser for a small sub- set of English as two of the regular exercises, and a term project. In a very large measure, the course owed its success to the b o o k under review, which was used as the t e x t b o o k 75 percent of the time. A questi- onnaire distributed to the participants showed that the text was an unconditional success. Predictably, a ma- jority of the students polled would have preferred Lisp or some other programming language to DL. There were no complaints a b o u t excessive difficulty, although the b o o k is intended for graduate courses. In the course, we covered Chapters 1 through 5 and the case grammar part of C h a p t e r 6. C h a p t e r 7 was sug- gested for independent reading.

College teachers of c o m p u t a t i o n a l linguistics should be very grateful to Terry W i n o g r a d for the a m o u n t of time and effort he d e v o t e d to this fundamental text. It is a b e a c o n for the field. I have no d o u b t that this b o o k will b e c o m e a standard reference b o o k for the developers of syntactic parsers. There are all reasons to believe that the f o r t h c o m i n g second volume, devot- ed to meaning, will as authoritative and c o m p r e h e n - sive, and even more t h o u g h t - p r o v o k i n g and stimulat- ing.

Sergei Nirenburg, Colgate University

R e f e r e n c e s

Akmajian, A. and Heny, F. 1975 An Introduction to the Principles of Transformational Syntax. MIT Press, Cambridge, MA. Bates, Madeleine. The theory and practice of augmented transition

network grammars. In Bolc, L., Ed., Natural Langauge Commu- nication with Computers. Springer, New York, NY: pp. 191-259. Charniak, E. and Wilks, Y., Ed. 1976 Computation Semantics.

North Holland, Amsterdam.

Quirk, R., Greenbaum, S., Leech, G., and Svartvik, J. 1972 A

Grammar of Contemporary English. Seminar Press, New York, NY.

Raskin, V. to appear On the boundary between linguistic and encyclopedic knowledge. Quaderni de Semantica.

Schank, R., Birnbaum, L., and Mey, J. 1982 Integrating semantics and pragmatics. In Preprints of the plenary session papers of the X l l l International Congress of Linguistis. Tokyo, pp. 129-140. Woods, W.A. 1 9 7 0 Transition network grammars for natural

language analysis. CACM 13 10, pp. 591-606.